Controlling A Robotic Arm Using Handwritten Digit Recognition Software

Abstract

Repetitive tasks in the manufacturing industry is becoming more and more commonplace. The ability to write down a number set and operate the robot using that number set could increase the productivity in the manufacturing industry. For this purpose, our team came up with a robotic application which uses MNIST data set provided by Tensorflow to employ deep learning to identify handwritten digits. The system is equipped with a robotic arm, where an electromagnet is placed on top of the robotic arm. The movement of the robotic arm is triggered via the recognition of handwritten digits using the MNIST data set. The real time image is captured via an external webcam. This robot was designed as a prototype to reduce repetitive tasks conducted by humans. 

Keywords

• MNIST Handwritten Digit Recognition,Deep Learning,Embedded System Robotic Arm Control

References

1. Y. Lecun, C. Cortes, C.J.C. Burges, MNIST handwritten digit database, http://yann.lecun.com/exdb/mnist/
2. K. Sato, N. Shimoda, Build your own machine-learningpowered robot arm using tensorflow and google cloud Google Cloud blog, 2017.
3. Keras documentation, https://keras.io/
4. TensorFlow, https://www.tensorflow.org/
5. A. Elfasakhany, E. Yanez, K. Baylon, R. Salgado, Design and development of a competitive low-cost robot arm with four degrees of freedom, Modern Mechanical Engineering, pp.47-55.
6. OpenCV library document, https://opencv.org/
7. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. Arxiv - Computer Vision and Pattern Recognition .
8. S. Raschka, V. Mirajalili, Python machine learning (pp. 341-385).

9. P. Bezak, P. Bozek, Y. Nikitin, Advanced robotic grasping system using deep learning, Procedia Engineering, 96, pp. 10-20, 2014.
10. A. Dhawan, A. Bhat, S. Sharma, H. K. Kaura, Automated robot with object recognition and handling features, International Journal of Electronics and Computer Science Engineering, ISSN- 2277-1956.
11. E. B. Mathew, D. Khanduja, B. Sapra, B. Bhushan, Robotic arm control through human arm movement detection using potentiometers. 2015 International Conference on Recent Developments in Control, Automation and Power Engineering (RDCAPE), 2015.
12. B.Iscimen, H. Atasoy, Y. Kutlu, S. Yildirim, E. Yildirim, Smart robot arm motion using computer vision, 2015.
13. M.A. Jayaram, H. Fleyeh, Convex Hulls in Image Processing, A Scoping Review, American Journal of Intelligent Systems, 2016.
14. N. Rai, B. Rai, P. Rai, Computer vision approach for controlling educational robotic arm based on object properties, 2nd International Conference on Emerging Technology Trends in Electronics, Communication and Networking. 2014
15. T. S. Tonbul, M. Sarıtas, Beş eksenli bir edubot robot kolunda ters kinematic hesaplamalar ve yörünge planlaması. J. Fac. Eng. Arch. Gazi Univ. Vol 18, No 1, 145-167, 2013
16. A. B. Rehiara, Kinematics of adeptthree robot arm, Robot Arms, ISBN: 978-953-307-160-2, 2011.