AN EMOTION ANALYSIS ALGORITHM AND IMPLEMENTATION TO NAO HUMANOID ROBOT

Abstract

Humanoid robots are extensively becoming an essential part of the social life.  It is crucial for humanoid robots to understand the emotions of the people for efficient human-robot interaction. Even though a great number of facial emotion analysis algorithms have been developed and a number of them have been implemented to humanoid robots, there are still gaps in improving accuracy, computational burden and speed of these algorithms.  This paper proposes a 4-stage emotion analysis algorithm and then presents its application to NAO humanoid robot. Initially, the robot detects the face using Viola-Jones algorithm. Later, important facial distance measurements are taken with geometric based facial distance measurement technique. Then, facial action coding system technique is used to detect movements of the measured facial points. Finally, measured facial movements are evaluated to understand instant emotional properties of the person. Although this algorithm can be implemented to all humanoid robots, in this research, it has been specifically applied to NAO humanoid robot. The reliability of the emotion analysis is verified by analyzing each terminal decision made based on the facial distance measurements. In addition, the accuracy, computational burden and speed of the algorithm are assessed to show the effectiveness of the algorithm. 

Keywords

• Emotion analysis,facial action coding system,geometric based facial distance measurement technique,facial expression,Viola-Jones algorithm

References

1. Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International journal of computer vision, 57(2), 137-154. Jensen, O. H. (2008). Implementing the Viola-Jones face detection algorithm (Master's thesis, Technical University of Denmark, DTU, DK-2800 Kgs. Lyngby, Denmark). Schneiderman, H. & Kanade, T. (2000). A Statistical Method for 3D Object Detection Applied to Faces and Cars.. CVPR (p./pp. 1746-1759), : IEEE Computer Society. ISBN: 0-7695-0662-3 Henry A. Rowley, Shumeet Baluja, and Takeo Kanade,“Neural network-based face detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no. 1, pp. 23–38, 1998. T. Kanade, "Picture Processing System by Computer Complex and Recognition of Human Faces," Kyoto University, Japan, PhD. Thesis 1973. Schapire, R. (n.d.). Explaining AdaBoost. 1st ed. [ebook] Princeton. Available at: https://www.cs.princeton.edu/~schapire/papers/explaining-adaboost.pdf [Accessed 30 Nov. 2014]. R. Brunelli and T. Poggio, "Face recognition: features versus templates," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.15, pp.1042- 1052, 1993. Brunelli, Roberto, and Tomaso Poggio. "Face recognition: Features versus templates." IEEE transactions on pattern analysis and machine intelligence15.10 (1993): 1042-1052. Ekman, P., & Friesen, W. V. (1976). Measuring facial movement. Environmental psychology and nonverbal behavior, 1(1), 56-75. Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4), 384. Loutfi, A., Widmark, J., Wikstrom, E., & Wide, P. (2003, July). Social agent: Expressions driven by an electronic nose. In Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2003. VECIMS'03. 2003 IEEE International Symposium on (pp. 95-100). IEEE. P. Ekman and W. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978. Asteriadis, S., Nikolaidis, N., & Pitas, I. (2010). A review of facial feature detection algorithms. Advances in Face Image Analysis: Techniques and Technologies: Techniques and Technologies, 42. Dhall, S., Sethi, P. (2014).Geometric and Appearance feature analysis for facial expression recognition. International Journal of Advanced Engineering Technology. Gongor, F., Tutsoy, O. (2017). NAO Humanoid Robot Makes Facial Character Analysis. International Journal of Social Robotics. https://www.ald.softbankrobotics.com/en Chown, E., & Lagoudakis, M. G. (2014, July). The standard platform league. In Robot Soccer World Cup (pp. 636-648). Springer, Cham. “Choregraphe User Guide.” Aldebaran Robotics. Web. Aug 2012. http://opencv.org/ “Python NAOqi API .” Aldebaran Robotics. Web. Aug. 2012. Sebanz, N., Knoblich, G., & Prinz, W. (2005), How two share a task: Corepresenting stimulus-response mappings. Journal of Experimental Psychology: Human Perception and Performance, 31, 1234–1246. Michael, J. (2011). Shared emotions and joint action. Review of Philosophy and Psychology, 2(2), 355–373. Barros, P., Weber, C., & Wermter, S. (2015, November). Emotional expression recognition with a cross-channel convolutional neural network for human-robot interaction. In Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on (pp. 582-587). IEEE. L. Ballihi, A. Lablack, B. Amor, I. Bilasco, and M. Daoudi, “Positive/negative emotion detection from RGB-D upper body images,” in Face and Facial Expression Recognition from Real World Videos, ser. Lecture Notes in Computer Science, Q. Ji, T. B. Moeslund, G. Hua, and K. Nasrollahi, Eds. Springer International Publishing, 2015, vol. 8912, pp. 109–120. Marsella, S., Gratch, J., and Petta, P, “Computational Models of Emotion.” In Scherer, K.R., BA.nziger, T., & Roesch, E. (Eds.) A blueprint for an affectively competent agent: Cross-fertilization between Emotion Psychology, Affective Neuroscience, and Affective Computing. Oxford: Oxford University Press, 2010. Le, Q.A., Hanoune, S. and Pelachaud, C. “Design and Implementation of an expressive gesture model for a humaoid robot.” 11th IEEE-RAS International Conference of Humanoid Robots (Humanoids 20122), 2011. Breazeal, C., “Emotion and sociable humanoid robot”. Int. J. of Human Computer Studies, vol. 59, pp.119-155, 2003. Islam, M. N., & Loo, C. K. (2014, November). Geometric feature-based facial emotion recognition using two-stage fuzzy reasoning model. In International Conference on Neural Information Processing (pp. 344-351). Springer, Cham. Tsalakanidou, F., & Malassiotis, S. (2010). Real-time 2D+ 3D facial action and expression recognition. Pattern Recognition, 43(5), 1763-1775. Kharat, G. U., & Dudul, S. V. (2008). Human emotion recognition system using optimally designed SVM with different facial feature extraction techniques. WSEAS Transactions on Computers, 7(6), 650-659. Zhang, L., Jiang, M., Farid, D., & Hossain, M. A. (2013). Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot. Expert Systems with Applications, 40(13), 5160-5168.