Adding Virtual Objects to Realtime Images; A Case Study in Augmented Reality

Year 2020, Volume: 3 Issue: 3, 188 - 200, 30.12.2020

Çağla Ediz

https://doi.org/10.35377/saucis.03.03.730505

Cited By: 2

Abstract

Augmented reality applications related with faces such as make-up, hair design, wearing glasses are mostly prepared for entertainment purposes. Facilitating the preparation of augmented reality applications and more accurate analysis of real-world data in applications will enable these applications to be used more widely in different sectors such as R&D, education and marketing. In generally, the steps in image-based augmented reality applications can be listed as follows; detection of the targeted object, finding two reference points for each targeted object in 2D images, determining the boundaries of virtual object in its image and inserting the virtual object in real time. In this study, the problems that may be encountered in preparations of these augmented reality applications expected to be used more in the future are examined through a case study. Firstly, haar cascade classifiers, used to find different face areas, are compared and as a result of the comparison, it is decided to use eye haar cascade. Afterwards, rule-based approaches have been used to eliminate the wrong ones among the found eyes and to match the eyes of the same face. Then the position, size and angle of the virtual object to be added are calculated and it is added to the face using affine transformations. The problems encountered in augmented reality and algorithms used for problem solving are explained through the virtual hat application, but these simply prepared algorithms, can be used for different objects such as hair and glasses by changing the target points.

Keywords

augmented reality, opencv, image processing

References

• P. Chen, Z. Peng,., D. Li, and L.Yang, , "An improved augmented reality system based on AndAR", Journal of Visual Communication and Image Representation, 37, 63-69, 2016.
• T. İçten and B. A. L Güngör, "Artırılmış gerçeklik üzerine son gelişmelerin ve uygulamaların incelenmesi", Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji, 5(2), 111-136, 2017.
• N. Wang, , X. Gao, D. Tao, H.Yang and X. Li, "Facial feature point detection: A comprehensive survey", Neurocomputing, 275, 50-65, 2018.
• P Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features", In Proceedings Of The 2001 IEEE Computer Society Conference On Computer Vision And Pattern Recognition, CVPR 2001 (Vol. 1, pp. I-I). IEEE, December, 2001.
• E. Skodras and N. Fakotakis, "Precise localization of eye centers in low resolution color images", Image and Vision Computing, 36, 51-60, 2015.
• D. Chen, S. Ren, Y.Wei, X. Cao, and J. Sun, "Joint cascade face detection and alignment", In European Conference On Computer Vision, pp. 109-122, Springer, Cham, September, 2014.
• V. Schwind, "The golden ratio in 3D human face modeling", Stuttgart Media University, Stuttgart, 2011.
• A. M. Rahman, T. T Tran, S.A. Hossain and A. El Saddik, "Augmented rendering of makeup features in a smart interactive mirror system for decision support in cosmetic products selection", In 2010 IEEE/ACM 14th International Symposium on Distributed Simulation and Real Time Applications, pp. 203-206, IEEE, October, 2010.
• A. Javornik, Y. Rogers, A. M. Moutinho and R.Freeman, "Revealing the shopper experience of using a" magic mirror" augmented reality make-up application", In Conference on designing interactive systems, Vol. 2016, pp. 871-882, Association for Computing Machinery (ACM), 2016.
• Z Feng, F. Jiang and R.Shen, "Virtual Glasses Try-on Based on Large Pose Estimation", Procedia Computer Science, 131, 226-233, 2018.
• J. J. Lv, X. H. Shao, J. S. Huang, X. D. Zhou and X. Zhou, "Data augmentation for face recognition", Neurocomputing, 230, 184-196, 2017.
• M. Kim and K. Cheeyong, "Augmented reality fashion apparel simulation using a magic mirror", International Journal Of Smart Home, 9(2), 169-178, 2015.
• K. Diaz-Chito, A. Hernández-Sabaté and A. M. López, "A reduced feature set for driver head pose estimation", Applied Soft Computing, 45, 98-107, 2016.
• E. Murphy-Chutorian and M. M. Trivedi, "Head pose estimation and augmented reality tracking: An integrated system and evaluation for monitoring driver awareness", IEEE Transactions on intelligent transportation systems, 11(2), 300-311, 2010.
• H.Peng, "Application research on face detection technology based on OpenCV in mobile augmented reality", International Journal of Signal Processing, Image Processing and Pattern Recognition, 8(2), 249-256, 2015.
• Z. Mahmood, T. Ali, N. Muhammad, N. Bibi, I. Shahzad and S. Azmat, "EAR: Enhanced Augmented Reality System for Sports Entertainment Applications", KSII Transactions on Internet & Information Systems, 11(12), 2017.
• F.Pereira, C. Silva and M. Alves, "Virtual fitting room augmented reality techniques for e-commerce", In International Conference On Enterprise Information Systems, pp. 62-71, Springer, Berlin, Heidelberg, October, 2011.
• V Singh, V Shokeen and B Singh, "Face detection by haar cascade classifier with simple and complex backgrounds images using opencv implementation", International Journal of Advanced Technology in Engineering and Science, 1(12), 33-38. 2013.
• S. Soo, "Object detection using Haar-cascade Classifier", Institute of Computer Science, University of Tartu, 1-12, 2014.
• T.Kumar and K. Verma, "A Theory Based on Conversion of RGB image to Gray image", International Journal of Computer Applications, 7(2), 7-10, 2010.
• F.Patin, "An introduction to digital image processing", [Online]. Available: http://www. programmersheaven. com/articles/patin/ImageProc. pdf, 2003. [Accessed: 20-Jan-2020]
• G. Wolberg, " Digital image warping", Vol. 10662, pp. 90720-1264, Los Alamitos, CA: IEEE Computer Society Press, 1990.
• R. C. Gonzalez and R. E. Woods, Digital image processing, Pearson International Edition, Thid Edition, USA., 2008.
• R. Fang, K. D. Tang, N. Snavely and T. Chen, "Towards Computational Models of Kinship Verification", IEEE International Conference on Image Processing, Hong Kong, September 2010 (ICIP '10)