Comparative Assessment Of Mobile Augmented Reality Applications Developed for Visually Impaired Persons

Öz

Thanks to the widespread use of augmented reality (AR) development tools and mobile devices such as tablets and smartphones, many new assistive AR applications have found a place in our lives, especially in the last decade. These applications provide reality augmentation with the information that comes from the inertial sensors of the devices such as camera, infrared, ToF, and lidar. With the spread of such applications, the development of visually impaired focused applications has become both a need and an active research field. The needs, sensitivities and comforts of the target audience should be taken into account in evaluating the suitability of these applications for the use of visually impaired people (VIP). In this study, selected studies from both the software industry and the academic literature are evaluated in terms of their suitability, accessibility and outputs provided to VIP. Studies selected from the software industry are products that have gained a place in the market and reached an evident user base, and academic studies are actual and original studies published from 2013 to present. In addition to the comparison of the studies, it has also been examined how an application that is most suitable for the mentioned criteria can be put forward.

Anahtar Kelimeler

• Augmented Reality
• Mobile Applications
• Assistive Technologies
• Visually Impaired Persons

Görme Engelliler İçin Geliştirilmiş Destekleyici Mobil Artırılmış Gerçeklik Uygulamalarının Karşılaştırmalı Değerlendirmesi

Öz

Tablet, akıllı telefon gibi mobil cihazlar için artırılmış gerçekliğe (AR) yönelik sunulan geliştirme araçlarının ve mobil cihazların kullanımının yaygınlaşmasıyla, özellikle son on yılda, birçok yeni destekleyici AR uygulaması hayatımıza girmiştir. Bu uygulamalar cihazların kamera, kızılötesi, ToF, lidar gibi sensörlerinden gelen bilgiyle gerçekliğin artırımını sağlamaktadırlar. Böylesi uygulamaların yaygınlaşmasıyla, görme engelli odaklı uygulamaların geliştirilmesi de hem bir ihtiyaç hem de faal bir araştırma alanı olmuştur. Bu uygulamaların görme engelli bireylerin (GEB) kullanımına uygunluğunun değerlendirilmesinde hedef kitlenin ihtiyaçları, hassasiyetleri ve konforları göz önünde bulundurulmalıdır. Bu çalışmada kendine yer bulan, gerek yazılım sektöründen gerekse akademik literatürden seçilmiş çalışmalar, GEB’e sağladığı çıktılar, gerçek zamanlı kullanıma hız ve doğruluk kriterleri yönünden uygunluk ve ulaşılabilirlik açısından değerlendirilmiştir. Yazılım sektöründen seçilen çalışmalar kendine pazarda yer edinip belirli bir kullanıcı kitlesine ulaşmış ürünler olup, akademik çalışmalar ise 2013’ten günümüze yayınlanmış güncel ve özgün çalışmalardır. Çalışmaların mukayesesinin yanı sıra, söz edilen kriterlere en uygun bir uygulamanın nasıl ortaya konulabileceği yönünde de irdelemelerde bulunulmuştur.

Anahtar Kelimeler

• Artırılmış Gerçeklik
• Mobil Uygulamalar
• Destekleyici Teknolojiler
• Görme Engelliler

Kaynakça

1. Akın, A.T. and Cömert, Ç., “The development of an augmented reality audio application for visually impaired persons.”, Multimedia Tools and Applications, 82(11), 17493-17512, (2023).
2. Elmannai, W., and Khaled E., "Sensor-based assistive devices for visually-impaired people: Current status, challenges, and future directions.", Sensors, 17.3:565, (2017).
3. Csapó, Á., et al., "A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research.", Journal on Multimodal User Interfaces, 9.4: 275-286 (2015).
4. Croce, D., et al., "Enhancing tracking performance in a smartphone-based navigation system for visually impaired people.", 24th Mediterranean Conference on Control and Automation (MED)., IEEE, (2016).
5. Tapu, R., Bogdan M., and Titus Z., "Wearable assistive devices for visually impaired: A state of the art survey.", Pattern Recognition Letters, 137: 37-52, (2020).
6. Maidenbaum, S., Sami A., and Amir, A.i., "Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation.", Neuroscience & Biobehavioral Reviews, 41: 3-15, (2014).
7. Gallo, P., et al. "ARIANNA: pAth recognition for indoor assisted navigation with augmented perception.", arXiv preprint, arXiv:1312.3724, (2013).
8. Lo Valvo, A., et al., "A navigation and augmented reality system for visually impaired people.", Sensors, 21.9: 3061, (2021).

9. Ahmetovic, D., et al. "NavCog: a navigational cognitive assistant for the blind.", Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services., (2016).
10. Alghamdi, S., van Schyndel R., and Ibrahim K., "Accurate positioning using long range active RFID technology to assist visually impaired people.", Journal of Network and Computer Applications, 41: 135-147, (2014).
11. Plikynas, D., et al., "Indoor navigation systems for visually impaired persons: Mapping the features of existing technologies to user needs.", Sensors ,20.3: 636, (2020).
12. Martinez-Sala, A., et al., "Design, implementation and evaluation of an indoor navigation system for visually impaired people.", Sensors, 15.12: 32168-32187, (2015).
13. http://gorengoz.aile.gov.tr,, “Gören Göz”, (Erişim: 18 Mayıs 2022).
14. https://wewalk.io/tr/, “Dünyanın En Akıllı Bastonu ve Mobil Uygulaması”, (Erişim: 18 Mayıs 2022).
15. Wang, W., "Understanding augmented reality and ARKit.", Beginning ARKit for iPhone and iPad., Apress, Berkeley, CA, 1-17, (2018).
16. Du, R., et al., "DepthLab: Real-time 3D interaction with depth maps for mobile augmented reality.", Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology., (2020).
17. Liu, W., et al., "Ssd: Single shot multibox detector.", European conference on computer vision., Springer, Cham, (2016).
18. Szegedy, C., et al., "Going deeper with convolutions.", Proceedings of the IEEE conference on computer vision and pattern recognition., (2015).
19. Oufqir, Z., El Abderrahmani A., and Satori K., "ARKit and ARCore in serve to augmented reality.", 2020 International Conference on Intelligent Systems and Computer Vision (ISCV)., IEEE, (2020).
20. Lin, B.-S., Lee C.-C., and Chiang P.-Y., "Simple smartphone-based guiding system for visually impaired people.", Sensors, 17.6: 1371, (2017).
21. Redmon, J., et al., "You only look once: Unified, real-time object detection.", Proceedings of the IEEE conference on computer vision and pattern recognition., (2016).
22. Girshick, R., "Fast r-cnn.", Proceedings of the IEEE international conference on computer vision., (2015).
23. Sun, M., et al. "“Watch Your Step”: Precise Obstacle Detection and Navigation for Mobile Users Through Their Mobile Service.", IEEE Access, 7: 66731-66738, (2019).
24. Marder-Eppstein, E.,“Project Tango”, https://dl.acm.org/doi/10.1145/2933540.2933550, 25-25, (2016).
25. https://www.theverge.com/2017/12/15/16782556/project-tango-google-shutting- down-arcore-augmented-reality, “Google’s Project Tango is shutting down because ARCore is already here”, (Erişim: 18 Mayıs 2022).
26. Bauer, Z., et al. "Enhancing perception for the visually impaired with deep learning techniques and low-cost wearable sensors.", Pattern recognition letters, 137: 27-36, (2020).
27. Laina, I., et al. "Deeper depth prediction with fully convolutional residual networks.", 2016 Fourth international conference on 3D vision (3DV)., IEEE, (2016).
28. Lee, J. H., et al. "From big to small: Multi-scale local planar guidance for monocular depth estimation.", arXiv preprint, arXiv:1907.10326, (2019).
29. Akın, A. T., and Cömert Ç.. "Testing of a Deep Learning Model Providing Monocular Depth Estimation on Mobile Devices via Web Service.", 2021 5th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT)., IEEE, (2021).
30. Kastanakis, B., “Mapbox Cookbook”., Packt Publishing Ltd, (2016).
31. Venkat R., Tarun, A. H., Yogeeshwar, S., Vishwath Kumar, B. S., and Sofana Reka, S., A realtime portable and accessible aiding system for the blind–a cloud based approach., Multimedia Tools and Applications, 1-14, (2023).
32. Maya-Martínez, S. U., Argüelles-Cruz, A. J., and Guzmán-Zavaleta, Z. J., Pedestrian detection model based on Tiny-Yolov3 architecture for wearable devices to visually impaired assistance. Frontiers in robotics and AI, 10., (2023).
33. Fahme, M.U.S., and Khan T. H., "Choose Your Arsenal." How to Make a Game. Apress, Berkeley, CA, 31-40, (2021).
34. Tandel, S., and Jamadar, A., "Impact of progressive web apps on web app development.", International Journal of Innovative Research in Science, Engineering and Technology 7.9: 9439-9444, (2018).
35. Metzger, F., et al. "The Prospects of Cloud Gaming: Do the Benefits Outweigh the Costs?.", (2016).
36. Kang, W., and Han,Y., "SmartPDR: Smartphone-based pedestrian dead reckoning for indoor localization.", IEEE Sensors journal 15.5: 2906-2916, (2014).
37. Lin, T. Y., et al., Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13., 740-755, Springer International Publishing., (2014).
38. Explore connected papers in a visual graph, https://www.connectedpapers.com/, Son Erişim: 31/05/2023