Mobile Application Based Indoor Routing System Using Transfer Learning

Abstract

Nowadays, indoor routing in places with complex multi-storey architecture such as hospitals, shopping malls, parking garages and public buildings is traditionally carried out using signage or devices in a fixed position. When we examine the literature, it is generally seen that indoor orientation studies for certain needs are seen. The fact that the routing systems are fixed, and the signage is not an effective tool constitutes the motivation of this study. In this study, an image-based mobile application that is hardware-independent and adaptable to other interior spaces has been implemented using a mobile device. The application basically consists of two parts. In the first part, transfer learning based MobileNetV2 architecture is used to determine the initial store location. The proposed model detects the store signage image taken from the camera with 96% success. In the second part, the user is successfully guided to the target using the Dijkstra algorithm. With the developed mobile application, the user can reach the targets on the same or different floors in the fastest way without wasting time and without asking anyone. The application was tried in real time in a shopping center and successful results are obtained.

Keywords

• Dijkstra algorithm
• Indoor navigation
• Transfer learning

Transfer Öğrenme Kullanılarak Mobil Uygulama Tabanlı İç Mekan Yönlendirme Sistemi

Abstract

Günümüzde hastaneler, alışveriş merkezleri, kapalı otoparklar ve kamu binaları gibi karmaşık çok katlı mimariye sahip yerlerde iç mekan yönlendirmesi geleneksel olarak tabelalar veya sabit konumdaki cihazlar kullanılarak gerçekleştirilmektedir. Literatürü incelediğimizde genel olarak belirli ihtiyaçlara yönelik iç mekan yönlendirme çalışmalarının yapıldığı görülmektedir. Yönlendirme sistemlerinin sabit olması ve tabelaların etkili bir araç olmaması bu çalışmanın motivasyonunu oluşturmaktadır. Bu çalışmada, mobil cihaz kullanılarak donanımdan bağımsız ve diğer iç mekanlara uyarlanabilen, görüntü tabanlı bir mobil uygulama gerçekleştirilmiştir. Uygulama temel olarak iki bölümden oluşmaktadır. İlk bölümde, ilk mağaza konumunu belirlemek için transfer öğrenme tabanlı MobileNetV2 mimarisi kullanıldı. Önerilen model, kameradan alınan mağaza tabela görüntüsünü %96 başarı ile tespit etmektedir. İkinci bölümde kullanıcı Dijkstra algoritması kullanılarak hedefe başarılı bir şekilde yönlendirilmektedir. Geliştirilen mobil uygulama ile kullanıcı aynı veya farklı katlardaki hedeflere zaman kaybetmeden ve kimseye sormadan en hızlı şekilde ulaşabilmektedir. Uygulama gerçek zamanlı olarak bir alışveriş merkezinde denenmiş ve başarılı sonuçlar alınmıştır.

Keywords

• Dijkstra algoritması
• İç mekan navigasyon
• Transfer öğrenme

References

1. [1] K. Braden, C. Browning, H. Gelderloos, F. Smith, C. Marttila, L. Vallot, “Integrated inertial navigation system/Global Positioning System (INS/GPS) for manned return vehicle autoland application,” IEEE Symposium on Position Location and Navigation Conference, Las Vegas, NY, United States,1990, pp.74-82.
2. [2] P.K. Doyle-Baker, A. Ladle, A. Rout, P. Galpern, “Smartphone GPS Locations of Students’ Movements to and from Campus,” ISPRS International Journal of Geo-Information, vol.10 no.8, pp. 517-530, 2021.
3. [3] A.A. Başak, “Izgara Tabanlı Parmak İzi Algoritmalarıyla Kapalı Alan Konumlandırma Optimizasyonu,” Yüksek lisans tezi, Bilgisayar Mühendisliği, Ankara Üniversitesi, Ankara, Türkiye, 2017.
4. [4] I. Kırbaş, K. Arslan, “Developing Node Prototype For Indoor Positioning Systems,” Journal of Engineering Sciences and Design, vol.8 no. 2, 612-624, 2020.
5. [5] M. Murata, D. Ahmetovic, D. Sato, H. Takagi, K.M. Kitani, C. Asakawa, “Smartphone-based localization for blind navigation in building-scale indoor environments,” Pervasive and Mobile Computing, vol. 57, pp. 14-32, 2019.
6. [6] J.C. Torrado, G. Montoro, J. Gomez, “Easing the integration: A feasible indoor wayfinding system for cognitive impaired people,” Pervasive and Mobile Computing, vol. 31, pp. 137-146, 2016.
7. [7] S. Jung, S. Lee, D. Han, “A crowdsourcing-based global indoor positioning and navigation system,” Pervasive and Mobile Computing, vol. 31, pp. 94-106, 2016.
8. [8] R. Ayyalasomayajula, A. Arun, C. Wu, S. Sharma, A.R. Sethi, D. Vasisht, D. Bharadia, “Deep learning based wireless localization for indoor navigation,” MobiCom’20: Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, New York, NY, United States, 2020, pp. 1-14.

9. [9] H. Rizk, A. Elmogy, H. Yamaguchi, “A Robust and Accurate Indoor Localization Using Learning-Based Fusion of Wi-Fi RTT and RSSI,” Sensors, vol. 22 no. 7, pp. 27, 2022.
10. [10] A. Nessa, B. Adhikari, F. Hussain, X.N. Fernando, “A Survey of Machine Learning for Indoor Positioning,” IEEE Access, vol. 8, pp. 214945-214965, 2020.
11. [11] H. Mehmood, N.K. Tripathi, T. Tipdecho, “Indoor Positioning System Using Artificial Neural Network,” Journal of Computer Science, vol. 6 no.10, pp.1219-1225, 2010.
12. [12] A.A. Abdallah, C. Jao, Z. Kassas, A.M. Shkel, “A Pedestrian Indoor Navigation System Using Deep-Learning-Aided Cellular Signals and ZUPT-Aided Foot-Mounted IMUs,” IEEE Sensors Journal, vol. 22 no.6, pp. 5188-5198, 2022.
13. [13] X. Feng, K.A. Nguyen, Z. Luo, “A survey of deep learning approaches for WiFi-based indoor positioning,” Journal Of Information and Telecommunication, vol.6 no.2, pp.163-216, 2022.
14. [14] S. Tomazic, “Indoor positioning and navigation,” Sensors, vol. 21 no.14, pp. 4793, 2021.
15. [15] J. Kunhoth, A. Karkar, S. Al-Maadeed, A. Al-Ali, “Indoor positioning and wayfinding systems: A survey,” Human-centric Computing and Information Sciences, vol.10 no. 1, pp. 41, 2020.
16. [16] F. Zhang, F. Duarte, R. Ma, D. Milioris, H. Lin, C. Ratti. (2016, Oct 7) Indoor Space Recognition using Deep Convolutional Neural Network: A Case Study at MIT Campus (1st ed.) [Online]. Available: https://arxiv.org/abs/1610.02414
17. [17] W. Chen, T. Qu, Y. Zhou, K. Weng, G. Wang, G. Fu, “Door recognition and deep learning algorithm for visual based robot navigation,” IEEE International Conference on Robotics and Biomimetics, Bali, Indonesia, 2014, pp.1793-1798.
18. [18] M. Afif, R. Ayachi, Y. Said, M. Atri, “Deep Learning Based Application for Indoor Scene Recognition,” Neural Processing Letters, vol. 51, pp. 2827–2837, 2020.
19. [19] A.K.T.R. Kumar, B. Schäufele, D. Becker, O. Sawade, I. Radusch, “Indoor localization of vehicles using Deep Learning,” IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks, Coimbra, Portugal, 2016, pp. 1-6.
20. [20] A. Sultana, K. Deb, P.K. Dhar, T. Koshiba, “Classification of Indoor Human Fall Events Using Deep Learning,” Entropy, vol. 23 no.3, pp. 328, 2021.
21. [21] S. Tavasoli, X. Pan, T.Y. Yang, “Real-time autonomous indoor navigation and vision-based damage assessment of reinforced concrete structures using low-cost nano aerial vehicles,” Journal of Building Engineering, vol.68, 2023.
22. [22] B. Ludwig, G. Donabauer, D. Ramsauer, S. Karema, “URWalking: Indoor Navigation for Research and Daily Use,” Künstl Intell, vol. 37, pp. 83-90, 2023.
23. [23] M. Mallik, A.K. Panja, C. Chowdhury, “Paving the way with machine learning for seamless indoor–outdoor positioning: A survey,” Information Fusion, vol. 94, pp.126-151, 2023.
24. [24] B. Singh, D. Toshniwal, S.K. Allur, “Shunt connection: An intelligent skipping of contiguous blocks for optimizing MobileNet-V2,” Neural Networks, vol. 118, pp. 192-203, 2019.
25. [25] Y. Li, Y. Zhuang, Lan Q. Zhou, X. Niu, N. El-Sheimy, “A Hybrid WiFi/Magnetic Matching/PDR Approach for Indoor Navigation With Smartphone Sensors,” IEEE Communications Letters, vol. 20 no.1, 169-172, 2016.
26. [26] M. Ullah, S. Khusro, M. Khan, I. Alam, I. Khan, B. Niazi, “Smartphone-Based Cognitive Assistance of Blind People in Room Recognition and Awareness,” Mobile Information Systems, pp. 1-14, 2022.
27. [27] B. Li, J.P. Munoz, X. Rong, Q. Chen, J. Xiao, Y. Tian, A. Arditi, M. Yousuf, “Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People,” IEEE Transactions on Mobile Computing, vol.18 no.3, 702-714, 2019.
28. [28] E.J. Alqahtani, F.H. Alshamrani, H.F. Syed, F.A. Alhaidari, “Survey on Algorithms and Techniques for Indoor Navigation Systems,” 21st Saudi Computer Society National Computer Conference, Riyadh, Saudi Arabia, 2018, pp.1-9.
29. [29] Y. Xu, Z. Wen, X. Zhang, “Indoor optimal path planning based on Dijkstra Algorithm,” Proceedings of the 2015 International Conference on Materials Engineering and Information Technology Applications, Guilin, China, 2015, pp. 309-313.
30. [30] H. Gao, Q. Yun, R. Ran, J. Ma, “Smartphone-based parking guidance algorithm and implementation,” Journal of Intelligent Transportation Systems, vol. 25 no.4, pp. 412-422, 2021.
31. [31] J. Li, Y. An, R. Fei, H. Wang, “Smartphone based car-searching system for large parking lot.,” IEEE 11th Conference on Industrial Electronics and Applications, Hefei, China, 2016, pp. 1994-1998.
32. [32] M.A. Uddin, A.H. Suny, “Shortest path finding and obstacle detection for visually impaired people using smart phone,” International Conference on Electrical Engineering and Information Communication Technology, Savar, Bangladesh, 2015, pp. 1-4.
33. [33] V. Prudtipongpun, W. Buakeaw, T. Rattanapongsen, M. Sivaraksa, “Indoor Navigation System for Vision-Impaired Individual: An Application on Android Devices,” 1th International Conference on Signal-Image Technology & Internet-Based Systems, Bangkok, Thailand, 2015, pp. 633-638.
34. [34] K. Kasantikul, C. Xiu, D. Yang, M. Yang, “An enhanced technique for indoor navigation system based on WIFI-RSSI,” Seventh International Conference on Ubiquitous and Future Networks, Sapporo, Japan, 2015, pp. 513-518.
35. [35] N.Y. Ko, S.W. Noh, Y.S. Moon, “Implementing indoor navigation of a mobile robot,” 13th International Conference on Control, Automation and Systems, Gwangju, Korea (South), 2013, pp. 198-200.
36. [36] Buyaka. “Anasayfa,” buyaka.com. Accessed: Nov. 11, 2023 [Online]. Available: https://www.buyaka.com.tr
37. [37] A. Oğuzlar, “Data Preprocessing,” Erciyes University Journal of Faculty of Economics and Administrative Sciences, vol. 21, pp. 67-76, 2003.
38. [38] S. Eltanashi, F. Atasoy, “A Proposed Speaker Recognition Model Using Optimized Feed Forward Neural Network And Hybrid Time-Mel Speech Feature,” International Conference on Advanced Technologies, Computer Engineering and Science, Karabük, Türkiye, 2020, pp. 130-140.
39. [39] A. Tasdelen, B. Sen, “A hybrid CNN-LSTM model for pre-miRNA classification,” Scientific Reports, vol.11, 2021.
40. [40] E. Somuncu, N. Aydın Atasoy, “Realization of character recognition application on text images by convolutional neural network,” Journal of the Faculty of Engineering and Architecture of Gazi University, vol. 37 no.1, pp.17-28, 2021.
41. [41] A. Sengur, Y. Akbulut, Y. Guo, V. Bajaj, “Classification of amyotrophic lateral sclerosis disease based on convolutional neural network and reinforcement sample learning algorithm,” Health Information Science and Systems, vol.5 no.1, pp. 9, 2017.
42. [42] Y. Kim, “Convolutional Neural Networks for Sentence Classification,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp.1746–1751.
43. [43] M. Sandler, A. Howard. (2018, April 3). MobileNetV2: The Next Generation of On-Device Computer Vision Networks, [Online]. Available: https://ai.googleblog.com/2018/04/mobilenetv2-next-generation-of-on.html
44. [44] A.G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam. (2017, April 17) MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (1st ed.) [Online]. Available: https://arxiv.org/abs/1704.0486
45. [45] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” The IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4510–4520.
46. [46] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A.C. Berg, L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), vol.115 no. 3, pp. 211-252, 2015.
47. [47] Buyaka. “Kat Planları,” buyaka.com. Accessed: Nov. 22, 2023 [Online]. Available: https://www.buyaka.com.tr/kat-planlari/
48. [48] Ebru ÇIRACI, Bartın, Türkiye. Video_sameFloor.mp4 dosyasını indirme sayfası. (Oct. 23, 2023). Accessed: Oct. 24, 2023. [Online Video]. Available: https://s2.dosya.tc/server27/ewb6ha/Video_sameFloor.mp4.html.
49. [49] Ebru ÇIRACI, Bartın, Türkiye. Video_differentFloor.mp4 dosyasını indirme sayfası. (Oct. 23, 2023). Accessed: Oct. 24, 2023. [Online Video]. Available: https://s2.dosya.tc/server27/6mb0hx/Video_differentFloor.mp4.html.
50. [50] L. Zhang, Z. Yingjie, L. Yangfan, “Path Planning for Indoor Mobile Robot Based on Deep Learning,” Optik, vol. 219, pp. 1-17, 2020.
51. [51] T. Ran, L. Yuan, J.B. Zhang, “Scene perception based visual navigation of mobile robot in indoor environment,” ISA Transactions, vol.109, pp. 389-400, 2021.
52. [52] A. Poulose, D.S. Han., “Hybrid Deep Learning Model Based Indoor Positioning Using Wi-Fi RSSI Heat Maps for Autonomous Applications,” Electronics, vol. 10 no.1, pp. 2, 2021.
53. [53] F. Li, C. Guo, B. Luo, H. Zhang, “Multi goals and multi scenes visual mapless navigation in indoor using meta-learning and scene priors,” Neurocomputing, vol.4 no. 49, pp.368-377, 2021.
54. [54] Y. Himeur, S. Al-Maadeed, I. Varlamis, N. Al-Maadeed, K. Abualsaud, A. Mohamed, “Face Mask Detection in Smart Cities Using Deep and Transfer Learning: Lessons Learned from the COVID-19 Pandemic,” Systems, vol. 11 no.2, pp.107, 2023.
55. [55] A. Wibowo, C.A. Hartanto, P.W. Wirawan, “Android skin cancer detection and classification based on MobileNet v2 model,” International Journal of Advances in Intelligent Informatics, vol. 6 no.2, 135-148, 2020.