Internet of Things-Based Smart Security System with Face and Object Detection Using Machine Learning

Abstract

The rapid advancement of the Internet of Things (IoT) has enabled significant progress across various sectors such as finance, healthcare, smart homes, and smart cities. One key application is the development of smart security systems, which are gaining traction due to their efficiency, reduced need for human input, and enhanced threat detection. However, many existing systems face challenges such as limited coverage, false alarms, weak authentication, privacy concerns, slow response times, and dependency on external resources. To overcome these issues, this paper introduces an IoT-based smart security system that uses machine learning for facial and object recognition. The system employs a Convolutional Neural Network (CNN) to detect faces and a Single Shot Detector (SSD) for identifying suspicious objects. When an unknown individual is identified, a push notification alerts the administrator for further action. The system demonstrated high performance, with CNN achieving 94\% accuracy and an F1-score of 95.24\%, while SSD achieved 90\% accuracy and an F1-score of 94.7\%. This intelligent security solution has potential applications beyond the scope of this study and could be effectively implemented across multiple industries, enhancing safety through advanced technology and AI-driven methods.

Keywords

• Convolutional neural network
• Facial recognition
• Internet of things
• Machine learning
• Object recognition
• Single shot detector

References

1. D. Surie, O. Laguionie, and T. Pederson, Wireless sensor networking of everyday objects in a smart home environment, in Proc. Int. Conf. Intelligent Sensors, Sensor Networks and Information Processing, (2008), 189–194.
2. L. Atzori, A. Iera, and G. Morabito, The internet of things: A survey, Computer Networks, 54(15), (2010), 2787–2805.
3. E. Fleisch, What is the Internet of Things, Auto-ID Labs White Paper WP-BIZAPP-053, St. Gallen, Switzerland, (2010).
4. O. Vermesan and P. Friess, Internet of Things Applications: From Research and Innovation to Market Deployment, Taylor & Francis, (2014).
5. H. Gharavi and R. Ghafurian (Eds.), Smart Grid: The Electric Energy System of the Future, IEEE, Piscataway, NJ, USA, (2011).
6. B. Afzal, G. Anwar, A. Q. Muhammed, and S. Shahaboffin, Smart security framework for educational institutions using the Internet of Things (IoT), Computers, Materials and Continua, 61(1), (2019), 81–101.
7. R. K. Kodali, V. Jain, S. Bose, and L. Boppana, IoT-based smart security and home automation system, in Proc. Int. Conf. Computing, Communication and Automation (ICCCA), (2016), 1286–1289.
8. M. Xue et al., Machine learning security: Threats, countermeasures, and evaluations, IEEE Access, 8, (2020), 74720–74742.

9. L. Chen, M. Tseng, and X. Lian, Development of foundation models for Internet of Things, Frontiers of Computer Science in China, 4, (2010), 376–385.
10. M. Saifuzzaman et al., Smart security for an organization based on IoT, International Journal of Computer Applications, 165(10), (2017), 33–38.
11. M. Bagaa, T. Taleb, J. B. Bernabe, and A. Skarmeta, A machine learning security framework for IoT systems, IEEE Access, 8, (2020), 114066–114077.
12. S. Patel et al., Face recognition based smart attendance system using IoT, International Journal of Computer Sciences and Engineering, 6(5), (2018), 871–877.
13. H. K. Bharadwaj et al., A review on the role of machine learning in enabling IoT-based healthcare applications, IEEE Access, 9, (2021), 38859–38890.
14. J. K. Thavil, V. P. Durdhawale, and P. S. Elake, Study on smart security technology for women based on IoT, IRJET, 4(2), (2017).
15. R. Sagar, R. Jhaveri, and C. Borrego, Applications in security and evasions in machine learning: A survey, Electronics, 9(1), (2020), 97.
16. P. K. Medapati et al., LAMSTAR: For IoT-based face recognition system to manage the safety factor in smart cities, Transactions on Emerging Telecommunications Technologies, 31(12), (2020).
17. M. Upala and W. K. Wong, IoT solution for the smart library using facial recognition, in IOP Conf. Series: Materials Science and Engineering, 495, (2019).
18. H. I. Salehin et al., Development of an IoT-based smart baby monitoring system with face recognition, in Proc. IEEE World AI IoT Congress (AIIoT), (2021), 292–296.
19. S. Pawar et al., Smart home security using IoT and face recognition, in Proc. Int. Conf. Computing Communication Control and Automation (ICCUBEA), (2018), 1–6.
20. V. Ravishankar et al., Sensor integration and facial recognition deployment in a smart home system, in Proc. Int. Conf. Recent Trends in Machine Learning, IoT, Smart Cities and Applications, (2022), 759–771.
21. A. B. Shetty and J. Rebeiro, Facial recognition using Haar cascade and LBP classifiers, Global Transitions Proceedings, 2(2), (2021), 330–335.
22. I. Aydin and N. A. Othman, A new IoT combined face detection of people by using computer vision for security applications, in Proc. Int. AI and Data Processing Symp. (IDAP), (2017), 1–6.
23. N. Patil, S. Ambatkar, and S. Kakde, IoT-based smart surveillance security system using Raspberry Pi, in Proc. Int. Conf. Communication and Signal Processing (ICCSP), (2017), 344–348.
24. Z. Balogh, M. Magdin, and G. Molnár, Motion detection and face recognition using Raspberry Pi as a part of the Internet of Things, Acta Polytechnica Hungarica, 16(3), (2019), 167–185.
25. A. O. Bang and U. P. Rao, Context-aware computing for IoT: History, applications and research challenges, in Proc. Int. Conf. Smart Energy and Communication, (2021), 777–787.
26. K. Ashton, That ‘Internet of Things’ thing, RFID Journal, (2009), 97–114.
27. A. Al-Fuqaha et al., Internet of Things: A survey on enabling technologies, protocols, and applications, IEEE Communications Surveys & Tutorials, 17(4), (2015), 2347–2376.
28. Vodafone, The history of the Internet of Things, (2016), https://www.vodafone.com/business/iot/what-is-iot/history-of-iot
29. MongoDB, What is IoT architecture?, (2022), https://www.mongodb.com/cloud-explained/iot-architecture
30. D. Bandyopadhyay and J. Sen, Internet of Things: Applications and challenges in technology and standardization, Wireless Personal Communications, (2011).
31. R. Khan et al., Future Internet: The Internet of Things architecture, possible applications and key challenges, in Proc. Int. Conf. Frontiers of Information Technology, (2012), 257–260.
32. M. Wu et al., Research on the architecture of the Internet of Things, in Proc. Int. Conf. Advanced Computer Theory and Engineering, (2010), V5-484–V5-487.
33. A. L. Samuel, Some studies in machine learning using the game of checkers, IBM Journal of Research and Development, 3(3), (1959), 210–229.
34. BriefCam, What is video analytics?, (2020), https://www.briefcam.com/technology/video-analytics/
35. IBM, What is machine learning?, (2021), https://www.ibm.com/topics/machine-learning
36. Y. Liu, J. Mao, and X. Liu, Smart community security system based on sensor web, Applied Mechanics and Materials, 571–572, (2014), 1037–1041.
37. Q. Lyu, Z. Ning, and H. Liu, Remotely access ‘my’ smart home in private: An anti-tracking authentication and key agreement scheme, IEEE Access, 7, (2019), 41835–41851.
38. MyAutomatedPalace, Smart home security vs traditional, (2021), https://myautomatedpalace.com/smart-home-security-vs-traditional/
39. A. K. Tran et al., Designing efficient smart home management with IoT smart lighting: A case study, Wireless Communications and Mobile Computing, 2020, (2020).
40. Y. Sun, X. Guan, and T. Liu, A cyber-physical monitoring system for attack detection in smart grid, in Proc. IEEE INFOCOM Workshops, (2013), 476–481.
41. H. Rawat et al., Analysis of various smart security systems, Lecture Notes in Networks and Systems, (2023).
42. M. Imran, S. Rana, and J. Grewal, Technological advancements in smart security systems: A comparative analysis and future prospects, IJNRD, 8(11), (2023).
43. H. Güney, AWC-NIDS: Attack-wise customized network intrusion detection system, Concurrency and Computation: Practice and Experience, 35(26), (2023).
44. J. G. Shanahan and L. Dai, Introduction to computer vision and real-time deep learning-based object detection, in Proc. ACM SIGKDD, (2020), 3523–3524.
45. V. Mašinska, OpenCV introduction, (2018).
46. J. Liang, Face recognition based on Python library, Network Security Technology and Application, (2019).
47. Z. Li et al., A survey of convolutional neural networks: Analysis, applications, and prospects, IEEE Transactions on Neural Networks and Learning Systems, (2021).
48. N. Saxena and S. Shukla, A review on LBPH-based enhanced technique for multiple face detection, in Int. Conf. Intelligent Technology and Science, (2021).
49. D. S. Cardona-Pineda et al., Face recognition—Eigenfaces, in Handbook on Decision Making, (2022), 373–397.
50. A. Nemavhola et al., A systematic review of CNN architectures, databases, performance metrics, and applications in face recognition, Information, 16(2), (2025), 107.
51. R. K. Shukla et al., Face recognition using LBPH and CNN, Recent Advances in Computer Science and Communications, 17(5), (2024), 48–58.
52. M. Şerifi and Ü. Şerifi, Dual CNN and texture-based face-iris multimodal biometric system, Signal, Image and Video Processing, 19(4), (2025).
53. P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proc. IEEE CVPR, (2001).
54. J. Redmon et al., You only look once: Unified, real-time object detection, in Proc. IEEE CVPR, (2016).
55. J. Redmon and A. Farhadi, YOLOv3: An incremental improvement, arXiv preprint arXiv:1804.02767, (2018).
56. T.-Y. Lin et al., Microsoft COCO: Common objects in context, in Proc. ECCV, (2014).
57. W. Liu et al., SSD: Single shot multibox detector, in Proc. ECCV, (2016).
58. Z. Huang et al., Speed/accuracy trade-offs for modern convolutional object detectors, in Proc. IEEE CVPR, (2017).
59. R. Ahamad and K. Mishra, Hybrid approach for suspicious object surveillance using video clips and UAV images in cloud-IoT-based computing environment, Cluster Computing, (2023), https://doi.org/10.1007/s10586-023-03977-0
60. K. Chaichana, P. Netinant, and S. Pukdesree, Design and implementation of an IoT-based surveillance system using Raspberry Pi, camera, and motion sensor, in Proc. 7th Int. Conf. E-Commerce, E-Business and E-Government, (2023), https://doi.org/10.1145/3599609.3599638
61. M. Zahrawi and K. Shaalan, Improving video surveillance systems in banks using deep learning techniques, Scientific Reports, (2023), https://doi.org/10.1038/s41598-023-35190-9
62. N. Bushra, G. Shobana, U. Maheswari, and N. Subramanian, Smart video surveillance based weapon identification using YOLOv5, in Proc. Int. Conf. Electronic Systems and Intelligent Computing (ICESIC), (2022), https://doi.org/10.1109/ICESIC53714.2022.9783499
63. I. Chatisa, Y. Syahbana, and A. Wibowo, Object detection and monitor system for building security based on Internet of Things (IoT) using illumination invariant face recognition, KINETIK: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 8(1), (2023), https://doi.org/10.22219/kinetik.v8i1.1622