AIoT Driven User-Centered Privacy Protection From Smart Home Assistant Devices

Abstract

Smart Home Assistant Devices (SHADs) always listen for wake words such as “Alexa”, “Okay Google,” or “Hey Siri.” As a result, these devices continuously listen even when they are not being used, which can compromise users’ privacy. Existing solutions largely focus on modifying network protocols or enforcing policy changes, but implementation of these approaches depends on authorities and therefore remains inaccessible to users. Other solutions rely on mechanical switches or tactile interfaces, which reduce usability since the main advantage of SHADs lies in their natural audio-based interface. This paper presents an AIoT (Artificial Intelligence and Internet of Things) driven user-centered approach to this privacy issue, named LampGuard, which outperforms existing solutions through two user-centered benefits: (i) its implementation does not depend on authorities, giving full control to the user, and (ii) it uses an audio user interface that aligns with the default interaction model of SHADs. LampGuard incorporates an ultrasonic interference system in the form of a desk lamp that provides localized privacy protection in shared living spaces by confining ultrasonic interference specifically to the home assistant device’s microphone area rather than the entire room. Thus, LampGuard blocks only the SHAD’s microphone and does not interfere with other microphones in the living space, such as cellphone microphones or TV remote microphones. LampGuard uses a separate microphone placed apart from the SHAD to detect wake words like “Hey Google.” Once detected, it briefly disables its interference so the SHAD can wake and operate normally. LampGuard was evaluated using 30 spoken commands at close range (≤20 cm) and extended range (25–50 cm). At close range, the Word Error Rate (WER) increased from 7.64% to 96.56% (p < 0.001), demonstrating effective blocking of the SHAD’s microphone. At longer distances, the WER decreased significantly, confirming that LampGuard does not interfere with other microphones in the room.

Keywords

• AIoT
• Ultrasonic Jamming
• Physical-Layer Security
• Smart Home Privacy
• MEMS Microphones
• Distance-Dependent Protection
• Voice Assistant Security
• Nonlinear Signal Distortion

Akıllı Ev Asistanı Cihazlarından AIoT Odaklı Kullanıcı Merkezli Gizlilik Koruması

Abstract

Akıllı Ev Asistanı Cihazları (SHAD'ler) her zaman "Alexa", "Tamam Google" veya "Hey Siri" gibi uyanma kelimelerini dinler. Sonuç olarak, bu cihazlar kullanılmadıkları zamanlarda bile sürekli dinlerler, bu da kullanıcıların gizliliğini tehlikeye atabilir. Mevcut çözümler büyük ölçüde ağ protokollerini değiştirmeye veya politika değişikliklerini uygulamaya odaklanır, ancak bu yaklaşımların uygulanması yetkililere bağlıdır ve bu nedenle kullanıcılara erişilemez hale gelir. Diğer çözümler mekanik anahtarlar veya dokunsal arayüzlere dayanır, bu da kullanılabilirliği azaltır çünkü SHAD'lerin ana avantajı doğal ses tabanlı arayüzlerinde yatar. Bu makale, bu gizlilik sorununa yönelik AIoT (Yapay Zeka ve Nesnelerin İnterneti) destekli kullanıcı merkezli bir yaklaşım olan LampGuard'ı sunmaktadır. LampGuard, iki kullanıcı merkezli avantajla mevcut çözümlerden daha iyi performans gösterir: (i) uygulanması otoritelere bağlı değildir, kullanıcıya tam kontrol sağlar ve (ii) SHAD'ların varsayılan etkileşim modeliyle uyumlu bir sesli kullanıcı arayüzü kullanır. LampGuard, paylaşılan yaşam alanlarında yerel gizlilik koruması sağlayan bir masa lambası şeklinde bir ultrasonik parazit sistemi içerir ve ultrasonik paraziti tüm odaya değil, özellikle ev asistanı cihazının mikrofon alanına sınırlayarak gizlilik koruması sağlar. Böylece, LampGuard yalnızca SHAD'in mikrofonunu engeller ve cep telefonu mikrofonları veya TV uzaktan kumanda mikrofonları gibi yaşam alanındaki diğer mikrofonlara müdahale etmez. LampGuard, "Hey Google" gibi uyanma kelimelerini tespit etmek için SHAD'dan ayrı bir yere yerleştirilmiş ayrı bir mikrofon kullanır. Tespit edildikten sonra, SHAD'in uyanıp normal şekilde çalışabilmesi için kısa bir süreliğine müdahalesini devre dışı bırakır. LampGuard, yakın mesafede (≤20 cm) ve genişletilmiş mesafede (25–50 cm) 30 sözlü komut kullanılarak değerlendirildi. Yakın mesafede, Kelime Hata Oranı (WER) %7.64'ten %96.56'ya (p < 0.001) yükseldi, bu da SHAD'ın mikrofonunun etkili bir şekilde engellendiğini gösteriyor. Daha uzun mesafelerde, WER önemli ölçüde azaldı ve LampGuard'ın odadaki diğer mikrofonlarla etkileşime girmediğini doğruladı.

Keywords

• Ultrasonik Karıştırma
• Fiziksel Katman Güvenliği
• Akıllı Ev Gizliliği
• MEMS Mikrofonları
• Mesafeye Bağlı Koruma
• Sesli Asistan Güvenliği
• Doğrusal Olmayan Sinyal Bozulması
• AIoT

References

1. [1] J. S. Edu, J. M. Such, and G. Suarez-Tangil, “Smart Home Personal Assistants: A Security and Privacy Review,” ACM Comput Surv, vol. 53, no. 6, pp. 1–36, Dec. 2020, doi: 10.1145/3412383.
2. [2] P. Cheng and U. Roedig, “Personal Voice Assistant Security and Privacy—A Survey,” Proc. IEEE, vol. 110, no. 4, pp. 476–507, 2022, doi: 10.1109/JPROC.2022.3153167.
3. [3] G. Zhang, C. Yan, X. Ji, T. Zhang, T. Zhang, and W. Xu, “DolphinAttack: Inaudible Voice Commands,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, in CCS ’17. New York, NY, USA: Association for Computing Machinery, 2017, pp. 103–117. doi: 10.1145/3133956.3134052.
4. [4] N. Zhang, X. Mi, X. Feng, X. Wang, Y. Tian, and F. Qian, “Dangerous Skills: Understanding and Mitigating Security Risks of Voice-Controlled Third-Party Functions on Virtual Personal Assistant Systems,” in 2019 IEEE Symposium on Security and Privacy (SP), 2019, pp. 1381–1396. doi: 10.1109/SP.2019.00016.
5. [5] E. Alepis and C. Patsakis, “Monkey Says, Monkey Does: Security and Privacy on Voice Assistants,” IEEE Access, vol. 5, pp. 17841–17851, 2017, doi: 10.1109/ACCESS.2017.2747626.
6. [6] X. Lei, G.-H. Tu, A. X. Liu, C.-Y. Li, and T. Xie, “The Insecurity of Home Digital Voice Assistants - Vulnerabilities, Attacks and Countermeasures,” in 2018 IEEE Conference on Communications and Network Security (CNS), 2018, pp. 1–9. doi: 10.1109/CNS.2018.8433167.
7. [7] T. Ni, Y. Chen, W. Xu, L. Xue, and Q. Zhao, “XPorter: A Study of the Multi-Port Charger Security on Privacy Leakage and Voice Injection,” in Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, in ACM MobiCom ’23. New York, NY, USA: Association for Computing Machinery, pp. 1–15, 2023. doi: 10.1145/3570361.3613293.
8. [8] Y. Chen et al., “Wearable Microphone Jamming,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, in CHI ’20. New York, NY, USA: Association for Computing Machinery, 2020, pp. 1–12. doi: 10.1145/3313831.3376304.

9. [9] K. Sun, C. Chen, and X. Zhang, “‘Alexa, stop spying on me!’: speech privacy protection against voice assistants,” in Proceedings of the 18th Conference on Embedded Networked Sensor Systems, in SenSys ’20. New York, NY, USA: Association for Computing Machinery, 2020, pp. 298–311. doi: 10.1145/3384419.3430727.
10. [10] M. Gao, Y. Chen, Y. Liu, J. Xiong, J. Han, and K. Ren, “Cancelling Speech Signals for Speech Privacy Protection against Microphone Eavesdropping,” in Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, in ACM MobiCom ’23. New York, NY, USA: Association for Computing Machinery, pp. 1–16, 2023. doi: 10.1145/3570361.3592502.
11. [11] Z. Han, J. Ma, C. Xu, and G. Zhang, “UltraJam: Ultrasonic adaptive jammer based on nonlinearity effect of microphone circuits,” High-Confidence Computing, vol. 3, Art. no. 100129, 2023, doi: 10.1016/j.hcc.2023.100129.
12. [12] X. Qiao et al., “NUSGuard: Smart Device Anti-Eavesdropping Protection Based on Near-Ultrasonic Interference,” IEEE Trans. Inf. Forensics Secur., vol. 20, pp. 7839–7852, 2025, doi: 10.1109/TIFS.2025.3592558.
13. [13] A. Sabir, E. Lafontaine, and A. Das, “Hey Alexa, Who Am I Talking to?: Analyzing Users’ Perception and Awareness Regarding Third-party Alexa Skills,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, in CHI ’22. New York, NY, USA: Association for Computing Machinery, 2022. doi: 10.1145/3491102.3517510.
14. [14] I. Ahmad, T. Akter, Z. Buher, R. Farzan, A. Kapadia, and A. J. Lee, “Tangible Privacy for Smart Voice Assistants: Bystanders’ Perceptions of Physical Device Controls,” Proc. ACM Hum-Comput Interact, vol. 6, no. CSCW2, Nov. 2022, doi: 10.1145/3555089.
15. [15] E. Cho, S. S. Sundar, S. Abdullah, and N. Motalebi, “Will Deleting History Make Alexa More Trustworthy? Effects of Privacy and Content Customization on User Experience of Smart Speakers,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, in CHI ’20. New York, NY, USA: Association for Computing Machinery, 2020, pp. 1–13. doi: 10.1145/3313831.3376551.
16. [16] V. Z. Lin and S. Parkin, “Transferability of Privacy-related Behaviours to Shared Smart Home Assistant Devices,” in 2020 7th International Conference on Internet of Things: Systems, Management and Security (IOTSMS), 2020, pp. 1–8. doi: 10.1109/IOTSMS52051.2020.9340199.
17. [17] Y. Chen et al., “Big Brother is Listening: An Evaluation Framework on Ultrasonic Microphone Jammers,” in IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, 2022, pp. 1119–1128. doi: 10.1109/INFOCOM48880.2022.9796834.
18. [18] P. Walker and N. Saxena, “Evaluating the Effectiveness of Protection Jamming Devices in Mitigating Smart Speaker Eavesdropping Attacks Using Gaussian White Noise,” in Proceedings of the 37th Annual Computer Security Applications Conference, in ACSAC ’21. New York, NY, USA: Association for Computing Machinery, 2021, pp. 414–424. doi: 10.1145/3485832.3485896.
19. [19] A. Radford et al., “Robust speech recognition via large-scale weak supervision,” in Proc. 40th Int. Conf. Mach. Learn. (ICML), vol. 202, pp. 28492–28518, 2023.