Design of the Integrated Cognitive Perception Model for Developing Situation-Awareness of an Autonomous Smart Agent

Öz

This study explores the potential for autonomous agents to develop environmental awareness through perceptual attention. The main objective is to design a perception system architecture that mimics human-like perception, enabling smart agents to establish effective communication with humans and their surroundings. Overcoming the challenges of modeling the agent's environment and addressing the coordination issues of multi-modal perceptual stimuli is crucial for achieving this goal. Existing research falls short in meeting these requirements, prompting the introduction of a novel solution: a cognitive multi-modal integrated perception system. This computational framework incorporates fundamental feature extraction, recognition tasks, and spatial-temporal inference while facilitating the modeling of perceptual attention and awareness. To evaluate its performance, experimental tests and verification are conducted using a software framework integrated into a sandbox game platform. The model's effectiveness is assessed through a simple interaction scenario. The study's results demonstrate the successful validation of the proposed research questions.

Anahtar Kelimeler

• Autonomous smart agents
• Cognitive perception
• Attention modelling
• World model.

Kaynakça

1. [1] Yan, Z., Schreiberhuber, S., Halmetschlager, G., Duckett, T., Vincze, M., & Bellotto, N. (2020). Robot Perception of Static and Dynamic Objects with an Autonomous Floor Scrubber. arXiv preprint arXiv:2002.10158.
2. [2] Freud, E., Behrmann, M., & Snow, J. C. (2020). What Does Dorsal Cortex Contribute to Perception?. Open Mind, 1-18.
3. [3] Bear, M., Connors, B., & Paradiso, M. A. (2020). Neuroscience: Exploring the brain. Jones & Bartlett Learning, LLC.
4. [4] Chin, R., Chang, S. W., & Holmes, A. J. (2022). Beyond cortex: The evolution of the human brain. Psychological Review.
5. [5] Thiebaut de Schotten, M., & Forkel, S. J. (2022). The emergent properties of the connected brain. Science, 378(6619), 505-510.
6. [6] Li, B., Solanas, M. P., Marrazzo, G., Raman, R., Taubert, N., Giese, M., ... & de Gelder, B. (2023). A large-scale brain network of species-specific dynamic human body perception. Progress in Neurobiology, 221, 102398.
7. [7] Devia, C., Concha-Miranda, M., & Rodríguez, E. (2022). Bi-Stable Perception: Self-Coordinating Brain Regions to Make-Up the Mind. Frontiers in Neuroscience, 15, 805690.
8. [8] Taylor, A., Chan, D. M., & Riek, L. D. (2020). Robot-centric perception of human groups. ACM Transactions on Human-Robot Interaction (THRI), 9(3), 1-21.

9. [9] Ronchi, M. R. (2020). Vision for Social Robots: Human Perception and Pose Estimation (Doctoral dissertation, California Institute of Technology).
10. [10] Suzuki, R., Karim, A., Xia, T., Hedayati, H., & Marquardt, N. (2022, April). Augmented reality and robotics: A survey and taxonomy for ar-enhanced human-robot interaction and robotic interfaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (pp. 1-33).
11. [11] Farouk, M. (2022). Studying Human Robot Interaction and Its Characteristics. International Journal of Computations, Information and Manufacturing (IJCIM), 2(1).
12. [12] Müller, S., Wengefeld, T., Trinh, T. Q., Aganian, D., Eisenbach, M., & Gross, H. M. (2020). A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots. Sensors, 20(3), 722.
13. [13] Russo, C., Madani, K., & Rinaldi, A. M. (2020). Knowledge Acquisition and Design Using Semantics and Perception: A Case Study for Autonomous Robots. Neural Processing Letters, 1-16.
14. [14] Cangelosi, A., & Asada, M. (Eds.). (2022). Cognitive robotics. MIT Press.
15. [15] Iosifidis, A., & Tefas, A. (Eds.). (2022). Deep Learning for Robot Perception and Cognition. Academic Press.
16. [16] Lee, C. Y., Lee, H., Hwang, I., & Zhang, B. T. (2020, June). Visual Perception Framework for an Intelligent Mobile Robot. In 2020 17th International Conference on Ubiquitous Robots (UR) (pp. 612-616). IEEE.
17. [17] Mazzola, C., Aroyo, A. M., Rea, F., & Sciutti, A. (2020, March). Interacting with a Social Robot Affects Visual Perception of Space. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 549-557).
18. [18] Mariacarla, B. Special Issue on Behavior Adaptation, Interaction, and Artificial Perception for Assistive Robotics.
19. [19] Sanneman, L., & Shah, J. A. (2020, May). A Situation Awareness-Based Framework for Design and Evaluation of Explainable AI. In International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems (pp. 94-110). Springer, Cham.
20. [20] Kridalukmana, R., Lu, H. Y., & Naderpour, M. (2020). A supportive situation awareness model for human-autonomy teaming in collaborative driving. Theoretical Issues in Ergonomics Science, 1-26.
21. [21] Tropmann-Frick, M., & Clemen, T. (2020). Towards Enhancing of Situational Awareness for Cognitive Software Agents. In Modellierung (Companion) (pp. 178-184).
22. [22] Gu, R., Jensen, P. G., Poulsen, D. B., Seceleanu, C., Enoiu, E., & Lundqvist, K. (2022). Verifiable strategy synthesis for multiple autonomous agents: a scalable approach. International Journal on Software Tools for Technology Transfer, 24(3), 395-414.
23. [23] Sakai, T., & Nagai, T. (2022). Explainable autonomous robots: A survey and perspective. Advanced Robotics, 36(5-6), 219-238.
24. [24] Inceoglu, A., Koc, C., Kanat, B. O., Ersen, M., & Sariel, S. (2018). Continuous visual world modeling for autonomous robot manipulation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(1), 192-205.
25. [25] Kim, K., Sano, M., De Freitas, J., Haber, N., & Yamins, D. (2020). Active World Model Learning in Agent-rich Environments with Progress Curiosity. In Proceedings of the International Conference on Machine Learning (Vol. 8).
26. [26] Kim, K., Sano, M., De Freitas, J., Haber, N., & Yamins, D. (2020). Active World Model Learning with Progress Curiosity. arXiv preprint arXiv:2007.07853.
27. [27] Riedelbauch, D., & Henrich, D. (2019, May). Exploiting a Human-Aware World Model for Dynamic Task Allocation in Flexible Human-Robot Teams. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 6511-6517). IEEE.
28. [28] Rosinol, A., Gupta, A., Abate, M., Shi, J., & Carlone, L. (2020). 3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans. arXiv preprint arXiv:2002.06289.
29. [29] Venkataraman, A., Griffin, B., & Corso, J. J. (2019). Kinematically-Informed Interactive Perception: Robot-Generated 3D Models for Classification. arXiv preprint arXiv:1901.05580.
30. [30] Persson, A., Dos Martires, P. Z., De Raedt, L., & Loutfi, A. (2019). Semantic relational object tracking. IEEE Transactions on Cognitive and Developmental Systems, 12(1), 84-97.
31. [31] Zuidberg Dos Martires, P., Kumar, N., Persson, A., Loutfi, A., & De Raedt, L. (2020). Symbolic Learning and Reasoning with Noisy Data for Probabilistic Anchoring. arXiv, arXiv-2002.
32. [32] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436.
33. [33] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press.
34. [34] LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10), 1995.
35. [35] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
36. [36] Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015, April). Convolutional, long short-term memory, fully connected deep neural networks. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 4580-4584). IEEE.
37. [37] Chiu, H. P., Samarasekera, S., Kumar, R., Matei, B. C., & Ramamurthy, B. (2020). U.S. Patent Application No. 16/523,313.
38. [38] Wang, S., Wu, T., & Vorobeychik, Y. (2020). Towards Robust Sensor Fusion in Visual Perception. arXiv preprint arXiv:2006.13192.
39. [39] Xue, T., Wang, W., Ma, J., Liu, W., Pan, Z., & Han, M. (2020). Progress and prospects of multi-modal fusion methods in physical human-robot interaction: A Review. IEEE Sensors Journal.
40. [40] Guss, W. H., Codel, C., Hofmann, K., Houghton, B., Kuno, N., Milani, S., ... & Wang, P. (2019). Neurips 2019 competition: The minerl competition on sample efficient reinforcement learning using human priors. arXiv preprint arXiv:1904.10079.
41. [41] MineRL: A Large-Scale Dataset of Minecraft Demonstrations
42. [42] Frazier, S., & Riedl, M. (2019, October). Improving deep reinforcement learning in Minecraft with action advice. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (Vol. 15, No. 1, pp. 146-152).
43. [43] Aluru, K. C., Tellex, S., Oberlin, J., & MacGlashan, J. (2015, September). Minecraft as an experimental world for AI in robotics. In the 2015 AAAI fall symposium series.
44. [44] Angulo, E., Lahuerta, X., & Roca, O. (2020). Reinforcement Learning in Minecraft.
45. [45] Eraldemir, S. G., Arslan, M. T., & Yildirim, E. (2018). Investigation of feature selection algorithms on A cognitive task classification: a comparison study. Balkan Journal of Electrical and Computer Engineering, 6(2), 99-104.
46. [46] Akinci, T. Ç., & Martinez-Morales, A. A. (2022). Cognitive Based Electric Power Management System. Balkan Journal of Electrical and Computer Engineering, 10(1), 85-90.