IoT uygulamalarında çok etmenli sistemlerin (MAS) birlikte çalışabilirliğinin rolü: Akıllı üretim sistemlerindeki son gelişmeler üzerine bir araştırma

Öz

Çok Etmenli Sistemler (MAS), Nesnelerin İnterneti (IoT) uygulamaları içerisinde, akıllı nesnelerin etmenler (agent) olarak tasarlanması aracılığı ile birden çok akıllı cihazdan bilgilerin algılamasını, toplamasını, paylaşmasını, pekiştirilmesini ve revize edilmesini mümkün kılmaktadır. Çok etmenli sistemler ve ilgili araçlar, sistemlerin optimizasyonunda kullanılan paradigmalar için önemli değişikliklere katkıda bulunmakla birlikte, MAS hakkındaki mevcut literatür, IoT cihazlarının gelişmiş işbirliğini açıklayan modern dağıtılmış hesaplama yöntemlerini açıklamak için yeterli düzeyde değildir. Bu araştırma, IoT içerisinde yer alan akıllı cihazların, MAS teknolojisi ile birlikte çalışabilirliği hakkında kapsamlı bir araştırma sunmaktadır. Bu kapsamda, önce etmen tabanlı sistemler ve IoT teknolojisi hakkında genel bir bilgilendirilme sunularak, çok etmenli sistemler içerisinde birlikte çalışabilirliğin önemi ve üretim sistemleri özelinde çoklu etmen teknolojisinin, sensör verilerini yakalamadan karar vermeye kadar çeşitli amaçlarla nasıl kullanılabileceğini açıklanmıştır. Çok etmenli sistemlerde birlikte çalışabilirlik için geliştirilen ve üreticiden bağımsız olarak birbirini anlayabilen iki ana dil hakkında bilgilendirme sunulmuştur. MAS'in IoT tabanlı kablosuz sensör ağında (WSN'ler) veri toplama ve entegrasyonu açıklanmıştır. Akıllı üretim sistemlerinin tasarımında, MAS'in geliştirmeye açık alanları ve gelecekteki trendlerinde IoT uygulamasını ele almak, bu makalenin bir diğer katkısı olarak sunulmuştur. Araştırmanın, IoT ve MAS üzeride çalışan araştırmacılar, tasarımcılar ve yöneticiler için bir yol gösterici olması beklenmektedir.

Anahtar Kelimeler

• IoT
• Çoklu-Etmen Sistemleri
• Birlikte Çalışabilirlik
• İşbirliği
• Üretim Sistemleri

The role of interoperability of Multi-Agent Systems for IoT applications: A survey on recent advances in smart manufacturing systems

Öz

Multiple Agent Systems (MAS) makes it possible to detect, collect, share, consolidate and revise information from multiple smart devices by designing smart objects as agents within the Internet of Things (IoT) technology. While multi-agent systems and related tools contribute to significant changes to the paradigms used in optimizing systems, the current literature on MAS is not sufficient to describe modern distributed computing methods that explain the advanced collaboration of IoT devices. This research provides comprehensive research on the interoperability of smart devices in IoT with MAS technology. In this context, first, a piece of general information about agent-based systems and IoT technology was presented. The importance of interoperability in multi-agent technology and how multi-agent technology can be used for various purposes, from capturing sensor data to decision making, are explained. Information on two main languages that are developed for interoperability in multi-agent systems and that can understand each other independently of the manufacturer are presented. Data collection and integration in MAS's IoT-based wireless sensor network (WSNs) is discussed. Addressing the IoT application in the design of smart manufacturing systems, areas for improvement and future trends of MAS is presented as another contribution of this article. The research is expected to guide research, designers, and managers working on IoT and MAS.

Anahtar Kelimeler

• IoT
• Multi-Agent System
• Interoperability
• Cooperation
• Manufacturing Systems

Kaynakça

1. [1] Popkova, E.G.; Egorova, E.N.; Popova, E.; Pozdnyakova, U.A. The model of state management of economy on the basis of the internet of things. In Ubiquitous Computing and the Internet of Things: Prerequisites for the Development of ICT; Springer, 2019; pp. 1137–1144.
2. [2] Kabalci, Y.; Kabalci, E.; Padmanaban, S.; Holm-Nielsen, J.B.; Blaabjerg, F. Internet of Things applications as energy internet in Smart Grids and Smart Environments. Electronics 2019, 8, 972.
3. [3] Calis, B. Agent-Based Simulation Model for Profit Maximization. Journal of Management and Information Science 2016, 4, 26–33.
4. [4] Rocha, J.; Boavida-Portugal, I.; Gomes, E. Introductory Chapter: Multi-Agent Systems. In Multi-Agent Systems; IntechOpen, 2017.
5. [5] Januário, F.E.P.; Leitão, J.; Cardoso, A.; Gil, P. Resilience Enhancement in Cyber-Physical Systems: A Multiagent-Based Framework. In Multi-agent Systems; IntechOpen, 2017.
6. [6] Konak, A.; Cabrera-Mora, F.; Kulturel-Konak, S. Agent-based simulations for multi-robot systems exploration of tree-like environments. 2018 IEEE International Conference on Real-time Computing and Robotics (RCAR). IEEE, 2018, pp. 172–176.
7. [7] Calegari, R.; Denti, E.; Mariani, S.; Omicini, A. Logic programming as a service in multi-agent systems for the Internet of Things. International Journal of Grid and Utility Computing 2019, 10, 344–360.
8. [8] Uslu, B. Ç. Capability Model and Competence Measuring for Smart Hospital System: An Analysis for Turkey. International Journal of Health Services Research and Policy 2020, 5, 41–50.

9. [9] Pantoja, C.E.; Soares, H.D.; Viterbo, J.; El Fallah-Seghrouchni, A. An Architecture for the Development of Ambient Intelligence Systems Managed by Embedded Agents. SEKE, 2018, pp. 215–214.
10. [10] Shen, S.; Han, Y.; Wang, X.; Wang, Y. Computation offloading with multiple agents in edge-computing–supported IoT. ACM Transactions on Sensor Networks (TOSN) 2019, 16, 1–27.
11. [11] Pico-Valencia, P.; Holgado-Terriza, J.A. Agentification of the Internet of Things: A systematic literature review. International Journal of Distributed Sensor Networks 2018, 14, 1550147718805945.
12. [12] Dorri, A.; Kanhere, S.S.; Jurdak, R. Multi-agent systems: A survey. Ieee Access 2018, 6, 28573–28593.
13. [13] Calvaresi, D.; Marinoni, M.; Sturm, A.; Schumacher, M.; Buttazzo, G. The challenge of real-time multi-agent systems for enabling IoT and CPS. Proceedings of the international conference on web intelligence, 2017, pp. 356–364.
14. [14] Muscalagiu, I.; Popa, H.; Vidal, J. Large Scale Multi-Agent-Based Simulation using NetLogo for implementation and evaluation of the distributed constraints. proceedings of IJCAI DCR 2013 (23rd International Joint Conference on Artificial IntelligenceWorkshop on Distributed Constraint Reasoning, 2013.
15. [15] Queiroz, J.; Leitão, P.; Barbosa, J.; Oliveira, E. Agent-Based Approach for Decentralized Data Analysis in Industrial Cyber-Physical Systems. International Conference on Industrial Applications of Holonic and Multi-Agent Systems. Springer, 2019, pp. 130–144.
16. [16] Xie, J.; Liu, C.C. Multi-agent systems and their applications. Journal of International Council on Electrical Engineering 2017, 7, 188–197.
17. [17] Novikova, G.; Azofeifa, E. Domain theory verification using multi-agent systems. Procedia Computer Science 2017, 103, 120–125.
18. [18] Rathore, S.; Kwon, B.W.; Park, J.H. BlockSecIoTNet: Blockchain-based decentralized security architecture for IoT network. Journal of Network and Computer Applications 2019, 143, 167–177.
19. [19] Azofeifa, E.J. A Multiagent Approach for Cost Estimating in Big Data Analytics. Information and Telecommunication Technologies and Mathematical Modeling of High-Tech Systems 2019 (ITTMM 2019), 2019, pp. 355–360.
20. [20] Azofeifa, E.; Novikova, G.M. Development of a crowdsourcing multiagent system for knowledge extraction. Artificial Intelligence and Decision Making 2020, pp. 40–48.
21. [21] Bakliwal, K.; Dhada, M.H.; Palau, A.S.; Parlikad, A.K.; Lad, B.K. A multi agent system architecture to implement collaborative learning for social industrial assets. IFAC-PapersOnLine 2018, 51, 1237–1242.
22. [22] Palau, A.S.; Dhada, M.H.; Bakliwal, K.; Parlikad, A.K. An Industrial Multi Agent System for real-time distributed collaborative prognostics. Engineering Applications of Artificial Intelligence 2019, 85, 590–606.
23. [23] Palau, A.S.; Dhada, M.H.; Parlikad, A.K. Multi-agent system architectures for collaborative prognostics. Journal of Intelligent Manufacturing 2019, 30, 2999–3013.
24. [24] Fasanotti, L. A distributed intelligent maintenance system based on artificial immune approach and multi-agent systems. 2014 12th IEEE international conference on industrial informatics (INDIN). IEEE, 2014, pp. 783–786.
25. [25] Kong, J.; Lomuscio, A. Symbolic model checking multi-agent systems against CTL* K specifications. Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, 2017, pp. 114–122.
26. [26] Iannino, V.; Vannocci, M.; Vannucci, M.; Colla, V.; Neuer, M. A multi-agent approach for the self-optimization of steel production. Int. J. Simul. Syst. Sci. Technol 2018, 19, 1–20.
27. [27] Zheng, G.; Kovalenko, I.; Barton, K.; Tilbury, D. Integrating Human Operators into Agent-based Manufacturing Systems: A Table-top Demonstration. Procedia manufacturing 2018, 17, 326–333.
28. [28] Kubo, R.H.; Asato, O.L.; dos Santos, G.A.; Nakamoto, F.Y. Modeling of allocation control system of multifunctional resources for manufacturing systems. 2016 12th IEEE International Conference on Industry Applications (INDUSCON). IEEE, 2016, pp. 1–8.
29. [29] Pavlatos, C.; Vita, V. Linguistic representation of power system signals. Electricity Distribution. Springer, 2016, pp. 285–295.
30. [30] Leusin, M.E.; Kück, M.; Frazzon, E.M.; Maldonado, M.U.; Freitag, M. Potential of a multi-agent system approach for production control in smart factories. IFAC-PapersOnLine 2018, 51, 1459–1464.
31. [31] Ciortea, E.; Tulbure, A.; Hu¸tanu, C.t. Multi-agent for manufacturing systems optimization. IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2016, Vol. 145, p. 022007.
32. [32] Coroiu, A.M. Emotional intelligent agent in decision-making process with implications in manufacturing. International Journal of Modern Manufacturing Technologies 2015, 7, 44.
33. [33] de Hoz Diego, J.D.; Saldana, J.; Fernández-Navajas, J.; Ruiz-Mas, J. Decoupling Security From Applications in CoAP-Based IoT Devices. IEEE Internet of Things Journal 2019, 7, 467–476.
34. [34] Diego, J.D.D.H.; Saldana, J.; Fernández-Navajas, J.; Ruiz-Mas, J. IOTSafe, decoupling security from applications for a safer IOT. IEEE Access 2019, 7, 29942–29962.
35. [35] Cagnin, R.L.; Guilherme, I.R.; Queiroz, J.; Paulo, B.; Neto, M.F. A multi-agent system approach for management of industrial IoT devices in manufacturing processes. 2018 IEEE 16th International Conference on Industrial Informatics (INDIN). Ieee, 2018, pp. 31–36.
36. [36] Javed, F.; Afzal, M.K.; Sharif, M.; Kim, B.S. Internet of Things (IoT) operating systems support, networking technologies, applications, and challenges: A comparative review. IEEE Communications Surveys & Tutorials 2018, 20, 2062–2100.
37. [37] Luo, T.; Xu, Z.; Jin, X.; Jia, Y.; Ouyang, X. Iotcandyjar: Towards an intelligent-interaction honeypot for iot devices. Black Hat 2017, pp. 1–11.
38. [38] Vieira, G.; Barbosa, J.; Leitão, P.; Sakurada, L. Low-Cost Industrial Controller based on the Raspberry Pi Platform. 2020 IEEE International Conference on Industrial Technology (ICIT). IEEE, 2020, pp. 292–297.
39. [39] Sycara, K.; Paolucci, M.; Van Velsen, M.; Giampapa, J. The retsina mas infrastructure. Autonomous agents and multi-agent systems 2003, 7, 29–48.
40. [40] Cai, Y.; Genovese, A.; Piuri, V.; Scotti, F.; Siegel, M. IoT-based architectures for sensing and local data processing in ambient intelligence: research and industrial trends. 2019 IEEE International Instrumentation and Measurement Technology Conference (I2MTC). IEEE, 2019, pp. 1–6.
41. [41] Mas, M.; Riera, C.; others. Systematization of the psychomotor activity and cognitive development. Educational Psychology 2017, 24, 38–41.
42. [42] Omicini, A.; Calegari, R. Injecting (micro) intelligence in the IoT: Logic-based approaches for (M) MAS. International Workshop on Massively Multiagent Systems. Springer, 2018, pp. 21–35.
43. [43] Korzun, D.; Balandina, E.; Kashevnik, A.; Balandin, S.; Viola, F. Ambient Intelligence Services in IoT Environments: Emerging Research and Opportunities: Emerging Research and Opportunities; IGI Global, 2019.
44. [44] Mas, J.; Panadero, T.; Botella, G.; Del Barrio, A.A.; García, C. CNN Inference acceleration using low-power devices for human monitoring and security scenarios. Computers & Electrical Engineering 2020, 88, 106859.
45. [45] Wang, B.; Tao, F.; Fang, X.; Liu, C.; Liu, Y.; Freiheit, T. Smart Manufacturing and Intelligent Manufacturing: A Comparative. Engineering 2020.
46. [46] Alzubaidi, L.; Al-Shamma, O.; Fadhel, M.A.; Farhan, L.; Zhang, J.; Duan, Y. Optimizing the performance of breast cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics 2020, 9, 445.
47. [47] Mostafa, A.E.; Gadallah, Y. Uniqueness-based resource allocation for m2m communications in narrowband iot networks. 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall). IEEE, 2017, pp. 1–5.
48. [48] Javaid, N.; Sher, A.; Nasir, H.; Guizani, N. Intelligence in IoT-based 5G networks: Opportunities and challenges. IEEE Communications Magazine 2018, 56, 94–100.
49. [49] Uslu, B.Ç.; Okay, E.; Dursun, E. Analysis of factors affecting IoT-based smart hospital design. Journal of Cloud Computing 2020, 9, 1–23.
50. [50] Nasralla, M.M.; García-Magariño, I.; Lloret, J. Defenses Against Perception-Layer Attacks on IoT Smart Furniture for Impaired People. IEEE Access 2020, 8, 119795–119805.
51. [51] Ayala, I.; Amor, M.; Horcas, J.M.; Fuentes, L. A goal-driven software product line approach for evolving multi-agent systems in the Internet of Things. Knowledge-Based Systems 2019, 184, 104883.
52. [52] H De La Iglesia, D.; Villarrubia González, G.; Sales Mendes, A.; Jiménez-Bravo, D.M.; L Barriuso, A. Architecture to embed software agents in resource constrained internet of things devices. Sensors 2019, 19, 100.
53. [53] Jiménez-Bravo, D.; others. Architecture to Embed Software Agents in Resource Constrained Internet of Things Devices. Sensors (Basel, Switzerland) 2018, 19.
54. [54] Khan, M.W.; Wang, J. The research on multi-agent system for microgrid control and optimization. Renewable and Sustainable Energy Reviews 2017, 80, 1399–1411.
55. [55] Abbas, A.; Siddiqui, I.F.; Lee, S.U.J.; Bashir, A.K. Binary pattern for nested cardinality constraints for software product line of IoT-based feature models. IEEE Access 2017, 5, 3971–3980.
56. [56] Olaru, A.; Sorici, A.; Florea, A.M. A flexible and lightweight agent deployment architecture. 2019 22nd International Conference on Control Systems and Computer Science (CSCS). IEEE, 2019, pp. 251–258.
57. [57] Mehmood, F.; Ullah, I.; Ahmad, S.; Kim, D.H. A Novel Approach towards the Design and Implementation of Virtual Network Based on Controller in Future IoT Applications. Electronics 2020, 9, 604.
58. [58] Farhan, L.; Shukur, S.T.; Alissa, A.E.; Alrweg, M.; Raza, U.; Kharel, R. A survey on the challenges and opportunities of the Internet of Things (IoT). 2017 Eleventh International Conference on Sensing Technology (ICST). IEEE, 2017, pp. 1–5.
59. [59] Gan, K.S.; Chin, K.O.; Anthony, P.; Hamdan, A.R. A FIPA-ACL Ontology in enhancing interoperability multi-agent communication. International Conference on Computational Science and Technology. Springer, 2017, pp. 151–160.
60. [60] Alaca, O.F.; Tezel, B.T.; Challenger, M.; Goulão, M.; Amaral, V.; Kardas, G. AgentDSM-Eval: A framework for the evaluation of domain-specific modeling languages for multi-agent systems. Computer Standards & Interfaces 2021, 76, 103513.
61. [61] Asici, T.Z.; Tezel, B.T.; Kardas, G. On the use of the analytic hierarchy process in the evaluation of domain-specific modeling languages for multi-agent systems. Journal of Computer Languages 2021, 62, 101020.
62. [62] Mascardi, V.; Weyns, D.; Ricci, A.; Earle, C.B.; Casals, A.; Challenger, M.; Chopra, A.; Ciortea, A.; Dennis, L.A.; Díaz, Á.F.; others. Engineering multi-agent systems: State of affairs and the road ahead. ACM SIGSOFT Software Engineering Notes 2019, 44, 18–28.
63. [63] Kardas, G.; Bircan, E.; Challenger, M. Supporting the platform extensibility for the model-driven development of agent systems by the interoperability between domain-specific modeling languages of multi-agent systems. Computer Science and Information Systems 2017, 14, 875–912.
64. [64] Weyns, D.; Mascardi, V.; Ricci, A. Engineering multi-agent systems. Proceedings of the 6th International Workshop, EMAS. Springer, 2018.
65. [65] Challenger, M.; Tezel, B.T.; Amaral, V.; Goulão, M.; Kardas, G. Agent-based cyber-physical system development with SEA_ML++. In Multi-Paradigm Modelling Approaches for Cyber-Physical Systems; Tekinerdogan, B.; Blouin, D.; Vangheluwe, H.; Goulão, M.; Carreira, P.; Amaral, V., Eds.; Academic Press, 2021.
66. [66] HoseinDoost, S.; Adamzadeh, T.; Zamani, B.; Fatemi, A. A model-driven framework for developing multi-agent systems in emergency response environments. Software & Systems Modeling 2019, 18, 1985–2012.
67. [67] Bircan, E.; Challenger, M.; Kardas, G. Interoperability of MAS DSMLs via horizontal model transformations. 2016 Federated Conference on Computer Science and Information Systems (FedCSIS). IEEE, 2016, pp. 1555–1564.
68. [68] Tezel, B.T.; Kardas, G. A Conceptual Generic Framework to Debugging in the Domain-Specific Modeling Languages for MultiAgent Systems. 8th Symposium on Languages, Applications and Technologies (SLATE 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
69. [69] Silva, J.; Barišic, A.; Amaral, V.; Goulão, M.; Tezel, B.T.; Alaca, O.F.; Challenger, M.; Kardas, G. Comparing the Usability of two Multi-Agents Systems DSLs: SEA_ML++ and DSML4MAS Study Design. proc. 3rd International Workshop on Human Factors in Modeling (HuFaMo 2018), held in ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018), 2018, pp. 770–777.
70. [70] Fatehah, M.; Mezhuyev, V.; Al-Emran, M. A Systematic Review of Metamodelling in Software Engineering. Recent Advances in Intelligent Systems and Smart Applications 2021, pp. 3–27.
71. [71] Miranda, T.R.B.d. Software Language Engineering: Interaction and Usability Modeling of Language Editors. PhD thesis, Universidade Nova de Lisboa, 2017. [72] Abrahao, S.; Goulao, M.; Heymans, P.; Le Pallec, X.; Renaux, E. Third International Workshop on Human Factors in Modeling (HuFaMo 2018)-Preface. CEUR Workshop Proceedings, 2018, Vol. 2245, pp. 728–729.
72. [73] KARLOV, D.N.; POLOZHENTSEVA, Y.S.; KREMLEVA, L.V.; KALIMYLLIN, D.D. The implementation of the IoT concept in the post-industrial economy. Revista ESPACIOS 2019, 40.
73. [74] Zemzem, W.; Tagina, M. Cooperative multi-agent systems using distributed reinforcement learning techniques. Procedia Computer Science 2018, 126, 517–526. [75] Zhou, P.; Shen, H. Multi-agent cooperation by reinforcement learning with teammate modeling and reward allotment. 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). IEEE, 2011, Vol. 2, pp. 1316–1319.
74. [76] Zemzem, W.; Tagina, M. Cooperative multi-agent reinforcement learning in a large stationary environment. 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS). IEEE, 2017, pp. 365–371.
75. [77] Calis, B. Cooperation & Coordination of Distributed Intelligent Agents for Manufacturıng Systems. PhD thesis, Marmara University, Istanbul, 2015.
76. [78] Zemzem, W.; Hosni, I. A New Distributed Reinforcement Learning Approach for Multiagent Cooperation Using Team-mate Modeling and Joint Action Generalization. Adv. Sci. Technol. Eng. Syst. J 2020, 5, 1–12.
77. [79] Sandita, A.V.; Popirlan, C.I. Developing a multi-agent system in JADE for Information management in educational competence domains. Procedia Economics and Finance 2015, 23, 478–486.
78. [80] DiLeo, J.; Jacobs, T.; DeLoach, S. Integrating ontologies into multiagent systems engineering. Technical report, Air Univ Maxwell AFB Al Center For Aerospace Doctrine Research and Education, 2006.
79. [81] ShaoPeng, T.; Jun, Z. A research on multi agent modeling language. Procedia Engineering 2011, 15, 1842–1847.
80. [82] Zemzem, W.; Tagina, M. Cooperative multi-agent learning in a large dynamic environment. International Conference on Modeling Decisions for Artificial Intelligence. Springer, 2015, pp. 155–166.
81. [83] Julian, V.; Botti, V. Multi-agent systems, 2019.
82. [84] Leitao, P. Multi-agent systems in industry: Current trends & future challenges. In Beyond Artificial Intelligence; Springer, 2013; pp. 197–201.
83. [85] Ioni¸ta˘, L.; Ioni¸ta˘, I. NM-MAS: A multi-agent system for network management in oil industry. 2014 RoEduNet Conference 13th Edition: Networking in Education and Research Joint Event RENAM 8th Conference. IEEE, 2014, pp. 1–6.
84. [86] Upadhyay, D. A review paper on Multi agent base intelligent manufacturing system. Int. J. Adv. Eng. Res. Sci 2014, 1.
85. [87] Ahmed, M.; Umar, A.; Bute, M. Multi-agent based architectural framework for the prevention and control of oil pipeline vandalism. 2017 International Conference on Computing Networking and Informatics (ICCNI). IEEE, 2017, pp. 1–8.
86. [88] Gupta, J.K.; Egorov, M.; Kochenderfer, M. Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems. Springer, 2017, pp. 66–83.
87. [89] Torreno, A.; Onaindia, E.; Komenda, A.; Štolba, M. Cooperative multi-agent planning: a survey. ACM Computing Surveys (CSUR) 2017, 50, 1–32.
88. [90] Ismail, Z.H.; Sariff, N. A survey and analysis of cooperative multi-agent robot systems: Challenges and directions. In Applications of Mobile Robots; IntechOpen, 2018; pp. 8–14.
89. [91] OroojlooyJadid, A.; Hajinezhad, D. A review of cooperative multi-agent deep reinforcement learning. arXiv preprint arXiv:1908.03963 2019.
90. [92] Khan, M.; others. Speeding up GDL-based distributed constraint optimization algorithms in cooperative multi-agent systems. PhD thesis, University of Southampton, 2018.
91. [93] Egger, G.; Chaltsev, D.; Giusti, A.; Matt, D.T. A deployment-friendly decentralized scheduling approach for cooperative multi-agent systems in production systems. Procedia Manufacturing 2020, 52, 127–132.
92. [94] Gottesman, O.; Johansson, F.; Meier, J.; Dent, J.; Lee, D.; Srinivasan, S.; Zhang, L.; Ding, Y.; Wihl, D.; Peng, X.; others. Evaluating reinforcement learning algorithms in observational health settings. arXiv preprint arXiv:1805.12298 2018.
93. [95] Nguyen, T.T.; Nguyen, N.D.; Nahavandi, S. Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications. IEEE transactions on cybernetics 2020, 50, 3826–3839.
94. [96] Kuhnle, A.; Kaiser, J.P.; Theiß, F.; Stricker, N.; Lanza, G. Designing an adaptive production control system using reinforcement learning. Journal of Intelligent Manufacturing 2021, 32, 855–876.
95. [97] Du, W.; Ding, S. A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications. Artificial Intelligence Review 2020, pp. 1–24.
96. [98] Lowe, R.; Wu, Y.; Tamar, A.; Harb, J.; Abbeel, P.; Mordatch, I. Multi-agent actor-critic for mixed cooperative-competitive environments. arXiv preprint arXiv:1706.02275 2017.
97. [99] Park, Y.J.; Lee, Y.J.; Kim, S.B. Cooperative multi-agent reinforcement learning with approximate model learning. IEEE Access 2020, 8, 125389–125400 . [100] Hu, J.; Zhang, H.; Song, L.; Schober, R.; Poor, H.V. Cooperative internet of UAVs: Distributed trajectory design by multi-agent deep reinforcement learning. IEEE Transactions on Communications 2020, 68, 6807–6821.
98. [101] Zhou, M.; Jin, J.; Zhang, W.; Qin, Z.; Jiao, Y.; Wang, C.; Wu, G.; Yu, Y.; Ye, J. Multi-agent reinforcement learning for orderdispatching via order-vehicle distribution matching. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 2645–2653.
99. [102] Plinere, D.; Aleksejeva, L. Production scheduling in agent-based supply chain for manufacturing efficiency improvement. Procedia Computer Science 2019, 149, 36–43.
100. [103] Wang, Y.; Liu, H.; Zheng, W.; Xia, Y.; Li, Y.; Chen, P.; Guo, K.; Xie, H. Multi-objective workflow scheduling with deep-Q-networkbased multi-agent reinforcement learning. IEEE Access 2019, 7, 39974–39982.
101. [104] May, M.C.; Kiefer, L.; Kuhnle, A.; Stricker, N.; Lanza, G. Decentralized Multi-Agent Production Control through Economic Model Bidding for Matrix Production Systems. Procedia CIRP 2021, 96, 3–8.
102. [105] Glavic, M. Agents and multi-agent systems: a short introduction for power engineers. Technical report, Electrical Engineering and Computer Science Department, University of Liege, 2006.
103. [106] Salazar, L.A.C.; Mayer, F.; Schütz, D.; Vogel-Heuser, B. Platform independent multi-agent system for robust networks of production systems. IFAC-PapersOnLine 2018, 51, 1261–1268.
104. [107] Peng, C. Optimization Based Control for Multi-agent System with Interaction. PhD thesis, UC Berkeley, 2019.
105. [108] Zhang, H.; Ji, H.; Ye, Z.; Senping, T.; Zhang, H.; Li, C. Impulsive consensus of multi-agent systems with stochastically switching topologies. Nonlinear Analysis: Hybrid Systems 2017, 26, 212–224.
106. [109] Lu, A.Y.; Yang, G.H. Distributed consensus control for multi-agent systems under denial-of-service. Information Sciences 2018, 439, 95–107.
107. [110] Sampaio, R.F.; Melo, L.S.; Leão, R.P.; Barroso, G.C.; Bezerra, J.R. Automatic restoration system for power distribution networks based on multi-agent systems. IET Generation, Transmission & Distribution 2017, 11, 475–484.
108. [111] Sujil, A.; Agarwal, S.K.; Kumar, R. Centralized multi-agent implementation for securing critical loads in PV based microgrid. Journal of Modern Power Systems and Clean Energy 2014, 2, 77–86.
109. [112] Zhang, Z.; Yang, J.; Zha, H. Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization. arXiv preprint arXiv:1909.10651 2019.
110. [113] Wang, S.; Zhang, P.; Fan, Y. Centralized event-triggered control of multi-agent systems with dynamic triggering mechanisms. The 27th Chinese Control and Decision Conference (2015 CCDC). IEEE, 2015, pp. 2183–2187.
111. [114] Simões, D.; Lau, N.; Reis, L.P. Multi-agent actor centralized-critic with communication. Neurocomputing 2020, 390, 40–56.
112. [115] Sayed, A.S.; Ammar, H.H.; Shalaby, R. Centralized Multi-agent Mobile Robots SLAM and Navigation for COVID-19 Field Hospitals. 2020 2nd Novel Intelligent and Leading Emerging Sciences Conference (NILES). IEEE, 2020, pp. 444–449.
113. [116] Ackermann, J.; Gabler, V.; Osa, T.; Sugiyama, M. Reducing overestimation bias in multi-agent domains using double centralized critics. arXiv preprint arXiv:1910.01465 2019.
114. [117] Khan, A.; Zhang, C.; Lee, D.D.; Kumar, V.; Ribeiro, A. Scalable centralized deep multi-agent reinforcement learning via policy gradients. arXiv preprint arXiv:1805.08776 2018.
115. [118] Dobbe, R.; Fridovich-Keil, D.; Tomlin, C. Fully decentralized policies for multi-agent systems: An information theoretic approach. arXiv preprint arXiv:1707.06334 2017.
116. [119] Mhamdi, E.M.E.; Guerraoui, R.; Hendrikx, H.; Maurer, A. Dynamic safe interruptibility for decentralized multi-agent reinforcement learning. arXiv preprint arXiv:1704.02882 2017.
117. [120] Ponomarev, S.; Voronkov, A. Multi-agent systems and decentralized artificial superintelligence. arXiv preprint arXiv:1702.08529 2017.
118. [121] Samadi, E.; Badri, A.; Ebrahimpour, R. Decentralized multi-agent based energy management of microgrid using reinforcement learning. International Journal of Electrical Power & Energy Systems 2020, 122, 106211.
119. [122] Zhang, K.; Yang, Z.; Liu, H.; Zhang, T.; Basar, T. Fully decentralized multi-agent reinforcement learning with networked agents. International Conference on Machine Learning. PMLR, 2018, pp. 5872–5881.
120. [123] Krishnan, V.G.; Ram, N.S. Analyze traffic forecast for decentralized multi agent system using I-ACO routing algorithm. Journal of Ambient Intelligence and Humanized Computing 2018, pp. 1–8.
121. [124] Khan, M.M.; Kasmarik, K.; Barlow, M. Toward computational motivation for multi-agent systems and swarms. Frontiers in Robotics and AI 2018, 5, 134.
122. [125] Omidshafiei, S.; Pazis, J.; Amato, C.; How, J.P.; Vian, J. Deep decentralized multi-task multi-agent reinforcement learning under partial observability. International Conference on Machine Learning. PMLR, 2017, pp. 2681–2690.
123. [126] Giret, A.; Trentesaux, D.; Salido, M.A.; Garcia, E.; Adam, E. A holonic multi-agent methodology to design sustainable intelligent manufacturing control systems. Journal of cleaner production 2017, 167, 1370–1386.
124. [127] Tang, H.; Li, D.; Wang, S.; Dong, Z. CASOA: an architecture for agent-based manufacturing system in the context of industry 4.0. IEEE Access 2017, 6, 12746–12754.
125. [128] Woo, J.; Shin, S.J.; Seo, W.; Meilanitasari, P. Developing a big data analytics platform for manufacturing systems: architecture, method, and implementation. The International Journal of Advanced Manufacturing Technology 2018, 99, 2193–2217.
126. [129] Sahin, C.; Demirtas, M.; Erol, R.; Baykasog˘lu, A.; Kaplanog˘lu, V. A multi-agent based approach to dynamic scheduling with flexible processing capabilities. Journal of Intelligent Manufacturing 2017, 28, 1827–1845.
127. [130] Koren, Y.; Gu, X.; Guo, W. Reconfigurable manufacturing systems: Principles, design, and future trends. Frontiers of Mechanical Engineering 2018, 13, 121–136.
128. [131] Zhang, D.; Feng, G.; Shi, Y.; Srinivasan, D. Physical safety and cyber security analysis of multi-agent systems: A survey of recent advances. IEEE/CAA Journal of Automatica Sinica 2021, 8, 319–333.
129. [132] Wang, J.; Zhang, Y.; Liu, Y.; Wu, N. Multiagent and bargaining-game-based real-time scheduling for internet of things-enabled flexible job shop. IEEE Internet of Things Journal 2018, 6, 2518–2531.
130. [133] Qu, Y.; Ming, X.; Liu, Z.; Zhang, X.; Hou, Z. Smart manufacturing systems: state of the art and future trends. The International Journal of Advanced Manufacturing Technology 2019, 103, 3751–3768.
131. [134] Tao, F.; Qi, Q.; Wang, L.; Nee, A. Digital twins and cyber–physical systems toward smart manufacturing and Industry 4.0: correlation and comparison. Engineering, 5 (4), 653–661, 2019.
132. [135] Anvari-Moghaddam, A.; Rahimi-Kian, A.; Mirian, M.S.; Guerrero, J.M. A multi-agent based energy management solution for integrated buildings and microgrid system. Applied energy 2017, 203, 41–56.
133. [136] Shi, L.; Guo, G.; Song, X. Multi-agent based dynamic scheduling optimisation of the sustainable hybrid flow shop in a ubiquitous environment. International Journal of Production Research 2021, 59, 576–597.
134. [137] Khan, M.W.; Wang, J.; Ma, M.; Xiong, L.; Li, P.; Wu, F. Optimal energy management and control aspects of distributed microgrid using multi-agent systems. Sustainable Cities and Society 2019, 44, 855–870.
135. [138] Mohseni, S.; Moghaddas-Tafreshi, S.M. A multi-agent system for optimal sizing of a cooperative self-sustainable multi-carrier microgrid. Sustainable cities and society 2018, 38, 452–465.
136. [139] Wang, B.; Chen, W.; Wang, J.; Zhang, B.; Zhang, Z.; Qiu, X. Accurate cooperative control for multiple leaders multiagent uncertain systems: A two-layer node-to-node communication framework. IEEE Transactions on Industrial Informatics 2017, 14, 2395–2405.
137. [140] Barenji, A.V.; Barenji, R.V.; Roudi, D.; Hashemipour, M. A dynamic multi-agent-based scheduling approach for SMEs. The International Journal of Advanced Manufacturing Technology 2017, 89, 3123–3137.
138. [141] Barenji, A.V.; Barenji, R.V.; Hashemipour, M. Flexible testing platform for employment of RFID-enabled multi-agent system on flexible assembly line. Advances in Engineering Software 2016, 91, 1–11.
139. [142] Shaygan, A.; Barenji, R.V.; others. Simulation platform for multi agent based manufacturing control system based on the hybrid agent. arXiv preprint arXiv:1603.07766 2016.
140. [143] Karavas, C.S.; Arvanitis, K.; Papadakis, G. A game theory approach to multi-agent decentralized energy management of autonomous polygeneration microgrids. Energies 2017, 10, 1756.
141. [144] Yu, W.; Wang, H.; Hong, H.; Wen, G. Distributed cooperative anti-disturbance control of multi-agent systems: an overview. Science China Information Sciences 2017, 60, 110202.
142. [145] Nouri, H.E.; Driss, O.B.; Ghédira, K. Solving the flexible job shop problem by hybrid metaheuristics-based multiagent model. Journal of Industrial Engineering International 2018, 14, 1–14.
143. [146] Wang, D.; Wang, W. Necessary and sufficient conditions for containment control of multi-agent systems with time delay. Automatica 2019, 103, 418–423.
144. [147] Singh, A.; Jain, T.; Sukhbaatar, S. Learning when to communicate at scale in multiagent cooperative and competitive tasks. arXiv preprint arXiv:1812.09755 2018.
145. [148] Lv, Y.; Lin, D. Design an intelligent real-time operation planning system in distributed manufacturing network. Industrial Management & Data Systems 2017.
146. [149] Tao, F.; Cheng, Y.; Zhang, L. Nee, a. YC (2015). Advanced manufacturing systems: socialization characteristics and trends. J Intell Manuf,(Chan 2001), pp. 1–16.
147. [150] Shen, W.; Norrie, D.H. Agent-based systems for intelligent manufacturing: a state-of-the-art survey. Knowledge and information systems 1999, 1, 129–156.
148. [151] Darbari, M.; Ahmad, H. Application of multi agent system in supply chain for e-products of government with special reference to government e-marketplace. Proceedings of the Third International Conference on Advanced Informatics for Computing Research, 2019, pp. 1–3.
149. [152] Howell, S.; Rezgui, Y.; Hippolyte, J.L.; Jayan, B.; Li, H. Towards the next generation of smart grids: Semantic and holonic multi-agent management of distributed energy resources. Renewable and Sustainable Energy Reviews 2017, 77, 193–214.
150. [153] Siderska, J.; Jadaan, K.S. Cloud manufacturing: a service-oriented manufacturing paradigm. A review paper. Engineering Management in Production and Services 2018, 10.
151. [154] Wang, X.V.; Givehchi, M.; Wang, L. Manufacturing system on the cloud: a case study on cloud-based process planning. Procedia CIRP 2017, 63, 39–45.
152. [155] Hmida, F.B.; Seguy, A.; Dupas, R. MultiAgent Systems for Production Planning and Control in Supply Chains. In Distributed Computing and Artificial Intelligence; Springer, 2012; pp. 205–212.
153. [156] Hmida, F.B.; Séguy, A.; Dupas, R. MultiAgent Simulation and Evaluation of Supply Chain Control Architectures. IFAC Proceedings Volumes 2012, 45, 781–786.
154. [157] Caridi, M.; Cavalieri, S. Multi-agent systems in production planning and control: an overview. Production Planning & Control 2004, 15, 106–118.
155. [158] Herrera, M.; Pérez-Hernández, M.; Kumar Parlikad, A.; Izquierdo, J. Multi-agent systems and complex networks: Review and applications in systems engineering. Processes 2020, 8, 312.
156. [159] Frayret, J.M.; D’Amours, S.; Montreuil, B. Coordination and control in distributed and agent-based manufacturing systems. Production Planning & Control 2004, 15, 42–54.
157. [160] Frayret, J.M.; D’Amours, S.; Rousseau, A.; Harvey, S.; Gaudreault, J. Agent-based supply-chain planning in the forest products industry. International Journal of Flexible Manufacturing Systems 2007, 19, 358–391.
158. [161] Han, D.; Yang, B.; Li, J.; Wang, J.; Sun, M.; Zhou, Q. A multi-agent-based system for two-stage scheduling problem of offshore project. Advances in Mechanical Engineering 2017, 9, 1687814017720882.
159. [162] Nielsen, P.; Banaszak, Z.; Bocewicz, G.; Janardhanan, M.N. Intelligent manufacturing/production systems: Modeling, algorithms, and optimization, 2018.
160. [163] Leitão, P.; Barbosa, J.; Geraldes, C.A.; Coelho, J.P. Multi-agent system architecture for zero defect multi-stage manufacturing. In Service Orientation in Holonic and Multi-Agent Manufacturing; Springer, 2018; pp. 13–26.
161. [164] Amaral, C.J.; Cranefield, S.; Hübner, J.F.; Roloff, M.L. Integrating Industrial Artifacts and Agents Through Apache Camel. arXiv preprint arXiv:2006.11694 2020.
162. [165] Battaïa, O.; Sanmartin, L.; Pralet, C. Dealing with disruptions in low-volume manufacturing: A constraint programming approach. Procedia CIRP 2019, 81, 1372–1375.
163. [166] Barbosa, J.; Leitão, P.; Ferreira, A.; Queiroz, J.; Angione, G.; Duca, G.L. Smart Inspection Tools Combining Multi-Agent Systems and Advanced Quality Control. 2018 IEEE 16th International Conference on Industrial Informatics (INDIN). IEEE, 2018, pp. 828–833.
164. [167] Sormaz, D.N.; Sarkar, A.; Ghosal, S. IMPlanner-MAS: A Multiagent System for Distributed Manufacturing Process Planning. Procedia manufacturing 2018, 26, 1242–1254.