A novel verification tool suite for industrial robotic systems evaluation results

Yıl 2026, Cilt: 32 Sayı: 2, 276 - 294, 16.03.2026

Alim Kerem Erdoğmuş , Uğur Yayan

https://doi.org/10.5505/pajes.2025.37200

https://izlik.org/JA89ES67WX

Öz

This study introduces a new tool suite for robotic system safety and reliability in manufacturing. Developed for a vehicle chassis inspection project, it includes a simulation-based verification tool and a camera fault injection tool. Evaluated across three scenarios with robust testing, the tools demonstrated significant improvements in system efficiency and robustness. Unique for being open-source and ROScompatible, they fill a gap in robotic system verification and validation. Results showed a 95.39% robustness in fault detection, a 27% efficiency increase in planning, and an 80% resistance to camera faults. This advancement marks a significant leap in reducing testing costs and time for industrial robotics

Anahtar Kelimeler

verification tool , robotics , industrial quality control , safety trajectory optimization , manipulation of sensor data , fault injection

Endüstriyel robotik sistemler için bir doğrulama aracının değerlendirme sonuçları

https://izlik.org/JA89ES67WX

Öz

Bu çalışmada, üretimde robotik sistem güvenliği ve güvenilirliği için yeni bir araç paketi tanıtılmaktadır. Bir araç şasisi denetim projesi için geliştirilen bu araç, simülasyon tabanlı bir doğrulama aracı ve bir kamera hata enjeksiyon aracı içermektedir. Sağlam testlerle üç senaryoda değerlendirilen araçlar, sistem verimliliği ve sağlamlığında önemli gelişmeler olduğunu göstermiştir. Açık kaynaklı ve ROS uyumlu olmasıyla benzersiz olan bu araçlar, robotik sistem doğrulama ve onaylama alanındaki bir boşluğu doldurmaktadır. Sonuçlar, hata tespitinde %95,39 sağlamlık, planlamada %27 verimlilik artışı ve kamera hatalarına karşı %80 direnç göstermektedir. Bu ilerleme, endüstriyel robotik için test maliyetlerini ve süresini azaltmada önemli bir sıçramaya işaret etmektedir.

Anahtar Kelimeler

Doğrulama aracı , robotik , endüstriyel kalite kontrol , güvenli yörünge optimizasyonu , sensör verilerinin manipülasyonu , hata enjeksiyonu

Kaynakça

• [1] Aiello F, Garro A, Lemmens Y, Dutre S. “Simulation-based verification of system requirements: An integrated solution”. IEEE 14th International Conference on Networking, Sensing and Control, Falerna, Italy, 16-18 May 2017.
• [2] Bauer T, Agirre J, Furcho D, Herzner W, Hruska B, Karaca M, Pereira D, Proenca J, Schlick R, Sicher R, Smrcka A, Yayan U, Sangchoolie B. “Cross-domain modelling of verification and validation workflows in the large scale european research project valu3s”. International Conference on Embedded Computer Systems, Taipei, Taiwan, 23-25 August 2022.
• [3] Kanak A, Ergun S, Ozkan M, Cokunlu G, Yayan U, Karaca M, Arslan AT. “Verification and validation of an automated robot inspection cell for automotive body-in-white: a use case for the valu3s ecsel project”. Open Research Europe, 1(115), 115, 2021.
• [4] Chance G, Ghobrial A, Lemaignan S, Pipe T, Eder K. “An agency-directed approach to test generation for simulation-based autonomous vehicle verification”. 2020 IEEE International Conference on Artificial Intelligence Testing (AITest), Oxford, United Kingdom, 3-6 August 2020.
• [5] Utting M, Pretschner A, Legeard B. “A taxonomy of model-based testing approaches”. Software testing, verification and reliability, 22(5), 297–312, 2012.
• [6] Huck TP, Ledermann C, Kroger T. “Simulation-based testing for early safety- validation of robot systems”. 2020 IEEE Symposium on Product Compliance Engineering, Portland, Oregon, USA, 16-20 November 2020.
• [7] Yang Y, McLaughlin K, Littler T, Sezer S, Im EG, Yao Z, Pranggono B, Wang H. “Man-in-the-middle attack test-bed investigating cyber-security vulnerabilities in smart grid scada systems”. International conference on sustainable power generation and supply (SUPERGEN 2012), Hangzhou, China, 8-9 September 2012.
• [8] Vara JL, Bauer T, Fischer B, Karaca M, Madeira H, Matschnig M, Mazzini S, Nandi GS, Patrone F, Pereira D. “A proposal for the classification of methods for verification and validation of safety, cybersecurity, and privacy of automated systems”. International Conference on the Quality of Information and Communications Technology, Algarve, Portugal, 8-11 September 2021.
• [9] Hobbs A, Lyall B. “Human factors guidelines for remotely piloted aircraft system (rpas) remote pilot stations (rps). Technical report”. San José, California, USA, 34128, 2016.
• [10] Simrock S. Control theory. 1st ed. Bern, Switzerland, CERN, 2008.
• [11] Fisher M. An Introduction to Practical Formal Methods Using Temporal Logic. 1st ed. John Wiley & Sons, Portland, USA, 2011.
• [12] Xiao A, Bryden KM. “Virtual engineering: A vision of the next-generation product realization using virtual reality technologies”. International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Salt Lake City, Utah, USA, 28-30 September 2004.
• [13] Robert C, Guiochet J, Waeselynck H. “Testing a non-deterministic robot in simulation-how many repeated runs?”. 2020 Fourth IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan, 9-11 November 2020.
• [14] Cavalcanti A, Sampaio A, Miyazawa A, Ribeiro P, Filho MC, Didier A, Li W, Timmis J. “Verified simulation for robotics”. Science of Computer Programming, 174(1), 1–37, 2019.
• [15] Garoche PL. Formal Verification of Control System Software. 67th ed. Princeton, New Jersey, USA, Princeton University Press, 2019.
• [16] Webster M, Western D, Araiza-Illan D, Dixon C, Eder K, Fisher M, Pipe AG. “A corroborative approach to verification and validation of human–robot teams”. The International Journal of Robotics Research, 39(1), 73–99, 2020.
• [17] Bogaerts B, Sels S, Vanlanduit S, Penne R. “Connecting the coppeliasim robotics simulator to virtual reality”. SoftwareX, 11(1), 100426, 2020.
• [18] Son TD, Bhave A, Auweraer HV. “Simulation-based testing framework for autonomous driving development”. 2019 IEEE International Conference on Mechatronics (ICM), Ilmenau, Germany, 18-19 March 2019.
• [19] Pedersen TA, Glomsrud JA, Ruud EL, Simonsen A, Sandrib J, Eriksen BOH. “Towards simulation-based verification of autonomous navigation systems”. Safety Science, 129(1), 104799, 2020.
• [20] Hsueh MC, Tsai TK, Iyer RK. “Fault injection techniques and tools”. Computer, 30(4), 75–82, 1997.
• [21] Parasyris K, Tziantzoulis G, Antonopoulos CD, Bellas N. “Gemfi: A fault injection tool for studying the behavior of applications on unreliable substrates”. 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Atlanta, Georgia, USA, 23-26 June 2014.
• [22] Aidemark J, Vinter J, Folkesson P, Karlsson J. “Goofi: Generic object-oriented fault injection tool”. 2001 International Conference on Dependable Systems and Networks, Gothenburg, Sweden, 1-4 July 2001.
• [23] Hari SKS, Tsai T, Stephenson M, Keckler WS, Emer J. “Sassifi: An architecture-level fault injection tool for gpu application resilience evaluation”. 2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Santa Rosa, California, USA, 24-25 April 2017.
• [24] Svenningsson R, Vinter J, Eriksson H, Torngren M. “Modifi: a model implemented fault injection tool”. International Conference on Computer Safety, Reliability, and Security, Vienna, Austria, 14-17 September 2010.
• [25] Erdogmus AK, Yayan U. “Development of simulation-based testing for automated robot cell for quality inspection of automotive body-in-white system”. TOK 2021 - Otomatik Kontrol Ulusal Kongresi, Van, Türkiye, 2-4 September 2021.
• [26] Yayan U, Erdogmus AK. “Endüstriyel robot hareket planlama algoritmaları performans karşılaştırması”. Journal of Science, Technology and Engineering Research, 2(2), 31–45, 2022.
• [27] Yayan U, Erdoğmuş AK. “Development of a fault injection tool & dataset for verification of camera-based perception in robotic systems”. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi, 30(3), 328–339, 2022.
• [28] Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY. “Ros: an open-source robot operating system”. ICRA Workshop on Open-Source Software, Kobe, Japan, 12-17 March 2009.
• [29] Chitta S, Sucan I, Cousins S. “Moveit! [ros topics]”. IEEE Robotics & Automation Magazine, 19(1), 18-19, 2012.
• [30] ROS Wiki. “SMACH article”. http://wiki.ros.org/SMACH (23.02.2024).
• [31] GAZEBO Website. “GAZEBO Description”. http://GAZEBOsim.org/ (23.02.2024).
• [32] Kuffner JJ, LaValle SM. “Rrt-connect: An efficient approach to single-query path planning”. Proceedings 2000 ICRA Millennium Conference, IEEE International Conference on Robotics and Automation, Symposia Proceedings, San Francisco, California, USA, 24-28 April 2000.
• [33] Sucan IA, Moll M, Kavraki LE. “The open motion planning library”. IEEE Robotics & Automation Magazine, 19(4), 72–82, 2012.
• [34] BTC Embedded. “When and How to Qualify Tools According to ISO 26262”. https://www.btc-embedded.com/when-and-how-to-qualify-tools-according-to-iso-26262/ (23.02.2024).
• [35] Osadcuks V, Pudzs M, Zujevs A, Pecka A, Ardavs A. “Clock-based time synchronization for an event-based camera dataset acquisition platform”. 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 1-4 June 2020.
• [36] Özkan K, Seke E, Işık Ş. “Derin öğrenmeye dayalı görünür yakın kızılötesi kamera kullanılarak buğday sınıflandırması”. Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 27(5), 618-626, 2021.