Simulation of deformation on 3D organ models using haptic device

Year 2023, Volume: 38 Issue: 2, 1263 - 1278, 07.10.2022

Nazmiye Ebru Bulut Emre Dandıl

https://doi.org/10.17341/gazimmfd.869134

Abstract

Nowadays, thanks to the improvements in computer graphics, remarkable improvements are achieved in 3D modelling in virtual environment in accordance with many original objects. In medical training, simulators are developed to enable physician candidates to make numerous repetitions using various scenarios and gain practical experience. In this study, a simulator software platform is developed to simulate the deformation of 3D organ models with soft tissues such as gallbladder, kidney and spleen in the human body using deformation algorithms. In the system, firstly, organs are modelled in 3D according to their anatomical structure. On the developed simulator, feedback is provided by the deformation realized on the model when the organ is touched or applied tensile force by designed haptic device. In the study, mass spring method and molecular modelling method are used to obtain deformation 3D models. In addition, collision detection method is preferred to determine which point is contacted in the mesh structure modelled with haptic device. All simulations on the simulator developed in the study are performed in real time using the functions available in the OpenGL library. In addition, simulation feedbacks for 11 novice and expert users are collected from the simulator and the operation times obtained by the users for each soft tissue are compared. As a result of the experimental studies and statistical analysis, it has been seen that the collision detection, reaction force, real-time simulation and deformation results of the software platform are at the desired level visually and in terms of speed.

Keywords

Computer graphics, deformation algorithms, haptic device, simulation, mass-spring system

Üç boyutlu modellenen organlar üzerinde dokunsal cihaz kullanılarak deformasyonun simülasyonu

Abstract

Günümüzde, bilgisayar grafiklerindeki gelişmeler sayesinde, birçok nesnenin asıllarına uygun şekilde sanal ortamda 3B modellenebilmesinde kayda değer gelişmeler elde edilmektedir. Tıp eğitiminde geliştirilen simülatörler ile organlar üzerinde doktor adaylarının çeşitli senaryolar kullanarak sayısız tekrar yapmalarına olanak sağlanarak pratik kazanmaları gerçekleştirilmektedir. Bu çalışmada, insan vücudu içerisinde bulunan safra, böbrek ve dalak gibi yumuşak dokuya sahip organların deformasyon algoritmaları ile yüzeysel benzetimlerinin yapılabilmesi için bir simülatör yazılım platformu geliştirilmiştir. Geliştirilen simülatör üzerinde, tasarlanan dokunsal cihaz donanımı ile anatomik yapılarına uygun şekilde 3B olarak modellenen organlar üzerine dokunulduğunda ya da çekme kuvveti uyguladığında model üzerinde gerçekleşen deformasyonun gerçekleşmesi sayesinde geri bildirim sağlanmıştır. Çalışmada, deformasyonun alınabilmesi için kütle yay ve moleküler modelleme metotları kullanılmıştır. Ek olarak, dokunsal cihaz ile modellenmiş örgü yapısı içerisinde hangi noktaya temas edildiğinin tespiti için çarpışma tespiti yöntemi tercih edilmiştir. Çalışma kapsamında geliştirilen simülatör üzerindeki tüm benzetimler, OpenGL kütüphanesi içerisinde bulunan fonksiyonlar kullanılarak gerçek zamanlı olarak gerçekleştirilmiştir. Ayrıca simülatörden toplamda 11 acemi ve uzman kullanıcı için simülasyon geri bildirimleri toplanmış ve her bir yumuşak doku için kullanıcıların elde ettikleri operasyon süreleri karşılaştırılmıştır. Yürütülen deneysel çalışmalar ve istatiksel analizler neticesinde, geliştirilen yazılım platformunun temas tespiti, tepki kuvveti, gerçek zamanlı benzetim ve deformasyon sonuçlarının görsel olarak ve hız açısından istenilen seviyede olduğu görülmüştür.

Keywords

Bilgisayar grafikleri, deformasyon algoritmaları, dokunsal cihazlar, simülasyon, kütle-yay sistemleri

References

• 1. Volkaner B., Sozen S. N. Omurlu V. E., Realization of a desktop flight simulation system for motion-cueing studies, International Journal of Advanced Robotic Systems, 13, 85, 2016.
• 2. Ruiz S., Aguado C. Moreno R., Educational simulation in practice: a teaching experience using a flight simulator, Journal of Technology and Science Education, 4, 181-200, 2014.
• 3. Gómez A. E., Santos T. C. d., Massera C. M., Neto A. d. M. Wolf D. F., Driving Simulator Platform for Development and Evaluation of Safety and Emergency Systems, arXiv preprint arXiv:1802.04104, 2018.
• 4. Slob J., State-of-the-art driving simulators, a literature survey, DCT report, 107, 2008.
• 5. Taheri S. M., Matsushita K. Sasaki M., Virtual reality driving simulation for measuring driver behavior and characteristics, Journal of transportation technologies, 7, 123, 2017.
• 6. Hamza-Lup F. G., Bogdan C. M., Popovici D. M. Costea O. D., A survey of visuo-haptic simulation in surgical training, arXiv preprint arXiv:1903.03272, 2019.
• 7. Varalakshmi B., Thriveni J., Venugopal K. Patnaik L., Haptics: state of the art survey, International Journal of Computer Science Issues (IJCSI), 9, 234, 2012.
• 8. Lin Y., Wang X., Wu F., Chen X., Wang C. Shen G., Development and validation of a surgical training simulator with haptic feedback for learning bone-sawing skill, Journal of biomedical informatics, 48, 122-129, 2014.
• 9. Westwood J., Development of a Patient-Specific Surgical Simulator for Pediatric Laparoscopic Procedures, Medicine Meets Virtual Reality 21: NextMed/MMVR21, 196, 360, 2014.
• 10. Chen X., Sun P. Liao D., A patient-specific haptic drilling simulator based on virtual reality for dental implant surgery, International journal of computer assisted radiology and surgery, 13, 1861-1870, 2018.
• 11. Dankelman J., Surgical simulator design and development, World journal of surgery, 32, 149-155, 2008.
• 12. Nealen A., Müller M., Keiser R., Boxerman E. Carlson M., Physically based deformable models in computer graphics, Computer graphics forum, 809-836, 2006.
• 13. Bro-Nielsen M., Finite element modeling in surgery simulation, Proceedings of the IEEE, 86, 490-503, 1998.
• 14. Koch R. M., Roth S. M., Gross M. H., Zimmermann A. P. Sailer H. F., A framework for facial surgery simulation, Proceedings of the 18th spring conference on Computer graphics, 33-42, 2002.
• 15. Chanthasopeephan T., Desai J. P. Lau A. C., Modeling soft-tissue deformation prior to cutting for surgical simulation: finite element analysis and study of cutting parameters, IEEE transactions on biomedical engineering, 54, 349-359, 2007.
• 16. Wu X., Downes M. S., Goktekin T. Tendick F., Adaptive nonlinear finite elements for deformable body simulation using dynamic progressive meshes, Computer Graphics Forum, 349-358, 2001.
• 17. Meriç D. Gedikli H., Numerical investigation of energy absorption behaviors of variable thickness tubes, Journal of the Faculty of Engineering and Architecture of Gazi University, 35, 1939-1956, 2020.
• 18. Nealen A., Müller M., Keiser R., Boxerman E. Carlson M., Physically based deformable models in computer graphics, EUROGRAPHICS 2005 STAR–STATE OF THE ART REPORT, 2005.
• 19. Mollemans W., Schutyser F., Nadjmi N., Maes F. Suetens P., Predicting soft tissue deformations for a maxillofacial surgery planning system: from computational strategies to a complete clinical validation, Medical image analysis, 11, 282-301, 2007.
• 20. Lloyd B., Székely G. Harders M., Identification of spring parameters for deformable object simulation, IEEE Transactions on Visualization and Computer Graphics, 13, 1081-1094, 2007.
• 21. Kang Y.-M., Choi J.-H., Cho H.-G. Park C.-J., Fast and stable animation of cloth with an approximated implicit method, Proceedings Computer Graphics International 2000, 247-255, 2000.
• 22. Duysak A., Zhang J. J. Ilankovan V., Efficient modelling and simulation of soft tissue deformation using mass-spring systems, International Congress Series, 337-342, 2003.
• 23. Frisken-Gibson S. F., Using linked volumes to model object collisions, deformation, cutting, carving, and joining, IEEE transactions on visualization and computer graphics, 5, 333-348, 1999.
• 24. Duysak A. Zhang J. J., Fast simulation of facial tissue deformations using mass-spring chain algorithm, 2005.
• 25. Duysak A., Triangle propagation for mass-spring chain algorithm, International Symposium on Computer and Information Sciences, 306-315, 2006.
• 26. Cicek Y. Duysak A., The modelling of interactions between organs and medical tools: a volumetric mass-spring chain algorithm, Computer methods in biomechanics and biomedical engineering, 17, 488-496, 2014.
• 27. Wang T., Pan B., Fu Y., Wang S. Ai Y., Design of a new haptic device and experiments in minimally invasive surgical robot, Computer Assisted Surgery, 22, 240-250, 2017.
• 28. Escobar-Castillejos D., Noguez J., Neri L., Magana A. Benes B., A review of simulators with haptic devices for medical training, Journal of medical systems, 40, 104, 2016.
• 29. Coles T. R., Meglan D. John N. W., The role of haptics in medical training simulators: A survey of the state of the art, IEEE Transactions on haptics, 4, 51-66, 2010.
• 30. 3dsystems. https://www.3dsystems.com/haptics-devices/touch. Erişim Tarihi: 06.11.2019.
• 31. Forcedimension. http://www.forcedimension.com/products/delta-3/overview. Erişim Tarihi: 06.11.2019.
• 32. Sekercioglu A. S. Duysak A., Application of molecular modeling with mass-spring systems for computer simulation and animation, International Journal of Physical Sciences, 4, 500-504, 2009.
• 33. Man R. R., Zhou D. S. Zhang Q., A Survey of Collision Detection, Applied Mechanics and Materials, 360-363, 2014.
• 34. Kockara S., Halic T., Iqbal K., Bayrak C. Rowe R., Collision detection: A survey, 2007 IEEE International Conference on Systems, Man and Cybernetics, 4046-4051, 2007.
• 35. Bauszat P., Eisemann M. Magnor M. A., The Minimal Bounding Volume Hierarchy, VMV, 227-234, 2010.
• 36. Klosowski J. T., Held M., Mitchell J. S., Sowizral H. Zikan K., Efficient collision detection using bounding volume hierarchies of k-DOPs, IEEE transactions on Visualization and Computer Graphics, 4, 21-36, 1998.
• 37. Löfstedt M. Akenine-Möller T., An evaluation framework for ray-triangle intersection algorithms, Journal of Graphics Tools, 10, 13-26, 2005.
• 38. Segura R. J. Feito F. R., Algorithms to test ray-triangle intersection. comparativestudy, 2001.
• 39. Majercik A., Crassin C., Shirley P. McGuire M., A ray-box intersection algorithm and efficient dynamic voxel rendering, Journal of Computer Graphics Techniques Vol, 7, 2018.
• 40. Kadleček P., Haptic rendering for 6/3-DOF haptic devices, 2014.