Automated liver segmentation using Mask R-CNN on computed tomography scans

Öz

Due to changes such as shape, border and density that occur in the slices of computed tomography (CT) images, liver segmentation remains a difficult process. Compared to other segmentation methods, more successful segmentation results with deep learning models are general phenomenon. In this study, a method accelerated with a multi-GPU is proposed for computer-aided automatic segmentation of the liver on CT scans obtained from the abdominal region using Mask Regional-Convolutional Neural Networks (Mask R-CNN). Experimental studies are conducted on liver CT image datasets to specific for this study with both single and double GPU hardware structure. The results obtained using the proposed method and the segmentation results realized by the specialist physician compared with parameters such as Dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), volumetric overlap error (VOE), average symmetric surface distance (ASD) and relative volume difference (RVD) metrics. In experimental studies carried out on the test images with the proposed approach, DSC, JSC, VOE, ASD and RVD segmentation performance metrics are gained as 97.32, 94.79, 5.21, 0.390, -1.008, respectively. With these results, it is seen that the proposed method in this study can be used as a secondary tool in the decision making processes of physicians for the segmentation of the liver.

Anahtar Kelimeler

• Computed Tomography
• Liver Scans
• Image Segmentation
• Mask R-CNN
• GPU

Bilgisayarlı tomografi taramaları üzerinde maskeli bölgesel-evrişimsel sinir ağları ile karaciğerin otomatik bölütlenmesi

Öz

Bilgisayarlı Tomografi (BT) görüntülerinde her bir kesitte ortaya çıkan şekil, sınır ve yoğunluk gibi değişikliklerden dolayı karaciğerin bölütlenmesi zor bir süreç olarak durmaktadır. Diğer bölütleme yöntemleri ile karşılaştırıldığında, derin öğrenme modelleri ile daha başarılı bölütleme sonuçları genel fenomendir. Bu çalışmada, abdomen bölgesinden alınmış BT taramalarındaki kesitler üzerinde karaciğerin bilgisayar destekli otomatik bölütlenmesi için, Maskeli Bölgesel-Evrişimsel Sinir Ağları (Maskeli B-ESA) kullanılarak çoklu-GPU ile hızlandırılmış bir yöntem önerilmiştir. Bu çalışmaya özgü hazırlanan karaciğer BT görüntü veriseti üzerinde, hem tek hem de çift GPU donanımsal yapısı ile deneysel çalışmalar yürütülmüştür. Önerilen yöntem kullanılarak elde edilen sonuçlar ile uzman hekim tarafından bulunan bölütleme sonuçları Dice benzerlik katsayısı (DSC), Jaccard benzerlik katsayısı (JSC), volumetrik örtüşme hatası (VOE), ortalama simetrik yüzey mesafesi (ASD) ve oransal hacim farkı (RVD) ölçüm parametreleri ile karşılaştırılmıştır. Önerilen yaklaşım ile test görüntüleri üzerinde yürütülen deneysel çalışmalarda DSC, JSC, VOE, ASD ve RVD bölütleme başarım metrikleri, sırasıyla 97.32, 94.79, 5.21, 0.390, -1.008 olarak hesaplanmıştır. Bu sonuçlar ile bu çalışma kapsamında önerilen yöntemin, karaciğerin bölütlenmesi için hekimlerin karar verme süreçlerinde yardımcı bir araç olarak kullanılabileceği görülmüştür.

Anahtar Kelimeler

• Bilgisayarlı Tomografi
• Karaciğer Taramaları
• Görüntü Bölütleme
• Maskeli Bölgesel-Evrişimsel Sinir Ağları
• GPU

Kaynakça

1. 1. Bogovic J. A., Prince J. L. Bazin P.-L., A multiple object geometric deformable model for image segmentation, Computer Vision and Image Understanding, 117, 145-157, 2013.
2. 2. Lu X., Wu J., Ren X., Zhang B. Li Y., The study and application of the improved region growing algorithm for liver segmentation, Optik-International Journal for Light and Electron Optics, 125, 2142-2147, 2014.
3. 3. Kaya H., Çavuşoğlu A., Çakmak H. B., Şen B. Delen D., Supporting the diagnosis process and processes after treatment by using image segmentation and image simulation techniques: Keratoconus example, Journal of The Faculty of Engineering and Architecture of Gazi University, 31, 737-747, 2016.
4. 4. Dandıl E., An Application for Computer-Assisted Automatic Segmentation of Liver on Computed Tomography Images, Gazi University Science Journal: PART:C ‘Design and Technology’, 7, 712-728, 2019.
5. 5. Selvi E., Selver M. A., Kavur A., Güzeliş C. Dicle O., Segmentation of Abdomınal Organs from MR Images using Multi-Level Hierarchical Classification, Journal of the Faculty of Engineering and Architecture of Gazi University, 30, 533-546, 2015.
6. 6. Von Landesberger T., Bremm S., Kirschner M., Wesarg S. Kuijper A., Visual analytics for model-based medical image segmentation: Opportunities and challenges, Expert Systems with Applications, 40, 4934-4943, 2013.
7. 7. Huang Q., Ding H., Wang X. Wang G., Fully automatic liver segmentation in CT images using modified graph cuts and feature detection, Computers in biology and medicine, 95, 198-208, 2018.
8. 8. Gotra A., Sivakumaran L., Chartrand G., Vu K.-N., Vandenbroucke-Menu F., Kauffmann C., Kadoury S., Gallix B., de Guise J. A. Tang A., Liver segmentation: indications, techniques and future directions, Insights into imaging, 8, 377-392, 2017.

9. 9. Moghbel M., Mashohor S., Mahmud R. Saripan M. I. B., Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography, Artificial Intelligence Review, 50, 497-537, 2018.
10. 10. Heimann T., Van Ginneken B., Styner M. A., Arzhaeva Y., Aurich V., Bauer C., Beck A., Becker C., Beichel R. Bekes G., Comparison and evaluation of methods for liver segmentation from CT datasets, IEEE transactions on medical imaging, 28, 1251-1265, 2009.
11. 11. Kainmüller D., Lange T. Lamecker H., Shape constrained automatic segmentation of the liver based on a heuristic intensity model, Proc. MICCAI Workshop 3D Segmentation in the Clinic: A Grand Challenge, 109-116, 2007.
12. 12. Zareei A. Karimi A., Liver segmentation with new supervised method to create initial curve for active contour, Computers in Biology and Medicine, 75, 139-150, 2016.
13. 13. Boykov Y. Funka-Lea G., Graph cuts and efficient ND image segmentation, International journal of computer vision, 70, 109-131, 2006.
14. 14. Lu F., Wu F., Hu P., Peng Z. Kong D., Automatic 3D liver location and segmentation via convolutional neural network and graph cut, International journal of computer assisted radiology and surgery, 12, 171-182, 2017.
15. 15. Qiao Y., Cappelle C., Ruichek Y. Yang T., ConvNet and LSH-Based Visual Localization Using Localized Sequence Matching, Sensors, 19, 2439, 2019.
16. 16. Andrew W., Greatwood C. Burghardt T., Visual localisation and individual identification of holstein friesian cattle via deep learning, Proceedings of the IEEE International Conference on Computer Vision, 2850-2859, 2017.
17. 17. Pinheiro P. O., Collobert R. Dollár P., Learning to segment object candidates, Advances in Neural Information Processing Systems, 1990-1998, 2015.
18. 18. Pinheiro P. O., Lin T.-Y., Collobert R. Dollár P., Learning to refine object segments, European Conference on Computer Vision, 75-91, 2016.
19. 19. He K., Gkioxari G., Dollár P. Girshick R., Mask r-cnn, Proceedings of the IEEE international conference on computer vision, 2961-2969, 2017.
20. 20. Wu W., Zhou Z., Wu S. Zhang Y., Automatic liver segmentation on volumetric CT images using supervoxel-based graph cuts, Computational and mathematical methods in medicine, 2016, 2016.
21. 21. Liao M., Zhao Y.-q., Wang W., Zeng Y.-z., Yang Q., Shih F. Y. Zou B.-j., Efficient liver segmentation in CT images based on graph cuts and bottleneck detection, Physica Medica, 32, 1383-1396, 2016.
22. 22. Liao M., Zhao Y.-q., Liu X.-y., Zeng Y.-z., Zou B.-j., Wang X.-f. Shih F. Y., Automatic liver segmentation from abdominal CT volumes using graph cuts and border marching, Computer methods and programs in biomedicine, 143, 1-12, 2017.
23. 23. Zeng Y.-z., Liao S.-h., Tang P., Zhao Y.-q., Liao M., Chen Y. Liang Y.-x., Automatic liver vessel segmentation using 3D region growing and hybrid active contour model, Computers in biology and medicine, 97, 63-73, 2018.
24. 24. Yang X., Do Yang J., Hwang H. P., Yu H. C., Ahn S., Kim B.-W. You H., Segmentation of liver and vessels from CT images and classification of liver segments for preoperative liver surgical planning in living donor liver transplantation, Computer methods and programs in biomedicine, 158, 41-52, 2018.
25. 25. Lu X., Wu J., Ren X., Zhang B. Li Y., The study and application of the improved region growing algorithm for liver segmentation, Optik, 125, 2142-2147, 2014.
26. 26. Foruzan A. H., Zoroofi R. A., Hori M. Sato Y., A knowledge-based technique for liver segmentation in CT data, Computerized Medical Imaging and Graphics, 33, 567-587, 2009.
27. 27. Lim S.-J., Jeong Y.-Y. Ho Y.-S., Automatic liver segmentation for volume measurement in CT Images, Journal of Visual Communication and Image Representation, 17, 860-875, 2006.
28. 28. Huang L., Weng M., Shuai H., Huang Y., Sun J. Gao F., Automatic liver segmentation from CT images using single-block linear detection, BioMed research international, 2016, 2016.
29. 29. Liu Z., Song Y.-Q., Sheng V. S., Wang L., Jiang R., Zhang X. Yuan D., Liver CT sequence segmentation based with improved U-Net and graph cut, Expert Systems with Applications, 126, 54-63, 2019.
30. 30. Budak Ü., Guo Y., Tanyildizi E. Şengür A., Cascaded deep convolutional encoder-decoder neural networks for efficient liver tumor segmentation, Medical hypotheses, 134, 109431, 2020.
31. 31. Jia W., Tian Y., Luo R., Zhang Z., Lian J. Zheng Y., Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot, Computers and Electronics in Agriculture, 172, 105380, 2020.
32. 32. He K., Gkioxari G., Dollar P. Girshick R., Mask R-CNN, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 386-397, 2020.
33. 33. Yu Y., Zhang K., Yang L. Zhang D., Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN, Computers and Electronics in Agriculture, 163, 104846, 2019.
34. 34. Jiang J., Bie Y., Li J., Yang X., Ma G., Lu Y. Zhang C., Fault Diagnosis of the Bushing Infrared Images Based on Mask R-CNN Algorithm, High Voltage, 2020.
35. 35. Lin T.-Y., Dollár P., Girshick R., He K., Hariharan B. Belongie S., Feature pyramid networks for object detection, Proceedings of the IEEE conference on computer vision and pattern recognition, 2117-2125, 2017.
36. 36. Qiao Y., Truman M. Sukkarieh S., Cattle segmentation and contour extraction based on Mask R-CNN for precision livestock farming, Computers and Electronics in Agriculture, 165, 104958, 2019.
37. 37. Zimmermann R. S. Siems J. N., Faster training of Mask R-CNN by focusing on instance boundaries, Computer Vision and Image Understanding, 188, 102795, 2019.
38. 38. Abdulla W., Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow: matterport/Mask_RCNN,
39. 39. Dice L. R., Measures of the amount of ecologic association between species, Ecology, 26, 297-302, 1945.
40. 40. Yuan Y., Chao M. Lo Y.-C., Automatic skin lesion segmentation using deep fully convolutional networks with jaccard distance, IEEE transactions on medical imaging, 36, 1876-1886, 2017.
41. 41. Ahmad M., Ai D., Xie G., Qadri S. F., Song H., Huang Y., Wang Y. Yang J., Deep belief network modeling for automatic liver segmentation, IEEE Access, 7, 20585-20595, 2019.
42. 42. Wang K., Mamidipalli A., Retson T., Bahrami N., Hasenstab K., Blansit K., Bass E., Delgado T., Cunha G. Middleton M. S., Automated CT and MRI liver segmentation and biometry using a generalized convolutional neural network, Radiology: Artificial Intelligence, 1, 180022, 2019.