Automatic Image Stitching Using SURF and RANSAC Based Homography Estimation

Öz

This study presents a fully automatic method for image stitching on two-dimensional uncontrolled images. It provides a solution to the field-of-view limitation, which is especially needed in areas such as stereo and multi-angle camera applications, satellite and underwater mapping systems, and medical imaging. The study is based on reference points detection between the images with the SURF algorithm, image alignment with RANSAC and homography techniques, and image blending with the Poisson method. The quantitative and qualitative experiments show that the proposed method produces higher quality results than previous approaches.

Anahtar Kelimeler

• Image stitching
• SURF
• RANSAC
• homography
• poisson blending

SURF ve RANSAC Tabanlı Homografi Tahmini Kullanarak Otomatik İmaj Birleştirme

Öz

Bu çalışmada kontrolsüz ortamda çekilen iki boyutlu imajların birleştirilmesi için tam otomatik bir yöntem sunulmaktadır. Özellikle stereo ve çok açılı kamera uygulamaları, uydu ve sualtı haritalama sistemleri, tıbbi görüntüleme gibi alanlarda ihtiyaç duyulan bu yöntem, görüntülerdeki görüş alanı limitasyonuna bir çözüm sunmaktadır. Çalışma, görüntüler arasında bulunan referans noktaların SURF algoritması ile tespiti, bu noktalar üzerinden RANSAC ve homografi teknikleri ile imajların hizalanması ve Poisson yöntemi ile imajların harmanlanmasına dayanmaktadır. Yapılan nicel ve nitel deneyler, önerilen yöntemin diğer yaklaşımlardan daha kaliteli sonuçlar verdiğini göstermektedir.

Anahtar Kelimeler

• İmaj birleştirme
• SURF
• RANSAC
• homografi
• poisson harmanlama

Kaynakça

1. [1] Szeliski R., “Image alignment and stitching: A tutorial”, Found. Trends Comput. Graph. Vis., 2 (1): 1-104, (2007).
2. [2] Sadi, S., Perdices-Gonzalez, S., Budhiraja, R., Lee, B. D., Khwaja, A. M. ve Mistry, P., “Image stitching for three-dimensional video”, U.S. Patent No. 10,313,656, (2019).
3. [3] Li, Y., Randall, C.J., van Woesik, R. ve Ribeiro, E., “Underwater video mosaicing using topology and superpixel-based pairwise stitching”, Expert Syst. Appl., 119: 171-183, (2019).
4. [4] PyarWin, K. ve Kitjaidure, Y., “Biomedical Images Stitching using ORB Feature Based Approach”, IEEE International Conference on Intelligent Informatics and Biomedical Sciences, Bangkok-Thailand, 221-225, (2018).
5. [5] Hügl, S., Eckardt, F., Lexow, G. J., Majdani, O., Lenarz, T., ve Rau, T. S., “Increasing the resolution of morphological 3D image data sets through image stitching: application to the temporal bone”, Comput. Methods Biomech. Biomed. Eng. Imaging Vis., 5(6): 438-445, (2017).
6. [6] Chen, W., Fu, Z., Yang, D. ve Deng, J., “Single-image depth perception in the wild”, 30th Conference on Neural Information Processing Systems (NIPS), Barcelona-Spain, 730-738, (2016).
7. [7] Han, B. ve Lin, X., “A novel hybrid color registration algorithm for image stitching”, IEEE Trans. Consum. Electron., 52(3): 1129-1134, (2006).
8. [8] Schmid, C., Mohr, R. ve Bauckhage, C., “Evaluation of interest point detectors”, Int. J. Comput. Vis., 37(2): 151-172, (2000).

9. [9] Chavel, I., “Eigenvalues in Riemannian geometry”, Academic Press, Florida, USA, (1984).
10. [10] Harris, C. G. ve Stephens, M., “A combined corner and edge detector”, Alvey vision conference, Manchester-UK, 147-151, (1988).
11. [11] Kang, J., Xiao, C., Deng, M., Yu, J. ve Liu, H., “Image registration based on harris corner and mutual information”, IEEE International Conference on Electronic and Mechanical Engineering and Information Technology, Harbin-China, 3434-3437, (2011).
12. [12] Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T. ve Van Gool, L, “A comparison of affine region detectors”, Int. J. Comput. Vis., 65(1/2): 43-72, (2005).
13. [13] Lowe, D. G., “Distinctive image features from scale-invariant keypoints”, Int. J. Comput. Vis., 60(2): 91-110, (2004).
14. [14] Wu, J., Cui, Z., Sheng, V. S., Zhao, P., Su, D. ve Gong, S., “A Comparative Study of SIFT and its Variants”, Meas. Sci. Rev., 13(3): 122-131, (2013).
15. [15] Bay, H., Tuytelaars, T. ve Van Gool, L., “Surf: Speeded up robust features”, European Conference on Computer Vision (ECCV), Graz-Austria, 404-417, (2006).
16. [16] Bauer, J., Sünderhauf, N. ve Protzel, P., “Comparing several implementations of two recently published feature detectors”, International Conference on Intelligent and Autonomous Systems (IAV), Toulouse-France, 40(15): 143-148, (2007).
17. [17] Fischler, M. A. ve Bolles, R. C., “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography”, Commun. ACM, 24(6): 381-395, (1981).
18. [18] Rousseeuw, P. J., “Least median of squares regression”, J. Am. Stat. Assoc., 79(388): 871-880, (1984).
19. [19] Hartley, R. ve Zisserman, A., “Multiple view geometry in computer vision”, Cambridge University Press, Cambridge, UK, (2003).
20. [20] Legendre, A. M., “Nouvelles méthodes pour la détermination des orbites des comètes” [New Methods for the Determination of the Orbits of Comets] (Fransızca), F. Didot., Paris, France, (1805).
21. [21] Özkaya, N., Sağıroğlu, Ş., ve Beşdok, E., “Genel Amaçlı Otomatik Parmakizi Tanıma Sistemi Tasarımı ve Gerçekleştirilmesi”, Journal of Polytechnic, 8(3): 239-247, (2005).
22. [22] Lu, G. ve Wong, X. I., “Taking me to the correct place: Vision-based localization for autonomous vehicles”, IEEE International Conference on Image Processing (ICIP), Taipei-Taiwan, 2966-2970, (2019).
23. [23] Muzaffer, G. ve Ulutaş, G., “Detection of copy move forgery based on color SURF”, Journal of the Faculty of Engineering and Architecture of Gazi University, 34 (3): 1539-1548, (2019).
24. [24] Porter, T. ve Duff, T., “Compositing digital images”, International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), Minnesota-USA, 253-259, (1984).
25. [25] Burt, P. ve Adelson, E., “The Laplacian pyramid as a compact image code”, IEEE Trans. Commun., 31(4): 532-540, (1983).
26. [26] Burt, P. ve Adelson, E., “A multiresolution spline with application to image mosaics”, ACM Trans. Graphics, 2(4): 217-236, (1983).
27. [27] Brown, M. ve Lowe, D. G., “Automatic panoramic image stitching using invariant features”, Int. J. Comput. Vis., 74(1): 59-73, (2007).
28. [28] Efros, A. A. ve Freeman, W. T., “Image quilting for texture synthesis and transfer”, International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), California-USA, 341-346, (2001).
29. [29] Long, J. ve Mould, D., “Improved image quilting”, Graphics Interface Conference (GI), Québec-Canada, 257-264, (2007).
30. [30] Pérez, P., Gangnet, M. ve Blake, A., “Poisson image editing”, International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), California-USA, 313-318, (2003).
31. [31] Hu, C., Huo, L.Z., Zhang, Z. ve Tang, P., “Multi-Temporal Landsat Data Automatic Cloud Removal Using Poisson Blending”, IEEE Access, 8: 46151-46161, (2020).
32. [32] Chen, W. C. ve Tsai, W. J., “Region-based poisson blending for image repairing”, SAI Intelligent Systems Conference, London-UK, 416-430, (2018).
33. [33] Chen, S., Jin, M., Zhang, Y. ve Zhang, C., “Infrared blind-pixel compensation algorithm based on generative adversarial networks and Poisson image blending”, Signal Image Video Process., 14(1): 77-85, (2020).
34. [34] Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D. ve Cohen, M., “Interactive digital photomontage”, International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), California-USA, 294-302, (2004).
35. [35] Nirkin, Y., Keller, Y. ve Hassner, T., “FSGAN: Subject agnostic face swapping and reenactment”, International Conference on Computer Vision (ICCV), Seoul-South Korea, 7184-7193, (2019).
36. [36] Wang, Z. ve Yang, Z., “Review on image-stitching techniques”, Multimed. Syst., 26: 413-430, (2020).
37. [37] Li, N., Xu, Y. ve Wang, C., “Quasi-homography warps in image stitching”, IEEE Trans. Multimedia, 20(6): 1365-1375, (2017).
38. [38] Lin, K., Jiang, N., Cheong, L. F., Do, M. ve Lu, J., “Seagull: Seam-guided local alignment for parallax-tolerant image stitching”, European Conference on Computer Vision (ECCV), Amsterdam-Netherlands, 370-385, (2016).
39. [39] Jovanovski, B. L. ve Li, J., “Image-stitching for dimensioning”, U.S. Patent No. 10,402,956, (2019).
40. [40] Zaragoza, J., Chin, T. J., Tran, Q. H., Brown, M. S. ve Suter, D., “As-Projective-As-Possible Image Stitching with Moving DLT”, IEEE Trans. Pattern Anal. Mach. Intell., 36(7): 1285-1298, (2014).
41. [41] Lalonde, J. F., Hoiem, D., Efros, A. A., Rother, C., Winn, J. ve Criminisi, A., “Photo clip art”, ACM Trans. Graphics, 26(3): 3-es, (2007).
42. [42] Brandt, J., “Transform coding for fast approximate nearest neighbor search in high dimensions”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), California-USA, 1815-1822, (2010).
43. [43] Microsoft Research, “Image Composite Editor (ICE) 2.0”, https://www.microsoft.com/en-us/research/product/computational-photography-applications/image-composite-editor, Erişim tarihi Ekim 30, (2020).
44. [44] Wang, Z., Bovik, A. C., Sheikh, H. R. ve Simoncelli, E. P., “Image quality assessment: from error visibility to structural similarity”. IEEE Trans. Image Process., 13(4): 600-612, (2004).
45. [45] Herrmann, C., Wang, C., Strong Bowen, R., Keyder, E., Krainin, M., Liu, C. ve Zabih, R., “Robust image stitching with multiple registrations”, European Conference on Computer Vision (ECCV), Munich-Germany, 53-67, (2018).
46. [46] Chen, Y.S. ve Chuang, Y.Y., “Natural image stitching with the global similarity prior”, European Conference on Computer Vision (ECCV), Amsterdam-Netherlands, 186-201, (2016).
47. [47] Adobe, “Adobe Photoshop User Guide: Create panoramic images with Photomerge”, https://helpx.adobe.com/in/photoshop/using/create-panoramic-images-photomerge.html, Erişim tarihi Ekim 30, (2020).
48. [48] Lee, K.Y. ve Sim, J.Y., “Warping Residual Based Image Stitching for Large Parallax”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle-USA, 8198-8206, (2020).