Video Motion Magnification Using Split Spectrum Processing

Öz

Video motion magnification is the process of enlarging small-scale movements that cannot be detected by the human eye. To achieve this, first, the video frames are processed in both spatially and temporally via magnifying the subtle movements and vibrations and then, these processed frames are embedded back into the video to create visibility. Various applications in different fields are studied, i.e., magnification of pulse motion on the wrist, oscillations of bridges, vibrations of buildings, and sound recovery of trembling surfaces from video. In this study, a well-known signal processing method, namely, split spectrum processing method which is used to increase the signal-to-noise ratio of returning signals in radar and ultrasound, is successfully employed on the temporal processing layer of popular Euler motion amplification technique for video magnification. Our proposed and classical Euler magnification methods are compared in terms of their structural similarity index and improvements are demonstrated.

Anahtar Kelimeler

• motion magnification,split spectrum processing,structural similarity index

Bölünmüş İzge İşleme ile Video Devinim Büyütme

Öz

Video devinim büyütme, insan gözünün algılayamadığı küçük mertebedeki devinimlerin uygun bir yöntemle büyütülmesi ve bu devinimlerin videoda gözle görülebilir hale getirilmesi işlemidir. Bu algoritmalar, video çerçevelerini hem uzamsal hem de zamansal alanda işleme tabi tutarak minik devinim ve titreşimlerin büyütülüp videoya geri gömülmesi temeline dayanmaktadır. Örneğin, nabız atımının bilekte oluşturduğu devinim, köprü salınımları ve bina titreşimleri gibi algılanması zor olan devinimlerin yanında ses geriçatımı ve optik gibi çeşitli alanlarda da bu yöntemler kullanılmaya başlanmıştır. Bu çalışmada video devinim büyütmede kullanılan Euler yönteminin zamansal işleme katmanında, radar ve sesötesi (ultrasound) gibi sinyal işleme alanlarında uygulanan ve sinyal-gürültü oranını arttıran Bölünmüş İzge İşleme yöntemi kullanılmıştır. Önerilen yöntem ve Euler devinim büyütme yöntemi, yapısal benzerlik indeksi üzerinden karşılaştırılmış ve iyileştirmeler gösterilmiştir.

Anahtar Kelimeler

• devinim büyütme,bölünmüş izge işleme,yapısal benzerlik indeksi

Kaynakça

1. [1] Liu, C., Torralba, A., Freeman, W. T., Durand, F., ve Adelson, E. H. “Motion magnification”,ACM transactions on graphics (TOG), 24(3), 519-526, 2005.
2. [2] Wu, H. Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., ve Freeman, W., “Eulerian video magnification for revealing subtle changes in the world”. ACM transactions on graphics (TOG), 31(4), 1-8, 2012.
3. [3] Wu, H. Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., ve Freeman, W.,“Eulerian video magnification for revealing subtle changes in the world”,ACM transactions on graphics (TOG), 31(4), 1-8, 2012.
4. [4] Terán, L., Ordóñez, C., García-Cortés, S., ve Menéndez, A., “Detection ve magnification of bridge displacements using video images”,In Optics and Measurement International Conference, 2016 (Vol. 10151, p. 1015109).
5. [5] Wadhwa, N., Chen, J. G., Sellon, J. B., Wei, D., Rubinstein, M., Ghaffari, R. ve Kang, S. H.,“Motion microscopy for visualizing and quantifying small motions”,Proceedings of the National Academy of Sciences, 114(44), 11639-11644, 2017.
6. [6] Zhu, Y., Hao, J., Yang, J., ve Dong, E., ”Subtle Vibration Sensing and Dimension Measurement with a Bio-inspired Optical Tactile Sensor”,IEEE International Conference on Mechatronics and Automation (ICMA), 2019, (pp. 1248-1253).
7. [7] Peng, W., Hong, X., Xu, Y., ve Zhao, G. “A boost in revealing subtle facial expressions: A consolidated eulerian framework”,14th IEEE International Conference on Automatic Face & Gesture Recognition, 2019, (pp. 1-5).
8. [8] Perrot, V., Salles, S., Vray, D., ve Liebgott, H. “Video magnification applied in ultrasound”,IEEE Transactions on Biomedical Engineering, 66(1), 283-288, 2018.

9. [9] Davis, A., Rubinstein, M., Wadhwa, N., Mysore, G. J., Durand, F., ve Freeman, W. T.,“The visual microphone: Passive recovery of sound from video”,ACM Transactions on Graphics, 2014.
10. [10] Zhang, Y., Pintea, S. L., ve Van Gemert, J. C., “Video acceleration magnification”,In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, (pp. 529-537).
11. [11] Wadhwa, N., Rubinstein, M., Durand, F., ve Freeman, W. T.,“Phase-based video motion processing”, ACM Transactions on Graphics (TOG), 32(4), 1-10, 2013.
12. [12] Wadhwa, N., Rubinstein, M., Durand, F., ve Freeman, W. T., “Riesz pyramids for fast phase-based video magnification”, IEEE International Conference on Computational Photography (ICCP), 2014, (pp. 1-10).
13. [13] Kamble, K., Jagtap, N., Patil, R. A., ve Bhurane, A., “A review: eulerian video motion agnification”,International Journal of Innovative Research in Computer and Communication Engineering (IJIRCCE), 3, 2384-2390, 2015.
14. [14] Elgharib, M., Hefeeda, M., Durand, F., ve Freeman, W. T.,“Video magnification in presence of large motions”,In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, (pp. 4119-4127).
15. [15] Kaur, V. veKaur, J., “Comparison of Image Fusion Techniques: Spatial and Transform Domain based Techniques,” International Journal of Engineering and Computer Science, vol. 4, no. 5, (pp. 12109–12112), 2015.
16. [16] Kalaivani,K. ve Phamila, Y. A. V., “Analysis of image fusion techniques based on quality assessment metrics”,Indian Journal of Science and Technology, vol. 9, no. 31, 2016.
17. [17] Garcia, A. L.,Numerical methods for physics. Prentice Hall Englewood Cliffs, NJ, 2000.
18. [18] Karaoguz, M., Bilgutay, N., Akgul, T., ve Popovics, S.,“Defect detection in concrete using split spectrum processing”,Ultrasonics Symposium. Proceedings, 1998,IEEE 1, (pp.843-846).
19. [19] Karaoguz, M., Bilgutay, N., Akgul, T., ve Popovics, S., “Ultrasonic testing of concrete using split spectrum processing” Materials Evaluation,(pp.1183), November 1999.
20. [20] Popovics, S., Bilgutay, N., Karaoguz, M., ve Akgul, T.,“High-frequency ultrasound technique for testing concrete”, ACI Materials Journal, 97 (1), (pp. 58-65), 2000.
21. [21] Pedram, S. K., Mudge, P., & Gan, T. H., “Enhancement of ultrasonicguidedwavesignalsusing a split-spectrumprocessingmethod”, Appliedsciences, 8(10), 1815, 2018.
22. [22] Wahyuni, I. S.,“Multi-focus image fusion using local variability. Université Bourgogne Franche-Comté (Doctoral dissertation), 2018.