Parmak Hareketlerinin Bilgisayarlı Yorumlanmasıyla Tek Oktavlı Notaların Seslendirilmesi

Year 2010, Volume: 7 Issue: 2, - , 01.04.2010

Vasif V. Nabiyev

Abstract

In this paper, the task of synthesizing virtual music without adopting any
musical instruments has been accomplished by detecting the changes in hand position with
the help of computer vision techniques. The melody of one octave has been studied taking
the possible diversities in the number of fingers into consideration. Vocalizing proper notes
corresponding to the current hand position has been carried out through computerized
interpretation of finger motions on a hand image recorded by a video camera. Finger
positions have been determined by preprocessing the input hand image. Feature vector
has been composed of the distances from hand’s center of gravity to finger tips. As a result,
the feasibility of real-time computerized synthesis of virtual music has been demonstrated
by evaluating the finger motions without the need for heavy musical instruments such as
piano.

Keywords

Human-computer interaction, hand moves recognition, artificial intelligence, music

References

• [1] F. Rossant, A global method for music symbol recognition in typeset music sheets, Pattern Recognition Letters 23 (2002), 1129–1141.
• [2] C. Nolker and H. Ritter, Visual recognition of continuous hand postures, IEEE Transactions on Neural Networks 13 (2002), 983–994.
• [3] D. O. Gorodnichy and A. Yogeswaran, Detection and tracking of pianist hands and fingers, Proceedings of the Third Canadian Conference on Computer & Robot Vision (CRV’06) Qu´ebec, Canada, June 7-9, 2006.
• [4] Y. Bulatov, S. Jambawalikar, P. Kumar and Saurabh Sethia, Hand recognition using geometric classifiers, Proceedings of the First International Conference on Biometric Authentication (ICBA 2004), Hong Kong, China, July 15-17, 753–759.
• [5] Y. Wu and T. S. Huang, Hand modeling, analysis, and recognition: For vision-based human computer interaction, IEEE Signal Processing Magazine 18 (2001), 51–60.
• [6] Ra L. N. Wong and P. Shi, Peg-free hand geometry recognition using hierarchical geometry and shape matching, Proceedings of the IAPR Workshop on Machine Vision Applications (2002) 281–284.
• [7] E. Y¨or¨uk, E. Konuko˘glu, B. Sankur, and J. Darbon, Shape-based hand recognition, IEEE Transactions on Image Processing 15 (2006), 1803–1815.
• [8] V. V. Nabiyev, Yapay Zeka: Problemler, Y¨ontemler, Algoritmalar, Se¸ckin Yayıncılık 2005.
• [9] V. V. Nabiyev , C. Kose, and S. Bayrak, An artificial neural network approach for sign language vowels recognition, The International Scientific Conference of Problems of Cybernetics and Informatics, Baku, Azerbaijan, 24-26 October 2006.
• [10] M’lou Dietzer, First Impressions: An Intermediate Piano Method, Alfred Music Publishing 2007.
• [11] W. K. Pratt, Digital Image Processing, John Wiley & Sons, New York 1991.
• [12] N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man and Cybernetics 9 (1979), 62–66.