GENERATING TURKISH LYRICS WITH LONG SHORT TERM MEMORY

Year 2020, Volume: 62 Issue: 1, 71 - 78, 30.06.2020

Mehmet Güzel Hakan Erten Erkan Bostanci

https://doi.org/10.33769/aupse.584380

Abstract

     Long Short Term Memory (LSTM) has gained a serious achievement on sequential data which have been used generally videos, text and time-series. In this paper, we aim for generating lyrics with newly created “Turkish Lyrics” dataset. By this time, there have been studies for creating Turkish Lyrics with character-level. Unlike previous studies, we propose to Turkish Lyrics generator working with word-level instead on character-level. Also, for employing LSTM, we can’t send the words as string and words must be vectorized. To vectorize, we tried two ways for encoding the words that are used in dataset and compared them. Firstly, we sample for generating one-hot encoding and then, secondly word-embedding way (Word2Vec). Observational results show us that word- level generation with word-embedding way gives more meaningful and realistic lyrics. Actually, there have not been good results enough to be used for a song because of Turkish Grammar. But, this study encourages authors to work on this field and we do believe that this study will initialize research on this area and lead researchers to contribute to this as well.

Keywords

LSTM, machine learning, one-hot encoding, word embedding, Turkish lyrics estimation

References

• Crash Course in Recurrent Neural Networks for Deep Learning, https://machinelearningmastery.com/ crash-course-recurrent-neural-networks-deep-learning/, accessed: 2019-06-15, 2010. 2
• Hochreiter, S., Schmidhuber, J., Long Short Term Memory, Technical Report FKI-207-95, URL http://citeseerx.ist.psu.edu/viewdoc/ similar? doi=10.1.1.51.3117&type =ab.2.
• Hochreiter, S., Schmidhuber, J., Long Short Term Memory, Neural Compu- tation, 9(8):17351780,November 1997. ISSN 0899-7667. doi: 10.1162/neco URL http://www.bioinf.jku.at/publications/older/2604.pdf. 2
• Understanding LSTM Networks, https://colah.github.io/posts/ 2015-08-Understanding-LSTMs/, accessed: 2019-06-17, 2015. 2
• Karim, A. M., Güzel, M. S., Tolun, M. R., Kaya, H., Çelebi, F. V., A new generalized deep learning framework combining sparse autoencoder and taguchi method for novel data classification and processing mathematical problems in engineering, Article ID 3145947, (2018), 13 pages.
• Karim, A. M., Güzel, M. S., Tolun, M. R., Kaya, H., Çelebi, F. V. A new framework using deep auto-encoder and energy spectral density for medical waveform data classification and processing, Biocybernetics and Biomedical Engineering, 39 (2019), 148–159.
• Sutskever, I., Martens, J., Hinton, G., Generating Text with Recurrent Neural Networks, in: Proceedings of the 28th International Conference on Machine Learning, ICML, (11 March 2011), 1017–1024,
• Graves, A., Mohamed, A.R., Hinton, G., Speech recognition with deep recur- rent neural networks, in: IEEE international conference on acoustics, speech and signal processing, IEEE, (March 2013), 6645–6649.
• Potash, P., Romanov, A., Rumshisky, A., Ghostwriter: Using an lstm for automatic rap lyric generation, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, (March 2015), 1919–1924.
• Malmi, E., Takala, P., Toivonen, H., Raiko, T., Gionis, A., DopeLearning: A Computational Approach to Rap Lyrics Generation, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, (March 2016), 195–204.
• “Sezen aksu sarkisi yazan yapay zeka diyorum”, 15/05/2019. https://medium.com/@tuncerergin/ sezen-aksu-sarkisi-yazan-yapay-zeka-diyorum-cd327001b7c4, Access date: 2019-05-31, 2019.