Detection and Identification of Stuttering Types Using Siamese Network
Abstract
Stuttering is a complex speech disorder characterized by disruptions in the fluency of verbal expression, often leading to challenges in communication for those affected. Accurate identification and classification of stuttering types can greatly benefit persons who stutter (PWS), especially in an era where voice technologies are becoming increasingly ubiquitous and integrated into daily life. In this work, we adapt a simple yet effective Siamese network architecture, known for its capability to learn from paired speech segments, to extract novel features from audio speech data. Our approach leverages these features to enhance the detection and identification of stuttering events. For our experiments, we rely on a subset of the SEP-28k stuttering dataset, initially implementing a single-task model and gradually evolving it into a more sophisticated multi-task model. Our results demonstrate that transitioning the network from a single-task learner to a multi-task learner, coupled with the integration of auxiliary classification heads, significantly improves the identification of stuttering types, even with a relatively small dataset.
Keywords
References
- [1] Amruth, V., Lavanya, K., Manoj, N., Umme, H., and Deepika, M. B. (2020). A novel approach for stutter speech recognition and correction. International Journal for Research in Applied Science and Engineering Technology (IJRASET), 8:544–547.
- [2] Arjun, K. N., Karthik, S., Kamalnath, D., Chanda, P., and Tripath, S. (2020). Automatic correction of stutter in dysfluent speech. Procedia Computer Science, 171:1363–1370.
- [3] Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, pages 12449–12460.
- [4] Bayerl, S., von Gudenberg, A. W., H¨onig, F., N¨oth, E., and Riedhammer, K. (2022a). KSoF: The kassel state of fluency dataset – a therapy centered dataset of stuttering. In Proceedings of the Language Resources and Evaluation Conference LREC, pages 1780–1787. European Language Resources Association.
- [5] Bayerl, S. P., Gerczuk, M., Batliner, A., Bergler, C., Amiriparian, S., Schuller, B., N¨oth, E., and Riedhammer, K. (2023). Classification of stuttering – the compare challenge and beyond. Computer Speech & Language, 81.
- [6] Bayerl, S. P., Wagner, D., N¨oth, E., Bocklet, T., and Riedhammer, K. (2022b). The influence of dataset partitioning on dysfluency detection systems. In Sojka, P., Hor´ak, A., Kopeˇcek, I., and Pala, K., editors, Text, Speech, and Dialogue, pages 423–436, Cham. Springer International Publishing.
- [7] Bayerl, S. P., Wagner, D., N¨oth, E., and Riedhammer, K. (2022c). Detecting dysfluencies in stuttering therapy using wav2vec 2.0. In Interspeech 2022. ISCA.
- [8] Dash, A., Subramani, N., Manjunath, T., Yaragarala, V., and Tripathi, S. (2018). Speech recognition and correction of a stuttered speech. In 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 1757–1760.
- [9] Heeman, P., Lunsford, R., McMillin, A., and Yaruss, J. S. (2016). Using clinician annotations to improve automatic speech recognition of stuttered speech. In Interspeech, pages 2651–2655.
- [10] Howell, P., Davis, S., and Bartrip, J. (2009). The UCLASS archive of stuttered speech. Journal of Speech, Language, and Hearing Research, 52:556–596.