Machine learning in audio mastering: a comparative study

Abstract

Machine learning in audio mastering: a comparative study

Abstract

Machine learning approaches now utilized in audio mastering are transforming traditional workflows. This comparative study examines the effectiveness of supervised and unsupervised methods in the mastering process. Platforms such as LANDR employ supervised models that emulate expert engineers, offering cost-effective options for independent artists, while unsupervised techniques aid spectral balance and dynamic range optimization. The methodology relies on objective metrics—including Distortion Meter, Dynamic Range, Loudness Penalty, Intelligibility, and High Frequency Distortion—along with subjective listening assessments. Statistical analyses show that human engineers surpass AI systems in preserving dynamic range, minimizing distortion, and maintaining sonic clarity, particularly for complex genres like classical and jazz. Empirical research reveals AI mastering causes greater distortion, narrower dynamic range, and higher loudness penalties. In contrast, engineers deliver superior audio quality through broader dynamic range, lower distortion, and enhanced intelligibility. While AI quickly provides reasonable results for simpler styles like Pop and Electronic, human expertise offers advantages for complex compositions where aesthetic judgment is key. These findings indicate that despite technological progress, human know-how remains critically vital in creative decision-making. The study also points to potential for human-machine collaboration in mastering, with AI initially optimizing parameters and engineers making refined aesthetic adjustments to enhance quality. This hybrid approach could unite technological efficiency with artistic excellence. Future work should focus on improving AI's ability to emulate human aesthetic decisions, developing genre-specific mastering, and incorporating techniques like generative adversarial networks to mastering. These advancements may pave the way for hybrid systems fusing human creativity and machine efficiency.

Keywords

• Audio quality
• Automatic audio mastering
• Dynamic range
• Machine learning
• Unsupervised learning

References

1. Abel, J., Kaniewska, M., Guillaume, C., Tirry, W., Pulakka, H., Myllyla, V., Sjoberg, J., Alku, P., Katsir, I., Malah, D., Cohen, I., Tugtekin Turan, M. A., Erzin, E., Schlien, T., Vary, P., Nour-Eldin, A., Kabal, P., & Fingscheidt, T. (2016). A subjective listening test of six different artificial bandwidth extension approaches in English, Chinese, German, and Korean. IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1109/icassp.2016.7472812
2. Alexander, L. (2020). An introduction to audio content analysis: Applications in signal processing and music informatics.
3. Ballou, G. (2002). Handbook for sound engineers (3rd ed.). Focal Press.
4. Barbosa, Á. (2006). Computer-supported cooperative work for music applications. Doctoral dissertation. http://www.tesisenxarxa.net
5. Benade, A. H., & Johns, R. H. (1977). Fundamentals of musical acoustics. American Journal of Physics, 45(1), 111–111. https://doi.org/10.1119/1.10883
6. Birtchnell, T. (2018). Listening without ears: Artificial intelligence in audio mastering. Big Data & Society, 5(2). https://doi.org/10.1177/2053951718808553
7. Cera, C. D., Regli, W. C., Braude, I., Shapirstein, Y., & Foster, C. V. (2002). A collaborative 3D environment for authoring design semantics. IEEE Computer Graphics and Applications, 22(3), 43–55. https://doi.org/10.1109/mcg.2002.999787
8. Cheng, K., & Olechowski, A. (2021). Some (Team) assembly required: An analysis of collaborative computer-aided design assembly. https://doi.org/10.1115/detc2021-68507

9. Cronhjort, A. (1992). A computer-controlled bowing machine (MUMS). Royal Institute of Technology Speech, Music and Hearing Quarterly Progress and Status Report, 33(2-3), 61-66. https://doi.org/10.1051/aacus/2024035
10. Gómez, A. M., Schwerin, B., & Paliwal, K. (2012). Improving objective intelligibility prediction by combining correlation and coherence-based methods with a measure based on the negative distortion ratio. Speech Communication, 54(3), 503–515. https://doi.org/10.1016/j.specom.2011.11.001
11. Hormozi, H., Hormozi, E., & Nohooji, H. R. (2012). The classification of the applicable machine learning methods in robot manipulators. International Journal of Machine Learning and Computing, 2(5), 560-563. https://doi.org/10.7763/IJMLC.2012.V2.189
12. Ivanović, M., & Radovanović, M. (2015). Modern machine learning techniques and their applications. In Electronics, Communications and Networks IV (pp. 833-846). CRC Press. https://doi.org/10.1201/b18592-153
13. Kates, J. M., & Arehart, K. H. (2004). A metric for evaluating speech intelligibility and quality in hearing aids. The Journal of the Acoustical Society of America, 116(4_Supplement), 2536–2537. https://doi.org/10.1121/1.4785122
14. Katz, B., & Katz, R. A. (2015). Mastering audio: The art and the science (3rd ed.). Focal Press.
15. Khanum, M., Mahboob, T., Imtiaz, W., Abdul Ghafoor, H., & Sehar, R. (2015). A survey on unsupervised machine learning algorithms for automation, classification and maintenance. International Journal of Computer Applications, 119(13), 34–39. https://doi.org/10.5120/21131-4058
16. Kirby, D. G., Feige, F., & Wustenhagen, U. (1994). ISO/MPEG subjective tests on multichannel audio coding systems: Practical realisation and test results. In IBC 1994 International Broadcasting Convention (pp. 132-139). IET. https://doi.org/10.1049/cp:19940741
17. Kotsiantis, S. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31(3), 249-268.
18. Kour, H., & Gondhi, N. (2020). Machine learning techniques: A survey. In Lecture Notes on Data Engineering and Communications Technologies (pp. 266–275). Springer International Publishing. https://doi.org/10.1007/978-3-030-38040-3_31
19. Kristijansson, A. M., & Aegisson, T. (2022). Survey on technique and user profiling in unsupervised machine learning method. Journal of Machine and Computing, 9–16. https://doi.org/10.53759/7669/jmc202202002
20. Kumar, K., Ravi, V., Carr, M., & Nampally, R. K. (2008). Software development cost estimation using wavelet neural networks. Journal of Systems and Software, 81(11), 1853-1867. https://doi.org/10.1016/j.jss.2007.12.793
21. Lazzarini, V. (2021). Fundamental aspects of audio and music signals. In Spectral music design (pp. 50–72). Oxford University Press. https://doi.org/10.1093/oso/9780197524015.003.0003
22. Loizou, P. C. (2011). Speech quality assessment. In Studies in Computational Intelligence (pp. 623–654). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-19551-8_23
23. Martinez Ramirez, M. A., Wang, O., Smaragdis, P., & Bryan, N. J. (2021). Differentiable signal processing with black-box audio effects. IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1109/icassp39728.2021.9415103
24. McAlpine, K., Miranda, E., & Hoggar, S. (1999). Making music with algorithms: A case-study system. Computer Music Journal, 23(2), 19–30. https://doi.org/10.1162/014892699559733
25. Naeem, S., Ali, A., Anam, S., & Ahmed, M. M. (2022). An unsupervised machine learning algorithms: Comprehensive review. International Journal of Computing and Digital Systems, 13(1), 911–921. https://doi.org/10.12785/ijcds/130172
26. Nag, H., Karthikeyan, M., & Kumar, V. (2020). Automation in audio enhancement using unsupervised learning for ubiquitous computational environment. https://doi.org/10.1109/icict48043.2020.9112473
27. Najduchowski, E., Lewandowski, M., & Bobinski, P. (2018). Automatic audio mastering system. https://doi.org/10.1109/acoustics.2018.8502427
28. Narayanan, U., Unnikrishnan, A., Paul, V., & Joseph, S. (2017). A survey on various supervised classification algorithms. In 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS) (pp. 2118-2124). IEEE.
29. Omankwu, C., Ugwuja, E., & Kanu, C. (2023). Comprehensive review of supervised machine learning algorithms to identify the best and error free. International Journal of Scholarly Research in Engineering and Technology, 2(1), 013-019. https://doi.org/10.56781/ijsret.2023.2.1.0028
30. Pasquier, P., Eigenfeldt, A., Bown, O., & Dubnov, S. (2016). An introduction to musical metacreation. Computers in Entertainment, 14(2), 1–14. https://doi.org/10.1145/2930672
31. Prince, S., & Shankar Kumar, K. R. (2012). Survey on effective audio mastering. In Communications in computer and information science (pp. 293–301). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-35594-3_41
32. Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S., & Sainath, T. (2019). Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2), 66-83. https://doi.org/10.1109/JSTSP.2019.2908700
33. Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S., & Sainath, T. (2019). Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2), 66-83. https://doi.org/10.1109/JSTSP.2019.2908700
34. Rai, A., Rout, B. P., Bhushan, & Mehta, A. (2022). Unveiling the power of data: A journey through machine learning techniques. International Journal of Advanced Research in Science, Communication and Technology, 42–47. https://doi.org/10.48175/ijarsct-17807
35. Reddy, R., & Shyam, G. K. (2018). Analysis through machine learning techniques: A survey. https://doi.org/10.1109/icgciot.2018.8753050
36. Regli, W. C., Hu, X., Atwood, M., & Sun, W. (2000). A survey of design rationale systems: Approaches, representation, capture and retrieval. Engineering with Computers, 16(3-4), 209-235. https://doi.org/10.1007/PL00013715
37. Relaño-Iborra, H., May, T., Zaar, J., Scheidiger, C., & Dau, T. (2016). Predicting speech intelligibility based on a correlation metric in the envelope power spectrum domain. The Journal of the Acoustical Society of America, 140(4), 2670–2679. https://doi.org/10.1121/1.4964505
38. Saravanan, R., & Sujatha, P. (2018). A state of art techniques on machine learning algorithms: A perspective of supervised learning approaches in data classification. In 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 945-949). IEEE. https://doi.org/10.1109/ICCONS.2018.8663155
39. Singh, A., Thakur, N., & Sharma, A. (2016). A review of supervised machine learning algorithms. In 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom) (pp. 1310-1315). IEEE.
40. Sterne, J., & Razlogova, M. (2019). Machine learning audio effects: The acoustic entanglements of artificial intelligence and music production. Popular Music and Society, 42(2), 178-196.
41. Talbot-Smith, M. (2001). Audio engineer's reference book (2nd ed.). Focal Press.
42. Talbot-Smith, M. (2010). Sound engineering explained. Routledge. https://doi.org/10.4324/9780080498171
43. Välimäki, V., & Reiss, J. (2016). All about audio equalization: Solutions and frontiers. Applied Sciences, 6(5), 129. https://doi.org/10.3390/app6050129
44. Vickers, E. (2010). Metrics for Quantifying Loudness and Dynamics 1.
45. Vinoth, K., & Datta, S. V. (2021). Core machine learning algorithms: A comprehensive study. International Journal of Innovative Technology and Exploring Engineering (IJITEE), 8(11), 2595-2604.
46. Whitaker, J., & Benson, K. B. (2001). Standard handbook of audio and radio engineering. McGraw-Hill.
47. Wong, P. C. (2021). A survey on unsupervised pattern recognition algorithms for data analytics. In International Conference on Innovative Computing and Communications (pp. 561-571). Springer.
48. Wu, X., Qiao, Y., Wang, X., & Tang, X. (2021). Unsupervised feature learning for audio. In Advances in Neural Information Processing Systems (pp. 1096-1104).
49. Zahra, F., Bostanci, Y., & Soyturk, M. (2024). Unsupervised machine learning for anomaly detection in Wi-Fi based IoT networks. IEEE. https://doi.org/10.1109/ICCSPA61559.2024.10794232