Professor Jeong Dasem of the Department of Art and Technology at Sogang University and his research team have co-authored a paper with Professor Mark Gotham of King's College London that was awarded the Best Paper Award at the International Society for Music Information Retrieval Conference (ISMIR) 2024, a top conference in the field of music artificial intelligence (AI). This is the first time a domestic research team has received the Best Paper Award at ISMIR.
ISMIR, a leading international academic conference in the fields of music informatics and music artificial intelligence, is a prestigious conference where various universities worldwide, as well as corporations such as Meta, ByteDance (TikTok), Adobe, Sony, Spotify, and Yamaha, present papers. It is a venue for the presentation of influential papers in the fields of music and musicology. ISMIR 2024 was held in San Francisco, USA, from Nov. 10 to Nov. 14.
The research team proposed a method to regenerate the 15th-century compositions 'Chihwapyeong' and 'Chwipunghyeong' by King Sejong, which are currently transmitted only as song scores, using deep learning to play them with six instruments in the traditional court music notation.
The research team first developed an optical music recognition technology capable of automatically recognizing the traditional court music notation from the National Gugak Center, establishing a traditional court music notation dataset. They also devised an encoding method to utilize the notation efficiently and employed a transformer-based language model to regenerate the melodies of the 15th-century 'Chihwapyeong' and 'Chwipunghyeong' in a form suitable for modern performance. Based on the regenerated melodies, they completed a performance score for the current arrangements of traditional court music. They implemented and released an interactive web demo so that general users could easily try the model, and the final generated output was performed during the 627th anniversary of King Sejong's birth held at Gyeongbokgung Palace on May 14.
The research team noted, "This study is significant in that it proposes the first encoding method that directly utilizes the traditional notation format and publicly shares a dataset built with optical recognition, as well as in expanding the realm of music AI by addressing non-Western music like Korean traditional music in a suitable manner."
References
arXiv (2024), DOI: https://arxiv.org/abs/2408.01096