An Explainable Recommender System for Singing-music according to Individual’s Vocal Abilities Using Modified K-Nearest Neighbor
This project is the final project of RAC Data Science Camp (2023) with contributions of tangnatta and IcyAlmondE, published on IEEE Xplore at https://doi.org/10.1109/ICCI60780.2024.10532228.
Singing activities are becoming popular these days and many people sometimes reach the point that they don't know what to sing or don't know which songs are suitable for them. So the objective of this project is to make a recommendation system that can suggest songs that are suitable for the user's voice while also providing explaination to make them understand how to improve.
Why did we came up with this nearest neighbor thing? I can't really remember, but with the weighting of notes occurrences, it kinda worked pretty well.
The project is divided into 2 main parts: the song database creation and the recommendation. Here is the overview system of the project.
In the song database creation part, we gathered around 100 songs in Thai, English, and Japanese and extracted only the vocal part using Hybrid Demucs. Then we sliced the singing audio of each song into 10ms intervals and converted each interval to notes using CREPE while counting the occurrences of each note. The data was collected in a table with notes as binary and numeral occurrences of each note (See in the Data Prep folder).
In the recommendation part, we created an algorithm to find the similarity between user's singing data and songs in the database by converting the user's data to notes in each 10ms interval and using physics theory of music to calculate the singing score of each note. Then we used modified K-Nearest Neighbor, finding the Euclidean distance of the binary notes and weighting with the occurrences, to get the similarity score and rank them if the song is suitable for the user's singing voice.
We also designed a recommendation page showing the score of each note that the user can sing. The insight of each song is also available containing the number of notes in the song.
The system was tested by our friends and here's the results on the satisfaction of the recommendation and and the understanding of the explaination.
| Evaluation Criteria | Score |
|---|---|
| Satisfaction | 8.75 |
| Understanding | 7.05 |
Future work may focus on refining the algorithm and exploring additional features for more vocal styles.
The demo of the project is available at HuggingFace Space.