dataset-quality · foscarin2024-beat-this

While exploring the datasets, we found multiple problems in the annotations, and we think this hinders the development of better models, especially for downbeat predictions. Even the GTZAN dataset, which is commonly used for evaluation, is not immune to quality problems. Some of them are evident and not debatable [...]. Furthermore, there are pieces where even for experts it would be hard to agree on a unique beat and downbeat annotation [...]. Finally, some pieces question the primary assumption of beat tracking, i.e., that there is a beat/downbeat to track [...]. correcting the existing commonly used datasets, and producing new annotated data for underrepresented genres is a crucial step for further development.