CITATION — REFERENCE ENTRY
Annotator subjectivity in harmony annotations of popular music — Journal of New Music Research
- Key
- koops2019-annotator-subjectivity
- Authors
- Koops, Hendrik Vincent; de Haas, W. Bas; Burgoyne, John Ashley; Bransen, Jeroen; Kent-Muller, Anna; Volk, Anja
- Issued
- 2019
- Type
- article-journal
- Container
- Journal of New Music Research
- Volume
- 48
- Issue
- 3
- Pages
- 232-252
Raw CSL JSON
{
"DOI": "10.1080/09298215.2019.1613436",
"URL": "https://pure.uva.nl/ws/files/62264010/Annotator_subjectivity_in_harmony_annotations_of_popular_music.pdf",
"page": "232-252",
"type": "article-journal",
"issue": "3",
"title": "Annotator subjectivity in harmony annotations of popular music",
"author": [
{
"given": "Hendrik Vincent",
"family": "Koops"
},
{
"given": "W. Bas",
"family": "de Haas"
},
{
"given": "John Ashley",
"family": "Burgoyne"
},
{
"given": "Jeroen",
"family": "Bransen"
},
{
"given": "Anna",
"family": "Kent-Muller"
},
{
"given": "Anja",
"family": "Volk"
}
],
"issued": {
"date-parts": [
[
2019
]
]
},
"volume": "48",
"container-title": "Journal of New Music Research"
}
Claims
-
In the Chordify Annotator Subjectivity Dataset, four expert annotators labelled 50 songs and overlapped on only 73% on average for major/minor labels and 54% for the most complex chord labels.
"Between annotators, we find only 73 percent overlap on average for the traditional major–minor vocabulary and 54 percent overlap for the most complex chord labels."
-
Koops and colleagues report that Ni and colleagues (2013), with five annotators and 20 songs by the Beatles and Queen, found that annotators disagree on roughly 10 percent of harmonic annotations.
"annotators transcribing the same music recordings disagree on roughly 10 percent of harmonic annotations."
-
Comparing MIREX 2017 results with their own annotations, Koops and colleagues suggest that the best chord estimation algorithms are overfitting to a single subjective annotation.
"overfitting to a single subjective annotation"
-
The annotators agreed on only 76% of root notes on average, and on only 59% for the most complex vocabulary.
"we find that annotators agree on average on only 76 percent of root notes. This disagreement increases with the complexity of chord labels, with only 59 percent agreement for the most complex vocabulary."
-
Koops and colleagues argue for a harmonic 'subjectivity ceiling', an upper bound for evaluation, and report that state-of-the-art chord estimation systems score beyond it by about 10 percent.
"Current state-of-the-art chord-estimation systems perform beyond this subjectivity ceiling by about 10 percent."
Available in