method-results · wu2024-realchords

We propose ReaLchords, an online generative model for improvising chord accompaniment to user melody. We start with an online model pretrained by maximum likelihood, and use reinforcement learning to finetune the model for online use. The finetuning objective leverages both a novel reward model that provides feedback on both harmonic and temporal coherency between melody and chord, and a divergence term that implements a novel type of distillation from a teacher model that can see the future melody. [...] Each chord must be generated without knowing in advance which melody note it will accompany. [...] Likelihood models, however, suffer from exposure bias due to being trained entirely on ground truth data, and transfer poorly to the online settings where mistakes, imperfections and stylistic differences are common [...]. We train our models on an updated version of the Hooktheory dataset [...] now contains 38K melody-chord pairs. [...] We recruited ten musicians, and collected 192 ratings