Music Plus One and machine learning — Proceedings of the 27th International Conference on Machine Learning

Claims

1. architecture-latency
   Music Plus One (also called the Informatics Philharmonic, named after Music Minus One play-along records) has three modules: Listen, an HMM score follower; Predict, a Kalman-filter-like Gaussian model of timing; and Play, which phase-vocodes an orchestra-only recording. Because detection latency is usually 30–90 ms, a purely responsive system cannot work, so the system continually predicts and revises the timing of accompaniment notes. The score follower waits until confident a note has begun, since reporting early can make the accompanist arrive before the soloist.

   "Our system is known interchangeably as the "Informatics Philharmonic," or "Music Plus One" (MPO), due to its alleged improvement on the play-along accompaniment records from Music Minus One that inspired our work. [...] Our system is composed of three sub-tasks called "Listen," "Predict," and "Play." [...] our detection latency is usually in the 30-90 ms. range, enough to prove fatal if the accompaniment is consistently behind by this much. For this reason we model the timing of our accompaniment on the human musician, continually predicting future evolution, while modifying these predictions as more information becomes available. [...] Predict, is a Gaussian graphical model quite close to a Kalman Filter [...]. The Play module uses phase-vocoding [...] to construct the orchestral audio output using audio from an accompaniment-only recording. [...] One of the worst things a score follower can do is report events before they have occurred. [...] the musical result often involves the accompanist arriving at a point of coincidence before the soloist does."

   Locator: section: 1–3.2 · Quote language: en
2. motivation-and-new-music
   At Indiana University's Jacobs School of Music, about 200 student pianists study concertos but the orchestras perform only two piano concertos a year with student soloists, motivating an accompaniment system. Raphael also sees new music composed for human–computer partnerships as a major opportunity, since the computer has nearly unlimited virtuosity but is weak at aesthetically satisfying interpretation; the Swiss composer Jan Beran wrote pieces for the system. For music without a regular pulse, a natural strategy is to wait for solo events and respond, as the IRCAM score follower does.

   "the JSoM contains about 200 student pianists, for whom the concerto literature is central to their daily practice and aspirations. However, in the JSoM the regular orchestras perform only two piano concerti each year using student soloists [...]. If the approach is based solely on audio, a natural strategy is simply to wait until various solo events are detected, and then to respond to these events. This is the approach taken by the IRCAM score follower [...]. The computer offers essentially unlimited virtuosity in terms of playing fast notes and coordinating complicated rhythms. On the other hand, at present, the computer is comparatively weak at providing aesthetically satisfying musical interpretations. [...] Several compositions of this variety, written specifically for our accompaniment system by Swiss composer and statistician Jan Beran"

   Locator: section: 1 · Quote language: en
3. rehearsal-learning
   Music Plus One must perform reasonably well without rehearsal and improve with rehearsal: after rehearsals it re-estimates its timing model to the soloist using the EM algorithm, training on the most recent several rehearsals because the soloist's interpretation drifts. It learns musicality by imitation, which suits the concerto setting but not settings needing independent musicality, such as accompanying early-stage students or opera, where its performance was poorest.

   "Our system must improve with rehearsal. [...] the system must "sightread" (perform without rehearsal) reasonably well. [...] Using one or more such rehearsals we can iteratively reestimate the model parameters {µn} using the EM algorithm, resulting in both measurable and perceivable improvment of prediction accuracy. [...] For this reason we train the model using the most recent several rehearsals, thus facilitating the continually evolving nature of musical interpretation. [...] Our system learns its musicality through "osmosis." [...] This manner of learning by imitation works well in the concerto setting [...]. Such a situation occurs with the early-stage accompaniment problem discussed in Section 1, as here one cannot learn the desired musicality from the live player. Perhaps the accompaniment antithesis of the concerto setting is the opera orchestra [...]. We observed the nadir of our system's performance in an opera rehearsal where our system served as rehearsal pianist."

   Locator: section: 4–5 · Quote language: en
4. scenarios
   Musical accompaniment systems are computer programs that serve as musical partners for live musicians, usually in a supporting role. Approaches include processing the musician's audio, improvisatory systems (e.g. jazz or call and response) that may compose their own part on the fly, and concerto-like systems that perform a precomposed part while following a live soloist.

   "Musical accompaniment systems are computer programs that serve as musical partners for live musicians, usually playing a supporting role for music centering around the live player. The types of possible interaction between live player and computer are widely varied. Some approaches create sound by processing the musician's audio [...]. Other orientations are directed toward improvisatory music, such as jazz, in which the computer follows the outline of a score, perhaps even composing its own musical part "on the fly" [...], or evolving as a "call and response" in which the computer and human alternate the lead role [...]. Our focus here is on a third approach that models the traditional "classical" concerto-type setting in which the computer performs a precomposed musical part in a way that follows a live soloist"

   Locator: section: 1 · Quote language: en