Making an orchestra speak

 “Speakings” is a piece for live electronics and large orchestra composed by Jonathan Harvey in 2008 and in collaboration with computer musicians Gilbert Nouno and Arshia Cont. The world premiere occured in August 2008 in Royal Albert Hall of London by BBC Scotish Orchestra. The composition was awarded the Prince Pierre of Monaco Prize in Musical Composition in 2009. A recording by BBC Scottish Symphony is available on Outhere Records. And a paper published in the 2009 Sound & Music Computing conference on the making of the piece was awarded the best paper.

The realization of “Speakings” was undertaken in two stages: The first, through Computer Aided-Composition and as an aid in preparing score materials; and second, real-time effects and performance realizations. On top of these were added various effects and controls not discussed here. We discuss the two phases below and share Demo Patches in the attachment to reproduce each item:

Voice to Music Score Transcription

In the first phase using OpenMusic and its OM-SuperVP library, selected voice soundfiles were transcribed to music scores. The OpenMusic patch is shown at the right. It uses SuperVP capabilities to analyze voice and capture partials, and uses RepMus Libraries to transcribe them to notes and quantize them rhythmically. This OpenMusic patch is available in the Attachment.

In addition, the Orchidée system suggested several orchestration schemes over selected patterns. These two steps provided enough material for the composer to create an orchestral score.

Stamping Voice Envelopes on Live Orchestra

While transcription and automatic orchestration provide orchestral patterns close to speech structures, they are far from rich spectral morphologies observed in speech. To reach these morphologies, a real-time algorithm based on FTM technologies was developped to enrich the spectral content of real-time orchestra using provided speech. The patch demonstrating this live process is available in the Attachment.


Speakings by J. Harvey, Performance Scheme

Live Performance

To assure synchronicity between live electronics and the orchestra during live performance, electronic queues are tracked via a keyboard player placed in the orchestra using score following technologies. Through Antescofo, the tempo of the performance is tracked and driven into SuperVP for Max and Gabor technologies in FTM for best synchronization of live treatements and the orchestra. In addition to this, a computer musician controls live spatializations and movements as well as electronic levels through a controller.

During live performance, computer music performers mix and control spatial parameters using multi-touch interfaces communicing with Max patches and live processings synchronize with the orchestra following the tempo and position of the orchestra with the electronics.

For more details, refer to the article by the composer and involved musicians and researchers published in SMC 2009 proceedings available online.