Fictions-Science : Alan Turing - screening of film in creative process

Fictions-Science : Alan Turing Screening of a film in creative process

Thursday, the 9^th of June 

8:30 PM

Centre Pompidou, Petite salle 

Axel Roebel and Frederik Bous from the Analysis-Synthesis team give us some information about the transformation of the voice during the event Fictions-Science: Alan Turing which will take place at the Centre Pompidou on June 9, 2022

The trajectory of the mathematician Alan Turing will be highlighted with various artistic and scientific events.

The screening of the video tale in progress by Judith Deschamps is part of the event. Still in the process of being created, the film is built around the recreation of Farinelli's song, which the visual artist is conducting within her artistic research residency with Ircam researchers Axel Roebel and Frederik Bous of the Analysis-Synthesis team.

More precisely, it is a question of recreating by means of deep neural networks the song Quell'usignolo che innamorato that Farinelli sang every night to the king of Spain to cure his melancholy. In order to recreate a realistic castrato voice, it was necessary to "augment" the voice of an alto singer by "hybridizing" it with a coding system. The technologies used in this project were developed in the ARS project. This residency is the first artistic use of the developed methods. It allowed to improve and refine the objectives of the software development.

"The basic principle is to teach a system made up of several deep neural networks to reconstitute the timbre of singers of various tessituras from a given signal and a target pitch. This makes it possible to transpose the passages of the song that the viola's voice cannot naturally reach," says Axel Roebel.

A very similar approach was used for the "creation" of Dalida's voice in the film "Hotel du Temps" by Thierry Ardisson, a project carried out in collaboration with IrcamAmplify. For both applications one of the networks is the same. It is the vocoder, it is used to convert a Mel spectrogram into a corresponding voice. The other network transforms the Mel spectrogram in order to adapt it to the indications given by the user. It differs depending on the task. For Judith Deschamps' project, the aim is to modify the pitch while keeping the identity of the alto singer who is the basis of Farinelli's song. Conversely for the project of Dalida, it is the identity that is changed by keeping the height. The method used is thus modular and allows adaptations for various applications.

Vocoder networks are now widely used to synthesize speech from text. However, the question of transformation with deep networks is still little studied. It is one of the current and main research activities in the Sound Analysis and Synthesis team.

The neural vocoder coupled with the pitch transformation network is developed in the framework of the ANR ARS project (https://ars.ircam.fr). The identity transformation network has been developed in the framework of the ANR project theVoice (https://www.ircam.fr/projects/pages/thevoice/) with a strong support of IrcamAmplify.

Axel Roebel and Frederick Bous from the Analysis-Synthesis team give us some information about the transformation of the voice in the creation of Fictions-Science : Alan Turing - projection of the videography tale.

Demo on the use of Mel spectrograms for voice transformation:

To go further in the research work around the voice :

A Bottleneck Auto-Encoder for F0 Transformations on Speech and Singing Voice.
Analysis and transformations of intensity in singing voice. 
Neural Vocoding for Singing and Speaking Voices with the Multi-band Excited WaveNet.