Mangrove (=KANDEL) of Memory ~ HOA Proximity Acoustic Expression through Integrated Wearable Auditory AR and Multichannel Loudspeaker Systems - Hiromichi Kitazume, Jin-Young Lee and Ken Ito

HOA Proximity Acoustic Expression through Integrated Wearable Auditory AR and Multichannel Loudspeaker Systems

Présenté par Hiromichi Kitazume, Jin-Young Lee et Ken Ito

Mangrove (=KANDEL) of Memory
…Proximity Acoustic Expression through Integrated Wearable Auditory AR and Multichannel Loudspeaker Systems

ITOKEN   +  JIN-YOUNG LEE  +  HIROMICHI KITAZUME
Division of Conducting and Composition The University of Tokyo

 -

 -

α

In 2017, under the supervision of Dr. Hideki Shirakawa, we developed the above Auditory AR Actuator by laminating a conductive polymer onto an organic piezoelectric film.

     

 Then, in 2023, using IRCAM technology, we established PAAR <Proximal Auditory AR> system, which sends sound from external speakers to wearable AR headset.

In 2017, under the supervision of Dr. Hideki Shirakawa, we developed the above Auditory AR Actuator by laminating a conductive polymer onto an organic piezoelectric film.

Then, in 2023, using IRCAM technology, we established PAAR <Proximal Auditory AR> system, which sends sound from external speakers to wearable AR headset.

.   

In 2007, when "Gruppen (1957)" was re-performed at the Lucerne Festival 50 years after its premiere, we had the opportunity to support Kürten's Stockhausen, Pierre Boulez and Peter Eötvös as a liaison.

On 1958, Boulez composed Poésie pour pouvoir in response to "Gruppen", which was groundbreaking in terms of spatiality in music, but Boulez was not satisfied with the result. Finally, Boulez solved this problem by founding IRCAM, developing the core technology and completing his Répons (1981), but THIS SPATIALITY of sound always remained BETWEEN SPEAKERS: FAR AWAY from the listeners' EARS. OUR WORK is the first musical invention in which sound is DELIVERED to the listener's EARS from distant speakers DIRECTLY.

β

Along with the issue of spatiality, one of the topics Boulez sought to pursue in Poésie pour pouvoir was the expansion of Schoenberg's ``Sprechgesang'' problem, which asks the difference between singing and speech, Boulez attempted to extend his efforts in Le marteau sans maître (1953-55) to electro-acoustic music, with the idea of treating the spectrum of instrumental timbre and the human vocal formant as continuum/discreteness. It also has close connection to his "temps strié" and "temps lisse".However, not only the analog technology during 1950's, but also the early digital technology of 1981, could not bridge these profound gaps among timbre, voice and speech.

In 1995 We began collaboration with Pierre Boulez in Tokyo, but prior to this, in 1987, Luigi Nono (1924-90), who visited Tokyo only once during his life, asked us about the problem of ``Sprechgesang'' and "Klangfarbenmelodie''; these two could be treated in parallel, and also with "spatiality'' Between 1997 and 98, we partially solved this problem by applying sinusoidal decomposition, which tears apart spoken language into minute sinusoids, according to the mechanism of the human auditory system, particularly the hair cells in cochlea. We had opportunity of short discussion with Gérard Grisey at the Théâtre du Nord on this, but unfortunately shortly after that he suddenly passed away, and 25 years had rapidly passed.

      

In our work, vocal document of the poet Frank Diamand (1939-), a survivor from Bergen-Belsen concentration camp, reciting his own poetry, is torn like threads of sinusoidal fragment.
       

Following this process in reverse, as hysteresis of brain cognition, reconstruction of linguistic sound image from sinusoids, and even from instrument-like timbre, is possible.

 

γ

In 1987, during his once-in-a-lifetime visit to Japan, Luigi Nono had a seminar held at the University of Tokyo about his "Prometeo - tragedia dell'ascolto" (1981-85). He mentioned of the "spatiality" as an attribute of music that is "neither pitch nor rhythm". An idea of positioning "spatiality'' as an extension of "timbre" is referred. Schoenberg's original definition of "Klangfarbenmelodie" is "changes in timbre" themselves could be conceived as ``melodies''. In this vision spoken language could be also a kind of "changes in timbres" and the problem of "Sprechgesang'' is also treatable as a part of "Klangfarbenmelodie", and so was the "spatiality" as a special extension of the concept of timbre.
When we add up the ``sinusoidal parts'' of a linguistic sound, simple fragments consist of complex timbre change, and finally the meaning of the spoken language becomes audible.

     

      1 sinusoid piece.                         2 sinusoid pieces.                       3 sinusoid pieces                  Verbal meaning audible

If we apply a similar operation to instrumental timbres, adding or deleting specific partial spectra, even 
only with instruments --- without electronic system can make things different.
Creating new fingering of instruments would change whole the music experience.

     

   Bass clarinet longtone                         Pulses                                Double color trill                      Triple color trill

In this work, by adding or removing spectra either instrumentally or electronically, we prepared for the situation in which the changings and spatially floating timbre themselves create rhythm and melodic motives.

 

Ω

Our friend Frank Diamond (1939-), a Jewish Dutch poet and film director, is a Holocaust survivor who was liberated from the Bergen-Belsen concentration camp as a 6-year-old boy. Below are excerpts from his poem.

           

“I was looking
for the lady oblivion
but she wasn’t there.
I needed her like death,
my complement.

So, we’ll meet again when I’m old.
You’ll suck my womb, and burn my brain.”
 

“Mangrove=KANDEL de memoire” is composed of three layers of sound. The first layer is an "external world" characterized by the pulse of a bass clarinet reverberating inside the room, in which only a limited spoken languages are audible. The second layer is "voices of border crossers'' that jump into and run away from listener's ears from "external'' when wearing auditory AR headset. The third layer is "Hidden Voices'' , read by Frank himself, which can only convey verbal meaning through AR head set.
Once hearing this ``hidden voice,'' listeners realize that victims' voices are omnipresent in the external echoes which are previously thought meaningless. This "Hysteresis" of memory was designed after the thoughts of an originally historian and psychoanalyst, who left Vienna at "Kristallnacht"(1938) at the age of 9. He evacuated to the US and later on known as Nobel neurophysiologist of memory, Eric Kandel(1929-), who succeeded in tracing the "memory suppression" through the process of molecular science.

The work is dedicated to Frank, and to Mrs. & Dr.Kandel.

-

Back to the event