nn/mémoire: Embodied Latent Space Walk by Jiatong Liu

Ai field recordings heritage ircam forum workshops 2026 nn~ sound walk

Exploring expressive manipulation of machine-learning models as a sound-design tool for storytelling. Drawing on cultural soundscapes, nn/mémoire provides an alternative lens to artifical intelligence, memory and heritage.

➡️ This presentation is part of IRCAM Forum Workshops Paris / Enghien-les-Bains March 2026

Presented by: Jiatong Liu

Biography

Rationale

Sound is a time-based media - the act of recording captures moments past. Site specific field recordings encapsulate the moment of time in that particular space and the story behind it.

nn/memoire grasps the temp-aurality of Beijing Hutong by recording its cultural soundscapes - sounds of community activities popular from the 1980s to today that are organically shifting and slipping away.

Modifying such recordings deforms and alters our perception of memory and past time. Some memories are lost, while others float to surface through the uncanny experience of remembrance.

The project uses Machine Learning as a form of sound design to alter our perception of Hutong soundscapes and memory, curating an artificial practice of remembrance.

Artistic Approach

By collecting a small dataset of Hutong soundscapes, I mapped the sounds into an immersive environment. The player’s movement (trajectory, velocity, direction) generates the audio in the space forwards, backwards and sideways, creating an eery virtual soundwalk of Hutong sounds.

Machine Learning as Sound Design

The project inquires the potential of Machine Learning in producing novel musical affordances as a form of sound design in storytelling. Here, by altering the inference of a VAE model, I reproduced audio textures reminiscent to snippets of memory.

Moreover, the spatial mapping of audio - made possible with the chosen Machine Learning technology - explores embodiment in Human-Computer Interaction.

Technical Implementation

  • Making the audio terrain: applying Expressive Manipulation to a VAE model using Latent Terrain Synthesis, creating a 2D audio terrain
  • Polishing the audio terrain: development of helper tools and techniques to eliminate unwanted audio artefacts in the audio terrain, from a latent vector level and audio engine level
  • Matching the audio terrain: creating an immersive environment in Unreal Engine with local 3D scans, with 2D coordinates matching the audio terrain
  • Connecting the audio terrain: connecting Max and Unreal engine with OSC