Crossroads AR - Colorful Paintings and Spatial Music in Augmented Reality - Bill Parod, Teresa Parod
Painter Teresa Parod and Composer and Software Developer Bill Parod have been collaborating on multi-media public art. Their work combines mural paintings with situated spatial music and sound art using a custom Augmented Reality mobile app.
Crossroads is a new project to bring this approach into a gallery setting by combining Teresa Parod's paintings with associated works of Bill Parod's spatial music, realized in a multichannel exhibition space. Each of the Crossroads paintings contribute interacting spatial music to the gallery space when viewed with the Crossroads AR mobile app. The music is formed from animal and musical characters interacting in the 3D personal space of the mobile app and also heard in the shared space of the gallery through multichannel speaker dispersion, forming an interplay of visual, aural, and viewer/listener participation.
We come to crossroads all the time in our lives. When two roads intersect, they present three choices in where to go or which direction to take. That impact might be immediate or delayed, minor or profound. You make a decision which direction to go. Either direction could have been a very nice life, but we have pivotal choices. Why do we make the choices? Do we have regrets for not making a different choice? Do we think of Robert Frosts’ “The Road Not Taken”?
Working on Crossroads AR with Bill (my lifelong partner) is a joy and I welcome his interpretation and collaboration in this work. Bill took the shape and colors of my paintings’ roads and put them in flight and movement within the AR. We have always considered Stevie Wonder’s “Ribbon in the Sky” as “our song”. When I saw Bill’s first graphics in Crossroads AR, I thought - these are like ribbons in the sky. They reflect and elevate the outcomes of our our lives’ choices at their more spiritual level. - Teresa Parod
I am fortunate to be surrounded by Teresa's art and art making. Her eye for color, shape, and joy in the world surrounds me every day. I've lately been working on a music arising from the social behaviors of musical characters. This draws much from my enjoyment of listening to birds in the forest or the park - the rise and flow of their daily dramas in the passing day. So, birds inhabit this music as well, affecting each other but also the musical characters they live with. Because these events are spontaneous, the music is realized in a mobile app, in order to make emergent drama possible and though characteristic to the characters, also unpredictable. My aim is that the vividness of the characters’ personalities are shown in their interactions as they cross paths, and perhaps change course, making the experience feel like a living music as possibilities present themselves, develop, and resolve in ways both natural and surprising. It seemed like an apt musical approach to this Crossroads collection of Teresa's paintings. Our hope is that by combining our work in Crossroads AR, the viewer and listener will be drawn to their own reflections of life’s choices. - Bill Parod
The app offers a few types of interactions to the listener as well. In addition to zooming closer or farther from the paintings and moving within the space, turning the phone face down will rest the musical characters, leaving sound to just the birds. Bring the phone face-up and the sounding birds will invite instruments back in. They in turn cause others to react, and the music will begin to form again, though in a different way. Shaking the phone will alert other musical characters. Those in turn will alert others in the piece, forming new dramas, harmonies, and textures. The app can be played as an ensemble in his way. Or (our preference) just put the phone down and go deep, listening to the piece evolve over longer stretches of time.
The app can also use pitch detection to interact with musical characters based on audio input. This is intended for player improvisation with the ensemble characters. This feature is still early in its development and requires more work and feedback of players before its general release in the app. If you would like to work with this, let us know.
As a technical note, the app can send the characters’ play and position events to an external listener port via a simple OSC grammar. This is used to cast the experience to external multichannel spatial dispersion and recording strategies.
The first Crossroads AR gallery exhibit will be at IRCAM Forum Workshop - Special edition VR/AR Spatialization, Spring 2023 at Ircam, Paris, France.
The AR app is built using the Unity Game Engine, sending OSC messages of audio characters' sound file and position information to a central computer running Max/MSP/Spat5 for multichannel 3D dispersion in the gallery space.
For more information, see Crossroads AR.