Virtuosic Embodiment of a Modalys String through Color Tracking Technology by Yongwoo Lee
ABSTRACT
This paper introduces the use of Modalys (a physical modeling synthesis tool) String in live settings by utilizing color tracking technology through MaxMSP/Jitter. This essay explores the approach of creating highly detailed and creative sounds by adjusting parameters such as the density, radius, and length of various physical materials. Furthermore, it discusses methods for effectively performing and composing in a live environment.
1. INTRODUCTION
Modalys String is divided into a mono-string object, primarily used for plucking, and a bi-string object, used for bowing. Each object generates sound by receiving values for horizontal and vertical positions, simulating the actual friction between the bow and the string. Additionally, factors such as weight, rosin, and access position significantly influence sound generation.
2. PARAMETERS OF MODALYS STRING FOR GENERATING SOUND
In the experimentation of bowing access points for Modalys String, several parameters were analyzed. The vertical position ranged from -0.001 to 0.001, where values below the minimum (upper side) produced a light, harmonic, sul tasto sound, while values at the maximum (lower side) created a harsh, tough, sul ponticello sound. The horizontal position varied between -4 and 4, determining the lateral smooth movement. Weight was tested between 0.7 and 1.0, affecting the pressure applied to the bow and thereby influencing the dynamics of the sound. Finally, the access position ranged from 0.5 to 0.01, controlling the precise point of contact on the string and significantly impacting the tone quality.
Figure 1. MaxMSP patching for the primary parameters for Modalys.
3. A DIVERSITY OF MATERIALS FOR MODALYS STRING
Modalys creates strings from a variety of materials by adjusting the density and Young's modulus values of different metals, woods, and synthetic materials. The range of material properties studied at IRCAM spans from spider silk to uranium, allowing for extremely nuanced changes in sound. This information can be input into Max/MSP, making it easy to implement and manipulate sounds in real-time.
Physical modeling synthesis with Modalys requires recomputation each time parameters change, which can lead to clicking and other issues. Traditionally, the melt-hybrid object has been effectively used when changing materials or pitches. Therefore, I connected the two strings in a hybrid configuration and implemented a separate mono-string for plucking, thus completing the Modalys coding.
Figure 2. Modalys coding in the Max/MSP environment.
Additionally, materials and pitch changes through the hybrid system are designed so that while String 1 is producing sound, String 2 undergoes modifications over time. These changes and visual information are then patched and displayed to the performer.
Figure 3. Example of the pictures of the pitches and materials in the hybrid object section.
4. COLOR TRACKING APPROACH VIA JITTER
Important parameters of Modalys String are adjusted by tracking colors captured by the camera through Jitter. Specifically, the blue globe on the left controls the hybrid position, enabling changes to the pitches and material properties between the two strings. The green and red globe on the right is used, with the red side for bowing the string and the green side for plucking the string. Three-dimensional gesture control is used for manipulating these parameters through Jitter.
Figure 4. RGB globes for the color tracking.
The three-dimensional gesture control for Modalys String operates as follows: the horizontal value obtained through Jitter adjusts parameters related to bowing, such as the horizontal and vertical positions and the weight. The vertical position value modifies the rosin, access position, and timbre (including constant-loss and frequency-loss).
Finally, the third dimension, the Z-axis, uses the amount of color tracking as its value. This is represented as white on a black screen to provide an instant visualization of the amount.
Figure 5. The degree of color exposure is represented in black and white.
5. DETAILED APPROACH TO IMPROVISING STRATEGIES
The strategies applied for using this patch for live improvisation are as follows:
The green and red globe on the right-hand side is used to control different aspects of the string. Specifically, the red side is designated for bowing the string, while the green side is used for plucking. Since these colors are opposites, the Z-axis value, which reflects the amount of green color, indicates their contrast. As a result, an increase in the Z-axis value of the green color reduces the amplitude of the bowing sound, while the red color, being its opposite, enhances it. This method effectively intercrosses the bowing and plucking sounds by utilizing the color contrast to adjust their relative intensities.
Additionally, by using a MIDI pedal for pedaling, random materials and pitches will appear. These images and pitches will provide visual information that influences the performer in a virtuosic manner.
Specifically, the left hand (blue color globe) controls pitch and material changes by smoothly adjusting the hybrid position.
Moreover, anticipated benefits include precise cues for sound processing (such as AM, RM, etc.), the ability to work with limited scales for specific pitches, and the integration of audiovisual elements with material properties. This approach allows even those who do not know how to play a string instrument to perform immediately, while a deeper understanding of material properties opens up numerous possibilities for creating innovative music.
6. CONCLUSIONS
Modalys is known to be a challenging coding program to handle in live settings. However, efficient coding through color tracking in a controlled environment, with a focus on key parameters, has proposed the possibility of effective performance and creation in live scenarios. This patch is designed to be usable even by those who do not have prior knowledge of Max/MSP/Jitter, or Modalys, and will become even more efficient through optimization and updates on some details.
This talk will be presented during the Ircam Forum Workshop in Seoul at the Seoul National University