Concerts Conferences Music Workshops

Piano-AI Dialog and Heart-Breath Visuals in Mystery Machines at MES2024

A collaboration between ERC projects REACH and COSMOS, Elaine Chew and Gérard Assayag present Elez / Boulaine – a semi-improvised dialog between a pianist and generative electronics with real-time visualisation of performer physiology – at Mystery Machines, the closing concert at Mathemusical Encounters in Singapore, an NUS Institute for Mathematical Science workshop. The concert took place at 8pm on Thursday, 22 February 2024, in the Concert Hall of the Yong Siew Toh Conservatory of Music.

Program Notes
Conceived as an anticipated hommage to Pierre Boulez, a year before the centenary celebrations, this duet piece creates an open, semi-improvised dialog between dueling musical streams drawing from the rich musical history that shaped Boulez and his music, and from original sonic materials (« Boulez Reloaded »).
Elaine Chew performs, adapts, reimprovises excerpts of pieces including  Boulez’s Sonata (1946) and Fragment d’une ébauche (1987), Messiaen’s Vingt Regards (1944), and Ravel’s Jeux d’eau (1901). The Somax2 cocreative AI program, based on REACH research, listens to Elaine’s playing and uses its generative model to react to it and intervenes in an improvised manner, continuously keeping pace with both the live musician and its own underlying musical logic. Elaine’s performance transforms the pieces through extreme distortions of timings and expressions in response to Somax’s generative agent repartees to her actions at the keyboard, so the musician and the machine’s reciprocal listening and adaptation produce rich and previously unheard combinations. Gérard Assayag guides the somax2 agents in a way that is itself an improvisation, by steering now and then the system towards different interaction strategies or sonic materials depending on the musical situation, exemplifying a multifactorial « cocreativity ». In this cocreative extreme performance, the players’ electrical heart activity, breathing, and stress will be projected visually in real time thanks to COSMOS-HeartFM technology.

Credits
European projects ERC REACH (Raising CoCreativity in CyberHuman Musicianship, PI G. Assayag), ERC COSMOS (Computational Shaping and Modeling of Musical Structures , PI E. Chew).
Thanks to Jean-Brice Godet, Valérie Philippin, Benny Sluchin, Francesco Guerri, and other REACH music research residents, who help us by training Somax2 with their wonderful musical material, and the illustrious engineers Joakim Borg (Somax2) and Charles Picasso (HeartFM) for their software development.

Boulez Homage : Elez / Boulaine : somax2 + heartfm
I. One-minute Highlights
II. Full-length (with HeartFM visualisation)
III. Full-length (without HeartFM visualisation, only Somax2)

I. One-minute Highlights

One-minute highlight of Elez / Boulaine : Fragment d’une ébauche and somax2 + heartfm (highlight)

One-minute highlight of Elez / Boulaine : Ravel Jeux d’eau and somax2 + heartfm (highlight)

II. Full-length Somax2 with HeartFM

Full version of Elez / Boulaine : Fragment d’une ébauche and somax2 + heartfm

Full version of Elez / Boulaine : Ravel Jeux d’eau and somax2 + heartfm

III. Full-length Somax2 only

Full version of Elez / Boulaine : Fragment d’une ébauche and somax2 only (without heartfm visualisation)

Full version of Elez / Boulaine : Ravel Jeux d’eau and somax2 only (without heartfm visualisation)


Photos

Boulez homage : Elez / Boulaine in Mystery Machines, Mathemusical Encounters in Singapore, YSTCM