In the early days of the pandemic, Giampaolo Martinelli, Consultant Cardiothoracic Anaesthetist at St.Bartholomew’s Hospital in London collected ECG recordings of a heart transplant COVID-19 patient and the medical team during a ward round. He approached us to sonify this data to show the medical team coming together to support the patient.
On Saturday, 26 June 2021, Emma Frid presented the resulting work as reported in “Mapping Inter-Cardiovascular Time-Frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a COVID-19 Patient and the Medical Team” at the International Conference on Auditory Display (ICAD2021). The presentation was followed by a Q&A session in Gather Town.
Frid, E, M Orini, G Martinelli, E Chew (2021). Mapping Inter-Cardiovascular Time-Frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a COVID-19 Patient and the Medical Team. In Proceedings of the International Conference on Audio Display, online, 25-28 June 2021.
The paper is available on HAL and will be archived in the collections of the ICAD community.
Title: Mapping Inter-Cardiovascular Time-Frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a COVID-19 Patient and the Medical Team
Authors: Emma Frid (KTH, CNRS–UMR9912/STMS (IRCAM)), Michele Orini (University College London), Giampaolo Martinelli (Barts Health NHS Trust), Elaine Chew (CNRS–UMR9912/STMS (IRCAM))
Abstract: This paper presents exploratory work on sonic and visual representations of heartbeats of a COVID-19 patient and a medical team. The aim of this work is to sonify heart signals to reflect how a medical team comes together during a COVID-19 treatment, i.e. to highlight other aspects of the COVID-19 pandemic than those usually portrayed through sonification, which often focuses on the number of cases. The proposed framework highlights synergies between sound and heart signals through mapping between time-frequency coherence (TFC) of heart signals and harmonic tension and dissonance in music. Results from a listening experiment suggested that the proposed mapping between TFC and harmonic tension was successful in terms of communicating low versus high coherence between heart signals, with an overall accuracy of 69%, which was significantly higher than chance. In the light of the performed work, we discuss how links between heart- and sound signals can be further explored through sonification to promote understanding of aspects related to cardiovascular health.
