Ever since Johannes Kepler published his treatise intrinsically linking music, geometry and astronomy in 1619, science has looked to the universal language of music for inspiration to understand the relationship between time, speed, repetitions and cycles.
One can only wonder if the author of Harmonices Mundi (the "harmonies of the world") imagined that the theories he proposed would one day be used in the medical clinic to diagnose and treat patients suffering the painful effects of sports injuries.
The musical language of resonance, periodicity, patterns and spectra are well-known principles used. But it was meeting at a teaching conference that brought Dr. Domenico Vicinanza, Senior Lecturer and co-director of Sound and Game Engineering at Anglia Ruskin University and GÉANT network expert in Cambridge, and Dr. Genevieve Williams, an expert in Sports Biomechanics in the Anglia Ruskin University’s Department of Life Sciences, to realise that their divergent fields had more in common than meets the eye – or in this case the ear.
It is at this junction between sound and music analysis and scientific exploration that the two are pioneering new ways to apply the synergy in the field of biomedicine with remote sensing and data sonification.
MUSIC TO MOVE BY
Anyone who has ever seen the Bolshoi Ballet, Riverdance or a figure skating competition can appreciate the notion that music and human physiology can be intrinsically intertwined. If music is a structured language that enables us to examine and communicate periodicity, fluctuations, patterns and relations, what an ideal tool it can be to study the human body – a complex collection of cycles, periodicity, fluctuations and transitions. These properties are related to musical concepts such as pitch, timbre and modulation, vibrations and oscillations. Take, for example, the glucose regulation. Insulin that is produced in the pancreas creates periodic oscillation in blood insulin that stops the downregulation of insulin receptors in target cells. Oscillations in the metabolic process are so vital that constant inputs of insulin can jeopardise the delicate system from functioning properly.
Drs. Vicinanza and Williams are taking these concepts a step further in their lab. They are combing Vicinanza’s experience in applying scientific data sonification in physics and neurobiology, with Williams’ experience in kinesiology, biomechanics, quantifying coordination of human motor skills and dynamical systems theory in movement, and applying this combined knowledge to use remote diagnostics and analysis and data sonification to motor control and biomechanics treatment and diagnostic models.
“Our infrastructure in the lab at Anglia Ruskin is pretty unique,” says Dr. Williams. “We joined together my sport science lab with Domenico’s state of the art sound and game engineering lab. We use a variety of sensors that are in everyday use in a sports medicine clinic. Like flex sensors to determine bending or tilting, or a position sensor that tracks movement in 3D with sub- millimetre precision.”
“Rehabilitation, sports training and coaching increasingly rely on quantitative data and measurements,” explains
Dr. Vicinanza. “For that reason, biomechanics labs are equipped with devices able to measure 3D position, force and acceleration in an extremely accurate way. Remote sensing allows us to potentially collect data at spatial and temporal scales that is either unfeasible or difficult to implement with existing instrumentation. Until now though, very little has been done in the actual deployment of networked sensor-based infrastructures for sport and
THE POWER OF SOUND
The next step is to communicate the scientific data in unique, accessible and understandable ways. “Our approach is based on the idea of analysing the waveforms and their relationship by translating them into audible signals,” explains Dr. Vicinanza. “We use the natural capability of the ear to distinguish, characterise and analyse waveform shapes, amplitudes and relations.”
The sonogram or spectrogram is one of the main tools used in this process, known as data sonification, to investigate the structure of sound is. “We can use sonograms to examine the phase relations between a large collection of variables without having to reduce the data. Spectral analysis is used in many fields like high-energy physics. Now we are showing how it can be used in biomedicine. Sonification is a veritable ‘magnifying glass’ to analyse data.”
When coupled with remote sensing and networks, sonification can provide physicians, physiotherapists and patients a uniquely effective way to analyse data and provide accurate and personalised feedback without having to travel to a the clinic or hospital. Physicians can analyse sonograms generated by sonification in real time from anywhere in the world and give immediate and accurate feedback.
“Networking is the final, and in my mind, probably one of the most innovative ingredients,” says
Dr. Vicinanza. “Low latency, extremely high availability and a global footprint are crucial to make this possible. National Research and Education Networks (NRENs) provide just that high-quality connectivity and extended reach. We connected our lab to the GÉANT network, through the UK research and education network JISC so that we are able to send or receive measurements across the globe.”
Their work has far-reaching ramifications in the fields of physical therapy, rehabilitation, sports medicine and athletic training. “We are experimenting with mapping data to sound in the very creation of this music. Our brains are ‘hard-wired’ to understand and internalise this audio feedback in ways much more powerful than visual perception,” explains Dr. Williams.
But quantitative audio feedback is not only one of the most promising ways to help athletes perform better; it is an ideal tool to reduce the risk of injury. “For an athlete in training, audio feedback
can keep him or her on track even when they are not under the scrutinising eye of the coach. Also patients in physical therapy who need to perform their exercises on their own. With the benefits of remote sensors and networking, athletes and patients can join together to share experiences and support one another.”
Drs. Vicinanza and Williams are quite literally “making their own kind of music” – music that is forming a foundation for new diagnostic and treatment modalities. With the help of R&E networks, physicians, physiotherapists and patients will one day benefit from models that lower costs, facilitate faster response time in the clinic, improve training, and bring about more effective recovery and rehabilitation from sport injuries
WHAT IS DATA SONIFICATION?
Data sonification is the use of audio signals to convey information or perceive data. Auditory perception of complex, structured information could have several advantages in terms of temporal, amplitude, and frequency resolution when compared to visual representations and often opens up possibilities for an alternative or complement to visualisation techniques. These advantages include the capability of the human ear to detect patterns, recognise timbres and follow different strands at the same time. This would offer, in a natural way, the opportunity of rendering different, interdependent variables into sound in such a way that a listener could gain relevant insight into the represented information or data. Presented at TNC 2016 https://tnc16.geant.org/core/presentation/729