Listening to Data: The Promising Sounds of Sonification
What is sonification?
Turning data into sound, or sonification, has been in the headlines lately, fascinating as it is because, well, “music from space” is part of it. Our interest in sonification is also reflected in pop culture: Have you seen the recent X-Files episode, in which agents Mulder and Scully investigate strange noises seemingly coming from the skies, which Mulder describes as “God blowing his own horn”?
Of course, music from space isn’t quite that. In a November 2015 article by Rossella Lorenzi, published on the Discovery News website, “Hear What the Earth Sounds Like,” Domenico Vicinanza, director of Anglia Ruskin University’s Sound And Game Engineering (SAGE) research group, talked about the algorithms he has developed with his colleague Genevieve Williams, to “give a specific pitch and melody to each image sent back from the satellite.”
“Sonification gives space research a new dimension. When you hear the resulting music you really are hearing the data,” commented Vicinanza, who, in addition to being a physicist, is a classical composer.
Sonification was born in the 1960s
As was described in the excellent March 19 article in The Economist, sonification came to be in the 1960s, when the seismologists started using instruments to “record local tremblings as frequency modulations on magnetic tape.”
The article says:
Days and weeks of data lay on these tapes, and the only way to sift through them for interesting events was to play them back at high speed, listening for any anomalies. […]
When the Voyager 2 space probe passed Saturn in 1981 and sent back a stream of data that its keepers could not decipher, they sonified it. The hailstorm-like sound, they determined, came from debris in Saturn’s rings striking the craft.
Audification and Dr. Robert Alexander
As the The Economist explains, most of scientific research and other efforts in the field of sonification fall under the category of audification:
When a single measure is taken at regular intervals, the result is a simple stream of regularly spaced points. Audification is simply the shifting of such “time-series data” to a frequency within the audible range — exactly what the seismologists, Voyager 2 scientists and lots of early radio astronomers did. As a reflection of change in just one variable in time, audification results in simple tones and rhythms.
Robert Alexander, a design scientist from the University of Michigan (and a classically trained composer) spent some time at NASA’s Goddard Space Flight Center working with data from satellites “measuring the stream of particles from the sun.”
Dr. Alexander has apparently found “that ions of carbon are better at indicating the regions from which the solar wind originates than the oxygen that was previously employed, and determined the cause of a long-lived storm of swirling particles within the wind. He even spotted an undocumented source of noise in the instruments aboard the Ulysses spacecraft.”
Visual analysis of data is still the preferred method in science, but vision has its limits, and hearing, in turn, offers some considerable strengths. Did you know that humans can “hear frequencies across three orders of magnitude, from about 20 to 20,000 hertz, and can discern tiny differences in those frequencies”? Hearing’s temporal resolution is 100 times finer than that of vision, yet, science still relies on visual display technology, and is overloaded by data as a result.
Dr. Alexander realizes the limitations of visual data analysis, and is championing audification. “NASA produces a vast amount of data from its satellites,” he was quoted as saying in an article published on NASA’s website. “Exploring such large quantities of data can be difficult. Sonification offers a promising supplement to standard visual analysis techniques.”
Sonification is everywhere, so let’s use it
The term “sonification” means that any type of data or measurement can be displayed as sound, which describes a vast amount of those — from the heart rate monitor beeps to a ringing door bell, to satellite recordings of electromagnetic fluctuations. Other everyday examples include the Geiger counter, used to measure radiation; or a sonar technique, used underwater.
So, why not use it for visual analysis? One example is how together with space scientist Robert Wicks, Dr. Alexander analyzes “observations of the sun,” studying the solar wind using the data from NASA’s Wind satellite. Using audification, Dr. Alexander translates raw satellite data into audio files.
A Penn State professor Mark Ballora thinks sometimes using ears is better than using eyes, “particularly when it comes to detecting patterns and dynamic changes.”
In an April 4 article for Foreign Affairs magazine, he explained that while data sonification is certainly valuable for the visually impaired, it could have wider usage, as “sound can perhaps present information with more urgency and clarity than a visualization,” as the article notes.
… And make it into art
If science can be experienced via an audio file, why not also make art using sonification? For example, Ballora collaborated with the Nobel prize-winning physicist George Smoot and the drummer for the Grateful Dead Mickey Hart on a project called “Rhythms of the Universe — an exploration of the cosmos.”
Also, Ballora’s project featuring sonifications was exhibited in November-December of last year at Borland Project Space. Titled “Sonifications of the Universe (and more),” it displayed “data sets describing natural phenomena… as sound,” exploring “how sound can envelop viewers, creating a visceral experience that is not possible to achieve when looking at imagery alone.”
The exhibit featured sonifications of hurricanes, among others, and even “arctic squirrel body temperatures and data about Antarctic ice.”
Sonification has a way to go, in terms of research and acceptance, but it sounds promising. The Economist quoted Paul Vickers of the University of Northumbria, in Britain, who notes that “progress is being made in breaking down scientists’ conservative nature and bringing in aesthetic elements in a serious and rigorous way.”
“We’re seeing people working in the scientific domains engaging with philosophers of music and psychologists,” says Vickers, “and thinking about how we listen in the real world to inform how we design sonifications.”
The trees are making music
Finally, here’s a little extra nugget I couldn’t resist including: Someone managed to translate thesounds of tree rings into music, and the results are hauntingly beautiful. Bartholomäus Traubeck used sensors that could “read” the wood’s color and texture, and employed an algorithm to put it into piano notes.
Turns out, each type of tree has a different sound! You can hear snippets or download the album, aptly titled “Years,” featuring beech, spruce, oak, walnut, and other trees, here.