Audio Visualizers, Signals and EEG
| January 24th, 2011Since I last wrote for this blog I’ve been further exploring data visualization in Processing. For practical reasons, I’ve been primarily dealing with audio data. This has given me a few opportunities to test out this software, displaying visuals at a few house parties as well as at the New Bridge Project’s regular after parties. I’ve largely been developing Anthony Mattox’s music visualizer code, combining it with the lessons learned in Dan Shiffman’s Learning Processing. Readers of Shiffman’s book should recognize the use of the nested push and pop sketch to develop orbiting structures. The most recent visualizer, with its rotating mobius strips, uses these coding techniques whilst rotating in 3 dimensions. Demos of my visuals are available to watch on my Vimeo account
http://vimeo.com/user5033658
Working on these projects, it has become increasingly apparent that audio, despite all the aesthetic, romantic and artistic notions often applied to it, is just a series of floating decimals when interpreted with a computer. I am fully aware that this is not a particularly new or profound concept. Ryoji Ikeda’s minimal electronic pieces have long explored the notion of signal composition, whilst the workings of the Cochlear implant seems based on the technological conversion of kinetic energy to electrical impulse. To put it simply, our audial perception can be seen as the conversion of a signal from kinetic energy, to chemical reaction, to electrical impulse. I am interested in exploring this concept further in my work.
Another issue that has emerged working on this project is the limitations of working with audio data. Many of the visualizers have limited user interactivity. Some use audio alone as the variable within their equations, with sound controlling the size, translation and colour of the imagery. All of them include boolean switches, with which various imagery is toggled on off using keystrokes, with audio controlling variables within these preset images. This is useful in a performance setting, allowing the user to manipulate the imagery with a series of keystrokes and movements of the mouse. However, in an installation environment, this setup is somewhat limiting, and would require an interface and instructions with which to manipulate the installation. I am eager to explore the concept of audio visualization in an installation environment, because I believe it could be developed into a highly dynamic and interactive installation piece. Furthermore, by using the ambient audio of the exhibition space, the piece would appear to subvert the traditions of the art gallery. Galleries have often been seen as a place for quiet reverence and appreciation of aesthetics. An installation that uses the audio data from the gallery to create the pieces presented within could be seen as an inversion of such traditions as it encourages viewers to be loud. In a silent exhibition space, such a piece would appear blank. However, for the purposes of this project I was still interested in finding an alternative interface that I could utilize for an interactive installation piece.
After working on the Wii nunchuk oscillator last summer, I started researching various new controllers that I might be able to hack. I noticed a growing number of apparent neurological controllers, utilizing EEG data as a control interface. There appears to have been a real push in consumer EEG products in recent years, despite the apparent age of the technology. OCZ Technology’s NIA (Neural Impulse Actuator) is one such controller, that offers an interface method based on electrical activity from the head. After a little bit of research, I learned that the controller was significantly less neural than implied, being based more in muscle activity than neuronal. Furthermore, I learned that the process of key mapping would make the NIA less useful for installation purposes, as the installation would have to include instructions so that the user could learn how to use the controller. This would significantly limit the accessibility and interactivity of the piece. I also looked at Neurosky’s Mindset, a unit which already explores the idea of EEG data visualization. The Mindset’s data visualizer raised some interesting issues. Audio from the users MP3 player is played via the Mindset’s headphones. The user’s brain neuron activity, arguably their reaction to the audio, is measured via EEG and displayed visually. Conceptually, I found this very interesting: audio data, a float, is presented to the user. The users EEG activity is measured, again as a float, and represented to the user visually. Further research into EEG measurement revealed that the data was represented via an FFT, much like my beloved music visualizers. However, the research revealed the limitations of the Mindset, whilst the additional cost of the research tools was unappealing. I shall continue to investigate other pieces of kit (the Emotive EPOC looks very interesting, and offers many more features for a similar price to the Mindset), but for the meantime, I look forward to experimenting with the BioWave I am picking up this week.
Despite the limitations of the Mindset, I liked the idea of this apparent data feedback loop, as it bears certain similarities with my work with Max/MSP and the Echo Chamber, a piece that utilized interactive audio feedback looping in a surround sound setup. I’m interested in developing this idea into an installation piece where the user is able to experience data signals (as audio, visual and tactile stimulus), whilst their neurological reactions to this data is presented to them visually. In doing so, I might be able to look for correlations in stimulus and response, essentially comparing an audio visualizer with an EEG visualizer where the data is rooted in the same signal (the data signal, and the users reaction to that signal).