Joseph Pochciol

I’ve been developing the code I wrote about in my last entry with some interesting results. I wanted to do something using waveforms, trying to create a visual representation of audio waves. I took Shiffman’s Sine wave code (http://www.learningprocessing.com/examples/chapter-13/example-13-7/) and used the audio spectrum analysis from Mattox’s code and the techniques I previously wrote about. Although the Sine wave is controlled by the mouse, the dimensions of the shapes which make up the wave are controlled by the audio spectrum analysis, making quite a neat little sound visualizer.

I played about using black and white colouring which had some interesting results, and also tried using some randomly generated colours for the stroke. The results can be seen in the attached stills. Though the images are quite striking, the visualizer is much more effective as video. As with the previous piece of code, the sketch can be used with live audio input or by routing the sound using Soundflower.

I’m currently working on combining these techniques with some of the 3D shapes and rotation sketches from Shiffman’s book. I’ll stick these up along with some stills soon.

Hope you enjoy the visualizers. I’ve found that glitchy tracks have the best results when routed through them (check out the drop from Amon Tobin’s remix of Noisia’s Machine Gun, [available as a free download - http://www.amontobin.com/news/free-amon-tobin-noisia-remix-download] with the sketch I posted previously).

//Colour wave visualizer – modified from Shiffman’s and Mattox’s original sketches

import krister.Ess.*;

FFT myfft;
AudioInput myinput;
int bufferSize=512;

float time = 0.0;
float increment = 0.01;
float theta = 0.0;

void setup() {
size(640,480);
frameRate(30);
smooth();
background(255);
noStroke();
fill(0);

Ess.start(this);
myinput=new AudioInput(bufferSize);
myfft=new FFT(bufferSize*2);
myinput.start();

myfft.damp(.3);
myfft.equalizer(true);
myfft.limits(.005,.05);
}

void draw() {
background(0);
smooth();

// With each cycle, increment the “time”
time += increment;

// Increment theta (different values = ‘angular velocity’)
theta += (0.04*mouseX)/100;

float x = theta;

stroke(mouseX,random(255),random(255));
noFill();
// Draw wave with ellipse and circle at each location
for (int i = 0; i <= 80; i++) {

//calculate y based on Sine function using mouse Y to determine wave height
float y = sin(x)*(mouseY/4);
//draw ellipses and rectangles using audio spectrum for length and width
rect(i*10, y + height/2,myfft.spectrum[i]*-400,myfft.spectrum[i]*-400);

ellipse(i*10, y + height/2,myfft.spectrum[i]*-400,myfft.spectrum[i]*-400);

//Move along x-axis
x +=(0.2*mouseX);
}}

public void audioInputData(AudioInput theInput) {
myfft.getSpectrum(myinput);
}

—————————————-

//Black and white

import processing.opengl.*;

// Starting angle
float theta = 0.0;

import krister.Ess.*;

//define variables: names ‘myfft’ and ‘myinput’ can be whatever

FFT myfft;
AudioInput myinput;
int bufferSize=512; //variable to determine frequency bands

void setup() {
size(1280,800,OPENGL);
frameRate(30);
background(255);
noStroke();
fill(0);
smooth();
//define audio objects using ‘bufferSize’ variable
Ess.start(this);
myinput=new AudioInput(bufferSize);
myfft=new FFT(bufferSize*2);
myinput.start();

//Effects look of FFT.
myfft.damp(.9);//smooths the band movement – change number for effects
myfft.equalizer(true); //smooths the spectrum
myfft.limits(.005,.05);//limits output scale – change for effects
}

void draw() {
background(0);
smooth();
//for (int i=0; i<bufferSize/2;i++) {

// Increment theta (try different values for ” angular velocity ” here)
theta += 0.02;

stroke(255);

noFill();
float x = theta;

// A for loop is used to draw all the points along a sine wave (scaled to the pixel dimension of the window).
for (int i = 0; i <= 128; i++) {
// Calculate y value based off of sine function
// calculate height of wave using mouse Y
float y = sin(x)*mouseY/2;
// Draw an ellipse
ellipse(i*10,y + height/2,myfft.spectrum[i]*200,myfft.spectrum[i]*200);
strokeWeight(myfft.spectrum[i]*10);

// Move along x-axis
//Move mouse to right to increase complexity of heli
x += 0.02*mouseX;
}
}
public void audioInputData(AudioInput theInput) {
myfft.getSpectrum(myinput);
}

I’ve been reading Dan Shiffman’s Learning Processing and have found the book really, really helpful in getting to know this program. At times the pace can be a bit slow, but the book is a great way to get to know Processing and has some excellent turtorials. I had a go at sketching and drew myself some shapes, started using variables as controls in animating them and started to notice a few things about the relationship between sound and video. I drew a sketch which involved a series of randomly generated vertices and colour fills, creating a garish, animated pattern that wouldn’t appear out of place in a New Order video. I played Blue Monday over the top of it for full effect and noticed that the pattern and the beats seemed to sync up. They weren’t really synced, but the illusion was quite effective.

I noticed something similar a couple of days later. I was trying to decide whether to watch the TV or listen to some music. I turned the television on before deciding to mute it and listen to some records I’d just had delivered. One record in particular, a bit of dark, downtempo drum and bass, seemed to sync up perfectly with the muted footage of a chocolate bear production line, part of How its Made on the Discovery Channel. There were some pretty surreal moments: particularly when the bass dropped just as an automated nozzle spewed pink sugary goo into the bear’s stomach.

Though this sort of footage of automated machinery is effective for visualizing break-beats and emphasized bass (check out Baz’s Max tutorial Amplitude Video Zoom – http://www.youtube.com/watch?v=ztB7d8c3GMk&feature=related), I wanted to find a way of visualising sound waves using Processing and other software. After searching the web I came across Anthony Mattox’s blog, who appears to be into the same sort of thing. His blog is really useful for anyone who is interested in this. One entry (conveniently titled Visualizing Sound with Processing – http://www.anthonymattox.com/visualizing-sound-with-processing) is particularly useful and contains a nice piece of code.

I decided to take Anthony’s code and have a little play with it, incorporating some of the ideas from Shiffman’s book. I used the audio input to control data in some simple drawing I’d done in processing and had some interesting results. I enjoyed being able to see a visual representation of the sounds picked up by my microphone, but I was interested in seeing an interpretation of a recorded sound. This was particularly the case when I tried using the sketches to visualise pieces by Ryoji Ikeda, who I’ve been listening to now and again since I saw his performance in Berlin. Because my laptop speakers couldn’t effectively reproduce the extremes of the frequency range that Ikeda uses, my laptop mic was unable to pick them up. I looked into ways of routing audio in processing and managed to use Soundflower to effectively route the sound, and then Soundflowerbed to monitor it.
(note – be careful when using Soundflower to route the sound: if the audio-output lead should come lose [something I find happens far too often on the MacBook], the result is some really painful noise that I find can only be stopped by restarting the computer)

The results of these experiments can be seen in these stills. I have been working on a few variations on this visualizer which I will put up here when I’ve cleaned up and annotated the code.   I’m also looking into recording a music video incorporating these sketches. I will post this video when its done.

import krister.Ess.*;

//define variables: names ‘myfft’ and ‘myinput’ can be whatever

FFT myfft;
AudioInput myinput;
int bufferSize=512; //variable to determine frequency bands

void setup() {
size(532,400);
frameRate(30);
background(255);
noStroke();
fill(0);
//these can be changed later in the program

//define audio objects using ‘bufferSize’ variable
Ess.start(this);
myinput=new AudioInput(bufferSize);
myfft=new FFT(bufferSize*2);
myinput.start();

//Effects look of FFT.
myfft.damp(.9);//smooths the band movement – change number for effects
myfft.equalizer(true); //smooths the spectrum
myfft.limits(.005,.05);//limits output scale – change for effects
smooth();
}
//redrawn every frame
void draw() {
background(255);
for (int i=0; i<bufferSize;i++) {
stroke(0);
strokeWeight(myfft.spectrum[i]);//stroke thickness determined by audio
noFill();
ellipse(width/2,height/2,myfft.spectrum[i]*400,myfft.spectrum[i]*400);//draw an ellipse at the center using the audio to determine

}
}

//one more function to linking the input and the FFT
public void audioInputData(AudioInput theInput) {
myfft.getSpectrum(myinput);
}

//Check out Anthony Mattox’s orignial code at http://www.anthonymattox.com/visualizing-sound-with-processing