This is all code for three projects I’ve done for DMS8013:

Webcam Synthesizer

//some functions are learned from Minim library

import processing.video.*;
import processing.sound.*;
import ddf.minim.*;
import ddf.minim.ugens.*;

Minim minim;
AudioOutput out;
MoogFilter moog;
Capture cam;
Oscil wave;

int scanX = 0, scanY = 250;

void setup() {
cam = new Capture(this, 1280, 720);
cam.start();
size(1280, 500);

minim = new Minim(this);
out = minim.getLineOut();
moog = new MoogFilter( 1200, 0.5 );
out = minim.getLineOut(Minim.STEREO);
wave = new Oscil( 440, 0.5f, Waves.SINE );
wave.patch(out);
Noise noize = new Noise( 0.5f );
noize.patch(moog).patch(out);
}

void draw() {

if (scanX >= cam.width || scanX >= width) {
scanX = 0;
scanY++;
if (scanY >= cam.height || scanX >= height)
scanY = 0;
}

if (cam.available()) {
cam.read();
}

image(cam, 0, 0);
int scanCol = get(scanX, scanY);
float scanHue = hue(scanCol);
int highlight = color(255, 255, 255, scanHue);
fill(highlight);
noStroke();
ellipse(scanX-5, scanY-5, 10, 10);

float freq = map(scanHue, 0, 255, 900, 250);
wave.setFrequency(freq);

float rez = map(scanX, 0, width, 0, 7);
moog.resonance.setLastValue(rez);

}

Slit-scan Box

//The slit-scanning technique (code) below was learned from Daniel Shiffman’s Slit-scan video tutorial on YouTube https://www.youtube.com/watch?v=WCJM9WIoudI

import processing.video.*;

Capture cam;

int videoSliceX;
int drawPositionX;

void setup() {
size(1200, 480);

cam = new Capture(this,1200, 480);

cam.start();

videoSliceX = cam.width / 2;
drawPositionX = width – 1;
background(0);
}
void keyPressed () {

if(keyCode == LEFT) {
videoSliceX = cam.width / 3;
}
if(keyCode == RIGHT){
videoSliceX = cam.width / 2;
}
if(keyCode == DOWN){
drawPositionX = width – 1;
}
if(keyCode == UP){
videoSliceX = cam.height / 2;
}

}

void draw() {
if (cam.available()) {
cam.read();
cam.loadPixels();

loadPixels();
for (int y = 0; y < cam.height; y++){
int setPixelIndex = y*width + drawPositionX;
int getPixelIndex = y*cam.width + videoSliceX;
pixels[setPixelIndex] = cam.pixels[getPixelIndex];
}
updatePixels();

drawPositionX–;
if (drawPositionX < 0) {
drawPositionX = width – 1;
}

}
}

Particle Modulation

 

//The Particle class code below was learned from Daniel Shiffman’s tutorial https://www.youtube.com/watch?v=WH31daSj4nc and Tom Schofield’s lecture back in October 2015

import processing.video.*;
import ddf.minim.*;
import ddf.minim.ugens.*;
Minim minim;
AudioOutput out;
Particle[] particles;
Oscil fm;

int scanX = 320, scanY =180;

 

Capture cam;

void setup() {
size(640, 360);

cam = new Capture(this, width, height);
cam.start();
particles = new Particle [300];
for (int i = 0; i < particles.length; i++) {
particles[i] = new Particle();
}
background(0);

minim = new Minim( this );
out = minim.getLineOut();
Oscil wave = new Oscil( 200, 0.8, Waves.TRIANGLE );
fm = new Oscil( 10, 2, Waves.SINE );
fm.offset.setLastValue( 200 );
fm.patch( wave.frequency );
wave.patch( out );
}

void captureEvent(Capture video) {
video.read();
}

void draw() {

for (int i = 0; i < particles.length; i++) {
particles[i].display();
particles[i].move();
}

int scanCol = cam.get(scanY, scanX);
float scanHue = hue(scanCol);

int scanCol2 = cam.get(scanX, scanY);
float scanBright = brightness(scanCol2);

{
float modulateAmount = map(scanBright, 0, height, 220, 1);
float modulateFrequency = map(scanHue, 0, width, 0.1, 100);

fm.setFrequency(modulateFrequency);
fm.setAmplitude(modulateAmount);
}
}

This is the third project I did for DMS8013. Well actually this project is pretty similar to Webcam Synthesizer, the difference is on this project I distort the screen using “particle” and synthesized it with modulation sound from Minim. Minim is very interesting to explore, there are so many possibilities. However the main issue I face on this project is I’m still figured it out how to use all the pixel hue and brightness on the screen to synthesized the modulation, so instead of creating one layer of modulation it will created many layers of sound.

Click here to view the embedded video.

Code:

//The Particle class code below was learned from Daniel Shiffman’s tutorial https://www.youtube.com/watch?v=WH31daSj4nc

import processing.video.*;
import ddf.minim.*;
import ddf.minim.ugens.*;
Minim minim;
AudioOutput out;
Particle[] particles;
Oscil fm;

int scanX = 320, scanY =180;

Capture cam;

void setup() {
size(640, 360);

cam = new Capture(this, width, height);
cam.start();
particles = new Particle [300];
for (int i = 0; i < particles.length; i++) {
particles[i] = new Particle();
}
background(0);

minim = new Minim( this );
out = minim.getLineOut();
Oscil wave = new Oscil( 200, 0.8, Waves.TRIANGLE );
fm = new Oscil( 10, 2, Waves.SINE );
fm.offset.setLastValue( 200 );
fm.patch( wave.frequency );
wave.patch( out );
}

void captureEvent(Capture video) {
video.read();
}

void draw() {

for (int i = 0; i < particles.length; i++) {
particles[i].display();
particles[i].move();
}

int scanCol = cam.get(scanY, scanX);
float scanHue = hue(scanCol);

int scanCol2 = cam.get(scanX, scanY);
float scanBright = brightness(scanCol2);

{
float modulateAmount = map(scanBright, 0, height, 220, 1);
float modulateFrequency = map(scanHue, 0, width, 0.1, 100);

fm.setFrequency(modulateFrequency);
fm.setAmplitude(modulateAmount);
}
}

The class code

class Particle {
float x;
float y;

float vx;
float vy;

Particle() {
x = width/1;
y = height/1;
float a = random(TWO_PI);
float speed = random(1,2);
vx = cos(a)*speed;
vy = sin(a)*speed;
}

void display() {
noStroke();
color c = cam.get(int(x),int(y));
fill(c,25);
ellipse(x, y, 12, 12);
}

void move() {
x = x + vx;
y = y + vy;
if (y < 0) {
y = height;
}

if (y > height) {
y = 0;
}
if (x < 0) {
x = width;
}

if (x > width) {
x = 0;
}
}
}

Thanks to another Daniel Shiffman‘s tutorial for the code on creating the “particle”

I finally found the process to make the best looking slit-scan result. I took the lesson from a YouTube video posted by Daniel Shiffman which is really helpful. My idea is to create an interaction between the slit scan and some audio, but at the end I feel it will be more engaging if the interaction is involving the user rather than just audio and slitscan. So I make a keyPressed code, where the user could change the type and style of the slitscan. While the camera scanning the user, the user can immediately change the style of the slitscan by pressing the arrow keys on keyboard.

Here’s a trial video:

Click here to view the embedded video.

And the code:

//The slit-scanning technique (code) below was learned from Daniel Shiffman’s Slit-scan video tutorial on YouTube https://www.youtube.com/watch?v=WCJM9WIoudI

import processing.video.*;

Capture cam;

int videoSliceX;
int drawPositionX;

void setup() {
size(1200, 480);

cam = new Capture(this,1200, 480);

cam.start();

videoSliceX = cam.width / 2;
drawPositionX = width – 1;
background(0);
}
void keyPressed () {

if(keyCode == LEFT) {
videoSliceX = cam.width / 3;
}
if(keyCode == RIGHT){
videoSliceX = cam.width / 2;
}
if(keyCode == DOWN){
drawPositionX = width – 1;
}
if(keyCode == UP){
videoSliceX = cam.height / 2;
}

}

 

void draw() {
if (cam.available()) {
cam.read();
cam.loadPixels();

loadPixels();
for (int y = 0; y < cam.height; y++){
int setPixelIndex = y*width + drawPositionX;
int getPixelIndex = y*cam.width + videoSliceX;
pixels[setPixelIndex] = cam.pixels[getPixelIndex];
}
updatePixels();

drawPositionX–;
if (drawPositionX < 0) {
drawPositionX = width – 1;
}

}
}

An update of the earlier version of Cam synth, which I think I will name it Webcam Synthesizer now. I found Moog Filter in Minim really interesting. I use it to create a really nice low frequencies detail on the sounds. Also, I put the stroke off the screen, to give it more playful sense and curiosity.

Click here to view the embedded video.

Code:

import processing.video.*;
import processing.sound.*;
import ddf.minim.*;
import ddf.minim.ugens.*;

Minim minim;
AudioOutput out;
MoogFilter moog;
Capture cam;
Oscil wave;

int scanX = 0, scanY = 250;

void setup() {
cam = new Capture(this, 1280, 720);
cam.start();
size(1280, 500);

minim = new Minim(this);
out = minim.getLineOut();
moog = new MoogFilter( 1200, 0.5 );
out = minim.getLineOut(Minim.STEREO);
wave = new Oscil( 440, 0.5f, Waves.SINE );
wave.patch(out);
Noise noize = new Noise( 0.5f );
noize.patch(moog).patch(out);
}

void draw() {

if (scanX >= cam.width || scanX >= width) {
scanX = 0;
scanY++;
if (scanY >= cam.height || scanX >= height)
scanY = 0;
}

if (cam.available()) {
cam.read();
cam.loadPixels();
copy(cam, (cam.width/2), 0, 1, cam.height, (scanX++%width), 0, 1, height);
}

int scanCol = get(scanX, scanY);
float scanHue = hue(scanCol);
int highlight = color(255, 255, 255, scanHue);
fill(highlight);
noStroke();
ellipse(scanX-5, scanY-5, 10, 10);
image(cam, 0, 0);

float freq = map(scanHue, 0, 255, 900, 250);
wave.setFrequency(freq);

float rez = map(scanX, 0, width, 0, 7);
moog.resonance.setLastValue(rez);

}

On the last post I talked quite a bit about the possibilities of interaction between slit-scan and sound. Although I still find the idea of slit-scan is still limited, I tried to experiment more on the synthesis between webcam and sound. So I kinda postponed the idea of slit-scan and try new thing. I found the code to read a hue on the screen, which I found really usable. And for the sound, at first I was trying to use SinOsc in order to create sine oscillator but I got stuck then I found minim. Minim is really helpful for synthesizing sound.

With the help of this code by fine gentlemen on openprocessing.org I managed to experiment with this cam synthesizer.

Click here to view the embedded video.

Here is the code, which is really a mess and it mixed up with the previous slit-scan code:

import processing.video.*;
import processing.sound.*;
import ddf.minim.*;
import ddf.minim.signals.*;

Minim minim;
AudioOutput out;
SineWave sine;
Capture cam;

int scanX = 0, scanY = 250;

void setup() {
cam = new Capture(this, 1280, 720);
cam.start();
size(1280, 500);

minim = new Minim(this);
out = minim.getLineOut(Minim.STEREO);
sine = new SineWave(440, 0.5, out.sampleRate());
sine.portamento(10);
out.addSignal(sine);
}

void draw() {

if (scanX >= cam.width || scanX >= width) {
scanX = 0;
scanY++;
if (scanY >= cam.height || scanX >= height)
scanY = 0;
}

if (cam.available()) {
cam.read();
cam.loadPixels();
copy(cam, (cam.width/2), 0, 1, cam.height, (scanX++%width), 0, 1, height);
}

image(cam, 0, 0);
int scanCol = get(scanX, scanY);
float scanHue = hue(scanCol);
int highlight = color(255, 255, 255, scanHue);
fill(highlight);
noStroke();
ellipse(scanX-5, scanY-5, 10, 10);
float freq = map(scanHue, 0, 255, 987.77, 261.63);
sine.setFreq(freq);
float pan = map(scanX, 0, width, -1, 1);
sine.setPan(pan);

}

 

Hi it’s been awhile!

As I mentioned on the earlier post, I have been playing around with this Slit scan idea. At that time I felt this idea is somehow limited (it is still though!), but when it comes to Processing code I find it quite fun to explore. Little background on why I like Slit-scan technique:

Around 13 years ago, I saw a film that really changed my life. It was 2001: A Space Odyssey. On the film there is this “gate” scene that really mesmerized the 11 years of me. I was so amazed with that scene. That scene haunted me, and to be honest it was actually that got me into cinema (beside many 80′s-90′s B horror film I watched when I was kid) and so called experimental cinema later. Anyway, fast forward to many years later I learned that scene was done by Slit-scan technique using several component of film reel by Douglas Trumbull (which if you’re reading this and you are film geek, I know you worship this guy but please remember this is the same guy who also try to “explore” commercial cinema experience with his idea about 48fps real thing yadda yadda The Hobbit yadda yadda). I’ve learned the technique on analog device and also digital camera, but as I was looking around the example on Processing library I was quite intrigued when I found there is a way to create a slit scan technique using code. Well, I think there is always a way to create anything with code, but hey! It’s slit-scan! So it should be fun!

So I’ve seen the example, and it was a slit scan code for video library. It was fun, but then I was thinking about interaction. I think it could be more interesting if there is an interaction going on between the user and this slit-scan technique. So I tried to explore Capture on video library, which is really simple. Anyway so here is the code I came up at the end.

import processing.video.*;
import processing.sound.*;
WhiteNoise noise;

Capture cam;
int x=0;

void setup() {
//println(Capture.list());
cam = new Capture(this, 1280, 720);
cam.start();
size(1280, 500);

noise = new WhiteNoise(this);
noise.play(1);
}

void draw() {
if (cam.available()) {
cam.read();
cam.loadPixels();
copy(cam, (cam.width/2), 0, 1, cam.height, (x++%width), 0, 1, height);

noise.pan(map(mouseX, 0, width, -1.0, 1.0));
}

}

As previously stated, it is really simple. There is also a WhiteNoise going on there because I’m still on the early stage to have some experimentation with pixel reader (this code hasn’t been update). For this project I was thinking something synthesis between the slit-scan and sound, so for further development I hope I could try create a simple interactivity between the slit-scan that scan through webcam and sound. I will use something like pixel reader and transfer the data use OSC to Puredata.

Anyway, this is my first idea for this module and I think I spent to much time learning annotated portfolio instead on working with project.

—–

For some random recommendation. What I’ve been into this past week:

The Defence of Food by Michael Pollan. As a food aficionado, I’d say this book is amazing. It touches an important issues of homecooking, food cultures, corporate food, and consumer culture theory.

A Girl on The Shore by Asano Inio. Oh man, I think Asano is the most sadistic mangaka ever live. After I read his utterly nihilistic manga Oyasumi Punpun, I need like couple years to touch his another work because of the trauma created by that manga. And now he creates something similar with MORE emotional wreckage. If you are not a fragile person when it comes to feeling, this one might for you.