andybest.net

5 Processing Video Tutorials

andy — Thu, 28 Jan 2010 21:50:45 +0000

Not had the chance to post for a while since I’ve been very busy with work, but I thought I’d share this series of 5 Processing video tutorials that I did for Switchboard back in November. They explore the basics of Processing as well as the use of FFT to create audio reactive sketches.

You can find the original post on Switchboard here, and I have embedded the 5 videos after the break for the sake of convenience.

Twitter Generative Music

andy — Tue, 21 Jul 2009 16:31:56 +0000

I haven’t written a post for a while, and I think that it is about time that I put some of my MA work on here, so I’ll just show a quick sample of one of the prototypes.

In my MA project, I explore ways to produce sound from sources of data like Twitter. I not only wanted to produce something that sounds interesting, I also wanted to have some meaning behind what was produced, so I am developing a system to relate the sounds produced to the content of the messages. I shall explain more about this system in a later post once I have finished the work I am doing on it at the moment, but for now, I shall share a sample from a prototype of the project written in Processing.

What you’re hearing is Processing …processing… the Twitter feed and outputting MIDI notes to Ableton live. This was an early prototype which was keyword based.

Experiments in Social Network Recursion

andy — Wed, 11 Mar 2009 23:08:48 +0000

http://twitter.com/andybest/status/1313198417

Processing Sketches in a Tweet

andy — Thu, 12 Feb 2009 00:28:15 +0000

Today I have been writing Processing sketches that fit within Twitter’s 140 character limit. Why, I don’t quite know, but it certainly is fun. Here are some of my attempts.

My first attempt- a resizing circle thingy…

int c,f;void setup(){size(640,480);c=0;f=255;colorMode(HSB);}

void draw(){background(color(c,f,f));ellipse(320,240,c,c);c=(c++>255)?0:c;}

My second attempt- printing the colour spectrum in squares.

int w = 160;int c=0;colorMode(HSB);size(w,w);for(int i=0;i<w;i+=10){

for(int j=0;j<w;j+=10){fill(c,255,255);rect(i,j,10,10);c++;}}

My third (and favourite!) attempt- Oldschool plasma effect (Online version)

int t,f,x,y;void draw(){f=256;colorMode(HSB);for(x=0;x<f;x++)

for(y=0;y<f;y++)set(x,y,color(f/4*(sin(x/16.0)+sin(y/16.0))+t%f,f,f));t++;}

I hearby challenge you all to come up with Processing sketches that will fit into 140 character tweets! (post your sketches as a comment :D )

Processing.org Twitterers

andy — Wed, 11 Feb 2009 21:05:22 +0000

I thought that since I’ve found Twitter such a valuable tool, I would put together a quick list of some of the Twitterers in the Processing world that I am following.

Memo Akten

Bio: mad scientist
(Twitter ID- memotv )
(website: http://www.memo.tv/)

Peter Kirn

Bio: Digital visualist-musician / technologist and creative tech blogger for CDM, see @cdmblogs
(Twitter ID- peterkirn )
(websites: http://www.createdigitalmotion.com http://www.createdigitalmusic.com/)

LennyJPG

Bio: CodeFormTypeColorLove
(Twitter ID- lennyjpg)

Stefan Goodchild

Bio: Jack of all trades, master of some. All round web geek and wonky music producer.
(Twitter ID- stefangoodchild )
(website: http://stefangoodchild.com/ )

Daniel Shiffman

(Twitter ID- shiffman )
(website: http://www.shiffman.net/)

Robert Hodgin (Flight404)

(Twitter ID- flight404 )
(website: http://www.flight404.com/)

Toxi

Bio: design, sweat & code and anything between
(Twitter ID- toxi)
(website: http://postspectacular.com/)

Golan Levin

Bio: We fling flongs all day long.
(Twitter ID- golan)
(website: http://www.flong.com/ )

John Maeda

Bio: President of Rhode Island School of Design (RISD)
(Twitter ID- johnmaeda )
(website: http://www.maedastudio.com/index.php )

This is by no means a complete list and there are tonnes of cool people that I’ve left out, but you should definitely check out their Twitter feeds. If you’ve still got room for some more Twitter love, you can always check out my twitter stream ;)

Processing OpenCV Tutorial #2- bubbles

andy — Fri, 06 Feb 2009 18:45:39 +0000

This is my second Processing and Open CV tutorial! (The first ran on CreateDigitalMotion.com)

I decided to write some tutorials because I am very much into Processing as a tool. As an Interaction Designer/ visualist, I find Processing very useful as it allows me to put my ideas into practice very quickly without the hassle of having to set up an environment first. The fact that Processing is based in Java means that while it isn’t as fast as a higher level language like C, it makes up for it with the huge amount of libraries availiable to the Java platform. If speed is what you’re after though, I’d definitely recommend the C++ framework OpenFrameworks.

Edit: I should probably note that I’m not trying to stop people using Processing and move to Open Frameworks. Processing is a very powerful tool, and is much easier to get to grips with- I was just pointing out that there are alternatives :)

In this tutorial, I will show you how to use a thresholded frame difference (motion) image in order to perform collision detection with objects onscreen. Essentially we will be creating something similar to one of the old webcam games where you can ‘pop bubbles’ with your hands (or indeed anything that moves). I’ve embedded a video below to illustrate what I mean:

Here is an outline of how it will work:

A frame is captured from the camera
An absolute difference is taken of the current frame and the previous frame. This is converted to greyscale and run through a threshold filter so that only pixels that have changed (i.e. where there is movement) will be white. All other pixels will be black.
For each bubble onscreen, check whether there are any white pixels (movement) in the difference image in the space that the bubble occupies.
If there is movement, “pop” the bubble. (Remove it from the screen)
If there isn’t movement, draw the bubble.

We will first create a class for bubbles to store information about each bubble (location, size, etc.) If you are not familiar with classes, I suggest you read a tutorial on the subject, such as this one.

The class will need to store information about each bubble:

The X, Y location of the bubble’s top left corner,
and the width and height of the bubble.

It will also contain an ‘update’ function that will check for collisions and draw the bubble to the screen. There is a basic outline of the class below:

class Bubble
{
int bubbleX, bubbleY, bubbleWidth, bubbleHeight;  //  Variables to hold the bubble's coordinates and width/height
Bubble ( int bX, int bY, int bW, int bH )  //  Class constructor- sets all the values when a new bubble object is created

{

bubbleX = bX;

bubbleY = bY;

bubbleWidth = bW;

bubbleHeight = bH;

}
int update()

{

/*  The bubble update code goes here  */
}

}

You may have noticed that the update function for this class returns an integer. This is because the return value (0 or 1) will be used to determine if the bubble is “popped” or not.

We shall now write the setup and draw routines for the sketch.

In order to manage a number of Bubble objects, we shall use a Java ArrayList (documentation). This has a number of benefits over a regular array, as we can add and remove objects at any point and the array will automatically resize and elements will automatically be shifted around.

So, before the setup and draw routines, we have to import some stuff and declare some variables.

import hypermedia.video.*;          //  Imports the OpenCV library
OpenCV opencv;                      //  Creates a new OpenCV object

PImage movementImg;                 //  Creates a new PImage to hold the movement image

int poppedBubbles;                  //  Creates a variable to hold the total number of popped bubbles

ArrayList bubbles;                  //  Creates an ArrayList to hold the Bubble objects

PImage bubblePNG;                   //  Creates a PImage that will hold the image of the bubble

PFont font;                         //  Creates a new font object

The comments should hopefully explain what all of the stuff is for well enough. I am using the built in font functionality of Processing to display some stuff onscreen. You can check out the reference page for PFont if you are unfamiliar.

So now we create the setup routine:

voidsetup()
{
size(640, 480);                      //  Window size of 640 x 480

opencv = new OpenCV(this);            //  Initialises the OpenCV library

opencv.capture(640, 480);             //  Sets the capture size to 640 x 480

movementImg = newPImage(640, 480);   //  Initialises the PImage that holds the movement image

poppedBubbles = 0;
bubbles = new ArrayList();              //  Initialises the ArrayList
bubblePNG = loadImage("bubble.png");    //  Load the bubble image into memory

font = loadFont("Serif-48.vlw");        //  Load the font file into memory

textFont(font, 32);
}

So, nothing too out of the ordinary happening here, just initialising and setting some stuff. By the way, I’ll include a zipped sketch folder at the end of the post that will have both the bubble.png that I made as well as the font file.

So now onto the buisiness end of the sketch. The draw routine.

voiddraw()
{

bubbles.add(new Bubble((int)random( 0, width – 40), -bubblePNG.height, bubblePNG.width, bubblePNG.height));   //  Adds a new bubble to the array with a random x position
opencv.read();                              //  Captures a frame from the camera

opencv.flip(OpenCV.FLIP_HORIZONTAL);        //  Flips the image horizontally

image( opencv.image(), 0, 0 );              //  Draws the camera image to the screen

opencv.absDiff();                           //  Creates a difference image
opencv.convert(OpenCV.GRAY);                //  Converts to greyscale

opencv.blur(OpenCV.BLUR, 3);                //  Blur to remove camera noise

opencv.threshold(20);                       //  Thresholds to convert to black and white

movementImg = opencv.image();               //  Puts the OpenCV buffer into an image object
for ( int i = 0; i < bubbles.size(); i++ ){    //  For every bubble in the bubbles array

Bubble _bubble = (Bubble) bubbles.get(i);    //  Copies the current bubble into a temporary object
if(_bubble.update() == 1){                  //  If the bubble's update function returns '1'

bubbles.remove(i);                        //  then remove the bubble from the array

_bubble = null;                           //  and make the temporary bubble object null

i–;                                      //  since we've removed a bubble from the array, we need to subtract 1 from i, or we'll skip the next bubble
}else{                                        //  If the bubble's update function doesn't return '1'

bubbles.set(i, _bubble);                  //  Copys the updated temporary bubble object back into the array

_bubble = null;                           //  Makes the temporary bubble object null.

}

}
opencv.remember(OpenCV.SOURCE, OpenCV.FLIP_HORIZONTAL);    //  Remembers the camera image so we can generate a difference image next frame. Since we've

//  flipped the image earlier, we need to flip it here too.

text("Bubbles popped: " + poppedBubbles, 20, 40);          //  Displays some text showing how many bubbles have been popped

}

I think that my code comments should explain it all well enough, but I’ll quickly run through how it works.

A new Bubble object with a random X coordinate is added to the ‘bubbles’ ArrayList
A frame is grabbed from the camera and is flipped horizontally. It is then drawn to the screen.
The image is then converted to greyscale, an absolute difference image is made and it is then thresholded to get a black and white difference image.
A for loop then cycles through each bubble in the ‘bubbles’ ArrayList and runs the bubble’s update function. If the function returns a 1, the bubble is destroyed- if it returns a 0, then the bubble is put back into the array.
The opencv remember function is called. Since we flipped the image earlier, we need to tell the remember function to flip this image.
Some text is drawn onscreen displaying the amount of bubbles popped.

So that is the draw routine. All that is left now is the update routine for the Bubble class. I’ll go ahead and paste it below.

int update()//   The Bubble update function
{
int movementAmount;          //  Create and set a variable to hold the amount of white pixels detected in the area where the bubble is

movementAmount = 0;
for( int y = bubbleY; y < (bubbleY + (bubbleHeight-1)); y++ ){    //  For loop that cycles through all of the pixels in the area the bubble occupies

for( int x = bubbleX; x < (bubbleX + (bubbleWidth-1)); x++ ){
if ( x< width && x > 0 && y < height && y > 0 ){             //  If the current pixel is within the screen bondaries

if (brightness(movementImg.pixels[x + (y * width)]) > 127)  //  and if the brightness is above 127 (in this case, if it is white)

{

movementAmount++;                                         //  Add 1 to the movementAmount variable.

}

}

}

}
if (movementAmount > 5)               //  If more than 5 pixels of movement are detected in the bubble area

{

poppedBubbles++;                    //  Add 1 to the variable that holds the number of popped bubbles

return 1;                           //  Return 1 so that the bubble object is destroyed
}else{                                 //  If less than 5 pixels of movement are detected,

bubbleY += 10;                      //  increase the y position of the bubble so that it falls down
if (bubbleY > height)               //  If the bubble has dropped off of the bottom of the screen

{  return 1; }                      //  Return '1' so that the bubble object is destroyed
image(bubblePNG, bubbleX, bubbleY);    //  Draws the bubble to the screen

return 0;                              //  Returns '0' so that the bubble isn't destroyed

}
}

The update function accesses the pixel array of the difference image directly, and looks at the brightness of each pixel in the space that the bubble occupies. If there are more than 5 white pixels, the function returns a 1 so that the bubble is removed from the ‘bubbles’ ArrayList. The poppedBubbles variable is also increased by 1. If the bubble falls off of the bottom of the screen, it is also removed, but the poppedBubble variable is not increased in this instance. If neither of the previous two things have occured, than The bubble is drawn to the screen and the function returns 0.

I tried to avoid the need to talk too much about the workings of the code by commenting each line. If you have trouble with anything, don’t hesistate to leave a comment on this post and I will try to answer your question. I hope that you have found this useful.

Here is a zipped copy of the sketch folder for this project. It contains the bubble picture and the font that I used.

If you end up making anything cool with it, please leave a comment here with a link to it- I’d love to see it :)

Once you pop, you just can’t stop! Or something… ;)

Processing OpenCV Tutorial #1

andy — Fri, 06 Feb 2009 18:44:42 +0000

Hi there! I have written a tutorial for the brilliant OpenCV library for Processing which Peter Kirn has kindly published on Create Digital Motion. You can find it here. Also, don’t miss the next tutorial, which I shall post up on this site :)

Here is a video of what the tutorial will show you how to make:

Psychedelic Sphere

andy — Wed, 28 Jan 2009 00:58:34 +0000

While messing around in Processing with Sphere drawing code, I came up with this…

I’m pretty pleased with the result. I used the OpenCV library for the blur. Processing is so great for producing little things like this in next to no time :)

Disco Mesh

andy — Sun, 04 Jan 2009 02:27:52 +0000

I’ve worked a bit more on the sketch that I was working on earlier, and I have rendered it as a mesh:

I need to look into smooth shading in OpenGL, but I’m not quite sure where to look yet. A push in the right direction would be appreciated ;)

Disco Lines

andy — Sat, 03 Jan 2009 18:01:43 +0000

I’m not quite sure what my fascination with rendering lines in 3d space is, but I’ve done it again! This time I came across a Flickr Picture by Stefan Goodchild. In the description for the Flickr picture, he left the following function:

for (int x = -2000; x &lt; 2000; x+= 9) {

for (int y = -2000; y &lt; 2000; y+= 9) {

z = ((cos(radians(y/3))*sin(radians(x/6)))*200)*sin(radians(x/2));

aStar = new Star(x,y,z);

s.add(aStar);

}

}

I found it pretty irresistable, so I decided to make something of my own using the function. I ended up with this:

I ended up modifing the function somewhat to achieve the animation, and I have another function to calculate the colours. I’ll continue to play around with this- it needs some optimisation since I’m rendering it all with lines- I think I might be better off just rendering it as a mesh. So watch this space!