Breach Candy Group: Blog

the dance of the in-between

Tue, 19 Oct 2010 12:42:48 -0700

I've always been fascinated by what happens in-between frames in a video. Let me explain. When a computer renders effects, say, in adding a transition between two different frames in a sequence, what it does is it it uses complex algorithms to create things that did not exist in either of the original frames/images. The technical term for this is interpolation and this is commonly used for all kinds of effects from ghosting to crossfades and what have we in video postproduction. Normally, when we look at video we of course do not pay attention to these in-between frames as the focus is on the moving image at 24+ frames a second. The interpolated frames here are merely to add on to the effect we want to create.

But, what if we reverse the equation. That is, we start with the premise that it is, in fact, in these in-between frames that we want to exploit as the focus of our art process? What if we stop focusing on the original images that somehow supposedly still capture reality (whatever this means) but rather focus on these ghostly mysteries of the in-between that get created when we add something extra in-between two frames in a video? What if we make this in-between the goal of our art? How could we best exploit these interpolations in a way that was not originally intended - what kinds of aesthetics effects could we achieve by doing this?

I experimented a bit with this kind of quasi-random composition with my "-Dividual City" series in 2006. Here I used HDR software the "wrong" way to create semi-random compositions where movement and people merged together by combining five images. This was the best way to capture the experience of the the city of Mumbai, India, known for its crowds and buzzing streets.

Recently, when I was doing some quick experiments during the Future Places festival in Porto, Portugal, I played around with a second way of doing this I want to experiment with next - here are my rough thoughts:

Basically, in its tentative phase, the idea here is to take a series of still images that we make into a timelapse ( say, a few hundred that you then play 24 frames a second ). What we get is a quite standard sped up video effect. We have seen used many times from clouds moving fast to plants opening etc. But here is the trick: if we slow this timelapse down significantly, the computer automatically creates the frames in-between these stills. By doing this we can theoretically slow down a timelapse to normal speed, reverse-engineer it in a very strange kind of way. By itself, this slowing down does of course not yet produce anything as the images will remain the same - only slower. But if we use a post-production software such as After Effects to add artificial effects in-between these still images - such as echoes, ghosting, motion blur or whatever - quite interesting things start happening. What initially was two still images in a timelapse sequence can now produce complicated movement even if there was initially no movement. A world of the in-between opens up.

Many questions here: could we control this time warping mathematically? Could we create the original still images with the explicit purpose of creating the kind of in-between images we want out of the mix? Or is this process random? Lots of things to experiment out practically.

But while this idea is of course still in development, you can find below a few images from some of the experiments I did when I walked around an empty shopping mall during the Future Places festival. These are shot without paying much attention to the composition or the exposure but to provide a sequence of the different corridors in the place. From these images, I then created a timelapse, slowed it down, and used some post-production magic to add all the extra frames in between. The end result was this video which we projected during a music show (I've added here some quick non-related music afterwards that I made on the way home in the airport).

But what is even more interesting about this video than the cute timelapse effect is that when we look in detail at the in-between frames that have been produced from nothing, they look look like art-pieces by themselves. I love the compositions that have been created; compositions which are not real, nor artificial, but something else - of a third order. And I have about a 1000 images to choose from kindly generated for me by the rendering processes of the postproduction software.

Could this end result then be painted into final art pieces? Created by a bizarre man-machine hybrid and generative process and finally given an aesthetic form by the artist.

Hmmmm ....

Breach Candy Group: Blog

the dance of the in-between

Sounds Like Graffiti showreel

How do you "play" a building?

Portugal - Here We Come.

Anybody has a mobile here and has been raped?

Quick notes from Mobile Brain Bank, Africa, conf (Helsinki, Finland) #2

Quick notes from Mobile Brain Bank, Africa, conf (Helsinki, Finland) #1

Mirror, Mirror, On The Wall

are we married yet?

The Internet Of Things & What It Implies

particles, space and 3D

theory:praxis in emerging digital technologies and cultures

the pirates and their loot!

uncanny valley: the prototyping machines

initial experiments with Olympus E520

random recursions

Tricks and technicks: the '-dividual' city

Tiny persistent automatons!

intern with us!

3.3 billion mobiles