I looked at some online tools to convert MIDI to something text-based, and some broken node.js modules, but didn’t find anything that worked for me. So I decided to create my own tools. The result is a Processing sketch that can convert a MIDI file into a JSON structure with timestamps, note numbers, velocity, … The JSON file looks like this:

{
  "PPQN": 96,
  "messages": [
    {
      "octave": 2,
      "noteName": "C2",
      "tick": 0,
      "velocity": 100,
      "noteNumber": 36,
      "timestamp": 0
    },
    {
      "octave": 3,
      "noteName": "D#3",
      "tick": 0,
      "velocity": 103,
      "noteNumber": 51,
      "timestamp": 0
    },
    {
      "octave": 2,
      "noteName": "C2",
      "tick": 96,
      "velocity": 101,
      "noteNumber": 36,
      "timestamp": 455
    },
    {
      ...
    }
  ],
  "BPM": 132
}    

I’ve also created a small WebGL project with BabylonJS to show you how this data can be used to trigger animations in sync with audio. Note that this small demo project was only tested in Chrome on Mac OS X, running on a local web server. I haven’t optimised anything yet for mobile devices or other browsers/operating systems, so there’s a chance that it may not fully work. The code is documented quite well, and should be clear enough to give you an idea on how to get started.

You can see the result right here: https://playground.vormplus.be/webgl-music-video/.

The repository with all code can be found on Github.

I still have a lot of work to do on the full music video. I’ll publish the project online once I’m finished mixing and mastering my new album. So keep an eye on this blog if you want to stay up to date.

I wanted a synth with a lot of controls, so I can play it like an instrument in many different ways. That’s why I chose to add some controls you won’t find easily in “commercial/off-the-shelf” synths. The synth has two distance sensors, that can be used to play it like a theremin. There’s also an extra stereo jack input for a Behringer foot pedal, so you can push buttons with your feet while your hands are playing with the other controls.

But enough chatter for now. This is what it looks like.

Build Process

I did a lot of prototyping with electronics, and different materials for the case at timelab, the local fablab in Ghent. I took a lot of photos during the design and build process. You can find all photos on Flickr.

Electronics

The synth has quite some different controls. Here’s the full list, and the GPIO pads they are connected to.

• 6 Potentiometers, connected to PA0, PA1, PA2, PA3, PB0 and PB1.
• 1 Thumb Joystick, connected to PC2 and PC3.
• 1 LKM1638 board, connected to PA4, PA5, PA6 and PA7.
• 1 Stereo Jack to connect a Behringer foot pedal, connected to PB6 and PB7.
• Toggle, connected to PB8.
• Push Button, connected to PB9.
• 2 IR Distance Sensors, connected to PC0 and PC1.

The image below shows some photos taken during the prototyping phase, and one of the final electronics.

Case

The case was made with wood for the main box, and black acrylic for the top plate. I’ve also made some custom 3D-printed parts for the Axoloti bracket, the thumb joystick mount, and the buttons for the LKM1638 board.

Before designing the final top plate, I did a lot of prototyping to find the best way to mount each individual control. The photo below shows some of these prototypes. Everything was laser cut from 3mm thick acrylic.

The synth also contains some 3D printed parts. I did a few iterations on each part to make sure everything had the right dimensions. These prototypes were printed on the Ultimaker 2 at Timelab, the final prints for the synth were made professionally at Shapeways. The photo below shows the process of prototyping the 3D-printed parts.

Graphics

All artwork for the top plate of the synth was made with the Adobe Source Code Pro font. The exact location for each controller was based on the layout. The Adobe Source Code Pro font contains a lot of interesting glyphs. You can see them in the image below. If you combine these glyphs, you can create great illustrations.

This is what the final graphics for the synth look like.

Software

I developed some custom objects to control the LEDs and display, and read button states on the LKM1638 board. These objects are available in the Axoloti community library. I’ve shot some short videos of the software for the LKM1638 board in action. You can find them on Vimeo.

• Axoloti + LKM1638: Light Organ
• Axoloti + LKM1638: Set LEDs Individually
• Axoloti + LKM1638: Toggle Leds
• Axoloti + LKM1638: Light up LED when button is pressed

Since the first chapter is available for free, I've decided to publish it over here as well. So here we go…

In this first chapter, you will learn how to build a basic monophonic synthesizer by creating objects and connecting them. Your synth will only be able to play one note at a time, but that should be enough. The Minimoog was monophonic as well, and a lot of great music was made with it. Time to get started.

Part 1: Making Noise

Start by creating a new document. Double-click anywhere in the empty document to bring up the object selector window. Type osc, and select the osc/sine object. Press enter to add the object to your document.

Now that you know how to create objects, it’s time to add some more. These are the four objects you need to create your first synthesizer.

• midi/in/keyb mod: This object represents a MIDI keyboard, and will handle all incoming notes.
• osc/sine: A basic sine wave oscillator. This is the object that will generate the sound.
• math/*c: This object multiplies the signal running through it with a constant value. You will use it to control the volume of your synthesizer. Make sure you use the one with the red inlet and outlet, as this object will be used on the audio signal.
• audio/out stereo: This object will send the audio signal to the audio outputs on the Axoloti board.

Once you’ve added the objects, your document should look like Figure 1.2. This might be a good time to save your document.

The next thing you need to do, is connecting the objects with “patch cables”. Inlets are displayed on the left to the object, outlets on the right side. You’ll notice that these inlets and outlets have different colors. In the objects you’ve created, you’ll see that there are blue, red and yellow inlets and outlets. Best practice is to connect inlets and outlets of the same color, but there are some exceptions you will learn about later. These are the connections you should make now.

1. Connect the note outlet of the keyboard object to the pitchm inlet of the oscillator.
2. Connect the outlet of the oscillator to the inlet of the math/*c object.
3. Connect the outlet of the math/*c object to both inlets of the audio out objet.

Once you’ve connected the objects, your document should look like Figure 1.3.

So that’s it. Your first synthesizer is finished, and ready to play.

1. Make sure your Axoloti board is connected to your computer with the Micro USB cable.
2. Plug in some headphones, or connect the main audio out to your amplifier or speakers.
3. Press the Live checkbox at the top left of your document to compile and upload your first patch to the Axoloti board.
4. Go to Window > Keyboard to bring up the virtual keyboard. You can also plug in a MIDI keyboard, as this is more fun to play.
5. Press a note on the virtual keyboard. At this moment you should hear nothing.
6. Rotate the dial of the math/*c slowly to turn up the volume.
7. Press some more notes, and hear the sound of your first synthesizer.

Part 2: Making the Noise Stop

As you may have noticed, the sound keeps playing once you’ve pressed a note on the keyboard. This is probably not the behavior you want. A good synthesizer should make noise when you press a key, and stop when you release it. Let’s take a look at how you can fix this by adding only one object.

1. Double click your document to bring up the object selector window.
2. Type gain, and select the gain/vca object.
3. Press enter to add the object to your document.

Your document should now look like Figure 1.5.

It’s time to change how the objects in your patch are connected.

1. Disconnect the wire between the oscillator and the math/*c object. You can do this by right-clicking on the oscillator outlet and selecting Disconnect outlet from the menu. Another way to do this is to click on the outlet, and dragging the cable away.
2. The new gain/vca object you’ve just created needs to be placed between the oscillator and the math/*c object.
3. Connect the outlet of the oscillator to the red inlet of the gain/vca object.
4. Connect the outlet of the gain/vca object to the inlet of the math/*c object.
5. Connect the yellow gate outlet of the MIDI keyboard object to the blue inlet of the gain/vca object. This is a one of the possible connections between different colors.

If you’ve done everything correct, your document should look like Figure 1.6.

The new patch is now ready to play. Press the Live checkbox at the top left of your document, and start playing some notes on the keyboard. You’ll notice that the sound stops when you release the key.

Part 3: Adding Velocity

Your current patch is starting to work the way most synthesizers work. But there is a small problem: all notes are played at the same volume. In this third step, you will add the functionality to play softer notes between the loud ones. This will make your music a bit more interesting.

1. Double click in your document to bring up the object selector window.
2. Type gain, and select the gain/vca object.
3. Press enter to add the object to your document.
4. Disconnect the wire between the first gain/vca object and the math/*c object.
5. Connect the outlet of the first gain/vca object to the red inlet of the second gain/vca object.
6. Connect the outlet of the second gain/vca object to the inlet of the math/*c object.
7. Connect the velocity outlet of the MIDI keyboard object to the blue inlet of the second gain/vca object.

Your patch should now look like Figure 1.7.

The synth is now ready to play. Press the Live checkbox on your document window and start playing the keyboard. Drag the velocity slider and listen how the volume of the notes change. If you use a real MIDI keyboard, you can play it harder or softer to hear the difference.

Part 4: Adding an Envelope

Most synthesizers have envelopes to control the volume of the oscillator or the frequency of the filter. ADSR stands for Attack/Decay/Sustain/Release. In this part, you will learn how to add an envelope to control the volume.

1. Double click in your document to bring up the object selector window.
2. Type adsr and press enter to create an ADSR envelope.
3. Disconnect the wire from the gate outlet of the keyboard to the first gain/vca object.
4. Connect the gate outlet of the keyboard to the inlet of the ADSR envelope.
5. Connect the outlet of the ADSR envelope to the blue inlet of the first gain/vca object.
6. Double click in your document to bring up the object selector window.
7. Type kscope, to add a kscope 128 p object. This object will be used to display the signal of the ADSR envelope.
8. Connect the outlet of the ADSR envelope to the blue inlet of the new kscope object.

If you did everything correct, your patch should look like Figure 1.8.

Envelope Parameters

Let’s take a look at what the different dials of the ADSR envelope do. The white line in Figure 1.9 shows the amplitude of the envelope.

• Attack: The time it takes to reach the maximum volume of the sound when you press a key on your keyboard. If you set this dial to the left, the sound will reach its maximum volume very quickly. If you set this dial to the right, it will take some time to reach the maximum volume.
• Decay: The time it takes to reach the sustain level of your sound after the attack.
• Sustain: The volume of your sound until you release the key.
• Release: The time it takes to go from the sustain level to zero after you release the key. If you set this dial to the left, the sound will stop fast when you release the key on your keyboard. If you set the dial to the right, the sound will slowly fade out.

You can now press the Live checkbox and start playing the keyboard. Change the parameters on the ADSR envelope and listen how it changes your sound. You can see what the different parameters do by watching the signal in the kscope object.

Part 5: Adding a Filter

The last thing needed to create an even more interesting sound is a filter. There are lots of different filters available in the Axoloti object library. You will find them in the filter directory of the object selector.

1. Double click in your document to bring up the object selector window.
2. Type filter, select the bp svf m filter, and press enter to add the object to your document.
3. Disconnect the wire between the second gain/vca object and the math/*c object.
4. Connect the outlet of the second gain/vca object to the red inlet of the filter you’ve just created.
5. Connect the outlet of the filter to the math/*c object.

Your final synthesiser is should look like Figure 1.10. Press the Live button and play around with the pitch and resonance dials and hear what happens to your sound if you play the keyboard.

Exercises

Now that you know how to create a basic monophonic synthesizer, it’s time to test your knowledge. Try some of these exercises and hear the broad range of sounds you can make with just one oscillator, an envelope and a filter.

1. You’ve created a sine wave oscillator at the beginning of this chapter. Replace it with a saw or square oscillator and see how this affects the sound of your synthesizer.
2. Change the Envelope. Use a Attack/Decay Envelope (AD), Attack/Hold/Decay Envelope (AHD) or Hold/Decay Envelope (HD). You’ll find these objects in the env directory in the object selector.
3. Change the Filter, or connect an extra ADSR envelope to the resonance inlet of the filter in your final patch.

Last year, Johannes Taelman invited me to play with the beta version of a new open source hardware/software music platform for he was building. He just launched a crowdfunding campaign, so it’s a good time for me to review the Axoloti for you.

Axoloti Core

Let’s take a look at the Axoloti Core first. This board is the hardware part of the platform. If you are familiar with electronics prototyping: think of it as an Arduino on steroids, for creating music. This is what the board looks like.

From left to right:

• Stereo audio input and output with 24bit sampling.
• Headphone jack.
• Micro USB port to connect the board to your computer so you can program the board.
• Micro SD Card slot to store patches and samples.
• Midi input and output.

At the other side of the board, there are a lot of holes where you can attach other electronic components like faders, knobs or buttons. This makes the board really interesting because it allows you to create your own personal hardware instrument. The board is quite small, so it’s also ideal to hide it in an interactive installation.

Axoloti Patcher

The software for the Axoloti allows you to model your own sounds by connecting small objects with patch cables. It’s a bit like building a modular synth. What’s really great is that you can decide for yourself in which order these objects are connected. If you need eight oscillators, each with a ADSR envelope and a filter, you can build it. With commercial software or hardware synthesizers, this isn’t possible as their specifications are fixed. They are usually designed to generate a certain kind of sound, so you aren’t free to do with them what you want. The Axoloti Patcher comes with over 400 different objects. Here’s a screenshot showing some of them.

Programming the Axoloti is a bit like using PureData or Max/MSP. You connect some objects with virtual cables. Once you’ve created a patch, you can push the “Live” button. The software will then compile the patch to C code and upload it to the Axoloti board.

A great feature in the software is subpatching. You can easily create a polyphonic synthesizer by loading a monophonic synth patch and telling the software how many voices you need.

Axoloti isn’t limited to create synthesizers. So far I’ve used it to create guitar effects, synths, drum machines and even a small sampler. The fun part is that you don’t need your computer after you’ve programmed the board. You can use it as a standalone instrument. Just plug in a MIDI keyboard/controller and focus on the music!

Axoloti Needs Your Support

If you are into making electronic music, you should support the Axoloti crowd funding campaign on Indiegogo. For only €60, you will receive an Axoloti Core board so you can create any musical instrument you like. That’s really cheap compared to small standalone music boxes from Korg or Roland. And those commercial boxes usually only do one thing, whereas the Axoloti can be multiple things at the same time.

Your money will be used to create the first batch of Axoloti boards, and to continue developing the firmware and software. The goal is to build a small community of music enthusiasts around this piece of open-source hardware/software.

Free Patches for Everybody!

I’ve open sourced some of the patches I made so far on GitHub. There’s only three of them right now, but I’ll publish more of them soon.

If you want to know what they sound like, you can find some more info about each patch and an audio file on this page I’ve created on www.beatfreak.be/axoloti.

Links

• Axoloti Website: All information you need about the Axoloti.
• Axoloti YouTube Channel: Tutorials and demos.
• Axoloti Crowdfunding Campaign: Go buy yourself a board so you can create music. You’ll love it.
• Follow Axoloti on Twitter.

One of the first things I tried was using a laser cutter to engrave thin plastic plates, and later copper plates with a hard ground, and printing them by hand. What I like about this process, is that it adds some imperfection to my work, something I can't add in the software I write.

I also experimented with other techniques like screen printing, woodblock printing, photopolymer printing, etching, … In December 2013, I generated some small patterns with Processing, and had some office stamps made with them.

The stamps were used to create the dyptich Cipher/Decipher. It took quite some work to print this piece, just over 20 hours, as the paper size is 70 x 100 cm.

The problem with using these kinds of stamps is that they only come in boring office colors. To use other colors of ink, I had to create my own stamp cushions. I printed these little orange boxes at the local fab lab, and added a layer of felt so I could work with Letraset Pantone ink.

And this is what the final stamps looked like.

Dubbelwerk, the Exhibition

In April, there was the Dubbelwerk exhibition in Ghent. The idea behind the exhibition was that teams of two artists would collaborate to create a single piece. I decided to ignore the curator, and collaborated with the visitors of the exhibition. So I used the stamps to create an interactive installation without computers. The name of the installation is "Apparatus for Creating Generative Drawings Without the Use of a Computer". The visitors of the exhibition were invited to use the stamps, and generate the drawing during the four days the exhibition was open.

Setting up the installation

The installation looked like this before the opening of the show. A clean white wall with five stamps hanging from the ceiling.

Kids vs. Adults

When the exhibition opened, I noticed that adults are bit hesitant in participating. Probably because they know that you can't touch art in a museum or gallery setting. But once they had a few beers, and saw other people using the stamps, they got over it. Kids on the other hand, don't have these boundaries. They see the stamps, and immediately reach for them. Some of them stayed over an hour doing nothing but placing stamps on the paper.

Expected interactions

There were a few things I expected visitors would do with this installation. People will use the canvas as a graffiti wall to draw hearts, smilies or even write their name.

Unexpected interactions

Sometimes it's great to see what people do if you hand them your tools. Somebody started rotating the stamp while pressing it, which gave a great result. People also dragged the stamp across the paper leaving a smudge of ink. These are things I probably wouldn't have done myself when using stamps to create art.

Drawing "Offscreen"

And then there is always the aspiring graffiti artist who starts drawing on the wall. It started with a single yellow stamp, and by the end of the opening night, somebody wrote on the wall next to the piece.

And when one person starts to draw and write on the wall, others will follow…

The Next Step

This is what the installation looked like after the exhibition.

As four days was a bit short to fill the entire wall, I want to setup the installation at a place where it can be displayed for a longer time. The good news is that I've found a place for it. The installation will be part of The New Sublime during the Brighton Digital Festival. Opening night is on September 5, the installation will be on display until the end of the festival.

These are the products I'll be showing in Milan. The exhibition is at Via dei Canzi 14, Milan from Tuesday April 8 - Sunday April 13. If you are in Milan during the fair, come say hi.

CloudOne Pendant Lamp

CloudCups

Import these Java classes at the beginning of your sketch and declare a PImage variable. You also need some basic code in the setup() and draw() functions.

import java.awt.Robot;
import java.awt.*;
import java.awt.image.*;

PImage screenShot;

void setup()
{
    size( displayWidth, displayHeight );

    takeScreenShot();

    background( 0 );
}

void draw()
{
    image( screenShot, 0, 0 );
}

The function to take a screenshot is fairly straightforward. You need to create a Java Rectangle and copy the screen dimensions into that Rectangle. The Robot class is used to capture the screen into a BufferedImage using the createScreenCapture() method. The pixels from the BufferedImage are then copied into a PImage object with the same dimensions. Make sure you use this code within a try/catch block or you'll get an error.

void takeScreenShot()
{
    Rectangle screenRect = new Rectangle( Toolkit.getDefaultToolkit().getScreenSize() );

    try {

        BufferedImage screenBuffer = new Robot().createScreenCapture( screenRect );

        screenShot = new PImage( screenBuffer.getWidth(), screenBuffer.getHeight(), PConstants.ARGB );
        screenBuffer.getRGB( 0, 0, screenShot.width, screenShot.height, screenShot.pixels, 0, screenShot.width );
        screenShot.updatePixels();

    } catch ( AWTException e ) {
        e.printStackTrace();
    }
}

The code for this tutorial is available on GitHub.

Equilibrium

In a time where we are constantly touching our connected devices, we hardly take time to stand still and look at something. If you want to enjoy this artwork, you need to stand completely still. Any movement will disturb the balance between you and the artwork.

The first thing we need to do is to declare some PShape variables, right before the setup() function.

PShape rectangle;
PShape circle;

Inside the setup() function, we'll create the shapes. The first parameter is the type of shape, then you have the x and y coordinates of the shape, and the width and height.

rectangle = createShape( RECT, 40, 80, 160, 160 );
circle = createShape( ELLIPSE, 250, 80, 160, 160 );

To change the appearance of these shapes, you can use the setFill(), setStroke() and setStrokeWeight() methods of the PShape class.

rectangle.setFill( #FF823A );
rectangle.setStroke( #F04155 );
rectangle.setStrokeWeight( 4 );  
circle.setFill( #95CFB7 );
circle.setStroke( false );

In the draw() function, you can draw your shapes with the shape() function. The result should look like this.

If you want to create your own custom shapes, you'll need to create an empty shape, and add vertices to it. This is the code to create a hexagon shape.

hexagon = createShape();
hexagon.beginShape();
for ( int i = 0; i < 6; i++ ) {
    float x = cos( i * THIRD_PI ) * 20;
    float y = sin( i * THIRD_PI ) * 20;
    hexagon.vertex( x, y );
}
hexagon.endShape( CLOSE );

hexagon.setFill( color( 255, 64 ) );
hexagon.setStroke( color( #0E8D94, 128 ) );

This shape will be drawn at (0, 0). In the draw() function, I've used the translate() function to move the point to where the mouse pointer is, so you can paint with the custom shape you've just created. The result looks like this.

Note that not every method of the PShape class is documented in the Processing 2 reference. If you want to know everything you can do with it, you should check out the JavaDoc for PShape.

The code for this tutorial can be found in the GitHub Repository for this series of articles.