Onderstekop Weblog

mingus, mingus, mingus 0.4!

Finally, after almost five months, version 0.4 of mingus the music package for Python has been released. A lot has happened since version 0.3:

A separate midi sub-package for playing, reading and writing midi data was added; The reading part is still buggy and should not be relied upon, but the rest works like a charm.
The fluidsynth module now uses the pyFluidSynth bindings instead of depending on a running server to play notes.
A new module named value was added to make dealing with note durations easier.
Added chord substitution algorithms in the progressions module.
More functions, updates and bugfixes in the existing packages (see the change log for detailed information).
Up to date documentation, tutorials and exercises.
Packages for Ubuntu and Arch (NB. these packages are not maintained by me and may not always be up to date).

Furthermore, the mailing list and bug tracker have been seeing some activity in the past couple of months, which has been great for finding and squashing bugs and getting a sense of what people are doing with mingus. I hope this will continue in the future and I would like to thank everybody who contributed.

mingus' homepage

Qt Frontend for Music Generation Program

Last vacation I did a lot of work on the improviser program, which is called Improviser, because I have a lot of imagination. The program has a new Qt frontend, better control over macro and micro structure, produces MIDI files and can switch from black metal to blues to dance without blinking. Here some new demos, recorded this afternoon:

Swinging Boogie Woogie

Modern Dance

A download is available at PyPi. More simulations can be found at YouTube.

Write Your Own Musician!

I got a bit carried away with the improvisation program I started writing yesterday, because it was only meant as a simple example application, but turned out way too complicated.

I started writing a new, better program from scratch today so I could do some more experiments, compose some songs, etc. and I think the resulting music is much better than before.

This time you can write your own musicians and plug them into the song, which is following a progression in a structure. Sounds vague? Yeah, it's pretty vague, alright, and I won't waste everybody's time explaining it because the system is pure magic. Magic, I tell you!

Let me just tell you that a collection of musicians has been written today that you can throw in any song you want. It's hilarious -they don't even know what's happening-.

I wrote myself a stride pianist, some soloists and a couple of drummers and let them play together in my computer. They are pretty cool guys:

Catchy Computer Improvisation

You can find more examples on my YouTube channel.

I need some sleep now.

Automated Improvising

I've added another example application to the mingus library today, which will generate drums, arrangements and solos for a given chord progression in real-time.

The program can be found under mingus_examples/improviser and knows how to swing as well. Check these example runs:

Test: Algorithmic Rock

Test: Modern Jazz

Nice Song

You can find some more examples on my YouTube channel.

The mingus Drum Computer

I just finished the next example application for the python music package mingus. Yesterday I presented a piano, today it's a drum computer with simple recording and playback functionality:

Piano-ex-machina: mingus Keyboard

My US-keyboard technique could use some work, but here goes:

This program will soon be available in a new example program directory in the mingus repository and is the first application demonstrating mingus' new fluidsynth support. The program is controlled by the keyboard (that is - the one you use to type) and chords get recognized while you play (not shown in this version). The graphical stuff and event handling is done by pygame for now.

The video was recorded with gtk-recordmydesktop. Audio was channeled through Jack Rack for a little (TOO MUCH!?) reverb using jack, which I finally got working. YouTube messed up the synching a bit, but I was happy they could handle ogg at all.

/shuffles of stage.

Update @ Dec 5, 2008:

/shuffles back on stage.

Today another example application was made (a drum computer) that was added together with this piano to the new mingus_examples directory in the repository.

mingus + MIDI sequencer = awesome!

Just a short post to let you know that I've added fluidsynth support and a simple MIDI sequencer to the mingus package, which means there is now cross platform realtime MIDI playback to use with all the theoretical goodness.

This little example requires a fluidsynth server running at port 9800:

>>> from mingus.extra import fluidsynth

>>> from mingus.container import *

>>> fluidsynth.init_fluidsynth()

>>> fluidsynth.play_Note(Note("A"))

This only plays an A on the piano, but playing chords, bars and tracks with other instruments is just as simple.

See mingus.googlecode.com for documentation, tutorials, downloads and the svn repository.

PS. I've also added bash tab completion to the directory bookmark tool I released earlier. I will try to upload that as soon as possible to another googlecode site - Update: see here.

Update @ Dec 4, 2008: A first demo program using this new features - a software keyboard - was made. See here

Command Line Bookmarks

Update Dec 2, 2008: The 1.0 release can be found at google code.

bm is a program that can make life at the terminal a little easier when you often have to deal with a couple of directories at the same time. It can be used to keep bookmarks of directories you use regularly, but are a drag to type and switch between.

Of course, there are the popd, pushd and dirs commands, but a stack is often not sufficient to work fast. That's why I wrote bm.

bm is a tiny program written in C that reads from a simple, clear text bookmark file to generate an index. All you need to do to set it up is type the following in a terminal:

wget http://cdbm.googlecode.com/files/bm-1.0.tar.gz

	tar -xvf bm-1.0.tar.gz

	cd bm-1.0

	make && sudo make install

Now you can type the bm command and it will show you nothing, because you haven't added any bookmarks yet. Lets add the current directory:

$ bm -a

	0. /home/bspaans/bm-1.0

The directory has been added! Lets add another one:

$ bm -a /var/log/apache2/

	0. /home/bspaans/bm-1.0

	1. /var/log/apache2/

As you can see we have built up a very small index. Using the index as the only argument we can now get the data out and use it with cd, for instance.

$ bm 0

	/home/bspaans/bm-1.0

	$ bm 1

	/var/log/apache2/

	$ cd `bm 1`

This works, but the backticks get a little tiring. It would be nicer if we could just type something like:

$ cdbm 1

And be done with it. This is possible by adding a tiny function to your ~/.bashrc script: [...]. Update @ Oct 17, 22:24: The last version automatically adds the commands cdbm and lsbm to the global bashrc file, so you can use them right after you have restarted your terminal.

You can see more options by using the --help switch or by reading README.txt.

UPDATE @ Oct 17, 21:22: I added tags to the program to make it more usable. When you add a directory you can now give a third argument that specifies the tag (use '.' or './' to add the current directory as a tag in this mode). The remove and list commands support tags as well, as you can see in this little example:

$ bm -a /var/log/apache2 aplogs

0. /var/log/apache2 [aplogs]

$ bm aplogs

/var/log/apache2

$ cdbm aplogs

:wq

Ah Um: mingus 0.3

Version 0.3 of mingus, the music theory package for Python I started working on a while back has been released. This new release contains a couple of new features since the last time I posted something:

The structures for notes (bars, tracks, compositions, etc.) are ready and allow you to express a truck load of musical concepts from within Python.
A lot of time went into building and refining chord, scale, progression and interval recognition (more about this later), but it all works now.
A LilyPond generator to create sheet music (as suggested by rahul) has been added to the extra sub-package.
Up to date, complete documentation and tutorials in the wiki. (Good documentation is crucial for packages like this, so don't hesitate to be a pedant in the wiki comment section.)

A complete, up to date feature list and download can be found at mingus' google code page.

Since this library is mostly aimed at developers, I will now take some time to discuss some of the things I ran into while working on this release, that you may find interesting.

Searching Problems

Interval Recognition

This new release contains a couple of functions that let you analyze groups of notes. This begun relatively simple, with a function that could recognize intervals between notes. For instance, intervals.determine('E', 'F') would return 'minor second', and intervals.determine('E', 'Ebb') would return 'diminished unison'.

I started out naively; treating this as a counting problem. To determine the interval between E and F I simply counted the steps between them (1) and used that number as an index in a simple ordered list to get the proper naming. This works for the E and F example, but fails horribly in the E-Ebb case (which would probably return a minor seventh, instead of a diminished unison). Music notation and theory is pretty elegant in itself, but it doesn't really play well with the list processing stuff I usually pull to get stuff done. So I added some unit tests, knocked myself over the head a couple of times and eventually whipped out the cycle of fifths again to fix that function.

The trick is to count the steps of fifths in the note names and compare this to the measured half note steps. For example: the steps of fifths in the interval F-C# is 1 (F-C), however, there are 8 half note steps in the interval F-C#. 8 is bigger then 7 (which is the expected number of half note steps in a fifth), so the interval must be augmented and the function can happily return an augmented fifth. This method still chokes on unisons though (eg. E-Ebb: 0 fifth steps, 10 half note steps...augmented unison?? No.), so I added them as a little side case. The actual function is a little bit more complicated, but this is the gist of it.

Once the interval recognition was working as it should, I could build another underestimated part of the library on top of it:

The Chord Recognizer

The chord recognizer would have to be able to recognize all kinds of triads (chords of three notes), sevenths (a triad + a seventh), extended chords (a seventh chord + a ninth, eleventh or thirteenth), altered chords (every chord mentioned before, but in a slightly changed version (flat five, flat ninth, etc)), and the very lovable polychords (chords with other chords above them; eg. Gm|C is a C major triad with a G minor triad above it). Already this has become a huge task. The task becomes even bigger when you also have to recognize every inverted chord (eg. 'C', 'E', 'A' is still an A minor triad) and when you realize that a chord made of notes 'A', 'C' and 'G' is also a valid seventh chord, even when the fifth ('E') is missing.

So yeah; the chord module turned out to be a lot of work and lines of code, but in the current release it does all the things mentioned above and can recognize almost every chord in the universe (missing chords should immediately be reported). It can probably do with a bit of refactoring and memoization (planned for the next release), but it's pretty fast already - all unit tests run within a second on my machine.

The way it works is a bit hard to explain, but I'll give it a generalized shot. The chords.determine function first measures the length of the note list and delegates the request to the right function for the job. Eg. a call to chords.determine with a list of three notes will be redirected to chords.determine_triad. Then the function looks at the intervals and figures out if it's a chord; every hit will be added to a list, other things will be ignored. This will find a lot of chords already, but can't really find inversions by itself (you would have to manually enter every possible inversion; an error prone process). That's why the function gets recursively called again, inverting the list of notes each time until it's at its last inversion. The function keeps track of the number of inversions performed so far and adds the results accordingly.

Finding polychords is separated from this process and the function responsible for this will just be called before returning the results, because once you can determine chords of size 3-7, it turns out you can easily find chords made of a combination of them. The actual determine_polychord function is only about 15 lines of actual code and a nice example of the aforementioned list processing stuff.

Scales and Progressions

mingus is also equiped with a scale and progression recognizer, which are a lot cleaner and easier to grasp from reading the source code. The scale.determine function basically starts out with all the good answers in a list and then removes (sieves out) all the wrong ones. I thought it was an interesting approach to pattern matching and it works much faster than 'brute forcing' it with endless `if` statements.

The scales.determine function will be altered however, because it's quite useless for automatic recognition in ie. a single bar, since it needs all the notes in the scale to find the scale. The plan is to alter the behaviour so that it will return all the scales that you could possibly play within, using the notes passed as an argument. ie. ['A', 'C', 'E'] could be played in one of the minor scales, or in a plain diatonic C, etc. The more notes are given, the more precise you can determine the scale (your ear does the same thing, so I like this).

Contributing to mingus

There are a couple of things that would be very nice to have: alsa, csound and midi support (in extra) and more analytic tools for containers are just a few of them. I can imagine people having their own ideas of cool extras or other things that would be nice to have and that's why I'm trying to open up the development process.

I have transferred all my stuff from my own repository into google's and also added my bugzilla entries into their issue tracker, so the development can take place over there. Building and distributing a new version and its reference documentation is 98% automated as well, so almost everything is in place.

The only thing that's missing now is a proper way to communicate about the project using a mailing list, google group or forum, but that will be coming soon (I'm open for suggestions). I will also add some new pages to the wiki aimed at people who may want to contribute to mingus.

mingus' downloads, docs and tutorials are here

mingus: Music Theory for Python

Last time I posted a rather chaotic and not-very-to-the-point journal of some of the things that are wrong with the use of naive algorithms to represent the ideas in music theory. I couldn't get this thing out of my head and decided to start working on a fundamental music theory and notation package for Python to make writing open source music applications easier.

Today I'm happy to announce that mingus, as the package was dubbed, is currently at version 0.1.7.5 and encapsulates most of the things I set out to make (and much more). The features, so far:

Work with notes, intervals, chords, progressions, scales, keys and meters in a simple and theoretically sound way.
Generate natural diatonic intervals (seconds, thirds, fourths, etc) and absolute intervals (minor seconds, perfect fifths, etc.)
Generate natural diatonic triads and seventh chords, and absolute chords (min7, m/M7, etc). mingus also knows about inverted chords.
Refer to chords by their diatonic function (tonic, subtonic, etc. or I, ii, iii, IV, etc).
Work with diatonic scales and their modes (ionian, mixolydian, etc.),
generate the minor (natural, harmonic and melodic) and chromatic or whole note scales.
Recognize intervals and chords from lists of notes.

Here are some simple examples:

 >>> from mingus.core import *

	>>> chords.minor_seventh("C")

	['C', 'Eb', 'G', 'Bb']

	>>> progressions.to_chords(["I", "IV", "V"], "C")

	[['C', 'E', 'G'], ['F', 'A', 'C'], ['G', 'B', 'D']]

	>>> intervals.minor_fifth("C")

	'Gb'

You can download mingus, follow its development and learn more about the package at mingus project's page on
googlecode. To give you a preview of what's to come, here is what a simple bar currently looks like:

The representation is still missing a couple of things as you can see, but work continues.

More mingus here

UPDATE: Version 0.3 of mingus has been released.