Quartz Composer is a free graphics and imaging tool on the Mac platform which is available as an optional install that can be found on your Mac OS X DVD since version 10.4.

It allows very easy creation of complex audio-reactive animations or effects which can now be played in GrandVJ (read the previous article about GrandVJ and Quartz Composer). As I’m writing these lines, the last version of Quartz-Composer is 4.0 on Mac OS 10.6 (Snow Leopard).

In this article we will demonstrate how to quickly create a bass-reactive Zoom/Blur effect like the one you can see below:

Important: This tutorial requires that you have some basic knowledge of Quartz Composer: if you have never used it, you really should first go through the “Getting Started”. You will also find some helpful readings at the Apple Developer Connection web site about Quartz Composer.

To create our effect we will simply edit Quartz-Composer’s default template named “Image Filter” to make it audio reactive. Basically, we will connect bass levels of the computer’s audio input signal to the “Amount” parameter of our Zoom/Blur filter. (Read more below..)

• Start the Quartz-Composer.app which is located in /Developer/Applications/

• From the menus, choose (File –> New from Template –>) Image Filter :
  

• Here’s the patch that will open:

• In the menus choose Editor –> Edit Protocol Conformance

• We will now tell the ArKaos software what inputs must be transmitted to the Composition. The ArKaos engine can send visuals or sound for processing by the Composition but only if he knows that these feeds will be accepted.
  To enable this you must check “Music Visualizer” and click “Done”

• You’re back to the Composition and several input modules have been added on the left:

• Remove every unnecessary input modules, so they won’t be listed as input parameters in GrandVJ’s effect parameter panel:

• Quartz composer can hide blocks inside blocks, it’s called macros. It’s very powerful and in this case it holds the most important part of our composition: the processing part, which will affect the visual before copying it to the output.
  Double click on the “Process Image” macro to open it, Quartz Composer details each block contained in the macro:

• We want to connect the audio volume to the “Amount” input of the “Zoom Blur” filter contained in this macro. To do that we need to make the “Amount” parameter available in the macro block so that it can be accessed from the top level, it’s called publishing an input.
  Publish the “Amount” input as in this screenshot:

• When it’s done, we can go back at the top level to add some processing.
  In the top bar, click the “Edit Parent” button to go back the parent patch:

• Now we will add a processing block to create a value from the sound input.
  From the Library, add an “Audio Processor” module:

• You’re back to the patch with the “Audio Processor” module added. The magical thing about Quartz Composer is that you can connect the modules in many different ways, when you add a module you just have to tell Quartz Composer how to feed the inputs and connect the outputs.

• First we want to process the audio received from the ArKaos engine. To do that we connect together the “Audio Spectrum” and the “Audio Processor” module:

• Now the audio processor will create three values from the audio we are feeding, Lows, Mids and Highs. For this effect we just want to use the lower value which is the one reflecting the bass volume. So we connect the Lows output from the “Audio Processor” module to the “Amount” input we just published in the “Process the Image” macro:

At this point we need to get an audio signal so we can test our effect within Quartz Composer. For this purpose we will add an “Audio Input” that we will disconnect in the end so GrandVJ can input its own audio signal into the “Audio Processor” module.

• From the Library, add an “Audio Input“ module..

• ..and connect its “Audio Spectrum” output to the input from “Audio Processor”

Now we can see the image reacting to the audio input signal, but the effect is definitely too low, so we need to boost the audio signal values. A simple way to do that is to add a new processing module, specifically a math module:

• From the Library, add a “Math Module”..

• ..and configure it to operator Multiply, set Operand #1 to value 100:

• Connect the Lows output from “Audio Processor” to “Multiply” then connect “Multiply” to the “Amount” input from “Process the Image”

• You can now disconnect the “Audio Input” from the “Audio Spectrum” so that, once this composition is loaded in GrandVJ, the software will automatically send the audio input signal directly to “Audio Spectrum”.

• Finally save the composition as “MyBassZoomBlur.qtz” in /Users/MyAccount/Library/Application Support/ArKaos/GrandVJ/Quartz Composer/Effects/

Now is time to start GrandVJ and find your hard work rewarded by playing with your very own brand new effect. You will find it located in the Effects panel under the “Quartz-Composer (User)” effect category.

Notice how GrandVJ has made available the X and Y effect parameters from the Composition in the Effect parameter panel so you can use them to control the behavior of your effect in real time.

Now, that wasn’t hard, was it? If you want to go further, a good start will be to create new effects based on this Composition by replacing the “Zoom Blur” effect by another filter.

Don’t hesitate to share your creations on the forums :)

Related posts:

1. How To: Create audio-reactive Flash for GrandVJ
2. Quartz Composer integration in GrandVJ 1.2
3. Using QuickTime or FFMPEG on a PC?

But as Chief Software Architect behind the ArKaos real time video mixing engine and also CEO and founder of the company, I feel the need to explain today a little more about how we develop software here and more wider, about the company’s philosophy, as this is something that can’t really be expressed in the Job Description.

Actually, if finding a good software developer with all te required skills is something totally possible, finding one that will fit right into the team is not an easy task. As I always say – and I am proud of that – we are not a typical company. Yes, we always try to stay open to the broadest possible range of technologies, we are cross platform, we use lots of different video API’s and our products go from very inexpensive software to customized and very powerful hand crafted media servers.

The only way we can achieve this involves tons of passion and spending a lot of time “learning on the job” every day. What we do at ArKaos is optimizing our products to push the limits as far as we can. It can be by trying to work around undocumented bugs in OpenGL drivers, hacking our way through poorly documented API’s or spending days tracking hypothetical memory performance issues..

Another important aspect of working at ArKaos is what I call our “Punk Mentality”. We have to be self-sufficient, we don’t expect others to help us. When you want to do things in a way that has not been done before, you can expect a lot of problems and many failures before you eventually succeed. In this sense, we do not write code in a way that will make our life easier, instead we code with the constant will to find new ways to avoid unnecessary dependencies from the operating system, to use the least possible memory and the least possible disk bandwidth.

But the good thing is that the world of real time video is fascinating, so many people did not yet perceive the potential of creating perfectly synchronized audio visual performances. And so many things are still to be created and defined. Personally i’m feeling a bit old writing this because I started creating real-time video tools at the beginning of the 90s.. yes, 20 years ago already.. And since then, just look what has been achieved in this area, all the different tools used by VJ’s and all the different media servers used in the lighting industry, these are so many different solutions that can be used and combined to reach the same goal: create amazing real time audiovisual performances.

One last important thing when working at ArKaos is our social behavior; we are mainly a software company but we love to speak with hardware companies. This is something that can only be done by spending a lot of time networking during trade shows and exhibitions. We like to build friendship relations and try to convince companies – sometime much bigger than us – that a collaboration is always the way forward. That’s how we forged many alliances during our history, that helped promote the concept of real-time video performance and – I like to think so – contributed significantly to what you can see today when you go to a concert and enjoy a nice moment.

You don’t go to a concert just to “listen to the music” but to “see a great show!” Right ?

If you think the same way and feel you can be a part of our team, here is our more formal job description for Graphics Software Engineer

No related posts.

Please test it and report any problems you might encounter.
The download links are at the end of this post.

Update: The beta went fine, thanks for your feedback: we have made available the RC1 on the forums

New Features

• Support for movie’s audio through FFMpeg and Quicktime (supports most formats). MediaMaster 1.2 comes with the new ArKaos Audio Output engine. The application will now play the audio track embedded in a movie file. We’ve added a new Audio preferences panel in which you can select your audio input and output devices, the audio output engine buffer size and its main volume volume. You can deactivate audio playback by choosing “No Audio” as output device.

Audio format and sampling rate have been added in the Visual Info box that appears when you hover the mouse on a movie thumbnail in Media Library or in the main window:

• New Time Reference: In MediaMaster 1.1, we introduced a new ultra smooth playback engine, synchronizing movie playback with the graphical card display (read more about this here and here). In version 1.2, you now have the choice from the Display preferences to synchronize either to the display or to the system clock. Synchronizing to System clock will make MediaMaster play content like version 1.0 did, but since we’ve now added audio playback, this setting is mandatory to get a good audio/video synchronization.

• Support for Flash Actionscript3: the flash player has been updated to cope with AS3 (you can find more info about how to write your AS3 to work with MediaMaster here)

Please report any problem you may find with this beta in the dedicated forum topic (and not in this article’s comments please).

Enjoy!

Related posts:

1. GrandVJ 1.2b4 with support for audio
2. ArKaos MediaMaster 1.1: The Smooth Operator!
3. GrandVJ 1.2 RC1 with a final audio engine

Meanwhile we have released GrandVJ 1.2 and we took the time to include a special template for the Ohm64 in this release. As we did for the Akai APC40, this new dedicated template allows even more bidirectional communication between both products and takes advantage of all the control and feedback possibilities offered by the device.
It’s just like if your GrandVJ was made for your Ohm64.. and vice-versa :)

And thus, below is a new video from Livid showing their controller in action with GrandVJ:

Related posts:

1. GrandVJ and Livid’s Ohm64
2. Sharing MIDI templates in GrandVJ
3. GrandVJ works out of the box with Livid’s Block

Read more for changes and download links..

Changes in this version:

• Quicktime audio playback has been re-implemented as fallback solution for movie files with soundtrack not supported by FFMpeg. GrandVJ 1.2 should be able to play sound from 99.9% of the movies files you throw at it :)
• We’ve added an Audio setup panel in Preferences, giving the ability to choose both input and output devices as well as their buffer size:
• Fixed the problem with Layer preview switching when using an nVidia graphics card on Windows 7.
• Enhanced graphic engine performances on Mac, giving the ability to play high-definition visuals even more smoothly.
• Two new demo patches are available and we’ve reorganized the Default Content folder with a new selection of visuals from the DVD that comes with the box. These visuals have been kindly provided by various artists, be sure to have a look at the “about” pdf in each folder of visuals.

Download links:

• Mac: ftp://www.arkaos.net/pub/seeding/GrandVJ/ArKaos_GrandVJ_1_2_RC1.dmg
• Windows: ftp://www.arkaos.net/pub/seeding/GrandVJ/ArKaos_GrandVJ_1_2_RC1.msi

For more info about the new features in GrandVJ 1.2, checkout the announcements about each beta release in the GrandVJ forum topic

Enjoy!

Related posts:

1. GrandVJ 1.2b4 with support for audio

Now why not start this brand new year by looking at what were the most popular articles from this blog in 2009 (second half of 2009 that is, since we started this Blog in May..).
This way we can better understand what kind of posts our dear readers like the most…

1 >> How To: Create audio-reactive Flash for GrandVJ

(1,221 unique visitors)
Here’s the first in a series of tutorials showing how to take advantage of the Flash capabilities within ArKaos GrandVJ. Note that it will also work for NuVJ and MediaMaster since the variables are the same. In this tutorial Ben Guerrette explains how to tap into the EQ data to create some simple audio-reactive animations..

2 >> GrandVJ, Ableton Live and Akai MPD24

(1,010 unique visitors)
This tutorial involves using an Akai MPD24 Drum Controller in conjunction with Ableton Live as a host sequencer and ArKaos GrandVJ as a ReWire slave for MIDI synchronised video clip launching..

3 >> GrandVJ 1.2b1 supports Flash AS3, Quartz Composer, APC40 and more

(773 unique visitors)
GrandVJ 1.2b1 is out for testing! It comes with ton’s of cool new toys such Actionscript 3 support, Quartz Composer player, Akai APC40 template and much, much more!..

4 >> Advantages of using Flash for VJ content

(755 unique visitors)
While a lot of VJ content is produced with high-end motion graphics, 3D modeling, and video editing software, Flash provides some advantages over these platforms not immediately obvious..

5 >> Quartz Composer integration in GrandVJ 1.2

(660 unique visitors)
GrandVJ 1.2 now allows to use Quartz Composer .qtz files directly, both as sources and effects, let’s have a quick look at how you can use Quartz compositions with GrandVJ..

Ok, “Tutorials and How To’s” you said? We got it!

Finally, here are a few ones that did not made it to the top five although we think they should have been ranked higher, just in case you missed them :)

1 >> ArKaos MediaMaster 1.1: The Smooth Operator!

ArKaos is pleased to announce the release of MediaMaster 1.1 with major improvements pushing the boundaries of professional video projection another step further..

2 >> Why software genlock at 60 FPS does matter!

Here’s why the “software genlock” optimization from ArKaos engine introduced in MediaMaster 1.1 is very important to ensure the best possible fluidity on computers equipped with multi core processors..

3 >> Using the right decoder for your video content

To get the most out of GrandVJ or MediaMaster, benefit from the advantages of multi threading and perform frame blending you should make sure that your content is always played through the best decoder..

No related posts.

One thing that appears obvious to us after more than 10 years of working with the various solutions that can be used on Mac and Windows to decompress video is that not all solutions are equal.

We believe that our users should be able to play almost any kind of media with the ArKaos software and at the same time we do everything a team of developer can to get the best performance on the platforms supported.

Practically the ArKaos software can play media files through the following decoders: QuickTime, FFmpeg, the Flash plugin, Quartz Composer and Windows Media Player. Currently (December 2009) we don’t allow the user to decide through which solution a media will be decoded, instead we try to make the best decision based on our many years of experience and tests. In the case where several video codecs (such as Mpeg4) can be played through different decoders like QuickTime, FFmpeg and Windows Media Player, we always prefer, no matter the platform, to first try to handle those codecs with FFmpeg as we have always had the best results with FFmpeg during our tests, and here’s why..

The other solutions..

Of course, on the Mac we could stick to QuickTime.. But sadly it appears that QuickTime has become almost obsolete today and is just used to decompress the codecs needed by the iPods. Most codecs are not properly multi threaded in QuickTime and this is a big problem for us. With GrandVJ or MediaMaster taking advantage of multi core machines by using each core separately to decompress the video layers you are mixing, it’s too bad to miss that great improvement by playing videos that absolutely need the QuickTime support.

On the PC we could be using Windows Media Player, it is a wonderful technology but it’s architecture is completely incompatible with the sophisticated engine we use to allow frame blending and frame prediction in the latest version of GrandVJ and MediaMaster.

Flash animations and Quartz Compositions, of course, are unique in the sense that they can only be read by their own dedicated player.

As you now understand, once you open a media file in ArKaos your content can go through various roads and it’s important to consider that. To get the most out of GrandVJ or MediaMaster, benefit from the advantages of multi threading and perform frame blending you should make sure that your content is always played through FFmpeg.

How does ArKaos process the movie file?

On the Windows platform it’s pretty easy because the software first tries to play a file with the FFmpeg player and then falls back to the others (Windows Media Player or QuickTime) if needed.

But things get harder with the Mac platform because some files that could be played by FFmpeg may be rejected and fall back to QuickTime just because they have not been saved the correctly. Many QuickTime files have been created in a way that make them compatible with Mac OS X but also it’s ancestors Mac OS 9 and 8 and 7 and… Those files make use of what is called a Resource Fork.

To make it short, when Apple introduced the Macintosh years ago they decided that a single file would in fact be two files: the data file itself and an optional resource file. Many software have been using that Resource Fork to store additional data and information. This is now a problem for many .mov files that were created in the past and even still today with some applications that were born under the Mac OS 9 era and still generate QuickTime .mov files that have a Resource Fork.

This is quiet an issue for us because when the FFmpeg decoder can’t handle those files and makes ArKaos fall back to QuickTime, you end with QuickTime’s very bad multi core support even for very common codecs such as Photo JPEG. Another bad side effect of this Resource Fork is that most of the time when you will copy a movie file that has a Resource Fork to a Windows or Linux machine it will simply not be readable anymore because that tiny bit of information contained in the Resource Fork is not read on other platforms although it is vital to play the content.

How to get rid of the Resource Fork?

Thankfully it’s very easy to remove that annoying Resource Fork from a QuickTime file. You can simply open the file in today’s QuickTime player and do a “Save As”, QuickTime will reprocess the file correctly. The process is called “flattening” a movie file, it is very fast and will not re-compress your movie, this is very important to avoid re-compressing when possible.

Open your file in QuickTime and go to File, Save As..

.. and enter a new name

Of course, you will only need to do that if your file was created on a Mac, the free version of the QuickTime player is enough to do this and it works on all Mac OS X versions up to Snow Leopard with the latest QuickTime player. Now if you have lots of movie files to convert, you can of course look for some professional software that can queue and “flatten” a bunch of files from a directory.

FYI, here is a QuickTime file that has a Resource Fork (moov), inspected with a programming utility called Resorcerer:

And here is the same file that has been flattened:

How do I know what decoder is used?

In GrandVJ it’s very easy to see what player is used by the ArKaos engine when processing a file; simply hover your mouse over the cell that contains the video and the contextual help appears.

In this one, the video file with a resource fork will be played through QuickTime:

In this one, the same video flattened with the QuickTime player will go through the FFmpeg path:

(The same information is also available in GrandVJ for each movie listed in the visual library and in MediaMaster you can get that information from the file library window.)

Conclusion

During the first days of the release of GrandVJ 1.2 beta 3 with the new audio engine (which relies on FFmpeg) some users have complained that some old .mov files were not playing audio correctly with the beta version, this was most of the time caused by the Resource Fork and flattening the files fixed the problem. As a bonus, it will make your files cross platform.

This should give you some lights on what is happening when you drop a file on GrandVJ or MediaMaster, as you can see a lot of work is being done under the hood :)

Related posts:

1. Using QuickTime or FFMPEG on a PC?
2. New video codec based on texture compression
3. Advantages of using Flash for VJ content

We’ve been a bit quiet lately but we’ve been working hard trying to move sound support from the old system (based on QuickTime) to an implementation using FFMPeg. This is because the QT support is getting worse and worse and we think it’s important we move to a more controllable medium.

The Beta 3 released last week on our forums contained the first result of this implementation but wasn’t very smooth or even stable especially with backward/forward playback.

The Beta 4 fixes the backward playback bug and relies on a new FFMPeg build that supports AC3. This means you can now drop VOB’s & MKV and get proper audio running… If the vob is 5.1, we also do stereo mixdown.

Surf to the forum post for more details about this Beta and the download links..

Related posts:

1. ArKaos MediaMaster 1.2 beta1 with audio
2. Using Flash ActionScript 3 in GrandVJ 1.2b1
3. GrandVJ 1.2b1 supports Flash AS3, Quartz Composer, APC40 and more

If the consoles have changed the club continues to trust the 3 ArKaos media servers installed by Sebastien to display visuals on the LEDs of the DJ / LJ booth and on the various Plasma screens around the place. And apparently, Nicolas Bailly put these to a good use: here’s a video from a light show he set up at the club on Pryda’s track “Miami to Atlanta”.

Nicolas also has a very interesting blog (in French) about lighting design, check it out if you want to know more about him.

Related posts:

1. mondo*dr article: MediaMaster at Zouk KL
2. Vickie Claiborne and MediaMaster in PLSN

We have optimized the Quartz Composer integration in GrandVJ 1.2 so that the compostions are rendered natively through the fastest available interface.

Finally, when using a composition as an effect the application will scan the published inputs and map them as effect parameters, if the format is compatible with the engine. GrandVJ can automatically map parameters that are float values, indexes, colors and booleans and there’s one visual input of the composition that receive the cell visual.

Now let’s have a quick look at how you can use Quartz compositions with GrandVJ..

Using compositions as sources

The quick way to use a composition as source is to import it just like you do with any movie, by drag and drop on a cell or by reaching your .qtz through the file browser. When doing so, GrandVJ will save the path to the .qtz file in your project as it does for any other visual.

For your convenience, GrandVJ installs a set of source compositions in “/Users/<username>/Library/Application Support/GrandVJ/Quartz Composer/Visuals”, these are listed in GrandVJ under the “Quartz-Composer (User)” section of the sources browser so that they are always available when you create a new project.

The idea is that you can also add your favorite source compositions there so that you always have them ready when you work on your projects.

Playing compositions as source does not allow modifying the variables from within GrandVJ.

Path for user compostions as Sources	User compositions as Sources

Using compositions as effects

System compositions as Effects

For effects, GrandVJ will add all the compositions from your system that are supported. It does it by automatically scanning the system folder “/System/Library/Compositions”. These compositions will appear in the application under the “Quartz-Composer (System)” section of the effects browser. You can use them as starting point to develop new effects. You can also add your own compositions as effects, just drop them in the local directory “/Users/<username>/Library/Application Support/GrandVJ/Quartz Composer/Effects”. They will appear in the “Quartz-Composer (User)” section of the effect browser.

Path for user compostions as Effects	User compositions as Effects

About the parameters

As a quick example, let’s have a look at the effect parameters from the system composition Image Hose. With the Quartz Composer application (found in Developer > Applications – you need to have installed Apple’s developer tools of course), open the file “/System/Library/Compositions/Image Hose.qtz” (ore better, open a copy) here’s what you should see:

Image Hose composition

Open the inspector (Editor > Show Inspector) and from the menu, look at the Published Inputs & Outputs, here’s what you have:

Image Hose composition parameters

In GrandVJ, the first 4 parameters that can be mapped automatically will be displayed. In this case it will be PrimaryColor, SecondaryColor, Pace and Size:

Quartz composition parameters in GrandVJ

As you can see, GrandVJ will only display the meaningful part of the parameter name and remove all _ProtocolInput strings from parameter names – this allows to easily display the full parameter name in the panel – it will also make things a little nicer by capitalizing the first letter of the parameter if necessary.

There are many resources on the web where you can find information and help about creating your own Quartz compositions, you can start by looking at Apple’s Developer Connection.

Note: you can download GrandVJ 1.2 beta 2 from our forums

Related posts:

1. How To: Create audio-reactive effects for GrandVJ with Quartz Composer
2. GrandVJ 1.2b1 supports Flash AS3, Quartz Composer, APC40 and more
3. Using QuickTime or FFMPEG on a PC?