Of course, every end is a new beginning. For me this means that I am going to start a whole new adventure. Luckily I am not alone in this because starting from scratch is never easy. Together with my colleague Tim Richter we are going to pursue our ideas of a multi-platform development kit.

2013 will be a roller-coaster. I am all in.

The reason why I integrated it into our system was quite simple: the only person who could perform a deploy was me. This was hindering the rest of the team of course.

Our architecture allowed “hot” deployments already. That means users who were already using the Audiotool application did not notice a deploy happened at all. This is of course only possible as long as there are no new dependencies on the API.

The Audiotool startup process is a very important ingridient. We have a boot sequence which loads a configuration file upfront. In this configuration file is a version number. The version number is used to load all dependencies the application needs to start. We have put a repository server in place which serves SWF files based on this version and they get cached till the end of time.

If a user started Audiotool when version 1.1 was online and we update to 1.2 in the meantime the user would still load all audio plugins for version 1.1. There are some cases when a hot deployment is not possible (yet) and we call this a scheduled update.
This pushes a message into Audiotool, notifying users that they should save their song and restart the application. But this is not part of the blog post and a scheduled update is very rare. We did it twice last year, once this year.

When we did a deploy it was basically done like this:

1. Make sure all changes are checked in.
2. Update a default version key in some source files in case the boot sequence cannot load the configuration for whatever reason.
3. Update the Nginx configuration so that some verison-less files like our embed player are routed correct.
4. Create a tag for the repository.
5. Create a clone of the tag.
6. Execute mvn -Pdeploy-to-live deploy in the clone. This command already updates all plug-ins on audiotool.com and puts new metadata in place.
7. Copy all SWF files form a local directory to S3.
8. SSH into a server and update the configuration file Audiotool parses at startup.
9. Reload the Nginx configuration.

A lot of steps. This means a lot can go wrong of course. And even though we had all those steps written down in an internal Wiki it is still hard to do all this. You need the appropriate SSH keys to log into some servers, uploading files to S3 requires a tool which is also configured and not everyone is comfortable with editing a Nginx configuration file on an Amazon server through the terminal.

There was only one deployment that happened without me. I was at FITC Amsterdam in 2010 at that time and it was a long phone call. Obviously something had to change.

The first step was to get rid of all the manual configuration hassle. The configuration file that one needed to change via SSH had to go so I made it a dynamically created file by the web server. The actual version is pulled from the database and this made our my life much easier already. But still this was not what I wanted. Of course nothing really changed. You still had to perform the S3 upload, SSH into a server and so on.

But since the configuration was already served via the web server I could start automating more things. Even better: with our last major update we dropped the version number from Audiotool. User’s would stop expecting to see big changes and a version number that increases. Instead we focus on being much more active and to push changes online as quick as possible.

Since I already wrote some shell scripts for myself to deploy various server applications with a single command I started doing the same for the whole Audiotool application.

The first step was to get rid of the default version in some of the ActionScript source files. I simply [Embed] a text file now which contains the version information. A single text file is so much easier to change.

Then I added an API call which allows me to change the version information online. That way a new version can be released easily without any human interaction. With the API call in place and a text file containing all important configuration parameters the last issue was the Nginx configuration. But a small script on the server should do the job.

So I started writing a little script which performs all the necessary tasks:

1. Create a clone of the repository.
2. Get the id of the current revision.
3. Create a tag for the revision like YYYY-MM-DD_REVISION.
4. Replace the default version with the revision.
5. Execute the Maven command to deploy some metadata and build all SWF files.
6. Upload all SWF files to S3.
7. curl our API with the revision information.
8. SSH into a server at Amazon to update Nginx.
9. Cleanup.

That’s all there was to it. Since I do not have that much experience with shell scripting the most annoying part was to figure out a way to replace the text in a file. After looking through all examples of possible/impossible sed and awk options I got lucky with sed -i "/@build.version@/ s//${HG_VERSION}/g" ${HG_CLONE}/default.version.txt. This basically replaces @build.version@ with the content of ${HG_VERSION}. I know, magic. I just want to write it down here so Future-Joa can Google this and come back in five years getting a quick answer.

With the shell script at our disposal we simply had to hook it into TeamCity, a fantastic continuous integration solution by the way. After configuring some command line tools on the CI server we were ready to go.

The results speak for themselves. We made already twenty deployments during the last two weeks. That is about as many deployments as we did since we started Audiotool. Because deployments become less scary and everyone can trigger them we are able to iterate quicker and get rid of a lots of bugs. In fact it makes it also much easier to reason about your program. If you push hundreds of new features and get a null pointer exception: good luck. However if you just changed one little thing and get loads of bug reports it is quite easy to identify what could be the possible culprit. Of course this is only a Flash problem since there are no stack traces in the release player. But I guess if you build a JavaScript application you would have similar problems.

I can only recommend doing the same. Especially for your own sanity. 100% automated deployments are really cool and stress free! It is also much easier to setup than you might expect.

I am no longer committed to supporting any Flash related open-source projects.

Here is why. When I started using the Flash Player it was quite easy to reach its limits. However you were able to get around those limitations with clever hacks and debatable optimization techniques. I was always keen to share my knowledge with the community and to explore all possible options to achieve best performance.

The Flash Player has been hibernating for half a decade now. The only glimpse of performance was finally a set of specialized op-codes which allow you to modify an array of bytes. In layman’s terms this means it was finally possible to do a[b] = c with an acceptable performance. So I wrote a tool which allows you to do just that and many other things. I have spent a good time of my free time trying to improve the performance of the Flash Player and contributing all my code to the community.

As a reminder: I showed some drastic performance improvements at Flash on the Beach in 2009. That was three years ago. It was not necessary to modify the Flash Player and it was not necessary to modify the ActionScript language.

The Adobe roadmap for the Flash runtimes states that Flash Player “Dolores”

• will support ActionScript Workers
• comes with improved performance for Apple iOS
• and ActionScript 3 APIs to access the fast-memory op-codes

This player should be released in the second half of 2012. The “Next” Flash Player will finally include

• modernizing the core of the Flash runtime
• work on the VM
• updates to the ActionScript language

This is planned for 2013 apparently. And what can we expect? Type inference, static typing as a default, and hardware-oriented numeric types. Hooray, so it will be finally possible in 2013 to write a[b] = c without having to use some weird fast-memory op-codes. If we look back to the year 2009 this makes me really sad.

With the introduction of the speed tax you will now have to license your application. No matter if you make money out of it or not. Now I think that 9% is a decent number and I can understand Adobe’s position on this. In fact it is much more friendly than the 30% Google or Apple take. However the AppStore was an invention. What is the invention here? Squeezing money out of an already existing feature, and suddenly making it unavailable after people have been relying on it for years to push the boundaries of the platform and actually innovate?

But for the hell of it, a[b] = c is not a premium feature. Nor are hardware accelerated graphics. That is what I would expect from any decent runtime.

Limiting the capabilities of a runtime — by defaulting back to software rendering for instance — will make it less attractive to use it in the first place. You are probably not interested to go through a signing progress for a small demo. So your performance might be crap, people will complain about the Flash Player taking 100% CPU because its using software rendering (YEY! 2013!), laptop fans will start to dance and you will look like a bad developer because that other guy got the same thing running with hardware acceleration. Or you could use a different technology.

Why is this bad? Because apparently this signing with a $50k threshold targets the enterprise and small developers seem to be acceptable collateral damage. However thinking about the next five to ten years: who is going to write ActionScript code if it is no longer attractive to play around with it in the first place?

We still rely on the Flash Player at audiotool.com. I am still developing for it and we will probably have to use it as long as there is no alternative. Me no longer supporting open-source tools is just me no longer spending my personal time for a platform that I would not use for private stuff. Work is of course not always about fun. But fortunately I am able to spend 90% of my time writing Scala code.

I will finish this blog post with some bad karma:

It’s also worth noting that the new Adobe license will prohibit scenarios where you’d have the first level of a game in the Flash Player, and the full experience inside the Unity Web Player. Alas, this is something you’ll need to be aware of if you were considering such a route.

You will not only pay for the features. You are also welcome to cede some of your rights.

But then again, the playback was done using JavaScript only and calculated in realtime.

Audiotool is a complex piece of software so you might ask how one could torture themselves by implementing it in JavaScript? We didn’t. Instead we started building our own vision of a cross-platform application framework a couple of months ago.

Introducing project Hiddenwood.

Hiddenwood is a collection of libraries and tools specifically designed to support different devices and platforms. The core libraries are the “driver layer” and always platform-specific with a platform-independent interface.
On top of that we provide a basic layer of libraries like our UI system, animation framework or managed collections which guarantee 0% garbage collection activity and have been battle-tested in Audiotool.

The framework is all about speed and consistency. The rendering pipeline is optimized for OpenGL and although we offer something similar to Flash’s display list a lot of features are not available because they would compromise the speed.

Speaking about speed: we are always interested in staying as native as possible on our target platform. So for the browser we emit JavaScript, for Android you will get the full DalvikVM performance and for the desktop you will get JVM performance. This approach has also another very important aspect. If you want to go platform-specific for certain features you can do that.
For instance if we want to render Audiotool songs on the server using a fork-join pool for our audio calculation this is possible and might not make sense on an Android device.

You write Java code and the supported platforms are native desktop applications, Android (minimum requirements are Gingerbread and OpenGL ES 2.0) and modern browsers. Now for browsers we even go one step further and support multiple options. That means if WebGL is not available we simply fallback to a normal canvas based render-engine. The same applies to some of the Android drivers.

iOS is of course important as well and we are actively researching the best option that will give us the most flexibility and performance.

We are currently working on two real applications built with Hiddenwood. So far it is a real pleasure to enjoy quick build times and simply test what you want on the desktop with great debugging capabilities. When you are ready you can try the same app on Android or in the browser — which might take a little bit longer to compile.

Because we see Hiddenwood as an application framework there are a lot of goodies built-in like a sprite-sheet based class generator. Think Image mixerBackground = Textures.mixer.background(); where mixer was the folder name and background the name of the file.

We believe that as a developer you really do not care about what kind of technology you are using and just want a great result. We also think that you should be able to reuse platform-independent code across multiple projects. However we do not want to take power away from the developer because if you know what you are doing: go for it.

Of course we are not the only ones with this idea. Nicolas Cannasse saw the signs years ago and invented haXe which gives you a comparable experience and Google released playN a couple of weeks ago which takes a similar approach (and requires another 25 committers :P).

But when we started Hiddenwood we wanted the Java tooling experience and playN was not public at that time. We also think that a game engine is not what you want to use for all kinds of applications. So we like to be able to give people the freedom to build their own game engine on top of Hiddenwood — and calculate physics in a different thread peut-être.
Speaking about threading: the only possible solution that works across all platforms is a shared-nothing architecture which we put in place. However if you write platform specific code you can use of course everything the platform offers and a lot of the Hiddenwood core libraries like the network- or cache-layer make use of multiple threads.

In the end what makes Hiddenwood special in my opinion is that we do not believe in write once run anywhere because that just does not make sense. The essence and philosophy behind Hiddenwood is to write platform-agnostic code using kickass-libraries and being able to reuse that. Audiotool on a tablet would look completely different from Audiotool running in a browser. And Audiotool on iOS would probably be also a little bit different from Audiotool on an Android device because there are simply different paradigms you should respect.

I hope that we can share more information with you soon. With the news of mobile Flash Player being deprecated and the ongoing demand for cross-platform development we have exciting times ahead of us. I am also super excited about the (beautiful <3) applications which we are going to release in the not so distant future.

Personally I am quite disappointed that Dart looks a lot like Java 8 with some tweaks here and there. Although the language is in its early stages I wonder why pattern matching was not a top-priority to the language team since Dart relies heavily on message passing. More on that in a second.

Since Dart should scale from small scripts to full applications it is nice to see support for generics from the start, optional typing and functions as first-class citizens.
However that makes Dart look more like a better Java than JavaScript.

What makes Dart more of a web language is how you perform concurrent computations. This is done by shared-nothing message passing. The same way you do it with workers in JavaScript, actors in Erlang or Scala and hopefully quite soon in the Flash Player. We should not forget Go in this equation since it was also an effort to try out concurrency via channels that can send and receive messages. If we look a little bit more into Dart we can see some of the same ideas.

A worker in Dart are called Isolate [1]. This makes sense since it runs completely isolate from the rest of your program. What is nice about the Dart approach is that you do not have to deal with files anymore when using an Isolate. JavaScript requires you to perform the task of having some special file lying around somewhere that adheres to the Worker protocol. When I write my application, I do not want to think about files that have some special top-level logic in them. Especially, I do not want to write a custom file at runtime using a blob-builder.

Dart embeds this from the start just like Go does. You will have a lot of Isolate objects communicating via a SendPort [2] and a ReceivePort [3]. The Promise<T> [4] is just like Java’s Future<T> a holder for a value that can be computed later. I just wonder why they did not implement a read-only and write-only view on them. It would be nice if Promise<T> would be extended with methods like map, flatMap etc. because your code would be less spaghetti.

With no further ado I would like to explain some code now that you know how Dart works. Basically we calculate the n-th Fibonacci number with a very expensive approach. You would never do this in reality. Let us define fib(x) = fib(x - 1) + fib(x - 2) with fib(1) = 1 and fib(2) = 1. So fib(3) is fib(2) + fib(1) which is 1 + 1. Sorry for to bore you to death.

If we implement this in pseudo-code we get something like this:

int fib(int x) {
  return x < 3 ? 1 : fib(x - 1) + fib(x - 2);
}

If you would like to make this multi-threaded with Scala you could wrap the calls to fib into a Future[Int] like this:

def fib(x: Int) =
  x match {
    case 1 | 2 => 1
    case _ =>
      val n1 = future { fib(x - 1) }
      val n2 = future { fib(x - 2) }
      n1() + n2()
  }

We want to do the same with Dart. Since it is a language targeted for the web we get no access to blocking calls. The Scala code blocks when n1() or n2() is called. In layman's terms: we wait until the value has been computed.

Since there is no support for continuation passing style like C#'s async we have to write a lot of callbacks now.

This is the entry-point for the Dart version of this:

main() {
  int n = 7;

  print("Computing fib($n) ... ");

  new FibIsolate().spawn().then(
    (port) =>
      port.call(n).receive(
        (value, port) => print("fib($n) = $value")
      )
  );
}

First of all we see string-interpolation which is nice. Then we create a FibIsolate which I will show you in a second. When you create an Isolate it does not do anything so we have to call spawn() to perform the actual computation. However spawn returns us only a Promise<SendPort> which we can only use when available. This is done with the then method. then takes a function as an argument.
There are multiple ways we could do this. then((x) => ...) is the same as then((x) { ... }). You are only allowed to use the first form for a single expression but you do not need to write all that boilerplate. In fact you are even allowed to omit a semicolon. Hell Yeah!

So when we get a SendPort which happens to be the case when our function is called we can send some value via the call method. Why use call and not send? call returns a ReceivePort which allows us to wait for the result. We do this by calling receive on the ReceivePort with a closure to print the value we get back.

Now let's have a look at the implementation of FibIsolate.

class FibIsolate extends Isolate {
  main() {
    port.receive((n, replyTo) {
      switch(n) {
        case 0:
          replyTo.send(0); break;

        case 1:
        case 2:
          replyTo.send(1); break;

        default:
          Promise<int> n1 = new Promise<int>();
          Promise<int> n2 = new Promise<int>();

          new FibIsolate().spawn().then(
            (port) =>
              port.call(n - 1).receive(
                (n, port) => n1.complete(n)
              )
          );

          new FibIsolate().spawn().then(
            (port) =>
              port.call(n - 2).receive(
                (n, port) => n2.complete(n)
              )
          );

          n1.then(
            (x) => n2.then(
              (y) => x + y
            )
          ).flatten().then(
            (x) => replyTo.send(x)
          );
      }
    });
  }
}

First of all we have to extend from Isolate. The main() function of an Isolate is its entry-point. Now something that really bugs me is the amount of indentation. Nearly every second Isolate which you are going to write has this form. Your actual logic starts at the fifth indentation level.

In the main method we immediately start listening for messages via the Isolate's ReceivePort. Then we need to react on the message via a switch case. Now although it might look simple in this case it is very sad that Dart does not come with proper pattern matching since you will need to write a lot of switch-case in Dart code. If you have ever seen pattern matching in action using Scala you never want to go back to a dumb switch statement.

In the default case it gets a little bit more interesting. We need to spawn two new instances of FibIsolate since we want to do this recursive. We need to receive their result and when we have both results we want to perform the addition and send our value back to the caller.

Since there are no blocking calls we create a Promise<int> for each FibIsolate. When can complete the Promise<int> once we receive a value from an isolate. This allows us to combine both Promise<int> objects and wait on their results. Why do we not nest the FibIsolate you might ask. Because we want to spawn both at the same time so the computation can happen in parallel.

The actual nesting happens in the n1.then((x) => n2.then((y) => x + y)) construct. However the return type of n1.then((x) => n2.then((y) => ...) is no longer Promise<int> but Promise<Promise<int>>. Fortunately someone thought about this case and there is a flatten() method which turns the Promise<Promise<int>> into a Promise<int> we just have to await and the finally reply with the computed value.

However remember the definition of fib(x)? The case for 1, 2 and 0 is quite simple because we can reply with the value immediately. And we do this by calling send on the SendPort which I named replyTo.

You can take a look at the full example running in the browser here.

What I like about Dart is that someone thought about workers and made them first-class citizens. Pattern-matching is on the road map and they will hopefully copy Scala. There are however a lot of things that I dislike.

1. Semicolons are required but optional for shorthand (x) => x
2. The return keyword is required but optional when used with (x) => x
3. No control about concurrency. Are my isolates CPU or I/O bound?!
4. No syntactic sugar for writing an Isolate
5. Not DSL friendly like Scala or maybe even Kotlin
6. int disguises itself as a primitive but is an object instead
7. Missed the opportunity to embed continuation passing style into the language from the start
8. Function is the only function-type

The type-system can be argued as well. I did not find anything about type erasure yet. Personally Dart feels much more like Java and Go than JavaScript. But in the end I would be quite happy if Dart could replace JavaScript since it is a step in the right direction from my personal point of view.

[1] http://www.dartlang.org/docs/api/Isolate.html#Isolate::Isolate
[2] http://www.dartlang.org/docs/api/SendPort.html#SendPort::SendPort
[3] http://www.dartlang.org/docs/api/ReceivePort.html#ReceivePort::ReceivePort
[4] http://www.dartlang.org/docs/api/Promise.html#Promise::Promise

When understanding Scala it is sometimes important to have a look at some real-world examples — especially when it comes to functional programming. When I first started using Scala I had a hard time understanding some of the constructs used by other people. In this blog post I want do explain some of them. The first example is a classic one. You want to debug some structure and output all elements which have a condition set.

Printing Elements of a Collection

You will find this in every first-steps guide for Scala. What I like about it is the fact that you have only little amount of code to write if you want to inspect a certain structure.

list filter { _.condition } foreach println

Flattening a Collection

Convert a collection of collections to a flat collection. For instance you have a List<List<String>> in Java and want to get a List<String> instead.

final List<String> newList = new LinkedList<String>();
for(final List<String> innerList : oldList) {
  newList.addAll(innerList);
}

A lot of code for a common operation. Probably the reason why the Google Guava library supports Iterables.concat(oldList) to get the same result. So how is this done in Scala?

val newList = oldList.flatten

A simple operation which is very powerful. Let me give you another example. Suppose you have a List[Option[T]] for some T. Before understanding Scala very well and before reading this excellent article about monads I once wrote shameful code like this.

val listOfOptions: List[Option[Int]] = 
  List(Some(1), None, Some(3), None, Some(5))

// don't do this!
val listOfDefinedValues: List[Int] = 
  listOfOptions filter { _.isDefined } map { _.get }

println(listOfDefinedValues) //List(1, 3, 5)

When encountering such a construct an alarm should raise in your head, telling you there must be a better way to do this. flatten to the rescue. println(listOfOptions.flatten) would do the job.

val listOfOptions: List[Option[Int]] =
  List(Some(1), None, None, Some(2))

println(listOfOptions.flatten) //List(1, 2)

Using flatMap for good

Although flatMap might sound esoteric at first it is a very helpful operation. So first of all you know there is the map operation which is extremely useful in a sense like this.

"hello world" map { _.toUpper }

So what is the deal with flatMap? It would not work in this case and the compiler would raise a type error. That is because a collection is expected as the result.

flatMap makes a lot of sense when you have a collection and need to pass every element into a function that would then again return a collection.

val list = List(1,2,3,4)

def intToListOfChars(i: Int) = 
  i.toString.toList

val listOfListOfChar: List[List[Char]] =
  list map intToListOfChars

As you can see calling map with intToListOfChars creates a list of lists and we could later flatten that list agin. Or we simply use flatMap which does all of this in one step.

val listOfChars = list flatMap intToListOfChars

Still this example is pretty awkward. But imagine you have a method like trimToOption(value: String): Option[String] which trims a string and returns None if the string is empty or null (for whatever reason).

If you write a webapplication which expects user input you are probably working with a lot of Option[T] values (or Box[T] in case of Lift). For example like this:

val description: Option[String] = 
  extractDescription(request)

Now you also want to trim that description. Here are a couple of ways on how to do it using trimToOption.

// not so nice
val trimmed1 = trimToOption(description getOrElse "")

// ugly as hell
val trimmed2 = description match {
  case Some(desc) => trimToOption(desc)
  case None => None
}

// kind of ugly
val trimmed3 = (description map trimToOption).flatten

// yey! :)
val trimmed4 = description flatMap trimToOption 

So you see that flatMap is actually a pretty usable function.

Passing a Tuple to a Function

Sometimes you have to pass a tuple to a function and instead of the tuple you would like to pass each value of the tuple as a parameter.

val listOfStrings =
  List("Hello", "World")

val withIndex = listOfStrings.zipWithIndex

println(withIndex) //List((Hello,0), (World,1)))

So now imagine you have a method like this:

def printStringWithIndex(value: String, index: Int) =
  println(index+": "+value)

In this case withIndex foreach printStringWithIndex would not work because we pass a tuple to that method. Scala has a nice built-in method called tupled which takes a function and converts it to one accepting a tuple.

withIndex foreach (printStringWithIndex _).tupled

Using the Compiler

Imagine you have a class with a type parameter like CustomList[T] and now you want to have a method which is implemented only for some kind of T. Obviously one way would be to use traits and mix them in on demand. But it gets more and more complex.

Fortunately we can let the compiler proof the evidence of a type at a position in our code.

class Foo[T](bar: T) {
  // only allow this method if T is a String
  def length()(implicit proof: T =:= String) = 
    bar.length
}

val foo1 = new Foo(123) //works just fine
foo1.length //does not compile.

val foo2 = new Foo("hello")
foo2.length //5

This is great because it allows us to introduce phantom types and to write less code for certain structures. For instance we have a Connector[A] in Audiotool. This connector accepts connections to other connectors. It is either an input or an output and it transmits an abstract signal. You can only connect an output with an input. A Connector accepts only connections of its own type and virtual connections. This means the following example would be uber-cool code. Unfortunately we did not write Audiotool in Scala ;)

class Connector[A] {
  add[B](conn: Conn[A, B])(implicit proof: A =:= B)
    { ... }

  add(connection: VirtualConn[_, _])
    { ... }
}

Note: This is in fact not a real example since it would not work. You could only connect an output with another output but I think you might get the idea and it is easier to follow.

Conclusion

Functional programming might seem weird. Constructs like =:= or flatMap are awkward at first but make complete sense. Using Scala and picking some of the functional idioms which you think are right and well understood in your team can increase productivity and lead to less bugs.

Go comes with a different concepts for synchronization. Channels are a construct baked into the language. They allow you to communicate between different go-routines etc. Basically I stopped reading after I came across the Channels section.

A channel allows you to send and receive data from it. Much like a Scala actor. By default a channel is performing blocking send and receive operations. Sounds pretty much like a blocking queue to me. The syntax is something like the following.

// Create a channel of int
var channel = make(chan int) 

func send() {
  // Send 1 to the channel.
  // This operation blocks ...
  channel <- 1
}

func receive() {
  // Receive from channel
  // This operation also blocks!
  message := <- channel 
  fmt.Println(message) //1
}

go send()
go receive()

So ultimately I thought it should be quite easy to achieve something similar using Scala. I think this is an excellent example of the scalable part when it comes to Scala.
Go channels are diveded into reciver and sender channels, with a bidirectional channel type. The same can be achieved using Scala traits very easily. I could also add a couple of nice syntactic sugar like adding a foreach method to a channel which allows you to use it with Scala for comprehensions.

Have a look at this Gist for the complete source code. First of all one part is a little bit unfortunate. You are not able to define <- or <~ as a unary operator in Scala. So I thought I stick with the ! operator which you already know from actors and it is the analog to Go’s <-.

At the end of the file you will find a re-implementation of the Go Producer-Consumer example.

Something really cool is the fact that you can use Scala’s for-comprehensions which automatically exit once a channel is being closed. I think Go allows you do the same by specifying range when iterating over a channel. However I am not sure since I only had a brief look.

So is this useful? I do not know since I would probably stick with Scala actors for most of my multi-threading needs but it is still a great example of Scala’s flexibility as a language.

Lift is very nice because of its excellent Ajax and Comet support. My main problem getting started with Lift was the lack of documentation and real world examples. E.g. if you want a custom 404 page without a redirect your only source of information is a thread in the mailing list. Lift also comes with its own terminology and a lot of custom DSLs.

Cricitcs aside: I am absolutely happy that Lift exists. You simply have to start using it. The documentation has also improved a lot. Simply Lift and Exploring Lift are available for free and the Wiki contains a lot of useful information too.

I want to give you a simple example of why I think that Lift is simply awesome and what sold me to the framework. You probably know those endless scrolling pages.

Let’s see how to develop something similar with Lift and Scala. First of all we have to prepare a little template.

<div class="lift:surround?with=default;at=content">
  <div class="lift:endless">
    <ul>
      <li>item0</li>
      <li>item1</li>
      <li>item2</li>
    </ul>
  </div>
</div>

That is all the HTML we need. In fact we need even less but if you are going for designer friendly templates you could extend it with your usual page chrome.

There are two calls to Scala code in this template. lift:surround will trigger the SurroundSnippet and insert the current template into the default template at the content position.

Our duty is now the implementation of lift:endless. Lift comes with a dynamic convention-over-configuration lookup for the so-called snippets. I am not a fan of COC so I will stick to the manual mechanism of binding endless to a snippet. Lift offers the DispatchSnippet in this case.

You will have to add this line to Boot.scala:

LiftRules.snippetDispatch append Map("endless" -> EndlessSnippet)

It tells Lift to invoke the dispatch method of the EndlessSnippet object whenever a snippet named endless is encountered.

So the only missing piece is the implementationof the EndlessSnippet and this is where Lift really shines.

import net.liftweb.common._
import net.liftweb.util._
import net.liftweb.http._
import net.liftweb.util.Helpers._
import net.liftweb.http.js.jquery.JqJsCmds
import net.liftweb.http.js.JsCmds
import net.liftweb.http.js.JE
import scala.xml.{NodeSeq, Unparsed}

object EndlessSnippet extends DispatchSnippet with Loggable {
  def dispatch = {
    // If you want to execute render only when "embed.xyz" is called
    // you would match on "xyz" here.
    case _ => render 
  }

  def render =  {
    // Generate a random identifier
    val containerId = nextFuncName

    // The page which we are currently looking at.
    var page = 0

    // Method to be called by JavaScript.
    def process(in: Any) = in match {
      case id: String if id == containerId =>
        page += 1
        logger.debug("Generating page "+page+".")

        // Generate new markup and append it to the list.
        JqJsCmds.AppendHtml(id, getItems(page))
      case _ =>
        logger.error("Illegal data has been sent.")
        JsCmds.Noop
    }

    // Here we marry the template with our code.
    // "*" is a CSS selector that matches anything and #> binds it to the content
    // we specify.
    // The most interesting part here is the JavaScript  which we bind to the process
    // closure. Lift does all the magic for us and calls the server-side method once the
    // user scrolls to the bottom of the page.
    "*" #> (<ul id={containerId}>{getItems(page)}</ul> ++ JsCmds.Script(JE.JsRaw(
"""$(window).scroll(function(){
  if($(window).scrollTop() == $(document).height() - $(window).height()) {"""+
    SHtml.jsonCall(containerId, process)._2.toJsCmd+"""
  }
})""").cmd))
  }

  def getItems(offset: Int): NodeSeq = {
    val count = 100
    val start = offset * count
    val end = from + count

    // We generate a couple of <li> elements here.
    for(i <- start until end) yield {
      <li>{"item"+i}</li>
    }
  }
}

This little amount of code really shows the power of Lift. Simply write a closure and bind it to your JavaScript. Lift does all the plumbing for you. Then you only have to implement some logic on the server. In most cases you do not even have to write any JavaScript since Lift offers Ajax links, forms etc. already. SHtml.a(...) generates a link which will trigger a function on the server for instance.

But please beware: I am not sure if SHtml.jsonCall is actually the way how you should do this. There might be a better way of calling a server-side method via JavaScript.

There is of course more to Lift. Comet is where things get really cool and the record framework seems interesting as well. Hopefully this little excursion into Scala and Lift could quicken your appetite. Since I am usually not a frontend fan this made it interesting and fun again.

I will try to post a complete project in the next days as well. Before starting with the Audiotool website I wrote some test cases that might help other people getting started.

Wikipedia defines STM as:

In computer science, software transactional memory (STM) is a concurrency control mechanism analogous to database transactions for controlling access to shared memory in concurrent computing.

First of all when reading this you might think that an STM is not necessary since ActionScript does not support true concurrency and also the shared-nothing worker proposal will not allow you concurrent memory access. While this holds true you may not forget that concurrency becomes an issue when you have just access to a single core but on multiple machines.

A lesser known fact about Audiotool is also that we have a built-in system for real time collaboration since its beginning. Due to conceptional issues and other prioritized features we were never able to roll it out as a full feature but the technology base is there.

When it comes to multiplayer gaming and network latency you will always have to figure out a system which allows you to share a certain state across multiple machines. In some games the state is synchronized across a certain server. An example is an FPS like Quake3. The Audiotool is unlike Quake3 not a FPS and different conditions have to be met. And this is the reason why I have choosen to implement an STM with very optimistic concurrency control.

The basic idea of my system is to use atomic transactions with commit and revert operations. Those are handled by a transaction manager which performs the basic tasks of an STM. That means: execute all transactions until an error occurs, roll back and try again. Each transaction in the Audiotool contains a guard. The guard defines whether or not one should still execute the transaction.

Here is an example of why a guard is important and how all this stuff works. You have two Audiotool instances running that share the same state. We call them A and B. A and B are connected via P2P no an arbitrary network. This means exchanging messages introduces latency and we cannot guarantee that A is at the same state of B.

So what happens? t₀ is committed by A, serialized and sent to B who is committing the same transaction. The same applies to t₁. t₂ is initiated by B, serialized and sent to A. Everything works fine until B commits t₄. While t₄ is being transferred to A, A created t₅ and committed it. This means A never saw t₄ when it committed t₅ and B would continue with t₆ ignoring t₅. A will actually receive t₄ after he commited t₅ so the system will revert t₅, commit t₄, commit t₅ if the guard does not prevent it, and both continue at t₆.

To explain the guard: Imagine the t₅ transaction includes modifications on a device in the Audiotool that is deleted by t₄. In that case t₅ will never be executed.

So what does this imply? You can guess that each modification of the Audiotool state is in fact handled by a transaction and we carefully have to pick the guards. E.g. if an editor is referenced by a transaction it is part of the guard. However this introduces a second issue: references. Sharing references of objects across a network is a tricky task. I am using “boxes” to solve this issue. A box can hold a value and it is guaranteed that it will hold the same value at the same state across the network. A box can be serialized and will be shared between commit and revert states. This means if t_n creates an object which is referenced by t_n+1 both will use the same box. This also means that I can safely revert t_n+1 and then t_n and commit them without having to worry that both still reference the same object. And this even works across the network. And in fact we can now also identify whether or not one transaction would conflict with a different one which is a very important condition for the actual implementation.

Right now you might have figured out that a transaction in the Audiotool has the following properties:

• It is serializable so it can be transferred across the network.
• All object references must be stored via “boxes”.
• A guard is used to determine whether or not to commit a transaction.
• Transactions can be committed and reverted.

A lot of stuff one has to reason about. But although we do not support live collaboration at the moment we are using the STM. Why? The reason is quite simple. The STM gives us history for free. Since everything is serializable via a very lightweight protocol we can take the last 500 transactions for instance and store them as a compressed ByteArray in memory instead of each as a single object with all its references.

Another very nice option is testing. A serializable history means that one simply has to press a super secret shortcut to dump the whole history and send it to the responsible developer who can reason about every single step that happened which ultimately led to the error. In development mode I usually commit, revert and commit each transaction. Then I also serialize and deserialize them and do the whole thing again.

Since transactions are known by the system we can also write a fuzzer that executes lots of crazy stuff. Sort of Android’s monkey mode. And ultimately I personally hope that we will get to the point where we will enable live collaboration in the Audiotool of course.

The last feature I implemented is the automatic transformation of arbitrary transactions to a compound transaction. You can guess what this feature actually means for the user :). In fact it is very complex and I implemented it with the STM using only a couple lines of code (of course that is just half the story …).

If you are interested in writing your own multi user application I think this is a very nice approach since collisions do not happen very often from my experience and the optimistic concurrency in a P2P environment achieves great results in terms of overall latency and feeling. The Google Wave protocol and their operational transform is also an interesting read.

It is very easy to understand an implicit conversion since most programming languages do this already for you. Here are two examples (AS3/Java):

trace("Implicitly converted: "+1)
System.out.println("Implicitly converted: "+1);

We only have to care about the expression "Implicitly converted: "+1 which will result in “Implicitly converted: 1″. Why does this happen? We have a string on the left hand side of the expression and an integer on the right hand side. When the compiler of your choice needs to emit code for this kind of expression it will widen the type of both operands. In this case the resulting type would be String since an implicit conversion from integer to String exists in both ActionScript and Java. If no such conversion exists you would probably get a compile time error.

The compiler inserts this conversion for you. In this example we can make use of an explicit conversion:

trace("Explicitly converted: "+1.toString())
System.out.println("Explicitly converted: "+Integer.toString(1));

Here we specify that we really want to concatenate two strings which is of course a tedious task. We are lucky to have implicit conversions. Whenever you want to go from type A to B a compiler either inserts an implicit conversion for you or tells you that it cannot do it with an error message.

In certain scenarios you might want to use implicit conversions for your own types. In ActionScript I might want to go from Sprite directly to BitmapData.

Here is an example. Please do not do this at home. There is a of course a much better way (and less memory intensive) to do a hit test on a Sprite.

function hitTest(bitmapData: BitmapData, x: int, y: int): Boolean {
  return bitmapData.getPixel32(x, y) != 0x00000000
}

const sprite: Sprite = new Sprite()
hitTest(sprite, 1, 2)

Now this code would not compile of course since hitTest(sprite, 1, 2) is incorrect. The compiler would tell you that it expects a BitmapData as the first parameter. Let us fix the example:

function hitTest(bitmapData: BitmapData, x: int, y: int): Boolean {
  return bitmapData.getPixel32(x, y) != 0x00000000
}

function toBitmapData(displayObject: DisplayObject): BitmapData {
  const result: BitmapData = new BitmapData(
      displayObject.transform.pixelBounds.width,
      displayObject.transform.pixelBounds.height)

  const matrix: Matrix = displayObject.transform.matrix

  matrix.tx = matrix.ty = 0.0
  result.draw(sprite, matrix)

  return result
}

const sprite: Sprite = new Sprite()
hitTest(toBitmapData(sprite), 1, 2)

Now it works but every time we would want to go from Sprite to BitmapData we have to insert our new toBitmapData function.

And here is what makes Scala so special. You can tell the compiler that toBitmapData may be used as an implicit conversion.

Here is the same example in Scala without making use of implicit conversions:

def hitTest(bitmapData: BitmapData, x: Int, y: Int) =
  bitmapData.getPixel32(x, y) != 0x00000000L

def toBitmapData(displayObject: DisplayObject) = {
  val result = new BitmapData(
      displayObject.transform.pixelBounds.width,
      displayObject.transform.pixelBounds.height)
  val matrix = displayObject.transform.matrix

  matrix.tx = 0.0
  matrix.ty = 0.0

  result.draw(sprite, matrix)
  result
}

val sprite = new Sprite()
hitTest(toBitmapData(sprite), 1, 2)

Not so different. However we can annotate the toBitmapData definition as implicit and the code would look like this:

def hitTest(bitmapData: BitmapData, x: Int, y: Int) = ...

implicit def toBitmapData(displayObject: DisplayObject) =
  ...

val sprite = new Sprite()
hitTest(sprite, 1, 2)

So I think now it is time to take a deep breath. If you think that this is dangerous you should keep two more things in mind. First of all the compiler will only use implicit conversions that are in the current scope of your method. This means if you define an implcit conversion in class A and want to use it in B you have to import it. Also it could happen that you have two implicit conversions in the same scope. Since the compiler does not know which one it should pick it will give you an error message. This makes a lot of sense.

So in the end it is a great way to enable a specific implicit conversion for parts of your code. You are also able to use an import statement only in a method body like this:

def myComplexMethod(a: A) = {
  import ImplicitConversions.fromAtoB
  ...
}

But this is just the one part about implicit conversions. In my A Year Of Scala post I did mention that implicit conversions can be used to build domain specific languages.

Right now we get rid of our explicit casts which is nice but a true DSL needs obviously much more. For instance I might want to say 0xff00ff.toHexString instead of toHexString(0xff00ff). This can also be achieved with implicit conversions. And this is not what you might already know from Java or ActionScript.

Whenever we invoke a method on an object, and the object’s type does not support that method, but an implicit conversion to a different type containing the method exists, it is applied. So remember the Sprite and BitmapData example?

This would be illegal in ActionScript:
sprite.getPixel(x, y)

This would be legal:
toBitmapData(sprite).getPixel(x, y)

And with an implicit toBitmapData method in Scala we get:
sprite.getPixel(x, y)

In fact this is quite simple. So in case you did not notice: this is also statically typed monkey patching. And because you can omit dots and parenthesis we get code like G("vertex1" --> "vertex2"). Where is the implicit conversion? A string has no --> method defined but we could define an implicit conversion to a class VertexPair that has the --> method defined.

Last but not least there are also implicit parameters and values in Scala. In fact only an implicit value may be used as an implicit parameter.

If you are always carrying the same parameter around you, why not define it as implicit? An OpenGL context for instance. render(float t, GL11 gl) is some kind of code that might occur very often. Let us make the gl parameter implicit.

def render(t: Float)(implicit gl: GL11) = ...

Now if you do not have an implicit value available, you can still call this method via render(1.0f)(gl). But if you declare a GL11 object once as implicit it can be passed down the chain very easy.


implicit val gl = framework.getOpenGLContext()
render(1.0f)
render(1.5f)


Now if you call a method inside render that needs an implicit GL11 parameter as well you would not have to declare it as implicit again. But the same rules apply here. If there are two implicit GL11 typed objects available the compiler will complain.

I hope this makes working with implicit conversions and values a little bit easier.

I have been trying to spread the word about Scala during various Flash conferences and always tried convincing my colleagues and friends to learn this great language. A blog post by Alan Shaw about his experience with Haskell inspired me to write about my journey.

Why Scala

I was very unhappy with ActionScript and Java last year. ActionScript is a good start but needs to adapt the features of the dropped ECMAScript 4 proposal to become interesting and competitive in my humble opinion. Java on the other hand is only interesting to me because of its virtual machine. The language itself is simple and has the features you need to get the job done.

I got very late into the Java game and missed the bumpy early years. However I was never earning money with Java since I was doing only open source.

Open source is a hobby. I like doing it. So why not use a language that is fun too? I finished the first version of Apparat in 2009 using Java. But the language became more and more annoying because of all the boilerplate code you have to write and the massive violation of the DRY mantra. A particular example are Java’s generics. I was dealing very often with graphs in Apparat. I had a parameterized graph class Graph<V, E> and of course sometimes the vertices and edges are parameterized too. I was tired of writing code like this over and over again.

final Graph<BasicBlockVertex<AbstractOp>, Edge<BasicBlockVertex<AbstractOp>>> graph = new BasicBlockGraph<BasicBlockVertex<AbstractOp>, Edge<BasicBlockVertex<AbstractOp>>>();

So I was searching for a new language that allows me to focus on ideas. I could now talk about languages like Lisp, Clojure, OCaml, Haskell and C#; but I will just explain why I did not use C# — a language that is superior to most other languages. If Scala would not exist, I would probably do more C# work.

I do not think that interoperability is an issue with C# since Mono is a great project that gives you Linux and OS X support. But I used C# already for Apparat’s predecessor AS3C. So why not learn something new? This is why I went for Scala . I thought I can learn more about programming if I start using it — and boy was I right: I am learning something new each day since then.

Learning Scala

Most people that see Scala for the first time are immediately irritated by the syntax they do not understand. Same happened for me. When I first saw Scala I just gave it five minutes and went on to something else. The second time I came back to Scala I spent more time on the examples and tried understanding them.

Here is the problem: learning Scala is hard. My main issue was not the syntax but the fact that most people simply slap you with a lot of academic topics right in the face. As an autodidact you have to read a lot before you will write your first line of code if you follow that path. If you studied computer science and took a Haskell or Lisp course you will probably know about algebraic data types and monads already.

Instead I started ignoring most of the cool features and tried understanding the syntax first by writing actual code. You will find a lot of high quality blog posts on Scala very easy. When it comes to understanding Scala from an ActionScript perspective it is a little bit easier since you know anonymous functions already. Let us demistify a little bit of the syntax.

Take this ActionScript example.

const square: Function = function(x: int): int { return x * x }

You can remove nearly all type annotations because of Scala’s type inference. You still have to annotate method arguments. The compiler would complain if you leave those types out. This is a trade-off for readable error messages and compiler speed.

const square = function(x: int) { return x * x }

Next you can get rid of the return keyword in Scala. Because it is a functional language every usual statement is an expression. val x = if(true) 1 else 2 has a value for instance and is equivalent to var x: int = true ? 1 : 2. What does val mean? It is the const keyword in Scala. I prefer it this way. You have variables and values.

val square = function(x: int) { x * x }

This looks already much better in my opinion. And here comes the catch. Scala is a functional language. It is also an object oriented language but more on that later. But it borrows a lot of the nice features from other languages like OCaml or Haskell.

Because you are working with functions so often there are multiple short-hand notations available. But the key is to drop the function keyword and replace it with the shorter =>. Last but not least we do not even need the curly braces in Scala and our example finally becomes

val square = (x: Int) => x * x

You will figure out that => is used more often. This is another catch. Two particular things are confusing for the beginner. => and _. => is used in multiple places like definition of anonymous functions, types, scope definitions and self type annotations. I will probably not talk about the other two in this post.

It is worth noting that Int => Int is the type of a function that takes one parameter typed Int and returns a result typed Int.

So Imagine you have this definition:

val applyIntFunction = (x: Int, f: Int => Int) => f(x)

This anonymous function takes two parameters. One is typed Int and another one is a function. So we could write applyIntFunction(2, square). But what if we want to increase x just by one? One option would be applyIntFunction(2, x => x + 1). You can see here that Scala knows the type of my anonymous function already and I am allowed to omit it. This is not too bad. But since it is so common in functional programming to work with functions, an even shorter syntax exists. Meet the underscore.

applyIntFunction(2, _ + 1)

This is where it gets confusing at first. However you will love this feature and it is not so hard to understand. The Scala compiler expects the type Int => Int as the second parameter. _ + 1 is in fact an anonymous method that takes one parameter. But we do not know anything about the type if we look at the method alone. It is always important to think about the context. _ * _ is a method that takes two parameters for instance. This would not work in our example. What does _ * _ do? Probably multiply two things. But again it is a question of the context and the required types.


val applyTwoIntFunction = (x: Int, y: Int, f: (Int, Int) => Int) => f(x, y)
applyTwoIntFunction(2, 2, _ * _)


One of my main problems with Scala was the usage of the _ character and how it works. Fortunately a great post I want to highlight again already exists. It explains very well which purpose the underscore fulfills in which context.

So even if you think that Scala is hard to understand at first keep in mind that it is just a different dialect of expressing your ideas. If you have less to write you can do that not only faster but more concise. Once you start thinking in Scala it becomes of course absolutely clear that _ * _ means multiply those two things whatever they are.

A Language For Hackers

Scala is a great language for hackers. Four features make it very easy to write any library that feels like real code. Namely anonymous functions, the ability to omit braces and dots, type parameters and implicit conversions. Okay, what does it mean? I already talked about anonymous functions but omitting braces and dots? In Scala the code 1.plus(2) is identical to 1 plus 2. Whenever you have a method with exactly one parameter and an object on the left hand side you are allowed to omit both braces and the dot. First of all this is quite cool but also I did the mistake to make extensive use of this feature. In the beginning of Apparat I was omitting nearly all dots just because I can. It does not make your code more readable and I inserted dots and braces back into the code.

So when does it help? When it makes sense. This sounds weird but lets think about another example. Lists are a key element of Scala and any other functional language. "Hello World".toList returns a list of characters. If we want to have all of them as lower case we would map that list. Even ActionScript has a predefined method for this task, called Array.map. Noone uses it. The explanation is that it is simply to complicated. Imagine we have a List[Char] in Scala or an Array in ActionScript and we want to convert it to an Array where all characters are lower case.

This is the ActionScript version:

array.map(function(x: String): String { return x.toLowerCase() }).

This is the Scala version:

list.map(x => x.toLowerCase())

But lets keep in mind what we have learned and simplify it a little bit:

list.map(_.toLowerCase())

Remember what I said about omitting dots and braces?

list map { _.toLowerCase }

Et voilà. You can decide which one is better to read. The Scala version is short and nice. However if I would simply give you this snippet of code you do not know what it does as long as you do not know about the context. But we are dealing with language and context all day. Words have different meaning in a different context and it is not hard to guess that toLowerCase probably converts a character to its lower case representation.

Please note two more things. I left out the () after toLowerCase because it is a method without side effects. This is the case whenever your method does not modify any state in your system. Those methods are usually written in ActionScript like public function get toLowerCase(): String { ... }. And we omit the () as well.

One more example to convince you why this makes sense: Imagine we want to get rid of all the space characters in our now lower-case List (or Array in ActionScript).

array.map(function(x: String): String { return x.toLowerCase() }).filter(function(x: String): Boolean { return x != ' '})

vs.

list map { _.toLowerCase } filterNot { _ == ' ' }

This also explains why Scala is a great language for hackers. You can bend the language to your needs. Usually those hacks are categorized as domain specific languages. A language for a specific task. I have written a CSS DSL using Scala for instance.

In order to do such things you will need implicit conversions. A very cool thing that is out of scope for this article (Update: I wrote a post on implicits).

Conclusion

Scala can do much more than you might expect. I just covered a small part of it. For me it is important to explain why I like it so much and why I believe that it is an elegant language that makes programming fun again.

After one year of extensive Scala use I do not regret a single minute spending confused in front of my screen. First of all I like that someone spent a good amount of time thinking about the language. It is simply great that you can distinguish between a class, something you want to instantiate multiple times, and an object, Scala’s notion of a singleton, at compile time. No need for private constructors and the other boilerplate fuzz.

The Scala community is filled with very intelligent and academic people who are not arrogant and always treat you with respect.

Speaking for myself I can only agree that Scala makes you more productive and it was a great decision to start learning it. Scala is also not a pure functional language. In my opinion a great decision although not everyone will agree. But you can always decide whether or not you want to write a loop using a simple while or to go for tail recursive functions with immutable values. It is up to you.

I started to use Scala also at work as well. It is very nice for smaller projects like determining the similarity between users and storing that in a database. Currently I am the only one writing Scala in our office so most projects are still either Java or ActionScript based.

If you are into speedcoding: Scala is your weapon of choice. The one time I was using it on stage I was finished after about 75% of the time which is great if you target ten minutes but then my IDE crashed. Tooling is a different topic but has greatly improved in the last couple of months. If you want to learn Scala why not simply start with an old experiment you wrote in Processing and convert that to Scala? Since Processing is written in Java you can call its methods from Scala without any problem. I did not mention it yet but since both languages run on the JVM you have full interoperability between them.

Apparat is also a great example of marrying Scala with Java. JITB is for instance written in Java and Scala. I have choosen to go for Java because I am working with Java Bytecode in Apparat and at that point the Scala compiler was not very stable regarding the bytecode it emitted. This does not mean it was broken but it changed during versions which would have made my job much harder.

However even if you will not use Scala on a daily basis it will make you a much better programmer. This is also true for any other functional language. I used immutable lists in my as3-signals fork to simplify the whole dispatch logic and was able to delete a lot of unnecessary code.

Regarding Apparat, in the end I got rid of writing

final Graph<BasicBlockVertex<AbstractOp>, Edge<BasicBlockVertex<AbstractOp>>> graph = new BasicBlockGraph<BasicBlockVertex<AbstractOp>, Edge<BasicBlockVertex<AbstractOp>>>();

Which became

val graph = BasicBlockGraph.empty[BasicBlockVertex[AbstractOp], Edge[BasicBlockVertex[AbstractOp]]]

Oh wait, it did not. In fact it is now

def createGraph() = BasicBlockGraph.empty[BasicBlockVertex[AbstractOp], Edge[BasicBlockVertex[AbstractOp]]]
...
val graph = createGraph()

What does def mean? Go figure it out and enojy learning Scala. You will not regret it!

This year I had the honor to be part of the opener show for Adobe MAX 2010. I really appreciate that the people at Adobe working on the keynote took the risk to try this out.

Mike Chambers asked me already in April this year if I would like to do it. He had the idea of Erik Natzke generating art, me doing live coding and a DJ playing music. Eric Clark has been a fantastic supporter of Audiotool so I thought it would be really nice to have him do it. Luckily he agreed.

We had several meetings and finalized the idea of the performance. We really wanted to connect the work from everyone of us and create one piece. Otherwise it would have been only three persons doing their stuff. But eventually we got it to a point were I was visualizing the music Eric Clark was playing using pictures that Erik Natzke has been drawing on stage.

I as a developer always want to know how things work. So I also want to explain the technological aspect and challenge of such a performance.

Eric D. Clark was using Audiotool on a touch screen and we really wanted to show how Audiotool can shine there. The idea was of course to support gestures one might intuitively expect. This was done in Germany already. We are using a custom built event handling system so I had to add gesture support there. We do currently support only zooming the desktop since the schedule was so tight. In fact the week before going to MAX we also launched a collaboration of Audiotool and Coca-Cola. Everything was a little bit crazy.

However not as crazy as it went when we came to LA. When doing livecoding I usually prepare my final result already so I know what I want to create. You know where the little problems are and you can make the most out of the time you have. But because of the launch I did not have that much time to prepare. So I wrote one demo for me already in Germany. It was also my first time using the new 3D capabilities I was allowed to use which will be available as a public beta in 2011. Building software with a beta version of something is always fun because it can break for unknown reasons etc.

I also wrote a server that streams arbitrary recording devices into Flash at a very low latency. This was very important of course since I needed to access what Eric Clark was outputting. I did a version that does FFT and normal amplitudes but went only with the amplitudes since an FFT terrain looks not as interesting. It can. But then you have to do more analysis.
On the flight to LA I build another program that would help me save some time during the performance. It is basically automatically compiling and launching a Flash Player. A lot of people asked me about this actually. But you can built it for yourself very easy or I can even release the source code. It is really not much to do. I also contributed this code to Flexmojos as the continuous compilation goal. In fact this was my first idea. But since I did not have internet access most of the time and I was developing this on my tiny netbook I thought I want to have something as slik as possible.
Then there was another server involved I quickly wrote for Erik Natzkes machine. That one was also nothing special. It simply watches a folder and serves always the latest file. That way Erik only had to save an image to a certain folder with AIR and I would get it.
Last but not least we also used FlashMate. My old companion when it comes to multi monitor development. FlashMate hooks into the window procedure of the Flash Player and simply repositions the window always where you left it.

Because of the tools I built I never had to worry about starting the compiler, moving the Flash Player window around etc.

The program I wrote was actually quite simple. It was a subdivided plane in 3D space and I was drawing the audio feed I got from Eric Clark onto a BitmapData. That BitmapData was used as a heightmap for the vertices of the plane. In fact each pixel was assigned to exactly one vertex. That way I could get a really nice and smooth terrain. Thanks to Molehill this was rendering even at 1920×1080 without any hickups.
The heightmap was also used as a lightmap to fake some kind of ambient occlusion. I know that this was rather cheap. But keep in mind that we also had a limited amount of time. Even worse: We had some cue points in the performance. Like when Serge Jespers came on stage and was giving me the audio cable. At that point everyone was expecting me to be ready to visualize the amplitudes. Otherwise the whole thing would not have made any sense.

However we also wanted to show even more at the end because it would have been boring to see only the terrain flight the whole time. That is the reason why I flipped the terrain and started to rotate it. We agreed to this on Saturday, two days before the keynote. But luckily this is a rather cheap copy and paste operation. You take the code you already have and rotate the terrain by 180° around the z-axis. The rest of the animation was really only rotating everything around the y- and z-axis.
During the keynote we were also on IFB and at one point we were told that we have to stretch for five more minutes. That is the reason why the terrain was bouncing, freezing and whatever at the end.

I really enjoyed this whole experience. I love to have a look behind the scenes and it was an absolut pleasure to work together with everyone at the keynote. Friendly people all over the place. And a lot of hackers too.
However it was stressful. It is really not easy to type and concentrate in front of such a big audience. And it is not like a Flash event where you know most people anyways. All of a sudden there is the international press and all the executives sitting front row.

But I really liked the whole thing. Especially the last five minutes freestyle coding. I also want to thank Erik and Eric. The artwork matched the music very well. And I love the idea of us going on stage, each with a blank canvas on its own, creating something in a limited amount of time and having it connect as one piece.

Also worth mentioning: JITB ships now by default with Apparat. What does it mean? Quadruple-rainbow in your face. You can now run your SWF files with JITB if you follow the five golden steps to pure happiness:

1. Download and install Scala 2.8.0
2. Type scala in a terminal, then type println(System getProperty "java.library.path" split java.io.File.pathSeparatorChar mkString "\n") and remember one of the folders
3. Download LWJGL 2.5 and extract somewhere
4. Copy the native libraries which are located in lwjgl-2.5/native/YOUR OS/ into one of the folders that the Scala command showed you
5. Open a terminal, go into the Apparat installation directory and type ./jitb some.swf to launch JITB

Now here is the problem. JITB is not complete and so it will probably fail with your SWF — In fact I can nearly gaurantee you that it will not launch your SWF. I included JITB basically because it already allows you to use GLSL shaders very easy and I know a lot of people would like to play around with that.

However I have created an archive of all files I showed at FOTB besides the files that came from David’s laptop since I do not have them on my machine. You might be interested in checking out Example12.as which is the one that uses GLSL shaders. If you want to compile your own version you will need to link apparat-ersatz.swc as an external library since it contains the ContinuousFileLoader and GLSLShader class. Happy hacking!

• Download the JITB Examples
• Apparat on GoogleCode

Update: Also read Ray tracing with Pixel Bender using JITB by David Laenarts for details on the FOTB demos.

A part of the Flash Player is the ActionScript Virtual Machine (AVM) which is responsible for executing the code that is bundeled in your SWF file. This piece of software called Tamarin is open source and released under the MPL/GPL/LGPL Triple License.

Being able to look at the heart of the Flash Player does reveal a lot to the interesting reader. However such a virtual machine is a complex piece of software and you can be lost without any guidance and will maybe not even be able to compile it unless you are familiar with such systems. Jean-Philippe Auclair’s blog contains a lot of interesting Tamarin and Flash Player releated articles if you want to know more about the VM. Of course you can also read about MMgc — the garbage collection — directly at the Mozilla developer network.

This article is titled “Opening The Blackbox”. Although Tamarin might be some kind of blackbox to most of us the source code is available. We can dig into it and understand how it works. The Flash Player however is not completely open source and we do not know about its implementation details. Unless Adobe is opening up the source code we will not know which exact algorithm is used for BitmapData.noise for instance.

Fills an image with pixels representing random noise.

This quote taken from the documentation does not tell you much about how to calculate this noise. In this situation I would create some algorithm that fills a BitmapData also with random noise. But until we know which algorithm is used to calculate that noise it will never be the exact same noise as the one of the Flash Player. For now we will have to live with it. There is no (legal) way to figure out the algorithm.
Of course there are also other classes like ByteArray which have methods like readByte that do not need any extra explanation. But ByteArray also has some implementation specific details. How does it grow when you exceed its length for instance? Fortunately ByteArray is something you can find in the Tamarin source code. JITB uses a different algorithm however which you can find here.

But what else do we got? There are some funny things you will notice. Usually Sprite(x) is an explicit cast. This is true for any statement like this. However Number(x) is a conversion with specific details how a value is converted. This is also true for some other expressions like Boolean().

The playerglobal.swc contains all classes the Flash Player knows about. It should and it does with some minor differences. Of course it could be a bug on my end but some interfaces are missing. And they are suspicious since they cause/have special behaviour. Have a look at the AbcFrontend code. The Synthetic object contains part of the AST that is synthesized since those interfaces are missing in playerglobal.swc.

A lot of the Flash Player implementation details have made it into the DisplayObject and its subclasses. You cannot extend from DisplayObject and you cannot implement IBitmapDataDrawable. Even more interesting is the amount of classes that you cannot really use. flash.geom.Transform is such a class. How is concatenatedMatrix working if you would create your own instance of the Transform object?

Of course somewhere in the Flash Player will be some code that creates a Transform object. This will hold a reference to the original DisplayObject and concat the matrices up to the stage if you call concatenatedMatrix. The JITB implementation adds some special methods to the Transform class in such cases. Those methods are always prefixed with JITB$ if they are public. Another interesting fact is that you can modify the appearance of a DisplayObject by setting the matrix of the Transform object. This also has to change the scale, x, y and rotation values of the DisplayObject. There are a dozen of souch cases. Graphics would be an obvious candidate to talk about. I would advise you to take a look at Claus’ blog for everything you never wanted to know about shapes.

Also something you probably never cared about is the way how URLLoader works. I do not care about the specific implementation details either. However one thing is important to keep in mind. Loaders have to work asynchronous since they do not block. This means your Flash content runs while waiting for an event like Event.COMPLETE. This is where it gets interesting since threads enter the game. This would be a simple view: ActionScript runs in one thread, the loader in a second thread. First of all we have to keep in mind that the ActionScript code runs in a loop which is basically only triggered by events like Event.ENTER_FRAME or user input. Only the code in your DocumentClass is the one that is initiated without an event. So how does this look like in pseudo-code?

while(NoErrorOccurred && Running) {
  Dispatch Event.ENTER_FRAME to each DisplayObject
  Render DisplayList
  Dispatch Event.EXIT_FRAME to each DisplayObject
  Sleep(1000 / FrameRate - Elapsed)
}

Okay but how does an Event.COMPLETE make it into this loop when it comes from a different thread in which the loader runs? If the loader would simply dispatch this event when it completes the code would not run in the main thread. This means you could end up with ActionScript code running in parallel which would cause chaos and destruction. So what do you do? The current approach in JITB is to gather all events and dispatch them at the beginning of a frame. So basically the code looks now like this:

while(NoErrorOccurred && Running) {
  Dispatch All Gathered Events
  Dispatch Event.ENTER_FRAME to each DisplayObject
  Render DisplayList
  Dispatch Event.EXIT_FRAME to each DisplayObject
  Sleep(1000 / FrameRate - Elapsed)
}

The implementation specific details can be found in EventSystem.java class. There are also some utility methods like futureDispatch to make development easier. Dynamic audio requires to dispatch an event that is filled with content. The thread in which the audio is passed to the sound card has to wait until SampleDataEvent is filled with new content. This also reveals a problem. The audio calculation has to wait for a complete ActionScript frame until new material is available. In fact such synchronization points are problematic for a task that requires very low latency. You can hear this when your ActionScript code takes very long to process the new audio and you get a stutter-noise or audible gap. Unfortunately there is nothing you can do about this problem at the moment besides keeping the cost of your ActionScript code as low as possible.

Of course everything is only my interpretation. The Flash Player might not suffer from the problems I describe here since someone might have come up with a better implementation. In fact I would doubt that the Flash Player works for dynamic audio the same way I do it currently.

The last blog post of this series will be more about how you can implement parts of the Flash Player API in Java so that JITB understands them and vice versa. Of course we can create our own playerglobal.swc as well and add new native classes like GLSLShader which I demoed at FOTB.

ActionScript 3.0 is a dynamically typed language with function closures. Furthermore it has native XML support, scope-changes, implicit getters and setters etc. The main issue however is to convert the set of bytecodes ahead of time to statically typed Java code. JITB does not know anything about what is going on since it compiles all the code ahead of time.

JITB and TAAS

JITB makes use of TAAS. A three address code intermediate language. TAAS has its own set of rules and typesystem that I defined for myself. Method overloading is supported for instance. And there is no concept of array access. Code like x = a[0] is converted to x = a.getIndex(0). In TAAS terms the name of a method is also not worth a penny since it is always using a reference based lookup.

What I love about TAAS is that it has a really small instruction set. Array access is just a method call like any other method call. No extra instructions added. This makes optimizations really slick and easy to develop. Since the code is statically typed you also always know which method is being called and you could construct a call graph and do whole program optimizations which is a huge bonus. Escape analysis is something that is now finally possible for me.

JITB uses a frontend that parses ActionScript bytecode and creates an AST which is then transformed and passed to a backend that creates Java bytecode. The class and package structure is stored in a tree form. Method bodies are always stored in a graph representation. The interesting problem is to convert ActionScript into TAAS since it is no direct mapping and some statements are not supported in the same way you get them form the ActionScript bytecode. Then the next interesting problem is to convert this code into Java bytecode which is also not a direct mapping.

An example for such a problem are implicit getters and setters. TAAS does not have a concept of getters and setters but understands methods. So whenever the ActionScript bytecode has a GetProperty instruction I try to figure out if this is a field or not. If it is a field I generate a TLoad instruction, if it is an implicit getter it will be converted to a TCall instruction. However if it cannot be determined since the class might be dynamic a TCall instruction will be generated which uses the special TGetProperty method and this will be resolved at runtime.
When converting to Java a TLoad can be easily translated to a GETFIELD operation. When it is a TCall I first lookup whether or not it is a special native function. If it is a special function like TGetProperty I insert Java code like object.JITB$getProperty(name). If it is a normal function I simply insert an invoke operation.

Closures for Java

Java 1.6 does not support closures. This means Java does not understand the concept of a reference to a function. Code like addEventListener(Event.ENTER_FRAME, onEnterFrame) is not possible with Java since onEnterFrame is a reference to the onEnterFrame function which is defined somewhere else in your code.

I took the idea from Scala and how they support closures in the JVM. Basically when code like addEventListener(Event.ENTER_FRAME, onEnterFrame) occurrs it will be translated to the TAAS instruction TCall(TReg(0), TaasMethod(addEventListener, ...), List(TString("enterFrame"), TClosure(TaasMethod(onEnterFrame, ...))), None). This basically means that the method addEventListener is called on the object in register zero with two arguments of which one a closure and it has no result which is indicated by None.

This instruction is expanded to the following Java code. Please remember that it is always Java bytecode. I think it is just easier to understand if I write the corresponding Java code here.

this.addEventListener("enterFrame", new Function1() {
  override Object call(Event arg) { callVoid(arg); return null; }
  override void callVoid(Event arg) { onEnterFrame(arg); }
});

Function1 represents a function of arity one. The AVM can call any method as either void or with an expected result. If the method has a result and you call it via CallPropVoid the resulting object is ignored. If however you have a method that does not have a result and you call it via CallProperty the null object will be pushed on the stack.

The Java code in EventDispatcher which would call closures basically now only looks like this:

void dispatchEvent(Event event) {
  for(Function1 listener : listenersFor(event.type)) 
    listener.call(event)
}

Dynamic code in Java

Since I want to target Java 1.6 I do not make use of the Java 1.7 opcodes like invokedynamic. Because TAAS is already statically typed we have only in very rare cases the need to make a dynamic lookup. This is done by some helper classes or methods. Every ActionScript based class extends from jitb.lang.Object. The jitb.lang package represents the ActionScript’s toplevel. The Object class has certain helper methods like JITB$getProperty and JITB$setProperty. Those methods throw an error by default if an object is not dynamic but they can implement the behaviour to store values in a HashMap for dynamic classes.
However currently dynamic method lookup is not supported. Code like a['method'+Math.random()]() fails since it is impossible to know upfront which method is called. But this can be supported as well. It is just very cumbersome and annoying.

Emulating the AVM in the JVM

What kind of error do you expect for this kind of code: var x: String = null; trace(x.toString());? This creates a TypeError in ActionScript but a NullPointerException in Java. Same for things like var x: A = new A(); var y: B = B(x); which would result in a Java ClassCastException if A is not assignable to B.
JITB offers an option which is by default turned on the get the exact same exceptions like the AVM would throw them. This means that a Java cast is converted to a call like AVM.cast(x, B.class) with the option turned on or the simple Java CHECKCAST bytecode when the option is turned off. Basically expecting something to be null in a try-catch block is very silly in my honest opinion. So to get rid of this overhead you can disable this extra step of correctness.

Another interesting issue is the fact thatthe AVM can throw any object. Java however expects that a throwable object extends from the Throwable class. Of course the basic jitb.lang.Object class could simply extend Throwable. However I do not want that overhead and at any time. Also I do not want to break how classes extend each other. If you do something like throw "string" it is converted to throw new Throw("string"). Throw does extend Throwable and whenever it is catched the value is simply extracted.

To box or not to box?

ActionScript allows you do use int as an object with a lot of special behaviours. Same happens for values like Boolean or Number. var x: int; trace(x); outputs 0 but not null. But it is also possible to call 0.toString() which indicates that the integer value 0 is treated like an object that has methods like toString().
In Java however you have a real int primitive and the Integer object. 0.toString() does not compile but you would rather do Integer.toString(0).

Of course Java is much faster when using primitive data types when appropriate. JITB does this since it handles primitive types like real primtives for Java. This means a method like function add(x: int, y: int): int { return x + y} is converted to int add(int x, int y) { return x + y }. However when you have a method that expects an Object type a primitive will be boxed into a compatible type.

Last but not least what happens for code like 0.toString() or other methods? A possibility would be to emit code like new jitb.lang.Number(0).toString() but that does not make sense in most of the cases. Instead those calls are treated specially with a call like jitb.lang.Number.JITB$toString(0) and we get rid of an extra allocation.

Conclusion

I hope you understand a little bit better now how JITB works internally and what the main gotchas are in such an implementaiton. The journey is not over since I have to add support for anonymous methods and although I talked about things like throw I still have to add support for try-catch. Also a lot of other basic features are still missing but I doubt that they would cause any problems.

The other side of the story is the Flash Player API inside of JITB which is maybe worth a blog post like this as well since it reveals a lot of other funny and interesting gotchas.

Step #1: Parsing PixelBender data

PixelBender compiles down to PBJ which is a two-address based bytecode format. Basically it contains instructions like this one which I have taken from Nicolas’ blog:

  sampleNearest f2, text0[f0.rg]
  sampleNearest f3, text1[f0.rg]
  mov f4, f2
  mul f4, f3
  mov f1, f4

Those PBJ files are parsed by Apparat into high level structures. This happens basically the same way I do it for ActionScript bytecode. Thanks to the work of Nicolas Cannasse this was a walk in the park.

Step #2: Optimizing PixelBender bytecode

The PixelBender compiler does not produce optimal code as noted by Nicolas. You can see it in the code I just posted. Or you can simply compile and empty shader and take a look at the disassembled output. Apparat performs only two very simple optimizations for PixelBender which are written especially for PBJ. Thos include copy propagation and dead code elimination. The compiler does create code which is not bad thanks to LLVM but there are unnecessary copies going on which are removed by Apparat.

Step #3: Loop detection

PixelBender still has no loops since its initial introduction in October 1st in 2007. Tomorrow it is October 1st in 2010. Go figure. So since there are no loops the only way people write loops in PixelBender is basically by copy and paste or using preprocessor directives. With JITB I want to send real shaders to the graphics card and I do want to make use of loops. So I wrote a PBJ bytecode based copy and paste detector which is based on ideas of the PMD copy and paste detector. When a segment is detected in a sequence I convert it to a loop. That way I can reduce the amount of data I have to send to the graphics card. This is very important since in the case of the demos I showed at FOTB they contained around 50.000 instructions. And my cheap graphics card was complaining that there are too many instrucitons. With the loop conversion this number drastically decreased.

Step #4: Converting PixelBender bytecode to GLSL

It is very easy to convert PixelBender to GLSL. The two address code format can be translated with some minor tweaks to GLSL with a very simple mapping. However this is not really optimal since you have only a two address code. When exporting to GLSL I perform one additional step which expands the two address based code to a three address based code. That way I do not emit a lot of unnecessary copies again.

That is basically the recipie to convert PixelBender PBJ files to GLSL shaders that perform well on current graphics card hardware. There are additional steps involved to get OpenGL shaders to work like PixelBender in Flash but they are only minor tweaks. I have no comparable metrics right now. However I will create a JITB applet version so you can have a look for yourself soon.