Journal | Michael Zehender

The next programming language? Ceylon and Kotlin

2013-12-13T00:00:00-08:00

There are already a lot of resources that make some comparison between ceylon, kotlin and Xs, where Xs stands for some JVM-based languages. I didn't include these in this post as I haven't spent enough time to analyze them in detail or they expose some major issues (regarding design, philosophy, readability, etc.) from my point of view.

I probably have to stress that I don't say Xs are flawed or bad, they're just not right for me or I haven't spent enough time to evaluate them.

Also I should note that this post is about my feeling to code in these languages and whether I could use them for products where a Java base already exists.

Why am I looking for a new language?

If you ask someone why they are looking for a new language, they'll probably mention something like boilerplate, expressiveness or productivity. I'm not one of those guys:

boilerplate: yes, in Java you write a lot of boilerplate code, but it is semi-automated by your IDE.
expressiveness: what does expressiveness exactly mean to you? I have always been able to express my design intentions in Java, so it can't be the expressive power - most likely you mean concise code and readability issues, however, I do think that Java code can be written in a readable way. And that you can write unreadable code in any non-trivial language.
productivity in a language: I sometimes read people claiming that they were almost instantly (a few days) more productive in a new language. How is this possible? To read up on the enormous API of todays languages takes weeks or months. In Java I don't think about how to do a specific task, I just do it. In a new language I have to learn how to do it, then I have to review whether there are better ways to express the idea or if there are hidden glitches. To cover the whole range of APIs and development tools that I have to use for my development tasks at work (from embedded development, to android app development, to desktop IDE-like tools, to web service and web application development) it takes at least 6 months until I can get as productive as I am in Java right now - with (about) the same quality of my code. However I believe it is much longer than that.

So why am I looking for a new language?

Firstly, I'm continuously looking for new languages out of interest. I like to see new concepts and ideas - and often enough these ideas inspire me to improve something in the products I work on in Java. That is, because it is a concept and even though it might not have support for syntactic sugar it is expressible in Java and might be worth implementing.

Secondly, there are issues with Java that I simply can't ignore:

Often enough we run into problems because of incompatibilities of libraries as one transitive dependency is library X version 1 and another is library X version 2, of course they are incompatible. For some time we used an OSGi container to circumvent the problem, however there is overhead in managing the dependencies (e.g. not all Java libraries are osgified), so the solution was not ideal (for us). Instead, we restrict the libraries that we use, which works well, even though sometimes developers complain that 'it' would be much easier with library 'X'.

Another issue is repository management, build systems and continuous integration. There are no standard tools. So if I need to replace an administrator I know I will have to plan at least 3 months of training. Worse, if something happens during this time it could cost us a lot of development time as our developers had to support the new team member.

Deployment and the Update of our products are another hot topic, but I don't expect it to be better with a new language (it would really be a surprise), so I'm not going into it. I wanted to note it down though.

This starts trivial: there are some features of Java that have knowingly been implemented in a suboptimal way because of backwards compatibility issues - additional to the usual design flaws that creep into a large system. I won't list them here, they were covered enough. However the design flaws themselves are not the issue, the issue is the current strategy of how to deal with them - which is to do nothing. I'll probably work for another 50 years (yes, I am a workaholic...), if I keep true to my current view of things I'll always be coding (at least one third of my week), so even if Java is the new Cobol it won't be usable in 20 years.

Lastly, even though I previously was hard on the boilerplate/expressiveness/productivity argument, it is nice to have a clean, regular and immediately understandable language. These reasons are just not driving my cause.

First Impressions of Kotlin and Ceylon

After spending a lot of time reading about the languages I implemented a simple project in both Ceylon and Kotlin. The most surprising thing for me was that even though the languages share a lot of ideas they have a completely different feel.

Considering that Ceylon is about 18 months 'older' than Kotlin (they were announced only 6 months apart, however, RedHat had been working on it for 'almost two years' and Jetbrains for only 'almost a year') it is understandable that they are in a different stage of development. So don't take my comments regarding Kotlin too seriously. Also Jetbrains has made it clear, that they are in the process of designing the language and if something doesn't work they change it.

Kotlin

One thing I noticed almost immediately is that Kotlin (at M6.2) is by far not finished. Some parts of the API are not consistent, e.g. for Java lists there are Extension Properties first and head as well as Extension methods first() and firstOrNull(). My opinion is that there shouldn't be a multitude of methods how to perform the exact same trivial task. But it also shows that there is no coherent design in place. I have the feeling that it is missing (or not published in any way) and that the current code does not express this design (if it exists).

One more thing is that there is a real lack of introductory material on Kotlin. There is documentation, but it is in a very rough state and it doesn't make the vision of Kotlin clear. I don't know what I can expect from the language. For example, if there hadn't been a talk about reflection I'd be completely in the dark whether reflection for Kotlin was supported or I'd have to 'decode' the JVM structures. Also I'd like to know what to expect in M7 and M8 (so at least two milestones in advance).

However, it does make a few topics very clear:

No compromises in Java compatibility. This is one of the things that drastically separates Kotlin from Ceylon. While it is possible in Ceylon to call Java-Code, the other way round it seems to be more difficult (when creating a ceylon project, the IDE has an option (sic!) to enable Java code calling Ceylon code and a warning that this might affect performance)
Compilation Speed matters, that is, Kotlin code should compile at least as fast as Java code (it doesn't at the moment).
Safe & Concise completes the mantra.

Especially the first point piqued my interest in Kotlin.

One thing I have to remark is that Kotlin has a very familiar feeling, aside from syntax and API differences it still feels somewhat similar to Java coding - which is a good thing. I feel that all my experience with Java is helping me to write Kotlin code.

Ceylon

Ceylon's documentation is overwhelmingly complete and readable, I can only recommend reading the Tour of Ceylon. Also the language itself feels complete and consistent, however, it really feels like a new and completely different language. I'd say that the barrier of using an existing Java library in Ceylon is higher than in Kotlin.

What I find most impressive and intriguing about the language is the concept of union and intersection types. In a recent blog post Gavin King said that Ceylon had the most satisfying solution for combining subtype polymorphism with parametric polymorphism. I completely agree with this statement. I like the language not only for their solution of generics, but also because it seems like a lot of consideration has been invested in every single feature of the language. This just makes the language very harmonic.

Also infrastructure has been taken into account:

Modularity is built into the language
Dependencies are built into the language
There is a standard repository management system
It has a great meta-model of the language (event though I haven't found out how legacy Java modules are supported in the meta-model)
Simple, regular and powerful visibilities using a single annotation shared

The concepts left a really good feeling about the language, however, there is one thing that is bugging me. My application took ages to compile! In the IDE it wasn't so apparent as I haven't used Eclipse for a long time and still remember Eclipse being really slow - but it might just be that it was the compiler that slowed things down. I really thought that I did something wrong when I built the whole application on the command line for the first time. I actually aborted with CTRL+C before I realized that it actually might take this long to compile.

Direct Comparison

I generally have to say that I like both languages a lot. Ceylon's design is much cleaner but Kotlin has a very familiar feel when coding (and regarding its design, it's not finished yet). Where Ceylon has brought OOP type systems to a whole new level, Kotlin concentrates on the most important pieces that improve upon Java like null-safety. Both languages allow me to 'overload' a predefined set of operators and provide a way to define hierarchical structures easily. In the latter case I must say that I like Kotlin's solution of using anonymous extension methods more that the construction based approach taken by Ceylon. According to the Ceylon FAQ extension methods might make it into a subsequent release, so my hopes are still up that I get more flexibility for defining my hierarchical structures.

It took me longer to get used to Ceylon's keywords and also its rhythm as they are really different compared to Java, but I was very satisfied with my result. In contrast, I caught Kotlin's syntax faster but the result didn't feel as satisfying. Probably that is because it felt easier so I didn't have the same sense of accomplishment. In my Kotlin application I mostly used classes from the Java Platform as Kotlin's stdlib is not really complete yet whereas Ceylon provides their own language module where I needed to lookup a lot of things - like how to append a list to another one and similar trivialities.

What I really like about Kotlin is that I could start using it in any of my projects right now without a fuzz. There are a gradle plugin, a maven plugin and ant tasks for build integration and Kotlin even works on Android which is a big plus. It's different with Ceylon, I couldn't just add Ceylon code to my existing projects to add a new feature, from what I gather now (correct me if I'm wrong), I'd have to transform the modules to Ceylon modules where I could mix Ceylon with Java - IF I can do this with my build system. There is proof that it is possible to run Ceylon on Android, however it doesn't give me the confidence to use it in my projects right now.

What came as a surprise to me is that the Ceylon-IDE is a much nicer IDE than the Kotlin plugin for IntelliJ. I thought that Jetbrains who specializes in IDE's would invest more time in their plugin, but probably they are waiting until the final design decisions of the language have been made. As soon as Jetbrains starts to dogfood Kotlin I'm sure the plugin will get better, however for the time being I prefer the Ceylon-IDE.

Conclusion

I'm a little dissatisfied with my findings, while I like both languages I am not able to use any of them now. On the one hand I have a really nice and elegant language that I can't use in my projects as it is not integrated in the toolchain and IDE; we use IntelliJ and we'll never change our Java IDE to Eclipse so we won't mix our code. On the other hand I have a language that I could use in my projects but which is unfinished and thus the risk of using it is too high (and also there are a lot of rough edges to it that need to be fixed before investing more time).

If I'd start a new project, I'd at least investigate whether I could use Ceylon for it.

Building your Xtext DSL with Gradle

2013-08-24T00:00:00-07:00

Xtext is almost what I expect from a tool that is designed to help with designing custom languages. I say almost, because it is heavily incorporated into eclipse IDE/Structures/etc. and therefore brings its own kind of problems. In this post I describe a sample gradle build that lets me compile my xtext languages outside of eclipse's grasp.

The example project

I used the 'MyDsl' stub that eclipse uses to initialize an xtext project i.e. our grammar looks like this:

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

Model:
	greetings+=Greeting*;
	
Greeting:
	'Hello' name=ID '!';

Then I wrote my MWE generator script that uses a different structure than eclipse does.

module org.xtext.example.mydsl.GenerateMyDsl

import org.eclipse.emf.mwe.utils.*
import org.eclipse.xtext.generator.*

var grammarURI = "classpath:/org/xtext/example/mydsl/MyDsl.xtext"
var projectName = "mydsl"
var runtimeProject = "../${projectName}"

Workflow {
    bean = StandaloneSetup {
    	scanClassPath = true
    	platformUri = "${runtimeProject}/.."
    }
    
    component = DirectoryCleaner {
    	directory = "${runtimeProject}/src/main/gen"
    }
    
    component = Generator {
    	pathRtProject = runtimeProject
    	projectNameRt = projectName
    	srcPath = "/src/main/java"
    	srcGenPath = "/build/xtext-gen"
    	
    	language = auto-inject {
    		uri = grammarURI
    
    		fragment = grammarAccess.GrammarAccessFragment auto-inject {
				xmlVersion = "1.0"
			}
    		fragment = ecore.EcoreGeneratorFragment auto-inject {
    		}
    		fragment = serializer.SerializerFragment auto-inject {
    			generateStub = false
    		}
    		fragment = resourceFactory.ResourceFactoryFragment auto-inject {
    		}
    		fragment = parser.antlr.XtextAntlrGeneratorFragment auto-inject {
    		}
    		fragment = validation.JavaValidatorFragment auto-inject {
    		}
    	}
    }
}

Anyone who has gone through the experience of customizing the generated mwe script should find his way with ease. However, for those of you who haven't had the pleaseure I'll briefly explain the differences that I introduced.

No generation of UI and test sources

As I don't use eclipse I don't value the ui plugin and the test sources are not really helpful, they are more like a hint how you would do it on your own. Moreover it would complicate the setup extremely (at least for the tests) as xtext wants to generate the tests into its own project. You could configure the test project as your dsl project, however, then the test sources would be generated into the same directories as your application source.

Change structure where the source files are read and generated

The generated source lives in the build folder, as gradle suggests it should. Your main source folder lives in src/main/java of your project, but that you can change according to your preferences.

Generate an XMI file instead of the new default xtextbin

I mainly included this change for you to see how it is done, so if you stumble upon a "missing resource /org/xtext/example/mydsl/MyDsl.xmi" then eclipse has probably upgraded the xtext version in your IDE and generates a MyDsl.xtextbin instead.

Use a java validator fragment instead of the default xtend generated one

Using a java validator instead of an xtend one is just a move of convenience. I didn't want to compile xtend sources with gradle but still show that we can directly generate java helper classes like the validator (its practically the same for scoping).

The gradle build

I'll leave out the dependencies part (with one exception), I wouldn't expect any reaction other than "those are a lot of dependencies". The exception is that we have to put our source folder (or the folder where you decide to put your *.xtext files) on the classpath. Xtext searches for the xtext file referenced from the mwe file on the classpath, therefore we have to provide it in this way.

apply plugin: 'java'

repositories {
	flatDir name: 'local', dirs: 'libs'
}

configurations {
	xtext {
		extendsFrom compile
	}
}

sourceSets.main {
	output.dir(new File(buildDir, "xtext-res"), builtBy: 'processLang')
}

dependencies {
	// compile dependencies ...

	xtext files("src/main/java")
	// other xtext dependencies
}

task(generateLang, type: JavaExec) {
	inputs.file new File("src/main/java/org/xtext/example/mydsl/GenerateMyDsl.mwe2")
	inputs.file new File("src/main/java/org/xtext/example/mydsl/MyDsl.xtext")
	outputs.dir new File(buildDir, "xtext-gen")

	standardOutput = new OutputStream() { public void write(int i) {} }
	classpath configurations.xtext
	main = "org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher"
	args "src/main/java/org/xtext/example/mydsl/GenerateMyDsl.mwe2"
	
	doFirst {
		def projectFile = file(".project")
		projectFile << "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
		projectFile << "<projectDescription>"
		projectFile << "<name>" + project.name + "</name>"
		projectFile << "</projectDescription>"
	}
	doLast {
		def projectFile = file(".project")
		projectFile.delete()
		
		def pluginFile = file("plugin.xml")
		pluginFile.delete()
	}
}

task(processLang, dependsOn: generateLang, type: Copy) {
	from "${buildDir}/xtext-gen"
	into "${buildDir}/xtext-res"

	exclude "**/*.java"
	exclude "**/*.g"
}

compileJava.source generateLang.outputs.files, sourceSets.main.java
compileJava.dependsOn processLang

Basically we just define a new JavaExec task, that is a task that executes a java program. We define our inputs to be our xtext and mwe files and our outputs to be in the build/xtext-gen folder. We set the classpath to the dependencies that we defined in the xtext configuration and provide our mwe file as an argument to the program.

That would have been the whole story if it weren't for the fact that eclipse is *cough* well eclipse... so we do some extra work.

Generate eclipse project file

When I first saw the message : java.io.IOException: The path '/mydsl/build/xtext-gen/org/xtext/example/mydsl/mydsl/Greeting.java' is unmapped I was puzzled. Of course it wasn't there, your task is to generate it! After a few hours of frustrated searching and looking through the source code I stumbled upon the solution by accident. I was retracing my steps as it had already worked and it wasn't until I deleted the .project file that I had to really breath very slowly and keep calm on purpose. After digging deeper it seems that xtext doesn't know the folder our project lies in because it uses emf resources that is a logical representation of the platform resources hiding their actual physical location. Actually I'm not surprised at all, it just wouldn't have been enough that you have to configure the project name and location several times in the mwe file, it just needed to forget the location of the project. The worst part is that it knows that it knows no location of *any* project but keeps quiet about it.

After you know why the .project file is important, you know why it's beeing generated and deleted afterwards again. We also delete the plugin.xml file that is automatically generated (and honestly I didn't invest the time search for a way to deactivate the generation).

The copy task makes sure that the java files don't and the tokens file does end up in the distributed jar file.

Finally we add the generated java files be compiled by the compileJava task and let the compileJava task depend on our processLang task.

Conclusion

If Xtext wasn't so involved with how eclipse projects are setup (and it isn't really difficult to make something like that configurable) and the dependencies would be more manageable (just look at the dependencies in the project) it would be the ideal tool for language development. Unfortunately this is not the case, but gradle gives us the possibility to circumvent most of the obstacles - but not all of them, e.g. I've hidden an exception that the META-INF/MANIFEST.MF file is not found in swallowing the complete standardOutput.

The sources of the complete project can be downloaded here.

Signing Your Custom AOSP Build With Your Release Keys

2013-07-12T00:00:00-07:00

The procedure described here actually works. I only had to edit sign_target_files_apks to change the display property (that was added in 4.2 or 4.1) so the version is correctly identified as a release version.

I am writing this because I searched the net for a solution - even after I found the web page above, as I didn't trust it - and didn't find sufficient proof to support the procedure.

Building android AOSP on Mac OS X Mountain Lion

2013-06-15T00:00:00-07:00

I have been working with the Android AOSP source code for some time now and spent endless hours searching the internet for problems that arose on different occasions. Today I decided to note them down, on the one hand it is convenient for me to just look it up on my own blog, on the other hand you might find some of these notes useful.

What I ran into today was upgrading from Android 4.0.3 to 4.2.2. I didn't expect any difficulties in compiling the new source as compiling Android 4.0.3 on my Mac OS X (Mountain Lion) worked like a charm for me. But of course I discovered something disturbing - when I ran lunch 12, I got an error:

xcrun: Error: failed to exec real xcrun. (No such file or directory)
build/core/combo/HOST_darwin-x86.mk:62: *****************************************************
build/core/combo/HOST_darwin-x86.mk:63: * Cannot find SDK 10.6 at /Developer/SDKs/MacOSX10.6.sdk
build/core/combo/HOST_darwin-x86.mk:65: * If you wish to build using higher version of SDK, 
build/core/combo/HOST_darwin-x86.mk:66: * try setting BUILD_MAC_SDK_EXPERIMENTAL=1 before 
build/core/combo/HOST_darwin-x86.mk:67: * rerunning this command 
build/core/combo/HOST_darwin-x86.mk:69: *****************************************************
build/core/combo/HOST_darwin-x86.mk:70: *** Stop..  Stop.

** Don't have a product spec for: 'full_maguro'
** Do you have the right repo manifest?

What the hell is it complaining about?

Quickly looking up what xcrun is doing I discovered it's some helper program do select the correct XCode version/sdk etc. It didn't suprise me that it failed as I don't have XCode installed. But why would it want somthing from XCode?

I don't really have an answer to that, but I discovered that some genius decided that everyone building on Mac OS X has XCode installed. There's no option nor flag nor anything else to disable the use of these XCode specific tools - at least I didn't find them. However, I don't want to complain about it (too much) so without further ado I'll tell you how to compile Android 4.2.2 on Mac OS X Mountain Lion. I'll even try to do it in a way that you can do it when starting from scratch. Please keep in mind that I had my system running for a long time so I might have missed some steps when trying to backtrack the essentials.

Setting up the build environment

At this point I should point out that there is an excellent documentation for setting up the environment here and if anything goes wrong you should check it and it might solve your problem.

Homebrew and Prequesites

I am using brew to install packages that are not shipped with the os and will setup the build environment using it. Of course you are free to use whatever you prefer, but I can highly recommend brew. After your system is ready to brew (check it with brew doctor), execute the following commands:

brew install automake
brew tap homebrew/dupes
brew install apple-gcc42

Moreover you have to install Java 1.6, for that just open the Terminal and type java -version. If you haven't installed it the system will prompt you whether you want to install it.

Prepare a virtual disk to hold the android source

The default file system on Mac OS X is case-insensitive, meaning that Test and test would refer to the same file or directory. For some reason there are files in the linux kernel that differ only in the case of a letter - this is something you'd probably do after 72 hours of not sleeping and having a few drinks, at least that's the situation I imagine when trying to grasp why someone would do that... Anyhow, either you have a disk with a case-sensitive filesystem or you create a virtual disk and mount it. To create the virtual disk, mount it and enter the directory run:

hdiutil create -type SPARSE -fs 'Case-sensitive Journaled HFS+' -size 40g ~/android.dmg
hdiutil attach ~/android.dmg.sparseimage -mountpoint /Volumes/android
cd /Volumes/android

Downloading the source

First we need to install repo which is a program that helps to organize the git repositories.

mkdir ~/bin
PATH=~/bin:$PATH
curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
chmod a+x ~/bin/repo

If you have followed along, you're still in the root directory of the mounted disk. If not you'll have to cd /Volumes/android. Finally to download the code run:

repo init -u https://android.googlesource.com/platform/manifest -b android-4.2.2_r1.2 --config-name
repo sync

This will take a while, go do something else until it finishes.

Building for a device

These steps depend on the device you're building for, I will give you the steps for Galaxy Nexus (maguro) and you have to adapt them accordingly. First download the necessary drivers from the nexus driver page and extract them to /Volumes/android. Now you should have a directory listing similar to this one:

$ cd /Volumes/android
$ ls
Makefile			external			libcore
abi				extract-broadcom-maguro.sh	libnativehelper
bionic				extract-imgtec-maguro.sh	ndk
bootable			extract-invensense-maguro.sh	out
build				extract-nxp-maguro.sh		packages
cts				extract-samsung-maguro.sh	pdk
dalvik				extract-widevine-maguro.sh	prebuilts
development			frameworks			sdk
device				gdk				system
docs				hardware			vendor

Now run each of the extract-... shell scripts. You will be asked to accept its license and you should (well, otherwise the drivers won't be extracted). Also you might have to unlock your bootloader etc. but I'll leave that to you; it's a well documented procedure.

Building the source code

Just to be on the safe side, the android build scripts depend on being executed by bash therefore run it if you don't already. Moreover I had problems with perlbrew, if you have it installed it'd be wise to switch to the perl version that is shipped with Mac OS X now.

Now there comes the time for the hack, download this file ( android-4.2.2_r1.2 only !!! ) and overwrite the file in build/core/combos/HOST_darwin-x86.mk. I changed the file to allow for manually overriding the host compilers HOST_CC and HOST_CXX and skip the xcode test if they are both present. I don't use makefiles at all so there might be a more elegant solution to the problem but this works. After you have replaced the files you can execute:

. build/envsetup.sh
HOST_CC=gcc-4.2 HOST_CXX=g++-4.2 lunch 
make HOST_CC=gcc-4.2 HOST_CXX=g++-4.2 -j8

This will again take a while and if you followed all the steps the compilation succeeds.

Final Notes

Most of the steps I presented here are from the official android documentation and I recommend reading these docs for further information or troubleshooting. I hope I could help some of you :)

Using the new Android Build System and TDD

2013-02-28T00:00:00-08:00

Today I ported our android build to use the new gradle based build system. As we heavily employ Test Driven Development I also wanted to be able to test drive our application without resorting to test on devices - that proved to be a challenge.

The new android build system

It is still in development and I guess far from being released, nevertheless it seems very promising to us as we build almost all of our Java-based software with gradle. The default setup is very simple, there are only 4 things to remember:

the android plugin must be on the classpath for the build process
the project must apply the android plugin
you have to put the sources in the correct directories
you have to tell the android plugin where the android sdk is

The following snippet shows an example build.gradle like the one published in the official documentation:

buildscript {
	repositories {
		mavenCentral()
	}

	dependencies {
		classpath 'com.android.tools.build:gradle:0.3'
	}
}

apply plugin: 'android'

android {
	compileSdkVersion 15
}

You should also notice that the directory structure is different from a standard android project. Basically you have to put all the sources and resources into src/main. Device tests have to go into src/instrumentTest.

Finally create a new file local.properties and put something like: sdk.dir=/usr/local/android-sdk/r21.1 into it. Please replace the path with your own.

Adding a task for unit tests

The android plugin doesn't have a task for standard unit tests which is quite unfortunate, therefore we have to add that. First we have to add a sourceSet for our test. Reading the documentation we discover that the sourceSets are configured in the android section. Naturally you would think you could just add an additional sourceSet, but the following doesn't work.

android {
	sourceSets {
		test {
			java.srcDir file('src/test/java')
			resources.srcDir file('src/test/resources')
		}
	}
}

First it seems to work, but when you try to use it:

task unitTest(type:Test, dependsOn: assemble) {
        description = "run unit tests"
        testClassesDir = project.android.sourceSetsContainer.test.output.classesDir
        classpath = project.android.sourceSetsContainer.test.runtimeClasspath
}

It just tells you:

Could not find property 'output' on source set test.

In other words, we can find the source set, but we cannot access the directory where the classes are - and therefore cannot run the tests.

Hacky workaround

I wasn't in the mood to give up, probably there is another way to access the correct directories but here is what I came up with:

apply plugin: 'android'

android {
	compileSdkVersion 15
}

sourceSets {
	unitTest {
		java.srcDir file('src/test/java')
		resources.srcDir file('src/test/resources')
	}
}

configurations {
	unitTestCompile.extendsFrom runtime
	unitTestRuntime.extendsFrom unitTestCompile
}

dependencies {
	unitTestCompile files("$project.buildDir/classes/release")
}

task unitTest(type:Test, dependsOn: assemble) {
	description = "run unit tests"
	testClassesDir = project.sourceSets.unitTest.output.classesDir
	classpath = project.sourceSets.unitTest.runtimeClasspath
}

check.dependsOn unitTest

We define a standard java sourceSet, this works as the android plugin uses the JavaBasePlugin. In the dependencies section we declare our dependency to the application code - as you might see, this is the hacky part and I will search for a better solution in the next few days. If there is one I will post an update. Last we define our Test task and declare it to be executed in the check phase.

Robolectric Tests

This gives us unit tests, but it doesn't give us the capabilities to test source code that depends on the android API. Of course that doesn't make a lot of sense therefore we will integrate Robolectric to drive our tests.

First we add additional dependencies to our dependencies section so it looks like this:

repositories {
	mavenCentral()
}

dependencies {
	unitTestCompile files("$project.buildDir/classes/release")
	unitTestCompile 'junit:junit:4.10', 'org.mockito:mockito-core:1.9.0'
	unitTestCompile 'com.google.android:android:4.0.1.2'
	unitTestCompile 'com.pivotallabs:robolectric:1.2'
	configurations.unitTestCompile.exclude group: 'com.google.android.maps'
}

We just add a dependency on the android API and robolectric and exclude the maps API which isn't necessary if you don't use it. Otherwise you just have to download it yourself as google doesn't permit its upload to maven central. Don't forget to declare a repository where gradle can find the dependencies.

Writing the Test

As our project structure is now different from the standard android project structure, we have to tell Robolectric where our AndroidManifest.xml and the other resources are. To do that we define our own runner:

import com.xtremelabs.robolectric.RobolectricTestRunner;
import org.junit.runners.model.InitializationError;
import java.io.File;

public class HelloWorldRunner extends RobolectricTestRunner {
	public HelloWorldRunner(Class testClass) throws InitializationError {
		super(testClass, new File("src/main"));
	}
}

With this runner we can write a simple test case:

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(HelloWorldRunner.class)
public class HelloWorldTest {
	@Test
	public void testInstantiation() {
		new HelloWorldActivity();
	}
}

I know it doesn't look like much, but try to run this testcase without Robolectric and you will see that it is really necessary.

Conclusion

Gradle is a great build system and we could easily extend the android plugin with ordinary unit tests. I think it is the right choice for the new android build tool and I am looking forward to using it.

Download the whole example project.

Go Controversial 1 - What about proper Constructors?

2013-02-27T00:00:00-08:00

When I first started programming in go, I found it irritating that there is no custom initialization phase when an "object" is being constructed. As I have heard some complaints about that and I've already made up my mind, I will share my thoughts here. In this article I use the term constructor to refer to the concept as it is used in C++ or Java - not go's concept of constructors (which are factory functions).

Initialization

Each type has a zero value, e.g. the zero value for int is 0. Optimally a type is ready to use with its zero value, but sometimes this isn't enough and we will use composite literals. A composite literal creates a new instance of the specified type and allows one to provide initialization for the fields. Suppose we have the following piece of code:

type SomeStruct struct {
	A, B string
	C int
	p int
}

SomeStruct is a complex type and has 3 exported fields and a private one. To initialize an instance we would write something like this:

x := SomeStruct{"first", "second", 3}

In this way you can initialize all the exported fields of a struct. Now there's a question, how would you fit in a magic function that is automatically called upon instance creation. It obviously wouldn't make much sense to call it before the fields had been initialized, therefore it must be after that.

What would such an initialization function do?

It would probably allocate data for private fields and do something with the already initialized fields.

What if there is an error?

The only way to react is to panic as you don't have a return value to pass an error. But that is against the philosophy of go, look at the statement again:

x := SomeStruct{"first", "second", 3}

It does look harmless, doesn't it? We just declared a new variable that is initialized with this composite literal.

How can we reason about it?

Very simple, nothing bad can happen (except if you run out of memory). However, if a magic function was called instead, you could not reason about this statement without looking into the code that implements the type. It could do anything, like opening a connection to the SETI project and wait for them to announce that they have found aliens.

But there's a more pressing problem, what about the following piece of code:

var x SomeStruct

When do you suggest this magic function should be called?

But I really do need a constructor!

Actually I've never been fond of constructors, they don't really communicate the intent I would like to see. When writing Java code I prefer static factory methods and similarly that's the way to go - just write a factory function. That's how the concept of go constructors are defined. A function that returns an instance of a type is a constructor for that type. It also solves the problem of reasoning, everyone knows that a function call can cause/detect errors in one way or the other.

I've heard the argument that it is still possible to work with not initialized structures. Firstly, I think that is always possible and secondly, you shouldn't worry about that. You can (should) provide documentation so that everybody knows how to use your API and if you really want to prohibit the use of an uninitialized type, return an interface instead and don't export your actual type.

Conclusion

The concept of Java/C++ constructors doesn't fit into the philosophy of go, there's a conflict with reasoning about variable declarations and composite literals. If you need initialization, provide a factory method. If you want to prevent the possibility of using your instances uninitialized return an interface and don't export your actual type.