The job comprised of roughly 50,000 tasks submitted recursively over a multi-GB data set that took 5hrs to process. The bug left a single task in the queue hanging indefinitely, thus the job never completing. There’s no way to kill a Thread in Java, and if you have blocking I/O it can be difficult to implement interrupt() such that it works correctly.

This is the point I found it’s quite difficult to actually expose at run-time what the hell Java’s Executor is actually doing at any point in time.

The successful choice I found was to create a custom ThreadPoolExecutor which trapped the creation of a FutureTask and the start/stop of any given task. I expect these hooks are there for this purpose, but it sure isn’t obvious how to do this.

It seems odd that the API doesn’t include any way to gather info about what’s happening inside the Executor, and also, there’s not even a toString() implementation for wrapping classes like FutureTask which would bubble your Runnable or Callable classes’ toString() methods.

Oh well!

Here’s an example:

private static final int NUM_THREADS = 4;
private final Logger log = Logger.getLogger(getClass());

private LinkedBlockingQueue< Runnable> taskQueue = new LinkedBlockingQueue< Runnable>();
private List< Runnable> running = Collections.synchronizedList(new ArrayList());

public void doSomeStuffConcurrently() {
	Executor executor =
		new ThreadPoolExecutor(NUM_THREADS, NUM_THREADS,
			0L, TimeUnit.MILLISECONDS,
			taskQueue,
			Executors.defaultThreadFactory())
	{

		@Override
		protected < T> RunnableFuture< T> newTaskFor(final Runnable runnable, T value) {
			return new FutureTask< T>(runnable, value) {
				@Override
				public String toString() {
					return runnable.toString();
				}
			};
		}

		@Override
		protected void beforeExecute(Thread t, Runnable r) {
			super.beforeExecute(t, r);
			running.add(r);
		}

		@Override
		protected void afterExecute(Runnable r, Throwable t) {
			super.afterExecute(r, t);
			running.remove(r);
			log.info("RUNNING: " + running);
		}
	};

	executor.execute(new Runnable() {

		@Override
		public void run() {
			// so some stuff
		}

		@Override
		public String toString() {
			return "describe the task here!";
		}
	});
}

Related posts:

1. Implementing a “Draft Mode” with Apache Wicket Forms
2. Clustering Guice Java Web Applications
3. Implementing Facebook OAuth 2.0 Authentication in Java

It also notable that tests like this will not just cover code that exists today, but code that get’s written in the future too.

Generally these tests won’t actually execute classes found the class path, but instead reflect on class structure, annotations or method IO. But you could also conceivably write tests that instantiated classes and tested behavior too.

Here’s an example of a classpath scanning, reflective test case.

Background: I’ve used warp-persist in the past to do JPA transactions with Guice projects. This gives you a @Transactional annotation which will apply a transaction across the method, unless exception(s) of specified types are thrown. Unfortunately, the warp-persist library has a design flaw, in that the default is that checked exceptions ROLLBACK, but runtime (non-checked) exceptions DO NOT. Why you’d want this is unclear to me, so we always end up writing {Exception.class, RuntimeException.class} to cover all cases – something that is easy to forget to do!

The test below, find’s all uses of the @Transactional annotation and verifies that each use of the annotation lists both RuntimeException.class and Exception.class as the rollback conditions.

By scanning the classpath, any new (or old) code will be flagged if the developer forgets this bit of housekeeping (this test has saved my bacon more than once).

package com.mycom;

import com.visural.common.ClassFinder;
import com.wideplay.warp.persist.Defaults.DefaultUnit;
import com.wideplay.warp.persist.Transactional;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import junit.framework.TestCase;

public class TransactionalTest extends TestCase {

    public void testTransactionalDoesntCreateBadBugs() throws ClassNotFoundException, Exception {
        ClassFinder cf = new ClassFinder("com.mycom", true);
        List < String >  invalid = new ArrayList < String > ();
        Set < Class >  classes = cf.find();
        for (Class c : classes) {
            for (Method m : c.getDeclaredMethods()) {
                if (m.getAnnotation(Transactional.class) != null) {
                    List < Class < ? extends Exception >  >  ex = Arrays.asList(m.getAnnotation(Transactional.class).rollbackOn());
                    if (!ex.contains(Exception.class) || !ex.contains(RuntimeException.class)) {
                        invalid.add(c.getName()+"#"+m.getName()+"\n");
                    }
                }
            }
        }
        if (!invalid.isEmpty()) {
            Collections.sort(invalid);
            throw new Exception("The following methods use @Transactional, but do not rollback on Exception & RuntimeException: "+invalid.toString());
        }
    }
}

This is a slightly contrived example, in the sense that it is working around a design flaw in a 3rd party library. But, this technique is useful in many cases where you have design patterns, standards or anti-patterns which you want to enforce or prevent.

Here’s a few examples:

• Enforce all model classes should be Serializable
• Enforce class or method naming conventions
• Enforce specific package structures / hierarchies
• Ensure that arguments/returns are not overly specific e.g. return List not ArrayList

The tests can be as broad or narrow as your design rules. I’m not suggesting that the above examples are good ideas in all cases. But writing tests like these is a way to formalise a team/project standard in a more useful way, than writing it down in a document or wiki.

There be exceptions to many of the rules or patterns being tested, but you can always write the exceptions into the test.

Overall the advantages of this approach are:

• An opportunity to express the design pattern in the actual product code base as an executable test.
• A way to ensure that a new developer is made aware of the design pattern, at the time it is most relevant: when they need to apply it.
• A place to document the exceptions to the pattern, and why they are allowable.
• A way to keep everyone honest. Even experienced developers take short cuts, or are forgetful sometimes.

It’s a win all round. Not all Java developers are comfortable with class-path scanning and reflection/introspection (maybe they should be), but not all developers will need to write these tests, as they are probably best handled by a technical lead.

Footnote: if you want a general-purpose utility for doing class path scanning, then my library visural-common has ClassFinder.java – a way to search the class path for specific classes (as illustrated above).

Related posts:

1. NetBeans 6.7 Broke My Parameterized Tests
2. Testing Abstract Classes With Mockito and JUnit
3. Using Bottom-Up Iterative Object/Database Layer With hbm2java and warp-persist

Apparently he created it while at Sun for Hudson. The other popular Java Service Wrapper for Windows (by Tanuki) is GPL licenced and has commercial licences available. Kohsuke created “winsw” so there was an option for Hudson available under a more permissive licence than GPL.

Conveniently, “winsw” can actually be used with pretty much any process you want to turn into a windows service, not just Java apps.

It’s got really simple configuration with an XML file e.g.

<service>
  <id>fisheye</id>
  <name>dev.net FishEye</name>
  <description>FishEye</description>
  <executable>%FISHEYE_HOME%\bin\fisheyectl.bat</executable>
  <logpath>P:\dev\logs\os\windows\services</logpath>
  <logmode>roll</logmode>
  <depend>Spooler</depend>
  <startargument>run</startargument>
  <stopargument>stop</stopargument>
</service>

And it’s basically just a single EXE file (31KB) and the XML config (although it does require .NET 3).

Anyhow, this is a nice little tool to have in your toolbelt, particularly for getting non-Windows friendly stuff running on Windows servers!

Related posts:

1. Use Hudson (now Jenkins) to restart a Windows service…
2. Setting Up Memcached As A Windows Service
3. 1 Minute Guide – Installing Redmine on Windows

You can get the downloads here (note there is a separate package for Wicket 1.4 and Wicket 1.5) -

• visural-wicket 0.7.0 for Wicket 1.5
• visural-wicket 0.7.0 for Wicket 1.4

Related posts:

1. visural-wicket 0.6.5 release is available!
2. visural-wicket 0.5 released – ready for action!
3. visural-wicket 0.6 released – lots of new components!

• Instantiation
• Render
• Enable

You can build your own security framework using the IAuthorizationStrategy interface and setting an appropriate implementation in your Wicket Application class.

Alternatively, you can leverage a simple security framework that is included in my open source library visural-wicket.

Visural-wicket’s security framework really is just the “glue” between Wicket’s IAuthorizationStrategy and something that you can start using immediately to control component level security.

There are two key interfaces – IClient and IPrivilege. The former represents some client of the application, generally a logged in user. The latter represents some security rule which is applied on rendering or enabling of a component.

They’re really simple interfaces -

/**
 * A client is any user of your system, i.e. in most cases an authenticated User.
 */
public interface IClient < T extends Serializable> {
    /**
     * Return any unique identifier.
     * @return
     */
    T getId();
}

/**
 * A privilege is any condition that can be evaluated with respect to a {@link IClient}.
 */
public interface IPrivilege < T extends IClient> {

    /**
     * Determine whether the privilege applies to the client.
     * @param client
     * @return
     */
    boolean isGrantedToClient(T client);
}

In your application class you provide some glue to define how to retrieve the current IClient (e.g. from the session) –

public class MyApp extends Application {

    public void init() {
        // ...
        getSecuritySettings().setAuthorizationStrategy(new com.visural.wicket.security.AuthorizationStrategy(new IClientProvider() {
            public IClient getCurrentClient() {
                return userService.getCurrentUser(); // for example
            }
        }));
        // ...
    }
}

Then any component which implements ISecureRenderInstance or ISecureEnableInstance can define the privilege required to display or enable it –

public class MyPage extends WebPage implements ISecureRenderInstance {
    // ...

    public IPrivilege getRenderPrivilege() {
       return new IPrivilege < User>() {
           public boolean isGrantedToClient(User client) {
               return client != null && client.isAdmin();
           }
       };
       // instead of returning a anonymous class like this, you could also
       // package up common privileges into a singleton instance,
       // e.g. return Privilege.ADMIN;
    }
}

Wicket and visural-wicket then work in conjunction to ensure that the privileges get evaluated automatically at run-time. Unlike manually coded security which is mixed with other visibility logic, you can guarantee that no component that fails its security check can get rendered to a user that shouldn’t see it. You can also define common security rules as privileges in your code base and refer to them from many pages and components rather than duplicating similar logic across the application.

It’s a simple security mechanism which is designed to fit in with pretty much any app. I hope others find it useful too.

Related posts:

1. visural-wicket 0.6 released – lots of new components!
2. visural-wicket 0.6.5 release is available!
3. visural-wicket 0.7.0 release with support for Wicket 1.5

The method naming and Javadoc is pretty poor for this part of the Java API and it’s always confusing as to how to interrogate the request to figure out which bits of information you want from the request URL and parameters.

I thought I save myself (and you) the effort of doing this again, once and for all, by writing a quick post which summarises all the interesting “getters” on the HttpServletRequest:

Say I have a request going to a Java web application deployed at context path “WebApplication1″ on my localhost sever, running on port 8080 –

URL = “http://localhost:8080/WebApplication1/test?q=123&r=456″

Related posts:

1. Getting a non-relative (absolute) URL for a Wicket page
2. 5 Minute Guide to Clustering – Java Web Apps in Tomcat
3. 301 Redirects Made Easy In Java

You might think that the Atari VCS was one of the early 2D home console systems, which had a number of 2D sprites to move around a two dimensional plane, but you’d be wrong.

The original Atari was more like a “one dimensional” console. That is, its graphics processor had no notion of a two dimensional surface whatsoever, no frame buffer, and not even the concept of two dimensional sprites. The TIA (the Atari VCS’s graphics processing chip) supported -

• Two one-line 8 pixel wide player sprites, of a single colour
• A one-line ”ball” up to 8 pixel wide single colour blob of pixels
• Two one-line ”missiles” up to 8 pixel wide single colour blob of pixels, tied to the colour of the player sprites
• A one-line 20-bit (20 pixel stretched) single colour “playfield” which can be mirrored or copied across the two halves of the screen

Oh, and the system had 128 bytes of RAM. That’s right BYTES.

The above graphic elements are one dimensional in the sense, for each scanline down the display, the programmer had to dynamically reprogram the objects if they needed to change appearance. If you didn’t do anything between scanlines the same row of pixels would get repeated down the screen.

This is what the book’s title “Racing the Beam” references, since for each scanline the Atari VCS developers were racing the CPU’s clock to setup the next scanline during the horizontal blank of the screen.

The names of the various graphic elements were named after Combat and Pong, the two games the system was actually designed to play. The company figured that if they could create other games for the system, then it was a bonus, but it was designed around Combat and Pong, which were popular arcade games at the time. It’s incredible to think that the VCS had one of the longest production runs in home console history with over 10 years of service!

Atari VCS ROMs were also 2K-4K for the most part (with 8K and 16K appearing late in the life of the system using bank switching). This small size meant developers would go to huge lengths to save space in the programs, often using graphic data as code, or code as graphics data.

A simply astonishing number of games were created for the Atari VCS which found innovative ways to overcome the limitations of the system (even when it probably wasn’t a good idea to try). The book covers a number of landmark games in the system’s history, such as Pacman (which is sometimes blamed for the North American game crash of 1983), Pitfall, Star Wars – The Empire Strikes Back (the first licenced Star Wars game) and others.

Pacman was interesting since the game requires Pacman and four ghost characters to be somehow fit into the “two player sprites” Atari VCS. This was accomplished by only drawing each ghost on every fourth frame, which makes the game pretty much unplayable in an emulator -

www.youtube.com/watch?v=HL2p2ANFlQ4

But as you can see, on a traditional CRT television, it looks fine (thanks to the eye’s persistence of vision) -

www.youtube.com/watch?v=juH2qHYX9aI

That said, it’s still a pretty poor rendition of Pacman.

Pitfall was also interesting as it’s pretty much the first “platform game” that has many recognisable elements of the genre by contemporary standards. The way the game was fit into the cartridge and how it came about from a design perspective is also fascinating and there’s an excellent talk on it by David Crane given at this year’s GDC.

Anyhow, I’d highly recommend picking up this book, if you were a kid in the 80′s, if you have an interest in early computing hardware, or just if you’re generally interested in computing, it’s a great read.

I’m actually tempted to write an Atari VCS game in assembler as a programming challenge. I used the 6502 quite a bit during my university degree, and the simplicity and purity of coding in against an 8-bit CPU like that is a lot of fun.

Amazon.com - Racing the Beam: The Atari Video Computer System (Platform Studies)

No related posts.

I was horrified though when I opened a project to find every single source file was being shown as “modified” by Netbeans, although there were no diffs shown inside the files!

However, it turns out that it’s not Netbean’s fault.

If you’re using msysgit on Windows to check out your projects, you probably installed it and did the typical “next->next->next” through the installer. Including through the following screen….

Well by default msysgit does line ending mutation so that if you checkout a source file to a Windows box, it will add Windows line ending (“\n\r”) to all source files, and when you push it back to the server, change the line ending to Unix style (“\n”). Unfortunately this causes Netbean’s Git plugin to (rightly) say, “Hey! You changed all these files!”.

Really the auto line ending change stuff is a really bad idea, and you should disable it.

The command to do so is as follows -

 git config --global core.autocrlf false

After disabling, the Git support seems to work really well.

Related posts:

1. 5 Reasons Why NetBeans Rocks
2. Automatic release packaging w/ version numbers for Netbeans Java Projects…
3. NetBeans 6.7 Broke My Parameterized Tests

I’ve been extremely busy through the first half of 2011, mostly because my wife and I are expecting our first baby (only a few weeks to go), and most of my free time has been spent getting our house in order, doing baby shopping and doing my part to support as best I can.

Work has also been very hectic with a lot going on. What little time I’ve had to work on personal projects, was spent implementing the new search features at onmydoorstep.com.au, doing the initial Wicket 1.5 branch for visural-wicket and learning a bit about Android development.

I do have some blog posts sorted out though for the coming months. Mostly they’re not going to be long form in-depth posts, but there are a few things I’ve learned or been working on lately that I’d like to post about, but haven’t had a chance to write up before now.

Of course, once our son arrives in a few weeks, who knows what could happen!

Related posts:

1. onmydoorstep.com.au – officially launched!

Here’s an example of how we can get Hudson to restart Apache running on the same host -

1. Create a new Job, say “Restart Apache” as a standard Hudson job.
2. Add a new “Windows batch command” step, and in the box enter -

   net stop Apache2

3. Add another new “Windows batch command” step, and in the box enter -

   net start Apache2

4. Run the job, and behold!

This is handy if you’re automating deployments usnig Hudson on Windows hosts. Particularly if you’re trying to avoid PermGen issues with hot-redeployments on say, Apache Tomcat.

Related posts:

1. Quick Tip – Make Anything A Windows Service
2. Setting Up Memcached As A Windows Service
3. 1 Minute Guide – Installing Redmine on Windows

This seems like a job for a face detection algorithm. Searching for appropriate open-source Java implementations didn’t yield too many results, however I was successful with JJIL – Jon’s Java Imaging Library, which is open sourced under the LGPL licence.

JJIL is targeted at Java ME / Android platforms and doesn’t have much documentation or a particularly intuitive API (not complaining, as clearly some stellar work has gone into it, kudos to Jon Webb, it’s creator).

In the end I muddled through, detecting faces in an image in a standard Java project, but given it took me a while to get everything working, I thought I’d write a quick guide to help out others that are trying to achieve similar results.

Getting the Right JARs

First things first, I had trouble getting the published JAR files to work happily together. There seems to be some sort of version mismatch issue between the core and J2SE versions.

So I built my own copy – you can download it here – JJIL-visural-build-20110112.zip

This is a build of the current trunk JJIL code, and the Java SE additions (jjil-j2se).

This build is guaranteed to work with the code examples below.

Basic Process

I’m going to try to explain the basic process of detecting the faces in terms of input and output data.

The key file provided by JJIL for easy face (and other body part) detection is Gray8DetectHaarMultiScale.java

This operation is applied to an 8-bit greyscale input image, in combination with a pre-defined Haar Cascade profile. The profile determines which areas of the image are “detected”. So you would want (for example) a profile to detect the frontal face features. JJIL provides several profiles out of the box.

The output image is a mask of the area of image where faces are detected (white) and the areas where no face was detected (black). This isn’t tremendously useful, as we’d usually rather just have the rectangular areas in coordinate form – I’ll address this later, after walking through the process.

1. Read an image from disk (.JPG, etc.)

2. Convert it into a jjil.core.Image

3. Generally we’ll have an RGB image (colored image) and so need to convert it to 8-bit greyscale, which is what the Gray8DetectHaarMultiScale class requires.

4. Create a new instance of Gray8DetectHaarMultiScale with the Haar profile for detecting faces (or other body part if that’s what you’re looking for).

5. Apply Gray8DetectHaarMultiScale to our 8-bit grey image.

6. Retrieve result from Gray8DetectHaarMultiScale.

The figure below shows, the source image, overlayed with the resulting mask, from step #6

So as you can see, the masks correctly identify the faces the two people in the image.

The Code

Here’s a small Java class that demonstrates how to read an image and apply the process described above.

Note: all the code in the article can be downloaded as a full project at the end of the article.

package jjilexample;

import java.awt.image.BufferedImage;
import java.io.File;
import java.io.InputStream;
import javax.imageio.ImageIO;
import jjil.algorithm.Gray8Rgb;
import jjil.algorithm.RgbAvgGray;
import jjil.core.Image;
import jjil.core.RgbImage;
import jjil.j2se.RgbImageJ2se;
import jjil.algorithm.Gray8DetectHaarMultiScale;

public class Main {
    public static void findFaces(BufferedImage bi, int minScale, int maxScale, File output) {
        try {
            // step #2 - convert BufferedImage to JJIL Image
            RgbImage im = RgbImageJ2se.toRgbImage(bi);
            // step #3 - convert image to greyscale 8-bits
            RgbAvgGray toGray = new RgbAvgGray();
            toGray.push(im);
            // step #4 - initialise face detector with correct Haar profile
            InputStream is  = Main.class.getResourceAsStream("/jjilexample/haar/HCSB.txt");
            Gray8DetectHaarMultiScale detectHaar = new Gray8DetectHaarMultiScale(is, minScale, maxScale);
            // step #5 - apply face detector to grayscale image
            detectHaar.push(toGray.getFront());
            // step #6 - retrieve resulting face detection mask
            Image i = detectHaar.getFront();
            // finally convert back to RGB image to write out to .jpg file
            Gray8Rgb g2rgb = new Gray8Rgb();
            g2rgb.push(i);
            RgbImageJ2se conv = new RgbImageJ2se();
            conv.toFile((RgbImage)g2rgb.getFront(), output.getCanonicalPath());
        } catch (Throwable e) {
            throw new IllegalStateException(e);
        }
    }

    public static void main(String[] args) throws Exception {
        // step #1 - read source image
        BufferedImage bi = ImageIO.read(Main.class.getResourceAsStream("test.jpg"));
        // onto following steps...
        findFaces(bi, 1, 40, new File("c:/Temp/result.jpg")); // change as needed
    }
}

Getting Face Rectangles Instead

It would be more useful in many cases, to get a collection of Rectangles, in coordinate form, instead of an image mask.

There is a version of DetectHaarMultiScale in the JJIL project SVN, which implements a “getRectangles” method to retrieve this data. Unfortunately the source in incompatible with the rest of the library in SVN, so it may be WIP or an abandoned version of the code.

To get around this, I created my own version of Gray8DetectHaarMultiScale, which you can download here – Gray8DetectHaarMultiScale

Here are the important changes below -

    public void push(Image image)  throws jjil.core.Error {
        pushAndReturn(image);
    }

    public List pushAndReturn(Image image) throws jjil.core.Error
    {
        List result = new ArrayList();
....
                    if (hcc.eval(imSub)) {
                        // Found something.
                        nxLastFound = imSub.getXOffset();
                        nyLastFound = imSub.getYOffset();
                        // assign Byte.MAX_VALUE to the feature area so we don't
                        // search it again
                        result.add(new Rect(nxLastFound*nScale, nyLastFound*nScale,
                                this.hcc.getWidth()*nScale,
                                this.hcc.getHeight()*nScale));

                        Gray8Rect gr = new Gray8Rect(nxLastFound,
                                nyLastFound,
                                this.hcc.getWidth(),
                                this.hcc.getHeight(),
                                Byte.MAX_VALUE);
                        gr.push(imMask);
                        imMask = (Gray8Image) gr.getFront();
                     }
....
        return result;
    }

So now we can call “pushAndReturn(…)” instead of just push() to apply the process to our image, and get a List back of the detected faces. Perfect!

Using this here is a version of the example code above which prints out the rectangles where faces were detected -

package jjilexample;

import java.awt.image.BufferedImage;
import java.io.File;
import java.io.InputStream;
import java.util.List;
import javax.imageio.ImageIO;
import jjil.algorithm.Gray8Rgb;
import jjil.algorithm.RgbAvgGray;
import jjil.core.Image;
import jjil.core.Rect;
import jjil.core.RgbImage;
import jjil.j2se.RgbImageJ2se;

public class Main {

    public static void findFaces(BufferedImage bi, int minScale, int maxScale, File output) {
        try {
            InputStream is  = Main.class.getResourceAsStream("/jjilexample/haar/HCSB.txt");
            Gray8DetectHaarMultiScale detectHaar = new Gray8DetectHaarMultiScale(is, minScale, maxScale);
            RgbImage im = RgbImageJ2se.toRgbImage(bi);
            RgbAvgGray toGray = new RgbAvgGray();
            toGray.push(im);
            List results = detectHaar.pushAndReturn(toGray.getFront());
            System.out.println("Found "+results.size()+" faces");
            Image i = detectHaar.getFront();
            Gray8Rgb g2rgb = new Gray8Rgb();
            g2rgb.push(i);
            RgbImageJ2se conv = new RgbImageJ2se();
            conv.toFile((RgbImage)g2rgb.getFront(), output.getCanonicalPath());
        } catch (Throwable e) {
            throw new IllegalStateException(e);
        }
    }

    public static void main(String[] args) throws Exception {
        BufferedImage bi = ImageIO.read(Main.class.getResourceAsStream("test.jpg"));
        findFaces(bi, 1, 40, new File("c:/Temp/result.jpg")); // change as needed
    }

}

If you run this, you will note that it actually detects 3 faces in the image. This is common, as the way the Haar algorithm works, is by resizing the image to different scales and running a fixed size matrix over the image. It is possible for the same face to be detected at different scales and so you end up with rectangles within rectangles. It is a pretty trivial matter to remove these “extra” rectangles though by just checking if they are fully contained by another and ignoring them accordingly.

Download the Project

To save you some time, here’s the full example as a Netbeans project that you can download and run, play with, etc. Have fun!

JJILExample

Photo used in this article courtesy of Gill Penny.

Related posts:

1. Adding a Dynamic Watermark to an Image – Java Web Application
2. 5 Minute Guide to Clustering – Java Web Apps in Tomcat
3. Detecting Which Browser In Java Servlet/Filter

This year was a very busy year for me. February saw the initial release of my open source projects, visural-common and visural-wicket, with two subsequent releases during the year. The visural-wicket library was much more successful that I had guessed and has been downloaded by hundreds of developers of the course of the year.

The rest of the first half of the year were a combination of juggling work, personal life and building onmydoorstep.com.au. Ultimately I was not successful in the AppMyState competition that I entered in to, but I continued to work on the site throughout the year with many of the features that I had planned initially finally making their way into the site over the following months.

Depsite not fulfilling its intended purpose, building On My Doorstep was a fantastic experience for me, and expanded my skills greatly, particularly in the areas of CSS, SEO, UI design and server administration. Having work mainly on internally hosted apps in the past, this site was a crash course in startup-dom which has been invaluable.

I dabbled in a little Python this year, having felt like I’ve “run my course” with Java recently. I like Python, but I don’t think I’m ready to switch over as yet. I think I’d like to dabble with a little Scala and Ruby first, before undertaking any “proper” project with a new language.

One thing that I felt consistently over the past year, is that I’ve been spreading myself too thin. I need to spend less time writing code, and try some new things in life.

This is reflected in my goals for 2011 below -

• Get “On My Doorstep” to a “parkable” state.
  By parkable, I don’t mean to abandon the project, but simply finish off the work in progress stuff and put it into “maintenance” for the time being.
• Write a novel.
  This will be a new and much needed creative experience. I went from working on animated films in 2006 to working two jobs and cutting code 7 days a week. It’s time I put my creative energies in a less work-oriented direction.
• Dropping 10kgs.
  Over the past 4 years I’ve slowly gained around 10 kgs in weight. This is since I dropped roughly 25 kgs between 2005/2006, dropping from 105kgs to 80kgs. This time it’s been mainly caused by too much good food, as opposed to bad food, and also excessive hours working with not enough exercise. My plan is to decrease the amount of food I eat (particularly carbs), but also try to schedule in some sustainable exercise.

I’ve kept it to three, as I feel it’s not realistic to have more than that at a time.

That’s not to say everything else falls by the way-side, I expect to make another few releases of visural-wicket during the year, and other random projects. I will keep making new blog posts, but maybe not quite as frequently as I have this year (at one stage I was managing 1-2 posts per week).

Some things I’m looking forward to -

• spending more time with my family
• getting through my bluray home-theatre backlog
• Dragon Age 2 (hope they don’t screw it up, I’m being optimistic!)
• Mass Effect 3 (hope it makes it for 2011)
• Diablo 3 (I might be disappointed – 2012?)

Anyway, here’s to a new year! I hope everyone has a good one, and finds it to be bigger, better and more enjoyable than the last.

Related posts:

1. Back to Dreamhost…

Doug asked Oracle to acknowledge that it was asking the ECs to condone breaking the JSPA rules. He said that if he was put in a position where he had to condone breaking the rules he would have to resign. Ken said that he understood, and would regret it if Doug resigned, but pointed out the importance of allowing the conversation to go forward. Josh Bloch asked again whether Oracle, who were on record as saying that to deny Apache a license without FOU restrictions is a violation of the JSPA, still feel that way. Ken Glueck responded that Oracle were not prepared to answer a legal question. Josh responded that Oracle had been willing to vote on this matter twice in the past. Ken responded that this is the situation now. Tim Peierls noted that Oracle has had plenty of time to prepare an answer.

Ken pointed out again that the platform is stuck and we need to move forward.

(my emphasis).

Firstly, the “conversation” that Oracle are saying “must go forward” is the proposed Java SE 7/8 JSRs which were not the focus of this part of the discussion. Secondly Oracle is on the record (in the past) as saying that the TCK licence issue is at conflict with the JSPA rules, but is now unwilling to fix it or even discuss it.

Instead it is apparent from the above and the rest of the discussion too, that Oracle has used the stagnation of the Java platform as leverage for JCP members to turn a blind eye on Oracle’s unwillingness to address the problems with TCK licencing the overall governance of the JCP.

I can’t help but be reminded of “The Empire Strikes Back” -

LANDO: You said they’d be left in the city under my supervision!

VADER: I am altering the deal. Pray I don’t alter it any further.

Given the current state of affairs, JCP members may find they are not equal negotiating terms with Oracle and have a difficult choice between two risky options. Stay with Oracle and hope they make favorable choices, or leave the JCP and hope for a change coming from the community or from the pressure back on Oracle.

Oracle declared at the start of the call that they were prepared to act with or without the ECs blessing. Those remarks should not have gone unchallenged.

The JCP is a sham.

No related posts.

This version is mostly a minor enhancement and fix release, but includes almost 6 months worth of patches based on production use of the library.

There are a number of fixes and enhancements based on user requests, and some new features and fixes based on my own work.

Note: this release separates the visural-common and visural-wicket library JARs.

Previously they were bundled together into a single JAR file, however this caused issues with release cycles for each library. However, I have included a visural-common-and-wicket-0.6.5.jar in the release package which leaves them bundled if you’d rather not have two JAR files attached to your project.

Enhancements

New Component: Client-side Tab-pages component

This is a javascript based tab control, which renders all tabs contents into the page at render time, allowing instant tab switching after render.

Unlike other implementations I’ve seen, this tab control retains tab state even when updating via Ajax.

It also has an intuitive implementation pattern that requires only adding a wicket:id to a single wrapping <div> tag containing the tab’s <div>’s.

Fancybox updated to latest v1.3.1

The new version of Fancybox (the image gallery component) has been integrated into visural-wicket, and it features an improved API with more options.

These options have been exposed in visural-wicket and javadoc has been included for each based on Fancybox’s API docs.

Example app includes InputHint example

The “InputHintBehavior” hasn’t changed at all, but there is now an example for it in the “live demo site” and examples application.

DropDown control changes

The DropDown control has had several fixes and improvements.

• The “Show All” / “Filter List” feature that would appear as the first line in the list is now optional and can be disabled.
• The “arrow icon” that appears at the right of the control can now be switched off. This may be desirable when working as a “suggest” or “auto-complete” style control, rather than a DropDownChoice style control.
• The drop list now appears as the width of the overall box and is positioned more consistently with the CSS styling of the control.

Dialog’s Open / Close Javascript methods now protected access

The dialog open and close Javascript generation methods are now protected. This allows developers to pre-fix or post-fix the open and close events with their own Javascript callbacks.

All Components – Allow disable of JS & CSS static header contributions

To assist with static resource packing and packaging all components now have a -

    /**
     * Override and return false to suppress static Javascript and CSS contributions.
     * (May be desired if you are concatenating / compressing resources as part of build process)
     * @return
     */
    protected boolean autoAddToHeader() {
        return true;
    }

Overriding, and returning false here will cause the component to avoid adding it’s resource contributions during construction. You are then free to repackage them as you see fit as part of your build process.

No longer necessary to manually add excanvas.js for BeautyTipsBehavior

Previously users had to manually add the ExCanvas.js header contributor for Internet Explorer support for the tool-tip behaviour – BeautyTips. This contributor will now be added automatically, so everything works “out of the box”. This won’t affect code that adds it manually, so it’s just a simplification for new users.

JQuery 1.4+ and other JS library compatibility

JQuery 1.4 comptability has been introduced and all components should now work under v1.4.

The JQuery resource reference now takes a version parameter and 1.3 and 1.4 versions of JQuery are included with visural-wicket out of the box.

For backward compatibility, if no version is provided 1.3 is used as the default. For existing projects this is no change.

Additionally, all of the generated code now uses “jQuery(…)” rather than “$(…)” so if you are using multiple libraries overriding the “$(..)” function then you should now be able to use visural-wicket in your project.

All components implement security interfaces and have a serialVersionUID

All of the visural-wicket components now implement ISecureEnableComponent, ISecureRenderComponent which lets them work seamlessly with the security features built into the library.

In additional all of the classes that may end up Serialized in a user session now have a serialVersionUID = 1, to reduce deserialization issues between releases.

Add German date form (dd.mm.yyyy) to DateInputBehavior

As per user request, the german-style date format of dd.mm.yyyy has been added DateFormat.DD_MM_YYYY_DOTS.

Bug Fixes

• Issues with rendering of modal dialog when using AJAX refreshes Fix #49 Fix #51
• Input Hint and InvokeClientSideFormSubmitHandler broken by Wicket 1.4.11 (Fix #58)
• Date Input popup can appear in the wrong location (Fix #59)
• Depending on DOM placement, modal dialog overlay doesn’t cover whole screen (Fix #60)
• Modal Indicator components do not appear above Modal Dialog (Fix #62)
• Can’t select bottom record in a DropDown (firefox)

So as you can see, this is a pretty decent release even if it was a while coming

Hope you find these changes useful, as always I welcome feedback, so if you have any ideas, questions or problems then please drop me a line.

Related posts:

1. visural-wicket 0.7.0 release with support for Wicket 1.5
2. visural-wicket 0.6 released – lots of new components!
3. Announcing – visural-wicket

In theory, it should be possible to do “backup” operation from Album 2.0 and import into it’s successor Photoshop Elements.

My research suggested that this may be hit-and-miss with several people experiencing bugs that rendered the backup useless on import. I tried – and experienced exactly the same issue.

Anyhow, after some consideration I decided to switch to Picasa, since it’s -

• free
• multi-platform
• has face recognition
• integrates with my Google account
• stores meta data inside the image files by default (i.e. captions & tags)

So the next challenge was – how do I get all the photos from Photoshop Album to Picasa with all the meta-data intact?

Step 1 – Get the meta-data out of Photoshop Album

Fortunately someone has already solved this part of the problem. I found this post here which links to a (no longer available) tool for reading the meta data out of Album’s (Access format?) ODBC data source.

I’ve cached a copy of the tool here – psatools.zip

I tried running the tool on the actual Photoshop Album database in the C:\Program Files folder, however it failed with a weird error. I found a workaround to that issue – export a full backup through Photoshop Album, and then run the PSATool on the backup file, e.g.

psatool /if-print /fp /xml "backup.psb" > output.xml

This will output the content of the Photoshop Album backup as XML with all the meta-data intact (captions, tags, albums). The “/fp” switch is required to output the full path to the original image file. We’ll need this for the following step.

Step 2 – Clean up, reorganise and apply meta-data to image files

There are two components required for this step.

One is exiftool - a command line tool for reading and writing meta data from image files.

The second is a custom Java program which I wrote (listed below). The Java program iterates through the XML meta data exported from the Photoshop Album backup and simultaneously reorganises (renames and copies) the images based on their captions and collections and adds this meta data (captions and tags) into the image using exiftool.

You’ll need a couple of libraries for the Java code to run -

• Javolution – a fast XML parser
• visural-common – my utility library which short-hands a lot of common stuff

You’ll probably also need to revise the code below to suit your own needs. It’s posted as a starting point only.

Be aware this is really rough and nasty code – it’s not intended to be used more than once.

It assumes that the output XML from step 1 is in “I:\Temp\output.xml”, the source picture files are accessible on the current PC and that you want to output to “I:\Temp\output” folder for the converted images.

import com.visural.common.Function;
import com.visural.common.Function;
import com.visural.common.StringUtil;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.util.HashSet;
import java.util.Set;
import javolution.text.CharArray;
import javolution.xml.stream.XMLInputFactory;
import javolution.xml.stream.XMLStreamConstants;
import javolution.xml.stream.XMLStreamReader;

/**
 *
 * @author Richard Nichols
 */
public class Main {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws Exception {
        XMLInputFactory factory = XMLInputFactory.newInstance();
        XMLStreamReader reader = factory.createXMLStreamReader(new FileInputStream("i:/temp/output.xml"));

        Set<Photo> photos = new HashSet<Photo>();
        Photo photo = null;
        CharArray blank = new CharArray("");

        while (reader.getEventType() != XMLStreamConstants.END_DOCUMENT) {
            switch (reader.next()) {
                case XMLStreamConstants.START_ELEMENT:
                    if (reader.getLocalName().equals("file")) {
                        // Reads primitive types (int) attributes directly (no memory allocation).
                        String type = Function.nvl(reader.getAttributeValue(null, "type"), blank).toString();
                        String name = Function.nvl(reader.getAttributeValue(null, "name"), blank).toString();
                        String caption = Function.nvl(reader.getAttributeValue(null, "caption"), blank).toString();
                        if (type.equals("photo")) {
                            if (name.toLowerCase().endsWith(".jpg") || name.toLowerCase().endsWith(".jpeg")) {
                                photo = new Photo(name, caption);
                                photos.add(photo);
                            } else {
                                System.out.println("Skipped photo: "+name);
                            }
                        }
                    }
                    if (reader.getLocalName().equals("folder")) {
                        if (photo != null) {
                            String type = Function.nvl(reader.getAttributeValue(null, "type"), blank).toString();
                            String value = Function.nvl(reader.getElementText(), blank).toString();
                            if (StringUtil.isNotBlankStr(value)) {
                                if ("Tag".equalsIgnoreCase(type)) {
                                    photo.getTags().add(value);
                                }
                                if ("Collection".equalsIgnoreCase(type)) {
                                    if (value.startsWith("\"")) {
                                        value = value.substring(1);
                                    }
                                    if (value.endsWith("\"")) {
                                        value = value.substring(0, value.length()-1);
                                    }
                                    photo.getCollections().add(value);
                                }
                            }
                        }
                    }
                    break;
                case XMLStreamConstants.END_ELEMENT:
                    if (reader.getLocalName().equals("file")) {
                        photo = null;
                    }
                    break;
            }
        }
        reader.close(); 

        // fix filenames & collections
        Set<String> collections = new HashSet<String>();
        int numTwo = 0;
        for (Photo p : photos) {
            // remove any unwanted collections here
            p.getCollections().remove("unwanted collection");

            if (p.getCollections().isEmpty()) {
                p.getCollections().add("No Collection");
            } else if (p.getCollections().size() > 1) {
                numTwo++;
            }
            collections.addAll(p.getCollections());
        }

        for (String s : collections) {
            new File("i:/temp/output/" + safeFilename(s)).mkdirs();
        }

        int n = 0;
        for (Photo p : photos) {
            n++;
            if (n%50 == 0) {
                System.out.println(n+" of "+photos.size());
            }

            String collection = p.getCollections().iterator().next();
            File i = new File(p.getFilename());
            if (!i.exists()) {
                System.err.println("MISSING FILE: "+p.getFilename());
                continue;
            }
            String outname = p.getCaption();
            if (StringUtil.isBlankStr(outname)) {
                outname = p.getFilename().substring(p.getFilename().lastIndexOf('\\'));
            }
            File o = new File("i:/temp/output/" + safeFilename(collection)+"/"+safeFilename(outname)+".jpg");
            if (o.exists()) { // allows continue after partial conversion
                continue;
            }
            copyFile(i, o);
            tagAndCapFile(o, p.getCaption(), p.getTags());
        }
    }

    private static String safeFilename(String name) {
        for (char c : StringUtil.FILENAME_INVALID_CHARS) {
            name = name.replace(String.valueOf(c), " ");
        }
        return name.trim();
    }

    public static void copyFile(File in, File out) throws IOException {
        FileChannel inChannel = new FileInputStream(in).getChannel();
        FileChannel outChannel = new FileOutputStream(out).getChannel();
        try {
            inChannel.transferTo(0, inChannel.size(),
                    outChannel);
        } catch (IOException e) {
            throw e;
        } finally {
            if (inChannel != null) {
                inChannel.close();
            }
            if (outChannel != null) {
                outChannel.close();
            }
        }
    }

    private static void tagAndCapFile(File o, String caption, Set<String> tags) throws IOException, InterruptedException {
        //System.out.println("Tagging and capping: "+o.getCanonicalPath());
        {
            String run = "lib\\exiftool.exe \"-iptc:caption-abstract="+caption.replace('\"', ' ')+"\" \""+o.getCanonicalPath()+"\" -overwrite_original";
            Process p = Runtime.getRuntime().exec(run);
            int i = p.waitFor();
            if (i != 0) {
                System.err.println("Failed running: "+i+"\n"+run);
            }
        }

        for (String tag : tags) {
            String run = "lib\\exiftool.exe \"-iptc:keywords+="+tag.replace('\"', ' ').trim()+"\" \""+o.getCanonicalPath()+"\" -overwrite_original";
            Process p = Runtime.getRuntime().exec(run);
            int i = p.waitFor();
            if (i != 0) {
                System.err.println("Failed running: "+i+"\n"+run);
            }
        }

    }
}

/**
 *
 * @author Richard Nichols
 */
public class Photo {
    private String filename;
    private String caption;
    private final Set<String> collections = new HashSet<String>();
    private final Set<String> tags = new HashSet<String>();

    public Photo(String filename, String caption) {
        this.filename = filename;
        this.caption = caption;
    }

    public String getCaption() {
        return caption;
    }

    public void setFilename(String filename) {
        this.filename = filename;
    }

    public String getFilename() {
        return filename;
    }

    public Set<String> getCollections() {
        return collections;
    }

    public Set<String> getTags() {
        return tags;
    }

}

After an hour or so while this buzzed away on 10000 odd pics I then ended up with a nicely arranged image collection with all the meta-data attached!

Related posts:

1. NetBeans 6.7 Broke My Parameterized Tests
2. ResourceTransformFilter – DataUris, LessCSS, Javascript and CSS Compression Made Easy!

Think something along the lines of “save draft” in a mail client like GMail.

It’s possible but requires a bit of Wicket-foo. I did post a short snippet on the mailing list, but I thought a longer explanation might be helpful.

First of all, let’s get something out of the way; it’s not possible to accept data that the model backing your components do not accept.

For example, you can’t accept “asdf” into a model field backed by an Integer – it ain’t going to work whichever way you slice it. (Though, you can accept any input if you build a model that consists only of Strings and add validation for numerics etc. further down the line).

Here’s the key steps to making a “draft mode” Wicket form -

1. Override the form’s process() method to set a “draft mode” flag based on the component that submitted the form.
2. Create validators that apply based on the mode that form is in.
3. You’re probably going to have multiple submit buttons on the screen which do different things, thus your form’s onSubmit() handler will probably do nothing.

This means that you can’t use Wicket’s built in validators as they are applied regardless of any notion of “mode” that you may have in your form.

Here’s an example of how this might hang together:

class MyForm extends Form {
    private final SubmitButton submitButton;
    private final SubmitButton saveDraftButton;
    private final TextField textField;
    private boolean draftMode = true;

    public MyForm(String id) {
        super(id);
        add(textField = new TextField("textField"));
        add(saveDraftButton= new SubmitButton("draft") {
            @Override
            public void onSubmit() {
                myService.saveDraft(); // or whatever
            }
        });
        add(submitButton = new SubmitButton("submit") {
            @Override
            public void onSubmit() {
                myService.submit(); // or whatever
            }
        });
        // works like a required field validator
        textField.add(new IValidator() {
            @Override
            public void validate(IValidatable validatable) {
                if (validatable != null && !MyForm.this.draftMode &&
                    validatable.getValue() == null || validatable.getValue().toString().equals(""))
                {
                    validatable.error(new IValidationError() {
                        public String getErrorMessage(IErrorMessageSource messageSource) {
                            return "Required.";
                        }
                    });
                }
            }
        });
    }

    @Override
    public void process(IFormSubmittingComponent submittingComponent) {
        draftMode = !(submittingComponent == submitButton);
        super.process(submittingComponent);
    }

    @Override
    public void onSubmit() {
        // do nothing
    }
}

This work can be standardised in your own sub-class of Form for example, with a bunch of validators that integrate with it.

Also, you can support even more complex use-cases by using a “mode” enum instead of a simple “draft” boolean. And make all the validators conditional on Mode.

This approach works just fine, but it would be nice if there was some sort of formal abstraction in Wicket to do this, since the current approach assumes quite a bit about the type of applications that will be implemented.

Related posts:

1. Implementing security in Wicket with visural-wicket
2. LessCSS available in Java and Wicket – less.js
3. visural-wicket 0.6 released – lots of new components!

Sometimes you need to get a “fully qualified” URL such as “/pages/something/mypage” or “http://localhost:8080/myapp/pages/something/mypage“.

Although there is nothing on the core Wicket objects to do this, there is a little utility included with Wicket which will allow you to convert a relative URL to an absolute URL – org.apache.wicket.protocol.http.RequestUtils

It’s dead simple to use – just call RequestUtils.toAbsolutePath(…) and pass in the relative URL as a string. Assuming the relative URL was generated from the current Wicket Page, it will convert from “../../something/mypage” to “http://localhost:8080/myapp/pages/something/mypage”

Handy!

Related posts:

1. Wicket – forcing page reload on browser back button…
2. Implementing security in Wicket with visural-wicket
3. Demystifying HttpServletRequest properties…

So far it has been doing a good job of making inversion of control / dependency injection approachable, light-weight and helpful for the whole team.

The biggest issue we’ve had with Guice has been unexpected bugs relating to object scoping misunderstandings.

These are bugs that originate from a mistake on the part of the developer, either forgetting or misunderstanding the scope of objects being interacted with and subsequently accessing the objects outside of their intended scope, or accessing the objects in a scope that turns out to be more expensive than intended (e.g. an object intended to be a singleton, being accessed with no scope and re-instantiated over and over).

What we’ve found is that as a standard it is best to use Guice’s annotation based configuration mechanism to configure objects in their “intended configuration”, rather than using “Guice modules”.

Let me give an example of the two mechanisms -

With module –

public interface MyService {
   void serviceOperation1();
   void serviceOperation2();
   void serviceOperation3();
}

public class MyServiceImpl implements MyService {
   @Inject
   public MyServiceImpl() {
   }
   public void serviceOperation1() {
      // ...
   }
   public void serviceOperation2() {
      // ...
   }
   public void serviceOperation3() {
      // ...
   }
}

public class MyServiceModule extends AbstractModule {
    @Override
    protected void configure() {
        bind(MyService.class).to(MyServiceImpl.class).in(Scopes.SINGLETON);
    }
}

With annotations –

@ImplementedBy(MyServiceImpl.class)
public interface MyService {
   void serviceOperation1();
   void serviceOperation2();
   void serviceOperation3();
}

@Singleton
public class MyServiceImpl implements MyService {
   @Inject
   public MyServiceImpl() {
   }
   public void serviceOperation1() {
      // ...
   }
   public void serviceOperation2() {
      // ...
   }
   public void serviceOperation3() {
      // ...
   }
}

These two examples are functionally identical, the only difference is how the Guice injector gets bootstrapped.

The common argument for Guice modules over annotations is that the application configuration is kept separate from the code, and can be changed in one central location. On the face of it, that seems like a pretty good argument.

In practice though, few objects can be arbitrarily rescoped without affecting the assumptions on which they were built, particularly where they have dependencies on other objects which have implicit scope. The only way to guarantee everything works as intended is to use Provider<MyDependency> everywhere to make sure things can continue working even if you monkey with the scoping in some unexpected way.

To me that seems like code-pollution which has rapidly diminishing returns.

The argument for @annotation based configuration is simple – understanding the intended scope and default implementation of an interface / object is an important signal to a developer reading the code for the first time. The best place for that information is in the code where the developer is reading it.

And best of all, this isn’t an either-or decision.

The annotation based configuration only specifies the default configuration of your Guice application. If you specify a different scoping or binding in a Guice module, it will override anything specified using annotations.

So with this approach you retain the flexibility (and explicit definition) provided by modules, but still solve the “developer usability” issue of in-your-face scoping/bindings.

Related posts:

1. Clustering Guice Java Web Applications
2. The 5 Minute Guice Primer
3. AOP, Annotation-based caching solution for Guice projects…

Exception in thread “main” java.lang.StringIndexOutOfBoundsException: String index out of range: -1

That’s easy right? You’ve passed “-1″ as an argument to a Java String’s “substring” method.

Wrong.

Here’s some code that gives the same error -

"fred smith".substring(4, 3);

See the bug? The start index is after the end index (by 1 character).

To me, this is pretty poor API design. Neither of the arguments are out of bounds of the String’s length, so why raise a StringIndexOutOfBoundsException?

Worse, the index reported is neither of the indices passed.

And finally, the exact same error message is reported for another common mistake, e.g.

"fred smith".substring(-1, 3)

Here’s the offending code in Java’s core String.java –

    public String substring(int beginIndex, int endIndex) {
	if (beginIndex < 0) {
	    throw new StringIndexOutOfBoundsException(beginIndex);
	}
	if (endIndex > count) {
	    throw new StringIndexOutOfBoundsException(endIndex);
	}
	if (beginIndex > endIndex) {
	    throw new StringIndexOutOfBoundsException(endIndex - beginIndex);
	}
	return ((beginIndex == 0) && (endIndex == count)) ? this :
	    new String(offset + beginIndex, endIndex - beginIndex, value);
    }

For what it’s worth, this should probably be an IllegalArgumentException, since the argument’s have nothing to do with being out of bounds of the string and are just illegal with respect to one another. This would also allow for a meaningful error message such as “Start index (4) is before end index (3)”.

It’s funny the stuff you find when fixing gnarly bugs

Related posts:

1. Implementing Facebook OAuth 2.0 Authentication in Java
2. Face Detection in Java – Haar Cascade with JJIL (how-to)
3. 301 Redirects Made Easy In Java

I prefer using the Kindle app on the iPad to read ebooks, and the Amazon Kindle book store seems to have the best price/range as well.

But I recently had a situation where I had bought a digital copy of Programming in Scala: A Comprehensive Step-by-step Guide which only comes in PDF and “.mobi” versions. .mobi is the format used by the Kindle, so I thought “woo hoo” this should be easy, right?

Wrong.

It’s not possible to get .mobi books into the Kindle app that weren’t purchased from Amazon right now. I did some googling and found a number of forum posts recommending iPad/iPhone file system tools to drop the ebook into the Kindle folder. Unfortunately on Windows I wasn’t able to get these to show the Kindle app’s folder.

So I tried looking for another solution, and although the iBooks app isn’t as good as the Kindle app for reading, I figured converting the .mobi book to an .epub book for iBooks would be an easier task.

As it turns out, it was! Here’s how to do it -

The same process can also be used to convert books from/to other ebook formats.

1. The key tool here is “Calibre” – an open source ebook management tool. Download a copy from http://calibre-ebook.com/
2. Install it, I’ll assume that you’re installing on Windows and with the default settings.
3. Start the program when installation is complete. When you’re prompted as below, choose “Apple” as your default reader.

   

4. Once it’s loaded, use the “Add button to add the .mobi (or other format) book to Calibre.

   

   It should then show up in the list of books.

5. Click the “Convert” button.

   

   When the conversion options dialog appears, use the default settings and click OK.

6. You should see a “Job” appear in the Jobs monitor in the bottom right hand corner of the screen. Wait for the book to be converted and the Job count to return to zero.

   

7. Right click the book and choose the “Save to disk -> Save only EPUB format to disk” option and choose where to save the resulting ebook.

   

8. The “.epub” book can now be dragged from the file system to iTunes and synced on the iPad.

Calibre seems to be another great open source tool that I’ve never heard of. It’s a shame the ebook distributors such as Apple and Amazon can’t  settle on a single format, and/or provide better interoperability between their formats/devices, but I guess vendor lock-in is still the modus operandi for the time being.

Anyhow, open-source ftw!

Related posts:

1. Thoughts on the iPad…
2. eBooks + DRM = Fail