Blog.Xam.de

Developing GWT Faster

2011-12-06T02:04:00.001+01:00

Xydra has a nice GWT module. Compiling it was too slow. How to speed it up?

This is the short version of what happened:

After exploring the easy ways we went from 5 min compile to 1,5 min. Not bad, but still too slow.

We tried to get (hosted) development mode to run with GWT and Maven. A bit tricky. Maybe -noserver (doc is for GWT 1.6) helps.

That worked up to a certain point. We reached a strange exception from the development mode GWT pseudo-compiler (then one that communicates with the browser-plugin). The error was something about gwt-log. So maybe gwt-log is not compatible with our GWT version? So we wanted to upgrade GWT. But we did not use the GWT version consistenly. So we moved it to a new parent pom. Can you have conditional sections in the parent pom that you turn on in the children, when needed? After consulting Stackoverflow:

Q: Can Maven compose a pom out of multiple parts? A: Only, if you're clever. It will be supported well in Maven 3.1.
Q: Will 3.1 come out soon? Rather not.
Q: Is it worth upgrading to Maven 3.0? Yes, because its faster. And compatible with 2.0 poms. And the error reporting is much clearer.

Ok. So we upgraded to Maven 3.0.3. And we use profiles in a parent pom. This means, the files "uses-gwt.marker" etc are now really important. Yes, it is much faster!

So now that we were in the pom-business, we upgraded all kinds of libraries. Minor upgrades:

Upgraded from SLF4J 1.5.0 to 1.6.4, which has some bug fixes. Details at http://slf4j.org/news.html.
Upgraded maven-antrun-plugin from 1.2 to 1.7
Upgraded commons-io from 1.4 to 2.1
Upgraded to gwt-log 3.1.6, just in time they deploy to Maven Central now. Of course, like every good (*cough*) maven plugin, they changed the groupId.
Upgraded from JUnit 4.8.2 to 4.10
Upgraded Log4j from 1.2.14 to 1.2.16 - seems to be a pretty stable lib.

Following the main goal, we also upgraded the GWT and GAE monsters:

Upgraded to Google AppEngine 1.6.0 with the compatible maven-gae-plugin 0.9.2. Cool, its in Maven Central now. Yes, one repo less to use!
Changed from GWT 2.x to GWT 2.4. Also using gwt-maven-plugin 2.4.0.

... some hours later...after updating many many poms... hosted mode runs!

The app still runs. And much complexity from the poms is gone and has been moved to the parent pom.

State of GWT Widgets, Themes, and Styles

2011-11-18T19:47:00.001+01:00

We used GWT some years ago and are now starting to use it again for our web and mobile apps. Before diving head-first into code we did some research on the market for GWt widgets, themes or styles. Overall, the situation is not so promising.

Themes:

How to use themes: http://code.google.com/intl/de/webtoolkit/doc/latest/tutorial/style.html
How to style widgets: http://code.google.com/intl/de/webtoolkit/doc/latest/DevGuideUiCss.html
GWT (since version 2.x) comes with 3 default styles: Standard, Chrome and Dark. They look like this (Screenshot).
Since Feb 2011: Clean theme. Demo: Default widgets in Showcase sample app

Widgets:

Widget gallery in Standard theme
Same widgets in Showcase sample app (Clean theme)
Experimental widgets from GWT team
Sencha GWT = ExtGWT, cool look with effects for $330 per developer | Demo - 4 themes, choose top right
Why are there not more widget sets here?

Mobile widgets

gwtmobile.com. Demo: Desktop browser: demo.gwtmobile.com | Mobile browser: demobile.gwtmobile.com. Supports PhoneGap. (from guy who build gwt-phonegap wrapper)
gwt-mosaic | 1 developer, last commit July 2011, some real-world usage, no live demo
Sencha Touch -- there IS a free license | Demo
NEXT very nice, but seems rather unknown | Demo looks very mac-ish

Helper frameworks

For using MVP: http://code.google.com/p/mvp4g/ OR http://code.google.com/p/gwt-platform/
Gquery is like jquery, but in Java

Ten Finalists of German Silicon Valley Accelerator

2011-11-04T18:09:00.003+01:00

Today, the ten finalists for the German Silicon Valley Accelerator (GSVA) have been selected. As they list only the company names on their blog, lets have a shorts view on which teams qualified. I've only quickly copied some stuff together so spare you the clicking. Obviously You should have your company claim and motto in copy-and-paste form on your homepage to get better press coverage :-)

Eviscan GmbH from Koblenz, 2 founders

Claim: "Spurensuche und Sicherung leicht gemacht. Mit EVISCAN!"

Fair Observer, 44 listed team members, unclear legal form

Claim: Fair Observer is a platform for shared understanding of our world through analysis sourced from across borders and disciplines.

Friendsurance from Alecto GmbH in from Berlin,

Claim: Versichere dich über Friendsurance und sichere dir im Schnitt 50% Rückzahlung deiner Beiträge zum Jahresende.

HPI Remarketing GmbH from Sulzbach
Claim: Mit dem Fullserviceangebot von HPI Remarketing sind teure Restwertfresser und lange Verkaufszeiten Schnee von gestern.

Mashero GmbH from Berlin
Claim: Erfolg durch zielgruppengenaue Gestaltung und Optimierung von Online-Werbung

Nelou from gestyled.com gmbh in Berlin
Claim: Best of independent style.
Mission: nelou ist ein innovatives Off- und Online-Konzept, das Mode, Shopping und Unterhaltung zu einem neuen Erlebnis verbindet. Dank nelou vernetzen sich endlich professionelle Modedesigner und interessiertes Publikum auf einfache Art und Weise – weltweit und rund um die Uhr. Und dank der regelmäßig stattfindenden nelou Events und Shows werden Kreative zum Anfassen nah.

Pixray founded as a HIT Labs UG in May 2009 by 3 founders
Claim: Manage and contraol digital image content.

Mission: PIXRAY stands for cutting edge research and technology in the field of computer vision. Our unique PIXRAY Engine™ revolutionizes the way companies handle their digital image content. PIXRAY's fully automated image processing solutions alter businesses notion of dealing with their image assets in an effective and reliable way. We provide the answers to industry-specific questions in the image processing and object recognition domain.

Signavio GmbH from Berlin
Claim: Prozesse verstehen, optimieren und umsetzen. Dies erreicht man mit professionellem Prozessmanagement – und professionellen Werkzeugen.

Trifense from Velten, Spin-Off from TU Berlin.
Claim: Intelligent network defense

Ubivent from Mannheim

Claim: Europe's leading virtual event specialist.

Analysis of datastore operation cost for Google AppEngine for the new pricing

2011-11-03T00:54:00.001+01:00

AppEngine new pricing is making me think how to reduce costs. Here is my analysis of the datastore operation costs:

[Prices based on http://code.google.com/intl/de-DE/appengine/kb/postpreviewpricing.html#similar_vm]

High-Level Operation	Cost (Low-Level Operations Required)
Entity Get (per entity)	$0.07 per 100k operations (1 Read)
New Entity Put (per entity) with k indexed property values and c composite index values	$0.20 + k $0.20 + c $0.10 per 100k operations (2 Writes + 2 Writes per indexed property value + 1 Write per composite index value) Example with k=5 and c=1: $0.20 + 5 * $0.20 + 1 * $0.10 = $1.30 per 100k operations
Existing Entity Put (per entity) with k indexed property values and c composite index values	$0.10 + k $0.40 + c $0.20 per 100k operations (1 Write + 4 Writes per modified indexed property value + 2 Writes per modified composite index value) Example with k=5 and c=1: $0.20 + 5 * $0.40 + 1 * $0.20 = $2.40 per 100k operations
Entity Delete (per entity) with k indexed property values and c composite index values	$0.20 + k $0.20 + c $0.10 per 100k operations (2 Writes + 2 Writes per indexed property value + 1 Write per composite index value) Example with k=5 and c=1: $0.20 + 5 * $0.20 + 1 * $0.10 = $1.30 per 100k operations
Query with r entities returned	$0.07 + r $0.07 per 100k operations (1 Read + 1 Read per entity returned) Example with r=100: $0.07 + 100 * $0.07 = $7.07 per 100k operations
Query (keys only) with r entities returned	$0.07 + r $0.01 per 100k operations (1 Read + 1 Small per entity returned) Example with r=100: $0.07 + 100 * $0.01 = $1.07 per 100k operations
Key allocation (per key)	$0.01 per 100k operations (1 Small)

Are my example values helpful?

Notes from Google AppEngine User Meetup at Barcamp Karlsruhe

2011-06-20T17:14:00.001+02:00

Notes from Session Google AppEngine User Meetup

at Barcamp Karlsruhe

on 19.06.2011

Attendants:

Sebastian Helzle | @sebobo, pyhton and java
Tobias Günther von Elaspix , java
Tobias Müller, 1&1 @tobiasmueller, mostly python
Max Völkel, Calpano (host) @xamde | Calpano.com, java

Topics & Tips:

Use the cookbook: http://appengine-cookbook.appspot.com/

Exception handling can be tuned nicely

to deliver errors over XMPP to GTalk
to include relevant details such as userId involved in error

Nice frameworks:

Objectify (mature, small, fast)
Xydra.org - less mature, more features (versioning, access rights, synchronisation, ...)

Common issues:

Privacy regulation in Germany make it hard/impossible to use GAE for German customers (depending on the kind of data to be stored)
Deadline exceeded error - Solutions:

TaskQueue, now easier to use with Deferred Task, also in Java
Caching (static variables, mem cache, datastore)
Asynch & batch operations (more concurrency, less datastore hits)
Report generation - commons solutions for this special case:

use bulk loader to download datastore, compute locally
use task queue to generate report cleverly in pieces, deliver in 30 seconds
use Backends (a new GAE feature) for this

Nice tools:

JRebel for debugging
A local embedded Jetty for debugging
AppStats!

Hint: Go to Google developer days and talk to real people (some come form the US GAE team, some come from partner companies)

--
max@xam.de | http://Xam.de

Was jeder SaaS-Unternehmer wissen muss

2011-02-25T14:04:00.001+01:00

Beim Schreiben unseres Business Plans für Calpano.com haben wir sehr viel Material zu Saas-Metriken, Preisgestaltung, Freemium-Modellen und viralem Marketing gelesen. Es gibt relativ wenige richtig gute Inhalte. Hier sind sie:

Der Blog http://www.forentrepreneurs.com/ enthält alles, was man über SaaS-Metriken wissen muss
Unter http://www.neildavidson.com/dontjustrollthedice.html findet sich ein gutes E-Book mit ca. 50 Seiten leicht verdaulichem und trotzdem sehr umfassendem Material zum Thema Preisgestaltung.
Unter http://sixteenventures.com/blog/the-reality-of-freemium-in-saas.html wird das Freemium-Modell sehr kritisch beleuchtet. Sehr kompakt, sehr gehaltvoll.

Welches andere Material kennt Ihr? Bitte kommentieren.

MediaMonkey rockt!

2011-01-25T22:33:00.000+01:00

Nach langen Jahren des kostenlosen Nutzens habe ich es jetzt endlich gekauft. Nun kann ich noch bessere Auto-Listen anlegen mit denen ich Musik entdecken kann.

Studienarbeit oder Diplomarbeit: Was jeder Student wissen sollte

2011-01-13T15:32:00.001+01:00

In meine Karriere als Doktorand habe ich ca. 10 Studien- und Diplomarbeiten betreut. Dabei habe ich jede Betreuung damit angefangen, dass ich meinen Studenten das Buch Studien-Arbeiten [Amazon-Link] in die Hand drückte. Meine Studenten waren sehr zufrieden und stellten die ganzen Anfänger-Fragen nicht mehr (Was ist eine Studienarbeit? Was ist wichtig? Wie kann ich die gliedern? Wie gehe ich mit Literatur um? ...).

Das Buch ist auf jeden Fall absolut gleichwertig für Diplomarbeiten zu empfehlen.

Is memory the new database?

2010-12-30T10:28:00.001+01:00

The current episode of SE Radio (Episode 169: Memory Grid Architecture with Nati Shalom) made me think. Is this the end of databases? In so many uses cases, all data will fit easily into memory, so persistency can be achieved by simply sending the data also to a second and third location where it can also live in memory.

How to delete all data in the Google Appengine datastore with Java tools

2010-09-10T02:47:00.003+02:00

A nice subtitle would have been: "The power of frustration". But in the end, hey, I solved it. Click the title of the post to learn how.

Lumix DMC-TZ10

2010-08-15T13:01:00.001+02:00

Since some weeks I own a Lumix DMC-TZ10 (US name: DMC-ZS7). The camera has a built-in GPS-module, but...

The GPS is quite slow, it takes several minutes to get a fresh location info
The GPS is weak, it takes a clear sky view to get any location info. It does not work indoors. It works sometimes close to a window.
If the camera has now fresh location info, it just puts the last location info it got into the EXIF metadata headers in the pictures.

I went to a hiking trip and took many pictures (ca. 600). How could I find out the pictures with correct GPS data and the wrong ones? How could I automatically fix the wrong ones?

Algorithm:

Sort pictures by EXIF:DateTimeOriginal.
Cluster pictures by EXIF:GPS-Latitude and EXIF:GPS-Longitude. The earliest picture (according to EXIF:DateTimeOriginal) is the only one with correct GPS info.
Let A be the location of the first picture in a cluster. Let B be the location of the first picture in the following (by DateTimeOriginal) cluster. Move each picture C in the first cluster (except A) from A towards B on the line AB. How far to move? Compute the time difference between A and C and normalise by time difference A to B. This yields a factor between 0 and 1 and gives the percentage how far to move C. Moving is simple: C.lat = A.lat + factor * (B.lat - A.lat), similar for longitude. Yes, this assumes a constant moving speed between two GPS located photos which in reality is not really true.

If you are interesed in more details about the Java code or the resulting GPX files (original and fixed), just email me.

Java copies files 80 times fast than commercial backup software

2010-06-28T22:13:00.000+02:00

I cannot believe what I saw today. I bought a brand new hard drive, a 500 GB 2.5" USB hard disk. It's a Western Digital My Passport Essential. It was about 15 Euros more expensive than a another 500 GB 2.5" USB drive from Western Digital - so it must be the software. I thought: Okay, let's try this backup software, I really need one.
So I installed everything and ran my first backup. After almost half an hour I took a look at the result. I was shocked. At this speed, the full backup would take days! I startet a self-written Java program called syncr with the sole purpose to copy files fast. Syncr was over 80 times faster. Of course, the comparison is not entirely fair. The software from Western Digital, called WD SmartWare 1.2.0.8, Copyright 2009 by Western Digital, can do much more than my hand-written syncr. The table below lists all differences. Nevertheless, 80 times is really a lot.

Syncr uses very few ticks to achieve this speed:

It does not copy redundant information - However, the speedup was measured by those files that were copied

It skips redundant folders with these names Temp", "RECYCLER", "System Volume Information", "Google Desktop"
It always skips files named "hiberfil.sys", "pagefile.sys", "Thumbs.db", "autorun.inf", "UsrClass.dat", "UsrClass.dat.LOG", "ntuser.dat.LOG","ntuser.dat", "parent.lock"
It does not copy files that are already present at the target drive with the same name and change date

It writes some kind of directories directly into a zip file in the target drive - this is much faster than copying first and zipping later and even faster than not zipping. Many small files cost a lot of file system operations. By default, only Eclipse projects and Eclipse workspaces are zipped.Overall, not many files are zipped.
It uses Java NIO, which is faster than the old Java.io.

All in all I cannot explain the slow speed of WD SmartWare nor the fast speed of Syncr. As a side remark: Syncr is also faster than the Windows Explorer (which is why Syncr has been written initially).

Quick facts about Syncr:

Files: 1
Lines of code: Less than 600, including comments
License: BSD
Dependencies: slf4j for logging
Source code: http://code.google.com/p/syncr/source/browse/trunk/de.xam.syncr/src/main/java/de/xam/syncr/Syncr.java

Because of all this, I released Syncr into the wild today. Enjoy.

Feature	WD SmartWare	Syncr
Business model	Commercial	Open source (BSD)
Front-end	graphical	Java source code, run from Eclipse
Installer	Windows, Mac	None
Languages	28	English only
Usability	very easy	easy for Java developers of any level, impossible for non-developers
Design	Aesthetic visualisation of current hard drive content	Accurate log messages
Backup storage format	Special folder system that contains the original files plus additional .dcm files	Original folder structure with original files. Some folders are automatically stores as .zip archives.
Product URL	http://www.wdc.com/en/products/wdsmartware/	http://code.google.com/p/syncr/
Versioning (can get back an older version of a file)	Yes	No
Incremental backup	Yes	Yes
Constant incremental background backup	Yes	No
Experimental results	Experimental results	Experimental results
CPU usage	20-40%	20-40%
Files created to backup 100 files	ca. 200	100 or just one zip file
Memory usage	164 MB	10-400 MB
Files copied after first 25 minutes	4200 files	5152 files (1 resulting zip file counting as 1 file)
Data copied after first 25 minutes	136 MB	11 GB
Estimated time for a full backup (of my 88 GB)	11 days, 5.6 hours	3.4 hours

Unifying Slf4j, gwt-log, GWT, AppEngine, Part II

2010-06-17T22:17:00.000+02:00

After some attempts to GWTify the existing slf4j-api it runs out, the approach of using linking API and implementation will not work in GWT, because it relies on reflection.

Hence, we use a primitive solution without any magic. The API provides the class LoggerFactory and a log implementation is required to register itself via a static method.

Finally I managed to write a first working code base for this. A description about how to use it can be found here.

Unifying Slf4j, gwt-log, GWT, AppEngine, Part I

2010-06-17T11:13:00.000+02:00

I like the API of SLF4J very much and used in all my code so far.

Can I re-use the same API for code that runs via GWT in the browser?

Experiment:

Check out source code from http://github.com/ceki/slf4j from trunk on 2010-06-17
Add GWT dependencies in Maven, add get-maven plugin in POM
Create a dummy entry point and a get.xml file
Run mvn gwt:compile

Result:

[INFO] Compiling module org.slf4j.GWTTest
[INFO]    Validating newly compiled units
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/LoggerFactory.java'
[INFO]          [ERROR] Line 28: The import java.net cannot be resolved
[INFO]          [ERROR] Line 207: The method getClassLoader() is undefined for the type Class
[INFO]          [ERROR] Line 217: URL cannot be resolved to a type
[INFO]          [ERROR] Line 217: URL cannot be resolved to a type
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/helpers/MessageFormatter.java'
[INFO]          [ERROR] Line 27: The import java.text cannot be resolved
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/MDC.java'
[INFO]          [ERROR] Line 74: No source code is available for type java.lang.NoClassDefFoundError; did you forget to inherit a required module?
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/MarkerFactory.java'
[INFO]          [ERROR] Line 53: No source code is available for type java.lang.NoClassDefFoundError; did you forget to inherit a required module?
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/helpers/BasicMDCAdapter.java'
[INFO]          [ERROR] Line 47: No source code is available for type java.lang.InheritableThreadLocal; did you forget to inherit a required module?
[INFO]       [ERROR] Errors in 'file:/O:/2010/slf4j-api/src/main/java/org/slf4j/helpers/NamedLoggerBase.java'
[INFO]          [ERROR] Line 40: No source code is available for type java.io.ObjectStreamException; did you forget to inherit a required module?
These errors show that SLF4j is not GWT-ready by itself. Now there are two options:

Create a drop-in replacement that implements some parts differently
Create a new SLF4j-like API which IS GWT-ready

The second option is probably less work, but the first option would be easier to use.I'll keep you updated.

2nd workshop on Personal Knowledge Management

2010-05-17T15:36:00.000+02:00

Call for papers: 2nd workshop on Personal Knowledge Management at Mensch und Computer 2010, September 12, Duisburg, Germany.

June 13, 2010 (Sun.) Deadline for paper submission
July 4, 2010 (Sun.) Notification to authors
July 14, 2010 (Wed.) Camera-ready copy due
September 12, 2010 Workshop

Runtime optimisation on AppEngine for Java (GAE/J)

2010-05-13T21:03:00.004+02:00

1. Install AppStats from http://code.google.com/appengine/docs/java/tools/appstats.html
Its basically a servlet provided by Google that intercepts GAE inner workings and logs it to your applications data store. It comes with a servlet that is able to display the data in nice ways. Leave all settings as described.

2. Log into the appstats servlet and keep the browser open

3. Open the Firebug network tab

4. Make a request to your appengine application

5.In the response headers: You see messages about the cost and CPU usage. The cost is for 1000 requests, the usage numbers are for this request. Sample:

X-AppEngine-Estimated-CPM-US-Dollars: $0.003959 X-AppEngine-Resource-Usage: ms=172 cpu_ms=156 api_cpu_ms=40

Google explains:
If you access your site while signed in using an administrator account, App Engine includes per-request statistics in the response headers. The header X-AppEngine-Estimated-CPM-US-Dollars represents an estimate of what 1,000 requests similar to this request would cost in US dollars.

6. Go into the Appstats and click to see your last request. You get a timeline diagram and each point you called the data store. Click on these points to see the stack trace that called them.

7. Use Firebug to determine the page loading time from a users point of view.

Our appengine application remains for ca. 20 seconds active before the next request causes another cold start (clearly distinguishable by page loading times; time measures with stop watch by hand). A cold start every 20 seconds is in the worst case 4320 cold starts per day. Given a cold start cost of $ 0,00377529 for 12 seconds startup time, we end up with ca. $ 49 per month. All other request hit certainly a warm JVM and are hence much cheaper.

There is also a little Firefox plugin to display the costs in the Firefox error console.

Open Letter to Alfresco [update 2]

2010-03-21T12:31:00.003+01:00

Original Post (see updates at the end):

Dear Alfresco,

I am trying to buy something from you. I downloaded the source code from your site, an I am pleased with what I see. In order to embed JLAN in my product, I need to know the price of your product. I want to embed your JLAN prodcut [1]. On the product site you write:

Alfresco offers either an end user and developer license that allows companies to use Alfresco JLAN for internal deployments, or an OEM license that allows you to add Alfresco JLAN features and functionality to your products. For more information please contact jlan@alfresco.com.

Ok, so I wrote an email to that address, asking for a quote. No response.
I also asked the same question at the German sales email contact as listed at [2], twice. Additionally, I called the German hotline number, twice, but only got an answering machine at normal business hours. Also no reaction from the US sales email, twice.

After these seven (!) attempts to just get a price quote from you, I feel lost.
But customer service can get even worse!
Now I get emails like these:

Besuchen Sie Alfresco auf der CeBIT
Doing More with your Alfresco Trial. 30-Day Full Enterprise Download Trial.
Webinar: Alfresco-SAP-Anbindung in der Praxis, 18 März 2010 um 16:00
Upcoming Webinars: Alfresco Share Customization and SAP Integration

Apparently you have neither an idea what language I speak, nor a clue what I want from you. I do not care about SAP integration.

Why do you spam your potential customers instead of selling your software?

[1] http://www.alfresco.com/products/aifs/
[2] http://www.alfresco.com/about/contact/

Regards,
Max Völkel

Update 1:
On 22.03.2010 at 10 am, I got an Email from Alfresco with various contact details. Whatever caused their hickup, the Web 2.0 seems to have worked better than their internal structure, this time. Maybe more and more future communication will turn in Web 2.0 social network communication?

Unfortunately, I cannot predict my future sales numbers and had to learn that OEM business works not as I expected it. I expected a simple price table for 1-10 licenses, 11-100, 101-1000 and another per-license-price for more than 1000. This is not the case. Alfresco wants to calculate my business case, determine how much of the value is contributed by their software and then come up with a quote. This makes sense for them to obtain a fair price for their software. On the other hand, for smaller players like me, this means I have no way to start using Alfresco software.

As all the code is open source anyway, all they would need to do is set up a contract valid for, let's say, up to 10.000 licenses and let me pay by credit card for the number of licenses used at the end of each quarter.

Right now I could secretly take the source code, use it, and get sued (rightly so!). In the future, I could pay for a number of licenses, and if I would not have paid enough, get sued in the same way. So there is almost no overhead for them to earn more money. They can just set up self-service web-shop for licenses and continue to sue people without proper licenses.

Does anybody knows of another CIFS/Samba server written in Java that can be embedded for a reasonable license fee even for a small or unknown number of licenses?

At the end of the week, Alfresco will tell me more details about licensing options with them, I'll keep you updated.

Update 2:
Today, 26.03.2010, we received our final verdict from Alfresco: We are too small to talk too. Of course, they did not use these words. But they are definitely not interested in negotiating a deal with unknown results, most likely because their negotiation process is too costly. Their product is really nice, they have a SMB/CIFS+WebDav-server in Java called JLAN and fits our current APIs really well. But since its either GPL (killing all our potential business models) or a minimum deal of ca. 10.000 Euros, we'll now need to go ahead an continue to implement our own WebDAV server. So SemFS will not have CIFS support in the near future, but we will open-source (BSD license) our WebDAV server. We looked at Slide (project is dead), JackRabbits WebDAV (really complicated OO-architecture with way too many interfaces), and other.

Logging in the Google AppEngine for Java (GAE/J) with slf4j, log4j and jul - and using Maven

2010-03-03T10:50:00.001+01:00

Logging is an invaluable tool for debugging web applications, especially if you can't run them in debug mode. Developping an application on Google AppEngine allows you to do these steps prior to uploading it to the real server

Run local JUnit tests (alhough that has its own problems if you use Jersey framework and an embedded Jetty)
Run the application or the JUnit tests in debug mode (e.g. use mvn gae:debug and a remote debug session in Eclipse)
Use logging

On the real appengine, you cannot run JUnit tests (at least I don't know how. Maybe you can somehow.) and you cannot run a remote debug session. Therefore you need logging. It's your only way to debug things that are different on the real appengine compared to the local test stub environment. And there are differences.

AppEngine uses java.utils.logging (also often called j.u.l. or just jul). As the blog from Fred Sauer (the man behin gwt-log) shows, the AppEngine web console nicely displays the differen log levels:

DEBUG
INFO
WARN
ERROR
CRITICAL

AppEngine uses jul for logging, although ironically jul defines different log-levels, namelly:

FINEST (lowest value)
FINER
FINE
CONFIG
INFO
WARNING
SEVERE (highest value)

The mapping to the GAE levels is not completely clear to me yet.
To add more confusion to the mix, most code projects that I start today use slf4j, the Simple Logging Facade for Java. Slf4j allows to delegate to another logging framework and involves almost zero runtime overhead. Underneath I usually use log4j, an old, mature logging framework.

Clickable hyperlinks in log output
Log4j has nice things such as a PatternFormatter. If you use "(%F:%L)" in your pattern, you get "(Customer.java:33)", which your Eclipse console parses and turns it into a clickable link to the java file "Customer.java" in line 33. That is a very quick way from an unexpected log message to you fixing it: One click!
Be aware that line numbers in source code are rather expensive to generate, you should therefore not use this pattern in a production environment. However, you can simply use another confguration file, how nice.
There are other reasons for using framework such as log4j instead of just jul.

Situation
In my project I have this (not uncommon) situation:

GAE uses jul
Jersey uses jul
Other bundled libraries and my code use slf4j

Goals

Clickable hyperlinks for local testing
Nice mapping from log levels to GAE icons
Same dependencies for local testing and production deployment

Failed Attempts

Use slf4j-jdk14 to route all log output to jul. This works and nicely uses the five different categories on the AppEngine, but at the cost of getting a default ugly pattern. Jul has no PatternFormatter.
Write a custom formatter for jul. This does not work, because a custom formatter can only be set to a known handler, such as the ConsoleHandler form jul. Unfortunately, the ConsoleHandler does not exist on AppEngine. Attemtps to get the existing handlers by creating a log and listing its handlers or parents failed: Its all emtpy on appengine.

Chosen Solution

Use log4j with PatternFormatter
Log4j put log messages to Sysatem.out which in turn are rendered by AppEngine as INFO level.
Jersey and AppEngine still use all five levels and an ugly pattern.

Configuration
This is the main piece of this long post, because it didn't get it right first.

In \src\main\webapp\WEB-INF\appengine-web.xml   you need to have

    
    <system-properties>
        <property name="java.util.logging.config.file" value="WEB-INF/java-util-logging.properties"/>
    </system-properties>

to tell GAE where jul is configured.
In \src\main\webapp\WEB-INF\java-util-logging.properties you need

.level = ALL

or another of the jul level names, as you like (i.e. 'TRACE' does not work).

In \src\main\resources\log4j.properties (if you put the file as usual in \src\test\resources\log4j.properties, it won't be deployed to the production server and hence you will have no logging there). In this file you will have

log4j.rootLogger=ALL, console

or a lower log level such as DEBUG, INFO or WARN.
And you can customize your output like:

log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%-5p: %m at %C.(%F:%L) on %d{ISO8601}%n

You may not use FileAppenders, as GAE does not allow to write files.
That's it. here is an overview how log messages will flow:

generates its own log messages and sends them to jul
GAE checks if this level should be displayed (java-util-logging.properties), if yes:
message appears at correct level with ugly pattern in GAE log

Jersey

generates log messages and sends them to jul
GAE checks if this level should be displayed (java-util-logging.properties), if yes:
message appears at correct level with ugly pattern in GAE log

Other libraries and your code

sends log messages to slf4j
slf4j sends them to log4j
log4j checks if this package & level should log (log4.properties), if yes:
log message is formatted nicely
and sends to System.out which is interpreted as a jul-messages at INFO level in GAE
GAE checks if INFO level should be displayed (java-util-logging.properties), if yes:
messages shown with nice pattern at INFO level in GAE log, real level represented as text. E.g. a complete log message then looks like this:
03-02 02:05PM 23.948
[myappname/1.340252283834355681].<stdout>: DEBUG: this is debug at org.example.app.Logtest.(Logtest.java:13) on 2010-03-02 22:05:23,947

So in the end you have 3 barriers for a log message until you see it:

Log4j
GAE's jul
The drop-down box in AppEngine manager app (Show only requests will show everyhing including requests with no errors).

I hope this helps other people struggling with logging an maybe somebody has an even nicer solution.

Fifth Workshop on Semantic Wikis – Linking Data and People [SemWiki2010]

2010-02-26T17:23:00.002+01:00

Paper Submission: ~~26th February 2010~~ Extended: 7th March 2010.

Google AppEngine Java - The Essential Links

2010-02-17T17:30:00.003+01:00

I keep searching for the same websites over and over again. This is my collection essential links for Google AppEngine for Java. Using maven and low-level data-access only.

Provided by Google

http://code.google.com/p/googleappengine/ - on googlecode

http://code.google.com/appengine - product homepage
http://code.google.com/intl/de/appengine - product homepage, too

Low-level data access

GAE/J Google Group

Provided by third parties

The maven-gae-plugin - it works great. Use version 0.5.3 or higher.
GAE/J more complete class whitelist

Getting started with maven-gae-plugin and GAE/J 1.3.1:
1) Installing

Download early access, e.g. to C:\
Go to the download directory and type in a command prompt (as a single line):
mvn install:install-file -DgroupId=com.google.appengine -DartifactId=appengine-java-sdk -Dversion=1.3.1 -Dpackaging=zip -Dfile=appengine-java-sdk-1.3.1.zip
Use a local pom like the one linked here
Go to your projects root dir and call "mvn gae:unpack"

2) Using

Just execute the maven goals listed at the maven-gae-plugin page

e.g., mvn gae:run runs the local appengine. It runs now, but I still have issues to find my local servlets...

iPhone gegen Kratzer schützen

2010-02-17T16:41:00.000+01:00

Es gibt viele Display-Schutzfolien und iPhone-Hüllen. Die meisten iPhone-Hüllen sind unnötig klobig. Eine relativ gute Lösung ist die Folie von "invisibleSHIELD" die man in Deutschland am günstigsten, schnellsten und bequemsten über Amazon bekommt: invisibleSHIELD Apple iPhone 3G / 3Gs (Full Body)

Die Folie schützt dabei das Display gegen Kratzer, klebt aber auch auf der Rückseite und macht so insgesamt das iPhone griffiger und kratzgeschützt.

Das Auftragen der Folie ist allerdings eine ganz schöne Friemelei. Vorher unbedingt Hände und Telefon 100% fettfrei machen, das ist am wichtigsten. Dauer eine knappe Stunde, bis alles so klebt wie es soll. Aber danach hält es auch richtig gut.

Fernskiwanderweg Schonach-Belchen (100km) in 3 Tagen

2010-02-17T00:39:00.000+01:00

Letztes Wochenende sind wir drei Tage im Schwarzwald unterwegs gewesen. Jeden Tag habne wir zwei Etappen gemacht. Da sich Mittags auch das iPhone stets laden durfte, konnte ich den Track komplett aufzeichnen. Das Ergebnis ist zu sehen auf

EveryTrail (mit Höhenprofil und Google Earth .kml download)
Hik (mit Fotos)
und hier ist die offizielle Karte

Alles in allem eine wunderbare Tour, man braucht drei Tage lang nur den orangenen Schildern zu folgen, sehr angenehm und perfekt gespurt. Also los, solange es noch genug Schnee gibt...

SemFS making progress

2010-02-16T22:47:00.000+01:00

Today I refactored SemFS (our semantic file system) heavily. Now at least the VirtualFileSystem API is beautifully documented with nice clickable dependency graphs and stuff. WebDAV server implementations are really nasty.

The one in Jakarta slide is weird and discontinued.
The implementation from JackRabbit is full of funky factories, providers and workspaces.
The server in Alfresco is LGPL and huge.
Do you know another decent WebDAV server in Java where we can hook our own semantics underneath?

For WebDAV clients, there is the nice sardine project which has everything you want (besides a pom - inside joke).

SoftwareArchitekTOUR (Podcast)

2010-02-16T12:21:00.000+01:00

Eine schöne deutsche Einführung zu REST gibt es in Folge 17.

iPhone Wiki Usability

2010-01-30T14:57:00.000+01:00

This week I started to experiment with the Trunk Notes wiki for iPhone. The wiki engine on the iPhone is impressive, it has many features that makes a power user happy. I really like the clever use of the application badge, which can show the number of wiki pages tagged with a certain tag, e.g. with 'ToDo".

As mentioned in my last post, I want sync. I looked at Trunk Notes export format, and it looks nice. It uses clean UTF-8 encoding, and a simple syntax. The basic syntax is:

Header: Data

Header: Date

Wiki text ...

So the headers look just like HTTP headers, separated by colon and terminated by new lines.
However, I am not entriely sure how the engine know where the real content starts, i.e., what happens if I start a page with:

Today: Big goals ahead

Tomorrow: The world :-)

Next I looked into ways of creating an online wiki with Markdown syntax. Finally I stumbled over this sentence on the markdown page: "The overriding design goal for Markdown’s formatting syntax is to make it as readable as possible." That is certainly a nice design goal and markdown syntax reads really nice in a text editor, but I don't want to have to type this on my iPhone.

So, what would be a good wiki syntax for the iPhone? Having researched many wiki syntaxes and written many converters, I have settled for STIF. STIF is a subset of HTML that contains only the essential elements. No wiki syntax needs to support more, but most wiki syntax should at least support STIF. That is my opinion after having looked into many wiki syntax.

Ok, so here are the next two steps that should be taken for an iPhone wiki syntax:

Analyse the iPhone text entry capabilities
Map them to STIF

iPhone Text Entry Analysis

Reachable with 1 click: A-Z (at the begin of a sentence), a-z within a word. Equally easy: " " (space).
With 2 clicks: 0-9, "-", "/", ":", ";", "(", ")", "€", "&", "@" '"', ".", ",", "?", "!", "'"
With 3 clicks: "[]{}#%^*+=_\|~<>$",(pound sign), (yen sign), (bullet), ".,?!'"

Best characters to mix with letters and numbers are obviusly ".,?!'", as they are reachable in both cases with 1 or 2 clicks.

The fastest way would be to create a wiki syntax from letters only, e.g., write

XXX my headline

here we list fruits

o Apple

o Banana

to mean

my headline

here we list fruits

Apple

Banana

However, we see already that using "XXX" (or "Xxx", or "xxx") might (a) conflict with other syntax and (b) already requires to tap three times. Obviously we need to find out what the most frequently used formatting instructions are. For me, they are in order of appearance:

Bulleted lists (also nested ones)
WikiLinks -- are perfectly handled by the iPhone keyboard
http:// links (usually created by copy and paste)
Strong (to find stuff easier in long wiki pages)
Headlines (one or two types suffice for me)

Let me know your preferences from the STIF specification before I will actually create an iPhone wiki syntax and try to convince the Trunke Notes developer to implement it as an option ;-)