JaanusSiim

Blast from the past

jaanus@jaanussiim.com (Jaanus Siim) — Wed, 22 Feb 2012 00:00:00 +0200

Starting J2ME and games

At the end of 2004 Aqris Software AS was looking into different technologies. Something that peaked our interest was J2ME. As I had previously shown interest in game development I was introduced to it. And this is how one man J2ME development team was born :)

Balloon pop

As first try we decided to make a Bejeweld clone. I came up with the balloons theme for it. Instead of jewels we would use balloons with new ones floating up from bottom. And this is when fun started.

Coming to J2ME was a challenge. Wrapping my head around platform fragmentation… Finding ways how to work around them… But this was not my only problem. When I was finished then the game consisted of one big run method with gazillion if statements. It was a absolute mess. Who ever wrote that should have been shot in place.

After working through all these issues the game actually got published at Zed Italy. Aqris had previous contacts with them and this is probably only reason why this deal happened. As I heard nothing from business side, I assume it was a total flop.

Duck-Up!

Aqris had also other contacts. One of these was Sulake. Creator of Habbo hotels. They wanted to have their version of Balloon Pop. For that all the application code went through a massive rewrite. And the result was also internally something beautiful.

There was a unwritten rule about not using object oriented code in J2ME. Because of classes overhead, size on disk and memory, and limited application JAR sizes this was frowned upon. But after I had seen beautiful code I have never looked back. Always make maintainable code and figure out JAR size limitations later.

Anyway - Duck-Up! was released in Habbo shops. At some point I did hear a number like 1000 downloads in one month, but I never got it confirmed.

Application promo from their site:

Puzzle pipes

I guess Duck-Up! had to be some success, because Habbo wanted another game. This time they came with the idea. They wanted to have ‘connect pipes’ type of game. Backstory was - new hotel was almost finished, but some plumbing work was still left. And your mission was to finish it before time runs out.

Basic app architecture from previous project could be used. But there was one challenging part, that I never believed we can do. Animating the fluids running in pipes. On slow J2ME devices we could never have kept the frame rate at acceptable levels. What I ended up doing was craziest fun code ever. Internal pixels of pipe were counted together and as fluids did flow only pixels changing were updated. Somehow this worked perfectly.

Again, I got no updates about app sales…

This was when it clicked…

Thinking back to that time… Figuring out how to shave few bytes off jar size so that it would fit into 64kB device limit. Finding common base for fragmented devices. Hunting down test devices. This kind of tinkering was real fun. And that is where I got the mobile bug. Although there were a few years between when it was only a hobby. I found my way back to it and am still going strong!

Starting new project - Wesnoth skirmishes

jaanus@jaanussiim.com (Jaanus Siim) — Thu, 24 Nov 2011 00:00:00 +0200

There is a plan. The plan is to push out multiple small projects to improve my portfolio and increase my street cred. Goal of the plan - become freelance and spend some years chilling in Italy. Because this is the place where best things in the world come from - pizza, pasta and Ferrari! During execution of the plan it would be great to do something fun and also learn something new.

For years I have tought about making my own version of Battleship. Some years ago I also completed a prototype of it in J2ME. It was called ‘Pillow Fight’ and was using some prototype code and characters from Habbo hotels. Unfortunately I never showed it to anyone.

Now is the perfect time to make this really happen on iPhone. For years I have enjoyed playing The Battle For Wesnoth and as it’s licensed under GPL, resources from it would be ideal for a small fun game. When I mix in learning Cocos2D then all the preconditions for fun time and skills increase are met!

The idea is to start with only a simple Battleship clone, but in time add some more variations and Wesnoth specific features/elements to it. In the future the game could be more then just a simple hit and miss game. It could make use of different landscapes and character skills. Maybe also some story mode could be added…

iPhone and CellID

jaanus@jaanussiim.com (Jaanus Siim) — Wed, 04 May 2011 00:00:00 +0300

Since my first contact with GIS in general and after finding OpenCellID database, I have developed an obsession with CellID-s. I need to map them. I need to see mapped cell towers on the map. I need to figure out the range of cell towers. And if the tower was 2G only or was also supporting 3G. And of course I needed to create my own GIS server to support all this.

In the old days You could access mobile towers cell id on almost any feature phone. With J2ME on SonyEricsson and Nokia phones it was a cake walk. With Android everything got even better. Loved my HTC Hero (European version with the chin). But since my day job consists mainly of iOS development, I have also moved to iPhone 4. And the problem is - there is no way to access CellID data programmatically on iOS. You can access MNC and MCC, but no CellID.

As a developer this little detail can’t stop me from writing a map application with CellID mapping support. I still have the old Hero laying around and with the iOS 4.3 update I got the most brilliant idea - would it not be possible to connect Hero to iPhone’s WiFi hotspot and access needed data somehow this way?

Initial tests have been positive. Using i-jetty on Android I was able to access the device from iPhone providing WiFi connection. Now I just need to check if I can also access required device data from Jetty servlets. If everything works out, I could connect multiple Android devices simultaneously and map all towers for operators in Estonia at the same time.

What will I do with the data? Probably just some pretty graphics and nothing more. It’s more about the journey…

Eclipse with preprocessing

jaanus@jaanussiim.com (Jaanus Siim) — Sun, 07 Sep 2008 00:00:00 +0300

When working with code that was written by somebody else, one of the most annoying issues for me was inconsistent code formating. Actually in this particular case the indentation was totally screwed up.

Only option was automatic formatting with Eclipse. But there was a problem with preprocessing directives in code. They all got screwed up, a space was added between // and #, and nothing worked. To my luck (and for the luck of other J2ME developers :P) there is a setting ‘Enable line comment formatting’ buried inside code formating settings. With this option disabled, preprocessing worked as a charm.

This particular code was also using code preprocessing in imports, that got still screwed up. But this should be considered a bad style anyway. And it is really not needed. You should preprocess code/class usage and let obfuscation do rest of the work – removing unneeded imports/classes.

J2MEDB version 0.2.0 has been released

jaanus@jaanussiim.com (Jaanus Siim) — Sun, 02 Mar 2008 00:00:00 +0200

Updated version of J2MEDB has been launched on j2mehandsets.com. Main work has gone into usability and handset detection improvement.

On MIDlet side key mappings test was altered a bit. I hope that new graphics will make the test more understandable. Also it has been made clearer, that on every upload only data for current test will be uploaded. I was thinking about uploading all test data in one go, but because of media test crash on Nokia 7650 I decided to drop it.

On server side handsets approving and some minor data editing options have been added. Handsets listing by supported APIs has also been enabled. I tried to improve handset detection based on current data available, but probably more issues can come up in that area.

At the moment the database only contains Nokia and SonyEricsson phones, which is bad :) But I am working on plan for adding more handsets (also from different manufacturers) to database.

For next version I would like to add some simple wiki implementation for known issues and general test descriptions.

So, please, keep testing :P

J2MEDB - version 0.1.0

jaanus@jaanussiim.com (Jaanus Siim) — Tue, 22 Jan 2008 00:00:00 +0200

After playing with the idea really long time, initial version of J2MEDB is up and running on www.J2MEHandsets.com

The Idea

The idea for my own database came from dissatisfaction with other databases available at that time. There were databases, that had a lot of useful data, but were a bit too generic. For example they listed all available sound formats, but did not mention, which protocols supported these. There were also databases, that were just filled from official specs, even if Java implementation did not support it.

At some point I would like to add database support to midlet-builder. In theory it should be possible to just tell midlet-builder to generate build for all handsets contained in database. And if midlet-engine implementation is also mature enough, it could really happen.

Current implementation

In current implementation I have mostly focused on data upload and simple data display. I feel, that I could have made it a bit better, but wanned to have some initial feedback. And also see, what kind of data could I expect. And where might the bottlenecks be.

At the moment SonyEricsson and Nokia support should be decent, but I did not have any other devices available during development – so, you may never know…

Future plans

Make it more prettier :)
Add some kind of simple screening support. At the moment all devices will be added in ‘pending for approval’ status.
Improve handset detection on midlet/server

MIDlet localization done right

jaanus@jaanussiim.com (Jaanus Siim) — Mon, 05 Nov 2007 00:00:00 +0200

I have used two ways for MIDlet localization since I started coding J2ME. I will be describing my changes in localization handling based on demand from clients.

Actually I have used three solutions. The first solution was having all your strings hard coded in source files and never change them. It worked until we found a client in Italy and we needed to support two languages.

Then we opted on using the tools that we knew at the time. We used ant’s replace task for processing translation strings. It worked really well at that time. Of course we had to create two builds for every targeted phones group, but we could handle it. What made things a bit difficult was that the build system we used then produced some times broken builds. So the release schedule also included me running every build in emulator. Just in case. If some code path did not work we made a new build for that device and everything was fixed.

Then we got a client, that wished to support twelve languages. It was clear, that our procedure at that time did not work anymore. I think that the idea for current implementation (used also in midlet-builder/midlet-engine) came from J2ME-Polish. It was how they were handling localization (disclaimer - I can’t remember, if I have ever actually looked at localization code in J2ME-Polish), or at least how I understood it.

In first step of localization midlet-builder will generate two files from input directory. The actual localization file and a file used by preprocessor later on. The localization file generated has following format:

< number of languages > at the moment this is byte
< number of translation keys > number if different strings used
    in MIDlet. This number does not include some special keys
    described later.

Next we will have stings used on localization screen for language selection by user

< language codes > two character language codes that can be used for
    automatic language selection on the phone (comparing it to
    microedition.locale property. A note – this will not work on
    some Samsung phones).
< language names > localized languages names. Preferably in native language.
< special select keys > also used on localization key. If you
    were moving up/down in languages list, 'select' string will
    be shown in highlighted language.
< help strings >
< about strings >
< all keys for every language > This is where all translation
    strings for every language are entered. All the keys will be
    same for every localization set. And they will be entered in
    the same order.

The other file generated is used in source files preprocessing. We’ll call it ‘key_map.txt’. It has key-number pairs for key index in localization set. For example ‘menu.help = 0’ would mean that translation for ‘HELP’ string in main menu is located in first position for every language.

In the source code we have languages screen for language selection. It will load language codes, language names and select strings for displaying these on the screen

    InputStream is="".getClass().getResourceAsStream("/loc.bin");
    DataInputStream dis=new DataInputStream(is);
    final byte languages=dis.readByte();
    languageNames=new String[languages];
    languageCodes=new String[languages];
    selectStrings=new String[languages];

    //ignore translations length
    dis.readByte();
    for(int i=0;i &lt; languageCodes.length;i++){
      languageCodes[i]=dis.readUTF();
    }

    for(int i=0;i &lt; languageNames.length;i++){
      languageNames[i]=dis.readUTF();
    }

    for(int i=0;i &lt; selectStrings.length;i++){
      selectStrings[i]=dis.readUTF();
    }

Here the translation keys length is ignored, because it has no meaning here. Maybe it should be moved to different position.

Because all languages are shown in the order, as they are in localization file, we can search for used language keys start position, after user has selected one. Until now no actual translations have been loaded. To load the keys into array we skip all special keys and also all different language keys that can come before selected language.

For accessing localization keys from code we have two methods in translation handler: get(String keyName) and get(int keyIndex). The first method will have no implementation and is there only for compiler and will be replaced by translation preprocessor. The preprocessor will replace all get(“key.name”) calls with get(index) calls based on values loaded from ‘key_map.txt’.

Because help and about text are usually not displayed that often and take more space in memory, they are handled a bit differently. They are loaded ‘on demand’ and are not kept in memory (unlike other strings for loaded language).

So this is my opinion of ‘localization done right’. I should really create a wiki page describing the process and all included file formats.

In the future I would also like to add localization loading from server and maybe just keep actually used translations in memory (for example only main menu elements). But this would require an automated way for detecting keys used on every screen… Looks like an idea for future projects!

How I made a backup from public wiki

jaanus@jaanussiim.com (Jaanus Siim) — Mon, 29 Oct 2007 00:00:00 +0200

Background

Lately the J2ME forums wiki has been under constant spam attacks. As the owner of the forum seems to be rather busy, to implement some spam protection, there was an idea to create a backup from the data in wiki before spam. I have no access to the wiki server, so I had to use other means available. Hpricot to rescue.

Know the enemy

To me it looked like there is no way to be sure, if site contains spam or not. Some users experimented with adding spam links into hidden div in hopes that spam robot will be fooled. But it had no effects. So we may have spam links on all pages. There was also no 100% sure way to know, when spam posts started, so I opted for full backup of the wiki.

So I needed a history for all discovered pages. In MediaWiki the pages history is available using address:

index.php?title=#{page_name}&amp;action=history

As I had no idea how many pages there were in history I had to add parameters

&amp;limit=1000&amp;offset=0

to the url. As default I would have gotten

limit=50

From history page I searched links for every available revision for the page. Armed with this knowledge I could scan every revision of the page, search for additional internal links, and save page content.

The code

Can be downloaded from s3 bucket

I created a Parser object, that will be initialized with the wiki url. After that You just call the

parse(entry_page)

method and it will start making backups of the wiki pages. Saving these into ‘data’ directory. All saved pages will have modification date in their name.

The result

After 10 hours of running and 3GB data transferred I had 850 MB of knowledge in ‘data’ directory. Probably a lot of it is also spam. The next step should be converting these pages automatically to DocuWiki format and removing spam. At the moment I will probably create clean pages by hand and look at it later.

I also tried to use urls blacklist from blacklist.chongqed.org but in my experiments it turned out to be rather useless. The latest urls were not listed there… In the end it actually did not make any difference. I just made a dump of data on the page.

Was it worth it - for me definitely. I added my url as user agent in request and until now nobody from J2ME forums has also complained.

Lessons learned

Hpricot made the task a lot easier. I must admit, that I had some small problems with it at first… But it got better.

In the parser, there is also one redundant step. There is no point for getting internal links from actual history page. This data is also available on edit page, that will be visited anyways… If this step would be removed, then the parser could have completed the backup in 50% less time transferring 50% less data. Too bad I realized that next morning, when script was already running.

And the revelation today - I need to start commenting my code better. You never know, when it will get published.

Anyway - have fun! Now I’m going back to mobile development.

A library repository for midlet-builder

jaanus@jaanussiim.com (Jaanus Siim) — Wed, 17 Oct 2007 00:00:00 +0300

One feature for midlet-builder would be dynamic jars selection for source compilation. The selection would be handled based on supported device properties. This should, in theory, add additional security level, that the compiled code is also suitable for the device. You could immediately see, if you are using classes from unsupported packages.

First of all I need a cheap location for the repository. And at the moment Amazon’s S3 looks like a perfect solution. It is lacking possibility for browsing, but I’m sure there are ways around it.

After playing a bit with S3 I have one tip for you - go straight to s3sync. In my opinion the fastest way to test S3. I started experimenting with Ruby code. And it was really not going anywhere… But it was my problem - some times I just can’t stand, if something is holding me back from working on my ideas.

Anyhow - the tough part will be dependency handling. At the moment I’m leaning towards using Ivy for it. It could be used as standalone from command line. What I do not like about it is, that it will create additional dependency inside midlet-builder. Already I’m relying on too many external programs, that can’t be configured.

As another solution I should check dependency handling in Raven. If I only found some time for it…

And of course midlet-bulder could also handle the dependency resolving. As I figure - it does not have to me something fancy… Just some dirty version comparing and http requests… But at the moment it should be really low priority… I have more important issues to resolve.

Midlet-engine - J2ME on Rails?

jaanus@jaanussiim.com (Jaanus Siim) — Thu, 11 Oct 2007 00:00:00 +0300

Ok, ok - the title is a bit bold. But the idea behind the framework for J2ME games should be the same - to handle all common behavior and let You just create.

midlet-engine is based on the idea that only places, where games code will change, are in main menu animation and game logic/rendering. Everything else can be handled by common code and can be reused. There is no need for creating screens flow handling, localization, resources loading, etc. from scratch for every new game. Just create a common framework for implementation, that would be reused, and innovate. The additional benefit of a framework would be, that it would contain all the knowledge about device specific bugs and workarounds for these.

Currently midlet-engine handles:

localization
screens flow (from intro screen to outro screen)
custom font implementation
a custom sprite implementation

Should handle:

network communication
generic high scores handling
generic saved game handling
handling and separation for thread driven and event driven MIDlets
resource packages for screen/level

The problem with the midlet-engine at the moment is, that it’s not really flexible. To use it for event driven MIDlet I had to use some hacks in midlet-builder, to ignore not needed classes. This should be some how configured and handled better.

Under resource packages I was thinking that there should be a way to determine automatically, what resources are needed by every step of the application. Basically between load screens. If it could be detected, then we could combine these resources in different resource packages, to gain some jar space. But at the same time the resource packages wold not get too big and make loading noticeably slower…

But the method for detecting the used resources is basis for another project…

So many ideas and so little time…