Sharing Technologies

Unit tests, Mock objects, and App Engine

2009-11-20T18:49:00.010-05:00

For my [still a secret] project which is running on Google App Engine infrastructure [1], I want to make it as solid as possible from the beginning by applying most of the best practices of the Agile methodology [2].

Update 2009/12/05:
With the release of the App Engine Java SDK 1.2.8 (read release notes, I had to update my code and this post on two points:

Without the specification of the JDO inheritance type, the environment assumes it's superclass-table. This type is not supported by App Engine. Only subclass-table and complete-table are supported. In the Entity class described below, I had to add @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE). Read the documentation about Defining data classes for more information.
With the automation of the task execution, the MockAppEngineEnvironment class listed below had to be updated to serve an expected value when the Queue runs in the live environment. Read the details on the thread announcing the 1.2.8. SDK prerelease on Google Groups.

Now, all tests pass again ;)

As written on my post from September 18, I had to develop many mock classes to keep reaching the mystical 100% of code coverage (by unit tests) [3]. A good introduction of mock objects is given by Vincent Massol in his book “JUnit in Action” [4]. To summarize, mock objects are especially useful to inject behavior and force the code using them to exercise complex control flows.

Developing applications for Google App Engine is not that complex because the system has a good documentation and an Eclipse plug-in ease the first steps.

Use case description

Let's consider a simple class organization implementing a common J2EE pattern:

A DTO class for a Consumer;
The DAO class getting the Consumer from the persistence layer, and sending it back with updates; and
A Controller class routing REST requests. The Controller is an element of the implemented MVC pattern

Use case illustration

The code for the DTO class is instrumented with JDO annotations [5]:

Consumer DTO class definition

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable="true")
@Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE)
public class Consumer extends Entity {
    @Persistent
    private String address;
 
    @Persistent
    private String displayName;
 
    @Persistent
    private String email;
 
    @Persistent
    private String facebookId;
 
    @Persistent
    private String jabberId;
 
    @Persistent
    private Long locationKey;
 
    @Persistent
    private String twitterId;
 
    /** Default constructor */
    public Consumer() {
        super();
    }
 
    /**
     * Creates a consumer
     * @param in HTTP request parameters
     */
    public Consumer(JsonObject parameters) {
        this();
        fromJson(parameters);
    }
 
    public String getAddress() {
        return address;
    }
    
    public void setAddress(String address) {
        this.address = address;
    }
    
    //...
}

My approach for the DAO class is modular:

When the calling code is doing just one call, like the ConsumerOperations.delete(String) method deleting the identified Consumer instance, the call can be done without the persistence layer knowledge.
When many calls to the persistence layer are required, the DAO API offers the caller to pass a PersistenceManager instance that can be re-used from call to call. With the combination of the detachable="true" parameter specified in the JDO annotation for the Consumer class, it saves many cycles.

Excerpt from the ConsumerOperations DAO class definition

/**
 * Persist the given (probably updated) resource
 * @param consumer Resource to update
 * @return Updated resource
 * @see ConsumerOperations#updateConsumer(PersistenceManager, Consumer)
 */
public Consumer updateConsumer(Consumer consumer) {
    PersistenceManager pm = getPersistenceManager();
    try {
        // Persist updated consumer
        return updateConsumer(pm, consumer);
    }
    finally {
        pm.close();
    }
}
 
/**
 * Persist the given (probably updated) resource while leaving the given persistence manager open for future updates
 * @param pm Persistence manager instance to use - let opened at the end to allow possible object updates later
 * @param consumer Resource to update
 * @return Updated resource
 */
public Consumer updateConsumer(PersistenceManager pm, Consumer consumer) {
    return pm.makePersistent(consumer);
}

The following piece of the abstract class BaseOperations shows the accessor made availabe to any controller code to get one handle of a valid PersistenceManager instance.

Excerpt from the abstract BaseOperations DAO class definition

/**
 * Accessor isolated to facilitate tests by IOP
 * @return Persistence manager instance
 */
public PersistenceManager getPersistenceManager() {
    PersistenceManager pm = getPersistenceManagerFactory().getPersistenceManager();
    pm.setDetachAllOnCommit(true);
    pm.setCopyOnAttach(false);
    return pm;
}

To finish the use case setup, here is a part of the controller code which deals with incoming HTTP requests and serves or operates accordingly. This specific piece of code replies to a GET request like:

Invocation: http://<host:port>/API/Consumer/43544"

Response:

{key:43544, displayName:"John", address:"75, Queen, Montréal, Qc, Canada", 
locationKey:3245, location: {id:3245, postalCode:"H3C2N6", countryCode:"CA",
latitude:43.3, longitude:-73.4}, ...}

Excerpt from the ConsumerRestlet Controller class definition

@Override
protected JsonObject getResource(JsonObject parameters, String resourceId, User loggedUser) throws DataSourceException {
    PersistenceManager pm = getBaseOperations().getPersistenceManager();
    try {
        // Get the consumer instance
        Consumer consumer = getConsumerOperations().getConsumer(pm, Long.valueOf(resourceId));
        JsonObject output = consumer.toJson();
        // Get the related information
        Long locationKey = consumer.getLocationKey();
        if (locationKey != null) {
            Location location = getLocationOperations().getLocation(pm, locationKey);
            output.put(Consumer.LOCATION, location.toJson());
        }
        // Return the complete set of information
        return output;
    }
    finally {
        pm.close();
    }
}

Simple mock

Now, it's time to test! To start slowly, let's deal with the Restlet getResource() method to verify:

Just one and only one instance of PersistenceManager is loaded by the function;
The PersistenceManager instance is cleanly closed at the end of the process;
There's a call issued to get the identified Consumer instance;
There's possibly a call issued to get the identified Location instance;
The output value has the expected information.

In the corresponding unit test series, we don't want to interfere with the App Engine infrastructure (the following chapter will address that aspect). So we'll rely on a mock for the PersistenceManager class that will be injected into the ConsumerRestlet code. The full source of this class is available on my open source project two-tiers-utils: javax.jdo.MockPersistenceManager.

Custom part of the mock for the PersistenceManager class

public class MockPersistenceManager implements PersistenceManager {
    private boolean closed = false; // To keep track of the "closed" state
    public void close() {
        closed = true;
    }
    public boolean isClosed() {
        return closed;
    }

    // ...
}

Here are the unit tests verifying the different flow paths:

When an exception is thrown, because the back-end does not serve the data for example;
When the Consumer instance returns without location coordinates;
When the Consumer instance is fully documented.

Three tests validating the behavior of the ConsumerRestlet.getResource() method

@Test(expected=IllegalArgumentException.class)
public void testUnexpectedError() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            throw new IllegalArgumentException("Done in purpose!");
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
}

@Test
public void testGettingOneConsumer() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            Consumer consumer = new Consumer();
            consumer.setId(consumerId);
            return consumer;
        }
    };
    final Long locationId = 67890L;
    final LocationOperations locationOps = new LocationOperations() {
        @Override
        Location getLocation(PersistenceManager pm, Long id) {
            fail("Call not expected here!");
            return null;
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
        @Override LocationOperation getLocationOperations() { return locationOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
    
    // Post-test verifications
    assertTrue(pm.isClosed());
    assertNotSame(0, response.size());
    assertTrue(response.containsKey(Consumer.ID);
    assertEquals(consumerId, response.getLong(Consumer.ID));
}

@Test
public void testGettingConsumerWithLocation() {
    // Test prepration
    final PersistenceManager pm = new MockPersistenceManager();
    final BaseOperations baseOps = new BaseOperations() {
        boolean askedOnce = false;
        @Override
        PersistenceManager getPersistenceManager() {
            if (askedOnce) {
                fail("Expects only one call");
            }
            askedOnce = true;
            return pm;
        }
    };
    final Long consumerId = 12345L;
    final Long locationId = 67890L;
    final ConsumerOperations consumerOps = new ConsumerOperations() {
        @Override
        Consumer getConsumer(PersistenceManager pm, Long id) {
            assertEquals(consumerId, id);
            Consumer consumer = new Consumer();
            consumer.setId(consumerId);
            consumer.setLocationId(locationId);
            return consumer;
        }
    };
    final LocationOperations locationOps = new LocationOperations() {
        @Override
        Location getLocation(PersistenceManager pm, Long id) {
            assertEquals(locationId, id);
            Location location = new Location();
            location.setId(locationId);
            return location;
        }
    };
    ConsumerRestlet restlet = new ConsumerRestlet() {
        @Override BaseOperation getBaseOperations() { return baseOps; }
        @Override ConsumerOperation getConsumerOperations() { return consumerOps; }
        @Override LocationOperation getLocationOperations() { return locationOps; }
    }
    
    // Test itself
    JsonObject response = restlet.getResource(null, consumerId.toString, null);
    
    // Post-test verifications
    assertTrue(pm.isClosed());
    assertNotSame(0, response.size());
    assertTrue(response.containsKey(Consumer.ID);
    assertEquals(consumerId, response.getLong(Consumer.ID));
    assertTrue(response.containsKey(Consumer.LOCATION_ID);
    assertEquals(locationId, response.getLong(Consumer.LOCATION_ID));
    assertTrue(response.containsKey(Consumer.LOCATION);
    assertEquals(consumerId, response.getJsonObject(Consumer.LOCATION).getLong(Location.ID));
}

Note that I would have been able to override just the PersistenceManager class to have the Object getObjectById(Object arg0) method returning the expected exception, Consumer, and Location instances. But I would have pass over the strict limit of a unit test by then testing also the behavior of the ConsumerOperations.getConsumer() and LocationOperations.getLocation() methods.

App Engine environment mock

Now, testing the ConsumerOperations class offers a better challenge.

As suggested above, I could override many pieces of the PersistenceManager class to be sure to control the flow. But to do a nice simulation, I almost need to have the complete specification of the Google App Engine infrastructure to be sure I mock it correctly. This is especially crucial when processing Query because Google data store has many limitations [6] that others traditional database, like MySQL, don't have...

Because this documentation is partially available and because Google continues to update its infrastructure, I looked for a way to use the standalone environment made available with the App Engine SDK [1]. This has not been easy because I wanted to have the test running independently from the development server itself. I found first some documentation on Google Code website: Unit Testing With Local Service Implementations, but it was very low level and did not fit with my JDO instrumentation of the DTO classes. Hopefully, I found this article JDO and unit tests from App Engine Fan, a great community contributor I mentioned many times in previous posts!

By cooking information gathered on Google Code website and on App Engine Post, I've produced a com.google.apphosting.api.MockAppEngineEnvironment I can use for my JUnit4 tests.

Three tests validating the behavior of the ConsumerRestlet.getResource() method

package com.google.apphosting.api;
 
// import ...
 
/**
 * Mock for the App Engine Java environment used by the JDO wrapper.
 *
 * These class has been build with information gathered on:
 * - App Engine documentation: http://code.google.com/appengine/docs/java/howto/unittesting.html
 * - App Engine Fan blog: http://blog.appenginefan.com/2009/05/jdo-and-unit-tests.html
 *
 * @author Dom Derrien
 */
public class MockAppEngineEnvironment {
 
    private class ApiProxyEnvironment implements ApiProxy.Environment {
        public String getAppId() {
          return "test";
        }
 
        public String getVersionId() {
          return "1.0";
        }
 
        public String getEmail() {
          throw new UnsupportedOperationException();
        }
 
        public boolean isLoggedIn() {
          throw new UnsupportedOperationException();
        }
 
        public boolean isAdmin() {
          throw new UnsupportedOperationException();
        }
 
        public String getAuthDomain() {
          throw new UnsupportedOperationException();
        }
 
        public String getRequestNamespace() {
          return "";
        }
 
        public Map getAttributes() {
            Map out = new HashMap();

            // Only necessary for tasks that are added when there is no "live" request
            // See: http://groups.google.com/group/google-appengine-java/msg/8f5872b05214...
            out.put("com.google.appengine.server_url_key", "http://localhost:8080");

            return out;
        }
    };
 
    private final ApiProxy.Environment env;
    private PersistenceManagerFactory pmf;
 
    public MockAppEngineEnvironment() {
        env = new ApiProxyEnvironment();
    }
 
    /**
     * Setup the mock environment
     */
    public void setUp() throws Exception {
        // Setup the App Engine services
        ApiProxy.setEnvironmentForCurrentThread(env);
        ApiProxyLocalImpl proxy = new ApiProxyLocalImpl(new File(".")) {};
 
        // Setup the App Engine data store
        proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY, Boolean.TRUE.toString());
        ApiProxy.setDelegate(proxy);
    }
 
    /**
     * Clean up the mock environment
     */
    public void tearDown() throws Exception {
        // Verify that there's no pending transaction (ie pm.close() has been called)
        Transaction transaction = DatastoreServiceFactory.getDatastoreService().getCurrentTransaction(null);
        boolean transactionPending = transaction != null;
        if (transactionPending) {
            transaction.rollback();
        }
 
        // Clean up the App Engine data store
        ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate();
        if (proxy != null) {
            LocalDatastoreService datastoreService = (LocalDatastoreService) proxy.getService("datastore_v3");
            datastoreService.clearProfiles();
        }
 
        // Clean up the App Engine services
        ApiProxy.setDelegate(null);
        ApiProxy.clearEnvironmentForCurrentThread();
 
        // Report the issue with the transaction still open
        if (transactionPending) {
            throw new IllegalStateException("Found a transaction nor commited neither rolled-back." +
                    "Probably related to a missing PersistenceManager.close() call.");
        }
    }
 
    /**
     * Creates a PersistenceManagerFactory on the fly, with the exact same information
     * stored in the /WEB-INF/META-INF/jdoconfig.xml file.
     */
    public PersistenceManagerFactory getPersistenceManagerFactory() {
        if (pmf == null) {
            Properties newProperties = new Properties();
            newProperties.put("javax.jdo.PersistenceManagerFactoryClass",
                    "org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory");
            newProperties.put("javax.jdo.option.ConnectionURL", "appengine");
            newProperties.put("javax.jdo.option.NontransactionalRead", "true");
            newProperties.put("javax.jdo.option.NontransactionalWrite", "true");
            newProperties.put("javax.jdo.option.RetainValues", "true");
            newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns", "true");
            newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns", "true");
            pmf = JDOHelper.getPersistenceManagerFactory(newProperties);
        }
        return pmf;
    }
 
    /**
     * Gets an instance of the PersistenceManager class
     */
    public PersistenceManager getPersistenceManager() {
        return getPersistenceManagerFactory().getPersistenceManager();
    }
}

With such a class, the unit test part is easy and I can build complex test cases without worrying about the pertinence of my mock classes! That's really great.

Excerpt of the TestConsumerOperations class

public class TestConsumerOperations {
 
    private MockAppEngineEnvironment mockAppEngineEnvironment;
 
    @Before
    public void setUp() throws Exception {
        mockAppEngineEnvironment = new MockAppEngineEnvironment();
        mockAppEngineEnvironment.setUp();
    }
 
    @After
    public void tearDown() throws Exception {
        mockAppEngineEnvironment.tearDown();
    }
 
    @Test
    public void testCreateVI() throws DataSourceException, UnsupportedEncodingException {
        final String email = "unit@test.net";
        final String name = "Mr Unit Test";
        Consumer newConsumer = new Consumer();
        newConsumer.setDisplayName(name);
        newConsumer.setEmail(email);
        assertNull(newConsumer.getId());
 
        // Verify there's no instance
        Query query = new Query(Consumer.class.getSimpleName());
        assertEquals(0, DatastoreServiceFactory.getDatastoreService().prepare(query).countEntities());
 
        // Create the user once
        ConsumerOperations ops = new ConsumerOperations();
        Consumer createdConsumer = ops.createConsumer(newConsumer);
 
        // Verify there's one instance
        query = new Query(Consumer.class.getSimpleName());
        assertEquals(1, DatastoreServiceFactory.getDatastoreService().prepare(query).countEntities());
 
        assertNotNull(createdConsumer.getId());
        assertEquals(email, createdConsumer.getEmail());
        assertEquals(name, createdConsumer.getName());
    }
    
    // ...
}

Conclusion

As a big fan of TDD, I'm now all set to cover the code of my [still a secret] project efficiently. It does not mean everything is correct, more that everything I thought about is correctly covered. At the time of this writing, just for the server-side logic, the code I produced covers more than 10,000 lines and the unit tests bring an additional set of 23,400 lines.

When it's time to refactor a bit or to add new features (plenty of them are aligned in my task list ;), I feel comfortable because I know I can detect most of regressions (if not all) after 3 minutes of running the test suite.

If you want to follow this example, feel free to get the various mock classes I have added to my two-tiers-utils open-source project. In addition to mock classes for the App Engine environment, you'll find:

Basic mock classes for the servlet (see javax.servlet) and javamocks.io packages -- I had to adopt the root javamocks because the JVM class loader does not accept the creation on the fly of classes in the java root).
A mock class for twitter4j.TwitterUser -- I needed a class with public constructor and a easy way to create a default account.
A series of mock class for David Yu's Project which I use to allow users with OpenID credentials to log in. Read the discussion I had with David on ways to test his code, in fact the code he produced and I customized for my own needs and for other security reasons.

For other details on my library, read my post Internationalization and my two-tiers-utils library.

I hope this helps.

A+, Dom
--
References:

Google App Engine: the homepage and the SDK page.
See my post on Agile: SCRUM is Hype, but XP is More Important... where I mentionned the following techniques: Continuous Integration (CI), Unit testing and code coverage (CQC), and Continuous refactoring.
I know that keeping 100% as the target for code coverage numbers is a bit extreme. I read this article Don't be fooled by the coverage report soon after I started using Cobertura. In addition to reducing the exposition to bugs, the 100% coverage gives a very high chance to detect regressions before pushing the updates to the source control system!
Vincent Massol; JUnit in Action; Editions Manning; www.manning.com/massol and Petar Tahchiev, Felipe Leme, Vincent Massol, and Gary Gregory; JUnit in Action, Second Edition; Editions Massol; www.manning.com/tahchiev. I was used to asking any new developer joigning my team to read at least this chapter 7: Testing in isolation with mock objects.
JDO stands for Java Data Objects and is an attempt abstract the data storage manipulation. The code is instrumented with Java annotations like @Persistent, it is instrumented at compile time, and dynamically connect to the data source thanks few properties files. Look at the App Engine - Using the Datastore with JDO documentation for more information.
For general limitations, check this page Will it play in App Engine. For JDO related limitations, check the bottom of the page Usin JDO with App Engine.

Canadian Wireless Management Forum - my review

2009-10-16T19:58:00.000-04:00

Last week, I attended the Canadian Wireless Management Forum in Montreal. Honestly, I've been a bit disappointed because none of the presenters were as great as the ones I met last year. Over the day, I've still been able to gather bits of information I want to share here ;) Thanks to my company Compuware to have allowed sparing one day there.

iPhone and other smart phones deployed in enterprises

The main focus of the conference is around managing wireless (read mobile phone) communications in enterprises. Some presenters talked about how to control expenses, from sharing guidelines up to using monitoring tools suggesting policies from the statistical analysis of the monthly bills. Testimonies showed that applying a strict control on the mobile phone usage and on the contracts allowed cutting costs from 10 to 30%!

At one point, Nicolas Arsenault made a strange point. Here is what I remind from his talk:

Credits: smoothouse

Credits:Josh Bancroft

Deploying an iPhone application for your employees, when compared to a BlackBerry application, has a much lower cost because the employee already owns the device or he's more likely to buy it. With the employee paying for the phone, probably paying to get new phones regularly (like to move from a iPhone 3G to the latest 3GS), employers can just assume the data plans and can spend more resources on the application development, that at one point can be offered to the company customers. Another benefit is related to the technical support: phone owners stop annoying enterprises' help desk for their phone, they contact the manufacturers or the software vendors directly (they are on their own)...

If this approach have evident economical benefits, I disagree with it because:

In general, letting employees owning their work mobile phones cause problems when they leave the company. Most of employees have a non concurrence clause in their contract, usually valid 6 months after the employment contract termination. During that period, they cannot compete with their previous employer. Imagine a salesperson, an account manager, or a consultant who owns his phone number, who is referenced in the phone directories all his previous contacts. When these contacts want to deal with the company, who do you think they'll call: the mobile phone or the company front line? Now that these employees are out, they don't have to submit their phone bills anymore (as they did with their expenses reports). No one can detect the issues anymore… And this is without mentioning that the employee replacement or his colleagues have no way to get the mobile phone contact list to continue the business as usual!

In the company I worked, none let me use my own computer on their network! For the IT departments, this practice would raise too much security concerns. I even know cases that just installing a VPN software on your own machine install silently a bunch of monitoring tools that can mess up your systems (thanks to VirtualBox, it easy to limit these nasty side effects ;). In the old days, when phones were just stupid, just able to handle voice call and exchange text messages (SMS), the risk of phone infection by viruses were pretty negligible. The widely used phone operating system is Symbian, used on Nokia and Sony Ericsson phones, and J2ME is a common application framework, even on phones running Windows Mobile. If the identified viruses are not a lot, and if they are not some damageable (send on your behalf SMS to expensive services, for example), the newest smart platforms provide much more threats because the corresponding phones can host tons of applications. In such an environment, how can a company force software upgrades on systems it does not own? How can it force an employee to upgrade to a new hardware because the current one is compromised?

My last point is more ethical: how far companies should go with putting the burden on their employees? An iPhone costs around 800$. It's often offered around 200$ with a 3-year contract, which costs basically around 60$/month with a simple data plan. If the telecom operators (telco) subsidize so aggressively such a phone, they surely expect higher average revenue per user (ARPU). In addition to be linked to the telco for a long period of time (compare 3 years with the 3 to 6 months between technological evolution: 6 to 12 times longer!), employees have to support costs the company should assume. Usually, companies try to provide a comfortable work environment to get most of their employees, and that's fair to me: if the company gives more than the salary, employees are more likely to deliver better work. With the incitation of employees assuming the cost of the new mobile phones, I see a regression: companies give less while expecting more (reachable—possibly traceable—outside the office hours, for example). IMO, it's yet another example of a technological progress that might worsen fragile people condition...

Mobile payment and Near-field-communication (NFC)

Shortly on this topic, I want to mention presentation made by Daniel Martin for Atlas Telecom Mobile, David Robinson for Rogers Wireless, and Prakash Hariramani for Visa. They talked about an experiment conducted downtown Toronto where they were able, thanks to Motorola phones equipped with a NFC emitter and a good number of retailers there, to allow mobile payment over the mobile phone network. Visa's solution, called PayWave™ (MasterCard's one is called PayPass™), was inserted into the Motorola phone extension and allow consumers to pay for their purchase quite easily.

During the discussion, Mr. Robinson talked about Rogers Wireless approach being strictly based on standards. He mentioned the recent u-turn of telco who continued to invest in closed and proprietary solutions (like CDMA) and now move to standardized ones (like HSPA which is an upgrade of GSM, on the road to LTE—see example of Bell and Telus in Canada). Rogers Wireless' approach is then to work with the rest of the major industry players (Orange, Vodafone, etc.) to define a solution for anyone anywhere. Mr. Robinson talked about the possibility to define an extended SIM card (for Subscriber Identity Module; the card contains a micro-SIMCARD processor which manage very securely information). This new SIM card will have NFC capabilities and will be able to interact with contactless payment terminals. It's possible that these SIM cards will contain additional information like driver license identifier that police officers will be able to read, insurance number for the government agencies, etc. The phone will provide the interface to enable the data access, and smart phones with touch screens open the door to various and robust verification techniques.

Because more and more people are more attached to their phone than to their wallet, this approach will possibly be more convenient, more secure, and smaller (no more cash, business cards, credits cards, etc. ;) Who said that implanting the SIM under anyone's skin, a SIM that can unlock your phones, cars, houses, computers, etc, is just science fiction?

Telco business model in danger!

iBwave offers solutions improving in-building wireless coverage. Knowing the fact that 60%-80% of mobile usages are indoor, that telco have difficulties to boost the power or to multiply outdoor antennas near high density areas, these places stay mostly uncovered. Mr. Bouchard illustrated his point with the simulation of the poor performance of the traditional networks on the McGill campus—really amazing! In conclusion, iBwave sits in a very nice and promising niche ;)

The last point of interest to me came from Mario Bouchard, from iBwave.

To introduce his company activities, Mr. Bouchard shows two diagrams which made me think for a while. Let me try to reproduce them.

Mr. Bouchard states that 15 years ago, the innovation came from the network manufacturers: they invented the technology, others created cell phones to connect to the new network, some operators offered (very expensive) plans, and consumers (locked with long term contracts) tried to communicate.

Credits: DigitalAlan

Today, the order has been scrambled:

Thanks to the rapid technology evolution, designers of mobile devices can embed many types of sensors into communicant machines, and with the increasing miniaturization, such machines are pervasiveness!
Because of the reduced delay between big technology evolutions (think about the iPhone which is just 2 years old), consumers choose their devices carefully, and bargain more to get the best quality-price ratio.
Network manufacturers now to their best to provide networks than can deliver at the rhythm the devices can consume. When the European community created the Groupe Spécial Mobile (initial meaning of the GSM acronym, known now for Global System for Mobile communications) in 1982, we had to wait up to 1991 to experiment the first GSM network. GSM is also known the 2G technology. The EDGE (Enhanced Data rates for GSM Evolution, or 2.5G) has been introduced in 1999, the first 3G technology (HSPA/UMTS, and EV-DO/CDMA) has been delivered late 2001, and the coming 4G ones (LTE for Long Term Evolution or WiMax) are on the bench.
At the end of the line, now there are the telcos:

Investments are phenomenal
The competition is rude (not rude enough in Canada, IMO)
Customers are volatile and they always want the latest phone
They have to subsidize the phones to lower the barriers to entry
They have to provide the best coverage everywhere
Customers are quick to leverage social tools to complain about them
Customers don't respect the old rules (read: jailbreak their phone)
Customers are not the cash cows they used to be...

I don't think the telco future looks very nice. As traditional telecommunication service providers, they are more and more just Internet providers. Personally, I'm fine with communication on Internet (VOIP/SIP), with the possibility to stream on Internet (Qik.com, Layar.com), to receive instant messages (IM) instead of text messages (SMS). And look, when I've a chance to connect my phone to a wifi network, I'm happy to get a better connectivity while saving few bucks.

Interesting developments to follow, aren't they?

A+, Dom

Internationalization and my two-tiers-utils library

2009-10-05T21:31:00.006-04:00

This is a follow-up article of Internationalization of GAE applications, which itself is part of the series Web Application on Resources in the Cloud.

In my initial article, I explained some hurdles of globalizing applications, especially the ones being implemented with many programming languages. In this article, I'm going to describe few use-cases and how my open-source library two-tiers-utils can ease the implementation. Here are the covered topics:

Get the user's preferred locale
Display messages in different locales
Handle localized messages with different programming languages
Generate the localized bundles per programming language
Bonus

Get the user's preferred locale

For this use-case, let's only consider the Java programming language. Another assumption is the availability of the localized resources in the corresponding Java format (i.e. accessible via a PropertyResourceBundle instance).

In a Web application, the user's preferred locale can be retrieved from:

The HTTP headers:

locale = ((HttpServletRequest) request).getLocale();

The HTTP session (if saved there previously):

HttpSession session = ((HttpServletRequest) request).getSession(false);
if (session != null) {
  locale = new Locale((String) session.getAttribute(SESSION_USER_LOCALE_ID));
}

The record containing the user's information:

locale = ((UserDTO) user).getPreferredLocale();

To ease this information retrieval, the two-tiers-utils library provides the domderrien.i18n.LocaleController class

Excerpt of public methods offered within domderrien.i18n.LocaleController

This class can be used in two situations:

In a login form, for example, when we can just guess the desired locale from the browser preferred language list or from an argument in the URL.
In pages accessible to identified users thanks to the HTTP session.

Usage example of the domderrien.i18n.LocaleController.detectLocale()

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<%@page
    language="java"
    contentType="text/html; charset=UTF-8"
    pageEncoding="UTF-8"
    import="org.domderrien.i18n.LabelExtractor"
    import="org.domderrien.i18n.LocaleController"
%><%
    // Locale detection
    Locale locale = LocaleController.detectLocale(request);
%><html>
<head>
    <title><%= LabelExtractor.get("dd2tu_applicationName", locale) %></title>
    ...
</head>
<body>
    ...
    <img
        class="anchorLogo"
        src="images/iconHelp.png"
        width="16"
        height="16"
        title="<%= LabelExtractor.get("dd2tu_topCommandBox_helpIconLabel", locale) %>"
    />
    ...
</body>
</html>

The same message in different locales

The previous example introduces also a second class: domderrien.i18n.LabelExtractor. Being given an identifier, an optional array of Object references, and a locale, the get static method loads the corresponding string from the localized resource bundle.

Excerpt of public methods offered within domderrien.i18n.LabelExtractor

A series of localized entries like en:“Welcome {0} {1}”, fr:“Bonjour {2} {1}”, and ja:“お早う {0}{1}” can be easily invoked with a simple command like: LabelExtractor.get("welcome_message", new Object[] { user.getFirstName(), user.getLastName() }, user.getLocale());.

The same message used from different programming languages

Java is a pretty neat language with a large set of editors and code inspectors. But Java is not the only languages used for Web applications. If the two-tiers-utils library provides nice Java features, the delivery of the same library interfaces for the programming languages JavaScript and Python libraries makes it way more valuable!

Code of the domderrien.i18n.LabelExtractor.get() method for the JavaScript language.

(function() { // To limit the scope of the private variables

    /**
     * @author dom.derrien
     * @maintainer dom.derrien
     */
    var module = dojo.provide("domderrien.i18n.LabelExtractor");

    var _dictionnary = null;

    module.init = function(/*String*/ namespace, /*String*/ filename, /*String*/ locale) {
        // Dojo uses dash-separated (e.g en-US not en_US) and uses lower case names (e.g en-us not en_US)
        locale = (locale || dojo.locale).replace('_','-').toLowerCase();

        // Load the bundle
        try {
            // Notes:
            // - Cannot use the notation "dojo.requirelocalization" because dojo parser
            //   will try to load the bundle when this file is interpreted, instead of
            //   waiting for a call with meaningful "namespace" and "filename" values
            dojo["requireLocalization"](namespace, filename, locale); // Blocking call getting the file per XHR or <iframe/>

            _dictionary = dojo.i18n.getLocalization(namespace, filename, locale);
        }
        catch(ex) {
            alert("Deployment issue:" +
                    "\nCannot get localized bundle " + namespace + "." + filename + " for the locale " + locale +
                    "\nMessage: " + ex
                );
        }

        return module;
    };

    module.get = function(/*String*/key, /*Array*/args) {
        if (_dictionary == null) {
            return key;
        }
        var message = _dictionary[key] || key;
        if (args != null) {
            dojo.string.substituteParams(message, args);
        }
        return message;
    };

})(); // End of the function limiting the scope of the private variables

The following piece of code illustrates how the JavaScript domderrien.i18n.LabelExtractor class instance should be initialized (the value of the locale variable can come from dojo.locale or a value injected server-side into a JSP page) and how it can be invoked to get a localized label.

Usage example of the domderrien.i18n.LocaleController.get()

(function() { // To limit the scope of the private variables

    var module = dojo.provide("domderrien.blog.Test");

    dojo.require("domderrien.i18n.LabelExtractor");

    var _labelExtractor;

    module.init = function(/*String*/ locale) {
        // Get the localized resource bundle
        _labelExtractor = domderrien.i18n.LabelExtractor.init(
                "domderrien.blog",
                "TestBundle",
                locale // The library is going to fallback on dojo.locale if this parameter is null
            );

        ...
    };

    module._postData = function(/*String*/ url, /*Object*/ jsonParams) {
        var transaction = dojo.xhrPost({
            content : jsonParams,
            handleAs : "json",
            load : function(/*object*/ response, /*Object*/ioargs) {
                if (response == null) {
                    // Message prepared client-side
                    _reportError(_labelExtractor.get("dd2tu_xhr_unexpectedError"), [ioargs.xhr.status]);
                }
                if (!response.success) {
                    // Message prepared server-side
                    _reportError(_labelExtractor.get(response.messageKey), response.msgParams);
                }
                ...
            },
            error : function(/*Error*/ error, /*Object*/ ioargs) {
                    // Message prepared client-side
                _reportError(error.message, [ioargs.xhr.status]);
            },
            url : url
        });
    };

    var _reportError = function(/*String*/ message, /*Number ?*/xhrStatus) {
        var console = dijit.byId("errorConsole");
        ...
    };

    ...

})(); // End of the function limiting the scope of the private variables

The following series of code excerpts show the pieces involved in getting the localized resources with the Python programming language.

LabelExtractor methods definitions from domderrien/i18n/LabelExtractor.py

# -*- coding: utf-8 -*-

import en
import fr
 
def init(locale):
    """Initialize the global dictionary for the specified locale"""
    global dict
    if locale == "fr":
        dict = fr._getDictionary()
    else: # "en" is the default language
        dict = en._getDictionary()
    return dict

Sample of a localized dictionary from domderrien/i18n/en.py

# -*- coding: utf-8 -*-
 
dict_en = {}
 
def _getDictionary():
    global dict_en
    if (len(dict_en) == 0):
        _fetchDictionary(dict_en)
    return dict_en
 
def _fetchDictionary(dict):
    dict["_language"] = "English"
    dict["dd2tu_applicationName"] = "Test Application"
    dict["dd2tu_welcomeMsg"] = "Welcome {0}."
    ...

Definitions of filters used by the Django templates, from domderrien/i18n/filters.py

from google.appengine.ext import webapp
 
def get(dict, key):
    return dict[key]
 
def replace0(pattern, value0):
    return pattern.replace("{0}", str(value0))
 
def replace1(pattern, value1):
    return pattern.replace("{1}", str(value1))
 
...
 
# http://javawonders.blogspot.com/2009/01/google-app-engine-templates-and-custom.html
# http://daily.profeth.de/2008/04/using-custom-django-template-helpers.html
 
register = webapp.template.create_template_register()
register.filter(get)
register.filter(replace0)
register.filter(replace1)
...

Django template from domderrien/blog/Test.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>{{ dictionary|get:dd2tu_applicationName }}</title>
    ....
</head>
<body>
    ...
    <div class="...">{{ dictionary|get:dd2tu_welcomeMsg|replace0:parameters.loggedUser }}</div>
    ...
</body>
</html>

Test handler from domderrien/blog/Test.py

from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template

def prepareDictionary(Request):
    locale = request.get('lang', 'en')
    return LabelExtractor.init(locale)

class MainPage(webapp.RequestHandler):
    def get(self):
        parameters = {}
        parameters ['dictionary'] = domderrien.i18n.LabelExtractor.init(self.request)
        parameters ['loggedUser'] = users.get_current_user()
        path = os.path.join(os.path.dirname(__file__), 'domderrien/blog/Test.html')
        self.response.out.write(template.render(path, parameters))

application = webapp.WSGIApplication(
    [('/', MainPage)],
    debug=True
)
 
def main():
    webapp.template.register_template_library('domderrien.i18n.filters')
    run_wsgi_app(application)
 
if __name__ == "__main__":
    main()

Generate the localized bundles per programming language

In my previous post Internationalization of GAE applications, I suggest to use a dictionary format that would be programming lnaguage agnostic while being known by translator: TMX, for Tanslation Memory eXchange.

Snippet of a translation unit definition for a TMX formatted file

<tu tuid="dd2tu_welcomeMessage" datatype="Text">
 <tuv xml:lang="en">
  <seg>Welcome {0}</seg>
 </tuv>
 <note>{0} is going to be replaced by the logged user's display name</note>
 <prop type="x-tier">dojotk</prop>
 <prop type="x-tier">javarb</prop>
 <prop type="x-tier">python</prop>
</tu>

The two-tiers-utils library provides a Java runtime domderrien.build.TMXConverter that generates the resource bundles for Java/JavaScript/Python. If a simple series of XSL-Transform runs can do the job, the TMXConverter does a bit more by:

Comparing the modification dates of the generated files with the TMX one to generate them only if needed
Check the uniqueness of the label keys
Generate the list of supported languages

Invoking the TMXConverter runtime from an ant build file is very simple, while a bit verbose:

Ant target definition invoking the TMXConverter

<target name="step-tmx-convert">
    <mkdir dir="${temp.dir}/resources" />
    <mkdir dir="src/WebContent/js/domderrien/i18n/nls" />
    <java classname="domderrien.build.TMXConverter" fork="true" failonerror="true">
        <classpath refid="tmxconverter.classpath" />
        <classpath location="${temp.dir}/resources" />
        <jvmarg value="-Dfile.encoding=UTF-8" />
        <arg value="-tmxFilenameBase" />
        <arg value="${dd2tu.localizedLabelBaseFilename}" />
        <arg value="-sourcePath" />
        <arg value="${basedir}\src\resources" />
        <arg value="-jsDestPath" />
        <arg value="${basedir}\src\WebContent\js\domderrien\i18n\nls" />
        <arg value="-javaDestPath" />
        <arg value="${temp.dir}/resources" />
        <arg value="-languageFilenameBase" />
        <arg value="${dd2tu.languageListFilename}" />
        <arg value="-buildStamp" />
        <arg value="${dd2tu.stageId}" />
    </java>
    <native2ascii
        src="${temp.dir}/resources"
        dest="${temp.dir}/resources"
        encoding="UTF8"
        includes="*.properties-utf8"
        ext=".properties"
    />
    <copy
        file="${temp.dir}/resources/${dd2tu.localizedLabelBaseFilename}.properties"
        tofile="${temp.dir}/resources/${dd2tu.localizedLabelBaseFilename}_en.properties"
    />
    <mkdir dir="src/WebContent/js/domderrien/i18n/nls/en" />
    <copy
        file="src/WebContent/js/domderrien/i18n/nls/{dd2tu.localizedLabelBaseFilename}.js"
        todir="src/WebContent/js/domderrien/i18n/nls/en"
    />
</target>

With the TMX file as the source of thruth for the label definitions, it is just a matter of altering the value a <prop/> tag and running the build once again to move one label definition from one programming language to another. No more error prone copy-and-paste of text between different file formats!

Excerpt of the generated Java resource bundle

bundle_language=English
unit_test_sample=N/A
dd2tu_applicationName="Test Application"
dd2tu_welcomeMessage=Welcome {0}
...
x_timeStamp=20091001.1001

Excerpt of the generated JavaScript resource bundle

({bundle_language:"English",
unit_test_sample:"N/A",
dd2tu_applicationName:"Test Application",
dd2tu_welcomeMessage:"Welcome ${0}",
...
x_timeStamp:"20091001.1001"})

Excerpt of the generated Python class definition

# -*- coding: utf-8 -*-
 
dict_en = {}
 
def _getDictionary():
    global dict_en
    if (len(dict_en) == 0):
        _fetchDictionary(dict_en)
    return dict_en
 
def _fetchDictionary(dict):
    dict["_language"] = "English"
    dict["dd2tu_applicationName"] = "Test Application"
    dict["dd2tu_welcomeMsg"] = "Welcome {0}."
    ...
    dict["x_timestamp"] = "20091001.1001"

Bonus

The TMXConverter being part of the build process and going over all localized TMX files, it generates the list of supported languages.

JSP code fetching a HTML &ltselect/> box with the list of supported languages

<span class="topCommand topCommandLabel"><%= LabelExtractor.get("rwa_loginLanguageSelectBoxLabel", locale) %></span>
<select
    class="topCommand"
    dojotType="dijit.form.FilteringSelect"
    id="languageSelector"
    onchange="switchLanguage();"
    title="<%= LabelExtractor.get("rwa_loginLanguageSelectBoxLabel", locale) %>"
><%
    ResourceBundle languageList = LocaleController.getLanguageListRB();
    Enumeration<String> keys = languageList.getKeys();
    while(keys.hasMoreElements()) {
        String key = keys.nextElement();%>
        <option<% if (key.equals(localeId)) { %> selected<% } %> value="<%= key %>"><%= languageList.getString(key) %></option><%
    } %>
</select>

The following figures illustrates the corresponding code in action.

Part of a login screen as defined with the default (English) TMX file.

Part of a login screen as defined with the French TMX file.

Conclusion

The two-tiers-utils library is offered with the BSD-like license. Anyone is free to use it for his own purposes. but I'll appreciate any feedback, contribution, and feature requirement.

See you on github.com ;)
“May the fork be with you.”

A+, Dom

Progress update

2009-09-18T23:39:00.001-04:00

Almost three months without publishing anything! I am definitively not proud of this score...

I have been busy on three fronts:

At work, I continue working on mobile related development. The easiest platform to play with is the Android one (see my post on Android Dev Phone 1) and I thrilled to see new handsets being made available, like the HTC Dream and Motorola Cliq, with their respective HTC Sense and Moto BLUR custom UIs (see videos below). Even if its SDK is not as rich as Android's one, even if the gadgets are not as polished as Android's ones, I also like developing for BlackBerry phones, like the BlackBerry Storm.

For my side project, running on Google App Engine infrastructure, it is progressing very well. Today, I reached a milestone: the first part of [still a secret] runs live. In terms of coding, it represents ~6,000 lines for the source files and ~12,000 lines for the test. I have always considered source vs test as being 50-50; it seems I should re-evaluate the balance to 1/3-2/3 ;)

Thanks to different contributors, I have developed a series of mock classes allowing to test transactions with BigTable, the database used by App Engine. A really neat piece of code I am going to describe here later.

On the social side, I am working with the board of Diku Dilenga Canada (board I am member of) to move as its Executive Director (still as a volunteer). The move has been inspired by Jean-Pierre Tchang, founder of IRIS Mundial. I hope this update will make Diku Dilenga (Canada) as successful as IRIS Mundial.

I had a lot of activities on this front recently: a fundraiser thanks to Louis Lamontagne walking between Saint Jean Pied de Port (France) and Santiago de Compostela (Spain), a trip of 780km, a first series of computers to be shipped to Kananga, Democratic Republic of the Congo, etc.

The plan to link the [still a secret] project with Diku Dilenga activities have been formally approved by the board during the summer. This side, there is a possibility I will do a presentation to the 2010 Africa/Middle East Regional Microcredit Summit (AMERMS) to be held in Nairobi, Kenya April 7-10, 2010. It would be nice to participate, isn't it?

Stay tuned, I should be back in few days with technical information about unit testing transactional code on Google App Engine ;)

A+, Dom
--
Videos:

HTC Hero, the first phone with the HTC Sense, a customized UI scheme on the top of Android.

Motorola Cliq, the first phone with Moto BLUR, a customized UI scheme on the top of Android.

BlackBerry Storm, first BlackBerry phone with a touch screen

JavaOne Conference

2009-06-12T14:55:00.003-04:00

Duke and myself ;)

It has been a long week in San Francisco while I was attending the JavaOne conference. Sessions started at 8:30 AM and finished after 9:30 PM!

Among all reviews that have been published, read this ones: Community day on Monday, Conference day 1 on Tuesday, Day 2 on Wednesday, Day 3 on Thursday, and Day 4 on Friday. Sun's website contains also a complete list of conference articles. Pictures of the event are available on Sun's Photo Center website.

I went there with two objectives:

See JavaFX in action;
Look at all efforts around JavaME and the MSA initiative.

JavaFX technology stack

Eric Klein, Vice President, Java Marketing, Sun Microsystems

I came without preconceived idea around JavaFX, just with my background as a JavaScript/Java developer and my knowledge of Flex and AIR.

The first deception came from looking at the scripting language: Man! they invented yet another language :( For sure, the JavaFX scripting language is nicely handled by NetBeans 6.5+ (except the code formatting) but new paradigms and new conventions are a big barrier to adoption to me. Can you figure out what the following code is doing?

public class Button extends CustomNode {
  public var up: Node;
  content: Node = up;
  public override function create(): Node {
    return Group {
      content: bind content
    }
}

If the visual effects to animate texts, pictures, and video are really great, the native support of a sound library is really missing! For example, I would expect to be able to synchronize KeyFrame objects with sound tracks but the KeyFrame.time attribute has to be set manually—not very flexible when it's time to change the sound track...

The pros are:

A coming visual editor to prepare clips;
A large library of image and video effects;
The multi-platform support, especially for mobile devices.

Platform dependent packaging is nicely handled: NetBeans project properties pane provides choices among {Standard Execution, Web Start Execution, Run in Browser, Run in Mobile Emulator}. As a Web developer, I am just sorry to see that the application window size cannot be expressed in percentage, that the auto-resizing is handled transparently, which is not better than usual Flex applications.

Last point: I have not seen how to invoke JavaFX handlers from JavaScript ones, and vice-versa, when the application is deployed to run in browsers. If you have a source for these information, please, drop the link in a comment.

JavaME technology stack

Christopher David, Rikko Sakaguchi and Patrik Olsson
during Sony Ericsson General Session

This was definitively the most interesting domain to me. During one session, it has been announced that 60% of shipped mobile devices in 2008 are Java-enabled. In 2009, the market share should grow up to 70%. In relation with the iPhone, Scott McNeally did this joke: the possible future Sun owner Larry Ellison might succeed to open the iPhone platform to Java because he is a well known friend of Steve Jobs.

Compared to the Java Standard Edition (J2SE) and the Java Enterprise Edition (J2EE), the Java Mobile Edition (J2ME) has more external contributors. Under the Mobile Service Architecture (MSA) initiative, a lot of mobile device manufacturers and telecommunication operators participate to the Java Community Process (JCP) to deliver Java Specification Requests (JSRs). Note that MSA itself is defined as a JSR: JSR 248. As of today, most of the recent phones are MSA 1.1 compliant (this is a mandatory requirement for the telco Orange, for example). Nokia and Sony Ericsson have shipped a lot of MSA-compliant handsets, LG, Samsung, and Motorola shipped very few ones. The standard MSA 2 (JSR 249) is being finalized and it contains the promising JSRs:

Additional APIs I imagine many developers are looking forward: JSR 257 Contactless communication API and JSR 229 Payment API.

MSA evolution and its JSR set
(from the MSA specification documentation)
(click to enlarge)

All major manufacturers have opened or are opening "App Stores" a-la Apple. They open also their development platforms. More companies will be able to adapt their software offering to mobile devices. Even Sony allows anyone to write application to run in Blu-ray Disc players. The main difficulty on developer-side is the fragmentation: there is no standard API allowing to discover the features supported by a device! Developers have to rely on each manufacgturer's feature list and on exception handling :(

The Blackberry platform is pretty well controlled and should be easy to develop on. Then follows Sony Ericsson which provides consistent phone classes (i.e. what works for one phone in the JP 8.4 class work for all phones in that class). The delivery of the Sun JavaME SDK 3.0 containing many third-party emulators (even one for Windows Mobile devices) added to better on-device deployment and on-device debugging capabilities, should motivate more and more developers.

I have not enough experience with Android (just got one Android dev phone two weeks ago) to compare it to the JavaME technology stack. I don't know neither about Symbian (Nokia devices) or LiMo (Motorola devices) platforms.

Exhibition hall

Besides the visit of mobile device manufacturers (RIM and Ericsson) booths, I visited:

Sun's project Fuji (open ESB) with a Web console using <canvas/> from HTML5, like Yahoo! Pipes.
Convergence, the Web client for Sun's communication suite, built on the top of Dojo toolkit ;)
INRIA (French national R&D lab) for its static code analysis Nit.
Isomorphic Software for its SmartClient Ajax RIA System.
eXo Platform (on OW2 booth) for its eXo Portal offering.
Liferay, Inc. for its eponymous portal.

Other discoveries

James Gosling and myself ;)

I attended very good presentations, like the opening keynote which was fun. Among the good presenters, I can mention (ordered alphabetically):

Ben Galbraith and Dion Almaer (see also Ajaxian.com), Mozilla.
BJ Hargrave, IBM, and Peter Kriens, aQute, about OSGi.
Doug Tidwell, IBM, who gave a funny speech about REST-WS* in the cloud.
Erik Hellman, Sony Ericsson.
James Gosling, Sun Microsystems.
Patrick Chanezon, Google, about Open Web.

If you have a Sun Developer Network (SDN) account (by the way, it's free), you can view the slides of the technical sessions at: http://developers.sun.com/learning/javaoneonline/.

Special mention

I want also to mention the call to developers by MifOS people who have been awarded by James Gosling during the Friday morning general session. This organization develops open source software for microfinance institutions (MFIs) to help them managing loans and borrowers (see demo). Really nice initiative started by the Grameem Bank!

Excerpt from James Gosling Toy Show report:

Microfinancing Through Java EE Technology

Gosling next introduced a group whose great innovation with Java technology was social and not technical. Sam Birney, engineering manager and Mifos alumnus, and Van Mittal-Hankle, senior software engineer at the Grameen Foundation, took the stage to receive Duke's Choice awards for their work using Java EE to serve 60,000 clients worldwide in microfinancing, a highly successful means of helping poor people get small loans and start businesses.

Mifos is open-source technology for microfinance that is spearheaded by the Grameen Foundation.

"Sometimes excellence comes not from technical innovation but in how technology is used," explained Gosling. "This is an example of using Java technology to really improve people's lives."

With an estimated 1.6 billion people left in the world who could benefit from microfinance, the men put out a call for volunteers to contribute to the Mifos project..

What's next?

What are my resolutions? Get a Mac as the development platform (Eclipse works on MacOS and I can use a Win7 image within VirtualBox), and start development on Java enabled phone (at least MSA 1.1 compliant).

A+, Dom

Internationalization of GAE applications

2009-06-10T15:08:00.000-04:00

Costs of badly planned internationalization

In my experience, internationalizing an application is very expensive if it is not planned upfront.

The first source of costs is due to developers who are used to lazily hard-coding labels. Extracting them at the end of the development process is always error prone because developers may have done assumptions on exact labels. Good regression test suites can help detecting such situations but they cannot avoid the cost of the required fix and its corresponding test runs. With late label extraction, developers without enough context tend to add extra dictionary entries. With the label extraction near the release milestone, developers in a rush and without the initial context tend also to produce non documented dictionaries—that is limit their re-usability and increase the difficulty of possible defect fixings.

Non localized Java code

protected String formulatePageIndex(int index, int total) {
    String out = "Page " + index;
    if(0 < total) {
        out += " of " + total;
    }
    return out;
}

In the example above, I have seen the extraction of the labels Page and of instead of Page %0 and Page %0 of %1. The quick extraction leads to the impossibility to invert the two arguments! Think about the people naming conventions: in many countries, the last name is displayed before the first name, in others the first name precedes the last name (%last %first compared to %first %last), and in Japan both names are printed without separator in between (%last%first).

Localized Java code

protected String formulatePageIndex(int index, int total) {
    // Get the resource bundle already initialized for the correct locale
    ResourceBundle rb = getCurrentResourceBundle();
    // Prepare values
    Object[] values = Object[] {Integer.valueOf(index), Integer.valueOf(total)};
    // Get the right localized label
    String label = rb.getString("PageIndexLabel_Page");
    if(0 < total) {
        label = rb.getString("PageIndexLabel_PageOf");
    }
    // Return the localized label with injected values
    return Message.format(label, values);
}

The second source of costs is due to the missed opportunities of deploying localized builds early in the development process. In Agile environments, we expect to get runnable builds on regular basis (at the end of each sprints, each 4 to 6 weeks, for example). And these fool-proof builds can be demoed to customers to get early feedback. If the development organization can work with translators iteratively, there are a lot of chance to detect localization defects while their fixing cost is not too high. In the past, I have seen product developments hugely hit when bi-directional languages (like Arabic and Hebrew) had been introduced...

Different aspects of the internationalization

Internationalization (i18n) [1] has two aspects:

The translation of the labels;
The localization (l10n) of these labels.

The localization takes into account the language and the country, sometimes with variants in a country. For example, the Spanish language spoken 19 identified countries. In Mexico (ES_MX), the language is slightly different from the one generally spoken in Spain (ES or ES_ES). In Spain, there are many regional languages like the Catalan (CA_ES). The different locales are normalized by the Unicode consortium (ISO-639 and ISO-3166). Codes are composed of a sequence of two letters for the language plus two letters for the country plus two letters for the region. If letters are missing after the language, most of programming languages fallback on common defaults.

In order to ease application localizations, Unicode references a Common Locale Data Repository (CLDR) [2]. This repository is used and updated by many companies like IBM, Sun, Microsoft, Oracle, etc. The repository describes rules on how to:

Localize currencies;
Localize metrics (distance, speed, temperature, etc.);
Localize dates and calendars.

As of today, I think only timezone definitions are still not centrally managed... This is especially bad because conversions between Universal Time (UTC) dates and local dates are operating system dependent (Sun Solaris have small differences with Microsoft Windows, for example). Many tools use the Unicode CDLR information. For example, each release of the Dojo toolkit use its information to provide the Calendar widget for 27 locales [3].

Internationalization with different programming languages

Almost all programming languages have ways to facilitate application globalization. Java provide resource bundles (*.properties file), Microsoft .Net has resource files (.rc files), Python has dictionaries, etc. If JavaScript lacks of native support for globalization, some libraries offer various support. To my knowledge, Dojo toolkit is the first providing a full support.
If developing an application on Google App Engine infrastructure can be done with only one programming language, Python or Java as this time of writing, it is highly possible that developers will use some JavaScript libraries to speed up their development. This is without counting the delivery of a similar program front-end as a native application (made with Adobe AIR, Microsoft .Net, C/C++, Groovy, etc.).

In different situations, I have seen developers moving manually label definitions from one environment to another one. Sometimes, definitions were left over, cluttering the system. In Agile environments, developers should focus on the requirements for the current sprint, leaving some tuning for later sprints. For example, at one point during the development, some labels defined in a JavaScript bundle might be moved to a Java bundle because the localization will be done server-side into a JSP file.

My solution is to put all labels in one localized central repository. The dispatch among the different programming languages is done at build time. When I looked for this repository format, my solution was selected against the following criteria:

Easily editable;
Has a standard format;
Usable by static validation processes;
Has excellent re-usability factors;
Easily extensible to new programming languages.

I chose the TMX format (TMX for Translation Memory eXchange [4]). This is an XML based format (good for edition, extensibility, and use by static validation tools) which has been defined to allow translation memory export/import between different translation tools like DejaVu. The XlDiff format would have been another good candidate.

The following table illustrates the flow of interactions between the different actors in a development team. This sequence diagram shows that, once the developers have delivered a first TMX file, testers and translators can work independently to push tested and localized builds to the customer. As explained later, if developers tune the TMX entries without updating the labels themselves, translators and testers (at least from the l10n point-of-view) can stay out of the loop—only steps [1, 7, 8] are replayed.

Simplified view of the overall interaction flow
Developers	Testers	Translators	Build process	End-users
1. Write labels in one language into the TMX file. These labels are extracted from design documents.
			2. Generate the application with for one locale.
	3. Produce a generic bundle to identify non extracted labels.
			4. Generate the application with for two locales.
				5. Use the application in one locale (switching to the `test` language is hidden).
		6. Use the initial TMX to produce `n` localized TMXs.
			7. Generate the application with for `2 + n` locales.
				8. Can use the application in `1 + n` locales.

The following code snippet shows how an entry into the base TMX file is defined.

Snippet of a translation unit definition for a TMX formatted file

<tu tuid="entry identifier" datatype="Text">
 <tuv xml:lang="locale identifier">
  <seg>localized content</seg>
 </tuv>
 <note>contextual information on the entry and relations with other entries</note>
 <prop type="x-tier">dojotk</prop>
 <prop type="x-tier">javarb</prop>
</tu>

The key features of the TMX format are:

The format can be validated with an external XSD (XML Schema Description);
One entry (tu: translation unit) can contain many localized contents (tuv: translation unit value);
Developers have a normalized placeholder (<note/>) to register contextual information;
Extensions are used by the build process to target the type of resource bundle to receive the localized label.

With such an approach, I have seen a drastic reduction of translation mistakes, especially thanks to the <note.> tag. Sometimes, graphical elements contain inter-related labels that cannot be grouped under a generic entity. The following set of elements illustrates the situation. The TMX approach saves translators headaches because they are simply informed about the relation between four entities.

Search:
The conversion from the TMX to the various resource bundles is done by an XSL-Transform. With the continuous integration handled by ant, the corresponding task generates the output after having appended the XSLT file coordinates to a copy of the TMX file and after asked for the transformation with the corresponding <xsl/> ant task. Depending on the machine performance, depending on the TMX file size, I found that the process can be time consuming. If this is your case too, I suggest you write your own little Java program to handle it. You can also use mine ;)

Stylesheet transforming label definitions for the Dojo toolkit

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 <xsl:output method="text" />
 <xsl:template match="/tmx/body">
  {
  <xsl:for-each select="tu">
    <xsl:for-each select="prop">
      <xsl:if test="@type='x-tier' and .='dojotk'">
        "<xsl:value-of select="../@tuid" />":"<xsl:value-of select="../tuv/seg" />",
      </xsl:if>
    </xsl:for-each>
  </xsl:for-each>
  "build", "@rwa.stageId@"}
 </xsl:template>
</xsl:stylesheet>

Use of the stylesheet above to convert Dojo toolkit related definitions from the TMX files by an Ant task [5]

<target name="convert-tmx">
  <style
    basedir="src/resources"
    destdir="src/resources"
    extension=".js"
    includes="*.tmx"
    style="src/resources/tmx2dojotkxsl"
  />
</target>

A+, Dom
--
Sources:

Introduction to internationalization and localization on Wikipedia.
Unicode Common Locale Data Repository (CLDR).
Dojo toolkit API: dojo.cldr, dojo.i18n, private dijit._Calendar, dojox.widget.Calendar, and dojox.widget.DailyCalendar.
Definition of the Translation Memory eXchange (TMX) format.
Reference of the XSLT/tyle task for Ant scripts.

Android Dev Phone 1 Setup

2009-06-05T18:05:00.004-04:00

To start investigations on mobile application development for Compuware, I have just acquired a first Android Dev Phone, also known as G1 [1]. Last week at Google I/O, Vic Gundotra delivered an Oprah event by offering to the audience a second generation Android phone, also known as G2 or HTC Magic [2, 3]. The G1 is more limited but it is still the only Android platform legally available.

With a bit of luck, few of 18 new phones Google expects [4] will be made available to developers during the year.

I have been able to activate the phone with the Fido pre-paid plan (10 $/month) [5]:

I inserted the SIM card as illustrated into the documentation, put the the battery in place, and connected the phone to the AC adapter.
Before signing in with my Google account, I had to create an Access Point Network (APN) entry:

Name: Fido
APN: internet.fido.ca
Username: fido
Password: fido
MCC: 302
MNC: 37

In some forums, it is reported that new Fido SIM cards use 370 as the MNC value.
A post of Olivier Fisher's blog [6] gives also the coordinates to connect to Rogers network, another GSM provider in Canada.
To limit interference, I deleted all pre-loaded APN entries (related to T-Mobile networks).
At one point, a popup asked me to enable data transfers. It is important to enable it and to activate the Data roaming, disregard how expensive are the costs for a prepaid plan.
Then I specified my Google account credentials and let the phone contacting Google servers via Fido network.
Once the activation has been successfully reported, I disabled the Data roaming, even before the synchronization of the applications {GMail, Contacts, Calendar} ended. The impact on my plan should be limited ;)
Then I added the description of my home Wi-Fi network.
I found the MAC address of the phone in the menu Settings > About phone > Status, with the entry near the end of the list. I used it to let my Wi-Fi controller accepting connections from the phone.
At this step, I was able to use my phone for regular calls over Fido network, and for data transfers over my Wi-Fi network.

The phone comes installed with Android 1.0 installed. I will blog later about updating the phone OS to the 1.5 version (also known as Cupcake)...

Update 2009/06/16:
Instructions on how to upgrade the Android dev phone to Android 1.5 is pusblished on HTC website: Flashing your Android Dev Phone with a Factory System Image.

Update 2009/07/15
Because of some restrictions to access Internet from the office, I have decided to pay for a 1GB/month data plan with Fido (30$/month). The activation has been made pretty quickly but none mentionned the following limitation:

On HTC website, you can see the network specifications for the G1: HSPA/WCDMA (US) on 1700/2100Mhz.
Fido/Rogers GSM only operates on 850/1900Mhz, so there's no possibility to go at a 3G speed in Canada!

Using this phone mainly for development purposes, it is not a blocking issue. It is just sad to not benefit from a better bandwidth...

A+, Dom
--
Sources:

Order the Android dev phone 1 from Android Market.
Techcrunch reports the Oprah moment by Vic Gundotra.
G2 review by MobileCrunch
Google expects 18 to 20 new phones on the market by the end of 2009.
Fido pre-paid plan.
Android G1 Phone in Canada on Rogers by Olivier Fisher. Posted comments are especially useful.

Le cap des 40 ans !

2009-05-31T23:34:00.015-04:00

1969-2009

Me voilà 40 ans révolu ! Ce 30 mai, j'ai eu le plaisir de recevoir de proches amis que j'ai rencontrés au long de mes 10 années passées à Montréal. Ce n'est pas tant le chiffre des 40 ans qui est magique, c'est plus celui des 10 ans et tous les événements qui se sont produits depuis.

En effet il y a 10 ans, ma femme Sophie et moi avons pris un aller simple Nantes-Montréal avec 2 valises, pendant que le reste de nos affaires traversait l'atlantique en bateau. Rapidement, nous avons loué un haut de duplex à Ville LaSalle. Et après quelques temps de vacances au Lac Saint Jean et dans le Saguenay, nous nous sommes mis à la recherche d'un boulot.

Pour moi, c'est la compagnie CS&T (plus tard renommée en Steltor) qui a décidé de me faire confiance. J'y ai d'abord réalisé une console d'administration Web pour le serveur de calendrier, un outil de collaboration instantanée et distribué avant l'heure. Je dois en partie mon recrutement à Patrice Lapierre, toujours employé par Oracle à Montréal ;)

Une fois acquise Steltor par Oracle, j'ai travaillé pour une équipe californienne. Mon expérience des applications Web a retenue d'Attila Bodis. Avec quelques autres, nous avons construit un environnement pour une application Web 2.0 avant même que le buzz existe ! Une première version a été commercialisé. Les débuts de la seconde version étaient très prometteurs, notamment grâce à la vision d'Amir Borna, et avec la bonne coordination de Vince Wu. Mais des aléas politiques ont cependant plombé le projet et l'équipe s'est quasiment dissoute...

Quand la semaine dernière, j'ai vu la présentation de Google Wave par Lars Rasmussen et Stephanie Hannon, j'ai trouvé beaucoup de similitudes avec ce que nous avions sur la planche de travail pour OCS (Oracle Collaboration Suite) en 2005-2006. Parce qu'Oracle évolue dans le monde de l'entreprise, les médias comme le courriel ou l'événement de calendrier devaient perdurer—j'attends à ce propos de voir quelle stratégie Google va offrir aux entreprises pour migrer leur legacy systems. Il y a sûrement des concepts OCSiens de l'époque qui s'appliqueraient à Wave aujourd'hui. Chapeau bas les gens d'en bas ;) car vous avez réussi où ceux d'en haut n'ont pas su persévérer !

C'est alors qu'IBM Rational m'a offert un poste d'architecte technique. Le projet n'était pas sensationnel (il est même mort depuis), mais les gens rencontrés et l'expérience de Big Blue ont été sans pareil. Chapeau à Steven Milstein, à Toufik Bahloul et Arthur Ryman.

Présentement, je suis chez Compuware dans un groupe d'experts multi-disciplinaires et j'aide à définir les stratégies de l'entreprise dans les domaines des applications Web et des mobiles. Évoluant dans une compagnie très conservatrice, certaines situations ont tendance à me frustrer royalement, mais les résultats que j'arrive à obtenir par ma persévérance font que finalement l'équilibre est positif. Je ne sais pas combien de temps cela va durer, mais j'en apprécie la valeur. Et je remercie Paul Czarnik, Mathieu Pageau et Abdel Belkarsi.

Du côté extra-professionnel, il y a eu mon passage dans le groupe RÉSULTATS, merci Sunnie Kim, pour lequel l'action des citoyens fait pression sur le gouvernement fédéral pour que l'aide au développement vise les plus pauvres. Maintenant, je suis impliqué dans Diku Dilenga au Canada, dont je remercie le co-fondateur de l'organisation en République démocratique du Congo révérend Tambwe Musangelu.

Je n'oublie pas non plus ma relativement récente entrée dans le monde du karaté au Dojo de Don Lorenzetti, pour le style Chito Ryu. C'est d'autant plus passionnant que toute la famille Derrien y participe : Sophie, Erwan, Goulven et moi-même chacun à des horaires différentes deux fois par semaine.

Merci Sophie, mes enfants, et mes amis pour votre amitié et les richesses que vous m'apportez. J'espère être à la hauteur et vous donner autant de plaisir.

A+, Dom

Revue du livre « le dip » de Seth Godin

2009-05-06T22:25:00.005-04:00

Image des Éditions du trésor caché.

Une fois n'est pas coutume, je prends le parti d'écrire une entrée de blogue en français. Ce qui me motive à le faire c'est la récente lecture du livre de Seth Godin qui n'était disponible qu'en français à la bibliothèque de mon quartier. Depuis quelques années, j'utilise la langue de Shakespeare mise au goût du jour par Ron McDonald pour mon travail. En plus, 95 % de mes sources d'information sont anglophones. C'est donc assez déroutant de lire du Seth Godin en français, mais aussi très rafraîchissant.

Le thème de ce court livre publié en 2007 (100 pages au format d'un livre de poche) traite de la description de différents profils d'activité et sur les stratégies à adopter pour avoir du succès. Seth est très honnête sur la notion de succès : s'il incite le lecteur à être le « meilleur du monde », il précise que ce « monde » est propre à chacun et qu'il peut varier au court du temps. Voici les trois profils d'activités qu'il identifie :

Il y a le profil que tous connaissent quand ils démarrent une nouvelle activité, où l'excitation rend les choses faciles. Cette phase est suivie d'un creux où l'évolution est lente et pénible. Quand l'obstination porte ses fruits, la sortie de ce creux est souvent couronnée de succès. Seth appelle ce creux le dip.
Il y a le profil de la falaise où les activités progressent bien jusqu'à s'écrouler totalement. Seth fait le parallèle avec le monde de la presse écrite : tout allait bien jusque récemment mais la démocratisation d'Internet a tout chamboulé. Plus besoin d'attendre le quotidien du lendemain pour connaître les nouvelles car la plupart des grands médias [2] diffusent sur Internet. Plus besoin du journal local pour vendre ou acheter un bien car les services comme craigList [3] offre de meilleures couvertures. Plus besoin du mensuel pour voir des photographies extraordinaires, il suffit de se rendre sur Flickr [4]...
Il y a enfin le profil de la progression plate, infiniment plate, sans perspective de succès. C'est le chemin vers une voie sans issue.

Seth explique que les deux derniers profils sont à abandonner rapidement, car il n'y a que de la médiocrité à en retirer. Le premier profil, celui avec le dip, est à évaluer consciencieusement.

Il arrive des situations, ou malgré les perspectives de réussite évidentes, le palier à traverser pour atteindre le succès est trop pénible. En fait, Seth écrit que la péniblité du palier est proportionnel au succès à atteindre. Si ce n'était pas pénible tout le monde serait couronné de succès, ce qui n'est évidement pas le cas. Parmi ses exemples, Seth cite le cas des grands sportifs, des dirigeants de grandes sociétés, des créateurs, etc. Il dit, par exemple, que c'est facile d'être un grand patron, mais c'est difficile de gravir les échelons pour arriver à ce poste.

La partie qui m'interpelle particulièrement, c'est son discours sur la nécessité d'identifier les situations de dip que l'on peut traverser et celles qui sont insurmontables compte tenu de nos propres contraintes. Quand une situation de dip trop difficiles, tout comme celle de la falaise ou de la progression, il est important d'y renoncer tôt. Il affirme même qu'il est important d'identifier les critères de renonciation avant d'être dans la situation de blocage pour être sûr de quitter pour les bonnes raisons. Si les raisons sont trouvées alors qu'on est dans le trouble, elles provoqueront des remords par la suite. Avec l'analogie d'un marathon, Seth dit que par exemple se mettre des objectifs de temps de parcours et de s'évaluer par rapport à eux plutôt qu'aux sensations du moment (crampes, fringale, etc.) donne de meilleurs chances de succès. Si on déjà parcouru 35 km, les sensations rendront la fin plus pénible à supporter s'il n'y a pas l'estimation initiale de succès établie en fonction de temps établis par avance.

Seth dit que le renoncement à des situations trop difficiles laisse la chance à la réalisation d'autres défis plus à sa mesure. Comme le critère de réussite est d'être « le meilleur de son monde », peu importe que d'autres pensent que l'abandon d'un situation soit un échec. Il arrive même que ceux qui vous jugeront mal sont eux-mêmes incapable de sortir du dip, qu'ils nagent dans une médiocrité relative.

Si je fais un peu d'introspection, j'en arrive aux constats suivants :

Autant mon immigration au Canada, mes derniers changements d'emploi (de Oracle à IBM Rational, puis à Compuware [5]), ainsi que le déplacement de mes activités para-sociales (de RÉSULTATS à Diku Dilenga [6]) ont été motivés par des constats de dip devenus infranchissables (en tout cas, avec des perspectives moindres lorsque comparées à celles offertes par les changements).
J'aime les difficultés, les challenges qui sont rudes, dans la mesure où la perspective de grandir est bonne. Instinctivement avant, clairement maintenant, j'abandonne les situations qui sont sans issue, ou pour lesquelles les compromis à faire sont sans commune mesure avec la satisfaction à en retirer. Cette attitude peut paraître fruste et égoïste mais c'est relatif : j'imagine qu'en étant le « meilleur de mon monde » cela bénéficie aux gens de « mon monde ». Ce qui est mieux, à mon avis, que d'affecter mon entourage parce que je reste « un médiocre dans mon monde ».

Étonnant comme un petit livre peut faire gamberger, n'est-ce pas ? À votre tour d'y consacrer quelques heures et de partager votre propre cheminement en me laissant un commentaire.

Pour le plaisir, voici la vidéo d'une présentation donnée par Seth Godin en 2003 [7]. Tout ce qui dit est du « gros bon sens » et c'est ce qui le rend pertinent. Le livre est du même acabit.

Seth Godin at Gel 2006 from Gel Conference on Vimeo.

A+, Dom
--
Sources :

Le livre décrit sur le site de Seth, et la présentation francophone sur le site des Éditions du trésor caché.
Google News, Associated Press (AP), Agence France Presse (AFP), Cable News Network (CNN), Radio de l'information (RDI, du réseau de Radio Canada), France24, Al-Jazeera, etc.
craiglist, kijij, lespac, etc.
Flickr, PicasaWeb, PhotoBucket, etc.
Oracle, IBM Rational, Compuware.
RÉSULTATS, Diku Dilenga.
Entrée sur le blog de Seth : This is broken.

Meet you at JavaOne Conference

2009-04-29T17:45:00.000-04:00

As the title let you imagine, I will be in San Francisco the week of June 2-5 for the JavaOne 2009 conference.

It will be my first visit to the event, not to the Moscone Center. While working for Oracle, I attended one Open World event there. With the recent acquisition (not officially closed yet) of Sun by Oracle, I am going to go in California for Oracle again ;)

With another Compuware colleague, we will mainly focus on mobile related sessions, but cloud computing and rich interactive application related ones will have my attention too. With sessions finishing at 10:20 PM, days will be surely long. I expect a fruitful trip.

Are you going to attend too? Tweet me your schedule to meet you there ;)

A+, Dom

Agile: SCRUM is Hype, but XP is More Important...

2009-04-13T19:52:00.001-04:00

(This post is part of the series Web Application on Resources in the Cloud.)

I have been doing “Agile development” for more than 5 years. I am used to saying that an organization is Agile at the level of its weakest element. So I cannot claim having worked on any fully Agile projects. However, I have always tried to apply as many as possible Agile principles to my work. This blog entry goes over different practices and identifies the ones that worked best for me and my teams.

Agile

The Agile methodology is a not pure invention, this is the compilation of best directives gathered from various practices:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Agile principles and sub-principles have been defined by a group of technical leaders: Beck from eXtreme Programming (XP), Schwaber (Scrum), etc. The Agile Manifesto [1] is the result of their collaboration.

Scrum

“Scrum is a lightweight Iterative and Incremental (Agile) Development Method that focuses on delivering rapidly the highest priority features based on business value.” It has been defined by Ken Schwaber and Jeff Sutherland in early 1990s.

Scrum promotes high collaboration and transparency. There are different backlogs helping delivering the best business values at each iteration. Capturing and integrating feedback (from business users, stakeholders, developers, testers, etc.) is a recurrent task. Deliveries occur often and their progression is continuously monitored.

Scrum in Under 10 Minutes by Hamid Shojaee

The points I really like about Scrum:

Task reviews done with all actors, in Poker Planning [2] sessions, for example.
Product designed, coded AND tested during the Sprint.
Sprint deliveries are workable products, with limited/disabled features, but working without blocking issues.
Defined roles: Product Owner, Scrum Master (aka Project Manager), and Project Team (composed of cross-functional skills: dev., QA, DBA, Rel. Eng., etc.).

Pig and Chicken are traditional roles in the Agile teams [3].

eXtreme Programming

While Scrum is mainly managers (chicken) oriented, eXtreme Programming (XP) focuses more on do-ers (pigs).

XP is more a matter of having the right tools and having real technical exchanges within the Scrum team. For example, XP strongly suggests the adoption of peer-programming: two developers per computers, one coding and the other thinking about the coding and correcting the code on-the-fly.

Applying peer programming in teams with actors from various backgrounds is sometimes too constraining. Matching peers is a difficult exercise. However, enforcing peer code reviews allows to get almost the same benefits without too much frustration. With code reviews, junior developers can see seniors' work in action, and senior developers can learn new programming paradigms. I found it's good also for the team cohesion, because team members really know about each others' work.

Among the practices XP incites to follow, there are:

Continuous Integration: every time a developer or a tester delivers new materials, a complete build process starts to compile, package and test the entire application. Ideally, the build time is short enough to prevent committers to start any other tasks, so they can fix it right away. A safe strategy is to put “fixing the build” as the top priority whenever a problem occurs.
Unit testing and code coverage: when developers write unit tests, they provide the first piece of code consuming their own code, and experience shows that it really helps delivering better code. Unit tests without code coverage measurements does not mean much. And not trying to reach 100% coverage leaves too much space to defective code... Using mock objects [4] is an essential tool to test accurately. Test Driven Development (TDD) methodology is pushing this practice up to writing the tests before the code.
Continuous refactoring: during each sprint, developers should focus on the immediate requirements, because they have very little control on future sprints (the product owner can adjust the development plan anytime). This is sometimes difficult to limit them to their immediate tasks because many do not like the perspective of having to rewrite their code later. Investing in tools like IntelliJ IDEA which provides extended refactoring features is really worth it because developers can adapt their code efficiently while being secured by the continuously growing regression test suites.

Best of both approaches

In medium to big companies, they are often many layers of management. In such environments, when managers should be facilitators[5], they often add weight to the processes.

About the issues in shipped products, here is an anecdote about IBM:

An internal team reviewed the quality of the released products and came to the initial conclusion that minor and maintenance releases contain more flaws than major releases. The conclusion was made after studying the number of defects reported by customers: this number was sometimes twice higher for intermediate releases. But the team pushed its investigation further and polled many customers. At the end, it appears that very few customers were installing major releases immediately, most of them would wait for the first maintenance release to deploy in pre-production environments (one stage before production one).

In this story, IBM used the results of this study to size and train the support teams according to the product release plans. As you can expect, more support people are trained and made available on releases following major ones. It did not help IBM delivering better products up front, it mostly smoothed the experience of customers reporting problems ;)

Development labs are often known for delivering over the budget, over the allocated time, and with too many issues. Many times, I have seen the maintenance being operated by specific teams, without relation with the development ones. In such environments, development teams focus on delivering features and maintenance teams fix issues: each team has its own budget and life goes on!

The combination of the relatively poor delivered software, the accumulation of managers, and the Scrum burn-down chart (the chart that shows how the work progress on a daily basis [6]) favors Scrum adoption in IT organizations.

Burn down chart sample

My problem with Scrum as I see it in action is related to its usage by managers: it is a one-way communication channel for them to put the pressure on Scrum teams. And because Scrum is task oriented, if the task set is incomplete (or deliberately cut through), these managers mostly follow the feature completion rate, and sometimes the defect detection and fixing rates.

In my experience, with organizations transitioning from waterfall methodologies to Scrum, the feature check list has always precedence on the quality check list! If tasks have a risk to break the deadline, the test efforts are cut. And because these organizations have very few ways to measure the delivered quality (because they adopted Scrum but refused to invest in XP), results are not really better for customers...

This is why I think it is important to balance the importance of Scrum with the one of XP, why as the same time managers should tools to monitor the work progress, Scrum teams should publish quality metrics about all delivered pieces of code. With both sides being instrumented, it is be easier to identify decision impacts and product owners can make informed decisions.

A+, Dom
--
Sources:

Principles of the Agile Manifesto, and definition of Agile methodology on Wikipedia.
Description of Poker Planing on Wikipedia.
The Classic Story of the Chicken and Pig on ImplementingScrum.com, and the role definition by Nick Malik.
Mock object definition on Wikipedia, and Chapter 7: Testing in isolation with mock objects from the book JUnit In Action, by Vincent Massol.
My personal view on the facilitator role managers should have: Manager Attitude.
Burn-down chart described as a Scrum artifact on Wikipedia and Burn Baby Burn on ImplementingScrum.com.

Google App Engine Meets Java

2009-04-10T19:12:00.011-04:00

On April 7, three days ago, Google people announced and demonstrated the new support of the Java programming language in Google App Engine ecosystem [1].

Before starting my side project [2], I was mostly a Java[3] adopter on back-end servers. Java is widely supported and have a large and active contributing community (to its core via the Java Community Process (JCP) [3] or to 3rd party libraries).

Once I decided to go with GAE, I invested a bit in improving my knowledge of Python [4], for example by looking at the WSGI specification [4] and at Django [4]. I have been impressed about the integration done by GAE people, about how easy it is to program complex steps in very few lines! My favorite part is the main function to dispatching events:

# -*- coding: utf-8 -*-

# Handlers
...

# Dispatcher
application = webapp.WSGIApplication(
    [
        ('/API/requests', RequestList),
        (r'^/API/requests/(.+)$', Request),
    ],
    debug=True
)

def main():
    run_wsgi_app(application)

if __name__ == "__main__":
    main()

Implementing a REST API, the RequestList class defines get(self) to return selected resources, and put(self) to create the proposed resource. The Request class defines get(self, key) for the identified resource retrieval, post(self, key) to update the identified resource, and delete(self, key) to delete the identified resource.

In the J2EE world, with the web.xml file forwarding /requests URLs to a servlet (as done in the app.yaml file), the servlet code will have to get the URI (with HttpServletRequest.getPathInfo()) and will have to parse it in order to detect the possible request identifier and the possible version number. IMO, Python offers slicker interface.

Another example is the support of the JPA and the JDO [5] specifications: few annotations decorating the Data Transfer Object (DTO) class definitions allow GAE/J to deal with the persistence layer (i.e. BigTable). Compared with the Python model definitions, the getters and setters plus the annotations seem overkill, but are necessary.

With Python allowing to rely on a more compact code, why would I switch to Java?

Even if I gave directions on how to test GAE/P applications [6], I should admit testing GAE/J code is easier: JUnit is de facto standard copied by many frameworks, and JCoverage is the tool helping to determine the quality of these unit tests. While working on an open source project, with possibly contributors from various horizons, relying on a strong testing infrastructure is a top priority. It is then possible I will go over the Java tradeoff in a near future...
The Java Virtual Machine (JVM) ported on GAE opens the door to many other languages, as reported by App Engine Fan [7]. I have a personal interest in the port of JavaScript language, processed by Rhino, the Mozilla JavaScript engine. I would be nice to be able to run the Dojo build process on GAE itself ;)

A+, Dom
--
Sources:

New features and an early look at Java for App Engine on Google official blog and Seriously this time, the new language on App Engine: Java™ of Google App Engine Blog.
Announcement in preparation this time.
Java history on Wikipedia, on Sun microsystems website, on IBM developerWorks website; site of the Java Community Process.
Key components of GAE/P: Python, Django and its template language, WSGI
Description of standards supported on GAE/J: Standards-based Persistence For Java™ Apps On Google App Engine.
Automatic Testing of GAE Applications from the series Web Application on Resources in the Cloud.
Hand me the Kool-Aid :-) by App Engine Fan.

Guy Kawasaki's keynote in Montréal

2009-03-26T21:19:00.014-04:00

Yesterday, I attended a keynote presented by Guy Kawasaki in Montréal. It was a real fun to listen to him. He is a great communicator.

I found many commonalities with my blog post Career Advice'08, and with Tim O'Reilly's message “Work On Stuff That Matters.” So the gap to adhere to Guy's recommendations was small.

Here is a report written by Jean-François Ferland, for the magazine Direction Informatique. If I want to reproduce it here, it's mainly because it is then ad-free ;) You can find the original version on Direction Informatique website.

Good reading,
A+, Dom

Kawasaki: innover est un art... sous le signe de l'humour
26/03/2009 - Jean-François Ferland

Guy Kawasaki, un ancien « évangéliste » d'Apple, suggère aux entreprises en démarrage dix règles pour se démarquer auprès des consommateurs et se faire remarquer par les anges investisseurs. Compte-rendu d'une allocution qui a bien fait rire l'auditoire.

Guy Kawasaki
Photo: David Sifry.
Licence: CC-by-2.0

La plupart des allocutions se suivent et se ressemblent. Or, il arrive qu'un conférencier se démarque par ses propos, par son ton et par la forme de son allocution, sans causer l'ennui ou un sentiment de déjà-vu.

Au Club St-James de Montréal, dans le cadre de la deuxième édition de l'événement Capital Innovation qui réunissait des entreprises en démarrage du Québec et des investisseurs, le conférencier Guy Kawasaki a décidément retenu l'attention des invités.

M. Kawasaki, un Californien d'origine hawaïenne, est le fondateur de Garage Technology Ventures, une entreprise qui fait du maillage entre les anges investisseurs et les entreprises en démarrage, en cherchant « deux gars, un gars et une fille ou deux filles dans un garage qui développent 'la prochaine chose importante' ».

Il est surtout connu pour son ancien rôle « d'évangéliste » pour les nouvelles technologies chez Apple lors de son deuxième séjour chez le fabricant de produits de 1995 à 1997. Lors de son premier passage, de 1983 à 1987, dans la division Macintosh, son rôle était de convaincre les gens d'écrire des logiciels pour les ordinateurs d'une entreprise « qui comptait le plus grand nombre d'égomaniaques, un record qui a été depuis battu par Google ».

Après avoir clamé son amour pour le hockey, un sport qu'il a commencé à pratiquer en Californie en même temps que ses fils à l'âge de 48 ans (!), M. Kawasaki a entamé sa conférence qui consistait en dix recommandations à l'intention des entreprises en démarrage dans le domaine des TIC.

Ses propos étaient parsemés d'humour, ce qui a plu à la foule de plus d'une centaine de personnes. « J'ai écouté bien des chefs de la direction lors de conférences comme le Comdex. Souvent ils étaient 'poches' et prenaient beaucoup de temps », a-t-il lancé en riant.

Voici brièvement ses dix recommandations portant sur l'art de l'innovation, accompagnées de courtes explications (et de remarques amusantes).

1. Faire quelque chose qui a du sens M. Kawasaki affirme qu'une entreprise en démarrage ou un innovateur doit vouloir faire quelque chose en premier lieu avec l'intention que cela fera du sens, en opposition à vouloir faire de l'argent avant tout, ce qui sera une conséquence naturelle de la première intention.

« Si une entreprise est démarrée avant tout pour faire de l'argent, elle attirera les mauvais cofondateurs, et les détenteurs de MBA sont les pires, a dit M. Kawasaki. Comment évalue-t-on la valeur d'une entreprise en prédémarrage? Ma règle est que chaque ingénieur à temps plein fait monter sa valeur d'un demi-million, mais chaque MBA la fait baisser d'un demi-million. »

2. Se faire un mantra M. Kawasaki s'est demandé à voix haute pourquoi les entreprises ne pouvaient décrire leur raison d'être en deux ou trois mots. Il a décrit la méthode nord-américaine de création d'un énoncé de mission, où les dirigeants d'une entreprise de réunissent deux jours dans un hôtel près d'un terrain de golf, avec un consultant en motivation « parce que personne dans l'équipe ne sait comment communiquer ». La première journée est consacrée à faire des activités de mise en confiance et la deuxième à écrire des idées au crayon feutre sur du papier adossé à un présentoir.

« On tente alors d'énoncer ce qui est bon pour les actionnaires, les dirigeants, les employés, les consommateurs, les baleines et les dauphins. C'est souvent trop long, il y a trop d'expertise et on ne peut comprendre si on enlève le nom de l'entreprise. Il faut dire en trois mots ce que l'on fait », a-t-il dit.

3. Sauter sur la prochaine courbe M. Kawasaki a affirmé que l'innovation survient lorsqu'une organisation sort de la trajectoire qu'elle suit ou qu'elle saute dans une nouvelle courbe. Il a donné l'exemple des coupeurs de glace des années 1900, de la version 2.0 à l'ère des usines de fabrication de glace, puis de la version 3.0 des réfrigérateurs à la maison. « Aucun des coupeurs de glace n'a bâti de fabrique de glace », a-t-il souligné, et sont carrément disparus.

4. Rouler les dés L'innovation prend forme lorsque l'entreprise prend le risque de faire le saut vers une autre courbe. En faisant un acronyme avec le mot Dicee - une altération inventée du mot « dé » en anglais, dont toutes les définitions sur Internet réfèrent au conférencier - M. Kawasaki a suggéré que l'innovation impliquait de la profondeur (Deep) par l'ajout de fonctions, de l'Intelligence lorsqu'on soulage une difficulté pour le consommateur, une expérience totale (Complete), de l'Élégance parce que le produit fonctionne lorsqu'on le branche et de l'Émotivité parce que les personnes l'aiment ou le détestent, sans zone grise.

5. Lancer maintenant, corrigez plus tard C'est ainsi qu'on pourrait traduire l'expression Don't worry, be crappy - inspirée par la chanson de Bobby McFerrin. M. Kawasaki a affirmé qu'une innovation a forcément des défauts et que ceux qui attendent qu'elle soit parfaite ne font pas de ventes entre-temps. « Le produit mis en marché ne doit pas être tout mauvais (crap), mais avoir juste un peu de mauvais. Le Apple 128 était un mauvais produit révolutionnaire! »

6. Polariser les gens M. Kawasaki a répété qu'une innovation audacieuse sera inévitablement aimée ou détestée par les gens, en donnant l'exemple de l'enregistreur numérique personnel Tivo que n'aiment pas les agences de pubs parce qu'elle permet d'éviter de regarder des messages publicitaires. « Le véhicule Scion de Toyota est vu comme étant cool par les gens de 27 ans et comme un réfrigérateur par les gens de 55 ans. On ne peut pas plaire à tout le monde, mais il ne faut pas en faire fâcher de façon intentionnelle. Cela n'arrive jamais que tout le monde aime un produit. »

7. Laisser cent fleurs éclore M. Kawasaki a indiqué qu'un produit innovant peut mener à des utilisations non intentionnelles, par des utilisateurs qui n'étaient pas ciblés à l'origine. Il donne l'exemple d'une crème de la compagnie Avon qui rend la peau douce, mais que les mères utilisent... comme chasse-insectes! Il a aussi évoqué la console de jeu vidéo Wii de Nintendo, destiné aux enfants, qui fait un malheur auprès des personnes âgées.

« Si cela vous arrive, prenez l'argent!, dit-il en riant. En 1984, Apple pensait que le Macintosh servirait au calcul dans les chiffriers, aux bases de données et au traitement de texte. Arrive PageMaker, qui a créé l'éditique et a sauvé Apple. Sans l'éditique, nous écouterions encore de la musique sur des cassettes de 60 minutes! »

« En ingénierie, je recommande d'aller voir ceux qui achètent les produits pour savoir ce qu'ils veulent. Chez Apple nous avions demandé aux entreprises Fortune 500 pourquoi elles n'achetaient pas nos ordinateurs. Nous leur avons créé un pilote d'impression, comme ils avaient suggéré, mais ensuite ils ont trouvé d'autres excuses... », a-t-il ajouté.

8. Vivre dans le déni M. Kawasaki a indiqué que la chose la plus difficile à faire en innovation est de refuser d'écouter ceux qui diront que c'est impossible à faire, que personne n'achètera le produit ou que personne ne fournira du financement. « On vous donnera 60 raisons. Ignorez-les. Mais une fois que le produit est lancé, virez votre capot et passez en mode 'écoute'. [Entre ces deux approches,] c'est le passage en zone neutre qui est le plus difficile », a-t-il confié. Comme au hockey.

9. Trouvez votre niche M. Kawasaki a évoqué un graphique pour une innovation où l'axe vertical décrit son niveau d'unicité et l'axe horizontal sa valeur, en affirmant que l'entreprise veut se situer en haut et à droite. « Un produit qui est unique et n'a pas de valeur ne doit pas exister. C'est comme offrir le curling aux États-Unis! »

Pour décrire une innovation qui n'a ni valeur ni unicité, M. Kawasaki a relaté le cas de Pets.com qui vendait en ligne de la nourriture pour chien. « L'enjeu en était un de gestion de la chaîne d'approvisionnement. Ils se disaient capables d'éliminer les magasins, cet intermédiaire qui prenait une marge de 25 %, en livrant directement aux propriétaires de chiens. Mais des vaches mortes en conserve pèsent lourd [en frais de livraison]. C'était plus cher et pas plus pratique... »

10. La règle du 10-20-30 La dernière règle suggérée par M. Kawasaki aux entreprises en TIC en est une qui servira lors des présentations aux anges investisseurs. « Utilisez 10 diapositives. Ne lisez pas vos diapositives - les gens savent lire. Le mieux est de ne pas avoir de diapositives! Aussi, expliquez votre produit ou votre projet en 20 minutes. De toute façon, 95 % des gens prennent 40 minutes d'une présentation d'une heure pour brancher leur portable avec leur projecteur. »

« Enfin, utilisez une taille de caractères de 30 points. Prenez la plus vieille personne dans l'auditoire, divisez son âge en deux et vous aurez la taille idéale. Comme les anges investisseurs deviennent plus jeunes, bientôt vous utiliserez une taille de 8 points! » a ajouté le conférencier, ce qui a suscité l'hilarité dans la salle.

(En bonus) 11. Ne laissez pas les « bozos » vous décourager En terminant, M. Kawasaki a suggéré aux entrepreneurs de ne pas se laisser abattre par les clowns qui les décourageront.

« Il y a deux types de clowns : le premier est mal coiffé, n'a pas d'aptitudes sociales et démontre qu'il est un perdant. Le deuxième, qui conduit une voiture allemande et porte un beau complet, est le plus dangereux. La moitié du temps, il est devenu riche et célèbre par chance », a-t-il déclaré.

Au terme de cette allocution, nous pourrions recommander aux entreprises une douzième règle : faites des présentations amusantes et animées comme celle de M. Kawasaki!

Jean-François Ferland est journaliste au magazine Direction informatique.

MVC Pattern and REST API Applied to GAE Applications

2009-03-17T21:35:00.006-04:00

(This post is part of the series Web Application on Resources in the Cloud.)

I have been developing applications for various categories of end-users since 1990. I coded front-ends in X/Motif [1] for Solaris, then in Borland OWL and Windows MFC for Windows [2], then HTML/JavaScript for Web browsers. Most of the time for good reasons, JavaScript has been considered as a hacking language: to do quick and dirty fixes. Anyway, I have always been able to implement the MVC pattern:

My first enterprise Web applications was relying on a back-end written in C++ for a FastCGI run by Apache [3]. I wrote a HTML template language which, in some ways, was similar to the JSP concept [4].
The second version of this administrative console was relying on JavaScript for the rendering (a-la Dojo parser) and was using frames to keep context and to handle asynchronous communication.
Then I learned about XMLHttpRequest [5], and with a great team, I started building a JavaScript framework to be able to handle complex operations and screen organizations within an unique Web page. We came up with a large widget library and a custom set of data models.
After a job change, I discovered Dojo [6] in its early days (0.2 to 0.4) and I closely followed the path to get to 1.0. With Dojo, I am now able to relax client-side because the widget library is really huge, while still easy to extend, and it has advanced data binding capabilities. Now, I can focus on the middle-tier to build efficient REST APIs.

Among the design patterns [7], Model-View-Controller (MVC) is maybe one of the most complex (it is a combination of many basic patterns: Strategy, Composite, Observer) and maybe the one with the most various interpretations. I did write my own guidelines when I ported the MVC pattern browser-side (proprietary work). Today, Kris Zip blog entry on SitePen side summarizes nicely my approach: Client/Server Model on the Web. The following diagrams illustrate the strategy evolution.

Credits: SitePen.

In Google App Engine documentation, Django [8] is the template language proposed to separate the View building from the Model manipulation. Django implements the “Traditional Web Application” approach illustrated above.

My initial concern about the traditional approach is the absence of a clean and public API to control the Model. It is not rare that APIs are considered as add-ons, as optional features. IMHO, APIs should be among the first defined elements: it helps defining the scope of the work, it helps defining iterations (in the Agile development spirit), it helps writing tests up-front, it helps isolating bottlenecks.

With the move of the MVC pattern browser-side, the need to define a server-side API becomes obvious. The Model is now ubiquitous: interactive objects client-side have no direct interaction with the back-end server, they just interact with the Model proxy. This proxy can fetch data on demand, can pre-fetch data, forwards most of the update requests immediately but can delay or abort (to be replayed later) idempotent ones.

My favorite API template for the server-side logic is the RESTful API [9]. It is simple to implement, simple to mock, and simple to test. For my side project [10], the repository contains descriptions of products made available by micro entrepreneurs (see table 1).

Table 1: Samples of RESTful HTTP requests
Verb	URL pattern	Description
GET	`/API/products`	Return HTTP code 200 and the list of all products, or the list of the ones matching the specified criteria.
GET	`/API/products/{productId}`	Return HTTP code 301 with the versioned URL for the identified product, or HTTP code 404 if the identified product is not found.
GET	`/API/products/{productId}/{version}`	Return HTTP code 200 and the attributes of the identified product for the specified version, or HTTP code 301 "Move Permanently" with a URL containing the new `version` information, or HTTP code 404 if the product is not found.
DELETE	`/API/products/{productId}/{version}`	Return HTTP code 200 if the deletion is successful, or HTTP code 303 or 404 if needed.
POST	`/API/products`	Return HTTP code 201 if the creation is successful with the new product identifier and its version number.
PUT	`/API/products/{productId}/{version}`	Return HTTP code 200 if the update is successful with the new product version number, or HTTP codes 303 or 404 if needed.

Table 2: Partial list of HTTP status codes (see details in [11]).
HTTP Code	Description
200	OK
201	Created
301	Moved Permanently
303	See Other “The response to the request can be found under another URI using a GET method. When received in response to a PUT or POST, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.”
304	Not Modified
307	Temporary Redirect
400	Bad Request
401	Unauthorized
404	Resource Not Found
410	Gone
500	Internal Server Error
501	Not Implemented

Parsing RESTful API for Google App Engine application is not difficult. It is just a matter of using regular expressions in the app.yaml configuration file and in the corresponding Python script file.

application: prod-cons
version: 1
runtime: python
api_version: 1

handlers:
- url: /API/products.*
  script: py/products.py

- url: /html
  static_dir: html

- url: /.*
  static_files: html/redirect.html
  upload: html/redirect.html

Code 1: Excerpt of the app.yaml configuration file.

# -*- coding: utf-8 -*-

import os

from google.appengine.api import users
from google.appengine.ext import db
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app

from prodcons import common
from prodcons import model

class ProductList(webapp.RequestHandler):
    def get(self):
        # Get all products
        # ...
    def post(self, productId, version):
        # Create new product
        # ...

class Product(webapp.RequestHandler):
    def get(self, productId, version):
        # Get identified product
        # ...
    def delete(self, productId, version):
        # Delete identified product
        # ...
    def put(self, productId, version):
        # Update identified product
        # ...

application = webapp.WSGIApplication(
    [
        ('/API/products', ProductList),
        (r'^/API/products/(\w+)/(\d+)$', Product),
    ],
    debug=True
)

def main():
    # Global initializations
    # ...
    run_wsgi_app(application)

if __name__ == "__main__":
    main()

Code 2: Excerpt of the Python file processing product-related requests

Note that the second code sample shows an ideal situation. In reality, I had to change the verb PUT when updating product definitions because the method self.request.get() cannot extract information from the stream—it does only work for GET and POST verbs. The corresponding client-side code relies on dojo.xhrPost() instead of dojo.xhrPut(). If you know the fix or a work-around, do not hesitate to post a comment ;)

While developing application front-ends, developers should always rely on the MVC pattern to separate the data from user interface, to separate the data flows from the interaction processing. IMHO, organizing the server-side interface as a RESTful API is very clean and efficient. If you use Dojo to build your JavaScript application, you can even rely on their implementation of various RESTful data sources [12] to simplify your work.

Credits: SitePen

Pushing the MVC pattern browser-side has nasty side-effects when too much information are managed by the Model proxy:

Large data sets consume a lot of memory.
HTTP connections being a rare resource sometimes unreliable, rescheduling requests (important ones first, non important to be replayed later) or replaying requests (because Microsoft Internet Explorer status reports WSAETIMEDOUT for example) complexify the data flows.
Fine grain API consumes a lot of bandwidth especiallywhen the ratio data vs envelope is negative).
Applications have often too few entry points, hiding then the benefit of one URI per resource (intrinsic REST value).
In highly volatile environments, data synchronization become rapidly a bottleneck if there is no push channel.

So the application performance (response time and memory consumption) should be carefully monitored during the development. If applications with the MVC pattern organized browser-side and relying on RESTful APIs cannot do everything, they are definitively worth prototyping before starting the development of any enterprise application or application for high availability environments.

A+, Dom
--
Sources:

Motif definition on wikipedia and online book introducing X/Motif programming.
History of Borland Object Windows Library (OWL) on wikipedia and its positioning against Microsoft Foundation Class (MFC) library.
FastCGI website, and its introduction on wikipedia.
Presentation of the JavaServer Pages (JSP) technology on SUN website.
History of XMLHttpRequest on wikipedia, and its specification on W3C website
Dojo resources: introduction on wikipedia, Dojo Toolkit official website, Dojo Campus for tutorials and live demos, online documentation by Uxebu.
Introduction of the Design Patterns on wikipedia and reference to the “Gang of Four” book (Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides). Specific presentation of the Model View Controller (MVC) pattern on wikipedia.
Django : default template language available in Google App Engine development environment.
Definition of the Representational State Transfer (REST) architecture on wikipedia.
Future post will describe the nature of this project ;)
HTTP status codes on wikipedia, and section 10 of the RFC2616 for the full status code list. Don't forget to look at the illustrations of the HTTP errors codes by the artist Adam "Ape Lad" Koford (license CC-by).
RESTful JSON + Dojo Data by Kris Zyp, with details on the dojox.data.JsonRestStore introduced in Dojo 1.2.

Telcos vs Internet providers

2009-03-12T20:25:00.003-04:00

What a big break, isn't it? I did not give up ;) I was just busy at work to help tuning the performance of a product which is going to be “GA” this month. I mostly focused on the browser-side code, trying to mitigate effects of flawed designs. Anyway, I'll try to use some of the collected materials in a future post.

From jopemoro, with CC-ByND.

Telecommunication operators (telcos) used to have a captive market:

Land lines subscribers do not switch very often from one provider to its competitor, even for long distance call plans. There is some inertia that helps securing their investments in other technologies.
Cellular phone subscribers are more susceptible to change, especially now that they can keep their number when subscribing to another operator. But 3-years plans are efficient tools used by operators to keep their subscribers under control.
Usage of the communication bandwidth is very much predictable. Companies have sized the minutes allocated to each plan so they can get the most from users who do not consume their quotas.

In this industry, the rule of “segment the market to maximize your revenues” is well applied.

Late last year, few operators [1, 2] started offering month-to-month plans, without term contract. Why? Because subscribers want more than just voice communication, they want more flexibility at a reasonable price.

With services like Twitter [3] or Google Calendar [4], any cellphone users can get notifications by SMS. When incoming SMS are billed, the incentive to look for an alternative is big!

Nowadays, a growing number of people have (or will to have) smart phones with multiple communications capabilities: to send pictures, to get live information, to access a map and benefit from their embedded GPS, to use VOIP services, etc. [5]

If local communication plans have usually a fair price (not the long distance plans or the fees when on roaming), prices of the “data” plans are usually crazy. With a smartphone equipped with a 8 MegaPixel digital camera, sending quite a few images over the telco networks is prohibitive...

From mag3737, with CC-BA-NC-SA

So people tend to use more and more direct Internet accesses, in the offices, at home, in cafes, libraries, etc. It so demanded, that non-profit organizations offers networks of Internet hotspots [6].

Thanks to direct Internet accesses which provide better bandwidth than broadband ones, smartphone users can use long distance calls (with Skype [7]), get their voicemail (GrandCentral [8]), stream videos (Qik [9]).

If I can see the month-to-month plan offering as a tentative to keep their customers, telcos should also revisit the data plan offering to avoid more customer base erosion! Because carrying data means carrying almost anything, I think should transform themselves from pure telecommunication operators to Internet providers.

A+, Dom
--
Sources:

No Contract Required — New Month-To-Month Agreement Gives Verizon Wireless Customers Even More Freedom, on Verizon website.
Fido removing system access fees, by on CBCNews website.
Twitter is a free social software where people post updates (limited to 140 characters, same limitation with SMS) and that followers can get automatically.
Google Calendar is time management platform where people can manage their agendas and invite other people. Event reminders can be sent by e-mail and by SMS for free.
For a better description of possible services, see my post: Hand held devices and sensors.
Free Wifi initiatives: Île Sans Fil of Momtréal, Wireless Toronto, etc. More in this directory.
Skype is a service allowing users to make phone calls over Internet. This popular service belongs to eBay (acquired in Sept. 2005).
GrandCentral: see the recent update posted on TechCrunch, about GrandCentral which is going to reborn as Google Voice.

Google App Engine: Free Hosting and Powerful SDK

2009-02-09T22:42:00.028-05:00

(This post is part of the series Web Application on Resources in the Cloud.)

Google App Engine (GAE) [1] is an open platform made available by Google to host Web applications:

It can serve static pages (HTML, CSS, JavaScript, images, etc.).
It can serve dynamic pages. The programming language is Python [2] (with limited features). The default template framework is Django [3].
It can persist data in Google BigTable (with a query language similar to SQL, but with restricted features).
It offers transparent scaling and load-balancing.
Its sources are open and freely available. Google allows to host up to three applications by account, as sub-domains of appspot.com (at least during the preview period).

Some people, like Dare Obasanjo [4], consider GAE as implementing “Platform as a Service” paradigm. I agree and think GAE offers a core element to implement “Software as a Service” (the hype SaaS). In general, I think that SaaS can help IT companies delivering value to their customers at a better quality/price ratio. Understanding GAE strengths should encourage development teams to give a close look at the entire SaaS concept.

Free Service

GAE is offered free of charges during the preview period. In the future, customers will be billed only for what they have consumed (disk space, bandwidth, CPU time, etc.). This practice has been adopted by many providers of services in the cloud, like Amazon [5] and it Amazon Web Services (AWS) offer.

The Software Development Kit (SDK) [1] is open, and anyone can take a look at it, can customize it for his own needs, and can even submit patches. For now, only one programing language is supported: Python [2]. The SDK is delivered with a standalone runtime environment.

Python is also an open system, created by Guido van Rossum who has been working for Google since 1995. In my opinion, this combination is an argument against developers complaining about the need to learn yet another language: Python is a really powerful language and will continue to have a full support by Google as their favorite language.

In association to Python, Django [3] is the template language helping to create applications compliant with the Model-View-Controller (MVC) pattern [6]. Django is also an open source software.

To get the best of the languages and of the standalone GAE runtime, I strongly suggest setting up Eclipse (another open-source software) [7, 8]. Eclipse might not be the ideal candidate for GAE application development, but it provides an extensible platform easy to leverage. For example, egit [9] is a Eclipse plug-in handling transactions with Git repositories (like Github.com [10]).

Servicing static and dynamic pages

GAE can host 1,000 files, each one smaller than 1Mb, for a grand total of 500 Mb per application [11]. Usually, the static files are accessories: images, style sheets, etc. But the offered space allied to Google's scalable infrastructure can be also leverage to host almost any file (HTML, FLV (Flash), CSS, JavaScript, etc.). App Engine Fan describes how to setup GAE for this usage [12], as Matt Riggott [13].

The following handler definition, which should be located into the app.yaml configuration file, indicates that all requests should be served from the corresponding files located in the directory static.

handlers:
- url: /
  static_dir: static/

Dynamically generated content, like developers are used to producing with PHP for example, can be implemented with Django templates [3]. The following template defines the general Web page pattern. And the second template is just extending it by overriding the extension points.

Common.html template with the Web page organization and the extension points.

Producer.html template overriding the extension points with the page specific elements.

Note: because of internationalization concerns, I strongly recommend to NOT code Web pages as the ones above. Refer to my post on Internationalization of GAE Applications [14] for a better implementation.

Quickly, it is possible to use GAE to host static and dynamic pages on the domain appspot.com (pattern is http://[application-name].appspot.com/). Integrating these pages transparently in your own domain allows future updates without having your readers to point to a new Web address. You need to setup Google Apps for your Domain and follow their instructions [15].

App Engine Fan explains how to prevent access to your application from unknown domains [16]. In a private network, you can even open the GAE server to remote access [17].

Access to Google BigTable

In the Web application world, data persist mainly in databases. Databases scale, maintain indexes (providing quickly search results), support transaction (update, then commit or revert). Most databases are relational databases [18]. Among the well know relational databases, there are: Derby, Oracle, DB2, MySQL. SQL (Structured Query Language) is often the query language of relational databases.

GQL (Google Query Language) is very similar to SQL [19]. The discrepancies are due to GAE architecture. For example, to preserve its scalability of the underneath database, GQL does not offer the possibility to JOIN tables. I am not database expert, but I consider all limitations being workable and some of them are very sane.

One important issue with database is related to their central place: if they are corrupted, system can stop working. Being able to backup and restore them is critical. In April 2008, Google communicated about possible export file formats [20]. I have not found if this feature has been published... However, I found Aral Balkan's Gaebar application (GAE Backup And Restore) [21] which covers the basic functionality and even more (like the staging concept).

Update 2009/02/10:
In the SDK release 1.1.9, Google describes ways to upload data from a CSV into BigTable, and to download data into a local development server. Refer to the documentation on GAE Website [1].

Going further

Google has developed a GAE application that is gallery for other GAE applications [22]. Many applications are described there. Interviews of successful implementers are also available on GAE Website [1].

On April 10, 2008, Niall Kennedy posted a detailed article describing GAE architecture [23]. Many others people continue to publish on GAE and on Cloud computing issues in general [24]. It is a really hot topic ;)

Update 2009/02/10:
Dare Obasanjo published another post on Dare Obasanjo: Google App Engine on the road to becoming useful for building real web applications

A+, Dom
--
Sources:

Google App Engine Website, and GAE Service API documentation.
Official Python Website. Python history on Wikipedia. Guido van Rossum's blog (Python inventor).
Django Website, with the section on its template language.
Cloud Computing Conundrum: Platform as a Service vs. Utility Computing by Dare Obasanjo.
Amazon Web Services Website.
MVC Pattern applied to GAE Applications... (another post to be published soon).
Eclipse Website.
Article Configuring Eclipse on Windows to Use With Google App Engine from GAE documentation site.
Eclipse plug-in for Git repositories: egit. Check egit short installation guide.
My post on Git as my New SCM Solution.
Quota description on GAE Website, on GAE blog, and on Wikipedia.
Free Webhosting, Google App Engine style, by App Engine Fan.
Using Google App Engine as your own Content Delivery Network by Matt Riggott.
Internationalization of GAE Applications... (another post to be published soon).
Access to the Standard Edition of Google Apps for Your Domain (GYAD) service and instruction on how to setup a GAE application for your domain.
The darker side of multiplexing, or how to prevent site hijacking by App Engine Fan.
Access Google App Engine Development Server Remotely, by Josh Cronemeyer.
Definition of relational database by Wikipedia.
GQL reference page on GAE Website.
Getting your data on, and off, of Google App Engine on GAE official blog.
Google App Engine Backup and Restore (Gaebar) by Aral Balkan
Google GAE Application Gallery, being itself a GAE application.
Google App Engine for Developers, by Niall Kennedy.
Architectural manifesto: An introduction to the possibilities (and risks) of cloud computing on developerWorks.

Automatic Testing of GAE Applications

2009-01-25T21:35:00.030-05:00

(This post is part of the series Web Application on Resources in the Cloud.)

I am not a purist when it is time to apply a development method, I am a practical guy!

Update 2009/11/20
At one point, I switched to App Engine Java (see my post Google App Engine Meets Java). One of my argument at that time was the lack of tools Python-side to produce code coverage numbers... I've just posted a lengthy article about Unit tests, Mock objects, and App Engine for the Java back-end. The techniques and the code I share in this post allow me to reach the mystical 100% of code coverage, for a project with over 10,000 lines for the business code. See you there ;)

For example, I know by experience how important testing code as soon it is produced. And these tests must run after each code delivery to detect defects and regressions as soon as possible [1]. I allocate as much time to write tests (unit tests most of the time, functional tests occasionally) as I allocate to write code. And no task is considered complete if the corresponding tests do not cover 100%* of its code! I admire adopters of the Test Driven Development (TDD [2]) approach.

When searching information to test Google App Engine (GAE) applications, I was happy to find a post [3] written by Josh Cronemeyer who says:

My favorite way to start any project is by doing TDD.

If you follow GAE tutorial [4], you can create a correctly designed application implementing the Model-View-Controller (MVC) pattern [5]. Testing such an application which organizes its behavior in different logical area means testing each areas independently:

Testing Python models
Testing Python business logic
Testing data injection in Django templates
Testing rendered HTML pages
Verifying the use-cases' implementation

In the following sections, I am going to summarize information I gathered. If I missed any element, please, let me know by posting a comment ;)

Testing Python models

Verifying that your models work as expected is really important. If we can rely on the robustness of Google Big Database implementation (after all, more and more of their applications run on App Engine), you need at least to detect regressions that our future refactoring operations might introduce.

To learn how to setup your environment, read the detailed post of Josh Cronemeyer: Unit Test Your Google App Engine Models.

What follows is some information to get you started writing unit tests against a GAE model. First, a list of the tools you need to install.

Nose is a tool for running your python unit tests.

NoseGAE is a plugin for nose that bootstraps the GAE environment.

An alternative is to use gaeunit [6] but tests might impact the persistence layer.

Josh Cronemeyer suggests to create a new stub data store before running all tests (function called in your test setUp()):

from google.appengine.api import apiproxy_stub_map
from google.appengine.api import datastore_file_stub

def clear_datastore(self):
   # Use a fresh stub datastore.
   apiproxy_stub_map.apiproxy = apiproxy_stub_map.APIProxyStubMap()
   stub = datastore_file_stub.DatastoreFileStub('appid', '/dev/null', '/dev/null')
   apiproxy_stub_map.apiproxy.RegisterStub('datastore_v3', stub)

Testing Python business logic

What do I mean by Python business logic? This is the piece of code that deals with the end-user requests and that does some computations before pinging the persistence layer. For example, a function verifying the format of e-mail addresses (using regular expressions) belongs to that category.

This last 6 years, I have mainly used Java as the programming language to develop the server-side logic. Java is a simple and powerful programming language. Java benefits from big companies' support and has a large set of helper tools. As helpers, I can count: ant and maven, cruisecontrol and hudson, junit and code coverage, tools for static and dynamic code reviews, powerful IDEs, etc.

To developers starting to write unit tests with the target of covering 100% of the code, I have always suggested to read at least the chapter 7 of the book “JUnit in Action” [7], written by Vincent Massol. He introduces the Inversion Of Control (IOC) pattern and the Mock object concept:

Mock objects (or mocks for short) are perfectly suited for testing a portion of code logic in isolation from the rest of the code. Mocks replace the objects with which your methods under test collaborate, thus offering a layer of isolation.

Python has also many frameworks to mock objects. Among them, Python Mocker seems to be very popular: it consists in recording behaviors and setting expectations on mock object, before letting them being used by real functions. See [9] for a detailed “how to use Mocker.”

Reminder: with the tip from Cronemeyer (see above with the stub for the data store, for more details look at [10]), there is no need to mock the data store.

Unit Test Sample in Python

The following piece of code defines utility methods which return 1) a list of supported languages and 2) a dictionary with localized labels (fallback on English one).

# -*- coding: utf-8 -*-

import en
import fr

def getLanguages():
    return {
        "en": en._getDictionary()["_language"],
        "fr": fr._getDictionary()["_language"],
    }

def getDictionary(locale):
    global dict
    if locale == "fr":
        dict = fr._getDictionary()
    else:
        dict = en._getDictionary()
    return dict

The series of tests in the following pieces of code verifies the list of languages contains at least English and French, verifies that the expected dictionaries contains the mandatory key with their name (“English” and “Français”), and that requiring an unexpected dictionary gives the English one.

# -*- coding: utf-8 -*-

import unittest
from prodcons.i18n import accessor

class SuccessFailError(unittest.TestCase):

    def test_getLanguages_I(self):
        """Verify at least 2 languages {en, fr} are in the dictionary"""
        self.assertTrue(len(accessor.getLanguages()) >= 2)
        self.assertEqual("English", accessor.getLanguages()["en"])
        self.assertEqual("Français", accessor.getLanguages()["fr"])

    def test_getDictionary_I(self):
        """Verify we can get a valid {en} dictionary"""
        self.assertTrue(accessor.getDictionary("en"))
        self.assertEqual("English", accessor.getDictionary("en")["_language"]) # Mandatory key

    def test_getDictionary_II(self):
        """Verify we can get a valid {fr} dictionary"""
        self.assertTrue(accessor.getDictionary("fr"))
        self.assertEqual("Français", accessor.getDictionary("fr")["_language"]) # Mandatory key

    def test_getDictionary_III(self):
        """Verify the fallback on the English dictionary"""
        self.assertTrue(accessor.getDictionary("no_NO"))
        self.assertEqual(accessor.getDictionary("en"), accessor.getDictionary("no_NO"))

Testing data injection in Django templates

Django comes with its own test runner! So its templates can be simulated in a stand alone mode.

With the help of Mocker [8], as described in [10], it is relatively easy to verify that your templates extract data as expected. If there is an unexpected data access to the Mock object, or if one attribute has been forgotten, the mock object will report it (a call to verify() ensure that all expectations were met.)

Testing rendered HTML pages

To verify that Django templates display the data as expected. Selenium [11] offers probably the best framework (and it's free):

Selenium IDE which runs as a Firefox extension and that can record, edit, and debug test scripts.
Selenium Remote Control (RC): it is a server that starts/stops browsers and that makes them running test scripts.
Selenium Grid: to control and run tests on remote Selenium RC instances (on WinXP, Win7, RedHat, Suse, etc.)

At that step, Selenium tests are considered being functional tests because they involve many parts of the application (do not work in isolation). It's fine to run them extensively in development environment, but should be carefully used in production (see next section).

The following presentation (3:30) shows how I write a test against my local deployment. The basic test ensure the language switcher works correctly:

Starting from the English homepage, it checks page title (in head>title and in div#title) against the label “Producer-Consumer”.
After having switched to the French page, it verifies that the URL has been updated correctly (no more lang=en, but lang=fr in place). It checks also the page title against “Producteur-Consommateur”.
After having switched back to the English page, it verifies the URL has been updated correctly.

Rapid definition of a test case with Selenium IDE.

Check points defined with Selenium IDE.

Verifying the use-cases' implementation

In Agile development, a sprint is a period of time allocated to complete development tasks. Each task or group of tasks implement a use-case, a story. Use-cases are used to validate the deliveries. Here is a simple story:

End-users fine-tune searches with the prefixes name:, producer:, and unit-price:

At the beginning of a project, use cases are simple and can be covered by automatic tests. After a while, use cases become more complex and running them takes quite a long time and requires a heavy setup. These complex use cases are usually processed manually by human operators (Quality Engineers).

If these qualified workers can focus on complex use cases (all dumb ones are processed automatically), they have more possibility to find real bugs, I mean behaviors that coders forgot to cover, that product owners forgot to specify, etc. Sooner these bugs are discovered lower is their cost in terms of engineer time spent to fix them and in term of lost credibility!

I hope this helps!

Update 2009/01/30:
Josh, alias Shlomo, updated his post with an excellent reference of a discussion thread in Google App Engine group: Testing Recommendations. Look specifically at two messages posted by Andy about using Selenium tools, like this one:

Selenium-RC is not necessary since Selenium core can be added to your App Engine project directory and run from there. Collected steps below to save you time:

Download the core: http://selenium-core.openqa.org/download.jsp

Unzip selenium core zip file

Copy this selenium core directory into your App path somewhere.

Edit your app.yaml file to permit static access (static because it doesn't need a sever interpreter. selenium is written in javascript, so it runs in your browser.) Under "handlers:", add something like this:
- url: /selenium static_dir: ./tests/SeleniumTests/selenium-core-0.8.3

static_dir is where I copied my selenium-core directory. To access selenium, I now open the app engine url, http://localhost:8080/selenium/index.html and run the TestRunner. From there, open your test suite and go. I had trouble loading individual tests for some reason, but suites work.

I could have of course done all of this using the Firefox Selenium-IDE plugin with fewer steps, but using the core approach does make it cross browser supporting.

I'm still trying to figure out how to use Python for this. I'll continue that in a separate message.

A+, Dom
--
Note:

Providing 100% of code coverage by unit or functional tests is sometimes not possible. The test case shutting down remotely the application server to verify that pending transactions are persisted correctly cannot, for example, be scripted for automation. In such a situation, we should rely on manual testing. The goal of test automation is to provide the best coverage possible, so manual testers can focus on edge cases and on trying to discover unexpected issues.

--
Sources

Agile: SCRUM is hype, but XP is as important... (another post to be published soon)
Test Driven Development approach description on Wikipedia, and information on the book Test Driven Development by Kent Beck
Unit Test Your Google App Engine Models by Josh Cronemeyer, and Josh's short bio from a speech he gave at OSCON in 2007.
Google App Engine tutorial on Google Code.
MVC Pattern applied to GAE Applications... (another post to be published soon)
gaeunit is a test environment that runs on development environment (not on Google infrastructure).
JUnit in Action, by Vincent Massol, edited by Manning Publications Co. Chapter 7, freely available, explains the IOC pattern and the benefits of using Mock objects. Vincent Massol is now technical lead of the open source project XWiki.
Python Mocker: Mock object framework which allows to record behaviors and expectations before replaying them with real functions.
Unit tests for Google App Engine apps, by App Engine Fan.
Proper Unit Testing of App Engine/Django, by App Engine Guy
The open-source framework for Web application automatic testing: Selenium IDE, Selenium RC, and Selenium Grid. Check the core features for information on the automating the tests -- See also a Japanese wiki on Selenium: JavaScud / Selenium IDE.

Book "Power of Surprise" by Andy Nulman

2009-01-21T07:20:00.004-05:00

Because I read the announcement of Six Pixels of Separation "Surprise! Andy Nulman Is Giving Away Copies Of His New Book", a blog I have been following for a long time, I decided to give a try ;)

Hey Andy Nulman, here an address for another Montreal tech guy:

7943, Duranceau, LaSalle (Qc) H8P 3R8.

Don't hesitate to post it yourself, Andy gives away 200 copies of his book! Check his blog powrightbetweentheeyes.typepad.com or his post on the offer. Don not forget to link www.andynulman.com for him to track you back ;)

Web Application on Resources in the Cloud

2009-01-13T23:50:00.008-05:00

“What? Why? When? How?” are the four questions I am going to answer to introduce my first series of posts.

What? I am going to build a modern Web application using resources on the Cloud. Specifically, I am going to build an open Web application for consumers to find and review products in a big database, and for producers to offer products and find consumers. The infrastructure will use Google App Engine infrastructure [1].

Why? There are many aspects:

There is the professional benefit: In my post about Gary Reynolds' presentation “Career Advice '08” [2], or as posted by Tim O'Reilly in his post “Work on Stuff that Matters: First Principles” [3], it is mentioned that delivering value is a key differentiator. Building this working application will demonstrate my various expertise.
As an active member of Diku Dilenga [4] which delivers microloans to small enterprises in Democratic Republic of the Congo, I know that such application will help microentrepreneurs finding customers, and vice-versa. This project helps sharing my expertise with people who need help.
This project gives me also a chance to contribute to the open source community which has already given so much to me, my life, and my work.

When? I have already played with the Google App Engine SDK on my machine. I wanted to be sure no blocking issue would prevent me building the application. The application is already available at: http://prod-cons.appspot.com/

How? I am a Agile Methodology Adopter [5] so I do not plan far ahead. I am going to prepare a backlog with the tasks to implement, and I am going to address them according to their priority order. The code is regularly pushed on github [6]: http://github.com/DomDerrien/diku-dilenga/tree.

If you like the idea, if you want to learn new mechanisms, if you are OK with writing lots of unit tests, contact me. Because I am pretty busy at work, and I have to assume responsibilities for Diku Dilenga, the project will move slowly. But it will move and it will be fun!

Hints on the coming post in that series:

A+, Dom
--
Sources:

Google App Engine website.
My post on Career Advice '08
Tim O'Reilly post Work on Stuff that Matters: First Principles.
Diku Dilenga website.
Agile Manifesto, and Agile Methodology description on Wikipedia
Github.com offers free hosting of open sources (charges applied to personal and commercial hosting).

Git as my New SCM Solution

2009-01-09T06:19:00.012-05:00

As a developer in collaborative environments, I am used to relying on source control management systems [1]. In different companies, I worked with CVS [1], ClearCase [1], and subversion [2]. For my side projects (personal ones and ones related to my social involvements), I was a big fan of subversion.

But subversion has limitations, like the importance of the connectivity with the subversion server whenever you want to gain access a file history or to revert an update. Because I have just started a new side project [3], I have decided to go with git and the hosting service github [4].

I am going to describe the straightforward steps to setup, connect, and get working with git/github.

First thing: install git runtime. git has been developed in replacement to a tools used to manage linux kernel code. It has been preliminary built on linux. For Windows users, you can install it over cygwin or use a fork built over MSys. I am setup with msysgit [5].

The authentication mechanism on git repositories relies on SSH [6]. When creating an account on github, a public key is asked. Then anytime, the git runtime on your machine operates with the github server, it signs commands with your private key, and your public key is remotely used to verify the git runtime works on your behalf. To generate a pair of keys, just type the following command in a git bash window:

$ ssh-keygen -C "your@email.com" -t rsa

In order to use git on another machine, you have to copy over the generated SSH file (id_rsa and id_rsa.pub, from %USERDIR%/.ssh). To backup them, I suggest two services I mentioned in my list 2009 Products I Cannot Live Without: KeePass (as a personal encrypted repository) and DropBox (as the replicator from the cloud).

The first time, it is easier to use github website to create an initial repository. You can verify that your SSH key is correctly configure.

$ ssh git@github.com

I have created the project diku-dilenga. If the SSH key is recognized, you can ready to clone (checkout equivalent for subversion) any project you have access to, like mine:

$ git clone git://github.com/DomDerrien/diku-dilenga.git

Adding file in your own project is easy. Note that the add is recursive (so adding . (dot) from the root folder will add all files for the next commit).

$ git add doc/README

Persisting an addition or an update is simply done with the following command. Note that giving a meaningful comment or a project task identifier is highly recommended ;)

$ git commit -a -m "..."

For additional commands (to remove files, to create branches, to merge branches, etc.), you have to refer to reference sites like:

Once all additions, updates, and removal have been locally done, updating the remote server (github in my case), is fairly simple:

$ git push

Getting updates made by others, or from another computer, are also very simple:

$ git pull

Update 2009/01/13:
This post is now part of the series Web Application on Resources in the Cloud.
I recommend you read the introduction of my side project which is going to be visible on github.com/DomDerrien/diku-dilenga/tree.

Update 2009/01/15:
An illustrated guide to git on Windows [7] has been published on github website. I am not a big fan of complex graphical user interfaces (GUIs) where end-users loose the focus of their current task! I think pushing/pulling are relatively rare operations (compared to commit/revert/history) and should have a very light interface... My two cents

Update 2009/03/24:
Thanks to some follower's feedback, it appears that dealing with Git commands is still fuzzy. Here is aSequence diagram initially produced by Olivier Steele and described on git ready blog.

A+, Dom
--
Sources:

Source Control Management description, CVS and ClearCase histories on Wikipedia.
subversion is a popular cross platforms open source replacement to the aging CVS.
Future posts will describe the nature of this project ;)
Official git website, git history on Wikipedia, and github hosting service.
Package of git for Windows: msysgit
SSH network protocol and public-key cryptography principles on Wikipedia
An Illustrated Guide to Git on Windows on github website.

2009 Products I Can't Leave Without

2009-01-05T12:36:00.019-05:00

Today, I saw Michael Arrington blog post about his annual list of products he can't live without. As a TechCrunch reader, I read Michael list few times in the past and have been maintaining my own list of favorite applications. Because of the success of Michael's post and because I use different tools, I have decided to publish my list too!

Michael and I have different backgrounds: he is the founder and the editor of a popular technical site and I am just a developer ;) So do not compare lists.

Here is my list of applications, in alphabetical order, I used often. And to make it consistent with Michael's post, the list is followed by a comment for each tool.

2009

AllChars
Blogger
Dabbleboard
Dropbox
Eclipse
Firefox
FriendFeed (private room)
Gimp
Git
GMail
Google Calendar
Google Reader
Google Search
KeePass
LinkedIn
VirtualBox
YouTube and Google Video

AllChars
For programmers or English writers, keyboards with the US layout are just fine. If you use another layout, then it is a compromise between being able to generate specific characters and staying efficient. AllChars is a Windows tool that can generate any Unicode character with few key sequences (like Ctrl+e+e for €). It is fully customizable and support even macros.

Company:	FLOSS
Website:	allchars.zwolnet.com

Interesting post from Jeff Atwood: We Are Typist First, Programmers Second.

Blogger
This is the tool used to publish this blog. I have not invest enough time in customizing the page layout, but I know I can easily do it later and updates will be propagated to all posts. A good and efficient tool, IMHO.

Company:	Google
Website:	blogger.com
Launch Date:	August 18, 2007

Blogger is a blog publishing platform formerly known as Pyra Labs before Google acquired it in February 2003. Blogger blogs are mostly hosted internally with the “dot Blogspot” domain but they can also be hosted externally on a user’s own server.

Blogger provides bloggers with a WYSIWYG editor that allows them to drag-and-drop widgets, change fonts and edit page elements. Also, Feedburner’s feed management tools are tightly integrated with Blogger blogs due to Google’s recent acquisition.

Credits: CrunchBase

Dabbleboard
DabbleBoard is a tool I blogged about in a post about Effective Communication principles.

Company:	Dabbleboard Inc
Website:	dabbleboard.com

Dropbox
Dropbox is a composed of plugins to install on your machines (Windows and MacOS) and a website which host your virtual folders. It is a silent tool, working smoothly, and synchronizing folders in background. I use it in conjunction with KeePass to backup my sensitive information.

Company:	Plum
Website:	getdropbox.com

Eclipse
For an early adopter of IntelliJ IDEA by JetBrains, I had to use Eclipse (company's tool when working with IBM Rational, cheap when working with Compuware). I should recognize that it is going better and better (especially with the refactoring features and the JavaScript support) and it has more plugins than IntelliJ. It is also a platform for OSGi and for Rich Applications.

Eclipse Website
IntelliJ IDEA Website
OSGi Alliance Website and OSGi introduction on Wikipedia.

Company:	FLOSS
Website:	eclipse.org

Firefox
Having been a Web application developer for a long time, I adopted Firefox (then know as Firebird) in 2003. With the introduction of the Firebug extension (in 2005), it became with primary browser and it had never lost this status. Its early integration of Google search was also a serious advantage. These days, with the faviconize extension and Firefox ability to start with the previous configuration, my browser always starts with: iGoogle, GMail, Google Calendar.

Company:	Mozilla
Website:	getfirefox.com
Launch Date:	November 9, 2004

In February 2008 Mozilla announced that they had reached 500 million downloads of Firefox, and 150 million active users.

Credits: CrunchBase

FriendFeed
I started to use FriendFeed at its early stages, mainly because it offered the possibility to share information from Web page I can visit randomly. Because few months after its introduction, Google Reader added the Share with note function, I stopped using FriendFeed... until recently when I have invited to a private room: this is a pretty nice service for offline chats. It is not as heavy as e-mails and not as disruptive as instant messages.

Company:	FriendFeed
Website:	friendfeed.com
Launch Date:	October 1, 2007

In February 2008 Mozilla announced that they had reached 500 million downloads of Firefox, and 150 million active users.

Credits: CrunchBase

Gimp
I never the budget and training for Adobe Photoshop. So I started using Gimp. If you can pass over its weird interface (too many windows, IMO), Gimp offer tons of features for Web application developers: to adjust pictures, to generate textures, to resize images, etc. And there are plenty of free tutorials on the Web.

Company:	FLOSS
Website:	gimp.org

My favorite video series on Photoshop, starting with the first episode: You Suck at Photoshop #1: Distort, Warp, & Layer Effects.

Git As a developer, I always want to put my code into a source control system. It is not just because I am afraid that my laptop crashes, then wasting hours of work. It is mainly because I want to keep track of the update history. At work, over the years, I used ClearCase, CVS, and Subversion. For my personal development, I used Subversion a lot and now I use Git.

Company:	FLOSS
Website:	git-scm.com

Free hosting service of open-sources Github - Charges applied for private hostings.

GMail When I started working, I dealt with many machines and I hated having to start one just to look at a specific inbox. With GMail, my account is available anywhere. When I read Turn Gmail Into Your Personal Nerve Center, I started to use GMail as my knowledge database.

Company:	Google
Website:	gmail.com
Launch Date:	April 1, 2004

Gmail, also known as Google Mail, is a free email service with innovative features like “conversation view” email threads, search-oriented interface and plenty of free storage (almost 7GB). Gmail opened in private beta mode in April 2004 by invite only. At first, invites were hard to come by and were spotted up for sell on auction sites like eBay. The email service is now open to everyone and is part of Google Apps. Paul Buchheit, an early employee at Google, is given credit for building the product.

Another Gmail feature is the organization, tracking and recording users’ contact lists. For instance, if you start typing the letter C into the “To” field Gmail will bring up a list of every email address and contact name starting with the letter. This feature helps when you can’t quite remember a name. Plus, Gmail automatically adds and updates email addresses and names to your contact list when you write emails.

Credits: CrunchBase

Google Calendar My first job in Montréal, Canada, was with a small company named Steltor (bought few years later by Oracle). The core business was the development of a distributed calendar system (servers in cluster, native clients, web client, mobile client, etc.). Since then, I am used to tracking my work with an electronic calendar. Google Calendar and its ability to mix many agendas is excellent.

Company:	Google
Website:	google.com/calendar
Launch Date:	August 21, 2007

Google Calendar lets users create events, manage multiple calendars and share calendars with teams and groups. Users can view their calendar by day, week or month. Calendar has a “Quick Add”� feature that lets you input natural language entries fast. For instance, you can type “Dinner with Michael 7pm tomorrow” into the entry box and Calendar will add “Dinner with Michael” into tomorrow’s agenda at 7pm without needing a specific date. Calendar can also be set-up to send you SMS or email alerts for upcoming calendar entries.

Credits: CrunchBase

Google Reader In the old day, reading blogs was time consuming and annoying because of the ad banners. With the introduction of Google Reader, reading them from one central place while just using keystrokes was a pleasure. With the introduction of the Share this feature, I became to be a media myself. The Share with note function is an quite recent addition that allows me to point posts that are more important to me.

Company:	Google
Website:	google.com/reader
Launch Date:	October 1, 2005

Google Reader is an online RSS feed reader.

Credits: CrunchBase

Google Search Google Search is an amazing tool: recently, I was trying to find a solution to a tough technical problem and I found it thanks to Google Search which pointed toward a blog post written the same day, just few hours before, in Europe! Incredible... When I give a conference into universities, I often say: “If I asked a question today and you have no clue about the response, that's fine. If you still have no clue tomorrow, you're in trouble...”

Company:	Google
Website:	google.com
Launch Date:	September 4, 1998

Search is Google’s core product and is what got them an official transitive verb addition to the Merriam Webster for “google”. The product is known for its Internet-crawling Googlebots and its PageRank algorithm influenced heavily by linking.

When users type keywords into the home page search box they are returned with relevant results that they can further refine. Google also has more specific search for blogs, images, news and video. Google will also return search results from your own computer files and emails via Google Desktop.

Credits: CrunchBase

KeePass KeePass is an open source Password Safe. I use it in conjunction with Dropbox. Much usable that PGP/GPG in the sense I don't have to save the decrypted file and then being at risk if I forget to wipe this copy from disk.

Company:	FLOSS
Website:	keepass.info

LinkedIn I started using LinkedIn when a wonderful team I worked with at Oracle exploded because of stupid political decision. Then some friends left to go with Google, one colleague went to Adobe, etc., and LinkedIn was the tool to stay connected. Since then, LinkedIn allowed to get many job propositions, like Compuware's one I accepted. I have been also able to retrieve friends I had in France, even one I may engage a partnership with.

Company:	LinkedIn
Website:	linkedin.com
Launch Date:	May, 2003

LinkedIn is a free business social networking site that allows users who register to create a professional profile visible to others. Through the site, individuals can then maintain a list of known business contacts, known as Connections. LinkedIn users can also invite anyone to join their list of connections. LinkedIn offers an effective way by which people can develop an extensive list of contacts, as your network consists of your own connections, your connections’ connections (2nd degree connections), as well as your 2nd degree’s connections (called your 3rd degree connections). From this network, individuals can learn of and search for jobs, business opportunities, and people. LinkedIn also serves as an effective medium by which both employers and job seekers can review listed professional information about one another. LinkedIn follows strict privacy guidelines wherein all connections made are mutually confirmed and individuals only appear in the LinkedIn network with their explicit consent. Other LinkedIn features include paid accounts that offer more tools to find people, and “LinkedIn Answers” developed in January 2007. A free feature, “LinkedIn Answers” allows registered users to post business-related questions that anyone else can answer.

Credits: CrunchBase

VirtualBox Developing software requires sometimes specific configurations. Testing them requires always specific configurations (at least to replay always the same test cases every time the source control system, like Git, is updated). There are the famous VMWare products (Workstation, Player, ESX) and Microsoft VirtualPC. VirtualBox is an open source product provided by SUN Microsystems, and it has nice features while being powerful.

Company:	Sun Microsystems
Website:	virtualbox.org

VirtualBox is a family of powerful x86 virtualization products for enterprise as well as home use. Not only is VirtualBox an extremely feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source Software under the terms of the GNU General Public License (GPL).

Presently, VirtualBox runs on Windows, Linux, Macintosh and OpenSolaris hosts and supports a large number of guest operating systems including but not limited to Windows (NT 4.0, 2000, XP, Server 2003, Vista), DOS/Windows 3.x, Linux (2.4 and 2.6), Solaris and OpenSolaris, and OpenBSD.

YouTube and Google Video YouTube is famous because of fun videos. But it also hosts technical videos.

Company:	Youtube
Website:	youtube.com
Launch Date:	December 11, 2005

YouTube was founded in 2005 by Chad Hurley, Steve Chen and Jawed Karim, who were all early employees of PayPal. YouTube is the leader in online video, sharing original videos worldwide through a Web experience. YouTube allows people to easily upload and share video clips across the Internet through websites, mobile devices, blogs, and email.

Everyone can watch videos on YouTube. People can see first-hand accounts of current events, find videos about their hobbies and interests, and discover the quirky and unusual. As more people capture special moments on video, YouTube is empowering them to become the broadcasters of tomorrow.

In November 2006, within a year of its launch, YouTube was purchased by Google Inc. in one of the most talked-about acquisitions to date.

Credits: CrunchBase

I hope it helps,
A+, Dom

Career Advice '08

2008-12-28T21:10:00.009-05:00

In my previous post, I wrote mostly about how managers should deal with their subordinates. I think most of the time the real work is achieved by regular employees and the delivered quality depends on how the management support the do-ers.

At the same time, having subordinates just doing their job and waiting for their management to come with all solutions is not realistic. I was trying to organize a post on how subordinates should differentiate themselves from the mass when I came across this presentation (thank you Steven[1]): it's a presentation by Gary Reynolds about Dave Pink's bool “Six Career Lessons”.

Career Advice '08

View SlideShare presentation or Upload your own. (tags: presentation pink)

The slide 174 summarizes the six lessons:

Slide 174 from Gary Reynolds' presentation.

Being a technician in a domain with a fantastic innovation trend, I have to emphasize on lessons 2 to 4:

Go beyond being just good at something: become an expert! For example, you can be a proficient developer, push to cover everything you produce with unit tests: you will be able to prove that your production is rock solid. If you know many natural languages, develop easily localizable applications (we produce for a global market).

Work to maintain the excellency of your expertise! Most companies do not let employees thinking freely during the regular hour - to my knowledge, only Google clearly allows it [2]. So you will have to invest your personal time to stay up-to-date.

Share your expertise! Your expertise is useless without you sharing it. Sharing it does not expose you dangerously because potential thieves will not have your experience and because they will not stay up-to-date. In reverse, sharing it will expose you to valuable experiences and feedback from your audience, adding up to your existing sources of information.

These days, there are so many ways to build and share an expertise:

Write an insightful blog;
Participate to an open-source project;
Publish technical articles (developerWorks [3], MSDN [4], OTN [5], etc.);
Speak to conferences (require a lot of energy and often a lot of money);
Build a successful project which delivers values to your customers!

Everyone should make his own path, whatever the environment is. If conditions are too tough (because the management is not supportive, for example), look for another place. If they are not so bad, find any occasion to build your expertise and grow yourself ;)

Happy new year 2009!

Update:
Tim O'Reilly, on O'Reilly Radar blog, has published a post aligned with my position. Here is principles:

Work on something that matters to you more than money.
Create more value than you capture.
Take the long view.

What? You haven't started demonstrating your value? Just kidding...

Second update:
Tim O'Reilly has been interviewed on the top Work On Stuff That Matters.

The video is on mutliple parts. Check O'Reilly radar webiste for additional parts.

A+, Dom
--
Sources:

Steven Milstein's blog
Google Jobs: Work Environment: 20% project concept
IBM developerWorks
Microsoft Developer Network
Oracle Technology Network
Tim O'Reilly post on Work on Stuff that Matters: First Principles

Manager Attitude

2008-12-10T09:15:00.001-05:00

A long time ago, I read the famous Joel Spolsky [1] entries on management:

As an experienced worker, I know my limits: for example, I do not think I will be a good typical manager... With my expertise based on emerging technologies (read “Web 2.0+” and “mobile”), I lost sometimes my patience while trying to explain new concepts or convince conservative or ignorant people. When it is time to deal with personal problems of your subordinates, I really think that such attitudes are absolutely unacceptable.

Despite this fear, I really like mentoring people. I can focus on transferring knowledge (what is the point of being an expert if you do not share your knowledge?) without having to deal with risky personal situations! I act as a mentor in different situations:

With new employees and interns: to teach them the best practices (efficiency is not that important at the beginning), to have them fully understanding the requirements before thinking about implementation details (to avoid the risk of delivering something that does not match the requirements), etc.
With experienced developers having a different technical background: to identify their strengths and to make them more productive (better designer, delivering high quality code, documenting their production, more agile).
And with managers for reasons I am going to developed below.

I have been introduced to the formal process of pairing mentors and mentees while working for IBM Rational. I would say that IBM has the best personal development program I have ever seen. And the mentorship is a corner stone of that program. This was the first time I had a formal mentor (Arthur Ryman, IBM Distinguished Engineer [2, 3]). Because it was a successful experience, I started then to become a mentor myself.

Usually, implementers (or do-ers, or pigs [4, 5]) are receptive to suggestions. Managers (or chickens [4,5]) are much less willing to listen. In my experience, I met many people that followed the path governed by the Peter Principle [6]. Usually, the only way to get promoted is to become a manager; very few people start their career as managers. Promoted managers often lack of training, and they mostly following the pattern managers have serve to them.

What is the big difference among managers and regular employees? The first ones have been empowered to take decisions and the second ones usually follow orders. Many managers I worked and I work with have a technical expertise reversely proportional to their management experience. And they have most of the time a better salary than their reports.

Managers have many responsibilities, and there is one that is often incompletely assumed: ensure their reports can deliver their best.

I find dangerous when decisions are taken without having the big picture. Some managers follow Joel Spolsky's advice and delegate the decision instrumentation.
At the same way, managers should delegate, I think reports should be able to do the same. For example, a developer needing a second hard drive to host big virtual machines should be able to go to his manager, to expose his requirement, and then go back working while the manager fills up the purchase request. How many times have you seen such a situation? Personally too rarely, and just from junior managers who had still fresh in mind some past frustrations. Most of the time, managers think their time is too expensive for such basic tasks and they prefer wasting the time of their smart reports.
A good manager should delegate to their reports and accept delegation from their reports. Each aspect of the delegation makes the reports more motivated and then producing a better job. I think benefits gained by applying only one delegation aspect are easily compensated by the frustration generated the non application of the other aspect.

For the fun of it, and because I endorsed most of lessons described in the article 14 Things Corporations Can Learn from Seasoned Web Workers, I sent it to many managers (development managers, human resources manager, product managers, etc.). Was it because they are “control freaks”, because they are going to lost their power, because they cannot do basic things anymore for others (especially for reports), but none came back saying: “Good article. Let's see what we can make happening.”

This conclusion is another argument why I do not want to be another manager. Hopefully, my current expertise allows me to focus on technical works, to be just a mentor, to give advice and to accept suggestions in return, and to stay on the implementers' side where true collaboration is common.

A+, Dom
--
Sources:

Joel on Software blog by Joel Spolsky
Arthur Ryman's profile on eclipse.org
Eclipse Web Tool Plateform: book co-authored by Arthur Ryman.
Chickens and pigs in situation by Nick Malik
Chicken and pig roles in SCRUM context on wikipedia
Description of the Peter Principle on wikipedia -- read the part on Dilbert Principle ;)

Mobile devices and mobile Web applications

2008-11-25T23:24:00.000-05:00

Beginning of 2000, I had colleagues developing applications for mobile phones. The calendar service offered by the company then named CS&T (renamed in Steltor later, and then acquired by Oracle in 2003) was using the Wireless Access Protocol (WAP [1, 2]) and SyncML [3] to convey scheduling information up to mobile devices and to return end-users' updates. Event notifications were sent by SMS [4]. Because WAP and SMS are standards implemented by most of the wireless networks (GSM, CDMA, PDC, DECT, etc. [5]), and because they can be processed by many devices (mobile phones, pagers, smartphones, etc.), it was a successful feature.

Today, mobile devices are not just connected to one Telecommunication Operator (telco) network: they can be connected to computers via Bluetooth [6, 7] and directly to Ethernet networks by Wi-Fi [8, 9] and soon by WiMAX [10].

And these additional networks offer more bandwidth to mobile applications.

At the same time networks offer a better service, the power of mobile devices increases [11]. With their relatively wide screens, there is less and less reason to define specific User Interfaces (UI) for the mobile applications. What we did in the past with SynchML can be easily achieved with a modern browser (Webkit based for the iPhone and Android smartphones) running a JavaScript engine. In that sense, I would say like others [12, 13] that the “Mobile Web is dead”.

Web applications are good because they provide a sandbox (no general file system access, limited connection capabilities) and because they can be rendered seamlessly almost everywhere thanks to the browser Firefox [14]. A key element of Web applications is the URL addressing: this piece of information can be embedded everywhere (in an e-mail, in a document, in a Web page, etc.). URLs can be bookmarked by the browser, by a remote service, or as a desktop shortcut. End-users can check the navigation history, can go back and forth, can reload, can open in a new tab or a new window. Thanks to their ubiquity, their safety, and their usability, Web applications deliver really good user experiences.

Because rich feature set of the new handheld devices [11], there are new integrations that Web applications cannot deliver! Consider Google Maps on the iPhone [15] or its equivalent Map View on Android [16]: there are both native applications that use the GPS, the Compass, the multi-touch screen (to zoom in and out, for example). A Web applications can hardly do it today without requiring the end-user to download a browser plugin.

It seems that Web applications will have a hard time competing with native applications on top handheld platforms. If browsers on such platforms do not evolve, all the benefits of Web applications (as mentioned above for end-users, but also for service providers and development teams) will be lost because end-users will mostly look for fully integrated applications!

Does it means that “Mobile Web is dead” can be transformed in “Web is dead on Mobile”? I hope not. What is your opinion?

A+, Dom
--
Sources:

Wireless Access Protocol (WAP) by Wikipedia
WAP by the International Engineering Consortium
SyncML by Wikipedia
Short Messaging Service (SMS) by Wikipedia
Global Service for Mobile communications (GSM), Code Division Multiple Access (CDMA), Personal Digital Cellular (PDC), Digital Enhanced Cordless Telecommunication (DECT): list of mobile phone standards
Bluetooth by Wikipedia
How Bluetooth works by the Bluetooth Special Interest Group (SIG)
Wi-Fi by Wikipedia
Wi-Fi Alliance
WiMAX by Wikipedia
Handheld devices and sensors
End of the mowser by Russel Beattie
Russell Beattie was Right, the Mobile Web is Dead by Dare Obasandjo
Get Firefox ;)
iPhone3G: Maps with GPS
Android Map View

OLPC XO available on Amazon

2008-11-17T12:17:00.001-05:00

I blogged before about the One Laptop per Child [1] initiative and I am pleased to remind you that the computer is available on Amazon starting today [2]!

Image from Amazon.com website [2]

A+, Dom
--
Sources: