Stephen on Software

Hadoop World videos

2012-01-05T23:53:00.002+08:00

Hadoop World videos can be found here http://www.cloudera.com/resources/Hadoop+World/. Seems like I will be spending some time watching these for the next few weeks.

Links:
Hadoop project

Bing 2.0 API is useless

2011-12-29T22:03:00.000+08:00

Bing's 2.0 Search API is basically useless. The results dosen't match up with the Bing.com website and so testing is almost impossible.

And worse the results returned by the API are mostly irrelevant. I think Microsoft needs to seriously look at what are they targeting the Bing API for. Because as a developer i won't want to use it.

But there is a lack of freely available search apis out there.
Google Custom search requires you to search only specific websites that you specify and not google itself.
Yahoo BOSS requires payment so its out for me.

Anyone knows of what other options are out there?

Links to the issue:
Stack Overflow
Bing Community

Sheevaplug Debian Install

2011-12-14T12:02:00.000+08:00

Upgraded from the default Ubuntu install to Debian today, mainly followed the instruction by Martin Michlmayr. Mostly worked but the instructions for after installation of Debian to the MMC card was slightly different for mine.

These are the ones that worked for me.

setenv bootargs_console console=ttyS0,115200
setenv bootcmd_mmc 'mmcinit; ext2load mmc 0:1 0x00800000 /uImage; ext2load mmc 0:1 0x01100000 /uInitrd'
setenv bootcmd 'setenv bootargs $(bootargs_console); run bootcmd_mmc; bootm 0x00800000 0x01100000'
saveenv

The main difference is the mmcinit command, on my uboot it was "mmcinit" together, for the instructions that Martin gave was "mmc init" with a space.

Product Owner's Guide - Breaking down estimates

2011-07-25T16:58:00.004+08:00

Image: Ambro / FreeDigitalPhotos.net

Overhead this while at work.

"The Product Owner wants us to break down our inital estimate on the development effort into parts like analysis, design and coding ...".

That sounds to me like the Product Owner(PO) thinks that whatever estimate that the team came up with is too long and wants to cut the estimate. If the team gives such such a breakdown, then be prepared for the negotiation about the time needed for the project to be discussed.

But how is the team going to come up with such a figure anyway? I always found the statement like "Most of the time is spent in design but little in coding", or using the 80-20 rule that 80% is spent in design and 20% in coding. Is that statement still relavant today? In the past (I mean way way past), people do spent a lot of time in design because computers process ur programs in batches and you better get it right the first time if not you wait a day for the next run.

Fast forward to today, where we have powerful ides, programs run almost instantly, where you can code a little, test a little, you are more likely to try a program a vertical slice and then build up your system from there, with refactoring tools so powerful, changing methods, implementing interfaces no longer is such a chore.

Now back to the story, not matter what the project team tries to justify their number the PO will just try to cut it. The problem with most people is that they have some prenotion of when the project will end, but they do not realised that an estimate is just that, an estimate, it's a educated guess base on whatever information you have. An estimate is not an accurate prediction of the future and couple with the fact that software estimates are usually wrong. Steve McConnell says it well in his book Software Estimatation.

"Estimation should be treated as an unbiased, analytical process;
planning should be treated as a biased, goal-seeking process"

So if you ask someone for an estimate, you are asking for his opinion on how long it will take, and you shouldn't try to cut his estimate down.

More likely is the product owner is trying to get a plan from the team not an estimate, and the team's estimate isn't matching his plan. His plan can be anything, could be trying to meet some timeline for marketing, or some trade show demo.

Product Owners please just tell the team what is your plan, it may be because of marketing, budget and other reasons that you need to slim it down. But please tell the real reason and not by trying to get the estimate within your plan.

For the development team, try to find out the underlying reason for doing such an exercise, buy a copy of Steve McConnell Estimation book and give it to him. Negatiate on the scope, not all features need to be a mercedes, sometimes a vespa is good enough.

The only thing that can be reliably cut down to meet the target is actually the scope of the project. Does the Product Owner need 100% of all the features of the software to be done to be really useful? More likely than not something like 50-70% will probably be good enough.

Main takeaway - Estimation and Planning are 2 different things, try to make sure what is it you are talking about.

Expanding your toolbox - Picking a new language to learn.

2011-04-28T00:08:00.004+08:00

Image: graur razvan ionut / FreeDigitalPhotos.net

One day a developer on my team came up to me and asked, what should he programming language should he learn next to be marketable (and by the way he also thinks that Java is slowly dying).

Well as to whether Java is dying a not, I don't think so, a quick check with TIOBE shows that Java is still ranked among the top few languages and lots of places still runs Java. So i do have a job for the next few years.

But the thought came to me, it is worth learning things that are marketable or should you learn things that broaden your knowledge? What's your take on it? Should you expand your toolbox or have more of the same tools?

Steamhead takes the languages from Seven Languages in Seven weeks and puts it into a nice flow chart. I think all should go check that out and pick something out from that list. After doing quite a lot in Java I begin to appreciate the simplicity of using interpreted languages like python and php, and the fact that their frameworks are usually simple as compared to Java ones where sometimes we descend into XML configuration hell.

Steve Yegge has a interesting way of categorizing the various languages, he breaks them down into Nouns (Java, C# or OO languages), Verbs (LISP, Clojure or Functional Programming) and Verbs and Nouns (Scala, Ruby, Python, C++ or a Mixture of both).

Well for me I am more for the expand your toolbox model, go learn something that makes you have a new way of looking at things. A lot of people learn languages but they don't really learn design, think stuff like design patterns (Gang of Four, Java EE Design ...), SOLID principles, OO Metrics. Even if you know the syntax, you simply can't write maintainable programs and all you write lacks a certain craftsmanship to it.

By learning a different language and looking at the api, you can have a different idea about how to design your programs. An example will be like in Java. You have Math.abs a static function to get the absolute value of a number, have you ever wonder why its so "un OO" like? Compare with Ruby its just number.abs(), simple and elegant and fits into the OO paradigm.

In short

If you know Java, C# go learn Lisp Dialects (Common Lisp, Clojure), Haskell, Ruby, Python...

If you know Python, Ruby, you are pretty much there, since you straddle the limits between the 2 kingdoms.

If you know Common Lisp, Clojure, dont bother learning any other thing since you will think why in the world Java, C# programmers are typing so much code and configuring so much XML.

If in doubt just learn PHP to be marketable :)

Amazon.com Widgets

One hour early a day...

2011-02-02T11:14:00.001+08:00

My latest read is Robin Sharma's book (The Leader who had no title), interesting read. But one statement struck me (I more or less paraphrase it)

"If you wake up one hour early each day for a month, in 30 days you will have almost one weeks worth of working time, think of the things you could do"

When i told this to some of my colleagues/friends they say wha sleep less one hour how ? Not enough time to sleep already.

"You have enough time to sleep when you are dead"

I guess this is the difference between people who do impressive/great things (physically fit, be it in doing voluntary work, open source projects, making lots of money, hobbies, making a difference, side business, execise often) and the rest of us. They spent the time not just dreaming and talking about it but actually doing it. I am not actually saying that you forgo sleep altogether but think of the possibilities that could happen if you think about how to spend your time wisely.

At the end of our life, lets not spend time regretting what it might have been, or what we might have done better.

Remember the saying that you need 10,000 hours to become good at what u do, maybe i should start doing things instead of writing things on my blog :)

Ebooks from top StackOverflow answers

2011-01-26T23:37:00.002+08:00

Have you tried StackOverflow, if you are a developer its probably one of the best resource for asking questions.

Its a great site, what is even greater is that someone converted the best answers into an Ebook format.

Great for bring around, the link is here http://hewgill.com/~greg/stackoverflow/ebooks/. Whats nice is that it is broken up nicely into the different subject like ajax, python, c#.

Posting data with Node.js

2011-01-25T17:53:00.001+08:00

Recently i needed to post data using node.js, I was not trying to post a long string of data but rather parameters from a form. But the Node.js help page isn't too helpful, it just say use request.write() to write data to the stream

This is the first interation i got.

var http=require('http'); var post=http.createClient(80,'localhost'); var request=post.request('POST',/test.php'.{'host':'localhost'}); request.write('test=10'); request.end(); request.on('response',function(response){ response.setEncoding('utf8'); reponse.on('data',function(chunk){ console.log(chunk); } });

test.php is a simple php script that prints out the parameter value.

I got nothing... nothing was printed out. What was wrong?

Using curl and Fiddler, i did a little investigating about what went wrong, what was missing was the Content-Length and the Content-Type which needs to be application/x-www-form-urlencoded.

Adding these 2 parameters to the script

var http=require('http'); var post=http.createClient(80,'localhost'); var request=post.request('POST',/test.php'.{'host':'localhost','Content-Length':'7','Content-Type':'application/x-www-form-urlencoded'}); request.write('test=10'); request.end(); request.on('response',function(response){ response.setEncoding('utf8'); reponse.on('data',function(chunk){ console.log(chunk); } });

And yeah ! It works !

JRex, Embedding Gecko (Firefox) in eclipse

2011-01-11T17:29:00.001+08:00

Jrex is a Java component that embeds Mozila Gecko in your application. The project has been dead for some time since 2008 from what the mozdev.org page. So the version of Gecko may not be the latest but should be good enough for whatever you would like to try out.

However looking at the code from Vijay (http://www.vijaykiran.com/2006/09/05/using-jrex-in-eclipse-rcp/
), i got it working and it seems to work fine, but Vijay dosen't really go into how to setup the project like where to download the jars and where to put the dll and so on, and I did have some hard time trying to find out where to find the jars.

Downloads
1. At first i took apart the jnlp file on jrex.mozdev.org to find out what are the dependencies, but after looking around the mozdev site, i finally found the downloads for the jars, source and docs in here (http://www.mozdev.org/source/browse/jrex/downloads/#dirlist).

2. But it dosen't include the jrex_gre files that are in the jnlp, those you can get them here (http://www.mozdev.org/source/browse/jrex/www/releases/jrex-1.0b1_dom3/). Look for the jrex_gre.jar

Setting up your eclipse.
Put the jrex.dll in your eclipse directory, if not you will get a UnsatisfiedLinkError when you try to run the view. Put the jrex.jar in your lib folder of your project.

Unzip the jrex_gre.jar contents to a nice location you can remember. You will see in Vijay's code a place that you will need to supply the path to the file in org/mozilla/jrex/jrex_gre files.

That should fill up the missing pieces in Vijay's writeup.

I will try to package the source code into github at some later time.
&n bsp;

Fedora 14 - Using the DVD as a software repository

2011-01-06T09:32:00.001+08:00

Installed Fedora today at office in order to try out some web acceleration stuff lile squid and varnish. I used the DVD iso version, so that i can have most of the essential software already.

When i rebooted i found that the dvd is not recongnised as a software repository, because there is no entry in /etc/yum.repos.d/.

No biggie, just mount the dvd, in the root of the dvd there is a file media.repo, jus copy the file to /etc/yum.repos.d/ as packagekit-media.repo and you are all set, your dvd becomes a software repository.

Resolutions 2011

2011-01-05T09:53:00.001+08:00

My resolutions for 2011.

For the personal geek in me (Probably have little to do with work)

I declare this to be the year of functional programming for me!

Common Lisp

Begin by first completing Practical Common Lisp and Land of Lisp, and the little pet project that i do when i have free time, the idea of closures and high order functions really appeal to me.

Erlang

Another Functional Language that I hopefully have time for this year. Invented by Ericsson for using in telephony applications, it has some interesting properties like fault tolerant and hot swapping. Since it's designed for telephony (which has something like 9 9's of availability) you can actually swap in code while the system is running which i think its pretty cool.

Hadoop/Map Reduce

Always heard about MapReduce and Hadoop, I shall attempt to read google's paper on it and hopefully setup a working instance.

NoSQL

NoSQL is getting big nowadays with Twitter and Facebook all moving/developing part of their infrastructure on top of NoSQL type of storage.

Learn at least one implementation and see how it can be worked into a project.

For the bookworm in me

Read The Elements of Style, one of the great books on the writing for the English language. Time to get better at my writing, better written communication is essential as you go along ur working life.

Read more "management" style books especially in the realm of making people happy at work, I am quite interested in how great organizations retain their people and keep them happy.
So far i have FISH! tales, Why work sucks from the library here with me.

For the not so fit me

Finish taking my IPPT before June and pass it.

Run at least once a week at least 2.4km.

Attempt to run a half marathon by then end of the year. (Haha this will the hardest one in my list)

For the bad habits in me

Second guessing or interrupting people midway

I don't know if people noticed but i do have a bad habit of either completing other people's sentence or second guessing what they will say. I will let others finish what they want to say before I start, I will try to follow the rule of One Conversation at a time.

Procrastination

I have a habit of procrastinating, enuff said.

For the father and husband in me
Vacation

Hopefully I will have time to go for a vacation, hopefully it will be the first time that I am bringing Chloe overseas.

Better Father/husband

Its quite hard trying to be specific as to what makes a better father, could be a variety of factors but i sorta like this one that I got from FISH! Tales book, "Love for child. is simply being there".

Lets look back again in 2012.

Reflections on 2010

2010-12-28T21:07:00.001+08:00

Well 2010 has come and soon will be gone, so time to look back.

I have been married for 2 years now although there are ups and downs but I am happy that I have a nice wife and a cute (and she continues to amaze me everyday) daughter.

Being a father does entail some sacrifices, no longer will you have "your" time, but rather it will be the "family's" time or more accurately the "baby's" time. But the smile of Chloe when she see you wake up and when she is scared and runs to hugs you ( only when the mum in law not around) makes it all worth while. But as for having another one .... lets wait and see :)

Hdb finally decides to give me my house, and it is in the process of renovating, left the painting and the carpentry to be done and it should be ready to move in.

Work wise has been good, finally saw a project from the start to the OSAT phase, couldn't see it all the way to the end though, got rope into another project, as a Software Develop ment Manager, its sort of like a Project Lead kinda thing. Challenging though because the project had already been running for sometime before I came in and there are lots of stuff that I need to pick up and be very familiar with. I am aware of the Peter Principle and the Dilbert Principle, hope that this is not the end yet :)

On more personal note, I have been trying to pick up the Diatonic Harmonica, practicing while waiting for my wife to go off work. Always sort of regretted that I never did pick up a musical instr ument, haha now trying to catch up. Seems like a lot of people around my age are playing catch up in music, I know a few who are trying to pick up the piano, trumpet and the violin.

Hope to play like this guy below someday!

or even this guy!

Till the next time...

Winston W Royce - Father of waterfall

2010-11-26T21:53:00.001+08:00

Uncle Bob posted an article, What killed waterfall could kill agile.http://cleancoder.posterous.com/what-killed-waterfall-could-kill-agile. He mentioned about a person named Dr Winston W Royce (whom uncle bob called the "Father of waterfall") who published a paper, Managing the Development of Large Software Systems: Concepts and Techniques, in which a system of developement now know as the Waterfall model was described.

What shocked me in that paper was not that he advocate the waterfall model, but rather he condemns it. In his own words
"I believe in this concept, but the implementation described above is risky and invites failure".

The funny thing is so many generations of programmers who studied in colleges and university were brought up believing that it was one of (or depending on how old you are) or "the" model of how software actually gets built. Now we know better, that's why we have things like the Agile movement, Kanban and so on

If you read the paper, you will discover he has views on how to improve the process and what is surprising is how similar it is to practices that we are advocating today.

His 4th and 5th recommendation looked remarkably like Agile practices.

Step 4: Plan, Control and Monitor Testing
Testing

Code inspection
He advocates the use of code inspection by a second party, he said the use of computers would be too expensive for this task. But with the computing power we have today. Software like Findbugs or JTest can help in the first line of bad code detection and Pair Programming helps in that aspect too.

Code Coverage
Code coverage is also advocated in fact he advocates testing every logical path if possible and if he is the customer he would not accept delivery until this step is done. Although he acknowledges the difficulty in accomplishing this step for large programs.

Step 5: Involve the customer
He knows that software design and specifications is open to a wide interpretation even after agreements, customer involvement in his words should be "formal, in-depth and continuing".

Let's not remember Winston W Royce as "Father of the waterfall" but rather one who saw how software development could be done better 40 years ago.

Relational selects in YII

2010-11-20T08:27:00.001+08:00

How do you do relational selects in YII? The kind where you have a many to many relationship on a table. The YII page on relational queries didn't have many good examples on how to do it, so I tried it out and found it to be actually quite easy, its quite like how you would do for single table query.

The above figure was taken from the YII page on relational queries (I was too lazy to do up my own example).

We will just concentrate on the tbl_post , tbl_post_category and the tbl_category tables.

I will assume you know how to overwrite the relations function in CActiveRecord to setup the relations correctly.

So the scenario is, I would like to retrieve posts that are in a certain category.

The idea is to use the with() method that is part of CActiveRecord, we would like to retrieve the categories together with the Posts.

So the query would look like this:

$posts=Post::model()->with(array('categories'=>array('condition'=>'name="FunStuff"')))->findAll();

In the with portion you specify the relations that you want to load in this case categories, and with it you can specify the different relation query options like select, condition and so on. In this case we use the "condition" option to specify what from categories that we want to retrieve. For this we want to retrieve all the Post with categories name = "FunStuff".

Simple rite!

LOTW - 11 Oct 2010

2010-10-11T21:09:00.001+08:00

Design Principles Behind Smalltalk - One of the most influential language although it didn't become popular. Java, Ruby all draw inspiration from it. If you are a programmer you owe it to yourself to go try Smalltalk. Squeak and Pharos are probably the more famous of the free implementation around.

Google's answers to Oracle's allegations

Doing quite a bit of PHP programming these days so there are the few links that might be useful
Using Emacs to edit php files
Geben + XDebug to debug php code on emacs - Haven tried this, so far with PHP, doing print statements still works for me.

AptOnCD Backs up all the packages that you have installed, making it easy to restore your packages instead of downloading them all again.

LOTW - 4 Oct 2010

2010-10-04T11:24:00.000+08:00

A really simple introduction to Map Reduce - Great intro to what Mappers, Groupers and Reducers are all about. I was scratching my head about Map Reduce before this came about.

How big are Facebook's data centers? This FAQ will tell you all that you need to know including servers, software, location and expansion plans.

Google URL shortener goo.gl opens for business

Heard of the Arduino, unknown to me there is actually a Netdunio yeap, and it runs on .Net. Scott Hanselman talks about it.

Quote: Paul Graham on Beautiful software

2010-09-28T08:36:00.000+08:00

"Great software requires a fanatical devotion to beauty.

If you look inside good software, you will find parts no one is suppose to see are beautiful too."

Paul Graham from Hackers and Painters

LOTW - 20 Sept 2010

2010-09-20T01:00:00.001+08:00

I missed last weeks LOTW due to lots of work and basically I was lazy haha, so this week gonna do a bumper issue. Enjoy

97 things every programmer should know - I guess most would already know about this, every programmer should go read what's in there, there are so many pearls of wisdom, now we have a video by Kevlin Henney at the keynote at jazoon talking about. Slides can be found at the video link.

Epic failures: 11 infamous software bugs

Google is going to release Wave in a Box, an installable version of Google Wave, since they have decided to discontinue Google Wave. The version is not going to have all the features of the original wave, but they are going to support federated across other Google wave boxes.

Lisp cabinet - For those who wants to tinkle with Common Lisp "Lisp Cabinet is a set of configuration files, tools and utilites bundled with automated installer to deliver full-fledged Emacs based Lisp development environment for Microsoft Windows"

Check out all the cool videos from NDC 2010 in this torrent
interested to hear Uncle Bob and Mike Cohn speak, Uncle Bob's slides from his Clean Code I Args are here.

Lamda the Ultimate has two posts that focus on Software Transactional Memory, one post quotes from a study that STM actually helps reduce errors as compared to the traditional locking model, while the other post talks about the support for STM in .NET and the decisions they have to make. I haven't really tried STM yet in any language, but basing modifications around a database like transactional model can probably make understanding and programming easier.

Designing Interfaces guidelines.

2010-09-15T10:11:00.006+08:00

We all write interfaces when programming, these are the few rules that I found useful to keep in mind when designing and implementing interfaces.

1. Do not write interfaces for Model classes
Model like classes are classes like your DB model, or any other kind of class whoose primary function is to hold values. I usually do not write interfaces for these kind of classes for their main purpose is to hold values, and so let say you have Person class with getters and setters for Name and Address. And you have a interface call IPerson so you have getters and setters in the interface for IPerson. If you add a Telephone to Person, you need to add both the getters and setters to the IPerson. And it just goes on and on.

So far I haven really found a nice use case for implementing interfaces for simple value/model kind of classes.

2. Reduce your external dependecies
The reason you are writing interfaces is so that you can reduce the dependcies and change implementations if necessary. Consider the following

import org.internal.*
import org.external.NameValuePair

public interface HttpPost{
public void setPostParameters(NameValuePair[] pairs);
....
}

You are writing an interface to HttpPost and you have a method to add the post parameters. Your NameValuePair class actually comes from an external library. Now you have a dependency to that external library and you will need to package that every time you are using this interface.

I will try to keep this to a minimium, the reason why I write interfaces is to keep dependencies simple not to introduce new ones. So i might introduce a new class to break that dependency if needed or change the method signature to something like addPostParameter(String,String);

3. Keep your interfaces small and cohesive
Try not to have interfaces that does sort of everything, keep your interfaces small and cohesive (that means the methods in the interfaces are related to each other).

Bob martin in his article on the Interface Segregation Principle talks about using the Adapter pattern and multiple inheritance to join multiple interfaces together.

I like the line from Uncle Bob, "Clients should not be forced to depend on interfaces they do not use". Keep that as a rule when designing and implementing interfaces.

Martin Fowler Role Interface article also talks about this.

Interfaces are great for breaking up dependency and when use together with the Dependency Inversion principle you will have a easier time coping with changes to your software.

These are the book I recommend to improve your code.
Effective Java

Clean Code: A Handbook of Agile Software Craftsmanship

Agile Software Development, Principles, Patterns, and Practices

LOTW - 6 Sept 2010

2010-09-06T07:16:00.000+08:00

Yahoo releases the Yahoo YUI Testing Interface it allows running to units tests in different browsers simultaneously.

The first step is to start from 37 signals - If you want to start something, just start, dont worry about it. As Yoda says "Do there is no try". Good advice for me ...

James Gosling creates a "Free Java from Oracle" t-shirt. Frankly I wonder what if Java was free from the beginning, would it reach the popularity that it has today? Look at those with ISO like organizations behind it like C++ or Ecmascript, they tend to have abit too much political maneuvering and arguing and it take a long time for them to agree on standards. For the one person system that Python, Ruby have, the seem to lack the mass acceptance that Java commands (because SUN was backing it up those days), yeap they are popular but are they really being used a lot in the Enterprise space?

Scott Hanselman talks about 2 tools he use for a more readable web, Instapaper and Readablility.

Google Code University - Offers courses on C++, Python, Android, Web security, lots of good stuff comes with slides, assignments and videos.

Let's face it programmers are probably not the best designers in the world (me included), hope the following 2 links can help out.

20 useful free pdf books for designers and bloggers

10 resources for design challenged programmers

What's your motivation for programming?

2010-09-02T10:18:00.001+08:00

Studies since the 1970s that show that Intrinsic motivation (from within) rather than Extrinsic motivation (external factors like money) is the main motivation for people doing good work. In Drive: The Surprising Truth About What Motivates Us, Daniel Pink talks about the various research and studies and even gave some suggestions on how to improve your workplace, if you are a lead or manager go read the book. If you don't have time to read the book, RSA has a nice video (about 10 mins) that sums up book quite nicely.

So what is your motivation for doing your job as a programmer? What are some of the motivations that you have for continuing to do programming as work? Here are some that I came up with, feel free to add more in the comments section.

1. Joy of knowing that there is always something new to learn

I am always amaze that there are still so many things that I can learn in this field, and that's a motivation for me, because I will always have some new framework, language, technology to play with.

2. Joy of finding a solution to a problem.

Solving a problem be it in programming, performance or a process issue is always satisfying

3. Joy of seeing a user liking your solution.

For most of us, we program for real users and its a good feeling when a user tells you that you did good work.

4. Joy of doing work with people that you like or like minded.

I am glad to be able to work with a bunch of guys who I like and feel comfortable with, the pantry talks and the lunches help make for a more interesting day.

Exploring Google Analytics Api - 1 Client Login

2010-08-31T08:51:00.001+08:00

Blogger dosen't have a "popular posts" widget, and since I am using Google Analytics and there is a popluar posts metric, i decided to roll my own and host it using App Engine. I am roughly following the examples on the Google Analytics Api site but adapting it for AppEngine use.

There are 3 methods to authenticate to Google Analytics and retrieving data.
1. Client Login
2. AuthSub
3. OAuth

The explanation for all these 3 types are provided here with the various scenarios on what to use.

If you are planning to serve out data from a server you should use OAuth, its the most secure and there is no passing of passwords anywhere, however since I am still playing around with the API and I don't want to mess with the complexities of OAuth authentication. I will be using ClientLogin for now.

Google does provide client libraries in Java for accessing the Google Data API (which Analytics is a part of), however they don't play well with App Engine because they use some restricted classes. So we will have to go with the protocol version, i.e using POST, GET calls to get the data.

You will need to POST the following data to https://www.google.com/accounts/ClientLogin with the following parameters.The table below from Google shows the parameters.

accountType	Type of account to request authorization for. The default is `GOOGLE`, which is currently the only option supported by Google Analytics.
Email	The user's email address. It must include the domain (e.g. joe@gmail.com).
Passwd	The user's password.
service	The Analytics service name is `analytics`. (For other service names, see the service name list.)
source	A string identifying your client application in the form `companyName`-`applicationName`-`versionID`.

Table 1: Post Parameters (Credits Google)

In App Engine, to do POST/GET request you will have to go quite low level, you will need to use the URL and HttpURLConnection classes. Below is a sample of using the classes to do a POST to the client URL.

URL loginUrl=new URL("https://www.google.com/accounts/ClientLogin");
   
StringBuffer message=new StringBuffer();
message.append("accountType=GOOGLE&");
message.append("Email="+URLEncoder.encode(username)+"&");
message.append("Passwd="+URLEncoder.encode(password)+"&");
message.append("service=analytics&");
message.append("source=stephenonsoftware-sitedata-1");
   
HttpURLConnection connection=(HttpURLConnection) loginUrl.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
   
OutputStreamWriter writer=new OutputStreamWriter(connection.getOutputStream());
writer.write(message.toString());
   
writer.close();
   
if(connection.getResponseCode()==HttpURLConnection.HTTP_OK){
BufferedReader reader=new BufferedReader(new InputStreamReader(connection.getInputStream()));
    
String line;
StringBuffer result=new StringBuffer();
while((line=reader.readLine())!=null){
 result.append(line);
}
reader.close();
return getAuthString(result.toString());
}else{
return "";
}

You will get back a long string containg the SID,LSID and Auth parts, you will just need the Auth part so just split the string at 'Auth=' and you can get the authentication token, you will need this token to request for the data from Analytics.

Now you have successfully login using ClientLogin, next I will post about how to get your account details from analytics.

Links:
Using java.net

LOTW - 30 Aug

2010-08-30T07:14:00.001+08:00

“My husband is a programmer; I have no idea what that means.” - Nice blog post about a non programmer and a programmer geek getting together and learning about each other. I agree with Renae, the important thing is to take an interest in what your other half does during work and his hobbies.

Patterns and Practices for Improving Personal Productivity, Time Management, and Effectiveness

Tim Daly writes about writing big Lisp applications.- Tim Daly responds to Steve Yegge's Lisp is not an acceptable Lisp. I let you make your own decision about whether Lisp is good. For me as a developer/programmer wanting to have more "breath" in your knowledge (I guess most of us are brought up in the OO kind of world), you owe your self to go try one of the functional languages, for me I am going to try Common Lisp and Clojure.

Pascal Costanza's Highly Opinionated Guide to Lisp - Pascal talks about why use Lisp and some of the common pitfalls and where to find more information on your discovery of Lisp. I was always a bit confused by the backquote operator ` in Lisp, his explanation is the easiest I have found. Gotta spend more time reading Practical Common Lisp !

LOTW - 23 Aug 2010

2010-08-23T16:46:00.000+08:00

Oracle Sues Google - This must be the big thing this past week.
CNet's Report has a copy of the letter from Oracle's lawyers.
Top 5 grievances with Oracle - A list of complaints about oracle and open source.
James Gosling foretold
Google saying that Oracle's suit is baseless.

Writing an operating system - Something that I always wanted to try. Lets put that in the todo list !

F# in Visual Studio - Functional programming is getting lots of press these days, for the JVM; Clojure and Scala and for .Net F#, the video shows how F# can be use most effectively and introduces F# features like parallelism, succinct expressive syntax, rich access to .NET libraries.

You can get the beta Windows 7 Phone developer tools already. Singapore is one of the launch countries so we will have our own application marketplace here.

Scott Hanselman in his 56th Weekly source code talks about the Visual Studio 2010 training kit and some of the cool stuff in there. I was particularly interested in the Parallel Framework

Oracle sues Google over Android - Possible outcomes.

2010-08-14T12:27:00.002+08:00

Oracle finally sues Google over Java. CNET has a copy of the complaint letter from Oracle. Other than a list of patents that was infringed nothing much was said about how it was infringed. I always thought that with the coming of Android, Java finally has a great platform in the mobile arena, I know people will agree with me that Java ME sucks most of the time.

I agree with Miguel that Java as a product has never been profitable for Sun, when Sun was around the big names in the J2EE app server market was Oracle, Bea and probably IBM that made money from selling middleware products. So i guess since Oracle paid quite abit for Sun and so they need to milk it for all it's worth.

How will all this pan out? Below are my 4 guesses.
1. Google pays Oracle
Straight forward way to solve the problem, but that will mean anyone else who has the same idea as Google needs to cough up money big time and that affect's Java, although people might not be moving away quickly but stagnation may occur and that's bad for any language.

2. Google goes to court and wins ! And all is well in the Android world.
Might be a long and drawn out battle, but sets the scene for even more innovation in the Java eco system. The thing is what will Oracle do about Java if it loses this battle.

3. Google goes to court and lose, Google pays Oracle
Same as 1

4. Google moves Android to alternative languages (Python, .Net).
That would be interesting, considering that Goggle is quite the advocate for Java and Python (look at App Engine). Would love to see Android supporting Python in the future, quite like how Symbian has some support for Python (although I think Symbian as a platform is dead also), the next closest alternative would be a move to C#, since they are quite close in terms of syntax and perhaps popularity, and it doesn't cost much for a Java programmer to move to C#.