Thought Bytes

Graph visualization with Neo4J and Arbor.js

2011-02-03T13:24:00.011+05:30

As you might have gathered from my previous post, I have been tinkering with Neo4J lately and I am beginning to appreciate the power and elegance of graphs.
I also came across Arbor.js and was bowled over by it. Loved the force directed layouts and the fact that if you are patient, you will watch your nodes traveling far and wide to settle in their final destination. No need to drag, drop or re-organize the graph- what emerges is a visual pattern, obvious to the eye. I can think of tons of usages for such a visualization, but thought I'd start with something simple just to get the feel of it.

I decided to model a simple organization chart or graph. So off I went to create that model in Neo4J, fed data into it, and my graph was ready very quickly.
On to Arbor.js- I must admit, I was rather stumped, even with the reference doc they publish. I did not know where to start and where to end- of course, my lack of knowledge around jQuery syntax contributed a lot.

So, I did the next best thing which was to get the sample project working locally. Tweak main.js to play with the layout and understand what's really happening. Scroll right down to the $(document).ready function to see the 'ParticleSystem' getting initialized. The sample project initializes with three parameters: repulsion, stiffness and friction. There are more available like frames per second and precision, but I have not tried them yet. Play with those values to see how they affect the graph. I settled on: var sys = arbor.ParticleSystem(500, 600, 0.5)
I also left gravity=true since I definitely want the settling graph.

You can then add edges and nodes to the ParticleSystem object you just initialized.
Adding a node is as simple as sys.addNode(name,data).
The name you supply functions as an identifier of the node and the data is an object you can use to associate any set of parameters with this node. These are especially useful when you need additional information about the node to render it- for example, set its color.
Adding an edge is just as simple: sys.addEdge(source,target,data). Source and target nodes are identified by their node names, and data again is an object of parameters to associate with the edge.

Now the sample project displays little black boxes for nodes. I was looking for something more like circular nodes with their labels in them- much like the arbor.js home page.

Just adding nodes and edges does not do anything- the ParticleSystem needs a renderer to well, render the graph. The arbor.js reference will tell you that you need to create an object with two methods- init and redraw, and set your ParticleSystem renderer to this object.
If you look at main.js, this is exactly what it does (viewport is the canvas object in which to render the graph)-

sys.renderer = Renderer("#viewport")

The init method in this case simply sets up screen dimensions.
The redraw method is the one responsible for painting nodes and edges. You will see those in the
particleSystem.eachEdge and particleSystem.eachNode loops.

So then this is the place to tweak the actual representation of nodes and edges.
Borrowing from the source of the arbor.js home page, I changed the code to:
particleSystem.eachNode(function(node, pt){
        
var w = Math.max(20, 20+gfx.textWidth(node.name) )
if (node.data.alpha===0) return
  if (node.data.shape=='dot'){
      gfx.oval(pt.x-w/2, pt.y-w/2, w, w, {fill:node.data.color, alpha:node.data.alpha})
      gfx.text(node.name, pt.x, pt.y+7, {color:"white", align:"center", font:"Arial", size:12})
      gfx.text(node.name, pt.x, pt.y+7, {color:"white", align:"center", font:"Arial", size:12})
  }else{
      gfx.rect(pt.x-w/2, pt.y-8, w, 20, 4, {fill:node.data.color, alpha:node.data.alpha})
      gfx.text(node.name, pt.x, pt.y+9, {color:"white", align:"center", font:"Arial", size:12})
      gfx.text(node.name, pt.x, pt.y+9, {color:"white", align:"center", font:"Arial", size:12})
  }

})
You will note that the color of the node is pulled from the data object attached to the node.
The font, size, width etc. can all be parametrized by setting them in the data object when you create the node.
This example uses the graphics library, so be sure to include graphics.js in your html.


For the edges, I changed the color and also parametrized the thickness of the connector line by the width parameter.
particleSystem.eachEdge(function(edge, pt1, pt2){
if (edge.source.data.alpha * edge.target.data.alpha == 0) return
      gfx.line(pt1, pt2, {stroke:"#151B8D", width:(edge.data.weight/2), alpha:edge.target.data.alpha})

       })


That is sufficient for a basic graph representation. Now to get the data out of Neo4J and into the graph.
The ParticleSystem has a graft method which accepts an object in the form {nodes:{}, edges:{}}.
These nodes and edges are added to the particle system.
I navigated through my Neo4J graph and spat out the data in this json like format.

Then, in the JS, you simply do a
var data = $.getJSON("orgChart.json",function(data){
  sys.graft({nodes:data.nodes, edges:data.edges})
})
and you're done.
Just make sure you enclose each attribute in quotes- for example, color and shape in the code below.
And it also freaks if you supply a .8 instead of a 0.8 for the length attribute in this example.


The final graph rendered in my browser looks like this(click to enlarge)-

This really just scratches the surface of what you can do with arbor.js
Add to it the various types of graphs or models you can extract out of Neo4J, and you could build some great stuff!


orgChart.json contains the following-
{"nodes":{
   "Aidan":{"color":"green", "shape":"dot", "alpha":1},
   "Sofia":{"color":"green", "shape":"dot", "alpha":1},
   "Liam":{"color":"GoldenRod", "shape":"dot", "alpha":1},
   "Charlotte":{"color":"green", "shape":"dot", "alpha":1},
   "Noah":{"color":"GoldenRod", "shape":"dot", "alpha":1},
   "Ava":{"color":"green", "shape":"dot", "alpha":1},
   "Jackson":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Addison":{"color":"green", "shape":"dot", "alpha":1},
   "Ethan":{"color":"green", "shape":"dot", "alpha":1},
   "Olivia":{"color":"green", "shape":"dot", "alpha":1},
   "Kayden":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Amelia":{"color":"green", "shape":"dot", "alpha":1},
   "Mason":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Lily":{"color":"green", "shape":"dot", "alpha":1},
   "Logan":{"color":"green", "shape":"dot", "alpha":1},
   "Isabella":{"color":"green", "shape":"dot", "alpha":1},
   "Jacob":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Emma":{"color":"green", "shape":"dot", "alpha":1},
   "Benjamin":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Abigail":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Caleb":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Scarlet":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Owen":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Audrey":{"color":"GoldenRod", "shape":"dot", "alpha":1},
   "Gavin":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Grace":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Connor":{"color":"green", "shape":"dot", "alpha":1},
   "Claire":{"color":"GoldenRod", "shape":"dot", "alpha":1},
   "Riley":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Emily":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Nolan":{"color":"lightblue", "shape":"rectangle", "alpha":1},
   "Quinn":{"color":"red", "shape":"dot", "alpha":1}
},
"edges":{
   "Quinn":{"Liam":{"length":2.5,"weight":2},
           "Audrey":{"length":2.5,"weight":2},
           "Noah":{"length":2.5,"weight":2},
           "Claire":{"length":2.5,"weight":2}},
   "Liam":{"Sofia":{"length":2.5,"weight":2},
           "Ethan":{"length":2.5,"weight":2},
           "Amelia":{"length":2.5,"weight":2}},
   "Audrey":{"Aidan":{"length":2.5,"weight":2},
               "Ava":{"length":2.5,"weight":2},
           "Jackson":{"length":2.5,"weight":2}},
   "Claire":{"Charlotte":{"length":2.5,"weight":2},
           "Olivia":{"length":2.5,"weight":2}},
   "Noah":{"Addison":{"length":2.5,"weight":2},
           "Kayden":{"length":2.5,"weight":2}},
   "Olivia":{"Mason":{"length":2.5,"weight":2},
           "Lily":{"length":2.5,"weight":2}},
   "Ethan":{"Logan":{"length":2.5,"weight":2},
           "Isabella":{"length":2.5,"weight":2}},
   "Addison":{"Jacob":{"length":2.5,"weight":2},
           "Emma":{"length":2.5,"weight":2},
           "Benjamin":{"length":2.5,"weight":2}},
   "Emma":{"Abigail":{"length":2.5,"weight":2},
           "Caleb":{"length":2.5,"weight":2}},
    "Logan":{"Scarlet":{"length":2.5,"weight":2},
           "Owen":{"length":2.5,"weight":2},
           "Gavin":{"length":2.5,"weight":2}},
    "Lily":{"Connor":{"length":2.5,"weight":2},
           "Grace":{"length":2.5,"weight":2},
           "Nolan":{"length":2.5,"weight":2}},
    "Connor":{"Riley":{"length":2.5,"weight":2},
           "Emily":{"length":2.5,"weight":2}},
   }
}

RuntimeExceptions and Neo4J

2010-12-13T16:52:00.006+05:30

I'm currently working with Neo4J(http://www.neo4j.org) - in the evaluation stage really, trying out various use cases. TestNG is my test framework of choice, so most of my work is driven by it.
What really irritated me is the fact that some tests failed. When I looked at the TestNG report, I found that in some cases my test data was bad, so in effect, I was creating a self relationship (which Neo4J frowns upon). There were also cases where, during trial and error, I did not clean the db so I ended up doing a node.getSingleRelationship and instead of one relationship, there were more.

So why was I irritated? My test cases did fail due to an exception. However, they were Runtime exceptions. When I wrote the code, I did not think they may be thrown- I did not look at the JavaDoc in detail- just used IntelliJ to show me what I could possibly do with Node, and winged it from there. So, when my test failed, I was a bit disbelieving. First checked the exception- and it was a Runtime exception (duh). Then pulled up the JavaDoc and saw that Neo4J does indeed indicate an exception can be thrown in these cases.
This caused me to tweet about Neo4J and their runtime exception philosophy. (Pleasantly surprised to see @emileifrem respond to it)

Anyway, somewhere in one of many open windows, I happened to search for any such issues related to Neo4J. I came across Stephen Colebourne's blog instead (http://www.jroller.com/scolebourne/entry/checked_exceptions_bijava)

So really, did it matter whether Neo4J threw RuntimeExceptions or not? Apart from the test results, which were surprising because I neglected to read the JavaDoc in detail, I went through some exceptions that Neo4J throws (leaving out the very obvious Runtime exceptions).
Now if a DeadlockDetectedException, HaCommunicationException, NotFoundException(thrown if a request is made to a node, relationship or property that does not exist), NotInTransactionException, ReadOnlyIndexException or TransactionFailureException were thrown, what would I really do about it? Nothing much, unfortunately. There is no way I could be smart and recover from these. At the most I'd log it and re-throw it.

Now on to the ones I was disturbed with- methods mostly on the Node.

getSingleRelationship() which (as the name indicated) returns the only relationship of a given type and direction. Well, the fact that I'm using this method does mean that I really expect there to be just one relationship, so when a runtime exception is thrown, there is no way to recover from it.

delete() throws a RuntimeException if it has relationships attached to it. Can I do anything about it? Maybe- depending on functionality, I might attempt to do a cascade delete and re-try the operation. Not sure how many use cases would really want a cascade delete though.

createRelationshipTo() throws a RuntimeException if you try to create a self relationship. Note here, that the JavaDoc does not specify this in the Throws, but rather in the description. This arguably could be dealt with by creating an intermediate node- but at the same time, this situation can easily be avoided by defensively coding for it.

So did it really matter that Neo4J threw Runtime exceptions? After a lot of thought, not really. Why some exceptions are true classes of RuntimeException and why some (the Node exceptions) are just RuntimeExceptions in themselves, I am not sure. The source does not indicate that they do anything over and above being a RuntimeException.
Does it really matter that there is a split convention? I'd think if I needed to recover from a delete exception, then it does matter.
It will be interesting to see as I develop more with Neo4J, how much these Runtime exceptions come into play and whether they are really as disturbing as I initially thought.

Disclaimer: I still do not believe that we should do away with checked exceptions altogether :-)

If you aint going to use it, don't write it

2010-04-16T08:47:00.003+05:30

It's interesting to see how engineers "follow processes" that they don't agree with. They might not agree with it because maybe they haven't understood the value it provides to them (after all, they are typically not dumb guys), or they are not provided with adequate time to follow them, or they are just plain stupid (the processes- not the engineers).

I've written about testing before. Once I realized its value, I don't go anywhere without my tests (so to speak). Now you can preach about it all you want but unless engineers try it, do it properly, and realize the value they lend to our complicated lives, they're not going to write them. Now take those engineers and tell them that they must write tests. First, everyone ignores the new law. Then they're reminded and now it's an exit criteria with one more requirement- coverage numbers.
So, what happens? They produce tests. They run them and get them to pass and management is happy. But what are the value of those tests? Are they maintained? Are they used to help verify that they didn't break anything when the next feature comes along? There are even interesting cases where tests haven't been run as part of builds for months on end and nobody noticed. Which says to me "delete those tests". If nobody cares about those poor things, it is more merciful to end their lives.

If you're not going to get value out of code you've written, then it is better not to have written it in the first place. I'm sure you can think of better ways to use your time (even if it's taking a nap).

-Luanne

Marker Interfaces- what are they marking?

2010-04-01T22:42:00.003+05:30

This might be a controversial statement, but I dislike marker interfaces. Yes, I get that the Serializable marker interface, for example, has a purpose. Let me amend that statement to 'I dislike marker interfaces with no clear intent'.

An interface specifies a contract. If you implement me, then you must provide these services.
An empty interface cannot specify a contract because, well, it doesn't have one to offer. It is more of a tagging mechanism. It says, if you implement me, then you're tagged in this manner. But, in most of the cases I've seen, nothing is done with this information. OK, so the class is tagged- but what the heck does that mean? Does it mean the class behaves in a certain manner? Does it mean the class can offer these services? Who knows? More often than not, marker interfaces are used as a handy mechanism to refer to classes that implement it, in a uniform manner, without the hassle of inheritance. That's about it. Which is why you'll find code riddled with if-then-elses that do an instanceOf. Kind of defeats the purpose.

What is worse about these badly designed marker interfaces, is that it's one large, no-invite-needed party; anyone and their pet elephant can tag along. Not that having the elephant would break up the party. But it isn't appropriate for the elephant to be there- the party wasn't really meant for him and passers-by would have a hard time figuring out what exactly was going on.

Let's take this very example. Your event planning application has people that are invited to parties. Just as you finish designing your Person class, along comes someone and says, we also invite entertainers to the party. The entire group is to be a single guest. Of course, they're paid and they're not expected to eat the snacks, but they're on the invite list. All your nice API's that returned collections of Person to represent the entire guest list is now left wondering what to do.

So what is usually done? Simple, lets introduce this marker interface:
public interface Invitee {} and have Person and EntertainmentGroup implement it. Then all we do is change our API to return Invitees and we're done.
What does it mean to be an invitee? Nothing.
Now you need to find out how many snacks to order assuming that each person will eat 5. The code I see more often than not is, get a collection of Invitees from the guest list, iterate through them, check if the Invitee is an instanceOf Person- and increment the snack counter by 5.
What did it mean to be an invitee? Nothing.
Tomorrow when the zoo keeper is invited to the party and insists on bringing his pet elephant, what do you do? Simple. Have Elephant implement Invitee. Then when you calculate the amount of snacks to order, you add yet another check to see if the Invitee is an instanceOf Elephant and you add 50 kilos of bananas to the list.
Ideally the Invitee class would specify what it means to be an invitee. It's behavior is predictable. When the event manager gets rich and decides that only movie stars can attend his parties, then you can no longer have Invitee extend MovieStar, unless you only know elephants that are movie stars.

Unless you are doing something very concrete with the marker interface, unless you are processing that tag information in some manner, I consider it a Bad Thing.
Tagging can be accomplished using annotations. You can selectively tag classes/methods and actually process them to do or mean something useful.

What do you think? Do you see marker interfaces being used as convenience mechanisms?

-Luanne

DeadlineExceededException on GAE with Grails

2010-02-10T23:45:00.003+05:30

Grails on the Google App engine takes substantial time to startup and will probably exceed the hard deadline limit set by GAE. If you see a lot of "DeadlineExceededException" errors in your logs, you should be aware of this. There is a jira for this - http://jira.codehaus.org/browse/GRAILSPLUGINS-1736 . Vote this up if it matters to you.

Notes on Google AppEngine 1.3, Grails 1.2.1, and app engine plugin 0.8.8.

2010-02-03T17:57:00.007+05:30

Currently in the process of upgrading a grails project to grails 1.2.1. Noted down some observations (and will keep updating as i test stuff)

1.
There doesn't seem to be a need to add the following to the Config log4j anymore (this was needed in grails 1.x see http://jira.codehaus.org/browse/GRAILSPLUGINS-1184)

'null' name:'stacktrace'

2.
Had some trouble with the sessions and kept getting this error

[java] WARNING: Nested in org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException: Session support is not enabled in appengine-web.xml. To enable sessions, put

true

in that file. Without it, getSession() is allowed, but manipulation of sessionattributes is not.:

As advised in the plugin documentation, added the following to Config, but it still didn't work!

google.appengine.sessionEnabled = true

Had to eventually update the /web-app/WEB-INF/appengine-web.xml file and uncomment the line below


3.
Can now user grails run-app to run the development server instead of grails app-engine

The Refactoring Quandary

2007-09-06T14:13:00.000+05:30

We all know by now that refactoring is important. The problem is- when do you refactor?
In my opinion, the time to refactor is when the thought first crosses your mind.

Let's say you're implementing a new feature or fixing a bug. You think 'hmm, I seem to be duplicating code' or 'looks like we could reorganize a bit of this'. Instead of pushing the thought 'this could be refactored later' to the back of your head where it will stay there for a very long time and come back to haunt you once your code is so messy that even you don't know how it got that way, the answer is to refactor immediately. Do what you can to improve your code. If you don't, then you and everybody else just pass the buck on and say "X will do that someday", and soon you are in programming hell. Moreover, if you refactor regularly, you most often need to just refactor small areas at a time. It's much better then saying we'll refactor this entire application- more often than not, throwing it out and restarting would be more appealing.

Of course there are reasons why people are frightened of refactoring. Acknowledging the need to refactor does not mean saying to the world "I have bad code". You do not have to have glaring bugs to think of refactoring. Think of it as general housekeeping. Just because the hotel housekeeping staff cleans your room every morning does not mean you left it looking like a pig sty the night before.
Another important reason is the fear of breaking code. Which is why to refactor confidently, you need to be backed by a solid suite of test cases. If you don't, then this is a very real risk you run.

Refactoring is easier said than done. You rarely find that you've been told to spend the next week refactoring. There are always more pressing tangible deliverables. However, it is extremely important to realize that the intangible ones will actually affect your tangible deliverables in the near future. Just as we need to consciously budget time to write automated tests, refactoring does not happen magically either. A small bit of effort regularly will go a long way to contributing to better manageable, extensible and flexible code.

-Luanne

At the local Udipi...

2007-08-24T15:10:00.000+05:30

So I'm sitting in this udipi fast food restaurant waiting for my lunch (rawa idli - highly recommended), and I find myself observing the waiters going about their jobs. This is what typically happens:

1. Customer enters, sits, decides what he wants and tells the waiter.

2. Customer asks the waiter how long it will takes and the waiter gives an approximate waiting time.

3. Waiter heads over to the kitchen and bellows the order to the cooks.

4. If a customer gets impatient, the waiter goes to the kitchen to check and assures the customer 'it's almost done!'.

5. Once the order is ready, the waiter takes it to the customer's table and serves it to him.

6. If the customer didn't like the meal, the waiter gets a earful. He smiles and goes back and abuses the cook.

7. Waiter handles many customers at the same time.

8. Finally the waiter hands over the bill.

With a start I realise - it's no different from my current job. I'm sort of a Project Lead. This is what I do:

1. The client sends me the requirements.

2. Client asks me how long it would take to deliver it. I estimate and return a figure.

3. I hand the requirements over to my team, who then start working on it.

4. Client gets impatient and eager to see 'anything', so I tell them 'it's 90% done!'

5. When it's actually done then it's released to the client

6. If it's a poor quality release, then I get a earful from the client and I pass it on to my team.

7. Many such requirements are being handled simultaneously.

8. End of the month, I total up the time sheets for the billing.

!!!

I ate quickly, tipped the waiter handsomely and left.

Someday, I want to go back into the kitchen and be the cook...

Cheers,
Aldrin

How much can we test?

2007-06-11T09:22:00.000+05:30

A couple of years back, I was introduced to unit testing. At the time, I found it rather tedious. Though it did expose bugs in our code early in the cycle, maintenance of test cases were becoming a nightmare. So the question is, what should you test, and how much should you test it? I've formed some opinions along the way- tell me what your experience has been...

First, every automated test is NOT a unit test. A unit test is simply a test that validates the smallest "unit" in your application. It could be a class, and you would be validating each operation of that class by testing its methods, for example. A test which tries to validate a business operation that spans more than one class would probably be a functional test.
A simple example would be a Shopping Cart class. A unit test could verify that deleting items from an empty cart is not possible. A functional test would be one that validates the shopping process- add items to cart, checkout, verify bill amount.

Which kinds of tests should you write? Only unit? Functional? Both?

Well, unit tests are no doubt good. You should ideally write them before you write your actual code. Ideally. In the real world of the services industry at least, there are very many projects whose scope and functionality change so fast that programmers are always playing catch up with unit tests. It is simply not possible to maintain a vast number of test cases. So writing unit tests become an afterthought. And they are the very first thing axed from the budget :-)
If we were to write comprehensive unit tests that really add value, teams would require time to do so. They would also require time to maintain them. That time is rarely granted.

So, what is the next option? What about writing functional tests? Agreed, they will not provide 100% coverage, but let us assume that we could test major business flows/processes. The amount of test cases decrease, but the ROI is visible earlier on because major flaws can be uncovered before moving into system testing. There will always be bugs. Subtle bugs that even unit tests will fail to pick up. So is it worth spending time trying to think of many, many ways to break the code when the code will certainly break somewhere in a place you did not expect it to?
I personally would write functional tests, and enhance the suite over time. For example, when a bug is reported, a test could be written for it. It could very well be a unit test. But with bugs and change requests, the suite will grow in sensible proportions.

What do you think?

Preventing multiple user sessions with the same user account

2007-05-22T08:30:00.000+05:30

Often you want to prevent the same user account being used simultaneously in different sessions (business license restrictions being one reason, complex technical synchronisation problems being another)। There are two typical strategies to prevent थिस :-
#1. Prevent any new sessions while a session with the same user account exists.
#2. Allow a new session, but disconnect any previous sessions if they exists.

Imho, I think #1 can be very problematic for the user. If the user faces a system crash during his session, his subsequent attempts to login after recovery will be thwarted. Until his previous session times out, he will be left twiddling his thumbs. Using cookies to maintain the session information would be a good idea here. If you are supporting auto-login, then you are already doing something like this. A session id stored in the cookie, would be enough clue to server to reconnect to the same session. You need to ensure however, that the user does a proper 'logout' - even going to the extent of hooking your 'logout' into the body unload event if the user chooses to shut the browser window, without clicking on your 'logout' link.
Now what happens when a user logs into your site, shuts his browser, deletes his cookies and tries to login again, before his session times out ?? Let him suffer, i say ! :)

#2 is much easier to implement. All you need to do is maintain a Map of user accounts with their sessions in application scope. Add a listener for session timeouts, to remove these entries from the map when the session times out. When a new session if being created, simply look up the map for previous sessions by the user and invalidate them if present and also remove them from the map.

I'd like to hear your comments on how you achieved similar requirements।

Cheers
Aldrin

You minus IDE=helpless?

2007-04-16T11:07:00.000+05:30

Nowadays, the IDE is your workbench. You can generate your UML, debug, run embedded servers, inspect your code, read the Dilbert strip...all from within a single application.

With each release boasting nifty features, there is very little reason to leave the comfort of your IDE.

Or is there?

In the good old days, we had no such pleasures. I remember writing code in Notepad, and was thrilled when I discovered EditPlus. Debugging meant compiling on the command line, fixing error by error till you had a successful compile, starting tomcat and testing your web app. Today, you can do all this and more at the click of a button.
This is certainly great news for the developer...and a very bad habit as well.
No doubt IDE's save you a lot of trouble and time. IntelliJ IDEA has increased my productivity tremendously and helps me with a whole lot of things- including reading the Dilbert strip ;-)
But, if I had no IntelliJ for a day, then I would still be able to code in EditPlus and look up the JavaDoc from wherever if need be, deploy my application and debug it. And that is the difference.

Too often I see IDE whizzes who can do almost anything with CTRL, SHIFT or ALT and one or two keys. But take the IDE away...and they are helpless. This is Bad Thing in my opinion. The fallout of not knowing the basics is terrible. And this doesn't just apply to software.

I have seen cases where for some reason, the embedded web server plugin fails for whatever reason. Everyone is examining their code and can find nothing wrong...and they're re-examining it for hours and hours. I say shut it down and deploy manually...but few know how to :-)
In fact, very few know how to run javac !!! Turns out that the code base is fine...it was the embedded web server....hours and hours of valuable time wasted because we refused to abandon our beloved IDE.

Consider this scenario- you've developed this fantastic application, deployed it using your IDE, debugged it and it runs like a dream. Then you go to install it at the clients site. And horror of horrors, they don't have your IDE. And won't let you install it. Now what???
You should be able to say so what? You'd roll up your sleeves, check your code out of CVS or wherever, set your build properties, run ANT, deploy, start your webserver and go home in a couple of hours to have a nice drink.
But if you know no such thing...then you'll be pleading with IT to install your IDE and when they refuse you'll have to call someone in your team and have them guide you step by step. And you'll be drowning your sorrows in many drinks after many, many tense hours. Which is a better option??

I am not condemning IDE's. On the contrary. I have used a couple (and IntelliJ by far is THE best...though beware, after using it, like me, you will hate the likes of Eclipse). They are huge help. But use them after you know how to get up and running without their help. If you are developing a simple webapp, can you code a servlet for example, without the IDE? Can you compile it? Can you deploy it? Do you know how to run Tomcat? Where to find the log files?
If yes, then go ahead...open up your IDE and be happy :-)

-Luanne

Hello World with standalone Java Persistence, Hibernate and HSQLDB

2007-04-03T17:53:00.000+05:30

Lately just started reading (& learning) EJB3. Because of a slow PC at home, I needed to be able to run code for the Java Persistence API in the standalone mode, without the overhead of a huge database or app server. So, I rustled up this guide to get up and running with Java Persistence. Basically all you need is a database and an implementation of the persistence API. We will use HSQL and Hibernate Entity Manager respectively. Hibernate Entity Manager runs on the Hibernate core, so will need that too.

Step 1: You will need Java 5. Get it from http://java.sun.com

Step 2: Download HQSLDB from http://hsqldb.org. Extract into some folder (eg. D:\hsqldb)

Step 3: Download the Hibernate Entity Manager from http://hibernate.org. Extract into some folder. (eg. D:\hibernate-entitymanager-3.2.1.GA)

Step 4: Download the Hibernate Core from http://hibernate.org. Extract into some folder. (eg. D:\hibernate-3.2)

Step 5: Set up the database

Create a new folder for all database related files we will create in this step. Lets call it 'db_stuff'.

Create a server.properties file. We will name our database 'think'. Configuration parameters should be as below:

server.database.0 file:./think
server.dbname.0 think

Start the database in the server mode by running the following from the command line (Better still put this in a batch file)

java -cp D:\hsqldb\lib\hsqldb.jar org.hsqldb.Server

You can use the HSQLDB DatabaseManager to view this database. Run the following from the command line to invoke the tool.

java -cp D:\hsqldb\lib\hsqldb.jar org.hsqldb.util.DatabaseManager

In the "Connect" dialog, select the following options:
Type: HSQL Database Engine Server
Driver: org.hsqldb.jdbcDriver
URL: jdbc:hsqldb:hsql://localhost/think
User: sa
Password:

Step 6: Now the standalone. Add the following jars to your project classpath:

hsqldb.jar from D:\hsqldb\lib\hsqldb.jar
hibernate3.jar from D:\hibernate-3.2
all jars from D:\hibernate-3.2\lib\
hibernate-entitymanager.jar from D:\hibernate-entitymanager-3.2.1.GA
all jars from D:\hibernate-entitymanager-3.2.1.GA
Whew!

Step 7: Create the persistence configuration

Create a file named persistence.xml, the persistence configuration file. This file needs to be within a META-INF folder in your classpath. If you are compiling all your classes into a folder, say 'classes', create a META-INF folder under 'classes', then create the persistence.xml file in META-INF. The file should look like below:

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence" version="1.0">

<persistence-unit name="think" transaction-type="RESOURCE_LOCAL">
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect"/>
<property name="hibernate.hbm2ddl.auto" value="update"/>
<property name="hibernate.connection.driver_class" value="org.hsqldb.jdbcDriver"/>
<property name="hibernate.connection.username" value="sa"/>
<property name="hibernate.connection.password" value=""/>
<property name="hibernate.connection.url" value="jdbc:hsqldb:hsql://localhost/think"/>
</properties>
</persistence-unit>
</persistence>

Note that the transaction-type is RESOURCE_LOCAL. This is required for standalone environments. Also note the "hibernate.connection.url" matches the URL you use to connect from the HSQL DB Manager to the database, jdbc:hsqldb:hsql://localhost/think

Step 8: Create an entity class, BroadcastMessage.java

package com.blogspot.thought_bytes;

import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
public class BroadcastMessage {

@Id
private long id;

private String message;

public long getId() {
return id;
}

public void setId(long id) {
this.id = id;
}

public String getMessage() {
return message;
}

public void setMessage(String message) {
this.message = message;
}
}

Step 9: Create a harness to interact with the persistence layer, ClientHarness.java

package client;

import com.blogspot.thought_bytes.BroadcastMessage;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.EntityTransaction;

public class ClientHarness {

public static void main(String[] args) {
//Use the persistence.xml
EntityManagerFactory emf = Persistence.createEntityManagerFactory("think");
EntityManager em = emf.createEntityManager();

//
EntityTransaction tx = em.getTransaction();
tx.begin();

//Create and initialize an entity instance
BroadcastMessage mb = new BroadcastMessage();
mb.setId(1);
mb.setMessage("Hello World");

//Persist using the entity manager
em.persist(mb);

//Save
tx.commit();

em.close();
emf.close();

}
}

Run the above.

Go back to the database manager and refresh the view. You should be able to see a new tabel "broadcastmessage". If you query this table you should get the following:

ID MESSAGE
-- -------
1 Hello World

That's it! And now the real hard work begins... :)

Cheers,
Aldrin

Write pseudocode!

2007-03-30T13:32:00.000+05:30

Writing pseudocode is a good thing.

Just as it is not recommended that you dive into writing code before your analysis and design are done, so should you not dive into code before you have some sort of pseudocode in place. Unless of course you are writing trivial code.

Even if your design is all laid out before you, then there are many cases where pseudocode is a huge advantage.

Simply listing out the steps gives me a lot of clarity about the way forward. I will most likely find that I've discovered areas that were not thought of during design e.g. if X event takes place, or the application state changes to Y, then how do I handle it?
When I'm coding I am in a different mindset- purely focused on code. Side issues like these are unlikely to pop into my head. And worse, if I haven't worked out the steps, then I will most likely miss something.

Getting stuck on a code issue is very common- at that point, all you can think about is how to solve it/work around it. By the time you've achieved this, your train of thought is lost.

I still use paper and a pencil. Whether you prefer electronic pseudocode or hand written, is up to you but in my opinion, it is an invaluable practice.

I am amazed at the number of people who jump into code and flounder along the way and just refuse to write pseudocode like it's some demeaning activity. What is more amazing is the number of tech leads who prohibit it of their juniors-- saying things like "the client wants to see code". It is well worth the time investment- the bugs will definitely be less. And the amount of patching needed on that code will decrease.

So, it's not a "waste of time" or "an activity that only freshers do". Some of the best engineers I know write pseudocode. It would be worth getting into this habit to help you understand your problem, eliminate ambiguities, reduce bugs and above all, write cleaner code.

How to load balance Tomcat 5.5 with Apache on Windows

2007-03-07T17:18:00.000+05:30

It's rather amazing that I could not find a single how-to that could detail the steps for using the Apache web server to load balance multiple Tomcat 5.5 instances on Windows. I found tons of articles- but they were either for older versions of Tomcat or Apache or for Unix flavors and none were accurate enough. Apache itself has fragmented documentation on the subject, and after much trial and error and pulling of hair, I succeeded.

The least I can do is list the steps here for all others who have had trouble getting it right.
The goal of this blog is to set up two Tomcat 5.5 instances with Apache in front, load balancing them. Please note that this does not cover clustering. I'll save that for another blog once I actually do it :-)
Also, this is not a tutorial on load balancing. It will get you up and running- then you might want to look at the Tomcat documentation to find out what exactly all those options do.

So, this is what you need to download:
1. Apache HTTP server 2.2.4 from The Apache HTTP Server Project. I prefer the zip over the Windows Installer. You are free to choose the Windows Installer if you wish, but modify the service instructions below accordingly.

2. Apache Tomcat 5.5.20 from Apache Tomcat downloads

3. Mod JK Tomcat connector from here. Please note: You want to download the binary- click on JK 1.2 Binary Releases --> win32 --> jk-1.2.21 --> mod_jk-apache-2.2.4.so

Now let's start by installing Tomcat first.

1. Extract the Tomcat zip. Hereafter, the directory you extracted to will be referred to as TOMCAT_HOME

2. Test Tomcat to see that it works. Go to TOMCAT_HOME\bin and run startup.bat
You may need to add an environment variable called CATALINA_HOME which is set to TOMCAT_HOME in case Tomcat fails to start.

3. Open up your browser and access http://localhost:8080/
If you see the default page, then Tomcat Instance 1 is working fine. Shut down Tomcat.

4. Now set this up as a service. Go to your command prompt, change directory to TOMCAT_HOME\bin and set an environment variable called CATALINA_BASE. The value of this variable should again, be TOMCAT_HOME. For example:

SET CATALINA_BASE=C:\Tomcat-5.5.20

Then, execute the following: service install Tomcat5
This will set up the Windows service for Tomcat.

5. Go to your Services and see that Tomcat 5 is listed. Start the service, and then verify again that http://localhost:8080 is up.
That's all for the first Tomcat instance. Now for the second.

1. Make a directory called SecondInstance in TOMCAT_HOME

2. Copy the conf, logs, shared, temp, webapps and work directories from the TOMCAT_HOME directory into the SecondInstance directory.

3. Open up SecondInstance\conf\server.xml in a text editor. We've got to change the port numbers so that they don't conflict with the first instance.
I just incremented by 10 and changed them as follows, but you could use other port numbers:


<Server port="8005" shutdown="SHUTDOWN"> to <Server port="8015" shutdown="SHUTDOWN">

<Connector port="8080" maxHttpHeaderSize="8192"... to     <Connector port="8090" maxHttpHeaderSize="8192"...

<Connector port="8009" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> to <Connector port="8019" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />

Change the SSL port if you need it as well.

4. Now set up the second instance to run as a service. Go to the command prompt, change to the TOMCAT_HOME\bin directory. Set the CATALINA_BASE environment variable to TOMCAT_HOME\SecondInstance. For example:

SET CATALINA_BASE=C:\Tomcat5.5.20\SecondInstance
Then, type:
service install SecondTomcat

Go to Services and start the SecondTomcat service. Test it out by pointing your browser to http://localhost:8090/
Works ok? Great! Your second tomcat instance is now ready to be used.

Next, let's set up the Apache HTTP Server. It's pretty simple...

1. Run the installer you downloaded. The standard install will do.

2. Open the Apache Server Monitor and start the web server if it's not already running.

3. Point your browser to http://localhost/ to verify that Apache is running on port 80.

4. Stop Apache.

Finally, we reach mod JK. Let's set it up first just to delegate requests to the two Tomcat instances, and we'll load balance it a bit later.

1. Copy the mod_jk-apache-2.2.4.so to the modules directory in your Apache installation.

2. Rename it to mod_jk.so

3. Open up httpd.conf in the conf directory of your Apache installation in a text edit, and add the following line at the end of the set of LoadModule statements:
LoadModule jk_module modules/mod_jk.so

4. Create a file called workers.properies in the conf directory. Add these lines to it:

workers.tomcat_home=C:/tomcat-5.5.20

workers.java_home=C:/jdk1.5.0_03

worker.list=worker1,worker2

worker.worker1.port=8009
worker.worker1.host=localhost
worker.worker1.type=ajp13

worker.worker2.port=8019
worker.worker2.host=localhost
worker.worker2.type=ajp13

This file defines which workers Apache can delegate to. We've listed worker1 and worker 2 to correspond to our two tomcat instances. Remember to set tomcat_home and java_home as well.

5. Specify the worker properties in httpd.conf:

Add these lines just after the LoadModule definitions-

# Path to workers.properties
JkWorkersFile c:/apache2.2/conf/workers.properties

# Path to jk logs
JkLogFile c:/apache2.2/mod_jk.log

# Jk log level [debug/error/info]
JkLogLevel info

# Jk log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

# JkOptions for forwarding
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat set the request format
JkRequestLogFormat "%w %V %T"

JkMount /jsp-examples worker1
JkMount /jsp-examples/* worker1

JkMount /tomcat-docs worker2
JkMount /tomcat-docs/* worker2

Defining these tells Apache where to look for definitions of its workers and tells it that any requests for the jsp-examples context should be handed off to the Tomcat instance represented by worker 1, and any requests for tomcat-docs context should be handed off to Tomcat Instance 2, represented by worker 2.

5.5 [Added on April 11 2007]
Edit the server.xml for Tomcat and Tomcat's SecondInstance and add a jvmRoute attribute to the Engine element:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker1">
for the first instance and
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker2">
for the second.

6. Start Tomcat Instance 1 and 2. Start up the Apache webserver. Point your browser to http://localhost/jsp-examples/ and then to http://localhost/tomcat-docs. You should see the respective pages load. To distinguish which Tomcat is serving you the page, the easiest thing to do is edit the index page in the tomcat-docs and jsp-examples of Tomcat 2 and change the title for example. Then you can verify that tomcat-docs is being served only by the second instance.

Thats it!! Apache is now delegating requests to both Tomcats.
Now for our last task- we will load balance it so that Apache distributes load for jsp-examples between both instances of Tomcat. It also serves as a failover mechanism. If Tomcat 1 is down for whatever reason, Apache will automatically keep delegating to Tomcat 2 so your application remains accessible.

Load balancing is a simple configuration. First shut down your Tomcat instances and Apache as well.

1. Open workers.properties in a text editor.

2. Edit it so it looks like this (changed lines in bold)-

workers.tomcat_home=C:/tomcat-5.5.20
workers.java_home=C:/jdk1.5.0_03

#worker.list=worker1,worker2
worker.list=balancer

worker.worker1.port=8009
worker.worker1.host=localhost
worker.worker1.type=ajp13
worker.worker1.lbfactor=1

worker.worker2.port=8019
worker.worker2.host=localhost
worker.worker2.type=ajp13
worker.worker2.lbfactor=1

worker.balancer.type=lb
worker.balancer.balance_workers=worker1,worker2
worker.balancer.method=B

We've changed the worker list to a single worker called balancer, and specified that the worker type of balancer is 'lb' or load balancer. The workers it manages are worker1 and worker2 (these do not need to appear in the workers list). And finally, we set the balance method to 'B' or balance by busy factor. Apache will delegate the next request to the Tomcat instance which is least busy. Please note that there are a couple of options for method- consult the Apache/Tomcat documentation which lists out options for workers properties to help you decide the best method for your type of application.

3. Open httpd.conf and comment out the previous JkMount directives. Replace them with these:

JkMount /jsp-examples balancer
JkMount /jsp-examples/* balancer

Very simple- we've just pointed Apache to a single worker- the balancer.

4. Start up both Tomcats and Apache. Access http://localhost/jsp-examples
You will either be served by Tomcat 1 or Tomcat 2. To prove that both are capable of serving, shut down the first instance and refresh your browser. You should be served by instance two.

Congratulations! You've successfully set up Apache load balancing multiple tomcat instances!

Help! My page loads twice...

2007-03-07T14:07:00.000+05:30

I spent all morning looking at a jsp which was taking too long to load. It executed a query and then displayed results.
First, I assumed it was the query. So I timed it but found that it took 1/4th of the entire page load time.
Then, I put in timers in the jsp around certain blocks of code and watched my Tomcat console. To my surprise, I saw the timers printed twice on the console- one set, then the next.
So apparently, my page was reloading. I checked all pages that linked to this... put in alerts and everything possible to see which was trying to load the page a second time (this page is in an iframe...so the possibility does exist)- but all pages seemed clean.

Then, I decided to eliminate all jsp code and js as well. It still loaded twice. Finally, I had no option but to remove html bit by bit till I find the section forcing the reload. Believe me, that was a painful process.

It came down to a cell in a table. The contents of the cell were an anchor with content as an image. Remove the image...and voila! the page loaded only once.
Now why would an image cause a page to reload?

Here is where I demonstrate my stupidity:

We basically have a whole lot of thumbnail images displayed on that page. So we have image tags like

<img id="theId" src="dummyImg.jpg" />

etc.
Now, I am dynamically assigning the src via JavaScript. So I thought, why let all the dummyImg.jpg's load if I'm going to be replacing them anyway. I removed the src... yes I actually did this-

<img id="theId" src="" />

And that was my folly. When I looked at the Page Info in my browser, the media list contained this image, with address http://www.xyz.com/theBadPage.jsp
It just used a relative path and appended nothing because the src was empty. And obviously, it tried to fetch the image, which in this case was the page itself, and hence the reload.

That is why it took so long... re-executed the query, did whatever calculations and assigments all over again.... just because of the empty src.
Now I have set it to a 1 pixel transparent gif... and have learnt a jolly good lesson...

Java String performance

2007-03-01T17:00:00.000+05:30

We all know that misuse of Strings degrade performance. But more often than not, we go back to look at our String code only when we are asked to optimize performance. This is a short blog on the performance aspect of Strings which would be good to keep in mind when coding.

So, the first thing to remember is that Strings are immutable- once you create them, you can't change them. Oh yes, you might think you're modifying them, but in reality, a new String is generated. Warning bells should be ringing in your head by now :-)

If you are concatenating Strings over and over, lets say in a loop, then you know that because they are immutable, new Strings keep getting generated. The javac compiler internally uses a StringBuffer to do this- so for example, you have

 String itemList = "";

 itemList=itemList + items[i].description;

in a loop.

What happens is, within the loop, two objects are generated. One is a StringBuffer-

itemList=new StringBuffer().append(itemList).
          append(items[i].description).toString();

The other is the String that gets assigned to itemList via the toString().

A much better way out would be to create a StringBuffer outside the loop, and just append to it within the loop. More efficient.

Also beware of certain String operations like substring, trim or replace that actually work by creating new Strings. Consider using equivalent StringBuffer methods instead.

Another thing we do not think about is comparing Strings.
Comparing a String via equals() can be fairly expensive. It basically involves a character-by-character comparison, though every effort is made to avoid it. For example, if the lengths do not match, the character by character comparison is not done. But there are many cases where lengths are same and strings are long... in these cases, equals() can be time consuming.
Now we know that comparing Strings with == will not solve our problem because == compares the pointer references, and since both Strings are different instances, you will get a false, even if characters match.

Fortunately, there is a way out, if you know about the intern() method. According to the JavaDoc, intern() returns a canonical representation for the string object. A pool of strings, initially empty, is maintained privately by the class String. When the intern method is invoked, if the pool already contains a string equal to this String object as determined by the equals(Object) method, then the string from the pool is returned. Otherwise, this String object is added to the pool and a reference to this String object is returned.

What this means is that if you have two identical Strings and intern them, only one instance is maintained. So then you can safely use == and have it compare the references and hence return a true.

Now the compiler automatically interns String constants for you. But you could explicitly intern() other Strings that will be compared extensively and repetitively, for example.

As with everything else in life, make sure that if you are interning, do it responsibly. If you intern every String in sight, you are in deep trouble. A pool of Strings will have maintenance costs and you cannot rely on interned Strings getting garbage collected. Hence, if you use it rashly, your performance will actually degrade and you will possibly run into memory issues.
It is worth knowing about intern though- after tuning other areas of your code, you could consider intern especially if you know for a fact that a particular operation will be more efficient by interning the String in question.

Zimoz.com

2007-02-28T15:26:00.000+05:30

(i should state upfront that Luanne is part of the zimoz product team)

I've been hearing a lot about this new site http://www.zimoz.com/ (reasons - above ;) ) Zimoz.com is a way to create and communicate your feelings, thoughts, pictures or video with a postcard. It fits neatly into the Web 2.0 buzz that you hear of nowadays. The user interface is very easy to use without the clutter of options that you would rarely use. I've seen it evolve over the last few months and it seems to have somewhat stabilized in its current state. The zimoz team has been very responsive to user feedback which shows in its latest version. It's still in beta though. I'm looking forward to when it goes fully live. I won't go into another review here (here's a review you can read though). What really gets me excited about zimoz is the amount of technology that has gone into the product. They use ajax a lot and you can tell that a lot of effort has gone into the UI for creating postcards. Speaking to Luanne I hear that there are a number of new features planned and they all sound neat. It could really change the way you look at the web today. With this movement towards web 2.0, where the community drives the content and more control is with the user, zimoz plays its part and plays it well. I can see a myriad sites that could do with the 'zimoz way' and I'm sure the community will come up with creative ways to exploit all this wonderful technology. I'm impressed. Way to go, guys!!

Check my own postcard at http://www.zimoz.com/zimo/user/aldrin/1172602091492.postcard

Cheers,
Aldrin

IE and Firefox- Scroll Offsets

2007-02-27T09:49:00.000+05:30

Many times, one needs to know scroll positions- how much the user has scrolled in a web page. These are particularly useful to display a dialog for example, in the center of the visible space of the browser.
As usual, there are different ways to get this figure in IE and Firefox.

In Firefox, you would use the window.pageYOffset or window.pageXOffset to get these positions. In IE, you would use document.documentElement.scrollTop or document.documentElement.scrollLeft.

So, to handle both browsers, a simple function to do this would be:



function getYOffset() {
    var pageY;
    if(typeof(window.pageYOffset)=='number') {
       pageY=window.pageYOffset;
    }
    else {
       pageY=document.documentElement.scrollTop;
    }
    return pageY;
}

The SOA hype

2007-02-27T09:43:00.000+05:30

We were rather surprised at the response we received for the SOAD workshop.

Most of the participants came in awe of SOAD, hoping to hear the hype and the promise that SOAD would turn their projects into raging successes overnight.

Instead, they heard that services are nothing new- they've been around for ages. And that good designers have already been practising it implicitly.

Using services does not mean that all your project problems will be solved instantly- everything boils down to good analysis and design skills. Everybody wants to hear that this is some great new technology and that OOAD is old fashioned and should be shelved. Many were not willing to accept that you need good OOAD skills first. Only after this will you be able to grasp and apply correctly service oriented analysis and design to produce more flexible software.

Which just goes to show- how many people understand concepts and basic principles- and how many want a quick fix solution and an excuse to use the latest buzzwords...

Service oriented analysis and design workshop

2007-02-26T09:10:00.000+05:30

Unfortunately, I haven't been able to post anything for the past couple of days. I was part of a workshop on Service Oriented Analysis and Design conducted by Experience Commerce in Mumbai.
More on this and others later... keep reading!

Attaching JavaScript functions to the Window OnLoad event

2007-02-21T13:27:00.000+05:30

In an earlier post, I had griped about IE but forgot to mention details about the actual problem. The issue was with the way we were using the html web page's onload event. We were attempting to do something like this:


<body onload="someFunk()">
Testing the onload function

...<br />

 <body onload="anotherFunk()">
   You shouldn't be nesting another body tag out here !
 </body>

</body>

Yes, exactly, two body onload functions! I'll come to 'why' later. The above code doesn't work - only the outer BODY's onload event is really triggered (someFunk only is called)

There is in fact a better way to attach the second function (and any subsequent functions) to the onload event. A quick utility function for this can be found at http://www.dustindiaz.com/top-ten-javascript . This function simply appends your function to the existing list (meta function ?). Try this code:


<script type="text/javascript">
function addLoadEvent(func) {
   var oldonload = window.onload;
   if (typeof window.onload != 'function') {
       window.onload = func;
   }
   else {
       window.onload = function() {
           oldonload();
           func();
       }
   }
}
</script>

<body onload="someFunk()">
Testing the onload function

...<br />

   This is better

<script type="text/javascript">
 addLoadEvent(anotherFunk);
</script>

</body>

Neat, elegant solution and works consistently in IE and Firefox.

Now coming back to the earlier question of 'Why?'. Why did we land up with a nested body tag like that? Well we are using this framework that allows us to define a common template for a group of pages. Now some of these pages needed to trigger a javascript function on body load. This was conveniently achieved by adding a <body onload="..."> to these pages. This worked fine as long as the templates themselves did not have a <body onload="...">! The moment a template defined a body onload event, the content page body onload, if present, was ignored. Such situations happen more often than not with templates being written by one person and other content pages being developed by various other people. There was already a way to do this in our code base but we didn't spot it immediately. It took a while to figure out and it was interesting investigating this. Well, everyday you learn something new (and my writing has been deteriorating !! :)

The Indian Software Factory

2007-02-18T12:57:00.000+05:30

I have been having this feeling for quite a while now, that the Great Indian Software Industry is little more than wool over our bleary eyes. At first I thought it was just me. But of late, I have been meeting people who feel the same. And finally today, to confirm my suspicions, was an article published in the Times of India entitled 'A myth called the Indian Programmer'. The very industry we claimed was THE place to be, is now a monster. To put it a bit harshly, we are no more than slaves in a sweatshop. We are no more than robots with a $ per day tag stamped onto our foreheads. We are back in the dark ages where producing more and more lines of code in the shortest possible time, earning the maximum amount of money is the very cornerstone of our lives.
Who has time to think anymore? About how we do our work? Are we doing it the right way? Are we even thinking about doing it the right way? Of course not- the powers higher up do not pay you to think. Why would they- your thinking time has cost them money. The bottomline is- how much work can you pack into your day. And how much money can it earn. Not enough? Then work nights of course!!! Do whatever it takes.

We have increasingly become a crazier lot. Let's present the client with this impossible estimate and impossible schedule. And then do whatever it takes to meet it. At the cost of quality and our intelligence. The clients, poor suckers, are overjoyed. Little do they know that the bright Indian Programmers are toiling day and night to fulfil their every whim and fancy and that there is a limit as to how much ones brain can take before producing complete cr*p because they are too exhausted to do anything else. They'll either scrap the project sooner or later or spend a fortune maintaining it. Yeah!!!! More work outsourced to us.

You probably think I am a big cynic- and I don't deny it. I call it as I see it. And this is by no means a take on all the programmers out here in India. I personally know excellent ones who are simply boxed in by the industry. I know how it feels. I have been there. Some months back, I am ashamed to admit, I too stopped thinking. Completely. I rushed all over the place to complete work. Just do it. Don't think. And I knew I would regret that. I did. My failure to think was a a personal failure. A failure to my team by taking rash decisions and making poor choices, a failure to the work I loved, a failure to produce good quality software that I could be proud of. And so I quit. Till today, many are concerned about my future. Even bewildered at what I did. I had it all and threw it away. But I do not regret it. My mind is no longer enslaved. I can go back to doing the work I love and not insult it by churning out rubbish everyday.

I frankly do not see it getting better. Most (please note, I say most- there are most certainly exceptions) companies now work on this model. A pity.
To those who say well, this earns your bread and butter, I do not dispute it. To those to say this work enables you to have a grand lifestyle, again I do not dispute it. Everyone has different priorities. Everyone enjoys their work in different ways. But please, let us stop boasting about the great software industry. We don't do that much at all. To those excellent engineers out there- you know who you are- I know it is hard, and I know what conditions you face, but try never to give up on thinking...don't let our beloved field of work down...

Cynically yours,
Luanne

PS: Do read this very long, but interesting blog here

Here's to Firefox

2007-02-16T17:09:00.000+05:30

Firefox is my default browser by choice. Features that Internet Explorer (IE) has only recently have been present in Firefox for a long time. Which makes me wonder - Why do people still use IE?!! Lets take a poll here and hear your preference. Could you please take a moment to leave a comment to this post and mention the browser you prefer to use? Thanks!
The fact that Firefox scores over IE was reinforced again today, when we were facing a confounding problem with one of our web pages. There was a menu on this page that just refused to show up. By the way, we were required to test only on IE for this particular application. When I decided to investigate this problem, by default I fired up Firefox to check it out and surprise, surprise - the menu worked just fine! That proved one thing - that the HTML being generated from the server was all right. Next possibility - some Javsascript or CSS was failing badly. This is when I began to tear my hair out (whatever little is left of it). There was simply no straightforward way to debug this in IE. Debugging was and is an extremely painful and laborious process in IE, involving scrolling through lines of HTML code picking up suspicious sections. Contrast this with Firefox, which has these 'add-ons' where you can inspect the HTML source, see the CSS styles, debug the JavaScript, enable/disable styles and a hundred other nifty, useful thingamajigs - all designed to help web developers nail down problems. Highly recommended for debugging HTML, CSS, Javascript are the DOM Inspector, Firebug and Web Developer add-ons. It high time IE develops features like these. It's not that IE is such a bad browser (IE 7 with it's tabbed windows is pretty decent), but the thoughtful little customizable things that Firefox provides makes it a far more pleasant experience. And shouldn't the internet (and software engineering) be all about the experience ? :)

Cheers,
Al

Defend thyself!

2007-02-15T16:41:00.000+05:30

I have learnt many invaluable lessons from my friend and top notch software engineer Jock Murphy. One of them was to code defensively. If there is the remote possibility that something can be null, be prepared to handle it. Although I have never forgotten that lesson, I fail to implement it many times. The past couple of days have been particularly horrific. I've had NullPointerExceptions laughing their heads off at me. And at those moments, I remember Jock and curse myself.

I also remember the yellow bit of paper on which I wrote down his programming practices as he told them to me and a colleague about a couple of years back.
I still have that paper- and I am sure that he will not mind if I share the contents with you for the good of software-kind.

1. JavaDoc- Public and protected methods and properties should have JavaDoc. Things that you should put down is the purpose (what it does or what it is- not how it does it), inputs, outputs and exceptions. I have noticed that usually, JavaDoc falls into three categories- none, wrong, or excessive. I don't believe in adding JavaDoc for regular getters and setters, unless your getter or setter does more than meets the eye. In that case, one should consider why it does that in the first place. Otherwise, document it.

2. Magic Numbers- You should never, ever have a magic number in your code. For example,
int something=somethingElse * 329;

What the heck is 329? It should be:

int something=somethingElse * Constants.NUMBER_OF_VIEWS;

You could define that constant externally in an interface, or locally in the class if it is the only class that will ever use it. Just imagine if 23 classes use 329- and then someone tells you it should be 330. Instead of abusing the guy and doing a search and replace, just go change the constant.

Imagine if you did do a search and replace and class A had the line:
int foo= bar + 329;

329 here means something completely different. You have then shot yourself in the foot. Define another constant!!

3. Logging- Logging is your friend. Log exceptions, unexpected conditions, logic failures

4. Exception Handling- Sure it's nice to define our own custom exceptions. But define and use them wisely. Don't throw exceptions all over the place- they are expensive. When you encounter an exception, log it and then handle it, or pass it on. If you don't require to have an exception thrown, don't throw one. Usually, many custom exceptions can be handled in other gentle ways.

5. Defensive Programming- As mentioned above, be prepared to handle any bad data. And assume as little as possible.

It's not rocket science- there are no buzz words or impressive rules... just plain basics...but believe me, if you follow these, you will be much happier...

Thank you Jock...I am a better engineer today :-)

-Luanne

Welcome!

2007-02-15T15:27:00.000+05:30

Welcome to our tech blog!

Down the years [ok, we're not THAT old but makes this sound impressive ;-) ] we've formed a lot of opinions on goings on in the tech world. Experiences, learnings, mistakes, HUGE mistakes :-)

For those of you who don't know us (shame on you :-D), we've both been in the software field for as long as we can remember. And this is frequently our after-hours topic of discussion. Which in effect means, we do not have 'after-hours'!!
We both independently thought 'why not write them down for posterity'...and when we mentioned this idea, each of us said 'hey even I was thinking of doing that!'

So here it is- our thoughts and feelings and passionate debates on the only thing worth talking about in our lives- technology!

We hope you will like it, and read it frequently... bookmark it NOW!!! (shameless promotion)
Please tell your friends about it (more shameless promotion), and praise us to the skies (the ultimate ;-))

Do feel free to comment, disagree, argue, offer ideas and your own experience...[we can always delete the comments we don't agree with........joking ;-)]

Here's to technology and our thoughts about it...cheers!!! (hic hic)

PS: While we would prefer you to leave comments on the blog itself to ensure our everlasting fame, you can also email us at thought dot bytes at gmail dot com