$thought | blog

iCal attachments and emailing woes

2014-03-17T00:00:00+00:00

Calendar notifications are becoming more common in collaborative applications. Scheduling systems are easy to build but difficult to integrate as there are large number of clients and service providers. Our product went through a painful time regarding integrating with various email clients and online services regarding iCalendar integration.

This post is going to make sure, I document them for anybody who is facing similar problems. If you have any questions regarding specifics, I will be happy to respond on a stackoverflow question or comment below.

I am not going to create a HOW-TO list of articles here - but document things I bumped against. So if you’re looking for a detailed tutorial, you’ll need to research on your own.

Brief about our solution

Our application is built on PHP 5.4 + Symfony 1.4. We are using iCalcreator library for the generation of iCal files. In my research, this is the most comprehensive standard compliant library out there for PHP. It supports RFC2445 and RFC5545. If you’re on PHP platform and looking for iCal generating library, you’re search ends here.

Our calendar system generated events between only two parties; organiser and attendee. So the event generation is pretty straightforward. We generate iCal files and just dispatch as attachment. Fairly simple. Still I managed to bump against couple of nasty issues on the way.

Issue 1 - Email clients won’t welcome us

iCal files are creates a tricky issue for client compatibility. You can respond to users of your website depending upon their environment. You can define custom CSS classes, generate responses based on browser, location etc. When you’re dealing with iCal files, you don’t get to do any of that. Your file is attached and you let the chips fall wherever they might be. So there is no alternative to extensive testing with various browsers, email clients and email service providers. Our focus was on -

Microsoft Exchange, Outlook (Desktop), Outlook Web Access (OWA) with version 2007, 2010, 2013
Thunderbird with iCal plugin
Google Mail
Yahoo Mail
Outlook.com

These clients provided a broad spectrum of variation which gives us enough comfort in our solution.

We wanted email clients to recognise calendar emails and show action buttons to respond to the event. Usually you’ll see “Yes”, “Maybe” or “No” buttons with additional context information in a special bar by these email clients.

Our emails were not able to achieve this feature. We investigated a lot but didn’t happen. My investigations told me you’ll need to do few things while attaching iCal file to email before clients will recognise it. These were -

Email body header should be “multipart/alternative”
Optionally attach the ics file as well.
Attach iCal file to email with following specifications -
- Encoding base64 or 7-bit
- Content type “text/calendar”
- Header parameter “method=REQUEST”

For Swift mailer sample code might be like this -

$messageObject = Swift_Message::newInstance();
$messageObject->setContentType("multipart/alternative");
$messageObject->addPart("Email message body goes here", "text/html");

$messageObject->setSubject("Subject line goes here")
  ->setFrom("Sachin Dharmapurikar");

$messageObject->setTo(array('johnsmith@example.com'));
$ics_content = file_get_contents("/tmp/sample.ics");
$ics_attachment = Swift_Attachment::newInstance()
  ->setBody(trim($ics_content))
  ->setEncoder(Swift_Encoding::get7BitEncoding());
$headers = $ics_attachment->getHeaders();
$content_type_header = $headers->get("Content-Type");
$content_type_header->setValue("text/calendar");
$content_type_header->setParameters(array(
  'charset' => 'UTF-8',
  'method' => 'REQUEST'
));
$headers->remove('Content-Disposition');
$messageObject->attach($ics_attachment);

$mailObject = Swift_Mailer::newInstance($transportObject);
$mailObject->send($messageObject);

If you look carefully, the message is marked with headers appropriately and the ical attachment is inline rather than as file.

If you look at the source of email generated with above type of code -

Delivered-To: johnsmith@example.com
Received: by 10.220.78.135 with SMTP id l7csp244293vck;
        Thu, 13 Mar 2014 04:04:02 -0700 (PDT)
X-Received: by 10.224.151.130 with SMTP id c2mr1291334qaw.67.1394708642049;
        Thu, 13 Mar 2014 04:04:02 -0700 (PDT)
Return-Path: <00000144bb1cf5e9-fd06-4945-b4fc-386af484b339-000000@amazonses.com>
Received: from a8-35.smtp-out.amazonses.com (a8-35.smtp-out.amazonses.com. [54.240.8.35])
        by mx.google.com with ESMTP id f6si1004430qap.152.2014.03.13.04.04.01
        for <johnsmith@example.com>;
        Thu, 13 Mar 2014 04:04:02 -0700 (PDT)
Received-SPF: pass (google.com: domain of 00000144bb1cf5e9-fd06-4945-b4fc-386af484b339-000000@amazonses.com designates 54.240.8.35 as permitted sender) client-ip=54.240.8.35;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of 00000144bb1cf5e9-fd06-4945-b4fc-386af484b339-000000@amazonses.com designates 54.240.8.35 as permitted sender) smtp.mail=00000144bb1cf5e9fd06-4945-b4fc-386af484b339-000000@amazonses.com
Return-Path: 00000144bb1cf5e9fd06-4945-b4fc-386af484b339-000000@amazonses.com
Message-ID: <00000144bb1cf5e9fd06-4945-b4fc-386af484b339-000000@email.amazonses.com>
Date: Thu, 13 Mar 2014 11:04:01 +0000
Subject: Calendar appointment
From: Support <noreply@serviceprovider.com>
To: johnsmith@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="_=_swift_v4_1394708669_02698d0808f167b864966fe96816eec3_=_"
X-SES-Outgoing: 2014.03.13-54.240.8.35


--_=_swift_v4_1394708669_02698d0808f167b864966fe96816eec3_=_
Content-Type: multipart/alternative;
 boundary="_=_swift_v4_1394708669_9fe42676490416322d81ca192ce8c5f7_=_"


--_=_swift_v4_1394708669_9fe42676490416322d81ca192ce8c5f7_=_
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<div>Hello World!</div>

--_=_swift_v4_1394708669_9fe42676490416322d81ca192ce8c5f7_=_--


--_=_swift_v4_1394708669_02698d0808f167b864966fe96816eec3_=_
Content-Type: text/calendar; charset=UTF-8; method=REQUEST
Content-Transfer-Encoding: 7bit

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Service Provider Inc//NONSGML kigkonsult.se iCalcreator 2.18//
CALSCALE:GREGORIAN
METHOD:REQUEST
X-WR-TIMEZONE:America/New_York
BEGIN:VEVENT
UID:20140313T070425EDT-3998irwScV@Service Provider Inc
DTSTAMP:20140313T110425Z
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE
 ;CN=John Smith:MAILTO:johnsmith@example.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=
 TRUE;CN=Jane Smith:MAILTO:janesmith@example.com
DESCRIPTION:
DTSTART:20140316T103000Z
DTEND:20140316T110000Z
LOCATION:Meeting venue
ORGANIZER:MAILTO:johnsmith@example.com
SEQUENCE:0
SUMMARY:Meeting about discussion of future
END:VEVENT
END:VCALENDAR

--_=_swift_v4_1394708669_02698d0808f167b864966fe96816eec3_=_--

This works with all the popular email clients. This was the first problem we encountered.

Issue 2 - Despite of writing above code, email clients still didn’t respect us

Now thats a bummer. I spent lot of time and made sure that all the required parameters are satisfied but still no luck with showing calendar invitations properly. Spent lot of time reading original messages sent to the users and figured out that the email headers and everything is stripped out by our vendor.

Our email sending API vendor used to parse message and headers, and convert them to generic message for god knows reasons. Then I decided to switch email sending API from current vendor to Amazon SES. SES allows you to send raw messages which makes life extremely easy. That solved our problem completely.

So if you’re using above code and your email headers are not appearing correctly, try a different way to send your email. It might work for you.

Most of the modern email service providers allow you read the source of the email received. Keep checking that for the headers. That is incredebly helpful.

Issue 3 - Occasionly our users received error in Outlook

Some of outlook users started receiving “not supported calendar attachment.ics” attachment with the emails. This was weird as nobody else receive this error.

Again, I started investigating this issue and figured out that there is some issue with RRULE directive in Outlook if the iCal file is created from specific clients e.g. Lotus Notes. Our case didn’t have that issue as we don’t deal with RRULE (recurring appointments) or using Lotus Notes to generate iCal files.

Later I figured out that due to a bug in our code, we were occasionly sending 0 byte attachments. This was treated like no attachment in modern email clients but Outlook gave this error message to the users.

Issue 4 - Outlook used to show 3-4 hour differences in meeting timings

This one was the most weird issue I faced. Google, iCal on mac or Yahoo used to show correct meeting timings but Outlook used to show timings with 3-4 hour time difference. This one was a huge problem.

The reason for this problem lies in formatting the dtstart and dtend attributes of vevent.

There is a good example here. You’ll need to format time in the appropriate timezone. You’ll also need to specify X-WR-TIMEZONE property in the vcalendar section of the file.

Symfony Worker Tasks

2012-09-29T00:00:00+00:00

Every large application needs background worker tasks for sending emails, cleaning up files or archiving content. We are using symfony 1.4 for our current project. In symfony, this can be achieved by creating tasks. Tasks can be created by generator. You can trigger these using cron, hooked with a queuing system or invoked manually.

In my experience, writing a symfony task in general is very easy and fun. When you want to run a task for long time, that’s tricky! We created a task to send emails asynchronously. Our solution involves three components.

Web Application this is where we decide to send a notification email. Create email text and then save it in a persistent store. Notify access identifier to RabbitMQ to dispatch using worker.
sfRabbit this plugin allows to create consumer class, invoked by a symfony task. Each event consumer has a callback. When qualifying event is received, execute method of the callback is invoked.
Email Dispatcher we use Amazon SES for sending emails. We wrote a dispatcher class which handles the responsibility of fetching the content from persistent storage and then dispatch email.

Looks like a simple process which shouldn’t be difficult to implement. At least I thought so.

I quickly created a database backed symfony task. It will use doctrine to connect to MySQL and fetch email information, dispatch emails and I can go have a beer. Not so easily.

Issue 1

After some time, I saw an error message in log file -

MySQL server has gone away

So this means, there was some connection idle for more than wait_time configured in MySQL. In my case it was set to 28800 (8 hours). This was long enough duration to hit at least one query. It didn’t happen over the night and the task stopped sending emails. Snippet from my task code -

protected function execute($arguments = array(), $options = array())

$databaseManager = new sfDatabaseManager($this->configuration);

$connection = $databaseManager->getDatabase($options['connection'])->getConnection();

// more code with database goes here

}

When you instantiate sfDatabaseManager in symfony task, it will create a connection using provided configuration. This connection will be killed if it idles for wait_time. If you kept issuing ping queries, then you don’t have that issue. Upon using the connection to query database wait_time will keep resetting to 0 and you’ll reuse the connection again and again.

This is exactly what happens when symfony is processing a web request. Only 1 connection is opened and that is reused throughout the lifecycle of the request. At the end of the request, the connection is closed and that’s how no memory leaks or connection leaks happen.

To tackle this connection getting killed, I just started creating sfDatabaseManager every time I received a message. Just like the symfony web request procedure.

$databaseManager = new sfDatabaseManager($this-&gt;configuration);
// Use database to do operations
$databaseMangare->shutdown();

$databaseManager->shutdown() is actually supposed to close all the active connections and then shut itself down. Sounds reasonable and worked for me until I hit second issue.

Issue# 2

That trick worked for a while and I bumped across another MySQL error -

Too many connections

I was surprised. Database manager is getting shutdown, I debugged code and I saw that each connection is issued close call. Why there is database connection leak?! I kept investigating.

What I found was very interesting.

Symfony task used sfDatabaseManager to manage database activity. I used doctrine so sfDatabaseManager internally used Doctrine_Manager. Doctrine uses PDO to finally connect to the database server. I found that right way to manage a PDO connection is simple -

$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass);
$dbh = null;

Setting a PDO reference to null will close the connection. Simple enough.

From documentation -

Upon successful connection to the database, an instance of the PDO class is returned to your script. The connection remains active for the lifetime of that PDO object. To close the connection, you need to destroy the object by ensuring that all remaining references to it are deleted -- you do this by assigning NULL to the variable that holds the object. If you don't do this explicitly, PHP will automatically close the connection when your script ends.

My interpretation says, you assign null to the PDO (which symfony code did) and then PDO will close connection immidiately. If you didn’t assign null then connection will be held until script ends. This is not how it happens. If you assign null it doesn’t get freed immediately. When the script ends, that connection will be actually closed. Till that time, MySQL will think of it as an idle connection. Bummer!

I didn’t find any easy way to force close the connection immediately.

How I solved this issue?

After lot of Googling, reading I didn’t find anything which can be used as solution. I had two options:

Increase wait_time – Increasing wait_time to something like 72 hours. No matter what happens, I will send 1 email in 72 hours and that will keep the connection alive. I didn’t prefer this solution as it will just delay the problem instead of fixing it. This will allow me to use my first solution of instantiating sfDatabaseManager only once and keep re-using the connection.
**Reduce wait_time ** – Reduce wait_time and use second solution of creating sfDatabaseManager on each message receipt. This is as bad as first solution. Just different approach so I really resisted this solution as well.

Finally, I found the solution which is not my favorite but doing the trick –

Create another task which will carry out unit work for you e.g. in my case sending 1 email. This task will use database connection and will be good for only 1 email processing.
Make the sfRabbit callback task as lightweight as possible. Just receive message and then invoke the unit worker task using exec() function.

How does it solve the problem?

sfRabbit consumer task will keep running forever, and it will not use any database connections. This allows us not to worry about database connection leaks.

Using exec() function it will invoke the unit worker task which actually initiates the sfDatabaseManager which will create a connection. Unit worker will be destroyed once the task is carried out. As script ends, PDO will close the connection automatically.

Not the most elegant solution but seems like the only option to me.

Programming language and Developer's Perspective

2012-07-15T00:00:00+00:00

In last October, I left ThoughtWorks and joined a startup in Pune. The new company is developing a product for advising health professionals in United States.

The product is a web-application developed in PHP and Symfony framework. Not the cool stack for cool kids (I am referring to Ruby developers here). I never worked on PHP before and believe me, it is not the most awesome language to work when you have worked on Ruby, Java and C#. I hated many things about PHP in the beginning and I still do. But, as a geek at core, slowly I started getting comfortable and found lot of awesome things in it. I can live with it.

I started working with it, and faced huge challenges. We didn't have any tests (despite of PHPUnit). The code wasn't very modularized and object paradigm wasn't followed at all. So I just kept thinking, we should just rewrite entire thing in Ruby and move to a new world. That is when I realized that, I am not going to solve anything by moving to Ruby.

Yes, Ruby is a better language, it will reduce a lot of code and generally Rails will take care of plenty of the best practices without breaking a sweat. I just needed developer's agreement and their comfort level to move to Rails. The team was not comfortable at all. They were comfortable in PHP and not Ruby. It means, you'll be spending time in ramping up team in Ruby and migrating application. Bad way to move. Getting team to sign up for that, it was another challenge. Never forget that even if everybody agreed, its not very difficult to write bad code in ruby and create a royal mess.

So, in the end, we decided to stay in the PHP world, and do the right thing. Clean up the code and make it a better place. We started writing tests using PHPUnit. Used phactory for database factories. Introduced Jenkins as CI server. Started using jasmine for JS unit testing. Selenium was setup for functional tests. Got everybody licenses of PHPStorm :).

While all of this action takes place, we performed merciless refactoring and we deleted thousands of lines of duplicate/unnecessary code. This effort was happening in parallel to regular feature development. After nearly two months of effort, we reached our goal. We had a codebase which sucked less and worked better. Code was easier to manage and extend.

During this time the team learned a lot. Instead of switching to a different tool, we cleaned our own mistakes. That taught entire team to do things differently and in a better way. Now all we needed were few more helping hands like this and we are good to go!

We faced another huge challenge here.

When we put in word to recruit PHP developers in Pune, we got hundreds of resumes. It felt incredibly easy to hire PHP talent in Pune. We were clearly wrong!

We observed 90% of the resumes got rejected simply because those guys were not application/web developers. They were plugin developers for Wordpress, Drupal or Joomla! So that never exposed them to real programming at all!

So I started asking first question to them, what is a web application for you? If they say Drupal, they just got rejected. I might have lost some talent in this process but who cares?! I want a guy who can define an application which is not Drupal!

Then I tried to see if other people (Java, Ruby or .Net developers) want to work for us. Clearly they had their reservations to put PHP on their resume. Why?! Just because they think PHP is not a language which real developers use. I am sorry but thats not how it works!

It doesn't matter what language do you use, what matters is what you did using that language.

Clearly, it was impossible to convey this thought to plenty of real developers.

Finally, we started hunting for talent which was right for us. People who don't give a damn about programming language and focus entirely on programming. They like to solve problems and take pride in code they wrote.

If you belive you're one of those who care about what code you write and it doesn't matter which language, talk to me. My email address is - sachin at dharmapurikar dot in.

Mac OS: netstat and pid

2011-10-26T00:00:00+00:00

I am a Linux user by large. Lately I migrated to Mac as it worked better for me. (...and they look beautiful!) Although, I found from @hyfather that there is significant difference between Mac based utilities vs Linux-based. This difference exists as Linux has GNU coreutils and Mac has BSD based.

Many switches are different, output is different and it is very frustrating to experienced users.

I ran into netstat command recently. I needed to find out pid of memcached server on my mac. Spend few mins but no luck. Mac based netstat command won't understand -p switch. Aaargh! I had to use lsof to get my job done.

Here is how I did it -

sudo lsof -i -P|grep memcache

Worked great!

Ruby: Insecure World Writable Directory

2011-10-15T00:00:00+00:00

I keep getting following warning lately on my mac.

ruby warning: Insecure world writable dir /usr/local/bin, mode 040777

Little research and I found, Ruby warns you about any world writeable directory in your PATH. Not only writeable directories but parents as well. Fix is super quick.

chmod o-w /usr/local/bin

You can replace /usr/local/bin with any directory which ruby complains about. This fixes the issue.

How to display local time to users in browser?

2011-10-15T00:00:00+00:00

In 7 years web programming career, I never met this problem before. In a recent project we needed to show browsers local time on the page. Few folks have solutions to do this server side but none of them work reliably. Why?

Browsers don't send any timezone information
You can send offset in headers / parameters but its only reliable for dynamic requests

After trying a failing multiple times, I resorted to a browser-based JavaScript solution. This works great!

Convert all the server side timestamps to UTC. This makes server side code life way easier!
Send the dates to browser in iso8601 format which JavaScript can parse easily.
I used very good jQuery library (Localtime) to convert iso8601 formatted dates to format required by application.

You can also tag your date spans with a special class. This way you can convert all timestamps to different formats unobtrusively. Very simple and elegant solution.

Save a read-only file in VIM using sudo

2011-09-25T00:00:00+00:00

Have you opened a read-only file and modified it but couldn't save? Not anymore, I found a quick tip which allows VIM users to save the file as 'sudo' even if you didn't use 'sudo' at the time of opening.

Create a shorthand mapping in VIM as follows:

cmap w!! %!sudo tee > /dev/null %

You can save this to your ~/.vimrc file to persist it.

Now every time you want to override the read-only status and save changes as sudo, just hit 'w!!' and you'll be good.

Love the power of VIM!

Getting Things Done

2011-09-18T00:00:00+00:00

It is not easy! I have struggled with achieving higher productivity while working on my personal projects or working from home. I kept asking why? Obvious answers were -

Lots of distractions - @home usually my PS3 is in my sight, there is no pair to pull me back from reading a relatively long blog post (and then catching up on xkcd).
Plenty of things to do - I have plenty of things to do in my new house. So my home tasks list is never-ending. That somehow gets higher priority than my personal project or gets mixed up!

Initially, my reaction to all these things was simple. I need to do better time management! Easy!

Attempt 1 - ToDo lists and Time Tracking

I decided to get a time tracking tool and then plan 10 hours for a weekend strictly for personal work. Seemed very simple and achievable.

I used following set of tools for this -

RescueTime - To track and improve my time spent on computer
Toggl - Timer for tracking time spent on work
Pen and Paper - To make list of tasks and track them.

How did that go?

Well, not great. I was having very high hopes from this simple system. Then I analyzed where did I go wrong?

I made a good list of things to do. This was just a list! I will explain why this is an issue.
RescueTime allowed me to understand how I spent time on computer, and nothing beyond. It improved my productivity by recovering some lost time but not much.
Toggl was a complete failure for this purpose. Often I just forgot to start/stop timers!

Attempt 2 - GTD

This time I decided to go better prepared and with better tools. Yes, I know what you're thinking! You are thinking, that tools can't improve productivity loss and self-control is more important than anything else. I agree, and will add that better tools will help you to get better results. If your will power is strong and you're very disciplined, with better tools you might be able to use your time even more optimally.

This time I decided to go with -

Things - This is a very powerful task management application. It is very simple to use and good at managing your tasks properly. I got inspiration from Akshay Dhavle to use this tool. Expensive but great tool!
Pomodoro - This is a free tool for mac. For folks who want to know more about pomodoro, head over here.

How did it go?

Well I have tried this for past 3 weeks and great so far! I will keep updating about this in coming days. I achieved most of the things I planned without serious lag and I am getting better at it.

So what improved this time?

As I mentioned earlier that pen and paper to-do list was great to begin with, but it was just a list! That was the only issue with it. It told me that I have these 100 things in my pipeline and I need to do them. Well yes, but how? How should I pick a certain thing in my list and not other. Depends on how you make your list and I am sure you have a better way to manage your tasks using pen and paper. I wasn't!

Time tracking is not very useful! I tracked time and realized that I wasted time! I knew that already! It didn't push me in the right direction to do more out of my time. Pomodoro is nothing but time tracking, you might say. I felt that it was more powerful technique than just tracking time. You should carry out one task in a pomodoro sprint. So when you're starting a pomodoro sprint, you're committing yourself to finish that task in 25 minutes or less. That is a huge difference in just tracking; I spent 40 minutes on something vs I want to get this done in next 25 minutes!

First I imported (typed) all my tasks on paper into 'Things' application. Added few more tasks which were not listed on paper as well. Later, I organized my tasks in various projects and responsibilities. This built nice context around my tasks. I spent about 10 minutes for around 100 tasks to think about and put dates on them. Many tasks were immediate once. I knew it is impossible to do them. I rearranged my tasks and priorities. No false promises to self!

"Things" really helped me in getting tasks in perspective. Pomodoro really helped me in getting those tasks beyond finish line.

It helped.

Moved from TextMate to MacVim

2011-09-12T00:00:00+00:00

Being a developer, you should be very thoughtful about your tools. You pick programming language, operating system, system shell, font, color-themes and most favorite of all; Text Editor!

I have used many text editors in more than a decade long programming career. As I am working on Ruby on Rails my obvious choice was TextMate. I used it for a while and thought to give RubyMine a spin. Since I was a Java developer for most of my career, I extensively used IntelliJ IDEA. No doubt it is the best IDE for Java developers out there! Eclipse and Netbeans are good but they have no upper hand when it comes to extensive features which IDEA provides. That is not the point of this post so I will skip writing about it.

RubyMine and IntelliJ Idea share most of the key-bindings, a natural bonus for me. I didn't need to learn anything new to get productive in RubyMine. I loved it. Very soon I realized, all the features which made IDEA so great, are not very important in Ruby on Rails! e.g. refactorings, type checking, name changes etc. Since Ruby is dynamic language, type checking is not required. When it comes to refactoring variable names, you can't trust an IDE to make those changes by itself! Properties hidden behind meta-programmed methods are tricky to find and rename accurately. So in short all the great powers which IDEA boasts, are not required.

Did I mention RubyMine is resource heavy? :)

So I went back to TextMate tried it for second run. TextMate is great editor. Period. I enjoyed working in that software a lot. It has almost all the features any developer will need. It got its popularity in Rails community but it is useful for anybody for any text-editing needs.

When it came to searching full project or loading large files, TextMate isn't the king of the land. It often shows you pretty spinning wheel and lets you wait for long times. Sometimes it just refuses to load large files which you accidentally open. e.g log files.

So I thought, let's try VIM. I used VIM for last 10 years as my terminal editor of choice. My usage was very limited and not VIM specific. I could have done all the editing using pico without a doubt. So I wouldn't call myself a power user of VIM for 10 years.

I met a guy on my last project (Selvakumar Natesan) who introduced me to MacVim as Rails IDE. I used it for a while and loved it a lot. Although, I was fairly naïve in terms of its usage. After a while the enthusiasm to use VIM faded away and I was back to TextMate. On my current gig, I met two guys who were using VIM regularly (Priyank Gupta and Carl Leiby). This gave me another reason to start using it, and I did it this time!

I realized that I can't just depend on my pair to drive most power from VIM where I just stare at them and appreciate VIM. I needed to learn it and get used to that style. So I did spend some time reading cheat-sheet (more time in practicing!) and watching few screencasts where people showed few cool tricks.

After spending couple of weeks working daily on VIM, now I am fully productive and don't have any reasons to switch to TextMate or Rubymine anytime soon. I think I found my soul-editor. :)

If you're interested in switching to VIM as well, here are few links which helped me a lot during this transition:

A Starting Guide to VIM from Textmate: This guide will get you rolling pretty quickly on VIM if you are on Mac.
Janus VIM Configuration Bundle: Undoubtfully most comprehensive set of tools for RoR development using VIM. I use non-customized version of this currently.
PLAY BY PLAY: GARY BERNHARDT : Very good screencast if you want to see a pro using VIM in 100 min video. I paid $12 for that and it was totally worth it! There are bunch screencasts available for free as well. Look for them.

On Github you'll find all the help needed to bootstrap to your new development environment.

After learning this, all I can say, "It is easier than you think!"

Hacking with Arduino

2010-12-20T00:00:00+00:00

Recently I started learning about Arduino and its applications. Lately, I started coming across really cool projects done with this small wonder. Few interesting once are -

Secret knock detecting door (link)
High speed photography using Arduino (link)
Arduino project>

I didn't know that micro-controller programming is this fun. All these years, I wrote software in higher level languages and thought micro-controllers are for only electronics engineers. I was obviously wrong! Arduino is fully open-source and community supported. Arduino programming happens in C++ and all the software required for programming is available for free from its website. You can develop for Arduino on all popular platforms and its very fun to use. All the Arduino libraries are well documented and it won't take long before you build something cool in the world of physical computing. I have created a simple LCD display library as well. I am still learning my way around this and expecting to do better in coming days. You can find all my Arduino projects code on github. Want to do more hacking in coming days.

Android: Downfall or Prosperity for Google?

2010-11-21T00:00:00+00:00

Android, launched in 2007 by Google is a huge success. Recent sales reports show that Google is beating iOS by 2:1. This is no small feat. When Apple released iPhone, it felt like best thing ever. It was. Android had huge gap to fill before it can stand strong against apple. With huge innovation put in by google and other supporters, android soon became really popular. This success is coming as result of OHA (Open Handset Alliance). Mobile manufacturers and carriers poured in their efforts in customizing, marketing and making android a huge success.

This deal is a win-win for Google and OHA members. Google got their search and ad revenue, OHA members got buzzing hot operating system for practically free. I read Harvard Business Review article about android, I was seriously thinking about future of Android and Google. I am no business expert but I am seriously disappointed by authors narrow point of view on android and its future. Android has its serious issues, but the goldmine of google is not going to get shut-down just because default search engine. Let me step back. Google released Chrome browser. They allow you to change your default search engine to Yahoo, Bing. I wonder how many did that? Ubuntu changed default search engine provide for Firefox to Yahoo! I wonder how much search revenue Yahoo gained? People can switch back to Google if they want to. If you care, you will change it. Android is just not about search engine revenue. If you perceive google wants to just get more search hits, we are making a mistake here. Google makes money by ads in apps, search hits and apps sales. Navigation, Gmail, Google Voice is all done by Google. People want to use these services on iPhone as well if they could.

Fragmentation

Android suffers from fragmentation. OHA members keep maintaining their own version of android, resulting in splitting away for mainstream android. How many Linux distributions exist? 1500. How many are known to most of the users and well maintained? 10. This is same situation, if more forks will happen, away they go from benefits of android. It will be nightmare for application developers to support app on all the devices and operating systems. All OHA members should keep their changes to minimal from mainstream. Easy and fast upgrades to customers. I had android 1.6 on my mobile 6 months after Google released android 2.2! This is plain unacceptable. We shouldn’t forget, one of strengths of android lies in upgradable operating system. Better software is pushed to phones to improve performance or add new exciting features. Gone are the days when people purchased smart-phones and never updated their firmware. If Google keeps innovating on android future versions, customizations done by Baidu and Bing will not impact them much. This will keep user’s interest in using Google’s version of android and not a third-party. In the world of open-source best product always wins. People will always have freedom to replace their firmware if they are not satisfied with what they got.

Hardware Differences

Android has been around for 3 years now. Android has more than 225 handsets so far. Various hardware configurations and processing power. Optimizing android experience for all these devices is a complicated task. You can see the same problem with iOS family as well. Already 1st gen iPod touch can not receive updates of latest versions of iOS. Introducing Nexus One was very good move by Google. Unfortunately that didn’t fly well. Nexus one gave users option to have a phone which is free from customized versions and you can always install latest android community software without any special customization. In coming days, it will be interesting to see how Google does with second version of Nexus. Google should keep introducing successors of nexus on regular intervals. There is huge value in showing customers and OHA members what is value in plain vanilla android version.

Where Google Wins?

Google wins at innovation. That will be always a deciding factor in future on android. Maps, email, contacts, purchases all things tied down to your google id brings tighter integration to phone. This strategy puts android in-game for now, at least. Google makes money from iPhones, Windows phones and android devices. Quality of products will decide future of companies who are manufacturing these phones. iOS is a strong competitor for fair reasons. I have no intentions of declaring a winner here. One thing for sure, android gave a very strong alternate to iPhone and is pushing forward the competition on every milestone. This competition will bring better and better things for consumers no matter who wins.

DNS-323, PS3 and UPnP

2010-07-20T00:00:00+00:00

I recently bought Dlink DNS-323. I am using this in RAID 1 mode to copy my data and archive images. I am also using it as media server (UPnP) for streaming video / audio and image content to PS3. The whole setup is very easy and lot of fun! I copied entire catalog of movies / audio and digital image archive and realized the setup is not working properly. I kept getting error while movie streaming on PS3. "This content cannot be played (800288D8)" I was very frustrated by the error message. Earlier I thought this is happening due to lack of processing power at DNS-323 end. After careful examination that possibility faded. DNS-323 has Marvell 500 Mhz processor, 64 Mb RAM, 1000 Mbps Ethernet connectivity. There are two Samsung Spinpoint 1 TB each HDD placed inside. I wasn't expecting any bottlenecks for couple of users. I was about to return the unit and then I came across this fantastic online resource. I installed Mediatomb as UPnP server and disabled in-built application. This allowed me to have seamless access to my video content without any issues! I will recommend you to install this on your DNS-323 so that you can enjoy secure storage and media server. :) This setup is good for -

Archive of all of your files. No more nightmares of hard-drive failure!
Cheap and abundant storage.
Media serving capabilities at home.
Expose firewall-enabled FTP server over internet.

Don't expect miraculous speeds though. 3-4.5 MBps is average transfer speeds for me. This is enough for me to backup and access stored data. For faster access, I prefer firewire or USB 2.0 drive. Figure out yourself about your requirements. Ciao!

Rewards of "Frequent Check In"

2010-06-19T00:00:00+00:00

I have worked with many developers and many times, I ran into following conversation. Whenever we are in the middle of development and I ask for checking in the code, one of following reply is heard -

Umm, lets do check-in towards end-of-the day. We are not yet done with the code.
Let me clean up these things and make them perfect.
I don't want to check-in as I have some ongoing changes and I can't checkin partial files.
Lets checkin tomorrow when we complete major functionality.

I am sure you must have heard similar conversation (or had it). Can you give a moment and think about why do are we afraid of checking-in? I have been constantly thinking about this issue and there are many reasons which makes sense on this issue. Few of them are - Fear of bad code - Developers are afraid of peers criticizing their code. Its GOOD! Believe me, you're never going to write perfect code without receiving feedback about it. Sooner you receive the feedback, better you perform in short time. This is the reason, people don't make their changes public until they are convinced that they can't do any better. It is clearly a false fear. One of the way to improve and influence your team is to make your work public as it is in progress. This way, you make team aware of your approach. If there is any conflict in approach or disconnect in understanding, it becomes visible sooner and you can resolve it better. Not sure about impact on other parts of the system - Yes! This is very valid reason. You're in middle of a feature and you're not sure how it will impact rest of the system if you check-in early. Half baked features can take the whole system down. To get over this fear, unit testing your code, running a full build before check-in and having continuous integration makes sure your changes are fail-safe. Make sure that you are providing a working build to the team by writing solid unit tests and more comprehensive integration tests. Working on multiple parallel streams - It is often a bad practice to work on multiple parallel streams of work. Even worse when you don't commit any of those changes in VCS. If you're working with DVCS like Git, it is very easy to branch and commit code. If you're working on central VCS like subversion, then it is little tricky to keep your parallel working streams with minimal effort. There are few simple things you can do to make your life little easy - Use DVCS. This will prove immense value in long run, if you use it properly. There is a learning curve associated with it, but it is definitely worth it. If you are using SVN, create a branch if it is a big change. If it is small change, evaluate if you can checkin without breaking application. Taking patch is another option too. Use unit testing safety net on your side and make sure you progress smoothly. In any situation, its not helpful when you're working on parallel streams of work and those changes co-exist. I have often observed people checking in partial or missing content. This happens when they are confused and trying to understand what all files are required to make one of streams live. Why frequent check in is important?

Getting feedback from your application and team.
Never lose any of your changes and hard work. Machines do crash and when they do, its bad!
Avoid stepping on each others toes, particularly when team is distributed and code churn is faster.
Making whole team aware of your approach and remove silo operations. This helps bringing everybody on the same page and triggers conversations whenever required.
You can't explain in 30 minutes of speech what code explains in 10 minutes. It eliminates confusion.
Discipline of frequent checkin allows you to make sure, changes are not causing any conflicts.
Smaller changes, encourages you to refactor and improve code quality.

If you want to read more - Golden principle of development: Check-in-early, check-in-often. DVCS introduction from the master: Google Video on Git. Continuous integration: Wikipedia entry.

Nice utility for finding classes in Jar files

2010-05-06T00:00:00+00:00

When you're working on large java code-base, its not very difficult to run in situation where you get a ClassNotFoundException. Many times we have a daunting task to find where the class is located and it is not very easy task. I found this nice utility called "jarscan" which makes life much easier. Worked great for me today!

Export CSV from MySQL Query

2010-05-06T00:00:00+00:00

As you might be aware that I started a "Murmur" category on my blog. I discover or learn these tips on day-to-day basis and just forget them. Sometimes those takes hours to come with one or often it boils down to just a google search. In any case, you might want to refer to these tips in future as well. So I decided to keep a log of them publicly so that it can help others too. Many times I need to get dump of data which is output of a sql query. MySQL makes it very easy to output the query result as csv file from command line. Following command should create a names.csv file in $MYSQL_INSTALLATION/data/$DB_NAME/ directory. This command comes very handy when you're working on remote shell where you don't have any GUI tools.

SELECT first_name, last_name INTO OUTFILE 'names.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
from employee;

Very useful.

Delete My Account

2010-05-03T00:00:00+00:00

I have few questions for you -

For how many years, are you using Internet?
Can you remember all the sites where you created an account or shared your personal information? (Email, Phone number, Birth date etc.)
Although you can remember all the websites you registered, can you keep those accounts active forever?

Whenever I ask these questions to myself, I get uncomfortable. There are probably 50 websites, who have my online account and personal information. It might be more in most probability. Considering this might be similar situation for most of us, how to make sure that you can delete your account? Most of the websites like Google / Yahoo allows you to delete the account completely. Although few of my readers will love to debate about its actual process. :) Considering our faith in these online services, I am not quite sure what does complete account removal means for these online websites (Facebook / Gmail / Yahoo etc). Even if it is not complete removal at least I have an option to go and remove my online account information. For most of the websites this option is not available at all! I will love to make this as public awareness that we should have account removal as standard feature on all the websites who allow users to sign-up. It should be complete removal of account (including all personal information) and not limited to just un-subscribing user from their mailing list. This will make lot of users much more comfortable to create online account.

Things to Ponder with Alfresco (Part III: WCM - Improvements)

2010-01-04T00:00:00+00:00

Now days using web content management systems (WCM) is trivial. You should continuously update the website content to maintain a fresh look. It increases returning customers and popularity. Almost every major website uses a CMS to achieve this. It might be an in-house developed or an off-the-shelf product. In any case, it helps organizations to separate the content authoring team from the application development cycles. This helps in quicker content updates without being dependent on release schedules. I am going to point out certain things about WCM feature of alfresco. You can also apply the same principles to your WCM if you are using existing one, or planning to develop a new one! Content storage format - Alfresco provides a nice feature called "web-forms". There are two types of forms currently available i.e. WCM forms and ECM forms. Both of them capture the input data and store it as XML. You can also choose to apply content transformation using XSL. It helps to save final output as either HTML, Text or PDF etc. Not to forget, the content is actually captured and saved as XML. In my perception one of the strengths of Alfresco is its content repository. If XML is target format, it is saved in CR as a cm:content property and it defeats the whole purpose. If CR is not aware of the structure of the content, it defeats the purpose of content model definition? It is not very different from storing the content xml as plain string in database. Frankly I don't see any benefit of using CR over database in this method. To avoid such inefficient use of alfresco, you can take some effort and write a custom form component. Use custom forms (develop forms in your application and then communicate with alfresco using API) to capture user data and store them in content repository in defined content model and schema. This will help you in -

Content repository now understands the structure of your content. This is really important and if you have any doubts, please post that in comments. I might write about it later in my blog.
You can apply your custom content transformation logic so that content can be exported in XML / HTML / custom output format. Just generate this once every time you update the content. It improves the efficiency of content management system to deliver target content.
You will have a flexibility of own input format, use JSP or rails forms, its your choice!

Deployment benefits - Alfresco has another great strength, Deployment Infrastructure. I think its also one of its weaknesses. Use it wisely to improve your release management. On the release date of web application usually you come across content sanity issues, or format differences. To avoid that, setup proper environments where alfresco can sync content and test it before you push it to live. This might save you a lot of frustration. Although its a great feature, there is a caveat to that. Generally all the code for web application is stored in CVS systems. Alfresco code (web scripts, web form definitions etc.) is stored in alfresco. Keeping the versions of alfresco code and web application code is nightmare. Develop some automated web script deployment tool and every time you release, wipe out all the previous webscripts and deploy new ones from the CVS. This always makes you keep updated code in the CVS and versioning becomes much robust.

Ubuntu: Cisco VPN Client installation on Intrepid

2008-11-19T00:00:00+00:00

Cisco VPN client for Linux installation doesn't happen out-of-the-box. There is plugin of VPNC which gets integrated with NetworkManager of Ubuntu but I didn't focus on that. Maybe somebody would like to guide the steps for it. I investigated a little and found bunch of articles / blog post about VPN issue. Some of the articles needs an update. Following steps will get VPN client working on Ubuntu Intrepid and use certificate based authentication to connect. First get the distribution of VPNClient for Linux. This distribution is valid for 32-bit as well 64-bit installations.

wget http://tuxx-home.at/vpn/Linux/vpnclient-linux-x86_64-4.8.01.0640-k9.tar.gz
tar -xvzf  vpnclient-linux-x86_64-4.8.01.0640-k9.tar.gz
cd vpnclient
sudo ./vpn_install

This command should result in similar output below:

Cisco Systems VPN Client Version 4.8.01 (0640) Linux Installer
Copyright (C) 1998-2006 Cisco Systems, Inc. All Rights Reserved.

By installing this product you agree that you have read the
license.txt file (The VPN Client license) and will comply with
its terms.

Directory where binaries will be installed [/usr/local/bin]

Automatically start the VPN service at boot time [yes]

In order to build the VPN kernel module, you must have the
kernel headers for the version of the kernel you are running.

Directory containing linux kernel source code [/lib/modules/2.6.27-7-generic/build]

* Binaries will be installed in "/usr/local/bin".
* Modules will be installed in "/lib/modules/2.6.27-7-generic/CiscoVPN".
* The VPN service will be started AUTOMATICALLY at boot time.
* Kernel source from "/lib/modules/2.6.27-7-generic/build" will be used to build
the module.

Is the above correct [y]

Making module
make -C /lib/modules/2.6.27-7-generic/build SUBDIRS=/home/sachin/vpnsandbox/vpnclient
modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.27-7-generic'
CC [M]  /home/sachin/vpnclient/linuxcniapi.o
In file included from /home/sachin/vpnclient/Cniapi.h:15,
from /home/sachin/vpnclient/linuxcniapi.c:31:
/home/sachin/vpnclient/GenDefs.h:113: error: conflicting types for ‘uintptr_t’
include/linux/types.h:40: error: previous declaration of ‘uintptr_t’ was here
make[2]: *** [/home/sachin/vpnclient/linuxcniapi.o] Error 1
make[1]: *** [_module_/home/sachin/vpnclient] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.27-7-generic'
make: *** [default] Error 2
Failed to make module "cisco_ipsec.ko".

To get around this error you need to patch couple of files in the existing source. Download the patch and apply it.

wget http://projects.tuxx-home.at/ciscovpn/patches/vpnclient-linux-2.6.24-final.diff
patch < ./vpnclient-linux-2.6.24-final.diff

After successful patch of the files you can go ahead to the install procedure again -

sudo ./vpn_install

You can see all files getting compiled successfully. If it is successful you will see a message like below -

* You must run "/etc/init.d/vpnclient_init start" before using the client.
* This script will be run AUTOMATICALLY every time you reboot your computer.

Now you can copy your vpn configuration files (*.pcf) in /etc/opt/cisco-vpnclient/Profiles folder and you can start connecting the vpn network using command -

vpnclient connect

If you want to use certificate based authentication for the connection then following do following operation.

sudo cisco_cert_mgr -U -op import

This will ask you for the certificate file location and password for the file if required. Enter the passwords for importing certificates if required and try with connection.

Region Locked DVDs and Mac Book Pro

2008-10-16T00:00:00+00:00

Recently I got a MacBook Pro 2008 from my office. I love the machine but I was completely disappointed when I inserted Italian Job DVD. I was asked to change the region to 2/4 and I stay in region 5. I own the original copy of the DVD and unfortunately, there is no Region 5 copy available in India. It was never the issue for me because in Windows, VLC media player could easily bypass these region codes and ignore it conveniently. [caption id="" align="alignnone" width="400" caption="DVD Regions Map"][/caption] I own few more DVDs belonging to 2/3/4 regions. I own another DVD player (my PlayStation 3) and that is also region locked. Now I have two very expensive machines and lots of original DVD titles yet can't play any of them. I don't know who created this *stupid* protection system which prohibits user from playing legal copies! I tried various things on my Mac -

Using VLC media player for OS X
Using utilities like MacTheRipper for decrypting the dvd
RegionX for changing the region indefinite times

None of the above works. I have almost read thousand blog / forum posts related to DVD region issues with Mac but everything was completely failure. Apparently MacBooks have this "MATSHITA DVD-R UJ-867" super drive which has no way of getting around the protection. I don't want to replace the firmware to region free as it will void my company laptop warranty which makes me crazy. Now what reward I got by paying for the legal versions of the DVDs? In India any English movie DVD will cost you around Rs. 399-699 which is considerably expensive. You pay Rs. 800-1000 for 1 month of the internet connection. Why I will not be tempted to download nicely ripped DivX movie from torrent instead of buying freaking expensive DVDs which don't play at all? Now can I demand my money back for these movies? Can I illegally download the DivX rip of the movie and play it on my laptop and still it will be legal? I really don't know but I will never buy legal DVDs for sure!

Things to ponder with Alfresco (Part-II: Workflow Design)

2008-09-21T00:00:00+00:00

As I mentioned in my earlier post that I will be writing few more posts about the Alfresco. This is second installment in the series. If you are thinking to use Alfresco at the center of your application then it is hard that it will not have a workflow. If your business process is just more than simple workflows mentioned in Alfresco, then for sure you have to write your own business model definition.

If you are planning to do so, here are few lessons I have learned –

Simplicity – We got a lot of benefits just by keeping our process definition simple and neat. Don’t add unnecessary steps which will cause confusing steps for the end users.
Right things at right place – Alfresco is very flexible in terms of business logic implementation. You will always have multiple options to carry out same operation. Web Scripts, Java classes or Web Service and you are tempted to use different things at times. Decide one approach and stick to it. Put as much as logic in the workflow and don’t spread across the workflow business logic outside the workflow.
Loops are evil – Loops in the workflow actions are daunting at times. Rethink on the circular steps in the workflow and make sure you are not violating the “simplicity” rule.
Blocking is good sometimes – Next bigger decision is synchronous or asynchronous operations. All the operations which might result in inconsistent states can be synchronous i.e. transactional. Asynchronous operations are better in terms of responsiveness. Sending e-mails could be asynchronous tasks as you people can receive emails after couple of seconds, but if it is related to pushing records to a queue, let it be synchronous. You got the idea!

You should check Alfresco wiki for more details. There is plenty of documentation available for the administration and authoring custom workflows.

Things to ponder with Alfresco (Part-I: Content Model)

2008-09-13T00:00:00+00:00

For any content repository, content model definition is similar to the DDL (schema definition) for databases. It is expected in a big application to have evolutionary content model like databases. Updating the table structures or introducing new tables is common in any business application. In content repository, you should also keep in mind that your content model should be open to evolve. Define the content model which will be extensible. Most of the good practices of database design should be applicable here. Few tips for better content models -

Use aspects for specializing types instead of defining content types. There are few advantages of that, aspects can be applied or removed at run-time which gives you much more control over the dynamic nature of types.
Aspects also helps in keeping the default set of properties over a content type to be small. Alfresco recommends you to prefer aspects over custom content types.
Aspects helps you to logically group properties. This helps your data to be part of multiple groups at the same time. If you have aspects Publishable and Indexable; you can apply or revoke a nature at your will without much hassle.
Indexing is very helpful for searchable content. It is useful but use it wisely. The cost of indexing in Alfresco is much higher than cost of index in Databases as this will be full-text search indexes. It will take up more disk space as well as time to index the data. Don't index the content which people are not expect to search! You can define this at the time of content model definition.
Versioning is similar to indexing and use this wisely. The content which is versionable is handled separately by Alfresco. This might be affecting performance as your repository size grows. Don't mark content as versionable if you don't plan to maintain revisions for content.

I will continue this series with other experiences with Alfresco. Later.

Ubuntu: Firefox 3 beta and unhappy users

2008-05-29T00:00:00+00:00

Beta software prohibited! (Photo obtained from here)

I was really disappointed over past few weeks about Ubuntu 8.04 release. The major reason of disappointment was Firefox 3 beta 4-5. The browser might be having lot of new things which are amazing, the performance might is improved for JavaScript execution but in reality, the overall performance of the browser wasn't that good. I was experiencing slow browsing speeds, sometimes the browser wouldn't respond and sometimes even crash.

Firefox 3 is still gaining popularity and there are tons of plug-ins which are still not compatible with it. My favorite plugins are no longer usable with Firefox 3 and that was a big drag for me. Firebug was one of them. The main reason I feel very disappointed was Firefox 3 was in beta!

Ubuntu was simplifying Linux for all the users who don't want to touch command line for writing documents and browsing internet etc. They are doing a great job on that front too. Firefox 3 made me think is it going away from that goal by giving beta software distribution? I will urge Ubuntu community for not including any of the beta software in mainstream distribution for any reason. That might not only discourage non-technical users but also put wrong message on the table.

For those users who have similar problem, they can uninstall the Firefox 3 from synaptic package manager and install Firefox 2 browser instead. Happy browsing!

Alfresco: Workflow managed by JavaScript

2008-05-29T00:00:00+00:00

If you are a regular reader of this blog, you might have figured out that I am currently working on a project involving alfresco at the center and it's a content management system. Earlier I have worked with Content repository (Apache Jackrabbit) and that was a good experience. Alfresco is next step of a content repository which extends it into a full blown content management system and much more. I was working on a problem where we defined a custom (Advanced workflow) to be precise in Alfresco lingo. This workflow was needed to be monitored, managed and accessed only via a webscript as we have used webscripts to do almost everything in our project. I was very pleased to see this, but that is only available in 2.9 labs release (as of now). We are using alfresco 2.2.0 version for our development and that is production stable version, so I have very little choice but go with production version. Alfresco has very limited JavaScript API support for workflow manipulation and that was difficult job for manage. There was no other option than writing custom JavaScript API which allows to manage workflow in Alfresco. We did that and it was very powerful and easy! :) I wanted following features for JavaScript API -

Easy to extend
Can handle any workflows
Provides easy ways to retrieve tasks, definitions, paths etc.
Provides as much functionality possible which is available in Alfresco Workflow Console.

Alfresco workflow console is command line interaction component which allows you to manipulate, get information about workflows using simple commands and I wanted to do all that using JavaScript. I started exploring the source code and found a class 'org.alfresco.service.cmr.workflow.WorkflowService'. This class has generic API which allows you to do all the operations possible through workflow console. This bean is used for workflow console operations. So all I needed was expose this bean in JavaScript as root object and start using it. That simple! I wrote a simple java class to expose the bean as follows:

package com.xxxx.yyyy.workflow;

import java.util.List;

import org.alfresco.repo.processor.BaseProcessorExtension;
import org.alfresco.service.ServiceRegistry;
import org.alfresco.service.cmr.repository.NodeRef;
import org.alfresco.service.cmr.workflow.WorkflowService;

public class WorkflowManager extends BaseProcessorExtension {
	private ServiceRegistry services;

	public void setServiceRegistry(ServiceRegistry serviceRegistry) {
		this.services = serviceRegistry;
	}

	public WorkflowService getWorkflowService(){
		return services.getWorkflowService();
	}
}

The method getWorkflowService() exposes the workflow service to the JavaScript calls. To expose the Workflow Manager as root object in JavaScript create WEB-INF/classes/alfresco/extension/custom-script-context.xml and you will be good! (If you don't know how to do that, read this) Now, you can get the handle of workflow service in JavaScript code. Yet, we need to write a small component which will allow us to retrieve the active workflows, tasks etc. To do so, I wrote a small JavaScript object which provides those functionality to the user.

var workflowService = workflow.getWorkflowService();

var W = {
	transitions: function(noderef){
		var task = W._currentTask(noderef);
		var trans = [];
		try{
		var tasks = task && task.path.node.transitions;
		for(var index=0; index < tasks.length; index++){
			trans[trans.length] = tasks[index].id;
		}
		}catch(error){
			// Ignore error
		}
		return trans;
	},

	move: function(noderef, command){
		workflowService.endTask(W._currentTask(noderef).id
                , command);
	},

	_workflowPath: function(noderef){
		var workFlows = workflow.getWorkflows
                      ("workspace://SpacesStore/" + noderef, true);
		if(workFlows.size()>0){
			return workFlows.get(0).id + "-@";
		}
		return "";
	},

	_currentTask: function(noderef){
		var curTask;
		var tasks;
		try{
		tasks = workflowService.getTasksForWorkflowPath
                           (W._workflowPath(noderef));
		if(tasks.size() > 0)
		      curTask = tasks.get(0);
		} catch(Error){
			// Ignore error
		}
		return curTask;
	}
}

function initWorkflow(noderef)
{
	var wflow = actions.create("start-workflow");
	wflow.parameters.workflowName
             = "jbpm$xxxxyyyy:abc"; // your workflow name
	wflow.parameters["bpm:assignee"]
             = person.properties.userName;
	wflow.parameters["bpm:workflowDescription"]
             = "Put your description";
	wflow.execute(noderef_for_which_workflow_will_associate);
}

Thats it! The initiate workflow method is used to apply a workflow to a node. Once that is done, the W object can be used to do operations like retrieving the active tasks, ending the workflow and making the workflow transition. Hope that helps. If you need more workflow related information and reading material you can visit following places:

JBPM jPDL Documentation (here)
Alfresco workflow wiki (here)
ECMArchitect Advance Workflow Guide (here)

Guarding in JavaScript

2008-04-25T00:00:00+00:00

After looking at JavaScript programming sessions by Douglas Crockford (here), I couldn't stop but admire JavaScript. There are many many impressive things in JavaScript but one of the eye-catchy thing was guarding operator in JavaScript. The && operator in JavaScript can be acted in two ways. The normal 'and' operator which is used in determining the conditional logic results. The other lesser known feature of JavaScript is the guard operator. In following code -

/* The if is checking if specs is not undefined, false, null,
"" (empty string) or 0 (numeric zero) as these all values
are falsy conditions in JavaScript */

if(specs)
     width = specs.width;

// To avoid the if condition you can simply do like -

width = specs && specs.width;

/* This makes sure that specs is not falsy and they
returns the second operand which is convenient
way to write null check */

The same is with || operator too. If the first operand is falsy then it just returns the second operand instead. Consider following code:

name = specs.name || "Name not set";
/* Use the specs.name, but if name doesn't have a value, use default
value instead. */

Very simple and very efficient!

Alfresco: Server side JavaScript, anyone interested?

2008-04-22T00:00:00+00:00

I started with Alfresco and its amazing software suite. For people, who don’t know about content repositories, in short its hierarchical, versionable storage (read more). Alfresco is one step ahead of other content repository implementations and it is full content management suite. This means, you have workflow, content repository and a portal which allows you to do lot of standard things e.g. communicating with external world via API, portal, security/authentication etc. From my perspective Alfresco is really great product considering the amount of features it provides and the quality of them.

I am working on a project which involves some bit of alfresco customization to meet the customer needs. This involves configuring alfresco for Windows SSO, allowing users to map their home space in alfresco as network share, knitting a workflow tightly integrated with their business process and writing some dynamic content generation code for reporting, user interaction and for auditing purpose. We could do most of the things straight away and we are quite happy with kind of response alfresco was giving. Everything is clean and configurable. Basically alfresco is built using spring framework; so you can configure custom beans or extend existing beans very easily via some xml files. There are tons of different services and pre-configured beans which might be little overwhelming for the beginners but, as you started to get familiar it doesn’t feel so bad.

We were at a point where we wanted to write some server side logic to create a dynamic (web 2.0) UI which has lot of interaction based on data queries with content repository. So we needed easy approach to query repository, minimal xml configuration, ability to change / update logic easily, faster development cycle to start with. Alfresco provides lot of options to communicate to repository -

Java API (most raw access to the repository, repository instance is singleton so that will be injected as dependency via spring configuration)
Web Service - Very standard SOAP and XML RPC based communication
RMI - Java RMI which is another standard method looks similar to Web Services
RESTful JavaScript API :)

I was excited by last option the most. I thought, lets explore the last one as first three were mundane and I knew what are the pros-cons of most of them. Alfresco provides a clean MVC around content repository and other parts of Alfresco (viz. workflow, user homes and some root objects) where controller is a pre-configured servlet, model is JavaScript code and view is a freemarker template. The beauty of the system is lying in the simplicity of JavavScript code.

Alfresco provides a nice feature rich JavaScript API (more here) where some of the objects are injected from scripting engine as root objects. These object provides the building blocks for JavaScript API e.g. search, user home, workflows etc. You can write your own Java services and inject them as root object using simple configuration to extend this foundation. Once your basic services are in place, then you can start knitting or wiring your business logic using these foundation blocks. JavaScript is an ideal choice as it expects small, working code instead of heavy logic which Java can handle. With small fluff of code you will be looking at some fantastic results.

I was amazed and surprised by the simplicity but there are few things which I needed to finalize before I decided to make a choice -

Performance - JavaScript is interpreted in Alfresco using Rhino which is high performance engine by Mozilla. So, no worries about performance. I have evaluated Rhino some time back and that was really great in performance.
Testability - That was major concern. Unit testing with JavaScript was not easy. Crosscheck is a good choice to begin with. There are other options like JsUnit and some browser based frameworks which are not that popular.
Debugging - Alfresco provides a very nice JavaScript debugger where you have luxury of an IDE. Its launched from browser (firefox, IE) and works with ease. Standard debugger features like breakpoints, watches and navigation control are provided.
Ease of deployment - Most of the scripts are convention based (naming conventions, location conventions etc) and very standard. Server can identify updated scripts and no server restart required (in development environment thats a needed feature)
Ability to talk to Java logic - If you want to use a external third-party library from JavaScript, no problem! JavaScript code can use API which are written in Java and work with them with ease. You can even write your heavy-weight business logic components in Java and inject that as service to reduce time and achieve code-reuse.

After all concerns are addressed, I was pretty sure I will be writing most of my business logic code and writing the services in JavaScript. I haven’t stressed much in this post on view as its pretty standard freemarker template engine. I would like to touch upon that later.

If you are considering the Alfresco and writing your business logic in JavaScript (called as Web Scripts in Alfresco) then I would seriously recommend following things for reading/viewing -

Douglas Crockford — “The JavaScript Programming Language” series of videos (found here)
Alfresco Developer series - For basic hands-on tutorials for Alfresco customization and understanding (found here)

Have a great time functional programming! :)

Java: static and transient keywords

2008-04-20T00:00:00+00:00

I came across transient keyword and wanted to get few more details about it. This is one of the most ignored or rarely used keyword by me. It is really funny but, I never really used this keyword extensively. The main reason is, this keyword has a very specific use and most of the applications don’t need that for building the business applications. As per info here, when any object state is getting serialized, static and transient variables will be ignored for serialization. Still, Java compiler doesn’t give any warning or prohibit from making a variable static and transient at the same time. I don’t think this as error but a warning should make more sense in this case.

Don't put JavaScript validation, I'll Firebug!

2008-04-20T00:00:00+00:00

Recently, I was with my friend to book a cab. The cab service provided an online booking facility which allows you to provide some details (phone number, address, destination and time of cab etc.). We went ahead and tried to book the cab and it wasn’t accepting the request. It seemed that they put a validation to check that user shall not book a cab less than 4 hours prior to departure.

Unfortunately, we wanted to book the cab on a 3 hours notice and it was mandatory for us to book the cab. We couldn’t resist but started looking for options at our hand. We pulled the website in Firefox and opened firebug to see where the validation is executed. We were surprised by the validation was done in browser using a very simple JavaScript. We thought; let’s see if we could just pass the 4 hours validation by bypassing the validation or maybe mocking the action. The validation code was very simple; we just put the breakpoint in the function and then set the variable value via firebug to an acceptable level. Bang! The code ran successfully and cab service accepted the request.

We made a successful cab reservation and my friend could take a peaceful ride back home! Moral of the story: Please put all the business critical validations on the server side instead of browser based JavaScript. If you really want to do some browser based validation then it will be really good idea to obfuscate the code.

Change of 2008

2008-01-22T00:00:00+00:00

If you are reading this post in RSS feed reader, then I think you should visit this post on the blog. I have now hosted blog at new place. I owe a big Thank to Wordpress for providing me great services for over two years. I believe that wordpress is the most powerful yet simple blogging service of all. I wanted to move to new hosting provider on the first day of 2008 but somehow couldn’t make it. I have now migrated all the blog with posts and comments.

I have more things to add to this place now. I will keep you posted about that in coming days. I am even going to put another blog (personal) for my personal, friends and family updates. You can find that here.

More is coming on the way.

NANT: The webservice at 'localhost' does not exist or is not reachable.

2008-01-15T00:00:00+00:00

I am a newbie to .Net based development. I have been working on a project which involves web services, client / server applications and web applications. For a guy who spent more than 5 years in Java, this is not new but the environment change is definitely painful. In my developer life, I have kept switching between development environments. I tried Ubuntu, Red Hat Linux, Windows (98/Me/XP/Vista) and Solaris (for a little while). Whenever I used to switch the platform, I never felt too much pain as Java IDE support is great on many platforms. You don’t need to worry about platform specific things and the web server / application servers behaves exactly same on all the platforms (except the file separators)!

Enough with my rant about .Net. Lets come to the point.

Today, I was trying to create virtual directories using Nant in IIS. My environment details are as follows:

Windows Vista Business Edition
IIS 7 installed with default options.

Now with above environment, when I fire the command which creates virtual directories. I got following console output:

E:\Blah\tools\nant\NAnt.exe -buildfile:Blah.build makeVDir
NAnt 0.85 (Build 0.85.2344.0; rc4; 02/06/2006)
Copyright (C) 2001-2006 Gerry Shaw
http://nant.sourceforge.net
Buildfile: file:///E:/Blah/Blah.build
Target framework: Microsoft .NET Framework 2.0
Target(s) specified: makeVDir
[loadtasks] Scanning assembly "NAnt.Contrib.Tasks" for extensions.
[loadtasks] Scanning assembly "DBUpdaterTask" for extensions.
[script] Scanning assembly "4cliemw_" for extensions.
makeVDir:
[deliisdir] Deleting virtual directory 'Blah.WebServices'
on 'localhost:80' (website: ).
[deliisdir] The webservice at 'localhost' does not exist
or is not reachable.
[deliisdir] Deleting virtual directory 'BlahService' on
'localhost:80' (website: ).
[deliisdir] The webservice at 'localhost' does not exist or
is not reachable.
[mkiisdir] Creating/modifying virtual directory 'Blah.WebServices'
on 'localhost:80' (website: ).
BUILD FAILED - 2 non-fatal error(s), 0 warning(s)
The webservice at 'localhost' does not exist or is not reachable.
Total time: 0.8 seconds.

I googled for it and didn’t found anything useful. I saw posts here and here with no information or solutions. Fortunately, I got this working and thought to make a post about it.I tried following things and I am listing them all.

My machine was referring to localhost as “::1” which is a IPv6 address. I went ahead and commented that in %windows%\system32\drivers\etc\hosts and kept localhost pointing to “127.0.0.1”.
I also forcefully edited the “Default” website binding in IIS Manager to 127.0.0.1 which was previously to “All Unassigned”.
Last thing I tried was (and this made my installation really work) was installing additional IIS Features. By default IIS in windows vista installs features which doesn’t allow Nant to register webservices or create virtual directories. So I went ahead and installed all the features under IIS installation in windows vista. The screenshot is attached below.

I think even the Nant message is little misleading. I got this thing working by just trial and error. I am still investigating exactly what is going wrong behind the scenes. If anybody already know please update the comments or I will post them when I find root cause.

Dhishkyaanv: Are you game?

2008-01-14T00:00:00+00:00

ThoughtWorks is holding a Gaming tournament in Pune office. The event is open for all interested gamers. The event details and registration details are available here. The event is having limited spots and its invitation based. So hurry up and get signed up for the same.

The idea of the tournament is to select winner across three categories of games. The three categories are arcade (Need for Speed), strategy (Age of Empires) and action (Counter Strike). Based on the nature of games, the teams or individuals will compete against each other to get the winning spot. If you feel you’re ready for burning some rubber, conquering territories or saving hostages please do drop by the official site and register yourself.

For any questions or queries, you can contact me at “sachin” at thoughtworks dot com

Ubuntu: Will Skype jingle?

2007-11-08T00:00:00+00:00

Skype 2.0 beta has arrived for Linux. I read about this here. I have been trying to get Google Talk support for Linux for a while. I tried that in PLUG mashup event. So summary of what I did was as follows:

What I wanted to achieve?

I wanted to integrate support for voice chat using Pidgin for Google talk users.

What we did?

We started with analysis of libjingle which is xmpp based communication library from Google code. We faced lots of problems in integrating that as a plugin with Pidgin. I could see that there is already a feature request logged with Pidgin for this support. I tried to work with Ubuntu (Gutsy Gibbon). There were many problems for libraries which are either very old or sometimes difficult to maintain.

There are many issues with the library dependencies of libjingle as well, viz. linphone, gstreamer etc. Each of them require different version which is sometimes quite old than the current stable version.

I will be working on this more to achieve the goal. There seems to be quite few huddles before I could reach there though. I don’t know what is current state of this post. I am hoping this will start working soon. Till then, I’m there to see how I can add the talk plugin to pidgin.

Ubuntu 7.10: Multiple monitors chaos

2007-10-12T00:00:00+00:00

I have already started early testing of Ubuntu 7.10 (Gutsy Gibbon). I am very impressed so far with the stability, performance and features. Today, I tried to configure multiple monitors for presentation. After lots of efforts I couldn’t configure them in proper way. Whenever projector screen was being used, laptop screen resolution will get messed up.

After lots of efforts I couldn’t get it working and finally I gave up. I think, Gutsy is just not ready yet. So far, I couldn’t get the multiple monitors working. Somebody has already started a thread here. (Sorry… no replies yet :( )

I just hope that this feature will be better supported in main release or at least in a patch, as early as possible.

For now, I guess at least with beta this kind of humour still stands true. :)

Ubuntu 7.10: Early Impressions

2007-10-06T00:00:00+00:00

I downloaded beta of Ubuntu Gutsy Gibbon yesterday. Wow! That’s the only word comes to my mind after having a look at latest version. This version is much changed from the 7.04 (although I wasn’t expecting much from this release). This release has few very good features, to start a few:

Installation wizard has provision to migrate existing users data. This feature is just brilliant! Now, I don’t need to rush through my installation and create a backup before I want to try next version of my favourite operating system!
Display configuration comes now in very user friendly screen. I always faced problems with connecting and extending the second screen to the display configuration. I used to screw up something every time (… I know I’m dumb but isn’t why the operating systems are there in the first place). This screen will allow us to change the display configuration in predictive way and easily.
Restricted driver support for the nVidia, ATI and other devices. For the first time, I installed these restricted drivers as another package! To make my joy multi fold, Ubuntu by itself detected the driver and a nice pop-up comes near the clock informing about the restricted drivers. Very user friendly.
Faster boot and response time. The memory footprint is also very less (~200 MB on my laptop with 1 GB RAM).
GNOME 2.2 is just better than previous versions. My early impressions are very nice about this. No crashes so far. Seems to be stable and neat! I hope this impression continues for longer time.
Network configuration just became simpler. For my needs to switch between wireless networks, office, VPN and home networks was seamless. For the first time I was prompted to enter the necessary authentication and later on it’s doing on its own! This has reduced my all pain to pay attention to my network connections. Kudos to Ubuntu team and developers, job well done!
Not the last but for eye-candy lovers (including me) Compiz fusion is just superb. The default set of effects implemented are very nice and lightweight. All the effect options are available in user friendly screen and you can very easily turn them on and off.

… and one more important thing. I could manage to install Lotus notes on my Ubuntu. Now the last reason why I was dependent on Windows is also gone and I can happily keep only one operating system in laptop (not to mention that will be ubuntu)!

Shhhh... Gutsy is coming!

2007-10-04T00:00:00+00:00

Gutsy Gibbon is declared and its about to hit the road in couple of weeks. I was excited by the news of new Ubuntu. Breezy was good but I was having few issues with it. Many of those got fixed in Feisty. To quote a few issues were wireless connectivity, widescreen monitor drivers for specific nVidia chipsets etc.

Finally another version is on the verge of a release and I’m quite sure that, this release must be better than ever. Ubuntu 7.10 is declared and its code name is ‘Gutsy Gibbon’. For the developers and geeks who can’t wait for couple of weeks, there is a beta version download available which can be upgraded later on. Those can’t wait another second on the post please proceed here for download.

So what’s new in this release? (Features I liked the most are listed here)

GNOME 2.20 GNOME is coming reloaded with lots of features. A detailed description of all the features which will be improved / introduced can be found here.
Dynamic screen configuration for nVidia, ATI etc. chipsets. This was one of the features where Linux lacks a lot than Windows. I hate to compare Linux features with Windows yard-stick but sometimes it becomes necessary.
Automatic printer configuration! I remember spending about an hour in just getting one printout on Kubuntu. This seems to be pretty nice if this works as expected.

Apart from the tiny list mentioned above there are loads of features available in GNOME new release. Go checkout all of them and start writing your precious reviews about the improvements or bugs in the system. I would be hunting blogosphere for reviews related to gutsy.

Thank you very much Ubuntu team for the hard work and prompt releases. Please keep the good work going as you always did!

Don't say 'Yesterday' in a stand-up!

2007-09-17T00:00:00+00:00

At ThoughtWorks Pune office, we were observing a regular pattern in Stand-up. Everybody will start to speak with “Yesterday, I did this… I did that…” and whole team used to feel it more of a status update and not giving idea and sharing the context. To address this issue, one of our team-mate came with a suggestion to avoid using word “Yesterday” in the stand-up. Now you don’t have word yesterday with you, you will be automatically forced to give a update which involves not only yesterday’s work but more than that. This worked great until following conversation happened…

(Names of the team-members are changed)

Friday, 14 Sept, 9:30 IST

… Ganpat: “Yesterday, I worked on …”

Keshubabu: “Ganpat, don’t use word ‘yesterday’ or at least avoid using that…”

Ganpat: “Ok. Day-before-today I worked on…”

Whole team: !!!!!!!!!!!

That was a funny incident but, we really experienced value coming out of avoiding using word ‘Yesterday’. This helped us to achieve sharing of context of the story and not just yesterday’s work.

You are not allowed to book Indian Railway tickets from Linux!

2007-08-26T00:00:00+00:00

IRCTC is an online railway reservation portal. I always admired the functionality provided by this portal. It simplified life of millions of people by providing very prompt and efficient services in ticket booking and information. From technical perspective, this is really amazing; how they are knitting all the reservation information and maintaining speed of operation as well. I was always fan of this site, until the black day arrived two days back.

I occasionally travel by train. Most of the times, I take benefit of online ticket booking services from IRCTC. I was surprised by the fresh look of the site. I wasn’t knowing that enlightenment was about to end after I logged in the portal. I couldn’t see the reservation form at all! I thought, something is wrong in the server or network and somehow page is not loaded completely. I investigated a little bit and came to know that, the portal is not designed to work with non-IE browsers. I had to reboot machine to Windows and book the ticket but that was very disappointing exercise in all.

This happened because they designed site for only IE browsers. This is the problem I always wanted to hightlight through my previous posts like this and this.

I have already dropped a letter to concerned officials requesting Non-IE compatible website and waiting for response. Government and Public services portals should be designed very carefully considering there might be consumers of those services from every corner of the country. Everybody might be using different software to access those services and it is not impossible to cater all of them. Just follow Standards (…and stick to them)!

Linux Myth Buster: What is difference in Server and Desktop Linux?

2007-08-21T00:00:00+00:00

Disclaimer: This article is too long! This was done to explain lots of historical information which lead to this question. If you are not interested, you can skip to ‘Actual answer begins here…’ section.

Why this question is here?

Linux has been part of my technical life since 2001. I have seen Red Hat Linux 6.x early releases and fall of Red Hat Linux. Rise of Fedora and birth of Ubuntu. This whole journey is very interesting and enjoying in its own way. Linux is today having a position which is not dominating but surely challenging! Linux is not having aim to replace every desktop in the world (… some might say it just can’t!) but surely its putting a smile on people’s faces who have used it for a while. You can’t just put Linux out of the door and wait to disappear, the small tiny tux is now having a great foot tapping parade of people and organizations which will do whatever they need to make this tux growing!

I have been asked a question many times by my friends and colleagues who are new to Linux or just want to learn Linux. They started the question with, “Sachin, I have a stupid question…” this was always one of questions which I answered very religiously. The only reason I never made any nonsense while answering this question was, they were new to Linux and slight wrong answer might put a wrong foundation about Linux Information in their minds. My friend Ashish suggested me to write a blog post about the same so that, people can get right information.

The question is not at all stupid! In fact, most of people don’t know the answer and they hesitate to ask this. I would request you to read the whole post and if you have any suggestion or question feel free to post a comment about it.

In technical terms, Linux is simply a ~2 MB sized binary file which has all (…almost all) the stuff necessary to manage file systems, networks, processes etc. Usually Mandrake, SuSE, Fedora, RHEL and all other popular Linux flavors contain this ~2MB binary acting as heart and soul of their distribution. Remember the italicized word in last sentence; we are going to come back to that word shortly. You can obtain Linux in pure form from Kernel distribution site. That’s most pure and raw form of Linux. If you follow all the instructions and put lots of efforts (although targeted audience is not supposed to do that) you might get a text based console at the max!

Remember the word we italicized? Distribution, yes that is a very important term in this world! Since, Linux started way back in 90s and it was free, people started experimenting with it. People wrote tons of applications which could be run on this robust and naive platform. Most of the applications were coming out of university students, startups, research laboratories and programmers who are dying to try new things! This created a huge sea of application, system, utility software which was unmanageable. Clearly, an effort to make Linux easily adoptable was necessary. This problem lead to a solution called as Distribution. The idea was great because of its simplicity. People started delivering Linux along with few must have software nicely wrapped around with a GUI and installer. This contained system, utility, programming, internet and other software categories. A beginner had all things nicely wrapped which were needed a beginner. Soon, the distribution size exploded from few hundred megabytes to 4-5 gigabytes and the size was growing day by day. Tons of software was coming as part of default bundle which was not at all needed by a normal user. To address this chaos and make Linux more targeted to audience, people started creating their own customized distributions! This lead to Live CD, business, small and large sized distributions! Each distribution was targeted for an audience with considering their need of software and applications. Nothing got changed; still a nice GUI and rock solid base functionality was supported. Each distribution (a.k.a. distro) was addressing a very specific problem. There are around 1000+ distributions available to date at the time of writing this post, refer to (distrowatch).

Actual answer begins here…

Linux was really easy to configure and creating a distribution was real fun, which lead to “Problem of Many”. There were unlimited options and unlimited configurations! This created a great amount of chaos and making impossible for new comers to figure out which one of these 1000+ distributions they can use for day-to-day use or what they should use to install a Linux server. When, any organization was supposed to install a Linux powered web server, a ‘Super’ developer used to spend around a week to either create a really custom version of Linux. This used to be based upon some out-of-box distribution which matches close to their criteria. If this doesn’t work out well, organization will pay hefty amount of money to purchase Enterprise Linux from vendors like Red Hat or SuSE. Commercial software vendor were giving 24x7 supports and other business terms which were comforting to organization which were new to Linux. Today or another day this was supposed to happen. (Let’s leave the topic here as this might be a good discussion over Slashdot)

In simple terms, Linux is a simple binary. Distributions are nothing but customized suite of application, system software which is solving some business, personal computing problem for most of the cases in ‘out-of-box’ fashion. Server and Desktop versions of distributions are nothing but different set of software packaged together. Yes, that simple it is! Just another distribution might be called Server Linux and others might be called different names as well.

Ubuntu, Kubuntu are today’s most popular Desktop versions of Linux distributions (from my perspective). Ubuntu as well releases a server distribution with same quality of bundling and support.

What can I expect from a Desktop distribution?

You can expect all the functions (perhaps much more than that) supported by a Installed and Configured Standard Windows box from a Linux Desktop distribution. This includes E-mail client, Internet Browser (Mozilla Firefox etc.), Media Player, Media Browser, Games (basic), Network support, Office and Productivity suite etc. There are tons of other features coming which you just need to explore.

What can I expect from a Server distribution?

If you are planning to host a web server which can run database applications written in PHP or Ruby then perhaps this is the distribution you are looking for. This includes Telnet, SSH, FTP, Web and other remote servers. This includes few server applications which are normally needed by web administrators. In all normal cases you don’t need this distribution as you don’t need most of the applications in your day-to-day basis unless you are geek.

I am still confused, which one is right for me today? Will I miss something if I choose a wrong distribution today?

Hell No! There is no such phrase called as ‘can’t happen’ in Linux world! If you choose a desktop or server or for that matter any distribution, you will be able to install any application which can run on Linux. Let us take example of standard desktop distribution of Ubuntu. If you installed the standard 1-CD installation and then want to host a web server, please go ahead. You can always find very nice and friendly applications like ‘Apt’ and ‘Package Manager’ which will help you to install web servers, database server, FTP or SSH servers without typing a single command! You can just click the mouse, make sure you are connected to internet and choose any software from repository of ~25000 applications available for Ubuntu! Doesn’t it sound simple and great? So there is no problem in beginning with any distribution of your choice. There might be few distributions which may not support ‘Apt’ but they might be having some parallel application which does the same job. SuSE have ‘yum’, Red Hat has ‘up2date’ and others have similar tools. You just need to find tool which runs in your distribution.

Thank you for reading a lengthy post. Please help me if there is anything which needs rectification.

Feature Request: Ubuntu

2007-07-11T00:00:00+00:00

I have been using Kubuntu 7.04 (Feisty) from past month and it was unbelievable experience. I have used Windows for using Lotus Notes and Google Talk only. KDE is becoming better day by day. WPA integration in Kubuntu with support for latest drivers for NVidia was a great plus in Feisty release. I think, if I could run Google Talk somehow in Linux then that will be last nail in my Windows installation and the next day I will format my hard drive to have only one OS. (No need to mention that will be Kubuntu!)

Before that happens, I have one more feature request for Kubuntu / Ubuntu. I have spent lots of time in customizing my installation. Drivers, Beryl, Themes, Development programs and other small stuff. All the programs were installed on ad-hoc basis or when I needed them. I don’t want to go through the same cycle again. It will be great to have installation persistence utility. This utility should do following:

Remember all the package names which I installed using apt. This should be fairly small file having hundreds of entries of softwares.
I don’t want to take backup of installed packages /usr or /opt on my external hard drives etc. because if next time I go for 64 bit installation (currently I have 32 bit installation) then all my backup won’t work.
When next time, I install a fresh system, I should be able to give this configuration file generated by installation persistence utility. The utility should be intelligent enough to pull all the packages to their latest version and make my installation up-to-date. This way, I don’t need to repeat the same exercise I did in the last installation.

I know that, with few shell scripts that would have been easily possible, but I don’t want to write a custom shell script for that. I want this should be part of my Ubuntu installation. This way, I don’t need to port my shell scripts.

Once the basic infrastructure is in place, I would love to see more features like migration of documents and data files. Applying desktop and theme settings on fresh desktop etc. This way, the distribution installation and switches will become very simple.

Thank you Kubuntu!

Bollywood is Bollywood!

2007-06-02T00:00:00+00:00

Today, after watching ‘Shootout at Lokhandwala’ (SAL), I realized that there is huge difference in the quality of movies produced in Hollywood and Bollywood. Directors like Quentin Tarantino, Woody Allen and Steven Spielberg make movies which has lots of research and study. Not only the shot but the finest detail of shot gets enough attention. Movies which are having based on past events are shot with great precision to make sure that they don’t have any present day detail.

In SaL I found few blunt mistakes which could have been fixed while shooting or at least at the editing time. On the top of mind few things come are -

Tata Indigo! - The movie is shot in 1989-1992 timeframe. When Sunil Shetty makes a blank call to his wife, I could see a white colored Tata Indigo car passing by. Its impossible to see that car in 1991! The car was launched in India in year 2002. Strange to see that car in 1991. :)
AA Group posters! - In the song ‘Ganpat’ when group of people are dancing on the top of a bus, we could easily see ‘Anil Dhirubhai Ambani’ Group advertisements on the light poles. This group was founded in 2005 after division of Ambani group of companies. I thought roots of this group were founded in 1992 :)

In addition to that, there are other small mistakes too. Which you can easily find like the bikes which were not present in movie time line etc.

Looks like Indian directors have to put little more efforts in research and analysis to make movie more authentic and real. Movies like Hollywood ending, Schindler’s List were based on past events. They were so real that you will believe those were produced from original footage which was conceived in the movie timeline.

Off to see Shrek 3! :)

Update on performance of Scripting languages

2007-05-22T00:00:00+00:00

I wrote a post related to Beanshell and Rhino performance comparison about year ago. I received a comment to that post which talks about Groovy performance comparison which should be attached to these results. The latest performance of Groovy and Beanshell are posted on Groovy list which can be found here.

I will love to spend some time and get results for the latest releases of all the scripting languages. In meantime, there must be some improvement on Rhino and Beanshell side (I hope so :) )!

Kubuntu 7.04 Feisty Fawn - WPA support for Intel ProWireless

2007-04-24T00:00:00+00:00

The second huddle in the way to enjoy Kubuntu and my Laptop, WPA support for my home wireless LAN. Finally, I could get a workaround with it by installing the proper driver. You can find the drivers here. There is a great Install guide which helps to get most out of it.

Enjoy Wi-Fi on Kubuntu. :)

Kubuntu 7.04 Feisty Fawn - Getting right resolution

2007-04-24T00:00:00+00:00

Yesterday, on first day of vacation, I installed Kubuntu Feisty Fawn release. This time, I wanted to try something else than Ubuntu. I faced lots of usability issues with Ubuntu. It had some problems in auto detecting my network settings. I use my laptop in various network environments, office, home and public places. I had to every time select the proper network adapter to get running. I never figured out whether there is some other way to do things but out-of-box configuration didn’t allow that. There were similar problem on other grounds too.

Kubuntu was amazing in these regards, I plugged my WD passport drive and immediately it shown me the dialog (like windows :) ) to open or view pictures. I liked the way they picked up good features of windows. When I plugged in my network cable, it automatically configured itself to proper device and I had to do nothing except waiting for couple of seconds! So my first impressions of Kubuntu are really good!

I faced only one issue with Kubuntu, getting 1440x990 resolution my Dell Latitude D620. Laptop configuration is as follows:

Core 2 Duo T7200 CPU (2 64-bit 2.00 GHz processors)
WXGA+ screen, 1440x900
NVidia Quadro 110M Turbo Cache
1GB RAM
Integrated bluetooth, usual ports and networking (Wi-Fi etc.)

The problem I faced was, NVidia Quadro 110M has 64 MB onboard memory and it can use shared memory upto 256 MB. Somehow the out-of-box driver with Kubuntu was not able to allocated shared memory which was forcing me to use lower resolution than 1440x900. I hate that resolution as there comes some weird shadow effect around text characters and even the desktop seems to be skewed a little bit.

I was also facing problem in having widescreen display and was forced to use 4:3 display. I tried to google for the problem but no luck there too. I searched for Adept Installer for “nvidia” and I could see two packages in the “System” category.

I selected the second binary driver (as shown in the image) and applied the changes. Later, I restarted the X-Server by (Ctrl+Alt+Bksp) and I could see the NVidia logo which was sign that the driver is in action. After login, I could go to “System Settings” and in administrator mode, I set my monitor to be “16:9”. Then I restarted the X server again. Later I could set the proper resolution to 1440x900 and it needed another restart of X server.

There might be some optimal way of doing that in one restart (e.g. editing the xorg.conf file manually) but I prefer to use wizards whenever possible to avoid syntax errors in the configuration files.

Finally I can use my laptop in 1440x900 resolution. :)

Indian Media: Are they treating news as commodity?

2007-04-20T00:00:00+00:00

From past few days we are looking that news channels are crazy over marriage of Abhishek Bachhan and Aishwarya Rai. I don’t remember that even world cup was covered with this details. Right from animated graphics of marriage invitation card to list of people who were invited were furnished in great detail. I don’t see any value of such news and coverage being added to lay man’s life. This neither give any moral boost nor gives any direction.

Day by day media is turning to be very irresponsible and spicy. Small and unimportant news are covered for longer duration sometimes even whole day! A normal person doesn’t even care about such news.

Today, Janhavi Kapoor (also known as Haya Razvi), a struggling model and actress claimed that she was married once to Abhishek Bachhan in private. Abhishek promised to marry her and now he denies that and getting married to Aishwarya Rai. This crazy lady went ahead and in front of media she cut down her wrist and collapses on ground. This is all needed for media to go crazy and they covered this looser actress for next whole day! Every hour the lady kept changing her statements like it was child play, news channels kept broadcasting this to the world. This was height of stupidity and irresponsibility!

This is not the first incident when some small event is getting enormous media hype. Intimate scene of Kareena Kapoor and Shahid Kapoor, Madhur Bhandarkar and Preeti Jain case and the king of all events “Prince”! Sometimes, really disturbing and violent images are transmitted for whole day without thinking much about its impact on normal person. When I talk about impact of news, first thing comes to mind is Mumbai blast. All channels were showing the half minute long footage which shows people are carrying a really deformed human body from busted railway compartment to some other place. That evening million people were travelling across the city. This incident happened and immediately all cell phone networks were jammed, phone lines were dead and communication media was stopped. Did anybody from these news channel thought of those women, children who were watching this ridicules half minute video footage and expecting their husbands, fathers, brothers and sons coming home? We are kind of forgetting the impact of news on normal person’s life. Showing the same violent footage over and over sometimes makes you numb about the event. The sensible feeling of humanity for such events becomes weaker and weaker. Audience starts anticipating more violence. There is no censor or commissioning body which decides what to be shown and for how much time!

Something stupid happens in corner of Mumbai and thousand of reporters start gathering around and starts asking million questions to people around the spot where incident happened. I wonder if they check any credibility of person with whom they are talking. They keep asking senseless questions about the event and create a spicy version of whole scene. There will be no wonder if they are cooking up some stories to make their headlines! To increase the TRP of the show and keep people entertaining, these people are finding weirdest and most useless events which a normal person even can’t imagine.

I see today’s news channels as commodity sellers. They are viewing news as commodity. Larger they produce better they earn revenue. I remember of a Bond movie which showcases an over ambitious News maker who first write news and then makes it happen. Over a period of time, his news starts shooting the bars of violence and set new horror marks. It will be really sad if we see something like that coming true anytime soon in reality!

Get Piped!

2007-03-02T00:00:00+00:00

Have you checked Pipes? I am really bad at predictions but I think this is going to be a great tool for bloggers, personal websites and even for commercials! The simple concept involved in pipes is really so powerful! When I first used Google Reader the way Google creates a stream of RSS feeds appealed me a lot.

Pipes is sequence of different modules which can do variety of things for you. You can ask for user input, sort the output, filter posts and lots of other things. This gives you complete liberty to process the RSS feed!

In two minutes I created a pipe. To demonstrate its power, wanna read my blog in French? Here comes the feed!

I used Babelfish translate service to translate the feed content.

I don't agree with you! How about that?

2007-02-20T00:00:00+00:00

I recently completed my one month in my new company. I hold my temptation to express my first impression of company for more than three weeks. I believe, to be fair you should spend at least a month before writing about new company. I was new to concepts of “Flat Organization”, “Agile Practices”, “Stand-ups” and “Sign-ups”. I was hoping to spend a fair amount of time in learning this process and be a part of it. Surprisingly I found it very easy to learn basics of that. Practicing them was even easier because of “Pair Programming” and other similar approaches. I am not Odd man standing out anymore. I am a thoughtworker! (Somebody at my work will ‘disagree’ with that! :) ) ThoughtWorks, is amazingly different than other organizations I have seen, visited or been. Following are the things which come on top of mind when I compare ThoughtWorks with any other companies.

No cubicles! – This is a blessing! Everyone sits around tables which mostly look like dining tables. All of your team members, are at a proximity where you can directly call him and have a discussion across tables! This increases a little noise in office but it’s worth it. I learned to work in that now.
Very energetic team which is very rare to see. In a technical discussion we always run out of time because of overwhelming ideas to discuss on table.
Agile at its core which is practiced very religiously!
Resources availability. Company which provides high-end latitudes to everybody for personal use. There are virtually not restrictions for personal / technical growth.
Every ThoughtWorker is passionate about what he/she is doing!

Comparing ThoughtWorks with any other company is like comparing apples and oranges! Their fundamental structure is different. I met few of very interesting people in new workplace. I had lots of technical discussions. The very strange thing was, in ThoughtWorks one of my friends opens and closes the discussions with only single phrase –

“Hey, I don’t agree with you! How about that?”

Fighting Developer Abuse

2007-01-25T00:00:00+00:00

Updated: Sorry but following video is removed from YouTube! :(

http://www.youtube.com/watch?v=sjnxFyJT2I4

Last day in Hyderabad

2007-01-13T00:00:00+00:00

As I announced earlier, I am going to join ThoughtWorks, Pune development center. For past 18 months, I lived in Hyderabad. This was a wonderful experience. People of different language, culture and attire! One of the best times of my life!

For all the good time I had, I owe thank to following list of people. Shriram (raamya) for his coffee breaks and wonderful thought experiments. You’re the man! Akshay and group for helping me in first days @ Hyderabad. Madan and Surabh for providing occasional company! :)

At last my crew from Akanksha; Rambabu for wonderful political, regional and personal views, movies and entertainment. I had lots of good discussion with you. Kameshwar for his humor and enjoyable discussions. Sridevi, Padmavati for their co-operation and fun. Sneha for her brain biting discussions! ;) Manoj for wonderful times having technical discussion and countless fun parties!

Radio City 106.4 / 91.1 FM for their wonderful music! Aparna (RJ for ‘Chalo Chalte Raho’ show) for her wonderful talks and entertainment. Thank you for giving me opportunity to speak me on Radio for 90 seconds on 6th December, 2006. I specially enjoyed your show everyday on the way to office. You are my favorite RJ for the year 2006! Gautami (RJ for ‘Nato Vastara’ show) for her sweet talks and cute behavior. You are a wonderful person with a great smile. I don’t know Telugu but just your show taught me to understand some of words! Being a Marathi person, I enjoyed your show everyday on way back to home!

Thanks to Sify for providing me internet services for whole year. Not the best, but you are good provider. Without you, this was impossible to have wonderful discussion with my loving friend Neelesh in USA.

Thanks to Neelesh for talking me almost everyday and talking on lots of topics. Great company dude! Hope to continue same discussions from Pune.

Final thanks go to my movie library person (Anna) who provided me hundreds of movies throughout the year! You made my holidays enjoyable! Thanks to Prasads IMAX, PVR Cinemas, Sangeet, and Raja for showing me great movies for the weekends and holidays.

Altogether, the stay at Hyderabad was wonderful and definitely added lots of memories to my personal life. I learned lots of things which will help me through the rest of time. Tonight I start my final journey towards Pune. Given a chance, will love to be a Hyderabadi! Again!

Thank you for the all persons who became part of life in Hyderabad. I always owe a big Thank to you.

Good bye Hyderabad!

Java on Multicore: Is definately good!

2006-12-24T00:00:00+00:00

This is another new discussion topic in Java circle. To get the background for this post you would like this good article. This post has some great comments so don’t miss that either. Followed by this and this article. There are different opinions about the topic. I studied Parallel computing & architecture as my graduate studies. So I would like to poke my nose in this topic. ;)

This discussion is circling around GC threads and JavaEE threading model. Although experienced bloggers have already touch length and breadth of the topic, I would like to express my view in this regard. Operating system designers and architects thought a lot about Uniprocessor and Multiprocessor execution methods. There is huge literature present on these topics on public sources. But multicore techniques are relatively new.

Java scales very well on multi-processor servers from long time. Since multi-core is similar to multiprocessor in great sense, there should be increase in performance. Although as Billy mentioned there is GC bottleneck but, that is something can be worked out through JVM only. If JVM can utilize different processor for GC then why not different core?

In addition to that, there are very sophisticated parallel computing algorithms and utilities which may increase the performance by re-writing the existing application or designing new applications to work better on multi-core architectures.

This problem can be worked out and not impossible. If Java is having problem so do all the other monolithic applications which were designed by keeping in mind uniprocessor / unicore architectures.

And there is always option to use libraries like Pervasive Datarush and Javolution to increase application performance without bothering about details of architectures :)

Most ignored class of Java!

2006-12-23T00:00:00+00:00

I am working with java for more than 5 years. From college to professional life, Java deserved an important position as a programming language. Recently I came across a conversation in my team. One of junior developer used ArrayList to store bean objects. To avoid duplicate objects in the list, he used a simple loop for comparing state of the beans.

I suggested him to use HashSet instead of ArrayList (but forgot to tell how HashSet works ;) !) He just changed data structure and came back to me with a complaint of HashSet not working!!! (This is true! :) )

Then I suggested him to visit API doc for HashSet and then he realized that he also need to override equals() and hashCode() methods from Object class. But all this didn’t stop there, he was not clear about implementing hashCode method despite of extensive API doc. It made me think about our ignorance of fundamentals.

Then I continued my search and conducted a short survey with my friends. There are total 11 methods in Java 1.4.2 version of java.lang.Object class. Most of them were not knowing how to implement hashCode() method. I am sure if they needed it badly, they will do as they are talented developers.

I found that people know complete Java Collection hierarchy, they know details of Vector class and Concurrency utilities. They know socket programming and design patterns but they miss Object class in major portion.

We tend to find other ways to implement the functionality which is provided by equals() & hashCode() by introducing extra loops or else. Now I don’t wonder if I found few more highly ignored things from Java!

Do you know any similar ignored classes / interfaces?

Java: Not so stupid series

2006-12-23T00:00:00+00:00

I always read the “Not so stupid questions” series on Java Today. It was becoming difficult for me to keep the questions list. Sometime, you need the questions list and you don’t find it; don’t you hate that? First thing came to mind was my blog! After all we are supposed to use these tools to organize our personal information; isn’t it? Blogs were invented for that,so I thought to make a list as a post and let others find the entire list in one place.

I googled for stupid questions list but I couldn’t find a single page enlisting all of them. :( But this problem is not there anymore. You would like to bookmark this page for your reference ;)

(Not So) Stupid Questions series:

Q 01: Should I try to declare more of my methods to be static?

Q 02: Some side-effects of String equality don’t make sense

Q 03: Some uses of the private keyword don’t make sense.

Q 04: I have no idea when to create a new package and what should go in it.

Q 05: When should I implement an interface, over inheriting from a parent class?

Q 06: How can you justify Dimension java.awt.Component.getMinimumSize() when Dimension does not implement Comparable?

Q 07: There are some weird Java operators I don’t understand.

Q 08: What’s the deal with serialVersionUID?

Q 09: I’m attending my first JavaOne. What should I plan on?

Q 10: Other than bundling my classes, what good does a JAR do me?

Q 11: I have a question about a Java feature. Who do I ask?

Q 12: Can I use the ‘Java’ name in an open-source project?

Q 13: Why do constructors have to start with a call to super()?

Q 14: Why is if (true); considered valid Java syntax?

Q 15: How can a constructor be private?

At the time of writing this post, only 15 questions were available. If any additional questions are appeared, I will update the post.

Tagging blogs: Are we missing something?

2006-12-22T00:00:00+00:00

Everybody is blogging! We get thousands of posts everyday and a wide range of topics and related information. I must admit that I can’t even read all the information which I receive on “Technology” tag per day. There is so much to share and so much to read. I came across Technorati page this morning

The tag “Foto” is in the popular list. What does that mean? I believe this word belongs to a non-English language. I found it very strange to see that tag. To help other people, there is other way, we can tag with “Photo” and “Foto” so the tagging is bilingual and other people also can find that very easily.

Tagging is the only way which make us reachable to post. We should tag nicely. Who would like to give me some guidelines of tagging your posts? Please share your comments.

Need for Speed: Carbon - Whatâs next?

2006-12-21T00:00:00+00:00

Need for Speed: Carbon is the latest release from EA. This game is way ahead of its predecessors in physics and modeling. The stunning graphics, great music and nicely arranged plot made my day. I have been playing NFS since NFS 2. Since then I played every release and my all time favorite is Need For Speed Porsche Unleashed. That game was something really stunning graphics, Porsche theme and way too real than its competitors! I was wondering, what will be the next version of NFS: Carbon? If we see the trend of games, it can be categorized as follows:

NFS 2 - Plain racing
NFS 3 - Pursuit
NFS 4 - Racing with better physics and realism
NFS 5 - Theme based game (Porsche lineup)
NFS 6 - Pursuit (Reloaded version of NFS 3)
NFS 7 - Underground theme
NFS 8 - Underground theme reloaded in much better way
NFS 9 - Mix n’ match of pursuit and career mode
NFS 10 - Underground theme revisited again with strong pursuit and territory based career mode

EA has always tried to put new race modes and latest cars. A constant effort to make NFS more than just racing can be observed in previous games. So what can be the next theme? I think a more realistic betting could be another option. Player earns and bets in racing. If you loose all the money, you are out of the career mode and your cars will be lost. The customization is the key now. Autosculpt in NFS carbon has done a great job. NFS carbon was having many other things such as crew members and some racing modes. I miss drag racing though! :(

What you think?

I want leopard!

2006-12-21T00:00:00+00:00

After taking a look at fantastic sneak peak of leopard I want to have one! Hope Santa gives to me on Xmas :)

going to be a ThoughtWorker!

2006-12-19T00:00:00+00:00

… Back after a while! I was busy in lots of things which you can notice now on. I have a great announcement on personal front. I am going to be a ThoughtWorker! I am very excited to be there.

I will keep posted about new happenings as things happen. For the time being, enjoy fresh look of my blog! :)

JDBC: Java DataBase Chaos

2006-12-19T00:00:00+00:00

“Write once run everywhere!” this tagline has won many developers. Java has been there for more than a decade and I love the way it ports itself to various platforms with little or no platform specific code. But, when it comes down to JDBC I feel like loosing my patience. We recently wrote a Database specific layer for our application, and that took lots of energy and time of my team. We first analyzed the behavior of different databases, their error codes and then code specific handling.

When, Java seamlessly works with multiple operating systems, what happens to that when it comes to JDBC? For any exception or error it only throws “SQLException” and a long database specific error code and message string.

I find it very common to encounter ColumnNotFound, DuplicateKey, IncompatibleDataType and similar exceptions in my application. Since, my application is designed to handle data at runtime; we are not having control over strict scrutiny of the data to check compatibility. If we do that, it will consume our lot of bandwidth to check input query data against dynamically generated table structures. So we directly push incoming data traffic to database without any strict checking. This leads to 98% hit ratio of proper insertion which saves our lot of processing power as well code efforts. To handle different kinds of errors we have inherited our custom exception hierarchy which wraps SQLException object. This way we always get proper exceptions and the design remains easy to understand.

So far there seems no problem, the real problem starts when we try to make our application compatible with other databases. We have to study each exception or error codes, their messages and other tons of database specific information which seems pretty difficult to manage. Although rare but if any database vendor changes an error code our whole application is for toss and we are forced to run a rigorous test cycle to ensure integrity and working of application!

This becomes very difficult if we are supporting large number of databases. So I was wondering, instead of hundreds of database specific error codes why JDBC doesn’t cover exception hierarchy. Then database vendors have liberty to use their error codes while making implementation of JDBC driver generic. Since complete JDBC exception hierarchy will be arranged in similar to IOException, if any user doesn’t want to catch granular exceptions they are free to use generic SQLExceptions.

This will surely help me to chop my code size by 1.5 K of lines :)

Rhino, BeanShell comparison

2006-06-28T00:00:00+00:00

Update (22/May/2007): I have also included Groovy performance as per Tom's comment. The post is here.

Recently I got a requirement to provide scripting capabilities in our product. A while ago, I was reading about JSR 223 and thought to have hands on the scripting capabilities of Java. While searching Internet I came across a page which states number of programming languages for Java VM. The most appealing options for me were Rhino, BeanShell and JudoScript. Ofcourse Groovy, Ruby and others are good too, but I wanted a simple and close to Java scripting for my product.

To get a quick start, I chose Rhino and BeanShell. JudoScript sounded like lots of in-built capabilities and since I didn’t want a Swiss-knife kind of thing with me so I chose to go with generic approach of Rhino and BeanShell.

In our case, developers will write some small routines to process sql resultsets. Typically the operations will be getting values in different columns or creation of sets etc. So my job is to provide a framework which takes sql queries dynamically and pass on the resultset to script. Script will process resultset and return the Lists or HashMap to framework again so that further processing can be done. So from my perspective resultset processing abilities was a winning point. For those who are more interested in computational performance analysis, I hope they will love to see this page.

Coming back to data processing capabilities, I set a simple test case with following steps:

Get Resultset
Note start time
Process Resultset using Java / Rhino / Beanshell code.
Note end time

I kept number of records as a variable in all of the executions. Thus I wanted to see the effect of increasing recordset size on performance. I started with 50 records and final reading was taken at 50000 records. Result of tests is as follows -

To help you understand clear the tabular representation of above data is -

Num of Records	Java	Rhino	BeanShell
50	31	375	250
500	110	391	453
5000	359	844	1391
50000	2375	5515	11406

On your machine the figures may vary a little bit but the ratio should be roughly same. BeanShell’s performance is dropped suddenly with the larger size of record set. On the other hand Rhino performs consistently and balances better with the size record set. For Java the ratio for 50:50000 records is 1:37 but for Rhino its 1:15. For Beanshell same ratio stands as 1:45! This clearly shows that there is consistency in performance of Rhino compared to BeanShell.

Although Rhino is winner from the performance perspective, I need to admit Beanshell is homely for Java developers. The communication between beanshell and java is almost seamless. For Rhino you need to understand Scope, context and little more Rhino specific things but for BeanShell there is no complexity. So from simplicity perspective I vote for BeanShell.

My advice to others will be, there is not much difference in both of the scripting engines. Normally neither we process (or at least not supposed to process) 100000 element sized lists nor so big datasets. So its mere matter of choice for majority of users. Go with BeanShell if your end-user is a Java Developer as its very easy to understand the operation and there is very little learning curve.

If you like JavaScript or needs millisecond accurate performance guarantee then choose Rhino as it proves to be consistent. I didn’t get chance to test E4x and other features but overall Ecmascript 4 is good. I will love to hear any other experiences regarding above two or any new engines.

Like New Reader Feature

2006-06-28T00:00:00+00:00

Ever you wished about a feature in product and it happened right away? How quick? Well in my case it took 7 hours flat! Yesterday night before sleeping, I tried to organize my Google Reader subscriptions. I wanted to categorize them with labels. Man, that was real pain! :( For each label it will say “Fetching you subscription list” and the pane size keeps on changing. That was really painful for me to arrange my reading list.

I opened my mail editor to send feedback to Google Reader that I want you to make it simpler! But I didn’t send the mail and fall asleep. I wake in morning, go to Reader and amazingly I could see the screen below :)

Man, that was quick! I happily organized the reading list and I have decided to be with Google Reader for some more time.

Kudos to Google team.

What Ubuntu User say?

2006-06-27T00:00:00+00:00

I was on vacation for last few days. :) I got chance to talk to once of my friend who is using Ubuntu on his laptop. He was using Breezy Badger distribution and the laptop was little old and upgraded one. I was curious to have a talk with him regarding his suggestion / feedback.

He was also going through same cycle of happiness which I experienced before! What made me more curious was, he was using a 3-4 years old laptop which was upgraded in terms of RAM and other hardware. That was IBM Thinkpad with earlier Pentium 4 processors. He said, he could run it smoothly and there were no hardware issues at all! He could install everything very easily and comfortably. That is a reason for toast!

He was happy with performance and also features. I would like to mention his one quote, which everybody will agree… “It is better than Windows XP!”

Open Standards

2006-06-27T00:00:00+00:00

I read today about new protocol for reading and writing data on web. Google has put the client libraries and other information on their website. I really appreciate the style of Google for providing open standards as well open application interfaces. We all know that Google earns a huge amount of money from their Ad programs but still, they are doing many things for free. Their every effort to make the web better place is clearly visible. Take example of their vast range of applications, (Gmail, Calendar, Spreadsheet, Reader, Suggest, Maps…) all applications are wonderful and I love them. Undoubtedly they brought the Ajax wave to mass. Many people were using XMLhttp before Google too, but it was not to masses, Google not only made it visible but they demonstrated how you can really simplify the applications!

Similar things should happen for quite long from now. We need help from giants on the web. They need to help building and realizing the open standards so that Web 2.0 and subsequent versions of it will be much more powerful than anybody has seen or imagined!

Are we ready for Web 2.0?

2006-06-22T00:00:00+00:00

Hushhh… last week was a real rough! Got some time to breath and felt to write a post. I am currently developing software which has a web-based application component. We tried to ajaxify it to give better user experience. We did it really well. There is lot of buzz going on since last few months around Ajax. Ajax is going to play a vital role in Web 2.0. In this post I would like to shed some light on Ajax only. Before we go too ahead, what we need to think is, are we really ready for this?

Ajax gives a very rich experience to user. Ajax gives richness of web and feel of desktop applications in true sense. While development, I felt that we are not that ready to hit the road. The first and foremost important thing is, the platform which runs the Ajax i.e. browsers are not supporting it natively. If browsers provide a clean abstract level implementation of Ajax then a new developer will not get confused by tons of different frameworks. Instead of number of frameworks, we need native support for Ajax and similar things. That will save lot much efforts of developer and in turn it will allow us to concentrate more on the business logic.

GWT (Google Web Toolkit), Tibco GI, Echo2 and Laszlo DHTML are good efforts towards simplyfying the development cycle. Still, Echo2 and Laszlo needs server side runtimes and GWT has better still Java based approach. I will really appreciate if IE or Mozilla support the Ajax just like form handling or any other task in the JavaScript. There shouldn’t be any headache to handle responses, marshall / unmarshall the XML etc. XML responses should be handled properly.

The fact still remains that there is huge compatibility issue with JavaScript. ECMAScript 4 standard is out and I don’t know why browsers don’t support it. Flex people were successful by implementing the ECMAScript 4 standard natively which supports E4X, the new way to handle xml, and other many powerful features.

Now I am not interested to start any flame war or address to JavaScript machos who will say Ajax is simple though. Still majority of people find it painful and we run into trouble many times.

To be really ready for Web 2.0 we need Web 2.0 ready web-browsers. Instead of handful of frameworks we need native support from browsers. ECMAScript 4 should be supported by all leading browsers. Most of all, I plea to Microsoft for not making their own standard for browsers. That really creates huge application portability issues. If they really want to create own standard then let it be public so that other people can get benefit out of it!

Peace.

Will Callisto help eclipse to beat NetBeans?

2006-06-13T00:00:00+00:00

It’s been a while since Callisto has been announced. I don’t have statistics about how many downloads happened for Callisto since first release. Most of you might know that, Callisto is collection of 10 most popular eclipse projects which will be released as a distribution. That really helps! In fact, I was waiting for such thing from so long.

Eclipse has hundreds of plugins and each of them has different dependencies. Once I tried to update WTP 1.0.7 in Eclipse 3.0.1 and every time it gave me a version mismatch error. I did not even know how to resolve the plugin version dependency! :( That was good example of plugin hell! The same feeling I had in Windows when it used to say XXX.dll has this version and you need XXX.dll with some different version. Now where I am supposed to get the newer version? I finally downloaded wtp-1.0.7-all.zip which has all things in-built from eclipse 3.1 to wtp latest version. Then I faced another problem, after downloading WTP, I had two versions of eclipse on my machine!

Callisto is good effort to address all such problems. Many times, I ask one of my team-member that, “Hey pal, you don’t have xml editing feature installed in eclipse? Go get WTP and then it will allow you to edit/format xml in the editor.” That guy spends almost half-day to build his workspace environment for editing couple of xml files. :( That’s even worse example of increasing productivity by using tools!

Now Callisto will give me one URL which should broadcast throughout my team and within couple of hours everybody’s eclipse workbench will be restored to latest version of top 10 projects from eclipse.org. :) For those who don’t know which are those top-10 projects here is the list:

This was needed by eclipse from long time. One URL, one distribution and you are ready to go. Really it should be that simple!

I was wondering will it beat NetBeans. There are few things which NetBeans is doing good. Good examples would be Enterprise pack and Profiler. These packs integrate very tightly with NetBeans. WTP needs that simplicity. Be it anything, BPEL, Glassfish or your favorite relational database, they just get part of IDE! That’s really a great feature of NetBeans. Visual studio always has such kind of integration with all available tools. NetBeans has exactly delivered same power.

I like Xml Schema editor, Schema refactoring, Web Services development and lots of other features which are great! I tried almost each type of project creation in NetBeans since my last post. I must say, its worth to give a try to NetBeans by all those people who love eclipse.

No! No! No! I am not advocating NetBeans, I am just saying just try another project which is powerful, next-generation and perhaps best Java IDE on planet! :) And above all, it’s free! So get your latest copy of NetBeans, Enterprise Pack & Profiler and start working on that. Remember it’s just another open-source project so there is nothing I can give to this community than my praising words! Soon I will get involved in editor development of NetBeans. (My current schedule doesn’t permit me to blog too :()

Cheers Eclipse & NetBeans team! I really owe you a, “Thanks!”

Google and Next Generation Search

2006-06-11T00:00:00+00:00

Google researchers have published a paper (PDF) which was awarded as best paper in Euro Interactive Television Conference last week. The research says, with an audio clip recorded using simple computer microphone we can find relative web-content. They used this technique to demonstrate a simple audio clip from running conversion can find relevant web content. The ambient noise doesn’t hamper the search quality and accuracy. Amazing! I think, we are not very far when we could implement a voice / video & image search. There are billions of phones which have capability to capture images, record sounds and video clips. Simplest use can be capture a photo of a person to whom you don’t know and send it to online search engines using web-services. Then the services will find the images which contain the person and you will have all the information about the person in seconds (provided he has public existence on web). Though this sounds like movie scene from Matrix or Mission Impossible but it’s very useful for guys like me who forgets people faces. (I have many incidents of talking to people to whom I don’t remember anymore and just pretend that I know them ;))

Another application is very important for me about audio search. I heard a music snippet from an online movie or some place and I don’t know what kind of music that is or any artist information. Now, how should I use search engines to find the song name, artist name etc. The online audio search may be very useful in this sense, send an audio clip and you will receive list of songs which contain that audio portion. Amazing… I am so much excited to see all these technologies in action.

Although there are couple of issues regarding media protection, online privacy and other things to be solved but compared to facilities gained by end user, those issues doesn’t seem big.

NetBeans Vs Eclipse

2006-06-10T00:00:00+00:00

Let me clear in beginning, I am strong fan of Eclipse. I have been using Eclipse for past 3 years and its very powerful. I love it for whole lot of reasons which I can’t mention here.

In Javaone conference I heard a lot about cool NetBeans demos and features. I decided to give it a go. I wasn’t expecting much from NetBeans. I installed the IDE, enterprise pack & profiler. To begin with created a simple web-application project. Integrated Glassfish / Sun App server really makes testing extremely easy. Just out-of-curiosity I clicked on the debug button with a breakpoint in a bean and surprisingly it allowed me to debug my web-application. Earlier I used to spend hours of time by writing log traces and analyzing them to debug application. Such debugging feature comes really hadny and makes coding extremely productive.

Beyond that, UML modeling (both way), testing, Web-services & BPEL integration makes NetBeans a real powerful JavaEE IDE. Many wizards are available to perform mundane coding / modification tasks. It is aware of Java 2, 5 & 6 syntax hence, it makes development easier for any version of Java. JSF & Struts are integrated so no worry for web developers. But I need to admit the editor of Eclipse is far better than NetBeans. For everything else the NetBeans is just superb!

Waiting for more in coming days. :)

Ubuntu - Painful re-install

2006-06-09T00:00:00+00:00

I installed ubuntu 6.06 LTS on my machine. I was on the ride of happiness :) I liked everything, right from the starting sound to shutting down screen! Some notable features are -

Package manager - This really helps. You can choose which packages / libraries to download / install and just download them. That really gives you lot of choice to work with.
Device support - I experienced the seamless operation of many devices. My thumb-drive and digital camera were instantly installed and ready to use. No more website searching for drivers.
DVD, Mp3 & other playback - Though I was required to download libraries to run non-free media formats, but that’s OK. I could finally listen to my songs while I work ;)

This list could have many more points which you can easily see from other ubuntu related blogs. Enough praising!

There are still areas where Ubuntu needs to work-out. I felt a major reason was re-installing ubuntu. All the packages and libraries which were downloaded from internet are totally lost! :o You need to start-over again for setting up those applications! :( On the top of it, to increase our difficulties the names of libraries and applications are very difficult to remember or cryptic. To play DVDs you need to use “xine” and for playing songs “Rhythembox”. Why we don’t use simple names like MusicBox or MoviePlayer? Library dependencies, names and other problems might be solvable by geeks, but normal users will switch back to Windows than wasting time on googleling for How-To’s!

Even those people who have knowledge of installing will not do because its painful and time-consuming task. There should be a way by which we should be able to restore our application profile or at-least you should be able to generate a automatic script which you run for next time and restore your desktop as before.

Moved to New Place!

2006-06-09T00:00:00+00:00

I have been writing for a while on Blogger & JRoller. But, I wasn’t able to continue for long. I needed a break! Finally I decided to settle down on WordPress. This time, I have planned my blog nicely. I will keep posting in each category on regular basis. I have lot of things to share and document. You can expect few articles on technology and some fun paragraphs from my life. Keep looking :)

Love @ first sight

2005-12-04T00:00:00+00:00

Few days ago in some blogs, I read about Ubuntu. The strange name made me curious to know more about it. On the website there is enough information abuout their name and their purpose. I think the initiator of project has taken the marketing strategy of some of successful Linux distributions to his heart.

Well, open source doesn’t mean free beer. You may need to pay for it. Its all about freedom to use it. Well, I will not drive myself in explaining whats the open-source. But my whole point is, there is nothing wrong done by Red Hat, Novell, Sun, SuSE and Mandrake by selling their distributions.

To give a serious try, I just downloaded the ubuntu for AMD64 platform. The whole distribution was only on a single CD. You can find a live version of Ubuntu too! I burned it and started the installation. Its having very less differences than the Red Hat installation version. But without much questions it got installed into my system. I liked the way it got installed. Initially it installed only necessary and very frequently used packages like browser, mail client and other stuff. Most of the times, Red Hat made me impossible to download their distribution due to its heavy size. It will deliver you all the packages whether you want them or not. And at the time of installation you need to swim in the sea of package lists to identify what really you need. Otheriwise spend around 6-7 GB space on your hard disk for complete installation. I really dont mind to have all of them, but there is no choice, and thats only complain. Everytime I need to run for the burned CDs from some vendors or order them. I could easily add/remove programs from the system without any command line operation. I think, this is good for the persons who are habituated of using Windows control panel.

I was very glad to see that without any complex with windows, ubuntu team included MyComputer into their desktop. Actually, its needed. Non techie people find it very hard to locate their files on linux based systems in the beginning. The lay man’s problem is to learn linux by making anologies with windows and that creates very big problems sometimes. When linux has a cleaner implementation of file system people want to make it complex by splitting everything into drives and in MyComputer as root folder. I find Linux or any unix has very simple concept of ‘/’ file system. Every device is a file. If you understand this phenomenon very well, you dont need the concept of MyComputer and its related things.

But ubuntu team has finely integrated the things which will make any beginner or windows user to adjust with it quickly. My network card, graphics card and sound card was detected without any problem and everything started running smoothly. I simply fallen in love with ubuntu in first sight. Belive me, ubuntu is great desktop for business workstation. Where you can have office suite, email client, higher security than windows and competent software bundle. Worth to give a try. I think, there is a little room for improvement in the themes section. More choice for user interface and sound alerts should be provided in basic installation. Rest things are simply rocking.

Open Source Wave

2005-11-14T00:00:00+00:00

Open Source is no more geek’s job! Linux and other OS projects are already working in this model from years and now many commercial players are joining ‘em. This is really interesting. I get to much surprised when I see Microsoft is also trying to join OSS. Well, already they have made their couple of projects as open source and now they are also commenting on the “Shared Source”.

The shared source is something more liberal than the closed source but not the fully open source. Still, this may be first step towards the open source. In last week, in some conference Microsoft spokesperson also mentioned some drawbacks of the open source (Click here for actual news). Now that’s partially true. To add the proper value, you may need to charge customer sometimes. But hey, that’s one clause under open source too. You can modify the code and sell it but you should also provide the modified source code. I don’t see there is any wrong thing done by the Red Hat about which people are talking in the news. Moreover, we shouldn’t at all forget that Red hat’s remarkable contribution in making Linux popular and they contributed a lot by standardizing a Linux distribution. So if some people don’t like that Red hat made few of its Linux distributions paid, I disagree with them. Red hat has done nothing wrong and still they are following the OS license. One more news struck me hard, that came from Sun Microsystems. A legendry operating system with more than $500 million research is now available under open source license. (See Tom Goguen speaking about this on this). Now that’s great, a real open source and liberal license. There might be various good or bad reasons about why Solaris is being made open source. Being an end user, I enjoy its presence in the open source. Now I believe there is a good competition to Windows Server side by Solaris joining OSS. I will not say people will wipe out commercial server line by Solaris or Linux but certainly it will have effect. I believe the open source nature of Linux made it very popular and this popularity contributed a lot in the fast pace development of Microsoft too. Without such a tough competition Microsoft wouldn’t have seen this day. Welcome aboard Solaris!

Moving to OpenOffice.org

2005-11-14T00:00:00+00:00

I just had a look at OpenOffice.org 2.0 and it rocks! You just see how many features are loaded there. Moreover, its free. I have decided that from now on I will install only viewers for Microsoft Office and for my day to day office purpose I will use OpenOffice. There are few differences in them like the macro definition language. But almost any office document can be opened into OpenOffice. Good work OpenOffice team. Just have a look at here.

Market Buzz - AJAX

2005-11-14T00:00:00+00:00

In a week, I at least see around hundreds are blogs being published on Ajax. On “Technorati” in top searches also Ajax is present. When people first saw a freaky interface on google maps and google suggest, it made a click. How does it works? There came the AJAX into picture. AJAX is giving now ability to send/receive XMLHttpRequest and XMLHttpResponse from the client/server to and fro.

But any facility comes with many challenges in its implementation. When I get ability to fire any arbitrary number of requests from web page in background there are few issues of which we need to take care.

Connections over! - When a client application can make any number of requests the server traffic is going to increase. All application servers should scale to accommodate the increasing request demands. Or else the whole fun of the application goes with a Blah!
Komplicated! - While AJAX gives me some chunky wunky features. I also make my user interface much complicated. Developer need to take care of each small and big error response which arrives from the server. Any mistake in handling responses properly makes a very poor application.
Steal my information - Previously, if a innocent user visits a website with spyware or some malware. The application needed the user permission to install its ActiveX control or something like that at client side. But now the capturing and sending of the information became whole lotta easier. In the background any application can send arbitrary information to the server in XML format. I am sure, we can stop this by installing proper firewalls and disabling some features. But one more possibility of threat couldn’t be denied.

Still, I love AJAX. Compared to all the drawbacks, it gives me excellent freedom to manipulate the user input. Looking for more and more work in this area to improve its functionality. There are few frameworks available which works on AJAX like Sarissa but better frameworks are needed. Those who want to develop a good application should have look at Echo2 also. This is a good platform distributed on GPL license. Even the Echo2 Application studio is also available to author echo2 based application in eclipse.

Coocoon - Web Development Framework

2005-11-01T00:00:00+00:00

I would like to go with Cocoon for my first post on this blog. My first encounter with Cocoon happened, when I was in design of a web based application. I wanted a simple framework which doesn’t hogs much server side resources and gives me ability of a very dynamic and configurable web application.

I saw into Cocoon and the concept of Pipeline appealed me a lot. Its really like a assembly line, you can see your content is fetched by business logic, serialized to various forms, transformed to various formats and finally published over web. More to it, you can customize the way pipeline behaves. By just modifying a simple XML you can change the complete behaviour of the pipeline. Thats complete other story that I couldn’t fit the framework in my needs as I was needing more than just a web-development framework. But there were few things which I wanted to highlight about cocoon:

Continuation: Cocoon gave me ability to use the continuatoin in the web pages, this is a ability which I needed from many days, but I was not knowing somebody has already implemented it. This is something like you write and execute a function.

There is a excellent information on what is continuation in cocoon on this page. Just click here.
Content-Logic Seperation: This is something which was tried many times before but very few people got success into it. You write your logic somewhere else into java classes or something you like. Configure your control flow and thats it. You are ready to transform your output into the way you like. It allows to you to seperate the view tier completely from the business logic. You generate one output from the business logic. Convert it into XML (well nowdays cocoon has many inbuilt serializers for many interfaces.

Ending up the list here is unfair with cocoon. You may like to visit and see the actual product and its features on here.

All in all, another excellent product from Apache. I know after reading the list hardly you can resist to download and try it.

$thought | blog

iCal attachments and emailing woes

Brief about our solution

Issue 1 - Email clients won’t welcome us

Issue 2 - Despite of writing above code, email clients still didn’t respect us

Issue 3 - Occasionly our users received error in Outlook

Issue 4 - Outlook used to show 3-4 hour differences in meeting timings

Symfony Worker Tasks

Issue 1

Issue# 2

How I solved this issue?

How does it solve the problem?

Programming language and Developer's Perspective

Mac OS: netstat and pid

Ruby: Insecure World Writable Directory

How to display local time to users in browser?

Save a read-only file in VIM using sudo

Getting Things Done

Attempt 1 - ToDo lists and Time Tracking

How did that go?

Attempt 2 - GTD

How did it go?

So what improved this time?

Moved from TextMate to MacVim

Hacking with Arduino

Android: Downfall or Prosperity for Google?

Fragmentation

Hardware Differences

Where Google Wins?

DNS-323, PS3 and UPnP

Rewards of "Frequent Check In"

Nice utility for finding classes in Jar files

Export CSV from MySQL Query

Delete My Account

Things to Ponder with Alfresco (Part III: WCM - Improvements)

Ubuntu: Cisco VPN Client installation on Intrepid

Region Locked DVDs and Mac Book Pro

Things to ponder with Alfresco (Part-II: Workflow Design)

Things to ponder with Alfresco (Part-I: Content Model)

Ubuntu: Firefox 3 beta and unhappy users

Alfresco: Workflow managed by JavaScript

Guarding in JavaScript

Alfresco: Server side JavaScript, anyone interested?

Java: static and transient keywords

Don't put JavaScript validation, I'll Firebug!

Change of 2008

NANT: The webservice at 'localhost' does not exist or is not reachable.

Dhishkyaanv: Are you game?

Ubuntu: Will Skype jingle?

Ubuntu 7.10: Multiple monitors chaos

Ubuntu 7.10: Early Impressions

Shhhh... Gutsy is coming!

Don't say 'Yesterday' in a stand-up!

You are not allowed to book Indian Railway tickets from Linux!

Linux Myth Buster: What is difference in Server and Desktop Linux?

Why this question is here?

Actual answer begins here…

What can I expect from a Desktop distribution?

What can I expect from a Server distribution?

I am still confused, which one is right for me today? Will I miss something if I choose a wrong distribution today?

Feature Request: Ubuntu

Bollywood is Bollywood!

Update on performance of Scripting languages

Kubuntu 7.04 Feisty Fawn - WPA support for Intel ProWireless

Kubuntu 7.04 Feisty Fawn - Getting right resolution

Indian Media: Are they treating news as commodity?

Get Piped!

I don't agree with you! How about that?

Fighting Developer Abuse

Last day in Hyderabad

Java on Multicore: Is definately good!

Most ignored class of Java!

Java: Not so stupid series

Tagging blogs: Are we missing something?

Need for Speed: Carbon - Whatâs next?

I want leopard!

going to be a ThoughtWorker!

JDBC: Java DataBase Chaos

Rhino, BeanShell comparison

Like New Reader Feature

Need for Speed: Carbon - Whatâs next?