{proton: fever}

Annoying Geo Targeting

noreply@blogger.com (Varun Mehta) — Thu, 18 Mar 2010 01:06:00 +0000

Ever see those annoying ads on websites you are visiting, which advertise services or locations around where you are currently surfing. How do they figure out where you are??

Such kind of targeting in the online media world is called as "geo-targeting". Every computer or device that connects to the internet gets an IP address assigned to it. These addresses are assigned based on your location, so if you are in North America your IP will start from 63.*.*.* or 64.*.*.*. If you are in Asia, your IP address will range in 202.*.*.* or 212.*.*.*, so on and so forth. These addresses are further drilled to second and third level based on the subnet mask.

When ads are placed, to target people in a specific location, they get help from the IP address/subnet, and then the spam ads really want to desperately reach you, they use a trick like the one mentioned in the cartoon below...

I hate to analyze xkcd.com, but this one hits right on target my work domain.

Hot Deployment issues

noreply@blogger.com (Varun Mehta) — Wed, 17 Mar 2010 01:02:00 +0000

The build always makes it to the staging and production server after a series of rigorous tests. This ensures the build is complaint with the requirement and that none of the old features have been compromised in any way. Once in a while, a rare, but an annoying "Scope Creep" scenario does happen.

Just minutes before the demo some one would realize a missing feature or patch which missed the commit deadline and did not make it to the server. So as a quick temp fix we just deploy that file to ensure the code does not have to go through the grueling tests again. Some of the smart developers don't commit these hot patches to SVN or merge it with the patch branch, losing the feature again.

Writing a shell script or a program which can check the MD5sum of all the files deployed, with the files about to be deployed can give us an idea if anything had been hot deployed. Hudson has a neat trick called as fingerprints for jar files, which is similar in the same aspect of checking MD5, so you know if you need to redeploy all the jar files or only the changed ones.

Project Management and Issue tracking tools: Redmine vs Trac

noreply@blogger.com (Varun Mehta) — Wed, 17 Feb 2010 22:38:00 +0000

I've been beating around the web searching for a good solution, that can scale from a microISV team to a mid sized team. We've been using svnrepository.com for sometime. They provide svn and git as a SCM, also trac with multiple plugins for project management.

Off late they've upgraded to a new system which offers redmine, along with trac and also has support for mercurial. After searching the web for a few minutes, I hit upon this article at stackoverflow;
Trac vs. Redmine vs. JIRA vs. FogBugz for one-man shop? Check through the article, you might find it useful.

If you are looking for online svn hosting, this website might be useful to you. http://www.svnhostingcomparison.com. It provides you with multiple options available in the market. You can filter depending on the cost, disk space and other features.

Adhoc or Agile

noreply@blogger.com (Varun Mehta) — Tue, 16 Feb 2010 22:59:00 +0000

I wanted to hot link to an old post by Simon Brown, but I believe he's dropped his old blog and transformed into a more consultant type blog. I'm copy pasting the old post here, with link to the original post.

In the vast majority of cases where these phrases are used, agile has mistakenly been used to mean that the development process in use is ad hoc or even non-existent. It's not hard to get to the bottom of what people mean by agile, particularly in interviews, with a few simple questions. Usually, the flow of conversation goes something like this.

Me : So you're using agile ... how are you finding it?

Them : Good thanks, we've been doing XP for a while now.

Me : Oh right, which practices of XP are you following?

Them : We've got the users sitting near us and we're doing test-driven development. We don't do pair programming.

Me : How are you managing your iterations/time boxes?

Them : We do a release whenever a new feature is ready. The users test it and then tell us how they'd like it changed. Agile is really good for capturing this early feedback. We just respond to whatever the users want and they love it!

Me : So how long are your iterations?

Them : Oh, that depends on .....

Me : So then, how are you finding test-driven development?

Them : We're really liking that. It's such a great technique for understanding exactly what an API should do before you code it.

Me : Excellent. What sort of unit test coverage do you have?

Them : Oh, I don't know, but it's probably over 50%.

Me : 50%? Do you use a code coverage tool? Do you write unit tests before writing all of your code?

Them : No, we've not got around to using a coverage tool yet. We sometimes right tests before the code, but we usually don't have the time to do that so we do it afterwards.

Me : How do you ensure quality?

Them : We're supposed to do code reviews, but we don't usually have time.

Me : So then, how do you feel agile methods differ from others, such as ...?

For me, agile is about reducing the ceremony associated with software development while maintaining very tight control over the structure of the process and quality of the deliverables. Unfortunately, many people are now using "agile" as a buzzword to describe their projects, although in reality these projects are low in ceremony, structure and quality.

Hmmm ... are you really agile or just ad hoc?

Original Link: http://www.simongbrown.com/blog/2006/11/21/agile_or_ad_hoc.html

Customizing JIRA workflow

noreply@blogger.com (Varun Mehta) — Thu, 21 Jan 2010 22:05:00 +0000

Like other cool companies we also use JIRA for issue tracking; and we are agile using it. JIRA has posted a very useful article on how to go agile using JIRA [http://confluence.atlassian.com/display/CONFEVAL/Using+JIRA+for+Agile+Development].

Since JIRA gives you an option to customize the workflow. We've added some more details extending the normal 4 step JIRA workflow to a 6-7 step workflow.

The steps are closely related to what we follow in a normal development cycle, something that is not only natural to a developer, but also helps managers track them. A look at the major steps involved;

Open Issue: Issue created by the lead and assigned to a responsible developer.
In Progress: Once the developer starts working on it, they mark is as "In Progress", to let others know he's working on it.
More Info needed: Along the way is any information is needed from the business analyst or the decision maker, reassign the issue to them and mark it as "More Info Needed" who when provides more info, assigns back to you. Why the run around; well helps you document for future reference. Back "In Progress" the developer continues with the quest to complete it.
Development Complete: When finished coding, update the status as "Development Complete". The module is code complete but not unit tested for yet!
Unit Tested: Supporting unit test was written and integrated with the Ant/Maven build to run in Hudson/Cruise Control/Bamboo

There are also cases, when there are no unit tests needed or possible for the code, the developer can mark "No Unit Test" in that case.

QA Approved: This is where we get a bit waterfally, but only cos we think unit test may not always provide complete testing justice of your application. Here the QA is responsible for ensuring the business requirements are met as specked out.
Resolve Issue: Good to go, go to staging, pass some robust abuse, and if all goes well resolve the issue.
Close Issue: The build has been released and we are generating release notes of what made it and what was missed. Close the issue and let it rest. If there are any repercussions reopen the ticket.

*There is some debate about the Unit Test being written first and then the code; so you know what to expect; but in this case, it's a state for the developer and the manager for better understanding.

On how to create a custom workflow, please refer the JIRA documentation

Version Control Policy: Best Practices

noreply@blogger.com (Varun Mehta) — Tue, 27 Oct 2009 20:21:00 +0000

Your code base is always under version control (if it's not, it's high time you did!), and developers are always scared about taking updates or committing files, under some supernatural fear, that an update will wipe out their local changes. This causes a lot of trouble in the team, when some developers have not updated their code base since a long time. I've compiled below a checklist of best practices.

Before you check in your code to subversion, here are a few points to follow.

Ensure that there are 0 (zero) serious CheckStyle, PMD, FindBugs error (warnings should be reviewed). If you need an exception
Checked in Code must build with all dependencies (not just your module).
Checked in Code must not break unit tests.
Checkins without comments should be noted as a build break caused by the developer.
Before committing your changes to repository, it is advisable to synchronize the files, run an update and then commit your changes.

Some best practices for not getting your code out of date.

Synchronize your local SVN repository using eclipse instead of external third party tools.
Update all the projects religiously every morning, irrespective of you having to commit your code that day or not. This will ensure you are always working on the latest code base.
Instead of blindly running an update from eclipse, follow the following steps.
Right click on all the projects in the project explorer, which are under version control
Team --> Synchronize with repository.
If it asks you to open-up the team perspective view, say "yes".
At the bottom right corner of your eclipse you should see these three icons in blue, grey and red.
Blue stands for incoming changes, grey outgoing and red as conflicting.

If you have the count of red more than 0, then your repository has conflicting files.

Under Synchronize tab, select the similar "red arrow", the list below is filtered to show only conflicting files.
Double click on the file to open in compare editor, fix the errors, and then update the files.
You can safely run an update on all the files with the blue incoming arrow icon.
Under any circumstances, NEVER copy the .svn file from another use to your repository. This folder contains the svn connection and credential information.

Changesets

Most if the commits in the system span across sub-modules and generally contain multiple files. When "applying a patch", "fixing a bug", "adding a new features". Always commit these group of files as one commit and not individual files, with the same comment!

Example: Jira ticket: ABC-123, introduced a new feature which involved changes in totality of 15 files, don't commit the 15 files one by one, but commit them all together as one group (changeset) of 15 files with proper comments.

Every time you hit a commit, there is a new revision created. After committing 15 files separately you've created 15 versions (or changesets), and incase an issue occurs and you need to revert the changes, you'll have to revert the revisions one by one. If these changes were done as a group, all you need to revert the one big changeset. Group commit also reduce the noise level and useless incrementing of revisions in SVN (any version control).

BUT, Each ticket needs to have its own commit changeset, not grouped with other tickets, unless there is a dependency between the tickets.

Futher reading: http://subversion.tigris.org/faq.html#changesets

Software Versioning

noreply@blogger.com (Varun Mehta) — Sun, 27 Sep 2009 18:56:00 +0000

Software versioning for the end consumer is a very vital part of the software release cycle. It is one of the primary identifiers for consumers to report issues against in a software. It also helps the development team to identify, if the features/bug fix made it through the release.

We follow a similar pattern, which can be useful for you too. The version number is strongly tied to the SVN (also valid for other version control systems) branching and tagging convention. Hudson (CI server) also plays an important role in stamping the build number.

Convention: productRelease.majorRelease.minorRelease.patchRelease.buildNumber

productRelease: Also called as the platform release, generally used when there is a huge overhaul of the application framework or platform.
majorRelease: Is the interim release like a Q1, Q2 release. Q1 would be called at 1, Q2 as 2, when a huge set of features is released, then we increment this version number.
minorRelease: Since we follow a two week sprint, after every two weeks the counter will increment by 1.
patchRelase: So the minorRelease made it past QA, but has some issues, when these issues are fixed and redeployed as a patch, the counter is incremented by 1.
buildNumber: The build number is generated by Hudson and will be appended as the last id.

So how does this affect the usage of Subversion. Before we get into those details, there are a few ground rules for subversion you'll have to follow, once every Release is released, tag it and create a branch to maintain it.

Considering the starting stage, mark the platform release as 1;
Once we release the first sprint; a new branch called 1.0.1 will be spawned off and 1.0.2 will continue on the trunk.
Any patches to be released on 1.0.1, will be further branched as 1.0.1.1 and will be supplied to QA on blocker fixes post 1.0.1 release.
Once 1.0.1.1 is released, it'll be merged back to 1.0.1.
If further more bugs are found, they'll be released at 1.0.1.2 following the same process.
No one will dual commit the changes from the branch to trunk, if the fix is for a patch, it'll only be committed to the patch branch.
Once the patch branch has been cleared by QA, it'll be merged to the minorRelease branch.
Once the second patch has been cleared by QA, it'll be merged to the minorRelease branch and then the minorRelease branch to the trunk for the next minorRelease.
There should be not not more than 2 patches for one minorRelease, (considering a 2 week sprint)
The second patch merge date is a 2 days maximum before the next minorRelease.
The changes from patches will only make it to the minorRelease branch or the trunk, via branch merging.
Every release will also result in the tag of the same name.
The branch merging should be performed by all developers in a cyclic fashion, so everyone is well versed with the process.

Along this route, Hudson will keep building, automated testing the build, and this build number will also hold reference when releasing a build, every subsequent build need not always be released to QA, there can be 20 Hudson tests and builds before being released to QA.

The numerous branching and merging might sound like a overhead, but will result in a more cleaner release process. Patch releases should not be one class or js file hot-patched, but a complete new fresh build, for redeployment.

Online Recruitment Firm

noreply@blogger.com (Varun Mehta) — Thu, 30 Jul 2009 13:36:00 +0000

One of my friend’s over the past few months has been speculating starting her own recruitment firm, but was not sure how to go about investing & setting it up. So we got together and brainstormed a few ideas. We created a list of tools that are of utmost importance for her work.

It turns out Microsoft Office, MS Word more precisely is one of the most sought after tools by recruiters, followed by Microsoft Outlook for email.

95-98% of the people seeking jobs send their resumes in MS Word format.
Time immemorial email is one of the fastest and most efficient way of bulk communication.
Using MS Office is expensive, and word documents are a prime target for carrying viruses. We needed a document reader, an email client and a strong anti-virus software.

Not sounding too stingy doing business running of the mill free gmail, yahoo or hotmail account, we invested in a domain name and got ourselves a dot.com so we could user @ourdomain.com. We got Google Apps and used our domain name to host it, and we were ready as a recruitment firm.

Let me explain our choices; We choose the Google Apps package, because it comes with 3 major tools we needed.

Gmail: One of the best email clients on the web, supports grouped conversation, amazing search, apply labels.

The grouped conversation helps you track the posting for which you’ve received a reply.
You can apply labels such as Java, .NET, ruby, finance, hedge funds etc.. and classify incoming resumes. Applying multiple labels helps you pick up users from a wide range of skill sets faster.
Direct integration with google docs.
Amazing anti-virus policy. The attachment is scanned when the document is received and also when we download/open the attachment

Google Docs: Online document reader, allows you to export as PDF and MS Word.

No danger of being hit by virus.
Allows you to collaborate documents online, so if you need some more clarification from the job-seeker or need them to update the document, you can always share the document and ask them to modify it.
Once done, just export the document in MS Word format and mail it to the company. Saves you a lot of time and effort running behind the job seeker and tracking which email as the final version.
One of the main reasons for choosing Google Docs was “upload via email“.
Google docs gives you a long, obscure email address which is difficult to remember and, if we mail a word document or any text document to it. Google docs creates a word document for us online. We went ahead and created an email alias resume@domain.com which received resumes and stored it on the google server.

Google Calendar: A free bonus! Helps you schedule, track and invite others to your meetings.

One of the biggest benefits is you don’t need to carry your laptop around not sure when you receive the next email and will have to forward the resume, also tomorrow if the hard drive crashes, we have nothing to worry all the data is backed up on the google cloud, just pick up a new machine log on to the browser and we are back in business.

Staging environment using VMware

noreply@blogger.com (Varun Mehta) — Wed, 03 Jun 2009 03:45:00 +0000

(This document is aimed more towards Java & VMWare, but the same can be replicated for any other language & environment)

Abstract

You have a team of developers working on different modules for a project-product which are inter-dependant. Each developer diligently writes unit & integration tests supporting their code. You want to set-up an agile test environment to run the unit tests & staging server for integration test, but purchasing hardware for multiple machines is a constraint.

The following article provides a guide on how virtualization tools like VMware can be used to set-up staging or QA environment (Agile).

Set-up
You can either install Windows or Linux (we are on an Ubuntu system) as the HostOS. Use a 64-bit distribution.

Download VMWare server from [http://www.vmware.com/products/server] select the distribution type depending on your OS. The guest and the host OS can be completely different, they need not be same there is no relation between them.
eg: You can install Windows as host and Linux as guest or vice-versa.

You can either create the virtual machines all by yourself using the VMware step-by-step wizard, or download 'appliances' created by others from the VMware site [http://www.vmware.com/appliances], this saves you the initial installation effort.

When you create a new VM, you are also asked the default networking connection type. NAT and Bridged are the most common options one selects from. Use NAT if you want the VM to talk within its own subnet only, or Bridged if you want other machines on the same subnet as the host access them.
eg: Your host machine is on 192.168.10.50

Tip: If you installing the guest yourself, you can create one master guest, with all the common applications configured (eg: OpenSSH server would be great to have on all the guests for remote management or java), you'll have to change the host name after that.

Power up each of the VM's and install the required applications for which they've been set and use them like any other machine on the network.

Your source files are versioned on the Revision Control Server
Your tests are run nightly on the Cruise Control Server
Your team documents the whole project and process on the Wiki, and file bugs in Bugzilla
Your MySQL server is used for running the DB & the supporting DBs for other apps (bugzilla, wiki)

These VM's need to be maintained like any other normal physical machine on the network.

Pitfalls

Putting all your eggs in one basket, if the main VMware server fails, all the "machines" (VM's) fail.
Since the server is dependant on the native OS it sits on, the capabilities of the VMServer is restricted by the OS
Solution: Use commercial ESX Server or other alternatives

Alternatives

You can use the Xen too for virtualization. I've heard pretty good things about it too, with a few short comings & advantages over VMServer (that's a separate topic all together).

There is an article dedicated to Virtual Machines on Wikipedia, you can refer that depending on your needs and fix upon a solution that suits your needs better

Glossary

Virtualization - The virtual machine simulates enough hardware to allow an unmodified "guest" OS (one designed for the same CPU) to be run in isolation

Continuous Integration - Continuous integration describes a set of software engineering practice's that speed up the delivery of software by decreasing integration times

Revision Control - Revision control (also known as version control (system) (VCS), source control or (source) code management (SCM)) is the management of multiple revisions of the same unit of information

Host OS - The operating system installed on the physical machine running VMware Server.

Guest OS - The operating system installed on the virtual machine.

Bibliography

DWR & Hibernate Lazy Loading

noreply@blogger.com (Varun Mehta) — Thu, 19 Mar 2009 04:31:00 +0000

Time immemorial, we are all aware about the famous lazy load exceptions in Hibernate. Using a detached object does not exactly hide us from the issue. We are using DWR in our application and found that when marshalling (or unmarshalling if you insist) a detached object to JSON, DWR was erratically trying to call a null (lazyly loaded) object, causing LazyLoadExceptions.

We did not want to use OpenSessionInViewFilter as it is evil (some day I'll post all the consolidated rantings as one). You can google and find a lot of info around it.

As a quick fix we did what one would first think of, "It's a demo, just eager load it!". Well the demo was over and now was the time to investigate a better solution, and we did find it.

It was no rocket science, the solution was available in front of us all along, we just did not look hard enough. When using DWR, there are BeanConverters available which are responsible for this marshalling process. There was one for Hiberante, called "hibernate3". http://directwebremoting.org/dwr/server/hibernate

The HibernateBeanConverter tries to avoid reading from un-initialized properties. (If you just want something that blindly reads everything then just use a plain BeanConverter).

Bingo, so when exporting a detached hibernate object to DWR, use type="hibernate3" and that should resolve the lazy load issues.

By Contract or By Convention

noreply@blogger.com (Varun Mehta) — Sun, 08 Feb 2009 18:22:00 +0000

Should we be using interfaces for everything and making everyone explicitly implement them or should we trust that everyone follows the same convention.

By Contract

For use cases, where a certain contracts needs to be followed, we should use interfaces, this allows easier testing and loose coupling of implementation, allowing us more flexibility to change the implementation at run time as asked for.

eg: UserService interface has a method fetchUser();
We can have two implementations for this service method, one using a DAO and other the WebService. Thus the implementation is loose and "the contract" is set to return a User.

By Convention

In certain cases, we cannot implement a contract, esp where the concerns span across multiple layers of the app. This is when we need to define convention over contract. AOP is the perfect use case where this needs to be strictly followed.

Let's consider the service methods which fetch information for displaying info to the front end. They need to have a specific security restriction associated with it, or a use case where we need to assign "READ_ONLY" rights to certain users for an object. In order to keep the code as decoupled as possible we use AOP, which apply concerns on these methods, without having actual references in the code.
For these concerns to be properly applied, we need to follow conventions.

eg: All DAOs should persist an object using the create() method only, that ways we can add a concern to block access to all create() methods for a group of user. If someone plans to "be different" and follow their own convention, like calling the method persist(), this can open up a huge security hole in the application. The AOP concern will skip "persist()" as it does not follow convention. Such issues will not be caught unless there is a proper code review performed on the code base, or we have really really strong test case.

If your module needs to follow a convention, which cannot be controlled using a contract, please document it, so as the team can follow is correctly.

2 DTO or not 2 DTO

noreply@blogger.com (Varun Mehta) — Mon, 02 Feb 2009 04:44:00 +0000

Since the new project began I've been think about this over and over again reading as many posts and taking ideas from different posts and blogs. The more I read, the less I knew. Below is a collection of few links I've come across which have some conclusive and non-conclusive decisions/arguments, some of these post lean towards POJO's and some towards DTO(Data Transfer Object). Frankly I was not been able to convince myself on what to use. (Leaning more towards POJO's as detached objects).

http://forum.springframework.org/archive/index.php/t-27716.html
http://forum.springframework.org/showthread.php?t=11338
http://forum.springframework.org/showthread.php?t=9810

http://www.hibernate.org/124.html

http://api.blogs.com/the_catch_blog/2005/05/protecting_the_.html
http://hookom.blogspot.com/2004/12/corporate-developer.html

Here is a small tutorial talking about how to develop using POJO's. Makes you think the non-EJB2 way. (Reading this is strongly recommended)
Part 1 - http://www.developer.com/java/ejb/article.php/3590731
Part 2 - http://www.developer.com/java/ejb/article.php/3592341
Part 3 - http://www.developer.com/java/ejb/article.php/3594121

Since EJB3 is more revolutionary and uses POJO's unlike remoting in EJB2, here is a link from the sun java forums
http://forums.sun.com/thread.jspa?threadID=5302559

I strongly recommend everyone to read these articles before choosing. My brains had been deep fried already reading these before trying to reach any conclusion. I wanted some other heads to think about these issues and post in their opinions.

Some points where I played the devil's advocate and jotted my concerns with arguments before we said 'I do' to DTO or POJO.
ProDTO: DTO will shield the UI if there are any model changes
Argument: Either ways we'll have to modify the DTO, the marshaller/unmarshaller to propagate these changes from the POJO to DTO. DTO does not eliminate the problem completely, but obfuscates or hides it.

ProDTO: Our service layer shall talk to various 'services' webUI or web Service or RMI or Star Trek, so we need DTOs to provide loose coupling and don't want the service to expose the domain object directly.
Argument: We will use XML (WSDL or SOAP or RestFUL or ) for data transfer, which will involve marshalling/unmarshalling of the XML to DTO and then POJO & vice-versa, any changes will again have to propagated. through all the layers. We are just exposing the POJO to the XML transformer, not the DAO calls. The POJO by itself cannot persist if in a detached state. Unless we have a usecase of just exposing half the object to the webservice (do we have a usecase?)

ProPOJO: Less effort, lean and clean design, any changes will propagate at multiple places and will provide less points of transformation errors.
Argument: The inner implementation of the POJO with the Data model needs to be changed, but the view/web service does not have to change the implementation. We have to make the changes in these layers even if we don't have to, there by adding more work and complexity.

ProPOJO: No over head of transforming POJO to DTO and vice-versa. Less memory consumption.
Argument: It's a good practice to separate your business objects and data objects, provides better decoupling.

Anti-POJO: Handling Lazy Initialization of hierarchical objects, no openSessionInView Filter for ExtJS.
Anti-DTO: Locking of objects for optimistic locking. DTO's are not aware of the revision.

Conclusion: Use POJOs as detached objects where basic CRUD calls are required and for fetching objects with minimum hierarchy, but if you need to fetch loads of info from different tables with a very small subset of columns from each, then use HQL + DTO

Select new com.vtech.model.dto.CompanyEmployee(bla bla bla);

iPhone & CFA

noreply@blogger.com (Varun Mehta) — Mon, 04 Aug 2008 18:25:00 +0000

I've enrolled myself for the Level I CFA exam in December '08. Studying on and off, depending on the schedule and time availability. Off late I stumbled upon SenFinance an interesting site full of Accounting and Quantitative tutorials for CFA Level I.

I downloaded them all and wanted to view them on my way to work (a 20 minute train travel is good to study). So I got my hands on SUPER® which is an amazing software to convert any one video or auto file to another format. Now I finally have the tutorials in mp4 format and can study on my way to work!

Setting up a PHP development environment on your local system

noreply@blogger.com (Varun Mehta) — Wed, 30 Jul 2008 19:59:00 +0000

This is an old post I had written on google docs, before VMware Server was free for the general user. Thought could be still more relevant to the general developer for testing in Linux envoirment. VMPlayer I beleive is lighter than VMware Server.

There are 2 stages of setting up the development & testing environment for yourself. The preferred work environment is Linux. If you already on Linux then you can safely skip Stage I, you can directly proceed to Stage II, also if you have Linux already installed, just cross check if you have the LAMP (Linux + Apache + MySQL + PHP) server installed.

Pre Installation Software Download Links

http://www.vmware.com/download/player/ [VMPlayer]
http://www.ubuntu.com/getubuntu/download [Ubuntu]

STAGE I: Setting up Ubuntu on VMWare.

Download Ubuntu/Xubuntu/Kubuntu Desktop edition ISO, what ever you like... all the same just different filemanagers (Gnome/Xfe/KDE resp.). If you are comfortable with command line, then you can also install the Server edition with no XWindows and only commandline.
Download and install VMPlayer for your system.
Download qemu for your operating system and extract in any directory you like.
Open your commandline to go ahead to bin folder of qemu and type

qemu-img create -f vmdk ubuntu.vmdk 3G

Here we are creating a 3GB, virtual hard drive in VMWare format. This file will now act as our hard drive for VMWare to install an OS on it. The name of the file is ubuntu.vmdk, you can name it what ever you wish to.
Now the main vmx file (The VMWare configuration file). change the highlighted paths as per your system.

 #!/usr/bin/vmware
config.version = "8"
virtualHW.version = "4"
ide0:0.present = "TRUE"
ide0:0.filename = "ubuntu.vmdk"
# The amount of RAM you want to allot to the Operating system. For Desktop use 512 and server just 256.
memsize = "512"
MemAllowAutoScaleDown = "FALSE"
ide1:0.present = "TRUE"

#ide1:0.fileName = "auto detect"
#ide1:0.deviceType = "cdrom-raw"

ide1:0.fileName = "ubuntu-7.04-server-i386.iso"
ide1:0.deviceType = "cdrom-image"

ide1:0.autodetect = "TRUE"
floppy0.present = "FALSE"
ethernet0.present = "TRUE"
usb.present = "TRUE"
sound.present = "TRUE"
displayName = "Ubuntu LAMP Server"
guestOS = "ubuntu"
nvram = "ubuntu-server-three.nvram"
MemTrimRate = "-1"

ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d ce 99 e0 d2 2b bf-73 47 ac 62 65 13 57 86"
uuid.bios = "56 4d ce 99 e0 d2 2b bf-73 47 ac 62 65 13 57 86"

tools.syncTime = "TRUE"
ide1:0.startConnected = "TRUE"

uuid.action = "create"

checkpoint.vmState = "ubuntu-lamp-server.vmss"

isolation.tools.hgfs.disable = "TRUE"
virtualHW.productCompatibility = "hosted"
tools.upgrade.policy = "manual"

tools.remindInstall = "TRUE"

usb.autoConnect.device0 = ""

After setting all the paths correctly, if you have VMPlayer installed, just save the vmx file, (call it ubuntu.vmx) and double click on the file.
If all the paths are set correctly, the VMPlayer will boot up the virtual drive and show the ubuntu installation menu. This is easier than windows a million times.
At the end of the installation (server or desktop) the process will ask you, if you want to install LAMP server. Select it and let it install the LAMP server.
Reboot the system, and you are good to go. You have successfully installed Ubuntu on VMWare.

STAGE II: Check the configuration setup of Apache/MySQL/PHP

Now we'll check if PHP and Apache have been successfully installed on your system. Go the /var/www folder, you should be seeing a apache2-default folder out there. If you see these then apache seems to be installed. Just open the browser and type http://localhost you should be able to see the apache-default folder over there. If you a receive a page not found then apache is not running or installed properly.

If things look good, next stage is to check PHP installation. Create a file named index.php and type the following there.

 <?php phpinfo(); ?>

Just refresh the localhost (place the file in /var/www) That should give you loads of PHP info on the screen in blue, purple and a huge table. If that happens you are good to go!!! Else something is wrong!!!.

MySQL Check
Open the command line and type

mysql -uroot

If it opens up with a mysql> prompt then it's good else something is wrong. Your mysql password is blank.

If all sounds good, now go to Stage III

STAGE III: Applications to setup.

Download

JOOMLA
WORDPRESS

Extract them to the /var/www folders and access them using http://localhost/joomla and http://localhost/wordpress respectively if you extracting them to the folders on these names.

How to go about setting them, start the index.php or read the readme file and you should be good to go.

Duplicate file finder

noreply@blogger.com (Varun Mehta) — Sun, 13 Jul 2008 23:30:00 +0000

Disk space is cheap, starting from a 1.2GB hard drive from my first computer to a "spare" 500GB external hard drive, cheap data storage has come a long way. I click a lot of photographs ever since I got my first digital camera, and I store a lot of these photos too (locally), now since 2006 I have over 201,608 photos and some videos. My camera photo number counter has reset twice!!

A few days back I decided to hand over my drive to my brother, since he was leaving soon, I dumped all the stuff on my two laptops, and forgot about them. Of late my wife asked me to pick up good nice pics so we can print them. That's when I realized I had a lot of duplicate photos, now the simplest idea was to find the duplicate file names and delete them, but that was not possible, since I had already reset the counter so I technically had 3 files with the same name and atleast 10,000 of them!!

MD5 to the rescue. Since all the photos and movies are binary files, MD5 seemed ideal to me...

MD5 digests have been widely used in the software world to provide some assurance that a transferred file has arrived intact. For example, file servers often provide a pre-computed MD5 checksum for the files, so that a user can compare the checksum of the downloaded file to it.

So I started my eclipse and churned out a program to scan my HDD and compare MD5 keys and find all duplicates.

The method below generates the MD5 checksum for any file

  private static String generateMD5(String path) throws IOException {
MessageDigest digest;
InputStream is = null;
try {
digest = MessageDigest.getInstance("MD5");
is = new FileInputStream(new File(path));
byte[] buffer = new byte[8192];
int read = 0;

while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
byte[] md5sum = digest.digest();
BigInteger bigInt = new BigInteger(1, md5sum);
return bigInt.toString(16);
} catch(NoSuchAlgorithmException e) {
e.printStackTrace();
} catch(FileNotFoundException e) {
e.printStackTrace();
} catch(IOException e) {
e.printStackTrace();
} finally {
is.close();
}
return null;
}

Then go ahead and get a list of all the files on your system


missed the code

And finally run the main method.

public static void main(String[] args) throws Exception {
List filePaths = new ArrayList();
File file = new File("/home/varun/workbench/duplicates.csv");
FileWriter fw = new FileWriter(file);
SortedMap duplicates = new TreeMap();
filePaths = generateFileMap("/mnt/datastorage/photos", filePaths);
for(String path : filePaths) {
String hash = generateMD5(path);
if(duplicates.containsKey(hash)) {
fw.write(path + "," + duplicates.get(hash) + "\n");
} else {
duplicates.put(hash, path);
}
}
}

So go ahead and run this, you can also extend it to generate a list of any kind of duplicate files. Average file size is 1.5 - 2.0 MB

The output file when viewed on Open Office, Google Docs or Excel looks like this

This is how the output looks like this. It shows you which file a duplicate of which other file.

/mnt/datastorage/Photos/2008/Photos/Halloween NYC 2007/DSC01958.JPG /mnt/datastorage/Photos/2008/Photos/SORT ME/DSC01958.JPG
/mnt/datastorage/Photos/2008/Photos/Halloween NYC 2007/DSC01959.JPG /mnt/datastorage/Photos/2008/Photos/SORT ME/DSC01959.JPG
/mnt/datastorage/Photos/2008/Photos/Halloween NYC 2007/DSC01960.JPG /mnt/datastorage/Photos/2008/Photos/SORT ME/DSC01960.JPG

UPDATE: Ran the program with SHA algorithm also and here are the comparison times.

Time for MD5 858,953ms (14.31 minutes)
Time for SHA 1,191,656ms (19.80 minutes)

Bibliography