Shalin Says...

Apache Lucene 3.1.0 and Apache Solr 3.1.0

Fri, 01 Apr 2011 16:24:59 +0530

Apache Lucene 3.1.0 and Apache Solr 3.1.0:

This is the first release bringing Lucene and Solr release versions in sync. There are numerous bug fixes, optimizations and new features. Download from here

My Android phone - Samsung Galaxy S Review

Fri, 06 Aug 2010 02:15:00 +0530

I had been holding out on buying a more internet friendly phone for some time now, waiting for 3G service to start in India. After my iPad experience, it was clear to me that I couldn’t be happy with an iPhone but it was also obvious that none of the available Android phones were good enough.

Enter the new Samsung Galaxy S with Android 2.1, awesome 4” display, light weight and a good (enough) battery life. A little market research showed that Samsung’s Super AMOLED technology was best in class and a better phone, the Samsung Galaxy S2, will only be available next year. With 3G to be introduced (supposedly) around October, the stage was set. So, one fine July evening, I bought the Samsung Galaxy S.

First of all, a message to people who are surprised upon hearing the price - the Samsung Galaxy S is not a phone; it is a pocket sized computer that also happens to be a phone. And what have I been upto with this device? Here’s a list of ten things (in no particular order) that I have used it to:

Get email alerts via K9 Mail, read blogs, check twitter, buzz and facebook feeds
Take photographs and upload it to Facebook (darn you Airtel, I could never get MMS to work properly)
Watch movies on my desktop monitor and phone through VLC Remote and Gmote.
Stream music from GrooveShark - Yay! freedom from syncing my music
Play Asphalt5 while sitting at the back of an autorickshaw - something ironic about it.
Use AIM to answer questions from our QA team while having lunch. Note to myself, use a spoon next time if you want to use the phone.
SMS while walking down the road without the fear of banging into an obstacle.
Read a book on the Kindle app.
Navigate to Kolar and back with Google Maps
Impress people with the live wallpapers, gesture search and some not so useful tidbits such as a lie detector and a sky map.

Google integration means that all my contacts are backed up on their servers. I am still surprised that something as simple as backing up contacts has been so hard for phones till now. Another cool feature that I loved was that I could link phone numbers with Facebook contacts together. Now when a friend calls, I see his Facebook photo automatically. The device has a decent battery life; with on and off wifi use, it lasts for a little more than a day. The device has a 5 megapixel camera and can record a 720p video. And, by the way, flash sites works too.

Before I bought the phone, I wasn’t sure how easy it would be to use a touch keyboard. Well, it wasn’t too hard with the default keyboard but ever since I switched to the Swype keyboard, I’m insanely fast. I’d recommend Swype to everybody. Good thing that Samsung provides this keyboard application for free.

There are a few downsides though. The phone feels sluggish when more than a couple of applications are running. The “Advanced Task Killer” application is essential for a decent experience. There is no flash light so one must depend on external light source being available to avoid dark photos. Sometimes (and this happens rarely), after moving out of a wifi zone, it won’t automatically switch to a GPRS/EDGE connection unless I restart the phone.

All in all, I’m happy with the device and eagerly waiting for an Android 2.2 (froyo) upgrade and Airtel’s 3G service.

Solr 1.4.1 Released

Tue, 29 Jun 2010 13:57:23 +0530

Solr 1.4.1 Released:

From the mailing list announcement:

Apache Solr 1.4.1 has been released and is now available for public
download!
http://www.apache.org/dyn/closer.cgi/lucene/solr/

Solr 1.4.1 is a bug fix release. See the change log for more details.

The iPad experience

Sat, 26 Jun 2010 00:18:00 +0530

Now that I have had the chance to play with the Apple iPad, I thought I’d put down some observations and opinions about the device. I know that I’m late to the party but hey, they don’t sell these devices here in India.

First of all, I love that I do not have to sit before a desk to use this. There is a real need in the world for a device like this and even though many have tried in the past, hats off to Apple for pulling this off. No company other than Apple, without its army of fan boys, would have been able to make this kind of a product successful. But now that Apple has, we can all expect that the new shiny competing devices will be able to do at least as much as what the iPad does. So, thank you Apple!

The model that I have tried is the 32GB iPad. The device is slim, the touch experience is intuitive and the display is sharp. It is great as a browsing device. I guess it would have been easier to use if it were a little more lighter - which is not too much to expect from later versions. The dimensions at 9.56 x 7.47 inches feels a little too big at times. A 7-8 inches display may be more handy.

Typing on the touch keyboard takes a little getting used to but practice makes perfect. Still, without my favorite set of browser plugins, I find it hard to share stuff on twitter/buzz. I guess the iPad is designed for people who are mostly consumers of data rather than creators. The device does not have a USB port which is a bummer. A Linux user like me cannot transfer songs or movies from my laptop to the device because one must use iTunes. I had to borrow a friend’s laptop to use Windows(!) so that I can transfer some media into the device. But to do that, you need to convert your divx media to the format iPad can play. The App Store is not open to anyone outside the US so I couldn’t use it.

As a geek, one gets bored with the device very soon. There’s just too many things that I want to do with it but am not able to because Apple won’t let me. I am looking forward to the Samsung Android based tablet which is supposed to be coming out in Q3. No one can beat Apple in the wow factor category but I am hoping that the Android devices may allow me to do much more.

[Update] - I used the excellent Handbrake application to convert between media formats and I’d highly recommend it.

[Update 2] - An interesting discussion on Google Buzz on this topic.

Apache Mahout 0.3 Released

Thu, 18 Mar 2010 14:58:29 +0530

Apache Mahout 0.3 has been released. Apache Mahout is a project which attempts to make machine learning both scalable and accessible. It is a sub-project of the excellent Apache Lucene project which provides open source search software.

From the project website:

The Apache Lucene project is pleased to announce the release of Apache Mahout 0.3. Highlights include:

New: math and collections modules based on the high performance Colt library

Faster Frequent Pattern Growth(FPGrowth) using FP-bonsai pruning

Parallel Dirichlet process clustering (model-based clustering algorithm)

Parallel co-occurrence based recommender

Parallel text document to vector conversion using LLR based ngram generation

Parallel Lanczos SVD (Singular Value Decomposition) solver

Shell scripts for easier running of algorithms, utilities and examples

.. and much much more: code cleanup, many bug fixes and performance improvements

Details on what’s included can be found in the release notes. Downloads are available from the Apache Mirrors

Congratulations and thanks to all Mahout developers!

Merging Lucene and Solr

Wed, 17 Mar 2010 01:18:00 +0530

A couple of weeks back, Apache Lucene Committer and PMC member, Michael McCandless started a discussion on factoring out a shared, standalone Analysis package for Lucene, Solr and Nutch. During the discussions, Yonik Seeley, Solr Creator, proposed merging the development of Lucene and Solr. After intense discussions and multiple rounds of voting, the following changes are being put into effect:

Merging the developer mailing lists into a single list.
Merging the set of committers.
When any change is committed (to a module that “belongs to” Solr or to Lucene), all tests must pass.
Release details will be decided by dev community, but, Lucene may release without Solr.
Modularize the sources: pull things out of Lucene’s core (break out query parser, move all core queries & analyzers under their contrib counterparts), pull things out of Solr’s core (analyzers, queries).

The following things do not change:

Besides modularizing (above), the source code would remain factored into separate dirs/modules the way it is now.
Issue tracking remains separate (SOLR-XXX and LUCENE-XXX issues).
User’s lists remain separate.
Web sites remain separate.
Release artifacts/jars remain separate.

So what does it mean for Lucene/Solr users? Nothing much, really. Except that you should see tighter co-ordination between Lucene and Solr development. New Lucene features should reach Solr faster and releases should be more frequent. Solr features may also be made available to Lucene users who do not want to setup Solr use the RESTy APIs.

Already, Solr has been upgraded to use Lucene trunk (in branches/solr) and should soon become the new Solr trunk. There is talk of re-organizing the source structure to better fit the new model. Things are moving fast!

Personally, I feel that this merge is a good thing for both Lucene and Solr:

Solr users get the latest Lucene improvements faster and releases get streamlined.
Lucene users get access to Solr features such as faceting.
The in-sync trunk allows new features to make their way into the right place (Lucene vs Solr) more easily and duplication is minimized.
Bugs are caught earlier by the huge combined test suite.
More number of committers means more ideas and hands available to the projects
Other Lucene based projects can benefit too because many Solr features will be made available through Java APIs.

There are a couple of things to be worked out. For example, we need to decide where the integrated sources should live and whether or not to sync Solr’s version with Lucene’s. All this will take some time but I am confident that our combined community will manage the transition well.

Apache Lucene Java 3.0.1 and 2.9.2 Released

Sat, 27 Feb 2010 15:39:00 +0530

Congratulations to the Apache Lucene team on releasing Lucene Java 3.0.1 and 2.9.2. Both of these are bug fix releases and are backwards compatible with Lucene Java 3.0.0 and 2.9.1 respectively.

From the official announcement:

Hello Lucene users,

On behalf of the Lucene development community I would like to announce the release of Lucene Java versions 3.0.1 and 2.9.2:

Both releases fix bugs in the previous versions:

2.9.2 is a bugfix release for the Lucene Java 2.x series, based on Java 1.4

3.0.1 has the same bug fix level but is for the Lucene Java 3.x series, based on Java 5

New users of Lucene are advised to use version 3.0.1 for new developments, because it has a clean, type-safe API.

Important improvements in these releases include:

An increased maximum number of unique terms in each index segment.

Fixed experimental CustomScoreQuery to respect per-segment search. This introduced an API change!

Important fixes to IndexWriter: a commit() thread-safety issue, lost document deletes in near real-time indexing.

Bugfixes for Contrib’s Analyzers package.

Restoration of some public methods that were lost during deprecation removal.

The new Attribute-based TokenStream API now works correctly with different class loaders.

Both releases are fully compatible with the corresponding previous versions. We strongly recommend upgrading to 2.9.2 if you are using 2.9.1 or 2.9.0; and to 3.0.1 if you are using 3.0.0.

See core changes at:

http://lucene.apache.org/java/3_0_1/changes/Changes.html

http://lucene.apache.org/java/2_9_2/changes/Changes.html

and contrib changes at:

http://lucene.apache.org/java/3_0_1/changes/Contrib-Changes.html

http://lucene.apache.org/java/2_9_2/changes/Contrib-Changes.html

Binary and source distributions are available at http://www.apache.org/dyn/closer.cgi/lucene/java/

Lucene artifacts are also available in the Maven2 repository at http://repo1.maven.org/maven2/org/apache/lucene/

Microsoft dropping FAST search for Linux, Unix

Mon, 08 Feb 2010 02:09:47 +0530

According to a blog post from Microsoft Distinguished Engineer and CTO, FAST Bjørn Olstad, the 2010 products will be the last to have a search core that runs on Linux and UNIX.

Being involved in Apache Solr and the newly formed Lucene Connectors Framework (LCF) project, I’m very interested in the implications. Undoubtedly, at least some FAST customers will not be happy with this decision and will want to switch to something which can still run on Linux/UNIX.

I believe that this is a great opportunity for the Apache Solr/LCF combo. Perhaps, the newly proposed Apache Spatial Information Systems (SIS) will help as well. Of course, this is big news for Lucid Imagination, Sematext and other companies as well who offer consultancy, training and support for Lucene/Solr.

I’d like to ask people who have used FAST in the past, what would it take for Lucene/Solr/LCF to fill the gap?

Saw a Kurkure advertisement on a website titled “No...

Sun, 24 Jan 2010 16:48:20 +0530

Saw a Kurkure advertisement on a website titled “No plastic in Kurkure”. ROTFL!

Solr In Action Case Studies

Fri, 15 Jan 2010 14:55:38 +0530

Well, the cat is out of the bag. I’ve been working with Otis on Solr In Action. We’re looking for a couple of contributors to write case studies for the book describing how they have used Solr. Otis just posted this to his blog and to the Solr mailing list as well.

So, if you are are using Apache Solr in some clever, interesting or unusual way, or deal with large indexes or large number of cores or distributed search and are willing to share this information with the world, please get in touch. We are looking for between five to ten pages (soft limits) per case study.

You can contact either me or Otis by leaving a comment on this post with your contact info or contact @shalinmangar or @otisg on Twitter or email me on shalin at apache and we’ll get back to you right away.

It’d be great if you can share this message around too.

The Total Growth of Open Source

Tue, 12 Jan 2010 13:01:00 +0530

The Total Growth of Open Source:

Amit Deshpande and Dirk Riehle from SAP Labs have conducted and published a research on the growth of open source software.

The data has been culled from Ohloh.net and is based on the stats and activity of around 5000 open source projects written in 30 different languages and 103 open source licenses.

Some interesting quotes from the publication:

Successful open source projects like Linux, Apache, PostgreSQL and many others are growing super-linearly. Previous research showed that linear and quadratic growth is the dominant growth pattern of open source software projects

Our work shows that the additions to open source projects, the total project size (measured in source lines of code), the number of new open source projects, and the total number of open source projects are growing at an exponential rate. The total amount of source code and the total number of projects double about every 14 months.

Open Source has taken off handsomely and continues to thrive. It is not just about philosophy any more, it is good business sense.

In case you are interested about Solr stats, see the Solr project page at Ohloh.

Update - The report is quite old but I just discovered it now :)

SolrMarc vs DIHMarc

Mon, 11 Jan 2010 00:01:51 +0530

Erik has written about Solr’s usage in libraries on the Lucid Imagination Blog. Solr has found its way into many libraries and quite rightly so. However, one of the main things that Erik talks about in that blog post is the performance of DataImportHandler vs SolrMarc (the indexing library used by both VUFind and Blacklight).

Quoting from Erik’s email to the solrmarc-tech google group:

The difference in speeds:
SolrMarc: 22 docs / s
DIHmarc: 1,745 docs / s

W00t!

Well, I don’t know much about SolrMarc but I’ve seen DataImportHandler instances with comparable (or even better) throughput many times. There is something fishy inside SolrMarc for sure and I have a feeling that fixing it would be a low hanging fruit. However, Erik’s opinion is that DataImportHandler is a better way to index and that he will devote a portion of his time to helping the Solr using library community (thanks Erik!).

DIH has really taken off since it debuted in Solr 1.3 and it would be safe to call it the de-facto standard for indexing data into Solr. It may not be the most elegant way to index data but it is quick and it works great. With the planned features for DataImportHandler in Solr v1.5, it will continue to improve and it makes sense to base VUFind and Blacklight’s indexing infrastructure on top of it. I’m very excited to see this happening.

Exploding mobile web usage in India

Tue, 22 Dec 2009 19:12:46 +0530

Exploding mobile web usage in India:

Opera published a study titled State of the Mobile Web, November 2009 which I found through TechCrunch. I can’t help but notice the tremendous growth in web usage through mobile phones in India. Page views have grown by 228.5% Y/Y and unique users have grown by 208.4% but if you look at metrics like page views per user or the amount of data transferred per user, you’ll see that they are quite small.

One of the reason is that in India we don’t have 3G (actually we do but it is limited to only one provider - BSNL) and browsing the Internet is painfully slow on GPRS connections. With 3G finally coming next year, I’m quite sure that the mobile web usage in India will just explode.

Right now the only kind of mobile applications that can work in India are SMS based. The increase in download speeds will make more people use the Internet and SMS will be less relevant, though that may take more time than a year. If 3G indeed is introduced by mid next year by prominent providers like Airtel and Vodaphone, I won’t be surprised to see Y/Y growth rate exceeding 500%.

Companies building products/services around SMS should start thinking about a mobile web strategy now.

Migrating from Blogger to Tumblr

Fri, 18 Dec 2009 01:31:14 +0530

I had been thinking about moving away from Blogger to my own domain. Finally, I decided to give in and I was fortunate enough to buy this domain. Blogger has been a simple service but I wanted to try the new kids Tumblr or Posterous. After spending some time fiddling with both of them, I decided to go with Tumblr.

It took me some time to figure out the right way to move from Blogger. I used the import script written by Jonathan Tron to import my old posts. Sadly, there is no way to import comments from blogger. The least I could do was to import them into disqus and link the same disqus account into tumblr; which actually does not help much.

The bigger issue was to migrate without leaving RSS subscribers in the lurch. The slightly lesser issue was to preserve my earlier blog’s Google page rank.

The first issue was solved easily. You can do the same with the following sequence of steps.

Create a Feedburner account and import your blogger feed.
Set your blogger feed to redirect to Feedburner.
Edit your tumblr theme and replace the RSS link to point to feedburner
Edit your feedburner to import from tumblr’s rss

The second was slightly more tricky. The right way to move a website to a different domain is to use permanent redirects from the old page to the new page. However, Blogger (obviously) does not allow you to do that. Thankfully, Google recently announced support for specifying canonical links which can point to the preferred version of a URL. So, I hacked up a script to match pages of blogger’s RSS with tumblr’s RSS and generate conditional blogger template snippets which let me specify the canonical (tumblr) URL for each page on my Blogger account.

I couldn’t redirect so I had to fall back on meta-refresh to redirect anyone visiting an old page to the new page. I hate to break the back button like this but that was the only possible way in this case. This is what it looked like:

I used to think that blogging is a solved problem. After doing all this and trying out many service, I don’t believe that anymore.

AOL’s new logos, which one do you like?

Thu, 17 Dec 2009 15:06:25 +0530

AOL’s new logos, which one do you like?

AOL lists on NYSE

Thu, 17 Dec 2009 14:06:00 +0530

AOL listed on the New York Stock Exchange on 10th December 2009. This has been in the works for a long time and I’m glad we’re finally here. Things are changing around the company and I’m happy to be a part of this change.

AOL has a new logo (and yes, it is still to be written as AOL). I loved the new brand videos, watch them on youtube - http://www.youtube.com/watch?v=YlSL7svbooY

Seed.com was also launched a few days back. It is a new spin on crowd sourcing content which, I believe, is a great idea.

We’re just getting started!

Apache Lucene Java 3.0 Released

Thu, 26 Nov 2009 05:45:00 +0530

Apache Lucene Java 3.0.0 has been released. Lucene Java 3.0.0 is mostly a clean-up release without any new features. It paves the path for refactoring and adding new features without the shackles of backwards compatibility. All APIs deprecated in Lucene 2.9 have been removed and Lucene Java has officially moved to Java 5 as the minimum requirement.

See the announcement email for more details. Congratulations Lucene Devs!

Apache Mahout 0.2 Released

Wed, 18 Nov 2009 05:54:00 +0530

Apache Mahout 0.2 has been released. Apache Mahout is a project which attempts to make machine learning both scalable and accessible. It is a sub-project of the excellent Apache Lucene project which provides open source search software.

From the project website:

The Apache Lucene project is pleased to announce the release of Apache Mahout 0.2.
Highlights include:
Significant performance increase (and API changes) in collaborative filtering engine
K-nearest-neighbor and SVD recommenders
Much code cleanup, bug fixing
Random forests, frequent pattern mining using parallel FP growth
Latent Dirichlet Allocation
Updates for Hadoop 0.20.x
Details on what’s included can be found in the release notes.
Downloads are available from the Apache Mirrors

Apache Solr 1.4 Released

Tue, 10 Nov 2009 07:15:00 +0530

From the official announcement:

Apache Solr 1.4 has been released and is now available for public download!
http://www.apache.org/dyn/closer.cgi/lucene/solr/

Solr is the popular, blazing fast open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Solr is highly scalable, providing distributed search and index replication, and it powers the search and navigation features of
many of the world’s largest internet sites.

Solr is written in Java and runs as a standalone full-text search server within a servlet container such as Tomcat. Solr uses the Lucene Java search library at its core for full-text indexing and search, and has REST-like HTTP/XML and JSON APIs that make it easy to use from virtually any programming language. Solr’s powerful external configuration allows it to be tailored to almost any type of application without Java coding, and it has an extensive plugin architecture when more advanced customization is required.

New Solr 1.4 features include

Major performance enhancements in indexing, searching, and faceting
Revamped all-Java index replication that’s simple to configure and can replicate configuration files
Greatly improved database integration via the DataImportHandler
Rich document processing (Word, PDF, HTML) via Apache Tika
Dynamic search results clustering via Carrot2
Multi-select faceting (support for multiple items in a single category to be selected)
Many powerful query enhancements, including ranges over arbitrary functions, and nested queries of different syntaxes
Many other plugins including Terms for auto-suggest, Statistics, TermVectors, Deduplication

Performance Enhancements

Revamped All-Java Replication

SolrReplication wiki page
Works on Microsoft Windows Platforms too!

DataImportHandler improvements

Rich document processing

Dynamic Search Results Clustering

Multi-select Faceting

Query Enhancements

Ranges over functions
Nested query support for any type of query parser (via QParserPlugin). Quotes will often be necessary to encapsulate the nested query if it contains reserved characters. Example: _query_:”{!dismax qf=myfield}how now brown cow”

New Plugins

TermsComponent (can be used for auto-suggest)
TermVectorComponent
Statistics
Deduplication

SolrJ - Java client

Faster, more efficient Binary Update format
Javabean (POJO) binding support
Fast multi-threaded updates through StreamingUpdateSolrServer
Simple round-robin load balancing client - LBHttpSolrServer
Stream documents through an Iterator API
Many performance optimizations

Miscellaneous

Rollback command in UpdateHandler
More configurable logging through the use of SLF4J library
‘commitWithin’ parameter on add document command allows setting a per-request auto-commit time limit.
TokenFilter factories for Arabic language
Improved Thai language tokenization (SOLR-1078)
Merge multiple indexes
Expunge Deletes command

Upgrade instructions

Although Solr 1.4 is backwards-compatible with previous releases, users are encouraged to read the upgrading notes in the Solr Change Log.

There are so many more new features, optimizations, bug fixes and refactorings that it is not possible to cover them all in a single blog post.

A large amount of effort has gone into this release. Many congratulations to the entire Solr community for making this happen!

Great things are planned for the next release and it is a great time to get involved. See http://wiki.apache.org/solr/HowToContribute for how to get started.

Enjoy Solr 1.4 and let us know on the mailing lists if you have any questions!

Why you should contribute to Open Source

Mon, 26 Oct 2009 15:12:00 +0530

Note: The following material and presentation was prepared for students of the Indian Institute of Information Technology (IIIT), Allahabad. The aim was to get them excited about contributing to open source projects and in particular about Apache Lucene, Solr and Hadoop. The first talk was titled “Why you should contribute to Open Source” and was aimed at freshmen and has no technical content. The second was titled “Get involved with the Apache Software Foundation” and was given to sophomore, junior and senior students and it goes into some basic technical information on Apache Lucene, Solr and Hadoop projects. The following post comprises of some notes that I put together for the talks.

Work on what you like, when you like

Everybody wants to work on “cool” products. However, the reality is that most of you will get stuck in a job which although may pay well, it will hardly be about the things you wanted to work on. In your course, you will learn about algorithms, distributed systems, natural language processing, information retrieval, bio-informatics and other areas of computer science and its applications but in real life, the majority of the work done in software companies will have little direct application of things you will learn in your course.

Most of the times you will be using things built by others and writing glue code to build things needed by your company’s business. This is not to say that all that knowledge will go waste; it will definitely help you become a better programmer and you should learn it but there’s a fair chance that it may not be used directly in your job.

Open Source projects offer you a chance to work on something that you want rather than something that others want you to work on. It is a great opportunity to work on something that is both cool and useful as well as to associate with a well known brand and all the publicity and goodwill it brings. You are free to pick and choose between the thousands of open source projects out there. Moreover, you are free to decide on how much you want to contribute. You won’t have a boss and you won’t have the pressure of deadlines and schedules.

Development in the “real” world

Academic projects are insufficient to impart many of the skills that you’d need once you start developing software full-time. Many of these skills are “social” rather than technical in nature but are at least as important.

Most academic projects are “toy” projects. By that, I mean that their whole life cycle revolves around you. You are the designer, developer, tester and also the user. As a result, there are few key things missing in those projects.

No build system - Makefiles? Ant? Maven? Very few students are familiar with using them. Don’t even ask about creating a build from scratch. “Hey! Just open those files in a text editor or an IDE and hack away” is not an unusual thing to be heard
No source control - CVS? SVN? Git? A single person writing all the code or >80% of the code is very common
No bug tracker - “It is never going to be used after we demo it to the professors”
No user documentation - maybe you will write a research paper detailing your findings but there is little or no documentation written for “other” people
No mailing lists or forums for support - Nobody but you is going to use it

Moreover, under these circumstances, you never learn how to:

Discuss technical design or issues in writing
Resolve conflicts in matters of design, architecture and a project’s road map.
Build usable interfaces (whether command line options or a GUI or an API)
Write proper error handling and logging code
Identify hooks for monitoring systems in production
Think about backup and recovery
Identify components which can be extended or replaced to add or modify functionality of the system

Open source projects are the real deal. If you are involved for long enough, you will either see or be a part of many such discussions and conflicts. All of the above skills are things you will need when you get around to software development in the real world.

Learn from the best

How many great developers do you know about? How many of them work or have worked on an open source project? I bet there are many names common to both the lists.

Open Source development will help you observe how experienced developers work and their various ways of designing, coding and discussing solutions. You will learn new ideas and new ways of solving problems. The second and probably more important part is that many smart programmers will be looking over your code and will provide review comments which will help you improve yourself. You will learn more efficient or shorter (or both) ways to solve the same problem. That kind of feedback is invaluable to a budding programmer.

I know that I’ve learned a great deal since I got involved in Apache Solr.

Build a publicly verifiable resume

What you tell in your resume are things like contact information, performance in academia, programming languages you know, projects you’ve worked on and other such stuff. There is very little in this document which can be verified easily. This is a problem for you as well as for the prospective employer because:

It may not represent you, your skills and your hard work sufficiently enough
It makes hiring a game of chance for the prospective employer and prevents them from making more informed decisions

The best thing about contributing to an open source project is that everything you do is public. So you can say things like the following:

I have worked on this project for the last two years
I wrote features X, Y and Z on Project P
I have over two hundred posts on the user forum or mailing list
I have commit access to the project
I am the expert because “I wrote it”

And your prospective employer can search and verify such things easily. Congratulations, you have just landed on top of the stack of resumes!

Companies will find you

When a company evaluates that an open source Project X can save them a lot of money, it is likely that they will hire a few people who have experience on project X and can support its use internally. Many such companies also allow their developers to work on the project either part-time or full-time. And who else is more qualified to work on the project but you - an existing contributor!

More and more companies are starting up around providing training, consulting and support for open source projects. Many such companies exclusively hire existing contributors.

Even if an open source project is not used directly inside the company, many tech companies hire open source contributors because:

Hiring popular open source developers makes them more cool in the eyes of other developers
Developers who contribute to open source projects are good programmers

I’m sure there are many more reasons other than the ones I’ve given here. In the end, contributing to an open source project is a good investment of your time and it may well be your big ticket to finding that great job. Good Luck!