There are claims that Open Source will one day completely displace closed software. Same goes with free online advertising, sometimes also referred to as ad exchanges, where a person places an ad on his/her site and in turn gets his/her ad placed on somebody else’s site. There are claims that free online advertising will one day displace Google’s monopolistic hold on online advertising. Frankly I think most of these claims are exaggerated and baseless to a certain extent.

Am I the only one who has a problem with transactions that do not obey the traditional laws of Economics? The market, or as Adam Smith would say – the foundations of human society,  runs on the buy-sell model. When something is free, it means other non quantifiable commodities get traded, like goodwill and trust.  In such circumstances fairness is the biggest casualty. But is there really a model which can ensure fairness in these situations ?

Yes, there are plenty of trust based networks at play in our real life, but they all reach a saturation point. A typical game theoretic scenario, the person to defect first gets a very high payoff ; In a network, everyone else suffers. People will cheat, one way or another. Like I mentioned before, fairness becomes a higly debatable issue. Though, the categorical imperative bestows humans with decision making powers, it doesn’t state the thresholding for these decisions. The only final outcome of these networks is that one person is going to hit it big and make the others suffer.

Naturally the follow up question will be:  how does Open source survive? I explain this using results from game theory. In any homogeneous population, ie either a defecting or a co-operating , a small group of people,  co-operating with each other while defecting against the general consensus , always has the ability to hold their own and, in some cases, even win against the entire population. Examples of this are plenty in real life,  for ex: corruption or the mafia. So, as long as that small bunch of people stay loyal to each other, such networks will continue. Open Source, is slightly a more complex beast than I described it to be, but it broadly falls into the category I mentioned.

Incentive mechanisms like reputation points, virtual currency etc usually try and emulate a real world materialistic economy, but they are limited in their reach and appeal. Eventually, people loose the initiative to compete for these non materialistic resources. Besides, in a medium such as the internet, where identity itself is questionable, how will people trust each other. Has anyone ever wondered if those mails, which inform you of the billions you have won, were ever real? I guess not.

In conclusion,  I ask, is there some mechanism design or a strategy that can ensure fairness (provided its defined accurately) whilst ensuring the continuing growth of such trust based networks ?

I remember liking programming for lots of reasons, but one which I particularly remember. Unlike the other subjects in school, like math or physics,  programming had no boundaries or a legacy to deal with. By legacy, I mean there were no equations, constraints and I didn’t have to reciprocate what some mathematician proved 400 years ago. I have always enjoyed freedom and programming gave me that freedom to express myself.  I guess, I can claim that most good programmers have taken up programming because it lets them play God or be creationists of some complex entity.

Anyway, there are a lot of interesting and fun facts through these ten years and I shall pen some down.

1. BASIC was my first programming language. PASCAL was probably my first compiled language.
2. I wrote my first sorting routine ( an act I call, truly understanding the programming abstraction) sometime during Jan 2000.
3. I got my first computer in 2001. AMD Athlon 1.7 GHz with 256 MB Ram and 40 Gb harddisk . “OMG 40GB” , was my friends reaction.
4. I disliked C early on, due to my PASCAL roots, but grew fond of it later on when I completed my first large scale project ( A linux text editor, which I proudly wanted to call, “Better than VI“). This is also where I wrote my first recursive functionality which resulted in people actually wanting to read and understand my code.
5. My first exposure to Linux and OSS was in 2002, I think it was RedHat linux 7.1 with running Xwindows.
6. I almost lost my entire project due to a floppy malfunction. USB’s were expensive and uncommon then. CD writers expensive.
7. I learnt about data structures in 2002.  My lecturer, one of the few good ones, was kind enough to teach us the intuition behind data structures and write the implementations ourselves. Our other lecturers taught programming from a book and expected us to replicate the programs written in the book.
8. I would like to thank  my data structures lecturer who spent hours helping me debug and correct programs. It was probably the first time I looked at code from an outside perspective and yes, I realized the importance of indenting code then and there. I have never not indented my code again. (Remember we are still in 2002).
9. I learnt about OO during christmas vacations of 2002. Loved it. Loved the paradigm shift in the way I thought about programs
10. My first OO program was one class with 50+ methods in it (Yup, there was nothing OO about it, but its a start)
11. My first large scale OO project was building a Paint like utility using C++. I am still proud of this work because I learnt about programming abstractions and class reponsibilities here. I still remember writing my first button class which was fully reponsible for itself.
12. I learnt PERL in 2004 and wrote my first web application. An online messaging, calendar and collaboration utility for friends and corporates. ( yes, this was my first social app). When I saw Myspace very soon after that ( MySpace was still pretty bare in terms of features then) the only difference I saw was that other people could see who my contacts(friends) were and network with them.
13. When I first discovered databases (2003), I thought, “wow, somebody made my file I/O’s really easy”. It was also an amazement because I had thought of a unified file writing and reading mechanism to solve all my I/O woes. Problem was getting it to work with many languages. I couldn’t get my head around it and thats when I learnt about databases. Imagine my surprise and rejoice. The first database I used was Oracle and later MySQL ( it wasn’t even relational then). My database project earned me 100/100 in my finals.
14. Compiler Design was one of my favorite subjects. I loved the fact that I could understand how they build programming languagues. Lex Yacc was probably the best thing that ever happened to programming. Prateeksha and I wrote the specifications for a shift reduce parser for C++. We used an A1 sized sheet of paper to write down the shift reduce matrix. After two full days writing the Shift Reduce rules, we realized we had messed up somewhere.
15. I wrote a prototype of my CASE tool in 2004.  My proud entry to the world of .NET. I had my official copy of Visual Studio.NET 2003 and .NET 1.1 . This CASE tool would eventually become my fnal semester project and also enter Microsoft’s Imagine Cup. We reached the Nationals for the competition. I called it Dzine.
16. Even though I had learnt java in 2004, I didnt use J2EE till late 2005. when I started writing simple J2EE apps. I also figured that javascript, that language I had used to validate my controls in HTML (in 2004) , had suddenly taken the world by storm. Everybody was talking about it. Web 2.0.
17. I sat for two whole days to figure out how yahoo mail was autocompleting email addresses I typed. After two days and a whole lot of searching, I learnt about AJAX. I was already on Gmail by then and didnt really understand the underlying plumbing that was holding the application together.
18. Within the next 4 months I put AJAX and J2EE together to work on Samparkh with Prateeksha. I wrote an online chat application using AJAX ( inspired by Meebo). Remember this was a time when firebug wasn’t around and I used a tool called Venkman, which I am sure many of you haven’t even heard of.
19. Then, Grad school happened and so did Microsoft, and the list of wonderful projects that I did during the two years I spent there.
20. Special mention to BigKahuna, which took almost two years to perfect and won the Google Product Engineering Competition 2009.

Am I a good programmer? I dont know. But, I will continue to remain a programmer. Most of my friends know that I dont take sides. By sides, I mean OSS v/s Closed source, Linux vs Windows , Google vs Yahoo, and this is because of the vast and varied experiences I have had with all these different entities during programming. They all have a special place in my heart and I cannot choose one over the other. Programming is changing fast, and all I can hope for is that that I dont wear out of ideas or skills to call myself a programmer in the years to come.

A list of all the wonderful projects that I have pursued over the years is available here.

• Just over 450,000 students in India graduate with an Engineering degree
• 150,000 students amongst them with a degree in either Computer Science or Information Technology.
• There are about 1500 Engineering colleges in India.
• Many of these colleges don’t even have a full professor on their rolls.
• Currently there are about 750 students pursuing a Phd in 15 of the most reputed institutions in the country which means that, about 80 to 90 students graduate with a Phd from one of the 15 reputed institutions in India.
• The 15 reputed institutions include the IIT’s, NIT’s, two of the IIIT’s (Hyderabad and Bangalore) and some autonomous institutions like BITS and Vellore.
• The percentage of students who take up graduate education after their engineering in India is drastically low.
• About a quarter of the students who secure Phd’s from universities from the US are Indians.
• Students of Indian and Chinese origin make up half the graduate schools students in America.
• Most people who secure their Phd’s from universities in India either join small and focused research groups in IT companies or take up faculty positions.
• This year the amount of students applying for graduate education has increased dramatically, which only is reassuring evidence that graduate education is seen as a substitute for jobs and not as something of value.
• A couple of IIT’s got about 700 applications for masters and phd positions.

Apart from all this the research output in India is not very high. Groups doing theory are considered to be doing some of the state of the art research, the other departments are not very highly regarded (I have a problem with this generalization, but we will keep that for another discussion). The researchers present in the discussion had plenty of points to contribute for the dismal state of higher education and some of the points mentioned were :

• Lack of good, trained and motivated faculty members. This was attributed to the fact that salaries in academia were not on par with that of the industry. (pay commission’s revisions should do some good in this direction)
• Lack of exposure to opportunities, challenges and rewards of research careers. ( this is true for colleges that are not very reputed, the quality of the faculty members are not up to the mark, which means they don’t have enough exposure … you get the point)
• Societal pressures for securing jobs, that too through college placements, rather than pursuing something that the student really wants to do. A survey of the choices of the students during the engineering seat selection process will ascertain this fact. I even know of people who took up courses they had no interest in just because it was in a college where the placements were good.
• Lack of funding for graduate students to attend conferences, workshops etc. ( though this was contested by a lot of people, I think , the problem lies in making the students aware of the funds that are available for such purposes )
• Discrimination against the students who graduate from the IIT’s versus other institutions. (though strong alumni networks are not anything new, other colleges should target to strengthen their alumni networks and not work as silo’s )

This is where I found the IIIT’s (particularly Hyderabad and Bangalore) to be very innovative in their approach. They are situated in the heartland of what can be considered seat of innovation in India. Both of them have strong collaboration with the Indigenous and multinational companies based out of their respective cities and provide for a wonderful platform for students to explore a mix of both academic research and industry relevant parts of the information technology industry. Both IIIT-H and IIIT-Bangalore have achieved recognition for their quality in the industry and academia, and that too in good time. I am positive that in a few years time, these institutions will be deeply connected to the research and development communities of the information technology industry in India  and will contribute significantly to the intellectual output of the country.

disclaimer : the numbers mentioned in this post are thanks to Ashwani Sharma, part of the External Research Programs  team at Microsoft Research India.

Give me your inputs as to what should go into a resume and what shouldn’t. If this turns out to be a good discussion, I am sure it will help out a lot of people like me.

Update: After receiving some feedback about my own resume, I am adding some more tips.

• Even if you don’t believe that technology matters ( like I do), you have to put technologies that you know in your resume. This is required as the HR’s who look at the resume’s usually filter out resumes based on skills mentioned. Not having the skills column is only going to get your resume away from good opportunities.
• Its very important that you provide your contact information in multiple forms. Phone number, at least two email id’s, home phone etc.
• Nobody cares that you won a first prize in your school’s annual dancing competition or that you have helped organize your college fest. A recruiter told me that such things are good only if you are applying for BPO jobs where you have to prove your leadership skills.
• Do not write essays about your projects. Leave it short and let the recruiters/interviewers quiz you about the same. This gives more time for conversation and a healthy dialogue.

I was supposed to be involved in a discussion about cloud computing at Cloudcamp Bangalore, but due to other commitments, I could not attend the event. I had a small writeup about the limitations and challenges in Application clouds. Here is the full text of it.

Cloud Computing is a way of providing dynamically scalable and available resources such as computation, storage etc as a service to users who can use it to deploy their applications and data. Cloud Computing can handle data in both the public and the private domain. But this seemingly harmless way of thinking about building applications has its own set of issues.I am primarily referring to application cloud providers, the kind where you deploy your applications. Not storage and service clouds. Google AppEngine would be a good example for the cloud that I am describing. I note some of them here :

From the Users perspective:

1. New unstructured and non standard paradigm of programming: Each cloud has its own supported programming language and syntax requirements for programming, though most of these clouds expose the typical hashtable based cache and datastore interfaces. There is an urgent need for standardization of interfaces and methods of programming them. One of the reasons why shared hosting environments work great is because , as a programmer, I know that I can move my PHP/PERL code to another server and it will work without too much of a fuss. Moving from one of the dozen odd cloud providers to another requires considerable developmental efforts, not to forget time (for businesses, this could spell doom).  A look back at history shows languages like SQL, C etc being standardized to stop exactly this sort of undesirable proliferation.
2. Restrictions on the programming model : For cloud based applications to be highly available, they must be easy to dynamically mirror on multiple machines. Once these applications are mirrored, they can be served on demand by load balancing servers which makes them highly available and the user doesn’t face delays in being serviced. This is an old trick used by busy websites from the early days of web publishing but these solutions were custom built for websites. So, extending this concept to cloud based platforms, servicing thousands of applications, mandates the platform providers to automate this task of replication and mirroring. This job is easier said than done. This process can be made seamless when the program stores as little state information as possible. By state, I mean transactional variables, static variables, variables in the context of the entire application etc. These things are almost a given in traditional programming environments but are very hard to come by in cloud based environments. The unnatural way of dealing with this situation is using the datastore or the cache to store state of an application. There are a lot of restrictions like lack of privileges to install third party libraries, no access to file system to write files etc ( which forces you to use the datastore and pay for it)
3. A good local debugging experience: A good local development environment, debugging experience is a must for programming on the cloud. Most cloud providers do not provide good local development environments. There is also a lack of good IDE’s that can help with programming and debugging programs written for the cloud. The providers that do provide a local debug experience, do not simulate real cloud like conditions. Both from my personal experience and from conversations with other developers, I have come to realize that most people face problems when moving code from their local development servers to the actual cloud. This is only due to inconsistencies in the behavior of the local dev env compared to the cloud.
4. Appropriate metrics and documentation of programming best practices : On a cloud, since a user pays for almost every CPU cycle, appropriate metrics on usage of processing time and memory must be presented to the users. Typically a profile of the application with function names and their corresponding time taken, memory used, processing cycles used will definitely help the developer tune his/her code to optimize on usage of processing power. The best solution for this is for cloud providers to abstract common code patterns into optimal libraries so that the users can be assured that they are running the most optimal code for a certain operation. An example of this is Apache PIG, which gives a scripting like interface to Apache Hadoop’s HDFS for data analysis. Also, Most cloud providers do not provide enough statistics and also profiling capabilities.

From the providers perspective:

Here I look at challenges that cloud providers have to face:

1. Ensuring availability of the cloud: This proves to be crucial as Clouds host critical business applications, for whom, downtime would mean monetary losses. Effective monitoring and load balancing solutions are to be built. Most clouds employ virtualization technology to get the most out of any resource. In such cases, tools should be written to figure out a resource hog early and move the application to a more powerful grid or a machine, so that the other users get their share of the cloud without delays.
2. Ensuring Consistency: Both the data and code is replicated on the cloud and maintaining consistency of data is extremely crucial. This is the reason why most transactional updates are not allowed on the cloud. Example: sequence objects, which are almost a given in traditional databases are not provided, probably because maintaining state across machines for such statements is non trivial. Problems like distributed updates, locking, partitioning, sharding etc  arise when dealing with data. Such constructs are to be provided to the users as most of it is given in the non cloud deployment space.
   Most datastores provided by cloud vendors (except the ones that provide cloud based database services) do not support relational models. Which means all object relations have to be programmatically established. This could always lead to bad code, unnecessary joins, cascading problems and tons of other problems that developers faced before working with relational datastores.
3. Program verification : One of the biggest worries about deploying applications on the cloud is the correctness of the program in execution. Erroneous conditions, like infinite loops, can not only put the machine at the risk of being overloaded and unavailable, but also cost the user a significant amount of money. Tools like static analysis should be used to analyze code uploaded on the cloud and it should be checked for infinite loops, possible race conditions,  null references, unreachable code etc. The code uploaded should also be optimized or suggestions should be provided to the users about how they could optimize code to best utilize the available resources.

Conclusion : The cloud should become a complete nonrestrictive platform for applications. There should be no restrictions on the constructs, functionality and privileges on the cloud. Also, it should be dead simple to move everyday applications onto the cloud without too much of rework. This could mean writing migration utilities, import/export options and other artifacts that make the transition to a cloud much easier.  This will prove essential as most live applications, at least currently, do not run on a cloud and helping them migrate easily will mean more revenue and adoption.

I used friendfeed because I wanted to start a conversation based on the items I shared. But, again like all social media sites, my ability to get the conversation started depended directly on the number of people who were using friendfeed. So, I sat down to fix the problem myself and after one nights work, Moffe was born.

For the unitiated moffe stands for My Own FriendFeed Emulator. It gives the same features that friendfeed provides and also provides an easy way for people to leave comments on the items I share. Plus, the link that I shared, is federated directly into the page. The outcome is that now people can leave comments on items I share plus see the page all with one click. I have also incorporated canned comments for that restless user who doesn’t have time to write in comments. Plus, I get to monitor moffe on this cool dashboard.

In essence, its a microsharing service which lets me keep my content on my site and not rely on other services like friendfeed ,twitblogs etc. Thanks to Easy on the Slaw for giving me the twitter wrapper in PHP.

If the programming language, an abstraction of the real machine code, ever helped solve a problem, it was that of uncertainty. Take an example of the piece of code given below.

    // sample code to add two numbers  
    int a=10,b=20,c=0;
     c= a+b;
     Console.WriteLine(c.toString());

When I run this code on any machine, I am assured to get the value of c to be equal to be 30. I know when I access the variable “c” the next time, I will find it contains the value of 30. I know that two instructions from now, variables a,b,c will be available for further manipulation.

My recent attempts at programming on the cloud has taught me several lessons, the most important one being, programming to deploy on a cloud is almost like writing programs that you can never be certain about. You can never maintain application state. This means no static variables, no relational datastore, no freedom to write into the filesystem etc. Think about it for a second and it will make sense why these seemingly harmless actions are prohibited. Filesystem access is a big no-no anywhere, but as for static variables, persistent classes, singletons etc, running this on many actual/virtual machines means, all these entities with their values have to be moved/replicated across the cloud. This becomes a non trivial problem especially when the state keeps constantly changing. I could live with all these restrictions by coding, painful but effective, workarounds. What I can’t do is, work with uncertainty. Here is an example :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  from google.appengine.api import memcache
 
def get_greetings(self):
  """get_greetings()
 
  Checks the cache to see if there are cached greetings.
  If not, call render_greetings and set the cache
 
  Returns:
    A string of HTML containing greetings.
  """
  greetings = memcache.get("greetings")
  if greetings is not None:
    return greetings
  else:
    greetings = self.render_greetings()
    if not memcache.add("greetings", greetings, 10):
      logging.error("Memcache set failed.")
    return greetings

The code is an example on using the built in caching mechanism on appengine. Notice the line of code given below; its supposed to return the value of the item in the cache with the key greetings

greetings = memcache.get("greetings")

Here’s the question: what is the guarantee that the value, which I inserted into the cache with a large timeout, is actually available. Whenever I write this line of code, do I have to write the failsafe code also(line 15,19) ? I am trying to model state using variables in the cache, mainly because its the next best thing to persistent classes and is less expensive (computationally and financially) than the key/value datastore. How do I reliably do this ? I cant trust that the cache will be available and have to keep on constantly updating the failsafe mechanism ( in case of appengine, the datastore) which is inefficient and highly taxing on the application. What has given rise to this situation is the environment of the cloud. Its not a new problem by any means. With the introduction of new languages, language constructs and other programmatic abstractions, this kind of uncertainty in programming has always reared its ugly head. The lochness of the programming world. And it will continue to do so; which is why we will have constructs like the assert(). My greatest worry is that I don’t see an elegant solution in the foreseeable future.

/* Get the latest 10 photos from flickr where the photo name contains cat */
SELECT * FROM flickr.photos.search WHERE text='Cat' LIMIT 10

As you can clearly see the language makes querying a service and receiving its response really really simple. This is how most new psuedo languages are. They work with service end points and emulate an existing programming language’s syntax to do that. These languages are built with mashup’s in mind. The dangers of such an offering are already imminent. Services are good as long as they are up and live.   Take for example any of the Google or yahoo  Api’s and you will find wrappers written by people in such pseudo langauges to make your life simple. Even in the enterprise space there are such languages being built which query custom services and makes building applications really really simple.

Another observation of mine involves loose typing in these languages. Most new languages are loosely typed. Most of them take from python which lets the user take care of the typing. SQL by far has been the most emulated language amongst these pseudo langauges. Take for example JoSql to add SQL like capabilities to operations like file handling or Linq in .NET which exposes a sql like interface to datastructures. These improvisations have dramatically reduced time to turn ideas into code and rapidly prototype the application.

There are limitations to using such improvisations; some that even I can vouch for. Loosely typed and unstructured languages are good as long as you are not working on large scale systems. If you are hacking up a solution to a problem that you are facing, these pseudo languages look to be real problem solvers but when it comes to working in teams, projects that need to go into production, you start getting into big problems. Though I am a python fanboy, I faced problems when I was working on python and perl on a large project with a team. Interfaces would be unclear, poor documentation would literally spell doom and tons of other problems that we never thought we would face. There are others who complain of the very same thing. I am guessing we will see a flood of such languages in the future thanks largely to applications evolving slowly into services and it will be difficult to guage the quality of these services. Twitter’s API tried to make their service more stable but the mechanism they chose didn’t satisfy many developers.  Lets hope we figure out a way to make these more reliable and stable. I guess its the developers call to be judicious about what language and service to choose when building applications.

 The answer is of course non trivial and I will only make a fool of myself by trying to predict the answer, but there are certain inferences I have made based on my observation which I shall pen down. I divide the broad category of pc users into prospects and adless users. Prospects are users who are new or ignorant to concept of online advertising, people like my mother who doesn’t know people endorse links for money. This category could also include people who are open to targeted advertising and see a value add with it. These are people who wholeheartedly click on interesting links. The other category I call adless users, users who have been around the internet long enough to understand irrelevant ads and can spot and ignore ads in a page. Take the example of the ad below.

The adless users wont even realize they scrolled though the content and didnt even notice the ad, or read it. 

More time a user spends online, the probability of him/her realizing the web is filled with irrelevant ads and over time becoming adless users. As a result, almost all users tend to move towards becoming adless users. This is dangerous for marketers, ad companies, publishers etc as there is a whole eco system depending solely depending on money made out of ads. As new users discover the web, their prospect phase is what publishers can hope to cash in on, but eventually the shift will happen. What happens then ?

Search engines are arguably the best places for advertising and probably the best place for demonstrating the phenomenon I call intrusive or endorsed content. Take the example below. 

 Ads will stop being sidekicks and move into the foreground, I have shown the shift pictorically.  Payperpost got the next concept right, people wont read ads, but social media yes, so pay people to write about your product/service etc. More results on search engines will be  endoresed and most of them already are, how do you know a review you are reading of some product isnt already endorsed. Now here is the strangeloop bit, you could say you will search for bad reviews instead of good like this. It wont take long for the advertisers to see this trend as well and then pay for people to write moderately bad reviews inturn endorsing the product. You know that they know that you are looking for bad reviews !! 

 A surprising result on top caught my eye. A visit to the site will tell you immediately that the site isn’t half as good as the second or the third result, but still its on top. SEO has come a long way and to cheat search engines into making a page popular isn’t that hard. You can hire professionals to do that job. That in a wierd sense is a form of endorsing, a professional SEO group can start bidding for making pages more popular and start their own cartel for endorsed content. 

 The other strange phenomenon I see that people recognize big search brands, Google in particular, but don’t necessarily relate to the results( you can’t possible relate to the results). You could have the Google homepage serving ads from ask.com and nobody would know the difference if the results looked like Google returned them. Thats probably the reason there still are companies trying to capitalize on the search market. Take a look at the results page below.

In this case the difference between a result from the index and an endorsement is a mere patch of color. How difficult do you think it is to remove that demarcation during difficult times.  Ethical boundries as meagre as color differences can be crossed very easily and corporations have showed time and again it can be done. 

Thanks to the falling prices of bandwidth and also social media, video is the next big delivery mechanism and it was quite understandable that Google paid a billion and a half to capitalize on youtube’s huge market share and put intrusive ads on videos( you dont have a choice there, no adblock plus !! ). Same goes with pictures and audio. Radio, papers and the television have been doing it for years. 

The world thought that we moved away from pop up advertising but we have just made the situation far worse. Ads will become more and more intrusive and there could come a time when content and advertisement are indistinguishable. More on this later.

I know when I look at sites like Digg, Reddit and Delicious people have gone through these links and painfully tagged and saved these links. Which means that with a very high probability its not marketing gibberish or spam. Folksonomy should definitely be given the credit for making life more organized. While searching for certain tags(delicious), I also discover other related tags and then run more filtered searches to improve the relevance of the results. The web, at least for the moment, is said to be partial to content on computers (technology in general, iPhone in particular J ) and for the zillion other domains that the web doesn’t do justice to, traditional ranking methods do little to improve relevance. Human in the loop is definitely better for such queries.

One of the projects that I have worked on is targeting this very need for non trivial searches. Silverfish, is a semantics extraction engine for academic documents and courses. The indexed results and the social aspect of the site are used to update researchers on the latest in their fields of interest and also recommend fresh material. When more and more people start using the internet there will be an increased demand for searches not related to technology and in such cases human in the loop searches will definitely take front stage. Future of search is definitely going to be more interesting.

First and fore most you need to to get Wordpress. Wait for 2.7, its awesome, with Quickiepress and customizable screens, blogging will never be the same again. If you dont own your own site, then get a wordpress subdomain. Though such a step will mean you wont be able to use most of what I write about next. Good, now lets get some plugins :

1. Akismet : I think this is shipped with wordpress by default but is seriously one plugin that you need. Its stopped nearly 50,000 spam comments.

2. Adsense- deluxe : Manage ads in your Wordpress posts by using simple shortcuts to insert different types of ads. Really useful for people displaying ads ( multiple ads ) on a post.

3. Comment-email-responder : This plugin lets the author reply to a comment via mail if the user has provided his/her mail id. Its useful as it drives the discussion forward and also is a way to bring people back for a good discussion.

4. Syntax highlighter : Syntax highlighter is an awesome plugin for developers. Most developers end up writing some code on their blogs and lets admit it, code is not so elegantly displayed on most blogging engines. Syntax highlighter make code look really pretty on you blog. It supports C++ (cpp, c, c++), C# (c#, c-sharp, csharp), CSS (css), Delphi (delphi, pascal), Java (java), Java Script (js, jscript, javascript), PHP (php) Python (py, python), Ruby (rb, ruby, rails, ror), Sql (sql),VB (vb, vb.net),XML/HTML (xml, html, xhtml, xslt). Take a look at this section of code.

 # python's functional programming support is awesome
 # you can use three functions map, reduce and filter

 # map(function, sequence) calls function(item) for each of the sequence's items
 #returns a list of the return values

  >>> map(lambda x: x*x*x, range(1, 11))
  [1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]

 #reduce(function, sequence) returns a single value by performing tree based reduction on f(item)
 #For example, to compute the sum of the numbers 1 through 10: 

 >>> reduce(lambda x, y: x+y, range(1, 11))
  55

5. Google-Analytics plugin : This plugin lets you put your adsense code in a single place without having to edit any of the files. If you are using analytics for your blog then get this plugin, makes the job hasslefree.

6. Subscribe to comments: I subscribe to comments feed for a post when I am interested in a discussion, but for users who dont want to take the pain of managing rss, you can give them an option to follow further comments by email. Its great to drive a good discussion again.

Well thats about it, have fun blogging!!

A friend of mine along with a bunch of other contributors has started Desitech.in a blog site that covers a wide variety of upcoming technologies, events ( now thats something that I look forward to) and interesting startups. Here is their description about the site:

Desitech is a technology journal which covers events, products and technologies relevant to the Indian audience. The journal features various columns including event announcements, event coverage, interviews with personalities, startup profiles, product comparison, etc

Seems to be interesting. Prash, best of luck on the project.

These stats are for both me and my mirror, actually the place where I was born.

Posts : 322 posts
Comments: 378
Spam Comments :44,083
Pageviews: 75,289
Visitors: 32,877
Average 2.5 min/visit
Total money made redeemed = Rs 800
Total money made yet to be redeemed = Rs 1269 ( waiting for 4500, till it redeems, damn adsense)

One of my articles even made it to 25 newspapers in the US. Some have even been added to the contribute section of MSN India

Anyway, I hope I get the same response (or better) for many more years to come and hope my author can put in credible and creative content and not make do with such lame attempts at posting.

( Author : ah screw you!! )

#@!* you hippie !!

- Most Linuxes come with python interpreter, if not use a package manager to get the latest interpreter.

- Read the Python Tutorial by Guido Von Rossum. The most (100 pages) you will ever read about python, beacuse its faster to do than read about python.

- Any good editor will do, but if you are one of those IDEated individuals, use the pydev plugin for Eclipse.

- For a simpler feel, you can get the python plugins for Gedit from this location. Just install it and my favorite, GEdit becomes python wise.  My friend also suggests using Emacs for python development.

Thats about it !! Please feel free to add comments on what else you read, did, installed to become a python developer.

Also, people who are using rails 2.0, please go the 2.0 section directly. If you start using 1.0 tutorial, you may get lost after a certain time and the results wont show. Check what version of rails you have installed and then use the corresponding section. Best of luck!!

Tags: rubyon rails, ror, rails, itsmeritesh

  After a fight with my significant other who is trying really hard to improve relevance on Yahoo search results, I realized that I wasn’t doing justice to search results by ordering top results in top down fashion. I have changed the code to now show results in left-right fashion. Now the most popular results will be available on top of the screen and you don’t have to strain those precious eyeballs of yours to look all the way down to see the results. A simple change but for a goo cause.

 When I get the time, I shall also incorporate search suggestions on your toolbar, but till then make do with this.

Tags: BOSS, yahoo, search, More Y!

 If you are using a widescren display or a theatrical display with resolution sizes bigger than 1024 *768 then use More Y! . More Y! displays thirty results in three columns and looks legible on widescreen displays (no guarantees for normal displays though). Whats even more handy is that More Y! can be integrated into your favorite browser (Firefox 2+ or Internet Explorer 7+ ) by following the instruction provided on the More Y! page. Its a cinch.

 Have fun with More Yahoo! and do reply back with comments and feedback.

Tags: boss, search, More Y!, yahoo, itsmeritesh

BOSS is simple. Really really simple and can do wonders if you are planning to build a search engine with your own flavor. Unlike other searches, BOSS gives you XML/JSON, meaning you can re-order results and present them in any way you like. Add flash, css, javascript, canvas elements whatever to build that unique search experience. After the cuil ripoff Yuil, which got taken down and was relaunched again as 4Hoursearch (why did they call it that ? figure it out Einstein !!), I was sure BOSS would be easy, but didn’t know it would be this easy.

Its only recently that I started learning Python, and I suck at it, so I picked my old favorite PHP as the language of choice ( I suck at PHP too, but suck less compared to Python). Got myself an Application ID to use BOSS. Used PHP SimpleXML parser to get a URL of choice and Voila, I had my results in an array. Wrote some really rudimentary CSS to match the aesthetics of my site and my site search was done .

Check out my Yahoo BOSS powered site search here !!

If it wasn’t for my crappy PHP skill level, Im sure I could have wrapped up the entire thing, right from “Duh, what is BOSS ?” to the implementation, in under an hour. If I do find more time to kill whilst I wait for dinner, I shall experiment with different displays for search results from BOSS. Y! BOSS is truly open and in keeping with the Open Source spirit, I have shared my rudimentary site search code. You wont believe it but the code , with proper convention, HTML and CSS comes up to 65 lines . Isn’t it awesome ??

Get the code for site search here.

Another wonderful manifestation of this concept is that you can now build custom search engines that will search only the sites that you catalog for information you need. Check out Y! BOSS.

Tags: yahoo, BOSS, sitesearch, search

 Lets take the example of facebook. The apps killed the facebook profile page. There were days when a single page would take 15 minutes to load thanks to the 50 odd apps that people had added to their profiles. Facebook’s new design rocks coz they have removed the clutter. Facebook’s usp, the newfeed, is what is shown as the profile , which is nice considering you get to see what the person you are seeing is upto recently. The apps have moved to a completely new page which means, no clutter Also the profile information, which people hardly change is gone into another page. Another cool addition I saw is the “What are you doing now ?” is highlighted and big. Given the run twitter is having with its fans, Im sure facebook realizes that micotexting about yourself is the rage right now and integrates that at the head of your homepage. Pretty ingenious if you think about it. Not all redesigns work , for ex , I hated the analytics redesign for the first few days and so was the redesign of wordpress but regular usage and some groking made me very comfortable with the interface. When you are redesigning interfaces especially for sites like flickr, facebook then it becomes essential that you get a value add and Facebook certainly nailed this one.

 Now lets go back to del.icio.us. Firstly the dots are off and you can access it as delicious.com, which saves me the trouble of explaining people the position of the dots in the url. I like the old del.icio.us and its UI. It was very basic, no frill, plain text based UI. The reason was probably the speed which it would work. But one thing that annoyed me was the clutter.The URL, Description, tags, other taggers, edit  and remove links etc were fit into a really small space; the new UI had to break this clutter.  The other problem was the tag cloud on the right. The older version would buch up tags and it would be difficult to spot the tags if you had more than say 200 which is common for most delicious uses. The new UI does a wonderful job of reducing the clutter. All the above mentioned items of the URL are still present in the same space and look darling. The tag cloud has also become more navigable and the social aspect of the site is utilized superbly if you see your networks page.

 I am now looking at some of the properties that I have built that can possibly be redesigned for better usage.

Tags: interfaces, redesign, facebook, delicious