Regular Geek

Finding A Job That Fits

Rob Diana — Fri, 19 Mar 2010 14:40:56 +0000

Over the course of my 15-year career I have found some things that I liked about jobs and other things that I did not like. It has taken me a while to figure things out, but most people will have one type of job and company that “fits”. Of course, the logical question to ask yourself is what type of job fits me? I have previously written some tips for finding your next software development job, and there is one paragraph that needs to be repeated here:

The first thing you need to do in your job search is determine what type of work you want to do. Do you want to go into consulting for a services firm? Do you want to go totally independent and freelance? Do you want to go into a big corporation or even smaller company of maybe 50 to 500 people? Do you want to go into a startup and work with maybe 10 people?

So, what do you need to consider?

Autonomy: Please tell me what task to do next. If you really want to be given tasks, complete them, and get another task, then startups probably are not for you. Big corporations may be a better fit.
Bureaucracy: I want to fill out forms and follow process. Some people really need to follow predefined processes in order to get things done. These people probably do not mind filling out forms to get things done. Other people avoid the processes on purpose, just to see how much they can get done without them.
Stability: I want to know that I have a job tomorrow or next month. This is the biggest myth of job hunting. Basically, there is no stability anymore, just different types of instability. In a startup, the whole company could go under in the span of a month. In a large corporation, it would take years to close up completely but there would be several rounds of layoffs that you would need to survive.
Generalization: I can write code in eight different languages. If you are looking to be just a Java coder, then a large company is for you. Startups typically require you to work in many different technologies. In really small companies, you may be asked to do some server administration (yuck) or even sales work (double-yuck).
Durability: I want to work 9 to 5 with a 2 hour lunch. In software development, nobody works 9 to 5 anymore, and if they do they are also working at home after dinner. If the length of the day or typical working hours matter, this can greatly affect your choice of company.
Growth: I want to be in middle management. Even if you are a software engineer now, you may want to take the management track. You could also go towards becoming an application architect or some other fancy sounding technical title. Some people stay as a software engineer for decades too.
Functionality: I want to fix bugs all day. In software development there are various types of work that you can do. First, there are people who fix bugs all day. There are the consultants that generally do whatever work you want and you pay them money. You can build a product, or you can be more services oriented. There are also subcategories of these as well, like administrative/data entry applications, web applications, mobile applications, embedded systems, and several others.
Benefits: I want wealth and health. Any discussion of the appropriate job would not be complete without looking at the compensation package. Obviously, your salary or pay rate is important, but what other benefits are there? Health insurance, prescription plans, 401K plans, tuition reimbursement and training are just a sampling of what companies may offer.

So, how do you know what type of job to get? Well, first you need to figure out how these concepts relate to many companies. Because this would take thousands of words to describe, I am going to give you a handy table. The table maps these concepts to different types and sizes of companies. I have not included “Functionality” as part of the table as all sorts of companies can build products or be services companies. I have also excluded “Benefits” because they change with each company and have no relationship to the type or size of a company. Please note that these are generalizations of a typical company of the given type and size. Obviously, your current company may be somewhat different.

Company Type	Employees	Autonomy	Bureaucracy	Stability	Generalization	Durability	Growth
Tiny Startup	0 to 20	Very High	Very Low	Very Low	Very High	Very High	Low
Small Startup	21 to 100	Very High	Very Low	Low	Very High	High	Low
Medium Startup	101 to 300	High	Low	Medium	High	High	Medium
Large Startup	301 to 1000	Medium	Low	High	High	Medium	Medium
Small Corporation	200 to 1000	High	Low	Medium	High	Medium	Medium
Medium Corporation	1001 to 5000	Medium	Medium	High	Medium	Medium	High
Large Corporation	5001 to 20,000	Low	Very High	Very High	Medium	Low	Very High
Huge Corporation	Above 20,000	Low	Very High	Very High	Low	Low	Very High

The values above range from Very Low -> Low -> Medium -> High -> Very High. In some cases, the value is the expected value, in others it is the required value. For example, you should expect a very high level of Autonomy at a small startup and working there will require a high level of durability as well as require a very high number of general skills.

There is probably a better way to display this information, but at least this gives a starting point of what you may want your future employer to look like. Are there other basic concepts that you look for in a company?

NoSQL Job Trends – March 2010

Rob Diana — Sun, 14 Mar 2010 13:38:20 +0000

In the past, I have posted about job trends for traditional programming languages as well as Web 2.0 programming languages. This is really just the tip of the iceberg when it comes to skills that jobs may require. In this installment, I wanted to look at a current hot topic, NoSQL data stores. With the current state of the internet, RDBMS-alternatives are being sought due to the amount of data being generated and the various ways this data needs to be accessed. As was done for the previous job trends posts, I looked at the trends from Indeed and SimplyHired to see what technologies were garnering demand. In order to get a feeling for how these technologies were growing, I have not included traditional RDBMS systems (MySQL, Oracle, DB2, and SQL Server) as they would skew the data. The initial list of tools was source from Rick Cattell’s Scalable DataStores page, which also links to some excellent PDFs.

You will notice that two tools are missing, Hadoop and Cassandra. Cassandra is not included as there was too much noise in the job data that could not be reliably filtered out. Hadoop was not included, though HBase is part of it, because it skewed the results in a way that made the other trends hard to decipher. So, when looking at these trends, you should assume that both Hadoop and Cassandra should be reviewed as well. The tools that are included in this analysis are Redis, Voldemort, SimpleDB, CouchDB, MongoDB, HBase and Hypertable.

Let’s start by looking the the basic trends from Indeed.

As you can see, the NoSQL movement is fairly new with HBase being the earliest arrival around May 2008. Even for the short trends, you can see that all of the trends are increasing fairly rapidly. HBase looks like it will continue its lead due to it being a component of Hadoop, the most popular NoSQL framework and an Apache project. Hypertable seems to be struggling to maintain momentum, and Redis is a relative newcomer based on these trends.

Now, what do the trends for the same list of tools look like on SimplyHired.

Due to the shorter timeline, SimplyHired gives us slightly better insight into the trends. However, there seem to be fewer jobs on SimplyHired for this set of technologies, making the trends a litter harder to decipher. One interesting note is that SimplyHired does reflect the same trend spike that Indeed had for SimpleDB around October 2009. Otherwise, HBase continues to lead the pack by a bit, with CouchDB and SimpleDB not far behind. The trend lines for the others are fairly flat or just do not have much data behind them. One noticeable difference is that Redis seems to be more popular in this trend graph than it is in the Indeed trends.

Finally, let’s look at the relative growth trends from Indeed.

The relative growth does show some interesting information. HBase is clearly growing quicker than the others, so we can assume that HBase and its Hadoop siblings will continue to be in demand. SimpleDB shows the same growth spike as it does in the raw job data. MongoDB is one growth line that does stand out as it does not directly correlate to its current popularity. This shows that MongoDB could see some solid growth in the coming months. Voldemort is not showing the growth I would have expected based on its popularity, and Hypertable is not showing significant growth as it lags behind the rest of the field. Redis being a newcomer definitely affects its popularity, but it is showing solid growth so far.

So, if you are interested in these types of jobs, you may also want to take a look at this list of links to NoSQL required reading.

Are You Treating The Symptom Or Solving The Problem?

Rob Diana — Thu, 11 Mar 2010 19:51:44 +0000

A few months ago, I wrote about process rarely fixing the real problem. I remembered that post because of an issue that was reported to me, and Tac Anderson wrote a timely post about the same idea. He phrases the question a little differently by asking if you are solving problems or fixing problems, and he includes an interesting definition of both:

Solving a problem is what people do when they figure things out for the first time. When you solve a problem you create a solution. The next time you solve a problem you’re applying the solution to the problem to fix it.

Tac’s post is geared towards PR & marketing, but the idea is somewhat the same in software development. Software is never completely correct the first time it is written. There are always problems (issues, defects, bugs or your favorite term) after the initial release. Some of what software engineers do every day is problem solving in the traditional sense.

So, what happens when a defect is found? The software engineer will need to dive into the code and fix the problem. In this case, it could be that the defect occurs because a fundamental assumption of the code was violated. As an example, let’s say we have a method that takes a list of tweets (those Twitter message things). A fundamental assumption of this method would be that the list does not contain a null value when the list is not empty. So, if someone is passing a list with a null value to the method, how do you fix the defect? In many places, the originator of the error will need to fix their code to handle the situation. This type of “defensive code” litters your source code with various if (object != null) type of statements. This will ensure that you no longer throw a null pointer exception.

However, what if the fundamental assumption of the method is supposed to be true? The symptom of the problem is the null pointer exception. In this case, the source of the issue is really the calling method. It could be true that the calling method is actually passing a list that is a parameter to it. So, you would need to trace the issue back to the real source of the problem. This is diagnosing the disease instead of just treating the symptom.

Granted, my explanation is about software, but this type of process can really be applied to almost any industry. Are you fixing a problem to make it quickly go away? By doing that, the problem may go away in the short term, but it may reappear several times over the long term. If you look at this in terms of time, you may provide a short term fix within 30 minutes. However, every time that you need to fix the problem requires 30 minutes. Think about the long term savings if the long term solution only requires 2 hours.

If you put an hourly rate on this ($50/hr), the returns become more obvious. Let’s say the problem occurs once per week. If you choose the short term fix, you will have incurred 30 minutes for each of the 13 weeks. A simple calculation, 13 * (30/60) * 50, gives you a total cost of $350. If you choose the long term fix that is a one time cost of 2 hours, your total cost is only $100.

Obviously, this little example does not add up to big dollars, but you can quickly see why short term fixes tend to be more expensive. So, are you treating the symptom or solving the problem?

Are You Managing Resources Or Building Knowledge?

Rob Diana — Sun, 07 Mar 2010 13:47:04 +0000

Today I am going to write about a topic that has always bothered me. I had not thought that I would write about it, but a post this morning by Seth Godin really triggered this post. His post is really promoting his new book Linchpin (Amazon Link), but it starts with an excellent quote by Andrew Carnegie:

Carnegie apparently said, “Take away my people, but leave my factories and soon grass will grow on the factory floors……Take away my factories, but leave my people and soon we will have a new and better factory.”

I have no idea when Andrew Carnegie said this, but it is something that every company should remember. Employees matter. The question is why companies forget this fact. Godin talks a little about shareholders wanting more tangible assets. What if the company is not a public company? What if your company does not really have tangible assets like a factory? Then your people should matter even more. They should not be “resources”.

The first time I saw people mentioned as resources was during my consulting days. Not all consulting companies talk like this, but when a consulting company gets large enough, there is a tendency to move into this resource management direction. In software related companies, you see this trend quite often. As management looks at their budget, they need to determine what work needs to be done. Due to the financial focus, software engineers become yet another cost in the budget equation. This financial focus then trickles down to the people staffing the work, typically your first level managers, who then need to find the best allocation of people to work. This is the way many organizations staff their software projects.

Don’t get me wrong, there is some question of an engineer’s knowledge when staffing projects, but it seems to be a secondary concern. Earlier, I said employees matter. Why? During the time that an employee works for a company, they gain information about a company’s operations or even just gain some general experience that can help them in their jobs.

In the case of software engineers, the experiences are very valuable. There could be the knowledge gained within the specific products that a company makes, like a content management system, some enterprise software, or even development tools themselves. The other side of experience is the knowledge gained when working with new tools and technologies. In working at various companies, my experience and knowledge could be significantly different than someone else at the same company. I may have more knowledge in content management, analytics and social media integration, but my coworker may have more knowledge of XMPP, messaging (JMS/MQ/ESB) and general network protocols. In this example, can I be substituted as a resource on a project where my coworker was originally staffed?

Most people would quickly say no. However, when the time comes to staff a project you may hear excuses about the timing of projects, the availability of people and “other factors”.

Paper and money are resources because they have a finite capacity or duration of existence. They have no knowledge. People have knowledge, so why are they treated like resources?

Google Aligning Its Stars And Its Applications

Rob Diana — Fri, 05 Mar 2010 09:00:34 +0000

Earlier this week, Google announced that they were adding the ability to “star” search results. This was added and SearchWiki was removed from the search results so that there was not duplicate functionality. The basic idea is that people like the concept of saving good search results, but really could not be bothered with changing the ordering of the results. However, stars on search results are more integrated with the overall Google experience:

You don’t even have to remember whether or not you starred something. Simply perform a search and you’ll rediscover your starred items right when you need them. Stars sync with your Google Bookmarks and the Google Toolbar, so you can always see your list of starred items in one place and easily organize them. Even beyond the results page, while browsing the web you can quickly click the star icon in Toolbar to create a bookmark, and those pages will start showing up in the new stars feature.

This is a really interesting feature because you can quickly bookmark some search results, in Google Bookmarks of course. Then you can “star” any page on the web from Google Toolbar to bookmark that page. This little tidbit hidden in the search results announcement triggered an interesting thought.

Also, there were some interesting comments captured by various tech blogs, and summarily beaten to death, by a Google director regarding the future of the desktop PC:

Speaking at the Digital Landscapes conference at UCD, European director of Google’s online sales John Herlihy said that Google is mostly oriented towards mobile devices, claiming they’ll become more important than desktop PCs. “In three years time, desktops will be irrelevant. In Japan, most research is done today on smart phones, not PCs,” he said.

First, let me state that I read this quote differently than most people. He is not announcing the death of the desktop PC. He is stating that they will be irrelevant in a few years because of the rise of smarter mobile devices. The key here is that the mobile industry will be growing rapidly, and there is a lot of revenue to be won. On the desktop PC, revenue is traditionally flat because people are moving some applications to the web or they are moving to more portable devices. For an internet company like Google, the desktop PC may already be irrelevant to their revenue streams.

As things become more mobile, more data and more applications move to the web. Google has slowly been building a web application empire with things like GMail, Google Reader, and Google Docs. One of the problems they are having is the inconsistency of each interface. GMail has stars for important items, search results were personalized but used the search wiki concept, and Google Docs really looked like nothing else for a while. Slowly, all of these interfaces are starting to converge. Stars for important items are used in GMail, Reader, Buzz and now Google Bookmarks.

Google has started learning something. If they really want to be your only online resource, they have to provide the majority of your web experience. Wave has the same problem right now. It is a completely separate application that was far too much change at one time for even early adopters to adapt to. I have said previously that Buzz is a baby step in the direction of Wave, and now Wave will send email notifications. Is this a step for Wave towards a more Buzz-like experience? Again, these applications will likely converge into one basic interface in order to make it simpler for people to use them.

Google has started aligning all of their applications to be more of a unified Google experience. So, don’t be surprised when we see a bunch of these smaller updates to applications. We will see little tweaks in many, so that they slowly converge. You may not even notice it happening, until you realize one day that you spend most of your time on Google applications. Has it already happened? Or are we slowly seeing the evolution of the Google web operating system?

Web 2.0 Programming Language Job Trends – February 2010

Rob Diana — Mon, 01 Mar 2010 13:01:47 +0000

At the beginning of this month, I compared the job trends for traditional programming languages like C++, Java and others. That was initially done because of TIOBE releasing their yearly language rankings. I also noticed that is was approximately 6 months from my last job trends posts. At that time, I wrote a second post regarding Web 2.0 programming language job trends. In this post, I have changed the list of languages a bit. First, Objective C is now in the traditional programming list so it is not included here. In order to make the list a little larger I have added Flex and Groovy to the list.

So, what do the trends from Indeed.com look like?

Obviously, JavaScript demand continues to rise, growing by 100% over the last 5 years. Flex has mostly flat demand though it looks like there is some growth in the past six months. PHP is also growing rapidly, probably because it is simple to learn and you can get a decent application running fairly quickly. Python, Ruby and Rails are all showing solid growth as well, with Python being the leader. Groovy is showing a nice growth trend so far, but its demand is significantly less than the others.

Now, to compare the trends against another site, we look at the trends from SimplyHired.com.

Because the trend data is limited to 2 years, we really see the short term bumps. Interestingly, SimplyHired does not show JavaScript to have a dominant position over the other languages, though it does still lead significantly. Both Flex and Python have shown very nice growth over the past 6 months. In all of the languages, there is a growth surge in July or August of 2009, though some of the surges were smaller than others. SimplyHired does not show the same volume for PHP when compared to Flex as the Indeed trends do. Otherwise, the trends from both sites are fairly similar.

Lastly, lets look at the relative trends for job growth from Indeed.com. This is something that I missed in the last post and it shows an interesting perspective of the job trends.

What is obvious from this chart is that Groovy demand has skyrocketed over the past year. However, showing 2000% growth over a small number of jobs still means there could be a small number of jobs when compared to the other languages. Ruby and Rails have also shown an increasing growth trend that is much higher than the others. While most of the other jobs are increasing their growth, Python seems to be leading the way and could be poised for a hockey-stick growth period. Honestly, I am surprised at the rapid growth trends of Ruby and Rails, as I figured they had somewhat fallen out of favor. I also expected Python to be growing much faster, but all of this could be due to the people I listen to and blogs that I read.

Based on all of this, I can easily state that JavaScript is still something all web developers should know. This includes those mostly server-side programmers that want to build a web application. Otherwise, it looks like PHP, Ruby, Rails and Python would be solid language choices if you were looking to learn something new. Groovy looks like it is still at the point where you should learn it if your current environment uses it, but it may not be that useful it getting you your next job. Now, where did I put that Python book?

Simplification, Implicit Behaviour and Explicit Opt In

Rob Diana — Fri, 26 Feb 2010 15:48:37 +0000

Some recent events in the past few weeks have had me thinking about how we develop software, specifically software that deals with social sites. The release of Google Buzz has raised various privacy concerns from people. I am not as worried about privacy, as those issues tend to get resolved fairly quickly. The problem that we are now seeing is the difference between doing things automatically, or implicitly, and asking for explicit permission. How you handle the difference can have a direct impact on the design and usability of an application.

So, Google Buzz brought the implicit/explicit debate to the forefront mainly because of the privacy issues that it raised. However, this is not Google’s first attempt at building implicit information. Fred Wilson had a post about Buzz, but brings an interesting geolocation perspective:

If you compare Google Latitude, where you broadcast your location, to Foursquare, where you explicitly checkin to a location, I think you’ll agree that the explicit gesture is better.

And so it turns out that implicitly deriving social relationships is tricky and potentially dangerous. That doesn’t mean the idea isn’t powerful.

Obviously, email and implicit networks are a good idea, but how we expand out from email will be very difficult. Changing something as stable as email is drastic for most people. Building a social network automatically from your email contacts can be incredibly powerful, but it has some privacy concerns. Purely explicit network building does not carry the same privacy issues, but it is slow and people are frequently omitted that should not be. How many times have you gone to someone’s Twitter profile and asked yourself, “how am I not following this person?”

Take a look at FourSquare. There is a group of administrative pages where you can manage your friends. Twitter also has some basic friend management, but there is so little that it becomes difficult to do much of anything. Now, look at GMail and Google Contacts. When you email someone, a contact is created. Google automatically segments your 20 most contacted people. You can quickly assign people to groups as well. Given that this is your address book, this all makes sense. However, should all of the social applications have their own friend lists and management tools?

Where is the simplicity? Implicit or automatic tasks are the playground of simplicity. Two companies that continue to attempt to simplify things are Google and Apple. I have talked enough about Google, but look at what Apple did with the iPhone. They simplified the smartphone, made it accessible to almost anyone and did not sacrifice much in terms functionality. I have used a BlackBerry for the past 5 years or more, and refused to use many other phones. I recently purchased an iPhone, mostly because it was a good value compared to an equivalent BlackBerry, but I also had the option of returning the iPhone within 30 days. It took me a month to get comfortable with my first BlackBerry. It took me two hours to get familiar with the iPhone. I had loaded my email account, contacts, Outlook Calendar and Google Calendar and downloaded both FourSquare and Tweetdeck. The key was simplicity and the integration between the phone and its management software, iTunes. Google has been doing the same with much of its software as well, providing simple integration points to allow the user to have more power without the typical complexity.

Facebook is now trying to do the same things as Apple and Google. We can use Facebook Connect to login to many different sites. You can get your news feeds from the main page, and they are building a better messaging or email system. The integrated experience that Facebook is trying to develop is all based on implicit power of the network. As you build your network, you automatically get news or updates from your connections. There is no explicit request after the initial friend connection.

However, the question remains, where do you draw the line between implicit and explicit behaviour? The integrated experiences from Google, Apple and Facebook are winning the battle for your attention. The key to these implicit actions is to not be intrusive. As can be seen by Facebook’s Beacon debacle and the recent Google Buzz reaction, the line of intrusion is a very fine one.

How Do You Hire Programmers?

Rob Diana — Tue, 23 Feb 2010 14:33:52 +0000

Let me start this by saying that I do not consider myself one of those “10x” programmers. I am also a technology generalist, which means I may know how to use specific technologies but I am not considered a guru in many. However, I have been in the software and services industries for over 15 years and I have had my share of success. Today, my point is not to talk about me, but I wanted to give some background on where my ideas may be coming from.

Yesterday, I read a great post by Jeff Atwood on Coding Horror about The Nonprogramming Programmer. In the post, Jeff laments the fact that he wrote a post back in 2007 about the same problems:

I wrote that article in 2007, and I am stunned, but not entirely surprised, to hear that three years later “the vast majority” of so-called programmers who apply for a programming job interview are unable to write the smallest of programs.

Part of this problem is that interviews are hard. Most people tend to get very nervous during an interview which makes even simple programs somewhat difficult to write. Obviously, this means we need to take this into consideration when we are trying to make a hiring decision. So, assuming we upgrade the interviewer’s skill based on the fact that there were interview jitters, how do we appropriately rate skills? Do we rate based on how well they know a specific technology, like Java? In my past experience, this is what most companies do because they want to hire people that can be immediately productive in some way. However, does this really lead us to build great teams? Jeff makes a very appropriate comment regarding the industry’s hiring practices as well:

Three years later, I’m still wondering: why do people who can’t write a simple program even entertain the idea they can get jobs as working programmers? Clearly, some of them must be succeeding. Which means our industry-wide interviewing standards for programmers are woefully inadequate, and that’s a disgrace. It’s degrading to every working programmer.

Part of this problem is that most companies are driven by needs. If they really need a new employee to help them finish (or even start) a project, their hiring standards will likely go down. Obviously, this desperation is bad, leads to bad hires and can cause bigger problems within your teams.

How do we fix this? You need to find a system for rating your interviewees. One problem is that interviewing is a very subjective exercise. One person may see a lack of knowledge, while another may see someone lacking experience but full of potential. This is potentially a problem of differing expectations, but you can set a level of expectations after the interview as well. So, what kind of rating system am I thinking of? There are a few things you can do to make this interviewing problem simpler.

Set Position Expectations

If you are interviewing a software engineer, you probably have a title associated with the position, like senior software engineer. In order to really rate the prospect, you need to know what you expect from someone in that position. Even basic roles and responsibilities help in this regard. If you can not point to recommended years of experience, and daily tasks people will be doing, how do you know if someone fits the role you are hiring for?

Determine Why You Are Hiring

You have an open position that you are hiring for, and now you know what type of role this is. The question you have to answer next is why are you hiring this position? Are you trying to fill a particular role on a team? Or are you trying to build a team to create a new product? These are considerably different positions that may have the same title of software engineer. If you are hiring for a specific project, you may want to look for experience with specific languages and technologies. You would also need to answer the question of what this person will do when the project is completed. Will they continue to maintain the software or do they need a new role?

If you are building a team, specific experience is not as much of a concern. You really want people that are intelligent, can quickly learn new technologies and fit the culture of your team. This is where those weird puzzle questions and general programming theory can come in handy. You really want to see how a person thinks about problems and how they handle the stress of the interview.

Can They Work On Your Team?

The last major requirement in the interview process is where your opinions really matter. In many organizations that are larger than 200 people, you may be interviewing someone that will probably work for a different team. I have seen standards lowered in these cases because people think the requirements in other teams are lower than theirs. In these cases, you really need to be honest and ask yourself whether you would want this person on your team. If not, then why hire them for another group? If you do hire a questionable prospect, what happens when that person is transferred to your team after 6 months? If you are interviewing someone it is because there is a chance that you will have to work with them at some point.

This is not much of a framework for interviewing candidates, but I have seen many people go into interviews with much less thought. Employees are a major asset of every organization, so you should treat interviewing as critical a task as you treat development, QA or production support. You should treat interviewing like everything else, like your job depends on it, because some day your job may depend on that interviewee working on your project.