Jay Fields' Thoughts

Polyglot World

blogger@jayfields.com (Jay Fields) — Tue, 03 Nov 2009 15:06:00 +0000

At QCon 2008 Steve Vinoski told me he uses at least 4 languages pretty much every day. At the time I thought 4 seemed like a lot of languages to use. Are we ready for a world where programmers need to know so many languages?

If you think about building a web application, you probably need to know a server-side language, HTML, Javascript, CSS, SQL, etc. Of course, there's no easy way to draw a line and say that those are, or are not all languages. I'm not sure the distinction matters, what does matter is having effective tools to get your job done. Maybe 4 languages isn't surprising.

I've worked on projects where C# and SQL were all we used; however, I think those days are coming to an end.

As an example, let's start with books. I'm currently reviewing Martin Fowler's upcoming DSL book. For his examples, Martin chose Java and C# where possible, and Ruby where a dynamic language was necessary. I think his language choices are pragmatic and wise. It's easy to agree and not give the idea another thought. However, his choice is a departure from his traditional usage of Java and C# exclusively.

Martin didn't add a new language because he changed his mind about dynamic languages, he added a new language because our industry changed it's mind about dynamic languages. Dynamic languages are now an acceptable choice for many projects, and an appropriate language addition for authors looking to appeal to the widest audience.

If Martin ever publishes another version of Refactoring, I expect it will have examples in several different languages. Refactoring: Ruby Edition contains several refactorings that are impossible to do in Java and C#. I have no idea what languages will be popular at the time of another Refactoring release; however, I think we will see a diverse set of languages and refactorings that target specific languages.

However, the polyglot movement can be found in the industry even more so than in books. Twitter is known to use both Ruby and Scala. Google uses C++, Java, Python, and Javascript. At DRW Trading we use any language that will help us get to market faster. Any given day, working solely on my project, I could be working with Java, C#, Clojure, Ruby or other even more interesting options. Steve's "4 language" statement doesn't surprise me at all these days.

You can also look at the multiple languages on the CLR and the JVM as further proof that as an industry we are willing to look to multiple languages to solve individual problems. The interoperability allows us to use the languages and paradigms we prefer for specific tasks, while working toward a greater solution. From personal experience, using Java and Clojure together has been a big productivity boost on several occasions.

As I already mentioned, in the web world you often need to master several languages to be effective. As a young developer, learning each language is probably a daunting task. However, I think having several targeted languages is part of the reason that web application development has been so successful. Targeted languages often give you the power and simplicity that allow you to get your job done without thinking about overly complex abstractions.

People often forget that "one language to rule them all" also implies that the language will attempt normalize everything. However, there are obviously efficiency gains to be had if you choose languages that play to the strengths of a paradigm or platform. The web proves this by having languages that target and solve specific problems. I wouldn't want to write CSS or Javascript in another language, even if it allowed me to solve all my problems with only 1 language. I prefer languages that increase my effectiveness to languages that reduce my learning requirements.

Ruby took a bold and productive step by introducing features that worked exclusively on Linux. As a result, several tasks are very simple to do on Linux and impossible on Windows.I applaud the choice. It makes my life easier when I'm on Linux, and I know I have to find another solution when I'm working on Windows. I'll take those options over some poor abstraction that feels painful on both platforms.

And, that's where I see our industry moving. Languages targeted at specific problems and run on specific platforms. The languages will be easier to create and use. There's been a movement in our industry towards simplicity for some time. This is just another logical step in that direction.

The days of one language for all problems and platforms are dwindling, which is nice. I prefer the polygot world.

Refactoring: Ruby Edition available.

blogger@jayfields.com (Jay Fields) — Thu, 22 Oct 2009 01:08:00 +0000

Refactoring: Ruby Edition is available (and In Stock) on amazon.com.

Sorry it took so long, I hope it is worth the wait.

Pressure, Expressed in Initial Development Time

blogger@jayfields.com (Jay Fields) — Thu, 10 Sep 2009 13:18:00 +0000

def Initial Development Time: In software development projects, initial development time (IDT) is the length of time it takes from the project's first line of code until the business derives notable value from it.

I've done plenty of projects in my career, some with an IDT of a few months and some with an IDT of a year or more. Based on those projects I've decided that I like the following equation to express the pressure felt by a team at any given moment during the IDT.

pressure = Fibonacci(current month of IDT)

note:This equation assumes all other variables are normal. Obviously a team that must finish a project in a month or suffer dire consequences will feel a much different amount of pressure.

During the first month, the team feels 0 (which represents almost no) pressure. There's no legacy code, few broken builds, no requirements issues. During the second and the third month, the team feels 1 (which represents more, but still very little) pressure. The first mistakes and hurdles have been found. The architecture is being pushed.

The following months follow the fibonacci sequence. At month four, the team feels a pressure of 2. At month five, the team feels a pressure of 3. And, so on.

The sequence isn't meant to be out of 100, 1000, or any other number. It's purpose is to show the progression of pressure as the as months of IDT pass. In an ideal world, a manager would understand that a team under extreme pressure will underperform, and the manager would do everything in his or her power to reduce the IDT.

Consequences
Pressure can be good in moderation. Many Agile teams have iteration point goals which pressure the team to get a reasonable amount of work done per iteration. Unfortunately, pressure also tends to have the boiling frog effect.

Pressure often leads to compromises on quality. Far too often I've heard "let's just get this done, and then we'll clean it up." This is a good strategy if you are planning on cleaning immediately after completion (red/green/refactor). However, if you plan on cleaning it up "in the future", that step (the majority of the time) is forgotten.

Fibonacci works well for this (quality related) equation also.

% of code that "sucks" = min(80, Fibonacci(current month of IDT))

Most people wont admit that the code they are currently writing is poor, which helps explain why the percentage rarely reaches over 80%. Also, there's often some code that was written early, and never needed to be modified. Late in a project, looking at that code may be the only thing that keeps you sane.

sidenote: 50% of the time "this code sucks" can be directly translated to "I didn't write this code". On a project with many months of IDT you are bound to experience some turn-over, thus the amount of code that "sucks" is destined to go up. And, even if the original code didn't suck, the modifications done by the person who only half-understands the code probably do suck.

Innovation also seems to suffer on projects with many months of IDT. A typical (well functioning) project team tends to spend 20-30% of their time experimenting with new ideas that will make them more productive in the future. When innovations are found the dividends often greatly exceed the original time investment. Even when no innovations are found, the team often benefits from the experience gained.

Unfortunately, during IDT people begin to forget the value in innovation. They seem to visualize a path to the finish and charge down that path full speed with no regard for hurdles or environment changes. They are often unable to see the value in any time spent that doesn't already have "proven" value. Very soon, you begin to hear: "let's just get this done, and then we'll look at better solutions."

Between (unclean? dirty?) poorly written code and lack of innovation, the chances of dragging a code base out of darkness dwindle quickly as the IDT months drag on.

Lies
Many Agile believers will tell you that the problems with pressure can be alleviated with Milestones. This is patently false. A solution isn't valuable to a business until it provides notable business value. You may be 13 iterations into the project with 2 milestones behind you, but if the business isn't deriving value, you haven't done anything more than show them partially working software. Partially working software is great for building trust, but it alleviates zero pressure with respect to IDT.

Options
The only real remedy to the negative consequences of a long IDT is to create a shorter IDT. There truly is no alternative.

Unfortunately, some projects truly do take many months to deliver business value. I've found that some pressure can be alleviated if the team recognizes and adjusts to the pressure.

The 3 most helpful things I've found are:

Weekly (scheduled) 1 on 1 discussions between the team lead and each member of the team.
Take vacation. One person out for a week isn't nearly as catastrophic as a team full of burnt out people.
Set aside, and always use, experimentation time. Innovations can give you a big boost that the project will definitely need in the long run.

Thanks to Lance Walton for feedback, and the phrase "Initial Development Time"

Staying Current: A Software Developer's Responsibility

blogger@jayfields.com (Jay Fields) — Thu, 20 Aug 2009 13:44:00 +0000

I have a personal hatred for weekend conferences*. To me, a weekend conference ensures that I'll be "working" for 12 straight days.

I understand that opinion isn't universal.

Some people have problems getting time "off" to attend conferences. These situations feel like a fundamental misunderstanding of a software developer's responsibilities. Part of your (software developing) job is staying up on current technologies. That means doing some research during your day.

(almost directly stolen from Ward on technical debt)
If you spend your entire day coding and never looking at new things, you accrue productivity debt. In the short term (say the last week of a release), it makes sense to take on a little debt. However, in the long term, assuming little or no payment, the interest (where interest equals the gap between your skills and the current solutions) will render you a NZPP (Net-Zero Producing Programmer). In a typical organization you can coast as a NZPP for around 6 months and slowly transition to a NNPP.

It is your responsibility not to become a NZPP (or NNPP). Most talented software developers refuse to work with NZPPs. At the point that you become NZPP, you usually have to declare bankruptcy (with regard to software development). You generally have two choices: take a much lower paying job where you can learn new things or move into another role. If you want to be a software developer, neither of these outcomes is desirable.

Luckily, you have the power to avoid becoming a NZPP. Most employers will happily buy you technical books and send you to technical conferences. In my opinion, whether or not you took advantage of these benefits should be noted on your performance review. Not staying current as a software developer, especially when the opportunity is offered to you, is software malpractice.

I once created a list of things I look for in potential team-mates.

Have you tried Test Driven Development? Can you name something you liked and something you disliked?
What language(s) that are gaining popularity, but not yet mainstream, have you written Hello World in?
Do you read books or blogs looking for new ideas at least (on average) once every two weeks?
Do you at least attempt to learn a new language every year?
Have you ever run a code coverage or cyclomatic complexity tool against your codebase?

A commenter said something along the lines of

Not everyone has the personal free time to dedicate to doing all of these things

And, that is the fundamental flaw. Employees (and even some employers) seem to think that these are activities that should be done in your off time. I can't disagree more. These are things that a responsible developer needs to do as part of their job, thus it can be done during work hours.

20% time isn't something Google invented, it's just something they named, formalized and made popular. The activity itself is something good software developers have been doing for years. I applaud Google for making it a standard, thus ensuring that it's employees always have the opportunity to stay current. However, your company doesn't need to standardize on 20% time for you to stay current.

It's your responsibility to make time in your day to read a book or a blog.

You should also take advantage of a company sponsored trip to a conference. If you've attended conferences before and derived little value, I highly suggest the QCon conferences and JAOO.

Once you start doing research as part of your job you'll find that conferences are just like work, except the focus is 100% on research. And, it's not something you want (or should have to) spend your personal time on, it's just another productive day of doing what you have a responsibility to do.

* Which is why Josh and I run SpeakerConf Tuesday-Thursday. You can travel to, attend and travel home without missing a weekend day.

Macros Facilitate Expressive Code

blogger@jayfields.com (Jay Fields) — Tue, 18 Aug 2009 03:38:00 +0000

Someone once asked me if I thought Clojure was more expressive than even Ruby. I didn't have enough information to form an opinion then, and I still don't now. However, I recently noticed something that led me to believe the answer could actually be yes.

I was looking through the code of clojure.test on Friday and I noticed something interesting. In clojure.test, the form(s) passed to the "is" macro are wrapped by a try/catch. It caught my eye because I often want to do the same thing in other languages, and usually I have to settle for much less elegant solutions.

Here's a bit of example code to help create some context

(deftest a-test
  (is (= 1 (some-function-that-throws-an-exception)))
  (println "this code still executes"))

For this example to work you'll have to ignore the fact that you probably don't want this behavior. In practice I prefer my tests to abort on the first failing assertion; however, in this blog entry I'm focusing on what's happening, not what I theoretically prefer.

In the example I call a function the same way I would anywhere else, and the framework has full control over what happens if my function throws an exception. This is accomplished when the "is" macro takes the forms and manipulates them into something similar to the original code, but with additional capabilities.

This particular example struck me as one where macros allow you to write only what you want, and the macro adds the additional behavior that you desire. However, the key is, you don't have to do anything special to get this additional behavior.

Consider trying to do the same thing in Ruby. In Test::Unit you would need an assert method that took a block.

def safe_assert_equal
  result = yield
  assert_equal result.first, result.last
rescue Exception=>e
  puts e
end

class Testable < Test::Unit::TestCase
  def test_something
    safe_assert_equal { [1, 2] }
  end
end

The additional syntax isn't drastically invasive; however, you are forcing additional syntax and requiring the understanding of why it's necessary.

Of course, in Java things would be even less expressive. The most likely solution would be to put the assertEquals in a Runnable, but I'd be interested in hearing other ideas. Regardless of the solution, it would obviously be invasive and take away from the readability of the test.

Being able to only say what is necessary is powerful. Stuart Halloway likes to talk about Essence and Ceremony, where Essence is the ability to say only what you want, and Ceremony is all the additional things you are required to say.

Macros seem to be a powerful tool for those looking to write Ceremony-free code.