Regular Expressions

What Your Hiring Process Says About You

2010-03-10T15:55:00.003-08:00

What's your hiring experience like? How many "hoops" do you make candidates go through? Why do you make them go through them? In this post I'll recount some hiring processes I experienced and chat a bit about what (as a candidate) they said about the company. At the end I have a potential checklist.

1. Longest: Archiving Company

As a referral, I was spared an initial phone screen and instead was brought in person to meet an operations manager. His questions were non-technical and focused along the "How well do you deal with conflict? How well do you deal with challenging people to work with? Have you been in an environment where people were difficult to get along with?". We ran over time and I was asked to come back to talk to a project manager.

I returned a few days later and the interviewer was reading off situational HR questions, all with a tilt toward conflict resolution.

A few days later, I was asked to complete an assignment - write a service with certain features, complete with documentation and tests. Unfortunately I received the assignment on a Friday and was told it had to be completed before Monday morning. This was a surprise and a bit of a disappointment as it conflicted with birthday plans I had that weekend.

A few days later I was brought in for a third interview with a developer, project manager and product manager. Very few questions were asked about my assignment and the developer asked me some technical questions (some were unrelated to the problem domain they were in). One was my own question that my previous co-worker had brought with him to the company! Further general questions about process followed from their project manager and product manager.

As the interview was closing up, I discovered they'd like me to write two tests: a number matching test and an IQ test. Both timed, both while a loud lunch party was happening through the adjacent wall. The number matching was a doubled-sided, double-columned list. Each entry had a hand written number and a printed number; I was to mark an X on the ones that differed. The IQ test was pretty standard with tricky logic, English and math questions.

Two days later I received a call from their receptionist saying they were no longer interested. Naturally I contacted my reference to find out more. He responded that as far as he knew, they were still discussing it. He confirmed this later, it was a mistake on their part.

The next day he contacted me with their decision; they were no longer interested. Infact, he said that it was a mistake to even begin the interview process with me as they weren't looking for me.

(Had they been interested, I would have been asked back for an additional interview with the CEO.)

Notes

Needless to say I was disappointed in this process and if it wasn't for the initial reference, I would have given up on this path much sooner.

Explain the workflow to the candidate. Before I started the process I was told it was similar to a previous company I worked at. It was most definitely not. Not knowing the steps in the hiring workflow, a candidate has no idea that he's about to invest a dozen hours.

Screen for culture. Every organization has a different "culture", and it's important to double check that a person will fit in with the existing employees (or if they'll shake things up in a good way!)

Your questions will reflect what a candidate thinks is important to you. When three separate employees ask questions about navigating difficult people, constantly changing requirements and micromanagement, this is what your candidate will think your place is like. If you ask questions about networking protocols or multi-threaded programming, then your candidate will feel the position will involve both. When I asked "Do you use the technology you're asking me about?", I was surprised their answer was "Not really.".

Sell the company to the candidate. I was walked around the development area on two occasions and got the feel of the environment. I was told about some of the unique perks of the company too. This kept me interested.

Involve only the right people. Out of the five people who interviewed me, one was for technical purposes and the other I would potentially be reporting to. The other three were brought in "just to meet me" and "had no real serious questions to ask". As a candidate, this tells me your decision making process requires people who may be uninvolved with a decision to approve.

Limit an interview time. After about 90 minutes, my ability to continue to answer a barrage of questions begins to diminish. After two hours, I really felt that writing a surprise number-matching and IQ test was quite unfair.

Hiring processes should be fail-fast. As soon as an organization realizes it's not interested, the process should stop. Clearly this one suffered from poor internal communication and lack of direction. Consuming not only my time, but employee time too.

The "assignment" was an interesting project, but given the investment I made and how little it was referenced, I was quite disappointed to waste my time on it. I got the feeling the developer hadn't had time to read it, or that this was just an extra step he disagreed with. The length of this process gave me a lot of experience in what not to do. Had I been accepted, refactoring the hiring workflow was going to be a personal side project.

Shortest: Payment Company

Reading up about the company, I figured that I would be a good fit. I emailed a simple cover letter with some specifics of where I'd fit in. Their CTO responded back and we set up an in-person interview.

The interview with the CTO went well, but he was hoping to have me fill a different role than I was looking for. I met after with two of their developers who were prepared to ask me questions relating to the other role and were ill-prepared to ask me Java related questions.

The next day I heard back from the CTO; his team didn't feel I was Java-savvy enough.

Notes
Short, and sweet. Within a few days this company came and left my radar.

Know what you're looking for. Before bringing in candidates, clearly define the role you're looking for. If you're not sure (it happens sometimes), then spend time defining what you're not looking for.

Be prepared. Having stock questions to get a conversation going is fine, but if you haven't read the candidate's resume and don't know what to expect, you'll be ill-prepared to ask the right questions to find out their skills, and most likely opt not to hiring them simply because of this.

They did an excellent job explaining the technology, where they were going and what challenges were ahead for them. The developers I chatted with were interesting, but since they had a "systems guy" and not a "Java guy" mindset, they didn't know what to ask.

Accepted: eCommerce Company

Did some research to discover they'd been looking for someone to fill this position for a while, skipped their online application form and found the email address for hiring. I had customized my cover letter to ensure they realized I was looking for a specific role and highlighted how my experience meshes with what I was looking for.

I received a call back from their hiring team and we set up a phone screen. During the phone screen we discussed what I was looking for, salary expectations, and more about what the company does. From there I was told there was an online Java quiz to do.

The quiz was timed and fairly wide reaching.

I heard back a couple days later that they would like to schedule an in-person technical and HR interview. I asked if this was the interview or if there were multiple steps. At the time I was told that this was the interview, so I fit it in the day before I left on a trip.

I met with a developer there who knew his stuff, and he asked me a stock set of questions and asked me to clarify something from the quiz. Apparently I had got one question wrong but they were still interested. After, I met with the internal recruiter and we talked more about the company and where things were. She let me know that there were two more steps: a refactoring exercise and a second interview.

My first interview went well and I received the assignment while on my trip. Once I returned I sent back my answer and a couple days later I was asked in for a second interview. The assignment took probably an hour or two.

This interview I met with their development manager and we discussed my solution, my decisions, alternatives, drawbacks and technology in general. Later I met with their CTO who asked me some higher level questions.

Then I waited. After what seemed unusually long, I was asked to provide references. After I did, it took an additional week to check them; mid-week I was told why. The company was making some internal changes and this role may not be exactly what it initially was. After discussing it further, I was still on board and was asked for a third interview - more of a just a chat - with the person who would signoff on the hiring paperwork.

I was in the next day and chatted with the VP in question for approximately 10 minutes and he was sold; an offer was made later that day.

Notes

Internal political changes made this hiring process a lot longer than it should have been, and being accidentally mislead about the hiring process was a bit of a disappointment.

Funnel Requests. A month prior to emailing them directly, I had filled out an online application form on their site and never heard from them. Since I felt I was a strong potential, I skipped that step when I later saw the role was still unfilled. I still wonder where my first application went.. Make sure that all different interfaces candidates can apply through are given equal priority.

Have Discussions. In a previous company, I would bring whatever problem I was currently tackling to the interview and ask the candidate, providing as much background as I could. We'd work through it together and see where we got. Having a refactoring assignment to chat about simulated this and turned the interview from Q&A to more of a free flowing discussion of ideas.

Summary

The eCommerce Company with their phone screen and online quiz had a "fail-fast" process, and I met only with relevant people. They mostly knew what they were looking for, but internal change made the hiring more challenging. I only answered a handful of direct "fit" questions. Everyone I chatted with was open about the challenges they were facing but also quick to highlight some of the great things they really liked. I discovered that both the quiz and the assignment came from existing weaknesses in their team and code base.

The Checklist

We explain our complete hiring workflow to candidates up front and keep them informed of their progress.
We will stop the hiring process as soon as we feel the answer would be "no". Our process is optimized to make this happen as soon as possible.
We limit the time an interview will take, but will cut it short if the answer is a definite "yes" or "no".
All potential candidates which enter our hiring system are treated equally, regardless of source.
We ensure candidates will be a good fit with the rest of the people here, and we feel we can do this mostly by discussing relevant topics with them.
Our interviewers always take time to prepare before an interview.
Our questions are relevant to the company, technology and role we're looking to fill.
We know the specific skills needed to succeed in the role we're looking to fill.
Our hiring process includes conversations with candidates similar to the ones that we have internally already.
We involve the fewest people necessary, each person adds unique value to the process.
We're honest about our challenges, and we mention the great things about our company throughout the hiring process.

Winner of the Worst Java API Award Goes To.....

2010-03-06T09:46:00.000-08:00

One of my Java interview questions revolves around the best/worst Java APIs a candidate has worked with. Tiago Fernandez has thought to take this question to the next level and posted a poll. While he had less than 100 respondents, it meshes pretty closely with my experience, so I would say that the results are pretty accurate. Check out the winner of the worst Java API. Now, how long will we have to wait for JSR-310?

My Java Developer Interview Questions

2010-02-24T14:08:00.000-08:00

Here it is! Many people have asked for this list - it's one I built, borrowed and tweaked over the years. I place a lot of value in "I don't know" over inventing answers as that's what I'd expect a colleague to do.

It's really important to tailor questions to each candidate. I'm a strong believer that a smart, experienced developer can pickup new frameworks, APIs, etc, so a lot of these questions are not Java specific.

Know the answers to any of these? :)

Difference Questions

These are really great kinds of questions, candidates can talk approach the answer however they feel comfortable and aren't required to give "definitions" of things.

What's the difference between TCP and UDP? (if they know, follow up asking about UDP vs ICMP)
What's the difference between Bandwidth and Latency?
What's the difference between an Inner Join and an Outer Join?
What's the difference between NoClassDefFoundError and ClassNotFoundException?
What's the difference between SAX and DOM?
What's the difference between a Thread and Process?
Order from slowest to fastest: register read, disk seek, context switch, read from main memory.

Design Questions

Let's say you're reviewing (or writing) a simple system that requests an email address, a password and comments (in a textarea) then stores those values in a database. Pretending performance isn't an issue, what kinds of things would you ensure in place before it went live?Sometimes candidates would need some assistance on topics (security, usability, database schemas).
Let's say McDonald's has asked you to develop their "N millions served" counter for their main website. This value comes from a slower system (via HTTP). Talk a bit about how you would design this.
How do you traverse a cyclic graph?

Java Specific Questions

What is autoboxing? Had any problems with it?
Every worked with JMS topic/queue, clustered?
How are Servlets and JSPs different?
What does 'final' mean? Ever worked with a class marked final? (bonus points if they laugh on the second question)
What does 'static' mean?
List and describe different access modifiers.
How would you diagnose a memory leak?

Random Grab-Bag Questions

A bit of systems, object-orienteering, and such.

How do you check error conditions in bash? (If they list shell scripting)
What's the difference between lazy and greedy regular expressions?
What's a context switch?
Ever worked on an "AJAX"y system? What challenges did you run into? (a little bit dated now I guess!)
What's a deadlock? How do you prevent deadlocks?
What is Polymorphism?
What are the characteristics of a singleton?

General Opinion/Discussion Questions

What makes a good build tool? What makes a bad build tool?
What's your favourite/least favourite API/package? Why?
Have you worked on a high-volume system before? What is "high volume" to you?

What have you been asked before?

97 Things Every Programmer Should Know

2010-02-22T12:53:00.002-08:00

Short on the heels of the now-lost 97 Things Every Software Architect Should Know comes a similar list for developers. It covers a wide variety of ideas from ethics and requirements to code formatting and testing. You can buy it from O'Reilly or check out the 97 things online.

Four Easy and Practical Tips When Visiting Vancouver for the 2010 Winter Olympics

2010-01-26T13:10:00.012-08:00

I've lived in Vancouver for a while now, and realize that you (and 2.3 million of your closest friends) may be coming to visit for the Winter Olympics. Here are some dead easy tidbits that will make your stay more enjoyable. You may want to bookmark this post for while you're here.

Tip #1: Eating

Typical Vancouver food is Japanese food, mainly sushi. There are many good, cheap Japanese/Sushi places all over the city. If you've never had Japanese food before (and you're nervous), check out the all-you-can-eat places that offer appetizer sizes of anything. Leaving Vancouver without at least trying would be pretty close to a crime.

Once you've had your fill of sushi, use Urban Spoon to find other places to eat (Dine Here is good too - but a little less friendly). If you like greasy-spoon breakfasts, try Joe's Grill or Bon's Off Broadway. I like Templeton (don't let its appearance deceive you!) and Kam's Place (tasty Singaporean food) but they're probably going to be packed while you're here.

Kam's Place has had a two-for-one coupon near the back of the Georgia Straight (weekly free Vancouver newspaper) every week for the past five years. Lastly, there are a number of delicious restaurants in the Commercial Drive area.

Your life will be just as fulfilling if you didn't go to Denny's or White Spot.

Don't forget grocery stores for when you just want to pick up some snacks on the go. Vancouver has a few chains including IGA, Safeway, Save On Foods, Urban Fare.

If you're from outside Canada and want to try "Canadian food", don't ask us. Most of us will have no idea. Look around for nanaimo bars, poutine, bannock, tourtiere, lobster rolls, steamed fiddleheads, and maple sugar pie - all genuinely Canadian. (If you've got a sweet tooth, seek out nanaimo bars). Vancouver also has a number of local breweries, and BC has a number of wineries.

With over 120 Starbucks in Vancouver, we're definitely a coffee-culture city. Other chains include Waves and Blenz (both offering free wifi), as well as the more ubiquitous Tim Horton's (origin of the term double double). If you're a Starbucks fanboy, I encourage you to make your way to the corner of Robson and Thurlow - a famous intersection with two Starbuckseseses.

Tip #2: Getting Around

Don't Drive. I really can't stress this enough. The city has recently extended pay-parking hours to 10pm - seven days a week everywhere in the city. Even without the 2.3 million additional people, driving in Vancouver is a pain. I've read somewhere that all tickets to Olympic events include transportation to/from the event. Please check into this!

Public transportation in Vancouver is run by TransLink. It includes buses, subways (Canada Line), light rail (SkyTrain) and ferries (SeaBus). TransLink fares work on a "zone" system. Your fare corresponds to how many zones you'll be travelling through. I found a pretty good TransLink Map. As a rule, SkyTrain and Canada Line stops have fare machines which accept Interac, credit card and Canadian cash. Buses are exact-fare.

Google Maps has a "Directions by Transit" for getting around the city. I encourage you to use this as a backup during the Olympics as some operating changes just for the Olympics have been made and I'm not sure if Google has been updated. Instead, use the pretty decent
TransLink Trip Planner.

If you don't have prepaid tickets (FareCards), there will be a $5 surcharge (per person) traveling from the airport on TransLink (via the Canada Line). It's still cheaper than taking a taxi. If you see someone at the airport selling FareCards, you may want to take them up on the offer. To ensure it's legit, make sure the magnetic stripe side of the FareCard doesn't already have an expiry time printed on it.

If you use the Trip Planner and you see "get off one SkyTrain, get on another SkyTrain" around Broadway-Commercial, I will confess it is a bit confusing the first time you do so, but you won't be alone in the necessary confusion. It's all well-marked.

Looking for something kinda fun to do? There is a SeaBus that goes from Waterfront to Lonsdale, it's a two-zone ride (cheaper on the weekends). It's a fun little ride. The Lonsdale side has a market with fresh food stuffs too. You'll have a nice view of downtown from the Lonsdale side too. Great on a sunny day!

When walking downtown, streets that run NW/SE are just generally referred to as "North" or "South" by downtowners. "If you can see the mountains, you're facing North." is a rule many locals have. I think (at least downtown) locals are pretty happy to help a lost soul.

Lastly, if you can avoid taking public transportation from 7am-10am and from 3pm-6pm, those of us going to work would greatly appreciate it. :-)

If you are driving, and you're from outside BC, you'll want to know about our BC traffic lights.

Tip #3: Weather

Vancouver is generally a rainy city. The outskirts are cold enough to get snow, but usually not a lot. If you want to see what it looks like right now in Vancouver, the popular and mostly reliable Kat Kam web cam is a great start.

I pretty much swear by Environment Canada's Vancouver Seven Day Forecast.

Dress in layers (guys too), but leave your umbrella at home. Instead, bring a long a hooded rain jacket. You'll navigate the sidewalks a lot more effectively and you're not likely to leave your jacket behind somewhere.

Locals tend to need higher amounts of rain to trigger their "I should put my hood up" feeling.

Vancouver is stunning when it's sunny out. If you can, make your way to the Lookout at Cypress Mountain for pretty much the best view of Vancouver day or night.

Tip #4: Blending In

Here are some random tips you may want to keep in mind:

Just like the Tube, Vancouverites follow "Stand to the side to let people out before getting on" for public transportation. If you do this too, you'll blend in a lot better.

Most people here don't speak French; there are easily more Chinese/Japanese/Korean speakers than French speakers in Vancouver.

Generally speaking in the rain, if you're without a hooded jacket or umbrella, walk nearest to buildings. If you've got an umbrella, walk furthermost from buildings.

On stairs and escalators, we follow the "stand on the right, walk on the left" rule.

The weather is a popular conversation topic and can be used to talk to just about anyone - especially if it's nice out!

That's It

Enjoy your trip! And if you're looking for someone to go for sushi with, drop me an email. :-)

Xen vs KVM

2009-04-02T22:35:00.002-07:00

I wanted to chime in on the Xen vs. KVM discussions, and give you some food for thought.

I've been using Xen now for years, having replaced UserModeLinux on my personal server, and it's seen a lot of production use at my current job.

That said, KVM is ultimately the right way to go. Constant maintenance of Xen's hypervisor kernel is what causes Linux distributions grief with Xen. Those that keep doing it, do so because it's performant, tested and well-understood.

So you're stuck wondering which virtualization technology you should pick. Here it is: libvirt (with Xen).

Base your toolset, deployment, configuration, and maintenance on libvirt, and use the Xen hypervisor for now.

libvirt provides an easy upgrade path from Xen to KVM once KVM's performance and stability have been battle hardened. Same virtual machine configuration, same provisioning, same disk images. Just grab a new kernel, change which hypervisor you're using and you'll be in the land of tomorrow.

Provisioning Servers: An Afternoon with Cobbler

2009-01-04T16:46:00.007-08:00

I spent some time evaluating Fedora's Cobbler installation service. If you play with many machines - real or virtual - you should definitely take a look. Some handy features:

* Kickstart Setup: For Red Hat breed systems, existing Kickstart configurations (found on installation media) are exposed as profiles in Cobbler. You can add your own Kickstart configurations too (imagine a "rhel5-webserver" profile).

* PXE Management: Tied in with a DHCP server, cobbler will offer profiles up for installation for servers during PXE boot (often called Boot from network), since these profiles are tied with Kickstart configurations, servers can be setup or re-setup with just a few keystrokes. KVM seems to support PXE boots; less hoops for installation your VMs.

* Repository Mirroring: Cobbler can take the URL to a repository and mirror it locally, both yum and apt style repositories (rpm and deb) are supported. Use cobbler reposync to keep local mirrors up to date.

* Distribution Swallowing: Cobbler can copy installation media and copy it locally.

* Boot ISO / Live CD: If your server doesn't support PXE, or if tying into DHCP isn't an option, cobbler can make burnable ISOs (and thus USB thumb images) that can be used to perform network installs.

* MAC Address to Profile Management: Cobbler can associate a particular MAC address with a particular profile.

* Automatic Repository Setup: Cobbler can setup your local yum mirror in your newly created server automatically for you.

* Triggers / Puppet Integration: Cobbler can be configured to register the new machine with puppet, or run other triggers (imagine automatic graphs in Cacti, monitoring in OpsView, etc, etc).

* KOAN: Koan is a command line tool that talks with the Cobbler server. It can be used to set up new VMs (using libvirtd), or with a special --replace-self option, cause an existing installation to reinstall itself. (Only works with Red Hat breed systems.)

* Zeroconf: Avahi is among one of the coolest technologies I've played with recently. Specifying --server=DISCOVER on the koan command line will find the cobbler server on the network. I love stuff like that.

* Configuration Cloning: Move your test environment configuration into production (or mirror for HA, etc)

I really like the architecture of Cobbler too; an XMLRPC server that both the command line tools and web interface use. The web interface is handled by Apache and mod_python, so authentication and authorization are handled by Apache (ie, basically anything you can think of).

Lastly, one nice touch is cobbler check. Run right after a fresh install it checks your environment and lets you know of anything that could potentially hamper Cobbler from working.

I haven't yet tried Cobbler with an existing DHCP server, and I'm not sure it supports ISC DHCP's failover capabilities (even with OMAPI).

Ye Olde Job: Technology That Wasn't

2008-12-29T09:03:00.002-08:00

Our system could accept transactions over the phone (via a clerk) or through our website. The clerk on the phone used a semi-web based application to interact with the system.

The headache with this was that the code base for the "semi-web" and "actual web" only became a common code path much, much deeper in the code. So the plan was simple, redo the website code such that it could eventually be leveraged to replace the semi-web code.

We prototyped, mocked up, tested and developed a new part of the application on our website using GWT. It went through load testing, SEO-approval and I believe we even did a couple in-company demos of the technology.

The code was "done" (in the "given to QA to break" sense of the word) when I left, but when I check the website now I see it never made it to production.

I cared about this for the first month or so after I left, so I pinged around to find out what happened. The short version is that it got stuck in developer mythology, and with nobody there to lead it out, I knew it would never see the light of day.

I wonder just how much technology is "done" that has never seen the real world?

Ye Olde Job: First Post - So You're the Boss

2008-12-26T09:58:00.009-08:00

It's been seven months since I left my old job, and I feel it's alright to talk about some things.

If you're in a situation where you're finding yourself leading a team of developers without any mentoring from anyone on how the heck to do that, I offer you some food for thought.

As a developer, you measure your own progress by the code you write, the bugs you fix, stored procedures you debug, problems you solve and features that get released. You have days where you go home "having done nothing" because you have a dozen variables in your head and you've been nose deep in code, but whatever it is still isn't working.

Or maybe you wouldn't debug the stored procedures, but instead really wish that you could replace them with something else; a nice cup of Java, easy to unit test and a breeze to step through.

One thing I figured out that really helped me: When leading a team, put the team first.

What the heck does that mean?

It means getting to know the strengths and weaknesses of not only your team, but of the teams you interact with. One developer has a knack at networking, another at databases, another one refactors code from a city dump into a gorgeous mansion. When the work is important - major features on a tight timeline - use peoples strength. When it's less important - minor bugs, features that are 'nice to have' - let them work on their weaknesses. If other teams have weaknesses in areas, work to help them, or fortify against it.

It means being two steps ahead of your team, knowing where you're going next. If nobody else in your company can give you direction two steps ahead, then take matters into your own hands. Don't just start piecemealing things; always have a vision. If you don't know where you're going, you won't know when you've gone off course. Obviously, be flexible with your vision for that idle Wednesday when some manager presents his vision to you so you can embrace it and walk together.

It means giving that improvement away. This was the hardest for me: giving up writing that feature you really wanted or fixing that bug that drives you nuts. You will write less code, but your team will write more code than you ever could. You will want to use a bit of the downtime you'll get to assign yourself some work, but you know you'll never get to it. Use that downtime to better map out your plan, to make sure everyone has got the work they need and to fish around to see if maybe that company plan or vision is hiding somewhere... ..anywhere.. ..maybe?

It means accepting shades of grey. There are many wrong ways to do things and few right ways. But there is definitely more than one right way. The way someone on your team might craft something may not be bytecode-for-bytecode how you would do it, but it may still fall in the realm of right. Accept that it's not how you would do it. That's fine. But is it wrong?

It means keeping it in the team. Just like most code bases that earn a over a billion dollars, the code base I worked on was suboptimal. We all knew it, but we knew we were there not to whine about it, but to make it better. Bit by bit, day by day. Start with the biggest problems, work toward your vision.

It means that stored procedure is going to stay; both the DBA and I know how to tune, monitor and troubleshoot it. We've minimized the row locking in it and fixed a deadlock problem in the logic. Also, next week you'll be fixing it to be resilient against another team's code, because their code does table scans and table locks and it will soon be our busy season and the potential for headaches is not part of my vision.

Updated: While stored procedures vs ORM is part of a future post, I hope that people who are in the "You've been here the longest, you're in charge" situation can benefit a bit from keeping in mind that they need to put their team before themselves. I know it's hard to switch gears.

Seven Reasons I Didn't Pick Your Open Source Project

2008-11-09T10:36:00.001-08:00

After a really relaxing summer, I've picked up a job as a Systems Architect at local company suffering from growing pains. After almost a decade, I've put down my debugger and picked up a budget..for now. I've found myself looking at many different options of products some open source some not. I want to apologize to those open source projects I've rejected and wanted to let you know why:

1. Capacity Planning: You lack any realistic guidelines for server sizing. I have no idea how much memory, disk, bandwidth or cpu cycles I need to support 250 users checking their email every morning and you have no tools to simulate load.

2. You're Working On the "Next Gen" Version: You've stopped working on the version people are using in order to rewrite the application using technology that follows your new beliefs. The current version is now suffering from bitrot and your new version is undergoing its third major refactoring.

3. Your Community: Your forums are an abandoned town with tumbleweed rolling through waking up the sleeping armadillos. Questions go unanswered for months at a time, most of the posts are spam or people asking if the project is dead (which it is).

4. Installation: Your twelve-step installation process involving a different version of Perl, MySQL, and PHP plugins I don't have is a headache. This is the world of LiveCDs, embedded servlet containers, virtual machines, Web Start, VNC, hosted-demos or at least RPM/DEB packages.

5. Your Features: They're geared towards other developers and not users. I realize you're "not M$", and it's swell you think "Not Written in Java, so it's Fast!" is important, and it's sure great that you "use AOP extensively", but if those are the features you list instead of "works through NAT","supports 802.3ad bonding" or "integrates with iCal" then you're pretty far removed from your user base.

6. Maintenance Headaches: According to your documentation, your upgrade process is "Backup all data. Install new version. Restore all data." That's fine if "all data" is the dhcpd leases file, but not so great when it's 9TB of files that users access all day.

7. Price: Your project has a "commercially supported" version. That's cool, but when the rates listed are more expensive than the defacto standard you can pretty much guarantee I'm not going to be able to sell it upwards.

I'll let you know what I've picked and how it turns out in the next few months. I'm also looking for a really good Linux sysadmin. :)

Wave Tracing: Ray Tracing for Sound

2008-10-11T12:35:00.002-07:00

Here's the pitch: With regular ray tracing, rays of light are traced backward from a pixel of the camera, to an object and eventually to a light source (or lack thereof). If you can do that with light, why can't it be done with sound?

Over ten years ago I was having breakfast with a friend and I sketched out the idea on a napkin. This kind of math is definitely not my strong point. But instead of a camera, have a microphone. Instead of tracing rays, trace vibrations, and instead of light sources, there's air and friction; or so the napkin said.

Instruments aren't the goal, but a simple model to test would be a
grade school caliber recorder. After modeling materials, blown air resonates down the tube of the recorder generating sound.

As different quality of air and different materials are created, artists build wireframes of different instruments. The effects of room acoustics could be modeled too.

This would culminate in a completely rendered orchestra performing a wave traced rendition of Beethoven's Symphony No 5.

As I mentioned earlier, instruments aren't the goal: Voices are.

After modeling complex instruments, the next step is to model and animate the speech pathway from lungs to lips. The speech animated model (or "Sam" for short) would start with simple vowels like the [a:] sound in raw, or the [æ] sound in "bad". (Wikipedia's page on the IPA has some good introductory information). Eventually Sam would be animated to fluidly pronounce most phonemes. Linguistics are also not my strong point.

I'd imagine that Sam would sound very lifeless and flat. So we would capture data from sensors attached to voice model actors. Voice capture artists would smooth out the data and bring Sam to life. Motion capture is also not my strong point.

With only a handful of voice model actors, spoken phrases could be captured for an entire movie or video game! The captured data could be applied to dozens or hundreds of different sounding models by adjusting Sam's variables. Accents would easy to add by changing the way Sam enunciates words. Playing with Sam's breathing rate would simulate a person out of breath or scared. Sam's voice could be lowered or raised or even modeled after a real person.

Eventually the captured data could be used to animate 3D model too!

This would be a perfect for RPG games like Oblivion or GTA that can benefit from an almost endless supply of different voices.

Let me know if you ever do anything with this, or find something that does. Especially if you're Bethesda!

GWT: Generators HowTo (BuildStamp)

2008-09-26T14:08:00.000-07:00

If you're curious what generators are, or have never used them with GWT, this simple generator will give you a quick introduction to get your hamster wheels turning.

I do a lot of prototyping in GWT before I build the backend and I've recently had the need to generate a "build version" for my GWT interface. While there are about fiftybazillionmillion different ways to do this, I've chosen a GWT generator to create a buildstamp. This
lets me tar up the www results of clicking on "Compile/Browse" and fire them off for others to play.

In your Java code, you just add:


          ...
          BuildStamp buildStamp = GWT.create(BuildStamp.class);
          yourLogo.setTitle(buildStamp.getBuildStamp());
          ...

This will add a tooltip to yourLogo which looks like: "Built on kikai.linuxstuff.org at Tue Jul 29 19:07:05 PDT 2008".

When you compile your GWT application the generator is compiled and run. It generates Java code which is converted to JavaScript along with the rest of your application. When this happens the timestamp is "frozen".

Code and wiring follows. Imports omitted, use your favourite IDE to fill them in. This makes a pretty simple Hello World example.

Comments are more than welcome.

package org.example.rebind;

public class BuildStampGenerator extends Generator {

        private String className;
        private String packageName;

        @Override
        public String generate(TreeLogger logger, GeneratorContext context, String typeName)
                        throws UnableToCompleteException {

                try {

                        TypeOracle typeOracle = context.getTypeOracle();
                        JClassType classType = typeOracle.getType(typeName);
                        packageName = classType.getPackage().getName();
                        className = classType.getSimpleSourceName() + "Impl";

                        generateClass(logger, context);

                } catch (Exception e) {
                        logger.log(TreeLogger.ERROR, "Exception during BuildStamp creation.", e);
                }

                return packageName + "." + className;

        }

        private void generateClass(TreeLogger logger, GeneratorContext context) {

                PrintWriter printWriter = context.tryCreate(logger, packageName, className);

                /* Returns null if the package has already been generated. */
                if (printWriter == null) {
                        return;
                }

                ClassSourceFileComposerFactory composer =
                  new ClassSourceFileComposerFactory(packageName, className);

                // shouldn't magic just happen here?
                composer.addImplementedInterface(BuildStamp.class.getCanonicalName());

                SourceWriter sourceWriter = composer.createSourceWriter(context, printWriter);

                writeGetBuildStamp(sourceWriter);

                sourceWriter.outdent();
                sourceWriter.println("}");

                context.commit(logger, printWriter);

        }

        /**
         * Write the source code for {@code BuildStamp#getBuildStamp()}.
         */
        private void writeGetBuildStamp(SourceWriter sourceWriter) {
                sourceWriter.println("public String getBuildStamp() {");
                sourceWriter.println(" return \"" + Generator.escape(getBuildStamp()) + "\";");
                sourceWriter.println("}");
        }

        /**
         * Executed at *compile* time, this generates a string that is embedded in the
         * generated class. In hosted mode, this code is rebuilt each Refresh.
         *
         * @return A user-friendly build string, good for hiding in tooltips.
         */
        String getBuildStamp() {
                StringBuilder msg = new StringBuilder();
                msg.append("Built on ");
                try {
                        String hostname = InetAddress.getLocalHost().getCanonicalHostName();
                        msg.append(hostname);
                } catch (UnknownHostException e) {
                        msg.append("(hostname not available)");
                }
                msg.append(" at ");
                msg.append(new Date());
                return msg.toString();
        }

}

You will need a BuildStamp interface:

package org.example.client;
public interface BuildStamp { public String getBuildStamp(); }

And wire it up in your .gwt.xml:

<generate-with class="org.example.rebind.BuildStampGenerator">
  <when-type-assignable class="org.example.client.BuildStamp" />
</generate-with>

What do you think?

Spore on Linux

2008-09-15T00:16:00.003-07:00

If you're a Linux user wondering if Spore works, the answer is: Sorta.

I bought the download version from the EA Store. Nothing mentioned until after my purchase was complete that I needed a Windows-only downloader. I didn't test this under Wine.

Aside: The EA Store website allows + in email addresses, but the Spore online login does not.

I'm using Ubuntu 8.04 with a custom build of Wine 1.1.4 (includes a patch needed to make Spore run). The installer runs fine in Wine. Skip the DirectX install. While the game works, piracy prevention code doesn't. You'll need to find a cracked SporeApp.exe floating around to work around the issue. I believe a similar workaround is needed if you purchase the boxed game.

I'm still on the creature level, I've found a few graphical glitches:

Minor: Turn "Shadows" to "Low" to reduce some of the bizarre sploches of blackness.
Minor: The icon by the mouse pointer seems to not be transparent properly
Minor: Text describing your goals overlaps the text describing items you've picked up.
Medium: The water isn't rendered.

Check the the WineHQ page about Spore.

Chromium and Bees

2008-09-03T12:28:00.005-07:00

Watching what Google does feels like watching X-Files. I want to believe there's some grand master plan, but I'm pretty sure they're just making things up as they go.

Hank Williams over at Why Does Everything Suck pretty much covers my thoughts on Google's new web browser in his Huh? and Huh! posts.

Nine Handy Tips for Taking the Summer Off

2008-07-15T11:10:00.003-07:00

Six weeks ago I quit my job; six weeks later I've never been happier. I had grown jaded about technology and worn down about my work. This post isn't about how to quit your job, how to budget for the summer or how to get your next job; it's what I've learned in the brief oasis between those points in my life.

The Beginning

1. Do Not Look For Your Next Job: You know internally if you're at the point in your career that taking the summer off is realistic. Don't start looking for your next job right away. You'll find it (or it will find you), and it will fill your mind space, or even shorten your planned sabbatical (then you'll need another one!).

2. Break Your Old Routine: Three days and a plane ride after quitting my job I was crashing out on a friends couch across the country (thanks Lori!). Shake your life up enough that you don't realize you've left your job. You'll forget about your BlackBerry in two days.

3. Leave Your Identity Behind: If you've ever moved cities, you've said "Back home at originalCity....". It takes time to adjust to big changes, so correct yourself if you refer to your old team/workplace/etc in present tense. Many people and social circles define will assess your value based solely on your job; you'll need to feel comfortable telling people you're taking a break. It took me a while to really let go of this.

4. Travel: Find something interesting going on in a different city and go there. One friend suggested I go tour around Europe for a few months; if you can do it, go for it.

5. Turn Off Your Alarm Clock: Unless you already have a morning ritual, turn it off. Wake up when you feel like it. As new-agey as this sounds, give yourself permission to do nothing. All those "if I had time, I would do it" things can wait (see below).

6. Don't Code: Or if you do, don't code in your normal language (admit it: you really want to learn ActionScript :-). Your skills will still be around in two months.

The Middle

7. Turn On Your Alarm Clock: So, it's been a few weeks of laziness, make yourself a daily ritual. Set your alarm clock again. It's okay if you were up late the night before, just take a nap. Now is the time to make and start working through that list you've been ignoring.

8. Resist Gossip: I still hear from friends at my old job; nothing has changed. Each time, I I feel stress returning in my back muscles. When we hang out now, I try to guide the conversation away from all the usual rants. "You sound like you could use some time off to relax."

9. Hobby: Pick up a hobby or two to fill your time. Unlikely retirement, these can be hobbies you have no intention of keeping after the summer, so go out on a limb and try something really different.

The End

I still have six more weeks of planned vacation left before I start really getting into job hunting mentality. Time will tell how it ends.

Google's Conference on Scalability

2008-06-16T20:02:00.000-07:00

This is the second year I've attended Google's conference on scalability, the talks were more academic this time around. One organizer indicated that this was indeed the goal of this year's conference. I enjoyed last year how much of the information was really hands on, but there's some value in seeing what's coming down the research pipe.

The conference is an example of what googlers do with their "20% time".

I took lots of notes, but I'll summarize just the overarching concepts:

* Go Scale Yourself: Everyone seems to suffer from interruptions and it can take up to 1/2 hour to refocus after one. Asking the interrupter to file a ticket was suggested (something I've had lackluster success at). A better idea: keeping a pad of post it notes and writing down their request and telling them you'll get to it. The physical demonstration of accepting their request makes the requester feel they're being heard.

Good quote: "We're too busy mopping the floor to turn off the faucet." Sounds familiar.

* P2P: Lots of talk directly and indirectly on peer-to-peer, on the desktop, in the datacenter and under-water. Lots of interesting issues with lots of potential on both the small and large scale.

* Distributed Hashtables: Many talks mentioned distributed hashtables as a solution for one problem or another. I haven't read up on the details, so I don't have much to add.

* Multi-Core: We write a lot of plumbing code now to handle multiple threads, and really, we shouldn't have to. Cray's talk on Chapel showed how brittle our code is (because Doing It Right Is Hard) and Google had a presentation on Transactional Memory. If you've never heard of TM, then the talk is a great introduction.

I had an interesting chat with a database developer on the effects of solid state storage on database engines and a guy from Microsoft who had done a fair bit of P2P research.

The talks and slides will be up on YouTube over the next week. The talks were brought down a manageable 30 minutes from the originally scheduled hour, so if the presenters seem rushed, they probably were.

Four Days of Spring: Spring Core Training

2008-05-03T14:02:00.005-07:00

The other week I was fortunate enough to grab an extra freebie spot on Spring Source's "Spring Core" course. It's a marathon. Four days jam-packed with just about everything you could imagine Spring-related. I thought I'd post a bit about my impressions...

First the good side; the course material is exhaustive and the instructor knew his stuff cold. While I was the only one in the course with no practical Spring experience, those that had been using it extensively carried on some pretty detailed conversations; the kinds of conversations you have when you're shoulder deep in some technology and you're having esoteric problems with it.

The material was well organized and the instructor did a really good job at calibrating to the overall skill level of the class. As much as I believe in substance-before-style he also remained refreshingly professional the entire time.

The labs were really well prepared containing problem solution and sometimes alternative solution sets. As someone who gleans more from a 3 line example than a four page reference chapter, this was really worthwhile. Everyone got a copy of Spring In Action too which was a nice bonus.

The only fault I found with the course was its length; drinking from the firehose for four days is a lot of drinking. If I was more familiar with the underlying framework, or if I was still in school "input mode", my mental integration would have been higher.

A two-day introduction course would have definitely been enough. Any organization planning on flying an instructor out for four days would be better off bootstrapping twice as many developers with a double shot of the two day introduction. Only those who have already muddled through some of the integrations would get a lot more from the full course.

Six PDFs You May Find Interesting

2008-04-03T18:35:00.008-07:00

1. Failure Trends in a Large Disk Drive Population [ PDF ] A popular Google paper from 14 months ago, but a good read if it's something you may have to deal with. Fairly readable language too. Summary: Really burn your drives in for the first few months. Few metrics are good indicators of failure.

2. Pinpoint: Problem Determination in Large, Dynamic Internet Services [ PDF ] Tracing requests and statistical analysis to find faults. They wrote a framework for root cause analysis. Summary: Complex systems fail in complex ways. "What went wrong?" is easy to ask, hard to answer.

3. Deconstructing Paxos [ Google | Wikipedia ] Protocol for achieving group consensus over unreliable links. The PDF itself is hard to digest, the Wikipedia page does a much better job. Summary: I still think clustering sucks.

4. The importance of understanding distributed system configuration [ PDF ] An easy read, claiming (and I'd agree), that human error when adjusting system configuration is an extremely common reason for system outages. The paper discusses why that is and some ways to minimize it.

5. Addressing Human Error with Undo [ PDF ] A set of powerpoint slides in MS Comic from 2001. They discuss solving human error problems with having an undo facility. Summary: Font selection aside, this is a really good simple read, it pleads for you to stop trying to prevent human error, and give people ways to recover from it. I 100% agree. (Explodie was born from this.)

6. Release It! Design and Deploy Production-Ready Software [ Site ] Wow. This book (PDF download, $22) reads like a history of the company I'm at. Cascading failures, database contention, developers not understanding the Big Picture, simple changes with dire consequences, the works. If your software runs in a production environment, skip the movie and popcorn and buy this book instead. It's broken into patterns and anti-patterns with some great war stories. Summary: There's a good reason this won a Jolt award.

Logging Performance Exceptions (Part 1)

2008-03-15T17:22:00.004-07:00

You've probably wished you could find out why a certain block of code was occasionally slow in production. Wouldn't it be nice to turn on extra logging only when things were acting up? You can!

With a bit of logging in your code, you can get an output like this:

 [135ms] request share purchase
   [5ms] validate purchase request
   [40ms] check account balances
    [39ms] synchronous balance request
   [30ms] read uncommitted share availability check
   [55ms] queue request
     [30ms] persist message to queue
     [15ms] write audit log

The oddly named hatch-timer tool, will give you a breakdown by putting simple push and pop methods in your code. It's not as fancy as doing it with AOP, but it's trivial to add.

One cool feature is configuring Hatch to only log when your code takes over a certain duration. For example, to log if businessMethod takes 1 second or longer:

   public void businessMethod() {
     TimerStackUtil.enableAndPush("businessMethod");
     ...
     TimerStackUtil.setThreadLocalMetricsLogger(
        new ExceedsDurationMetricsLogger(BasicMetricsLogger.LOGGER,999));
     TimerStackUtil.pop("businessMethod");

Check out the getting started to try it yourself!

Java Developer Performance Reviews

2008-03-05T19:29:00.003-08:00

Despite years of promises to the contrary, I've found myself in the unfortunate situation of having to write performance reviews for my developers. For those other poor souls in this situation, I've collected my thoughts and present them here. This post is split into two parts. The first part I vent a bit, the second part has the useful content.

Before You Get Here

Warning: If you find yourself with a Technical Lead title and you would rather be doing technical things than dealing with politics, human resource issues, sitting in meetings discussing how to set up training programs or writing performance reviews, then keep your eyes open for how far you drift into management.

My experience has shown that nobody will understand that you aren't interested in pushing paper and you'd rather solve interesting problems. Find someone to take the management side of stuff off your plate yourself if you must.

The Technical Review

Since this review is for developers, I've focused my efforts on technical skills as super-smart developers tend to come with idiosyncrasies. Since those idiosyncrasies don't cause 3am phone calls about memory leaks, database deadlocks or network saturation, they're secondary to me. I wasn't given any guidance on how to do this, so if you've heard other ones, please comment!

Requirements

The start of most development processes involves gathering one, some, most or all of the requirements for whatever you're working on. Sometimes these are technical ("Make this code faster."), sometimes they're business oriented ("Add baseball support.").

I ask myself: How well do you handle vague or changing requirements? How well do you seek out the details you need? How much forethought do you have to determine potential problems and ask key questions to flush them out? How often do you misinterpret requirements? How good is your intuition or background reading on the task at hand?

Coding

The syntax, type usage, commenting, method names, scoping, etc of your code. Most senior developers will be naturally strong in this area, only sometimes suffering from a desire to use their own coding conventions or lacking documentation on more complex areas.

I ask myself: When others code against your APIs, how many problems do they have with documentation? When I read your code, how useful are the comments to me? Do you link to relevant other parts in the code? Do you leave waste in your code? Do you catch exceptions properly? Are your error messages useful? Do you (not) synchronize when you should(n't)?

Design

Beyond compilable code, there is an art to development.

I ask myself: How well does the code follow best practices (as they vary from year to year)? Do you use proper patterns when appropriate? Do you follow good OO design (encapsulation is a big one for me)? How well do you package your code? Is your code enjoyable to read? Are things intuitively in the right place? Do your classes do what you would expect (and nothing more)? Is your code testable, without he expense of it being usable?

Integration

Your code talks with other systems. How good is your "out of the JVM" skill? This includes networking, databases, security, message queues, mail servers, etc. Not everyone will have code that interacts with all other systems, so this section can be very specific to your environment.

I ask myself: Do you make proper use of the database? Do you cache query results? Do you try to use indexes when you can? Do you understand network concepts (ARP requests, DNS lookups, TCP / UDP, firewalling, NAT, latency, packet loss, TTLs, etc)? Do you handle database failures well? Do you consider what the effects of your code are on other components in the system? Do you understand (a)symmetric encryption and when to use both? PKI? Do you understand "be strict in what you send, forgiving in what you receive"?

Release

Not only does your code have to work and play nice, but it needs to make it out of the development sandbox and into the real world.

I ask myself: How often does your code gum up our load testing? How often does QA get stuck with your features? Do you forget to flag database updates for our release? Is your code backward compatible? Do you understand the effects of releasing your code in the real world? Is your code serviceable without hand editing web.xml and restarting our servers? Can it be monitored through our existing monitoring infrastructure? Would it make Michael Nygard happy?

Operational

Once your code is running in production, it may catch fire. If it does, this section deals with your ability to find the root cause and split your time coming up with a quick bandaid while ultimately fixing your bugs. If you don't have any code in production then I use this section on how well a developer finds and fixes bugs found by QA.

I ask myself: How quickly and accurately do you rule out red herrings? Do you have good troubleshooting toolset knowledge? Can you triage haystacks of log files to find the needle? Are you able to work around the issue creatively, or do you just rollback / reboot? When it's not your code, how quickly can you come up with a solution that doesn't involve a rewrite? Can you reverse engineer a cascading failure?

That's All

I hope this helps someone else out there that has been asked to do as I have. I'm curious to hear what others have done in the past. Oh, and of course, if this doesn't suit your needs, use something that does!

Testing Network Effects on Your Code

2008-02-16T13:59:00.004-08:00

Testing is a big deal. Very often though, we spend days writing unit tests only to forget that our code must play well with others. In the real world, how responsive will a new AJAX interface or RMI Swing application be?

In our load testing environment, we've introduced a network nightmare bridge. This handy little box sits between us and our server (or between any two networked devices). We enter maximum bandwidth, milliseconds of round trip delay, and percentage of packet loss and instantly we feel the effects. Most versions include monitoring and statistic interfaces.

We can simulate long links across multiple countries, poor people on dialup, a lossy VPN connection, WiFi, etc. Really, anything we can think of.

Thinking of opening a new data centre and wondering what the latency effects will be? Got a customer who's using your WAN replicating software over a lossy VPN link and wonder how it performs? This little box is excellent for all these sorts of things and more.

The best part is It Just Works™.

Using SQL on Java Collections

2007-12-08T21:11:00.000-08:00

On the project I work on, we have an in memory model of some particularly useful data (a POJO model as it were). A situation that regularly occurs with this useful data is that various groups want the data sliced and diced in various manners.

Each time a request comes in we have to write a new method that iterates through our collection, sometimes making a new comparator and then return a subset of the objects. I've often wished for a way to just query those objects without having to put them in a database or writing my own domain specific language. Anyone who tells you domain specific languages (DSLs) are a good idea is trying to sell you a DSL framework.

Hibernate's HQL looks pretty sweet, and if there was a way I could just toss objects into a second level cache without a database and do HQL on the cache, that would solve my problem. But I can't.

Java 6's jmap tool has OQL support, but it seems to be a custom OQL implementation for jmap. I did some digging around and the closest thing I could find that would OQL on a collections of objects was db4o which is effectively commercial for my purposes.

Then I ran across JoSQL. It's exactly what I'm looking for. From some poking around, it's fairly decent performing. A large amount of time is spent parsing queries so preparsed queries are in order. Plus, the query objects are not thread safe, so those preparsed queries will need to be in an object pool. Not a big deal.

Lastly, its syntax for handling objects with lists is a bit awkward. As an example, instead of "SELECT * FROM recipes WHERE ingredients.name = 'butter'", you would need to write "SELECT * FROM recipes WHERE (SELECT * FROM ingredients WHERE name='butter')". These sub-selects continue to nest as long as your model has collections of collections.

It's nice since most decent developers know enough SQL to get by, and it's a lot faster to write a query to say "show the top five most popular recipes with butter in them posted in the past 30 days" in SQL than it is in Java. JoSQL supports returning queries as a TableModel too, which could potentially solve some "I want to see my data in fifteen different configurations" issues I'm dealing with too.

Overall, I was quite impressed! If I get time to add it into my project, I'll post an update.

Security Anti-Pattern #1: The Very Secure Password

2007-11-21T20:45:00.001-08:00

Our security department decided to change passwords of some "common accounts" (anti-pattern #2) we have at work on our load testing network. They started this process without letting anyone know and set the new password to "G2xi]$7jB;". This leads to security anti-pattern #1: The Very Secure Password.

A good password has to meet four criteria:

1) It must be easy for you to remember.
2) It must be hard for others to guess.
3) It must be relatively immune to brute-force attacks
4) It must be as secure as what it protects.

For a load testing password, "$scale2Day?" meets all four of these criteria. As it's meaningful, it's easy to remember. As it has punctuation and slang in it, it would make it harder for others to guess, and since it has some punctuation and mixed characters and numbers, so it's relatively resilient to brute-force attacks.

The new password fails the first test, so it risks being "kept" somewhere; written down on paper, emailed to other people, in a text file on your desktop or otherwise circumvented with ssh keys or "remember this password locally" (making our load test network as secure as your password). This makes people work around security instead of work with security.

This opens point #2 to a social problem. The old password was easy to remember. So if someone sketchy feigned forgetfulness, it was suspicious. Since the new one is nearly impossible to remember, it becomes extremely easy to trick someone into giving you the password since "they forgot". Also, if it's on paper or your desktop, or in your "Secret Passwords" mailbox, then it's listed way more places than before.

Of course it's less likely to be brute force attacked, but it's way more complex than it needs to be.

So before you go and change passwords based on some insane line noise criteria, remember those four rules of thumb. If you need something more secure, then consider two factor authentication choices (and steer away from PKI. Honest.)

[ All passwords are made up, and details are altered. If you really want our passwords, send me your resume, we're hiring! ]

Breaking Continuous Integration Servers (Part 2)

2007-11-18T19:02:00.000-08:00

I wrote a few posts ago about how easily continuous integration servers break. This post got quite a bit of response!

Jon from Atlassian felt bad for the ordeal and sent over a couple of "I'M WELL BUILT" t-shirts. Thanks Jon! Owen over at Unreasonable Entertainment ran an initial version of the "hell build" maven plugin idea from my post against Hudson with less than stellar results. Damn!

Owen and I were discussing the results when I found a post from Donnie Pinkston on using premain to write a Java agent. While his post is about a code profiler, the idea is perfect for a VM monitor, taking performance information exposed via JMX, wiring up monitors and a custom MBean to kill the build.

I guess all the ingredients are there, it's only a matter of time for someone to cook them all together. Since switching from Ant to Maven, my releases have gone from 6 minutes to 133 minutes, so maybe I'll put that time to good use...

Cutting Through "Technological Mythology"

2007-10-29T23:47:00.000-07:00

"Java is slow." "Windows always crashes." "We have to do it this way; it's the way we've always done it!"

As we grow at work, I find the rules of thumb that I usually follow and ask others to do the same have become carved in stone. Those who don't fully understand "why" they're doing it have taken it as immutable doctrine. When I ask for minor deviations in these rules of thumb, I'm met with resistance! My own words are fighting against me!

Years ago, when I also wore the "build master" hat, I would put a cut of our weekly release to server #13 for QA to smoke test. Thirteen is a lucky number for me. Eventually we hired some poor soul who now does our builds. They still smoke test on server #13 every week.

When I suggested the other week that they rotate which server smoke testing is done on (as we've got a fair number now), I was met with "but we *always* test on #13! We have to test on that server!" QA wasn't quite sure why though.

Technological mythology forms when people are dealing with a complexity they don't understand. It forms easily, spreads like wildfire through all kinds of mediums (email, wikis, conference calls) and is hard to combat. Education doesn't work. It can be more painful to untangle than your CIO's SOA plans..