IndyNDA Bloggers

Coderetreatin’

2011-12-04T03:27:07Z

Today, the Global Day of Coderetreat came to Indianapolis. And it was great.

13 people chose to spend the whole day on Saturday, December 3rd, in the “common area” at Interactive Intelligence honing their skillz.

By the way, I want to extend a huge thanks to Interactive Intelligence for providing the space, a catered lunch, and some breakfast! It was awesome!

This was my first coderetreat, not to mention the first one I’ve hosted and facilitated. I was trusting Corey Haines’ experience when it came to the format, and I was very pleased with how it went. We did 6 45-minute sessions, each time pairing up and implementing Conway’s Game of Life. People used a wide variety of languages throughout the day: ruby, python, C#, java, javascript, even some C++.

Many of the people were new to practices like pairing, and TDD. We also got to explore a lot of design ideas and look at different ways to approach a problem. I really enjoyed being able to throw in various constraints during each session and then to see how it affected the approaches or designs that people came up with.

Overall, I got very positive feedback – it really seemed like a valuable day for everybody, for a wide variety of reasons. By far, people said that the pairing and the ping/pong TDD were the most enlightening aspects of the day. It was great to have a structured and guided environment for people to explore those concepts that can otherwise be difficult to dig into ad-hoc.

Other things people learned or enjoyed were designing with the Tell, Don’t Ask approach, and getting exposed to some new languages.

Overall I think it was a huge success. Or at least, I was very pleased with how it went and how it seemed to affect people. People grew as software developers today! That’s awesome in and of itself.

Thanks to all who attended!!

Life, Death, and Splitting Secrets

2011-11-21T13:43:00Z

(Summary: I created a program to help back up important data like your master password in case something happens to you. By splitting your secret into pieces, it provides a circuit breaker against a single point of failure. I’m giving it away as a free open source program with the hope that others might find it useful in addressing this aspect of our lives. Feel free to use the program and follow along with just the screenshots below or read all sections of this post if you want more context.)

Background

I just couldn’t do it.

My grandma died at this time last year from a stroke. She was a great woman. I still miss her. In that emotional last week, I was reminded of great memories with her and the fragility of life. I was also reminded about important documents that I still didn’t have.

When something happens to you, be it death or incapacitation, there are some important steps that need to occur that can be greatly assisted by legal documents. For example:

An advance health care directive (aka “Living Will”) specifies what actions should (or shouldn’t) be taken with regards to your healthcare if you’re no longer able to make decisions for yourself.
A durable power of attorney allows you to designate someone to legally act as you if you become incapacitated.
A last will and testament allows you to legally assign caregivers for minor children as well as designate where you'd like your possessions to go.

My grandma had these and it helped reduce stress and anxiety in this difficult time. We knew what she would have wanted and these documents helped legally enforce that.

I had assumed that these documents were expensive and time-consuming to create. Furthermore, as a guy in my 20’s, death still seems like a distant rumor. As a Christian, I’m not overly concerned about death itself, but my grandma’s death reminded me that these documents are not really for me, but rather the people I’d leave behind. I knew that if something happened to me, I’d potentially be leaving behind a mess, and that concern of irresponsibility compelled me to investigate what I could do.

It turns out that creating these documents is essentially a matter of filling out a form template. I bought a program that made it about as easy as preparing taxes online. In most cases, you just need disinterested third parties, such as friends or coworkers, to witness you signing them to make them fully legal. At most, you might have to get them notarized or filed in your county for a small fee.

One of the steps involved in filling out the “Information for Caregivers and Survivors” document is to list “Secured Places and Passwords.” It’s a helpful section that your executor can turn to if something happened to you in order to do things like unlock your cell phone or access your online accounts. Sure, your survivors might be able use legal force to get access without it, but only after months of sending official documentation. That’s a lot of hassle to put someone through. Also, it’s very likely that a lot of important things will be missed and no one would ever know they existed.

It’s probably rational to just write your passwords down and put them in a safe which your executor knows the location of and can access in a timely matter. Alternatively, you could pay for an attorney or a third-party service and leave your password list with them. However, this seemed like it would cause a maintenance problem, especially as I might add or update my passwords frequently. These options would also force me to trust someone I haven’t known for a long time. Most importantly, the thought of writing down my passwords on a piece of paper, even if it was in a relatively safe place, went against every fiber of my security being.

I just couldn’t do it.

DISCLAIMER: The above simple approaches are probably fine and have worked for a lot of people over the years. If you’re comfortable with these basic approaches, by all means use them and ignore this post. These simpler approaches have less moving parts and are easy to understand. However, if you want a little more security, or need to liven up this process with a little spy novel-esque fun, read on.

The Modern Password & Encryption Problem

As an online citizen, you don’t want to be that person. You know, the one whose password was so easy to guess that his email account was broken into and who “wrote” to you saying that he decided to go to Europe on a whim this past weekend but now needs you to wire him money right now and he’ll explain everything later: that guy.

You’ve learned that passwords like “thunder”, “thunder56”, and even “L0u|>Thund3r” are terrible because they’re easily guessed. You now know that the most important aspect of a password is its length combined with basic padding and character variation such as “/* Thunder is coming! */”, “I hear <em>thunder</em>!”, or “1.big.BOOM@thunder.mil”.

In fact, you’re probably clever enough that you don’t create or remember most of your passwords anymore. You use a password manager like LastPass or KeePass to automatically generate and store unique and completely random passwords for all of your accounts. This has simplified your life so that you only have to remember your “master password” that will get you into where you keep all the rest of your usernames and passwords.

You also understand that your email account credentials are a “skeleton key” for almost everything else due to the widespread use of simple password reset emails. For this very reason, you probably realize that it’s critical to protect your email login with “two-factor” authentication. That is, your email account should at least be protected by:

Something you know (your password) and
Something you have (your cellphone), that creates or receives a one-time use code when you want to login.

On top of all of this, you try your best to follow the trusty advice that your passwords should be ones that nobody could guess and you never ever write them down.

But what if something happens to you? If you’ve done everything “right,” then your master password and all your second factor details go with you.

And then there are your encrypted files. Maybe you’re keeping a private journal for your children to read when they grow up. Perhaps you’re living in some spy novel life where you’re worried that people will take you out to prevent something you know from being discovered. Wherever you fall on the spectrum, what do you do with such encrypted data?

Modern encryption is a bit scary because it’s so good. If you use a decent encryption program with a good password/key, then it’s very likely that no one, not even a major government, could decrypt the file even after hundreds of years. Encryption is great for keeping prying eyes out, but it could sadden survivors that you want to have access to your data. The thought of something being lost forever might make you almost yearn for the days when you just put everything into a good safe that’s rated by how many minutes it might slow somebody down.

On a much lighter note, the “something” that happens to you doesn’t have to be so grim. Maybe you had a really relaxing three week vacation and now you can’t remember the exact keyboard combination of your password. Given that our brains have to recreate memories each time you recall something, it’s possible that you could stress yourself out so much trying to remember your password that you effectively “forget” it. What do you do then?

When you put all your eggs into a password manager basket, you really want to watch that basket. Fortunately, creating a basic plan isn’t that hard.

A Proposed Solution

Let’s borrow an ancient yet incredibly useful idea: if it’s really important to get your facts right about something, be sure to have at least two or three witnesses. This is especially true concerning matters of life and death but it also comes up when protecting really valuable things.

By the 20th century, this “two-man rule” was implemented in hardware to protect nuclear missiles from being launched by a lone rogue person without proper authorization. The main vault at Fort Knox is locked by multiple combinations such that no single person is entrusted with all of them. On the Internet, the master key for protecting the new secure domain name system (DNSSEC) is split between among 7 people from 6 different countries such that at least 5 people are needed to reconstruct it in the event of an Internet catastrophe.

If this idea is good enough for protecting nuclear weapons, the Fort Knox vault, and one of the most critical security aspects on the Internet, it’s probably good enough for your password list. Besides, it can make a somewhat uncomfortable process a little more fun.

Let’s start with a simple example. Let’s say that your master password is “1.big.BOOM@thunder.mil”. You could just write it out on a piece of paper and then use scissors to cut it up. This would work if you wanted to split it among 2 people, but it has some notable downsides:

It doesn’t work if you want redundancy (i.e. any 2 of 3 people being able to reconstruct it)
Each piece would tell you something about the password and thus has value on its own. Ideally, we’d like the pieces to be worthless unless a threshold of people came together.
It doesn’t really work for more complicated scenarios like requiring 5 of 7 people.

Fortunately, some clever math can fix these issues and give you this ability for free. I created a program called SecretSplitter to automate all of this to hopefully make the whole process painless.

Let’s say you want to require at least 2 witnesses to agree that something happened to you before your secret is available. You also want to build in redundancy such that any pair of people can find out your password. For this scenario, you keep the can use the default settings and press the “split” button:

You’ll get this list of split pieces:

Notice that each piece is twice as long as your original message (about twice the size of a package tracking number). This is by design.

Now comes the hard part: you have to select three people you trust. You should have high confidence in anyone you’d entrust with a secret piece. It’s easy to get caught up in gee-whiz cryptography and miss fundamentals: you ultimately have to trust something, especially with important matters. SecretSplitter provides a trust circuit breaker just in case (because even well-meaning people can lose important things). The splitting process adds a bit of complexity, but so do real circuit breakers. If you trust no one, then you can’t have anyone help you if something happens.

For demonstration purposes, let’s say you trust 3 people.

You now have to distribute these secret pieces. You could do all sorts of clever things like send letters to people that will be delivered far in the future or read them over the phone. However, distributing them in person is a pretty good option:

It can make the upcoming holiday table discussions even more fun:

Let’s pretend that something happened to you. Two of the three family members that you gave pieces to would come together and agree that “something” indeed has happened to you. What happens now?

Well, either you included a note with each secret piece or you emailed them previously with instructions that they’d just need to download and run this small program. The pair comes together at a laptop and they each type their piece in quickly and then press “Recover”:

Oops... they typed so quickly that they mixed up one of the digits. It told us where to look:

They fix the typo and press recover again:

And immediately they see:

Password recovered! They could now use this master password to log into your password manager where you’ve stored further details.

This “message” approach is useful if you have a small amount of data such as a password that you could write on a piece of paper. One downside is that each piece is twice the size of the text message. If your message becomes much larger then it will no longer be feasible to type it in manually.

One alternative approach is to bundle together all of your important files into a zip file:

To split this file, you’d click the “Create” tab and then find the file, set the number of shares and click “Save”:

You’ll then be told:

And then you pick where to save the encrypted file:

Finally, you’ll see this screen:

This creates a slightly more complicated scenario because you now have 2 things to share: the secret pieces and the encrypted file with all your data. The encrypted file doesn’t have to be secret at all. You can safely email it to people that have a secret piece:

Now, if something happens to you, they’d run the program, and type in two shares and press “Recover”:

It’ll then tell them:

They’d then go to their email and search for the email from you that includes your encrypted file:

Then they’d find the single message (or the latest one if you sent out updates) and download your encrypted attachment:

They’d then go back to the program to open it up:

and then they’d see a message to be careful where they saved it:

and then they’d save it:

They'd then be asked if they want to open the decrypted file, which they’d say “Yes”:

Now they can see everything:

It might sound complicated, but if you’re familiar with the process, it might only take a minute. If you’re not tech savvy and have never done it before and type slowly, it might take 30 minutes. In either case, it’s faster than having to drive to your home and search around for a folder and it contains everything you wanted people to know (especially when things are time sensitive).

That’s it! Your master password and important data are now backed up. The risk is distributed: if any one piece is compromised (i.e. gets lost or misplaced), you can have everyone else destroy their secret piece and nothing will be leaked. Also, the program has an advance feature that lets you save the file encryption key. This feature allows you to send out updated encrypted files that can be decrypted with the pieces you’ve already established in person.

SecretSplitter implements a “(t,n) threshold cryptosystem” which can be thought of as a mathematical generalization of the physical two-man rule. The idea is that you split up a secret into pieces (called “shares”) and require at least a threshold of “t” shares to be present in order to recover the secret. If you have less than “t” shares, you gain no information about the secret. Whatever threshold you use, it’s really important that each “shareholder” know the threshold number of shares.

You can be quite creative in setting the threshold and distributing shares. For example, you can trust your spouse more by giving her more shares than anyone else. The key idea is that a share is an atomic unit of trust. You can give more than one unit of trust to a person, but you can never give less.

Another important practical concern is that you should consider adding redundancy to any threshold system. This is easily achieved by creating more shares than the threshold number. The reason is that if you’re going out of your way to use a threshold system, then you probably want to make sure you have a backup plan in case one or more of the shares are unavailable.

IMPORTANT LEGAL NOTE: It’s tempting to keep everything, including the important directives and your will in only electronic form (even when they’re signed). Unfortunately, most states require the original signed documents to be considered legal and most courts will not accept a copy. For this reason, you should still have the paper originals somewhere such as a fireproof safe. However, be careful where you put the originals: although it might sound convenient to put them in a bank safety deposit box, there’s usually a rather long waiting period before a before a bank can legally provide access to your box to a survivor, so don’t put any time sensitive items there. My recommendation at the current time would be to include copies of the signed originals in your encrypted file and also include detailed instructions on where the originals are located and how to access them.

How It Works

Given the sensitive nature of the data being protected, I wanted to make sure I understood every part of the mathematics involved and literally every bit of the encrypted file. You’re more than welcome to just use the program without fully understanding the details, but I encourage people to verify my math and code if you’re able and curious.

To get started, recall that computers work with bits: 1’s and 0’s that can represent anything. For example, the most popular way of encoding text will encode “thunder” in binary as

01110100 01101000 01110101 01101110 01100100 01100101 01110010

We can write this more efficiently using hexadecimal notation as: 74 68 75 6E 64 65 72. We can also treat this entire sequence of bits as a single 55 bit number whose decimal representation just happens to be 32,765,950,870,971,762. In fact, any piece of data can be converted to a single number.

Now that we have a single number, let’s go back to your algebra class and remember the equation for a line: y=mx+b.

In this equation, “b” is the “y-intercept”, which is where the line crosses the y-axis. The “m” value is the slope and represents how steep the line is (i.e. its “grade” if it were a hill).

This is all the core math you need to understand splitting secrets. In our particular case, our secret message is always represented by the y-intercept (i.e. “b” in y=mx+b). We want to create a line that will go through this point. Recall that a line could go through this point at any angle. The slope (i.e. “m” in y=mx+b) will direct us where it goes. For things to work securely, the slope must be a random number.

Although we use large numbers in practice for security reasons, let’s keep it simple here. Let’s say our secret number is “7” and our random slope is “3.” These choices generate this line:

With this equation, we can generate an infinite number of points on the line. For example, we can pick the first three points: (1, 10), (2, 13), and (3, 16):

You can see that if you had any two of these points, you could find the y-intercept.

It’s critical to realize that having just one of these points gives us no useful information about the line. However, having any other point on the line would allow us to use a ruler and draw a straight line to the y-intercept and thus reveal the secret (we could also work it out algebraically). Each point represents a secret piece or “share” and has a unique “x” and “y” value.

The mathematically fascinating part about this idea is that a line is just a simple polynomial (curve) and this technique works for polynomials of arbitrarily large degrees. For example, a second degree polynomial is a parabola that requires 3 unique points to completely define it (one more than a line). Its equation is of the form y=ax^2 + bx + c. In our case “c” is the y-intercept and “a” and “b” are random as in y = 2x^2 + 3x + 7:

Given this equation, we can generate as many “shares” as we’d like: (1,12), (2,21), (3,34), (4,51), etc.

Keep in mind that a parabola requires three points to uniquely define it. If you just had two points, as in (1,12) and (2,21), you could create an infinite number of parabolas going through these points and thus have infinite choices for what the y-intercept (i.e. your secret) could be:

However, a third point will define the parabola and its y-intercept exactly:

You’ve just learned that splitting a secret that requires three people is just a matter of creating a parabola. Requiring more people is just a matter of creating a higher-degree polynomial such as a cubic or quartic polynomial. If you understand this basic idea, the rest is just details:

Instead of using numbers, we translate the data to a big polynomial with binary coefficients.
Instead of using middle school algebra, we use a “finite field.” This helps keep results about the same size as the input and adds some security.

Don’t be intimidated by these changes. The core ideas are the same as the basic case. The only noticeable difference is that you have to think of operations like multiplication and division in a more abstract way. For details, check out my source code’s use of Horner’s scheme for evaluating polynomials, peasant multiplication, irreducible polynomials with the fewest terms, Lagrange polynomial interpolation to find the y-intercept, and using Euclidean inverses for division.

Again, it probably sounds more complicated than it really is. At its core, it’s simple. This technique is formally known as a Shamir Secret Sharing Scheme and it was discovered in the 1970’s.

I didn’t want to invent anything new unless I felt I absolutely had to. There was already a good tool called “ssss-split” that generates shares similar to how I wanted. This program adds a special twist by scrambling the resulting y-intercept point and therefore adds an extra layer of protection. Since this program was already the de-facto standard, I wanted to be fully compatible with it. To make sure I was compatible, I had to copy its method of “diffusing” (i.e. scrambling) the bits using the public domain XTEA algorithm. However, to ensure complete fidelity, I had to look at the source code. The only problem was that it was originally released under the GNU Public License (GPL) and it used a GPL library for working with large numbers. My goal was to make my implementation as open as I could, so I asked the author if I could look at his code to derive my own implementation that I’d release under the more permissive MIT license and he graciously allowed me to do this.

To prove the compatibility, you can use the ssss-split demo page and paste the results into SecretSplitter and it’ll work just fine. In addition, I created command line programs from scratch that are fully compatible with ssss-split and ssss-combine.

After some basic usability testing, I decided to make one small adjustment. The “ssss-split” command allows you to attach a prefix that it ignores. I wanted to add a special prefix that would tell what type of share it was (i.e. a message or a file) as well as a simple checksum because with all those digits it’s easy to mistype one.

Now, you can understand all the pieces of the long share:

In theory, you could “encrypt” a large file directly using this technique. In practice, it doesn’t work well because each share would be huge and not something you’d be able to write down by hand or say over the phone, even using the phonetic alphabet.

For lots of data, we use a hybrid approach: encrypt the file using standard file encryption with a random key and then split the small “key” into pieces.

For file encryption, I again didn’t want to invent anything new. I decided to use the OpenPGP Message Format, the same format used by PGP and GNU Privacy Guard (GPG). I didn’t want to have to worry about licensing restrictions or including a third-party library, so I wrote my own implementation from scratch that did exactly what I wanted. I read RFC4880 and started sketching out what I needed to do. A few bug fixes later and I had a working implementation that was able to interoperate with GPG. To simplify my implementation, I only support a limited subset of features:

I always use AES with a 256-bit key for encryption, even if users select a smaller effective key size. This means that users can pick any size key they want and thus balance security and share length. I picked AES because it’s strong and understandable with stick figures.
The actual file encryption key is always a hashed, salted, and stretched version of the reconstructed shares text.
The encrypted file has an integrity protection packet to detect if the file has been modified and ensure it was decrypted correctly.

Since I used common formats, you can verify the correctness of the generated files using a Linux shell. You can also create files using the shell and have them interoperate with SecretSplitter. I included a sample of how to do this with the source code.

Help Wanted / Future Possibilities

SecretSplitter still looks and feels like a prototype. There are lots of possible improvements that could be made:

Secret splitting is a relatively complicated idea. In Cryptography Engineering, the authors write “secret sharing schemes are rarely used because they are too complex. They are complex to implement, but more importantly, they are complex to administrate and operate.”

Although I tried to simplify the user experience for broad use, it could still use some user experience enhancements to simplify it further.
I wrote it in C# for the .net platform because that is what I’m most familiar with (and it has some built-in powerful primitives like BigIntegers, AES, and hash functions). I suspect that an HTML5 version using JavaScript, a nice interface, and coming from a trusted domain would get much broader usage. In addition, since this is a problem that affects everyone, having great internationalization support would be a nice touch. It also would be nice to have a polished look with a good logo and other graphics.
You could use more elaborate secret sharing schemes than what I implemented in SecretSplitter. I considered these, but ultimately wanted to use a technique that was already compatible with widely deployed tools. I also considered enhancing shares with two-factor support or using existing public key infrastructure, but decided that added too much complexity. Perhaps it’s possible to incorporate these in a good design.
It’d be neat if this scheme or something similar to it was integrated into LastPass and KeyPass as a core feature.
Obviously the shares themselves are long. I tried making them shorter but the downsides outweighed the upsides. Perhaps it could be better. Also, a compelling graphically designed share card might make it more fun for broader use. The long length is somewhat of a safety mechanism that prevents people from memorizing with a quick glance. Also, it discourages overhasty use much like freezing a credit card.
I kept the codes in a format that would be easy to write as well as read over the phone. I used a simple character set that avoids ambiguities like “O” vs “0”. One additional strategy could be to embed the share as a QR code or something similar. I didn’t pursue this approach in favor of simplicity, but this could be an option.
Really paranoid people might want to back up their encrypted file to paper. This is possible, but I’m not sure if it should belong inside the program itself.
It’d be good to have suggestions on how to exchange shares or perhaps borrow ideas from PGP key signing parties. I suspect that if secret splitting were to become popular, then “web of trust” scenarios would naturally occur (i.e. “I’ll hold your secret share if you hold mine”).
It’d be fun to compile a list of non-obvious uses for SecretSplitter to share with others. For example, it could make for interesting scavenger hunt clues.

If you’d like to donate your time to any of the above ideas, I’d encourage you to just give it a go. You don’t have to ask for my permission but it would be nice if you posted your results somewhere or left a comment to this post. You can use my code for whatever purpose you’d like. My only hope is that you might get some benefit out of it.

Conclusion

SecretSplitter is just a tool that gives another option for backing up very sensitive information by splitting it up into pieces. It’s not a full solution, only a tool. By relying on people I trust instead of a third-party company, it helped me remove one excuse I had for not preparing somewhat unpleasant but important documents that we should all probably have. I still don’t have this all figured out, but writing SecretSplitter help me get started.

If you’re young, don’t have any minor children, and don’t care at all what happens to your stuff, then you could run some mental actuarial model and convince yourself that the probability of you or your survivors needing these documents or password recovery procedure anytime soon is low, but you’re not given any guarantees.

At the very least, it’s a good idea to make sure all of your financial assets and life insurance policies have a named beneficiary and at perhaps at least one alternate. You can also declare things like organ donor preferences on your driver’s license instead of making declarations in other documents. It’s also a good idea to have an “ICE” entry in your cell phone. However, going the extra step and making very basic final documents doesn’t require that much more work. Besides, once you have baseline documents, keeping them fresh is just a matter of occasional updates due to life events.

The increasing digitization of our lives means that more personal things will only be stored digitally. From our journals to email to videos to health records, all of this will eventually only exist digitally and likely hidden behind passwords. This future needs some safety net for backing up sensitive things in a safe and accessible way.

Everything doesn’t need to be backed up. There are also lots of files, usernames and passwords that don’t really matter. Don’t include those. SecretSplitter was built with the assumption that everything that really mattered could be stored in a file small enough to email to others. This helps focus and pare down to what really matters.

It’s also good to have a healthy dose of common sense. Instead of holding out a secret until after your death, maybe you should get that resolved today. You’ll probably live better. My general view is that these final “secrets” should be mostly boring by just containing account details and credentials.

Finally, on a more personal level, I think it’s healthy to be reminded about our own mortality at least once every year or so. It’s a helpful reminder of how much a gift every day is and helps focus what we do and not worry about things that don’t matter.

If a little bit of fancy math can help you sleep better at night, well then, I’d consider it a success.

Special thanks to B. Poettering for creating the original ssss program and allowing me to clone its format.

Global Day of Coderetreat in Indianapolis

2011-11-11T01:52:28Z

I recently came across this thing called a “coderetreat” that Corey Haines puts on. The gist is that it’s a free full-day event where a small group of dedicated-but-busy software developers can come together and work on honing the skill or art of writing software, away from the usual pressures of business.

I thought it sounded awesome. I have yet to attend a developer event that was comprised primarily of the attendees coding, with the possible exception of some codemash sessions.

Then I saw something about a “Global Day of Coderetreat” — coderetreats organized to occur all around the world on the same day: December 3rd.

And I noticed that Indianapolis was not on the list.

That simply won’t do.

So I’m happy to announce that Indianapolis is going to participate in the Global Day of Coderetreat! My employer Interactive Intelligence (I contend it’s the best place to work in the midwest) has agreed to sponsor the entire event by providing a great space to meet and a free catered lunch! Check out these pics of our offices and in-house cafe on facebook.

You’re probably asking,

“Okay, so what’s a coderetreat anyway?”

I’ll talk about what ours is on December 3rd. I’ve opened it up to 26 people. (I said it was a small group…) You can see more information on the structure here, but the gist is that there is some meet & greet time at the start, an intro, then 5 to 6 sessions (with a long lunch in the middle) of dividing up into pairs and tackling a problem (same problem all day). After each session, we delete all our code, switch pairs, and start over again, but working to grow our designs, and explore alternatives and practices like TDD. This is language agnostic, we just try to make sure each pair knows the same language (regardless of proficiency level.) Then there’s a wrap-up and whoever wants heads out for a beer.

And did I mention it’s all free?

Sound good to you?

Yes? Me too!

See below for ticket info. I want to point out something first, though. With only 26 slots, I want to avoid it filling up quickly with people who won’t show up. So I’m charging $5 to attend. If you attend, you will get a full refund, or you can opt to donate your $5 to the Interactive Intelligence Foundation – a great not-for-profit that helps at-risk youth, especially with technology education.

You can get a ticket at http://coderetreatindy.eventbrite.com/

Any questions? Tweet to me at @aaronlerch or email me at aaronlerch at gmail

Hope to see you on the 3rd!!

What did I learn?

2011-08-22T15:57:54Z

Karl Scotland and I have helped coordinate the organization of the bag stuffing each year for the last three years. It’s a great time because of the awesome group of volunteers showing up and the mindset of “I’m here to learn something while I help out” that’s so pervasive.

Execute!
The flow of bags was very effective at the end of the day last year and this year we simply started where we left off last year with the same design. Thanks to some great pre-work, we knew where everything was and what needed to go in each bag, so the briefing of the volunteer staff took less than 15 minutes and we were ready to go.

The team got moving and each zone quickly solved any issues that arose, continually improving and adapting how they worked to the needs of their specific area. We got done a couple hours early, and everything pretty much “just worked” better than could be expected.

Retrospect?
So, when I’m asked to lead a retrospective, what do I do? Everything went so well, and learning was incorporated so rapidly, that a simple plus/delta seems utterly pointless. The plusses have been amplified, and the deltas have been changed, why should we write them down? So here’s what we did…

We gathered in a standing circle, grabbed a spare piece of swag to use as a talking stick, and each answered one simple question: “What did you learn?”. After sharing, the group reflected the individual’s statement back in a simple phrase appropriate to a sticky note. [1]. As each of us answered, we sat down to bring focus to the people still wanting to speak. Best of all, the “people in charge” [2] did and said very little, and learned quite a bit.

Understand.
Why did it work? Well, as I mentioned above, many of the volunteers were there to learn. The bag stuffing has filled up in the first volunteer sign-up round (of four) each of the last two years. The process execution was well beyond “good enough”, so it seems like the right thing to do is help people take away more than just their own perspectives. With a majority of volunteers having English as a second (or more!) language, the reflection portion ensured that the individual’s perpective was conveyed clearly and helped to ensure that people were able to absorb the statements effectively. More interestingly to me, the “what did I learn” style questioning seems to have drawn out positive emotions in a way that other retrospective formats haven’t been able to do in my experience.

Here’s to next year!

[1] Karl captured them here.
[2] Specifically, Karl and I as facilitators were able to keep our mouths pretty well shut.

Uninspired by pushing

2011-08-13T21:36:36Z

I’ve not been blogging lately. I haven’t really been pushing things to twitter, either. For a while, I thought I was just lazy, and perhaps tired from travel. I’ve lately come to see that this isn’t the case. Good or bad… I don’t write for myself.

I write for others. I write for people who care about what I have to say, who use it to improve their own lives. I write for people that have a passion for learning, for exploring ideas, for changing the world around them. What do these people want?

I really have no idea. Looking back at my blog, I realize that most of my posts are the result of an amazing conversation I’ve had with somebody, or the result of reading somebody else’s work and being inspired to extend or evolve it in some way. When I hear from others which bits they find valuable, they are uniformly the cases where I write because somebody says “you should post this.” My inspiration is a reflection of shared passions.

So what’s this mean? It’s changed my behavior a bit. I’ve not written anything “just to write it”. I’ve focused much more on the people and ideas that are around me, especially inside of Rally. I’ve paid more attention to which trends and areas I find myself passionate about. I brainstorm with my peers around ideas before starting my writing. I’m exploring crowdsourcing initial drafts internally to Rally, and may open that beyond. I’m looking at the needs of various groups to see which ones resonate as ways of selecting topics. I’ve started involving others in my writing.

Starting with this post, I’m actively inviting others in my community to pull content from me. Tell me what you feel I should write about, what ideas you’d like me to extend, what basics need better coverage. Help me to understand what’s valuable, what’s engaging, what will help you create your own reality. And I might just ask you to write with me.

Enter User Voice. On the right bar of the blog, you’ll see a tab flagged “Suggest Topics” Throw them out there, and I’ll listen. If you wish, vote up the ideas of others. Pull the content you value, the material that inspires yourselves and others. Join me in my new experiment.

…and for those who read… Thank you!

talkasaur.us – dirt simple conference calling

2011-08-08T06:35:31Z

Both at work and “for fun”, I’ve used join.me for any simple screen sharing needs. I remember the first time I used it. It was simple, and it just worked. You download a small app to share your screen, and everybody else visits a URL and gets instant in-browser viewing.

Let me emphasize the key points in case they were missed. It was simple, and it just worked.

The 19th century French novelist George Sand once said, “Simplicity is the most difficult thing to secure in this world; it is the last limit of experience and the last effort of genius.”

This is clearly a gentleman who coded against the MSHTML library. I would say I was kidding, except that I’m pretty sure MSHTML was around in the 1800s.

When I saw that Twilio announced that they released an in-browser soft phone and started a 2-week contest around it, I decided to create a simple in-browser ad-hoc conference call tool. My mind immediately went to join.me’s simplicity and just-work-edness. Imitation is the sincerest form of flattery, and with talkasaur.us I am seriously flattering join.me.

There have been a few situations in recent memory where having a tool like talkasaur.us would have been useful:

Scott Hanselman wanted to record a group conference call for an episode of his podcast. I think they went through at least 3-5 different services before settling on something, and if I recall correctly (and I probably don’t) even that service wasn’t very satisfactory.
Every group video chat I’ve done (which has not been many, admittedly) has ended up requiring me to a) put on clothes, and b) create an account or sign in with an existing account like twitter or facebook. When authentication gets added to the mix, simplicity decreases. Here’s a chart to illustrate, because data doesn’t lie:

So thus I decided to pick a horrible time, personally, to spend time on this idea of talkasaur.us – the name of which was conceived at 3AM in a state of delirium and because I’ve always been fond of any “*saur.us” domain for some reason.

The result is “Iteration Zero” – enough to submit my app to the Twilio contest, and test out the waters of interest. It works like this. The homepage gives you two choices: start or join.

When you start or join a conference, you get a view that shows you a few things:

The current status (“connected”, “disconnected”, etc.)
Sharing options – you need to invite people to your call, otherwise you could save yourself the time and just talk to the mirror.
Actions – currently only muting and leaving the conference call (watch this space, though!)
Participants – a realtime list, see who’s on the call, change your name, etc.

When you leave the conference, you get a single call to action: start a new conference again.

Bam. That’s it! Of course I can already think of features to add, and the list keeps on growing:

Moderation tools: first person in gets the ability to mute others, kick them out, etc.
Additional information: because the participant list is updated in realtime, we can display more information such as whether they are muted or not, and (if Twilio can implement my suggestion) even see who is currently speaking.
Call recording/archiving
External number integration – not excited about this, but it’s a possibility.
etc

The whole thing about my idea is that it isn’t brand new, but it’s applying the discipline of keeping things simple and coming out with a better product.

And, because this is my tech blog, I’ll just do a quick list of the technologies I built this site on. Ruby, sinatra, heroku, mongodb (via mongoid), jquery, and of course, twilio. Let me just say, for a simple app like this, these tools were a joy to use, even if I did spend the majority of the time fighting battles with them.

How Talentopoly is Built

2011-06-01T02:49:38Z

Today’s post is written by my guest and fellow Indianapolis-ite, Jared Brown. I asked Jared to guest post here because he has created an application using Ruby on Rails, Heroku, and a host of other bits and services that I’ve also been using to bootstrap my new business, beautifulsavings.com which I’ll post more about in the near future. He’s done a fantastic job with Talentopoly, and he’s a developer I respect. Talentopoly is on my short-list when it comes to finding useful tidbits to expand my tech horizons. Check out my profile here.

Jared Brown is the founder of Talentopoly.com, a community for programmers, designers, and IT professionals staying current by sharing the best of what they discover online.

I wanted to take a minute to give a glimpse into the platform Talentopoly is built on. A lot of this may not be familiar to non-Rails developers.

The site is hosted on Heroku. I use it for staging as well as production. To say Heroku makes deployment easy would be an understatement. To deploy Talentopoly all it takes is a quick “git push heroku” or “git push heroku-staging”. To setup your project is also one command.

The production server consists of two dynos (Heroku’s term for a virtual server unit) for a cost of $36/mo. This setup can handle dozens of concurrent users per minute. The site regularly peaks at 50 users and doesn’t break a sweat.

Add-ons are Heroku’s way of making it easier to integrate with other services. Often times no configuration is necessary when turning on an add-on.

My staging server mirrors production. It uses the same add-ons. The WebSolr folks were nice enough to setup a free staging instance. So the staging server is free. I populate staging with production data via Heroku’s handy PostgreSQL commands.

Heroku uses nginx as the http server acting as a cache store and reverse proxy. nginx is written in C and super fast. Behind nginx are a load-balanced cluster of Thin servers (modified Mongrel servers) running the application stack. Heroku is constantly monitoring the health of the dynes and will spin up new ones to replace hung processes if necessary.

Other than Heroku non of these things are exclusive to Ruby on Rails developers. The following services help make my life easy.

Search is provided by WebSolr and the excellent Solr search engine. Search is configured on a per-model basis. Only a few lines of code are responsible for making the models searchable. It provides geospatial search as well as full text search. Unlike Sphinx Solr doesn’t rely on fragment indexes. Everything is handled transparently. When new records are added or destroyed the index is updated. WebSolr costs $20/mo. for 75k documents.

The database is a shared PostgreSQL instance on Heroku. It performs well under load. There are other options such as Amazon’s RDS but for simplicity of deployment I have stuck with Heroku’s native solution.

SendGrid handles all the outgoing email. It’s accomplished via a SMTP gateway, which saved time over using their API.

S3 handles user uploaded image assets via the awesome paperclip gem. Paperclip re-sizes and stores the images on S3.

Scribd is used to store and convert uploaded resumes. This is why Talentopoly can accept and properly display so many different file formats.

New Relic is also provided as an add-on. It’s invaluable for investigating bottlenecks. It provides a breakdown of the request stack tracing it through the database.

Chartbeat provides real-time analytics and alerts. I can see how many concurrent users I have on the site, which is useful if it’s seeming slow. Google Analytics gives me the full stats to analyze.

I haven’t had a strong need to implement an in-memory key-value store like Mongo, Memcached, or Redis yet. Though hopefully the increasing traffic on the site makes that a priority soon.

There are other great services, gems, and plug-ins I could talk about but hopefully that gives you a sense for how Talentopoly is built and what it’s like to build a Ruby on Rails app on Heroku.

Instructional Design Applied to Agile Coaching

2011-04-28T16:32:03Z

[This is a cross-post from Marian's blog]

Agile coaching requires observant facilitation with an ability to engage learners actively. This parallels almost identically with accelerated adult learning at the university level. “Guide on the side” instead of “sage on the stage” is a common mantra necessary for all effective facilitators to embrace when motivating adult learners. There are scores of interactive instructional strategies that not only provide relevance to the learners, but also allows for quantifiable assessment of the learning growth. However, for this post, I will focus on the instructional design that is the moving force behind effective facilitation effort.

Instructional design is just putting together the course, right?

No.

It is a much more holistic picture of all curriculum/courseware/module, or whatever buzz term you favour. In my experience as the instructional designer for several degree programs at a small university, a huge takeaway has been the understanding of knowing what to standardize and what to customize.

Let’s look at the benefits of instructional design insight for developing multiple training programs or courses of any level.

Standardization allows focus on architectural improvements and shared collaboration across coaches.
Recreating content for different client contexts is no longer necessary.
The modularization of core components allows focus on tailoring courses to specific client needs at a much deeper and more effective level.
Everybody shares value and baseline improvements holistically, but can also modularize for personal use. Nobody teaches the same way, and never should courseware force a specific type of instructional methodology.

Any individual coach or organization that trains internally or externally wants a strong relationship with their learners. Internal coaching needs a strong relationship for better and smoother implementation efforts. External coaching needs to consistently improve relationships with their clients beyond the expectations of internal coaching. Another benefit to formalized instructional design techniques being applied to training material is that it improves relationships with the learners in the following ways.

Customization is improved for the client’s needed context before and after the course.
Coaches can apply beginner’s mind more effectively.
Modular pieces allow coaches to make courses more experiential.
Familiarity with content and structure allows coaches to breed active learning.

Of course, economics is always a question, even if it’s not the first one. Risk management questions need asked, including, is the cost worth the investment both financially and time-wise. Unfamiliarity with the purpose of instructional design can result in the belief that something so simple is not needed, and just one more step in the process anyway. Let’s look at the economic improvements for course development with instructional design techniques applied.

More quality time for coaches is available.
Consistent master documents represent a collection of all coach experiences for focused improvement.
Course development for a new client is faster with high quality.
Course development scales across personal techniques without changing the learning values.
Needed improvements are easily targeted.
Unnecessary re-invention is eliminated, allowing focused innovation.
Easier experimentation content is enabled.
Gap analysis is simplified for new course development.

While a book could be written on the value and techniques of instructional design for agile coaching, these are a few ‘down and dirty’ values that instructional design provides for training programs.

Written in collaboration with Marian Willeke

When teams work differently

2011-04-21T04:05:38Z

In a few weeks at Rally’s user conference I’ll be leading a session on resolving the problems that arise when different teams have to work together, but have entirely different models for how their work gets completed. Details are posted at http://www.rallydev.com/coachingblog/?p=55.

Office Prank

2011-04-12T21:47:31Z

Every now and then we like to have some fun with our boss. I’ve said it before, and I’ll say it again – he’s a fun-loving guy and we all love working together. Usually when you go on vacation for a week you can expect something to happen by the time you get back. Not too long ago I returned to find my office completely filled with black plastic. I mean it took 6 large garbage cans and a mini-dumpster, tightly packed, to remove it all.

We recently played yet another prank on my boss. This one probably takes the cake, and it was a ton of fun to pull off. We actually geeked out a bit and we had two hidden cameras, which were automatically recording any motion they saw, as well as live streaming their feeds via ustream.tv. I wrote an app that monitored the filesystem and when it detected a new motion-based recording, it tweeted a frame of the motion. I just subscribed to mobile notifications and I got an SMS with a link to a JPG file every time it detected movement over the weekend. Pretty easy monitoring system!

Check out the video of the prank. His reaction was priceless!
We love to work hard and have a lot of fun at the same time. If this sounds like the kind of software company you’d like to work for, we’re hiring! (Nobody asked me to post that, in case you wondered.)

More on expand collapse pattern

2011-02-25T01:44:38Z

There’s a thread on kanbandev today about the role of expand/collapse, and its value in helping to identify the BVI/MMF [1]. Chris’ post, linked above, talks effectively about the ability to discover work that doesn’t need done, as do a couple of Ron’s contributions in the same thread. Unfortunately, from my perspective the language there is still primarily about decomposition of scope, rather than focusing on the real nature of the work we do. The value I find in “expand / collapse” is very different. This learning and discovery is just one of several intertwining aspects important to understanding the nature of the work, but I believe that effective teams are focused on learning and discovery as a matter of course.

Focus the team

One major benefit of expand/collapse is that it focuses more of the team on related tasks. I often joke that at Inkubook we had an MMF WIP limit of 1.1, meaning that there would be one MMF that most of the team was focusing on and one that was just being finished up or just being started. We had a different limit (four most of the time, iirc) for the tasks we broke out of the MMF, and when there weren’t enough valuable tasks left for the MMF that was about to end, people would start on the next one (with the focus being on finishing, not on starting). From what I’ve seen, type of focus scales nicely across teams and planning horizons, allowing “team” and “set of related MMF’s” to be the next unit of abstraction outward. This effectively enables WIP limiting and goal focus at that level. If taken all the way, I should be able to see how my individual activity today rolls all the way up to my company’s True North [2]

Manage Learning in Progress

I spoke on this at the LeanSSC 2010 UK [3] and have been observing the concepts in practice since then. Teams are limited in how fast they can learn, primarily because learning takes research and conversation to achieve [4]. When you apply expand/collapse, keep an eye on the amount of stuff to “figure out” in each broken down item. The ones with lots of stuff to figure out will have the highest variability. I can’t say if you should concentrate it or balance it, or whether you should front-load it or back-load it, but be aware of it as you make your decisions and fit the breakdown to your context. If the process of breaking it out allows you a way to get a BVI without having to figure much out, then by all means defer the stuff that requires lots of figuring out.

Manage abstractions

As I mentioned above in the focus section, the levels of expand/collapse provide a very effective way of managing the abstractions inherent in the work. It’s far easier to have value and prioritization conversations about 5 large items than 50 (or 500) smaller items [5], especially if you’ve managed to effectively engage your senior management in the decision planning. On the other hand, asking a dev team to implement against a single large item taking over a month is pointless, they need the detail to effectively track and deliver against the goals. Visualization of progress against goals is also easier due to the aggregate data, enabling the use of goal-focused reports like a parking lot diagram.

Triggering conversation

The need to apply expand/collapse is also a trigger to figure a bunch of stuff out. The best way to figure things out is to have a conversation about what’s needed with the relevant people. The mere act of decomposition is a value add activity, and should not be done by a single individual. Scrum uses sprint planning and task estimation to drive these conversations, but dropping iterations doesn’t excuse you from the need to explore the meaning of work items and tasks as a group. The conversations need to happen, and the acceptance criteria need to be discovered: “Don’t start what you can’t finish” [6]

Enable Operational Languages

One very nice side-effect of the abstractions in use is that the language used at each level of abstraction can be made appropriate to the audience, and should be. I’ve often heard about deep misunderstandings between “development” and “the business” because the two are speaking completely different languages while using the same words. This is only made more complicated when you introduce other groups as valued stakeholders, such as legal, finance, compliance and operations. With effective expand/collapse and a good goal-oriented organization, the highest levels can be written in terms of markets and the organizational vision, middle levels in terms of the desired behaviors enacted in the market and the protections the organizations require, lower levels in terms of the features that enable those behaviors, and the development level in terms of the actions that need completed to create those features. This is not a problem, this is a positive step to communicate effectively across the organization! [7]

Finally, a perspective

At the lowest level, converting from the most detailed card (task/story/line item/whatever) to code (configuration/html/sql/whatever) is its own expand/collapse translation, and the conversation is one with your compiler and (hopefully) automated tests. There’s no expectation that you can clearly define what code you need when you pick up a task, you’re free to be a professional and add, remove, and modify the code until it solves the need at hand. The same logic should apply to the tasks to complete a story, the stories required to complete an epic, and any other expand/collapse relationship you may have in your particular environment. There are many effective patterns, practices, and guidelines, but the set of actual “rules” is fairly minimal. Do what makes sense, and always strive to be more effective in what you do.

[1] Business Value Increment, which Chris renamed from Minimum Marketable Feature to clarify it as different than what’s talked about in Software by Numbers

[2] Or whatever you call your organization’s guiding vision for the year/quarter/decade

[3] http://skillsmatter.com/podcast/agile-scrum/eric-willeke-on-value-stream-languages

[4] Dan North tasks about this as removing ignorance in the context of Deliberate Discovery.

[5] I don’t think I’ve seen a Scrum training deck in years that doesn’t have a pyramid diagram for story size indicating the desired backlog composition

[6] Paraphrasing from Alan S, Dean L, and Don R among many others.

[7] I plan on writing shortly about the role of a BA as linguist in this exchange. The post is languishing as a half-finished draft.

Away with “Active”

2011-02-15T14:46:34Z

I dusted off my personal kanban about 45 days ago when I started at Rally, and it’s been a very interesting path since then. When I first rebuilt and updated the contents, the columns were Eventually, Midterm, Nowterm, Active, and Done, and they were essentially prioritization buckets so that I didn’t have to mess around sorting each of the individual items. Roughly mapped to Must Have, Should Have, and Could Have [1].

Last weekend, that changed. I was getting frustrated because I kept having to prioritize things I didn’t really _care_ about, but that I needed to do. It bothered me a lot that I couldn’t get to the stuff I care about, so I updated my columns again. My columns became Options [2], Proactive, Reactive, Active, and Acted. Suddenly I felt much better about things, because I could prioritize the things in Proactive in order of how much I care about them, how engaged I would be, and how distant the payback on my investment of time would be. Reactive, I prioritize based on a combination of ease and cost of delay.

It’s been working, but a pattern quickly emerged that bothered me. Either my active column was empty, or my active column contained the top item moved over my my reactive column. It didn’t tell me anything useful, it didn’t help me prioritize, it was just… there, taking up screen real estate without providing any value. So I zapped it. I don’t have an active column anymore, I just know I’m focused on the top item of one of the two columns that’s in the view. If I’m full of energy and wanting to engage, I hit the top of proactive. If I’m dragging a bit and just want to get some things done, but not pour energy in, I hit the reactive column. When they’re done, they’re dropped into Acted and left to languish with all their finished friends.

[1] Why bother with Won’t have?

[2] Options represents ideas that I don’t want to forget, but don’t have any desire to prioritize or work on now. Approximately weekly I skim that column to see if anything inspires me, but they’re roughly the “transient” class of service.

Bring your ‘A’ game

2011-01-31T02:20:35Z

My passion is to engage. I am at my best both personally and professionally when I’m able to confront a situation and engage the people around me. I can then understand the goals they’re trying to achieve, explore the impediments they feel they’re facing, and collaboratively find solutions. Usually, these are not only acceptable to everybody, but exceed the hopes of the individuals. I’ve not only found a solution, I’ve shown people an example that more is possible, and that they shouldn’t have to settle.

That’s my ‘A’ game, and I’m not talking about a grade. I’m talking about the affective learning domain, and how it’s equally important as the cognitive domain [1]. When people talk about being passionate about what they do, they’re talking about having achieved engagement at an emotional level. This engagement is what drives my effectiveness. When I’m engaged, I’m much more capable of understanding the way others are feeling, and I’m able to intuitively match the situation and their concerns against the many mental models I’ve collected over the years. Even better, the people I’m engaged with are more likely to be engaged because they can feed off my engagement. These are the experiences when the most fascinating things happen, the biggest audacious goals are defined, and the world gets changed (at least in small ways).

There’s one big problem, though: It’s fragile. It’s extremely easy to knock a group out of this type of zone. All it takes is one person whining about expenses, or the waitress bringing the wrong beer, or a manager walking into the room with an “We’ve got a problem” look on her face. Please, please, don’t be that guy [2]. Team-level flow is slightly more resilient than individual flow, but it’s less able to ignore the context. As a manager, this is one of your jobs. As a team member, it’s your job too. Ditto ScrumMaster, product owner, and everybody else. Protect your environment, take care of the emotional state of your colleagues, and otherwise design the proper environment for the team to engage without distraction.

[1] Referencing Bloom’s learning domains. There’s also a psychomotor domain that deals with imprinting physical and repetitive behaviors, this is something I should write about after my next code retreat.

[2] Although, there’s not much you can do about last call, really. Learn to stack.

[edit] Forcing a feed update… playing with the blog

Just one thing…

2011-01-28T22:21:29Z

Yesterday, I was reminded of the power of commitment.

I spent the day shadowing Alan Atlas as he helped a room full of managers understand their role in a brave new agile world. At the end of the day I was invited to facilitate the retrospective of the day’s course and attempt to bubble up some of the individual learning to benefit the group. Last night, at the airport, I realized I’ve never written specifically about a powerful technique I’ve added to the end of every retrospective I’ve run over the last year with great effect.

When I’m helping teams reflect, I tend to use a four-stage starfish to structure the thinking process. Yesterday, I tried an ORID model [3] due to there being many individuals rather than a focused team. In the past, I’ve used the six thinking hats to guide the reflective paths, done standard plus/delta, and several other models. [1] Regardless of the retrospective format used, however, there is one thing I always do to close out the exercise: I ask for a commitment.

This is probably the least “nice coach” activity I do as an agile guide. I force members of the group to make commitments. No Weasel Words, no dodges, no non-commitive commitments. I insist on real commitments. “I will set up a meeting next week with Bob, Julie, and Josh to discuss my role as an agile manager.” I don’t accept “I’ll each out to the other managers about this.” I require “I will hang up visible burndowns for each of my three teams” instead of “I’ll start increasing the amount of visual management”. I’m not a nice person. I insist on real, committed language, and I insist it’s something they’re willing to have held accountable by the entire group.

Why?

Because it helps. I first observed this while coaching with Simon Bennett in the UK during early summer 2009, and this is where I first heard the term “Weasel Words.” I am amazed at how deeply it struck me about my own behavior, and how much it started focusing my interactions as a coach and team lead. I started doing action plans [2] with individuals and teams. It is one of the things I most appreciate here at Rally: Following through with commitments is part of the company’s core values. Too often, it’s not. Time after time I see teams and individuals make a fuzzy statement of intent and never ever follow through, because the commitment doesn’t _mean_ anything. It doesn’t connect with affective learning, only cognitive learning, and therefore doesn’t stick very well.

As a coach, I force people to connect with what they’re saying. I don’t put individuals on the spot, I put entire groups on the spot. “Stand up, tell your peers what you will _do_, and make that commitment to the group.” Starting yesterday, I even invited (didn’t insist) people to sign the written record of their commitment (i.e. the sticky on the board). It helps create focus when multiple people make very similar commitments.

Just one thing, one pebble, one little piece of progress can start a landslide. It’s even more powerful when you get ten or twenty little pieces of progress. People throughout an organization see many people suddenly DO something and take notice. They instinctively recognize that something important is happening and they watch, and maybe even engage.

[1] I’m more than happy to write about any of these… leave a comment or tweet at me if you’re interested.

[2] Do you want another post on Action Plans?

[3] Scroll down to “The ORID Model” and pay attention to the Kolb Learning Cycle too

Welcome to my new home!

2011-01-26T19:32:53Z

As many of you know from Twitter and other events, I’ve recently joined Rally as an agile coach, allowing me to engage with a much broader range of teams and contexts as I help guide people towards more effective and satisfying ways of providing meaning and value to their organizations. In the early years of my career, I was lucky in that I was exposed to a great many contexts that often were wildly different from engagement to engagement. As I progressed in my understanding, I began to recognize the value of the perspectives I gained from exploring these different environments. This knowledge allowed me to change my approach to my career, letting me seek out roles and situations that would accelerate this exposure, leading to my explorations of the product development environment as an employee of CSI and later Inkubook. Recognition that Indianapolis was not able to provide the wide exposure I desired led me to take on my role at EMC Consulting, allowing me visibility into many very large organizations over the last eighteen months.

Rally is the next natural step on my path. Coaching with Rally will offer me the continued opportunity to engage with larger companies at the right level to see entire systems and how they work. I’m excited, and I look forward to the stories I will be able to tell, and the ones I won’t…

To those who read this, thank you for your support and guidance over the last 167 posts! This is the new home for Rediscovering the Obvious, and it’s probably very obvious that it’s very much a work in progress. First step is a reasonable theme, then the various plugins and useful things for both myself and visitors. Only then will I start taking the time to try and clean up the older posts, with the expectation that the code blocks will take the most time; I’m hoping styles will take care of the rest of the mess in a reasonable way. Comments are lost, as are my previous view counts (a handful of posts are pushing 1800 views with most posts getting around 500). Thus ends the tale of my conversion from Community Server to WordPress.

Finally, I’d like to offer a very special appreciation to two people.

Michael Ruminer, owner and host of manicprogrammer.com, I appreciate you for making blogging very easy over the last few years.

Bernardo Heynemann, friend and fellow architect over the years, I appreciate you for getting me off my lazy ass 167 posts ago and getting me to put my thoughts on the web. I have gained far more than I ever would have expected, and I owe you.

Forgetting yourself in MEF w/ Prism

2010-12-10T05:49:19Z

Just a quick note in case anybody else runs into this issue in Prism for WPF.

In the code below, the line “return Container.GetExport<Shell>().Value” was failing and telling me with the following message:

System.ComponentModel.Composition.ChangeRejectedException was unhandled Message=The composition remains unchanged. The changes were rejected because of the following error(s): The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.

1) No valid exports were found that match the constraint

‘((exportDefinition.ContractName == “ABC.DEF.MenuViewModel”)

AndAlso (exportDefinition.Metadata.ContainsKey(“ExportTypeIdentity”)

AndAlso “ABC.DEF.MenuViewModel”.Equals(exportDefinition.Metadata.get_Item(“ExportTypeIdentity”))))’

, invalid exports may have been rejected.

Resulting in: Cannot set import ‘ABC.DEF.Shell.ViewModel (ContractName=”ABC.DEF.MenuViewModel”)’ on part ‘ABC.DEF.Shell’.

Element: ABC.DEF.Shell.ViewModel (ContractName=”ABC.DEF.MenuViewModel”) –> ABC.DEF.Shell

It turns out, this was for a very simple reason: I’d failed to actually include the line that sets the current assembly as one of the assemblies in the AggregateCatalog. Don’t do that!

    internal class Bootstrapper : MefBootstrapper
    {
        protected override DependencyObject CreateShell()
        {
            return Container.GetExport< Shell >().Value;
        }
        protected override void InitializeShell()
        {
            base.InitializeShell();
            App.Current.MainWindow = (Window)this.Shell;
        }
        protected override void ConfigureAggregateCatalog()
        {
            base.ConfigureAggregateCatalog();
            AggregateCatalog.Catalogs.Add(new AssemblyCatalog(System.Reflection.Assembly.GetExecutingAssembly()));
            AggregateCatalog.Catalogs.Add(new AssemblyCatalog(typeof(ABC.DEF.Module.ContactModule).Assembly));
        }
    }

That’s not X!

2010-11-09T21:00:18Z

Mark Graban has a great post [1] over at Lean Blog today. His overall premise is one we see over and over again in software: People take a good tool, apply it poorly or in the wrong context, and then blame the tool. Worse, they occasionally broadcast their blame, causing people to have a lower reputation of the tool.

In software, this is most recently seen in the certification discussion and in the “Scrum vs. kanban” conversation. As a result, I very much want to call out a couple of things Mark says in the healthcare context that apply equally well in the software context:

But the difference I draw from the Lean critics:

They see L.A.M.E. and leap to the conclusion that Lean is inherently bad. They throw Lean under the bus and take the tone that these leaders are “idiots.”

I see that Lean works well when leaders learn about and embrace the full management system. I work to try to educate healthcare leaders who are smart, but have a different viewpoint on management.

This is absolutely what I see with agile, lean, scrum, and kanban quite frequently. Teams try, fail, and blame the tool. Yes, it’s quite possible the team tried the wrong tool. It’s also quite possible they didn’t actually use the tool the way it was intended. [2] It could be their context is wrong for that tool in the first place, or that they changed their labels but didn’t change anything they actually do, or that they did things right but ran into a management wall, or faced struggles scaling out to other teams, or any of the hundreds of other reasons that adoptions “fail”.

We amplify this with the way our labels and language creates lines rather than identifying a body of perspectives and tools. It’s time to take a cue from BDD and define some scenarios and well-formed outcomes of the adoption. Mark presents a great list from a nursing perspective…

Why would nurses WANT Lean? When organizations really embrace Lean:

Nurses say, “Finally! Management is seeing the problems we face every day and we’re going to work together on fixing things.” Instead of feeling neglected, they are being properly supported and involved in process improvement.

The culture changes to fix routine “Every Day” problems – instead of jumping through hoops and constantly fighting the same fires, they get to focus more time on patient care.

Lean helps support save care – proven documented examples include reduced patient falls, fewer infections, and fewer cases of V.A.P. — nurses want the best for their patients.

With Lean, the space is designed and sized to match the processes for nursing care, instead of nurses being forced into spaces that don’t have enough bed/equipment storage space, for example.

They get to design their own standardized work and can improve it through daily huddles and kaizen boards.

The nurses get to work as part of a team (such as ThedaCare’s Collaborative Care model, where the RN helps direct doctors and pharmacists.

Nurses work less overtime because they have more time during the day to do proper charting instead of batching it at the end of the day.

These sure feel like things I can see and validate against when they happen. What’s our list look like for software? Here’s a few that come to mind immediately:

When a stakeholder asks for a release, it’s done quickly, easily, and with a minimum of fuss.
When a stakeholder identifies an opportunity related to the current application [3], it’s not painful to make it into reality.
When a problem is discovered with the app, it’s professionally and quickly fixed without attacks or defensiveness.
Simple and appropriate improvements are accepted and implemented with a minimum of fuss regardless of source.
Teams work in ways that suit them while still respecting their organization
People go home, have families, and maintain good balance despite loving their jobs
People care about the end result, not just their own specific job
Individuals are respectful across specializations, and attempt to use a shared language and goals rather than engaging in “legalese” or “geekspeak”.

What else?

[1] All quotes attributed to “Mark” are from this post, used with permission: http://www.leanblog.org/2010/11/why-nurses-should-reject-lame-and-demand-lean/

[2] Personally, this is where I stop because I feel like I’m approaching the “If it works, it’s Scrum, if it doesn’t, you did it wrong” perspective, but that’s not useful, so I’ll continue.

[3] Using “application” as shorthand for “software stuff we build” including systems, services, apps, etc.

EntityCommandShim – Silverlight MVVM helper

2010-10-25T20:15:13Z

I received a question yesterday around the shim pattern that I used, and I thought I’d clean it up a bit and share it. Again, if you’re used to reading about agile and lean here, this isn’t one of those posts (in case the title didn’t make that clear).

Refresher

The situation I’m addressing is that I have an element in my XAML that uses an ItemsSource (e.g. ItemsControl, ListBox) and want to receive an event specific to one item. This post will assume that I’m working with an items template like this, which will be bound against a DTO or Entity that has a property descriptively named “Value” containing the content to display.

<DataTemplate x:Key="TemplateSelectionItem">
   <HyperlinkButton Content="{Binding Value}" 
   />
</DataTemplate>

Now, whenever the user activates the hyperlink, the “Click” event will be raised. Without MVVM, I’d just add a handler in my code-behind that attaches to the click event and performs the logic. The testability and separation of concerns aspects of MVVM suggest this is a very bad idea. Instead, we should set up something that is called in response to the click. This leads to two options: put handling code on the bound entity, or put handling code in the ViewModel containing the ItemsControl. Since polluting the entity seems like a bad idea for a behavior local to a single view, let’s plan on putting it in in the ViewModel. And, since I like command objects for the flexibility they offer for a wide variety of scenarios, let’s target calling a Command instead of a method. Avoiding Prism-specific code, I’d write something like this:

        <DataTemplate x:Key="TemplateSelectionItem">
            <HyperlinkButton Content="{Binding Value}" >
                <i:Interaction.Triggers>
                    <i:EventTrigger EventName="Click">
                        <i:InvokeCommandAction Command="{Binding Command}" CommandParameter="{Binding }" />
                    </i:EventTrigger>
                </i:Interaction.Triggers>
            </HyperlinkButton>
        </DataTemplate>

Unfortunately, the “Command” binding will always fail, because the DataContext is currently set to my entity. Well, that’s fine, let’s just use a named source like this:

        <DataTemplate x:Key="TemplateSelectionItem">
            <HyperlinkButton Content="{Binding Value}" >
                <i:Interaction.Triggers>
                    <i:EventTrigger EventName="Click">
                        <i:InvokeCommandAction Command="{Binding DataContext.Command, ElementName=myListBox}" CommandParameter="{Binding }" />
                    </i:EventTrigger>
                </i:Interaction.Triggers>
            </HyperlinkButton>
        </DataTemplate>

Unfortunately, this is entirely too brilliant to keep working. It’ll work great as long as you don’t modify the ItemsPanel of your ListBox [1]. As soon as you do that, the binding will fail, and it took me far too long to figure out why the first time. Remembering back to my early programming education when I was taught things like “there’s no problem that can’t be solved with another level of indirection”, I decided to just put something in the middle that puts the Command where I need it. Thus the introduction of the shim, or as it’s now called, the EntityCommandShim. The comment prompted me to do some aggressive refactoring of what I was using, so thank you… here’s what it looks like now:

    public class EntityCommandShim<TEntity>
    {
        public EntityCommandShim(TEntity sourceObject, DelegateCommand<object> command)
        {
            Debug.Assert(command != null && sourceObject != null);

            this.Command = command;
            this.Entity = sourceObject;
        }

        public DelegateCommand<object> Command { get; private set; }
        public TEntity Entity { get; private set; }
    }

There’s not much to it. The Command and Entity references aren’t meant to change for the lifetime of the object, so no INotifyPropertyChanged implementation. The passed in Command is just a reference to one declared on the ViewModel, and doing the wrapping is quite simple with a line like this:

            this.AvailableTemplates = service.AvailableTemplates.Select(item =>
                new EntityCommandShim<FilterData>(item, TemplateItemSelectedCommand)).ToObservableCollection();

Once this is done, I’m all set. The ItemsTemplate becomes: [2]

        <DataTemplate x:Key="TemplateSelectionItem">
            <HyperlinkButton Content="{Binding Entity.Value}" >
                <i:Interaction.Triggers>
                    <i:EventTrigger EventName="Click">
                        <i:InvokeCommandAction Command="{Binding Command}" CommandParameter="{Binding Entity }" />
                    </i:EventTrigger>
                </i:Interaction.Triggers>
            </HyperlinkButton>
        </DataTemplate>

and the repeater can be as simple as this:

            <ItemsControl
                ItemTemplate="{StaticResource TemplateSelectionItem}"
                ItemsSource="{Binding AvailableTemplates}"
                >
            </ItemsControl>

Or as complicated as this: [3]

<input:AutoCompleteBox 
    FilterMode="Contains" 
    MinimumPopulateDelay="250"
    ItemsSource="{Binding Clients}"
    SelectedItem="{Binding SelectedClient, Mode=TwoWay}"
    ItemTemplate="{StaticResource ProviderSelectionItem}"
    IsDropDownOpen="{Binding IsSelectionActive, Mode=TwoWay}"
    ValueMemberPath="Entity.Value"
>
    <i:Interaction.Triggers>
        <i:EventTrigger EventName="SelectionChanged" SourceName="autoCompleteBox" >
            <ic2:InvokeCommandWithArgsAction 
                Command="{Binding SelectionChangedCommand}" 
                CommandParameter="{Binding RelativeSource={RelativeSource Self}, Path=InvokeParameter}"
                />
        </i:EventTrigger>
        <i:EventTrigger EventName="DropDownClosed" SourceName="autoCompleteBox" >
            <ic2:InvokeCommandWithArgsAction 
                Command="{Binding SelectionConfirmedCommand}" 
                CommandParameter="{Binding RelativeSource={RelativeSource Self}, Path=InvokeParameter}"
                />
        </i:EventTrigger>
        <i:EventTrigger EventName="DropDownOpened" SourceName="autoCompleteBox" >
            <ic2:InvokeCommandWithArgsAction 
                Command="{Binding BeginSelectionCommand}" 
                CommandParameter="{Binding RelativeSource={RelativeSource Self}, Path=InvokeParameter}"
                />
        </i:EventTrigger>
    </i:Interaction.Triggers>
</input:AutoCompleteBox>

[1] I haven’t tried this with ItemsControl, and I don’t know where this issue comes from… any thoughts? I also haven’t tried this in SL4, maybe it’s fixed? Will check that out soon, and if it works, all of this can be removed, making it a true “shim” in the sense that it goes away when the floor’s fixed.

[2] When I need to synchronize against SelectedItem in a list box, I often use CommandParameter={Binding} so I get the instance of the shim to reassign back to a property that’s Mode=TwoWay bound against SelectedItem.

[3] InvokeCommandWithArgsAction is adapted from http://weblogs.asp.net/alexeyzakharov/archive/2010/03/24/silverlight-commands-hacks-passing-eventargs-as-commandparameter-to-delegatecommand-triggered-by-eventtrigger.aspx and is a way of passing the arguments of an event as the CommandParameter. Check it out!

How does Google know this stuff?

2010-10-25T00:21:43Z

I get this – it’s a satellite picture of my condo building in Indy

On the other hand, how do they know where the individual units are inside? (Oh, and they’re RIGHT).

And it’s done for the apartments nearby…

What feed is this from?

Silverlight AutoCompleteBox and MVVM

2010-10-25T00:07:19Z

First, a warning

This is not an introduction to the AutoCompleteBox. If you’re looking for a basic overview of the there’s nothing better than Jeff Wilcox’s intro at http://www.jeff.wilcox.name/2008/11/autocompletebox-missing-guide/. Read that, then come back. It’s where I learned by example.

There’s no direct anchor, but about half way down the page you’ll find a section titled “Building a custom search filter” and I’ll be picking up from there. His advice is “Here’s a sample item filter, set in the Loaded event of my page”, which is the first indicator that this isn’t the way to go if you’re doing MVVM… what follows is a way around that.

Next, the backstory

One pattern I’ve adopted in Silverlight development with MVVM is to introduce a helper object when I’m binding to a ListView or ItemsControl’s ItemsSource property. I call this object a “Shim” because it a slightly hackish way of solving what would otherwise be a much bigger problem creating a lot of rebuilding needs. Essentially, the Shim becomes the new DataContext that sits behind each bound item, becoming a mini-ViewModel that matches the mini-View defined by the DataTemplate/ItemTemplate. My typical need is that I need to accomplish two goals:

Bind to the data in a DTO or Domain Object in some interesting way
React to a user’s operation against that item in some way specific to the data instance.

Unfortunately, markup like this doesn’t work in the ItemTemplate because the DataContext is now set to your specific DTO instance rather than to the original ViewModel instance.

<HyperlinkButton Content="{Binding Value}" 
             command:Click.Command="{Binding ItemSelectedCommand}" 
             command:Click.CommandParameter="{Binding}" />

Instead, I introduce the Shim with an assignment something like this:

this.Clients = insurer.Clients.Select(client =>

new ClientSelectedShim(client, SelectionChangedCommand)).ToObservableCollection();

and then bind as follows:

<HyperlinkButton Content="{Binding Value}" 
             command:Click.Command="{Binding ItemSelectedCommand}" 
             command:Click.CommandParameter="{Binding OriginalData}" 
             />

The Shim instance simply exposes the provided DelegateCommand as a property, along with the original DTO and often an intended display value. (Thus, I bind to “Value” instead of “OriginalData.Value”).

Overall, this feels like the cleanest of several different approaches I’ve tried to get around this binding issue in Silverlight. [1] I tend to do this using the Prism library, but it’s also applicable to other simple MVVM approaches. For example, the same would apply with using an EventTrigger and InvokeCommandAction to activate the command as such:

<i:Interaction.Triggers> 
    <i:EventTrigger EventName="Click"> 
        <i:InvokeCommandAction Command="{Binding ItemSelectedCommand}" /> 
    </i:EventTrigger> 
</i:Interaction.Triggers>

Finally, the Point

All this rambling is a lead up to the challenge I hit today: I added a Shim into the list of items used to populate an AutoCompleteBox, because I needed to respond to the user with some item-specific behaviors in the popup that contains the matching values. As soon as I added the Shim, I couldn’t use the simple FilterMode=”Contains”, because it was comparing against the type of the shim rather than the much simpler string I was giving it before. Because I suddenly had a Shim involved and couldn’t dodge the complexity of having a DTO by just yielding the display value any longer, I was forced to use FilterMode=”Custom”. {EDIT [3]} Problem is, MVVM strongly discourages me from writing code anywhere that looks like Jeff’s sample:

MyAutoCompleteControl.ItemFilter = (txt, i) => Stock.IsAutoCompleteSuggestion(txt, i);

The good news is that binding is incredibly powerful, so here’s what I can do instead:

<input:AutoCompleteBox  
   FilterMode="Custom" 
   MinimumPopulateDelay="250" 
   ItemsSource="{Binding Clients}" 
   SelectedItem="{Binding SelectedClient}" 
   ItemTemplate="{StaticResource ClientSelectionItem}" 
   ItemFilter="{Binding ShimFilter}" 
/>

<!—In Resources –> 

<DataTemplate x:Key="ClientSelectionItem">  
   <StackPanel Orientation="Horizontal" > 
      <HyperlinkButton Content="{Binding Value}" 
         command:Click.Command="{Binding ItemSelectedCommand}" 
         command:Click.CommandParameter="{Binding OriginalData}" 
      /> 
    </StackPanel> 
</DataTemplate>

And, in the ViewModel: [2]

public AutoCompleteFilterPredicate<object> ShimFilter 
{ 
    get 
    { 
        return new AutoCompleteFilterPredicate<object>( 
            (str, item) => { return IsFilterMatch( str, item ); }); 
    } 
}

private bool IsFilterMatch(string valueToCheck, object item) 
{


    ClientSelectedShim shim = item as ClientSelectedShim; 

    if (shim == null) 
        return false; 
    return shim.Value.Contains(valueToCheck); 
}

It wasn’t immediately obvious to me, but this definitely lets me put my comparison code in the ViewModel where it belongs while keeping the expression of the UI structure and behavior. Now I can wire the special effects!

[1] One of the most promising ways I’ve otherwise tried involved binding using ElementName to get “out of” the ItemTemplate and back to an element on at the parent level, typically the containing list itself. This will work as long as you don’t change the ItemsPanel to a non-default panel. Making that change hits some bug (feature?) in Silverlight I can’t properly characterize and causes lots of pain figuring out what happened. Be warned!

[2] The property has to stay templated to the <object> type or there will be a binding expression error when the View is first bound against the ViewModel… covariance doesn’t apply here. More info here. Second, it appears it must be an actual property, the binding failed when I tried to have a simple field initialized to the Predicate.

[3] After figuring this all out, I did of course come across the incredibly useful ValueMemberPath property, which allowed me to revert to a simpler design for my specific case. However, if you are doing custom filters (especially if you’re checking against multiple properties, which I’ve done in the past), this technique will still be quite valuable.