Lone Star Developer

Emacs!

2009-04-28T19:09:00.001-07:00

This has been posted with the use of e-blog, an Emacs addin that allows me to write my posts from this editor. So far so good, it seems!

Yeah, this is just a test.

Edit: Oh yeah, I published the post from Emacs too! How sweet is that?

New blog

2009-02-17T11:08:00.001-08:00

I have been honored recently with an invitation to blog on the Los Techies website. If you like my technical blurbs of knowledge, come on over and subscribe to my RSS feed! In fact, why don’t you go and subscribe to the main RSS feed? The bloggers there are all much smarter than me, really.

http://feeds2.feedburner.com/LosTechies

I plan to keep this blog going with slightly less technical postings.

Thanks for reading!

Did you feel that?

2009-02-12T15:04:00.001-08:00

A tingling sensation in the pit of my stomach, like a small flame that refuses to die as you turn off the gas… This is something I haven’t felt since my first days in college!

The passion for my craft and the pleasure I used to have at practicing it manifested themselves today. I’ve always known they were there, but for some reason they were hidden from my reach.

Through all the jobs I’ve had, I’ve yearned for that feeling to come back, that sheer joy of programming. I’m not talking about the exultation you feel when something that’s been bugging you finally works, like making an SSIS package work properly for example. No, I’m talking about loving the grind we all go through on a daily basis: writing lines of code.

I’ve been working on my behavior driven development skills and learning NHibernate today and it felt like I clicked. I was unable to write code before having written a spec and it eventually made me refactor in a few changes that made my code look… pretty!

I realized that I love doing well designed code. I thrive in such an environment and I’m able to enter the zone: spec-code-test-spec-code-test-spec-code-test… It makes me feel proud of what I did. I look at that big green bar and think that I can go home a happy man tonight. I’ve earned my diner. I feel like I went out of my cave and brought back a juicy mammoth for my family to eat. Aaarh! Something like that, anyway…

I don’t want to go back to my old ways, where passion had no room to expand and was all but extinguished! I shudder just to think of it. I don’t want to let go! I want to fan the flames.

Design patterns usage

2009-02-10T11:16:00.001-08:00

“What design patterns have you used?”

This used to be the killer question for me at all the job interviews I went to right after graduating from college. I’d stumble and mumble, even though I’d studied them the night before, like with so many previous employers. You’d think that I’d know them by heart! All my poor younger self could do is say that he used the singleton pattern extensively and cross his fingers… It is obvious that what I lacked was practice. I’ve had to self-teach those skills over the years to overcome that deficiency.

Most job postings these days mention that you need some sort of expertise in Design Patterns. I deplore the wording, somewhat, since there are far more patterns than the ones depicted in the Gang of Four book. For example, Design Driven Development introduces a couple of new patterns, Martin Fowler brought in the Active Record pattern with his books on enterprise patterns and Agile practices have their own library of very useful patterns like Dependency Injection, which requires you to use the bridge pattern. This is clearly a fast evolving field where patterns now use other patterns! Having been an active learner in the past two years, I have yet to come across a blog posting on how to use that old bridge pattern. Instead, I see a lot of Dependency Injection patterns articles, or Inversion of Control tutorials.

Separating your implementation from the interface is simply a good practice now. Is it really still a pattern? Do we still need to call it a bridge? And what about dynamic languages like Ruby, where there is no need for such a separation? Some patterns are so trivial to use in some languages that we are using them without even thinking about it.

Patterns are tools that allow us to do our jobs more efficiently, which is why employers keep asking us about them. You plug-an-play them into your object model, tweak them a little, run your tests and voila! It works.

Have you ever used Linq to SQL? Then you have used the unit of work pattern. Do you practice test driven development or behavior driven development? Then you most certainly have used dependency injection to make your system under test work with your mock objects.

I’m also willing to bet my life on the fact that you have used the infamous singleton pattern, where only one instance of the object can exist. I’m also going to go so far as to say that unit testing an object that uses a singleton internally has caused you to cry yourself to sleep.

When you are asked which design patterns you have used in the past, the interviewer typically wants to know if you know your craft’s tools. He wants to know if you’ll spend days, and company money, banging your head against a problem that could be easily solved using a design pattern. Am I personally an expert in design patterns? Do I know them all? No. The important thing is to know they exist and how to use them, and the only way to do that is to stay alert and keep up with what your peers are doing. Of course, a little practice goes a long way, because when the pressure is up and a deadline looms, we all tend not to do what’s right.

BDD testing environment for .Net

2009-02-07T13:33:00.001-08:00

In my last post, I mentioned I was going to use BDD and DDD to build a silverlight application. Well it took me a couple hours to settle on which BDD framework to use.

I started by trying out NBehave, but found it very verbose. I do not enjoy re-writing my stories in my tests. Stories should be defined, yes, but in a separate document. There is no point to repeating them again in the code.

Thus began a search for which framework I should try out. First, you have different unit testing frameworks, on top of which the BDD ones are built: nUnit, xUnit and MbUnit. Then you have a choice between xUnit BDD extensions, SpecUnit.Net, NBehave, StoryQ and mSpec.

Well, that wasn’t easy. In the end, I decided to go with SpecUnit, just because it was Scott Bellware’s project and he is quite vocal about what the others do wrong (see comments) and I’d like to see what he’s got to teach me with his framework. Yes, mSpec was greatly influenced by SpecUnit, but it is currently under development, so I’ll wait a bit before I try it. I guess the main point is that writing a few tests in SpecUnit was enjoyable, which is important if you’re going to be writing thousands of tests, I guess.

How to learn

2009-02-07T08:32:00.001-08:00

I’m going to Austin, TX in two weeks for a job interview where I’ll be asked to demonstrate my coding skills. Now, I’m the kind of guy who’s always humble about his talents. I mean, I grew up surrounded by people with mad hacking skills! They were doing object oriented programming, developing games and creating resident programs to allow us to use the mouse in the school lab when I was still trying to figure out how to program in Pascal.

Later, in College, those guys were rewriting C compilers, just for fun or putting in place a networked messaging system for their Quake-like gaming engine.

Great. I was having trouble balancing a binary tree in C++.

Okay, I’m selling myself a bit short here. After all, I did program a photon mapping engine and an image-based renderer. But we all know these guys. You know, like the one that created Linux in his spare time? These guys can make you seriously doubt your own skill set. On the other hand, they represent only 1% of all programmers so maybe I should really see how I compare to the other 99%. Well, not too bad, I suppose.

The key here is that they knew things I didn’t. They practiced when I wasn’t. So I’m a bit to blame here. However, I’ve recently taken back the “student” mantle and tossed it over my shoulders.

Learning

Here is a great article about learning and understanding. Go read it. I’ll wait.

Okay? Wasn’t that interesting? I mean, I knew practicing was part of the whole equation but I had never looked at the whole learning process like this before.

I’ve decided to put it into practice for the interview I’m going to. I’ve been reading a lot (and I mean *a lot*!) about being agile in the past two years without having the chance to put it into practice where I work. Well that changes now. I vowed to my interviewer that I would learn some Silverlight and show him what I can do. Thankfully, I’m already at the practice stage here since I’ve played with it in the past (a good thing for me since Scott Guthrie’s tutorials are a bit outdated and you’ll have to figure out a thing or two on your own.)

It might be backwards, but I’ve been mixing the learning stages a bit. For example, I’m reading the DDD book and creating domain models as I go, practicing without knowing everything. Same with BDD; I jumped into rSpec on a ruby on rails project without even knowing how it works.

I usually tend to jump into it and fight my way out. But this path can be frustrating to follow, with too much stuff on your plate all at once.

My Goal

I’ve decided to build my Silverlight application using BDD and DDD, two subjects I’m currently studying intensely.

However, I will start by practicing the technologies and practices in isolation, so as not to be burdened too much. I’ve already been going through some Silverlight tutorials. Next, I’ll learn MbUnit and develop a domain model for my app with it. Finally, I’ll create the UI layer with Silverlight. I have 13 days, a wife and two kids. Can I do it and manage to impress a potential future employer?

Never Eat Alone

2009-02-04T14:10:00.001-08:00

I’ve just finished reading Never Eat Alone, a great book written by Keith Ferrazzi about networking and connecting with other people. I feel somewhat inadequate in this respect and I hoped I could gain a few good pointers from such a book. And I did! Living in the deep south of the Texas plains, I definitely need to grok networking to get further with my career.

The book was overall very useful. It is filled with techniques and examples of people who have successfully used them.

This is my attempt are summarizing the book.

mind set

You need to want to become a member of the club, somehow. Don’t be afraid! You want to hang out with the top dogs. In the end, who you hang out with kinda dictates who you are. Don’t keep score, be out there and help without asking for anything in return.

What is your mission? Write down your goals. It has been shown that you are more likely to achieve them if you write them down. Ferrazzi suggests detailing where you plan to be three months, six months, a year and three years from now.

You also need to build the network before you need it. This thing takes time! You need to have helped before you can be helped. So get out there and be audacious. Get in front of people and don’t be afraid to try and meet who you admire. CEOs and gurus are people just like you and I. As far as I know, they haven’t found a way to reach godhood, yet… But be careful not to be that networking jerk who tails people at conferences just because they said hi. Don’t hand out business cards without having first established a meaningful contact.

the skill set

Do your homework. Research who you want to meet and find out what they like, where they hang out and with whom, and where you’re more likely to “bump” into them. This isn’t cheating. You did it for your girlfriend or boyfriend, didn’t you? Why should it be different for IBM’s CEO?

Whoever you meet, take their names and note it down. These people will be useful later. You’re expanding your network and everybody is worthwhile to know. Remember, you’re not out to abuse your network, you’re also there for them. One of these people will probably be the key to getting connected to somebody more important. They can be referrals that you can use when cold calling your idols, which makes it not-so-cold anymore.

Speaking of cold calling, befriend the secretary or personal assistant. They can become powerful allies. Did you know they can actually influence the person you’re trying to reach? Yeah. They can.

Take your people out to lunch, share your passions. Keep developing the relationship and strengthening the bond. Invite somebody else you know to your lunch and be a connector yourself! And when you meet new people, follow up before they forget about you. A gentle reminder of a meeting in an email, simply saying thanks for a great time or just saying hi will work. You want them to remember your name the next time you talk to them.

Be a conference commando. Go to conferences to network. Research who will be there and plan on “bumping” into them. Learn who is organizing the whole thing and volunteer to help out. You’ll meet a lot of people that way in addition of getting a heads-up on who’s coming. Also, try to be a speaker.

Connect with connectors and marvel at the power of crossing the streams. I don’t care what you learned in Ghostbusters, crossing the streams is a good thing when mixing networks together. But there are rules. Don’t give away your entire network all at once, because the other won’t. And don’t make your connector friend look bad with their network.

Always work towards expanding your circle. Find ways to meet new people at all times. Master the art of small talk, which can be hard. Try listening to local talk radio. You'll have some stuff to talk about. It also help if you know what the other person is into.

turning connections into compatriots

Health, wealth, and children. Those are the three things people really care about. If you can help somebody with any of those, you’ve just made a friend for life. Can you get that guy’s kids into the best college in town? He’ll be forever grateful. Or did you refer that gal’s daughter for a very coveted position that just opened up in your organization?

Connect people together. If you meet someone who needs something and know a friend who can provide it, please connect them. Learn more about arbitrage here.

Ping your contacts constantly. You don’t want them to forget you. Depending on how close they are to you, ping them monthly, semi-annually or annually. When you travel to their area, give them a call. Try to meet face-to-face.

Organize diner parties and invite anchor tenants: people with certain influence that keep coming to your parties. Almost by magic, other people will want to be there too.

trading up and giving back

Be interesting and build your brand. You have something to say and shouldn’t keep it to yourself. Form opinions, ideas, philosophies. Don’t be average. Stand out! Then broadcast your brand. Shout it out. Blog it, write guest columns, talk in conferences.

Get close to power by meeting assistants or people close to who you really want to meet. In gatherings, the important people will typically be extremely busy, while their aids will stand idle. Meet the aids! Again, we are not out to use them. We really want to know them.

Build a network that people will want to join, but don’t become arrogant or you’ll kill the golden goose. Find mentors who can teach you and mentees who will learn from you.

Finally, welcome to the connected age! Use LinkedIn, Twitter, blogs, FaceBook, Plaxo, etc. Connect!

Good Design

2009-01-29T08:41:00.001-08:00

Someone asked me on a phone interview how I recognize a good software design. I think I managed to mumble some kind of response after blabbering a bunch of “uuh” and “aah”. I didn’t get the job, obviously, but since then I’ve never put the question to rest. It kept hunting me, distracting me like an annoying fruit fly that just won’t go away.

So I put a lot of thought into this and managed to get only as far as realizing that the following are signs that you have a good design:

The code is readable. By reading it, you can extract knowledge from it with a minimum of effort.
The code is reusable.
The code is easy to change.
The code is succinct and to the point.

Maybe I need to call the guy up again and tell him all of this. “Hey, it took me a month, but here’s what I think is essential in a good design!” Sure, it all looks like common sense. But that also means it is pretty rare. I find it incredibly hard to live up to those four intuitive guidelines. Sometimes I even forget that they exist!

Finally, I got my feelings validated a day or two ago when I read a blog post that mentioned the four qualities of BAD design (I apologize, but I can’t find that post anymore!):

rigidity
fragility
immobility
viscosity

Rigidity means your code is hard to change. Fragility means a change will break everything. Immobility means your code isn’t reusable. Finally, viscosity means that your stuff is hard to read and understand.

Wow, that’s exactly what I was thinking about! Now let’s get to work and apply all of this on a daily basis, shall we?

The answer to anything and everything…

2009-01-13T09:54:00.001-08:00

… hasn’t been found yet in the case of software development.

We used to say that a good singer could sing the phone book without boring his or her audience. Nowadays, I feel like a good test for singers would be to make them sing all the Agile acronyms without putting our managers to sleep. Or when your tired at night, instead of counting sheep, recite the acrony… Zzzzzzzz…

In the past couple of years, we grew so fed up with the classic waterfall model of software development that we started shooting in every direction at once in the search for the right way to develop software.

We have Agile development, with short and fixed iterations. Then Lean software development came about a said to hell with fixed iterations! We do it like Toyota is doing it (so much so that I actually wonder why they didn’t call it Toyota Software Development, or TSD in short.) This is where a lot of our Agile terms come from, by the way. If you ever wondered what was up with all the Japanese terms in our field today, they come from Toyota: Kaizen, Andon, Kaikaku and Kanban are the ones I’ve heard the most, but there are about 10 others.

Sprinkle all of this with a bit of DRY, POCO, SOLID, DSL, BDD, TDD, DDD, IoC, MVC and PVC and you get a giant pool party where only the smartest ones that have had time to research all this are invited. It is our own Playboy mansion party!

It’s like being 15 all over again

It seems like we are entering our teenage years as a community. We are going through our identity crisis, trying to figure out who we are. Is writing software like building a house, or is it like manufacturing? Maybe it is like growing a garden? It’s probably none of those things. We don’t know for sure and these days, we are sticking with the manufacturing metaphor. Maybe in a couple of months we’ll think coding is like being in the police: you take your problem and circle it, make sure you have your snipers into place and don’t let any bug escape!

Eventually, however, we’ll start settling down in the same way other, more ancient fields have done. Engineering has it’s own set of rules and guidelines, and so do the people in manufacturing and construction, lawyers, cooks, doctors and party planners. They’ve been through it before. In the meantime, being teenagers, we get excited by the new stuff: “this new watchamacallit is soooooo gonna replace Kanban boards!” But it the end, we need to show restraint.

What has been very hard to me recently was how to learn everything and how to use the new methodologies. Behavior Driven Development and Test Driven Development, for example, are two ways of driving your development. There can be only one driver, however, so which one is it? It seems like BDD has superseded TDD and is the way to go. Thank God I skipped TDD!

I also started learning Domain Driven Design to be a better designer. However, the methodology calls for a lot of coding *while* you design your model. Ooookay, so do I have to code using BDD? I guess I could! And where does Activity Modeling fit into all this? Hmmm, I could use it to help me figure out my model. But where do DDD and Activity Modeling meet? At what point do I switch from designing the user experience to designing the domain model? I know they overlap a bit, but not completely. Do I use DDD to create my model layer and the create the app and UI layer using activity modeling and user stories? And can you use a Kanban board if you do DDD?

Etcetera…

This stuff is hard and people can’t do and test everything out. Some methodologies are known to be incompatible and some might repel each other, except that we don’t know it yet! Some very smart people are doing the hard work of figuring it all out and mostly end up saying that “you don’t have to follow every methodologies to the letter.” Great, now we can mix and match to circumvent things that won’t work together. What’s more, each team is different. What works somewhere won’t work everywhere. We need to adapt.

God damn it, I feel like a freaking headless chicken right now! Nobody has a perfect answer, everyone is still searching and every branch has its own dedicated evangelist saying that their way is THE way. I almost want to give up. The waterfall model seems so attractive compared the junk yard of methodologies in front of me…

Let’s make this work

Okay, let’s see. What have we got here…

Here’s what I know:

Coding takes time: a loooong time.
Coding always takes more time that you think
Coders are a constraint in the system. Software cannot be shipped faster than coders can code.
Modifying code is hard
Programmers and users never seem to understand each other

Being a constraint in the system, coding must at all costs be protected so that it never stops. While this is all good in theory, it won’t be so in practice because we have to talk to our users, fix build servers, etc. This is what I like about Lean development: the idea that one must look for waste and get rid of it. In my mind, waste is anything that slows down development, like interruptions, waiting for customer response, waiting for DBAs, business analysts or managers. All of this waiting should be minimized, and any technique that helps you do that is good.

Oh, and why not throw in some good design practices in the mix? Your code should be easy to understand, easy to modify, easy to test and easy to reuse. Phew! Now for the rest of us mortals, try to make it as easy as possible, okay? Taking the time to understand code or being stuck when adding new features to unmanageable code: this is waste.

Another form of waste is the fact that users never seem to speak our language. This is why I love DDD. It helps develop a domain model and language in unison with the users. If done right, it increases the chance that the software will actually do what the users want! Novel, I know…

Hence, every methodology that I will ever follow will do the following:

Protect the coder by minimizing waste, and
Help users and coders understand each other

That’s it. The rest are just tools to help me achieve the above points.

It took since the industrial revolution to get to the way Toyota manufactures cars today. Software development is barely 40 years old. Hang in tight, it’s going to be a bumpy road.

Caffeine

2009-01-07T13:31:00.001-08:00

Coffee tip#1:

Did you know that there is more caffeine in lighter roasts than in dark roasts? The roasting process burns away the caffeine in the coffee bean. While it might seem that the stronger taste indicates a stronger coffee, what you are tasting is just burnt oil.

Espresso beans, which are just regular beans that have been roasted darker to remove unwanted acidity in the taste profile, do not contain more caffeine than regular beans.

A 7 oz cup of drip coffee contains about 115mg of caffeine whereas a shot of espresso will contain 100mg. However, some espresso blends have a small amount of robusta beans. These beans have a higher caffeine content and might increase the dosage a bit. By the way, never get a cup of 100% robusta. That varietal is awfully bitter, almost undrinkable.

In the end, if you’re going to stay up all night programming, you’ll be better off drinking a light roasted cup of coffee then a darker one, or even espresso! What’s more, light roasts taste so much better. I purchased a small home roaster a year ago and have been roasting my own beans. In the morning, my cup of coffee leaves me licking my sticky lips, craving more. Yes, my lips get a bit sticky because of the very flavorful oils in the coffee beans that you don’t get from store bought beans. The oils disappears about a week or two after the roasting process.

Resolution

2009-01-06T14:45:00.001-08:00

I'm 6 days late but I found one goal for the new year, which I'll hopefully attain before the summer.

I want to run a sub-20 minute 5k.

I already broke the 30 minute barrier, which was enough to create a loud BANG noise. Er.. no, wait. That's the sound barrier. Never mind.

Why running a sub-20 5k? Because We have four 5k races at work, once a quarter, where the winner gets $150. That's $600 per year!

And I want it. The problem is, there's this guy in my age bracket who runs the 5k in about 21 minutes, hence the sub-20 goal.

Training

I'm planning a toned-down, cardio centered crossfit workout during lunch, with one or two days of strength training every week. I will also run in the morning, 3 or 4 times a week doing mainly sprints like 4x400m or 8x800m and a longer run, maybe on Saturdays.

I'll also go back to my no bread, no sugar diet for a while, which is going to slim me down quite a bit. I'm hoping I can lose 10 pounds, which I won't have to haul when I run.

Goals

It's good to have a goal again! After attaining my goal of reaching my BMI last year, I was kind of in limbo. I didn't know what to train for and I wasn't very motivated to go to the gym . On top of that, I started eating bread and sugar during the holidays and had to go back a notch on my belt. Not good.

But now, with this new goal, I should be all set! Waking up early is gonna be a bitch though...

Format change = process change

2009-01-06T12:09:00.001-08:00

I have just read that Dr. Dobb's Journal is going Web-only. My question now is will they try to fit a square peg in a round hole?

Traditional print magazines gather articles and publish them on a strict schedule. This is imposed on them by the publishing industry, where it is more cost effective to print one million copies every month then to make half a million every other week.

By going online, however, a magazine becomes free of that constraint and can release content every day or so. I hope that Dr. Dobb's Journal change their process this way instead of doing it the good old way just because "it's what we've always done!"

I have a subscription to the Crossfit Journal, a newsletter turned online magazine about fitness. They moved away from a monthly PDF to continuous content creation, which is a fabulous way to make your customers come back for more every day or two. Also, reactions to articles and discussions happen on a much shorter span where authors can respond or react almost immediately.

Changing your medium is a great opportunity to change your model for the best!

Alone in the back country might be a good thing

2009-01-06T11:52:00.001-08:00

The whole reason for this blog is to keep me connected, somehow, to the technological world where most uber-geeks live. I copied myself on the Internet through this blog, Twitter and Facebook to create the illusion that I live in this beautiful giant metropolis where bits fly down the highways and jump down from the high rises that are the layered applications of the World Wide Web.

My hope was to one day get back and live in a real city like Austin or San Fransisco, strong from all my digital connections. This all changed, however. I might not want to live in a big city anymore. Not since I've read this article on Boston.com, anyway.

Cities are bad for my brain? That is definitely a thinker. All the things that I grew up to love about cities prevent my brain from operating at its full potential by saturating it with useless sensory experiences. In contrast, nature tends to heighten and relax the mind, studies show.

This seriously makes me rethink my strategy. While I miss the sophistication of bigger urban areas, with their 20 varieties of blue cheese and exotic food and restaurants, I also crave the quietness of nature and the sight of water.

As software developers, we are incredibly fortunate to be able to telecommute and work at a certain distance from our customers. Technology is helping us by allowing us to stay connected without being physically together. IM software, emails, scanners, the Web, Google Docs and now iWorks from Apple now allow you to share data online. More and more, people have the tools they need to connect and meet on the Internet.

We already see user group meetings online where one can attend from anywhere and contribute to the discussion. Apple enables you to present a keynote presentation (similar to power point) through an IM session. I'm positive there is already a tech book club online. Hell, IRC has been very useful for years already! What about Linux? It's been developed by people who lived all around the world.

Plus, I can always order my blue cheese from Amazon.com. Hmm, maybe it is time to move further into the wild! All I need is a T1 line and I'm all set!

A message...

2008-12-25T07:21:00.000-08:00

def christmas
"Christmas"
end

def wish_a(holiday)
puts "I wish you all a merry #{send(holiday)}!!"
end

wish_a(:christmas)

Is Smart UI what we need?

2008-12-23T10:13:00.001-08:00

I just read a very perturbing section in Domain Driven Design called "Smart UI". This unthinkable, heretical pattern advocates that you put your business logic in the UI and that sometimes, it might even be worth it.

It was too late. I knew that the seeds of reason had started growing inside of me, like a vine on steroids. Soon, one of the thorns pierced my heart and I gave up.

According to Evans, Smart UI is useful for small projects or for teams that are not sophisticated enough to handle the complexity of Domain Driven Design. In that case, putting all the business logic in the UI will at least isolate it in the sense that all one has to do to modify it is to replace the UI. Of course, this means trouble in the long run. Things will get progressively harder and harder to change and you will end up with a monolithic beast that demands too much attention and sleeps on your side of the bed. But...

What if this is the best my team can do? Should I embark on a crusade to teach them Agility when I know deep inside that they don't really care? As far as they're concerned, they think things are just fine the way they are right now.

Why don't I think the same thing? Why am I on this quest for new methods and patterns?

I want to have fun programming like I used to in high school. This is a deep motivation of mine. Every company that I've worked for so far has somehow succeeded in making it more of a burden. I love the pureness of the craft, the manipulation of concepts, the massaging of data. On the other hand, I find that trying to make things fit is a bother. I love Quality things that just work.

This is who I am. I'm attracted by the beauty of mathematics, the usefulness of Ruby and the creativity and science of cooking food.

I'm put off by by the monotony of building data centric web applications that simply mirror the database schema with some extra validation and business rules sprinkled all over the place. I'm put off by anything that takes time and requires me to stop my creative flow, by things that create friction and that refuse to work even though it seems like you did everything right.

I also hate getting off track like I did just now... But it had to get off my chest. Sometimes I don't even know why I do it; why I sit at a desk and try to make code behave. This little rant helped me refocus on what makes me tick and on why I might want to go the Smart UI way with my team. In the end, it is the customer that counts. I might not become the equivalent of a Michelin three stars chef this way but at least I'll make some people happy... in the short term. The future, fortunately, will force us to change. By this time, I might gain enough influence and wisdom to help my team migrate towards a more... educated way of doing things.

Who is your Chief Engineer?

2008-12-20T08:13:00.000-08:00

I really like Scott Bellware's post about a chief engineer to drive the development of software within your organization, which is based on the definition of the same role at Toyota (which, incidentally, is referenced quite often when talking about lean software development.) I also like the though that such chief engineers need to be grown and cultivated within an organization with a culture of change already in place. This is very similar to what Mary Poppendieck calls master developers in her book Lean Software Development. These people are responsible for representing the customer as domain experts and oversee the design and development of software since they know about the technology and still use it!

In another post, Scott rewords the chief engineer's responsibility within a context of acceptance testing:

A Chief Engineer in a software product development organization can write acceptance tests without the burden of elaborate end-user testing tools. He can use the common tools of the trade. He understands the imperatives of using tests as documentation and uses usability-focused test authorship, and sets standards for authorship that his organization cultivates and follows.

If I'm reading between the lines correctly, it sounds like the chief engineer might not be writing code anymore, but can sure write tests to insure that the software is implemented in a way that its behaviors pass the tests that he wrote. I like that very much. I've read in so many places and truly believe that great managers and leaders still code. They have to have their hands wet and dirty. Writing acceptance tests, or BDD and other kinds of tests is a great way to do that while at the same time making sure that they are not in the way of the developers. Talk about having your feet in both the customer's world and in your own world at the same time! I don't think this concept exists any other way in software development.

Now, how do we get a chief engineer if we are a one man show like myself? I guess it is up to me to cultivate myself into one. But that can take years! I guess the only solution, in the meantime, is to surround myself with customers and use all the tools that we have so that I can understand them as best I can. BDD, DDD, acceptance tests, etc... Eventually, either I'll become a chief engineer in that domain or I'll end up cultivating another one, which will mean that the business that I haven't started growing yet will be making money (crossing my fingers!)

Thanks Scott for the great post!

SQL separation of concerns

2008-12-17T09:03:00.001-08:00

After writing my post last night, I started thinking about ways to remove the bad smell coming out of my SQL code. The issue is that I wrote a stored procedure whose

My thoughts followed this logic:

Well, that's okay. I'll just put an obvious comment in there asking anybody not to change the result set. I'll also rename the stored procedure, ending it with "ForInsert" so that it is obvious what it's for.

Oh man, that is even worse! Now it's like I just sprayed some strawberry fragrance to cover that nasty bathroom smell... You know what I mean!

I guess I could put in some output parameters in my stored procedure instead and use those to insert the data I want. That way, as long as nobody removes a parameter, I'm fine.

Yes, that's better. Compared to my previous thought, this one removes the problem where another programmer would change the order of the fields returned, remove or even add fields in the returned result set. Adding field is the biggest issue here. I know I like to "supplement" stored procedures here and there by adding extra information that is returned to me. Which makes me think...

If I want a stored procedure to return extra data, I'll write another stored procedure that returns the result set of the first one with my extra stuff added to it. That way, I won't break the initial stored procedure.

Hey, I'm learning as I go, Ok? Cut me some slack. I'm going through the usual programming learning curve here. I'm fairly advanced when it comes to regular code like C# or Ruby, but mediocre with SQL. It feels like I just came out of the "Look 'ma! SQL code!" stage and started wondering about encapsulation and separation of concerns. Of course, that is much harder to achieve in SQL than it is in any good object oriented language.

I had one last thought, which I haven't acted upon yet. What about using views and filtering the data on the view instead of passing parameters to a stored procedure? Or what about using common table expression? What about temporary tables? I haven't had time to think those through just yet, but I will...

SQL code can smell too

2008-12-16T19:00:00.000-08:00

I just wrote some bad SQL code tonight that I think I'll have to revisit tomorrow morning when I get back to work.

I think that using a stored procedure to populate an Insert statement is simply wrong. Should anyone modify that stored procedure or change the order in which the values are returned, my insert will fail. Can you smell that? Yeah, it smells bad.

I was simply trying to "encapsulate" things a little and make my code less cluttered. Maybe using a common table expression or temporary table would be a much better idea.

Losing your work

2008-12-12T15:56:00.001-08:00

Windows Live Writer, which I use to write this blog, decided to stop allowing me to use the undo function just at the moment when I had accidentally deleted 3/4th of my post. Of course, I had saved my work both before and after the accident. I guess the "save often" philosophy works only when you can trust the undo button to do its work properly.

I hope you guys were not hoping for my next installment of my SQL training recap. I really don't feel like writing it all again and I'm getting a little tired of SQL code. Yeah, it gets old.

Could somebody write a text editor that keeps track of the history of your file? Kthxbye.

State of my Digital Self

2008-12-06T08:45:00.000-08:00

When I created this blog last July, my goal was to brand myself online and to create an online persona that would represent me, my thoughts and my values with the goal of becoming visible and gaining the respect and recognition of my peers in the software development world.

I started blogging without caring about the whether I was right or not with the assumption that the process of learning that takes place while writing is more important to me than being right. I have, after all, the rest of my life to be right.

I opened a Twitter account with the intent of meeting new people and hoping that a few would follow me. What I discovered, however, is that Twitter is at its best when you use it to follow the right people. That is, following people you want to meet or that something that you don't, be it skills, money or fame. That is the magic of Twitter and that is how I got to go to the best conference I've ever been to and meet people who would have a lot to teach me, and still do continuously!

I'm also on LinkedIn. However, I'm not sure I have uncovered the key to that social networking site yet. I don't know how to make it work for me. It is a great place for people to examine your professional past and references, but that's about it so far. I'm sure I'm missing something, but what? I need to look into it some more.

Then there's Shelfari, a social site where you can advertise the books you've read, the ones you'd like to read and the ones you own. Seems interesting. I opened up an account just a couple days ago so we'll see where that takes me.

And last but not least: Facebook. I do not have an account on Facebook simply because I never understood it. I used to have an account on MySpace and I hated it. Should I do Facebook? Is it different? I guess I'll have to check it out. Today!

Advanced SQL training

2008-12-05T12:45:00.001-08:00

I ranted a bit about SQL in general in my previous post, which was supposed to be a quick intro to the SQL training I'm putting myself through at home. Instead, I rambled on about developers being scared of SQL and bending over backwards in order to avoid it.

Anyway, my boss asked me last week to go to a SQL training. He correctly identified my lack of knowledge and understanding about the database tools we use at work and wanted to correct that. But instead of going to a training that just covers the basics in a breath-first manner, I bought two books: Inside SQL Server 2005: T-SQL Querying and Inside SQL Server 2005: T-SQL Programming. Oh my God... The amount and depth of knowledge in those books is staggering. I have four days to go from "just getting by" SQL developer to a "data can't hide from him" database guru. Of course, I told my boss that I'd be blogging about my discoveries in order to cement the knowledge in my head but now I find that I will not have enough of my four days of reclusion at home to learn everything I want to learn. I need to focus on what I need for work.

For this training, I dove deep into two books and started with the fundamentals...

Fundamentals

What steps SQL Server goes through to evaluate a query is a very useful thing to know and helps explain why certain things can or can't be done inside a query.

Take the following query for the Northwind database, which select the total number of orders made by all american employees where that number is greater than 100:

SELECT COUNT(Orders.OrderID) AS TotalOrderCount, Employees.EmployeeID

FROM Orders

    JOIN Employees ON Orders.EmployeeID = Employees.EmployeeID

WHERE Employees.Country = 'USA'

GROUP BY Employees.EmployeeID

HAVING COUNT(Orders.OrderID) > 100

ORDER BY TotalOrderCount

SQL Server starts by executing the FROM and JOIN statements by performing a cross-join between the Orders table and the Employees table and storing the resulting rows in a virtual table that we'll call VT1. Then the ON statement is executed and only the rows that match the condition in the query are kept in another virtual table: VT2. The WHERE statement is then examined and SQL Server gets all the rows in VT2 where Country = 'USA' and stores them in VT3. VT4 is created by grouping together the rows based on employeeID, meaning that we are keeping only one row per employeeID and that we have to somehow aggregate the rest of the data from now on.

The HAVING clause indicates that we only want to keep the rows where the count of orderIDs per employeeID (aggregating the data) is greater than 100. The result is stored in VT5.

Then, we select the employeeID and the count of orders (remember, the rest of the data after executing the GROUP BY statement needs to be aggregated somehow) and assign, optionally, column aliases, which you can now use in the query, into VT6. Finally, VT7 is created when we use the ORDER BY clause.

Interstingly enough, VT7 is not returned. Instead, a cursor is what the query gives us back because of the ORDER BY clause. Without it, a virtual table would indeed by returned. This is why you cannot use ORDER BY inside subqueries, which are expected to return a table. After an ORDER BY statement, only functions that take cursors as input, like TOP(n), can be used.

So let's go back over the order in which SQL Server executes the query (I'll include every possible SQL statement this time):

1- FROM .. JOIN
2- ON
3- WHERE
4- GROUP BY
5- WITH
6- HAVING
7- SELECT
8- DISTINCT
9- ORDER BY
10- TOP

Any alias that you define in a query, be it a column alias, a table alias or a subquery alias can only be used in subsequent steps in the list or SQL Server will not recognize them.

On a side note, it is important to know about the UNKNOWN value in SQL. Usually, TRUE or FALSE is return when comparing values. But since some values can be NULL, comparing such a value with another one results in UNKNOWN. When a filter is applied to a query (WHERE, HAVING, ON), unknown values are considered like FALSE. But inside a check constraint, like making sure the salary in a column is greater than zero, unknown values are considered like TRUE.

Hey, I like SQL!

2008-12-04T11:58:00.001-08:00

SQL is one of my weak spots. I've known that for a while, since college actually. I never thought I'd need it, you know? Ha, add that to the list of things that seemed superfluous to me at the time! SQL? That's what DBAs are for, no? Math? That's what mathematicians are for, no? Parallel programming? I'm never going to work on more than one CPU at a time! Statistics? I'll just look up formulas online. Ah, being young...

Well my use of SQL has increased linearly since I got out of college and I always managed to learn just enough to get by. However, it has caught up with me. I work in a manufacture that creates pumps and valves for the oil fields. Tractability is important and we keep track of everything that is done to our parts during the manufacturing process. Add to that the HR system and our accounting software and we have massive amounts of data in the database. That data is there, just waiting to be used.

Not being great with databases in general is a flaw since I do not know how to use SQL as a tool. Here are a ton of nails, but I don't have a hammer, just a C# screwdriver (I do VB.Net, so I what I really have is a plastic screwdriver!)

Software developers and DBAs have been at war since they ever met. DBAs complain that software developers are nosy and don't know how to use a database properly and exist only to circumvent the systems they have put in place to safeguard against both data problems and intruders. Programmers, on the other hand, complain that DBAs take too long to take care of their requests and are just creating job security for themselves.

On top of that, software developers in general just don't like databases. It seems alien to us. We can't debug it, it gathers data all at the same time and we can't ever be sure that we even got the right data to begin with! Why not just grab everything, loop over it and grab what we need? After all, our data is never complex.

So we attempted to build object databases, which are far from being mainstream at the moment. We like them because we know Objects. But SQL just isn't going away. So we created ORM frameworks to help get rid of the repetitive CRUD operations in our models so we don't have to ever deal with SQL. Then, we declared a holy war on stored procedures, the only thing that can still possibly resist ORMs.

Stored procedures are not faster than code! They're just there so that the DBAs have something to do. There is nothing a stored procedure can do that code can't do!

To all of this I say: Bullshit! Programmers react to SQL the same way VB ASP.Net programmers react to functional languages. They don't want to touch them with a 10 foot pole simply because they don't understand them. All their arguments are a facade to hide the fact that they don't want to learn the new languages.

You don't think the same way when you program in Javascript or Lisp or Ruby versus VB.Net. In the same manner, using SQL forces you to think in SQL, an entirely different beast altogether.

I've really come around recently concerning SQL. I used to be like what I described above, but the more comfortable I became with the language, the more I started understanding it. I realized it is a tool in it's own right, used to solve a wide range of problems and that trying to resolve everything with my favorite language was not always the right thing to do.

If you deal with data and databases, learn SQL. Learn your database management system. There is really nothing else out there that can work this efficiently with relational data. Sure, use ORMs, since they are a tool that helps you keep a clean model. But do not be afraid to use SQL. Do not let the ORM do all the SQL thinking for you the same way you wouldn't let the politicians in Washington tell you what to think! ORMs can't do everything, after all. Sometimes, you just need the data without an associated object.

Lean Software Development

2008-11-23T10:07:00.001-08:00

A book review

I've just finished reading Lean Software Development, a book written by Mary and Tom Poppendieck that introduced the lean practices used in the manufacturing world to software development as an evolution of Agile methodologies.

Lean software development in general is still a new process, but the theories are solidly anchored by the manufacturers that have made it work for them. Toyota has introduced the kanban system and the "decide as late as possible" technique. 3M and the way they empower their teams. The Theory of Constraints which is used, incidentally, at the company I work for.

What Mary and Tom did was to translate every good practice to the software world. But because of the novelty of the approach, there are very few examples in the book of the lean techniques as applied to software development. Some time has passed, however, since the book has been written and today the reader can find a plethora of examples dispersed on the Internet, via developer blogs everywhere. Alt.Net in particular is a group whose members adhere to agile and lean practices and would be a good place to start searching.

I consider this book an excellent introduction to lean software development and highly recommend it to anyone looking into this methodology wanting to know more about it, or wanting to implement it in their workplace. The language of the book provides an easy read and most real life examples are taken from Mary and Tom's personal experiences at the various companies they have worked for and are always very insightful.

What is lean?

Lean development is the elimination of waste.

That's it... Really! You can stop reading now.

Seriously, however, lean development is about using tools and techniques in accordance to seven basic principles in order to eliminate waste in the value creation process.

The value creation process is simply how your company (or department) creates value. In our case, how we develop software is the process we are talking about here. Software is the value we create for our customers.

One way we can visualize this process is by using a value-stream map, which depicts all the steps taken when designing, developing and deploying software, along with the time taken to go through each step and the time that has been lost just waiting for something (feedback, testers, DBAs, etc.)

Then we apply lean methodologies to try to shorten the value-stream map as much as possible without sacrificing on quality. All and all, it means having shorter software creation cycles.

Let's talk a bit about each of the seven lean principles, as discussed in the book.

Eliminate waste

Well I've rambled a little about this already. The main thing is to learn to see waste, however. One way to do this is to search for anything that isn't coding and analysis and see if it adds value to the product. If it doesn't, then it is waste.

The following is considered waste:

Partially done work
Extra processes
Extra features
Task switching
Waiting
Motion (how far do you need to walk to for an answer to a question, for example?)
Defects

Another way is to map a value stream and try to reduce as much as possible the time spend on steps that don't provide a lot of value.

Amplify learning

Knowledge is key: knowledge about what the customer wants, knowledge about the system, knowledge about the constraints. We need to generate that kind of knowledge as fast as possible in order to make the best decisions we can.

Having short release iterations will generate customer feedback faster. Also, constraints might become more obvious sooner. Your code needs to speak volumes about intent and design without the need to spend time trying to decode it (a major waste!)

Try to do some set-based development also to generate as many ideas at the same time as you can.

Decide as late as possible

This is one of the things that Toyota does that makes them stand apart from the competition (and why they still make a profit and don't need their government to bail them out!)

Delay the critical, hard to undo, decisions until the last possible moment when delaying any further would reduce the value of the product or would eliminate important alternatives.

Decouple your modules, anticipate, code multiple options knowing that one might be chosen and the others discarded later. The price of losing the options in this case would be lower than not doing them at all. Use parameters, interfaces, abstractions. Encapsulate variation and do not repeat yourself.

Deliver as fast as possible

Use a Kanban system and code in short iterations. Enough said!

The key element in this principle is throughput: the rate at which value is created. The book mentions the theory of constraints, which deals with this in a manufacturing context where the entire shop cannot go faster than the slowest machine. The same can be said about software, which can't be written faster than the slowest process in the value-stream map. Everything must then be subordinated to the constraint since any extra effort by a non-constraint is waste. I suggest you read The Goal if you haven't already. It is an excellent book.

Empower the team

Teams are more efficient when managers give them free reins over their processes. Most successful companies have seen change happen within their walls from shop employees, not managers.

At 3M, for example, new inventions come from such employees, who are then expected to lead the effort to manufacture them!

Allow leaders within teams to emerge and influence others. Allow master developers to take charge. And make sure your team members keep improving on their skills. Set them free!

Build integrity in

Integrity is how the software behaves. Is is consistent? Are all the parts acting in synergy with one another? How is the user experience? This principle helps you answer those questions.

Use refactoring and testing to maintain a solid architecture and make sure you spend time designing a good user experience. The happier the user is, the less effort you'll spend down the road.

See the whole

Finally, focus on the whole. Do not spend time doing local optimizations like making the testing department faster when it is not the constraint of the system. Be weary of local measurements like lines of code per coder because you'll be tempted to improve on them (local optimization) in isolation of the big picture.

Instead, focus on you value-stream map! Work on reducing the amount of time it takes for an equal amount of value to be created. When you change a process, go back to this map and make sure you didn't make it worse, even though a local area might have improved.

Has throughput increased? Have operating expenses been reduced? Has the amount of work in process decreased? All three must be true.

Conclusion

Of course, there is much, much more being discussed in the book. But what I have written in this post should get anyone started with lean or should be a good reminder of the key points of the book if you have already read it. I know I will refer to this often in the future in my own quest to implement lean techniques with the rest of my team!

Activity Modeling and the Bag

2008-11-19T10:10:00.001-08:00

This is the third and final part in this series. You can read the first two posts here:

On our last day at Kaizenconf, I participated in an action type of session where our aim was to figure out what we could do about Activity Modeling, a design method. After reviewing it extensively, we decided that we would all try to fit it into a kanban pull system and blog about it.

Activity Modeling

In essence, Activity Modeling is a technique that allows a designer to create a set of actions and operations that different users, or personas, can do using a certain software application. It involves creating files on different personas to keep track of their goals, motivations and values using, among other methods, non-directed interviews where the main question is never asked directly.

Then, using a very specific activity statement, actions are defined. From the example that was given to us at Kaizenconf, the activity statement could be: "Providing patients acute pain management services." Now an action could be: "Admit patients into the acute pain management system." Finally, operations are determined for each action: "Specify patient demographics," for example.

Our group decided that our action plan would be to incorporate Activity Modeling to Jef Newsom's Kanban technique, which I depicted in the first post of this series.

In retrospect, this seems easy enough, at least on paper! The first thing to do would be to make sure we have enough information about our main users, or our personas. This is a very important step, as it has a direct link to the "People" column in our Kanban board.

Filling up the BAG

So we have our Goal, and our personas, which we can translate to our Kanban board. Using what we know about them, we can write down their personal goals. A doctor wants to be able to edit information and wants to know what is going on at all times. A nurse might just need the history of the patient, or just what the current health care procedures that need to be performed are. Activity modeling helps us know all of this.

Our bag used to be full at this point, but let's go ahead and add a new column at the end: the Actions column. This is a further refinement on each personal goal, normally a step we were reserving for the S.A.F.E. column in the Kanban. However, since we are still in the bag, actions remain at fairly a high level of abstraction, which is good since we might have the user helping us defining them.

S.A.F.E. scenarios benefit from Activity Modeling

Now that we have refined the bag, the designer and developers can work together to fill up the S.A.F.E. column at the beginning of an iteration. In Activity Modeling, operations are defined for each action and it just seems to me that this is exactly what we are doing by creating our S.A.F.E. scenarios. Our success, advance, fail and error scenarios are operations that have been classified!

This is what really struck me in our session. Activity Modeling benefits from the S.A.F.E. scenarios idea by having its operations classified and our Kanban benefits from Activity Modeling for filling up the Bag. There is an almost perfect synergy between the two methods and it has enormous usefulness potential for developers and designers alike.

From there, developers can simply pull operations into their iteration and can keep going with the rest of the Kanban.

The Bag and the Kanban, part 2

2008-11-11T12:06:00.001-08:00

You can view part 3 of this series here.

In my last post, I depicted the results of a workshop I attended at Kaizenconf called Driving Towards the Goal. Jef Newsom, our facilitator, had set up a little experiment which we all voluntarily and happily participated in.

As a whole, the experiment simulated the creation of software from Goal to Code, each step being a column in a kanban board.

Even though it was a very enlightening experience, we quickly discovered that the further along you went, the longer it took to specify work items. For one, each work item became much more specialized the closer it got to being coded. Second, as they became more detailed, we needed a lot more work items to cover the previous ones.

In terms of workflow, you can think of it like a sandglass. The thinnest part, where only a few grains of sand can flow through at a time, is the coding part. The sides of the sandglass are time and resources constraining your team and the distances between the sides is how fast the task takes to perform.

Of course, specifying the goal can take years of efforts, but in terms of software development, since it is usually defined in advanced (I hope!), it is the fastest step.

The Bag

So how does one manage the flow in such a system? Well the idea came to me that the kanban in the previous post could be separated in two. The first part focuses more on the creative aspect of defining the problem, the solution, the people involved and their goals. Here, chaos can rule. People can jump in anywhere and add items to the board, prioritizing and moving stuff around. This is what I call the BAG, which happens to stand for BAck loG. It also makes me think of a bagpipe, where the player (the stakeholder) fills up a bag with air, which is then pressed into the flutes to produce a sound.

The Kanban

The second part of the kanban is where the more technical stuff happens. Here, work items actually have to be pulled from previous columns before any work can happen. This discipline and organization is important in order to maintain a good flow. Remember, since coding is the constraint, it must be protected at all costs. Any disruption in this part (called "kanban" from now on) will cause a drop in throughput.

The reason that the S.A.F.E scenarios are outside the cloud in the picture above is that I believe the entire team should be working on them at the beginning of every new iteration. This will give every member of the team a general understanding of what is going on in the project.

Playing the bagpipe

To play the bagpipe, you start by filling up your bag with air. Then, using your arm, you compress the bag so that it pushes just enough air through the pipes to make the sound you want at the volume you want.

Filling up our bag is fairly easy. You get your stakeholders together and use your favorite method for liberating the creative juices of every people in the room. When things have stabilized a bit, the developers can come in and pull a couple "features" from the personal goal column in the bag in order to create their first iteration.

It is then absolutely important that the stakeholders leave what the developers are currently working on alone during an iteration. They can come back to the bag and play with work items, but what the software team is working on is sacred, at least until the end of the iteration.

At the end of every iteration, when the stakeholders have a chance to try out their newest release. Then they will have a chance to introduce changes in the personal goals. Each change can then be pulled from the bag by the developers, perhaps with a higher priority than other regular features.

As this process advances in time, stakeholders will probably find themselves asking for change during iterations. This is fine, as long as the change doesn't affect what the developers are currently working on. Changes then go in the bag and await the next pull.

Now, I've never played the bagpipes but I'm pretty confident that that what I've described further up is approximately how they work. In the same way, I've never implemented this process, but is sure looks nice on paper! I've also been called lots of names, but as a programmer, being compared to compressed air flowing through a pipe is a first, I must admit.

This process has a lot of potential and I'd like to keep defining it. Unfortunately, my current work conditions prevent me from experimenting on it in the short term, but rest assured that as soon as I can claim ownership of a project, I will definitely use the "Bag and the Kanban" model and see how far is takes me. In the meantime, I must realize that it is just an idea.

In my next post, I will use what I've learned in a session given at Kaizenconf on activity modeling to go into more details about how to fill the bag.