Los Techies

Take 3: Python, ISP, IoC, and OCP need a fundamental rethink.

Ryan Svihla — Fri, 20 Nov 2009 19:23:00 GMT

In response to Julian’s thoughtful Dynamic Languages and SOLID Principles I’d have to argue he is about 95% there but is missing the last critical links needed to view this in a whole different light. For ISP, Julian says it all:

Ultimately, I don't think ISP is changed by Python, it's just kind of irrelevant, for better or worse.

Exactly! Why even bother mentioning in the context of dynamic languages. A developer that understands it fully and one that does not will code the same way. Whereas LSP is still taught in dynamic languages, I’ve never seen it called that but I’ve seen many admonishments about subclass consistency, and not making client code aware of the differences, and is something that requires mentioning. With ISP we could ignore it’s entire existence and developers would code the same.

For OCP:

Well, the open closed principle is a goal, not a design practice, but let's take a look at the danger points:

    * You can't have non-virtual methods, so Python wins this hands down.
    * Your variable can't be made too specific, so you're safe there.
    * You can still compare against hard-coded values. It's just as easy to get this wrong in Python as it is in C#.
    * Same holds true for Law of Demeter violations. If you pass the wrong object around, your code will be just as fragile in Python as in C#.

Python certainly reduces the scope for some OC violations, but you've still got lots of rope to hang yourself. Think you still need to bear the goal in mind.

Bolding emphasis is mine. Enough of OCP is stripped and the primary ways its taught and demonstrated do not apply to the language at all. Focus on Law of Demeter, but having something that stands for “Open for extension” whenever everything is open and “Closed for modification” when nothing is closed would fall on completely deaf ears when presenting any of the associated ideas to a Python developer.

For Dependency Inversion/ IoC:

Let's quote Ryan:

Now all calls in this runtime, from any module, that reference the Output class will use XmlOutput or HtmlOutput instead.

Yes, but what if I wanted only half of them? Maybe there's Python techniques I don't know about (I'm barely competent in the language) but as I see it, I'm going to need to change the code. I don't think that dependencies can "always" be injected. They can only be done when it won't cause damage. In his case, he's worrying about testability. That's fine, but we all agree there's more to DI than testability.

Ok firstly, I didn’t just mention testability, in fact that the code you’re referencing there was no unit test for, it was about flexibility. Secondly, call Output an “Abstract Class”,and then call XmlOutput and HtmlOutput concrete types and then add them to an IoC container like Windsor then let me ask yourself the question you asked me “what if i wanted only half of them”. There would be a need for custom code to make this resolution work for this custom circumstance. All I’m advocating is taking that same exact custom code and place it somewhere else in Python (yes you can override per method, per instance, etc, etc).

I think there needs to be a principle that addresses the benefits of explicit interfaces and well formed custom composition cases…and in those cases DI makes a lot more sense, but certainly not as a defacto convention and certainly is not required to provide flexibility to existing implementations. It certainly becomes apparent when you start looking into the actual way IoC works in static languages its all too similar to how a dynamic language runtime works. That code you’d use in a static language IoC to apply AOP, or custom composition rules, could just as easily be applied in a component in Python, because the primary mechanism for an IoC to work is as a factory, and the benefits and features they bring are all derived from having access to how objects are created and then returned, when you always have access to how objects are created and returned anyway as in dynamic languages, why should you implement it a second time. 100% DI’d code will be more effort to maintain because it would be in C# too…if not for that fact the language is so “structured” that the flexibility and testability benefits gained far outweigh the slightly increased cost (which is only brought down with the awesomeness of auto registering components). In a language already with flexibility and testability whether you use DI or not AND without auto registering IoC containers the cost to maintain a full DI code base is I’d argue unacceptable.

Summary, for SOLID to make any sense whatsoever in a dynamic context it has to amended and modified to fit that world. Maybe make SOLID into SELL for:

S.RP

E.xplicit empty class for required overrides

L.SP

L.aw of Demeter

Maybe there is something containing other more valid concepts for the dynamic world that I’m probably not qualified to design or come up with. Anyway, I’ve enjoyed the conversation and I hope it brings about a firmer understanding of SOLID, and whatever we’ve not come up with yet for dynamic languages.

Why Chrome OS Will Succeed (in a limited market)

Eric Anderson — Fri, 20 Nov 2009 13:22:00 GMT

Randall Kennedy released an excellent analysis of the Chrome OS that I invite you to read. In that analysis, Mr. Kennedy outlines why he believes that the Chome OS is doomed to fail miserably. Though I haven't seen Chrome OS myself, I believe his read on the operating system is solid. This is a stripped down operating system that basically locks the user into web applications with the Chrome browser as the interface. The operating system is at its core a paired down version of linux and does not seem to support running desktop applications. Because this is a this OS that only supports the web browser, Mr. Kennedy's analysis is that no user will want this running on their computer. This is where Kennedy misses the point.

From reports including Mr. Kennedy's article, Google will be releasing Chrome OS on a limited set of hardware. Instead of having to deal with device drivers for a wide array of display adapters and whatnot, it appears that Google will be shipping Chrome OS on a constrained set of netbooks. Kennedy looks at this limited set of hardware combined with the limited features of Chrome OS and determines that with all these restrictions, nobody will use the operating system. This is where Kennedy has missed the point.

Personally, I would be happy to have a netbook running Chrome OS on it. With access to Google apps, I will still have simple spreadsheets and word processing inside of the full functional web browser. I would use a Chrome OS netbook like, well, a netbook. I would surf the web and check email. My wife and I would keep it in the kitchen for looking up recipes on our favorite recipe sites. My wife would use it to waste time on Facebook, er... "see what her friends are up to." I would have to find a web Twitter client that I like since I'm guessing I wouldn't be able to run TweetDeck. But, that's not a major loss.

I don't want to run VisualStudio on my netbook. I probably wouldn't care if I couldn't run Word. As long as the user is looking for a simple web appliance that is better than a mobile phone while being lighter and less expensive than an entry laptop, then I think a Chrome OS netbook could be a suitable device. Yes, this is a narrow market. No, Google won't be making a big dent in Windows (or Mac for that matter) with this device. However, this could get them a decent footing in a tough market to make headway into.

I, for one, welcome our new robot overlords

A better Model Binder addendum

bogardj — Fri, 20 Nov 2009 02:31:19 GMT

A while back, I wrote about a ModelBinder enhancement we use to do arbitrary filtering on types. The underlying matching algorithm only matches on one type, but we like to use layer supertypes for a lot of our domain objects, so we want to use a single model binder for every Entity or Enumeration type in our system.

However, I missed an important piece in our model binding story: ModelState. In the original implementation:

public override object BindModel(
    ControllerContext controllerContext, 
    ModelBindingContext bindingContext)
{
    foreach (var filteredModelBinder in _filteredModelBinders)
    {
        if (filteredModelBinder.IsMatch(bindingContext.ModelType))
        {
            return filteredModelBinder.BindModel(controllerContext, bindingContext);
        }
    }

    return base.BindModel(controllerContext, bindingContext);
}

I didn’t address ModelState. It turns out that this may lead to problems later on when I use the built-in HtmlHelper methods. These helper methods use ModelState to do things like set error message indicators, fill in values and so on. But if one of the IFilteredModelBinder implementations binds a value but doesn’t do the ModelState piece, I start to get some weird behavior in certain scenarios.

If you start to dig pretty deeply in the DefaultModelBinder code, you’ll find that the base implementation of BindModel will set the ModelState values for certain scenarios. In our case, we need to allow the IFilteredModelBinders to do something similar.

Our slight fix

First, we’ll need to fix the IFilteredModelBinder interface to return not just an object, but something that includes the ValueProviderResult object (used by ModelState). Why not just do object? Well, we can’t make an assumption on what the underlying model binder implementations need to do with a BindResult, it’s rather custom for each implementation. So, our IFilteredModelBinder interface becomes:

public interface IFilteredModelBinder
{
    bool IsMatch(ModelBindingContext bindingContext);
    BindResult BindModel(
        ControllerContext controllerContext, 
        ModelBindingContext bindingContext);
}

That BindResult object is not in MVC, it’s something we have to define:

public class BindResult
{
    public object Value { get; private set; }
    public ValueProviderResult ValueProviderResult { get; private set; }

    public BindResult(object value, ValueProviderResult valueProviderResult)
    {
        Value = value;
        ValueProviderResult = valueProviderResult ?? 
            new ValueProviderResult(null, string.Empty, CultureInfo.CurrentCulture);
    }
}

All object adds is the ValueProviderResult, providing a default null-object implementation if the implementing filtered model binder doesn’t support ValueProviderResults. With this in place, we need to fix our SmartBinder:

public class SmartBinder : DefaultModelBinder
{
    private readonly IFilteredModelBinder[] _filteredModelBinders;

    public SmartBinder(IFilteredModelBinder[] filteredModelBinders)
    {
        _filteredModelBinders = filteredModelBinders;
    }

    public override object BindModel(
        ControllerContext controllerContext, 
        ModelBindingContext bindingContext)
    {
        foreach (var filteredModelBinder in _filteredModelBinders)
        {
            if (filteredModelBinder.IsMatch(bindingContext))
            {
                var result = filteredModelBinder.BindModel(controllerContext, bindingContext);

                bindingContext.ModelState.SetModelValue(
                    bindingContext.ModelName, 
                    result.ValueProviderResult);

                return result.Value;
            }
        }

        return base.BindModel(controllerContext, bindingContext);
    }
}

It’s pretty much like the original, except that piece that does the SetModelValue part. That fills in the necessary pieces for the input HtmlHelper methods to do their thing. Interestingly, this made another problem go away, around ModelState.AddModelError. It turns out I just missed a piece in the custom model binding code.

So just a note to custom model binder implementors – don’t forget to populate the appropriate ModelState values, especially if the results of the binding operation will be used in conjunction with the HtmlHelper extensions.

Take 2: Why we use SOLID in static languages and how we get the same functionality for cheap in dynamic languages

Ryan Svihla — Thu, 19 Nov 2009 22:46:17 GMT

One of the things we do pretty well at Los Techies is explaining SOLID principles and why they make our code more maintainable and if you’re not familiar with our work on SOLID, read Chad Meyer's post to get an understanding on the Los Techies perspective. Now that you’ve taken a bit of time to read that the following chart takes SOLID’s benefits and then shows how they apply to dynamic languages:

Principle	Primary Benefit	Secondary Benefit	What dynamic langs do to replace said principle
Single Responsibility Principle (SRP)	Classes are in understandable units	Testability	Nothing
Open-Closed Principle (OCP)	Flexibility	Guidance	can override anything globally (monkey patch)
Liskov Substitution Principle (LSP)	Not bleeding subclass specific knowledge everywhere	Least surprise	Nothing
Interface Segregation Principle (ISP)	Interfaces are in understandable units	Less work for interface implementation	everything’s an interface (duck typing)
Dependency Inversion Principle(DIP)	Flexibility	Testability	can override anything globally (monkey patch)

The above is my opinion and I’m sure people will beat me to death because I’m just some dude not many people know about, but I think I’ve got a grasp of what SOLID means in C# and I’ve grappled with it from a practical perspective.

You’ll note when it comes to LSP and SRP dynamic languages really give you nothing at all, but OCP, ISP, and DI benefits wise are supplanted by language features in python, ruby, etc. How does this happen? In the case of ISP and “duck typing” this should be pretty obvious, but in the cases of OCP, DIP

DIP/OCP testability achieved through C#:

public class FooBar{

    private IFooService _service;

    public FooBar(IFooService service){

          _service = service;

    }

    public void DoFoo(){

        _service.run();

    }

}

//testcode

[TestFixture]

public class FooBarSpec{

  [Test]

  public void should_call_fooservice(){

     var mockfoo = new Mock<IFooService>();  

     var foobar = new FooBar(mockfoo.object);

     foobar.DoFoo();

     mockfoo.Verify(foo=>foo.run())

   }

}

DIP/OCP equivalent testability achieved through Python:

class FooBar(object):

    def DoFoo(self):

        service = FooService()

        service.run()

#test code

import unittest

class TestFooBar(unittest.TestCase):

    def runmock(self):

        self.fooservice_run_was_called = True

    def test_do_fooservice_is_called(self):

        originalrun = FooService.run #storing the original method in a variable 

        FooService.run = self.runmock  # I'm globally overriding the method call FooService.run AKA Monkey Patch

        FooBar().DoFoo()

        FooService.run = originalrun #globally put it back to where it was AKA Monkey Patch

        assert self.fooservice_run_was_called

Now rightfully you may say the Python version has a few more lines of code, and it certainly can be considered ugly…but don’t you see that I replaced an entire mocking library in a few lines of code. There are also mocking libraries and monkey patch libraries for Python that would shorten my line of code count further. Testability and flexibility dovetail, but the global accessibility of class definitions is understandably hard to grasp for the static mindset (it took me months), so we’ll continue on to flexibility:

public interface IOutput{

  void save(string fileloc, ResultObjects[] results);

}

public class XmlOutput:IOutput {

   public void save(string fileloc, ResultObjects[] results){

    // place xml specific code here

}

public class HtmlOutput: IOutput {

   public void save(string fileloc, ResultObjects[] results){

    // place html specific code here

}

public class TestRunner {

  private IOutput _output;

  private ITests _tests;

  public StoreTestResults(IOutput output, ITests tests){

    _output = output;

    _tests = tests;	

  }

  public void RunTestsFromDir(string testdirectory, string outfile){

    var results = _tests.GetResults(testdirectory);

    _output.save(outputfile, results)

  }

//later on we use IoC containers to make this chain easy 

pubilc static void main(string[] args){

  // imagine contextual resolution is happening intelligently 

 // on the resolver because someone did some cool stuff in windsor

  var runner = testcontainer.Resolve<TestRunner>();

  runner.RunTestsFromDir(args[0], args[1]);

}

In Python with “monkey patching” everything is “Open For Extension” and dependencies can “always” be injected. A code sample based on my last blog post but extended to include some more complete client code follows:

#in module outputlib

class Output(object):

  def save(fileloc, results):

    pass

class XmlOutput(object):

  def save(fileloc,results):

     """xml specific logic here"""

     pass

class HtmlOutput(object):

  def save(fileloc,results):

     """xml specific logic here"""

     pass

class TestRunner(object):

  def run_tests_from_dir(testdirectory, outfile):

    tests = Tests()

    output = Output()

    output.save(  outfile  , tests.get_results(testdirectory)

#rest of the script

import sys

import outputlib as o

testdirectory = sys.argv[1]

outfile = sys.argv[2]

if outfile.endswith("xml"):

  o.Output = o.XmlOutput #globally override calls to Output with XmlOutput. AKA Monkey Patch

elif outfile.endswith("html"):

  o.Output = o.HtmlOutput #globally override calls to Output with HtmlOutput AKA Monkey Patch

runner = o.TestRunner()

runner.run_tests_from_dir(testdirectory, outfile)

Again this is longer code than the c# example.. but we’ve replaced an entire IoC container and whatever custom code we would have had to add to create custom dependency resolver for selecting HTML or XML output options has now been replaced by a simple if else if conditional at the beginning of the code base execution. Now all calls in this runtime, from any module, that reference the Output class will use XmlOutput or HtmlOutput instead, all with “monkey patching”.

Hopefully, from here the rest of the dominos will fall into place once you get the concepts above, and the things we rely on SOLID for are in some cases already there in dynamic languages. Consequentially, my dynamic code looks (now) nothing like my static code because I no longer have to use DI/IoC to achieve my desired flexibility and access to use proper unit tests.

Pablo’s Fiesta Open Space Conference

chadmyers — Wed, 18 Nov 2009 15:51:22 GMT

In case you didn’t see it on the main Los Techies feed, John Teague and fellow Los Techian announced “Pablo’s Fiesta”, Los Techies’ first Open Space conference.

The sign-up page is here for more information: http://fiesta.lostechies.com/

John Teague has taken the lead on this thing and has done an amazing amount of work on this in the past few days (not to mention several other Los Techies who have dedicated many hours of their own time to make this happen).

We were hoping to make it free, but there’s some cost involved in the venue and probably a bunch of other stuff. At $15, I’m hoping you won’t think it’s too expensive.

You can read more details on the site, but the gist is that it’s an Open Spaces conference, so YOU determine what the subject matter will be. Personally, I’m hoping it’ll be focused on hard-core nuts and bolts stuff with real code and patterns and practices around code.

If you’re reading this blog, you’re the right person to come, so if you can make it to Austin in February, please try to come!

How We Do Things - The Team Room

Scott C Reynolds — Tue, 17 Nov 2009 14:30:16 GMT

This content comes solely from my experience, study, and a lot of trial and error (mostly error). I make no claims stating that which works for me will work for you. As with all things, your mileage may vary, and you will need to apply all knowledge through the filter of your context in order to strain out the good parts for you. Also, feel free to call BS on anything I say. I write this as much for me to learn as for you.

This is part 8 of the How We Do Things series.

Everyone should basically know what a team room is by now, so this post is going to just be a look into how we set ours up, how we got the team to buy in to the change from private cubicles, what we've noticed since creating the team room, and my thoughts on privacy and productivity.

When we first started growing our team, we were scattered about. A few of us had private offices, the rest were in cubicles with high walls. As we increased staff, I began to look at some alternatives to this situation. The space allocated to me was a rectangular room about 54'X18' with an adjacent office that was about 15' square. Not a lot to work with in a traditional cube setup, which is what I was asked to pursue. So I took this as an opportunity to build the team room I had been wanting.

What we came up with is "pods" of 3 desks each, with about 1' mesh dividers above the desk surface. This gave everyone their own desk, and their own little area, but they were still together, and everyone could see and freely talk to everyone else.

Steve Harman once asked on twitter if private desks were anti-productive, and I answered that I didn't think so when used appropriately. I know a lot of team rooms just do the flat table thing and everyone sits shoulder to shoulder at all times. That's fine, I guess, and great for pairing, but people do need a place of their own. Someplace they can check email and their bank account without everyone looking over their shoulder. Someplace where they can put their knick-knacks. People like desk drawers to store things in. It's psychology 101. People need a little space. At least the people on my team do.

One of the problems with some of the things we do in agile-type software is that we ignore the human factor, we ignore team dynamics and personalities, and try to make everyone the same. This is probably at least partly why a lot of teams push back on these practices, and why they end up falling down.

So we compromised, and people got private desks that were put together in such a way as to not hinder the "team" aspect of the team room. And I think it's worked well.

We outfitted the team room with plenty of whiteboard action:

It's important to have a lot of areas where people can collaborate and try out ideas. In addition to the team room we also have a "conference room" for just us that is separate, has a whiteboard and a nice projector, where people can gather and do design and planning sessions.

Since the desks aren't really conducive to long-term pair programming, we set up a pairing station at the end of the team room, and we turned the extra office into what we call "pairing jail". Pairing jail has two pairing stations and another whiteboard, plus all the equipment we need to test our hardware integrations (microscope, camera, barcode scanners, etc).

Each pairing station has a 24" monitor, extra keyboards and mice, and is a nice flat surface people can work on.

This is the compromise between pair programming and private desks. We pair most features, and most of the developers are pairing about 5-6 hours a day. However, we have an uneven number of developers, and sometimes people are working on something on their own, or spiking something, or just doing whatever else, and having a semi-private space keeps morale up.

We have 7 developers so 3 pairing stations works. When we add more staff, we add more pairing stations. Pretty simple.

We also work on a few different projects simultaneously, which makes the separated pairing room nice. That room usually becomes the war room for the two pairs working on the biggest thing, so they can have all their conversations and whiteboard sessions without interfering with the conversations of the third pair who is working on something totally different. It works for us.

People seem to organize themselves fairly well into various areas, they respect the whiteboards when there's real work on them (people outside the team don't always respect the whiteboard, which can get annoying, but that's another story)

At first there was resistance to the team room concept, and I'm sure a lot of people come up against this. To overcome the objections I did the simplest thing that could possibly work: I gave up my office and sat in the team room. When the boss gives up his office, people are less likely to complain about sharing space. Then they get used to it. Then, over time, they see the value, and the organic process of gelling a team happens.

There was also resistance externally to the idea at first (and still some today, a couple of years later). Other managers at my level didn't understand why I left my office for a desk in this room, and I took a lot of not-so-subtle teasing for it. People didn't understand the collaborative nature of a team room, because corporate America is used to peons in cubes, managers in offices, and communication via email and scheduled meetings. This is just something people have to get over or not. I stopped caring. The results spoke for themselves. Don't let the sheep drag you down.

The team room isn't just for developers. I sat in there until very recently (gave up my space for expansion since I'm moving soon anyway). The business analysts are in the team room. The QA guys are in the team room. Everyone is together, and everyone talks, and everyone is accessible.

Not everyone has the perfect big room for a team room, I didn't. You have to find a way to work with the space you have, be creative. The most important thing is getting the team together, in the same place, and removing the barriers.

A team isn't built overnight. It takes time. When people aren't used to a team environment, a collaborative environment, you can't just throw them together at desks and expect magic. It's a process. I will say that since we put the team room in place, after the first month or so, it's been amazing. People communicate. They talk. They whiteboard. They get together at each others' desks for impromptu design sessions. They pair on things they weren't even planning to pair on. They collaborate. Without the team room in place we would not be where we are today with either our process or our practices.

Technorati Tags: how we do it, improvement, lean, management, quality, software, team, planning

I recant my IoC! IoC containers in dynamic languages are silly.

Ryan Svihla — Mon, 16 Nov 2009 22:28:33 GMT

After a year or so of solid Alt Dot Net infection (as far as infections go its a pretty awesome one to have), I decided to give Python a try again for more than one off sysadmin tasks, and to actually dive into it as a newly minted “Agilista”.

However, I had a problem..there were no non-painful IoC containers in Python (sorry to the other authors of IoC frameworks in Python like Spring Python and Snake Guice, I know you try and I respect the effort). Ultimately, I could not imagine coding anymore without something to handle all my registration for me, that’d dynamically inject in my dependencies, give me hooks for contextual resolution, and give me rich interception support.

So I built my own, it was painful still, but I had some ideas to move it in an Convention Over Configuration direction, and ultimately get within shooting range of what fancy stuff we’ve come to expect in .Net that Windsor, StructureMap and many others provide.

Now as I got into the fancier aspects of dynamic languages, open classes and the ability to override all behavior easily I get it…a dynamic languages runtime is like a bit IoC container.

Now I’ve heard other people say this many times but rarely with explanation or example, or focus on frankly silly things like “Dependency Injection adds too many lines of code” which is a bit melodramatic. Two arguments to a constructor, two fields ,and then avoiding having to call new is not “adding too many lines of code”, especially with templates, IDE’s, scripting, etc to cut down the actual typing load.Today I’m going to actually try to explain how languages like Python, Ruby etc give us the same awesomeness we’ve come to expect in things like Windsor..but at a much cheaper cost of learning and dev time.

Take a typical resolution scenario where you want to output to a different file format depending on command line switches. With an IoC container you can either:

Change the resolution argument to load a different concrete type:

if(arg == "XML")
{
  container.Register(Component.For<IOutput>().ImpmentedBy<XmlOutput>());
}
else if(arg == "HTML")
{
  container.Register(Component.For<IOutput>().ImpmentedBy<HtmlOutput>());
}
else
{
  container.Register(Component.For<IOutput>().ImpmentedBy<NullOutput>());
}

or resolve different arguments or keys using a service locator approach in later client code (thereby depending on the container)

public void output(IKernel container, string key)
{
var output =container.Resolve<IOutput>(key);
output.save();

}

or implement a custom implementation of the resolver system (which I’ll leave out for the sake of brevity, but it’s not instant code). Also to do all this you have to depend on interfaces, add all your interchangeable code to the constructor and life is grand. You do this for many reasons in static languages, its the only way to get easy testability and code that is open to extension. In dynamic languages its always open for extension and easy to test . Let me demonstrate:

import outputlib as o

def outputselect(arg):
if arg == "XML":
o.Output = XmlOutput
elif arg == "HTML":
o.Output = HtmlOutput
else:
o.Output = NullOutput

def saveoutput():
o.Output().save() #will save whichever

Contextual resolution in a nutshell, and throughout your code if need be. “Interception” is even easier, take a look at http://docs.python.org/dev/library/functools.html and then start playing you’ll see you can trivially apply logging and security to methods without explicitly adding it. A short logging example follows:

import types
import functools

#applies a cepter to each non-underscored method.
def wrapcls(cls, cepter):
    publics  = [ name for name in dir(cls) if not name.startswith("_")]
    methods = [getattr(cls,method) for method in publics if type(getattr(cls,method)) == types.MethodType ]
    for method in methods:
        intercepted_method = cepter(method)
        setattr(cls, method.__name__, intercepted_method) #attaches intercepted_method to the original class, replacing non-intercepted one

#the magic all happens in the functools.wraps decorator
def loggingcepter(func):
    @functools.wraps(func)
    def logafter(*args, **kwargs): #for csharp devs view this as an inline delegate
        result = func(*args, **kwargs) #invoking function
        print "function name: " + func.__name__
        print "arguments were: "
        for a in args:
            print repr(a)
        print "keyword args were: "
        for kword in kwargs:
           print repr(kword) + " : " + repr(kwargs[kword])
        print "return value was: " + repr(result)
        return result
    return logafter

#default boring repository class
class StorageEngine(object):
    @property
    def connections(self):
        pass

    def _sessioncall(self):
        pass

    def create(self,user):
        print "running create now from the method"

    def delete(self, id):
        print "running delete now from the method"
        return "deleted from the database"

    def get(self, id):
        pass

    def __init__(self):
        pass

#placeholder storage object
class User(object):
    pass

wrapcls(StorageEngine ,loggingcepter)
repo = StorageEngine()
print "calling count this should not be intercepted"
cnncount = repo.connections
print "now get should be intercepted"
repo.get(1)
print "we should see keyword arguments here"
repo.delete(id=2)
print "session call should not be intercepted"
repo._sessioncall()
print "create should be intercepted and we should see a User object"
repo.create(User())

Running the above script should result in the following output

So this is all very cool, but what does it mean or why do I care? For those of us used to using a proper IoC container like Windsor or StructureMap, we’ve gotten used to have capabilities like the above easily available to us when we’ve needed them. It’s nice to find that in Python (or really any dynamic language ) we can easily build our own similar functionality that we’ve come to depend on. We’re never coupled and we’re always able to test and mock out behavior. It was a long time coming but I think I finally get it now.

Pablo’s Fiesta: Open Space Conference in Austin TX

jcteague — Mon, 16 Nov 2009 21:36:27 GMT

Pablo's Fiesta

Los Techies Open Space Conference on Software Quality & Craftsmanship

When: February 26, 27 & 28
Where: St. Edward's Professional Educational Center, Austin TX

I am proud to announce the first Los Techies Open Space Conference. We’ve been working on this for a while we are now finally ready to make the public announcement.

You can see all of the details and register here.

What I want the conference to be.

The great thing about Open Space conferences is that the content as well as the tenor of the conference is defined more by those attending than those who organize it. Here is what I think the conference should be.

The topic of the conference will be focused on Software Quality and Craftsmanship. This is a topic that is very near and dear to the hearts of the Los Techies crew. Quality is the most important feature of any product, and can be difficult to achieve and measure in software. Craftsmanship is a philosophy about how professionals in our field should approach their work and how, as a profession, we should focus on cultivating a community of software developers who learn, respect and give back to the profession.

There are several important aspects to being a craftsman that I to address:

Sharpening the Saw. It important to learn new techniques and different technologies to continue to grow in this profession. I’m the software developer equivalent of a gear head (or petro head you you UK guys). I have no doubt there will be some technical discussions.

Multi-Platform. I want to talk about more the C#. Let’s discuss iPhone, Ruby, Python and any other platform that help us become better software developers.

Selling Quality as a feature. How do we convince clients and business executives that quality is a feature of software that has a very big impact on their bottom line.

Come to the conference and influence help shape it to be to what you think it should be. Please join us in Austin TX, February 26,l 27 & 28.

The Passionate Programmer Book Review

Louis Salin — Sun, 15 Nov 2009 22:19:00 GMT

Earlier this year, I've been given the chance to read any book of my choice from the Pragmatic Programmer bookshelf and write a quick review. Being so focused right now on figuring out where I want to go with my career, I thought I'd give The Passionate Programmer a try. I mean, who doesn't want to be passionate about their job? Loving what you do makes it fun and this is the key to a very successful career! I don't think I can do it justice until I've tried to put into practice most of what the author is preaching in the book, so here is a quick and dirty review of the book. (spoiler: I liked the book!)

The author, Chad Fowler, didn't get his start by getting a degree in computer science. Yet, he has accomplished more in his career than most software developers will ever do in their lifespan. So he must be doing something right, no? Well, the book might not teach you how to be passionate about anything. However, it will give you great pointers to make a great career out of software programming, which might make you passionate about the whole thing, I guess.

The book is divided into five independent parts, each of which you need to give serious thought to if you're ever going to become great at what you do, and remain great. In each part are multiple, small chapters of about two to three pages long, that will give you some hopefully valuable insight about growing you skills, finding mentors, pleasing your managers, how not to panic in dire situations and being careful about market changes, among many, various others topics. All in all, there are 53 "tips" in the book and I must say that most of them are very useful.

I would've wished that not all of those tips be written as if it was a recipe for success. Every single one ends with a "Act on it" section with exercises that are bound to put some of us out of our comfort zone. But maybe that's a good thing! Any day, I can open up the book and re-read one of the tips that I know I should be working on. There is no doubt in my mind that I can become a remarkable programmer by working on my weaknesses, and Chad covers most of them in his book.

This isn't a book that you'll read cover to cover. The optimal way to get through it, in my opinion, is to read it one chapter or two at a time and quickly put into practice what was read before progressing any further. Incidentally, this makes the book a very easy read; you could be spending only 15 minutes every night reading this book. But to make it effective, you'll need to actively put it into practice. I know I will. And I know I'll probably end up re-reading it a couple of times.

I have read the 7 Habits of Highly Successful People and Who Moved My Cheese and Chad's book reminded me of them. Because of that, I can only recommend this book to all my peers, unless you are already at the top of your game. Then maybe you should write your own book too! Oh wait, that's one of the tips in The Passionate Programmer!

How We Do Things - Specification (Using the right tools)

Scott C Reynolds — Fri, 13 Nov 2009 14:41:52 GMT

This is part 7 of the How We Do Things series.

This post was co-written with Cat Schwamm, business analyst extraordinaire.

In the last post we talked about how we approach specification philosophically, as an iterative, JIT process. In this post we will take a look at the tools we use to create specifications.

User Stories and Text

User stories and other text documents are the bread and butter of defining work, and we use them like crazy. Our story practice has evolved over time, and we have come to a place where we feel that, when used appropriately, our stories are very effective. For a deeper look at the guidelines we use to construct stories, I recommend you go check out Cat’s post on the subject. Go ahead. I’ll wait.

Okay, welcome back. Here’s the thing, stories and tasks are only part of the equation. Stories alone aren’t enough to define a system, and trying to define everything in text is a fool’s task. (I’ve been that fool). You need a full arsenal of specification tools to do the best job possible.

Story Pros: captures textual data well, tells the story

Story Cons: not everything is meant to be captured in text

Mockups

Mockups are a very useful tool when specifying details of how things should work. With stories, you really can't add a lot of design and implementation details or the signal to noise ratio becomes too high and shit gets overlooked. A basic rule of thumb we employ on the team is "things that don't get communicated well in text shouldn't be forced into a text medium." Basically, if you're going to try to describe the way something should look in a story, a) it's probably not going to look the way you actually picture it b) that story is now noisy as crap and people are going to ignore more important parts. With a mockup, you don't have to take forever to do a flashy, literal, perfect screenshot in Fireworks or anything; you can just drag and drop Balsamiq controls around and voila. Ya got an aesthetically pleasing mockup that humans go nuts over. In five minutes I can mock something up in front of a developer, explain how it works, and they are ready to go.

Another great thing about mockups is that they are extremely useful for getting user feedback on specs without distracting the user that "this is the final product, no more input." You can use a mockup to discuss workflow and layout without getting mired in fine-grained detail. The last time I was at the lab, I went back to my hotel room for a couple of hours and mocked up apps for 4 workspaces, brought them back to the supervisors and was able to get plenty of good feedback and make edits right there in front of them. Gold.

Mockup Pros: Time-saver, gives the gist of what you want, keeps your stories clean while still conveying what you want, good to show to users.

Mockup Cons: Can fall into the trap of putting everything on a mockup just like you would put everything into a story and it's inappropriate

High Fidelity Design

How easy is it to develop from what basically amounts to a screenshot? You know exactly how everything should look, you can strip images out, you don't really have to think about it.

Wait a minute. There's a red flag.

You don't have to think about it? That's a paddlin'. A high fidelity screenshot, while beautiful and easy to work from, gives developers a signal that this screen is a specification set in stone. They see what it needs to look like, they build it like that. It's just like BDUF; the high level of detail and granularity means that people won't think about what they're actually building, they'll just duplicate what they are given.

Screenshot Pros: Hotness, high level of detail, easy to work from

Screenshot Cons: Removes developer thought, can take a long time to create such a design

Conversation and Whiteboarding

While each of these mediums has plenty of merit and many benefits, conversation and whiteboarding are my (Cat's..well, OK mine too) favorite method of specifying work. There is nothing like having the team (or pertinent members) together, talking through the workflow of a feature/app, mapping out how everything works, doodling out a rough idea of what things are going to look like and how things will come together. It is so damned valuable to have the working group together, talking through how things are going to work and getting their input. While business analysts and managers can come together to specify the general nature of how things need to work, having different members of the team around will help to eke out edge cases or problems that may not have been thought of in original discussion.

Conversation is obviously important by itself too; user stories are written to leave plenty of room for conversation. If you lose communication on your team and people just go off to code in the dark, a lot of the intent and original specification is lost.

Whiteboard Pros: Mapping workflow, multiple sources of input, easy to sketch out an idea/easy to change an idea, whiteboarding is fun as shit, conversation fully fleshes out ideas

Whiteboard Cons: Easy to get lost if not captured appropriately

While we’ve clearly chosen a favorite medium, you really can’t use just one. Each medium has a lot to offer depending on the scenario you are working with, and just like any other thing, you have to use what works naturally for the team in context with what you are doing.

Technorati Tags: how we do it, improvement, lean, management, quality, software, team, planning

How to annoy your teammates

bogardj — Fri, 13 Nov 2009 14:22:55 GMT

In 3 easy steps:

Step 1: Perform a change that affects many, many files

My favorite is a namespace rename. Other choices include deleting a core marker interface, renaming a layer supertype class, or changing the folder structure in a project.

Typically, VisualStudio will crash when I do this, though I’m not sure if it’s VS choking or ReSharper. Either way, a large automatic refactoring will cause a crash.

Step 2: After the VS crash, re-open the solution and choose to recover all unsaved files

ReSharper likely made it quite a ways through your refactoring before VS crashed. All those unsaved files were likely cached for recovery by VisualStudio, so you can probably get a lot of those changes back. Instead of choosing to revert your local changes, go ahead and recover all those files. It’s key for maximum annoyance.

Step 3: Merge these recovered files back to trunk

Since this was an enormous refactoring, maybe affecting hundreds of files, you’ll want to merge your changes back to trunk as soon as possible. After all, you don’t want to be the sucker merging after everyone else has finished their work.

When you’re done, your co-workers will be greeted with one annoying merge, and another fun dialog that never seems to quite go away:

Just when you think you caught the last of the recovered files that had its line endings screwed up, another one pops up. They’ll hate you once because of the core change that affected so many files, making it quite annoying to merge back to trunk. But they’ll form that long-lasting, persistent hate because of this dialog box that keeps coming up. Yes, there is that little checkbox at the bottom, but who reads these things anyway?

Willed and forced design

bogardj — Fri, 13 Nov 2009 02:32:19 GMT

Roy Osherove, as a TypeMock employee, presents quite a dilemma from opinionated TDD blog posts simply because whether he has one or not, there’s always the question of agenda. Which is quite unfortunate, everyone has some sort of selfish agenda at some level. One of those posts came up in a commentary on willed and forced design, with respect to the usage of mocking frameworks:

#1 Willed Design

By writing tests, you can observe the usability of your design from a consumer perspective, and can decide whether or not you like it, and change it accordingly

#2 Forced Design

By using a subset of the available isolation frameworks(rhino, moq, nmock) or specific techniques *manual mocks and stubs) you discover cases that are not technically “mockable” or “fakeable” and use that as a sign for design change.

After about 2 seconds of playing around with dynamic languages such as Ruby and JavaScript, the idea that TypeMock is some how “impure” seemed rather silly. Yes, it bends the CLR in crazy ways, but in and of itself, I believe it has its place.

However, I highly disagree with Roy that #2 is bad, that a tool shouldn’t force my hand in a design. But there’s a bit of a straw man here – it’s not the tool forcing my design, it’s the test telling me where I need to change my design. Quite simply, if I run up against something hard to test, my first choice is to isolate that piece. Here’s an example:

public ActionResult Index()
{
    var user = Session["CurrentUser"];

    return View(user);
}

This piece is awkward to test, even though HttpSessionBase or whatever is a class with virtual methods, and it makes the test ugly. Yes, I could mock the crap out of this heavyweight object, but that’s not really helping me out, is it? Instead, I’ll isolate the ugly:

private readonly IUserSession _userSession;

public HomeController(IUserSession userSession)
{
    _userSession = userSession;
}

public ActionResult Index()
{
    var user = _userSession.GetUser();

    return View(user);
}

Instead of using a dictionary directly, I’ll hide that business behind a facade service, keeping the dictionary ugliness and duplication in one highly cohesive class. But it was the _test_ that led me to do this, because I’m tuned in to pieces that are hard to mock. File I/O? Facade. Web service proxies? Facade. Registry, HttpContext, etc. etc? All facade. I want to isolate my core application from the untestable pieces, not because they’re necessary untestable, but because I don’t want to couple my application directly to these external services. Putting these behind targeted facades eliminates all that duplication that directly using these APIs tends to encourage.

There are plenty of APIs that slapping a simple facade over tends to make things worse, not better, such as:

Workflow Foundation
SharePoint

And any other framework that requires you to tiiiiightly integrate with in order to be successful. So where does that leave me and TypeMock? Well, I don’t use any of those frameworks that require tight coupling, and I don’t have any pain with my current tool of choice in .NET (Rhino Mocks), so why switch exactly? For those rare occasions I might need TypeMock (DateTime.Now), I already have a solution I quite like. Yes, TypeMock could be free, but the only compelling reason I’d switch would be to another free tool with a better, clearer to write, read and scan API.

How We Do Things - Evolving our Specification Practice

Scott C Reynolds — Thu, 12 Nov 2009 21:59:14 GMT

This is part 6 of the How We Do Things series.

This post was co-written on Google Wave with my colleague Cat Schwamm, who keeps me sane every day.

In this part we will explore how specification practice has evolved in our team, from a very waterfallish BDUF approach to what currently is a very lean approach to specification. In the next part of the series we will talk stories and mockups and other tools we employ to make the magic happen.

Where we came from

We began our large LIS project in late 2004, starting with about 6 full months of writing specifications and designing the database. We created reams of documentation for each application complete with use cases that we knew we wouldn't even implement until after the first version was released (sidebar: some of those still haven't been done). We defined more than 100 tables of heavily normalized database schema. Before one line of code was written, we had spent six months of project budget.

As development continued on the project we found ourselves diverging from the specifications that had been created. The documents became little more than a roadmap, with the bold headers on functional groups and the basic usage stories becoming the guidelines that we followed. Much of the detailed specification and database schema went out the window as the system took shape and people began to see and give feedback on what was being produced. Too much specification gave us a false sense of "doing it right" and led us down many wrong paths. Balancingrework with completing planned features became a costly chore.

By the time the project was complete, it was clear that much of that up-front work had been wasted time and money. Had we top-lined the major milestones of such a large project, detailed a small set of functionality, and started developing iteratively, the system would have taken form much sooner, allowing for a tighter feedback loop and much less overall waste.

How we do it now - Overview

Lessons learned, we set about improving how we do specification. I already talked a little about this in the posts on planning, so please review those if you haven't seen them yet. [part 1] [part 2]

When planning a new project of any size, we take an iterative approach. We start at a very high level, and strive to understand the goals of the project. What need does it serve? Who will be using it? Are we competing with something else for market share? Is there a strategic timeline? What's the corporate vision here? What are the bullet points? We will fill in details later. We only want broad targets to guide us further.

When we get closer to development, we start to identify smaller chunks of functionality within each of those broad areas. This is still pretty high level, but done by business analysts and management on the IT team. We start to identify MMFs (minimal marketable features) and group them up in order of importance to help determine next steps.

MMFs in hand, we take the first one and start defining it further. This is where the information from the planning post comes in. We start to write stories (Cat has a great post detailing how to build effective stories). The other information gathered to this point sits dormant, with as little specification work done as possible, until such time as we are getting closer to working on it.

Over time, and only as needed, we put more and more specification on the system, and this is done in parallel with development. In fact, often the most specific information can only surface during development of those features, as they take shape, and as we understand how users will react. Specification should be every bit as iterative as coding.

Reduced to its essence, we JIT specification at many levels to allow maximum flexibility to change direction with minimal wasted work.

Gemba is an essential part of specification

It's often the case that a team relies on "domain experts" to provide them with the specifications they need to build software. This is the BDUF way - gather in committee and have the all-knowing user tell you what to build. Fail.

Cat sez:

The dev team works out of the corporate office in Florida, while the actual laboratory is located in upstate New York. As a result, it is often an exercise of the imagination to figure out the best way to create things for them. Before I visited the lab, I relied on information from others, emails back and forth to the lab crews working all weird hours of the night, and my knowledge of the applications currently in existence. I really only ever got half the picture, and it was difficult to ensure that the things I was specifying would fit into the lab users workflow well without actually knowing what was going on up there. When I visited the lab my whole world was changed. Actually seeing how users interacted with the software and seeing ways they would work around what we didn't have (open Excel or Word documents on their desktop, a wall covered in post-it notes). Just from walking around talking to people for 2 days, I probably got 50 requests. And they never would have asked for them; they would have just kept suffering. Being there showed me everything about how they worked and a million ways I could improve their lives. The experience was invaluable to both me and them, and each subsequent trip has just improved my knowledge of the way we work and the way they interact with our software.

There is no substitute for a certain amount of domain expertise being resident in the team room. Your domain experts are experts only in their domain. They may know the workings of a histology laboratory inside and out, but if you ask them to design a system to support that lab, they'll come back with something that looks an awful lot like excel and post-it notes.

Yes...this is a workaround someone had for something our system didn't do. It's been fixed ;)

Lean has a concept of the gemba attitude and genchi genbutsu, essentially, being in the place where the action happens. Developers and business analysts have to work with the domain experts, in the place where the work happens, and observe the true workflows in place before hoping to design a system to support them. You cannot get this information from sit down meetings and emails with domain experts. You must go and see for yourself, or you will miss things.

In the next post we will talk specifically about the tools and techniques we use to specify work, and how we combine them to form complete pictures of a system (spoiler alert: it ain't just stories).

Technorati Tags: how we do it, improvement, lean, management, quality, software, team, planning

Going Away Geek Beers

Scott C Reynolds — Thu, 12 Nov 2009 14:47:43 GMT

Most of you have heard by now that I am leaving Florida for the concrete pastures of New York City.

I'd like to have a get-together of some of the Florida community guys (and gals) Monday, Nov 16, in Ocala at 7pm until we decide to leave.

It's truly been a pleasure working with this community over the last few years, and I will miss you guys. So..come on out and buy me a beer! ;)

Location: O'Malley's Alley

My phone is 352.209.0943 drop me a line if you get lost of want to confirm you're coming. So far looks like a real good group will be there, including Cory Foy, Sean Chambers, Scott Densmore, and Will Green, the amazing Cat Schwamm as well as members of my team. Hope to see you Monday!

Electronic Kanban Tools and WIP Limits: LeanKitKanban and AgileZen Get It!

derick.bailey — Mon, 09 Nov 2009 17:55:24 GMT

Most of the electronic Kanban boards that exist simply don’t understand what WIP limits are really about. They all seem to go down the path of forcing any step (column) in the process to have a maximum limit that is very hard and fast. They set it up so that you cannot add any more items to the step than the limit allows. But the truth about WIP limits is that they are not supposed to be hard and fast, mechanical limits. Putting in a hard limit like this is a great way to destroy the productivity of a team and usefulness of a tool when the team finds themselves in a crunch, in a situation with additional resources and needs, or any of a number of other circumstances.

I know I haven’t been vocal about this online, but it’s something I’ve been complaining about to my coworkers and friends for a while. It bugs me and is one of the major reasons that I’ve stayed away from the major (free) players in the electronic Kanban realm, other than trials and basic testing of the systems.

But then I got an email from Chris Hefley this morning, about some new features in LeanKitKanban. When I logged in to check them out, I was surprised to see this happen when I tried to exceed a column limit:

I can’t tell you how happy I was to see this! Finally, one of the tools vendors in this space is really starting to get it! For this, and a few other reasons that I don’t want to talk about yet, LeanKitKanban is now officially my favorite electronic Kanban / task board around. If you haven’t done so, I highly recommend that you check them out.

Do any other electronic Kanban boards are heading down this same path and allowing for limit overrides? If your current tool doesn’t support this, it should.

Update: Looks like AgileZen is getting this feature in the next release (see Nate’s comments below). that’s great news! I’ve been impressed by this tool as well. the support for soft WIP limits in multiple tools is starting to show some real competition and innovation in the market space. that’s good news for everyone!