you've been HAACKED

The Law of Demeter Is Not A Dot Counting Exercise

Haacked — Tue, 14 Jul 2009 17:48:29 GMT

Recently I read a discussion on an internal mailing list on whether or not it would be worthwhile to add a null dereferencing operator to C#.

For example, one proposed idea would allow the following expression.

object a = foo.?bar.?baz.?qux;

This would assign the variable a the value null if any one of foo, bar, or baz is null instead of throwing a NullReferenceException. It’s a small, but potentially helpful, mitigation for the billion dollar mistake.

Sure enough, it did not take long for someone to claim that this syntax would be unnecessary if the code here was not violating the sacred Law of Demeter (or LoD for short). I think this phenomena is an analog to Godwin’s Law and deserves its own name. Let’s call it the “LoD Dot Counting Law”:

As a discussion of a code expression with more than one dot grows longer, the probability that someone claims a Law of Demeter violation approaches 1.

Count the dots and win a prize!

What is wrong with the claim that the above expression violates LoD? To answer that let’s briefly cover the Law of Demeter. I’m not going to cover it in detail but rather point to posts that describe it in much better detail than I would.

The Many Forms of Demeter

The formal object form of the law can be summarized as:

A method of an object may only call methods of:

The object itself.

An argument of the method.

Any object created within the method.

Any direct properties/fields of the object.

A general formulation of the law is:

Each unit should have only limited knowledge about other units: only units "closely" related to the current unit. Or: Each unit should only talk to its friends; Don’t talk to strangers.

This of course leads to the succinct form of the law:

Don’t talk to strangers

In other words, try to avoid calling methods of an object that was returned by calling another method. Often, people shorten the law to simply state “use only one dot”.

One of the key benefits of applying LoD is low coupling via encapsulation. In his paper, The Paperboy, The Wallet, and The Law of Demeter (PDF) (it’s a relatively quick read so go ahead, I’ll be here), David Bock provides a great illustration of this law with an analogy of a paperboy and a customer. Rather than having a customer hand over his wallet to pay the paperboy, he instead has the paperboy request payment from the customer.

In answer to “Why is this better?” David Bock gives these three reasons.

The first reason that this is better is because it better models the real world scenario... The Paperboy code is now 'asking' the customer for a payment. The paperboy does not have direct access to the wallet.

The second reason that this is better is because the Wallet class can now change, and the paperboy is completely isolated from that change…

The third, and probably most 'object-oriented' answer is that we are now free to change the implementation of 'getPayment()'.

Note that the first benefit is an improvement not only in encapsulation but the abstraction is also improved.

Dot Counting Is Not The Point

You’ll note that David doesn’t list “50% less dots in your code!” as a benefit of applying LoD. The focus is on reduced coupling and improved encapsulation.

So going back to the initial expression, does foo.bar.baz.qux violate LoD? Like most things, it depends.

For example, suppose that foo is of type Something and it contains properties named Bar, Baz, and Qux which each simply return this.

In this semi-contrived example, the expression is not an LoD violation because each property returns the object itself and according to the first rule of LoD, “you do not talk about LoD” … wait … sorry… “a method is free to call any properties of the object itself” (in a future post, I will cover the class form of LoD which seems to indicate that if Bar, Baz, and Qux return the same type, whether it’s the same object or not, LoD is preserved).

This pattern is actually quite common with fluent interfaces where each method in a calling chain might return the object itself, but transformed in some way.

So we see that counting dots is not enough to indicate an LoD violation. But lets dig deeper. Are there other cases where counting dots leads do not indicate an LoD violation? More importantly, is it always a bad thing to violate LoD? Are there cases where an LoD violation might even be the right thing to do?

Go Directly to Jail, Do Not Pass Go

Despite its name, violating the Law of Demeter will not get you on an episode of Cops nor is it some inviolable law of nature.

As the original paper points out, it was developed during design and implementation of the Demeter system (hence the name) and was held to be a law for the developers of that system.

The designers of the system felt that this practice ensured good Object-Oriented design:

The motivation behind the Law of Demeter is to ensure that the software is as modular as possible. The law effectively reduces the occurrences of nested message sendings (function calls) and simplifies the methods.

However, while it was a law in the context of the Demeter system, whether it should hold the weight that calling it a Law implies is open to debate.

David Bock refers to it as an idiom:

This paper is going to talk about a particluar (sic) 'trick' I like, one that is probably better classified as an 'idiom' than a design pattern (although it is a component in many different design patterns).

Martin Fowler suggests (emphasis mine)

I'd prefer it to be called the Occasionally Useful Suggestion of Demeter.

Personally, I think most developers are guilty of bad encapsulation and tight coupling. I’m a bit more worried about that than applying this law inappropriately (though I worry about that too). Those who have deep understanding of this guideline are the ones who are likely to know when it shouldn’t be applied.

For the rest of us mere mortals, I think it’s important to at least think about this guideline and be intentional about applying or not applying it.

I Fought The Law and The Law Won

So what are the occasions when the Law of Demeter doesn’t necessarily apply? There’s some debate out there on the issue.

In his post, Misunderstanding the Law of Demeter, Daniel Manges argues that web page views aren’t domain objects and thus shouldn’t be subject to the Law of Demeter. His argument hinges on a Rails example where you send an Order object to the view, but the view needs to display the customer’s name.

<%= @order.customer.name %>

Counting two dots, he considers the change that would make it fit LoD:

<%= @order.customer_name %>

He then asks:

Why should an order have a customer_name? We're working with objects, an order should have a customer who has a name.

…when rendering a view, it's natural and expected that the view needs to branch out into the domain model.

Alex Blabs of Pivotal Labs takes issue with Daniel’s post and argues that views are domain objects and an order ought to have a customer_name property.

It’s an interesting argument, but the following snippet of a comment by Zak Tamsen summarizes where I currently am on this subject (though my mind is open).

because they don't. the primary job of the views (under discussion) is to expose the internal state of objects for display purposes. that is, they are expressly for data showing, not data hiding. and there's the rub: these kind of views flagrantly violate encapsulation, LoD is all about encapsulation, and no amount of attribute delegation can reconcile this.

The problem as I see it with Alex’s approach is where do you stop? Does the Order object encapsulate every property of Customer? What about sub-properties of the Customer’s properties? It seems the decision to encapsulate the Customer’s name is driven by the view’s need to display it. I wouldn’t want my domain object’s interface to be driven by the needs of the view as that would violate separation of concerns.

There’s another option which might be more common in the ASP.NET MVC world than in the Rails world, I’m not sure. Why not have a view specific model object. This would effectively be the bridge between the domain objects and the view and could encapsulate many of the these properties that the view needs to display.

Another case where an LoD violation might not be such a bad idea is in cases where the object structure is public and unlikely to change. While in Norway, I had the opportunity to briefly chat with Michael Feathers about LoD and he pointed out the example of Excel’s object model for tables and cells. If LoD is about encapsulation (aka information hiding) then why would you hide the structure of an object where the structure is exactly what people are interested in and unlikely to change?

Use It or Lose It

When I learn a new guideline or principle, I really like to dig into where the guideline breaks down. Knowing where a guideline works, and what its advantages are is only half the story in really understanding it. When I can explain where it doesn’t work and what its disadvantages are, only then do I feel I’m starting to gain understanding.

However, in writing about my attempts at understanding, it may come across that I’m being critical of the guideline. I want to be clear that I think the Law of Demeter is a very useful guideline and it applies in more cases than not. It’s one of the few principles that can point to an empirical study that may point to its efficacy.

In a Validation of Object-Oriented Design Metrics as Quality Indicators, the authors of the study provide evidence that suggests The Law of Demeter reduces the probability of software faults.

Still, I would hope that those who apply it don’t do it blindly by counting dots. Dot counting can help you find where to look for violations, but always keep in mind that the end goal is reducing coupling, not dots.

Technorati Tags: SOLID,Software Development,Patterns,Law of Demeter

Geek Your Momma Jokes

Haacked — Wed, 08 Jul 2009 21:52:14 GMT

Ok, I haven’t had a good track record with making up jokes before. Just see exhibit A - this groaner of an MVC joke.

However, I think I did okay recently with a few geeky Your Momma jokes that I can’t just leave to Twitter alone. Here they are:

Your momma so fat, Bloatware is her clothing line.

Your momma so fat I called her and got a stack overflow.

your momma so ugly its just best to forego the "V" in MVC with her.

So now it’s your turn. Please post your best ones in the comments to this post. :)

Technorati Tags: humor,jokes,your momma jokes

NDC09 Trip Report

Haacked — Mon, 29 Jun 2009 03:24:50 GMT

When you visit Norway, it takes a week to recover. Ok, at least when I visit Norway, it takes a week. But that’s just a testament to the good time I had. As they say, what happens in Vegas stays in Vegas, but what happens in Oslo gets recorded as a .NET Rocks Live episode.

The week before last, I spent the week in Oslo, Norway attending and speaking at the Norwegian Developer’s Conference (NDC 09). This conference was not your typical Microsoft conference I usually attend but was a conference on .NET with a heavy Agile Software bent.

Just looking at the speaker line-up will tell you that. Scott Bellware tweeted a blurb recently that succinctly summarized my impression of the conference:

how to know you're at a good conference: the speakers are going to sessions (at least the ones who aren't working on their sessions)

That is definitely true. I didn’t attend as many talks as I would have liked, but I did manage to attend two by Mary Poppendieck which really sparked my imagination and got me excited about the concept of a problem solving organization and learning more about Lean. She had promised to put her slides on her site, but I can’t for the life of me find them! ;)

While there, I gave three talks, one of them being a joint talk with the Hanselnator (aka Mr. Hanselman).

Black Belt Ninja Tips ASP.NET MVC
This covered several tips on getting more out of ASP.NET MVC and included the first public demonstration of David Ebbo’s T4 template.

ASP.NET MVC + AJAX = meant for each other
This covered the Ajax helpers included with ASP.NET MVC and drilled into some of the lesser known client aspects of these helpers, such as showing the Sys.Mvc.AjaxContext object and how to leverage it. The talk then moved into a demonstration of the client templating feature of ASP.NET Ajax 4 Preview 4. I showed off some of the work Jonathan Carter and I (mostly Jonathan) did to make two-way data binding work with ASP.NET MVC. The audience really dug it.

The Haacked and Hanselman Show
So named because we didn’t have any agenda until about a week before the conference, this ended up being a web security talk where Scott would present a common “secure” implementation of a feature, I would then proceed to “Haack” the feature, and then Scott would fix the feature, all the while explaining what was going on. I think this was very big hit as we saw messages on Twitter like “I’m now too afraid to build a web application”. ;) Of course, I hope more attendees felt empowered rather than fearful. :P

The conference was held in an indoor soccer stadium since it was a venue large enough for all the attendees. They curtained off sections of the bleachers to create rooms for giving the talks. On the outside of the curtains was a large screen which allowed attendees to walk around from talk to talk with a headset on the conference floor if they didn’t feel like sitting in the bleachers.

Plenty of bean bags on the floor provided a comfortable place to relax and listen in. In fact, that’s where I would often find some of my new friends lounging around such as the crazy Irishman.

On the second night of the conference, we all rocked out at the big attendee party featuring the band, DataRock which played some rocking music with geek friendly lyrics like:

I ran into her on computer camp
(Was that in 84?)
Not sure
I had my commodore 64
Had to score

-- DataRock, Computer Camp Love

Many thanks to Kjetil Klaussen for posting those lyrics in his NDC 09 highlights post because I had forgotten pretty much every lyric. :) After DataRock, we all went upstairs for the after party to enjoy a more intimate setting with LoveShack, an 80s cover band.

One interesting highlight of the show was a live recording of .NET Rocks. The show was originally going to simply feature Scott Hanselman, but while hanging out in the speakers lounge Carl Franklin, one of the hosts of the show, suggested I join in the fun too.

While it was great fun, Scott is a big, no ginormous, personality and I rarely got a word in edgewise, except for the few times I swooped right in just in time to put my foot in my mouth to apparently great comedic effect. In any case, you can listen to the results yourself, though I hope they post the video soon to get the full effect of how much fun everyone was having. :) Be warned, there’s not a lot of real software development content in the show.

The conference ended on Friday leaving all of Saturday for me to relax and actually get out and see Oslo. On Saturday, I headed out to Vigeland Statue Park with an eclectic group of people, Ted Neward, Rocky Lhotka and his wife, Jeremy Miller, and Anna K{Something With Too Many Syllables in a Row}, a conference organizer herself.

The park was very beautiful and I took a ton of pictures, but unfortunately I lost my camera on the flight home from Norway. :( So instead, I’ll just include this Creative Commons licensed picture taken by Cebete from Flickr. The main difference was the sky was a deep blue when we visited.

That evening, Sondre Bjellås, an attendee, was kind enough to invite several of us over to his flat for a little gathering. I headed over with Bellware and Anna since everyone else was pretty much flattened by the previous weeks activities. It was great to meet non-techie Norwegians such as his wife and friends in order to get a different perspective on what it’s like to live in Norway. The answer: expensive!

In an odd coincidence, on my connecting flight in Philadelphia, I ran into my good friend Walter who happened to be flying home from Belgium. In fact, we were on the same flight in the same exit row with seats right next to each other. How’s that for a funny coincidence?

Show me the Code!

Rune, one of the organizers of the conference, assures me that the videos of the talks will be posted online soon, so you’ll get to see them if you’d like. I’ve also posted my powerpoint slides and code samples here.

Please note that my talks tend to be heavy in demos so the Powerpoint decks don’t have much content in them. Likewise, the code samples represent the “before” state of my talks, not the “after” state. I usually write up a checklist for each talk which I use a to remind myself where I am in those cases where I have a total brain fart and forget my own name under the pressure of presenting.

Is It Too Late To Change JSON?

Haacked — Fri, 26 Jun 2009 12:00:00 GMT

In my last post, I wrote about the hijacking of JSON arrays. Near the end of the post, I mentioned a comment whereby someone suggests that what really should happen is that browsers should be more strict about honoring content types and not execute code with the content type of application/json.

I totally agree! But then again, browsers haven’t had a good track record with being strict with such standards and it’s probably too much to expect browsers to suddenly start tightening ship, not to mention potentially breaking the web in the process.

Another potential solution that came to mind was this: Can we simply change JSON? Is it too late to do that or has that boat left the harbor?

Let me run an idea by you. What if everyone got together and decided to version the JSON standard and change it in such a way that when the entire JSON response is an array, the format is no longer executable script. Note that I’m not referring to an array which is a property of a JSON object. I’m referring to the case when the entire JSON response is an array.

One way to do this, and I’m just throwing this out there, is to make it such that the JSON package must always begin and end with a curly brace. JSON objects already fulfill this requirement, so their format would remain unchanged.

But when the response is a JSON array, we would go from here:

[{"Id":1,"Amt":3.14},{"Id":2,"Amt":2.72}]

to here:

{[{"Id":1,"Amt":3.14},{"Id":2,"Amt":2.72}]}

Client code would simply check to see if the JSON response starts with {[ to determine whether it’s an array, or an object. There many alternatives, such as simply wrapping ALL JSON responses in some new characters to keep it simple.

It’d be possible to do this without breaking every site out there by simply giving all the client libraries a head start. We would update the JavaScript libraries which parse JSON to recognize this new syntax, but still support the old syntax. That way, they’d work with servers which haven’t yet upgraded to the new syntax.

As far as I know, most sites that make use of JSON are using it for Ajax scenarios so the site developer is in control of the client and server anyways. For sites that provide JSON as a cross-site service, upgrading the server before the clients are ready could be problematic, but not the end of the world.

So what do you think? Is this worth pursuing? Not that I have any idea on how I would convince or even who I would need to convince. ;)

UPDATE: 6/26 10:39 AM Scott Koon points out this idea is not new (I didn’t think it would be) and points to a great post that gives more detail on the specifics of executable JSON as it relates to the ECMAScript Specification.

Tags: JavaScript, JSON, security

JSON Hijacking

Haacked — Thu, 25 Jun 2009 19:16:24 GMT

A while back I wrote about a subtle JSON vulnerability which could result in the disclosure of sensitive information. That particular exploit involved overriding the JavaScript Array constructor to disclose the payload of a JSON array, something which most browsers do not support now.

However, there’s another related exploit that seems to affect many more browsers. It was brought to my attention recently by someone at Microsoft and Scott Hanselman and I demonstrated it at the Norwegian Developers Conference last week, though it has been demonstrated against Twitter in the past.

Before I go further, let me give you the punch line first in terms of what this vulnerability affects.

This vulnerability requires that you are exposing a JSON service which…

…returns sensitive data.
…returns a JSON array.
…responds to GET requests.
…the browser making the request has JavaScript enabled (very likely the case)
…the browser making the request supports the __defineSetter__ method.

Thus if you never send sensitive data in JSON format, or you only send JSON in response to a POST request, etc. then your site is probably not vulnerable to this particular vulnerability (though there could be others).

I’m terrible with Visio, but I thought I’d give it my best shot and try to diagram the attack the best I could. In this first screenshot, we see the unwitting victim logging into the vulnerable site, and the vulnerable site issues an authentication cookie, which the browser holds onto.

At some point, either in the past, or the near future, the bad guy spams the victim with an email promising a hilariously funny video of a hamster on a piano.

But the link actually points to the bad guy’s website. When the victim clicks on the link, the next two steps happen in quick succession. First, the victim’s browser makes a request for the bad guy’s website.

The website responds with some HTML containing some JavaScript along with a script tag. When the browser sees the script tag, it makes another GET request back to the vulnerable site to load the script, sending the auth cookie along.

The bad guy has tricked the victim’s browser to issue a request for the JSON containing sensitive information using the browser’s credentials (aka the auth cookie). This loads the JSON array as executable JavaScript and now the bad guy has access to this data.

To gain a deeper understanding, it may help to see actual code (which you can download and run) which demonstrates this attack.

Note that the following demonstration is not specific to ASP.NET or ASP.NET MVC in any way, I just happen to be using ASP.NET MVC to demonstrate it. Suppose the Vulnerable Website returns JSON with sensitive data via an action method like this.

[Authorize]
public JsonResult AdminBalances() {
  var balances = new[] {
    new {Id = 1, Balance=3.14}, 
    new {Id = 2, Balance=2.72},
    new {Id = 3, Balance=1.62}
  };
  return Json(balances);
}

Assuming this is a method of HomeController, you can access this action via a GET request for /Home/AdminBalances which returns the following JSON:

[{"Id":1,"Balance":3.14},{"Id":2,"Balance":2.72},{"Id":3,"Balance":1.62}]

Notice that I’m requiring authentication via the AuthorizeAttribute on this action method, so an anonymous GET request will not be able to view this sensitive data.

The fact that this is a JSON array is important. It turns out that a script that contains a JSON array is a valid JavaScript script and can thus be executed. A script that just contains a JSON object is not a valid JavaScript file. For example, if you had a JavaScript file that contained the following JSON:

{"Id":1, "Balance":3.14}

And you had a script tag that referenced that file:

<script src="http://example.com/SomeJson"></script>

You would get a JavaScript error in your HTML page. However, through an unfortunate coincidence, if you have a script tag that references a file only containing a JSON array, that would be considered valid JavaScript and the array gets executed.

Now let’s look at the HTML page that the bad guy hosts on his/her own server:

<html> 
...
<body> 
    <script type="text/javascript"> 
        Object.prototype.__defineSetter__('Id', function(obj){alert(obj);});
    </script> 
    <script src="http://example.com/Home/AdminBalances"></script> 
</body> 
</html>

What’s happening here? Well the bad guy is changing the prototype for Object using the special __defineSetter__ method which allows overriding what happens when a property setter is being called.

In this case, any time a property named Id is being set on any object, an anonymous function is called which displays the value of the property using the alert function. Note that the script could just as easily post the data back to the bad guy, thus disclosing sensitive data.

As mentioned before, the bad guy needs to get you to visit his malicious page shortly after logging into the vulnerable site while your session on that site is still valid. Typically a phishing attack via email containing a link to the evil site does the trick.

If by blind bad luck you’re still logged into the original site when you click through to the link, the browser will send your authentication cookie to the website when it loads the script referenced in the script tag. As far as the original site is concerned, you’re making a valid authenticated request for the JSON data and it responds with the data, which now gets executed in your browser. This may sound familiar as it is really a variant of a Cross Site Request Forgery (CSRF) attack which I wrote about before.

If you want to see it for yourself, you can download this ASP.NET MVC project and run it locally. Make sure to login first and then go visit http://haacked.com/demos/JsonAttack.html.

Note that this attack does not work on IE 8 which will tell you that __defineSetter__ is not a valid method. Last I checked, it does work on Chrome and Firefox.

The mitigation is simple. Either never send JSON arrays OR always require an HTTP POST to get that data (except in the case of non-sensitive data in which case you probably don’t care). For example, with ASP.NET MVC, you could use the AcceptVerbsAttribute to enforce this like so:

[Authorize]
[AcceptVerbs(HttpVerbs.Post)]
public JsonResult AdminBalances() {
  var balances = new[] {
    new {Id = 1, Balance=3.14}, 
    new {Id = 2, Balance=2.72},
    new {Id = 3, Balance=1.62}
  };
  return Json(balances);
}

One issue with this approach is that many JavaScript libraries such as jQuery request JSON using a GET request by default, not POST. For example, $.getJSON issues a GET request by default. So when calling into this JSON service, you need to make sure you issue a POST request with your client library.

ASP.NET and WCF JSON service endpoints actually wrap their JSON in an object with the “d” property as I wrote about a while back. While it might seem odd to have to go through this property to get access to your data, this awkwardness is eased by the fact that the generated client proxies for these services strip the “d” property so the end-user doesn’t need to know it was ever there.

With ASP.NET MVC (and other similar frameworks), a significant number of developers are not using client generated proxies (we don’t have them) but instead using jQuery and other such libraries to call into these methods, making the “d” fix kind of awkward.

What About Checking The Header?

Some of you might be wondering, “why not have the JSON service check for a special header such as the X-Requested-With: XMLHttpRequest or Content-Type: application/json before serving it up in response to a GET request?” I too thought this might be a great mitigation because most client libraries send one or the other of these headers, but a browser’s GET request in response to a script tag would not.

The problem with this (as a couple of co-workers pointed out to me) is that at some point in the past, the user may have made a legitimate GET request for that JSON in which case it may well be cached in the user’s browser or in some proxy server in between the victim’s browser and the vulnerable website. In that case, when the browser makes the GET request for the script, the request might get fulfilled from the browser cache or proxy cache. You could try setting No-Cache headers, but at that point you’re trusting that the browser and all proxy servers correctly implement caching and that the user can’t override that accidentally.

Of course, this particular caching issue isn’t a problem if you’re serving up your JSON using SSL.

The real issue?

There’s a post at the Mozilla Developer Center which states that object and array initializers should not invoke setters when evaluated, which at this point, I tend to agree with, though a comment to that post argues that perhaps browsers really shouldn’t execute scripts regardless of their content type, which is also a valid complaint.

But at the end of the day, assigning blame doesn’t make your site more secure. These type of browser quirks will continue to crop up from time to time and we as web developers need to deal with them. Chrome 2.0.172.31 and Firefox 3.0.11 were both vulnerable to this. IE 8 was not because it doesn’t support this method. I didn’t try it in IE 7 or IE 6.

It seems to me that to be secure by default, the default behavior for accessing JSON should probably be POST and you should opt-in to GET, rather than the other way around as is done with the current client libraries. What do you think? And how do other platforms you’ve worked with handle this? I’d love to hear your thoughts.

In case you missed it, here are the repro steps again: Download this ASP.NET MVC project and run it locally. Make sure to login first and then go visit http://haacked.com/demos/JsonAttack.html in the same browser.

I followed up this post with a proposal to fix JSON to prevent this particular issue.

Tags: aspnetmvc, json, javascript, security, browsers

And Get Rid Of Those Pesky Programmers

Haacked — Sat, 13 Jun 2009 03:49:54 GMT

Every now and then some email or website comes along promising to prove Fred Brooks wrong about this crazy idea he wrote in The Mythical Man Month (highly recommended reading!) that there is no silver bullet which by itself will provide a tenfold improvement in productivity, reliability, and simplicity within a decade.

This time around, the promise was much like others, but they felt the need to note that their revolutionary new application/framework/doohickey will allow business analysts to directly build applications 10 times as fast without the need for programmers!

Ah yeah! Get rid of those foul smelling pesky programmers! We don’t need em!

Now wait one dag-burn minute! Seriously?!

I’m going to try real hard for a moment to forget they said that and not indulge my natural knee jerk reaction which is to flip the bozo bit immediately. If I were a more reflective person, this would raised a disturbing question:

Why are these business types so eager to get rid of us programmers?

It’s easy to blame the suits for not understanding software development and forcing us into a Tom Smykowski moment having to defend what it is we do around here.

Well-well look. I already told you: I deal with the god damn customers so the engineers don't have to. I have people skills; I am good at dealing with people. Can't you understand that? What the hell is wrong with you people?

Maybe, as Steven “Doc” List quotes from Cool Hand Luke in his latest End Bracket article on effective communication for MSDN Magazine,

What we've got here is a failure to communicate.

Leon Bambrick (aka SecretGeek) recently wrote about this phenomena in his post entitled, The Better You Program, The Worse You Communicate, in which he outlines how techniques that make us effective software developers do not apply to communicating with other humans.

After all, we can sometimes be hard to work with. We’re often so focused on the technical aspects and limitations of a solution that we unknowingly confuse the stakeholders with jargon and annoy them by calling their requirements “ludicrous”. Sometimes, we fail to deeply understand their business and resort to making fun of our stakeholders rather than truly understanding their needs. No wonder they want to do the programming themselves!

Ok, ok. It’s not always like this. Not every programmer is like this and it isn’t fair to lay all the blame at our feet. I’m merely trying to empathize and understand the viewpoint that would lead to this idea that moving programmers out of the picture would be a good thing.

Some blame does deserve to lie squarely at the feet of these snake oil salespeople, because at the moment, they’re selling a lie. What they’d like customers to believe is your average business analyst simply describes the business in their own words to the software, and it spits out an application.

The other day, I started an internal email thread describing in hand-wavy terms some feature I thought might be interesting. A couple hours later, my co-worker had an implementation ready to show off.

Now that my friends, is the best type of declarative programming. I merely declared my intentions, waited a bit, and voila! Code! Perhaps that’s along the lines of what these types of applications hope to accomplish, but there’s one problem. In the scenario I described, it required feeding requirements to a human. If I had sent that email to some software, it would have no idea what to do with it.

At some point, something close to this might be possible, but only when software has reached the point where it can exhibit sophisticated artificial intelligence and really deal with fuzziness. In other words, when the software itself becomes the programmer, only then might you really get rid of the human programmer. But I’m sorry to say, you’re still working with a programmer, just one who doesn’t scoff at your requirements arrogantly (at least not in your face while it plots to take over the world, Copper-top).

Until that day, when a business analyst wires together an applications with Lego-like precision using such frameworks, that analyst has in essence become a programmer. That work requires many of the same skills that developers require. At this point, you really haven’t gotten rid of programmers, you’ve just converted a business type into a programmer, but one who happens to know the business very well.

In the end, no matter how “declarative” a system you build and how foolproof it is such that a non-programmer can build applications by dragging some doohickeys around a screen, there’s very little room for imprecision and fuzziness, something humans handle well, but computers do not, as Spock demonstrated so well in an episode of Star Trek.

“Computer, compute the last digit of PI” - Spock

Throw into the mix that the bulk of the real work of building an application is not the coding, but all the work surrounding that, as Udi Dahan points out in his post on The Fallacy of ReUse.

This is not to say that I don’t think we should continue to invest in building better and better tools. After all, the history of software development is about building better and better higher level tools to make developers more productive. I think the danger lies in trying to remove the discipline and traits that will always be required when using these tools to build applications.

Even when you can tell the computer what you want in human terms, and it figures it out, it’s important to still follow good software development principles, ensure quality checks, tests, etc…

The lesson for us programmers, I believe is two-fold. One, we have to educate our stakeholders about how software production really works. Even if they won’t listen, a little knowledge and understanding here goes a long way. Be patient, don’t be condescending, and hope for the best. Secondly, we have to educate ourselves about the business in a deep manner so that we are seen as valuable business partners who happen to write the code that matters.

Technorati Tags: software development,business analysts,silver bullet

ASP.NET MVC Installer For Visual Studio 2010 Beta 1 And Roadmap

Haacked — Tue, 09 Jun 2009 17:59:37 GMT

A little while ago I announced our plans for ASP.NET MVC as it relates to Visual Studio 2010. ASP.NET MVC wasn’t included as part of Beta 1, which raised a few concerns by some (if not conspiracy theories!) ;). The reason for this was simple as I pointed out:

One thing you’ll notice is that ASP.NET MVC is not included in Beta 1. The reason for this is that Beta 1 started locking down before MVC 1.0 shipped. ASP.NET MVC will be included as part of the package in VS10 Beta 2.

…

We’re working hard to have an out-of-band installer which will install the project templates and tooling for ASP.NET MVC which works with VS2010 Beta 1 sometime in June on CodePlex. Sorry for the inconvenience. I’ll blog about it once it is ready.

Today I’m happy to announce that we’re done with the work I described and the installer is now available on CodePlex. Be sure to give it a try as many of the new VS10 features intended to support the TDD workflow fit very nicely with ASP.NET MVC, which ScottGu will describe in an upcoming blog post.

If you run into problems with the intaller, try out this troubleshooting guide by Jacques, the developer who did the installer work and do provide feedback.

You’ll notice that the installer says this is ASP.NET MVC 1.1, but as the readme notes point out, this is really ASP.NET MVC 1.0 retargeted for Visual Studio 2010. The 1.1 is just a placeholder version number. We bumped up the version number to avoid runtime conflicts with ASP.NET MVC 1.0. All of this and more is described in the Release Notes.

When VS10 Beta 2 comes out, you won’t need to download a separate standalone installer to get ASP.NET MVC (though a standalone installer will be made available for VS2008 users that will run on ASP.NET 3.5 SP1). A pre-release version of ASP.NET MVC 2 will be included as part of the Beta 2 installer as described in the …

Roadmap

I recently published the Roadmap for ASP.NET MVC 2 which gives a high level look at what features we plan to do for ASP.NET MVC 2. The features are noticeably lacking in details as we’re deep in the planning phase trying to gather pain points.

Right now, we’re avoiding focusing the implementation details as much as possible. When designing software, it’s very easy to have preconceived notions about what the solution should be, even when we really don’t have a full grasp of the problem that needs to be solved.

Rather than guiding people towards what we think the solution is, I hope to focus on making sure we understand the problem domain and what people want to accomplish with the framework. That leaves us free to try out alternative approaches that we might not have considered before such as alternatives to expression based URL helpers. Maybe the alternative will work out, maybe not. Ideally, I’d like to have several design alternatives to choose from for each feature.

As we get further along the process, I’ll be sure to flesh out more and more details in the Roadmap and share them with you.

Snippets

One cool new feature of VS10 is that snippets now work in the HTML editor. Jeff King from the Visual Web Developer team sent me the snippets we plan to include in the next version. They are also downloadable from the CodePlex release page. Installation is very simple:

Installation Steps:

1) Unzip "ASP.NET MVC Snippets.zip" into "C:\Users\<username>\Documents\Visual Studio 10\Code Snippets\Visual Web Developer\My HTML Snippets", where "C:\" is your OS drive.
2) Visual Studio will automatically detect these new files.

Try them out and let us know if you have ideas for snippets that will help you be more productive.

Important Links:

CodePlex Download Page
Release Notes (also linked to from above page)
ASP.NET MVC 2 Roadmap
Troubleshooting Info for this installer

Tags: asp.net, aspnetmvc, visual studio 2010

An Alternative Approach To Strongly Typed Helpers

Haacked — Tue, 02 Jun 2009 15:56:16 GMT

One of the features contained in the MVC Futures project is the ability to generate action links in a strongly typed fashion using expressions. For example:

<%= Html.ActionLink<HomeController>(c => c.Index()) %>

Will generate a link to to the Index action of the HomeController.

It’s a pretty slick approach, but it is not without its drawbacks. First, the syntax is not one you’d want to take as your prom date. I guess you can get used to it, but a lot of people who see it for the first time kind of recoil at it.

The other problem with this approach is performance as seen in this slide deck I learned about from Brad Wilson. One of the pain points the authors of the deck found was that the compilation of the expressions was very slow.

I had thought that we might be able to mitigate these performance issues via some sort of caching of the compiled expressions, but that might not work very well. Consider the following case:

<% for(int i = 0; i < 20; i++) { %>

  <%= Html.ActionLink<HomeController>(c => c.Foo(i)) %>

<% } %>

Each time through that loop, the expression is the same: c => c.Foo(i)

But the value of the captured “i” is different each time. If we try to cache the compiled expression, what happens?

So I started thinking about an alternative approach using code generation against the controllers and circulated an email internally. One approach was to code gen action specific action link methods. Thus the about link for the home controller (assuming we add an id parameter for demonstration purposes) would be:

<%= HomeAboutLink(123) %>

Brad had mentioned many times that while he likes expressions, he’s no fan of using them for links and he tends to write specific action link methods just like the above. So what if we could generate them for you so you didn’t have to write them by hand?

A couple hours after starting the email thread, David Ebbo had an implementation of this ready to show off. He probably had it done earlier for all I know, I was stuck in meetings. Talk about the best kind of declarative programming. I declared what I wanted roughly with hand waving, and a little while later, the code just appears! ;)

David’s approach uses a BuildProvider to reflect over the Controllers and Actions in the solution and generate custom action link methods for each one. There’s plenty of room for improvement, such as ensuring that it honors the ActionNameAttribute and generating overloads, but it’s a neat proof of concept.

One disadvantage of this approach compared to the expression based helpers is that there’s no refactoring support. However, if you rename an action method, you will get a compilation error rather than a runtime error, which is better than what you get without either. One advantage of this approach is that it performs fast and doesn’t rely on the funky expression syntax.

These are some interesting tradeoffs we’ll be looking closely at for the next version of ASP.NET MVC.

Tags: aspnetmvc, asp.net, helpers, actionlink

Writing A Page To A String

Haacked — Fri, 29 May 2009 16:43:28 GMT

ASP.NET Pages are designed to stream their output directly to a response stream. This can be a huge performance benefit for large pages as it doesn’t require buffering and allocating very large strings before rendering. Allocating large strings can put them on the Large Object Heap which means they’ll be sticking around for a while.

However, there are many cases in which you really want to render a page to a string so you can perform some post processing. I wrote about one means using a Response filter eons ago.

However, recently, I learned about a method of the Page class I never noticed which allows me to use a much lighter weight approach to this problem.

The method in question is CreateHtmlTextWriter which is protected, but also virtual.

So here’s an example of the code-behind for a page that can leverage this method to filter the output before its sent to the browser.

public partial class FilterDemo : System.Web.UI.Page
{
  HtmlTextWriter _oldWriter = null;
  StringWriter _stringWriter = new StringWriter();

  protected override HtmlTextWriter CreateHtmlTextWriter(TextWriter tw)
  {
    _oldWriter = base.CreateHtmlTextWriter(tw);
    return base.CreateHtmlTextWriter(_stringWriter);
  }

  protected override void Render(HtmlTextWriter writer)
  {
    base.Render(writer);
    string html = _stringWriter.ToString();
    html = html.Replace("REPLACE ME!", "IT WAS REPLACED!");
    _oldWriter.Write(html);
  }
}

In the CreateHtmlTextWriter method, we simply use the original logic to create the HtmlTextWriter and store it away in an instance variable.

Then we use the same logic to create a new HtmlTextWriter, but this one has our own StringWriter as the underlying TextWriter. The HtmlTextWriter passed into the Render method is the one we created. We call Render on that and grab the output from the StringWriter and now can do all the replacements we want. We finally write the final output to the original HtmlTextWriter which is hooked up to the response.

A lot of caveats apply in using this technique. First, as I mentioned before, for large pages, you could be killing scalability and performance by doing this. Also, I haven’t tested this with output caching, async pages, etc… etc…, so your mileage may vary.

Note, if you want to call one page from another, and get the output as a string within the first page, you can pass your own TextWriter to Server.Execute, so this technique is not necessary in that case.

Tags: asp.net

A Fright on Mt Si

Haacked — Mon, 25 May 2009 03:28:19 GMT

Being that it’s a glorious Memorial Day Weekend up here in the Northwest, my co-worker Eilon (developer lead for ASP.NET MVC) and I decided to go on a hike to Mt Si where we had a bit of a scary moment.

I first learned about Mt Si at the company picnic last year, seen behind me and Cody in this photo. I remember seeing the imposing cliff face and thinking to myself, I want to climb up there. I imagined the view would be quite impressive.

Mt Si is a moderately strenuous hike 8 miles round trip with an elevation gain of 3100 feet taking you to about 3600 feet, according to the Washington Trails Association website. Given that it is a very popular hike and that this was a three-day weekend, we figured we’d get an early start by heading over there at 7 AM.

That ended up being a good idea as the parking lot had quite a few cars already, but it wasn’t full by any means. This is a picture of the trail head which starts the hike off under a nice canopy of green.

Right away, the no-nonsense trail starts you off huffing uphill amongst a multitude of trees.

Along the way, there are the occasional diversions. For example, this one won me $10 as the result of a bet that I wouldn’t walk to the edge of the tree overhanging the drop off.

When you get to the top, there’s a great lookout with amazing views. But what caught our attention is a rock outcropping called the “Haystack”, which takes you up another 500 feet. Near the base of the Haystack is a small memorial for those who’ve died from plummeting off its rocky face. It’s not a trivial undertaking, but I demanded we try.

Unfortunately, there’s nothing in the above picture to provide a better sense of scale for this scramble. In the following picture you can see some people pretty much scooting down the steep slope on their butts.

Once they were down, we set up and reached around two thirds of the way when I made the mistake of looking back and made a remark about how it’s going to be much more difficult going down. That started getting us nervous because it’s always easier going up than down.

It would have probably been best if I hadn’t made that remark because the climb wasn’t really that difficult, but introducing a bit of nervousness into the mix can really sabotage one’s confidence, which you definitely want on a climb.

At that point, the damage was done and we decided we had enough and started heading back down. Better to try again another day when we felt more confident. At that moment, a couple heading down told us we were almost there and it wasn’t so bad. Our success heading back down and their comments started to bolster our confidence to the point where I was ready to head back up, until I noticed that my shoe felt odd.

What I hadn’t noticed while climbing on the steep face was that my sole had almost completely detached from my hiking boot during the climb. Fortunately, Eilon had some duct tape on hand allowing me to make this ghetto looking patch job.

At this point I had a mild panic because I worried that the duct tape would cause me to lose grip with my boots on the way down. And frankly, I was pissed off as well, as I’ve had these boots for a few years, but haven’t hiked in them all that often. What a perfect time for them completely fall apart!

Fortunately, I didn’t have much problem climbing back down and we stopped at the first summit to take some pictures and have a brief snack.

Not having the guts today to climb the big rock, I scrambled up a much smaller one and got this great view of Mt Rainier in its full splendor.

The view from the top is quite scenic and using binoculars, I was able to check on my family back in Bellevue (joke).

Going back down was much quicker than the way up and we had a blast of it practically trail running the first part, until my other shoe gave out.

Guess the warranty must have run out yesterday. ;) Fortunately, Eilon, who was prepared with the Duct tape, also had all terrain sandals with him, which I wore the rest of the way. Next time, I think I’ll ditch the Salomon boots and try Merrells which other hikers I ran into were wearing.

Despite the mishaps, the hike was really a fun romp in the woods and I highly recommend it to anyone in the Seattle area to give it a try. Go early to avoid the crowds. I doubled my $10 in an over/under bet where I took 140 and over cars in the lot. We stopped counting at around 170 cars in the lot when we left.

This is one last look at Mt Si on our way back home. Eilon put together a play-by-play using Live Maps Bird’s Eye view (click for larger).

For more info on the Mt Si hike, check out the Washington Trails Association website.

Tags: Hiking