CodeBetter.Com - Stuff you need to Code Better!

Meffifying Windows Azure

Glenn Block — Fri, 03 Jul 2009 17:09:00 GMT

Magnus has been off doing some interesting work around integrating MEF with Windows Azure. The first question you might be asking is Why?

In his words, he set out to build a template for Windows Azure templates that:

enables testability
abstracts away storage
is extensible and easy to evolve during development

In the post he shows how to take the RoleManger and expose it through MEF, thereby making it pluggable. He then creates a mock Role Manager for use in his unit tests, thus removing the dependency on all the Azure infrastructure.

I am guessing this is the first of many posts to come on MEF and Azure.

For more, check out Magnus post here.

Move The Chair

Rod Paddock [MVP] — Fri, 03 Jul 2009 14:17:00 GMT

Last Sunday I took my family to Red Robin (a family style restaurant) after church. I requested a table with a high chair for my 3.5 yr old son. Upon bringing this high chair the hostess simply moved an adult chair to the side and put the high chair in its place. (See Image)

This is where the fun began…

For the next hour or so I watched countless customers, servers and managers limbo their way between these two chairs. A lot of them did it repeatedly, some even carrying trays full of drinks and/or food. Finally I provided evidence to to my family that I am 100% full on geek. I got excited about the insight that this little experience game me (to the point of taking pictures)

I was simply amazed that people would allow this “friction to continue” It was a simple problem in this case. Move the chair!

I find it interesting that it’s built into our DNA incur friction in our lives without really addressing it. In real life its called denial. I know the chair was not that big a deal but it did open my eyes to our basic nature. Our basic nature follows through to software development as well.

The example of the misplaced chair provides some interesting parallels when it comes to technical debt (which is a cause of software development friction) We all have technical debt in our applications and it’s in our nature to shimmy, shuffle, limbo, mambo, and shake around these issues whenever possible. It takes a constant vigilance to address technical debt. We must be active in our quest to remove technical debt. We must take the time to do two things:

Recognize the debt. In our case: The chair.
Remove the debt: In our case: Move the chair.

That’s it for this blog post. Just a simple observation brought to you by the staff and management of Red Robin :)

HandleUnknownAction in ASP.NET MVC – Be Careful

David Hayden — Thu, 02 Jul 2009 20:19:00 GMT

One of my team members was showing me some code in HandleUnknownAction based upon sample code in the online documentation. Per the documentation:

“The following example shows how to render views that do not have a matching ActionResult method. For example, if you have a Details.aspx view but no corresponding method exists that returns an ActionResult instance, the following example displays the Details view when a request to for the Details action is made on the controller. If there is no matching view, the error page displays a message.”

protected override void HandleUnknownAction(string actionName)

{

try

{

View(actionName).ExecuteResult(ControllerContext);

}

catch (InvalidOperationException ieox)

{

ViewData["error"] = "Unknown Action: \"" +

Server.HtmlEncode(actionName) + "\"";

ViewData["exMessage"] = ieox.Message;

this.View("Error").ExecuteResult(this.ControllerContext);

}

The only problem with this example is that it does not take into account attributes, like the ActionMethodSelectorAttribute, that can decide based on runtime information whether an action can fulfill a request. So, it may not be that the action does not exist, but that it is not supposed to respond to the request based on the context of the request.

For example, if I use the AcceptVerbsAttribute, an example of ActionMethodSelectorAttribute, to do something as simple as this on my HomeController:

[AcceptVerbs(HttpVerbs.Post)]

public ActionResult DisplayOnlyViaPost()

{

return View();

}

The DisplayOnlyViaPost View will not be displayed via the DisplayOnlyViaPost Action when someone types the URL in a browser ( HTTP GET Request ) – http://…/Home/DisplayOnlyViaPost.

Unfortunately, the HandleUnknownAction code above will kick in and notice that indeed the DisplayOnlyViaPost View does exist. Therefore it goes ahead and displays the view, undermining the use of our AcceptVerbsAttribute on the original action. Not good :)

Just something I noticed that you may want to be careful of if you are using similar code in HandleUnknownAction.

David Hayden

ASP.NET MVC Extensibility at Tampa MVC Developer Group

David Hayden — Wed, 01 Jul 2009 21:09:00 GMT

The Tampa ASP.NET MVC Developer Group is starting to kick it up a notch with this month’s meeting being about ASP.NET MVC Extensibility. We have concluded our introductory topics and are now focusing on more intermediate topics. This month’s meeting will hit more advanced topics, such as:

Creating Custom Controller Factories for Dependency Injection
Using Custom View Engines
Creating Custom ActionFilters for DataBinding, Validation, etc.
Using Custom ControllerActionInvokers
Developing Custom ActionResults for RSS Feeds, etc.
Building HtmlHelpers for Productivity

If you are interested in building your own opinionated version of the ASP.NET MVC Framework and have it bend to your will, you don’t want to miss this meeting.

In addition, we will also be going over last month’s homework assignment, which was to use the ASP.NET MVC AjaxHelpers to “ajaxify” an ASP.NET MVC Web Application. As a hint, you can find an answer in our last screencast, ASP.NET MVC Ajax Helpers - Ajax.ActionLink and Ajax.BeginForm – Screencast :)

Those completing the homework assignment will be included in a random drawing of Steven Sanderson’s Awesome book, Pro ASP.NET MVC Framework. Absolutely love Steve’s book. Huge thanks to Apress for sending us several copies to give away to our members.

Starting in August we will be working on voluntary group projects to strengthen our team development skills using ASP.NET MVC. Again, huge thanks to two of our sponsors for making this happen: Server Intellect and Unfuddle.

See you on Thursday, July 9 at the Microsoft Office in Tampa!

David Hayden

The Videos for NDC 2009 are online

Jeremy D. Miller — Wed, 01 Jul 2009 16:16:00 GMT

The links to the videos recorded for NDC 2009 talks are online and linked in the agenda pages here. My talks are at:

Lessons Learned from a Long Lived Codebase
Presentation Patterns for Composite Applications (I'd dearly appreciate feedback from folks on this talk)
Convention over Configuration applied to .NET
Software Design and Testability

Reporting against a domain model

pvanooijen — Wed, 01 Jul 2009 08:44:00 GMT

I have a love it /hate it relationship with reports. In many an application the reports are the main output. They are the prints which get filed or the invoices which have to ensure the ROI of the application. A good looking output rich with information does increase customer satisfaction. The downside is that fine tuning a report is tedious and requires quite different skills than coding. Building a system with a lot of reports can be pretty boring. So the tools better be good. Switching from Crystal Reports to sql server reporting services (RS) was quite a relief.

But RS still has two drawbacks. First is that it accesses data using plain sql. My application is built around a domain model using nHibernate to hide all db details. The domain model includes calculated properties which are to be included in the report. The calculations are done in the domain object. It would be repetitive and prone to error to repeat them in sql or reporting expressions. It would be quite a relief to report straight from the domain model itself.

The other drawback is that RS requires an ms sql server instance with installed reporting services. The actual data is just a sqlexpress instance and I don’t want to set up a full blown reporting services server.

New in Visual Studio 2005 were client side reports (rdlc’s). They are very much like RS reports (rdl’s) but can work with any data source and don’t require a sql reporting server instance. To include these reports in an app two client side report viewers are include in the .NET framework, one for WinForms and one for asp.net. This looked like the best of both worlds so I decided to give the winforms version a try.

For a good overview on the reports check the got reportviewer site. In name it supports VS 2008, but (some of ?) the samples are based on VS 2005. Moving to 2008 there has been a change in the namespace for the classes, after fixing the reference the samples run in VS 2008. How to work with a non sql datasource is covered but not that clear. Here I will describe my minimalistic approach to work with a domain model fed by nHibernate.

The domain model

The intent of the application is to print invoices. This class diagram describes the model

An invoice is for a contact, has a number of lines and an enumerated property. That is a number of values which contain a tax rate and have a clear name describing this value. Which gives the model additional value, something which would get lost when viewing the data as raw db data. An invoice has a reference to a contact and many Invoice lines. The (fluent) nHibernate mappings express this.

public class InvoiceMap : ClassMap<Invoice>
{
    public InvoiceMap()
    {
        WithTable("Factuur");
        Id(x => x.Id);
        Map(x => x.Number).ColumnName("Nummer");
        Map(x => x.OnDate).ColumnName("Dedato");
        Map(x => x.BtwPercentage).CustomTypeIs<BtwPercentage>();
        Map(x => x.Description).ColumnName("Inleiding");
        HasMany(x => x.Lines).KeyColumnNames.Add("idFactuur").Cascade.All();
        References(x => x.ForContact).ColumnName("idRelatie");
        Map(x => x.PrintIt).ColumnName("Printen");
    }
}

So we have a customtype, a hasmany and a references. Let’s see how these are used in the report.

The report

There are several ways to include a report in an application. The default when adding a new report (Add –> New Item –> Reporting –> report) is embedded. Which is clear and easy in deployment. When you need more flexibility you can switch to rdlc file or (back to) server side report.

The report needs a datasource. It will accept any datasource including an object datasource.

Add a new object datasource to the project and pick the Invoice domain object.

Just pick your POCO domain class. There is no need to set up any methods to get data, it is enough to have a model of the data.

This object data source is the datasource for your report. Having done so the domain invoice is available in the report designer. (To be precise the datasource has to be attached to a List, see the reportviewer docs for more information on that.)

A report is built from VB-like expressions. A simple one looks like this:

=Fields!Description.Value

Which displays the plain content of a property. But expressions can be more complicated. The report designer has a good expression builder which even includes intellisense. The next expression uses both the meaningful name and the value of my tax enumeration property in a string

="BTW " & Fields!BtwPercentage.Value.ToString & " "  & Fields!BtwPercentage.Value & "%"

The result is a string like “BTW Hoog 19%”.

Things get really interesting when displaying the contact information. In the domain model this expressed as

invoice.ForContact.AddressLine1

In the report the expression is quite similar

=Fields!ForContact.Value.AddressLine1

The subreport

The lines in the invoice are a one to many relation. The invoice datasource has a lines property but this property is not usable in the report. To display the lines I need a subreport. The subreport is based on invoice lines. After adding the InvoiceLine class as a datasource the subreport is drawn. It is a simple table displaying the lines.

A subreport is linked to its parent report using parameters. In my case an invoice and its lines are linked through a parameter named FactuurNummer.

All of this works just as in “traditional” reports. In the main invoice report this parameter is assigned the value of the invoice number

Check the gotreportviewer site for further information.

Binding the data to the report

Now the reports are ready. The next step is to get some real life data into them.

It takes code to bind real data to a report and display the result in the viewer. Also the subreport has to be fed with data. To wrap things up I have built a helper class which takes a reportviewer component and a list of invoices and wires them up.

internal class InvoiceReporter
{
    internal InvoiceReporter(ReportViewer viewer, IList<Invoice> reportData)
    {
        _reportData = reportData;
        _viewer = viewer;
    }
 
    private readonly IList<Invoice> _reportData;
    private readonly ReportViewer _viewer;
 
    internal void ShowReport()
    {
        _viewer.LocalReport.ReportEmbeddedResource = "Gekko.Administratie.FactuurPrinten.Invoice.rdlc";
        _viewer.LocalReport.DataSources.Add(new ReportDataSource("Gekko_Administratie_DomainModel_Invoice", _reportData));
        _viewer.LocalReport.SubreportProcessing += SubreportProcessing;
        _viewer.RefreshReport();
    }
 
    private void SubreportProcessing(object sender, SubreportProcessingEventArgs e)
    {
        var invoiceNumber = e.Parameters["FactuurNummer"].Values[0];
        var invoice = _reportData.First(fakt => fakt.Number == invoiceNumber);
        e.DataSources.Add(new ReportDataSource("Gekko_Administratie_DomainModel_InvoiceLine", invoice.Lines));
    }
}

The code requires some explanation. The constructor should be clear, it receives viewer and data.

The ShowReport method does the real work. The first line loads the report in the viewer. The second line loads the report data. This line is tricky. The name of the ReportDataSource has to match exactly the name of the datasource class used as model. The Invoice class name is Gekko.Administratie.DomainModel.Invoice which gets formatted as Gekko_Administratie_DomainModel_Invoice. This required magic string is not clear from any documentation, it took me quite some time to find out how essential it was.

All these datasource properties in the report definition are confusing. The model of the data is saved in the report definition. Changing the datasource to another “dataset” will also lead to errors at run time. The report will complain about the former datasources. To get rid of the error messages you have to check the report defintion by hand. The report definition file (.rdlc) is plain xml. The last part contains dataset definitions. What helps is just deleting the unused nodes by hand.

Upon rendering a subreport an event is fired. In the third line of ShowReport a handler is attached. This SubReportProcessing method feeds the subreport with data. From the parameters the number of the invoice is read. Which is used to find the corresponding invoice. The datasource for the subreport is set just like the datasource of the main report. Again spelling the name of the reportdatasource right is essential. The data is the Lines property. Which is an IList of InvoiceLines.

Putting it all together

With all pieces in place it’s time to get some results. I have a windows form with a reportviewer on it.

var repo = new InvoiceRepository();
var reporter = new InvoiceReporter(reportViewer1, repo.ListPrintableInvoices());
reporter.ShowReport();

This reads the invoices from the repository and sends them with the reportviewer to the helper class.

There is one more thing to watch. The repository is an nHibernate repository. By default nHibernate lazy loads related objects, like the Contact and the Lines of the Invoice. The contact and the invoicelines will not be read from the database until they are actually used. This default setting has some serious drawbacks.

In the first place the reportviewer‘s datareader does not operate very well (at all ?) with nHibernate’s proxies. In the report all the contact fields will contain error messages. The subreport on the lines will work. But still the lines are not read from the database until rendering the sub-report. So for this lazy loading to work the report has to be run against a dataset with an open dataconnection. That’s bad, I try do do as little as possible, and most likely not presenting a report, with an open connection. A far better scenario would be to fetch all data in one go, close the connection to the database and render the report after that. This can be done by eagerly loading the Contact and Lines properties and disposing of the session before returning the data.

In the nHibernate API the fetch mode can be set on a property level. The code in the repository takes care of fetching all data in one big snapshot.

public IList<Invoice> ListPrintableInvoices()
{
    using (var session = SessionFactory.GetFactory.OpenSession())
    {
        var query = session.CreateCriteria(typeof(Invoice));
        query.AddOrder(Order.Asc("Number"));
        query.Add(Restrictions.Eq("PrintIt", true));
        query.SetFetchMode("Lines", FetchMode.Eager);
        query.SetFetchMode("ForContact", FetchMode.Eager);
        return query.List<Invoice>();
    }
}
 

The repository reads all data from the database and closes the connection. After that the collected data are input to the reportviewer. Which now works completely as hoped for.

I have my report complete with printing and export functionality and am still working with my beloved domain objects.

This way reporting can almost be pure fun.

Check out Virtual ALT.NET

aaronjensen — Tue, 30 Jun 2009 23:14:00 GMT

Several months ago a few people started hosting a Virtual ALT.NET meeting right here on the information super highway. I've been to a few and they're always packed with information and good conversation. It's a great way to learn new things or show off cool stuff you've been working on. If you haven't been to one, I'd recommend you check one out. There are two great looking meetings hosted by Ryan Svihla coming up soon:

Development with Castle Project with Ryan Svihla July 1 and July 8
Ryan will be doing a two part series on the Castle Project. Mark your calendar for some Castle Project fun.
Ryan Svihla has been working as a C# developer Farm Bureau Bank in San Antonio since September 2007. Before that he worked as a Consultant in Lincoln, NE for 3 years, where he had working experience with Php, some Perl, Python and of course C#. Attemping Agile since early 2008 as an eager student with a focus on making programming more useful and relevant for the end user.

IoC and Dip through Castle Windsor
Ever wonder what acronyms like IoC and Dip mean? If you know what they mean do you wonder why anyone would use them in code? This talk aims to deal primarily with those two questions through the use of Castle Windsor IoC container. Intermediate level C# material with a couple of more advanced demos at the end for fun and pleasure.
Central Daylight Time
Start Time: Web, July 1, 2009 8:00 PM UTC/GMT -5 hours
End Time: Web, July 1, 2009 10:00 PM UTC/GMT -5 hours
Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Web Development with Castle Monorail, Active Record and Brail view engine
Have a look at the first popular MVC .Net based web framework. Also will be covering persistance with ActiveRecord, and view templates using Brail. Bonus, will demo a plugin framework for building CMS like applications.
Central Daylight Time
Start Time: Web, July 8, 2009 8:00 PM UTC/GMT -5 hours
End Time: Web, July 8, 2009 10:00 PM UTC/GMT -5 hours
Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Hope to see you all there!

Making your code easier to understand context/specification style unit tests

Ian Cooper — Mon, 29 Jun 2009 04:08:00 GMT

When we started our current project we did not use context/specification style testing, instead we used testcase-per-class with a four-phase test model (also known as arrange-act-assert). Although we followed story-test driven development (STDD) we were not explicitly Behavior-Driven Development BDD when we set out. Over time I began to see the path between STDD and BDD we shifted toward a BDD approach.

That is really background for what comes next. As part of this shift we started writing more context/specification style tests. We did not re-write our existing testcase-per-class fixtures (then and still now I would see the possibility of using multiple test organisational patterns within a project; some of our tests still seem better with testcase-per-class). Recently we have had new developers join the team. One interesting, if anecdotal, observation has been that those developers find it easier to use tests as a source of documentation for what the software is doing when they are in the context/specification style. This is a useful observation, because the rest of the team is able to supply much of the context for testcase-per-class fixtures by virtue of having worked on the codebase and so do not see the lack so easily.

To build a maintainable system you have to build one where the behavior can be understood. Tests offer this promise, but often we have found that tests failed us, because the tests were no easier to comprehend than the code under test. Context/specification style tests seem to be better on this account. Of course there is no silver bullet in software. You still have to put the effort into making your context/specification tests readable, but the form seems helpful toward achieving that goal.

I'll try to post more on my experiences with BDD style approaches as they come out. I would also recommend that anyone interested in the topic look at the RSpec book from the Pragmatic Programmers which offers a great overview of the topic.

Fighting Fabricated Complexity

Patrick Smacchia — Sun, 28 Jun 2009 10:12:00 GMT

Normal 0 21 false false false FR X-NONE X-NONE

During my interview on Code Metrics by Scott Hanselman’s on Software Metrics, Scott had a particularly relevant remark.

Basically, while I was explaining that long and complex methods are killing quality and should be split into smaller methods, Scott asked me:

looking at this big too complicated method and I break it up into smaller methods, the complexity of the business problem is still there, looking at my application I can say, this is no longer complex from the method perspective, but the software itself, the way it is coupled with other bits of code, may indicate other problem…

Software complexity is a subjective measure relative to the human cognition capacity. Something is complex when it requires effort to be understood by a human. The fact is that software complexity is a 2 dimensional measure. To understand a piece of code one must understand both:

what this piece of code is supposed to do at run-time, the behavior of the code, this is the business problem complexity
how the actual implementation does achieve the business problem, what was the developer mental state while she wrote the code, this is the implementation complexity.

Business problem complexity lies into the specification of the program and reducing it means working on the behavior of the code itself. On the other hand, we are talking of fabricated complexity when it comes to the complexity of the implementation: it is fabricated in the sense that it can be reduced without altering the behavior of the code. As an illustration here is a super/giant/complex method found inside the .Net Framework implementation System.Windows.Forms.DataGridView.GetClipboardContent():

GetClipboardContent() is made of around 300 Lines of Code and has a ILComplexity equals to 192. GetClipboardContent() does not have drastic performance requirement. As a consequence I don’t see any justification for not refactoring this massive method into smaller ones and maybe even a small classes hierarchy that could help in implementing the enormous switch/cases. Doing so would certainly discard a lot of fabricated complexity.

Fighting Fabricated Complexity with Simple Code Metrics

The simplest way to limit fabricated complexity is to abide by simple code metrics thresholds. This is why one of the default CQL rule proposed by NDepend is the following one:

// <Name>Quick summary of methods to refactor</Name>

WARN IF Count > 0 IN SELECT TOP 10 METHODS WHERE

( NbLinesOfCode > 30 OR

// http://www.ndepend.com/Metrics.aspx#NbLinesOfCode

NbILInstructions > 200 OR

// http://www.ndepend.com/Metrics.aspx#NbILInstructions

CyclomaticComplexity > 20 OR

// http://www.ndepend.com/Metrics.aspx#CC

ILCyclomaticComplexity > 50 OR

// http://www.ndepend.com/Metrics.aspx#ILCC

ILNestingDepth > 4 OR

// http://www.ndepend.com/Metrics.aspx#ILNestingDepth

NbParameters > 5 OR

// http://www.ndepend.com/Metrics.aspx#NbParameters

NbVariables > 8 OR

// http://www.ndepend.com/Metrics.aspx#NbVariables

NbOverloads > 6 )

// http://www.ndepend.com/Metrics.aspx#NbOverloads

Fighting Fabricated Complexity with Abstractions

Another popular way to limit fabricated complexity is to hide implementations behind some interfaces. Even though interfaces are not contracts, an interface itself often conveys enough information to make its purpose understandable. For example the IDisposable pattern is a complex topic but still, the IDisposable interface present only one method and we at least understand that it is an indication that some unmanaged resources must be released somehow. When a piece of code relies on an interface, at code review time, the interface relieves the developer from the burden of mastering every details of the implementation(s) hided by the interface. This is in essence the Liskov Subsitution Principle.

NDepend provides 2 ways to asses if and where your program should use more abstractions. First NDepend proposes the Robert C Martin metrics about Abstractness vs Instability. The idea is that the more a code element of a program is popular, the more it should be abstract. Or in other words, avoid depending too much directly on implementations, depend on abstractions instead. By popular code element I mean an assembly (but the idea works also for namespaces and types) that is massively used by other assemblies of the program. Abstractness of a group of types (like an assembly) is the ratio NbAbstractTypes / NbTotalTypes. There are debates about how these metrics should be computed and we plan in the future to make Robert C Martin's metrics more flexible but still, you get the point, it is not a good idea to have concrete types very popular in your code base. This provokes some Zones of Pains in your program, where changing the implementations can potentially affect a large portion of the program. And implementations are known to evolve more often than abstractions.

The second way to assess if more abstractions is needed is to rely on the Level metric. I won’t detail here this metric and its usage because I have already done it in the post Layering, the Level metric and the Discourse of Method. The idea here is that using more interfaces decrease the overall Level value of code elements (classes/namespaces). Thus, if many classes and namespaces have a high Level value (> 12), it means that you have a long stack of concrete layers sitting above each others. Introducing abstractions is then a good idea to split such long stack and benefit from the interface simplification describe above.

Fighting Fabricated Complexity with Immutability

A common source of fabricated complexity is mutable states. The human brain is not properly wired to anticipate what is really happening at run-time in a program. While reviewing a class, it is hard to imagine how many instances will simultaneously exists at runtime and how the states of each these instances will evolve over time. This is actually THE major source of problems when dealing with a multi-threaded program. If a class is immutable (meaning if the states of all its instances objects don’t change at runtime once the constructor is called) its runtime behavior immediately becomes much easier to grasp and as a bonus, one doesn’t need to synchronize access to immutable objects. For more information, I wrote about the benefits of immutable types and how NDepend can help in statically verifying immutability.

Fighting Fabricated Complexity with minimal Coupling

When trying to re-engineer/understand/refactor a large piece of code (I mean something made of dozens of classes like a big namespaces or an assembly), the difficulty is directly proportional to the coupling between considered code elements. Both these following graphs are made of dependencies between 23 classes, one with 53 edges and the other one with 175 edges: which one would you prefer to deal with?

While clear componentization is certainly the best way to fight against entangled/spaghetti code, keep in mind that using abstractions is also a good way to limit the over coupling-overhead. Indeed, if an interface has N implementations, then relying only on the interface is virtually like depending on the N underlying classes, except that from the static dependency point of view, you actually rely on only one type.

Could Fabricated Complexity be Measured?

There are plenty of other sources of fabricated complexity and I estimate that the 4 quoted ones are certainly the big 4 culprits. One could arguably appends also the ratio of code coverage + the number of automatic tests in the list, since a clean tests suite certainly forces the code to be better designed, simpler and more maintainable.

Interestingly enough, all these potential sources of problem can be controlled through NDepend. Could we find a formula that might process all these data to finally spit a number (a score) to measure the Fabricated Complexity? This approach is implemented in tools like Struture101 in the Java world that comes with a dedicated XS metric to measure fabricated complexity. Another example is the maintainability index range in VisualStudio that spits a number in the range 0-100. This number is linearly computed from potential source of fabricated complexity such as NbLinesOfCode, CyclomaticComplexity… and being lower than 20 is bad.

Such metrics are certainly interesting but I have a few caveats. The inventors of these formulas claim that in a certain range of value the code is crappy. The fact is that the value spitted doesn’t have any associated dimension. If a method has 120 lines of code and a Cyclomatic Complexity of 50, does it helps to add 120+50 and says that 170 something (something here is undetermined) is a bad thing? From the 2 values (120 Loc and 50 CC), isn’t it already obvious that the method should be splitted? And what about a method with 169 lines of code and no cyclomatic complexity (such as the massive Windows Form InitializeComponent() method)? I don’t think that such a method is as bad as the previous one and still they both measure 170 something.

Let's take for example the more and more popular CRAP metric that helps to detect crap code. The idea is that crap code is complex methods and poorly covered by tests.The proposed formula is

CRAP(m) = comp(m)^2 * (1 – cov(m)/100)^3 + comp(m) where comp(m) is the cyclomatic complexity of method m, and cov(m) is the test code coverage provided by automated tests.

By mixing it up, it is hard to understand the meaning of an estimation of the fabricated complexity value and worst, it makes harder to predict which refactoring will affect positively the value of the metric. This is the reason why so far in NDepend, we favored a more linearly independent/vectorial way to assess the exact causes of problems. To detect complex methods poorly covered with NDepend one just needs the following CQL rule and adjust thresholds at whim: Normal 0 21 false false false FR X-NONE X-NONE MicrosoftInternetExplorer4

WARN IF Count > 0 IN SELECT METHODS WHERE CyclomaticComplexity > 10 AND PercentageCoverage < 90 ORDER BY NbLinesOfCode DESC

The CQL result panel provides all needed information to work with the crap methods detected and lets jump directly into VisualStudio to view and edit culprit methods:

Alternatively, the CQL flexibility lets correlate these metrics with other kind of relevant information, like for example detecting crap methods added recently to the code base (or also crap methods where ChangesObjectState, ChangesTypeState, CodeWasChanged, is declared inside or outside certains assemblies/nalmespaces/classes, not IsGeneratedByCompiler, is used by a certain code element, is using certain code elements, BecameObsolete...etc):

Normal 0 21 false false false FR X-NONE X-NONE MicrosoftInternetExplorer4

WARN IF Count > 0 IN SELECT METHODS WHERE WasAdded AND CyclomaticComplexity > 10 AND PercentageCoverage < 90 ORDER BY NbLinesOfCode DESC

Also one can do experiments and compose metrics at whim:

Normal 0 21 false false false FR X-NONE X-NONE

This situation might evolve in the future. But so far I would prefer to avoid giving the NDepend user the feeling that fighting fabricated complexity is only a matter of reducing the value of a naked index (i.e an index without any dimension). Things must remain close to the code itself: if I have a method with 15 parameters it seems much more meaningful than having a class with a maintainability index equals to 18 or a CRAP value of 33.21.

Are you using these formulas? What’s your opinion on this?

MSpec... for Boo!

aaronjensen — Sun, 28 Jun 2009 05:41:50 GMT

Jeffery Olson (a fellow Eleutian guy) has put in some good effort to create a very readable, very clean MSpec DSL for Boo. You should give it a gander.

Talking conventions with the Hanselman

Jeremy D. Miller — Sat, 27 Jun 2009 15:22:00 GMT

Scott Hanselman posted a podcast he & I recorded at NDC last week. We nominally talked about applying conventions and opinionated software to .Net, then we went off on every possible tangent that presented itself.

The “Anti-For” Campaign

Matthew.Podwysocki — Fri, 26 Jun 2009 04:35:26 GMT

Recently, there has been an effort launched called the “Anti-If Campaign” in which they deride the use of if statements and instead, focus on Object Oriented Principles in order to create more flexible designs. Now certainly, I have a sympathetic ear to this cause as I’ve seen code that literally walks off the side of the screen due to nesting of if statements. Pattern matching to me, especially at the top level of the function is actually quite beautiful in a way, such as the implementations in Haskell:

-- Haskell
lucas :: Int -> Integer
lucas 0 = 2
lucas 1 = 1
lucas n = lucas (n - 2) + lucas (n - 1)

And in Erlang, this also holds true:

% Erlang
-module(lucascalc).
-export([lucas/1]).

lucas(0) -> 2;
lucas(1) -> 1;
lucas(N) -> lucas (N - 2) + lucas (N - 1).

Simple, easy to understand and best of all, no if statements. But, instead of focus on this debate, I’d like to propose another which strikes closer to this functional programmers heart, the “Anti-For Campaign”. This is simply to say that we should create and use composable functions instead of explicit for loops. This is actually an old post I had written months ago and until now had been unfinished, but now with some inspiration, it’ll finally be done.

What and Why?

Before you throw all sorts of questions asking what and why, let me instead ask a question. When you’re writing a loop, ask yourself the question, “What am I accomplishing in this loop?” Chances are, it might be one of the following:

Query (Map, Filter, etc)
Aggregation (Sum, Count, etc)
Perform some side effect

If you’re doing more than one of those in a single loop, then well, you’re probably doing too much. In fact, Martin Fowler’s Refactoring site has a refactoring called “Split Loop” which would solve that issue. It is better for future refactorings and readability if we keep those loops pointed to do one thing, and one thing only. Better yet, we could rid ourselves of that loop altogether, and that’s what we’ll focus on here.

Looking at first two bullet points, you’ll notice most of LINQ is indeed built around those two to be able to query data as well as aggregate. The final bullet point, we perform some sort of side effect, perhaps writing to a file, printing to the console, or even perhaps sending messages.

So, what’s my problem with them? My problem is that it focuses more on the How instead of the What. Let’s look at a quick example down below of what I mean. First, we’ll attempt to find all prime numbers under 100 using C# as an example language. First in the How:

static bool IsPrime(int i) {
    var lim = Math.Sqrt(i);
    
    Func<int, bool> check = null;
    check = j =>
        j > lim || (i % j != 0 && check(j + 1);
        
    return check(2);
}

static void Main(string[] args) {
    var numbers = Enumerable.Range(1, 100);
    var output = new List<int>();

    foreach(var number in numbers)
        if(IsPrime(number)) output.Add(number);
}

The problem is that if we then want to compose another operation, well, it’s really hard to do inside of these for loops. We’re too focused on the how at this point. Instead, getting to know generics and lazy evaluation, in .NET 2.0 and beyond, we were able to write generic functions to take advantage of some functional constructs. This will give us a more declarative style that we can now focus on the what.

static IEnumerable<T> Filter<T>(
    this IEnumerable<T> items,
    Func<T, bool> predicate) {
    
    foreach(var item in items)
        if(predicate(item))
            yield return item;
}

static void Main(string[] args) {
    var numbers = Enumerable.Range(1, 100);
    
    var primes = items.Filter(x => IsPrime(x));
}

Of course we realize that LINQ already has such constructs built in, so we could rewrite the entire code above in just one statement.

var primes = Enumerable.Range(1, 100)
    .Where(x => IsPrime(x));

And what’s better is that it is composable that we could do other operations such as aggregations (sum, count, etc) without much additional code:

var primesCount = Enumerable.Range(1, 100)
    .Where(x => IsPrime(x))
    .Count();

And there you have it. We not only have a query, but also an aggregation. Try doing that with those non-composable loops! Justin Etheredge has a nice writeup as well recently on the subject.

Coping Strategies

Functional languages tend to deemphasize the use of such constructs. In fact, Haskell has neither a for loop nor a while loop, and languages such as F# and OCaml have limited support for such constructs in terms of no break and continue. We tend to look at those two in particular with suspicion due to the fact that it cannot return a value and instead it mutates state in some fashion. With that in mind, how do we cope with the fact that those aren’t available to us? Above I showed a basic concept of a filter instead of an explicit loop, but what about some other considerations?

Some things we might want to consider with some links to some of my previous posts on the subject:

Explicit Recursion
Transforming each item (Map/Select & SelectMany in LINQ)
Selecting items (Filter/Where in LINQ)
Aggregating Data (Folds/Aggregate in LINQ)

In fact, many times that people could consider using explicit recursion could instead use a fold to aggregate the data which then cuts out the issue of tail call optimization. By truly understanding the goals of LINQ as well as the concepts of functional programming, we can realize that most of the looping that we do can indeed be replaced by the above, outside of side effects of course.

Now What About Those Side Effects?

Is there a place where we draw the line and say that explicit loops are ok? Eric Lippert was recently asked about the reasoning of the lack of the ForEach extension method on IEnumerable<T>. His response was that he was philosophically opposed to such an endeavor as the whole approach is to cause a side effect. As IEnumerable<T> collections are immutable, he doesn’t believe it makes as much sense because you wouldn’t be side effecting the collection itself. Not only that, but introducing closures can complicate object lifetimes and all sorts of potential reference issues.

What about me? I understand his concerns, and in C#, I can certainly see where he is coming from. However, in F# we have such constructs readily available to us in the iter and iteri functions as shown below in F# Interactive:

> let flip f y x = f x y
- [1..10]
-   |> List.map((*) 2)
-   |> List.filter(flip (%) 3 >> (=) 0)
-   |> List.iter(printfn "%d");;
6
12
18
> [1..10]
-   |> List.map((*) 2)
-   |> List.filter(flip (%) 3 >> (=) 0)
-   |> List.iteri(printfn "%d\t%d");;
0       6
1       12
2       18

Outside of logging, writing to the console and such are rare in functional programming, so once again, I can certainly understand the concern.

Conclusion

I hope by going through some basic scenarios that we will indeed question our code the next time we see that we are writing that explicit loop with a for and a while. With a tool chest filled with such functions as transforming every item, to filtering content, to aggregating data and so on, we can realize that we can create composable solutions instead of creating mutable collections or mutable variables and aggregating to them which are not as much. So, come and join me in the “Anti-For Campaign”.

Becoming a better developer, or “How to sidestep a question”

Kyle Baley — Thu, 25 Jun 2009 22:01:48 GMT

This post is over a week in the making. I attended AltNetBeers #9 last week in London which is making a quick rise in my list of top developer-related events, big or small. And that’s not just because I got one of the best compliments ever from someone who claims his non-IT friend reads a single IT-related blog (take that, Hanselman!). Anyway, it’s nice to know I’m reaching my target audience.

The event was not altogether removed from what Sebastien outlined at the one I attended last year which shows a degree of vision to which I only aspire. We spent a couple of hours talking on a topic of choice (as voted by attendees), then socialized until the place closed down at the unreasonable hour of 11pm. I’m told much merriment ensued afterward but, after posing for a picture, I made my way back to the hotel.

The topic at hand, and I’m paraphrasing: How do we become better developers today, and how do we create them tomorrow? Nice and subjectively vague, just how I like my open spaces. The runner up topic, brownfield applications, seemed suspiciously planted so I threw my weight behind the one we eventually chose.

Kinda concerned at how often this image is relevant to my posts

Conversation ran the usual gamut from apprenticeship programs (and thanks to Neil for the extensive historical thesis into the concept) to showing passion to defining professions to just going out there and doing it. My position through most of it was to focus more on the non-technical skillz, to the point where I think I pooh-poohed actual book learning more than I should have. If you decided, based on my advice, to drop out of university, I’d advise you to politely grovel to your dean to get back in. It’s still important. But take a psyche course or two when you go back.

Alas, we didn’t offer as much practical advice to the first part of the question as I would have liked so I’m expanding on the closing comment I made. It was essentially: Follow your instincts.

Now, there’s a danger with throwing out a broad statement like this because I made it in a very specific context. Namely, in a crowd of people who had taken time out of their lives to come to an IT-related event (albiet, one with more social activities than your average code camp; it *is* held in a pub after all). These are people who, at some level, have made a conscious decision to improve themselves on their own time. They could have had much better things to do but their instincts told them that this was an event that was more important than attending the new West End show, Deliverance: The Musical. (Side note: I know I just made that up now but MAN, the Duellin’ Banjoes scene would rock live; I suspect they’d lose a lot of people at intermission though.)

A lot of people have questions like, well, how do I make myself a better developer. The fact that they are asking those questions is the first step. The next step is to trust that they can muddle their way through their own answer. Because as we proved last Wednesday night, the same answer may not work for everybody. Me? I’m not much of a book learner. Nor, ironically, do I get much out of scanning blog posts (other than creating my own internal Google index so I know where to look later when the topic becomes relevant to me).

It’s dead simple to make yourself a better developer nowadays. Resources are plentiful and mostly free. And the ones you pay for, like conferences and training courses, are easy enough to rank with a little research (e.g. JP’s Nothin’ But .NET, Oredev, NDC). All it takes is a focused effort to make yourself better and to put some thought into how you go about it. You alone know which bloggers out there speak the same language you do. And you alone know which technologies will help you in your daily life and which ones interest you. Yes, SharePoint developers are in high demand but is it a product you want to learn and spend eight hours a day working with? (That’s not meant to be facetious actually, stop laughing.)

There will be much second-guessing and wondering which way you should go. And that’s fine. I won’t pretend I’ve made the right decisions all the time but like the software we’re supposed to be writing, most decisions are reversible. Humming or hawing about a contract in Dubai? Go for it. If you don’t like it, go home when it’s done. Worried about being away from your family that long? Well, you have some soul-searching to do but once you’ve made the decision, move on to the next one.

Was it the right decision? In my experience, unless you have direct and obvious evidence to the contrary, the answer is always yes.

Like I said, the topic is generic enough that pretty much everyone’s opinion will be valid, likely because it has worked for them personally. The underlying message I want to get across isn’t so much how to improve yourself (because you already know how to do it), but rather, if you’re in the London area, AltNetBeers is not to be missed. Many thanks to Neil, Toby, Scott, Andrew, Chris (both of them), Paul, Christian, and Lorenzo for the conversations and, of course, to the incredibly humble and reticent Sebastien Lambla for organizing and chairing the event, and also to Neil Robbins for the geek lunch two days later.

Kyle the Well-Shod

Ubiquitous Language in Product Development

Dave Laribee — Wed, 24 Jun 2009 21:15:00 GMT

Hi there.

I got this question from a fellow named Martin today:

Real simple ... most of the discussion on DDD surrounds the design in terms of the ubiquitous language.

What if you are developing a product that is designed to be a tool for a certain industry, say marketing, but the processes and terminology all differ according to the marketing company that is in discussion.

To be specific, one company may use the term Client and the other Customer?

How do you accommodate the difference in terms? Do you code to a common denominator or do you just pick one and go?

Great question; in product development we're often trying to make a general solution that solves the common problems of customers that have different languages. There's no way around that.

First, find those elements that are universal. Question if there's really a need for all of this customization or if your clients are OK with calling a client a customer. Having a ubiquitous language that can be used end-to-end, from developer to end user, is extremely valuable and kind of the original point.

For variable terms you should, in my estimation and experience, form an internal ubiquitous language that's adhered to across product management and software development. In cases like this, you'll can potentially also apply DDD guidance to a generic subdomain that handles terminology. This domain usually goes by the name "localization," only slightly extended to allow clients to change terminology. This is exactly how we handle the differences in terminology (silly) between agile methods like Scrum and XP in our product at VersionOne. We simply have a set of components that handle the change between backlog item and story, iteration and sprint, etc.

Providing Safe Alternatives

Matthew.Podwysocki — Wed, 24 Jun 2009 13:08:19 GMT

When I was reading through Real World Haskell, I was struck several times by the mention of providing safe function alternatives. The idea is to provide a function that in all cases returns a value as well as the one which is meant to accept valid input and throw exceptions should that contract be violated. There is a real performance consideration to be taken into account as a function which repeatedly throws exceptions as logic will tend to overwhelm a system and slow it down significantly. Imagine if you will an application which reads a large directory to check each file for an X509 certificate, whether it has one or not, and it throws an exception if one is not present. The problem of course is there is no way to determine whether a file was signed at the time using the .NET class without resorting to P/Invoke (my favorite).

Recently on Twitter, there was talk finally of the inclusion of Guid.TryParse in .NET 4.0, yet somehow, Enum.TryParse has not been addressed yet (gee, we’re consistent here). The idea is to provide a way of determining whether the string is a Guid format without throwing an exception, and instead returning a flag indicating success and an out parameter with the value which is set to the value if there is success, else the default value. Before we dig any further, let’s look at some terminology.

Partial Versus Total Functions

To get some terminology straight, let’s talk about partial functions versus total functions. Partial functions are those functions which only return values for a defined subset of valid inputs, as throwing an exception is not considered a return value. On the other hand, functions that return valid results over the entire input are considered to be total functions. Let’s look at a quick example of what that means in F# interactive, the first being a partial function and the latter being a total function.

> open System.Linq;;
> ([] : int list).First();;
System.InvalidOperationException: Sequence contains no elements
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at <StartupCode$FSI_0003>.$FSI_0003.main@()
stopped due to error
> ([] : int list).FirstOrDefault();;
val it : int = 0

We see by the above that the First() function is a partial function because it only has defined behavior for non-empty collections, whereas FirstOrDefault() has defined behavior for empty lists and returns the default value for the given generic type. When we’re writing our code, we better know which one we’re dealing with as default values can cause unintended consequences. Without proper tests around these calls, this can be hard to manage.

Mitigating these partial functions with calls to such things as the Count() extension method is not necessarily the answer either. When dealing with non-lazy lists such as an array or a List<T>, then such things as the number of items in our collection is well known and part of the state. In the case of the lazy sequence, this is not the case, so any call to Count() would cause an evaluation of all items in the list, which could lead to unintended consequences. So, if we’re chaining together 4 partial functions together, we could be doing something wrong. All in all, total functions should be preferred usage, but of course within reason.

Can We Do Better Than Try?

Going back to one of the original points, can we do better than the standard Try (Parse/GetValue/etc) pattern that is pervasive throughout .NET code? After all, it tries to do two things at once, and the success flag can be missed if one so chose. F# decided to take a slightly different angle to this problem by returning the success flag and the value as a pair tuple. Such an example would be like this memoize function below:

open System.Collections.Generic

let memoize f = 
  let t = new Dictionary<_,_>()
  fun n ->
    match t.TryGetValue(n) with
    | (true , value) -> value
    | (false, _    ) -> 
        let res = f n
        t.Add(n, res)
        res

What we’re doing above is calling the TryGetValue method which takes a key and returns a success flag as well as the out parameter for the value (set if there is success). F# automatically creates a pair tuple for us which we can then pattern match against to take appropriate action. Unfortunately, for this go around at least, C# does not treat tuples as first class data types, so a solution like this exclusive to F# at this time. Overall, a much cleaner design, but maybe there’s another option?

Maybe there’s an Option…

Instead of dealing with the clumsiness of the above code, I would much rather solve this in a clean way using the F# Option type. This allows us to definitively specify whether we have a value or not. Oh great you say, another version of null. Not quite, as sometimes null can be a value… More to the point, we can have a universal solution to having a value without having to resort to the Nullable<T> or reference type null-ness nonsense. Simply put, the Option type is no more than the following:

type Option<'a> =
  | Some of 'a
  | None

You’ll notice two constructors for the Option type, the Some which takes a value, and the None which takes no parameters. Now, using this to extend the Dictionary class much like the F# Map class, we can add a function which returns an Option type whether we have a value or not.

type System.Collections.Generic.Dictionary<'k,'v> with
  member this.Lookup(k:'k) =
    match this.TryGetValue(k) with
    | (true , value) -> Some value
    | (false, _    ) -> None

Then we can rewrite our memoize function to utilize this in a cleaner fashion using the Option type for pattern matching purposes to show our true intentions of whether we have a value or not.

let memoize f = 
  let t = new Dictionary<_,_>()
  fun n ->
    match t.Lookup(n) with
    | Some v -> v
    | None   -> 
        let res = f n
        t.Add(n, res)
        res

Much cleaner and shows better intent on our part.

Conclusion

How far do we take this? After all, making full functions everywhere sometimes isn’t feasible as we want to constrain our system. Adding both partial and full implementations might just clutter up our entire API. With this comes balance, but there are certainly places where this comes into play such as parsing values, querying values from collections and so on. Could Code Contracts in .NET 4.0 help us here in terms of statically verifying that we’re not failing on a chained partial function? We’re not quite finished here with this discussion as I’ve yet to cover the many values of null.

NDC 2009 Wrapup

Jeremy D. Miller — Wed, 24 Jun 2009 12:58:00 GMT

I’m on my way back from the Norwegian Developer’s Conference in Oslo this past week. I had a fantastic time meeting some new people, hanging out with friends, interacting with a couple heroes, and seeing a little bit of Norway. Special thanks to Anders Norås for inviting me to NDC. I had a blast with the Bergen gang: Mark Nijhof, John St Clair, and Jon Arild Tørresdal. I got to put a face to Andreas Öhlund who is developing the StructureMap integration with WCF and nServiceBus.

I think all the talks were recorded and are going to be posted to the NDC website for download. I’ll blog the links as soon as they go up. I gave 8 talks in 4 days across 2 cities. I’m not sure I want to be one of those professional speakers that do that a couple dozen times a year. I had an awesome crowd for my “Design and Testability” talk that I appreciated. I had a comparatively small crowd for my “Conventions over Configuration” talk, but I had a lot of good questions afterwards that probably made that my favorite session of the week. Most importantly, I presented some preliminary design patterns work on the “Screen Activation Lifecycle” from my book and I thought it went over pretty well (you can see a little preview of it on Ward Bell’s blog here).

I got even more book ideas from Jonas Follesø’s talk. <bias>Apparently there are some crazy people out there who think it’s not entirely insane to do “View First” navigation. I need to cover that pattern in the book and pretend it’s not a nutso thing to do.</bias>

Lastly, I gave my “Lessons Learned from StructureMap” talk one more time. I think I’m going to retire this one permanently, so I might try to blog it out over the next 2 months.

Upon reflection, I think my favorite thing about NDC was that there was a lot of more or less ALT.NET-ish content (and the older speakers from the bigger software world that we admire), but we weren’t in special camps or the “Agile gutter” track like in the DevTeach’s I went to a couple years ago. I love that the tee shirts for the conference said “Legalize Unit Testing.” I don’t ever see ALT.NET becoming completely the mainstream, but it’d be nice to stop being this strangely alternative stuff on the fringe.

I got ambushed by Hanselman and recorded for Channel 9 spouting off on 3 hours sleep about “Executable Requirements”, StoryTeller, apologizing for using angle brackets, and explaining why I’m not using Oslo/MGrammar for creating external DSL’s for testing (hint: I’m very unenthusiastic about the phrase “you just have to parse the AST”). There’s several other impromptu recordings from the speaker’s area here.

All in all, I had a great time and I’d be happy to recommend NDC next year for those of you in that part of the world.

A small web design rant: beware of the RADical designer

pvanooijen — Mon, 22 Jun 2009 10:09:39 GMT

A web designer is responsible for the look of a web site. She designs how the information is displayed on the pages, what illustrations are used and how the parts look together. Working with Visual Studio the collaboration between developer and designer can be streamlined. What usually works quite well is the designer creating an html page and style sheet with the look of the site. The places with dynamic content are marked as plain div’s, in the first draft usually filled with some dummy content to get a first impression. As a developer I turn this page into the master page substituting the content div’s with content placeholders. The classes in the style sheet are used in all the views and pages and I am freed from worries about the look. A clear separation of concerns.

Over time some adaptations might be necessary, due to content not fitting or specifications changing. This is usually no problem, I get a new css or updated html. Many a good designer does understand asp.net markup, making round tripping even easier. In the process both visual designers are used (by the designer) and mark up is edited (by me). This is no problem as good tools used are two way designers: when you change the look visually the markup changes and when you change the markup the visual appearance changes.

All of this is of course pretty obvious. But recently I had an experience where it did not work. Describing the scenario our designer assured us it was no problem. She (actually he) would make a good design and his tool would provide us with plain html and a decent css stylesheet. The visual appearance, in the browser, was nice and the design was approved. But alas the markup behind was not that nice. The html was over complicated with things like div’s whose only purpose was to wrap up yet another div. The css did contain a lot of classes. Most of them were quite verbose and defined many many style attributes. A later analysis of their usage showed they were not cascading style sheets, many an attribute was repeated, usually with exactly the same value. Trying to change the appearance of the site was hard. We had to wrestle our way through the trees of markup. And trying to change the font of a particular part was a quest through styles. Changing which one(s) would do the trick ?

Going back to the designer did not solve that much, his knowledge of the mark up side of web design was not sufficient for an effective communication. The lessons learned were several. First of all make sure that your designer does not only have good esthetical judgment but can communicate his ideas in a shared language, that of html and css. And when she has a tool which generates that make sure it is a two way tool, so proposals for change can be dealt with.

All reminded me very much of the RAD discussions. With again the same conclusion. RAD on itself smells but a good 2-way visual designer can still be nice. Coming from Delphi I used to believe that kind of tool existed. Having seen the haystacks of code Visual Studio produces led to serious doubt. And having seen what a visual design tool can do has shattered my last hope.

Agile Behavior: Nurture Knowledge Database

Patrick Smacchia — Sun, 21 Jun 2009 09:26:00 GMT

Normal 0 false false false EN-US X-NONE X-NONE

Code Query Language (CQL) can be used to write all sorts of code convention. I’ve described some CQL range of usage, and as an illustration I also exposed some CQL rules dedicated to the .NET Framework usage.

We discovered a weird issue in our last NDepend version: to select programmatically a particular row in a Windows Form DataGridView you need to call the setter DataGridView.FirstDisplayedCell. However if the DataGridView height is too small and doesn’t let show any row, then an exception will be raised:

Exception.Type {System.InvalidOperationException}

Exception.Message {No room is available to display rows.}

Exception.StackTrace {

at System.Windows.Forms.DataGridView.set_FirstDisplayedScrollingRowIndex(Int32 value)

at System.Windows.Forms.DataGridView.set_FirstDisplayedCell(DataGridViewCell value)

…

IMHO this behavior is so counter-intuitive and prone to error that I would qualify it as a .NET Framework bug. But Microsoft is not willing to fix such behavior in the name of ascendant compatibility. The solution is then to capitalize on your own knowledge database. Typically we included the following CQL rule in our set of rules. It warns as soon as there is a method that doesn’t check the rows available height before setting FirstDisplayedCell.

// <Name>Check height space before setting FirstDisplayedCell of a DataGridView</Name>

SELECT METHODS WHERE

IsDirectlyUsing "System.Windows.Forms.DataGridView.set_FirstDisplayedCell(DataGridViewCell)" AND

!(IsDirectlyUsing "System.Windows.Forms.Control.get_Height()" AND

IsDirectlyUsing "System.Windows.Forms.DataGridView.get_ColumnHeadersHeight()")

/* The pattern implementation is:

if ((rowToSelect.State & DataGridViewElementStates.Displayed) == 0) {

if (this.Height - this.ColumnHeadersHeight > 0) {

this.FirstDisplayedCell = rowToSelect.Cells[0];

}

We immediately spotted 3 others risky areas in our code.

F# – Async Running with Continuation Scissors

Matthew.Podwysocki — Sat, 20 Jun 2009 06:04:04 GMT

As you may have noticed, I’ve been covering a bit about concurrency on this blog lately, and for good reason. Between Axum, Erlang, Scala and F#, there is a lot to explore with actor model concurrency, task based concurrency, data parallel applications and so on. In the next couple of months, I have some talks coming up on concurrency and in particular with F#. I’ve been covering a bit of that with F# and mailbox processing, but I wanted to step back a bit into the asynchronous workflows and hitting against a well known API.

In this case, let’s walk through a simple example of using the Twitter HTTP API to get a user timeline using F# asynchronous workflows. Best of all, I’ve done all of the above code without ever once opening up Visual Studio and instead using the power of a simple text editor and F# interactive, I’m able to be quite productive. It certainly makes me rethink my utter dependence on certain tools…

Give me some tweets… eventually, when you get around to it

In order to get started, we need some helpers when dealing with LINQ to XML. In a previous post, I talked about the need for such a thing due to the lack of implicit operator support in F#. We can use the same operator that we had before such as the following:

let inline implicit arg =
    (^a : (static member op_Implicit : ^b -> ^a) arg)

let (!!) : string -> XName = implicit

Alternatively, we could look at utilizing the XNamespace as well due to it also having an implicit operator to convert from a string. This is helpful as it has a defined the op_Addition which allows us to add an XNamespace to a string to create an XName. To take advantage we could add an additional operator to take care of this. Below is a simple session of F# interactive to show how it might work:

> let (!?) : string -> XNamespace = implicit;;
val ( !? ) : (string -> XNamespace)

> (+) !?"";;
val it : (string -> XName) = <fun:it@205>

> !?"" + "foo";;
val it : XName = foo {LocalName = "foo";
                      Namespace = ;
                      NamespaceName = "";}

If you’ll notice above, I curried the XNamespace implicit operator function and sure enough it recognizes it needs a string in order to produce an XName. Now that we’ve defined our helpers, let’s start looking at how to get a user’s timeline in Twitter. We’ll need to define a holder for our status data as well as the URL we will be using to post data.

type UserStatus = { UserName : string; ProfileImage : string; Status : string; StatusDate : DateTime }

let timelineUrl = "http://twitter.com/statuses/friends_timeline.xml"

We’re interested at this point just of the user name, their image file location, their status and the date of the tweet. Now let’s move onto retrieving and parsing the data.

let parseDocument xml =
  let document = XDocument.Parse(xml)
  document.Root.Descendants()
    |> Seq.choose(fun node -> 
         if node.Element(!! "user") <> null then
           let status = HttpUtility.HtmlDecode(node.Element(!! "text").Value)
           let image = node.Element(!! "user").Element(!! "profile_image_url").Value
           let userName = node.Element(!! "user").Element(!! "screen_name").Value
           let statusDate = DateTime.ParseExact(node.Element(!! "created_at").Value, 
                                                "ddd MMM dd HH:mm:ss +0000 yyyy", 
                                                CultureInfo.CurrentCulture)
           Some { UserName = userName; ProfileImage = image; Status = status; StatusDate = statusDate}
         else None)  

let getUserTimeline userName password =
  async { let request = WebRequest.Create timelineUrl :?> HttpWebRequest
          do request.Credentials <- new NetworkCredential(userName, password)
          
          use! response = request.AsyncGetResponse()
          use reader = new StreamReader(response.GetResponseStream())
          let! xml = reader.AsyncReadToEnd()
          
          return parseDocument xml
        }

In the above code, we create a WebRequest, setting the credentials through Basic Authentication. We’ll cover this approach for now with some attention to OAuth at some point in the relative near future. After setting the identity, we get the response stream and read it to the end. This gives us XML to parse and by using LINQ to XML, we’re able to iterate through each node, and if the user node is not null then we parse out each element for which we’re concerned.

Now that we have the ability to do some asynchronous behavior, what do we do about it? In previous posts, I’ve taken the road of running these operations in parallel, which then runs all instances and returns an answer. But, if we’re only executing one instance, how do we enlist in asynchronous behavior overall? To illustrate, I’ll create a simple Windows Forms application to display the tweets in a textbox and have a refresh button to call the above function to repopulate our textbox. First, let’s create the basic skeleton:

let form = new Form(Visible=true, TopMost = true, Width=500, Height=300)

let status = new TextBox(Multiline=true, ScrollBars=ScrollBars.Vertical)
form.Controls.Add status
status.Width <- 400
status.Height <- 300

let refresh = new Button(Text="Refresh")
form.Controls.Add refresh
refresh.Left <- 410
refresh.Top <- 10

Now that we have a basic skeleton, let’s add the behavior to the button to refresh our textbox. To do this, we simply add a handler to the button click event. In there, we will call Async.RunWithContinuations which gives us the ability to handle not only the successful case, but also should an exception be thrown or some form of cancelation occurs. Before we look at the implementing code, let’s look at the signature of RunWithContinuations:

> Async.RunWithContinuations;;
val it :
  ('a -> unit) * (exn -> unit) * (OperationCanceledException -> unit) *
  Async<'a> -> unit = <fun:clo@0-42>

What this tells us is that it takes a success continuation as the first argument, the exception continuation as the second argument, the cancelation continuation as the third, and then finally the asynchronous operation. This allows us to uniquely handle each type of event, should they occur. In this case, our success continuation should populate the textbox, whereas both the cancelation and the exceptional case should show a message box containing the exception.

refresh.Click.Add
   (fun args ->
      status.Text <- ""
      Async.RunWithContinuations(
        (fun res -> 
           res |> Seq.iter(fun item -> status.Text <- sprintf "User: %s - %s\r\n\r\n%s" item.UserName item.Status status.Text )),
        (fun exn -> MessageBox.Show(sprintf "An error occurred: %A" exn) |> ignore),
        (fun cxn -> MessageBox.Show(sprintf "A cancellation error ocurred: %A" cxn) |> ignore),
        (getUserTimeline "username" "password")))

As you can see, the behavior is rather straight forward to define handlers for our three cases. Lastly, we provide our credentials to the getUserTimeline function and then when the button is clicked, we’ll get results much like the following screenshot.

Then we can keep pressing the refresh button and then asynchronously, it will handle the behavior appropriately, either in success or in failure.

Conclusion

This is a pretty simply and naive application, but it can show how you can take advantage of asynchronous behavior using the Async Workflows in F#. By being able to have the F# libraries handle the exceptions and cancelation, a major source of errors is reduced by quite a bit. Giving us the power then to handle each one of these scenarios, either in success, failure or cancelation is a great concept. There is still more yet to be covered including Futures and the inclusion of the Task Parallel Library as well as other adventures into Twitter and Bing APIs.

Revisiting Memoization

Matthew.Podwysocki — Thu, 18 Jun 2009 15:04:14 GMT

After revisiting the Haskell Wiki recently, I wanted to look at memoization again for a brief second after talking about it a while ago. In particular, there were two competing ideas, one around using a generic dictionary/map for storing the memoized values, the other using a lazy list approach. I wanted to briefly look into those as possible solutions.

The Baseline Approach

Let’s take a baseline approach, and instead of using the trite Fibonacci sequence, let’s use the Lucas Sequence instead. It’s a slight variation on the theme, and enough to avoid that dreaded function. The baseline should produce the following sequence: 2, 1, 3, 4, 7, 11, 18, 29, 47, 76, 123

It might be implemented something like this:

let rec lucas = function
  | n when n = 0I -> 2I
  | n when n = 1I -> 1I
  | n -> lucas (n - 1I) + lucas (n - 2I)

You’ll notice that F# cannot handle BigInt values as literals like you can for integers and other primitives, but can be easily worked around by using a when clause in the pattern. When we run the aforementioned function in F# interactive, we see that we get the expected results.

> lucas 2I;;
Real: 00:00:00.003, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : bigint = 3I

The problem with the above function is that when dealing with larger numbers, and calling this function repeatedly, you have to take a rather large performance hit each time. And worse yet, when dealing with those larger numbers say greater than 30, the function just may hang. We can solve this through a process called memoization, which I’ve talked about before. The main idea is to cache all values that we’ve seen before for quicker retrieval instead of calculating the value each and every time.

The LazyList Approach

There are two ways I’ll cover memoization here, one using a lazy list and the other using the dictionary approach. First, let’s look at how we might be able to use a list to store those values we’ve already calculated. Let’s first look at the Haskell implementation which is similar to the Haskell Wiki example.

memoized_lucas :: Int -> Integer
memoized_lucas =
  (map lucas [0 ..] !!)
  where lucas 0 = 0
        lucas 1 = 1
        lucas n = memoized_lucas (n-2) + memoized_lucas (n-1)

The approach here is to perform a map of the lucas from 0 to infinity and taking the nth value. Since this is partially applied, we need to wait on the last value to say which value we need. When we run this through GHCi, we can calculate the 1000th Lucas number rather easily with me omitting the result and only checking the time:

*Main> memoized_lucas 1000
(0.03 secs, 0 bytes)

Not only that, but the performance is rather good. Any subsequent calls to this function should now be memoized such as the following call:

*Main> memoized_lucas 1000
(0.00 secs, 0 bytes)

Now, how would we apply this to F#? Can we get the same results as a lazy language? If we switch over from using a list to a LazyList, the answer is yes, and the performance will be close. First, we need to define the nth function which returns the value at the index specified. Using pattern matching, we can then return the appropriate value, or recurse again. We could use Seq.nth, but this may be more efficient.

module LazyList =
  open Microsoft.FSharp.Collections.LazyList

  let rec nth list index = 
    match list with 
    | Cons(h, t) when index >= 0 -> 
        if index = 0 then h else nth t (index - 1)
    | _ -> invalidArg "index" "the index was outside the range of elements in the list"

Now that we’ve defined our nth function, it’s time to define our memoized Lucas function. The logic is the same as above, and I’ve introduced a new operator to get the nth value which could be used as an infix operator. The problem is that F# currently does not allow for the infix operator to be in the middle during partially applied functions, so instead I have to define it in the beginning.

open System
open Microsoft.FSharp.Math
let (<!>) s (n:bigint) = LazyList.nth s (int n)

let rec memoized_lucas : bigint -> bigint =
  let lucas = function
    | n when n = 0I -> 2I
    | n when n = 1I -> 1I
    | n -> memoized_lucas (n - 1I) + memoized_lucas (n - 2I)
  (<!>) (LazyList.map lucas (seq {0I..new BigInt(Int32.MaxValue)} |> LazyList.of_seq))

This is a rather clumsy approach in some ways as we’re downcasting our bigint value to an integer, so in reality, we’re only able to get values up to Int32.MaxValue, which in most cases, should be ok. Let’s look at the performance of this in the same way we did above:

> memoized_lucas 1000I;;
Real: 00:00:00.092, CPU: 00:00:00.093, GC gen0: 15, gen1: 0, gen2: 0

You’ll notice that we are taking a slight performance penalty in terms of our Generation 0 memory in the GC. Any subsequent calls are memoized given the following results:

> memoized_lucas 1000I;;
Real: 00:00:00.001, CPU: 00:00:00.015, GC gen0: 0, gen1: 0, gen2: 0

Let’s kick it up a notch to see what kind of penalty we may pay should we calculate the 10,000th Lucas sequence number:

> memoized_lucas 10000I;;
Real: 00:00:07.922, CPU: 00:00:07.815, GC gen0: 1511, gen1: 356, gen2: 1

Ouch! But, there is another approach that we can use and that I’ve shown before using the generic dictionary to store our values.

The Dictionary Approach

The generic dictionary approach is another approach rather than using the lazy list approach. We simply create a dictionary outside of our inner function which stores these values from our inner Lucas sequence function.

open System.Collections.Generic
let rec memoized_lucas' : bigint -> bigint =
  let t = new Dictionary<_,_>()
  let rec lucas n =
    match t.TryGetValue(n) with
    | (true , res) -> res
    | (false, _  ) ->
      match n with
       | n when n = 0I -> 2I
       | n when n = 1I -> 1I
       | n ->
           let res = lucas (n - 1I) + lucas (n - 2I)
           t.Add(n, res)
           res
  fun n -> lucas n

The code itself is rather self explanatory. We have the outer dictionary to store our values, then we have the inner function to calculate the value at the nth index. We then determine whether it is in the dictionary and, if so return it, else calculate, add to the list and then return. You’ll notice that we return not the literal value of the calculation, but instead a function which allows us to specify the nth at a later time. This is important!

Looking at the performance, we’ll notice the following:

> memoized_lucas' 1000I;;
Real: 00:00:00.006, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
> memoized_lucas' 1000I;;
Real: 00:00:00.001, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0

You’ll notice that the first calls is less expensive than the lazy list approach and has less impact on Generation 0 of the GC. Any subsequent calls, as above will be nil. But once again, let’s ratchet up the number to 10,000 much as we did above:

> memoized_lucas' 10000I;;
Real: 00:00:00.047, CPU: 00:00:00.062, GC gen0: 1, gen1: 1, gen2: 1

You’ll notice a drastically less performance hit than the above solution.

Conclusion

If nothing else, it was an interesting exercise in seeing different ways of memoization in action and what kind of performance penalties you may pay along the way. Now back to your regular programming…

NHibernate : Handling the Special Cases

karl — Thu, 18 Jun 2009 14:03:00 GMT

In my last NHibernate post I mentioned that NHibernate is great because it does the heavy lifting for the common tasks that make up 95% of my data access layer, while letting me tweak the remaining 5% as needed.

Let's look at some of the ways you can tweak, or even circumvent NHibernate for those queries that require some extra attention. Do note that there are a lot of different solutions – we're only looking at a few.

The most obvious method is to use named queries and custom SQL statements in your mapping. This provides you a lot of flexibility without diverging from NHibernate's realm. The NHibernate reference has good documentation on this approach, so we won't cover it (check out section 14.2 "Named Queries", 14.3 "Custom SQL for create, update and delete" and 14.4 "Custom SQL for loading").

The next option is to use an ISqlQuery via the ISession.CreateSqlQuery. Unlike an IQuery which works against your domain using HQL, an ISqlQuery is closely tied to your database and SQL. However, you still get access to NHibernate parameter substitutions, paging capabilities and O/R mapping (if you want). Again, the NHibernate reference has a very detailed section, check out 14.1 "Using an ISQLQuery".

You can optimize your Deletes by supplying the ISession.Delete command some HQL rather than the instance you want to delete. For example, if you want to delete all comments belonging to a blog post, you can do:

session.Delete("from Comment where PostId = :id", postId, NHibernateUtil.Int32);

More recently the NHibernate team made it possible to execute any statement (insert/delete/update) from an IQuery. You can find out more from this post by Ayende.

When necessary (typically bulk imports or other bulk operations), you can circumvent NHibernate's first-level cache by using an IStatelessSession rather than an ISession. Rather than calling OpenSession on your ISessionFactory, simply call OpenStatlessSession. Keep in mind that the IStatelessSession doesn't have the exact same API as an ISession.

When things get really crazy, you can always get an ADO.NET connection from the ISession's Connection property. With a raw database connection, the world is your oyster. You can always write your code in such a way that you have direct access to our connection string to create your own connection, thus completely and fully circumventing NHibernate.

Finally, any of these methods can be combined with some simple designs in order to ensure that everything is neat and tidy:

public abstract class Repository : IRepository
{
    public void Delete<T>(T entity)
    {
        WithinTransaction(s => s.Delete(target));
    }

    protected void WithinTransaction(Action action)
    {
        WithinTransaction(Session, action);
    }        
    protected static void WithinTransaction(ISession session, Action action)
    {
        var transaction = session.BeginTransaction();
        try
        {
            action();
            transaction.Commit();
        }
        catch (Exception)
        {
            transaction.Rollback();
            throw;
        }
        finally
        {
            transaction.Dispose();
        }
    }
}


public class PostRepository : Repositor, IPostRepository
{
    public void Delete(int postId)
    {
        WithinTransaction(s => 
            {
                session.Delete("from Comment where PostId = :id", postId, NHibernateUtil.Int32);
                session.Delete("from Post where Id = :id", postId, NHibernateUtil.Int32);
            });
    }
}

Once you understand NHibernate's flexibility and understand your code's hot spots, it generally takes a pretty trivial amount of tweaking to speed things up. While the changes you make, in the name of performance, to your models (both domain and data) may be disgusting rest assured that HHibernate will not stand in your way. Be warned though that many of the above solutions may have side effects with respect to NHibernate's 1st and 2nd level caches. However, if you were to custom write everything, including caching, you'd likely have similar synch issues.

CppDepend Beta now available

Patrick Smacchia — Sun, 14 Jun 2009 15:43:00 GMT

Normal 0 21 false false false FR X-NONE X-NONE

A few months ago I was happy to announce the Java version of NDepend: XDepend. Actually a third brother was under development and today, I am glad to announce CppDepend, aka NDepend for C++. A beta version is freely downloadable. You can use it right now to demystify that huge legacy C++ applications you are working on. Most of NDepend features are available in this beta. We are now entering a debugging phase to reach a RTM product. Feel free to report bugs and feedbacks on this google group. Alternatively you can contact directly Issam Lahlali (cppdepend at google mail), the product manager of the project.

To make things more practical, Issam just wrote an article showing what CppDepend has to say about the MFC code base.

I love 100% coverage!

Patrick Smacchia — Sat, 13 Jun 2009 09:02:00 GMT

Last week I wrote a post about non-trivial reasons why 100% coverage is useful anyway.

Yesterday, my quest for 100% coverage saved me one more time from some code written too quickly.

Code written too quickly, not 100% covered:

Code fixed 100% covered:

I am not sure that a proper code review would have detect this one character bug, human eyes are not trained for that.

Moreover, this code is in a IComparer<T>.Compare(...) method used for sorting. In this condition the bug side-effect could be hardly detected by unit tests and contracts.

You’re Not Your Data Access

robconery — Fri, 12 Jun 2009 03:50:33 GMT

Seems I touched off a bit of a “swirl” with a comment I made on my last blog post:

I think, in general, the .NET crowd overthinks and over-engineers just about everything

I said as much in my MIX presentation, where I basically challenged ASP.NET developers to “embrace their inner scripter” and stop building rockets. I have a fairly strong opinion on this – and I’ll save that for another time – but I think it’s high time to remind folks that there’s a lot more to an application then how you get your app gets it’s freak on with your database.

Good Morning!
I remember watching Fight Club for the very first time back in the thick of the very first DotCom bubble. I was feeling pretty ill that day so I stayed home and watched the DVD, and was pulled in pretty dramatically. One line still resonates with me:

You're not your job. You're not how much money you have in the bank. You're not the car you drive. You're not the contents of your wallet. You're not your f******* khakis.

I feel the same way about building software: You’re Not Your Data Access.

When I watched that movie I had just bought a nice pair of $180 Kenneth Cole shoes to wear to a client’s VC meeting, to go with my groovy business-casual Cornflower Blue button down shirt. I clearly remember looking over at my closet – the feeling of fashion sobriety coming over me – wondering what I had turned into (if you knew me then you’d know I don’t like wearing that kind of stuff. I’m happy in a white t-shirt and shorts).

I had this same feeling when I popped my head out of the .NET community just 4 years ago (before I worked at Microsoft) to see what this “Rails thing” was all about. I remember the feeling well – and it was very, very sobering and inspiring at the same time. I remember creating an application in fairly short order with all the requisite pieces put where they needed to be put, with every tutorial urging me to “stop thinking – just build”.

Camaro Programming
The .NET platform is pretty dang powerful. It lets you build your own personal Tower of Babel if you so choose, and it can abstract away the very fabric of space-time if you let it. In short it’s a killer set of tools that any mechanically-inclined person would love to use. So naturally it can be very easily abused!

Consider my good friend Eric. I’ve written about Eric a few times before – but in summary he’s my token wrench-head friend that loves his engines and American cars. I lived with this guy for years and he rapidly filled up our garage with parts and *crap* from every car he’d ever owned. And he could dutifully tell you which part went to which car and what that part did – no matter if it was useless.

Eric built and maintained a hunk of junk 1968 Capri which we used to call “The Crappy”. This thing could drive – and I mean really drive – I think Eric pushed it to 180 mph once on a closed race course. It had that sound that all muscle cars make – the sound that makes your ribs shake and your inner 16 yr old say “coooooolll!”.

The problem was that no one ever wanted to drive in it. It smelled, it was ugly, and it absolutely sucked the gas down. Eric would steadfastly defend his Crappy because “it was the easiest car in the world to work on” and “could blow doors on any production car from any country”.

“Yah but when do you really need to go 180 miles an hour man?”

“When I race dammit”

I don’t need to draw the parallels here. I think you get it.

Let it Go Already
I want to be perfectly clear about what it is I’m saying here, and that is that it’s very, very easy for the geek inside us to whisper sweetly in our ears, saying “this Super Massive Paradigm Shifter will be the biggest application EVAR… don’t mess it up!”. I felt this very sensation when starting … well every app that I ever made.

I don’t like being wrong – no one does. So defensively I’ll be sure to include patterning where I can, making sure to separate out my projects whenever it … seems like I’ll crap for not separating out what I’m doing into another project.

I’ll use enums instead of boolean parameters.

Oh wait, no I won’t.

I’ll be sure to avoid Singletons

I’ll absolutely never use Stored Procedures, I mean seriously never, ever, ever (unless Rob Howard tells me to). Or maybe unless my boss tells me to. I’ll let Jeff decide.

I’ll always use Stored Procedures.

I won’t use NHibernate

I won’t use any ORM for that matter. Seriously - Never.

Well I might use both…

I won’t use Code-coverage to tell me anything.

I’ll be sure to use Code-coverage data to tell me everything.

I’ll make sure my method and variables are named properly on Fridays.

Reflection? Not a chance. Unless Rick says it’s OK.

I won’t use Extreme Programming (I’ll be sure to use Lean… whatever that is)

I’ll be sure not to have scrums.

I’ll also be sure not to multi-task.

I won’t use configuration either – hard-coding FTW!

I won’t use static methods

I won’t use extension methods

I’ll keep Native American stereotypes out of my code.

I’ll make sure that all my methods are public and never sealed or internal

I’ll check twice before I use any of these evil keywords.

I’ll also make double damn sure not to use regions in my code – ever.

I won’t use VB (who would?) and I’ll be sure to use C# with great care.

I won’t use Composite Keys in my database

And I’ll be very, very sure to keep away from Norway.

You wake up at Seatac, SFO, LAX. You wake up at O'Hare, Dallas-Fort Worth, BWI. Pacific, mountain, central. Lose an hour, gain an hour. This is your life, and it's ending one minute at a time. You wake up at Air Harbor International. If you wake up at a different time, in a different place, could you wake up as a different person

Let.

It.

Go.

Summary
Simplicity is beautiful. Simple doesn’t mean hard-coding, it doesn’t mean cutting corners or being sloppy – it means building what you’ve been asked to build, not a rocket to Saturn and most of the time it’s a skateboard to the corner store.

Not every application needs to be stitched together from Codebetter posts and Twitter rants. Focus on what’s important – the experience, not what’s under the hood. You’ll change that quite a few times no matter what you think :).

F# – Duck Typing and Structural Typing

Matthew.Podwysocki — Thu, 11 Jun 2009 16:53:18 GMT

As you may have noticed on this blog lately that I’ve been focusing on Asynchronous Workflows. In those adventures, I’ve been taking well known APIs such as Twitter, Bing, among others and seeing what I can do with them. In this instance, when using LINQ to XML, I’ve run into a slight syntax problem.

The Problem with LINQ to XML

The syntax problem I had revolves around the retrieval of an XElement from the given XContainer. The signature of this is as follows:

public abstract class XContainer : XNode {
    public XElement Element(XName name);
}

But, what you may not realize is that there is an implicit operator which allows us to implicitly convert a string to an XName with little effort such as this:

public sealed class XName : IEquatable<XName>, ISerializable {
...
    public static implicit operator XName(
        string expandedName);
}

This allows us to use a string instead of an XName when specifying which element name we want, including namespace if desired, rather easily.

var document = XDocument.Parse(xml);
var statuses = document.Root.Descendants()
    .Select(node => node.Element("text").Value);

Unfortunately for us, this behavior does not work in the current F# release.

> document.Root.Descendants() 
-   |> Seq.map(fun node -> node.Element("node").Value);;

stdin(5,65): error FS0001: This expression has type
        string
but is here used with type
        XName

It does not recognize this implicit operator. What options do we have around this? One simple answer might be just a simple function that does the conversion for us, such as this:

> let ( !! ) : string -> XName = XName.op_Implicit;;
val ( !! ) : string -> XName

> let xFoo = !!"foo";;
val xFoo : XName = foo

What we’re now able to do is convert at will using the !! operator, or what I call the “Convert Dammit” operator. But, there is another solution worth exploring that is a bit more generic through the use of duck typing.

Duck Typing to the Rescue?!?

For those unfamiliar with the term duck typing, it is a style of dynamic typing in which the object’s current set of methods, properties, etc, determines its validity instead of its inheritance chain. In F#, we have the ability to use this mechanism to use this to our advantage to create generic functions. For example, let’s take the implicit operator example from above and use duck typing instead of the direct call to the XName class.

// Longhand
let inline implicit< ^a,^b when ^a : (static member op_Implicit : ^b -> ^a)> arg =
  ( ^a : (static member op_Implicit : ^b -> ^a) arg)
  
// Shorthand - let the compiler do the work
let inline implicit arg =
  ( ^a : (static member op_Implicit : ^b -> ^a) arg)

The two above statements are syntactically the same, although in the latter version, I don’t have to clutter up my declarations as much and instead let the compiler type inference magic do its work. From here on out, I’ll use the shorthand unless the longhand doesn’t quite work out. What you’ll notice is that I have a requirement for my ^a generic to have a restriction that it must have the implicit operator declared. Then we can execute said operation by passing in our argument. I can rewrite the convert dammit operator then to look something like this:

open System.Xml.Linq
let (!!) : string -> XName = implicit

let xFoo = !!"foo" // Our new XName

Now you can see here that I didn’t explicitly have to call anything on XName as I let the implicit operator do the work for me. Best of all here is that it’s now statically typed to that any call to implicit must have that operator defined on it, else it will not compile.

But, let’s not stop there, you can continue on all sorts of tangents. What about functions which have more than one argument? Let’s take the addition operator. Could we use duck typing in our favor for this? Although it’s not recommended, the answer is yes.

> let inline add arg1 arg2 = 
-   ( ^a : (static member op_Addition : ^a * ^b -> ^a) (arg1, arg2))
val inline add :
   ^a ->  ^b ->  ^a when  ^a : (static member ( + ) :  ^a *  ^b ->  ^a)

> open System;;
> let addedTime = add DateTime.Now (TimeSpan.FromSeconds 5000.);;
val addedTime : DateTime = 6/11/2009 1:00:32 PM

The reason why it’s not recommended is that a lot of primitive types are implicitly augmented with this operator such as int, float, etc, so this would not work for those plus not work on string as well since dynamic invocation is not allowed on this particular instance.

This is useful though in other scenarios. Take for example extracting a Name property from a given class. Once again, using the methodologies from above, it’s rather simple:

let inline getName arg =
  ( ^a : (member Name : string) arg)
  
open System
open System.IO
let fName = getName (new FileInfo("fsi.exe"))
let dName = getName (new DirectoryInfo(Environment.CurrentDirectory))

What about multiple restrictions? That’s not too difficult either. Let’s do the canonical Duck example with a little twist. First, we will define the overall function which will extract the Flying and Walking state of a given class.

let inline flyAndWalk arg =
  let flying = ( ^a : (member Fly : unit -> string) arg)
  let walking = ( ^a : (member Walk : unit -> string) arg)
  (flying, walking)

And now we can define our classes which have the methods which have our interest such as the Fly and Walk methods. We execute each of these, obtain the result and finally return the results of each as a tuple. Now, let’s define two classes below of a Duck and an Eagle.

type Duck() =
  member this.Swim() = "paddling"
  member this.Fly() = "flapping"
  member this.Walk() = "waddling"
 
type Eagle() =
  member this.Fly() = "soaring"
  member this.Walk() = "creeping"
 
let (eFly, eWalk) = flyAndWalk (new Eagle())
let (dFly, dWalk) = flyAndWalk (new Duck())

Our final result will be rather predictable as you might imagine. Just as well if we put a Penguin class which contains no fly, well, then this class won’t compile as shown below in F# interactive:

> type Penguin() =
-   member this.Walk() = "waddle"
-   member this.Swim() = "swimming";;

type Penguin =
  class
    new : unit -> Penguin
    member Swim : unit -> string
    member Walk : unit -> string
  end

> let (pFly, pWalk) = flyAndWalk(new Penguin());;

  let (pFly, pWalk) = flyAndWalk(new Penguin());;
  -------------------------------^^^^^^^^^^^^^

stdin(84,32): error FS0001: The type 'Penguin' does not support any operators na
med 'Fly'

As you can see, it’s a rather powerful concept, when used in moderation, can help you solve some interesting problems such as implicit operators as well as other scenarios.

Structural Typing

There is a concept that is prevalent in languages such as OCaml, ML, and Scala for Structural Typing as opposed to Duck Typing. This is a concept in which compatibility and equivalence is based upon the type’s structure and not the explicit declarations as I’ve had above. Equivalence is then measured by the structural sameness between the two even though they may be nothing more than anonymous objects.

In F#, we can accomplish this in a sense through the use of the record type quite easily. For example, we can define a class that has two properties and then check for sameness between them.

> type Foo = { X : int; Y : int};;

type Foo =
  {X: int;
   Y: int;}

> {X=32;Y=45} = {X=32;Y=45};;
val it : bool = true

Normally, given that these are reference types and not standard structures, they shouldn’t ordinarily equate one to another automatically, but F# through the use of structural typing, does allow for this. The same also applies for Discriminated Unions as well such as the following:

> type DU = A of string | B of int;;

type DU =
  | A of string
  | B of int

> B 42 = B 42;;
val it : bool = true

Once again, since they are reference types, they shouldn’t equal each other, but with F#, they do, through the use of structural typing.

Conclusion

Through the use of structural typing and duck typing, we realize how flexible the F# language really is. This gives us the ability to be creative in how we solve our problems generically in specialized cases such as our LINQ to XML example from above. Now that F# is a full citizen in the .NET space with Visual Studio 2010, it’s even more exciting about what we can do with the language.