In the last blog post Testing Trade-offs, I talked about the difficulties of verifying the decisions and dependencies of our classes with the current mainstream testing methodologies. Based on a recorded conference talk by Gary Bernhardt, the focus was on effectively testing the logic a class contains and the dependent collobrators it takes in for coordinating with other classes and objects to perform its responsibilities. Mixing the two concerns in the same object definition requires utilizing both isolated unit testing and integration/integrated testing in order to adequately test cover the class. However, code designed this way seems to play to the weaknesses of each testing strategy just as much as it plays to their strengths (please see the previous blog post if you would like more details on that discussion).
Today let’s go one step further and talk about ways that we could more cleanly separate the concerns of decisions and dependencies, with the hope that we can create objects that better lend themselves to one type of testing over the other. Gary proposes that a such a codebase could have better modularity, scalability, and even concurrency. I assert that your code will also be more maintainable and extensible as well. Most of today’s content will take a lesson from the functional programming paradigm, including practices they have espoused for decades.
If you wanted to test the mathematical addition operator (i.e., the plus sign
+), what frameworks, tools, and/or coding tricks do you have to employ to
sufficiently isolate it from all other concerns and objects? Absolutely nothing!
There are no dependencies to mock or stub; it isolates for free. Why is that?
Gary cautions against assuming it is because the addition operator is simple and
lacks complexity. He digs deeper in order to identify two properities the
implementation of plus sign exhibits that allows it to be naturally isolated.
The first property is that the operator takes values as arguments and returns new values as output without any mutation. The second property is that the operator requires no dependencies in order to perform its computation and logic. Thus there is nothing to mock or stub when testing it, which was the major weakness of isolated unit testing. Also because of the lack of dependencies, no integrated tests are required in order to better test how the operator will behave in a production environment where there are no mocks and stubs. To test the addition operator we merely need to write simple pass-values-in, assert-value-out tests with no extra setup required.
As Gary applies these concepts to existing code, we notice a few changes. Pieces of domain logic and pieces of code that coordinate dependencies are separated from each other, broken out into new objects created for a single purpose and responsibility. The nature of the communication between objects also changed, with values (inputs and outputs) becoming the boundary between objects instead of the emphasis being on several synchronous method calls. Value objects focused on data (and not behavior) become the new contract between collaborating classes.
You may notice that many of these concepts have a functional programming influence. The properties of immutability and focus on data at the boundaries allow us to write code that isolates very easily and lends itself very well to isolated unit testing that is simple and not brittle. It is very good at verifying the domain logic and decision paths of our objects. Of course, we can’t write the entirety of our codebase in this manner with no dependencies ever. Next time I will discuss a code architecture that Gary proposes which can utilize this style of code married with some more imperative glue code that coordinates the dependencies in the system. We will also discuss testing those portions of code as well.
]]>A common practice in unit testing classes and objects is to isolate the targeted class from its dependencies so that you can focus solely on its responsibilities and domain logic independent of any implementation details of the dependencies. This is typically accomplished by the use of stubs and mocks, which attempt to control and monitor the interactions with and data return from dependencies. There are a ton of gains afforded by this style of testing as Gary points out, but one very large criticism of this testing methodology is that it doesn’t exercise the code in the same way it would run in production. It is not exactly rare to have these isolated unit tests successfully pass and yet still have problems in production.
Integration testing tends to better emulate production because it maintains the
interaction relationships between objects in addition to acquiring and passing
the data around the system in the same way. The criticism Gary puts forth of an
integrated testing strategy is that it is very slow as you begin to attempt to
cover all the code paths through the system. Consider trying to cover all
logical branches through the system, including all branches of
try/catch/finally, conditional, and looping structures. Gary suggests the growth
in code to cover these scenarios is 2n, where n is the number of branches.
Many try to get the benefits of both types of testing in order to compensate for each strategy’s shortcomings. However, they still write code that doesn’t play to each testing methodolgy’s strengths. The strength of isolated unit testing is verifying that given certain inputs the expected output is always returned. The strength of integration testing is coordinating dependencies, making sure they utilize and interact with the API of other objects correctly. If you design your classes to be a mix of dependencies and logic, it becomes difficult to effectively test cover them without writing both sets of tests for all classes in your codebase.
OK, so you are probably wondering what the point is then. This sounds like a good plan to just be more disciplined in your test coverage, right? Well, let’s explore a way to better segregate dependency orchestration from actual logic and behavior. This could allow us to use the right testing methodology depending on which type of responsibility the object is meant to encapsulate: dependencies or decisions? Gary also asserts that it will lead us down a path that could yield a codebase with better modularity, scalability, and potentially even concurrency. I believe you can also add better maintainability and extensibility to that list.
Next week I will dive into how Gary suggests we can better seperate these concerns of dependencies and decisions. Stay tuned (or go watch the video and spoil the surprise).
Next post: The Value is the Boundary
]]>async and await keywords in C# 5 for a while now,
but I saw a use of it I hadn’t anticipated. I saw Jon Skeet create an
asyncronous lambda expression and was caught a bit off guard. After thinking
about it, it really isn’t that amazing, but I thought I would spread the word
anyway because I think it could be a really handy feature.
Anonymous functions are declared without a method name and can be passed around
and invoked at will. Lambda expression syntax creates anonymous functions that
can be used as delegates or expression trees. In C# 5, you can adorn functions
with the async keyword which signifies that the function will return a Task
object from the Task Parallel Library as a return value. You will also find
somewhere in the method body the keyword await next to a statement that could
take a while to process and be a blocking operation. Now your method can resume
operation after the asyncronous operation has completed and has its result.
Here is a simple example:
public void ButtonClick()
{
ProcessFiles();
}
private async void ProcessFiles()
{
var files = new string[] { "file1.txt", "file2.txt", "file3.txt" };
foreach (var file in files)
{
var fileContents = await ReadTextAsync(file);
// likely do something with file contents
}
}
private async Task<string> ReadTextAsync(string filePath)
{
// Some text reading implementation that supports the TPL
}
The button click handler method would return at the first sign of a blocking operation and callback delegates are set up under the covers by the compiler to finish executing the rest of the code after the blocking process finishes.
Here is the same example but using an async lambda expression:
public void ButtonClick()
{
var files = new string[] { "file1.txt", "file2.txt", "file3.txt" };
files.ForEach(async file => await ReadTextAsync(file));
}
private async Task<string> ReadTextAsync(string filePath)
{
// Some text reading implementation that supports the TPL
}
Please overlook my disregard of single responsibility and encapsulation, and so on so forth. This is an overly simplified example intended solely to show what the syntax looks like. Either way, I could see this lighter weight syntax coming in handy in the future.
In the same example where I saw async lambda syntax used, I also saw Jon Skeet
show off a really clever extension method that took an enumerable collection of
Task objects and reordered them so they could be enumerated and handled in the
order that they complete their asynchronous operations. Perhaps I will show that
piece of code next time.
One of the main things that hit me while talking to these graduating students was how well-rounded our developers have come. No, I’m not talking about the amount of soda and treats we consume. I’m referring to the varying skills each one of us have developed. From front-end to back-end or web sites to desktop apps, every single member of the team is capable of jumping into any of our codebases and tackling new features or bugs with a excellent degree of competentcy. I find this fascinating because it is not the norm at many medium to large size companies. It is typical for a developer to be hired to a position or team where he or she works on a single project and codebase for a few years until they get bored enough to change positions within the company or go somewhere else completely. This has not been the case for all of the developers at Extend Health.
We are currently divided into four teams of about four or five devs each. While each team may have past experience with a certain project or codebase, they are not automatically the chosen team to work on it again the next time. Because of this, there is shared knowledge across teams and team members, instead of all the knowledge residing in only one person’s head. This avoids the “what if someone gets hit by a bus” problem my coworker Ryan just blogged about. The other side effect is that our developers have all become well-rounded in varying languages and platforms, solving both front-end and back-end problems. Contrast this with how I used to classify my dev skill set four or five years ago before working at Extend Health. I used to consider myself a back-end systems developer, feeling it was important to have specialization in order to differentiate myself and provide a chance to develop focused and in-depth experience. I also fooled myself into thinking that I preferred this designation because web development was trivial and only the hardest and most worthwhile problems were in the back-end systems. Instead, I merely lacked some meaningful experience with front-end web development, which is harder to come by in the typical work enivronment I described above. However in my current employment, I’ve become a much more well-rounded software developer.
I’m not entirely positive if our environment and team dynamic were intentionally setup to ensure we were all well-rounded developers (though every other good aspect of our dev culture seems to have been carefully pieced together by Corey). Regardless, I am appreciatevly taking note of the advantageous results of such a configuration. This is one very important aspect of true job satisfaction and also in avoiding high turnover and attrition in workforce.
]]>I’m sure everyone can agree that the safest way to refactor code and be 100% positive that the functionality hasn’t regressed is to have a good suite of tests. And if you don’t have sufficient test coverage (or any at all), then now is the right time to add it before refactoring.
However, I have found myself skipping this step in the past, feeling justified because the quality of the code was so poor or the deadline was so short. It is irresponsible of me to assume that I have not changed the functionality of the code without proof in the form of passing tests that sufficiently cover all scenarios and use cases. Nonetheless, watching Katrina’s presentation has given me renewed commitment and a sound plan of attack against legacy code that may be ugly and very unfamiliar to me.
Here is the example method that Katrina uses during her presentation (which is later revealed as a piece of code that she vaguely remembers coding herself and now abhors):
module XYZService
def self.xyz_filename(target)
# File format:
# [day of month zero-padded][three-letter prefix] \
# _[kind]_[age_if_kind_personal]_[target.id] \
# _[8 random chars]_[10 first chars of title].jpg
filename = "#{target.publish_on.strftime("%d")}"
filename << "#{target.xyz_category_prefix}"
filename << "#{target.kind.gsub("_", "")}"
filename << "_%03d" % (target.age || 0) if target.personal?
filename << "_#{target.id.to_s}"
filename << "_#{Digest::SHA1.hexdigest(rand(10000).to_s)[0,8]}"
truncated_title = target.title.gsub(/[^\[a-z\]]/i, '').downcase
truncate_to = truncated_title.length > 9 ? 9 : truncated_title.length
filename << "_#{truncated_title[0..(truncate_to)]}"
filename << ".jpg"
return filename
end
end
It certainly isn’t the worst lines of code ever written and it is getting the job done, but it is definitely not very readable and quite difficult to understand its purpose and logic. About the only thing that can be discerned is that it is intended to generate a filename in a specific format.
Katrina starts by writing a very simple and seemingly useless test (which she calls a “Mickey Mouse test” in her GitHub commit message) in order to begin test coverage of this method:
require_relative './xyz_service'
describe XYZService do
let(:target) do
stub(:target)
end
subject { XYZService.xyz_filename(target) }
it 'works' do
subject.should eq('something')
end
end
Without much knowledge of what this method’s purpose or inputs are, we can only pass in an empty stub object as input and then assert something rediculous for the result. We expect this test to not only fail to pass, but it is also likely to error during runtime (or even at compile time if in a statically typed language like C#). The idea is to create a quick feedback loop that aids us in finding the context and inputs required to make the code execute without runtime error. Then we can finally see the actual return value and modify our dummy assertion to begin expecting that output.
After iterating in this fashion, Katrina arrives at the following test which is a bit more useful:
require_relative './xyz_service'
require 'date'
describe XYZService do
let(:target) do
messages = {
:publish_on => Date.new(2012, 3, 14),
:xyz_category_prefix => 'abc',
:kind => 'unicorn',
:personal? => false,
:id => 1337,
:title => 'magic & superglue'
}
stub(:target, messages)
end
subject { XYZService.xyz_filename(target) }
it 'works' do
subject.should eq('14abcunicorn_1337_cb6c53bc_magicsuper.jpg')
end
end
However, this test doesn’t pass because the cb6c53bc portion of the resultant
filename is different every execution of the test. A simple regex allows the
test to pass all of the time:
subject.should match(/14abcunicorn_1337_[0-9a-f]{8}_magicsuper\.jpg/)
Even though we now have a passing test, we have not yet exercised the different input values and all code paths and branches through the method. This must be done in order to fully understand, document, and test cover the logic and purpose of the method. This would be done in the normal unit testing fashion that is shown in just about every testing demo by adding more test scenarios and assertions that excercise the varying use cases.
One interesting development that occurred while Katrina was using this process
was that she actually found a bug in the implementation of the original
xyz_filename method, specifically an extra set of braces in the regex on line
14.
Now that we are fully covered with tests (and a bug fixed), we are free to refactor the code into smaller, more readable chunks of code logic that are more cohesive and more reusable as well. This is what Katrina does for the remainder of her presentation, and it is just as instructive as this first part. However, today I wanted to focus on her strategy for writing test coverage against a legacy codebase.
I like how what Katrina is demonstrating seems to utilize a black box testing approach at first, and then progresses to a more white box approach as we begin to explore the different parts of the implementation. Perhaps such a strategy should have been obvious to me, but Katrina’s conference talk really struck a chord with me and encouraged me to commit to doing better with test coverage in legacy codebases before I begin making modifications or additions.
I highly recommend viewing the entire video, as it is only 30 minutes long and an entertaining watch.
]]>PIVOT and UNPIVOT SQL keywords.
This last week I needed to track down some data inconsistencies in our database by joining a few view model tables. These tables are populated by event handlers that denormalize the relevant data from events into a view model table that serves the needs of a specific part of the UI. In one particular view model table, it made sense to arrange the data points specific to each US state as columns on a single row. This allowed for a grid-based UI that enables batch operations on many rows at one time. A second view model table was intended for a different use case and was designed so that the state-specific data points became a one-to-many relationship across multiple rows and only two columns for the key-value pair of the US state and its associated data point value. These were two of the tables I needed to join together during my database inquisition.
At this point I figured I was stuck, as I couldn’t see a sensible way to join two tables designed so contrary to
each other for the purposes of joining. I tried to think of any ways to dynamically join tables on a column name
in one table specified by the value in the other table’s row. I knew I was heading nowhere fast, so I asked our
DBA for some ideas. He informed me of the PIVOT and UNPIVOT keywords in SQL as one option to solve the problem.
We reasoned that even though it may not be the most efficient feature, the likelihood of it proving to be a
performance bottleneck would be essentially non-existent (given that the entire size of the data set was relatively
small, on the magnitude of hundreds or thousands of rows instead of millions of rows). And so I gave it a go.
I will now describe an example scenario in order to illustrate how I used the UNPIVOT feature to join two tables
that didn’t seem at all compatible. The goal will be to join a table representing what US states each product is
available for sale within to a table representing how many of each product are being sold in each state. We will
call the first table ProductAvailability and give it an integer identity column named Id, an integer column
named ProductId, and then 52 bit columns indicating the availability status of the product in each of the 50 US
states, the District of Columbia, and Puerto Rico. These columns will be named by their two letter abbreviations of
AK through WY. The second table we will call ProductSales and it will also have an integer identity column
named Id and an integer column named ProductId. In addition it will have a string column (specifically
char(2)) named StateCode and an integer column named TotalSales.
The unpivoting of the ProductAvailability table to prepare it for joining can be done like so:
SELECT
Unpivoted.ProductId,
Unpivoted.StateCode,
Unpivoted.IsAvailable
FROM ProductAvailability
UNPIVOT
(
IsAvailable FOR StateCode
IN (AK,AL,AR,AZ,CA,CO,CT,
DC,DE,FL,GA,HI,IA,ID,
IL,IN,KS,KY,LA,MA,MD,
ME,MI,MN,MO,MS,MT,NC,
ND,NE,NH,NJ,NM,NV,NY,
OH,OK,OR,PA,PR,RI,SC,
SD,TN,TX,UT,VA,VT,WA,
WI,WV,WY)
) AS Unpivoted
The interesting part is obviously the UNPIVOT clause. Here we are declaring a list of column names to unpivot and
also naming two new columns in which to place the old column’s name and value (StateCode and IsAvailable
respectively). The result is that instead of having one row of 54 columns for each product, we now have 52 rows of
three columns to describe the exact same availability of the product. But with this second schema, we can more
easily join with the ProductSales table.
Here is an example of such a query where we are trying to find the products which have no sales in a state for which it is marked as available for sale:
SELECT
ProductStateAvailability.ProductId,
ProductStateAvailability.StateCode
FROM
(
SELECT
Unpivoted.ProductId,
Unpivoted.StateCode,
Unpivoted.IsAvailable
FROM ProductAvailability
UNPIVOT
(
IsAvailable FOR StateCode
IN (AK,AL,AR,AZ,CA,CO,CT,
DC,DE,FL,GA,HI,IA,ID,
IL,IN,KS,KY,LA,MA,MD,
ME,MI,MN,MO,MS,MT,NC,
ND,NE,NH,NJ,NM,NV,NY,
OH,OK,OR,PA,PR,RI,SC,
SD,TN,TX,UT,VA,VT,WA,
WI,WV,WY)
) AS Unpivoted
WHERE Unpivoted.IsAvailable = 1
) AS ProductStateAvailability
INNER JOIN ProductSales
ON ProductSales.StateCode = ProductStateAvailability.StateCode
AND ProductSales.ProductId = ProductStateAvailability.ProductId
WHERE ProductSales.TotalSales = 0
I hope you find the PIVOT and UNPIVOT SQL keywords just as helpful as I did if you ever find yourself in a bind
with two tables that are seemingly impossible to join.
I just gave a webinar yesterday for JP Boodhoo’s Develop With Passion® webinar series on Compositional and Convention-based Web Development with FubuMVC. The recorded session is now posted online with a slew of referenced and related links. I apologize for the sound quality and will make corrections if I get another chance to do another webinar/screencast. Here is the link to the recording:
http://webinars.developwithpassion.com/webinars/3
I didn’t mention this in the screencast but before we began switching our company’s production website to FubuMVC, we first devised a way to get the benefit of FubuMVC Behavior Chains while still on the ASP.NET MVC framework. Please feel free to read more about that successful venture at Bob Pace’s blog post on the MvcToFubu project.
]]>For the last several months at work, we have been gradually migrating our production website to use the .NET open source FubuMVC web framework. Before committing to a whole new infrastructure for our site, some brilliant talent on our team decided to improve our experience on the ASP.NET MVC framework by integrating in one of FubuMVC’s most compelling features, behavior chains. Bob Pace originally blogged about the process in a two part blog post series (part 1 and part 2); since then, he has created a packaged solution called MvcToFubu which he just recently started another blog post series about.
Behavior chains are by far my favorite feature of FubuMVC. I believe they single-handedly revolutionized our use of the MVC pattern in a web environment. In fact, I like to think of it as a whole new pattern, the BMVC Pattern (I was told by a coworker that the ‘B’ should stand for “Better”, and while that’s true, I’ll stick with “Behaviors”). No, I’m not sincerely trying to coin yet another design or architectural pattern buzzword. However, I am most sincerely not joking when I say behavior chains add a whole new dimension to how I design web architectures. I find myself creating composable pipelines of cohesive, testable components in lieu of monolithic controllers and actions.
Have you ever looked at your controller actions and thought “Man, there’s a lot going on here!” (validation, authorization, transactions, ORM setup, logging, caching, etc.)? If you haven’t, I urge you to take a conscious look at your controller actions and try to identify code that doesn’t directly relate to the purpose of the page (or sub-view of the page). Shouldn’t there be a way to move some of this secondary infrastructure code out of our controller actions and have it be applied conventionally to the correct actions/routes? Yes, and this is exactly what Behavior Chains are meant to accomplish, the ability to create composable pipelines of reusable behavior nodes for each route in your site.
Behavior chains can be built up in any order you want for each route via Conventions and Policies, which are extension points that allow you to inspect and modify the entire Behavior Graph that was built by FubuMVC during startup bootstrapping. The behavior graph is the modifiable collection of all behavior chains in the site, initially built with just a few default behavior nodes for each route. A typical behavior chain by default usually has an ActionCall node (representing the call to a controller action) and an Output node (often a tie-in to a view rendering engine, but can also be JSON or anything else). Other behavior nodes can be sprinkled into the pipeline in any order in front of or in between the ActionCall and the Output nodes. As mentioned before via the use of Conventions and Policies, it is common practice to filter your list of behavior chains using LINQ extension methods and then add a specific behavior node into the matching chains at a specific location in the chain. This truly elevates coding to conventions to a new level; some of the filtering criteria regarding controller actions that you may come up with could be as follows:
One more aspect of behavior chains that is important to point out is the logistics in how behavior nodes are chained together. The chains are not merely a linked-list of behavior nodes that are traversed linearly. Instead the makeup of the chains is better conceptualized to be an outer behavior node wrapping around an inner behavior node, which is then itself an outer node wrapped around another inner node, and so on. In fact, the FubuMVC developers like to refer to this chaining architecture as the Onion Layer Model or the Russian Doll Model (conjuring up a mental image of the nesting Russian doll toy called a matryoshka doll or a babushka doll). When an outer behavior node has finished executing it’s logic, it can then optionally execute it’s inner behavior node logic (or stop the behavior chain entirely in order to return early with a response, an error, or even transfer to a different route and behavior chain). One of the key advantages that this wrapping architecture can afford is the ability for an outer behavior to run additional logic or clean up code after all inner behavior nodes have completed (i.e., transactions can be committed, connections can be closed, sessions can be disposed, etc.).
I find that my controller actions are much thinner and more focused on the base case and the successful code path. All the exceptional code paths and edge cases have been identified and deal with by the other behavior nodes in the chain, prior to my action call being invoked. My action call can now focus on the essential logic that differentiate this route from others. That essential logic doesn’t get lost anymore in the necessary infrastructure code that should be a secondary concern. And all of this logic can be broken up into cohesive, reusable components within a flexible request/response pipeline that can be built up automatically by coding to whatever conventions you deem beneficial. Behavior chains really do add a whole new first-class citizen to the MVC design pattern. Do yourself a favor and check out the MvcToFubu solution mentioned at the top of this post, and bring some FubuMVC behavior chain goodness to your ASP.NET MVC website. I think you might be as impressed as we were and yearn for the other killer features of FubuMVC as well.
It came to my attention recently that there are a few companies in other dev circles with all-star developers who give back to their communities by doing apprenticeship programs or open door programs at their workplaces. Basically the idea is to let people come into your workplace and pair program with them, sharing knowledge and best practices. I believe we need more of this sense of community going on in the .NET space. I’m sure I’ve probably overlooked something (or perhaps I’m about to spout off craziness and heresies), but it just seems like we should be able to work out the logistics to make this a reality.
The interesting thing is that this really isn’t a new concept in other professions. For example, the culinary world has had the idea of staging for quite a while. Sometimes it’s a final part of an interview process and other times it’s just to swap techniques and recipes. As mentioned above, I was recently informed of a few dev companies that have taken this idea to our profession. In the Rails community, Hashrocket has an apprenticeship program where you can apply to be a part of their team for a week and soak in some of the awesome skills and techniques they possess. It appears they vet their candidates a bit to ensure they get the right kind of people coming in who are motivated and ready to learn, and also because the code being worked on is for real Hashrocket projects for their clients. In the Node.js space, Joyent has a program called Office Hours where devs can register for free in advance to bring in something to work on with a Joyent employee at end of work day and get their feedback & expert advice. They provide snacks, drinks, and a comfortable work environment.
I think these ideas are brilliant and foster real community among developers. Although this idea alone can improve the interview and hiring process, I think it can go further than that to help our profession mature in the sharing of best practices and proven techniques. For most .NET companies out there, I’m willing to bet there are few legitimate roadblocks that would prevent instituting programs like these. Being completely honest with yourself, how much of what you work on each day is top secret intellectual property that is core to the success of your company and absolutely cannot be seen by outsiders? Even if there are parts of your code that should not be seen by strangers, I’m sure there are projects and tasks to be done on the other parts of code that will still prove instructive for your invited guests. I encourage our dev community to do whatever it takes to make this happen: have the visitors sign non-disclosure agreements, interview and re-interview potential candidates to meet satisfactory requirements, don’t allow them to hook up thumb drives to your machine and steal your secrets, whatever! It doesn’t matter if it’s a week-long thing or a few hours; let’s realize we’re all in this together and build this community up to have a higher standard of software craftsmanship.
]]>A very talented coworker and I recently went to JP Boodhoo’s Develop with Passion bootcamp training course (previously known as Nothin but .NET). I had originally heard of the course from Dave Tchepak who attended the course nearly two years ago. This blog post is a wrap-up of my thoughts and feelings about the course itself, JP Boodhoo as the instructor & experienced craftsman, and the .NET development community in general.
My prior impressions of the course from what I had read was that I would be in awe of JP’s coding and design skills and very overwhelmed with the feeling that I didn’t really know how to code; also that I would get no sleep at all that week, going from early morning to past midnight every day. I had also heard from a previous course member (off the record) that JP’s organization and direction with course material was a bit disorganized back then (luckily this has changed significantly since he had taken the course). I was really excited to see some good BDD in action, get my TDD passion ignited again (I had actually mentioned on twitter recently that my motivation for test was waning as of late), and see JP’s mad Vim skills (since I have gotten into that the last few months).
Overall, I was pleasantly surprised with the fulfillment of most of my positive expectations and initial impressions going into the course. JP’s personality and passion are infectious, and his teaching skills are pretty top-notch. The course material was well-structured and there was a clear direction for what we hoped to cover throughout the week (and yet, he was still flexible enough to be able to briefly entertain an off-topic direction the students wanted to take the course). He stayed true to the advertised 9am-9pm every day no matter what. He said that when he did the late nighters with previous courses, people came back the next day with little sleep and trashed as far as coding & learning capabilities go.
Watching him code so efficiently is quite a treat. He automates anything he does more than once so that a particular task doesn’t ever get in his way and hold him back the next time. His AutoHotkey scripts and automation tools that he has put together are absolutely incredible. He briefly touched the mouse two times the entire week, and it only for when one of his programs or Windows VM was wigging out. Screens and dialogs flew by so fast, you could tell he wasn’t going to let any UI (even Visual Studio) get in the way of his productivity to code. He mentioned he will be open sourcing some of his scripts that he uses to keep his development environment consistent across OS platforms. Definitely cool stuff.
As far as JP’s actual coding/design skills, I am absolutely floored. He certainly understands C# and .NET well. But it’s his Object-Oriented design skills that blew us away. I’ve realized I don’t really know OO design. I think too mechanically about the implementation of the solution instead of focusing on breaking down the problem into smaller manageable chunks. His TDD skills (both Top Down Design and Test Driven Development) are amazing. I’ve never been a fan of the TDD samples and intros I’ve seen (where the instructors do the absolute dumbest thing to make the test pass; JP admitted that style didn’t jive with him either). I’ve always thought that Behavior Driven Development and Top-Down Design were the way to go if I were to have tests drive my designs (or have tests at all for that matter; I’m still having trouble finding enthusiasm for tests written after development for modules and units). So it was great to see a master at that particular craft. It made me excited to use tests as a tool to guide my OO design, behavior discovery, and problem solving.
Some of you may recall that I had previously investigated BDD, Machine.Specifications, and Top-Down Design a year and a half ago doing the Anagram Code Kata. I think I can recognize better design and responsibility break-up and assignment nowadays. I likely will blog on these subjects again this year with new & improved passion and knowledge.
There was one sad takeaway for me personally from the course, which was a sense of disappointment in the .NET development community in general. Giving myself some time since the course to really think it through, I’m realizing I’m probably being too tough on our community, as I know each Dev community out there suffers from it’s own set of problems. But I don’t want to excuse our shortcomings either; rather, I would like to point them out so that we can focus on them and root those weaknesses out. I think our community in specific is unique in that there is a big shot company that dictates the direction our community should take, sometimes taking credit for all the innovation. Things are improving lately no doubt; it appears Microsoft is listening more and more and trying to let outsiders get involved and contribute.
But we as .NET devs need to step up and take matters into our own hands, we need to take better control of the code we write and encourage best practices within the realms of our responsibilities and influence. I believe many .NET shops rely on the tools sent down from above so heavily that we don’t really solve problems anymore. JP’s focus on problem solving and code katas from the course attack this deficiency at its core. I do somewhat blame our university Computer Science programs for not teaching real OO (a great discussion on formalism versus hermeneutics in a book I highly recommend, Object Thinking), but at some point we have to take what we were given and no matter our current standing, focus on sharpening our skills through practice and continuous learning. It boils down to individual responsibility for our current skill set and progression as a software craftsman. JP’s course does a terrific job of waking you up to that realization and inspiring you to be a better professional at your craft.
I have a few takeaways from the course that I’ll publicly spell out now. First, I (and hopefully our whole team) need to better understand design patterns and have them become integral to our code discussions. JP was all about efficiency in keystrokes and related dev tasks, and it seems natural that our language and code discussions could benefit from more efficient communication using design pattern terminology to convey a lot about a specific design concept in only a few words. Second, I hope that I (and our team again) can really put our tests to work for us, helping us recognize the problem to be solved or the behavior that needs to be captured, and then guiding us toward a simple solution. The third takeaway is related to that last part, and that is to have simple solutions that are distributed across simple object implementations (focusing on behaviors) that adhere to SOLID principles. It appears the key is to not focus on implementation specifics until the very last responsible moment; if done right, the implementations to create should be so clear cut and trivial, almost spelled right out in the tests themselves. It’s hard to put into words, but it seems crucial to focus first on the collaborators needed while designing code at the higher levels, finding ways to offload as much responsibility as possible (at least that’s what it felt like) onto dependencies that have yet to be designed and implemented. Designing Top-Down in this way gave such simple and elegant solutions that were totally extensible and adhered to the Open/Closed Principle better than any other code I have ever seen before. Perhaps as I blog more about this, I’ll be able to better put this style of coding into words (especially as I try to better understand it myself; it all seems so magical still!).
Overall, a great course that I highly recommend to all. JP has mentioned doing a similar course in Ruby, which I personally find especially interesting, given the same design/coding techniques would seem to run against a lot less friction compared to the heavy reliance on interfaces and mocking that C# require.
Stay tuned as I try to put some of this stuff into practice!
]]>In LINQ, you can query in-memory data collections using Method Syntax (the IEnumerable extension methods) or Query Syntax (the SQL-like, declarative syntactic sugar on top of the underlying method calls). I was using the query syntax to join a few in-memory collections and then project (like a SQL SELECT) the denormalized data into a new object that only contained a few pieces of information from each of the separate data collections. The code could look similar to this (we’ll go with a shopping example in honor of the upcoming holiday season):
var shoppingLists =
from person in people
join wishlist in wishlists on person.Id equals wishlist.personId
join store in stores on store.Id equals wishlist.storeId
select new
{
PersonName = person.Name,
StoreName = store.Name,
NumItems = wishlist.Items.Count
};
However, I ran into the following intellisense red squiggly error in the Visual Studio C# code editor on the second join statement:
The type arguments for method ‘IEnumerable<TResult> System.Linq.Enumerable.Join<TOuter,TInner,TKey,TResult>(this IEnumerable<TOuter>, IEnumerable<TInner>, Func<TOuter,TKey>, Func<TInner,TKey>, Func<TOuter,TInner,TResult>)’ cannot be inferred from the query.
The error might also end with a statement that says "Try specifiying the type arguments explicitly."
The underlying problem is best understood by examining the method signature of the LINQ Join extension method and how the LINQ query syntax for the keyword join maps to that method call in the compiler. The type signature for the extension method is as follows:
public static IEnumerable<TResult> Join<TOuter,TInner,TKey,TResult>(
this IEnumerable<TOuter> outer,
IEnumerable<TInner> inner,
Func<TOuter,TKey> outerKeySelector,
Func<TInner,TKey> innerKeySelector,
Func<TOuter,TInner,TResult> resultSelector)
Consistent with all extension methods, it is public static and has a this keyword in front of the first method parameter, allowing you to call the Join method as if it had been declared in the original definition of the IEnumerable<> class or one of its inheriting children classes.
The first two parameters are the two data collections being joined, and in the case of my LINQ query above, they would be the joined collection of people & wishlists and then the stores collection, respectively.
The next two method parameters take delegates or lambda expressions that select the key value from each collection participating in the join, outer collection key selector first and then the inner key selector second. The key values of each item in each collection will compared to each other for equality. It will use the normal definition of the Equals method for your object type (or if you specified a custom one in your class definition), or there is another Join extension method signature that takes in another method parameter that an object that conforms to the IEqualityComparer<TKey> interface. The key selectors from our LINQ query sample would be the two member access expressions in the "on store.Id equals wishlist.storeId" statement.
The last parameter is a delegate or lambda expression that takes in a item from each participating data collection and projects the items into a new form (like the Select LINQ query method and its SQL equivalent). The two items being passed into the Func have passed the equality comparison of each item’s key value, exactly the same as how rows from two tables match up in an inner join statement in SQL.
As for the generic type parameters in this extension method definition, TOuter and TInner are the types from the two collections participating in the join. TKey is the type returned by the key selectors; notice that in order to compare the equality of each key selected, they need to be of the same type. In our example, TKey is likely int or long (or maybe Guid) since we’re dealing with Id properties of all of our objects. TResult is the type returned by the projection Func<TOuter,TInner,TResult> resultSelector; in our case, we returned an anonymous type consisting of three properties (PersonName, StoreName, and NumItems).
To answer this, let’s take a look one more time at my LINQ query syntax, how it would map to LINQ method syntax, and the Join method signature again all side-by-side:
var shoppingLists =
from person in people
join wishlist in wishlists on person.Id equals wishlist.personId
join store in stores on store.Id equals wishlist.storeId
select new
{
PersonName = person.Name,
StoreName = store.Name,
NumItems = wishlist.Items.Count
};
// Equivalent LINQ method syntax
var shoppingLists2 = people
.Join(wishlists,
person => person.Id,
wishlist => wishlist.personId,
(person, wishlist) => new { person, wishlist })
.Join(stores,
store => store.Id,
something => something.wishlist.storeId,
(obj, store) => new
{
PersonName = obj.person.Name,
StoreName = store.Name,
NumItems = obj.wishlist.Items.Count
});
public static IEnumerable<TResult> Join<TOuter,TInner,TKey,TResult>(
this IEnumerable<TOuter> outer,
IEnumerable<TInner> inner,
Func<TOuter,TKey> outerKeySelector,
Func<TInner,TKey> innerKeySelector,
Func<TOuter,TInner,TResult> resultSelector)
The issue is lines 19 and 20 above; they need to be switched. The compiler can’t properly infer the generic type parameters because we have our key selectors out of order in the argument list when invoking the Join method. If you wanted to be explicit about your generic type parameters assignments, then Join<TOuter, TInner, TKey, TResult> would be Join<Temp1, Store, int, Temp2>, where Temp1 and Temp2 represent the magical type definitions that the compiler creates underneath the covers when you use anonymous types in your C# code. Notice the outerKeySelector needs to come before the innerKeySelector. It appears the "on store.Id equals wishlist.storeId" syntax is not a commutative operation.
One last thing, I just realized that the error(s) given in Visual Studio when you actually compile the code are different than the intellisense red squiggly error message. It appears an actual build of the code is able to be a little more omniscient than the quick compile that intellisense does as you type. The error (and intelligent suggestion) the compiler offers says something like this:
The name ‘store’ is not in scope on the left side of ‘equals’. Consider swapping the expressions on either side of ‘equals’.
While reading the MSDN article on the LINQ Join method, I thought the "Remarks" section has some really great stuff about deferred execution, default equality comparison, differences from SelectMany, sort order perseverance, and so on:
This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach in Visual C# or For Each in Visual Basic.
The default equality comparer, Default, is used to hash and compare keys.
A join refers to the operation of correlating the elements of two sources of information based on a common key. Join brings the two information sources and the keys by which they are matched together in one method call. This differs from the use of SelectMany, which requires more than one method call to perform the same operation.
Join preserves the order of the elements of outer, and for each of these elements, the order of the matching elements of inner.
In query expression syntax, a join (Visual C#) or Join (Visual Basic) clause translates to an invocation of Join.
In relational database terms, the Join method implements an inner equijoin. ‘Inner’ means that only elements that have a match in the other sequence are included in the results. An ‘equijoin’ is a join in which the keys are compared for equality. A left outer join operation has no dedicated standard query operator, but can be performed by using the GroupJoin method. See Join Operations.
Thanks, I hope this proves helpful to someone, as it was certainly eye-opening to me.
]]>I learned a lot from my last foray into implementing my own Maybe Monad. Apparently, it got noticed by and was actually useful to Derick Bailey, as it was one of the resources he used in his own personal exploration into Monads. Of course, Derick did the topic a bit more justice, as he has a propensity to do.
Nonetheless, I have done as I said I would and reworked my Murray Monads into a cleaner, more concise implementation using extension methods, much like the implementation I introduced from Dmitri Nesteruk in the last post. So without further ado, I’d like to introduce you to the MaybeMonadExtensions.
If you want to get an introduction to what Monads are and how I got to where I am today with this little library, then you really should check out the previous post on this topic. I will however give a few brief code samples to show you what it can do to tidy up all those null checks using the Maybe Monad.
Here is code we tend to write all the time:
string postCode;
if (person != null)
{
if (HasMedicalRecord(person) && person.Address != null)
{
CheckAddress(person.Address);
if (person.Address.PostCode != null)
postCode = person.Address.PostCode.ToString();
else
postCode = "UNKNOWN";
}
}
It can look like this with some Maybe Monad love:
string postalCode = person.If(HasMedicalRecord) .Access(p => p.Address) .Apply(CheckAddress) .Access(a => a.PostCode) .Convert(pc => pc.ToString(), "UNKNOWN");
I know! Groovy, huh?!
The source can be found here: http://bitbucket.org/murrayondotnet/maybemonadextensions/
Let me know if you put this to good use!
Off and on for the last six months, I have probably seen and read at least a half dozen introductions and tutorials on monads, a functional programming construct based on the mathematical monads from category theory. Monads are known for their headache-inducing explanations, and I’ve had my fair share. Every tutorial I’ve committed time to mentions the endless possibilities of using such constructs to improve your code, but I’ve had a hard time finding real-world examples of using this stuff out in the wild (much like this Stack Overflow poster, whose link I only found just now). It has recently become a quest of mine to find some C# code we all write that could be improved by a monad.
I am now going to attempt to introduce a Maybe monad that I cooked up, but I warn you that I have no idea what I’m doing or if I’m staying within the rules for what makes a monad by definition a monad. Don’t worry though, you have nothing to lose; I’m the only one taking a risk at looking foolish, and it’s all at the expense of my reputation and not yours. Should be fun!
From the Wikipedia article linked to above:
“In functional programming, a monad is a kind of abstract data type constructor used to represent computations (instead of data in the domain model). Monads allow the programmer to chain actions together to build a pipeline, in which each action is decorated with additional processing rules provided by the monad. Programs written in functional style can make use of monads to structure procedures that include sequenced operations, or to define arbitrary control flows (like handling concurrency, continuations, or exceptions).
…
“A programmer will compose monadic functions to define a data-processing pipeline. The monad acts as a framework, as it’s a reusable behavior that decides the order in which the specific monadic functions in the pipeline are called, and manages all the undercover work required by the computation.”
The promised tenets of monads are increased code readability and algorithm composability, an example of preferring declarative code over imperative, procedural code. In the case of the Maybe monad, it is a wrapper around types that might have an invalid state like null or nothing. The lure of the Maybe monad is the ability to deal with values that might be in this invalid state without having to explicitly check for null before continuing or exceptions being thrown in your algorithms.
I first found a post by Rinat Abdullin where he introduced his version of a C# Maybe monad. I downloaded the library and attempted to figure out how to use it in code I could possibly write any given day. I tried mimicking the HttpContext.Request.Params collection commonly used in ASP.NET pages and HTTP handlers to retrieve the passed in query string or post values on the Request object. If you ask that collection for a parameter that was not passed in on the request, it returns a null instead of the string value. Common practice is to then use a bunch of if statements and key off the fact that certain query string or post parameters may or may not exist.
The problem I ran into with Rinat’s implemenation is that I wrote the code using the monad how I thought it should help me, and then ran into compiler errors and runtime exceptions because of the expectation that Maybe objects couldn’t be constructed around null values. Here is the twitter conversation I had with Rinat in regard to this misconception that I had:
And so I did.
I don’t even like the name. I’m also not sure this code will ever be used again…by anyone, let alone me (especially since I like some other guy’s implementation better; read on to watch me embarrass myself). But whatever, Visual Studio asks for a project/solution name and so there it is. I need some cool theme like Rhino, as used by Ayende.
Speaking of Oren (I will use his proper name instead so that he doesn’t get stuck at the airport unable to board his flight), he posted a link on Twitter to an article where Dmitri Nesteruk created his own version of the Maybe monad, just as I was starting my experiment. He used extension methods instead of an explicit monad object wrapper in order to enable the pipelining composability that you can also do with Rinat’s implementation. Really clever, but there was one small thing that bothered me a bit, and that is how the monad chaining begins with Dmitri’s API. (Actually, I’ll prove I’m wrong by the end of the post; this is the cause of upcoming embarrassment mentioned above, hooray!) Because of this, I decided to use his example problem as the basis for my sample that I will show, in order to increase comparability.
So I set off to try and merge the good points of both Rinat’s and Dmitri’s implementations. I have put my source code up for viewing at Bitbucket.org: http://bitbucket.org/murrayondotnet/murraymonads
Yes, we are. Let’s see some code finally.
Here is the code we don’t like writing all the time:
string postCode;
if (person != null)
{
if (HasMedicalRecord(person) && person.Address != null)
{
CheckAddress(person.Address);
if (person.Address.PostCode != null)
postCode = person.Address.PostCode.ToString();
else
postCode = "UNKNOWN";
}
}
This is actually code that you wrote; I took it just yesterday from your current project at work. I know you’re feeling a bit defensive because of this, but I’m here to help so put down the attitude please. We’re gonna make your life easier…I hope…
All in all, it’s code you see every day and there’s not much really wrong with it per se. I just think it could be cleaner I suppose. So using Dmitri’s extension methods, you can get that logic down to this:
string postCode = this.With(x => person) .If(x => HasMedicalRecord(x)) .With(x => x.Address) .Do(x => CheckAddress(x)) .With(x => x.PostCode) .Return(x => x.ToString(), "UNKNOWN");
As long as that last ToString method call doesn’t return a null (which one of my use cases does in the TestConsoleApp included in my source code), you will either get the value or “UNKNOWN”. Looking good!
So my version of the Maybe monad comes out pretty close to this last one:
string postCode = person.If(p => HasMedicalRecord(p)) .Access(p => p.Address) .Apply(a => CheckAddress(a)) .Access(a => a.PostCode) .Convert(pc => pc.ToString(), "UNKNOWN"); // or even string postalCode = person.If(HasMedicalRecord) .Access(p => p.Address) .Apply(CheckAddress) .Access(a => a.PostCode) .Convert(pc => pc.ToString(), "UNKNOWN");
The difference between Dmitri’s and mine being how the chain is first started, on the Maybe person object instead of Dmitri’s use of the this keyword.
Here’s what you’ve been really waiting for where I eat my own words. Dmitri’s code can also be written like this:
string postCode = person.If(HasMedicalRecord) .With(x => x.Address) .Do(CheckAddress) .With(x => x.PostCode) .Return(x => x.ToString(), "UNKNOWN");
…which appears to take away any advantage of my library. I officially like his solution better, I just don’t like some of the method names. So perhaps I will create another blog post and Bitbucket repo with a new library of extension methods instead. I will leave up this mess-of-a-post public (as well as the source code) so that perhaps others can see the process I went through and learn from my mistakes.
Let me have it. Let me know how crazy I am or if this is even useful. Let me know if I’m completely misusing the monad principles. Or, by some weird miracle, let me know if I’m on to something. Actually, if Dmitri is on to something. Either way, I want to hear from you. Thanks.
I am currently reading the Real-World Functional Programming book from Manning Publications. It’s mostly an exercise in learning F#, but also to improve my C# code as well. I finally learned what the F# pipelining operator (|>) is all about, and I was so impressed I thought I would try to figure out how to bring it into C# if possible.
Here is some F# code that defines a list of integers and a pair of functions. They are then used in a few “higher-order” functions (functions that utilize other functions as parameters or return values, according to my book). List.filter is equivalent to the LINQ IEnumerable.Where() extension method, and List.map is equivalent to IEnumerable.Select().
let numbers = [ 1 .. 10 ] let isOdd(n) = n % 2 = 1 let square(n) = n * n List.filter isOdd numbers List.map square (List.filter isOdd numbers)
The interesting part is the last line of code, where it is argued that you lose a little bit of code readability because the function that is actually executed first (List.filter) is found second in the expression statement. Enter the F# pipelining operator:
let squared = numbers |> List.filter isOdd |> List.map square
Even though we’re no longer a compact, one-liner expression, I think we’ve gained some code readability. It says to take the list numbers and perform the following manipulations in the exact order listed. Cool stuff, if I say so myself.
(Sorry, I’ve been watching the movie Kung Fu Panda a lot with my kids lately. Funny stuff.) So can this awesomeness be brought over into C#? I opened up Visual Studio and tried to see if I could figure out how to define my own operator or reuse one of the existing ones.
I realize C# is a pretty tough language to flip keywords and operators around and invent new syntax with (ok, it’s impossible), but I tried anyway. The type of code readability issue I wanted to remedy is the same as with the F# code snippet above. I thought it would be nice to make C# code like the following statement more readable and digestible (especially the execution order):
Convert.ToInt32(Context.Request[“requestParam”].Trim());
As I started to (unsuccessfully) beat out some pretty rough code, I ran into problems with trying to make extension methods that are operator definition overloads and errors saying “One of the parameters of a binary operator must be the containing type.” It just wasn’t working and then I began to realize that C# already has a pipelining syntax, even if only in one small subset of the .NET API. You can find a decent pipelining syntax with the Fluent Interface used for the LINQ extension methods. Its power is in the fact that the functions return the object type that was passed in and acted upon, so that more functions can be chained on afterward in a composable manner. The ability to do things like collection.Where().Select().OrderBy() is a much more declarative and functional solution to imperative for loops (and it makes it more readable and intention revealing in my opinion). It’s a difference in describing more what you want to be accomplished rather than how it is to be accomplished (especially when the how in programming languages is more readable to a computer than it is a human).
So the good news is that you indeed can have a nice pipelining syntax in C#, especially for APIs that are designed with Fluent Interfaces like LINQ. However, it doesn’t solve code readability issues with code snippets like the C# fragment I showed above. For those regular, non-fluent method calls in the rest of the .NET API, you must use old-fashioned Object-Oriented Programming practices of breaking code up into meaningful, atomic units of statements and naming and encapsulating functions cohesively in order to get more readable code.
I have been intrigued by the Reactive Extensions for .NET (Rx Framework) for quite a while now, especially the fundamental shift from a “push” data model as opposed to the “pull” model most familiar to us. I have thought to myself that this must be a game changer in how data and events are consumed in every-day software. I didn’t quite understand how it all works or how to use it from the few samples I had seen, but I knew if I gave myself a little time to really sit down and look at it, my life would be changed forever.
However, after reading some articles, watching some recorded presentations, and even attending a MIX ‘10 session in-person on the Reactive Extensions, I felt increasingly lost and confused by what this framework could offer me. I wasn’t understanding some fundamental ideology of the framework and it was jading my learning experience when watching/reading sample code. Well, I think I have finally figured out what the mental roadblock was and will attempt to explain the framework here as I best as I understand it, in hopes that it can be of help to others not getting what the fuss is all about. I can sense that I’m not the only one who has been turned off by the unsuccessful attempts by Microsoft to explain this revolutionary framework to the average developer. Apparently those who had a hand in designing the framework are just so incredibly intelligent that they are having a hard time discerning that the common man is not immediately getting it.
Essentially what Microsoft did is create a mathematical dual of the IEnumerable and IEnumerator interfaces using Category Theory, and named them IObservable and IObserver respectively. In its bare simplicity, this is done by merely reversing the direction of your arrows in your mathematical functions and compositions. The following images of a few slides from Erik Meijer’s MIX ‘10 Session that were the best visual explanation I have seen:
Duality Example with De Morgan’s Laws
Creating a Dual of IEnumerable
Creating a Dual of IEnumerator
The reason the mathematical proof part is significant is because we can guarantee that LINQ syntax and operators will work on these new collection and data subscriber interfaces, just like they did with IEnumerable collections in LINQ-to-Objects. This new LINQ provider is commonly called “LINQ over Events.”
This is actually where I think some of the confusion came in for me. Mostly what I didn’t understand is how you can write LINQ queries into pushed data (already sent and also yet to be sent), when essentially that task is indeterminate by nature. What I have figured out is that the LINQ syntax is not entirely for the data being pushed, but also used for molding the subscriptions themselves. What the LINQ syntax provides you is a way to compose, filter, project, aggregate, and throttle data and event subscriptions in a powerful, declarative way.
A common workflow seems to be to create one or more observable collections (and there are many ways to do so) and then customize and compose a subscription based on the subscription(s) using LINQ syntax. All that’s left then is to set the subscription in motion, giving it some sort of callback or lambda expression to be called as data is asynchronously pushed in and processed.
Here is a very simple code example that illustrates this process quite plainly:
using System;
using System.Linq;
namespace TestRX
{
public class Program
{
public static void Main(string[] args)
{
var oneNumberPerSecond = Observable.Interval(TimeSpan.FromSeconds(1));
var numbersTimesTwo = from n in oneNumberPerSecond
select n * 2;
Console.WriteLine("Numbers * 2:");
numbersTimesTwo.Subscribe(num =>
{
Console.WriteLine(num);
});
Console.ReadKey();
}
}
}
It is interesting to watch the code run and think about what it’s actually doing. It sits on the Console.ReadKey() line while the observable collection is asynchronously pushing data to the lambda expression containing the Console.WriteLine() statement.
It looks like the Reactive Framework is not just a better way to handle events and pushed data, but it also appears from what I’ve read that it could be used as a nicer API for sending off asynchronous process threads.
Microsoft DevLabs has created the Reactive Extensions for .NET (and it’s associated languages), but it has also came up with the Reactive Extensions for Silverlight and JavaScript (can be found on same webpage). This proves very interesting as JavaScript (and Silverlight) tends to be quite event driven because of its close interactions to UI elements in HTML (or XAML). In fact, one of the most encouraging example use cases that I’ve heard was that of having some HTML text field that you wanted to auto-search as the user types, but not until the user pauses for a second; they can also hit Enter. What the Reactive Framework could provide you is a way to aggregate more than one event source (alphabetic and Enter key strokes and the timer) into a single event handler using clear, declarative framework statements.
Here are other links I would recommend for additional examples and explanation:
Enjoy! Let me know via comments if you start using it for something real cool and useful.
I was having fun working on the Toughest Developer Puzzle Ever 2 when I came across a problem that asked for the coordinates into a grid of numbers of the upper left corner for a 4x4 sub-grid with a magic sum of 34, meaning all rows, columns, and the 2 diagonals all sum up to 34. Instead of trying to figure it out manually, I probably did what many programmers would do and wrote a quick and dirty program to accomplish the task for me.
I didn’t try to use any optimized search algorithm or anything, as I knew that the problem space was very small and specific and that computers are really fast. So I figured I would just write a brute-force iterate through every row and column type of an algorithm (in Big O notation being O(mn) or just O(n2) if the original grid is square), knowing it would return in milliseconds.
I quickly realized that each iteration was not dependent any previous iteration for its calculations, so I decided it might be a good time to try out the new Parallel.For loops in the .NET 4 Framework that would multi-thread the iterations of the loops. Here’s the implementation I threw together:
using System;
using System.Linq;
using System.Threading.Tasks;
namespace Find34Grid
{
public class Program
{
private static readonly int[][] Grid = new []
{
new [] {16,3,2,13,15,10,3,6,41,15,14,4,12,8,7,1,12},
new [] {5,10,22,8,4,5,16,7,9,7,6,12,5,11,10,8,5},
new [] {9,6,7,12,14,11,2,13,16,3,2,13,15,10,3,6,5},
new [] {41,15,14,4,12,8,7,2,5,10,11,8,4,5,16,9,15},
new [] {16,3,2,13,15,10,3,6,15,10,16,2,3,13,16,2,3},
new [] {5,10,11,8,4,5,16,9,4,5,5,11,10,8,5,11,10},
new [] {9,6,7,12,14,11,2,13,14,11,9,7,6,12,9,7,6},
new [] {41,15,14,4,12,8,7,1,12,8,4,14,15,1,14,15,1},
new [] {9,7,6,12,5,11,10,8,5,11,10,3,6,41,15,14,4},
new [] {4,14,15,1,9,7,6,12,9,7,5,16,9,9,7,6,12},
new [] {12,8,13,13,4,14,15,1,4,14,11,2,13,16,3,2,13}
};
private const int MagicSum = 34;
public static void Main(string[] args)
{
Parallel.For(0, Grid.Length - 3, i =>
{
Parallel.For(0, Grid[i].Length - 3, j =>
{
int[] sums = new []
{
Grid[i].Skip(j).Take(4).Sum(), // Row 1
Grid[i+1].Skip(j).Take(4).Sum(), // Row 2
Grid[i+2].Skip(j).Take(4).Sum(), // Row 3
Grid[i+3].Skip(j).Take(4).Sum(), // Row 4
new [] { Grid[i][j], Grid[i+1][j], Grid[i+2][j], Grid[i+3][j]}.Sum(), // Column 1
new [] { Grid[i][j+1], Grid[i+1][j+1], Grid[i+2][j+1], Grid[i+3][j+1]}.Sum(), // Column 2
new [] { Grid[i][j+2], Grid[i+1][j+2], Grid[i+2][j+2], Grid[i+3][j+2]}.Sum(), // Column 3
new [] { Grid[i][j+3], Grid[i+1][j+3], Grid[i+2][j+3], Grid[i+3][j+3]}.Sum(), // Column 4
new [] { Grid[i][j], Grid[i+1][j+1], Grid[i+2][j+2], Grid[i+3][j+3]}.Sum(), // Diagonal 1
new [] { Grid[i][j+3], Grid[i+1][j+2], Grid[i+2][j+1], Grid[i+3][j]}.Sum() // Diagonal 2
};
if (sums.All(x => x == MagicSum))
{
Console.WriteLine("y (row): {0}\nx (column): {1}", i, j);
}
});
});
Console.ReadLine();
}
}
}
Nothing special and certainly did the job. I didn’t really notice a performance improvement in anyway, but that was expected for such a small problem. I don’t really expect a whole lot of feedback on such straightforward code, but if there is a cleaner or more efficient way to utilize Parallel.For or Parallel.ForEach, please don’t hesitate to let me know via commenting to this post. Thanks!
]]>This post is part of a series on coding Kata, BDD, MSpec, and SOLID principles. Feel free to visit the above link which points to the introductory post, also containing an index of all posts in the series.
In this post, we will discuss some reasons why you might want to avoid using language primitives directly in place of domain objects. Specifically, I have been using String variables and objects to represent words up to this point. Esteban suggested I create a Word class for a few good reasons which I’ll layout for you. Out of sheer luck and good timing, Dru Sellers of CodeBetter.com also wrote on this subject shortly afterward and confirmed Esteban’s reasoning.
I mentioned in the previous post that Esteban had suggested creating a Word class instead of just passing around strings everywhere through my application. One good reason is that we don’t own the String class and so further modification to our string-based implementation is difficult because the logic is scattered throughout the application, instead of centralized under the responsibility and definition of one domain class. It is much more difficult to refactor the word-specific logic with its behaviors and attributes spread throughout the code. Even if your Word class doesn’t grow any more beyond a seemingly unnecessary wrapper of the String class, the code is more cohesive and ready for change should the need arise.
But more importantly, you designed the code thinking in a true object-oriented mindset. You have to keep in mind that the String class is someone else’s implementation, and is hardly ever sufficient in and of itself as a domain object within your solution. Think about it, the behaviors that a string object performs are so generic and multipurpose that you likely don’t need two-thirds of the class as defined (and there’s likely a few behaviors you really need that aren’t there). Of course nearly every application on earth makes use of the String class; but because of this fact, it has no meaning in and of itself within any given application. You would have to look at how the stings are actually used within the code in order to understand its unique application within the app’s context. Of course that task of research is much easier for everyone (including the original author of the code) if it’s all encapsulated within a dedicated domain object class. True object-orientation means describing in code form the properties, behaviors, and interactions/relationships of real world objects within your problem domain.
Esteban gave me a great example to illustrate these points. He said that you can always represent money as a decimal (and even when you use a domain object, it’s got to have a decimal language primitive underneath the covers). However, what happens when you need to attach metadata to the amount (like currency denomination), or if you need to change decimal precision? You would have to go through all of the code and make sure your use of decimal language primitives is modified uniformly in order to retain consistency. Also, mathematic operations involving money are hardly ever the same as their counterparts involving standard decimals, because currency deals with discrete values to a certain decimal precision. Typically when the behaviors and properties within our system begin to get complex, we are cognizant enough to create domain objects in order to bring it all under one class. We definitely don’t want to over-architect features and interactions before we need them, but I think there is power in this principle of abstracting away language primitives and instead encapsulating their use within domain objects located in just one place in your codebase. I believe it is one thing we can keep in mind to help guide us to better object-oriented thinking, and avoid language-oriented coding.
As mentioned above, Esteban’s thoughts were confirmed nearly verbatim by Dru Sellers of CodeBetter.com in his blog post that he wrote just a few days after I had the conversation with Esteban. A great coincidence no doubt, and worth a read; here’s the link:
So basically I created the following class implementation and then replaced string with a reference to this new class, Word:
public class Word
{
private string wordStr = string.Empty;
public Word(string wordStr)
{
this.wordStr = wordStr;
}
public override string ToString()
{
return wordStr;
}
public override bool Equals(object obj)
{
return wordStr == ((Word)obj).ToString();
}
public override int GetHashCode()
{
return base.GetHashCode();
}
public int GetCanonicalHashCode()
{
char[] letters = wordStr.ToCharArray();
Array.Sort<char>(letters);
return new string(letters).GetHashCode();
}
}
I have defined an overridden implementation for Equals(object) (so that the test assertions and other IEnumerable.Contains() queries work) and GetHashCode() (solely to satisfy a compiler warning). I also moved the GetCanoncialHashCode() method from AnagramGrouper, in order to better encapsulate it as a behavior a Word knows how to do innately.
One other change I had to make was to convert strings into Word objects in our NewlineFileParser, which I accomplished by using an IEnumberable.Select() call as shown below:
return File.ReadAllLines(filePath).Select<string, Word>(x => new Word(x));
Trust me, I am groaning with you, even as I wrote that heading text. The last thing we need is for me to rehash the topic again and flip flop my stance yet another time. Yes, that’s right I’ve changed my mind again. First I couldn’t understand what utility asserting methods were called would have under normal test scenarios. Then I changed my mind that perhaps using it would help my test specifications have clearer intent of what I am asserting. After an email conversation with David Tchepak, I think I’m now back to my original stance. Here is what Dave said that had me reconsidering, and I think it’s pretty sound reasoning (emphasis added by me):
…
“But seeing as you asked for it, here goes. :)
“I try and avoid AssertWasCalled like the plague. Generally I don’t care that some method was called, I care that my class does the work it needs to. If that involves calling a dependency then great, but that is not my class’ reason for existence.
“I prefer your original approach of stubbing out everything in the setup and having that tested indirectly. One reason I prefer this is I find it makes it easier to refactor: the assertions in my test don’t change, the class still does the same thing. However I can add or change dependencies by changing some wiring in the setup, and then make sure I haven’t stuffed anything up as my assertions still pass. I prefer that my tests specify what I want, not how it does it. To me, AssertWasCalled reeks of over-specification. The one exception is where I hit the external boundaries of my code, so where I want to send an email or something without side effects that I can test. Then the core of the behaviour is the call itself, so I’m happy to assert on that then.”
And with that I’ll promise to never bring this up again…unless of course I get swayed by someone else. :) In all seriousness, I think this is an interesting discussion and so if you have insight, please share via comment below.
Please leave your thoughts in the comments below in regard to creating domain objects over using language primitives (or heaven forbid, the AssertWasCalled debate). As far as this coding kata exercise, I need to take a high-level look at where this should go next. Perhaps the next post will tie up loose ends and see how our code performs on large text files as input. It may be that we will need to refactor our architecture to achieve better runtimes. If not, maybe we can still discuss where would could have headed if it had been necessary.
]]>This post is part of a series on coding Kata, BDD, MSpec, and SOLID principles. Feel free to visit the above link which points to the introductory post, also containing an index of all posts in the series.
In this post, we work on implementing the other main dependency, IAnagramsFinder. In doing so, you will discover that I have been a little naïve with this BDD methodology. Apparently I expected a different experience than what I ran into doing the exercise. In gathering feedback, I had a very interesting discussion about Object-Oriented Programming with Esteban that I will summarize and ask for further discussion from all of you. I’ll also show you one way to debug your code when running the MSpec tests via the console test runner.
During Part 2 of this series, Tobias left a comment about using mocks and AssertWasCalled, rather than stubs and defining fake responses. I didn’t understand how it would help, as it appeared they would accomplish roughly the same thing. With stubs and fake responses, if you don’t get back the return you are looking for, then you can infer that the dependencies weren’t called since you had previously hard-coded their return values when called. With mocks and asserting methods were called, you directly assert that the manager class did in fact call it’s dependencies.
When I looked more closely into using the AssertWasCalled method in Rhino.Mocks, I noticed that my strategy of using stubs and faked return values wasn’t very explicit in signifying that the ultimate responsibility of the manager class was to delegate to its dependencies. I mean the test context does rigorously stub out the dependencies, but the actual assertions weren’t clear what we were testing. Therefore, I decided to pop in a pair of AssertWasCalled statements (as found below), leaving the rest of the test the same as it was back in Part 2.
It should_result_in_list_of_anagram_sets_and_both_dependencies_should_have_been_called = () =>
{
fileParser.AssertWasCalled(x => x.ExtractWordListFromFile(filePath));
anagramGrouper.AssertWasCalled(x => x.FindAnagramSets(wordListFromFile));
result.ShouldEqual(expected);
};
However, when I thought to simplify the test code by removing most of stubbing setup, it appeared I had to leave most of the stub logic in tact in order to test if anagramGrouper.FindAnagramSets() was called with parameter wordListFromFile, which was one of the stubbed fake responses. The one dependency (the file parser) needed to pass it’s return value to the next dependency (the anagrams finder). I could not think of a way to accomplish testing that the manager class facilitated that without the stubbing logic (fake responses) that I created.
It would be great if someone could enlighten me since I’m such a newbie to mocking and stubbing. Otherwise, I feel the intent of the test is now more clear with the AssertWasCalled checks that I added, even if that I means I’m needlessly mixing stubbing and mocking. But one last question: should I try to stick more closely to the rule of thumb guiding me to stick to one all-encompassing assertion per test if possible? Let me know your thoughts.
So let’s continue onward with some more actual design and code. We are tackling the AnagramGrouper dependency, whose sole responsibility is to take in the parsed collection of words and find anagram sets. Here is the test spec (one thing to note is that in some of my earlier tests for the AnagramsFinder manager class and IFileParser, I had used string arrays liberally; I decided to go with IList<string> collections instead):
public class AnagramGrouperSpecs
{
[Subject(typeof(AnagramGrouperSpecs))]
public class when_given_word_list_collection
{
static AnagramGrouper sut;
static IEnumerable<IList<string>> result;
static IEnumerable<string> wordList = new[] { "wordA", "wordB", "Aword", "wordC" };
Establish context = () =>
{
sut = new AnagramGrouper();
};
Because of = () =>
{
result = sut.FindAnagramSets(wordList);
};
It should_result_in_anagram_sets_collection_of_length_1 = () =>
{
result.Count().ShouldEqual(1);
};
It should_contain_2_specific_words_in_the_anagram_set = () =>
{
result.First().ShouldContain("wordA", "Aword");
};
}
}
Here too I have basically two assertions (but split it up differently than last time); should I also try harder to avoid this as well?
When contemplating how to further breakdown this responsibility of finding anagram sets into subtasks, I figure we would need one piece of code that would compare two words and answer whether they are anagrams of each other, and another piece of code that would keep track of the various anagram sets (perhaps a collection or dictionary of string arrays). With what BDD has revealed to me thus far, I figure this means we are going to see two more dependencies (and interfaces, and spec tests) added to the codebase.
So I start with the word comparison task, thinking I could perhaps leverage the String.GetHashCode() method for assigning meaningful value to the list of characters in the string. However, string hash codes give importance to letter casing and order of characters, so a modified strategy would have to be utilized, even though GetHashCode() seems quite close to filling our need.
Now I probably should have come up with this ingenious algorithm all by myself, but…it is what it is and this is the best, most concise solution that I can put together to solve the problem. With that said, I ended up doing some internet searching in order to proof-of-concept my GetHashCode() idea. I came across pure genius in the form of Brian Mullen’s blog post entitled LINQ group by and GroupBy, specifically the second-to-last code snippet. Just String.ToCharArray() the word and then Array.Sort() the resultant array; now turn it back into a string and do GetHashCode() now. (As a side note, we can worry about case-insensitivity later if desired by simply doing a String.ToLower() on the word before getting the hash code.) Now anagrams will have the same hash code (Brian uses the term “canonical”, which I think is very fitting); it’s literally perfect in every way!
So now for the other dependency, I figure we could use a Dictionary<int, IList<string>>, where the key is the canonical hash code and the value is a string list collection representing an anagram set of words. These two dependency implementations are so simple and straight forward that I have a hard time seeing a reason to continue breaking out full-fledged object dependencies with interfaces and Dependency Injection. However, I will make a case for moving the canonical hash code logic outside this AnagramsGrouper class, but that will come later.
So let’s write some code to pass our test/spec:
public class AnagramGrouper : IAnagramGrouper
{
public IEnumerable<IList<string>> FindAnagramSets(IEnumerable<string> wordList)
{
Dictionary<int, IList<string>> results = new Dictionary<int, IList<string>>();
foreach (string word in wordList)
{
int canonicalHashCode = GetCanonicalHashCode(word);
if (results.ContainsKey(canonicalHashCode))
{
results[canonicalHashCode].Add(word);
}
else
{
results.Add(canonicalHashCode, new List<string>(new[] { word }));
}
}
return results.Values;
}
public static int GetCanonicalHashCode(string word)
{
char[] letters = word.ToCharArray();
Array.Sort<char>(letters);
return new string(letters).GetHashCode();
}
}
Maybe you noticed already, but there is a bug in my logic. At that time, I didn’t see it and needed to debug the code while running. Setting a debug break point in your code doesn’t work like you’re probably used to, and this is because we’re running the MSpec tests via its console test runner. I had to do some searching to figure this one out, and that’s what we’ll cover next.
After a short ride on the Google super highway, I found a blog post containing a November 2009 comment from Aaron Jensen, author of MSpec, that gave a few possible solutions. I opted to go with the Debugger.Break() solution, which I put at the very top of my FindAnagramSets method. It is a tad awkward because it will show a Windows error dialog box that says the program encountered a problem and Windows is searching the web for a solution. If you wait through that, it will give you an option to debug the program via a Visual Studio dialog where you select the currently running instance of Visual Studio as your debugger instance. You just remove the statement when you’re done (and you’ll really want to remember to remove it from your production code). Does the trick, though it is a bit obtrusive to your code base.
If you haven’t already figured it out, the bug is that my FindAnagramSets method is returning all anagram sets found in the dictionary. An anagram set of just one word is not really an anagram set, which is why our test specification which is based on that assumption failed. The fix is to filter out anagram word sets with less than two words before returning, like this:
// Return anagram sets with more than one word in it return results.Values.Where(x => x.Count > 1);
That will make the code pass the tests now. Remember to remove the debugger statement we added.
Just kidding. But I do want to discuss a reaction I had to BDD after implementing this anagram finding solution. After I completed the code and finished the tests, I was feeling pretty great. But then I started thinking about how I didn’t really go down the path that I felt BDD was originally trying to lead me down with AnagramGrouper, which was to create two more dependencies, just as I had done for the first managing class AnagramsFinder. I mean I felt pretty frustrated because I was digging BDD to this point. I’m not quite sure what my original expectations were, but I might have thought BDD was magically going to right my algorithms for me as well.
I decided to discuss the whole thing with Esteban to get some perspective on the issue. He told me he didn’t know a thing about BDD and how it’s supposed to work, but he figured BDD would lead me to decoupled, cohesive code design, but he couldn’t see how it would aide with algorithmic solutions to the problems at hand. I’m not sure why I was so naïve, but it makes complete sense. I mean these methodologies are designed to keep you disciplined to some proven, guiding principles, but you still need to use your brain and creativity to actually solve the problem. I guess it would help to get some feedback and suggestions from you all in regard to knowing when it is time to switch from BDD architecture design mode to algorithm design mode within the broken down dependencies that handle the very specific sub-responsibilities. Please let me know if you have found some guiding concept or question you’ve come up with that you put to the test in order to lead you into the right coding mindset for that particular context (design or algorithm; or do I even have the right categories?).
Perhaps it’s just that this design process feels so new to me. Maybe I have solely been in the coding, algorithm mindset this whole time throughout my career. Perhaps it will become more natural with practice and experience. I guess it just feels like I’m learning to drive a manual, stick-shift car or something, and I am a little rough with the clutch when transitioning between gears.
To take this discussion further, Esteban made a comment in our conversation about his personal thoughts on these methodologies (like BDD) and Object-Oriented Programming. He has come to believe that we invent these methodologies like flavors of the week “because as programmers we don’t really get OOP.” Perhaps OOP just comes more naturally to his thought process that most people (or he admits it’s possible he could just be completely ignorant), but he feels that there’s really just a few simple rules and guidelines to keep in mind at all times in order to keep your objects behaving like real life objects and not acting like data constructs. I encouraged him to give his theory some additional thought and make it a little more formal by writing it down. He did this and the following is a link to the resultant blog post:
3 Simple Rules to Good Object Oriented Code
Please read through that post and comment on it. His idea needs to be criticized and/or validated. He admits he may have no clue what he’s talking about, but I said that’s why it needs to be put to the test by posting it in the open. We both think it’s an interesting theory, but we’re unsure if it even holds any water. Perhaps Esteban’s theory is too simplistic to be practical for many programmers. Provide feedback and lots of it.
So I think I need to wrap this post up about now. I think I have raised enough discussion points to write a book, but please do take the time to comment on what I’m doing wrong, what I’m doing right, or how to make the BDD mentality more flowing and natural. I promised to talk about having a Word class as opposed to using primitive strings everywhere, and that will definitely be in the next post. Also, I think our AnagramGrouper class may need some more testing and flushing out of how it should handle spaces, punctuation, casing, and so on.
Thanks for sticking with me thus far.
]]>This post is part of a series on coding Kata, BDD, MSpec, and SOLID principles. Feel free to visit the above link which points to the introductory post, also containing an index of all posts in the series.
In this post, we will discuss how far to take the idea of creating mockable contracts via interfaces for testing purposes and to keep responsibilities modular. As I solicited feedback from commenters, I was getting hung up on applying this concept to every object and dependency. We’ll talk about how I’m now thinking that overly strict adherence to the practice can drive you nuts and decrease productivity. It is meant to help guide your design process along rather than hinder it. We’ll finish this post off with one of our two dependencies implemented in an extremely simple manner.
When we left off last time, we had defined two separate responsibilities that AnagramsFinder was going to push down into its dependencies. As I started to write a specification test for the newly created NewlineFileParser, I realized it would be difficult to stub based on providing a file path to a text file as the only parameter. I began having thoughts that maybe I could break up another pair of dependencies for file parsing, namely obtaining a file stream (or file contents string) from a path and then parsing the file contents into a list of multiple lines. But then again, my first of the two dependencies is still dependent on the physical file system and I cannot get away from this unless I somehow mock up an in-memory file system. This began to feel silly to me as I was hoping I would just use the built-in .NET APIs for file systems and string parsing.
After emailing Dave and Esteban asking for guidance, the revelation I received via these two fine gentlemen is that file parsing is not an essential purpose of this application nor exercise (at least not yet; we won’t be focusing on performance unless it appears to be grossly deficient as we go along). What we could do is write integration tests instead, where we have a set of simple test files sitting in a folder that we run our NewlineFileParser against to ensure it working as we expect. But I don’t think I’m even going to do this at this point. I really do want to get to the core of the application. Plus, I feel certain that I can trust the implementation of File.ReadAllLines() from the .NET Framework.
Unless it needs to be changed further down the road, the following code implementation will likely be sufficient for the remainder of this coding exercise:
public class NewlineFileParser : IFileParser
{
public IEnumerable<string> ExtractWordListFromFile(string filePath)
{
return File.ReadAllLines(filePath);
}
}
I feel great about this because I haven’t gone insane trying to mock and test everything in sight, nor have I broken up every single responsibility in such small chunks that it becomes tedious. Instead, I have broken up the two core responsibilities of AnagramsFinder which means I can change and test the two dependencies independent of each other and from their manager class. Even though I haven’t written a ton of code yet, I feel the tests specs have driven my design into a cohesive, decoupled design that should prove flexible if requirements change or if I encounter friction later on and need to rethink my design. A flexible architecture for the application is being flushed out based on a “top-to-bottom” design view of the requirements.
Tobias Walter left a comment on my last post with some suggestions that I would like to get clarification on and discuss further with my growing audience of three now. He compared some of my implementation to Dave’s implementation on his Favour test driving logic over data post, but the comparison was lost on me. It seemed Tobias was saying that I had gone too far breaking up the responsibilities into such small dependencies. I am unsure what his argument was, but as far as I could tell both Dave and I broke up a managing class into two separate responsibilities so that the more crucial of the two could be tested independently. Also in doing so, we both created stubs that were given specific dummy responses to method calls using Rhino.Mocks. One more comment I would like to say is that I broke the file parsing responsibility away into its own dependency more because of the need to isolate the anagrams grouping than because file parsing is of itself a difficult task. If Tobias or anyone else could try to clarify his argument, that would really help me understand the merits of his suggestions.
Another suggestion that Tobias brought up that I would like to petition feedback on is to use the mocking feature of testing whether a method was called instead of creating a stub programmed with a dummy response. I have to admit, I’m not sure when to use one method over another. I guess I’ve felt in the past that testing whether a method was called seems silly, since it seems odd to have your test know the implementation and inner method calls of the subject under test. But I am now realizing that my stubs appear to be doing the same thing, but with dummy input and output programmed in. If you have an opinion or experience on this subject, please leave some feedback in the comments of this post.
In my next post I hope to summarize this mocking debate brought up above regarding two differing methods of verifying my manager class is correctly managing its dependencies. I also hope to begin driving out the design and implementation of the AnagramGrouper dependency via behavior specification tests. Thanks and any feedback is greatly appreciated.
This post is part of a series on coding Kata, BDD, MSpec, and SOLID principles. Feel free to visit the above link which points to the introductory post, also containing an index of all posts in the series.
In this post, we will rewrite our first specification/test from the previous post, where we didn’t feel very confident about the direction we were heading. We got some great feedback and will now better focus on test driving logic and not just data; we will also be more careful in extracting the requirements from problem statement and in giving our classes a Single Responsibility (the first of the SOLID principles of Object-Oriented Design). As we are test driving our design from a “top-down” perspective, we will encounter the need for dependencies not yet implemented while creating the higher level classes. Instead of halting our rhythm and progress, we will utilize a mocking framework to stub out non-existent implementations according to interfaces. This also encourages us to use Dependency Inversion (last of the SOLID principles; same link as above) and we can prove the validity of our modules by testing them in complete isolation from their dependencies.
I had a feeling I was focusing too much on testing data, and I also knew it seemed to odd to be writing my very first test to be dependent on solving a specific instance of the problem (especially a test case with thousands of words in it), instead of working on solving the problem generally. After some terrific feedback from David Tchepak in the comments of my last post, I now understand that wasn’t my only mistake. I was also trying to give my AnagramsFinder class too many responsibilities (especially with just one ParseTextFile method), namely extracting the words out of the text file and then grouping the anagrams together. Not only that, I also was returning a count of anagram sets when the problem statement specifically asked for output of the anagram sets themselves. Even though this is a simple problem scenario, it can be foolish to make assumptions or simplifications like I did that are not part of the requirements.
Let us first focus on the responsibilities required of our AnagramsFinder class. It needs to parse a list of words out of a text file given the file path and it also needs to group words together into sets that are anagrams of each other. However, we just named two responsibilities; implementing both in the same class would make it less cohesive and therefore less maintainable. This is because our class would have more than one responsibility that could change in the future, and because these responsibilities are in the same class, they could be considered coupled and one could be affected by modifications to the other. All of this train of thought falls under the Single Responsibility Principle from the SOLID principles (see links at the beginning of the post).
To solve this predicament, we will make AnagramsFinder a managing class of dependencies that individually solve these separate concerns. Each of these dependencies will adhere to the Single Responsibility principle, as does this higher abstraction manager class. Its responsibility can be summarized as managing the inputs, outputs, and execution order of its dependencies. We shall name the dependencies directly after each of their responsibilities, namely FileParser and AnagramGrouper. However, I don’t want to go implement these dependencies and throw off the flow of fleshing out the design and logic of my manager class. We are trying to design from a more “top-down” approach, instead of focusing on the lower level data concerns of the solution too early in the design process. To accomplish completing the design of our manager class without actually implementing the dependencies, we will code to interfaces (namely IFileParser and IAnagramGrouper).
The advantages of using interfaces are that we abstract out the actual implementation of the dependencies, allowing us to be more modular and less coupled between dependencies. One class is not strongly tied to a specific implementation of its dependency, but rather the general contracts made available by the interface’s defined method signatures. What this really means is that we can swap out actual implementations of the dependency without the consuming class being modified in the least bit. This makes for truly maintainable code, especially when requirements change down the road after version one of the application is up and running. This is what the Dependency Inversion Principle (again, from SOLID) is all about.
We can take this a step further by passing the necessary dependencies into our consuming class at construction. This allows for us to “inject” into the object any implementation of the interface we see fit; this technique is aptly named Dependency Injection (DI). Our manager class doesn’t have to fret about any concrete details regarding its dependencies’ actual implementations, nor about any of the construction ceremony associated with “newing” up and initializing said dependencies. We won’t be using any DI containers (as our dependency needs are very light in this exercise), but the use of the Dependency Inversion Principle does set us up nicely to use a mocking framework so that we can finish fleshing out our tests without implementing any of the dependencies yet.
We would like to finish the design of our top level AnagramsFinder manager class, but not fully commit to how its dependencies will be implemented yet. We have a general idea of how we want our consuming class to interact with its dependencies via interfaces. Let’s go ahead and look at our interfaces we named earlier:
public interface IFileParser
{
IEnumerable<string> ExtractWordListFromFile(string filePath);
}
public interface IAnagramGrouper
{
IEnumerable<string[]> FindAnagramSets(IEnumerable wordList);
}
Now, we could create concrete implementations of the interfaces to be used in our tests that return simple dummy results. An example could look like this:
public class TestFileParser : IFileParser
{
public IEnumerable<string> ExtractWordListFromFile(string filePath)
{
return new[] { "wordA", "wordB", "Aword", "wordC" };;
}
}
This is helpful because it keeps the one class we are interested in from being dependent on any real logic in its dependencies. We want to be able to test our modules in isolation from any dependencies, so that no secondary or outside influences skew the test results. We are basically providing dummy dependencies to our tests that have no logic that could cause side effects.
However, the software development community has provided tools called mocking frameworks that can create these dependency stubs for you. This can save you from creating concrete interface implementations that have no use except for in testing. The mocking framework I will try out is Rhino.Mocks from Ayende. To get set up, you merely need to download the latest build zip archive, extract out the Rhino.Mocks.dll library to our Libraries folder created in the last blog post, and add a reference to the DLL in our Visual Studio solution. I will next show you how to use Rhino.Mocks as we rewrite our first specification test.
Armed with SRP, DIP, mocking, a more strict adherence to the actual problem statement, and a renewed focus on “favoring test driving logic over just testing data,” we now rewrite our first specification as follows:
[Subject(typeof(AnagramsFinder), "Finding Anagrams")]
public class when_given_text_file_with_word_on_each_line
{
static AnagramsFinder sut;
static IEnumerable<string[]> result;
static string filePath = "dummy_file_path.txt";
static IEnumerable<string> wordListFromFile = new[] { "wordA", "wordB", "Aword", "wordC" };
static IEnumerable<string[]> expected = new[] { new[] { "anagram", "gramana" } };
Establish context = () =>
{
var fileParser = MockRepository.GenerateStub<IFileParser>();
fileParser.Stub(x => x.ExtractWordListFromFile(filePath)).Return(wordListFromFile);
var anagramGrouper = MockRepository.GenerateStub<IAnagramGrouper>();
anagramGrouper.Stub(x => x.FindAnagramSets(wordListFromFile)).Return(expected);
sut = new AnagramsFinder(fileParser, anagramGrouper);
};
Because of = () =>
{
result = sut.ExtractAnagramSets(filePath);
};
It should_result_in_list_of_anagram_sets = () =>
{
result.ShouldEqual(expected);
};
}
I should mention a few notes about the code. First, we need to add a using Rhino.Mocks statement at the top of our specification class file. Second, our AnagramsFinder instance variable named sut stands for the “Subject Under Test,” a mannerism I picked up from David Tchepak, whom I’ve mentioned several times throughout this post series.
Our use of Rhino.Mocks is found when we call the MockRepository.GenerateStub method against an interface. We then proceed to tell the stubbed object how to behave by specifying dummy return values when given methods of the object are called with given parameters. The last line of our “context” setup is to then inject these two newly generated, stubbed dependencies into our manager class for testing. It’s also interesting to note that none of my parameters and expected outputs really make much sense. This is done on purpose to show that the data really doesn’t matter to this class, as we are not testing the logic of any data processing by the dependencies (remember, test in isolation). It is true that this test really isn’t testing any real logic at this point, and it may even never evolve into testing any real logic either. However, I think the important point is that it aided us in fleshing out the design, which we previously didn’t know how we were going to implement. Not all tests will created equal in regard to validating our logic, but they all will play a part in driving the design of our code. Hopefully these statements are true, and I would love to hear feedback on this topic in the comments.
I also want to touch on the Specification pattern of Behavior Driven Design and testing. Specification tests are commonly set up into a workflow of context establishing, behavior performing, and results asserting. The MSpec framework is designed to encourage this test organization.
To make our first test run, we need to create the basic outline of our AnagramsFinder class as outlined in our test. This includes a constructor that takes our two dependencies and an ExtractAnagramSets method:
public class AnagramsFinder
{
private IFileParser fileParser;
private IAnagramGrouper anagramGrouper;
public AnagramsFinder(IFileParser fileParser, IAnagramGrouper anagramGrouper)
{
this.fileParser = fileParser;
this.anagramGrouper = anagramGrouper;
}
public IEnumerable<string[]> ExtractAnagramSets(string filePath)
{
throw new NotImplementedException();
}
}
I am using the ConsoleRunner according to the method described in Rob Conery’s introductory MSpec and BDD post, including output to an HTML report. After running our specifications, we get the following output:
Specs in AnagramCodeKata: AnagramsFinder Finding Anagrams, when given text file with word on each line ¯ should result in list of anagram sets (FAIL) System.NotImplementedException: The method or operation is not implemented. ... Stack Trace here ... Contexts: 1, Specifications: 1 0 passed, 1 failed
…and the HTML report:
A highly encouraged tenet of Test Driven Development is the practice of “Red, Green, Refactor.” It is meant to denote the evolution of the results and state of your tests. You are encouraged to write the test and then do the minimum work necessary to get the code base to compile and run. You are first running your test in a control state where you know it should fail. Most test runners will show the color Red in regard to failed tests, and thus the name of the first step. Your next stage is to implement the code to make the test pass and turn the output color to Green, commonly indicating passing tests. The Refactor stage is a time to pause and see if any code can be reorganized or simplified. Then “rinse and repeat as necessary”, an applicable instruction from shampoo bottles.
To make our test pass, we implement real logic to coordinate the calls into our dependencies, like so:
public IEnumerable<string[]> ExtractAnagramSets(string filePath)
{
var wordList = fileParser.ExtractWordListFromFile(filePath);
return anagramGrouper.FindAnagramSets(wordList);
}
…and the successful outputs of our test runner:
Specs in AnagramCodeKata: AnagramsFinder Finding Anagrams, when given text file with word on each line ¯ should result in list of anagram sets Contexts: 1, Specifications: 1
So what do you think of the new test? Are we better able to see the Single Responsibility of the AnagramsFinder manager class? Are we more inline with solving the actual problem statement? Your feedback would be much appreciated.
In the next post, we will begin work on test driving the design and implementation of the dependencies…unless of course I get feedback that I should be focusing on a different area first or that this last rewrite still needs more help.
This would also be a great intro to behaviors for the fubumvc guides.