ISerializable - Roy Osherove's Blog : Art Of Unit Testing

Test driven design – Willed vs. Forced Designs

RoyOsherove — Thu, 12 Nov 2009 21:59:48 GMT

I’m writing this as a typemock employee, but also as someone who has sat on the other side of the line for several good years, and can argue in both ways. The following, I feel, is true no matter where I work.

There are two ways people use tests to drive design, as far as I see. one is great, and I agree with, the other is not so great and I don’t agree with it. Sadly, both of them are categorized together these days, and the baby gets thrown out with the bath water - You either use both (BAD) or you use neither (BAD!)

Here are the two usage patterns:

#1 Willed Design

By writing tests, you can observe the usability of your design from a consumer perspective, and can decide whether or not you like it, and change it accordingly

#2 Forced Design

By using a subset of the available isolation frameworks(rhino, moq, nmock) or specific techniques *manual mocks and stubs) you discover cases that are not technically “mockable” or “fakeable” and use that as a sign for design change.

I highly agree with #1, and highly disagree with #2.

#1 makes sense.

You get to decide what is a good and bad design, and the experience of using that design from the test perspective is your guideline. But you make the rules on what you like and don’t like.

#2 is problematic for several reasons:

The tool decides. Not you.

You let an automated tool (rhino mocks, Moq etc..) tell you when your design is OK or not. That point alone should go against anything ALT.NET has ever stood for, doesn’t it? If you need a tool to tell you what is good or bad design, then you are doing it wrong. You should either know good design beforehad, or you shoud pair program together to find the best design, or you should learn by a mentor who can review your design mistakes, but don’t ever let a tool tell you what is right and what isn’t, especially when the only reason that tool works for that is by chance and not on purpose (as you’ll see in the next point)

A technical limitation that grew into something else

tools like rhino and MOQ and NMOCK just happen to support some OO ideas that seem good enough for design activities because of the underlying technology they use underneath. It’s pretty simple – they all use some form of either generating code at runtime that inherits from a class or interface, and then overrides methods on it (therefore they need to be virtual methods and non sealed classes, or an interface) or they use a proxy of some kind which underneath does pretty much the same things. (typemock is an exception since it uses a profiler api which has non of these requirements) . simply that means that the reason rhino, or MOQ or NMOCK “let you know” that you should use an interface somewhere, is a technical limitation of the tools, not a choice. Ayende was asked once what he’d do if he was technically able to fake static methods in rhino mocks – would he add that feature? “in the blink of an eye” he answered. and I agree. adding more options to the tool just extends the limitations of the possible design under test, not the “goodness” of it.

Languages like Ruby, Javascript, Python etc.. have isolation frameworks (or in some cases don’t even need such frameworks) that fully support any type of behavior changing, regardless of the design, since the language is less strict. yet, somehow, proper design arises in those languages tool. perhaps those languages are just “too powerful” and should not be used because they will cause you to do bad design? see the previous point for my answer.

What happens if tomorrow, or using C# 4.0 those tools get such abilities? will you all stop using them?

of course, you don’t have to use isolation frameworks to be dissuaded by the idea of limiting your design by using something – in this case a technique: using manual mocks and stubs in an object oriented language is just as “limiting” technically as is using one of those frameworks. You’re still bound to play within the simple laws of OO and using a design that is even a little out of place (even though it might make perfect sense for your application for security, performance or other reasons) is either untestable, or a general no no.

see the previous point for what i think about that.

the point here is that you’re using a technical limitation of a tool or a technique to tell you what to do instead of thinking for yourself and learning proper design guidelines. that limitation just happens to be somewhat partially consistent with what you might currently to believe to be true for design. but technically, it is a limitation that could end soon. when it changes it’s behavior, will you just change your design guidelines? switch or won’t upgrade to a new version of the language? or actually start using your head and your peers to see what’s right and what’s not?

The Typemock Dilemma

Typemock gets a lot of flack for not inhibiting the design of the program, and I can see how people would be afraid to lose that limitation in other tools, since all they head from alpha geeks in .NET is that if it’s not “testable” then your design is wrong. worse, they hear “if you need typemock your design is wrong”.

there’s nothing a silly as absolute “fact” theories in the software world. In fact, let me go out and say that all fact theories are wrong. How that’s for irony?

The message should be, I feel, more like “here are some principles of good design as we think of it today”, but instead it is based on tool choice and not on technicques or craftsmanship.

Unfortunately, I don’t think getting rid of #2 is possible in .NET today without using a tool like typemock, and that’s a shame, since it means that, because it costs money, people in the community will still want to use the free tools, which force design, instead of allowing them to decide on it like the mature developers they are.

Maybe it’s time to have some sort of free version of Isolator so that everyone can benefit. what do you think?

Art of Unit Testing on Hanselminutes

RoyOsherove — Tue, 07 Jul 2009 12:28:40 GMT

Scott Hanselman took me in for a 30 minute interview about Unit Testing Dos and Don’ts in his podcast, Hanselminutes. It was a pleasure, and I hope to be there once more about other topics. Maybe in TechEd Berlin..

Have a listen

NDC 2009 – Done!

RoyOsherove — Mon, 22 Jun 2009 11:46:14 GMT

NDC 2009 was a blast!

Thanks for all the great conversations :)

Here’s what it looked like when you look up at a full room in NDC

Questions every team and dev lead should ask themselves

RoyOsherove — Tue, 16 Jun 2009 05:43:34 GMT

here are the questions that teams and team leads should be asking themselves on a daily\weekly basis.

There are more, but these are the basics, to me. It’s part of the summary for the talk “Beautiful teams I am giving at SEConf and NDC. we do a lot of this stuff over at work, and it’s proving itself on a daily basis.

Whole team

What can we automate?
where are we "Reinventing the wheel"?
what are the tools that slow us down?
what tools can we use better?
are there bugs that I could have found earlier? how do I make sure I find them earlier?
when do we find out we built the right thing?
when do we find out our code\design sucks? how can we make that earlier?
How do we show progress at the team level? at the management level?
How many meetings does each dev have every week? how can we remove them?
Are we building by feature or by layer?
can we make all our team sit in the same place?

Team Lead

daily: what bottlenecks exist in the team? what have I solved?
will my devs be better in a month or two than they were before? if not, how do I make that happen?
what prevents my devs from working? what am I doing about this?

Art Of Unit Testing available at Amazon

RoyOsherove — Sun, 07 Jun 2009 00:43:57 GMT

My book, The Art Of Unit Testing, is now in stock at Amazon. If you’ve read the book, I’d love it if you put in a review on that page.

Using Explicit Arrange,Act,Assert scopes in tests – thoughts?

RoyOsherove — Wed, 03 Jun 2009 00:55:11 GMT

What are your thoughts on this style of writing?

Specifically the “Explicit” way of defining arrange, act and assert scopes.
also, given the name of the test, where should the call to “OnApplyTemplate” be (if you’ve never seen the code)?
which version makes more sense to you? A, or B?

version A:

version B:

Testing that an event was raised

RoyOsherove — Sat, 23 May 2009 18:25:20 GMT

This question keeps coming up: “How can I test that an event was actually raised from my class under test?”

actually, there is an easy way to check if an event was raised.
you subscribe to the event in your test, and in the event handler you set a boolean flag to true. then you just assert on the flag.
here is a quick example:

Code:

public void Test()
{
bool wasRaised=false;
var button = new Button;
button.Click += ()=> wasRaised=true;
button.DoSomethingThatShouldHaveTriggeredTheEvent();
Assert.IsTrue(wasRaised);
}

If you feel less comfortable using lambdas:

public void Test()
{
bool wasRaised=false;
var button = new Button;
button.Click += delegate

{

wasRaised=true

};

button.DoSomethingThatShouldHaveTriggeredTheEvent();
Assert.IsTrue(wasRaised);
}

Unfortunately, with VB.NET’s current version, doing this in a single method is next to impossible, so you are forced to register to the event with a method at the class level, and check that:

Code (VB.NET):

dim wasRaised as Boolean=false;

public sub test()

wasRaised=false
dim button = new Button()
AddHandler( button.Click , AddressOf(OnClick))

button.DoSomethingThatShouldHaveTriggeredTheEvent()
Assert.IsTrue(wasRaised);

end sub

public sub OnClick(source as object,e as EventArgs)

wasRaised=true

end sub

Art Of Unit Testing (The Samurai Book)– Get it now, it’s done.

RoyOsherove — Wed, 20 May 2009 21:22:14 GMT

The time has actually come. After 2.5 years, and two kids, my book is finally done and is available in full form as an EBook. at the end of the month it will be in print form. Now would be the time to get it, when it is still at a “pre-order” pricing. Get the preorder price either at Manning(along with the EBook) or at the amazon page.

I love the cover image. How about we call this “The Samurai book” from now on?

This is the book that I wished I had when I started out writing my first tests, and that combines my knowledge about unit testing from the past 5-6 years or so working with companies on real projects (and real failures).

It contains things I have not seen in other places – writing readable, maintainable and trustworthy tests, as well as guidelines on how to review someone else’s tests and what to watch out for.

I was lucky to have the foreword written by no other than Michael-Legacy-Code-Feathers himself, and with some great quotes including one from Kent Beck about the book.

Here is the Table of Contents.

Free Chapters

The book comes with two free chapters:

Chapter 1 – The basics of unit testing(PDF)

Chapter 3 – Using Stubs to Break Dependencies (PDF)

Or get the full book

Book description:

Unit testing, done right, can mean the difference between a failed project and a successful one, between a maintainable code base and a code base that no one dares touch, and between getting home at 2 AM or getting home in time for dinner, even before a release deadline.

The Art of Unit Testing builds on top of what's already been written about this important topic. It guides you step by step from simple tests to tests that are maintainable, readable, and trustworthy. It covers advanced subjects like mocks, stubs, and frameworks such as Typemock Isolator and Rhino Mocks. And you'll learn about advanced test patterns and organization, working with legacy code and even untestable code. The book discusses tools you need when testing databases and other technologies. It's written for .NET developers but others will also benefit from this book.

What’s inside:

Test Review Guidelines
How to create readable, maintainable, trustworthy tests
Stubs, mock objects, and automated frameworks
Working with .NET tools, including NUnit, Rhino Mocks and Typemock Isolator

Test Review #3 – Unity

RoyOsherove — Mon, 23 Mar 2009 05:58:55 GMT

Watch previous videos:

In this video I go over the tests for Microsoft Unity Application Block.

Overall the quality of the tests in Unity is pretty good! I could certainly recommend that people look at them as examples of a bunch of tests against a framework, which are mostly very readable and maintainable.

Things I walk through:

Using [Ignore]
exploration testing
magic numbers and strings
un-needed asserts
Stub hard coded behavior
handling config files in tests
Asserts hidden in a utility method
more naming conventions
Separating integration tests
“smart” code that is less readable
Meaningless “isNotNull” tests
cleaner ways of expecting exceptions

Test Review #2 – ASP.NET MVC Unit Tests

RoyOsherove — Sun, 22 Mar 2009 03:42:38 GMT

See other reviews:

Review #1: NerdDinner

Here’s the second video review of Unit Tests. This is another one written by Microsoft – ASP.NET MVC (source).

First, it’s important to state how surprised I was by the high quality of the tests in MVC. The tests are readable, maintainable and trustworthy, with very little issues that I could find. whatever Issues I found are rather easy to fix. In any case, if one is looking for examples of systems written in what seems almost entirely in TDD, or at the minimum with very good testing guidance, ASP.NET MVC should be a good stop to look at.

Issues discussed in this video:

Implementing RowTest with MSTest, and the importance of naming (14:00)
Verify() that is splitted from the mock expectations (17:00)
some naming conventions
Over-specification in tests (mainly more than one mock object per test)
verifying mocks when it’s not required
logic inside tests (concatenation)
test factory methods with too much logic
a very good example of when multiple asserts is really bad (11:50)

Again – I’m very pleased with the test quality. Now is the time to make sure the things above are fixed. they are still important!

Test Review #1 - NerdDinner

RoyOsherove — Fri, 20 Mar 2009 11:45:53 GMT

I’ve decided to start doing some test reviews of tests that I see in the wild. I figured it’s the best way to show people what I mean when I say they do not have readable, maintainable or trustworthy tests.

The first episode is the review of the tests from the NerdDinner MVC source code. It’s 30 minutes long. and it was shot at 2AM, so I’m quite cranky. But the tests sure don’t try to ease my mind, and only make me crankier.

If you have tests you’d like me to review send an email to Roy AT osherove.com with the subject “Test Review [project name]”

problems that are dealt with in this video:

Naming conventions
Magic strings in tests
Hard coded values in hand written stubs
Magic assertions (expecting a value that no one understand why it should be that value)
Lack of tests

API design values and decision matrix

RoyOsherove — Tue, 17 Mar 2009 14:56:56 GMT

We’ve come to a nice way of deciding upon features in the Isolator API. This is what happens when your end product is an API for programmers.

First, we realized that we must argue about it. For each type of new feature in the isolator API, we put up on the white board several version of the API that we can’t seem to decide about, and then ask everyone on the team to say what they think. If everyone agrees, then we ask people to defend the opposite side and explain why an API is not good.

We came to several agreed upon values for the C# API, that help us judge the usability of it:

Consistency – is the new API consistent with the other APIs that already exist, or does it go against the regular way things are done?
Discoverability – If a user knew they wanted to do thing X, would they know to use the API without help? simply from intellisense?
Explicitness – we decided early on to be as explicit as possible about the API, so that the least guessing needs to take place by the user
Single point of entry – everything should start from a single point (Isolate.Something())
Readability for the reader, not the writer (if you didn’t write the test would you find it easy to understand what it does?)
Single way to achieve things – is there more than one way to do a task with the new API?
Backwards compatibility – do we break an existing feature and cause some heartache for users?

We measure the various versions of an API by putting it in a matrix and setting “V” or “X” on each of them (if we can’t agree it’s a V with a line on it).

then we have a better idea of what makes more sense.

I admit we still haven’t come up with a single set of values for the VB API, but I’ll try to define it here:

Does not require use of Action<T>
Consistent with the way a VBer would use other APIs
Explicit
Readable
Single way to achieve things
Backwards compatible

note that “single point of entry” is not included. That was a design decision we made early on.

Art of unit Testing goes to print in April

RoyOsherove — Mon, 16 Mar 2009 23:33:45 GMT

The book of never ending production is now actually near the end.

the projected print date of “Art Of Unit Testing” is now April 30 and I’m happy to see the Amazon page for the book already has one review of the early access version (though he totally destroys my spelling and book formatting abilities, he likes the content, so I’m happy)

Unit Testing in silverlight land with Typemock Isolator

RoyOsherove — Sat, 27 Dec 2008 21:54:07 GMT

I’ve been asked quite a lot recently whether one can write unit tests and isolate logical code that runs inside a silverlight application. Up until today my initial answer was ‘no’ because silverlight runs under different versions of mscorlib.dll.

however.

Today I actually gave it a try and realized that writing unit tests (not integration tests, as the silverlight test framework allows) against silverlight based code is possible and quite easy. Just like any other code that relies on a third party platform (like sharepoint code) the silverlight related code might have various dependencies.

I’m going to show how to use Typemock Isolator to overcome a couple of simple silverlight dependencies (using HtmlPage) and how to setup a test project against a silverlight project (with NUnit or MsTest)

Assumptions:

You have an open solution
the solution contains a silverlight class library or silverlight application project

To setup a test project against silverlight using MSTest:

Add a new test project to the solution
Remove all the existing classes (Database test, ordered test etc..) so that you are only left with the unit test class (UnitTest1).
Remove all useless comments and crud code from the test class so that you are only left with a test method (no comments, not even the TestContext)
Add a reference to the silverlight versions of “System.dll”, “System.Windows.dll” to the test project. (usually located under “C:\Program Files\Microsoft SDKs\Silverlight\v2.0\Reference Assemblies\” (remove existing reference to system.dll if you need to first)
Add a reference to the project under test
You can now write tests against the object model (standard classes)

To setup a test project against silverlight using NUnit or MbUnit:

Add a new class library to the solution as your test project
Add a reference to the silverlight versions of “System.dll”, “System.Windows.dll” to the test project. (usually located under “C:\Program Files\Microsoft SDKs\Silverlight\v2.0\Reference Assemblies\” (remove existing reference to system.dll if you need to first)
Add a reference to the project under test
You can now write tests against the object model (standard classes)

How to break the dependencies

Now, here is a simple example of code that has a silverlight dependency we’d like to test:

Let’s say we have a class in the silverlight project called ChatSession (I’m basing this on ScottGu’s Chat demo). but it’s constructor looks like this:

What if we wanted to control during our test whether the page is enabled or disabled?

Here’s one way to do it using Isolator:

In your test project add two references under the .NET tab: “Typemock Isolator” and “Typemock ArrangeActAssert”
Write the following test:

Using Isolate.WhenCalled() we are able to circumvent any method (static or not) to return whatever we want, or throw an exception.

Here’s a more interesting case. Let’s say we have a method that modifies the current page and shows some html to the user in a span tag:

here is one way to write a test that makes sure that the right text is set into the message element in the page, without needing to have a real page present:

There are several things to note here:

HtmlElement is Sealed has an internal constructor, but we can still create a fake instance of it using Isolator. None of the other frameworks can do this.
We are returning a fake object from a method call chain (HtmlPage.Document.GetElementById ) without needing to create a separate stub for document. None of the other frameworks can do this (especially since it is based on a static method call to begin with)
We assert in the end that the method was indeed called with the correct arguments without needing to change a single piece of code. Granted, I wouldn’t write code like this (I like decoupling!) but sometimes you just don’t have the ability to change existing code.

If you are developing an open source silverlight project, it is important to note that there is a free full version of isolator for open source projects with full functionality.

These are just the beginning of my journey into silverlight unit testing. I’m looking for good code examples that you might want to test, with various dependencies that need breaking. your comments are appreciated.

Isolator Feature Focus: Duck Typing and Isolate.Swap

RoyOsherove — Sun, 02 Nov 2008 12:25:00 GMT

Typemock Isolator 5.1.1 has been released, and this release brings with it some awesome (seriously) features that are unique fro any other framework I've seen.

a good overview of features can be found here.

The Isolator Swap feature allows swapping calls between real and fake objects (kind like redirects) so that any relevant calls made against the real object will be redirected and invoked on the fake object.

Here's how you use it:

Unlike doing a standard "WhenCalled()" on some object method and telling what its custom behavior will be, "Swapping" objects redirects all relevant calls(I'll explain what "relevant" means in a second) to the fake object which you have created.

Duck Typing Awesomeness

Another cool thing about it is that "Duck" and "Dog" don't have to have a shared interface or base class. The "Swap" feature will redirect a method call if it exists on the "fake" object, but if it does not, it will invoke the original object. This is one interpretation of what's called "Duck Typing".

The "CallsOn" and "WithCallsTo" methods take an object type, so you can send in anything you want. if a method on the fake object matches the signature and name of a called method on the real object, the fake method will be invoked.