Continue reading »]]> The following article was originally printed in Methods & Tools

Introduction

For many years the developers who practiced Unit Testing were frustrated about numerous problems they had when trying to apply automated testing to ASP.NET sites, in particular, those that were built using the WebForms framework (which was, for many, a synonym of ASP.NET). Not so long ago, Microsoft developed a new ASP.NET framework, called ASP.NET MVC. One of the selling points of this framework was its testability. In this article, we’re discussing what the problems are with testing ASP.NET sites in general, and how the ASP.NET MVC framework tries to solve them.

What’s different about testing ASP.NET applications?

When you are developing a library, or an application, you “own” the code. It means that, provided you follow certain guidelines, you can automate testing all, or at least all worth testing, features of your product. While you usually use third party libraries, or system resources like database or Internet connection, they can be considered “enhancements” sitting on top of your code.

With ASP.NET, the situation is reversed. What you develop are, essentially, enhancements to the IIS (or another hosting process). This is not unique, of course, to ASP.NET – the same holds true for any plugin development. It is like your code is a “guest” in this system, and should behave accordingly. You can add as many classes as you wish, and make them testable, but there is always a boundary layer that communicates with the framework. The host system creates these boundary classes, and calls certain methods that you provide, so that you have a brief moment of control. In WebForms, this layer is made of ASPX pages and code-behind files. In ASP.NET MVC, it is Controllers and Views.

For testing, a developer can take two very distinct paths. A developer can either test the main system together with your custom code (integration testing), or test your code in isolation, somehow dealing with the absence of the main system (unit testing).

Integration Testing

End-to-end, or integration, tests verify the behavior of the system as a whole. One can start, for example, with a certain URL, and inspect the HTML output, plus the side effects (a DB record inserted, an email sent, some money written off of our account, etc). Based on this test, we can assume that everything works as expected, or something is wrong. Given the HTML response, for example, all we can is search for a certain string and hope that if we find it, then everything else is as expected as well. If not, we have an idea that something went wrong, but we don’t know what. This is known as a “black box” testing – we provide an input and inspect the output, but we don’t know what’s inside.

We can push this even further, and test our client code together with the server code. While the former tests mimic a browser (set up an HTTP request, inspect the response), the latter ones mimic a user (“click” a button, inspect the content of a certain HTML element). Typically, we write code that “drives” a browser, automating all the clicking for us. There are some important qualities that characterize such tests:

• We are testing the system behavior as it is presented to the end user, so if the tests pass, the system is expected to work correctly.
• If a test is broken, we don’t know whether it is a problem with the client or the server code, or the incompatibility between these two.
• Such tests are really slow. The clicking happens faster than a human would do it, but still much slower than the code processing. A complex system can take several hours to test.
• Complicated setup and cleanup: Often we have to perform several steps before we even get to the page we want to test. For example, testing a password protected page would require visiting the login page first.
• Tests are hard to maintain, as there is a lot of test code involved that is not directly related to the system.

Whether we use client code testing or not, there are some general problems with such tests, and these problems do not depend on a particular framework or programming language:

• If something’s wrong, we are not sure about what needs to be fixed.
• More often than not, some tests start to fail because of a non-functional change (such as changing an ID of an HTML tag).
• It is hard to control the external dependencies. For example, if we test an e-commerce application, we don’t want to bill somebody’s account each time we run a test, yet we need to get a positive response from the billing system, so that we can feed it into our system.
• Integration tests alone do not drive your system to a better design. After all, they don’t care about the implementation.

So, while integration tests are very important, and should be a part of any test suite in order to ensure that your system works properly as a whole, it is essential to use unit testing in your development process.

Unit testing

As we saw, when working with a framework that hosts your code, you have to create a number of “boundary” classes that interface with that framework. It is these classes that are sometimes hard to test, and have to be designed carefully in order to keep both your hosting framework and your test runner happy. One of the biggest challenges is to test your boundary class without the “real” hosting system up and running. For example, the WebForms framework is responsible for parsing the ASPX pages, assembling the controls into a web page, and firing certain events now and then. In order to write a sensible test for a WebForm page, you need at least the page itself built up properly, meaning that you should do a lot of work that normally is done for you by the framework. Another problem is that often the tested code depends on certain objects that only the framework can provide. For example, the infamous HttpContext object, that cannot be created manually, and its “children” – HttpRequest, HttpResponse, etc.

Typically, the only way out of this situation is make the boundary classes to serve as mere adaptors, and put the real “meat” in other classes, those that do not interact directly with the framework, or at least are easy to setup manually and tested in isolation. This supposedly leads to a better code as well, since, for example, moving the business and DAL logic away from the code-behind leads to more maintainable applications. The boundary classes themselves are left untested. One example is when, rather than put your data access in the code behind, you move it to DataObject classes that are used by ObjectDataSource controls. These classes, although being created and used by the framework, are relatively easy created and tested in isolation.

Here, the point whether the framework design cares for testability becomes important. A lot of people (me included) tried to follow this path for WebForms applications, trying to apply the MVP pattern. What happened is a lot of plumbing code that required an amount of effort far exceeding the testability benefits, and making the code harder, to maintain.

ASP.NET MVC: Testability Paradise

After listening to developers criticize the WebForms design and testability issues for several years, Microsoft gave us the ASP.NET MVC framework. It has been designed to solve many shortcomings of WebForms, including the testability problems. The main “boundary” classes are now Controllers. Although they are created by the framework, and have to conform to some rules, they are quite easy to setup in tests, and they can happily exist without the ASP.NET request pipeline. What’s important, we are not limited to default constructors: we can inject dependencies into our controllers, making it possible to mock our services and test controllers in isolation. Another major benefit is that controllers are totally independent of the UI. It is widely agreed that the UI is hard to test, and Codebehind classes were suffering from this “burden”. Now we’ve got Views, which are still hard to test (although some effort is made into that direction, one blog post from an architect on the ASP.NET team), but this does not affect testing our Controllers.

Another step toward testability is addressing the HttpContext nightmare. Microsoft took the “thin wrapper” approach, which, admittedly, is the only way when you deal with the vast amount of existing code which should be easily upgradeable to the new ASP.NET version. In terms of testability, I’d say it’s moving from impossible to very hard. And if we recall the popular belief that testability equals good design, this case is a great example of the opposite. So, what we gained here is a possibility to write unreadable tests with chained partial mocks, plus having to use Http** wrappers in your production code without any real benefit to the application design.

Most developers tend towards one of two extremities. One, the “Big Ball of Mud” approach (that MVC does not save us from), is to put all the request handling code in the Action method of the controller. Writing tests for such methods is essential, since the code can become very complicated. That would be, of course, integration tests: each would test everything that happens during the request. However, the size of the “black box” is now much smaller, and we can even peek at what’s inside it, since our test is now in the same process as the code being tested.

The other tendency, dubbed “Slim Controller”, follows the idea that the sole responsibility of a controller is to wire up various services and model classes. Each Action method looks like a piece of configuration in an executable form. Typically, a test for such a method involves lots of mocking, and essentially duplicates the method itself (although in an unreadable form). Such tests should be generally avoided, since they don’t add any value to the system, being only a maintenance problem. The rest of the classes involved in the Action method are either business layer services (which are testable or not regardless of the framework), or infrastructure classes like custom Action Filters, Model Binders etc., which are mostly testable, but with some considerable effort (see, for example, Scott Hanselman’s post about unit testing ASP.NET MVC Custom Model Binders).

So, there are two extremities: bad design with integration tests, or good design with brittle and useless unit tests. This is why many developers that tend towards good design tend to not write tests for controllers at all, instead concentrating their efforts on testing the underlying classes.

How about test-driving our design?

Controller testability can be a good encouragement for a better Action method design. For example, it’s a bad idea to use HttpContext.Current in your code. And while you can use HttpContextBase as a mockable dependency (see our discussion of the Http* wrappers above), the mere fact that your tests are hard to write and maintain should also give you an idea that you shouldn’t do it either. Fortunately, you can use Model Binders, Action Method Filters, and other infrastructure classes to shield your code from the framework. That still leaves you with a piece not covered by tests, but at least this piece is part of the infrastructure now, and is not supposed to be changed often. Such classes are very simple, since their responsibility is just to extract the information from the framework specific classes and present it in a framework agnostic form.

Is it perfect?

As with most applications, unit testing is not enough. This is especially true when your code interface with a framework that does not participate in your unit tests. Since the boundary part is not tested for the most part, and can become rather wide (especially if you start using the infrastructure classes, like custom Route Providers and Model Binders), it is essential that you write a number of tests that make sure all your parts work together as expected.

Another potential problem is testing the Views. While this problem should discourage you from using considerable amounts of code in your Views, sometimes in order to make your Views maintainable you have to use Helpers, Partials, and even render Child Actions, and the result can become much more complicated than a simple piece of data bound HTML. Add to the mix a lot of “magic” that the framework conveniently does for you, and you’ll find yourself in a situation when making a simple change suddenly breaks things in an unpredictable way. Having a safety net of at least simple smoke tests can save you from numerous regression bugs.

I guess the correct question would be not “is it perfect,” but rather “could they make it better.” My answer is, given the circumstances, not much. On one hand, looking at the FubuMVC framework which I started to use recently, I see that it can be better. On the other hand, the general idea of making a Ruby on Rails clone for ASP.NET must have forced a number of API decisions (e.g. returning an ActionResult from the Action method). If we go deeper and try to test some advanced infrastructure code (e.g. a custom convention for View location), we discover that the Microsoft’s infamous preference of inheritance over composition does a poor job for writing such tests.

However, I should admit that for 99% of development efforts these issues are not a big problem. It is definitely good that Microsoft realized the importance of automated testing, and made testability one of the cornerstone principles of the ASP.NET MVC framework. Thanks to that decision, and its proper implementation, developers can make Web sites that are fully tested and have much less bugs. Because of that, the Web has become a better place.

Artëm Smirnov, Developer of Ivonna for Typemock Isolator, www.typemock.com

Download Ivonna now and start testing ASP.NET MVC applications. Ivonna is an extension to Typemock Isolator. Download Isolator.

Continue reading »]]> Many Web Parts in SharePoint render for capturing data that is stored in a SharePoint list. This means that to test the Web Part you have to load the the Web Part within a SharePoint server. Then how do you test it?

During this webinar, Richard Fennell will show how you can use Typemock Isolator to allow the development and testing of your Web Part without the need to keep redeploying it to SharePoint to check every minor change

Learn how to speed up your SharePoint development and test web parts

When: Tuesday, May 22 at 7:00 AM Eastern US, 12:00 BST (UK), 21:00 Australia Standard Time

Sign up: http://www.typemock.com/using-typemock-isolator-to-speed-up-sharepoint-development-may-webinar

We will also be raffling licenses of Typemock Isolator V7 and Isolator for SharePoint. Attend to win

Continue reading »]]> Coworker won’t do unit tests? Or worse, is bugging you to start unit testing and don’t understand what’s all the fuss?

Discover the secrets of unit testing.

In order to prevent bugs and release quality code to market, you also need unit testing. Discover why you should start unit testing, and how you can get started with automated tests quickly.

When: Wednesday May 16 at 10:00 AM Eastern (NY), 15:00 United Kingdom

Sign up now

Learn:
• Benefits of unit testing
• xUnit Frameworks
• The difference between unit testing and test driven development (TDD).
Join this webinar to learn what you need to know in order to start testing today.
We will also be raffling away Typemock Isolator/Isolator++ licenses and t-shirts. You must attend to win.

Also develop in C/C++? This webinar will specifically focus on some of the issues and difficulties of testing in C/C++. Sign up now.

Continue reading »]]> Last week, software architect Raffaele Garofalo hosted a Typemock webinar on Agile architecture. There were lots of questions and we weren’t able to get to all of them in the webinar, so Rafe generously agreed to answer the questions after the webinar.

You can watch the webinar here.

Patrik asked:

Q: Can you elaborate on what you mean with the "design phase"_

Ok, during the first iteration, the envision, there is a light design phase. You should design some scratches of your envision, representing the architecture and the UI, but you should not invest too much time.

Design phase while modeling means adopting UML to design your models.

Q: I didn’t quite catch the TFS workflow and how Raf used it.. Can you show it again? (also, is that view new in the tfs preview? Looks awesome)

The TFS I was working with is the new version called 2011 preview. The workflow has been adapted by me, I will publish the Agile Architecture workflow for TFS in the future.

Q: How does your "Iter: 0" relate to the PO, backlog and sprint planning?

Ah ah, interesting. The envision or iteration 0 is tricky to fit into SCRUM. Take the classic meter to measure an iteration, like 1 or 2 weeks. Inside this timeframe you should have enough space and time to prepare the necessary analysis and mocks required by next steps. So you may consider iter 0 like a beginning sprint.

Q: Couldn’t the use-case diagram be enough for the modeling phase? I’m interested in hearing your reasoning because I’m currently in this situation in my current project. What level of detail is enough when it comes to architecture that lets developers use their brain and problem solving power?

No, the use case is not enough because it may incur into personal interpretation. With the use case and the acceptance you and your team are ready for the modeling phase, where you create the architectural style for that feature. The modeling should be enough to represent the current feature and it should not be implemented, the implementation occurs in the next phase where you apply TDD against the model discussed in the previous part. It is hard to keep the model the blueprint of the code and often it doesn’t provide value. It is important that the model represents what you are delivering.

Q: Does the Process workflow you describe impact the philosophy of embracing change? How can we perform radical change when we get a bigger up-front model?

No, it doesn’t because Agile Architecture pretends the flexibility into your model. Model just enough to represents the feature and remember that you can always come back and re-model and re-factor. Embracing Agile Architecture will force you to have a dynamic more agnostic and flexible model than using a classic architecture approach.

Why would you get a radical change request with a radical model upfront? Of you adopt Agile you should structure these changes into small iterations.

Q: How can it be TDD when you have specified the methods/attributes before writing tests that requires them?

I still believe that you are applying TDD and XP in the most strictly way. You can model UML and decide the behaviors of your model, Then with TDD you will implement the model. I don’t think that TDD should drives your naming convention or model terminology.

Huet asks:

"Q: Shouldn’t the arrow be going from the support team to the team members?

If you mean the dependency adopted in UML, yes the arrow should be in that way. My intention was to represent the transfer of knowledge between the various members of the various teams.

Q: Acceptance Tests should have been written before the model.

Not really, if you adopt the strict TDD all the acceptance tests should be written before, but when you envision the architecture, it is too early to write acceptance tests. When you start the iteration and you have your requirements define, you may start to model against the acceptance criteria for that iteration. But remember that the modeling part should be light enough to provide value for the next step.

Q: These tests should be built to support the acceptance tests.

Again, you are referring to a strictly TDD approach, which is not wrong but differ a little bit from the entire Agile Architecture process.

Q: ATDD – Acceptance Test Driven Development

In the webinar I was talking about Agile Model Driven Development.

http://www.agileModeling.com/essays/amdd.htm

Q: the functions of the architect are clear, but it is not clear how these functions are intended to fit into the Sprints.

The sprint is composed by three parts, modeling, brainstorm and coding. The architect will perfectly fit into each of this phase. I think I have explained this concept during the webinar, but just to be sure. During modeling and brainstorm the architect should help with his knowledge and collaborate in the modeling, during the coding he should be an active coder too and contribute to the technical decisions providing knowledge and expertise.

Q: What is a "mock"? – do you mean a wireframe mock-up?

A: Typemock will be hosting several webinars on mock objects. Check out http://www.typemock.com/webinars/

Thanks Rafe!

For more Typemock webinars, or to watch past webinars, and learn how to start unit testing go to http://www.typemock.com/webinars/

Continue reading »]]> More than a third of the cost of software bugs – over $25 billion in the US alone – could be avoided if better software testing was performed in the initial stages of development?

The cost of inadequate infrastructure for software testing in the United States was $59.6 billion, or about 0.6 percent of US GDP, according to a report by the National Institute of Standards and Technology (NIST). Over half of the costs are borne by software users and the remainder by software developers and vendors.

According to NIST, while software bugs can’t be completely avoided, more than a third of this cost could be avoided if better software testing was performed and bugs were found during the development stage.[1] Based on 2010-2011 labor costs, the cost of preventable software bugs has increased to at least $25.8 billion in the United States alone.

So, how does this break down? According to NIST’s estimates, approximately 10 percent of computer programmers’ and 35 percent of software engineers’ time is spent debugging and correcting errors. According to NIST, this is equal to at least 302,450 full time employees engaged in testing and debugging activities, representing approximately one-fourth of all computer programmers and software engineers in the United States. Half of labor when developing software is spent on testing.[2]

The median salary in 2010-2011 of American computer applications software engineer was $85,430 and the median wage of computer systems software engineers was $92,430. At the lower wages of American computer applications software engineers, assuming that no additional employees are required to engage in testing and debugging activities, this yields a minimum cost of at least $25,838,303,500 spent on debugging and correcting errors per year.

Currently, over half of all errors are found “downstream” in the development process, after the code has already been sent to QA or during post-sale software use. However, as the above shows, by engaging in development practices that are more likely to find bugs upstream, such as unit testing, overall costs can be reduced.

Ward Cunningham’s First Law of Programming states that “lowering quality lengthens development time.” According to Cunningham:

Quality software takes the least amount of time to develop. If you have code that is simple as possible, tests that are complete and a design that fits just right, additions and changes happen in the fastest possible way because the impact is lowest. Consequently, if you hack something out, the more you hack the slower you go because the cost of addition or change grows with each line of code.

By preventing bugs in the development process, long-term costs are saved. The average lifespan of a software system can be decades. According to David Laribee, “The idea that you have a development team that completes a project then throws it over the wall to a maintenance team is fundamentally flawed.”

Hence, if you unit test, you will:

• Find bugs quicker
• Provide confidence in your code
• Send less bugs to QA
• Release higher quality software
• Save your company lots of money.

Who wouldn’t like that? Download Isolator or Isolator++ today.

[1] The Economic Impacts of Inadequate Infrastructure for Software Testing. http://www.nist.gov/director/planning/upload/report02-3.pdf

[2] http://www.cs.washington.edu/homes/mernst/pubs/economics-foser2010.pdf

Continue reading »]]> What does Agile architecture have to do with unit testing?

How do architects build software in an agile environment.

Join software architect Raffaele Garofalo as he builds the pathway to Agile architecture.

Learn

• What is Software Architecture
• What is Agile development
• How they can live together?
• Discover principles of Agile architecture
• Lifecycle and process
• Modeling and development
• The right solution for the right problem
• How to deliver quality with (TDD)

When: Wednesday, May 9, 2012 8:00 AM Eastern US (EDT), 1:00 PM BST (United Kingdom), 3:00 PM Israel

Sign up: http://www.typemock.com/software-architecture-in-an-agile-environment-webinar

We will also raffle a Typemock Isolator license & t-shirt to an attendee.

Continue reading »]]> Behavior Driven Development, an agile software development technique, can help spread unit testing and TDD in your team.
This webinar will walk through how you can be driving your application development through behavior driven, one specification at a time in a test-driven manner. You will see how BDD could fit with an agile process.

Join Einar Ingebrigtsen, Technical Fellow, at DoLittle Studios as he walks you through how BDD can help drive your application development.

Learn:
- What is Behavior-Driven Development (BDD)
- The relationship between BDD & Test-Driven Development (TDD)
- Basic specifications, Cucumber-language
- How BDD makes a good fit with agile processes
- Driving out your units in a test-first approach
- Mocking / Fakes / Stubs – what are they and how to use them
- Decoupling in software

When: Wednesday, May 2, 2012 8:30 AM Pacific time, 5:30 PM Central European Time

Register Now: http://www.typemock.com/driving-application-delivery-through-behavior-driven-development-bdd

This will be a vendor-neutral webinar; we will also be raffling Typemock Isolator licenses and t-shirts to winning webinar attendees. Sign up for a chance to win.

Continue reading »]]> Are you using a legacy version of Typemock Isolator isolating your legacy code? Want the latest advances in bug detection?

Then upgrade to V7

For FREE*

Existing Typemock customers* can upgrade to Typemock Isolator V7 for free.

Download it now.

You can also run more than one version of Typemock Isolator. Typemock Isolator V7 is backwards-compatible, so, for example, you can try V7 in a sandboxed environment while continuing to use Isolator 6 in a production environment while you migrate your team to V7.

Get it now

* Typemock customers with a valid maintenance package are eligible. This includes customers with permanent licenses and valid maintenance and all customers with active maintenance. If you are unsure of your status, or would like to renew your maintenance, please email sales@typemock.com.

Continue reading »]]> It starts out pretty simple. You want to start writing tests, so you create a library. Before you know it, you’ve got a huge suite. Although there’s some order in the chaos, you probably wish there was some way to make it look better, and even more so, make the team work in a consistent manner.

But before we discuss how, let’s ask this: Can better test organization make us more effective?

The harsh answer, forgive me all you organized people, is that it probably won’t. In fact  spending time on organization is a waste. Your choice of tools has much more to do with your effectiveness.

Gil, Say It Ain’t So

It is. And why?

Once a test goes green, it becomes transparent to us. That’s because we run it with a huge group of other green tests. If it’s not red, it’s not important.

How does this relate to test organization? Well, since our test framework of choice flags the red tests immediately, and probably lets you navigate to a test in single click, it doesn’t really matter where the test resides. If the tool you’re using is not integrated with your IDE, it will tell you exactly where and in what line the failure happened (and you should get a better integrated tool).

There are a few things that you can do, that can make you more effective, though:

• Separate quick tests from slow tests. Yes, even if they test the same method. You want to get the quickest feedback you can to keep you going. If you have 100 tests which take 2 seconds to run, and 2 tests that take 1 minute to run, and you always run them together, you’re wasting your time. Moreover, you’ve got time to think if these testing stuff really is worth it. Put the slow ones into separate suites so you can run the quick ones more frequently.
• Don’t blindly put all the tests for all the methods in a class into one file. Imagine that few of tests pass and some fail – you want to get the best picture you can about the status in a single glimpse. So put different but related scenarios together. Separate unrelated scenarios to different files.
• Don’t use test inheritance. It may seem a clever object-oriented way of code reuse but when the test is broken, the last thing you want is to start debugging what it actually does, which setup it requires (in another file) and what happens after it. There are other ways to DRY. Or even better, you can sacrifice the DRY principle in terms of readability, and repeat yourself.

If you’re using Isolator v7, you don’t even have to bother with the first two, since it’s doing it for you automatically: It runs the quick tests automatically, and if a bug occurs, it shows all the tests (both passing and failing) related to the offending code. The third one is still up to you, but if you don’t like to shoot yourself in the foot anyway, you don’t need to worry about test organization at all!

Download Isolator now!