Alex Pop's technical trail

Announcing my upcoming book "Learning Underscore.js"

2015-10-08T01:09:00.002+01:00

After a period of silent and sustained effort I am happy to anounce that my newest book "Learning Underscore.js" has entered its production stage and it is available to pre-order on my publisher website or on the Amazon.com website with delivery in a couple of weeks.

In our ever changing software development landscape 2015 seems to be a great year for JavaScript. A couple of major events raised the profile of JavaScript for me: the release of its final specification ECMAScript 2015 (ES6), the stable release of Node.js 4 that reconciled the stagnant versions 0.1x with the effervescent io.js fork, the widespread adoption of the Babel.js library to facilitate ES6 adoption, and the public availability of the React Native framework that uses JavaScript for native mobile development. What better way for me to mark this year than to release a book about one of the most popular JavaScript libraries.

The book discusses most of the Underscore.js library features in the first part and the second part takes this library in a journey as an exploration vehicle to look at: different programming paradigms, how it can be used in server side and database environments, how it compares with related and alternative libraries, and how to use it in the context of ECMAScript 2015 (ES6). The last chapter discusses build automation and code reusability between client and server environments, while making the final point that Underscore.js and JavaScript can be used in any environment, employing a modular and clean development approach in the process.

All the book examples are hosted on GitHub and I have created an examples index page at https://github.com/popalexandruvasile/underscorejs-examples that should reveal more about the kind of content covered in the book.

I am also including an extract from the book preface that contains chapter details and who the target audience is.

What the book covers

Chapter 1, Getting Started with Underscore.js, introduces the reader to Underscore and explains the main problems addressed by the library together with a quick introduction to functional programming. The chapter describes some of the concepts and patterns that are used by Underscore or are helpful when using Underscore. The final part is a walkthrough to setup the development environment used throughout the book with a starter example.
Chapter 2, Using Underscore.js with collections, explores the Underscore functionality for collections by example and explains some of the concepts related to these area: scope resolution and execution context, map/reduce, functional programming concepts.
Chapter 3, Using Underscore.js with arrays, functions and objects, looks at functionality for arrays, functions and objects, and builds upon concepts and functions introduced in the previous chapter. In the functions section the reader will learn how to manipulate function scope and arguments and why this is an important aspect of JavaScript programming in general.
Chapter 4, Programming paradigms with Underscore.js, expands on the concepts and practices of functional programming to solve common programming problems. The chapter starts with a differentiation between object oriented programming and functional programming. It continues with exploring examples of functional programming with Underscore.
Chapter 5, Using Underscore.js in the browser, on the server and with the database, is about using Underscore in specific contexts starting with the browser environment, closely followed by Node.js server side applications or libraries. The reader will then learn how to use Underscore with databases like MongoDB and PostgreSQL that can execute JavaScript to query data.
Chapter 6, Related Underscore.js libraries and ECMAScript standards, discusses advanced topics like the link between Underscore and JavaScript standards, libraries that augment or can even replace Underscore, with a focus on taking advantage of the new ECMAScript 2015 (ES6) features.
Chapter 7, Underscore.js build automation and code reusability, introduces a build automation solution together with strategies to reuse Underscore based code between different application hosts.

Who this book is for

The book is for developers with fundamental JavaScript knowledge that want to use modern JavaScript libraries to advance their programming skills. Underscore is one of the most important libraries you should be familiar with and the book will help you achieve this goal by going through its fundamentals and using it in a wide variety of contexts. The book should be relevant to anyone who:

Is interested in building web applications, single page web applications or JavaScript based desktop mobile, or embedded applications;
Wants to use Node.js to build web applications or web services;
Wants to work with databases like MongoDB or PostgreSQL and leverage their JavaScript support.

Free e-book giveaway on 9th of February 2015

2015-02-01T11:03:00.001+00:00

Please answer a short survey here and qualify to win a free e-book copy of my "Learning AngularJS for .NET Developers" book on 9th of February 2015.

Free sample chapter "Testing and Debugging AngularJS Applications" from my recently launched book "Learning AngularJS for .NET developers"

2014-07-25T08:34:00.000+01:00

My book Learning AngularJS for .NET Developers (announced here) was published today and it offers a free chapter called "Testing and Debugging AngularJS Applications" available here.
This chapter showcases some of the first examples available of integrating Karma and Protractor test runners for AngularJS in Visual Studio through Node.js Tools for Visual Studio.

Karma and Protractor are test runners that were built specifically for testing AngularJS applications. Both can be also be used to test non-AngularJS applications while they have AngularJS specific functionality that is explored in the chapter.

Getting Karma and Protractor configured in Visual Studio was the essential step that allowed me to properly introduce AngularJS testing for the main sample application of the book. By using Node.js Tools for Visual Studio available at nodejstools.codeplex.com I was able to integrate the Node.js testing pipeline with the JavaScript files of my sample AngularJS ASP.NET MVC application. Of course all the tests can also be executed via the command line even if you don't have Node.js Tools for Visual Studio installed.

I was also able to use a Visual Studio extension VS Adapter for Google's Karma Test Runner that displayed Karma tests results in the Visual Studio Test Explorer which is another nice integration with the Node.js ecosystem.

The second part of the chapter tackles testing ServiceStack web services via the xBehave.net test framework and examples of unit testing classes using both xUnit.net and xBehave.net. As xBehave.net is built on top of xUnit.net you can switch between different styles of testing easily as showcased in the examples.

The chapter ends with the topics of debugging AngularJS application and ensuring Internet Explorer 8 and 9 compatibility. The examples index for "Chapter 5: Testing and Debugging AngularJS Applications" can be found on the main sample application repository wiki at github.com/popalexandruvasile/rentthatbike/wiki.

You can find the list of headings for the free sample chapter below under "Chapter 5: Testing and Debugging AngularJS Applications". Here is the full table of contents of the book:

Chapter 1: Introducing AngularJS

Presenting AngularJS with examples

A jQuery example
An AngularJS example
Introducing the AngularJS controller
An overview of the AngularJS architecture
The Model-View-Controller pattern
The structure of an AngularJS application

Bootstrapping an AngularJS application
Defining module dependencies

The JavaScript patterns and practices used in AngularJS applications

Immediately-invoked Function Expression
The revealing module pattern
The strict mode of JavaScript

Dependency injection
Introducing AngularJS services
Directives

The ngRepeat directive
The ngInclude directive
Creating a custom directive

Filters
Summary

Chapter 2: Creating an AngularJS Client-side Application in Visual Studio

Using Visual Studio with NuGet to manage AngularJS and related libraries

Introducing NuGet
Using Visual Studio extensions for AngularJS

Prototyping client-side components
Organizing client-side code
Integrating AngularJS with JavaScript libraries

Native JavaScript libraries and AngularJS
Third-party user interface libraries and AngularJS

Routing
Creating a bicycle rental management application

Creating the application layout
Implementing the Bicycles views

The Bicycles index view
The Bicycles new view
The Bicycles edit view

Implementing the Customers views
Implementing the Rentals views

Summary

Chapter 3: Creating .NET Web Services for AngularJS

RESTful web services
Choosing a RESTful web services framework
Why ServiceStack is a great fit for AngularJS
ServiceStack overview

Setting up the ServiceStack starter project
Creating ServiceStack web services
Using the ServiceStack C# client
The ServiceStack infrastructure

Routing
Request and response filters
Dependency injection
Automatic validation

Creating web service methods for the sample application

ServiceStack project structure
Implementing Bicycles-related web services

The GET web service methods
The POST web service method
The PUT web service method
Merging PUT and POST request DTOs

Calling RESTful web services in AngularJS using the ngResource module

Using the $resource service
Resource class methods
Resource instance methods
Custom resource methods

Summary

Chapter 4: Creating an AngularJS, ASP.NET MVC, ServiceStack Application

Using ASP.NET MVC as the application backbone

Why you should use ASP.NET MVC
Adding ASP.NET MVC to the main sample application
Integrating ASP.NET MVC with ServiceStack
Routing considerations for ASP.NET MVC and AngularJS
Using ASP.NET bundling and minification features for AngularJS files

Securing the AngularJS application

Using the ServiceStack.Mvc library
The building blocks of the ServiceStack security infrastructure
Introducing the AuthFeature plugin
Implementing authentication for web services
Implementing authentication for ASP.NET MVC controllers

Pushing initial data from the server-side application to the AngularJS application
How to ensure that AngularJS works well with the backend

Handling the waiting time for long operations
Handling exceptions and implementing generic animations

Adding a $http interceptor
Implementing spinner animations for all web service calls

Adding database persistence

Adding database persistence to the AuthFeature plugin
Adding database persistence for web services
Persisting the Bicycle class
Changing BicycleRepository to use the OrmLite API

Summary

Chapter 5: Testing and Debugging AngularJS Applications

Overview of testing AngularJS applications

Introducing Node.js
Setting up the Node.js environment

Unit testing AngularJS components

Installing Karma
Configuring Karma
Creating AngularJS unit tests with Jasmine

Jasmine essentials
Using the ngMock module for the first test
Mocking remote server calls with $httpBackend

Testing an AngularJS application end to end

Installing Protractor
Configuring Protractor
Writing Protractor tests

Testing web service endpoints

Configuring self-hosted ServiceStack web services
Adding BDD tests with xBehave.net

Unit testing .NET code
Techniques to debug AngularJS applications
Support for HTML5 validation and older browsers such as Internet Explorer 8 and 9
Summary

Chapter 6: Advanced AngularJS Topics

Internationalization and localization

Using the ngLocale module
Using the angular-dynamic-locale module
Other internationalization-related topics

Using AngularJS animations
Working with remote web services

Using JSONP
Using CORS

Template caching
Summary

Announcing my upcoming book "Learning AngularJS for .NET developers"

2014-06-16T22:32:00.000+01:00

UPDATE: The book was launched on 25/07/2014 and it has a free chapter "Testing and debugging AngularJS applications" that is described here.

My blog was a bit quiet in the past 6 months and I had a good reason for this. Since February I am working on a book called "Learning AngularJS for .NET developers" covering the intersection between AngularJS, ServiceStack, ASP.NET MVC and ... Node.js.
The book is an expansion of my previous blog post Using AngularJS with ServiceStack while keeping ASP.NET MVC in the background and is a concentrated introduction for building single page web applications with AngularJS and .NET.
The content is divided in three main sections:

An introduction to AngularJS the popular JavaScript MVC framework originally designed to develop testable web applications faster. This section finishes with building a sample client side application in Visual Studio;
A section about creating web services using ServiceStack. It highlights features that make this process very efficient with minimal code required;
A final section about integrating AngularJS, ServiceStack, ASP.NET MVC and securing the sample application. The section continues with adding data persistence support with the micro-ORM ServiceStack.OrmLite framework. There is a chapter on testing where I can use Node.js to test the AngularJS application thanks to the recent beta release of "Node.js Tools for Visual Studio". I am using Karma and Protractor with Jasmine to define and run the JavaScript and end to end tests. For .NET integration and unit tests I am using the xBehave.net BDD test framework. The section end with a chapter on advanced topics like internationalisation, using animations and working with remote web services.

The book is focused on a modern web development workflow meant to improve developer productivity within the context of Visual Studio. This workflow should give you great results when integrating AngularJS with .NET web applications and one of its highlights is the integration with Node.js based test runners for user interface tests.
All the examples from this book do not require a commercial license for Visual Studio or for any of the frameworks or libraries that were used. The minimum requirement is the free Visual Studio 2013 Express for Web that supports "Node.js Tools for Visual Studio" and Web Essentials 2013.
I provided some of the AngularJS examples on plnkr.co/tags/angularjs-dotnet-book and all the examples from this book are on GitHub so anyone can clone the book examples repositories. The examples are split in two repositories:

One covering examples for specific concepts around client side and server side development hosted at github.com/popalexandruvasile/angularjs-dotnet-book and at plnkr.co/tags/angularjs-dotnet-book so they can be executed directly. You can find the example index on the repository home page;
One representing the main sample application built incrementally throughout the book at github.com/popalexandruvasile/rentthatbike. All chapters examples are found as branches for this repository. You cand find the example index on repository wiki homepage at github.com/popalexandruvasile/rentthatbike/wiki.

Using AngularJS with ServiceStack while keeping ASP.NET MVC in the background

2013-12-12T08:12:00.000+00:00

UPDATE: This post is now only valid for ServiceStack v3. ServiceStack v4 has now built in support for a Windows Authentication Provider: https://github.com/ServiceStack/ServiceStack/blob/master/release-notes.md#windows-auth-provider-for-aspnet .

I mentioned in a previous post the productivity gains on the server side front when using the excellent ServiceStack framework. In this post I want to present a starter application that relies on AngularJS and ServiceStack to provide almost all its functionality while using ASP.NET MVC for very few and specific features.

Soon after ServiceStack has introduced support for the Razor view engine I started to question why would you use ASP.NET Web Forms or ASP.NET MVC features when similar functionality is now available in ServiceStack? Especially if you use a client side JavaScript framework like Backbone.js, AngularJS or Ember.js - you can now use ServiceStack on its own with no dependencies on ASP.NET.
However there are some constraints that you need to factor in when you design a web application that needs to integrate with existing systems. I think you cannot rule out completely the ASP.NET Web Forms and ASP.NET MVC frameworks if you answer yes to any of this questions:

do you need to use Windows authentication?
do you need to expose/reuse ".asmx" web services or WCF endpoints?

For my application the answer was yes to all questions and I decided to use the ServiceStack framework in combination with ASP.NET MVC. This allowed me to rely on the built in Windows authentication provided by ASP.NET MVC without having to write any code to support it and to be able to use almost any .NET server side endpoint.
The application structure has only one server side view created as a result of a default controller action and this view provides just two functions: build the application HTML skeleton by loading the layout file and initiate a single page JavaScript application powered by AngularJS.
All the application data is loaded by AngularJS resource objects from ServiceStack powered webservices that are secured by the existing ASP.NET authenticated session. With this structure I can create new controller actions, WCF services or even ASP.NET Web Forms pages and use the built in ServiceStack inversion of control container to instantiate any of its registered dependencies.
One of the highlights of this setup apart from using AngularJS was how easy and seamless I could test all my ServiceStack endpoints using the friendly xBehave.net testing framework and the indispensable test data generation framework AutoFixture.
Setting this all up was relatively straightforward thanks to many community contributions and the very good ServiceStack documentation. However in order to speed up the process for anyone looking for a similar solution I created a starter project that has all the features I mentioned above. I began with a simple ASP.NET MVC project and I tried to ensure all my subsequent commits are organised in logical steps so you can see how everything was put together.

The next steps for this project are to add some tests that exemplify how easy is to test ServiceStack endpoints, upgrade to ASP.NET MVC 5 and ServiceStack 4, create a similar project that can be used as an "app for SharePoint" web application.

NOTES:

the starter project can be found here;
a key part of this project around sharing the ASP.NET session with ServiceStack was taken from a StackOverflow answer.

Apps for SharePoint and modern web applications

2013-10-11T21:00:00.000+01:00

In a previous post I mentioned a couple of improvements that the new "apps for SharePoint" model brings for the wider web development community. After developing a couple of web applications using this model I want to elaborate on the new technologies and practices that it made possible.

First a quick overview of the new development model. This model does not use the SharePoint server side libraries and it does not require to develop your application on a computer with SharePoint installed. It introduces two new types of applications for SharePoint:

one is a collection of web pages with no server-side code behind. You can only use HTML, CSS and JavaScript to build your user interface and for application logic. This application will be deployed directly to and hosted by SharePoint.
one is a standalone web application that can use any technology(client or server side) to render the user interface and provide functionality. You have to handle the deployment of the web application alongside SharePoint or a standalone/remote server if you use the "provider-hosted" option. You can also deploy to Windows Azure if you use the "auto-hosted" option.

Both types of applications can expose "app parts" for SharePoint which are similar with classic SharePoint web parts and/or provide an immersive experience where it builds all the user interface. I will only discuss about the "provider-hosted" web application in its incarnation as a .NET implementation.

In Visual Studio 2012 the "app for SharePoint" project wizard creates an ASP.NET Web Forms application and adds a reference to the "App for SharePoint Web Toolkit" package that is responsible for integration with SharePoint.

You can change the default behaviour and switch to ASP.NET MVC by adding the relevant packages and slightly modifying the project structure(in Visual Studio 2013 you can create an ASP.NET MVC application from the project wizard). At this point your SharePoint application has become an ASP.NET MVC application that is easier to test and closer to a web application rather than a web site.

Speaking of testability you can now move all the interactions with SharePoint behind an interface making these easier to mock or simulate. You can debug and test your application without any SharePoint server available which is a useful and important change from the previous development model.

I was able to move one step further than ASP.NET MVC and use a JavaScript client-side framework for buiding most of the user interface. The main driver for this was the need to be able to build "SharePoint-hosted" applications which have a requirement for HTML, CSS and JavaScript.
For my first project I used a JavaScript framework based on the MVVM pattern but I soon discovered its limitations like verbose code for databinding, proliferation of string based html templates and making it too easy to mix application logic code with user interface generation code.
In my next project I used AngularJS as my client side framework and this helped me to:

reduce the JavaScript code size dramatically;
keep all user interface code in clean, nicely formatted and easy to read html files;
have a very clear separation between user interface code and the rest of the code;
test JavaScript code a lot easier.

I have extensively used its companion libraries from the AngularUI project and I was able to employ native Bootstrap directives and simplify all the user interface building tasks. AngularJS has great support for RESTful web services and this plained nicely with my server side code. I was very impressed with the power of this framework and it saved a lot of time and effort overall.
The user interface has built using a single ASP.NET MVC view to render a web page skeleton and handle the CSS and Javascript assets. It allowed AngularJS to take over and render the main content of the page and handle all navigation events. I hope to cover this aspects on a separate blog post soon.

On the server side I have used the ServiceStack web services framework to create RESTful endpoints and this allowed me to reduce a lot of my development costs compared to ASP.NET Web Api. As ServiceStack it is very testable I had a good coverage for my server side code(see my previous post for a brief overview of ServiceStack).

Switching to the "apps for SharePoint" application model allowed me to used modern web frameworks, build more testable and tested code and reuse a lot of tools and techniques employed for standalone web applications. It made it easy to introduce web developers to this type of applications as now the main requirement is a good command of modern web development technologies: HTML, CSS, JavaScript and RESTful web services.

NOTES:

My presentation slides on this subject can be found here;
Further details on the "apps for SharePoint" model can be found here.

Productivity gains with the ServiceStack web framework

2013-08-20T22:59:00.000+01:00

I am using the ServiceStack web framework with great success for almost a year and I want to expand on why I have chosen it, its benefits and why it should be an essential part of your software toolbox.

Building modern web applications is challenging due to the rapid rate of innovation within the industry. JavaScript frameworks have gained momentum and a significant number of client side web frameworks are now replacing or at least complementing established server side web frameworks. The number of devices that connect to a web application is increasing together with the display form factors and input interface types.
A good server side web framework should help in building a rich and versatile API, support the modern web and enforce a clean application architecture. It is a strategic asset and it should be chosen carefully.
Fortunately there are quite a few .NET web frameworks that match these requirements - all open source and accepting community contributions:

community based - ServiceStack, OpenRasta, Simple.Web, NancyFx, FubuMVC, Oak;
provided by Microsoft - ASP.NET Web Api and ASP.NET MVC(with some extra components on top).

At the time of my initial experiments with these frameworks(second half of 2012) I quickly found out that ASP.NET Web Api was missing important functionality out of the box: JSONP and CORS support, had only a handful of media formatters. You could either dig up some blog entries that provided some of the missing pieces or you could implement them yourself. The same thing was true for ASP.NET MVC: it needed open source components like Restful Routing to facilitate a resource oriented architecture and content negotiation.
If I had to rely on open source community contributions and also spend development time to compensate for missing features shouldn't I check if one of the alternative web frameworks is a better choice? At this point I already looked at code samples and read blog entries for many of the web frameworks out there but getting code to work was the only way to do a proper assessment.
All of the community based frameworks that I looked into were interesting and their authors invested a lot of effort and some even had a lot of community contributions. I think any of them could easily replace/complement the Microsoft ones, but in the end the ServiceStack framework came on top due to many factors that outweighed the benefits of the other frameworks:

Maturity - was around for a couple of years with consistent and frequent commits from its authors and its contributors;
Community involvement - over 200 contributors with a very active discussion group and lot of samples - I think it is the .NET project that has the biggest community involvement;
Excellent documentation: a very well written and thorough wiki, lots of blog entries, podcasts and screencasts;
Message based architecture - an interesting approach that contrasted with the rest of the frameworks;
One stop framework - covers all development needs while allowing to replace/enhance its parts. It embeds popular and fast open source components like Funq, MiniProfiler, FluentValidation and it also provides its own excellent components: serializers, cache clients, authentication, authorization, ORM.
Commercial support is on the way - this is a unique feature for ServiceStask and it is very useful if you need guaranteed support.

Everything looks good on paper until you use it in anger. After implementing a couple of projects with ServiceStack I can now look back and report that:

I didn't had any problems/bugs: everything just worked really well;
It stayed out of my way: I was able to concentrate and work on application code with minimal infrastructure code required;
It works very well alongside ASP.NET and ASP.NET MVC: I was able to use ASP.NET for windows authentication and as an application shell while relying on ServiceStack for API functionality;
Very useful for building resource oriented applications: the message based architecture really shines by helping you to think in terms of routes and data transfer objects for each request and response;
It works very well with JavaScript applications and AngularJS: if your API is resource oriented than it will work seamlessly with the AngularJS ngResource component;
The one stop framework is the biggest productivity gain: rather than chasing libraries and best practices you can use the built in functionality. The whole framework is based on programming best practices and proven design patterns and if you grow out of something you can easily replace it;
It helps build testable applications: I found the message based architecture a great benefit for creating outside-in tests. You can start by defining a request and a response class and build your test method starting with just that. You can then write your assertions and start making your test gradually pass by implementing the rest of the functionality.

By using a framework that does what described on its wiki I could get my code delivered as expected - no hidden surprises. Thanks to the work of its authors and its contributors ServiceStack is a feature rich and a reliable modern web framework. It is a benchmark for all the other frameworks and if you want to build modern web applications with minimal effort you should give it a try.

NOTES:

You can find more about the open source .NET web frameworks and libraries available here: http://www.asp.net/mvc/open-source;
You can find more about ServiceStack here: http://www.servicestack.net/.

Automated testing as a return on investment proposition

2013-06-08T14:23:00.000+01:00

A while ago I presented an automated testing overview which was geared towards developers at first. Following initial feedback I realized that I needed to provide a wider context where I explain the immediate business benefits that automated testing brings to the table. This post is a long delayed accompanying note for the first part of my original presentation.

There is lot of published literature, together with many blog entries and tutorials around the topic of automated software testing in general. The content is mainly targeted at developers and this allowed this topic to become well known and understood by many.
I also have encountered situations where developers met resistance from management when they tried to introduce automated testing. I think to overcome this resistance automated testing should be presented slightly different to a non-developer audience: with a stronger emphasis on the business costs and benefits rather than on pure technical merits.
A distinction is made between functional, integration and unit testing but I prefer to refer to all these types of tests as automated testing. This name helps me to capture two facts:
- they are coded tests;
- they can run unattended and on a schedule.
I think this name also makes the concept simpler to non-developers.

I have seen teams of developers that wanted to implement automated testing but faced a couple of difficult technical challenges:
- the third party frameworks used were not testable;
- their own code was not testable.
These challenges meant that in order to introduce unit testing or integration testing they had to start a refactoring exercise that had to be included in their planning schedule. Even if the effort was thinned out throughout many development cycles there were a couple of bigger refactorings that had to be undertaken, one at a time. Needless to say these couple of "extra tasks" caused heated discussions between developers, testers and managers leading to long delays in the introduction of automated testing for a particular software product.

In order to analyze the effort required to introduce automated testing a distinction needs to be made between:

automated tests that target the product endpoints: (web) service methods, web pages and the user interface in general;
automated tests that target the product internals: specific classes, integration points.

Usually the refactoring exercise is associated with the effort to test the product internals and these tests interests developers most because they increase the velocity of the development cycle.

But I think the biggest business value comes from implementing the endpoint tests first. This also follows the principles of outside-in testing and produces test results that can be understood by non-developers.

A key asset are the quality assurance testing plans that in theory should already describe the manual steps required to test all product endpoints. Having this asset allows you to present the automated testing as a return in investment proposition and the following points become relevant:

Implementing automated testing allows the team to lower the costs associated with tedious manual testing over time.
By planning the effort to automate the manual test plans you can easily contrast its cost with the total manual testing cost over the lifetime of the product(or a product version).
Reducing the cost of manual testing does not mean you can also reduce the size of the test team: you allocate more time for exploratory testing which is engaging and will add more value to the software product. Testers are more creative in their explorations, the overall work satisfaction increases and testers will work more with developers to enhance existing tests or discuss new scenarios.

A successful delivery of this initial effort paves the way for expanding automated testing for the product internals. The key asset here is the analysis of the current velocity for enhancing parts of the product. Having this asset allows to predict and measure the velocity gained after the introduction of automated tests for a specific part of the product.
Associating a saving cost with each refactoring exercise makes the process measurable and easy to understand by all product stakeholders.

NOTES

My initial presentation slides can be found here: Automated testing overview;
The code used in the presentation demos can be found here: Unit testing overview;
To implement outside-in testing I have successfully used a BDD framework based on xUnit : xBehave.net and I strongly recommend it.

SharePoint 2013 supports open source / non-Microsoft platforms

2013-03-03T21:23:00.000+00:00

At this point in time (early 2013) almost all the Microsoft web frameworks and libraries are distributed as open source software: ASP.NET MVC, ASP.NET Web Api, Entity Framework, Reactive Extensions, SignalR. You can contribute back to the original source or you can fork it and start developing and distributing your own flavour.
If you are a .NET web developer you are now using an open source development stack.

On the application server front Windows Azure can boot up Linux instances on demand and it can host Node.js, PHP and Java aplications. You can also deploy your Java application to Windows Azure directly from Eclipse. No, this does not mean that the hell has frozen over, it just shows that .NET is not the one and only development framework supported by Microsoft after it has consistently and strategically invested in support for open source software in the last couple of years.

SharePoint is one of the most profitable products for Microsoft and has recently got a new version SharePoint 2013 that will further strengthen Microsoft's support for the open source software ecosystem. It introduces a new application model called "Apps for SharePoint" that allows for web content to be delivered/hosted in SharePoint from any web application regardless of its language or its host operating system. If your web framework of choice can output HTML it can become an "App for SharePoint". The main vehicle to accomplish all these new possibilities is the improved SharePoint 2013 REST api.

New scenarios are now made possible:

As a software vendor you can now create or repackage your ASP.NET MVC or Ruby on Rails web applications as "App for SharePoint" applications.
As an organisation with many internal web based systems you can now gather and expose them under the same SharePoint roof leveraging a single point of entry for authentication and application discovery.
As an employer of software developers you can now recruit from a larger pool of people as you are no longer constrained by a single language and software framework. You can now experiment and pilot different languages and tools in the eternal quest for the software platform that will reduce development time and effort.

Microsoft has recognized and adapted to the emerging trends in the software industry and this change will ripple through to all web based enterprise software products that target SharePoint.

If you are web developer regardles of your programming language and web framework of choice you now have a new delivery platform with SharePoint 2013.

Entity Framework 5 performance improvements for testable applications

2012-10-23T23:53:00.000+01:00

Earlier this year I wrote a series of posts about how to automate Entity Framework(EF) testing but since then a lot of things happened: EF 5 was released, it was made open source and Microsoft has published an in-depth article about performance tweaks for older versions of EF and performance improvements in EF 5. Following these events I can now expand my previous post and address an important performance consideration around the EF testable interface that I described earlier.

A common and ugly sight for any application that uses EF in production is the explosion of horrendous CompiledQuery static code constructs: for any Linq to EF query that is called often within your application you have to create a Compiled Query instance to ensure its query plan is cached and to benefit from the extra performance that comes with this change. But CompileQuery objects are suited for the old style EF context class ObjectContext - they are not available for the new DbContext EF context class introduced in EF 4.1. This means that if your application has a lot of chatty database calls you are stuck with ObjectContext because you really need to use those CompiledQuery instances. And this also means you cannot actually do unit testing in this scenario... at least not until EF 5 was released.

With EF 5 and .NET 4.5 you benefit from automatic compilation of Linq to EF queries. This brings a major boost in performance for all EF models, removes the need for all those ugly CompiledQuery instances and makes the DbContext class (finally) a worthy replacement for the old ObjectContext class. And yes it means that you can use the IDbContext interface and benefit from unit testing the EF model if this is what your application needs.

Post-Silverlight remarks

2012-10-18T00:59:00.000+01:00

After spending a year and a half in XAML land using Silverlight extensively I leave with pleasant memories but also with a bitter feeling.

The main strength of Silverlight is creating complex applications for Windows/Mac OS X with ease while maintaining a low deployment footprint. Silverlight XAML is almost identical with WPF XAML minus a couple of features and this versatility seems to make Silverlight an essential tool for the enterprise application developer. But since Microsoft abruptly shifted focus from Silverlight to HTML5 in 2011 the Silverlight open source ecosystem has imploded rapidly.

Recently AutoMapper has dropped support for Silverlight (version 2.0 is the last version that supports Silverlight) and I took this rather lightly to begin with knowing that the library is mature and very popular. But soon I discovered that when open source libraries like AutoMapper abandon Silverlight it affects other open source projects too. I found this worrying fact when I was working on adding AutoFixture support for Silverlight 4. Thanks to some previous attempts to accomplish the same feature I was able to modify and compile the AutoFixture assembly for Silverlight. But when I tried to make the tests pass for the new assembly(and after getting Resharper to run the xUnit tests for Silverlight) I discovered that some tests were failing. The failed tests were for a bit of code that I had to change to cater for .NET classes and methods that are not available in Silverlight. And this bit of code was originally taken from the AutoMapper source code which now has dropped support for Silverlight: I have problems modifying an open source library to support Silverlight while the other open source library that tried to address similar problems has dropped support for Silverlight.

I can investigate other solutions - maybe the Moonlight project(Silverlight for Linux) could offer some relief ... but this project was recently abandoned as well. I should continue looking to find an open source project that tries to solve a similar issue but it feels like I am trying to build on an eroding structure. I think this also affects commercial projects as open source projects are facilitating or providing missing features for the Silverlight toolchain - xUnit, ReSharper unit testing, REST clients.

Other parting thoughts are that Silverlight done properly via MVVM and unit testing offers a good model for the emerging enterprise applications that are based on JavaScript, HTML and CSS. It is a very good benchmark for any framework that aims to replace it and it helped me grasp how a rich and complex JavaScript web application should be like.

Using a desktop shim with a PhoneGap app

2012-09-18T00:03:00.001+01:00

I wanted to look at some sample PhoneGap apps code but the cost of setting up an Android/iOS shell project seemed a bit high. Sure it is worth for a real app but if you just want to play around with code and after you paid it a couple of times you start to think "it has to be a better way". And ... wait for it ... there is.

The key was the search for "phonegap shim". This brings back a couple of GitHub repos but some of them look stale. One that looks fresh and juicy is jxp/phonegap-desktop - it even has a wiki. Yay.

The PhoneGap code that I wanted to poke around was for a jQuery Mobile sample app EmployeeDirectory created by Christophe Coenraets. The intended use of this app is within an Android/iOS project that has the PhoneGap integration already configured. But what happens if I drop the phonegap-desktop folder in the app folder and replace the phonegap.js link in the index.html with a link to phonegap-desktop.js? ... suspense, gasps of awe ... and (some) results.

Not all the way there yet : an error occurs when you try to access resources that are referenced with the "file://" scheme rather than the "http://" one. You can fix this by either serving all resources through a web server(... remember we are on a time budget here) or temporary enable an interesting command line switch in Chrome called allow-file-access-from-file(unsecure and dangerous and totally unrecommended ... but solves my problem fast and cheap).

After these tweaks I was able to run the EmpployeeDirectory app on my desktop with minimal configuration while modifying and viewing my changes very quickly - no emulators hurt or used in the process.
Of course all this can be found in my fork of the EmployeeDirectory app here .
You just need to open index.html in Chrome and the app will be alive.
Kudos to jxp and ccoenraets that made this possible.

NOTES:

alternatives for "phonegap-shim" are : phonegap-proxy which comes with some dependencies and Ripple which is more of a browser based emulator.
The command line for the Google Chrome switch is "Chrome --allow-file-access-from-files" for Windows and "/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --allow-file-access-from-files" for Mac.

Load testing .NET applications is (still) expensive

2012-04-20T01:30:00.000+01:00

Performance analysis is usually a late activity in the software development process but a vital one if you have to deploy your application in production.
After an intense performance analysis session during load testing I decided to share my experience about the tools that I used.

Redgate's ANTS Performance Profiler(RAPP) is the best tool for performance analysis when you are dealing with small scale application usage scenarios. Very easy to get it up and running and intuitive to use and read the profiling results.
Visual Studio 2010 Performance Profiler(VSPP) is a good tool as well but is less intuitive to use (you need to spend some time reading about the various configuration options). It is a lot more expensive being part of Visual Studio 2010 Premium Edition which is a couple of thousand $/£/€ more compared to a couple of hundred for RAPP. For small scale usage scenario analysis I would use RAPP exclusively given its price and usability.

The situation is different when you need to do load testing for a large number of users and when you want to reach the absolute limits of how much load your application can take. RAPP has a bigger overhead than VSPP and makes the load testing process very slow overall - I don't think RAPP is even meant to be used in this scenario. VSPP has a very small overhead and you can undertake performance analysis while bombarding your application with a lot of requests. It is like this tool has used a back door into the operating system and it just gets the data that it needs. You can download VSPP as command line tool so you can use it on a computer that does not have Visual Studio installed. Although you can gather the test results for free in order to read the final report you need to have at least Visual Studio Premium edition. If you can generate the load test manually or using a third party tool your expenses stop here but if you want to use Visual Studio to generate the load test you have to buy Visual Studio Ultimate edition (which is a lot more expensive than the price of Visual Studio Premium).

There is a lot more to be said on this subject but the takeaway is that if you your application needs support a couple of thousand users that perform frequent writes be prepared to invest a lot of many and a reasonable development effort or less money and a considerable amount of effort - load testing in .NET is not cheap.

Automate Entity Framework 4.x mocking using T4 templates - part 2

2012-03-10T16:32:00.004+00:00

In my previous post I described how to use T4 templates to customize the Entity Framework(EF) code generation in order to introduce EF testability. My test code sample showed a very simple way to inject data and test the business logic without using a physical database. However the code that injects the test data could do with a bit of automation itself.

I need a helper class that :

allows me to easily add test data;
can create a simple stub for the IDbContext interface – similar with the initial test example;
can create a mock with some default behaviour already specified - e.g. I need to check that the IDbContext.Delete method is called exactly 3 times for a specific entity but I don’t care about the IDbContext.Add method.

By using another T4 template the helper class is now created. This allows me to refactor the initial test into:

I still have some noise from the data creation code. In a lot of cases I don’t really care about the content of my data, except for a couple of fields or foreign keys. Ideally I should be able to generate objects with default data and set only the bits I need. Fortunately I can use an excellent library called AutoFixture to accomplish all this. AutoFixture will populate any new class instances with sensible defaults and supports a lot of customisations that allow you to skip properties and override default values. It is easily extensible and it has become an indispensible tool for all my unit testing needs.
Using AutoFixture my test is now a lot shorter:

At this point unit testing Entity Framework is production ready and the helper class described here should allow me to easily create test data, freeing me from the physical database dependency.

Automate Entity Framework 4.x mocking using T4 templates - part 1

2012-02-20T23:18:00.000+00:00

Every time I read about a new Entity Framework (EF) release I am hoping to find better support for mocking the physical database. I would like an in memory data provider for Entity Framework - thank you very much. It should enable seamless unit testing with a lot of benefits: fast tests, zero environment configuration, no database setup and tear down, ~~happy~~ less grumpy developers. Until this happens the story of building a testable software product with EF will involve some degree of pain.

If you use the code first model with EF 4.1 you experience the least amount of pain - as you build your classes representing entities you can gradually build an EF abstraction that allows unit testing. An approach using the repository and unit of work patterns is covered thoroughly in the article Testability and Entity Framework 4.0 by Scott Allen. A lighter approach using a simple interface for the DbContext class is presented in this blog post How to design a unit testable domain model with Entity Framework code first by Daniel Cazzulino. Both articles give a lot of context for exploring testability with EF. I prefer the lighter approach because it has minimal code noise and it behaves like the missing interface for the DbContext class. This interface makes your EF model abstraction look like a POCO: elegant, minimalistic and unit testing friendly.
In the original Daniel's article this interface is called IDomainContext - however due to the fact that it should have been part of the EF code I will rename it to IDbContext in my sample.
I wanted to use this interface in a project that was using a database first model rather than a code first model. I found out that I could easily do this by using the DbContext generator T4 templates. These templates are created and provided by Microsoft. One template will create the DbContext derived class and the second template will create the POCO entity classes for your EF model. Looking at the first template file I could see an easy tweak that allows me to generate the required DbContext interface. Soon I had a nice looking interface that was ready for unit testing:
From now on every time I modify the .edmx file the interface class is updated automatically.
The next step is to modify the T4 template that generates the DbContext derived class to ensure it implements the IDbContext interface. Some of the methods from the IDbContext interface are implemented in a partial class for convenience, but they can be easily embedded in the T4 template.
At this point I can create unit tests against the IDbContext interface by leveraginq Moq as a convenient mocking framework.

Without the EF model abstraction a test method would look like this:
Every time I run this test the physical database is hit. For this type of test I need to maintain configuration strings, database state, database backups and restores, clean up and setup scripts. This slows me down in scenarios where I just need to test the business logic code and I don't care about database roundtrips or I don't have the development budget to maintain the database infrastructure.
But thanks to the IDbContext interface I can forget all about these issues and do this:
I am creating the test data manually and then I am injecting it into my business logic class. From now on I am freed from the concrete EF model and the physical database and I can continue writing unit tests that are cheap to run and mantain.
The code that creates the test data looks a bit clunky and repetitive: in my next blog post I will present a T4 template that creates a unit testing helper class and I will create a new version of the unit test that looks more efficient.

The full source code for this series of posts is already available on Github.

NOTES:

Scott Allen has recently done a presentation where he demoes an EF interface that is similar with Daniel Cazzulino.
Daniel Cazzulino has packaged up his interface and can be used as part of the open source NetFx NuGet library: NETFx Patterns: Domain Context for Entity Framework.

Hello world

2012-02-12T22:23:00.000+00:00

I recently achieved a professional milestone: a round number of years working as a (.NET) software developer. The number does not mean a lot in itself apart from the fact that it made me think a bit more about my past and current experience.
This in turn helped me realise a couple of things:
I still want to find answers to some of the questions I accumulated so far.
I learned so much from the software community and it is great to be part of this unprecedented sharing of software knowledge.
And as a conclusion to my retrospective I want to try and give something back to the community.
I hope that starting this blog will help me accomplish that.