Something To Read On The Train

NCrunch -- A cool TDD Visual Studio extension

2012-02-01T22:27:00.001-06:00

I've got another dev tool that I wanted to pass along:

A few months ago, Jon Skeet posted a tweet about a new tool he was using called NCrunch. Since then, I've been playing with the tool and working with the author to resolve some of the issues (here and here) that were blocking it from working smoothly in the my environment. I believe it's now to the point where my coworkers who wish to take advantage can do so and where I can promote it's use to the world.

NCrunch, at its core, is a TDD extension for Visual Studio. It will run your unit tests in the background and provide *real-time* unit test results (no need to even save your file – runs as you type) by way of color-coded dots to the left of each line of code. (Green = passing, red = failing, black = not covered). It will provide details for exceptions that are thrown and many other cool features. This will allow you to get immediate feedback if changes you are currently typing break/fix any unit tests.

There's a very good demo video on the NCrunch homepage (about 6 minutes long) that I think is worth watching to get a feel for what the tool can do.

Key features (or at least "My favorites"):

Line-by-line, real-time status of unit test coverage
Context menu access to applicable unit test
Tool-tip/hover bubble with details on:
- number of covering tests,
- performance,
- exception details/stack trace
Visual indicators for performance metrics (slow tests have yellow centers -- with transparency based on level of slowness)
Quickly run covering tests, debug into a given line
Ability to configure how much CPU it will use.
It's FREE!!

Cons:

Feature overlap with TestDriven.Net and ReSharper test runners (although, the need for those may go away if you use NCrunch)
Some rough edges still (see below), but the developer is very actively updating and fixing bugs, and very responsive to users on the forum, twitter, etc.

There are a couple of things to note:

NCrunch does a lot of background compilation and running unit tests. It appears to be smart enough to only compile/run tests that are affected by changes you are making. In any case, if you have a slow machine, you may want to disable the automatic testing and run in manual mode. I have a very beefy development box (8 cores, 8GB memory) and don't see any issues (I also run ReSharper with full solution analysis mode with no issues).

NCrunch lets you designate which unit tests to run/ignore. In our case, we have both unit tests and system tests (db dependent) in the same solution, so developers would want to enable the unit tests but ignore the system tests. When you first enable NCrunch for a project it asks you if you want to ignore all tests – I'd suggest doing that and then using the Tests window (accessible from the NCrunch menu) to unignore the tests/assemblies you care about from the right-click context menu.

NCrunch has the option of running tests linearly or in parallel. If your tests are written such that they do not have side effects and don't share singletons, etc., then you should be better off running in parallel. However, you run the risk of having tests interfere with each other. For our code, we need to run the tests one at a time.(Update 2012-2-2: per the comment from the author, each test is run in a separate process, so no memory/static property sharing, so little risk to run the tests in parallel.)

NCrunch compiles each project in an isolated, shadow-copied environment in the background. There are some cases, though, where the Visual Studio configurations are such that NCrunch doesn't automatically determine all of the referenced assemblies that need to be copied. In those cases, you can flip a configuration setting to have NCrunch copy the output folder over into the shadow environment. This resolves the issue, but does have a performance impact. This shows itself as an error in the NCrunch Tests window with the message "Cannot register assembly XXX or one of its dependencies. The system cannot find the file specified."

To resolve this, you need to enable the "CopyReferencedAssembliesToWorkspace" option for the project by going to the NCrunch Visual Studio menu and choosing the Configuration option, selecting the project in the Configuration window and changing the property.

Setting up Sonar analysis for C# projects

2011-11-25T07:22:00.001-06:00

In an effort to better understand some of the problematic areas of the C# codebase I work on, I recently setup an instance of the Sonar code analysis platform. Sonar is originally written for Java analysis and later added C# support. This posting walks you through my experience attempting to setup, configure and run the analysis.

Sonar Overview:
So why did I even do this? Once up and running, Sonar provides some useful metrics for pointing out hotspots in your code that may be making it more difficult to maintain and extend your functionality. Through the web interface, you can drill-down on any of the metrics to the module, class, and method level, including full source code. Some of the metrics provided for each C# project include (screenshots are from the "nemo" demo site mentioned below, my own project, or other sources):

General analysis

Uses several rule-based static analysis tools

FxCop
StyleCop
Gendarme (part of the Mono project)

Details and statistics, with drill-downs, on rule violations

Cyclomatic Complexity

By method, class, file

Comments:

Percentage of code commented
Percentage of public APIs that are (un)documented

Duplication:

Percentage of code that is duplicated
Counts by duplicated lines of code, blocks, files

Unit tests:

Coverage
Success/Failure statistics
Time to run

Counts:

Lines of code
Count of files
Count of classes
Count of methods

In addition, it will track changes over time, so you can see where issues are increasing/decreasing in your code.

Sonar is open sourced under the LGPL and free to use, however some of the plugins used to perform the analysis are only commercially available, and in some cases come with steep licencing fees. For this blog, I focus on only freely available (ie: no fees) aspects of the product. Each of the tools I used are also freely available (FxCop, StyleCop, Gendarme, Gallio, PartCover, MySQL -- all have licenses that allow no-fee usage for most people).

There is a demo site provided by one of the commericial plugin providers to demo the system (including their not-free SQALE plugin) which shows analysis for several opensource Java projects and can give you a feel for the UI and the data that can be provided. Beware -- this includes data from some of the commercial plugins, so don't expect to see everything on that site after following this posting.

General Sonar Technical Architecture:
Generally, the Sonar runner consumes the source code and analyzes it via plugins. This information is published to the sonar database directly.

The Sonar server pulls the results from the database and provides a UI for reviewing them.

Step 1: Prerequisites and Assumptions:
You'll need a 32-bit machine to run your analysis on. There are issues with the 64-bit versions of Gallio and PartCover which caused errors. If you must run on 64-bit, there are sites with workarounds to getting this running, but I didn't go that route because I needed a valid 64-bit build environment for my day-to-day tasks.

In my case, I spun up a Windows 7 Virtual PC and installed everything on there, including my TeamCity build server.

This posting assumes you have a working Debug build of your codebase that compiles with no errors and generates .pdb files. You need access to the source code and the output, as they exists at the completion of your build. In my case, I have a CI server (TeamCity -- free for small/mid-scale configurations) running and just added another build step at the end to kick-off the Sonar analysis from the command line.

You'll need to install the .Net Reference Assemblies, which are part of the Win7 SDK. If you're using .Net 4, it seems the Reference Assemblies may only be available by installing Visual Studio 2010. Note that the FxCop 10 installer is included (exclusively) with the Win7 SDK.

If you have any Silverlight components in your build, you will need to install the Silverlight SDK for the version of Silverlight you are targeting.

Step 2: The Sonar Server
Download the Sonar server. At the time of this posting, it was version 2.11.

This will give you a .zip file. Decompress it into the location you want; I used C:/sonar-2.11

You can find the official installation notes here: http://docs.codehaus.org/display/SONAR/Install+Sonar

For the initial configuration/setup, I'd suggest running it at the command line until you know you have it fully configured. To do that, open a command window and run bin\windows-x86-32\StartSonar.bat This will run in your current window (no real output, but the command blocks) until you hit Cntl-Z or Cntl-Break.

Again, I had trouble with the 64-bit version even running (no console output -- just died), so I'd stay with the 32-bit side.

Once you get things working the way you want, I'd suggest installing it as a Windows service. To do this, first register Sonar as a service using the provided script: bin\windows-x86-32\InstallNTService.bat (There's an UnInstallNTService.bat script too). Then, start the service from the Windows Services control panel, or using the provided "StartNTService.bat" script. Personally, I would suggest just configuring it to be Automattic so it will service reboots, etc.

The server has a sonar.properties config file that drives some of it's functionality, including which database to use. You can find it in the \conf folder. The only change I made to this file was for the database (see Step 2), but if you want to use the built-in Derby database, no changes are needed.

Step 2a: Verify and Change Password
At this point, you should be able to access the sonar server at http://localhost:9000 (or whatever port you set in the sonar.properties file). I would highly recommend changing the admin password at this time.

Step 3: The Database
Sonar server comes with a build-in Derby database. While it is quick and easy (and pre-configured), I would suggest moving away from it almost immediately. I had MAJOR performance problems using it. For instance, for my solution of about 400K lines of code (50+ Visual Studio projects), when using the Derby database, it took about 2 hours to perform the analysis, 5 hours if the runner was on a differnt host. During that time, the server CPU would be pegged at 100% for 5 - 15 minutes per project as the runner reported it's results. Once I moved to a MySQL database, it takes about 20 minutes.

So, I would suggest downloading and installing the free MySQL Communit Edition from Oracle.

Once installed, you'll need to create the sonar database instance. This is pretty straightforward, but Sonar provides a script to make it even that much easier. From the sonar installation folder, the script is in extras/database/mysql/create_database.sql. This will generate the sonar database instance, as well as grant access from localhost and remotely. The sonar schema is created the first time you run the server.

You'll need to configure the Sonar server to use this db. This is as easy as changing a couple of lines in the sonar.properties config file. Chances are, the lines you want are already in the file (commented out), as samples are there for MySQL, PostgreSQL, Oracle and MS SqlServer. (Note: MS SqlServer is not officially supported, although people on the web have posted ways to get it working. I did not venture down this path).

Update: While getting Sonar up and running in our production environment, the person doing the work was able to get up and running with MS SqlServer and provided me with this feedback. Note, he was using v.2.13.1, which is newer than what I was using:

I followed the link you had on your page (http://www.sezok.de/sonar/sonar.html) but differed in the following ways:

The database must be setup with Case Sensitive and Accent Sensitive rules, so I created the database with collate option SQL_Latin1_General_CP1_CS_AS

The jdbc drivers that come bundled with 2.13.1 actually work, and the ones listed in the above link don’t.

For me, I commented out the Derby lines and used these for my MySQL instance:

#----- MySQL 5.x/6.x
# Comment the embedded database and uncomment the following properties to use MySQL.
# The validation query is optional.
sonar.jdbc.url: jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8
sonar.jdbc.driverClassName: com.mysql.jdbc.Driver
#sonar.jdbc.validationQuery: select 1

Restart the sonar server, if it's running.

I should note that there isn't a quick-and-easy way to migrate from the Derby db to another, so expect any data collected in Derby will be lost when you move. I found this page where someone found a way, if you really must try.

Step 3a: Verify and Change password
At this point, the sonar server should be up and running, albeit with no data. I would suggest verifying you can reach the server at http://localhost:9000 (or whatever port you used in the sonar.properties file).

I would highly recommend changing the admin password at this time!

Step 4: The Runner
Sonar was build for Java, so the docs almost all assume you're using Maven. You may be able to get this working with nAnt, but there's an easier route. A small, java-based runner has been created that can be kicked-off from the command line. This is what I used from within my TeamCity builds.

To use this runner, you will need to do a few things:
1) Download the runner. This will come as a Zip file. Decompress it in an appropriate location. For me, I put it in C:/Sonar-2.11/runner

2) Install the Java JDK. Yes, the *JDK*. It's big and ugly, I know. You're a .Net dev, I know. Do it anyway.

3) Set the JAVA_HOME environment variable to point to the JDK root folder. Make sure to set this environment var where the runner will get it. Typically, I would say set it as a System ENV var, but in my case, there is a bug in the current TeamCity build agent that does not properly pass system env vars when running as a remote agent, so while running in that config, I had to use User vars and run the agent as my user. If you're running this all on the same machine, you should be fine with a System Environment Var.

4) Test that it works. Open a command line to the sonar-runner/bin folder and run sonar-runner -h. This will either show you the usage statement (if things are working fine), or blow up. If it blows up, check you're JAVA_HOME is set correctly.

5) Once it works, set another environment var: SONAR_RUNNER_HOME to the root installation folder for sonar-runner. This is used when you want to run the sonar runner from a folder other than the /bin folder. (ie: when you run sonar-runner from your source folder).

6) Change the conf/sonar-runner.properties file. This has general configuration items used by the runner, some of which can be overridden in the project's config file (we'll get to that later). At a minimum, you'll need to set the sonar.host.url to point to where you have your sonar server running, and the database config. For the db, just copy/paste the lines from the sonar.properties file you created in Step 2. Again, example values are provided in the file (commented out) which will likely work just fine. Don't forget to comment out the Derby lines.

Step 5: The C# Plugins
You'll need to install the "C# Plugins Ecosystem" (ie: The plugins to analyze C# code). This will be a zip file with each of the C# plugins .jar files. Decompress them and place them into the /extentions/plugins folder of your Sonar installation folder. You can enable/disable each one individually, so no harm in putting them all in there at once.

The plugins consist of the following:

C# Core (support for the C# language - general API used by other plugins -- required)
C# Squid (support for parsing C# language -- required)
FxCop support (general analysis)
StyleCop support (formatting analysis)
Gendarme support (general analysis)
Gallio support (unit testing / coverage)

If you're going to use FxCop, you'll need to install FxCop 10 (the FxCop installer is installed as part of the Win7 SDK install).

If you're going to use StyleCop, you can either use a bundled version included with Sonar, or install you're own.

If you're going to use Gendarme, I've found that the version bundled with Sonar has a bug in it that caused it to error if you referenced System.Drawing from the .Net 4.0 library. I had to install the newest version (2.11 snapshot).

If you're going to use Gallio, you'll need to install it as well as PartCover (or you can use NCover, if you have a license).

Step 6: The sonar-project.properties File
Each Solution will need to have it's own sonar-project.properties file. This file will need to exist in the folder from which you execute the sonar-runner. To make this easy, I would suggest putting the file in the same folder as your .sln file.

The file will have a few sections, which I will describe here. Note that some of these, I believe, can be put in the sonar-runner.properties file if you have the same settings for each project.

Important Note: Any folder names in the config file will need to either escape the backslash with another backslash (\\) or use a forward slash (/). I've chosen the latter.

Project Identification:
This section will provide the project key used by the Sonar server to group analysis results over time, as well as provide a useful name in the UI, etc. This should be unique across projects. The project version can be used to track different branches, etc.

# Project identification
sonar.projectKey=Jwright:DemoApp
sonar.projectVersion=trunk
sonar.projectName=DemoApplication

Then, describe the source code layout. The "sources" field points to the top-level folder where the source code exists. If you're .sln and .csproj files have relative paths internally, then this should be the top-level folder. Assuming you don't have any strange layouts, this will likely be the same folder as your .sln file (which is likely where your .properties file exists), so can just be ".". Additionally, you need to denote that the language is C# using the "cs" value.

# Info required for Sonar
sources=.
sonar.language=cs

C#-specific settings:
Here, you'll need to provide information about where the .sln file is located and where key libraries are located, and which version of .Net you're using.

#Core C# Settings
sonar.dotnet.visualstudio.solution.file=DemoApp.sln
sonar.silverlight.4.mscorlib.location=C:/Program Files/Reference Assemblies/Microsoft/Framework/Silverlight/v4.0
sonar.dotnet.excludeGeneratedCode=true
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0

Plug-in Specific Sections:
For each plugin, there is a "mode" setting. If blank, then the plugin will run. If you want to skip/not run a plugin, set the mode to "skip".

For Gallio, you can stipulate if you want to use PartCover (free) or NCover (not free). You can also stipulate the runner mode. I had trouble using anything other than "Local". You will also need to stipulate the naming pattern (regular expressions, I believe) for the Visual Studio projects that include unit tests. You can have multiple patterns, seperated by semicolons.

Very Important: Note that there is a bug/typo in the Gallio plugin where the key for the Test Project Pattern does not have a "t" in "sonar.donet.visualstudio.testProjectPattern". You'll also need to provide the installation directory for PartCover.

#Gendarme
sonar.gendarme.mode=

# Gallio
sonar.gallio.mode=
sonar.gallio.coverage.tool=PartCover
sonar.gallio.runner=Local
sonar.donet.visualstudio.testProjectPattern=*UnitTest*;Testing*
sonar.partcover.installDirectory=C:/Program Files/PartCover/PartCover .NET 4.0/

# FXCop
sonar.fxcop.mode=

#StyleCop
sonar.stylecop.mode=

Step 7: Running an Analysis
From the folder with the sonar-project.properties file, run the command
$SONAR_RUNNER_HOME/bin/sonar-runner

You'll see the runner start up, listing some details like the working folder, etc, then it will kick off the source code parsing, then the plugins (such as running the unit tests). If there are any errors, a Java exception will be thrown. Sometimes these contain enough details to troubleshoot, sometimes not, so you may need to run with the -X command line argument to get additional details when errors occur.

After a successful run of the analysis, you can see the results in the sonar server webpage. Unless you have changed the port in the sonar.properties file, this will be at http://localhost:9000/

The homepage will have a link for each project (the value provided as sonar.projectName), which will take you to the project overview. Clicking on any of the links on the overview page will take you to the drill-down data for that metric.

References:

I used this page to start my journey, in addition to the Sonar project docs: http://www.ifunky.net/Blog/post/Install-and-Configure-Sonar-on-Windows-2008.aspx

The C# Plugins page: http://docs.codehaus.org/display/SONAR/C-Sharp+Plugins+Ecosystem

The Sonar installation page: http://docs.codehaus.org/display/SONAR/Install+Sonar

Update:
The Sonar Source page, complete with mailing lists, issue trackers, etc. http://www.sonarsource.org/support/support

Hope this helps!

Thoughts on Cyclomatic Complexity

2011-11-10T16:44:00.001-06:00

The following is an excerpt from an email/blog I wrote for internal consumption by my development team, but I think the concepts are universal and worth sharing. I've removed any specific references to company intellectual property and otherwise "sanitized" the details, so some of the details are lost in the process.

For those unfamiliar with the term, Cyclomatic Complexity is a formula for determining the varying logical paths that can be taken though a piece of code, thus determining a quantitative value for the code's complexity. A higher Cyclomatic Complexity score means that there are a lot of code paths and/or factors that affect the logic with the code. One way to think about it is this: to fully unit test a method, you should have one unit test for each logic path - thus a score of 10 would result in as many unit tests. Wikipedia entry: http://en.wikipedia.org/wiki/Cyclomatic_complexity

The higher the CC score, the more difficult a piece of code is to read and understand, to test, and therefore, riskier to modify and maintain over time. A general rule-of-thumb suggests keeping the CC score to under 15 greatly improves readability, maintainability and reduces defects in the code. Anything above that, and the code is likely violating the single responsibility principle and is trying to do too much.

So why do I bring this up?

Because over the last 9 months as I've been learning the company's codebase and delving into many areas of code where I have no prior knowledge, I've come across sections of code which have been difficult to pick up and understand, and/or which requires and extraordinary amount of knowledge about the inner workings of that code and the code around it (tight coupling). And, as a direct corollary, this code is has low (or no) unit test coverage to protect us against mistakes and misunderstandings.

From theory to concrete examples:

A little further down, I will discuss how I suggest we address this issue, but first I want to give some concrete examples of what I'm talking about.

I was recently tasked with making a fairly straightforward change to what data is displayed in one of our "legacy" controls; however, it took me almost a full day just to wrap my head around what the control was doing and all of the (inconsistent) ways other classes/controls interacted with it.

Method: private void SelectAll(SelectAllMode mode)

Cyclomatic Complexity: 23

So what makes this method particularly problematic? Nested conditionals.

At one point, there's:

an if block,
- with a nested if block
  - with a nested if block
    - with a nested foreach loop
      - with a nested if block
        
        with a nested switch block
        
        with a nested if block.

There are 11 if statements and 3 switch statements in a 96-line method -- 66 lines, if you don't count non-code lines.

Method: public void Initialize()

Cyclomatic Complexity: 17

This method does not have the level of conditionals of SelectAll(), but it does change behavior based on the type of a public property. In this case, the control accepts four types of DataSets (we utilize typed DataSets, which extend the DataSet class -- so there's a ProductDataSet, PricingPlanDataSet, etc) and acts differently based on the type of the dataset.

Another factor that affects code complexity is high levels of coupling with other classes, and in this case the Initalize() method is coupled with 36 other classes. This means that there is an increased potential that changes in those other classes will affect the logic in this method.

So, what can we, as developers, do?

Step 1: Knowing you have a problem

Aside from just generally being aware of Code Smells (http://www.codinghorror.com/blog/2006/05/code-smells.html) when you see them, there are a couple of tools that will help specifically call out Cyclomatic Complexity:

There is a free "CyclomaticComplexity PowerToy" ReSharper plugin that will mark any methods with a CC score above 17 with a ReSharper warning and display the CC score as a tooltip. It can be downloaded here: http://resharperpowertoys.codeplex.com/

(make sure to grab the v5.1 installer pack if you haven't upgraded from v5 -- the source code is already converted to use ReSharper v6). This will give you more "real time" warnings that things are ugly.

Additionally, Visual Studio 2010 Ultimate comes with code analytics features which will calculate several metrics for your code, including Cyclomatic Complexity, Depth of Inheritance, Class Coupling, Lines of Code, and something Microsoft calls a "Maintainability Index", where they try to apply an aggregation of the others to give you an overall score.

You can access this feature from the Analyze menu

This can be quite time consuming, though, depending on your hardware and the size of the Solution/Project you decide to analyze. At the end, VS2010 will spit out a report, which can be exported to Excel.

Step 2: Refactor to reduce complexity, improve understanding

I am a firm believer in a phase I picked up from Martin Fowler's writings (which he attributes to "Uncle Bob" Martin): "always leave the code behind in a better state than you found it". For me, this means small refactorings to address issues or improve understanding of the code. For example, if you're coming in to a piece of code and it takes you more than two minutes to figure out what the code is trying to do, you should look for ways to make the code more clear and embed the understanding you just gained into the code. Maybe that means pulling a set of lines out into their own method and naming that method in a way that makes it clear what those lines are doing. In other cases, it may be a larger refactoring effort.

The key here is make it better than you found it -- it doesn't have to be perfect. If everyone who touched the code makes it better each time, then those areas of the code that get touched a lot improve quickly -- which is good, because areas that change a lot are the places that have the most risk of getting broken.

So in my examples above, I would apply this rule in two ways:

1) pull out those nested conditionals in SelectAll() into sub-methods (assuming there isn't a clear way to remove them altogether)

2) utilize inheritance to reduce conditional blocks, such as those in Initialize(). By having the core/shared functionality in a base class, then having derived classes for each of the DataSet types, we could move the type-specific code into easier-to-maintain and smaller classes and greatly improve the readability and maintainability of the code.

Now, obviously I didn't do these things, otherwise they wouldn't still be here as examples. Why not? Well, because this class is tied to core functionality in our system and zero unit tests, so we would need considerable QA effort to regression test this functionality -- and had no QA resources available to do that. So from a risk standpoint, this was not the time to make changes. However, I do hope we can get scheduled time in the (near) future to do planned refactoring of this code.

Step 3: Don't make things worse

This one's easy: When writing new code, make sure it's clean, readable and testable.

Filtering DataTable with LINQ

2011-08-16T07:22:00.000-05:00

I had a situation today where I needed to modify an existing method that fetched a DataTable from the Data Access Layer, modified and returned it as a DataView. My task was to filter the rows in the DataTable based on a call into managed code (ie: not something that could be done at the db level).

Now, I'm somewhat new to DataTables, having used ORMs for most of my .Net experience, so this was actually more difficult than I initially expected. I was hoping to just set a value in RowFilter and be done. Unfortunately, as best I can tell, RowFilter does not allow row-specific dynamic filters (ie: you cannot call into a method with each row). Furthermore, the return value of the method needed to stay as a DataView, since I'm tasked next with backporting the change to our production branch and need to greatly limit the scope of my changes.

So after some googling, I was able to craft a solution using LINQ, which I wanted to document here for future reference. Obviously, the code has been changed to protect the guilty, which has the side effect of greatly simplifying the logic.

I needed to reference System.Data.DataSetExtentions to have access to the DataTable LINQ extensions.

Then I did this:

public abstract bool SecretFilteringMethod(int someId);

public DataView RetrieveFilteredRecords(int someId)

{

DataSet ds = DALServiceProxy.RetrieveRecords(someId);

DataTable myTable = ds.Tables[0];

// this is bound to a UI drop-list, so add some usability enhancing rows

ds.SuspendColumnValidation();

DataRow row1 = myTable.NewRow();

row1[Consts.Columns.Name] = "-- Select One --";

row1[Consts.Columns.ID] = -1;

myTable.Rows.InsertAt(row1, 0);

ds.ResumeColumnValidation();

// unlike most LINQ methods, this returns an EnumerableRowCollection<T> instead of IEnumerable<T>

var rowsAfterManagedCodeFiltering = myTable.AsEnumerable()

.Where(dpRow => !dpRow.IsNull(Consts.Columns.ImportantField))

.Where(dpRow => SecretFilteringMethod((int)dpRow[Consts.Columns. ImportantField]));

return rowsAfterManagedCodeFiltering.AsDataView();

}

The return value of AsDataView() is a LinqDataView object, which has its Table property set to the original DataTable, so the result is fairly close to what would happen if I'd set a RowFilter. However, instead of RowFilter, the RowPredicate property is set with the LINQ representation.

One thing to note: RowPredicate and RowFilter are mutually exclusive, so if a consumer later tries to set RowFilter to further refine the view, it will erase the RowPredicate, thus falling back to the base DataTable's full set of records.

Personally, I'd rather move away from the DataTable altogether and return an IEnumerable<T>, but that wasn't really a logistical option at this point.

Additionally, if you have an EnumerableRowCollection<T>, you can create a new DataTable with only the rows in the collection with the .CopyToDataTable() extension method.

References:

LINQ query on a DataTable:

http://stackoverflow.com/questions/10855/linq-query-on-a-datatable

DataTableExtensions.AsDataView<T>:

http://msdn.microsoft.com/en-us/library/bb908061.aspx

On Rhino.Mocks VerifyAllExpectations Vs AssertWasCalled

2011-06-21T09:17:00.001-05:00

After reading my Rhino.Mocks Quick Reference post, a colleague and I had a discussion about the proper way to validate expectations in Rhino.Mocks. Specifically, he questioned if my use of .VerifyAllExceptions() was correct for the ArrangeActAssert syntax, or, as he proposed, is it the "old" syntax, being replaced by the .AssertWasCalled() methods.

Since this is a person I respect, I decided not to smite him and instead took a mental note to do some ~~quick google searches~~ research. Having failed to actually find Ayende's opinion on the matter in the first 5 minutes, I did come across this post (http://lostechies.com/sharoncichelli/2010/03/19/rhino-mocks-examples-with-a-fix/), which raised a point I ~~did not realize~~ had forgotten:

In Rhino Mocks, expectations on stubs are not verified; only mocks are verified. If an object is created with GenerateStub instead of GenerateMock, then its VerifyAllExpectations method doesn't do anything. This is non-obvious because the AssertWasCalled and AssertWasNotCalled methods on a stub will behave the way you want them to.

In Rhino Mocks, a stub can keep track of its interactions and assert that they happened, but it cannot record expectations and verify they were met. A mock can do both these things.

So, if you are using a Stub, you must not use .VerifyAllExpectations(), because it will always pass.

Now, one might argue (and many, many on the Internet do argue) that a Stub should not have any expectations, so you shouldn't be calling either method. Stubs are for providing inputs to allow the code under test to run and limit your test to just the code under test, while Mocks are uses to validate behaviour. (they are one of the tests).

If someone does have a link to Ayende's opinion, please post as a comment or email/tweet me. I'll update this post if I find it.

Update: found this (http://ayende.com/wiki/Rhino+Mocks+3.5.ashx#ExpectExtensionMethod) where it seems Ayende expects people to use either case (ie: he doesn't state here that one is "proper"). I still seem to remember a blog he wrote stating a preference for one over the other, but I haven't yet found it).

Cool new Visual Studio Ultimate debugger

2011-06-14T20:13:00.000-05:00

Microsoft has release a new Visual Studio Power Tool called “Debugger Canvas” that looks to be a very useful way to debug your apps. Basically, instead of jumping from file to file while stepping through your code in debug mode, it will layout the callstack as method “bubbles” on a single tab, with local variable information available for each bubble:

There’s a really cool video demo on the download page that I would suggest you watch.

As well as this overview blog\announcement by Mary Jo Foley

Which points to this announcement page

Using Rhino.Mocks - quick guide to generating mocks and stubs

2011-03-21T17:15:00.006-05:00

This post is a general review of the Rhino.Mocks syntax. While it is intended primarily as a quick reference for myself, I hope anyone who stumbles upon it can also find it useful.

Updated 3/21/11: corrected Throw() example. (Thanks Joe for the catch!)

Updated 6/21/11: added link in Mocks vs Stubs to another post on VerifyAllExpectations vs AssertWasCalled, as well as link to posting with simple explaination

Updated 9/8/11: added “Testing non-public members” section

As a quick into: Rhino.Mocks (http://www.ayende.com/projects/rhino-mocks.aspx) is a .Net mocking framework which is extremely useful for stubbing out objects in your unit tests to ensure you're only testing what you want to test and to allow you to control the environment/state/surroundings under which your tests execute. It is written by Ayende Rahien, the same guy that created nHibernate. My typical dev unit testing environment would be VS (C#) + nUnit + Rhino.Mocks.

For readability, I suggest the Arrange, Act, Assert syntax (http://www.arrangeactassert.com/why-and-what-is-arrange-act-assert/).

This post is broken into several sections, starting with a general overview and then hitting on several specific use cases of interest:

- Types of Mocks

- Basics

- General Layer Abstraction

- Stubs vs Mocks

- Modifiers for the stubs/mocks

- Providing an alternate implementation / Using the input params

- Throwing an Exception instead

- Testing non-public members

Note: Rhino.Mocks can only mock/stub virtual members of a real class, so make sure the members you care about are virtual -- OR, my preference, mock/stub an Interface, in which case you can do whatever you want.

The surrounding code for the code used in my examples can be found at the end of this posting.

Types of Mocks

Rhino.Mocks supports three basic types of mock objects:

- Strict Mock

A strict mock requires you to provide alternate implementations for each method/property that is used on the mock. If any methods/properties are use which you have not provided implementations for, an exception will be thrown.

- Dynamic Mock

With a dynamic mock, any methods/properties which are called by your tests for which you have not provided an implementation will return the default value for the data type of the return value. In other words, you'll get back a 0 for number types, false for Booleans and a null for any object types.

- Partial Mock

A partial mock will use the underlying object's implementation if you don't provide an alternate implementation. So if you're only wanting to replace some of the functionality (or properties), and keep the rest, you'll want to use this. For example, if you only want to override the method IsDatabaseActive(), and leave the rest of the class as-is, you'll want to use a partial mock and only provide an alternate implementation for IsDatabaseActive().

Basics

To generate mocks, you'll use the static factory Rhino.Mocks.MockRepository. From this factory, you'll create you mock using one of a few generic methods (GenerateMock<T> for DynamicMocks, GeneratePartialMock<T> or GenerateStrictMock<T>) where the T is the class/interface being mocked.The method parameters, if you provide any, will be passed to the object's constructor.

As a general rule, I will generate all of my mocks in the test suite [SetUp] method to ensure everything is reset from one test to the next. You'll see in my test fixture file in the Code Used In Examples section that I have done just that.

General Layer Abstraction

Scenario:

I have a BLL object that I want to test, but it relies on a DAL object to fetch/update, etc records from the database. I want to test my BLL object in isolation from the DAL object, so I mock the DAL interface.

Things to note in this example:

I generate a stub for the dao's GetRecordFromDatabase() method so that when it's called with the recordId I care about, it will return my prepared value. This removes the dependency on the DAO layer (which is not even used since I'm mocking an Interface for the DAO) and ensure's my test is controlling the inputs and outputs to ensure I'm getting exactly what I want for my specific test condition.

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(myRecordId))

.Return(recordFromDatabase);

Code:

[Test]

public void TestGetImportantDataAndUpdateTheName()

{

//Arrange

int myRecordId = 100;

var recordFromDatabase = new ImportantData

{

Name = "Orignal Name",

RecordId = myRecordId

};

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(myRecordId))

.Return(recordFromDatabase);

//Act

var myRecord = _fancyBL.GetImportantDataAndUpdateTheName(myRecordId);

//Assert

Assert.AreEqual(myRecord.RecordId, myRecordId);

Assert.AreEqual(myRecord.Name, "All Your Base Are Belong To Us");

}

Stubs vs Mocks

A stub is simply an alternate implementation. A mock, however, is more than that. A mock sets up an expectation that

1) A specific method will be called

2) It will be called with the provided inputs

3) It will return the provided results

So when you setup a mock, you use the syntax .Expect() instead of .Stub(). Then, in your asserts, you can do . VerifyAllExpectations() on your mock to ensure reality matched your expectations.

In this example, the test will fail due to an ExpectationViolationException being thrown due to the Expect(101) not being called.

[Test]

public void TestExpectations()

{

////Arrange

int myRecordId = 100;

var recordFromDatabase = new ImportantData

{

Name = "Orignal Name",

RecordId = myRecordId

};

_mockDAO.Expect(dao => dao.GetRecordFromDatabase(myRecordId))

.Return(recordFromDatabase);

_mockDAO.Expect(dao => dao.GetRecordFromDatabase(101))

.Return(recordFromDatabase);

//Act

_fancyBL.GetImportantDataAndUpdateTheName(myRecordId);

//Assert

_mockDAO.VerifyAllExpectations();

}

Update: see my post for more on VerifyAllExpectations vs AssertWasCalled methods

Update: found a posting that I think does a good, simple explanation of Mock vs Stub, including a graphic!

Modifiers for the stubs/mocks

In the above example, I did _mockDAO.Stub().Return(). This will cause the mock object to return the provide value the first time it's called with the provided inputs. Sometimes we want to change this behavior, thus the following modifiers can be used between the .Stub() and .Return() calls.

Change how many times to use the stub:

Using the .Repeat.Any(), .Repeat.Once(), Repeat.Times(10) modifiers:

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(myRecordId))

.Repeat.Any()

.Return(recordFromDatabase);

Return the prepared value regardless of the input value:

Using .IgnoreArguments():

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(myRecordId))

.IgnoreArguments()

.Return(recordFromDatabase);

Advanced Argument Constraints:

You can provide very detailed conditions for when to use your return values by defining per-parameter constraints. For example, here I've said the input must be greater than 0.

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(Arg<int>.Is.GreaterThanOrEqual(0)))

.Return(recordFromDatabase);

Here's an example with more than one parameter: (There's a lot more than this – IntelliSense is your friend)

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(

Arg<int>.Is.GreaterThanOrEqual(0),

Arg<decimal>.Is.NotEqual(2.0),

Arg<List<string>>.List.ContainsAll(new List<string> {"foo", "bar"}),

Arg<object>.Is.NotNull,

Arg<object>.Is.Anything))

.Return(recordFromDatabase);

Additionally, you can put constraints on properties of objects used as parameters. For instance, if the input parameter had a bool property "IsSomethingICareAbout" and you only wanted to provide a return value when that property is true, you could do this:

_mockDAO.Stub(x => x.SomeMethod(myObject))

.Constraints(Property.Value("IsSomethingICareAbout", true)

.Return("foo");

You can put constraints on the input arguments in the same way:

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(0))

.Constraints(Is.GreaterThanOrEqual(0))

.Return(recordFromDatabase);

And constraints can be chained with boolean operators:

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(0))

.Constraints(Is.GreaterThanOrEqual(0) && Is.LessThanOrEqual(100) )

.Return(recordFromDatabase);

Note: Constraints must be listed in the order of the parameters (ie: the first set of constraints apply to the first parameter, the second set to the second param, and so on). And a constraint must be provided for each parameter (you can do Is.Anything() as well).

Providing an alternate implementation / Using the input params

Instead of using a .Return() with a simple value, you can provide a full implementation of the method using the .Do() method. This also allows you to get access to the input parameters. If you want, you can define a delegate and just call the delegate. I prefer to use lamdas unless the method is really long. So instead of my previous stub for GetRecordForDatabase which pre-configured a return value, I can do it on the fly:

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(0))

.IgnoreArguments()

.Repeat.Any()

.Do((Func<int, ImportantData>)(input => new ImportantData

{

Name = "Orignal Name",

RecordId = input

}));

Throwing an Exception instead

Instead of .Return(), you can use .Throw() to force an exception:

[Test, ExpectedException(typeof(NullReferenceException))]

public void TestNeverCallThisMethod2()

{

//Arrange

_mockDAO.Stub(dao => dao.GetRecordFromDatabase(0))

.IgnoreArguments()

.Repeat.Any()

.Throw(new NullReferenceException());

object inputValue = null;

//Act

_fancyBL.NullsNotWelcomeHere(inputValue);

//Assert

//nothing to do here -- ExpectedException

}

Testing non-public members

With Rhino.Mocks, you can't mock private or protected members, but you can mock internal members if you do a little extra work.

Specifically, you must allow your test assembly to access internal members of the assembly under test. This means adding two InternalsVisibleToAttributes to the AssemblyInfo.cs file of the assembly under test: one for the unit test assembly and one for Rhino.Mocks' DynamicProxyGenAssembly2. If you're using signed assemblies, you must put the full public key in the attribute.

You can get the public key for an assembly by using the "sn -Tp yourAssembly.dll" command in the Visual Studio Command Prompt.

For example: (no wrapping -- can't be any spaces in the public key)

[assembly: InternalsVisibleTo("DynamicProxyGenAssembly2, PublicKey=0024000004800000940000000602000000240000525341310004000001000100c547cac37abd99c8db225ef2f6c8a3602f3b3606cc9891605d02baa56104f4cfc0734aa39b93bf7852f7d9266654753cc297e7d2edfe0bac1cdcf9f717241550e0a7b191195b7667bb4f64bcb8e2121380fd1d9d46ad2d92d2d15605093924cceaf74c4861eff62abf69b9291ed0a340e113be11e6a7d3113e92484cf7045cc7")]

[assembly: InternalsVisibleTo("jwright.Blog.UnitTesting, PublicKey=00……ec")]

Code Used In Examples

Here are the code files I'll use in my example tests:

<DataAccessObject.cs>

using System;

namespace jwright.Blog

{

public class ImportantData

{

public string Name { get; set; }

public int RecordId { get; set; }

}

public interface IDataAccess

{

ImportantData GetRecordFromDatabase(int recordId);

void NeverCallThisMethod();

}

public class DataAccessObject : IDataAccess

{

public ImportantData GetRecordFromDatabase(int recordId)

{ throw new NotImplementedException(); }

public void NeverCallThisMethod() { throw new NotImplementedException(); }

}

<FancyBusinessLogic.cs>

namespace jwright.Blog

{

internal class FancyBusinessLogic

{

private IDataAccess _dataAccessObject;

internal IDataAccess MyDataAccessObject

{

get { return _dataAccessObject ?? (_dataAccessObject = new DataAccessObject()); }

set { _dataAccessObject = value; }

}

public ImportantData GetImportantDataAndUpdateTheName(int recordId)

{

var record = MyDataAccessObject.GetRecordFromDatabase(recordId);

record.Name = "All Your Base Are Belong To Us";

return record;

}

public void NullsNotWelcomeHere(object input)

{

if (input == null) { MyDataAccessObject.NeverCallThisMethod(); }

}

<FancyBusinessLogicTest.cs>

using System;

using NUnit.Framework;

using Rhino.Mocks;

namespace jwright.Blog

{

[TestFixture]

public class FancyBusinessLogicTest

{

private IDataAccess _mockDAO;

private FancyBusinessLogic _fancyBL;

[SetUp]

public void SetUp()

{

//reset all my objects under test and mocks

_mockDAO = MockRepository.GenerateMock<IDataAccess>();

_fancyBL = new FancyBusinessLogic {MyDataAccessObject = _mockDAO};

}

//-----

// Tests will go here

//-----

}

Blogger formatting frustrations

2011-03-15T20:29:00.000-05:00

I haven't been posting new blogs for a while after getting fed up with how difficult it is to get nicely formatted code snippets into Blogger. Using the site's rich text editor strips out much of the whitespace (tabs and carriage returns, in particular) making the code unreadable. I really just wanted to copy&paste my code from Visual Studio and keep the format, the font colors, etc.

I started the process of setting up my own WordPress instance (using MS WebMatrix), but was concerned about losing the links to my old posts, etc.

Then it occurred to me that Blogger has an email-to-post feature - and it supports HTML emails. So, I just need to craft my posts as rich text emails and submit them that way. Much easier!

Now if only I could draft HTML emails from my iPhone. At least I can edit them in another app and copy&paste into the mail client.

So, expect some blogs in the next week. I'm working on some unit testing related posts, beginning with a Rhino.Mocks overview (mainly for some of the folks at my new job).

.Net Knowledge Nuggets: A Note About Unicode

2010-12-14T12:54:00.000-06:00

A while back, one of my coworkers sent out a late-night plea for help with a unicode issue he was having. I thought I'd post the conversation here (slightly censored) for future reference:

Original email from coworker:

I’m looking for a little .NET help – particularly with the MailMessage class. I’m pulling the contents of an HTML page which is in French (and displaying properly in the webpage) and sending it via an email. I’m having difficulty getting the mail to use the correct encoding to show all the special characters correctly . If anyone has any experience doing this –please let me know.

My reply:

Make sure you're setting the .BodyEncoding property to Unicode. If you're getting "?" chars where special chars should be, it may actually be a problem with the way you're importing the content (the HTML could be getting munged before going into the MailMessage body). Take a look here: http://bytes.com/topic/asp-net/answers/345431-sending-mail-message-unicode-text

Coworker initial reply:

Thanks – I’m going to take a crack at this tomorrow. I’ve tried all the different encodings on the .BodyEncoding property but no luck. So I think you are right – it might be the way I am pulling the HTML. What a freaking pain.

And follow-up:

You were right. That worked. I owe you a beer – remind me to buy it for you next happy hour.

That reminds me -- I need to collect on that beer.

Remember that if you don't handle Unicode correctly in every place you manipulate the bytes (from initial read to final write), you run the chance of munging the bits because some string or character library is assuming 8-bit chars or assumes the wrong encoding. The key here was to ensure the initial read was reading it as Unicode -- something like this:
var myReader = new StreamReader(fileName, System.Text.Encoding.Unicode);

.Net Knowledge Nuggets: Dynamically Ignoring Unit Tests

2010-09-16T08:04:00.000-05:00

During a client project, we had a set of unit tests that connected to a webservice hosted on a client server which utilized Windows Authentication and would reject requests coming from our local dev environments (since we, as consultants, had laptops that weren't on the client's AD domain). This resulted in false-negative tests and in my experience, when you start having "known bad" tests that always fail, people start to ignore failing unit tests, which is not what you want.

So, to keep from having false-negative tests, I modified that particular suite of tests to auto-ignore the tests if the current environment is not on the right AD domain. This would keep them from showing as failures on our local dev environments, but they would fully run on our build server (vs. marking them as permanently ignored, where they wouldn't run on the CI server). Here's what I did:

/// <summary>

/// Will dynamically mark a test as Ignored if the system running the test is not

/// on the SOMECLIENT domain. This is because the API will fail authentication

/// otherwise and will provide false negative results

/// </summary>

public static void IgnoreTestIfOffDomain()

{

try

{

var domain = System.DirectoryServices.ActiveDirectory.Domain.GetCurrentDomain();

var domainName = domain.Name.ToUpper();

if (!domainName.ToUpper().Contains("SOMEDOMAINNAME"))

{

Assert.Ignore("NOT RUNNING API TESTS from current domain " + domain.Name);

}

catch (ActiveDirectoryOperationException)

{

Assert.Ignore("NOT RUNNING API TESTS from current domain (domain missing)");

}

Note that an ActiveDirectoryOperationException will be thrown if the system is not currently registered on a domain, thus the try/catch.

I put this into a method which I called in the TestFixtureSetup or individual test methods, depending if I wanted to ignore the whole suite or the individual tests.

West Monroe Partners again sponsoring Chicago Architects Group

2010-07-15T08:01:00.001-05:00

My employer, West Monroe Partners, will once again be sponsoring the Chicago Architects Group monthly meeting Tue, July 20th. Chicago has a strong technical community and as I've mentioned before, I'm proud to see WMP support the continued growth and learning of the community at large.

This month's topic is "Architecture And Business Intelligence" by David Leininger. Details and registration can be found at http://chicagoarchitectsgroup.eventbright.com/ Presentation begins at 5:30pm at the ITA/TexusNexus space in the Loop. Attendance is free thanks to corporate sponsorships, such as West Monroe Partners!

I hope you'll join me there!

Chicago Architects Group was founded on the goal to unit various technical architecture competencies into a community for collaboration and knowledge sharing. The group is for people to meet their peers and share some stories and ideas. Architects and those aspiring to understand technical architecture better will enjoy this group. (text from http://chicagoarchitectsgroup.com).

West Monroe Partners Sponsoring Chicago Architects Group

2010-05-18T08:05:00.000-05:00

For over a year now I've been attending technical user group meetings across the Chicago area and one in particular has caught my interest more than the others. The Chicago Architects Group ("CAG") has consistently provided presentations and discussion on topics relevant to me and by presenters with experience and knowledge on the subjects at hand. Presentations has ranged from Dependency Injection & Inversion of Control Containers, to Onion Architecture, to Agile Project Management strategies. And tonight will be Azure for Architects. All targeted to the technical architect audience, who discuss the topics, share personal experiences and insights.

Shift gears now to one of the reason I enjoy working for West Monroe Partners: WMP has a set of values core to the culture of the company that help make the company a great place to work:

A culture of learning and growth
The belief that our clients should learn and grow with us
The belief that diverse experiences and thought are required to develop a balanced set of knowledge and skills

So now I am happy to be able to bring these two groups together, I believe to the mutual benefit of all: this month West Monroe Partners will be sponsoring the CAG meeting by funding the food and drinks. And there are plans in place to further sponsor at least two more meetings this year. So what does this mean? Well, it means that CAG meets continue to be free to anyone who wants to attend. It means that people can continue to learn about new skills, new technologies or alternate views on existing skills for the purpose of learning and growing and picking up on the experiences of others. It's a small way for WMP to help a community group continue to grow and better the larger technical community and I hope it's a pattern that will continue.

And it means I continue to get free pizza -- which helps offset the long commute back to the 'burbs when staying downtown for after-hours meetings. :-)

Node.js, et al: Highlights from JavaScript Chicago January Meeting

2010-02-01T23:20:00.000-06:00

(Each month I plan to attend technical user groups in the Chicago area to (re-)learn from peers experiences with new and existing technologies and to network with like-minded techies. This blog is one in a series of recaps of some of the more interesting aspects of the meetings for my own purposes (this is a “web log” afterall) and for others to get a general taste of what’s available in the Chicago user group scene.)

User Group: JavaScript Chicago (JS.chi)

http://javascriptchicago.com

http://www.meetup.com/js-chi/

Twitter: @chicagojs

Location: CashNetUSA (200 W Jackson – In the Loop)

Meeting Date: Thursday, January 20, 2010

The js.chi meetings follow a format of two main presentations, followed by three “lightning talks” (5-min quick presentations – although I have only seen one actually achieve the 5-min timeframe). Fairly large attendance again (20 – 30 attendees, from what I could tell), but had audio problems again, which meant it was difficult to hear beyond the first five rows or so (hint: arrive early to get a good seat – beers/sodas and networking before the meeting to help it along).

Presentations:

Prototypal Inheritance

This month, the opening presentation was an overview of prototypal inheritance in JavaScript. Honestly, I didn’t find the topic that engaging, which combined with an apparently nervous presenter, meant that I didn’t take that many notes. Check out the website for a video, and http://javascript.crockford.com/prototypal.html for more info.

Node.js

Jason Huggins (twitter @hugs – formerly @jhuggins) of SauceLabs (“Selenium in the cloud”) presented an overview of Node.js. This presentation I did find intriguing, although may be more applicable to my Linux friends than those living in a Microsoft world. (If you don’t know what POSIX means, you probably won’t want to use Node.js)

Node.js, at a very high level, is a JavaScript environment that lives outside the browser. Think: server-side JavaScript or shell-scripting on crack. It’s a three-way marriage with the following actors:

Common.JS: An attempt at a common/standard JavaScript library, not tied to any specific framework. Most other high-level languages have some core library (.Net has – well, the .Net core libs, Perl has CPAN, etc). Includes library support for file access, database drivers, console drivers, etc. (http://commonjs.org)
Google’s V8 JavaScript engine: This is the JS engine in Chrome – written in C++, it’s very powerful, very fast, but really not a standalone system – intended to be run from within a browser.
Node.js: The glue that holds it all together. Provides a command line interpreter in addition to the JavaScript run-time environment. Adds additional libraries to round-out common.js. (http://nodejs.org)

Node.js is fairly new to the scene – started circa Nov 2009. It requires a POSIX environment, which means additional installs in Windows-land if you want to use it. Use out-of-the-box for Linux and friends.

Per the presenter, search Google for “node.js jsconf slides” for the originators presentation slides, which go into depth about the project. (You can also find video of this slide presentation with audio overlay at http://blip.tv/file/3152542 or just this presenter’s slides at http://www.slideshare.net/hugs/nodejs-javascript-chicago-meetup )

So what’s so great about using JavaScript outside a browser? Well, unlike most other scripting languages, JavaScript is (can be) event-driven – meaning you can provide anonymous functions to method calls to create async callbacks when handling results, providing a more responsive and scalable system, as well as being a feature-rich language:

db.query($sql, function(result) {

//do something with result

}

The framework provides library inheritance (like the “using” keyword in C#, #include in C).

var sys = require(“sys”);

var http = require(“http”);

http.createServer(…);

The presenter’s proof-of-concept demo was a ~20 line JavaScript application that bound itself to a port as an http listener and responded with an HTTP 200 request and “Hello World” body to any incoming request. Could be very useful for mocking up web interfaces.

Some drawbacks:

- Library is new and immature – but appears to be growing up quickly.

- Currently no “step debugging” support – start loving log output now L

Other notes:

The presenter is a founder of SauceLabs (http://saucelab.com), which provides cloud-based support for the UI testing framework Selenium (http://seleniumhq.org/). I’ve been wanting to play with Selenium, and if/when I do, this looks like an interesting model for grid-based/pay-as-you-go script agents.

Lightning Talks:

Chad Norwood presented his JavaScript-based project for using Google Maps as a filter for large, geography-based data sets. Specifically, he’s linked Google Maps and Google Calendar to allow the user to display/filter calendar entries by physical location. In his example, he used a usergroup calendar (such as http://chicagotechcal.com – love it!) and could zoom in/out to narrow/widen the list of events and chart them on the map. Cool idea. Play with it here: http://chadnorwood.com/gcm
The second talk was kind of a teaser for another presentation at a local Ruby group. MongoDB is a JSON-based database (apparently similar to CouchDB) which stores data as BSON (Binary JSON). Its schema-less, so stores the objects as you define them. I could see this useful for cases where you may want to cache AJAX responses for improved performance, but I struggle to see where/how you would use this in an enterprise environment (full disclosure: I’m a big fan of RDBMS, so you’d have to really wow me to move away from a relational design). More can be found here: http://mongo.kylebanker.com

That’s all for tonight. Typically, after the meeting a smaller group goes to the Elephant and Castle pub for drinks and networking – but after a long week, and a list of must-do’s for the client status meeting the next morning, I opted instead to head home.

Bonus Teaser:

I’m working on a blog (rather, the code which will lead to a blog) as an intro to Microsoft’s ASP.NET Charting controls and Google’s .NET Analytics API. Think: Building your own custom Google Analytics charts. Why? Because I want to see them on my Flash-less iPhone. And ‘cuz I’m a geek.

Dependency Injections and Inversion of Control Containers -- Chicago Architects Group January Meeting Recap

2010-01-19T21:59:00.000-06:00

User Group: Chicago Architects Group

http://www.chicagoarchitectsgroup.com/

Location: ITA (200 S Wacker - next to Sears Tower)

Meeting Date: Tuesday, January 19th, 2010

The CAG meetings I’ve been to have had between 10 and 20 attendees and tend to be more of the Architect/Team Lead types and less of the hard-core, in your face techies/developers (this is not a bad thing, in my opinion). Like most user groups, they have pizza and soda as well as giveaways at the end (today included Windows 7 Ultimate, Office 2007, a wireless mouse and several books). Personally, I find the meetings to be useful, although I know some of my coworkers who have come with me will likely not attend unless the topics are of great personal interest to them.

Presenter: Tim Murphy

Blog: http://geekswithblogs.net/tmurphy

Twitter: @twmurph

Topic: Dependency Injection and IOC Containers

I'll admit up front that this entry will be a little light on content, such as the pros/cons to using this pattern – mainly I’m making notes for me to reference later. Here is a link to the presenter’s wrap-up as well, with slides and code (using Unity and Windsor). http://geekswithblogs.net/tmurphy/archive/2010/01/19/cag-january-2010-wrap-up.aspx

But by all means, please read on :-)

The Dependency Injection pattern is intended to lessen the coupling of objects by having dependant objects provided to an object instead of having the object itself generate the dependant objects. For example, you have a Widget object which depends upon a LoggingManager object and a WidgetValidator object. Without Dependency Injection, you might instantiate your dependant objects insider your object constructor, like this:

namespace WrightThisBlog.blogspot.com

{

public static class MainApp

{

public static void main(object[] argv, int argc)

{

var myWidget = new Widget();

//do some widgetity stuff here

}

public interface IWidget {}

public interface ILogger { }

public interface IValidator { }

public class LoggingManager : ILogger { }

public class WidgetValidator : IValidator { }

public class Widget: IWidget

{

ILogger _logger;

IValidator _validator;

public Widget()

{

_logger = new LoggingManager();

_validator = new WidgetValidator();

}

But now you're tied to a specific implementation of ILogger and IValidator and if you ever wanted to change them, you'd have to modify your code anywhere you referenced those objects and replace them with your new ILogger object. This is less than ideal and makes your code a bit fragile.

So how does Dependency Injection change this? Basically, by having the caller provide the ILogger and IValidator to the Widget object, either as parameters in the constructor or as properties. This is further decoupled in my example by using factory classes:

namespace WrightThisBlog.blogspot.com

{

public static class MainApp2

{

public static void main(object[] argv, int argc)

{

IWidget myWidget = WidgetFactory.GetWidgetInstanceViaConstructor();

IWidget myNextWidget = WidgetFactory.GetWidgetInstanceViaProperties();

//do some widgetity stuff here

}

public class ImprovedWidget: IWidget

{

public ImprovedWidget(ILogger loggerToUse, IValidator validatorToUse)

{

LoggerToUse = loggerToUse;

ValidatorToUse = validatorToUse;

}

public ImprovedWidget() {}

public ILogger LoggerToUse { get; set; }

public IValidator ValidatorToUse { get; set; }

}

public static class WidgetFactory

{

/// <summary>

/// Could provide in constructor

/// </summary>

public static IWidget GetWidgetInstanceViaConstructor()

{

ILogger logger = LoggerManagerFactory.GetLoggerInstance();

IValidator validator = WidgetValidatorFactory.GetValidatorInstance();

return new ImprovedWidget(logger, validator);

}

/// <summary>

/// Or via properties (beware unset properites!)

/// </summary>

public static IWidget GetWidgetInstanceViaProperties()

{

return new ImprovedWidget

{

LoggerToUse = LoggerManagerFactory.GetLoggerInstance(),

ValidatorToUse = WidgetValidatorFactory.GetValidatorInstance()

};

}

public static class LoggerManagerFactory

{

public static ILogger GetLoggerInstance() { return new LoggingManager(); }

}

public static class WidgetValidatorFactory

{

public static IValidator GetValidatorInstance() { return new WidgetValidator(); }

}

Now you’ve created a loose coupling between your Widget and its supporting objects, which will come in handy when

· You want to use a mocking tool (like RhinoMocks) to unit test your code

· You want to replace your LoggingManager with something else that implements ILogger – now you only need to update the factory classes (technically, using a factory class is a different pattern, but works well here)

Still, you’re coding in a concrete implementation in your factory, so while you’ve isolated the number of places you need to update, you’re still hard-coding in an implementation. Additionally, if there are very deep dependencies (you’re Logger needs a FileManager which needs a PermissionsManager which needs a CurrentUserManager….), this can get pretty ugly to manage and you end up writing a lot of plumbing code that is only tangential to the application’s real purpose.

This is where the IOC Containers come into play. Using an IOC framework, you define which concrete classes implement your interfaces and what dependencies they have. Then you use the IOC container like a factory class to instantiate your objects. There are two primary means for defining the dependency trees: using XML in your app.config or via code. Per the group discussions, some IOC frameworks provide tools (such as Structure Map) which will auto-generate your code mappings, while others (like Microsoft’s Unity Framework) use [Dependency] attributes to denote where there are dependencies. This allows you to say “I need an IWidget” and the container framework will know that ImprovedWidget should be created and it has dependences on ILogger and IValidator, which are provided by LoggerMananger and WidgetValidator, and so on.

using Some.IOC.Framework;

namespace WrightThisBlog.blogspot.com

{

public static class MainApp3

{

public static void main(object[] argv, int argc)

{

var IOC_container = IOC_Framework.GetContainer();

IWidget myWidget = IOC_container.GetObject<IWidget>();

//do some widgetity stuff here

}

Resources and Reference:

Some IOC Frameworks:

· Structure Map

o Has “scanning” tool to auto-map dependencies and limit the amount of manual configuration needed. (If only one class in your project implements a given interface, that class will be used when that interface is requested).

· Ninject

· Castle Windsor

o Uses app.config to defined assemblies/classes to use for each interface. Uses constructor to provide dependencies.

· Microsoft Unity (part of P & P group)

o Uses [Dependency] attributes places on class properties to determine where dependencies are needed, as well as to get/set them.

Some links:

· Martin Fowler: http://martinfowler.com/articles/injection.html

· MSDN article on DI: http://msdn.microsoft.com/en-us/magazine/cc163739.aspx

· DI Tutorial: http://tutorials.jenkov.com/dependency-injection/index.html

CQRS Overview - Chicago Alt.Net User Group January Meeting Recap

2010-01-13T23:50:00.002-06:00

UserGroup: Chicago Alt.Net

http://chicagoalt.net/

Location: Willis (aka Sears) Tower

Meeting Date: Wednesday, January 13, 2010

If you're in the Chicagoland area and do any .Net programming, I recommend this group. It's got a decent turnout (usually). I've been to three meetings with anywhere from 20 - 40 people in attendance. And they have good giveaways if you stay till the end. Today was especially good: A copy of Windows7 Ultimate, Office 2007, the JetBrain's product of your choice (ReSharper, dotTrace, TeamCity, IntelleJ -- great products if you're not familiar), $50 Barnes & Noble card, and of course a few Microsoft tech books. And free pizza!

Presenter: John Nuechterlein (aka Jdn) (http://www.blogcoward.com).

Topic: CQRS in roughly an hour or so

(CQRS == "Command Query Responsibility Segregation")

So, I'll be honest: Going into this presentation, I had only a very vague idea of what CQRS is and even less idea why I should care. (I had actually planned to play poker that night instead, but poker got canceled L). Leaving this presentation, my mind was swirling with all of the projects where CQRS would have been great to use (and the many parallels with the "massively" distributed computing environment we developed at Wayport. -- where "massive" is defined in pre-Google, pre-Facebook terms).

So, what is CQRS? Frankly, I couldn't do it justice in this blog -- but I needn’t try, since my smarter, more eloquent people have done so before me. Actually, from what I understand, CQRS got its start in the blogosphere. For more depth, here is one key link (http://elegantcode.com/2009/11/11/cqrs-la-greg-young/), but here's my lame attempt at a quick definition:

CQRS is a system architecture/design pattern that separates the act of reading data (query) from taking action (command) in order to produce a system which easily scales and provides some useful benefits (such as "playable" event logs) that make the maintenance of the system less burdensome. (For my purposes, I'm going to designate CQRS as an "architecture", partly because I don't want to write "architecture/design pattern" anymore.)

In my mind, CQRS lends itself pretty well to web-based systems, and SOA/SaaS at that, although it could be applied elsewhere. I'm probably jumping ahead of myself a bit, but that statement is useful for backing this next statement. To consider using CQRS, you must buy into a fundamental assertion: your data is always stale. How stale depends on your system, but the fact that my last blog entry was about data caching techniques just goes to show that particularly in web-based system, we intentionally make some of our data stale by putting it into a cache.

But even without intentional caching, your data is stale. Consider this: you have an eCommere site (say, amazon.com). Your end user pulls up the product page for the Widget2010 product, which includes price and quantity in stock information. In the 60 seconds it takes the user to read the page and click the shopping cart button, five other people have ordered their own Widget2010, so the quantity in stock is now less than what's on the first user's screen -- thus, the data is stale. You know this -- you expect this -- and you've already coded a dozen ways to deal with this, so accepting that "your data is always stale" is really not that big a leap of faith.

So why is that important? Because the CQRS architecture separates the reading of data and the acting upon data into two separate logical areas, where the reading area has a data cache which is stale. Now, it may only be 1 ms stale, but stale none the less.

So let's dive in to my high level summary of Jdn's high level overview. (note: if you want to bypass my bias and possibly complete misinterpretation of the presentation, it is (or will soon be) available on the http://chicagoalt.net website in video form):

CQRS has four logic areas in your system design, as pictured in this drawing stolen squarely from the blog linked above:

Queries
This is the "reporting" piece -- or said another way, this is the read-only interface into your data. Basically, it's a lightweight data layer reading from your data store and proving DTOs back to your UI. It does NOT go through your Domain Model. It is simply a read-only view into your database. Your database, however, is really a cache of your data, and likely a local cache. In other words, since the data is just a cache of the "real" data, why not push it out as close to the UI as possible to minimize the latency to your UI (since the majority of your UI's interactions with your data is reading, making this link super-fast will in turn make your system faster). Since these are read-only views into your data, the DTOs are extremely simple and can/should be specific to the consumer (ie: no "product" DTO, but rather a productForOrderPage which only has the data needed by the order page) reducing the amount of data getting dragged around between layers, etc. Why pull your full product from the db if you only need 20% of the information for the current page? Need to scale your UI? Just add another data store/cache -- it's practically a rubber stamp.

Commands
The next logical area of this architecture is the Command Bus/Command Handlers. This is where the next tenant of CQRS comes into play: system actions (aka "commands") have specific intent. If you've practiced Agile Development and you're familiar with the concept of the User Story -- basically requirements as documented from the viewpoint of a user's interaction with the system -- then this may be easy to grasp. Basically, Commands are actions taken on your Domain objects. For example: CustomerChangedAddressCommand or AddProductToShoppingCartCommand.

Commands are the only way to update data in your system and are seen as atomic actions which can be wholly accepted or rejected. Commands are generated by the UI and pushed onto the Command Bus and picked up by Command Handlers. Command Handlers in turn pass those commands into the Domain. Since commands go onto a bus, they can be queued, prioritized, etc, just like any message on a messaging bus, thus allowing you to ensure your most important commands are handled appropriately (another scalability knob you can adjust to meet your performance needs).

Now, since commands require specific intent (ie: user updates their address/places item into shopping cart), this does have implications for your UI -- specifically, you can't have "Excel-like screens". There's no "update everything about this user and here's all the data" command (or there shouldn't be), so if that's what you're looking for, you may want to look elsewhere. But honestly, this may not be a bad thing, as it forces some system designs which will likely result in more user friendly, reliable systems in the long run.

Internal Events / Domain
This area is the authoritative knowledge source. Your business logic resides here. Your Domain objects reside here. Here is where you'll find the Event Store. This is the most complicated part of the system, and is where the presenter struggled at times to explain some concepts / answer some questions, so this is where you'll likely want to ensure you've done your homework. (To his credit, Jdn fully admitted up front he did not know all of the ins-and-outs of this area and did his best to explain).

As commands come into your Domain from the Command Handlers, your Domain objects validate business rules to determine if the command is valid for the current state of the world and either reject the command en-whole, or, in one atomic action, update the state of the world according to the command (thus, an “event” occurs).

Events are written to the Event Store, which is persisted, likely in an RDBMS, ODBMS, etc. A snapshot of the state of the world is taken periodically and if you want to recreate a place-in-time, just take the previous snapshot and replay the events in order from that snapshot until the place-in-time you care about. Your domain could just remain in memory, if you'd like -- otherwise you'd pull the most recently stored (via snapshot) version of your domain object(s) and replay any new events against that object until it's fully restored, then execute your new command and presto-chango!

Now this was of particular interest to me from a troubleshooting/audit standpoint. If a problem occurs on the site, pick a snapshot from right before the problem occurred and replay your events to reproduce the issue. (QA engineers applaud here.) New version deployment go ghastly wrong, rollback the events until right before the deployment. Theoretically, you could even re-run the events again against the older version of the software and (unless the events weren't supported in that version) recover the data changes. Try doing that when your domain is specific to your db schema or your audit history is at the db table level (how do you replay an "insert" when the columns have changed?)

One note here is that your Event Store is "write only" -- meaning you don't ever delete things from your domain, you just adjust them. The presenter used the analogy of an accountant ("accountants don't use erasers"). If an accountant finds an error in the ledgers, they don't edit that line item -- instead they create an adjustment line item to offset the difference. The Event Store is similar -- you create events to adjust/negate/otherwise manipulate your data.

External Events / Publication
Now, to go full circle, the Domain / Event handling system will publish any events that it handles (but not those that it rejects). Any data stores/caches will subscribe to that feed, and will update themselves based on those events. Thus, your data cache is only as stale as it takes to process the published events. Again, you use a message bus here (or a webservice, etc) and use prioritization queues if you wish to enhance performance/scalability.

This is where the concept of "Eventual Consistency" is used -- that is, your Domain and your Data Stores will eventually sync up, just not necessarily in "real time" -- but we're ok with that because we've agreed that latency is almost always ok and availability trumps correctness in the Data Stores -- because "your data is always stale" anyway.

And the big finish...
To conclude, the presenter touched on reasons why you wouldn't want to use CQRS, including:

· it's new, it's different

· multiple data stores (maybe you don't want this)

· operational complexity

· lots of commands, events, handles, etc

…and my conclusion
All-in-all, this is definitely something I will consider for large systems in the future, although it is likely overkill for most systems in the market I serve. I suggest doing some Google searches if you're interested at all in learning more (seems to be a good deal of data, videos, webinars, blogs, etc. out there).