The good stuff is that windows live writer is able to decide date of the post directly on the post page, I’m writing this post 03 July 2009, but it will be published only the next day. Moreover in the net there are some suggestions on how to show a list of future post in the sidebar.

I’m not sure if I’ll use this feature, I’m a fan of Blogging at the moment, but for technical question, especially for series of posts dedicated to the same argument, the ability to schedule post in advance can be really interesting.

Alk.

Tags: Blogging

Another interesting stuff to explain is how to execute custom code during the build, to demonstrate this concept I’ll show how to tweet a warning message when test phase fails. To execute custom code you simply need to create a custom msbuild task that does everything you want, then you simply need to call this custom task in the build script. The first step is creating the task.

public class TweetTask : Task
{
    [Required]
    public String Username { get; set; }
    [Required]
    public String Password { get; set; }
    [Required]
    public String Tweet { get; set; }

    public override bool Execute()
    {
        TwitterService service = new TwitterService();
        String tweet = Tweet.Substring(0, Math.Min(Tweet.Length, 140));
        return service.SendMessage(Username, Password, tweet);
    }
}

As you can see, creating a custom task is really simple, you should inherit from the Task base class and override the Execute() method. In this example everything is done in the TwitterService class that I took from this post. To pass parameters to this task you can simply create public properties in the task class, and use the [Required] attribute to specify that the caller absolutely needs to specify these properties to use the task.

Now you only need to compile this class into a dll, and make it avaliable to the build process. This is probably the most interesting part, because the build engine must be able to locate this dll in order to execute your custom action. Another requirement I want, is that you must be able to use your custom task without the need to physically go to tfs server and copy the dll into some directory. This because you must be able to fully configure the build with the same set of permission you have in tfs. I do not want to ask someone “Hey copy this dll into the tfs server”, I simply want to configure everything in visual studio.

This is a well known problem for everyone that works with Continuous integration machines, and the solution I like is inserting everything is needed for the build in a specific folder into the source control system of the project.

As you can see I created a folder called BuildTools, inside it another folder called MsBuildCustomTasks and inside the compiled dll with my actino. Now I’m sure that the dll with my custom task is included into the source control system, and I can refer to it inside the TFSBuild.proj build file.

<UsingTask
        TaskName="DotNetMarche.MsBuildExtensions.Twitter.TweetTask"
         AssemblyFile="..\sources\BuildTools\MsBuildCustomTasks\DotNetMarche.MsBuildExtensions.dll"/>

to import the custom task you simply need to specify the task name and the assemblyFile in a <UsingTask> directive, as you can see the path of the dll is ..\sources\xxx where xxx is the path of the dll relative to the root of the source control system. Now if you do a check-in the next build will comprehend your custom task. This technique is useful, because you can be sure that each build always do a “Get Latest” to build the latest version of the source, thus getting also every tool it need to do the build.

Now you can use this task wherever you want:

<TweetTask
    Condition="'$(IsDesktopBuild)' != 'true' and '$(TestSuccess)' != 'true'"
    Username="alkampfer"
    Password="xxxxxxxxxxx"
    Tweet="Unit Test Failure in build number: $(BuildNumber)" />

As for the previous post, I’m asking to tweet a message to my twitter account when test phase fails during the build, I made a test fails, commit everything and after integration machine finished to build I can see this in my twitterDesk

WOW you can have your CI machine tweet whenever you want with less than 20 lines of code. You can now create a dedicated twitter account for every project you have, and you can follow status of the build in every device that supports twitter :D.

Alk.

Tags: Team Foundation Server

In this post I’ll show how to automatically open an issue in TFS when a test fails. This is a standard requirement, because in such a way you are immediately aware when some test are failing.

<Target Name="AfterTest">

    <!-- Refresh the build properties. -->
    <GetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                                 BuildUri="$(BuildUri)"
                                 Condition=" '$(IsDesktopBuild)' != 'true' ">
        <Output TaskParameter="TestSuccess" PropertyName="TestSuccess" />
    </GetBuildProperties>

To execute some task after tests are run, you can simply override the target AfterTest then insert inside it all the tasks you like. To know result of tests we need to refresh some properties of the build, this is simply done with the GetBuildProperties task, you need to insert the serverUri, the BuildUri and ask to know the value of the TestSuccess parameter. Now you can know if some tests failed during test run, and you can create an issue if TestSuccess property is false.

<CreateNewWorkItem
    Condition="'$(IsDesktopBuild)' != 'true' and '$(TestSuccess)' != 'true'"
     TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
     BuildUri="$(BuildUri)"
     BuildNumber="$(BuildNumber)"
     Description="Test run failure in build"
     TeamProject="$(TeamProject)"
     Title="Unit Test Failure in build number: $(BuildNumber)"
     WorkItemFieldValues="$(WorkItemFieldValues)"
     WorkItemType="$(WorkItemType)"
     ContinueOnError="true" />
</Target>

The condition property is used to create the item only if the DesktopBuild is false and TestSuccess is also false, this means that issue are created only when the build is executed by the server and at least one test failed. You can configure every parameters but the most important one is the WorkItemFieldValues that can be specified in this way.

<PropertyGroup>
    <WorkItemFieldValues>
        priority =2; severity=3; rank =4; triage =Investigate
    </WorkItemFieldValues>
</PropertyGroup>

It must contains a list of issue properties that you want to set when you create the new work item. Now I reset back the TreatTestFailureAsBuildFailure to false, so the build will partially succeed even when some tests fails, but for each build that fails unit testing task an issue is automatically generated, with the desidered properties.

Configuring the build adding msbuild tasks is quite simple and permits you to fully configure your build.

alk.

Tags: Team Foundation Server

If you select “custom Configuration” you can fine tuning the configuration, you can choose as example the database you want to use.

Since I already configured sharepoint services and created sites I can specify them in the next configuration step.

You can configure almost anything with simple wizards, the overall process can be complicated if you are not familiar with the tecnologies involved, since you have to configure reporting services, integration with sharepoint etc. etc. When you finished configuring everything the wizard runs a check that tells you what is wrong and what is good.

From this simple list of errors and warning you can easily address every problem you can have, in this example I forgot to set the Database Agent in automatic execution, so I started it and do again a check.

During the whole installation I had only a little problem with error TF255275 that states

The following Web service for SQL Server Reporting Services could not be accessed: http://tfsd2010/ReportServer/ReportService2005.asmx

This is caused by a wrong configuration of reporting services, actually I completely forgot to create databases for Reporting services :), but the error is somewhat misleading me to wrong cause. you can find a detailed discussion of this error here.

I must admit that the installation process is more friendly and improved respect Tfs 2008.

Alk.

Tags: Team Foundation Server

The "ResolveKeySource" task failed unexpectedly.

System.InvalidOperationException: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.

This error happens because when the project has a certificate used by click once, you need to import that certificate in order to compile, and since my certificate is password protected the system need to show a form that ask for password to import certificate. Since cc.net runs as service it tries to shows the dialog, but it cannot gain access to a user interface and fail miserably.

The solution is to create a specific user to use for cc.net service (a good practice that you should always do), open a command prompt with credentials of that user, run your build script and wait for the form that ask you the password of the certificate. Once you give the password to msbuild script it installs into appropriate location and subsequent builds will run just ok.

Alk.

Tags: CC.Net

This kind of option is not present in the wizard, and it is time to make your hands dirty. To unleash the full power of team builds, you need to know msbuild, in order to configure the build editing the msbuild file of the team build. First of all go to the team explorer, open the source control node (step 1), here you will find a folder named "TeamBuildTypes (step2) that contains a subfolder for each build definition, in the BuildWithTests folder there are a couple of file, the one named TFSBuild.proj (Step 3) is the one that contains all operations of the build.

In order to edit that file, you first need to grab a copy from the source control to your local folder, simply right click in the teamBuildTypes folder to “Get the latest version”

Ok now you can double click the tfsbuild.proj file to edit it. This is a simple msbuild file, and in the first lines you find this instruction

<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets" />

it imports a file called Microsoft.TeamFoundation.Build.targets that contains base targets for a build definition. This file permits you to override some specific target to execute custom code in specific moments of the build process. You can find a list of overridable targets here. From that list you can verify that there is a target called BeforeTestConfiguration that can be used to specify some custom configuration for test execution.

To fully understand how overriding works, you can open the Microsoft.TeamFoundation.Build.Targets file and look at the BeforeTestConfiguration target.

  <!-- Override this target to execute custom tasks before the testing of an individual configuration -->
  <Target Name="BeforeTestConfiguration" />

Since its only purpose is running your own code, it contains no task. Then you can verify on how it is used by the basic test runner task.

  <PropertyGroup>
    <TestConfigurationDependsOn>
      BeforeTestConfiguration;
      CoreTestConfiguration;
      AfterTestConfiguration;
    </TestConfigurationDependsOn>
  </PropertyGroup>
  <!-- Batch target for individual configuration testing -->
  <Target Name="TestConfiguration"
          DependsOnTargets="$(TestConfigurationDependsOn)"
          Outputs="@(TestOutputs)" />

...

From that code you can verify that test configuration depends on BeforeTestconfiguration, CoreTestConfiguration, AfterTEstConfiguration.Of these three targets, only CoreTestConfiguration contains real actions, the other two are there only to insert own task into the build process. If you look at the CoreTestConfiguration you can find this code.

    <PropertyGroup>
      <ContinueOnTestError Condition=" '$(StopOnTestFailure)' != 'true' ">true</ContinueOnTestError>
      <ContinueOnTestError Condition=" '$(StopOnTestFailure)' == 'true' ">false</ContinueOnTestError>
    </PropertyGroup>

Ok this is exactly what we need, the Continue on test property is set to true or false depending on the StopOnTestFailure variable. Another interesting piece of code is this one

  <Target Name="RunTest"
          DependsOnTargets="$(RunTestDependsOn)"
          Outputs="@(TestOutputs)">

    <MakeDir
          Directories="$(TestResultsRoot)"
          Condition="!Exists('$(TestResultsRoot)')" />

    <MSBuild Projects="@(ConfigurationList)"
             Properties="TreatTestFailureAsBuildFailure=$(TreatTestFailureAsBuildFailure)"
             Targets="TestConfiguration"
             StopOnFirstFailure="$(StopOnFirstFailure)">

That shows you an interesting property called TreatTestFailureAsBuildFailure, that is initially set to false. Now you have every information needed to solve the original problem, you can simply edit TFSBuild.proj file inside the definition of your build, and add this code, after the import statement of the file Microsoft.TeamFoundation.Build.Targets.

<Target Name="BeforeTestConfiguration">
    <Message Text="Configure test runner to make the entire build fails if a single test fails" />
    <PropertyGroup>
        <StopOnTestFailure>True</StopOnTestFailure>
        <TreatTestFailureAsBuildFailure>True</TreatTestFailureAsBuildFailure>
    </PropertyGroup>
</Target> 

As you can see, overriding a target is done simply redefining it in the build file. With this simple addition, the msbuild engine will call your BeforeTestConfiguration task that sets the StopOnTestFailure property to True and the TreatTestFailureAsBuildFailure also to true, thus asking to msbuild to make the entire build fails when the test task fails. Now if I fire a build with a failing test I see.

As you can see the entire build is considered to be failed, because we override the BeforeTestConfiguration. If you click on the Log file you can find the Message that confirms that your override task was really called.

Building with tools version "3.5".
Target "CanonicalizePaths" skipped. Previously built successfully.
Target "BeforeTestConfiguration" in file "C:\Users\tfsservice\AppData\Local\Temp\FluentMsTest\BuildWithTests\BuildType\TFSBuild.proj" from project "C:\Users\tfsservice\AppData\Local\Temp\FluentMsTest\BuildWithTests\BuildType\TFSBuild.proj":
Task "Message"
  Configure test runner to make the entire build fails if a single test fails
Done executing task "Message".

Configuring Tfs build is quite simple, it is only a matter of overriding certain specific targets to insert custom msbuild code in the main build process.

Alk.

Tags: Team Foundation Server

Specifying test to run in a Tfs build is really simple, since you have a specific step of the wizard dedicated to this operation.

During the definition of the build, I can simply ask to automatically detect tests in all assembly that matches the pattern *.Test.dll. Now if you schedule a build you should see results of test run.  Here the first result of my test build.

Now we have a problem, because the task “running tests” is failed, but we also check that we have a “No test result”, something went wrong and weare in trouble :). The most important thing to avoid when you work with Tfs is having panic you will always get detailed error of a failing build. To look at these details, you need to look in the file BuildLog.txt located in the shared folder used by the build (in my situation is \\10.0.0.200\Builds\FluentMsTest\BuildWithTests_20090625.3). Since the failure is due to a Test failure, I need to look for string testtoolstask (the task related to running test) to find problem related to unit testing.

Here is what I found

SBUILD : warning : Visual Studio Team System for Software Testers or Visual Studio Team System for Software Developers is required to run tests as part of a Team Build.
  The previous error was converted to a warning because the task was called with ContinueOnError=true.
  Build continuing because "ContinueOnError" on the task "TestToolsTask" is set to "true".

Ok, it seems that I forgot to install Visual Studio Team System for Software Tester in the Tfs test machine (I’m doing posts using a test virtual machine installed ex novo). Visual studio is needed, because the build is done with msbuild tool, and if visual studio is not present the build machine does not have the correct tasks to use with msbuild. Another interesting stuff is that this error states that build can continue, because the ContinueOnError of “TestToolsTask” is set to true. This is the default setting, when something goes wrong with running tests, the build does not fail, and we have a  “partially succeeded” build.

Now when you install Visual Studio Team System for Software Testers on the build machine, you will finally see test result.

The interesting stuff here is that if you click on the hilighted link, you can download test results on your machine. The file you download can be opened directly from visual studio, and you can examine results as if you executed them locally.

Once again we can see that setting Tfs to run unit tests during the build is a simple process, and can be simply done with the wizard. In the next post of the series I’ll deal on more advanced configuration for test run during a build.

alk.

Tags: Team foundation Server Build

If you deselect this option the import wizard does not check the actual file size of the database that is being imported.

alk.

Tags: Visual Studio Database Edition

Now with VS DB edition I write test more easily using the data generation plan, but usually I run both the old tests and the new ones, and I noticed that the deploy of my DB project is incredibly slow when the db does not exists.

when I look at the database folder I saw that my test database is 40 GB big, I was astonished by this fact, since it is deleted at each test run and have small preload set. This problem was caused by the fact that VS DB edition imported the whole structure of the DB as well as file dimension. If you go to “Schema Objects”, then “Storage” then “Files” you can find several .sql files that contain definitions of the physical files of the database. Examining the files I found entries like this.

ALTER DATABASE [$(DatabaseName)]
    ADD FILE (NAME = [MyFileName], FILENAME = '$(DefaultDataPath)$(DatabaseName).mdf', SIZE = 16000000 KB, MAXSIZE = UNLIMITED, FILEGROWTH = 64000 KB) TO FILEGROUP [PRIMARY];

This means that each time VS DB edition does a deploy this file is 16 gigabytes big, and the operating system take times to find space on disks. The solution was changing this value to a reasonable value (10 megabytes for each file). Now deploy operation is considerably faster.

alk.

Tags: Visual Studio Database Edition

Sometimes it happens that someone calls me to solve a problem in code, and quite often i say “Write a unit test that reproduce the problem, then begin to investigate”. Quite often one I got a reply of

I’m in a hurry, I do not have time to write test, I must correct that bug as soon as possible

I’m aware that writing a test consume time, but quite often in the end a test save you time. After that people told me that he do not have time, he shows me the problem, then fire a big web site from a .NET solution with 60 projects, we have to wait about 40 seconds for the app to start, then browse to a web page, enter data, and verify if the answer is correct.

The problem is that we are testing a function that use some stored procedures and some .net code, so if the problem is still there, we need to return to Management Studio, change the procedure or change the code. If we must change the code, we need to recompile, fire the app again (and wait for the app to reinitialize) bla bla bla.

For big application the startup time of the whole application is just too big to think that F5 approach is a feasible solution to verify if a bug is corrected. One day, for a very difficult bug, after a couple of hours I realized that more than half of the time was spent waiting the app to start and insert data to exercise the function……. too bad. This means that we are developing slowly, and moreover it is frustrating to: “do a modification and wait for one minute to see if it corrects the problem”

Working this way lead to this pattern

1. Write a modification
2. Fire the program with F5
3. Goes to facebook, twitter, waiting for the app to start
4. The app started but you are reading some blog

This is incredible inefficient way to develop software.

My suggestion is, whenever you need to correct bug, or verify some feature of a low level layer of your application, do not exercise it from the User Interface but write a test. It will save you time in the long run. Most of the problem is that writing test is not so simple, expecially in complex product, but when you are experienced in unit testing, writing a test is really a breeze. Most of the time you will end with a test infrastructure specific for your project, where creating a new test is a matter of seconds.

Alk.

Team Foundation Server addresses this issue separating build machines from Team Foundation Server. If you find that the build machine is becoming slow, you can simply use another machine in the network to execute some of the builds. In real environment you can even avoid to install build engine in the Tfs machine, delegating builds to other servers. First of all go into another machine, fire the installer of Team Foundation Server, and choose to install a “Team Foundation Build”.

Note: if you do not have Active Directory you Must create in this machine an account with the same name/password of an account of the machine where Team Foundation Server is running and that account must have right to access tfs. This is a standar requirement, the Build Server needs to access Team Foundation Server to get sources to build and store build result, etc, etc. so it is of vital importance that it runs with sufficient privilege to access TFS server. If you have active directory you do not need to create a specific account, but you must select one AD user that have credential to access TFS during the installation.

When the installer finished, you can go to Visual Studio, Right Click on the Builds in the Team Foundation Server Explorer Tree, then choose “Manage Build Agents”

Now you can add additional Build Agent, and you can choose the new machine name, where you had previously installed the Team Foundation Build.

Now you can use this new machine to run Builds simply specifying this agent when you create a Build Definition, or you can use it when you manually queue new build. It is really important that the user credentials used to run the Build Service in the new Build machine has the right to access the Team Foundation Server Machine as stated before.

If you had already installed the team Build engine with a wrong user don’t panic, you can still manually change build machine settings to correctly access TFS. Go to Service control panel and change the credential used by the “Visual Studio Team Foundation Build” service to use an user that has sufficient credential for TFS. Now restart the service.

It is highly possible that when you restart the service you get “access is Denied” error, it means that the new account does not have sufficient rights to run, first of all the user needs rights to access the folder where you install the Build Agent, but this is usually not enough. The only way to understand exactly what is wrong is going into the Event Viewer where you can find detailed errors like this one.

Detailed Message: TF224004: The Visual Studio Team Foundation Build service failed to start because WS2008V1\alkampfer does not have the required access permissions for address http://ws2008v1:9191/Build/v2.0/AgentService.asmx.
Exception Message: HTTP could not register URL http://+:9191/Build/v2.0/AgentService.asmx/. Your process does not have access rights to this namespace (see http://go.microsoft.com/fwlink/?LinkId=70353 for details). (type AddressAccessDeniedException)

This message is one of the most dreadful, it means that the user has no rights to use that url to host  a web service. The solution is to run this couple of commands.

netsh http delete urlacl url=http://+:9191/Build/v2.0/AgentService.asmx
netsh http add urlacl url=http://+:9191/Build/v2.0/AgentService.asmx user=alkampfer

With these two lines of code you are giving to user Alkampfer (the one that has right to access Tfs and the one used to run build service) enough rights to create a web service in that url. Now you should be able to start the service. Verify if the Build Agent is running browsing the WebService page (something like http://10.0.0.11:9191/Build/v2.0/AgentService.asmx). Now if everything is ok, you should go to “Manage Build Agents” menu and reenable agent if necessary. This step is needed because Tfs disables all Agents that are unable to run. If you fire a build with an agent that has no right to access TFS the TFS disables that agent because it is useless. When you correct the problem, you can simply edit the Build Agent definition, and change its status back to active, now you can schedule again builds on it.

With such a configuration you can divide build tasks across multiple machines, and the whole build process is more scalable. The advantage is that you have a single point of access (the Tfs machine), and from that point you can subdivide works between machines. Since you can install how many build server you want, the whole system is highly scalable.

Alk.

Tags: Team Foundation Server

Since building a Dto is a mechanical process it is better to use a code generation tool to generate them with little effort. If you work with visual studio 2008 the best choice in Code Generator is T4, because it is included in Visual Studio and is really good. I do not want to cover T4 syntax, because you can find tons of information following this post links.

You can download here a little project that contains a simple DtoFActory.tt file used to generate basic Dto out from a domain object. It is written with little effort, and it can contain errors, but I’m quite satisfied by the general result. To use it simply add the DtoFactory.tt file to your solution, then you can use in other tt file. In the example I simply generate a EntityFramework edmx against northwind database, then I create a CustomerDto.tt file with this content.

<#@ template language="C#" hostspecific="True" debug="True" #>
<#@ output extension="cs" #>
<#@ include file="DtoFactory.tt" #>
<#
// <copyright file="CustomerDto.tt" company="Gian Maria Ricci">
//  Copyright © Gian Maria Ricci. All Rights Reserved.
// </copyright>
    GenerateDto(
        @"bin\Debug\DtoFactory.Exe",
        "DtoFactory.Customers",
        "CustomersDto",
        "DtoFactory",
        "http://www.nablasoft.com/dotnet/",
        1,
        new PropertyData[] {"CustomerId", "CompanyName", "ContactName"},
        true, true, true); 

#>

As you can see I import the DtoFactory.tt file, then call the GenerateDto function. The first parameter is the relative path of the assembly that contains the original object. Relative means respect to the tt file itself, this is one of the part I do not like very much because you need to find out relative path and refers to the debug directory, but it can be acceptable.

Then I specify the fullname of the class, the name of the Dto object, the name of the namespace were to put generated class. Then I need to specify the namespace used in the DataContract attribute (I want my Dto to be transferrable by WCF), then the number of base classes to examine to find properties, a list of PropertyData object with names of the property of the original object I want in my dto, finally three boolean values to specify if I need my dto to implement INotifyPropertyChanged, INotifyPropertyChanged, and IEditableObject.

You can see that this simple template will generate a CustomerDto.cs file that contains a CustomersDto class that contains that three properties. The interesting part is how I generate the assembler of the Dto. To build a Dto the standard way to proceed is to take the original object and copy all needed properties into corresponding properties of the dto object. This is a suboptimal approach, because using such a technique, you are forced to load the entire object from the database. In My situation CustomersDto has only three properties, why I need to load an entire Customers object into memory to build a CustomersDto object that contains only three properties? The solution is in how my dtoFactory generates the assembler.

public static Expression<Func<Customers, CustomersDto>> ExpressionSelector;
public static Func<Customers, CustomersDto> Selector;

static CustomersDtoAssembler()
{

    ExpressionSelector = obj => new CustomersDto()
    {
        CompanyName = obj.CompanyName,
        ContactName = obj.ContactName,
        CustomerID = obj.CustomerID,
    };
    Selector = obj => new CustomersDto()
    {
        CompanyName = obj.CompanyName,
        ContactName = obj.ContactName,
        CustomerID = obj.CustomerID,
    };
}

As you can see my generator generates an Expression<Func<Customers, CustomersDto>>, and simply initialize it in static constructor with a simple lambda expression. If you are wondering why I need such a strange object let’s see how I use it

using (NorthwindEntities context = new NorthwindEntities())
{
    var Query = context.Customers.
       Where(c => c.CustomerID.StartsWith("A"))
       .Select(CustomersDtoAssembler.ExpressionSelector);

    foreach (CustomersDto customerDto in Query)
    {
        Console.WriteLine("ID: {0}, ContactName: {1}, CompanyName {2}",
            customerDto.CustomerID, customerDto.ContactName, customerDto.CompanyName);
    }
}

Thanks to LINQ to Entities, I can simply build a query to the NorthwindEntites data context, selecting Customers, applying a filter and specify the assembler ExpressionSelector seen before. This produces this query against my database.

SELECT
1 AS [C1],
[Extent1].[CompanyName] AS [CompanyName],
[Extent1].[ContactName] AS [ContactName],
[Extent1].[CustomerID] AS [CustomerID]
FROM [dbo].[Customers] AS [Extent1]
WHERE (CAST(CHARINDEX(N'A', [Extent1].[CustomerID]) AS int)) = 1

Thanks to the Expression<Func<Customers, CustomerDto>>, Entity Framework knows how to retrieve only the data he need to create the Dto, wonderful :D. With such a powerful generator I can create Dto objects with few lines of code.

Since this generator is not still fully tested feel free to send me suggestions, bugs, feature you wish to have, etc, etc.

Alk.

Code Download.

Tags: Dto

As Oleg states here, if you need to analyze assemblies during code generation, you should use fxcop introspection vs standard reflection. Using introspection is really simple, first of all you need to locate a couple of dll in the fxcop installation folder, and references them in the solution.

Microsoft.Cci.dll contains a lot of classes to use Introspection. The main problem is that there is not great help on how to use these classes, but it can be quite intuitive.

AssemblyNode modelAssembly = AssemblyNode.GetAssembly(@"KilogWms.Model.dll");
Identifier namespaceIdentifier = Identifier.For("KilogWms.Model");
Identifier typeIdentifier = Identifier.For("Location");
TypeNode locationNode = modelAssembly.GetType(namespaceIdentifier, typeIdentifier);
Assert.That(locationNode.BaseType.Name.Name, Is.EqualTo("LocationBase"));

With this code I load an assembly specifying dll name, then create a couple of identifier, the namespace and the name of a class, finally with the GetType() method of the AssemblyNode I get a TypeNode that contains all information on the Type. The good part of introspection engine is that the assembly is not really loaded into the appdomain of the caller. You can verify with with a little test, start code and load an AssemblyNode, then try to delete the assembly from disk, you cannot do this because the file is in use, now call Dispose() on AssemblyNode object, and try to delete the assembly again, now you can delete it with no problem. This happens because Introspection does not really load the assembly file into the process, it simply inspect it, so when you dispose the AssemblyNode the file is free again.

Looking at the above code you can see that using Introspection is really intuitive, with intellisense is really simple to figure out how to find information about classes.

Alk.

Tags: FxCop Introspection

This command will open a menu that shows all objects that references this specific object, as well as all database objects that are referenced by this object. This function is of invaluable use when you need to change or modify something, and gives you an invaluable sight on dependency between database structure.

In this specific situation I removed some old features from the code, and I want also to remove all objects in database that are related to those features. Since I know that these feature are related to three tables, I can simply check all dependencies of those tables, that and verify that I’m not breaking anything. When I’m sure I can remove tables files from the solution.

After table files are deleted, I return in the Schema view. Now I can open the “Orphaned Objects” to view all object that are orphan.

This is an invaluable help because you immediately view all objects that are now orphan because I deleted some tables. As you can see in “orphaned Objects” you can view only keys, constraints, etc, and you do not see stored or view. If you watch in schema view to stored procedure section you can find that each stored that references one of those removed table is marked as “warning”

If I build the project I obtain a lot of errors, because I have stored and views that reference missing tables.

I simply check again each stored and view, some of them are to be deleted, and some other needs only modifications. Thanks to database edition I know when I have finished work because until I have a single object that references missing object the project will not compile. Finally I fire my database tests to verify that everything is still working.

If something went wrong (because at a certain point you find that you should not removed those tabled because they are still used somewhere), you can simply undo everything and starts again the process. The great advantage working with Database Edition is that you works only with  local files, and you can commit changes to databases only when you are really sure that everything is ok. Now database can be managed like other code, you modify it, you test it, and until you does not check in, nothing gets really changed.

alk.

Tags: Visual Studio Database Edition

The main disadvantage of using such a configuration is the need to make each tool communicate with others, the good part is that these tools are open source. Team Foundation Server on the other side is a Commercial tool, so you have to pay it, but it gives you a lot of features in a single unified tool, and this is Great. Let’s see as an example how to set up a continuous integration server for a simple project.

Leveraging the power of a Continuos Integration machine with open source tool is not difficult, but you need to do some work, in CC.net you have to create a NANT or MSbuild script, configure cc.net with ccnet.config and so on. The good part of Team Foundation Server is that creating a build is a matter of a right click on the builds node

The “New Build Definition” shows you a wizard that helps you in creating the build, the first time you run it tells you that it does not find any build file in the source code, so it suggest you to use a wizard to create one.

The wizard is really simple, first of all it asks you the list of solutions to build.

In this test project I have only one solution so I select it. Then you select the configuration (Debug/Release/Etc). You can forget any option for now. When you click finish you will return to the original wizard, now you can decide how many result you want to mantain

This is a great option, because it permits you to avoid keeping too many data, as example for failing builds. The next step asks you the build agent to use, since it is my first build, I create an agent for the first time

I’ve installed the Build Engine on the same machine where the Tfs runs, so I create an agent named StandardBuildAgent, then specify the name of the computer TFSALKAMPFER , when you press OK you can use that agent to run the build, now you must specify network share where to put the result

In the last step you simply decide when to trigger a build, and since I want really continuos integration I specify a build at each check-in-

Now the build is created, you can right click and schedule a new build (this is the equivalent to Force a build in CC.net). The result gives you a lot of informations on what is happened.

This build have some errors, if you scroll down the result of the build you can find detailed error, stored in file Debug.txt. This file contains the error output of the msBuild engine so you can understand what goes wrong. The good news is that, if the build goes ok, you can find a lot of information in the shared folder you specified during build definition.

And if you click into the Debug folder you can find all the artifacts produced by the build.

Respect to CC.Ner, Team Foundation Server is really simpler, with few clicks you can create a Build Definition that is triggered at each check-in, gives you a lot of detailed information, and copies all artifacts in a network share. I really care about Continuous Integration Machine, and the ability to directly manage it inside Visual Studio with few clicks is really a great feature respect CC.net.

Alk.

Tags: Team Foundation Server

While this is really good to create a set of unit test that are repeatable, it can slow up testing process during development. Let’s make an example, I need to made a change in a stored procedure, so I write some tests to verify the new features, verify that they fail, and then start to work.

The process usually is

1. Modify the stored in the Database Project
2. Run the new tests
3. If new tests are all green rerun the whole test suite to verify that you have not broke something else.

If the modification is really complex you need to modify/run test more than one time, but since you are always running the same little test suite, you really does not need to redeploy database structure and run data generation plan at each iteration. Deploying database and running data generation plan at each test run can be time consuming.

I’m used to setup database test with transaction management to automatically rollback a transaction after each test, you can use extensions project or you can code it by yourself, because is a simple matter of using a TransactionScope object. In the end you will end with tests that do not alter the database. In such a situation I prefer to proceed this way: first of all I configure my database project to deploy to the test database, simply open property of the database project and goes into Deploy tab

Then I go into the test project, goes into Database Folder, locate the DatabaseSetup.cs and comment out the two lines that actually does structure deploy and data generation

[AssemblyInitialize()]
public static void IntializeAssembly(TestContext ctx)
{
    //Setup the test database based on setting in the
    //configuration file
    //DatabaseTestClass.TestService.DeployDatabaseProject();
    //DatabaseTestClass.TestService.GenerateData();
}

Then I open the DataGenerationPlan I want to use and with the menu Data->DataGeneration Plan->Execute plan, I execute my plan against the test database.

Now I’m sure that

1. the database is prepared to be tested
2. No test will alter the content of the database (thanks to transaction maangement)

At this point I setup the build configuration to make a deploy of the DAtabase Project at each build, everything is now prepared for work.

I open the .sql file that contains the stored I want to modify, write some modifications, and then Build+Run Tests. With such a configuration I’m sure that at each build, my modification are propagated to test database, data are already there because I preloaded previously, and each test now run very fast. With this configuration I can really speed up development time avoiding unnecessary preload and structure syncronization.

When all new test passes I run the whole test suite, then remove the comment from the InitializeAssembly and do a check-in.

alk.

Tags: msTest

First of all goes into the machine where the CC.Net runs, prepare the IIS directory where the clickonce generated setup will be copied, open the solution into visual studio, goes to the project you want to publish and set everything needed for clickonce, set version to 1.0.0.0 and publish through visual studio IDE. This will create the folder structure in IIS and the application is published for the first time with the version number 1.0.0.0. Now take the publish.htm file and copy into the trunk of your project into a specific directory (I used Configuration\ClickOnce), rename it in OriginalPublish.htm, edit it and find the place where visual studio ide wrote the string 1.0.0.0, replace it with the string VERSIONNUMBERTOKEN.

This part is needed because msbuild is able to publish a clickonce enabled application, but it will not create the publish.htm file. Now create a target in nant, and inside it execute MsBuild against your project

<exec program="${MSBuild}"
        commandline="${ProjectDir}\Src\MyApp\MyApp.csproj"
      basedir=".">

    <arg value="/target:publish" />
    <arg value="/p:Configuration=${BuildConfiguration}" />
    <arg value="/property:ApplicationVersion=${assembly.file.version}" />
    <arg value="/property:PublishUrl=http://10.8.0.5:10444/MyApp/" />
    <arg value="/Property:InstallUrl=http://10.8.0.5:10444/MyApp/" />
</exec>

As you can see I simply delegate to msbuild the task of publishing the application, specifiying the url used to distribute it. This task will create all needed click once files in the bin\debug\app.publish subdirectory of the project. Now you need to create the publis.htm file

<property name="clickOnePublishDir" value="${ProjectDir}\Src\MyApp\bin\${BuildConfiguration}\app.publish" />
<ReplaceTokenInFile
    sourcefile="${ConfigurationDir}\ClickOnce\OriginalPublish.htm"
    destfile="${clickOnePublishDir}\publish.htm"
    token="VERSIONNUMBERTOKEN"
    replacetoken="${assembly.file.version}" />

This task takes the original publish.htm file and replace the token VERSIONNUMBERTOKEN, with the actual version number. The ReplaceTokenInFile task is c# code in nant script

    <script language="C#" prefix="FileOp" >

        <references>
            <include name="System.Dll"/>
        </references>
        <imports>
            <import namespace="NAnt.Core.Types"/>
        </imports>

        <code>
            <![CDATA[

          [TaskName("ReplaceTokenInFile")]
                public class ReplaceTokenInFile : Task {
                  #region Private Instance Fields

                  private string _sourceFile;
                        private string _destFile;
                        private string _token;
                        private string _replaceToken;
                  #endregion Private Instance Fields

                  #region Public Instance Properties

                  [TaskAttribute("sourcefile", Required=true)]
                  public string SourceFile {
                      get { return _sourceFile; }
                      set { _sourceFile = value; }
                  }

                  [TaskAttribute("destfile", Required=true)]
                  public string DestFile {
                      get { return _destFile; }
                      set { _destFile = value; }
                  }

                        [TaskAttribute("token", Required=true)]
                  public string Token {
                      get { return _token; }
                      set { _token = value; }
                  }

                        [TaskAttribute("replacetoken", Required=true)]
                  public string ReplaceToken {
                      get { return _replaceToken; }
                      set { _replaceToken = value; }
                  }

                  #endregion Public Instance Properties

                  #region Override implementation of Task

                  protected override void ExecuteTask() {
                      String source = System.IO.File.ReadAllText(SourceFile);
                            source = source.Replace(Token, ReplaceToken);
                            System.IO.File.WriteAllText(DestFile, source);
                            String message = "Replace token" + Token + " in file " + SourceFile + " with token " + ReplaceToken + " and saved into " + DestFile;
                             Log(Level.Info, message);
                  }
                  #endregion Override implementation of Task
                }

       ]]>
        </code>
    </script>

Now you can simply copy all the files into the folder where IIS maps the publishUrl and the game is done.

<copy todir="${ClickOnceDir}">
    <fileset basedir="${ProjectDir}\Src\MyApp\bin\${BuildConfiguration}\app.publish">
        <include name="**" />
    </fileset>
</copy>  

The advantage of this approach, is that I can reuse the automatic version numbering I use in my CC.Net machine,  and at each checkin I have an automatically publish of new version of the application that my testers can use. Clearly for a software in production this action must be run manually only with stable and tested builds.

If you do not like deploying at each checkin you can simply schedule this action to run periodically, but the important stuff is that now everything is automatic and you can update clickonce application with a single click.

alk.

Tags: Continuos Integration Click Once

Now I need to add another feature, I need to open an excel document with formulas, fill some cells, leaving formulas intact. My first version does not work as expected, I simply created an excel with simple formulas, then fire my function and when I open the resulting excel I see all zero on formula column, but the formula is there, and if I change some cell referenced by the formula I’ll obtain the right value.

This problem arise because formula are stored in original sheet with such a xml

<c r="C2">
  <f t="shared" ref="C2:C10" si="0">A2+B2</f>
  <v>0</v>
</c>

This means that the cell C2 contains the formula A2+B2, but the node <v>0</v> tells Excel that actual value is Zero. So when you open the resulting excel file, excel found that content of the cell is 0 and shows this value until some related cell changes content. To solve this problem I simply added a bit of code that removes the <v> element.

originalElement.Descendants(ExcelFiller.ns_s + "v").Remove();
row.Add(originalElement);

Now when excel opens elaborated document, it found no <v> (value) element, so it recalculate it based on formula.

alk.

Tags: OpenXml

Those exceptions I received this morning told me that my unit tests for that class are not so good, even if they cover the 100% of the class code. Before actually trying to review the code to find why it can give an IndexOutOfBoundException I fired Pex on that class.

It seems that there are really a lot of paths that makes my routine to fail. If you look at the Text column you see that some of the input strings that makes my routine fails are composed by string termination (\0) inside the string, or are null etc etc. Pex can find those specific input values because it analyzes the code of the Class under test, for this reason it is a tool of “White Box” testing. Another great feature of Pex is the “Suggestions Windows”, since Pex found errors because he really analyze written code he can also give you suggestion on how to avoid a specific error.

SElecting the specific error, you can find a suggestion on how to avoid it, in this situation it seems that I completely forgot to test the length of the text to be analyzed. My next step is to add validation of text lenght to avoid those errors.

A real cool feature is the possibility to save one or more failing Pex test as a standard unit test.

This will create one test for each one of the selected tests, and you can run those test as standard ones.

After you have modified your class you can run pex again to check if some other error remains. Running Pex analysis took time, Pex have a time limit to avoid running forever, but you can click on appropriate icon to change default values (this needs the creation of a test project that pex will use to store information)

Once the test project is created you can find a test file named in the same way of the class under test with the suffix Test, it is composed of some test methods decorated with Pex specific attributes.

[PexMethod(MaxConstraintSolverTime = 2)]
        public MultiRange FindHit(
             [PexAssumeUnderTest]WildCardBeginEndKeywordFinder target,
             string keyword,
             string text
        )
        {
            // TODO: add assertions to method WildCardBeginEndKeywordFinderTest.FindHit(WildCardBeginEndKeywordFinder, String, String)
            MultiRange result = target.FindHit(keyword, text);
            return result;
        }

As you can see I changed the MaxConstraintSolverTime to 2 so Pex can spend more time to analyze my function, now I can simply right click Pex generated class, and select “Run Pex exploration” to explore again the method with this new setting. Now Pex can do a more deep analysis (this does not means that will always find more errors).

Pex is really good to find input pattern that makes your code fail:

I’m sure I never think to pass 1****\0 to my routine :D. After some correction you usually find that inputs that makes your code fail are really strange, and you can tell “Ok, it is enough”, if someone passes “1*****\0” he should really got an exception as result, but in the end Pex is exceptional in finding obvious failing path in your code.

alk.

Tags: Pex Testing

Following the link “For More information .. “ you can read detailed log, in the end of the file there is an exception

To avoid being lost in error simply look at the last operation that it tried to do, in this case it was “Creating reports on the Sql Server reporting" Services” and the cause of the exception is “The permission granted to user “TFSALKAMPFER\Alkampfer are insufficient”. To solve this problem install on the server the Team Foundation Server Administration Tool

Then you can launch that tool with administrative privileges, connect to the server and verify permissions. Here are permissions on my newly installed server.

Uh Oh, it seems that my account does not have any rights on reporting services, and limited access to share point, now you can simply click on Reporting Services area.

Since this is a test server and I do not care a lot about security I give to myself full rights, then save changes. Now I’m able to create team projects.

When you work with a complex system like TFS do not forget to analyze detailed log files in case of errors, it can save you a lot of time.

alk.

Tags: Team Foundation Server

Developers usually end with one or more of those thought

1. Analyst is a moron, he did a bad job not finding the real requirement of the customer
2. Customer are stupid because they does not know what they want, they are constantly changing requirements.
3. We lack requirement gathering phase, now we must loose time following desires of Analyst/Customer

Finally the programmer complains a lot about too much code to change, and how is painful working in an environment where noone is able to catch the real requirement of the customer.

Situation is much more different if you understand that the needs of the customer quite often change with time, we are in a fast world where moving slowly quite often leads to death, so it is absolutely normal that requirements changes with time. The only solution is Plan in advance for change, if you are ready for changes, you will having really less problem when some requirement changes over time.

Technically speaking, to have a software that can react quickly to change the key concept is loose coupling, through heavy use of Inversion of Control, you can minimize dependencies between the various part of the software. The more the software is loosely copuled, the more you can modify a part without affecting other parts.

Then you need a lot of tests, that can spot immediately when a change in some part of the software has introduced some bug. Then You need to setup a Continuous Integration machine, and make your tests run at each check-in, this assures you that tests are run frequently and automatically. Then you must setup a real good logging infrastructure, to trace immediately what is going wrong with the system. Finally you should work in asynchronous mode when possible, minimizing the coupling between different tiers in the software.

But probably the most difficult step is changing the way we manage software developement. We must change the way we approach the software, not fearing changes and reacting differently to change in requirements.

alk.

Tags: Software Architecture

I followed all the instruction, but when team foundation server installer asked me the address of sharepoint site I tried some url but I got the infamous error

the specified windows sharepoint services site url is not the default site collection site

The missing part in the guide is that after step 19 you need to create a site collection. First of all verify that you are able to browse the root site you have created in step 18

Ok now verify that you can browse the site collection at site http://tfsalkampfer/sites/base/default.aspx where tfsalkampfer is the name of the machine, probably you get a 404 error, so goes to the administration pane of sharepoint services

You need to click “application management” and then “Create site collection”. Now you should see the page to create another site collection

Please be sure that you are working on the Web application you have created for Tfs, then simply use the /sites/ option in the combo and specify whathever name you want for the web site address, i’ve chosen “Base”. Now press ok and verify that the site collection is ok browsing http://tfsalkampfer/sites/base/default.aspx, now you can set the value http://tfsalkampfer/sites/ as share point site during Tfs installation, and now everything went ok.

alk.

Tags: Team Foundation Server

To avoid poor performance I eager load all valid ids in memory (each id is an In32 (4 bytes) and I have 100k of them, so I’m keeping in memory 100k * 4 bytes 400kbytes of RAM). Now I load all these id in a List<Int32>, then iterate into all files of a directory, for each file parse the name and verify that the List contains the id. To analyze 200k files the whole routine took 10.200 milliseconds. It is quite speedy, but if you profile it you can verify that most of the time was spent into checking that an id is contained in the list.

This is because search operations in List<T> are slow, a search is basically a scan of the whole list until you find the element or you find the end of the list. We are so used with List<T> that I saw a lot of developers completely forget the old Array class. Let’s see how we can speed up the lookup with the old Array class.

//Load all id in ascending order into ValidLink List<Int32>
Int32[] links;
links = ValidLink.ToArray();

I’ve simply change the query to the database, asking for id in ascending order, then I populate the list as usual, and finally creates a simple Array of Int32 with the function ToArray(). now you can use BinarySearch

if (Array.BinarySearch<Int32>(links, id) > 0)

A lot of people forget that the Array class natively supports Binary search for ordered array, when I ran the sample again execution time was dropped to 2.392 seconds, and now most of the time is spent reading Disk structure.

Pay attention to the data structure you use

Alk.

Amazing :D, WindowsDeath answered me, I’m impressed hahahahaha

alk.

[TestMethod]
public void BaseAssertFluentWithSyntaxHelperLtOrGtRo()
{
    Int32 obj = 400;
    Assert.IsTrue(obj < 100 || obj > 200);
}

It is quite clear, but with fluent syntax you can write test like this

[TestMethod]
public void BaseAssertFluentWithSyntaxHelperLtOrGtR()
{
    Int32 obj = 400;
    FluentAssert.That(obj, Is.LesserThan(100).Or.GreaterThan(200));
}

I find it really clearer than the first form. The advantage of a fluent syntax is that you can read the assertion as if were English text. I’m actually writing an helper assembly to support fluent syntax with msTest, if someone is interested in I can publish in source form. For now I’m able to support basic syntax with operator precedence and parenthesis

[TestMethod]
public void TestBasic2()
{
    IConstraint c = Is.LesserThan(100) | Is.GreaterThan(200) & Is.GreaterThan(300);
    Assert.IsTrue(c.Validate(99));
}

[TestMethod]
public void TestBasicParenthesis()
{
    IConstraint c = (Is.LesserThan(100) | Is.GreaterThan(200)) & Is.GreaterThan(300);
    Assert.IsFalse(c.Validate(99));
}

As you can see the first test pass because the And operator ( expressed by & because you cannot overload &&) has higher precedence over the or operator. The second test pass because you can alter precedence with parenthesis.

Alk.

In such a situation the fixture of the test is using different source files to test the class with different input. In a classic NUNIT unit testing you can solve this problem including the file in the project with “Copy if newer”. With such a setting Visual Studio at each build check if the file is changed, and eventually copies the file into the appropriate directory (bin/debug/pathofthefile in standard debug configuration)

This approach is good, but has some problems, if you simply change the XML file, and rebuild the project, the file gets no copied in the output directory . This happens because Visual Studio correctly checks that no source file was changed, so it has no need to recompile. Another stuff I do not like is that I need to find the file with the path I used to insert it into the test project.

[Test]
public void GrabMenuWithActionUrl()
{
 MasterLogic sut = new MasterLogic(new MyTestUrlHelper());
 List<MenuItem> menu = sut.CreateMenu("SampleFiles\\MenuType1.Xml").MenuItems;
 Assert.That(menu, Has.Count.EqualTo(2));
 Assert.That(menu[1].MenuItems[0], Has.Property("Url").EqualTo("/Photo/ManageAlbum"));
}

The same test can be written in msTest, but this time I have the DeploymentItem attribute, that can be used to specfy to mstest engine, that my test really need the file “SampleFiles\MenuType1.xml”.

[TestMethod()]
[DeploymentItem(@".\SampleFiles\BaseMenu1.xml")]
public void CreateMenuTest()
{
    PrivateObject param0 = new PrivateObject(new MasterLogic(new MyTestUrlHelper()));
    MasterLogic_Accessor target = new MasterLogic_Accessor(param0);
    string menuFileName = @"BaseMenu1.xml";
    MenuItem expected = new MenuItem("TEST");
    target.CreateMenu(menuFileName);
    Assert.AreEqual(expected.Text, "TEST");
}

This is a cleaner way to express the fixture, thanks to the DeploymentItem attribute it is clear that the test needs the file SampleFiles\BaseMenu.xml, and most important, the file was copied automatically before each test runs and it is located in the same dir of the test assembly, so i can simply refer to it with the BaseMenu1.xml name without any path.

alk.

1) it accepts a string parameter let it call P
2) Manipulate that parameter P, transforming in P1 with an algorithm that is contained in a specific class
3) uses P1 it to ask for database instance of some classes and does a lot of manipulations and database operations

Now I want only to test that the class calls the algorithm to transform P in P1. All other functions of that sut are already verified, I first begun with a skeleton test I used previously to test the sut, this skeleton makes heavy use of Mocks and stubs objects to FULLY simulate the database.

In the end I have a test that works, but it was too complex, mainly because it is full of stub expectation. Moreover it is Fragile, because if I’ll change the way how this test interact with the DB, I’ll need also to change expectations. So I changed the test in this way.

[Test]
public void VerifyThatBlaBla()
{
    IParamTransformer pt = MockRepository.GenerateStub<IParamTransformer>();
    try
    {
        using (IoC.Override(typeof(IParamTransformer), pt))
        {
            MySut sut = new MySut();
            sut.PerformStuff("MyParameter");
        }
    }
    catch (System.Data.Common.DbException)
    {
        //Ignore everything.
    }
    pd.AssertWasCalled(obj => obj.TransformParam(Arg<String>.Is.Equal("MyParameter"));

}

MySut is the class I need to test, it internally calls IoC.Resolve<IParamTransformer>(), to take a parameter transformer. Thanks to AAA syntax of Rhino Mocks I simply create a mock, use an internal override function to instruct the IoC component to return my mock when asked to resolve a IParamTransformer, then invoke sut method. Since I know that the test calls the DAL with transformed parameter I simply ask in the test to catch every DbException, ignore it and then assert that the TransformParam is called. The test is quick because I used SqlLite to have a Db in memory, the sut resolve the IParamTransformer, calls for parameter transformation, then calls the db; the db call failed because the test db is empty so a DbException is thrown. Now the exception is ignored and I finally verify that function TransformParam was called.

This test is less than half length respect to the first version that fully mimics the DAL with mock objects. I think that it is really clearer, because it is focused on “what I want to test” without distracting the reader with greedy details.

A still better solution could be a refactoring of the original class, making it more testable, but I think that this is enough.

Alk.

[AssemblyInitialize()]
public static void IntializeAssembly(TestContext ctx)
{
    //Setup the test database based on setting in the configuration file
    DatabaseTestClass.TestService.DeployDatabaseProject();
    DatabaseTestClass.TestService.GenerateData();
}

This is run first of every code in the test project, but I really have the need to run different data generation plan in different part of my code. If you look at the DatabaseTestClass with reflector you can be really surprised, it has a TestService class that is of type DatabaseTestService.

As you can see it has a GenerateData method that permits you to specify dgen file name, but it is protected. I really wonder why, but it seems that you cannot run database generation file programmatically…or not?

Here is the solution

public static void RunDataGenerationPlan(
    String path,
    String projectFile,
    String dataGenerationFile,
    bool clearDatabase, string providerInvariantName, string connectionString)
{
    FileInfo project = new FileInfo(Path.Combine(path, projectFile));
    FileInfo generation = new FileInfo(Path.Combine(path, dataGenerationFile));

    Microsoft.Build.BuildEngine.Engine engine = new Engine(ToolsetDefinitionLocations.Registry);
    engine.DefaultToolsVersion = "3.5";
    Project msBuildProject = new Project(engine);

    msBuildProject.Load(project.FullName);
    BuildPropertyGroup group = msBuildProject.AddNewPropertyGroup(false);
    group.AddNewProperty("ConnectionString", connectionString);

    group.AddNewProperty("SourceFile", generation.FullName);

    group.AddNewProperty("PurgeTablesBeforePopulate", clearDatabase.ToString());
    group.AddNewProperty("Verbose", "True");

    //FileLogger logger = new FileLogger();
    //logger.Parameters = @"logfile=D:\temp\build.log";
    //engine.RegisterLogger(logger);
    if (!engine.BuildProject(msBuildProject, "DataGeneration"))
    {
        throw new AssertFailedException("Cannot Generate Data plan");
    }

}

The solution is really simple, it is true that I have no way to invoke that protected method (Except reflection), but msbuild can be invoked programmatically, so I create an Engine class, load the main project file, then set some properties for the project such as ConnectionString, Verbose etc. With properties  you can specify the “SourceFile” that is the data generation file that you want to run. I’ve found a sample here that explain the xml syntax for msbuild, so I simply translate it into C# code using the Engine of msbuild directly from code. You need to imports Microsoft.Build.Engine to make this code compile. Now I can use that function in this way.

[TestMethod]
public void TestMethod1()
{
    DatabaseTestServiceHelper.RunDataGenerationPlan(
        @"..\..\..\NorthWindDatabase\",
        @"NorthWindDatabase.dbproj",
        @"Data Generation Plans\GenerationA.dgen",
        true,
        "System.Data.SqlClient",
        @"Data Source=localhost\sql2008;Initial Catalog=NorthWindForUnitTesting;Integrated Security=True;Pooling=False");
}

And I can decide programmatically when to run a specific data generation plan. The code to regenerate the whole structure is not important, it is enough for me to run at the beginning of test. The only thing I do not like is the need to use relative path to references project file and data generation plan file, but since it is uncommon to change relative path of the projects, I’m quite satisfied of it.

Alk.

Tags: data generation plan visual studio

Moreover asp.net has a not so good handling of themes, and all css that are in theme folder are included in final pages (if I compress files into 1 with youicompressor I’ll end with a lot of 404 request), I want to insert only a single css in the theme. Here is the solution, place in theme folder a single css files and include all original css files with include directive

@import url("../../ThemesCss/v2/BackOffice.css");
@import url("../../ThemesCss/v2/CollapsiblePanel.css");
@import url("../../ThemesCss/v2/control.css");
@import url("../../ThemesCss/v2/EMailEditor.css");

Now create a folder in the root named ThemesCss and put all your css there. Now everything works ok, but you have no compression and you still suffer from too many css in the site, but now you have only a css file included in the theme.

Now I created another file, with the same name of the single theme css file, but with extension .FileList, inside it I put all path of the css files that are used in the site.

~/ThemesCss/v2/BackOffice.css
~/ThemesCss/v2/CollapsiblePanel.css
~/ThemesCss/v2/control.css
~/ThemesCss/v2/EMailEditor.css

It is a simple text file, but now I’m able to write an handler that use this file list to compress all site.

    class YUICompressorHandler : IHttpHandler
    {
        #region IHttpHandler Members

        public bool IsReusable
        {
            get { return true; }
        }

        public void ProcessRequest(HttpContext context)
        {

            if (context.Request.FilePath.EndsWith(".css"))
            {
                String path = context.Server.MapPath(context.Request.FilePath);
                String compressedFileName = Path.ChangeExtension(path, ".csscompressed");
                String cssListFileName = Path.ChangeExtension(path, ".FileList");

                //Check if some source file was changed.
                if (File.Exists(compressedFileName))
                {
                    DateTime csscompressedDate = File.GetLastWriteTime(compressedFileName);
                    foreach (String cssfile in File.ReadAllLines(cssListFileName))
                    {
                        String realCssFile = context.Server.MapPath(cssfile);
                        if (File.GetLastWriteTime(realCssFile) > csscompressedDate)
                        {
                            File.Delete(compressedFileName);
                        }
                    }
                }

                //if not in debug mode activate caching. in debug mode we does not want cache.
                if (!context.IsDebuggingEnabled)
                {
                    //We are in release with debug="true" check for request cache
                    if (File.Exists(compressedFileName))
                    {
                        string ifModifiedSince = context.Request.Headers["If-Modified-Since"];
                        DateTime modifiedSince;
                        if (!string.IsNullOrEmpty(ifModifiedSince) && ifModifiedSince.Length > 0 && DateTime.TryParse(ifModifiedSince, out modifiedSince))
                        {
                            DateTime fileDate = File.GetLastWriteTime(compressedFileName);
                            TimeSpan difference = fileDate - modifiedSince;
                            if (difference.TotalSeconds > 0)
                            {
                                //Data is cachable.
                                context.Response.StatusCode = 304;
                                context.Response.StatusDescription = "Not Modified";
                                context.Response.AddHeader("Content-Length", "0");
                                return;
                            }
                        }
                    }
                    //If we reach here the caller does not specified If-Modified-Since, tells client that this is cacheable
                    context.Response.Cache.SetCacheability(HttpCacheability.Public);
                    context.Response.Cache.SetLastModified(File.GetLastWriteTime(compressedFileName));
                    context.Response.Cache.VaryByHeaders["If-Modified-Since"] = true;
                }
                //If we reach here we must stream the file, if is not present recreate.
                if (!File.Exists(compressedFileName))
                {
                    CompressCssFile(context, compressedFileName, cssListFileName);
                }
                context.Response.ContentType = "text/css";
                context.Response.TransmitFile(compressedFileName);
            }
        }

        private void CompressCssFile(HttpContext context, string compressedFileName, string cssListFileName)
        {
            StringBuilder sb = new StringBuilder();
            String[] lines = File.ReadAllLines(cssListFileName);
            foreach (string cssFile in lines)
            {
                //Each line of the file is a relative path to real file name
                sb.AppendLine(File.ReadAllText(context.Server.MapPath(cssFile)));
            }
            File.WriteAllText(compressedFileName,
                              CssCompressor.Compress(sb.ToString(), 80, CssCompressionType.StockYuiCompressor));
        }

        #endregion
    }

This is the very first version of the handler, it basically compress all original files into one, it checks if the previously compressed file is out-of-date because one of the original css is changed. If nothing is changed it returns the compressed file to the caller. It has even a simple check, if the site has debug=”false” and is in release mode, I honor cache of the browser, using appropriate headers.

It works really well, now even in test site deployed by CC.net I can have compression and I can enable/disable simply from the IIS control panel, if I send css request to asp.net engine I’ll have compression, if I do not map css extension to asp.net I have standard IIS behaviour.

In a site I’m working in I moved from 25 css files and 45Kb of uncompressed css, to a 1 css and a 28k of uncompressed css, that became 5.5k when gzipped.

alk.

Tags: asp.net css compression

First of all we have the syndrome of “It works on my machine”. The software run perfectly in developer’s machines, but when you move in production servers or into customers machines nothing work. If you ask developer, quite always you get this answer: “HAve You installed library XYZ? Then have you wrote into the registry the SuperSecretKey XYXCWKDSAF, after you done this you should run for a mile, turn yourself for three time with fingers on your nose, …..” This is the sign that you need at least a document that explain how to deploy software, or you’re gonna kill your developers

Then the software begin to work with real customer data, and you can see with horror that everything worked ok with test data, but when you import 10KK orders from the old system of your customer, the software stops working. But other problems can arise, maybe the customer works with order with negative amount and the software does not permits so. In the end, you find that test data were not representative, and now you are in trouble with a live system that is not capable to handle the data that the customer really needs. (you’ll end with unsatisfied customer)

Maybe the software begin to crash, when you see the log it happens that users insert “Hello Folks!” where the software expects a DateTime…Ok, it is not so simple, but it often happens that real user uses the software in a way we never thought that was possible, and this pattern lead to a crash.

You have your system in production, users begin to send feedback, after some time you have a new version ready, but… you now have to update database, update client machines, make sure that older version can work with the new data. Another typical problem is that you cannot stop production server, but you need to make changes online.

Customer Calls you telling “The system crashed”, if you ask more detail maybe you get “The System crash when we are working on it”, please do not expect customer to be technical skilled, he can never tells you real details of the error, if you want to laugh just ask to the customer “Can you send me the stack trace?”, the answer can range from “WHAT??” to “Hey does not bother me with technical terms, the system chrased, NOW YOU’LL FIX IT”. Then you can only go into production server and then you need to understand why, when, what part of the system crashed…. it sounds you need real careful logging system. Maybe you log everything in a file, but…. you have no access to production server because in the customer web farm, so you need to contact the IT manager, tell him you need log, typical answer is “in witch machine are those log…… this lead to pain and frustration

I can continue but the real concept is that a system in production can generate an high number of problems that does not arise during developement stage. So what is the solution? It turns out that it can be as simple as the sentence

“Go in production as often as you can”.

I only gives you a little scenario. Set up a CC.net or similar integration machine, create a test production server, at each check in the integration machine compiles code , runs tests and if everything is ok, it deploys the software in the test machine, let it be called preproduction machine . Fill this machine with real data, and make it available to the customer. Now you immediately faces all the problems of a system live in production, because you are actually developing against a software in production, where you virtually go in production at each checkin .

1. You are forced to create deploy script that automates the task of deploying the software
2. You are forced to maintain data of the customer in the preproduction machine, so you need to find a way to manage software updates.
3. You immediately works with real data and real user, moreover you immediately have feedback on the software
4. Since software is in development stage, bug arises, and you need a quick way to correct them, so you will setup a good error handling system such as elmah, or maybe you can setup a log system that email or send all data to a bug tracking system where developer will automatically see details about the error, without the need to go into the production machine.

This means that you begin to face immediately typical problems of a live software, and you do not postpone solving them when you really are in production. Solving a problem when you are really in production, can be really a pain, so you better work hard to make all possible problem arising during developement stage. Time spent to immediately start developing with a production environment can be difficult, but in the long run it really saves your life.

Alk.

Tags: Production