PrimordialCode

WP7 - Test if your ViewModel is serializable

guardian — Fri, 24 Feb 2012 22:47:24 GMT

Writing WP7 applications (and the new Metro UI Apps in Windows 8) you will have to deal with concepts like ‘Tombstoning’ or ‘Suspension’; if you follow the MVVM pattern, there’s a very good chance that the status of your application page is represented by the ViewModel itself. So a quick and dirty way to handle those suspended states is to ‘persist’ the ViewModel (or the part of it that have actual meaning) to the application state or to a file storage and retrieve it at a later time when the application is reactivated.

if you put something that is not a primitive type inside the Application State or the Page State dictionaries, it will be serialized using the standard DataContractSerializer (on the subject of serialization and deserialization in WP7 you can read these previous posts of mine: WP7 Understanding Serialization and http://www.primordialcode.com/blog/post/wp7-datacontractserializer-bug).

A good way to avoid headaches when using this approach to handle tombstoning/suspension is the to test not only the behavior of your ViewModels, but also if your ViewModels are serializable; you can do this by using an helper class similar to this one:

public static class DataContractSerializerHelpers
{
    public static string ToXml(object obj)
    {
        return ToXml(obj, null);
    }

    public static string ToXml(object obj, IEnumerable<Type> knownTypes)
    {
        Type objType = obj.GetType();
        DataContractSerializer ser = new DataContractSerializer(objType, knownTypes);
        {
            using (StringWriter sw = new StringWriter())
            {
                XmlWriterSettings settings = new XmlWriterSettings();
                settings.OmitXmlDeclaration = true;
                settings.Indent = true;
                //settings.NewLineOnAttributes = true;
                using (XmlWriter writer = XmlWriter.Create(sw, settings))
                {
                    ser.WriteObject(writer, obj);
                }
                return sw.ToString();
            }
        }
    }

    public static object FromXml(string data, Type type)
    {
        return FromXml(data, type, null);
    }

    public static object FromXml(string data, Type type, IEnumerable<Type> knownTypes)
    {
        using (StringReader sr = new StringReader(data))
        {
            XmlReaderSettings settings = new XmlReaderSettings();
            using (XmlReader reader = XmlReader.Create(sr, settings))
            {
                DataContractSerializer ser = new DataContractSerializer(type, knownTypes);
                return ser.ReadObject(reader);
            }
        }
    }

    public static T FromXml<T>(string data) where T : class
    {
        return FromXml(data, typeof(T), null) as T;
    }

    public static T FromXml<T>(string data, IEnumerable<Type> extraTypes) where T : class
    {
        return FromXml(data, typeof(T), extraTypes) as T;
    }
}

It’s basically a simple wrapper around the DataContractSerializer function to call it in a convenient way that mimic what the infrastructure does.

Using the standard DataContractAttribute and DataMemberAttribute on the ViewModels you can fine tune what will be persisted (obviously you should limit this to the properties that have some meaning for your application state, you will also have to design the ViewModels to provide lazy initialization for everything that depends on those data). Your tests will then look similar to this one:

[TestClass]
public class SerializableTypes : SilverlightTest
{
    // Test if we can save the VM to the state dictionaries
	[TestMethod]
    public void ViewModel_Serialize()
    {
		// todo: add proper members initialization
	    MainViewModel vm = new MainViewModel();
        DataContractSerializerHelpers.ToXml(vm);
    }

	// Test if we can retrieve the VM from the state dictionaries
    [TestMethod]
    public void ViewModel_DeSerialize()
    {
		// todo: add proper members initialization
	    MainViewModel vm = new MainViewModel();
        var data = DataContractSerializerHelpers.ToXml(vm);

        // test
        var loaded = DataContractSerializerHelpers.FromXml<MainViewModel>(data);
        Assert.IsNotNull(loaded);
		// todo: add proper testing on members values
    }
}

With this approach the code that you have to write on the views to handle tombstoning will become trivial; in your views you will have something similar to this:

...

private SingleFeedViewModel Vm
{
    get { return (SingleFeedViewModel)DataContext; }
    set { DataContext = value; }
}

protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
    base.OnNavigatedTo(e);

    // if we came up after a dormant state we do not need to do anything at this stage
    // I also have to check if the Vm is unassigned (we can come here from a normal navigation, after a resume)
    if (ApplicationHelper.IsApplicationInstancePreserved && Vm != null)
        return;

    // recover from tombstoning
    if (State.ContainsKey(StateKeys.SingleFeedVm))
	{
		//clear prev value
		SingleFeedViewModel itm = State[StateKeys.SingleFeedVm] as SingleFeedViewModel;
		if (itm != null)
		    Vm = itm;
	}
    // nothing was retrieved from the state dictionary, create an instance
	// of the ViewModel
    if (Vm == null)
    {
        Vm = new SingleFeedViewModel();
        Vm.LoadData();
    }
}

protected override void OnNavigatedFrom(System.Windows.Navigation.NavigationEventArgs e)
{
    base.OnNavigatedFrom(e);
    // save state to handle tombstoning
    if (State.ContainsKey(StateKeys.SingleFeedVm))
    {
        //clear previous value
        State.Remove(StateKeys.SingleFeedVm);
    }
    State.Add(StateKeys.SingleFeedVm, Vm);
}

...

WARNING! I was about to forget an important point: events cannot be serialized (once again a reference to another old post of mine: Serialization Exception: PropertyChangedEventManager is not serializable, it’s related to binary serialization or objects, but the concept is the same), so remember to not mark the events with the DataMemberAttribute or, if your ViewModel has everything public, and you didn’t used the DataContractAttribute to opt-in what you want to be serialized, it’s better to mark the events with [field: IgnoreDataMember] to exclude them from the serialization process.

WP7 Unit Testing - getting started

guardian — Wed, 22 Feb 2012 20:26:26 GMT

Testing is important and it can save you lots of time, to successfully setup a WP7 Unit Test project I recommend you to first obtain the ‘NuGet Package Manager’ Visual Studio 2010 extension, it will make it a breeze to retrieve the latest version of libraries and packages and add a reference to them.

Here is a step by step guide on how to create your test project using the testing framework kindly provided to us by Jeff Wilcox:

Open up Visual Studio and install NuGet if you do not have it: Tools –> Extension Manager, look for NuGet Package Manager in the Online Gallery.
Create a new Windows Phone Application project, do not modify it right now.
Use NuGet to get the latest version of the Unit Test framework for WP7, you can do it in two ways:
- for the ‘Tools’ menu chose ‘Library Package Manager’ and then ‘Manage NuGet Packages for Solution’
- right click on the project you wish the package to be added to and select ‘Manage NuGet Packages’
In the dialog that will open go to the Online section and look for ‘silverlight.unittest.wp7’
click on ‘install’ to download and add the package to the selected project.
it’s time to modify the test project following what’s written in the Test_Readme.txt file:
- open up the MainPage.xaml and handle the ‘Loaded’ event
- add the following line to the event to activate the test infrastructure: this.StartTestRunner();
Done. Running the project you should see something like this:

You are now ready to write you own custom tests using a syntax and rules very similar the ones used by the MSTest suite that is already integrated in Visual Studio.

Configure WebMatrix/IIS Express to work with WP7 on Windows 7

Guardian — Tue, 07 Feb 2012 17:15:34 GMT

Long time is passed after my last ‘notable’ blog post, I was just too busy and lazy in the past months, lots of things happened and I couldn’t follow everything, but it’s not that I want to talk about, let’s talk about some more WP7 development.

Actually I am ‘playing’ with WP7, jQuery Mobile, WCF services and some various mobile stuff these days, honestly I was tired of having my WP7 device connected to the USB cable in order to be able to surf websites and use services that are hosted on my development machine. So I started to scouting around looking for a solution.

My goal is to be able to host websites and WCF services using WebMatrix ad IIS Express at first and a custom made WCF self hosting solution at a later time for my developing everyday life (and for demo purposes).

So here’s my current setup:

IIS Express/WebMatrix
Zune
WP7 + USB cable connected
a fantastic website hosted on IIS Express

Everything is working correctly here, I can browse the website from WP7 and test it, but I have to keep the cable connected, which is a no go if I want to make a demo of something to someone with a ‘floating’ device. I do not want to create new DNS entries, nor touch the network infrastructure nor do any other complicated IT or network related stuff, that’s not for me.

I want to be able to switch to this setup (both for testing and demo environment):

IIS Express/WebMatrix
WP7
still my fantastic website hosted on IIS Express

you see: no Zune, no cable, just me and my device (and Visual Studio of course). To obtain this we need to make IIS Express able to accept connections from the outside, you can do that following this guide:

http://blogs.iis.net/vaidyg/archive/2010/07/29/serving-external-traffic-with-webmatrix-beta.aspx

Here’s what I had to do in order to make things work:

Get your computer IP, mine was: 192.168.1.10.
Pickup a port you will use to host your service (using port 80 is bad idea on a development machine), I choose: 24778.
Configure HTTP.SYS (the component that is used to handle http requests) to accept external connection when running as standard user (you can bypass this if you launch WebMatrix and IIS Express with admin privileges, but that’s not good for security reasons), open a shell prompt with admin rights and type:
```
netsh http add urlacl url=http://192.168.1.10:24778/ user=everyone
```
Configure the binding in WebMatrix: doing it by manually editing the applicationhost.config file is the best way. The file is located is your user profile at the following folder: “%USERPROFILE%\documents\IISExpress\config”. Another way to find it is: right click on the IIS Express icon in the system Tray, choose “show all applications”, highlight your website and then look at the configuration setting (it’s the last voice on the window), you can even click on it to open the file in notepad.

The image is in Italian, I know ; now that you have your file open you need to locate the your website configuration and change the binding settings to allow connection the pre-defined IP address and port. Here’s how I modified my configuration:
```
<site name="MyWonderfulSite" id="2">
 <application path="/">
     <virtualDirectory path="/" physicalPath="D:\XXXXXXXXXX" />
 </application>
 <bindings>
     <binding protocol="http" bindingInformation="*:24778:localhost" />   
     <binding protocol="http" bindingInformation="192.168.1.10:24778:" />   
 </bindings>
</site>
```
I have added a binding that explicitly configure the IP address and port that I want to use without assigning a hostname.
The last step is to open up the port in your Firewall, this is up to you depending on the Firewall you are using.

If you have followed all the steps you are now able to browse your test website (using the specified address like: http://192.168.1.10:24778/) from your WP7 without having the USB cable connected and Zune open, pretty cool isn’t it ?

These very same steps can be used for your Android and iPhone devices, but I personally do not own any of those devices so I wasn’t able to try it.

Doing this research I’ve learned a couple of things about IIS Express and how the http requests are served by the system.

As last note: to undo the modifications you’ve done to the HTTP.SYS configuration you can use the following command:

netsh http delete urlacl url=http://192.168.1.10:24778/

See you next.

Windows 7 ATIKMPAG.SYS BSOD Solved (in my case)

Guardian — Thu, 02 Feb 2012 17:01:09 GMT

After having updated the drivers for my graphic card to the latest version - Catalyst 10.12 as I write this post - taken directly from the ATI website, I started to experiment some Blue Screen Of Death (BSOD) at the startup of the OS.

More specifically it all was happening when the Catalyst CCC was about to be fully loaded.

The error message reported by the system was something like: “atikmpag.sys has stopped responding”, followed by the memory dump.

I tried some solutions found on the internet but nothing worked: I kept having the BSOD at the startup; I tried reinstalling the drivers but neither that has worked, I even tried some old versions of the drivers with no luck.

I was however able to startup the machine in recovery mode, so I started looking for some files left behind by the Uninstall manager: in my private profile directory I found a file that was storing all the drivers settings and customization that were made during the previous months. That file (called Profiles.xml) was not removed during the uninstall process, looking into that I’ve found some settings that turned to be the cause of my troubles with the new drivers.

To solve this issue just delete the “Profiles.xml” you can find in the user profile directory: ‘C:\Users\YOURUSERNAME\AppData\Local\ATI\ACE’; at the next restart this file will be recreated by the system with the default settings.

Maybe in the past I installed some gaming profile that caused some problems or tried some ‘experimental’ configuration to improve performances that was causing troubles, I don’t really know and, actually, I don’t really care , but resetting it to the default values solved the problem for me avoiding a complete OS installation from scratch.

HP ProLiant MicroServer, Windows 7 or Vista installation issues

Guardian — Tue, 13 Dec 2011 20:28:36 GMT

Stating at what HP says installing Windows Vista or Windows 7 on an HP ProLiant MicroServer (in my case the old N36L model) is not supported. This doesn’t mean it isn’t doable.

All you need are the correct drivers to start the setup properly, I had an old license of Windows Vista (32 bit...ouch!) I wasn’t using anymore, so I’ve decided to install it over this piece of hardware to do some testing at my home.

First problem I faced was the missing of a driver for the RAID controller (yes I set it up to use RAID 1 through the BIOS settings); once I figured out it has an AMD SATA/RAID Controller on board I started looking around for some drivers, the following package worked for me:

AMD SATA/RAID AHCI Controller Driver (VERSION: 3.2.1548.37 REV: G)

Just unzip it to a folder on your USB flash drive and extract the content of the ‘exe’ file (do not run or install it), you can do it using lots of free utilities out there (WinRar, just to name one).

During the setup you will need to chose on which driver install Windows, if you don’t see any drive available (as it was in my case, having built a RAID array and a logical disk), you will need to fed the setup with the correct raid controller drivers. The drivers are located in the following path inside the previously extracted directory: \sp48266\RAID7xx. There will be x86 or x64 versions of the driver, chose the one that match your OR.

At this point the setup will go on and everything will be installed correctly.

Next step is to use ‘Windows Update’ to finish off the installation (protect the system, keep everything updated, install eventually missing drivers and so on...), but wait...we have no valid driver for the Network Card...sob ; once again let’s look for some information and drivers.

The actual network card happens to be identified like: ‘HP NC107i PCIe Gigabit Server Adapter’ and it’s made by Broadcom, looking around a bit I was able to find a list of standard drivers from the Broadcom website:

NetXtreme I Server

Pick the ones that correspond to your OS and:

Extract the content in a folder or USB drive.
Open up the Control Panel and go to the device management, highlight the missing network card, right click on it and choose update drivers.
Navigate to the directory where you previously extracted the drivers and install them.
Reboot the system (this is required for the drivers to be able to work).

Forcing these drivers in my case it made the things work; now I’m able to browse my network and install updates.

Some information about the hardware you can find inside a ProLiant MicroServer were taken out from the HP product sheet and the ‘support and drivers’ pages at the following url:

http://h20000.www2.hp.com/bizsupport/TechSupport/ProductList.jsp?lang=en&cc=us&taskId=135&prodTypeId=15351&prodSeriesId=4248009

Maybe this post can help someone that is having my same issues.

Creating a ‘Windows Installation’ bootable USB drive

Guardian — Tue, 13 Dec 2011 01:19:44 GMT

Recently I was in need to setup some new hardware and I’ve found myself doing this procedure over and over (because I am a cheap b*****d and I only own few USB pen drives ) again. So even if there are plenty of posts out there offering more or less the same content, I’ll write it down here as a reminder to myself.

Consider you can use this very same procedure to create a bootable USB flash drive to install any Microsoft Windows OS (from Vista to 2008 Server R2).

Prepare and Format the drive

You can do these first steps in several way, I prefer to use the diskpart command line utility; BE CAREFUL HERE! If you do something wrong you may easily wipeout you disks!

Open a command prompt as administrator (Right click on Start > All Programs > Accessories > Command Prompt and select “Run as administrator”)
type the following commands into the Command Prompt Window:

diskpart

list disk

This will bring up a list of all your physical drives installed; look for your USB drive and take note of the number; if you have multiple USB drives attached I suggest you to disconnect them all but the one you want to use: the only way to identify the drive is looking at the drive’s size in this step, having multiple USB drives with the same capacity is not good!
Format the drive by typing the following commands into the same window. Replace “X” with the number of your disk.

select disk X

clean

create partition primary

select partition 1

active

format fs=NTFS

assign

exit

At this stage we have a clean formatted USB with an active partition ready to be made bootable. Some guides will tell you to format the pen drive using FAT32, I always prefer to use NTFS especially to create bootable winpe drives to use the imagex.exe utility to clone systems, in this scenario we can easily break through the FAT32 file size limit.

Make the drive bootable

To make the drive bootable we will use the bootsect utility that comes along with any Microsoft Windows OS:

Insert your OS DVD.
Change directory to X:\boot (where X is your dvd drive).
Type the following instruction to make the drive bootable (now ‘X’ is your pen drive).

bootsect /nt60 X:
Close the Command Prompt Window.

The last step is copy all your installation files from the DVD (or ISO) to your newly created bootable USB flash drive.

If you do not want to do everything by hand, there’s a tool from Microsoft you can use to do the very same work: Windows 7 USB/DVD download tool.

Build: Windows 8

Guardian — Wed, 14 Sep 2011 17:13:25 GMT

I bet almost all the developers followed the ‘build’ keynote Yesterday, we had some big news (nothing too much scary, thankfully) and we saw Windows running on a bunch of new devices (I honestly can’t wait to put my hands on them) that will hit the market sooner or later in the future.

On the Dev side, I’m very excited for what we saw: new potentials, some big improvements and a potentially rewritten and optimized infrastructure over which your apps will run on. I loved the metro style UI on WP7 and this ‘bigger brother’ seems very good to me. And yes the ‘pro’ will have their usual desktop view too (with productivity improvements on that side too).

On the consumer and business side, timing here is quite important: when this new beast will be released to the market ? will it be too late to compete against its ‘enemies’ ? when the new devices will be available ? and another one... pricing ? will it get a good welcome from the customers? we can go on and on and on...

Windows 8 should focus its target on the new devices and new installations, because hoping that a big part of the pre-existing windows installed machine will be updated to windows 8 in the close future is just...well...chose your word here!.

Here in Italy before updating any windows version to the new one the usual policy is to ‘wait for the SP1’, people is just scared of changes (and of course the ‘money factor’ plays its main role too, which will stops many businesses to early adopt it).

While we wait and think about all of this, we (devs) can happily put our hands on a Developer Preview of Windows 8, head up to MSDN for more info or just follow these links:

Windows Developer Preview: http://msdn.microsoft.com/en-us/windows/apps/br229516

Build Keynote & sessions info: http://www.microsoft.com/presspass/events/build/

Build sessions will be available through Channel 9: http://channel9.msdn.com/Events/BUILD/BUILD2011?sort=sequential&direction=desc&term

Time to have some fun (again)!

NHibernate.Envers - Customize the Revision Entity

Guardian — Fri, 05 Aug 2011 21:10:17 GMT

Sometimes the basic information provided by the default revision entity we have in NHibernate.Envers are not enough, when we need to extend those information and provide additional data we have the option to use a customized version of the Revision Entity class.

The default revision entity is defined like this:

[Serializable]
public class DefaultRevisionEntity
{
	[RevisionNumber]
	public virtual int Id { get; set; }

	[RevisionTimestamp]
	public virtual DateTime RevisionDate { get; set; }

	public override bool Equals(object obj)
	{
		if (this == obj) return true;
		var revisionEntity = obj as DefaultRevisionEntity;
		if (revisionEntity == null) return false;

		var that = revisionEntity;

		if (Id != that.Id) return false;
		return RevisionDate == that.RevisionDate;
	}

	public override int GetHashCode()
	{
		var result = Id;
		result = 31 * result + (int)(((ulong)RevisionDate.Ticks) ^ (((ulong)RevisionDate.Ticks) >> 32));
		return result;
	}
}

You can notice two properties that defines the information that a Revision Entity class MUST have, they are also marked by two configuration attributes:

[RevisionNumber] - states which property will represent the revision number of the version
[RevisionTimestamp] - states the timestamp of the version

Every custom Revision Entity class MUST have these two kind of properties, once again we will use the fluent by code configuration so we will not need to decorate our classes with attributes (I’ll show how to configure NHibernate.Envers using attributes in another post).

Our custom Revision Entity class for this example will be:

//[RevisionEntity]
public class REVINFO
{
	//[RevisionNumber]
	public virtual long Id { get; set; }

	//[RevisionTimestamp]
	public virtual DateTime CustomTimestamp { get; set; }

	public virtual string Data { get; set; }

	public override bool Equals(object obj)
	{
		var casted = obj as REVINFO;
		if (casted == null)
			return false;
		return (Id == casted.Id &&
				CustomTimestamp.Equals(casted.CustomTimestamp) &&
				Data.Equals(casted.Data));
	}

	public override int GetHashCode()
	{
		return Id.GetHashCode() ^ CustomTimestamp.GetHashCode() ^ Data.GetHashCode();
	}
}

As you can see we have the two basic properties plus an additional Data field that will represent our custom data added to each revision tracked.

To be able to fill these data we need to implement a particular type of interface: an IRevisionListener. This interface expose only a single member function - void NewRevision(object) - and it’s called every time a new revision object is generated. The demo code is:

public class RevInfoListener : IRevisionListener
{
	public static string Data = "test data";

	public void NewRevision(object revisionEntity)
	{
		((REVINFO)revisionEntity).Data = Data;
	}
}

The last step is to change the configuration to ‘inject’ our custom Revision Entity class and Listener; in my test project I do it with the following code:

public class NHinitCustomRevInfo : NHibernateInitializer
{
	protected override System.Collections.Generic.IEnumerable<System.Type> GetDomainEntities()
	{
		// I am using ConfORM to emit the mappings, so we add it to the mapped classes.
		return base.GetDomainEntities().Union(new[] { typeof(REVINFO) });
	}
	
	protected override void ConfOrmMapping(ConfOrm.ObjectRelationalMapper orm, ConfOrm.NH.Mapper mapper)
	{
		// I have to provide a mapping for a custom RevInfo class...NHibernate must know how to handle these objects
		orm.TablePerClass<REVINFO>();
		
		orm.TablePerClass<Person>();
		orm.TablePerClass<Game>();
	}

	public void InitializeAudit()
	{
		// initialize the NHibernate.Envers fluent configuration object
		var enversConf = new NHibernate.Envers.Configuration.Fluent.FluentConfiguration();

		// I prefer to not use attributes to configure the custom revision entity
		enversConf.SetRevisionEntity<REVINFO>(e => e.Id, e => e.CustomTimestamp, typeof(RevInfoListener));
		
		// the RevInfo class must not be in the auditing list
		enversConf.Audit(GetDomainEntities().Where(e => !typeof(REVINFO).IsAssignableFrom(e)));

		// to inspect the metadata
		//var mets = enversConf.CreateMetaData(Configure);

		// Configure.Properties.Add("nhibernate.envers.audit_table_prefix", string.Empty); // default
		Configure.Properties.Add("nhibernate.envers.audit_table_suffix", "_REV"); // default _AUD
		// Configure.Properties.Add("nhibernate.envers.revision_field_name", "REV"); // default
		// Configure.Properties.Add("nhibernate.envers.revision_type_field_name", "REVTYPE"); // default

		Configure.IntegrateWithEnvers(enversConf);
	}
}

I am using ConfORM and my mapping engine, so part of the configuration is related to how it’s used to generate the mappings, but the concepts are valid for any other mapping tools you use; Let’s go through it:

Line 6: we add the REVINFO class to my mapped domain, ConfORM need to be aware of it.
Line 12: we tell ConfORM how to map the class.
Line 21: we initialize the NHibernate.Envers fluent configuration engine.
Line 24: this is the crucial point, we specify our custom Revision Entity class type, we tell Envers which functions to use to fill in the Revision incremental number and the Revision Timestamp; plus we pass in the type of the IRevisionListener implementation if we want to specify additional data in the custom revision entity class.
Line 27: we tell to NHibernate.Envers which classes we want to track (be careful to exclude the REVINFO class from the tracked ones here).
Line 37: we add NHibernate.Envers to the standard NHibernate configuration.

If we create a simple test and execute it we can inspect the schema of the generated database to see the custom revision entity in place:

Figure 1 - Database Schema with Custom Revision Entity

That’s all for this post, next time we’ll see how to query for the extended revision entity properties we just added.

Sql Server - Scramble / Mask database production data

Guardian — Wed, 27 Jul 2011 18:59:49 GMT

Sometimes you might need to test your product using some ‘production’ data to better simulate what users do with your software.

There are situations in which having only your test database, with your own (well formed) test data is not enough: just think of a continuously growing database with frequent schema changes...you might likely have inconsistencies between the data as you upgrade it or unpredicted behaviors.

If you want use a copy or a snapshot of a production database for test purposes, you should at least try to hide the sensitive data away (in a way that is really hard or impossible for your development team or anyone else that have access to the database to reconstruct them).

As you can imagine the task here can be extremely difficult to accomplish and there might be some tools out there that can help you doing this job.

If you do not want to invest in them a quick and dirty solution (it took me more or less 15 minutes to put it all together, and I’m not a SQL expert) can be writing some SQL scripts to modify what you consider to be your most sensitive data, leaving everything else intact. It doesn’t guarantee that all the sensitive data are hidden/changed because it relies on the code you write.

The typical scenario is to modify the names and telephone numbers of some personal data records, but you can apply this technique to anything you want.

Fist we need some functions to create test data, here you can use anything you want but I prefer something that generates data based on a prefix I can pass in; you can easily create functions that generate completely random data, but it will be hard to talk to your team and say something like: “ehi...check the person named ‘XWggTRDnnns’!”, a simple string concatenation does the job here.

--concatenates a generic string and an incremental number
CREATE FUNCTION fn_GenerateText(@prefix varchar(255), @incr int)
RETURNS varchar(255)
AS
BEGIN
return @prefix + CONVERT(varchar(100),@incr)
END
GO

Next we need to scan all our data tables and update the data we find there, here I decided to use a cursor (it’s not efficient but you can handle any type of key in this way, from guids to strings), so the idea is:

read each line of the table
generate some test data for each of the sensitive columns with text like: ‘name0’, ‘surname0’, etc... the number will be incremented for each line
update the data row with the given data for each of the columns you need to modify

-- a variable we use to generate incremental numbers
Declare @i int
set @i = 0

-- let's use a cursor, i don't know the data type of the key column
declare @id uniqueidentifier;
Declare id_cursor CURSOR
	FOR select AP_ID from dbo.PersonalData;
open id_cursor;
fetch next from id_cursor into @id;

WHILE @@FETCH_STATUS = 0
BEGIN

update  dbo.PersonalData
set 
AP_Note = dbo.fn_GenerateText('test', @i),
AP_FiscalCode = dbo.fn_GenerateText('fc', @i), -- or use a valid fiscal code generation routine here
AP_Surname = dbo.fn_GenerateText('surname', @i),
AP_Name = dbo.fn_GenerateText('name', @i),
AP_Phone = dbo.fn_GenerateText('t', @i),
AP_CellPhone = dbo.fn_GenerateText('c', @i)
Where AP_ID = @id 

-- increment the suffix and get the new data from the cursor
set @i = @i+1;
fetch next from id_cursor into @id;
END
CLOSE id_cursor;
DEALLOCATE id_cursor;
GO

In the end we just cleanup the database from every support function we created:

DROP FUNCTION dbo.fn_GenerateText
GO

As you can see this is a very quick solution and you will have to write some SQL code and manage the relations by yourself if needed, nonetheless once you have your scripts in place it’s a quite easy solution to use and maintain.

Fetching collections with NHibernate, a mistake a I always do!

Guardian — Wed, 20 Jul 2011 00:52:37 GMT

I did it again! That’s why I’m writing this blog post...I do it almost every time and the main reason is: trying to optimize NHibernate queries without thinking.

As the title of the posts says the problem here is related to fetching the data of different collections attached to a single entity. To show you up the problem let’s first see the test domain, it is very simple:

public class Post : Entity<int>
{
	protected Post()
	{
	}

	public Post(string slug)
	{
		Slug = slug;
		Categories = new List<Category>();
		Tags = new List<Tag>();
	}

	public string Slug { get; set; }

	public IList<Category> Categories { get; set; }

	public IList<Tag> Tags { get; set; }	
}

public class Category : Entity<int>
{
	public string Name { get; set; }
}

public class Tag : Entity<int>
{
	public string Name { get; set; }
}

I generated some test data consisting of a single post which has 2 categories and 2 tags associated to it.

We want to query the Post object together with all it’s data, the first thing we can do is this:

[Test]
public void NoFetch_Success()
{
	using (ISession s = _nh.SessionFactory.OpenSession())
	{
		// look for a non existing revision
		using (var tx = s.BeginTransaction())
		{
			var p = s.CreateQuery("from Post p where p.Slug = :slug")
				.SetParameter("slug", "Slug")
				.UniqueResult<Post>();

			tx.Commit();

			Assert.IsNotNull(p);
			Assert.AreEqual(2, p.Categories.Count);
			Assert.AreEqual(2, p.Tags.Count);
		}
	}
}

Straightforward and simple; this thing produces 3 different queries, one for each kind of object:

select post0_.Id as Id0_, post0_.Slug as Slug0_ from Post post0_ where post0_.Slug=@p0;@p0 = 'Slug' [Type: String (4000)]

SELECT categories0_.PostId as PostId1_, categories0_.Id as Id1_, categories0_.Id as Id1_0_, categories0_.Name as Name1_0_ FROM Category categories0_ WHERE categories0_.PostId=@p0;@p0 = 1 [Type: Int32 (0)]

SELECT tags0_.PostId as PostId1_, tags0_.Id as Id1_, tags0_.Id as Id2_0_, tags0_.Name as Name2_0_ FROM Tag tags0_ WHERE tags0_.PostId=@p0;@p0 = 1 [Type: Int32 (0)]

The returned data are all correct.

But we really have too many queries here and we want to optimize it a bit (in order to limit/prevent the impact of the n+1 selects problem), so in my optimization berserker fury I started using fetch joins to retrieve more data in a single pass and I wrote this query:

from Post p left join fetch p.Categories left join fetch p.Tags where p.Slug = :slug

which is an absolute mistake, because it generates duplicated data; let’s see some test code:

[Test]
public void FetchALL_Failure()
{
	using (ISession s = _nh.SessionFactory.OpenSession())
	{
		// look for a non existing revision
		using (var tx = s.BeginTransaction())
		{
			// try adding a: select distinct p 
			var p = s.CreateQuery("from Post p left join fetch p.Categories left join fetch p.Tags where p.Slug = :slug")
				.SetParameter("slug", "Slug")
				.UniqueResult<Post>();

			tx.Commit();

			Assert.IsNotNull(p);
			Assert.AreEqual(2, p.Categories.Count); // <- big exception... we have 4 rows
			Assert.AreEqual(2, p.Tags.Count);
		}
	}
}

which produces 1 single query:

select post0_.Id as Id0_0_, categories1_.Id as Id1_1_, tags2_.Id as Id2_2_, post0_.Slug as Slug0_0_, categories1_.Name as Name1_1_, categories1_.PostId as PostId0__, categories1_.Id as Id0__, tags2_.Name as Name2_2_, tags2_.PostId as PostId1__, tags2_.Id as Id1__ from Post post0_ left outer join Category categories1_ on post0_.Id=categories1_.PostId left outer join Tag tags2_ on post0_.Id=tags2_.PostId where post0_.Slug=@p0;@p0 = 'Slug' [Type: String (4000)]

But the data are all wrong! In fact the Assert that should check for the correct numbers of categories are telling me that I have 4 rows inside that collection. I should have reminded earlier what the NHibernate documentation says about this situation:

It is possible to create a cartesian product by join fetching more than one collection in a query, so take care in this case. Join fetching multiple collection roles is also disabled for bag mappings. Note also that the fetch construct may not be used in queries called using Enumerable(). Finally, note that full join fetch and right join fetch are not meaningful. (Reference documentation, 13.3 Associations and joins)

Note that adding a ‘select distinct p’ will not help in this case.

To fix this we need to have just one fetch join at each level of the hierarchy; we can write something like this:

var p = s.CreateQuery("from Post p left join fetch p.Categories where p.Slug = :slug")
						.SetParameter("slug", "Slug")
						.UniqueResult<Post>();

This code will generate 2 queries (one to get the Post and the Categories and one to get the Tags) and the test will pass:

select post0_.Id as Id12_0_, categories1_.Id as Id13_1_, post0_.Slug as Slug12_0_, categories1_.Name as Name13_1_, categories1_.PostId as PostId0__, categories1_.Id as Id0__ from Post post0_ left outer join Category categories1_ on post0_.Id=categories1_.PostId where post0_.Slug=@p0;@p0 = 'Slug' [Type: String (4000)]
		
SELECT tags0_.PostId as PostId1_, tags0_.Id as Id1_, tags0_.Id as Id14_0_, tags0_.Name as Name14_0_ FROM Tag tags0_ WHERE tags0_.PostId=@p0;@p0 = 1 [Type: Int32 (0)]

But...wait! .UniqueResult<T>() here is smart enough to hide us a fetch join side effect; if we use .List<T>() instead we will see it clearly: the fetch introduces ‘ghosts’ in our returned resultset; the data will be a combination of Posts and Categories fields, so for any single Post object we will have ‘n’ records (one for each Category bound to the Post) and NHibernate will output ‘n’ Post entities, all of them containing the same data.

The solution here is pretty simple: just add the ‘distinct’ keyword to the select statement (or use the DistinctRootEntityTransformer), this way we are telling NHibernate to give us back all the unique instances of Post objects together with their Categories, here is the code:

[Test]
public void FetchSingleCollection_Distinct_Success()
{
	using (ISession s = _nh.SessionFactory.OpenSession())
	{
		// look for a non existing revision
		using (var tx = s.BeginTransaction())
		{
			var p = s.CreateQuery("select distinct p from Post p left join fetch p.Categories where p.Slug = :slug")
				.SetParameter("slug", "Slug")
				.UniqueResult<Post>();
			tx.Commit();

			Assert.IsNotNull(p);
			Assert.AreEqual(2, p.Categories.Count);
			Assert.AreEqual(2, p.Tags.Count);
		}
	}
}

If I remind it well, there’s no need to use the distinct on the queries generated using Linq to NHibernate, because the provider does it for you.

As usual here’s the test project for those who want to play with it:

NHibernateTestsCollectionFetching.zip

I am hoping, now that I’ve wrote it down, to not make the same mistake again ...

PrimordialCode

WP7 - Test if your ViewModel is serializable

Related Content

WP7 Unit Testing - getting started

Related Content

Configure WebMatrix/IIS Express to work with WP7 on Windows 7

Windows 7 ATIKMPAG.SYS BSOD Solved (in my case)

HP ProLiant MicroServer, Windows 7 or Vista installation issues

Related Content

Creating a ‘Windows Installation’ bootable USB drive

Prepare and Format the drive

Make the drive bootable

Build: Windows 8

Related Content

NHibernate.Envers - Customize the Revision Entity

Related Content

Sql Server - Scramble / Mask database production data

Related Content

Fetching collections with NHibernate, a mistake a I always do!

Related Content