Ramblings from Rhys

Lessons Learnt - Building an SDP

noreply@blogger.com (Anonymous) — Sun, 08 Jul 2012 23:06:00 +0000

I was recently asked what lessons had I learnt in regards to building an SDP and I thought it was an interesting question in many aspects and a question I would attempt explore in the form of a post on this blog.
Now to prefix this post I am obviously only a cog in the many wheels it takes to build a product like a Single Dealer Platform and these opinions are solely mine and many members of the team, past and present, will obviously ave differing opinions. The Alpha platform has had its ups and downs, as had the whole market, however I am pleased to say it is a successful and profitable multi asset platform and I am proud to have played a part in its success. That being said there are things that could have smoothed the sometime bumpy ride. However I dont feel it is fair to throw stones as many of these problems are industry wide and we have interesting issue of having two industries to deal with, finance and technology and to ease the thought process in this post I will take the question and rephrase to if you had to build an SDP what would focus on?

Form Key Relationships

Business Buy-In

The platform should be supporting the business and not conflicting with exiting goals and other teams strategies. If you have sales teams or team members that feel the product is competition you need to get them on board or you risk having to work against their negative energy. Sales and traders can have significant sway in the bank so obviously having them as allies make for a much smoother project. The opposition to new products is not isolated to these profiles. Anyone who has developed software internal in a corporation will have come up against this, so it is a common problem we must over come. Given the platform should enhance their daily dealings by being able to provide up and cross selling facilities as well as a portal-like experience into research and post trade facilities, the product should actually be a good thing for the sales desks.

Backend Systems

Be prepared - Front end systems like SDPs do not work without good service from the various providers required to price and book trades. There is more to an SDP than this, however these systems are key. Work closely with these teams - the are your life blood. When delivery times are tight you may have to call in favours from these teams. At a business level these teams goals must have some alignment with that of the SDP or their work slate will continually be serving other priorities. To have a beautiful fornt end that is not integrated is an effort in futility and will quickly raise concerns from the bill payers.

Clear Vision

Firstly know your strengths and play to them. Every bank cant not be the best in FX. Although most SDPs will have a strong FX and Fixed income feature set, trying to have the best prices and lowest latency is a costly venture. If you are going to try to take on the big boys you had better have a strong financial backer. If you are not one of those players, acknowledge it and find where your strengths are and use them. A strong product owner will be key here. His role will be to clearly understand the direction of the bank and to know what will drive the platform. He should have a clear understanding of our income sources and our user base. Defining core users is also key here. This will provide direction in the type of platform we will technically deliver : Desktop, RIA, Web, mobile or a combination should be a business decision based on the users of the platform, not based on the latest technical craze? Lastly we must have clear success criteria for each deliverable and the platform as a whole. When this is clear the team can easily all move in the same direction.

Deliver the Goods

In my opinion: Delivery is king. Continuous delivery is a logical progression of a well oiled development team. To achieve continuous delivery you should address some key, common concerns early in the project. Authentication and authorisation, handling of entitlements, security concerns, auditing, logging and other such cross cutting concerns are all usually defined by the bank at a department or group level. IMO these should be addressed in the first iteration while other team members could possibly be working on PoCs realting to functional deliverables. This is also a good time to investigate some standard deployment concerns such as the handling of independent and aggregated modules, dependency management, versioning, forward and backward compatible migrations. From a higher level architectural stance the team will also need to define how they could deal with future scalability, reliability and regional requirements. Do we have the infrastructure to provide five 9s to a global audience with latency under 10ms? If not now is a good time to let the business know so they can lead the charge on making the necessary changes if required or so they can have their expectations realigned appropriately. The reason I prefer to address this early is because the cost is so much higher when done later in the project and delivery becomes an expectation and a core team value. It also raises the notion of non functional cost to the business; fives 9's up time is not free, nor are follow the sun deployments, the sooner the business are aware of this the better for everyone.

Justify your Costs

By understanding what brings value to the business we can more appropriately prioritise our work. One major issue that I have encountered on many projects is the confusion of the prioritisation of features that generating income versus those that prevent losses or reduce the cost of doing business. Both are of value.
Features that I can see SDPs typically spending too much on are technical "play things" (new frameworks etc), performance and low value "nice to haves". A good Product owner that has a good relation with his technical leads and architects should be able to address these issues easily.

Typical neglected features are resiliency, monitoring and work related to minimising maintenance cost. No one wants to pay for any of these but we often dont realise we want the consequence of not having them even less. The unfortunate thing is we often discover this far too late. IMO the key ways of reducing maintenance cost is clean well tested code (ie employing BDD and TDD).
Another key issue in resource allocation. It is easy to think due to the large mount of money involved in building these platforms that a large number of people should be involved. For any one who has read Fred Brooks' Mythical Man-Month or has managed very large software teams will know that the communication overhead of throwing more resources at the problem will most likely only exacerbate the problem. In addition to this hiring the wrong resources can also be catastrophic. Because of this I prefer, like many agile teams, to have small self managing teams comprised of exceptional people that are completely accountable for the delivery of their feature. The cost IMO of having unaccountable people or inappropriately skilled people on the delivery team is often just too high.

Summary

The person that originally asked the question would possibly be surprised that my answer wasn't more technical as I think he wanted to hear "Use product X". I believe the tools used should ideally be representative of the way the team works and the product that needs to be delivered. Now this is not always feasible as there will be company wide mandates in place, however how that team works within those confines can often be quite impressive.

Like any project I believe a team led well, with a common clear direction and with members driven to deliver success will find a way.

A couple of books to me really resonate in this area, the Brooks classic as mentioned earlier and surprisingly to some The Art of Scalability. I originally bought this while working on the streaming price distribution components for Alpha but to be honest go more benefit in the non technical sections. The authors strongly believe a scalable system comes for a well oiled team that allows for scaling. If you are looking at starting a large scale system like an SDP would recommend this book for more than just the excellent technical read it is.

Back to the code

noreply@blogger.com (Anonymous) — Sun, 08 Jul 2012 20:31:00 +0000

The last couple of years have been a whirlwind and in my time running a relatively large part of a relatively large project I was often unable to spend time keeping up to date in any technology other than that at hand at the present time. I have finally been able to put my head up for some air so to speak and get my head back in the tech world that I feel I have turned my back on in the last 18 months. I may post more about this soon however for now it is updates on stuff I have been playing with lately.

Im quite keen to dive back into JavaScript, which would have sounded weird to me 10 years ago as it was a language that at the time I thought was just a hack fest. Having grown up somewhat and having more experience with the language it is obvious that it deserves a bit more respect than that. With HTML 5 and Windows 8 fully harnessing javascript and the acknowledgement of node.js as a legitimate backend technology, JavaScript is here to stay. As I started diving back into the JS world I realised that my knowledge of JS fundamentals was woefully lacking.

My first step back into the JS journey is getting my head around what JS actually is (vs what I thought it was). Being a prototype based language and not a class based language, it offers different ways of solving similar solutions to languages I'm more used to. This does not mean you cant use conventional OO based techniques, we know you can, but it means you may have to rethink how you could best solve those problems. Most of the change comes into the definition of the objects, not so much the use of them... well... kinda. For more good reading on JS for people with OO background have a look into this nice blog series on the topic.

One of the major steps forward in JS, as the language has not really changed, are the new APIs available. I have the fortune of having some friends that are in web based start ups that I can tap into as a source of up-to-date trends and whats hot in the "valley". A few of those things that I have picked up lately are UnderScore.js, Backbone.js and in the testing realm Jasmine and Sinon for isolation/mocking.

UnderScore is a handy utility library that provides a bunch of assistance functions especially helpfully, amongst other things, in the handling of collections. As it is a utility library its super helpful and at 1000 lines in a single file, if you really want to know how it all works, you are only an hour or two away from figuring it all out. I would certainly read through the website as it quickly and succinctly describes all its functions. Being a .Net fan-boi, Linq is dear to my heart, so I was please to see that method chaining was supported albeit not as cleanly as something like linq.js, but hey you cant have everything :).

Backbone to me is much more interesting. I have many times bumbled my way through DOM manipulation handling various client side events using JS. Backbone comes to the rescue by providing a Model-View binding approach that I am much more comfortable with (at least conceptually) and leads to clean SoC and a testable model, which is all good stuff.
Learning the basics of Backbone is not super obvious and the docs IMO are not super helpful either (I have been spoilt lately learning rails with a tonne of good docs). Fortunately the PeepCode series has been a fantastic resource with 3 video tutorials and the accompanying source code so you can follow along.

Now there is no point in saying Backbone is super testable without testing it and Jasmine with Sinon fit in nicely here. Jasmine and Sinon are covered in the PeepCode series and also in this nice blog series.
Backbone goes a long way in addressing my issues that I had with my previous "DOM manipulation" heavy approach, and Jasmine fits hand and hand with an improve testing approach. If you are used to BDD style frameworks like Cucumber or SpecFlow then Jasmine maybe for you, however if you prefer standard xUnit style testing there is always QUnit.

Sinon is also a nice fit with testing as Backbone has some convention based features that allow you to hit backend servers for model updates and persistence; Sinon steps in and saves you some pretty serious pain if you were serious testing your models over the wire.

Hopefully I will give some more feedback on these various APIs and approaches as I get a bit more experience in them, however for now it is just a link fest. Depending on how well my ventures into Backbone go I may or may not use Reactive Extensions for JS. Reactive Extension (RX) is something that is pervasive on the code base I currently work on so it is an obvious choice for me to experiment with. RX is a brilliant API for event based data streams and is something that I think will become common place in .Net in the very near future. For more info check out Intro To Rx

See y'all soon.

Links

https://peepcode.com/products/backbone-js/
https://peepcode.com/products/full-stack-nodejs-i

Agile Tools That Just Work

noreply@blogger.com (Anonymous) — Mon, 22 Nov 2010 11:23:00 +0000

Here are some nice tools that I use and generally make life easier in an agile world.

Scrumy

http://www.scrumy.com
Free kanban board for managing projects (and life).

WebSequenceDiagrams

http://www.websequencediagrams.com/
Quick way to create sequence diagrams and the input is text meaning its easy to share with other that don't have tools like Visio or EA etc. I actually prefer this to visio and our scrum team used it a lot in our last delivery.

SpecFlow

http://specflow.org/
I like Fitnesse but there is too much friction (not a lot, but enough) in terms of the development cycle (e.g. its odd source control management). SpecFlow is close enough to cucumber for me (in the .Net world) and does not rely on Ruby to be installed. Ruby installs on windows are still a complete PITA as far as I'm concerned and I'll generally do anything to avoid it. SpecFlow also runs with your existing (x/n/mb)unit testing tool of choice. I must say I didn't like this tool at first but having come back to it, either the experience is cleaner or I have opened up a bit more.

Gist

https://gist.github.com/
Like Pastie but associated to my GitHub account. I'm starting to use it more and more to do brain dumps of simple code snippets (e.g. PowerShell functions etc) that can be handy to reuse, but not really worth its own repo.

If I was running an external commercial project (e.g. at home) I would seriously consider the use of LowDown. It fits my development style very well without the clutter and bollocks associated with other ALM/project management tools. Git integration makes it very interesting too.

More AutoTesting - Powershell extensions and addins

noreply@blogger.com (Anonymous) — Mon, 18 Oct 2010 18:37:00 +0000

There is a nice library called PS-Eventing that makes the event handling a whole lot cleaner and generally a nicer experience. This post with the plugin is not a smaller and doesnt really do any else other than the last post except cleanly deregister the event, but is much more user friendly.
I think i will also post my code on gisthub as I am not and have not been at my real machine for so long... i miss it... sorry.
Heres the link : http://gist.github.com/632749

Auto test - Thanks Powershell

noreply@blogger.com (Anonymous) — Wed, 13 Oct 2010 13:57:00 +0000

Im doing some funky Nunit stuff* at the moment and sometimes the ReSharper or TD.Net runner wont pick up my inner fixtures, the console runner however runs them just fine. Here is a PS script to have auto test running on build:

$Script:initialdir = "C:\myroot\"

$fsw = new-object system.io.filesystemwatcher
$fsw.Path = "c:\mysln\myproj\Bin\"
$fsw.EnableRaisingEvents = $true

$action = {
$filter = "*Tests*.dll"
$file = $Event.SourceEventArgs.FullPath
if($file -like $filter)
{
$nunit = "C:\3rdParty\NUnit-2.5.2\nunit-console"
$currentdate = get-date -format g
write-Host "Running Tests on $file $currentdate" -ForegroundColor Yellow
&"$nunit" /xml:$_.Name-testresult.xml $file
if($LastExitCode -ne 0)
{
WRITE-HOST "Test have failed for $file"-ForegroundColor RED
}
else
{
WRITE-HOST "Test have passed for $file"-ForegroundColor GREEN
}
#$null = Remove-Event $_.EventIdentifier
}
}

Register-ObjectEvent -InputObject $fsw -EventName Changed "FileChanged" -Action $action
Register-ObjectEvent -InputObject $fsw -EventName Created "FileCreated" -Action $action

Note that you will either have to explicit unregister your event handlers or kill powershell to get rid of the handlers. Cancel the watcher by hitting ctrl+c and then Unregister by runnning these lines

unregister-event "FileCreated"
unregister-event "FileChanged"

*The stuff we are doing is to do with inheritance and inner test fixtures to give us a BDD experience without the drama of the BDD frameworks, that personally I still dont like. We are still a fair way off the TDD experience of the Ruby kids.

Threading - I am on a Journey

noreply@blogger.com (Anonymous) — Tue, 07 Sep 2010 17:23:00 +0000

Recently it has become more and more apparent that I had a crack in my understanding of a key aspect of not only the CLR but computing in general. No prizes for guessing that it was threading and the general parallel programming constructs that go along with it.
Now I am not a complete n00b in the threading world, I have written code that uses threads in client and server side but it was typically handing off a single task to a single thread and looking back it was probably pretty delicate and by now in need of some re-factoring love.
The first real kick in the pants was playing with my 8 core machine and never seeing all 8 core light up in the task manager. How wasteful... its like all the Aston Martins I see driving over Tower Bridge; what is the point?
Next major trigger for me was a computationally heavy project that I, up until recently, was working on. My colleagues and I knew that splitting up tasks could make or break this system, however given a distinct lack of human resources and not exactly compelling hardware, time was never allocated to fleshing out the best approach.
Well I am no longer at that job (its somewhat unfortunate as the project could have been epic had it not be butchered by red tape and ego's) so I have had time to scrub up on my threading knowledge... well I thought I would just be scrubbing up. Sometimes you just don't know how much you don't know.

So in an attempt to short cut any one else's learning curve on the wonderful world of threading and parallel programming in the .Net world here are some great places to get started.

As always C# in a Nutshell is a great place to start with anything C# related. I have both 3.0 and 4.0, Joe is a great author; IMO every C# developer should have the latest copy of this book on their desk.

Next up the deceptively good CLR via C#. This book looks like its going to be as much fun as having teeth pulled out however it is surprisingly good and dives deep in the areas you want and would expect. I read the 2nd edition however will be buying the 3rd as soon as I can find it.
Obviously these books are not completely dedicated to threading but they have some great chapters dedicated to it. If you are after a more in depth reference I have been recommend Joe Duffys book : Concurrent Programming On Windows

Next up is my favourite learning tool : Tekpub. There is a threading production under way, it only has 1 episode but it is great. As I have a yearly subscription it was a no-brainer. For me this was valuable as it has a bunch of code and a great visual comparison of auto and manual reset events, Semaphores and Mutexes. I highly recommend this video if these are confusing you at all.

For a more casual and much briefer talk on Threading in the pre .Net 4 days check out Jon Skeets articles . I have found the multiple angles these guys talk about the same topic good for bashing it into my head.

The piece that I am currently reading is the PnP new book Parallel Programming with Microsoft .NET that is also available as a hard copy book and eBook. Code snippets for the text are available here.

The last piece that is semi related is my reading of the Manning book Real-World Functional Programming. Although functional programming technically has no direct link to threading its basic tenants strongly support parallel programming and as such I have been endeavouring to improve my understanding and use of functional constructs where it is appropriate. So far the books is very good it is in F# and C# so it has bee a good (re)introduction to F# for me.

With all of the movement in not only hardware (hyper-threaded and multi-core processors) as well as architectures (Grid, Cloud, ESBs etc) and the software frameworks we use (.Net's TPL, Rx etc) and how we use them (i.e the movement towards functional programming in distributed/parallel systems) I think it is becoming very clear that as .Net developers we need to know how this stuff works, how we can use it and what it means for the systems we build and the users that use them.

As tempting as it may be to run off and just blindly use the new TPL I strongly urge you to get a good understanding of the history and the older APM practices, I am sure this will ensure you will make better choices when you write your next parallel task.
As for me I'm back to reading, I have a long way to go...

Getting started in Asp.Net

noreply@blogger.com (Anonymous) — Mon, 26 Jul 2010 04:55:00 +0000

I recently met a guy who was new to town and had picked up a job doing development for a place that appeared to be moving towards Microsoft focused solutions meaning he was likely to move away from his LAMP (Linux, Apache, MySQL, PHP) background to a .Net/Sql Server environment.

I realised after talking to him that there are a lot of ways you could go wrong coming into the .Net realm from a LAMP/Java/Python/Ruby background and this post, although intend for him, is generic enough that it may save other web developers valuable time to get up and running, and producing valuable software using the M$ stack

Asp.Net

Asp.net is a different beast to the many other frameworks out there and you could spend years (and many have) learning all the ins and outs of the beast.
Dont.
I would highly recommend skipping WebForms all together as IMO it is a dead framework. I have been doing MS MVC for the last year and did MonoRail (an open source ASP MVC framework) on the side prior to that and I have never looked back. Some of the significant benefits are :

Matches the way the web works (you explicitly work with http verbs)
clean html
easy to learn
pluggable
testable
simple
And finally, IMO, encourages better coding practices, which web forms simply does not.

I will assume you will trust me and accept that MVC is the way forward and I will go ahead and recommend how to start.

Asp.Net MVC in action
First port-of-call is to pick up this book. It will give you the no BS version of the truth and covers some third party tools that will help you along the way. There is a second edition on the way however if you are using VS 2008 then this books is for you anyway. Manning (the publisher) often have good discounts so register for emails or follow them on Twitter to know when promos are running.

TekPub : Build your own blog with MVC
Tekpub is one of my favourite resources at the moment. It is great value for money and video is an efficient way to learn new concepts. I had bought several series and now have the yearly membership and many of my colleagues have done the same. Go check out the BYOB which shows how Rob* built a blog in MVC. There are also MVC 2.0 and various other Asp.net series available. Buy one, buy a bunch or get a month or yearly subscription, go on, you know you want to ;)
*Rob was on the team that actually built Asp.net MVC

Asp.net
Yes the website is a good place to get lots of info too! There are lots of little articles and videos from from smart people who live and breathe this stuff. Just be aware that there is sometimes a very Microsoft slant to the information, not a bad thing but something to be aware of.

Next there is the Database

SQL Server

If you have a background in Oracle or MySql then picking up M$ Sql Server should be pretty easy. The standard practices apply e.g. dont run your web app as System Administrator. Nothing to really say here to be honest.

Joining the web and data world is your data access strategy

Linq2Sql or NHibernate

The 2 standout choices for new commers IMO are the light weight data acess strategy from Microsfot called Linq2Sql (pronounced "Link to Sequel", just dont ever write it like that) or the OpenSource and more heavy weight NHibernate. If you are from a Java background NH may be a good tool, if you are from a Rails background then Castle ActiveRecord (which sits on tope of NH) may even be a good idea. However for most Linq2Sql is probably the best bet when you are getting started and NH is the best if you are building up more complex domains. The caveat with NH is that there is a steep learning curve. If you are new to it then check out SharpArchitecture or one of the abstractions in the NHforge.com to help bootstrap yourself. If you have a simple domain or going to be building from the database up (i.e. create a database table and then the c# code to use it) Linq2Sql is a better fit anyway.

Tools

If you are playing with the M$ stack at home then you will most likely not want to shell out cash for the full blown enterprise versions of Visual studio and Sql Server. For this reason they have the express versions. I have only ever used the SQL express edition (which is great) so have no idea what VsExpress is like. In saying that if i did not have the full version of VS I probably would not be using .Net and just be doing rails/sinatra dev (but thats just me).
Assuming you are working with the full version at work then be sure to invest in the Resharper VS plug-in. Its not super cheap but the time it will save you will mean it will quickly pay for itself.

I terms of source control TFS is the M$ standard, but I personally don’t recommend it. I think tools like Git or even SVN are better tools that integrate better with other platforms/tools and are free. TFS however does more than source control, so if you are using make sure you use all the good ALM bug tracking bits etc

Extra links

Follow the team behind the framework, most importantly Scott Gu (all .net dev should read his blog) and Scott Hansleman. Mr Hanselman also has a good podcast called hanselminutes that is not Asp.Net specific but a good subscription nonetheless.
StackOverflow is a great QnA community place for lots of developer question. The are lots of .Net and Asp.Net people hanging out here so most of your initial question will most likely have already been answered. The site itslef is even built on top of Asp.Net MVC and Linq2Sql so you can see that this combination is a very real world and scalable solution.

If you are in Perth (like this person was) check out the Perth .Net Community of Practice and the Alt.Net lads that have monthly beers and discuss things that may or may not be from the Microsoft mother ship that affect .Net devs.

Perth Alt.Net - RavenDb

noreply@blogger.com (Anonymous) — Thu, 24 Jun 2010 02:25:00 +0000

Last I did a "talk" on RavenDb at the Perth Alt.Net beers. I must say I was really stoked at the turn out. Lots of banter, comments, queries, theories and general discussion. I sincerely hope that everyone else enjoyed it as much as I did. I also felt that last night was moving closer to what I hope these events should be. These shouldn't be one-way dialogue. IMO the presenter is there to just start discussion and then it becomes a group affair, there should be lots of interjecting, lots of crowd participation. Last night there certainly was that and best of all it stay on track, we didn’t veer off into off-topic subjects (much) and I think the night was better for it.

I really do feel lucky to be in the company of some very smart people and i want to thank everyone for letting me bleat on and not give me too much grief for being woefully underprepared.

However I put it out there that the people who are comfortable enough to ask the hard questions (and there were a few), please take the reins, we want to hear from you! I’m a strong believe that you can learn as much about some from the questions they ask as the questions they answer and there were some gems last night.
Please contact Mike (@wolfbyte) or myself if you are thinking you may be interesting is starting a discussion, or even if you want to hear about stuff especially if it is edgy and possibly not suitable for the standard M$ .Net community of practice stream (Mitch wheat is the contact for that if you do have M$ specific stuff however). Any feedback is good. The hardest thing for organisers is trying to figure out what the crowd wants and as we nerds are not the most vocal people, making it even harder to run nerd events.

A further incentive is we now have sponsorship. If you get up and present you get swag, swag you want. I worked out since I have been back in Perth I have received several thousand bucks worth of cool stuff from presenting or attending various .Net events. It’s worthwhile getting involved!

Garry Stewart is up next time (in about 2 weeks) talking about light wieght IDEs, specifically VIm and letting go of our addiction that is Visual Studio.

One big note I also want to make. Take the talks for what they are. The whole under current of Alt.Net is "options". We need to be aware there are options. We are not saying you must throw away SQL and use RavenDB exclusively or Vi will replace Studio, we just want people to open their minds. If last nights talk got you thinking about how you could get you python web stck talking to cassandra, sweet, most of what we are talking about is concepts, if there is a tool involved, its just there because it facilitates a different way of thinking. Thats all.

See you next time :)

DDD eXchange 2010 - London

noreply@blogger.com (Anonymous) — Thu, 17 Jun 2010 04:07:00 +0000

Un fortunately i dio not attend this event (which looked like an awesome event) however the Skills matter guys were good enough to post the videos of the event... sweeet!

DDD EXCHANGE 2010 - PODCASTS, SLIDES AND PICTURES

Thye have recorded all the DDD eXchange 2010 talks. The slides and podcasts can be viewed here:

You can also find some photos taken at the conference here.

Thanks SkillsMatter!

Test Fakes - Solving a Domain Entity Issue

noreply@blogger.com (Anonymous) — Fri, 11 Jun 2010 02:32:00 +0000

Follow on from :Testing with Domain Driven Design

Background on the design in question

We are using NHibernate for persistence on Oracle.
We have service that accept command that are coarse grained and perform a clearly defined business function, they are not chatty services.
We are using the notion of aggregate roots so all child objects are created and managed by the owning aggregate root. If you need to access the child it must be done via the AR
The domain is relatively large and complex.

Background problem

We had issues where we were dealing with domain objects in test and getting problems with interrogating child collections. For example we sometimes need to be able to update a child of an aggregate root and we were using the id of the child (within our command object) to indicate which child to update. We are using a basic database identity field (oracle sequence) for setting the ids on all of our user defined entities.
Herein lies our problem;* In our test we create an aggregate root complete with child objects. We then want to test, for example, updating a child of the aggregate root via a DTO based command (i.e. using an id to define the child to update) and we run into issues when all of the child object have the same id of 0 as they are not persisted (it’s a unit test). Now this would never happen in real life. Why would you issue a command to update something that has not been persisted, how do you even know about that object?
The quick solution that I have seen used a lot is to set the ids of all of the domain objects prior to running the test. I don’t like this if you are doing this by exposing the ID setter on the domain object. This is opening up the API of our domain for testability and is potentially the start of a slippery slope in to a pit of hell. An easy way around this is to use fakes. These object are just child object of the domain objects in question that help expose stuff the domain shouldn’t; in this case the ID setter.
The other alternative is to set the id on creation of the fake so the setter of the id is not exposed. This can also work but it will mean your fake will always have an id set. For the situation I was in this was not suitable.

The end solution

All domain objects have a fake create for it. All of the fake implement an interface ISettableId. This interface is defined as below:

**CODE GOES HERE**

public interface ISettableId
{
    bool HasIdBeenSet();
    void SetId(TId id);
}

With an implementation example (its id type is and integer) :

public class FooFake : Foo, ISettableId
{
    public FooFake(string name, TypeRequiredForValidConstruction myDependecy)
    : base(name, myDependecy)
    {}
    public bool HasIdBeenSet()
    {
        return _id != 0;
    }
    public void SetId(int id)
    {
        _id = id;
    }
}

This mean we can now create fake objects and emulate persistence later by reflecting down the object graph and setting all of the ids. This is much faster than hitting the database and has proved to be a very valid exercise as we can now run tests again transient and persisted versions of the object graph without having a db connection.
One additional thing I should mention is that the creation of child object now must be exposed as a hook. For example when we create child objects I do not just new up to object and add it to a collection. I call a protected virtual method that creates the child object and then add that to the list. This allows my fake to override the return type that is added to the list so children can also be fakes. This has not increased the exposure of the domain but has now facilitate a hook to allow me to create fakes for my child collection.

Caveats:

Fakes should not include any logic. The fakes I am using only allow the setting of ids. The only logic they have is to check whether the id is set (this allows us to skip objects that have had their ids set when we walk the object graph and not go around in circles). The only other methods that are required is the overrides of the creation of child objects.
Other methods you may find in the fakes are builder methods. Instead of having big factories for creating test object you may choose to put the object creation closer to the owner by putting it in the fake class itself.

Summary

As I mentioned, we have a pretty large and complex domain we were running into issues where the difference between persisted and non persisted objects graphs was becoming problematic; using the fakes has allowed us to keep the domain clean and only as open as possible while allowing us to emulate persistence without the cost of hitting a database. Over all I’m pretty happy with the result.

*I’m sure there are other way to solve this higher level problem

Alt.Net, Openspaces, whatever its coming...

noreply@blogger.com (Anonymous) — Thu, 27 May 2010 07:44:00 +0000

Google groups and Twitter has been busy today after I seemed to have started some hype about an Alt.Net styled openspaces event (#ozopenspace)

PLEASE spread the word. We need ideas when people can come and where they would go to (Australia is a big place, we are all very spread out)

The wiki looks like its going to be based here: http://ozopenspace.pbworks.com

Register you interest here. Seriously please do this otherwise we can not estimate or cater for your needs/wants etc. This will also show sponsors if we are serious or not and may help convince overseas guest to attend too, of which there is already interest :)

The discussion is on the oz alt.net google groups here

I cant wait!

Powershell and the web

noreply@blogger.com (Anonymous) — Wed, 26 May 2010 15:16:00 +0000

Lately I have been playing with PowerShell to see if it can help with some uptime/availability checks for our web sites and services at work; Yes i know what you are thinking but I'm not a sys-admin and I need to know if my stuff is working with out having access to the server... long story I'm sure you will appreciate.

In my reading I have seen lots of flaky posts about PowerShell and web services. I think people need to remember that web services still conform to the http protocol, there is no magic, you do not need to access any visual studio dll or do any weird crap like that, you just need to hit the url and receive a payload.

    $wc = new-object system.net.WebClient
    $webpage = $wc.DownloadData($url)

That's it. No weird proxies no importing custom dll's.

Here's an example calling the Google Maps API (which is a nice API BTW) to get a latitude and longitude of an address, specifically Perth's craziest bar Devilles Pad (think Satan living in a trailer park that hangs out at a Go-Go club that used to be a James Bond-villians liar, yeah its that cool:

    $url = "http://maps.google.com/maps/api/geocode/xml?address=3+Aberdeen+St,+Perth,+WA+6000,+Australia&sensor=false"
    $wc = new-object system.net.WebClient
    $webpage = $wc.DownloadString($url)#Edit -Thanks Ken
    $xmldata = [xml]$webpage
    $lat = $xmldata.GeocodeResponse.result.geometry.location.lat
    $lng = $xmldata.GeocodeResponse.result.geometry.location.lng
    write-host "Latitude = $lat - Longitude = $lng"

Like that? Go play: Google Maps API Doco

MSBuild 101

noreply@blogger.com (Anonymous) — Fri, 07 May 2010 08:40:00 +0000

Lately for some reason I have seen a fair bit of tension around MSBuild. I guess this is good thing, more people are using build tools like PSake, Rake etc and most of the time if you are building .Net stuff sooner or later you will have to call into MSBuild, unless your feel bold enough to uses csc... um... not for me.

MSBuild is, to be honest, very basic.

There are typical 2 things that I use MSBuild for

-Defining a build script

-Running a build

The term MSBuild is a little confusing to some as its covers a few things: a file type and an executable.

Firstly we need to be aware that all .Net solution files (*.sln) and project files (*.*proj) files in .Net are MSBuild files; that should hopefully ease some tension as it is now pretty obvious that you are already using MSBuild! The MSBuild files are XML and therefore human readable and modifiable. The files typically define properties and tasks. The properties can reference other properties and they can be collections too. The tasks can have dependencies on other task (i.e. Test requires Build which requires Clean) and the task can make use of our properties we have defined in our script as well as define items that are basically task level variables. This is starting to sound just like procedural programming, which I hope we are all au fait with. It is probably also a good time to note that pretty much all build tools use the notion of dependencies. This basically means if you call a task and it has a dependant task then that task will be called first. You can have chained dependencies like the example we mentioned before (i.e. Test requires Build which requires Clean) and you can also have multiple dependencies for one given task (Deploy Requires Package, Document, Test).

After reading that it is quite clear that MSBuild is actually very simple. It is the syntax and general XML noise that scares most people, including myself.

Running MSBuild

To achieve the most trivial usage out of MSBuild we need to know how the executable works so we can actually call MSBuild to do something. MSBuild comes with Visual studio, I don’t think it actually comes with the .Net framework (someone can correct me here) and it is also bound to a framework version.

To start with I will show the most trivial example of how to use MSBuild and that is to just build a solution.

@C:\Windows\Microsoft.NET\Framework\v3.5\MSbuild.exe MySolution.sln

Assuming that this command is run in the directory that MySolution.sln resides this will build that solution. It can’t get much easier than that. Note this will not do anything clever, it will just build to your default location ie bin/debug for each project in the solution. Personally for me this offers little value other than it is a bit faster than building in visual studio.

Typically if I am building using the command line it is because I am building from a build tool. Build tools like PSake allow me a lot more flexibility as they are not constrained by the bounds of XML and have powerful functions I can use that may be associated to builds and deployments that might not otherwise exist in MSBuild. If you are using a tool like PSake or Rake as your build script then it is more likely that you will use the following syntax:

&$msbuild "$solution_file" /verbosity:minimal /p:Configuration="Release" /p:Platform="Any CPU" /p:OutDir="$build_directory"\\ /logger:"FileLogger,Microsoft.Build.Engine;logfile=Compile.log"

This is from a PSake script so anything with a $ in front of it is a PowerShell variable (i.e. not MSBuild syntax). Walking through this I have defined the location of the MSBuild exe ($msbuild) and the sln file. Note that the sln file is note associated to a switch. It is the first argument and this indicates that this is the file we are building. Other arguments are prefixed with a switch. These switches begin with a forward slash and end in a colon and contain either full descriptive words (e.g. /verbosity:minimal) or short hand syntax for a word (e.g. /p:Platform="Any CPU" which is short for property and in this example defines the platform property)

FYI : The & at the start of the line is a PowerShell construct to say "run this command, don’t just print it to the screen"

Defining Build Scripts with MSBuild

First up we will walk through a very basic build script that justs cleans and builds a solutions. It will be pretty obvious that this scripts really offers little value but we will get to some of the nitty gritty stuff soon.

<ApplicationName>MySolutionApplicationName>

<SolutionFile>..\$(ApplicationName).slnSolutionFile>

PropertyGroup>

Target>

Target>

Project>

Let’s go over this line by significant line:

The xml documents root is the project where we define the xml namespace, the MSBuild Tools version and the default target. Make sure you tools version work with your .Net framework version. This first line is pretty standard for me. I like having a default target and im typically working with .net 3.5 at the moment so this is all good.

Next up we are importing an external extension. This is an example of importing the very useful MSBuild Community Extension toolset. If you are making use of MSBuild a lot then you will want this toolset. Another Import I often make is for Gallio a very handy test runner. Please note that this import demonstrated is not being used in the script so could safely be removed, it is just here to show the syntax

Next we define our property group section. The nodes in here are user defined. MSBuild does not have “ApplicationName in its schema, i made that name up. These properties are in effect you global variables. You will also note that you can reference other Properties from with properties as shown in the solution file property. This is often very handy when dealing with file paths. Then Syntax for referencing a single properties is $(MyProperty)

Next up we define our first target. Targets are just like methods in that they define functionality. The Clean target just calls an MSBuild task that cleans the solution. Not very impressive, I know. The next target shows how we can specify dependencies. This means any time we call the Build target the Clean target must run first.

Having a build file is all very nice but we need to be able to run it. As it is not a solution of project file I tend to treat them a little different to how I would call those files directly. I typically prefer to be explicit in calling target, even though it is a DRY violation.

I have an example extracted from a bat file that calls into an MSBuild script below to show you the syntax (this is the same syntax as if you were to run from the cmd line):

@C:\Windows\Microsoft.NET\Framework\v3.5\MSbuild.exe MyMSBuildFile.build /t:UnitTests /l:FileLogger,Microsoft.Build.Engine;logfile="UnitTests.log"

From this you can see:

-I am calling MSBuild 3.5; MSBuild is slight different for each .Net version, be sure you are not calling the .Net 2 version for your new shiny .Net 4 solutions!

-MyMSbuildFile.build is in the location I am running this command. If it was not I would need a relative or full path.

-The extension of an MSBuild script is not important. I personally prefer to call my MSBuild files *.build to make it clear that they are in fact build scripts

-The "/t:" switch defines the target I am calling: UnitTest. You do not need to specify this if you have defined the Initial Target in the project tag in you build file. I recommend defining a default target and make sure it is a safe one... you don’t want to accidently deploy something to production "by default"

-By using the "/l:" switch I can define a logger so I don’t lose the info once the console window disappears. I pretty much only use this syntax and change the name of the output file.

For more help you can just call the MSBuild exe and use the /? for a detail example of what the exe can do. I recommend doing this :)

Stepping Up

So everything I have shown you so far is all well and good but it is of little value in the real world. So here is a quick fire list of problem and MSBuild solutions, a cheat sheet of sorts:

Note the command line syntax used below is from PowerShell and to get the MSBuild variable assigned I have called:

[System.Reflection.Assembly]::Load('Microsoft.Build.Utilities.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') | Out-Null

$msbuild = [Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToDotNetFrameworkFile("msbuild.exe", "VersionLatest")

Change a property from outside the script

You have a property, say application name defined in a script, you want to reuse the script but not have to edit the script for everything that uses it.

Solution : use the property switch

&$msbuild "$build_file" /p:ApplicationName="RhysCNewApp"

Directories with spaces are failing when passing them to MSBuild

If you are passing in paths from a build tool that requires you to use quote (e.g. the path has spaces in it) then MSBuild may throw its toys and blow up on you. To dodge this use double backslashes (\\) at the end of the argument to escape the nastiness that is MSBuild

&$msbuild "$solution_file" /p:OutDir="c:\Rhys Hates Spaces\In His\ Directory Paths\"\\

Inside your Scripts

Display a message to the console

I want to display the paths of my Visual Studio tools folder for ‘05 and ‘08. Note VS80COMNTOOLS and VS90COMNTOOLS are environment variable, so yeah MSBuild can pick those up too J

Target>

I want to reference a whole Item group

Sometimes we have collections of related items and want to reference all of those items:

ItemGroup>

Target>

This will produce:

Foo;Bar;Baz

I want a conditional execution

I want to delete a folder, but only if it exists

<RemoveDir Directories="$(OutputDirectory)"

Condition="Exists($(OutputDirectory))">

RemoveDir>

Target>

This show the use of Exists() and the MSBuild command RemoveDir.

Error Messages

I want to display a meaningful error when an exception should be raised in my task. Solution use the Error command within you task

<Gallio

Assemblies="@(UnitTestProject)"

IgnoreFailures="true"

EchoResults="true"

ReportDirectory="$(ReportOutputDirectory)"

ReportTypes="$(ReportTypes)"

RunnerType="$(TestRunnerType)"

ShowReports="$(ShowReports)">

Gallio>

Target>

This examples show the use of the Gallio MSBuild task and how we can raises an error if the exit code is not 0 was the command is run

Conditional Properties

I want properties to have value dependant on conditions:

Html-Condensed

ReportTypes>

PropertyGroup>

Swap out Configuration files

This is last as it is a personal preference on how to manage environmental differences. I basically have all my configuration files in one directory and have N+1 files per file type where N is the number of environments. I may have a ConnectionString.Config and right next to it will be ConnectionString.Config.DEV, ConnectionString.Config.TEST, ConnectionString.Config.UAT, ConnectionString.Config.PROD with each environments values in the respective file. My build only acknowledges the ConnectionString.Config file and it copies that one, the rest are not included in the build. So when i build for an environment I swap out the config files as part of the build. A contrived example of how to do this is below:

ItemGroup>

<Copy SourceFiles="@(SourceFiles)"

DestinationFiles="@(SourceFiles->'$(ApplicationConfigFolder)\%(Filename)')" />

Target>

This grabs all of the files from the Application Config Folder that end in .DEV and copies them to the same location without the DEV.

Explaination: The %(Filename) means I get the filename with out the extension. MSBuild basially just removes anything after the last “.” in the file name, this means I over write ConnectionString.Config with the contents from ConnectionString.Config.DEV. Its handy, it works for me, take it or leave it.

Creating your own MSBuild Task

This was way easier than I thought it was going to be. To create a task you just need to inherit from Microsoft.Build.Utilities.Task, override the execute method and add some properties. Here is a SQL migration build task I built (please excuse the logging and the IoC Cruft). The code to do all the leg work was already done and it had a console to fire it up. At the time I was using MSBuild to call it and i was using the exec functionality of MSBuild when I thought: I could probably just make a plug in for this! Below is the result

using System;

using Microsoft.Build.Framework;

using Microsoft.Build.Utilities;

using Microsoft.Practices.ServiceLocation;

namespace SqlVersionMigration.MsBuildTask

{

public class RunAllScripts : Task

{

[Required]

public string ScriptLocation { get; set; }

[Required]

public string ConnectionString { get; set; }

public override bool Execute()

{

try

{

ContainerRegistration.Init();

var versionMigrationService = ServiceLocator.Current.GetInstance<VersionMigrationService>();

TaskHelper.LogEntry(Log, ScriptLocation, ConnectionString);

var result = versionMigrationService.RunAllScriptsInOrder(ConnectionString, ScriptLocation);

TaskHelper.LogResult(Log, result);

return !result.Isfailed;

}

catch (Exception ex)

{

TaskHelper.LogError(Log, ex);

return false;

}

Some hints to get the most out of MSBuild:

Download the MSBuild Community extensions, there is lots of good stuff here.

Use plugins where it is a logical idea as opposed to calling out to exes. This can be done by using the import tag at the start of your file:

Know when to use MSBuild. It is a good starting point if you are learning about automated builds, especially as you have to use it every day (if you are a .Net dev) whether you like it or not (remember all sln and proj file are MSBuild files). Be warned though that it is not really the best option for stuff out side of a pure build. Once you are comfortable with MSBuild you will soon hit a very low ceiling. At that point investigate other tools like PSake and Rake. Due to the XML nature of MSBuild certain task are just messy or impossible to perform; Loops is a good example, just don’t do it.

I hope that is enough to get you up and running, if not let me know and I will try to fill in any of the gaps

Castle - Reborn!

noreply@blogger.com (Anonymous) — Tue, 27 Apr 2010 15:36:00 +0000

It looks like Castle has had a shake up! I don't *know* who is behind all this but I know my my castle go-to-guy has his sticky fingers in it and I'm glad he does!
Of late there is a new Castle Wiki and thanks to Andy Pike a bunch of new screen cast dedicated to getting up and running with the various aspect of the Castle framework.

So the links are:

Current site which may get replaced in the future
New Castle wiki and general go to place
CastleCasts

Please check them out and if you want to add anything i have been told that the wiki is a real wiki, so contribute!
Thanks to Krzysztof Koźmic for the heads up :)

*OK so Castle is not reborn, just the doco ;)

Soft Deletes in Nhibernate

noreply@blogger.com (Anonymous) — Tue, 27 Apr 2010 15:06:00 +0000

The following was an email to my dev colleagues, sorry about the code formatting Im using the Blogger web interface (OSX really needs a WLW equivlient!):

Fellow nerds;
There are frequently requirements that we need to have soft deletes. These things are a pain, especially with an ORM as they can start becoming invasive to your domain code.

If you want soft deletes to be transparent as possibly in normal workings of the domain you can use a DeleteListener to intercept the deletion and, well, soften it.
Below is some code to show a parent and child relationship (ParentSampleDomainObject and SampleDomainObject).

Key points:
• The entities need to have a flag to show they are deleted eg IsDeleted
• The mapping on the parent needs to say that it is not interested in retrieving deleted items by using a where clause
• The mapping also need to say it want to delete orphan children, (we won’t actually delete them, but this needs to be here)
• We need to have an NHibernate delete listener to intercept the cascading delete and instead of deleting the entity mark it as deleted. This is why we need cascading deletes in the mapping
• We need to register the listener to the config.

The test code will be checked into the core soon.

Rhys

Nhibernate Configuration:

<?xml version="1.0" encoding="UTF-8"?>
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2">
    <session-factory name="applicationregister">
        <property name="dialect">NHibernate.Dialect.MsSql2005Dialect</property>
        <property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
        <property name="connection.driver_class">NHibernate.Driver.SqlClientDriver</property>
        <property name="proxyfactory.factory_class">NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle</property>
        <property name="connection.connection_string">Data Source=.\SqlExpress;Database=CoreTests;Integrated Security=True;</property>
        <property name="connection.isolation">ReadCommitted</property>
        <property name="default_schema">CoreTests.dbo</property>
        <mapping assembly="RhysC.NHibernate.Tests"/>
        <event type="save-update">
            <listener class="RhysC.NHibernate.AuditListener,RhysC.NHibernate"/>
        </event>
        <event type="save">
            <listener class="RhysC.NHibernate.AuditListener,RhysC.NHibernate"/>
        </event>
        <event type="delete">
            <listener class="RhysC.NHibernate.SoftDeleteListener,RhysC.NHibernate"/>
        </event>
    </session-factory>
</hibernate-configuration>

Parent Domain entities mapping file:

<?xml version="1.0" encoding="UTF-8"?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" namespace="NHibernate.Tests" assembly="NHibernate.Tests">
    <class name="ParentSampleDomainObject" table="ParentSampleDomainObject">
        <id name="Id" column="ParentSampleDomainObjectId">
            <generator class="identity"/>
        </id>
        <bag name="Children" inverse="true" lazy="true" cascade="all-delete-orphan" where="IsDeleted = 0">
            <key column="ParentSampleDomainObjectId" foreign-key="ParentSampleDomainObjectId_FK"/>
            <one-to-many class="SampleDomainObject"/>
        </bag>
    </class>
</hibernate-mapping>

[Serializable]

public class  ParentSampleDomainObject : BaseEntity<int>

{

    public ParentSampleDomainObject()

{

        Children  = new List<SampleDomainObject>();

}

    public virtual IList<SampleDomainObject>  Children { get; set;  }

    public virtual  void AddChild(SampleDomainObject  child)

{

        Children.Add(child);

        child.Parent = this;

}

    public virtual void RemoveChild(SampleDomainObject  child)

{

        Children.Remove(child);

}

}

[Serializable]

public class  SampleDomainObject : BaseEntity<int> , ISoftDeletable

{

    public virtual string Name { get; set; }

    public virtual bool IsDeleted { get;   set; }

    public virtual  void MarkAsDeleted()

{

         IsDeleted = true;

}

    public virtual ParentSampleDomainObject Parent { get; protected internal set; }

}

public class SoftDeleteListener  : DefaultDeleteEventListener

{

        protected override void DeleteEntity(IEventSource  session, object entity,

            EntityEntry  entityEntry, bool isCascadeDeleteEnabled,

            IEntityPersister  persister, ISet transientEntities)

{

            if (entity is ISoftDeletable)

{

                 var e = (ISoftDeletable)entity;

            e.MarkAsDeleted();

                if(entity is IAuditableRecord)

{

                     var a = (IAuditableRecord)  entity;

                     AuditListener.SetAuditInfo(a);//need to have a log of when this was actually deleted,  probably the intent of the soft delete!

}

                CascadeBeforeDelete(session, persister,  entity, entityEntry, transientEntities);

                CascadeAfterDelete(session, persister,  entity, transientEntities);

}

            else

{

                 base.DeleteEntity(session, entity,  entityEntry, isCascadeDeleteEnabled,

                    persister, transientEntities);

}

}

}

Begining with the end in mind

noreply@blogger.com (Anonymous) — Tue, 27 Apr 2010 14:19:00 +0000

A long time ago, what feels like a previous life, I used to train bodybuilders. Yeah, weird, I know. I actually learnt a lot from this subculture: discipline, sacrifice and hard work are things you can not escape in that life. One huge lesson I picked up that many missed was "Begin with the end in mind". The art of bodybuilding is often confused with weight lifting. The number of competitors that complained that a physically weaker contestant beat them amazed me. Being able to bench more than your opposition counts for nothing.
In the sport of bodybuilding the winner is decided by a panel of judges; that's right, you are judged by humans. It is the image you present that they must judge you on. You may even have a better physique but if you do not display it better than the others you can lose. For this reason all of my athletes posed at every training session. In the middle of the gym, in front of mirrors down to their underwear, unashamedly posing as if they were in front of a panel of judges. We would critique, take photos, film it, change the lighting... I never saw any other body builders do this. I am sure that why, collectively, they won several, national and international championships.

Now ask yourself: Are you training for game day?
For us this is deploying to the production environment.
Are you practicing it everyday?
You should be.

If you are doing manual deployments you are a weight lifter in a bodybuilding show. You will lose.
To be honest we have it better than body builders. We have no competitors and we have predefined requirements. We can measure our performance, they cant, they can only contrast and compare.

So what do we need to do to prepare for the big day? Well first and foremost don't make it a big day. Make it just like every other day. Make deployments part of your daily routine and start deploying from day one. Personally I like to have automated deployments working prior to writing any business code. Infrastructure cruft should be done in iteration zero and deployments are infrastructure cruft.

A daily routine makes deployments so easy the are a non event. This usually mean defining everything you need to do to do a deployment then scripting it - Writing the best code in the world means nothing if the deployment is botched... and manual deployments get botched.

Deployments should also cover all the steps you will do on production deployment day, they do not cover just what needs to get done to make the application work on a developers box.

Define where you production environment is at and then work from there. If your testing environments are not the same, you should question: why? The more similar they are the less likely you are of having deployment issues. Where I currently work we have 5 environments

Developers machine
"Development" Environment - automated CI deployment
Test
UAT
Production

The developer machine I consider an environment. I often get latest and (sometimes) hit F5 and run the application and I expect the application to work! This means I require a database that is in a valid state, external services to work, file paths to be correctly set up etc. For this reason I like developers to have local sandbox which includes local databases. Nothing pisses me off more than when I am running an app and some bugger has cleared the dev database for testing a script breaking my flow. Having your own database also forces you to properly script database changes in a sane manner. Checking in those changes and getting latest, running you build script should get you up and running every time. See Tarantino from Eric Hexter or RoundHouse from the ChuckNorris framework for a simple way to get database migrations working cleanly in a .Net-SQL world.

The Development Environment is a name sake. No development is done on it, but it is owned by the developers. We may use this for debugging on a production-like environment if things go pear shaped, I just never have. Its main purpose is for, IMO, automated deployments from our build server. If anything breaks here the red light on the build server goes on and the check in for that build fails. The steps to do this include

Cleaning the server - ie getting rid of the last deployment and backing it up
Setting up the server including

creating apps pools/virtual and physical directories,
ensuring dependencies are present eg MSMQ, DTC etc
ensuring the dependencies can run, ie queues are set up, services are running
setting up account privileges

Deploying the packages to the server and installing it if applicable
Running SQL scripts including

creating users roles and permissions
creating the database objects (tables, views, keys constraints, triggers etc)
creating required reference data

Testing the deployment

creating test data
running high level acceptance and smoke tests

If you can get to this stage then is it not obvious that doing a test deployment is going to be next to trivial? Migrating to the test environment should be the same as migrating UAT and therefore the same as Production. Production deployments should therefore be just a matter of going through the motions.

This also means that you may need various scripts or at least functions with in these scripts to carry out these various steps. Obviously if the Production environment is already set up we do not need to do it again, and the deployment scripts should reflect that. Just like in normal code use pre-conditions and post-conditions to enforce a correct deployment. If certain set ups are not required log it and move on, just make sure it is part of the agreed process.

DBAs are involved and decide how you want to manage your deployments. Keep reminding the team that this should be streamlined.
One thing that often trip up teams is permission issues. Personally I prefer not having access to anything outside of the development environments (I'm pretty sure I am alone on this one). As far as I am concerned the testers can deploy there own stuff. It will be the same script that the SysAdmins and DBAs will run in UAT and production, why should I do it? I have code to write! They can have permission to run the scripts in their own environment making sure that no manual step has been introduced by any developer along the way. This separation I feel further reduces risk of failed deployments. If the deployment to Test does fail then they can raise a bug, roll back and tell the developers what they think of them. Sure this will be embarrassing and it does happen, but would you rather it done in the confines of the IT department or in full view of the customers?

This brings us back to what we are here for : to fix a customers problem. I assume this typically means delivering working software. Working software on your development machine has not fixed the customers problem, that's like being a weight lifter in a body building show. Don't be that guy, think about the end game and make sure that each day you are working towards that end goal of providing you customer with a business solution, in the cleanest possible way.

*Sorry for putting the images of most nude men in your mind, it (probably) wont happen again

Continuous Improvement - Bringing back UML

noreply@blogger.com (Anonymous) — Mon, 26 Apr 2010 14:28:00 +0000

Continuous improvement is something we should all endeavour to pursue on a daily basis irrelevant of our chosen field. For me I am not a natural geek. I never built my own computer and never wrote my own gaming engine. I am much more interested in business than computers. What I love about computers is the way they can stream line our processes and, ironically, how unforgiving they are. They do what they are told, not you you meant. To me this poses a nice daily challenge and keeps my brain from turning to mush at work. Because i am not a natural geek I have to actively push myself to learn things with regard to IT.

When PowerShell came out a few years ago I looked at it in fear. I have never really been a shell guy. My 'nix mates were always playing on the green screens and I secretly knew there was a lot of power there but it was too much of a mind shift from the comfort of my drag-n-drop/intellisense laced world... But I knew it was a weakness and jumped in feet first.

Although i am still a complete n00b at PowerShell I can get what i need done with it. I am comfortable with the tools and have got real return on my learning investment. I got uncomfortable and it paid off in spades. I have even moved on to PSake and replaced most of my build and deployments steps to be completely PowerShell/PSake based, a feat I am pretty proud of. It has also helped immensely in me picking up Rails and Git, both of which are command line driven.

For me the next thing I think I really have to tackle is UML. I have dodged it for years.
I draw diagrams a lot. I use a white board every day and like to draw while I talk. I feel in mixed audiences it help get points across by using multiple forms of communication at the same time, voice body language and diagrams. However I am communicating I should be adhering to common practices where possible (as my last post pushed), when there is a common language, you should use it.
For diagrams in nerd world its all about UML.

My dislike for UML stems from my dislike of concrete documentation. I have inherited too many projects that have had documentation that did not marry up to the implementation, with UML typically being one of the worst offenders. It is incredibly frustrating and most importantly wasteful; wasted time, money and trees (paper has to come from somewhere)!

So I sit thinking that perhaps I was throwing the baby out with the bath water; UML is not bad, inaccurate documentation is bad. If I am to communicate with diagrams then I should use the common language. I also understand there is a time to be pragmatic and a time to be dogmatic. Most of the time a couple of boxes with some lines joining them will suffice but the ability to step up to the more accurate implementation is a worthwhile ability.
So my UML journey begins... or at least is reborn.

Common language and Patterns

noreply@blogger.com (Anonymous) — Mon, 26 Apr 2010 03:32:00 +0000

Imagine if craftsman from other industries called everything basic patterns they used they did by different names. Instead of a butterfly stitch a doctor called it "the sticky skin healer" or a carpenter calling a dovetail join "the zig zag corner"... it just wouldn't fly.
Well unfortunately this is not common in our industry, it is the norm.
Firstly I want to define what "our industry" is:
It is not computer science.
It is, if anything, Information Systems.
We push data around, make it easy for users to persist and transmit data so systems or other users can make decisions, business decisions. All of this is intended to some how improve the bottom line of the client and/or the company you are working for. People forget this. Our job is 99.999% of the time about making money. The Wiki on cucumbers git hub site has a good blurb about defining features and asking "why?" with regard to implementing features, you should get to the following reasons:

Protect revenue
Increase revenue
Manage cost
Increase brand value
Make the product remarkable
Provide more value to your customers

Now this is clearly for a product; I have no problem with team defining what the underlying drivers for their project, department or business are but being able to see how a feature helps enhance that is important.

Anyway... back to language.
Language and the words we use are very underrated. I think it is possible that, in our industry, communication skills are so rare (hey, we are nerds!) that we trivialise the importance of the words we use. Evans makes it very clear that the ubiquitous language is a key aspect of DDD and the same can be said about whole industries and the language used by the individuals that make up that industry.
This is why patterns books are popular. If I want to have a discussion with a developer at another firm to bounce questions off them, which I frequently do, if we do not share the same language then we spend half the time getting our terms aligned. There is a way to avoid this: understand the common language!
How? Well, I'm sorry but you are going to have to pick up a book or two... OK that's a lie. You are going to have to read a crap load. Hey, we get paid well, I expect senior developers and consultants (again, in our industry) to have read the books I am about to describe.

This list is an ordered list of pattern books, it focuses on patterns i.e. a language based term to describe a common approach to solving a problem. The language is as important as the fix

Refactoring - Fowler
PEAA - Fowler
Head First Design Patterns -or- Refactoring To Patterns
APPP - Uncle Bob
Refactoring Databases - Ambler
GoF
XUnit Test Patterns - Meszaros
EIP - Hohpe
DDD - Evans

There is also one I really cant wait for and that is the forthcoming Presentation Patterns by JDM; which should round out the whole stack.
To be honest this reading list would take the average developer that has a family and a life about a year to read. I'm sorry. Suck it up.

The order I have presented this list in is very deliberate. People will argue that GoF should be first. I heartily disagree. Gof is a hard book to read, with out having the understanding of refactoring and the softening blow of "Head First" the book is too much for the average nerd. I know this because it was my first patterns book. I have read it 4 times and only the last 2 time I have actually got anything out of it.

Refactoring
I feel this book is a minimum in a developers arsenal of pattern books. With this you at least can understand why resharper is doing what it is doing. It defines some basic terms that are low level enough that it is applicable to most developers irrelevant of the layers they work in. Some common terms like parameter object are introduced to the developers language

PEAA
Although he book references a metric tonne of other books, I think its aggregation of basics is its strength. The 2 book approach is great and the patterns introduced are very basic and high level enough that the are able to be picked up quickly. Read in conjunction with conversations with an experience developer that uses these patterns regularly and the developer should be very familiar with these patterns very quickly. This is a great book for Book Club type discussions too (hint hint @wolfbyte)

Head First Design Patterns -or- Refactoring To Patterns
These two books are a softer introduction to lower level patterns when compared to the godfather of patterns books, GoF. These are much easier to read and depending on the developer one may prefer one over the other. Ideally read both but read at least one of these prior to continuing down the stack. You may find it odd that the list starts with a low level pattern book (Refactoring) then a high level book (PEAA) then something in between. The pattern described in HFDP and RTP are more abstract patterns that require much more thought than the high level patterns in PEAA eg specific patterns like Table Gateway or ActiveRecord. I also find it easier to show what tool implements the PEAA pattern, eg Rails ActiveRecord or Castle Activerecord, NHibernate as DataMapper (all as examples of Lazy loading) etc

APPP
This book is great at helping developers step up to actually writing clean, readable, maintainable code by showing the patterns that you can use on a daily basis. ~~S.O.L.D.I~~ S.O.L.I.D is introduced to the reader. Unfortunately SOLID has just become another rote learnt acronym/pattern that is infrequently applied. I have found this does however allow code reviews to now have a common language and not just "um this code looks ugly" type code reviews, which i have been guilty of in the past.

Refactoring Databases
This book is in here by necessity. I doubt many .Net developers would get much from this. We have been force feed SQL drizzled in a Table sauce and Views jus and a side of Stored Procs for years. We know Databases, at times, it feels too well. However if you are from a non data base driven framework then this is a necessary read. I'm talking to you Ruby and Java kids. You lucky bastard get ORMs by default. In the twilight of the SQL age this book may seem a bit legacy but if you are truly and enterprise developer you will be dealing with SQL for years to come. I'm sorry, I really am ;)

GoF
Ahhh. The serotonin of pattern books. If you have insomnia this is for you. Its a great yet boring read, hmm what a paradox. This is just developer tax, its rite of passage, a necessity. Please note that just because a pattern exists it does not mean it must always be used, that it should ever be used or that it is not hard wired into you language. I'll let the reader guess/figure what I'm referring to.

If you have got through all of these books in my mind you have done you due diligence and are now allowed near an IDE.. OK I'm just kidding, but seriously I believe all of those previous books are minimum requirements for some one who leads a team of other software developers or consults. Below are the one you should continue on to if you want to be an above average developer

XUnit Test Patterns
By this stage you should be testing you software. The XUnit book, although not my favourite Testing book (that goes to The Art of Unit Testing) it is the best in describing the patterns of unit testing. Understanding these patterns will help you help other learn how to correctly add TDD to their skill set and firm up the teams language around key terms like doubles, fakes, stubs, mocks and spies; all things that are commonly misunderstood

EIP
This book cover various messaging patterns that relate to integrating system. Ideally you should read this book prior to using any ESB type framework like NServiceBus or MassTransit. If you have read this book and are using these patterns without having read or understood the previous books I fear you may have a big distributed mess on your hands. This is a great book but IMO this system you would tend to build using these patterns are most like system that should have been built on the patterns from the other books. If you do any integration you owe it to your self to read this, which means unless you are just doing the integration, reading all the other books first ;)

DDD
My favourite and I believe the most misunderstood book of the lot. This book is not just a code book it a process book. It raises the issue of language and introduces the notion of a ubiquitous language that the team including the stakeholders/SME/Users understand.
I believe this book has pushed what I can produce for a customer to the next level. The code feel more aligned with the business as opposed to just getting a task done. The closer code matches business processes and concepts the easier it can change with the business too.

So there it is. My list of pattern books I assume a Tech lead or Consultant has read, understands and implements. These are not the be all and end all of pattern books but I believe they cover the bases. Please note there are a suite of other books that are not pattern books that I believe are also "essential reads", but that is perhaps another blog post.

Would love to hear feedback,
cheers Rhys

NoSQL & Document databases

noreply@blogger.com (Anonymous) — Wed, 21 Apr 2010 07:56:00 +0000

I am really loving the NOSQL movement at the moment, however there seems to be a lot of confusion as to when it is appropriate to use them.

Significant things to considered are:

Atomic transactions are only for single operations. You can't really have long running or nested transactions (ACID rule a different).
Joins don't make sense so don't expect to use them when designing the system
Document databases are typically schema-less. This means adding new properties is much less of a hassle than in SQL-land, especially once you are in production.
The notion of an aggregate root fits perfectly with a document so the idea of using a DocDB for DDD is appealing (assuming transactions are not required)
DocDBs tend to scale horizontally very well unlike our SQL counterparts which tend to only scale vertically without huge headaches
Read and write performance is possibly the opposite of what is expected with very fast writes (I understand) being the norm.
Queries are done differently. MongoDB for example uses JavaScript as it query language (this does not mean it is used in the web tier!)

Several uses for document databases come to mind:

High volume, low value writes; eg user data entry on social sites; this is not business critical but potentially requires easy scaling options; ie no one is going to die if you last Facebook update doesn't go through to all the DB servers instantaneously.
Auditing; One area I'm keen on is command persistence. I like the idea of having a trail of all command sent to a component, it becomes a self documenting timeline of what users were trying to do to the system. When a command is handled by a component it can just write the whole serialized object to a DocDB, thereby capturing all the info without being bound to a schema (audit is version agnostic). The command can then be processed by the component. I will admit that an Object DB is also suitable for this.

Things not to use Document Database for is high value transaction heavy stuff i.e. banking transactions or thing that inherently require SQL... whatever use case that may be.

Hope this helps :)

Help Me Help You

noreply@blogger.com (Anonymous) — Fri, 16 Apr 2010 01:53:00 +0000

We have been lucky here in Perth that we have a very active community, well run by people who have stepped up to the plate to provide us these events. However these events are not free. We need venues, we often get prizes and some sweet swag. I don't think some of our attendees quite understand that without that support we have no event.
We are also fortunate enough to have sponsors we actully like! For this reason when we make a plug it not just because they are a sponsor but because we either use the product or service (or want to) ourselves. As our audiene is largely made of business developers I thought they would understand these basic back scratching processes.
Anyway, below is a list companies or products that have made many live better by helping out a technical community I'm involved in AND I recommend professionally:
JetBrains-ReSharper, Teamcity
TekPub
Readify
Beacon
Excom
Redgate

Thanks. I like your stuff, recommend you and thank you for helping us out. Also want to thank Mitch Wheat and Mike @wolfbyte for organizing us presentations every month; you are appreciated!

Testing with Domian Driven Design

noreply@blogger.com (Anonymous) — Tue, 13 Apr 2010 01:28:00 +0000

DDD is a hairy beast. It feels like everyone has a slightly different opinion of what it is. I personally thought I had a pretty good grasp on it until I caught up with Udi Dahan earlier this year where he (as per usual) completely turn my understanding on its head.
That being said I believe the way I am approaching the current project is in line with what I believe the majority of people believe DDD is, correctly or otherwise.
Key aspects in this project that help make it resemble a DDD domain are:

The level of understanding the devs have and the ubiquitous language that is in use and they way it is constantly evolving amoungts all team memebers, including our SMEs/Users
The general structure of the code - repositories, services, aggregate roots comprised of entities and value types etc etc

To be honest one could argue, that like most DDD projects, it is just clean OO coding. It is; that doesn't mean that there isn't something that others could glean of of the project, such as what we are doing to keep things clean and help with creating nice tests.

First and foremost follow TDD and BDD if possible
If you have a good understanding of the component you are building then writing explicit specifications should be easy. Do this with your BA and SME make sure the tests are not shallow low value tests. Investigate what BDD is and see if one of the many framework fit your teams needs. Personally I am still using xUnit frameworks and am creating elaborate contexts on which I make assertions. My fixture set ups can sometimes be complex but my test methods are very clear & clean and can often just be one line assert statements. I have not found a BDD framework that sits well with me in the .Net world like Cucumber does in my Rails development, so I will continue down this path till I find something I like better than explicit and somewhat elaborate xUnit styled tests.

The domain should remain as pure as humanly possibly
Anything that is public should have a good reason for being so. Unfortunately most developers use public as their default. This is bad practice as it produces a very non intuitive API. Use access to help show the other developers how you intended the API to be used.
Typically, IMO, this means protected virtual by default. (I'm an NH fanboi). Also restrict what you return. I personally don't like the idea of returning child entities from parent entities especially aggregate roots. What is the consumer going to do with these entities? Most of the time a value type or projection is more appropriate and helps keep the line of ownership clean.

Modifying the domain to be more testable should be frowned upon, especially if it confused the API
This ties into the first point. NB : There are concession that I make that I can live with (I will cover them soon) but I believe these do not negatively affect my domain and make life easier and my intentions more explicit to the next developer

Use subclass fakes to increase accessibility for testing
I often see people making fields, properties or methods public so tests can call them. Please don't do this. If these are private definitely do not expose them. This is an indication that you are not doing TDD. Private methods generally only come from refactoring once your test pass so testing private methods is a sure sign you are not doing TDD. Internals sometimes may warrant being tested. This is OK as we can make internals visible to the test project. I generally don't mind doing this as I put this in the AssemblyInfo file which is recreated in my deployment anyway so there is no dodgy test orientated code in my final deployed assembly. I must say that I don't really do this a lot and i think people use this as a crutch. True TDD generally will not warrant a lot of this, However i find myself doing it a bit in my domain, possibly out of fear more than any really rational reason.
If you really need to make something accessible to test and don't want to expose it fully then create a fake that inherits from the class and override the accessor in that class. Sometimes I do this to check things like IDs. Again question whether this is really needed, have faith in the TDD process!

Allow hooks in the domain for creating child objects
One thing I have used in the current project is protected factory method to create child entities. By opening up these one liners I can override the method in a fake sub class to create a fake subclass of the child: eg OrderFake will create and return an OrderLineFake in its CreateOrderLine method as opposed to the Order creating and returning an OrderLine in its CreateOrderLine.

When using fakes of real domain objects make sure you are not hiding any of the real object functionality. The fakes should be as plain as possible. Adding additional logic will surely corrupt your tests. One thing to watch for is to make sure your fakes implement the same ctors as their parent and call into those ctors. Failure to do this will create a big PITA :)

Next post I will talk about some issues and traps that we as a team have managed to fall into. Most of them have been things when consciously question know we shouldn't do but have managed to creep into the solution, hopefully we can help you avoid repeating our mistakes!

What is Regression Testing?

noreply@blogger.com (Anonymous) — Mon, 12 Apr 2010 14:10:00 +0000

We are on a journey. The company I work for is a very large one and has suffered some lag in migrating to modern ways of software development, a situation many companies of this size often find themselves in. Fortunately the team I work with have come together to make a conscious and consistent effort to improve our area of influence. First and foremost was the introduction of a structured developer oriented testing plan. Because we are a development team we started with unit testing, integration testing and some build scripts to run these tests. From there we have improved they way we do those tests, streamlined our scripts and made CI a solid part of our process. Automated deployments have become easier too leading to automated staged deployments. We are now at the stage where we are adding in Fitness and UI testing to the equation.

Each of these step have been driven from the ground up. We have had to, carefully, maneuver these processes into our daily process and show the benefits to management. It has taken time but they have responded very well; so well that they are jumping on board! This is a great thing and has made all the hard work, and it was hard work, worthwhile.
However there are still political games to play. The language is fractured and management want to throw around terms that they don't really understand and force their idea of these terms onto the very people that introduced them to the department, sound familiar? ;)

"Unit Testing" and "Regression Testing" are two of the terms du jour. Unit testing "means" something a developer does and regression testing is an automated UI test that a non techie can watch as it magically clicks away at the screen. This, as you may know, is not correct... well not strictly correct; and in a place where the language is so commonly fractured it is hard to justify fighting for seemingly such trivial victories.
So instead I vent here.

The things that i felt i needed to express most importantly are:
Any automated test is a regression test
- and -
You do not write regression tests.
What i am referring to in that last statement is merely the fact that you write a repeatable test that tests a certain unit, module or scenario, that can be automated and then your check it in to source control. The very notion that it is part of your build process/continuous integration means it now acts as a regression test.
So our tests that

Test a unit, i.e. an xUnit framework which may also uses mocks and stubs etc
Testing seams or integration points i.e. DB tests, file systems tests, web service tests etc
System smoke test
High level acceptance tests using tools like cucumber, fitness etc
UI testing - i.e. a button clicking framework like QTP

are all regression test because the are repeatable, automated tests.
Please don't make the mistake of thinking you will explicitly write a regression test. The test should test that the code produces a specific outcome e.g. enforces a business rule.
The automation and maintenance of a healthy CI environment makes those tests regression tests.

I guess the key here is really your CI process. If the tests can be dodged or accidentally not run somehow then this will need to be addressed.
Make the pit of success an easy one for you team to fall in to and let them know regression tests are a side effect of good daily habits.

InfoQ

noreply@blogger.com (Anonymous) — Sun, 21 Feb 2010 15:16:00 +0000

It still surprises me that people are not aware of the great resource that is InfoQ. If you are reading this post from my web page then there is probably a big InfoQ link right in front of you. If you are like the majority then this is probably a feed and you cant see it! Anyway its a great site that I have as my home page that gives a running report of the happenings of our industry. It doesn't say anything about the latest Intel processors or have Dilbert cartoons its just enterprise development and architecture, specifically .Net/Java/Ruby, SOA and Agile; Basically stuff I am interested in on a professional level.

Go check it out : http://www.infoq.com

Udi Dahan's Advanced Distributed Systems Design with SOA & DDD

noreply@blogger.com (Anonymous) — Sat, 23 Jan 2010 00:44:00 +0000

Over the last week I have been in Melbourne attending Udi Dahan's Distributed systems course and I thought now that I am home will do a quick review on the course.

Firstly the course is not about how to build a 3 tier system on top of a Microsoft stack. Udi is a well known M$ MVP and has a well known open source .Net project however the course is an architectural course focusing extensively on architectural concerns that are largely technology agnostic. Most of the examples at in C# and use (an abstraction of)MSMQ however I guess near on 100% of the examples could work in Java (assuming tool support). The course is also not about how to build a Thomas Erl style web of web services.

To be honest leave all your preconceived ideas of architecture at home... it will just make life easier.

There were jokes of requiring a support group for attendees of the course, you will feel like Neo after he took the Red Pill; naked alone and very... well humbled.. or stupid, depends how diplomatic you want to be. Because people inherently don't like change there was the typical amount of resistance; given there would have been several hundred years of experience in building large systems in the small room and I would consider these guys some of the best minds in the country in terms of .Net, the red pill was pretty hard to swallow. However like Neo with Morpheus, we put our faith in Udi and let him show us the way.

Seriously though, the course was incredible, I think the overwhelming majority learnt more in those 5 days about designing large scale scalable enterprise architectures than they had over the 5 years of their career.

Just a side note about Udi: he would have to be the most tolerant presenter and teacher I have ever had. These concept were hard to grok all in one go, Udi allowed for lots of questions lots and ensured that the audience knew enough to progress. Sure a lot of the time I was thinking... "what the?" but typically the very next slide answered my question. In terms of raw ability, I have never met anyone like him. Yes he is an outstanding architect, truly outstanding, seriously never met anyone that can walk the walk like Udi in those terms. However to be a truly great architect you have to be a bit more than the guy who is drawing the UML. He has a fantastic mind that has a great understanding of business, organisational management, psychology and his technical understanding is better than all but the absolute best coders. This man is no Ivory Tower Architect. He would be just at home in the largest corporation boardroom having discussions with CTOs as he would pair programming with the guy at the boiler plate writing the code. Possibly a genius.

Request Response is dead... well, its not but... well go on the course, its freaking awesome just be ready to start looking for a support group... you'll need it ;)

**********************

Course outline as it was for our course (from http://www.udidahan.com/training/ on 23 Jan 2010) :

The Rhys Campbell Course Rating: 10/10

Advanced Distributed Systems Design using SOA & DDD

Duration: 5 days

Introduction

Designing large-scale distributed systems is hard. New technologies make it easier to comply with today’s communications and security standards, but don’t auto-magically give you a robust and scalable system. Join Udi for a course packed with the wisdom of companies like SUN, Amazon, and EBay.

Tried-and-true theories and fallacies will be shown, keeping you from making those same costly mistakes today. Communications patterns like publish/subscribe and correlated one-way request/response will be used in conjunction with advanced object-oriented state management practices for long-running workflows. If you enjoy deep architectural discussion, if you are in charge of building a large-scale distributed system, if you want to know more about how the big guys run their systems, this is for you.

Audience

This workshop is targeted at team leads, application and solutions architects, as well as technologists who are involved in making decisions about the overall system design of software products and projects.

Course Topics

Module 1: Distributed Systems Theory
Decades of distributed systems development have taught us many lessons. In this module we’ll cover many historical mistakes as well as proven best practices for scalable and robust design. Topics include:

8 fallacies of distributed systems
Transactions

Module 2: Coupling: Platform, Temporal, & Spatial
Loose coupling has become the watchword of complex systems development, yet few understand its multiple dimensions. In the module we’ll be covering the three different dimensions of coupling as well as patterns for dealing with them.

Platform Coupling – XML/SOAP
Temporal Coupling – Synchronous/Asynchronous
Spatial Coupling – Endpoints/Topics

Module 3: Asynchronous Messaging Patterns
Although scalability is achieved through the use of asynchronous message passing, more advanced message exchange patterns are required to handle today’s complex integration scenarios. This module will cover the most commonly used patterns:

One way
Correlated Request/Response
Publish/Subscribe

Module 4: Bus & Broker Architectural Styles
Enterprise Service Buses are all the rage these days. In this module we’ll be covering what’s the difference between the Bus architectural style, and the more well-known Broker, found commonly in many EAI projects. Topics will include:

Architectural advantages and disadvantages
Technological advantages and disadvantages

Module 5: SOA Building Blocks
One of the goals of SOA is to develop systems which are more closely aligned with Business. In this module we’ll be covering an analysis methodology from moving from the business domain to executable systems that comply with all the principles of loose-coupling.

Business Services
Business Components
Autonomous components & Queues

Module 6: Scalability and Flexibility
In order to enable agility, services must be able to scale up, out, and down quickly. In this module we’ll see how autonomous components can be configured including transactional and durable aspects of message handling.

Configuring autonomous components
Scaling up and out

Module 7: Long running processes
The distributed communications patterns wouldn’t be complete without a discussion on orchestration. In this module we’ll see how to manage the state of long-running distributed communication flows as well as:

Encapsulating process logic
Advantages & disadvantages of orchestration

Module 8: Service / Autonomous Component Solutions
As developers go to implement autonomous components, guidance is required as to which concepts need to implemented in which project, what dependencies are there between projects, and how to bridge the worlds of messaging, business logic, and reporting. Topics include:

Messages + Handlers
Databases

Module 9: Service Layer – Domain Model Interaction
Logic-rich services require the use of advanced techniques for logic componentization. The Domain Model Pattern enforces a high level of Separation of Concerns, yet it must eventually be connected with Service Layer code that supports many concurrent users. In this module, the topics covered will include:

Domain Model introduction
Testing Domain Models
Optimistic, Pessimistic, and Realistic Concurrency Models

Module 10: Creating High-Performance Domain Models
The strong separation between the Domain Model and the database which stores and retrieves its data may enable a high level of testability, yet often causes performance problems. In this module, we’ll see the various aspects impacting the performance of persistence:

Transactions and Isolation Levels
Lazy Loading, Eager Fetching
Databases Tips & Tricks

Module 11: Web Services and User Interfaces
The ease of interacting with users over the web drives the need for service to UI interactions. Also, many integrations require exposing synchronous web services to customers. In this module, we’ll see what is required in both cases:

ASP.NET 2.0 Asynchronous Tasks
Rich Internet Applications and Services
Web Services for integration

Module 12: Smart Client / Service Interaction
The publish/subscribe semantics with which services communicate require smart clients to perform a great deal of background work. Also, certain service contracts lead to more performant clients. In this module, we’ll cover the first part of these interactions:

Multi-threaded client challenges
Client-friendly Service Contracts
Service Agents and Client Repositories

Module 13: Notifications & Smart Clients
After Message Handlers in the Service Layer create or update the relevant Model objects in the client Repository, Supervising Controllers are in charge of getting Views to show the updated data. In this module, we’ll describe the parts and interactions of these flows:

Client-side Model Objects
Supervising Controllers
Views and their Interfaces

Module 14: Commands & Smart Clients
Capturing user intent and synchronization between views are at the core of smart clients. After describing solutions that use Events on the View Interfaces, the Command Pattern will be introduced to further decrease coupling between Supervising Controllers. In this module, we’ll describe the parts and interactions of these flows:

View Interfaces, and how Entity Cloning affects them
Supervising Controllers and clone reconciliation
Commands, and Event-Based programming

Summary & Review
In order to make sure that attendees are able to put into practice all that they’ve learned throughout the course, here we strengthen the seams between the various topics. Q&A is also a core part of this final section.

Relearning WCF

noreply@blogger.com (Anonymous) — Thu, 24 Dec 2009 02:56:00 +0000

Of late I have been playing with WCF again. We have some projects here at work that require some integration and we are desperately trying to move away from the old ASMX based services. Unfortunately because I have not touched WCF the whole time I have been here (12 months now, wow! That has gone fast!) and I have found myself at a point where I really need to relook at WCF again and basically relearn it... oh well.
Anyway here is a bunch of stuff that here, that at work we have found to be useful that you may not otherwise be able to do with ASMX or may not be aware you could do with WCF.

IOC and WCF

You can in fact use IoC with WCF, there are some good blog posts and accompanying videos to show what to do and if, like me, you just want one ready to that uses the CSL then The Code Junkie has done it for you!

Dynamic KnownType Resolution

This always erked me that I had to put into the data contract that I knew of other types, it was like really bad tight coupling*. There are a bunch of way to declare known types with the bottom example seeemingly a little known alternative : a provider mechanism

in config

<system.runtime.serialization> <dataContractSerializer> <declaredTypes>...

Data contract with attributes


 [KnownType(typeof(PurchaseApprovalRequest))]
 [DataContract]
 public class ApprovalRequest
 {...

Knowntype provider

The way I have just found out is by declaring a knowntype provider on the service contract:


[ServiceKnownType("GetKnownTypes", typeof(ApprovalRequestKnownTypesProvider))]
[ServiceContract]
public interface IApprovalService
{...

with the following class (change the implementation to suit yourself, this is from some of my demo code, it’s not recommended!)


internal static class ApprovalRequestKnownTypesProvider
    {
        public static IEnumerable<Type> GetKnownTypes(ICustomAttributeProvider provider)
        {
            // collect and pass back the list of known types
            foreach (var module in Assembly.GetExecutingAssembly().GetLoadedModules())
            {
                foreach (var knownType in module.FindTypes(
                    (t, f) => ((Type)f).IsAssignableFrom(t), typeof(ApprovalRequest)))
                {
                    yield return knownType;
                }
            }
        }
    }

With these two little nuggets I have been able to produce a pretty handy little broker service that act as a very basic content based router that keeps the client messages very clean and does not expose any implementation details (i.e. no passing of service or workflow names in the message header!)

*NB: to paraphrase Krzysztof: "Polymorphism is an OO term, not SOA term, so I don't use it, and make my contracts explicit wherever possible." be wary that you are using known types for the right reasons