Developers 42

Time for a change

noreply@blogger.com (Jonathan van de Veen) — Fri, 12 Oct 2012 12:59:00 +0000

I've been considering this for several months and now got an opportunity to make a change. This post describes my next career move and why I decided to do this.

The history

The past four years I've been working for a software company that builds a product suite around Human Resources and Salary. The suite contains anything from salary calculation and HR registration to Employee and Manager Self Service systems. I was hired to help bring innovation and new technology to the existing team and work on the next version of the product.

As you might know we started with Silverlight 2 beta shortly after I joined the company. We worked on this product and upgraded it all the way to Silverlight 4 in two years time, until we decided that we needed to move away from Silverlight. For the past months our team has been working on delivering an ASP.NET solution.

My role has been to lead the team from a technical standpoint and architect the framework underlying the application. I've been happy to do this, however I still decided to change.

The Why

There are several reasons, some of which I'm not comfortable to discuss online, however the most important ones I can discuss.

The first reason, to me, is inherent to the assignment I had. If you complete the assignment, in this case developing a framework and getting a team up to speed on using it, you can't help but ask "What's next?". Unfortunately the answer was, "Finish the rest of the product." To me, this didn't (and still doesn't) sound very appealing. If you've been working on highly technical generic code and all of a sudden you're expected to do repetitive development, somethings got to give.

The second reason is more personal. I find it important to keep improving myself and I didn't see myself being challenged enough to improve on the skills I think are important to improve on as a senior developer, mainly soft skills. At some point you can get to comfortable in a certain situation.

The What

So the final question is, what am I going to do? I've though about this and figured that in order to improve on soft skills a more customer facing role would suit my ambitions better. As I still wanted to be doing technical work as well a consultant role was the obvious choice. However I didn't want to be working with just another 'hour-factory' where you're just expected to bill as many hours to a customer as possible and I didn't feel much for being just another SharePoint 'specialist'.

I've been looking around and thinking about different routes to get what I'd like until an opportunity came along that I really liked. As of October 15th, I am working as a Senior Consultant in Enterprise Search. I've been working with enterprise search technology between 2004 and 2008 and I enjoyed it very much and always did want to go back to it and now I got the chance!

So what does a consultant in Enterprise Search do? I'll be working for customers that want to use Autonomy technology within their organization. Assignments can range from a simple advice all the way to a complete one off implementation for the customer. I'll be working multiple projects at a time and I'll be working with different programming languages and platforms a long the way.

So what about Developers 42?

What does all this mean for Developers 42? I will try to continue writing posts as time passes. However it does mean a change in direction for the topics I'll discuss. As I'll be using multiple programming languages, chances are I'll be writing more about my experiences with those.

Further more, as I explore more of the world of Enterprise Search and it's surroundings, I might be able to give some insight into what goes on in this field of expertise I'll be a part of once more.

I do hope you'll still be following this blog and to learn and teach together with you.

Follow me on twitter as well.

To Tweet or not to Tweet. That’s the question.

noreply@blogger.com (Jonathan van de Veen) — Fri, 10 Aug 2012 11:09:00 +0000

In this post I’d like to give an explanation of why I decided to start using Twitter and how it has worked out so far.

“Why on earth do you think you need to write a post explaining why you started using Twitter?”, you might ask.

I don’t like social networks

Well, people who now me, may have learned that I’m not a great fan of social media. I tried using several and most of them still don’t seem to add value for me.

Especially a network like Facebook appears like a serious waste of my, and others, time. I do have a Facebook profile, but I mostly use it to promote my blog posts.

The main reason I don’t like social networks is because I think most of us don’t have such an interesting live that you need to post updates about it every five minutes. Does anyone really care that I get up in the morning, go to work, when I drink coffee and when I go back home? Another question is, do I want others to know when I go to work and when I get home? To me it’s all a lot of noise and I don’t feel like spending time creating that noise for others to listen to.

Why Twitter might be different

So why go and use Twitter then? It seems like a lot of noise. Two things occurred to me. First of all, not a lot of people in my personal network actually use Twitter, so I don’t feel obligated to actually follow tweets by people that are close to me. The other thing is, and that applies to all Twitter users, Twitter is unique in the sense that it’s not automatically two way traffic. I can follow someone, but that doesn’t mean that someone will follow me.

This means that I can follow people who I think might have something interesting to say without me worrying about being intrusive and it also allows me to not follow a follower who I think is unlikely to tell me something interesting.

Besides the advantages of the platform I saw, I was looking for a platform to get some more interesting information about what others are doing in the field of software engineering.

I was also looking for a platform to promote this blog a bit better and a platform that allows me to post short updates appeared to me as a good way to do that and a challenge in the process. As some of you may know, I’m not blogging at a preset frequency. This does mean you may not hear anything from me for months. The reason for that can vary from being to busy to write something worth reading up to just not having a topic worth writing an entire post about (then again, maybe writing a post about why I joined Twitter might not qualify as a topic worth writing a post about ). With Twitter I can just write a one liner, share a link or a thought and it’s done.

Experience so far

So what has been my experience up to now? Well, I don’t have any real followers, so I don’t know about noise to others. I did tweet about ten times now of which two were retweets. I did find several Twitter accounts to follow that provided me with interesting Tweets. I got to some information I wouldn’t have got if I wasn’t on Twitter. I also stopped following some accounts as they provided me with to much noise. All in all that makes it meet my expectations.

This is another thing that struck me on Twitter. It seems more natural that people stop following other people. I had a follower and she stopped following me after about a day. I don’t know why and to be honest it doesn’t really matter. I see it more as a long running conversation. Once it stops being interesting for one of the parties involved, they leave the conversation. It makes sense to me.

Conclusion

So I have joined Twitter and up to now it has been a reasonable experience. I will continue to use Twitter for the upcoming months. If you think I might tweet something interesting, or if you want a tweeted update whenever I post something here, please join me on Twitter: @jvandeveen.

Feel free to share your experience with Twitter or other social media in the comments section below.

A TED video I think you should watch

noreply@blogger.com (Jonathan van de Veen) — Tue, 31 Jul 2012 09:03:00 +0000

In this post I would like to share a video from TED, that I think anyone even remotely interested in computers should watch.

Every now and then I like to watch videos that teach me something and/or inspire me when it comes to innovation. I used to limit myself to Channel 9, which is a great source for anything even remotely (and sometimes not) related to Microsoft. It has been inspiring and educational. Remember this post I wrote back in 2009?

Last year I was pointed to TED. As TED says on their website, they are a ‘nonprofit organization devoted to Ideas Worth Spreading’. Find out more about TED here. You should note that most videos on TED are presentations or talks. Sometimes there are interviews and sometimes there is a video with some kind of performing art.

The video I’d like to share with you is this one. It is a talk by John Graham-Cumming and it’s about the first ever design for a computer by Charles Babbage. I don’t want to give you to much information on the details, but you should definitely watch it. It’s only twelve minutes long, which is a common thing about these talks, they usually are not very long, which makes it easier to find the time to watch some of them.

One thing I took away from this is that although it is ok to doodle around with a whole bunch of different ideas, we should every now and then execute one of them.

If you have other great places where you find inspiring and/or educational videos, feel free to share them with us in the comment section.

Innovation can't be planned

noreply@blogger.com (Jonathan van de Veen) — Tue, 29 May 2012 11:55:00 +0000

The profession of software developer is a relative young one. As such we learn new things about it that other professions already have figured out. This post is about one of those things that managers who need to manage developers rarely seem to get: innovation.

Software development is devided in two different types of work, the standard projects for which you can simply apply well known en documented methods that lead to a predictable result, and the innovative projects where features and requirements can change based on the technical possibilities and impossibilities.

For example, implementing a corporate intranet in a CMS is well understood and tends to be predictable. On the other end, building a new product where the UI has to be available on a wide range of platforms and devices is not only new, but also has numerous variables that make it different every step along the way. Sure, there are methods to execute a project like that in a structured way.

Most agile methods appear to be suited to do highly innovative projects. However, they do have a single flaw, which I've recently experienced. Although they do allow for great flexibility in the big picture, once it comes down to the detailed work, they don't allow that same flexibility because at some point there still has to be some predictability so it can be sold to managers and executives alike.

In practice, when using an agile method, developers are still required to commit to delivering a certain amount of work within a certain amount of time. The only thing that has changed is the amount of work they are asked to make an estimate for. It's like instead of asking to design and build a spacecraft you're now asking for only the propulsion system. Sure, it's less work, but it's still highly unpredictable, especially when you find out the maximum speed in the specification is not nearly enough to get the craft to the destination within a reasonable amount of time and the fuel consumption needs to be cut in half.

Obviously the example above is exaggerated, but the point is that with innovation things can only become apparent once you're working on it. You encounter problems you could not have thought of and you have to solve them in order to deliver a product that's up to standard.

So how do you go about building an innovative software product? The first thing is to make sure you have the right people leading the team and trust them. Also make sure there is a clear set of boundaries that the team needs to operate within. Then support them and facilitate their work as much as possible.

This doesn't mean you can not put a timeconstraint on a project. However if you do, you should be prepared to compensate for that in functionality (some would argue that you can also assign more resources to a project, but why I think that has great limitations is a topic for another post).
So please, stop trying to plan your innovative development as you will only be disappointed.

Writing code != developing software

noreply@blogger.com (Jonathan van de Veen) — Fri, 18 May 2012 08:33:00 +0000

There has been some fuzz online about whether or not should everybody learn how to write code. In this post I’d like to add my point of view and take into account what the profession of developer involves (or should involve).

If you have not followed this discussion, you might want to read up on some other posts firsts:

Please Don't Learn To Code - Jeff Atwood

Please Learn To Think About Abstractions - Scott Hanselman

In a nutshell the fuzz is caused by public figures announcing they are going to learn how to write code.

The car mechanic

There has been several comparisons with other professionals already. However I think they are not really suited for what is happening here. I like to explain why learning how to write code doesn’t mean anything by projecting this to the profession of the car mechanic.

Let’s say I announce I’m going to learn how to work on an old car engine. If you analyze the skills I need to work on a car engine it almost becomes laughable:

I need to be able to open the bonnet of the car
I need a basic understanding of tools in the toolbox (a spanner, a screwdriver, etc.)
I need to be able to start the car and drive it in order to test my work

Anyone can see that it I work on a car engine with only the above skills, I’m very likely to ruin the engine. At the very least we can conclude that being able to work on an old car engine doesn’t make me a car mechanic.

Let’s project this example back to the software developers profession. If you want to write code you need the following skills:

I need to be able to install and run an IDE
I need to be able to type code in the syntax for the language I’m using
I need to be able to use the IDE to compile and/or run my code

If you are going to use only the above skills to write a program, changes are you are not going to achieve the results you want. In any case the above skills are not nearly enough to become a software developer.

The software developers profession

Jeff Atwood makes an excellent point in his post when he states that being a software developer is not about writing code, but about solving problems. I completely agree with Jeff on this point. If you read the post he refers to when making this point (here), you’ll see he paints a very bleak picture about developers being extremely disconnected from the users they are building software for. I personally do not have that experience, but then I’ve always worked in relatively small teams (six developers being the max).

In my career I’ve coached several developers and I’ve always been an advocate of what to me is the single most important skill any developer should have is the analytical skill. I’ve even given short trainings in analysis. When another developer asks for my help solving some problem, I always try to be very verbose on how I get to my analysis.

And as Jeff Atwood also states in his post, writing less code is better. In order to do that, being analytical about what you’re trying to achieve is crucial.

Obviously you can have all the analytical ability in the world, but if you don’t have a proper understanding about what you’re dealing with, it being the inner workings of an operating system, or the politics around some customers feature request, you can not make a proper analysis. This means you do need knowledge.

This brings me to a problem that is fairly unique to the software developers profession. Code is not knowledge, however it is transferred and documented online in vast quantities. This has created a whole breed of so called developers who simply go online and copy code for what they think solves a particular problem they’re having, without actually understanding what the code does, nor why they are having a certain problem. The term Lamer comes to mind.

The danger with that is that someone who does that might think that they solved a problem, however they might very well have not. And because of the lack of understanding they are unable to verify if the code solves the problem, nor are they able to troubleshoot the code if needed.

The software developer needs to analyze the users problems, then come up with a solution that fits the users needs. Again it’s not about the code. It’s about the solution. It requires a deep understanding of computers and the users needs and it requires analytical skills and a creative mind to come up with a fitting solution.

This is just my two cents. Please leave any comments, questions or rants below. I’d love to hear how others perceive this.

A new blogging opportunity

noreply@blogger.com (Jonathan van de Veen) — Mon, 14 Nov 2011 09:18:00 +0000

A little over a week ago I was contacted for a new blogging opportunity and I decided to give it a try.

When having a blog it’s nice to be noticed by people every once in a while. So when I was contacted by the Technorati Editorial Team (TET, really?) last week with an invitation to write for them I decided to at least investigate. It clearly was a mass-mailed invitation, so I was not as keen as I was when joining SilverlightShow.Net.

So what got me on board then? Well, let’s think about why I stopped writing for SilverlightShow, because let’s be honest, SilverlightShow did pay me for my articles and Technorati is not. Well, obviously there is the decision on stopping with Silverlight as our go-to UI technology, but it was not a deal breaker. In the end I was just to busy to crank out lengthy posts with lots of code and it became increasingly hard to find good topics to write about.

What makes writing for Technorati different is that they are looking for news and opinion posts in IT and they are looking for relatively short posts. Another thing that they allow, which I like, is reposting a post I wrote for Technorati here.

I finished my first post for Technorati last Friday and they published it a few hours later. By Saturday morning the link was Tweeted sixteen times, which means it generated more Tweets on its own than I did with all the posts I’ve done so far. I also got a very informative comment on the post, so to me this was a successful experiment. I do plan on writing more posts for Technorati and I also plan on reposting right here, for your convenience.

Enjoy.

Parsing CSS for analysis

noreply@blogger.com (Jonathan van de Veen) — Mon, 31 Oct 2011 09:57:00 +0000

This time we’ll look into parsing CSS so we can use the result for analysis.

The story

As we are starting a new project using ASP.NET MVC3 one of the things I was thinking about was how to better manage our CSS and JavaScript. The first thing that I figured would be great was if you could to some static analysis on all of your CSS files to see if certain situations arise.

The issue with multiple people working on the same HTML based UI is that it can be hard to find out what CSS class to use. Because of that you might end up with classes doing the same thing, classes being defined more then once with different definitions, etc..

All this leads up to a less then ideal situation. Removing the cause of the problem, not knowing what CSS class to use, proves difficult. Helping counter the symptoms of the problem is proving a lot easier, so that’s the path I chose for, right now.

A two part solution

In order to do static analysis on something that is only available to me in plain text, I first need to parse the plain text into something that is easier to analyze. Then I can use the result to do the actual analysis on and communicate the result.

Analyzing how to parse CSS

To write a parser for any language requires a deep understanding of its syntax. Fortunately for me CSS is not a very complex language. W3Schools.com proves to be very helpful in providing us with an explanation on the CSS syntax. It comes down to this:

A CSS document consists of CSS rules
A CSS rule consists of a Selector and a set of Declarations
Each declaration consists of a property and a value
Comments can exist at the top level of the CSS document (outside of the CSS rules) or in between declarations

In order to do static analysis on this I came up with some interface definitions that allow me to query the structure of a CSS document:

        1: public enum SelectorType       2: {       3:     Tag,       4:     Id,       5:     Class       6: }       7:         8: public interface ICSSDocument       9: {      10:     string FilePath { get; set; }      11:     IEnumerable<IRule> Rules { get; }      12:     void AddRule(IRule rule);      13: }      14:        15: public interface IRule      16: {      17:     ISelector Selector { get; set; }      18:     IEnumerable<IDeclaration> Declarations { get; }      19:     void AddDeclaration(IDeclaration declaration);      20: }      21:        22: public interface ISelector      23: {      24:     string Name { get; set; }      25:     SelectorType SelectorType { get; set; }      26: }      27:        28: public interface IDeclaration      29: {      30:     string Name { get; set; }      31:     string Value { get; set; }      32: }

Note that this might not be final as I might come up with requirements implementing the static analysis.

For parsing text like this there are always several approaches. In this case I decided that using plain and simple text parsing would be the most flexible, as I might want to add features to the parser in the future. Here is what I came up with for the main parse loop:

   1: public void Parse()      2: {      3:     string data = File.ReadAllText(FilePath);      4:        5:     _position = 0;      6:     _isInComment = false;      7:     while (_position < data.Length)      8:     {      9:         if (IsEndOfFile(data))     10:         {     11:             break;     12:         }     13:         HandleBeginOfComment(data);     14:         HandleEndOfComment(data);     15:         if (!_isInComment)     16:         {     17:             HandleRule(data);     18:         }     19:         else     20:         {     21:             _position++;     22:         }     23:     }     24: }

As you can see I have a (private) field to keep track of the position within the CSS document and another field to keep track of comments.

You might find the IsEndOfFile method weird as I have a condition within the while loop that should do the same thing. However I need to check ahead one position in case I’m still checking for comments (or are in a comment for that matter). The definition of the method is quite simple:

   1: private bool IsEndOfFile(string data)

   2: {

   3:     return _position == data.Length - 1;

   4: }

The HandleBeginOfComment method checks for the start of a comment:

     1: private void HandleBeginOfComment(string data)      2: {      3:     if (data[_position] == '/' && data[_position + 1] == '*')      4:     {      5:         _position += 2;      6:         _isInComment = true;      7:     }      8: }

Basically it checks for the string /* and if it finds that string at the current position it moves the cursor by two characters and sets the _isInComment flag. HandleEndOfComment does the same thing for */ and sets the _isInComment flag to false again. Any comments are currently ignored, but it is easy to extend the main parse loop to allow for parsing comments as well.

The HandleRule method takes care of all the parsing magic, which makes sense as the Rule is the main component of a CSS document.

     1: private void HandleRule(string data)      2: {      3:     while (_position < data.Length && !StartOfRule(data[_position]))      4:     {      5:         HandleBeginOfComment(data);      6:         if (_isInComment)      7:         {      8:             return;      9:         }     10:         _position++;     11:     }     12:     string selectorData = GetSelector(data);     13:     string declarationsData = GetDeclarations(data);     14:       15:     IRule rule = _kernel.Get<IRule>();     16:          17:     ISelector selector = _kernel.Get<ISelector>();     18:     selector.Name = selectorData;     19:     selector.SelectorType = GetSelectorTypeFromName(selectorData);     20:     rule.Selector = selector;     21:       22:     HandleDeclarations(rule, declarationsData);     23:       24:     AddRule(rule);     25: }

The first loop deals with running into a comment later on in the document. If we do run into a comment we simply return to the main loop, which will then deal with finding the end of the comment. In the same loop it checks for the start of a Rule.

If we are still in the method after this loop, we have reached the start of a Rule. As we’ve noted earlier a Rule consists of a Selector and a set of Declarations. The methods GetSelector and GetDeclarations take care of parsing those portions of the CSS document. Once we have that data we can use it to create a rule. We use a Ninject Kernel to create instances of both the IRule and ISelector implementations.

Note that right now we handle the Selector like it’s a single entity. A future improvement might be to split up the Selector into parts and assign types to them individually.

The HandleDeclarations method takes the declarations text, parses it into IDeclaration implementations and adds them to the given IRule:

     1: private void HandleDeclarations(IRule rule, string declarationsData)      2: {      3:     string[] declarations = declarationsData.Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries);      4:     foreach (string declaration in declarations)      5:     {      6:         if (string.IsNullOrWhiteSpace(declaration))      7:         {      8:             continue;      9:         }     10:         int splitterIndex = declaration.IndexOf(":");     11:         string declarationName = declaration.Substring(0, splitterIndex).Trim();     12:         string declarationValue = declaration.Substring(splitterIndex + 1).Trim();     13:       14:         IDeclaration declarationInstance = _kernel.Get<IDeclaration>();     15:         declarationInstance.Name = declarationName;     16:         declarationInstance.Value = declarationValue;     17:         rule.AddDeclaration(declarationInstance);     18:     }     19: }

Note that I use String.Trim to make sure we don’t end up with white space in our declaration data, which could get in the way of our analysis (and is of no value any way in CSS).

So far so good. We can now parse CSS into an object model, which allows us to analyze the CSS in a structured way. I plan on writing a next post that shows the analysis based on this model.

No complete source included: Unfortunately, as this project is owned by my employer, I can not include full source code, however the parts I included should provide you with a good insight into parsing CSS.

Moving from Silverlight to MVC3

noreply@blogger.com (Jonathan van de Veen) — Tue, 20 Sep 2011 08:14:00 +0000

In this post I’d like to explain why we felt the need to abandon our Silverlight development in favor of MVC3.

As some of you may very well know, I’ve been pushing Silverlight within our company from as early as Silverlight 2 Beta 1, back in 2008. Still, this last summer I felt I had to raise the question: Is this still the right technology for our company?

Technology choices are about requirements

When we first chose Silverlight as our UI technology, we came up with a set of requirements. Our application should have a (near) zero footprint on client systems. It should be accessible through the web. Deployment on clients should be effortless or none existent. It should look great without a whole lot of effort. Silverlight checks all these boxes (it still does).

We write summer of 2010. The iPad has been around for several months now and it has been a great success. Smartphones are widely adopted to the point that we predict our customers will get interested in support for platforms like these when using our software.

Another thing we see is people working on more different platforms than before. Was it ok to ask for a certain browser in 2009, in 2010 our customers began to ask for a browser independent UI, not only on Windows, but also on a lot of other platforms.

To see what exactly would be the response of our customers we decided to build a native iPhone application with a limited set of functionality. We started the project early 2011 and published the first Dutch HRM app for any smartphone platform in may. It was a hit, not only because of responses by our clients, but also because of responses by our competitors.

Requirements shift, the world changes

This lead us to conclude that we needed to do something to support a new requirement. Our application, at least in part, should be available on mobile devices and tablets. And we should make sure we could drop the browser requirement we had active, even with our Silverlight 4 UI*.

That is why, during one of our meetings about what to build next, I felt the need to ask “Shouldn’t we move away from Silverlight and adopt ASP.NET MVC3 with HTML5?” It sure was painful, after investing over two and a half years in Silverlight to even suggest dropping it. However I felt then, and I still do, that a change had to be made. After some more debate on the consequences and some research, we are now putting in the effort to build the first parts of an MVC3 UI with the first steps into HTML 5 and CSS 3.

Luckily not all our efforts with Silverlight where in vain. We did deliver a good working product and we are committed to using, expanding and improving the middleware we developed for it. (Do you see the power of a Service Oriented Architecture?).

So is Silverlight really dead now?

Some of you may remember this post. In that post I state that Silverlight is actually the better technology for writing Line Of Business Applications. Even after researching ASP.NET MVC3 and HTML 5 further, I do still believe that, however because of the changes in how people use computers it simply fails to check one box for our customers.

Silverlight is no where near dead. It will still be used in many ways, including new platforms like Windows 8 and, for example Microsoft CRM 2011. Microsoft is integrating Silverlight in more and more of their own platforms. Also with the introduction of Silverlight 5 there will be many cool new features coming to the browser. Therefor I doubt Microsoft will drop Silverlight any time soon.

Consequences for Developers 42

Unfortunately our move away from Silverlight does have some consequences for this blog. First of all, my long running series of posts “Adventures while building a Silverlight Enterprise application has ended with part 41 (which is slightly painful for an obvious reason).

It also means I will no longer be writing posts for SilverlightShow.net. I’d like to use this opportunity to thank Svetla for supporting me for the past years.

And finally, it will also mean a lot less content on Silverlight, but hopefully a lot more content on MVC3, HTML 5, CSS 3, jQuery, etc..

It’s the end of another interesting period in my professional live and the start of a new one. I hope you have enjoyed my articles on Silverlight and I hope to see you around.

* The IE dependency was not actually caused by Silverlight, but by the integration with the Microsoft Report Viewer we used.

Trivialities about nullable types

noreply@blogger.com (Jonathan van de Veen) — Fri, 12 Aug 2011 07:01:00 +0000

Sometimes things in the world of software are not that hard. They are just poorly documented or documentation is hard to find. This surely applies for some use cases of nullable types. This post shows the use of nullable types in generic methods and how it’s easier than you might think.

An introduction to nullable types in C#

In order to understand the use of nullable types, it is imperative that you understand the difference between a value type and a reference type. For a complete description of these things have a look at chapter 4 of the C# 4.0 Language Specification (included with Visual Studio 2010). In short it comes down to this: value types directly contain their value, while reference types contain a reference to their value. Value types are always either a struct type or an enumeration type. Struct types include things like numeric types.

As value types always directly contain their value, they can not be null. However, in some cases, you might want to have a structure where you can identify if a value type is actually not defined. In lots of cases simply defining 0 as the undefined value for an int, for example. will not suffice. 0 might actually be a meaningful value for an int.

To provide a way of declaring a variable of a value type that can be null, C# introduces the question mark as a suffix to a value type:

int nonNullableInt = null;
int? nullableInt = null;

Line 1 doesn’t work, because null can not be assigned to a value type. Line 2 does work, because of the question mark behind the variable type.

Under the hood of nullable types

So what happens here? In fact the question mark in this context is syntactic sugar for Nullable<T>. So what happens is that nullableInt in the previous example is actually of type Nullable<int>. Because of this the variable now has a property Value, which contains the actual value if one is a available, and it gets HasValue of type bool, which indicates if a value is actually available.

Note that Nullable<T> is defined as a struct and is therefor a value type itself. Also note that T is limited to contain structs. Therefor it is not possible to apply Nullable<T> (or the question mark suffix for that matter) to a reference type.

Nullable types in generic methods

Now let’s get to the point. Let’s say you want to do some generic handling of input. So write a generic method with a signature like the following:

static string GetStringRepresentation<T>(T value)

You can simply call this with both non-nullable and nullable types. Because the compiler infers the type variable T I don’t even have to tell it what type value is explicitly. Let’s say we have the next program.

static void Main(string[] args)
{
	int nonNullableInt = 0;
	int? nullableInt = null;

	Console.WriteLine(GetStringRepresentation(nonNullableInt));
	Console.WriteLine(GetStringRepresentation(nullableInt));

	Console.WriteLine("IsNullable(int): {0}", IsNullable(typeof(int)));
	Console.WriteLine("IsNullable(int?): {0}", IsNullable(typeof(int?)));

	Console.ReadLine();
}

The output of line 7 is actually an empty string. This is because the call to value is actually boxed because the call to ToString() is actually a call to object.ToString(). Because nullableInt is in effect null, it will return an empty string.

Obviously there are many reasons why that might not be correct for a nullable int, so let’s find a way to handle nullable types better in our generic method. The first thing to do is to detect if the passed in value is actually a nullable type. To do this we can use the type system:

static bool IsNullable(Type valueType)
{
	return valueType.IsGenericType && valueType.GetGenericTypeDefinition().Equals(typeof(Nullable<>));
}

What this method does is use the type passed in and checks if it’s a generic type. If it isn’t, it can’t be nullable. It needs that check because otherwise our second check on GetGenericTypeDefinition would throw an exception as it would not have a generic type definition to return. Our second check compares the returned generic type definition with the Nullable<> type. Another task we need to accomplish is to get a default value for a nullable type, or, more specifically, get a default value for it’s generic argument:

static object GetDefaultValue(Type nullableType)
{
	Type valueType = nullableType.GetGenericArguments()[0];
	return Activator.CreateInstance(valueType);
}

This method takes a type and gets the first generic argument. This code assumes that you’ve already made sure this type is actually a nullable type. Normally you could use the default keyword to get a default value from a generic argument by writing default(T). As in this case we don’t have an actual type declaration, but a Type instance, we need a different approach to get a default value. If you’d check out the C# 4.0 reference you’d find out that the default keyword simply calls a default constructor on the type passed in and returns the result. To mimic that behavior, I simply call Activator.CreateInstance with the underlying type to get a default value. Now if we rewrite our GetStringRepresentation method to use the new methods above, we would end up with something like this:

static string GetStringRepresentation<T>(T value)
{
	if (IsNullable(typeof(T)) && value == null)
	{
		return GetDefaultValue(typeof(T)).ToString();
	}
	return value.ToString();
}

Now, if we call this method with a int? it would return 0 instead of an empty string. For your convenience, I’ve uploaded a sample project here. I hope you’ve found this an interesting tour around nullable types. If you have any questions or comments, please use the comments form below. Thank you for reading.

Bigger is not always better: Communication breakdown

noreply@blogger.com (Jonathan van de Veen) — Thu, 04 Aug 2011 12:52:00 +0000

Using Excel 2010 on Windows 7 to make an estimate for a new project, I ran into this problem that I think is both funny and a bit frustrating.

As we all know, Microsoft is a huge software company, with many developers and lots of QA people. Where they used to suffer from a reputation of poor quality, recent years have shown us Microsoft can make decent quality software with great features. Part of that ability comes from having lots of people working on projects. However, it can work against you as well, as Microsoft demonstrates in the following bug.

I’ve been a keen user of Excel for the past decade for a lot of different tasks. From simple estimations to long-list and complete data conversions, I’ve done it all with Excel. One of the long used features is the ability to block indent inside a cell. You know, these two buttons:

I use this feature frequently, so I figured a keyboard shortcut would make me more productive. Fortunately for us Microsoft makes finding out shortcuts very easy. Just hover over the button:

Mm, something is not smelling right, but hey, maybe the Office developers did a great job on implementing these shortcut keys. So let’s just try it…

…No. That’s not what I wanted to happen. Instead of indenting the contents of my cell, it showed the Windows 7 process switching dialog thingy or whatever it’s called. You know, the preview of your open windows?

Turns out both the Windows 7 team and the Excel team are using the same shortcut here. Excel has been using it for a while (since before Excel 97!), but in Windows this was only introduced in Windows 7 for the first time! Didn’t anyone figure “hey, we wanted to use this keyboard shortcut in Windows 7, but it’s already used in Excel 2010 so let’s have a meeting to discuss this”? In fact, shouldn’t a company like Microsoft have some sort of registration on what keyboard shortcuts are used where?

Even worse, Microsoft knows about the problem right now and has it published on their website. Is it really that hard to get a patch out for Excel to assign a different shortcut key to this function?

There is a lesson to be learned here. If you work at a small software company (with say, under 20 developers), make sure you utilize this advantage of a small team and communicate! It can make sure that your next product / project will be great.

Code generation is a basic developer skill

noreply@blogger.com (Jonathan van de Veen) — Thu, 30 Jun 2011 13:48:00 +0000

“Give a good developer a 40 hour task and he’ll spend 39 hours writing a program that can do the task in 1 hour.” – unknown

Really? This seems risky!

It can look risky, can’t it. You might end up with a half working program and not enough time to do the job. There are approaches that reduce this risk, but more on that later.

Why should you even bother? Simple math tells us it’s not faster to do so (at least not in the above example). However, experience tells us that in general most jobs will be repeated. Now, with the above example, if it’s only repeated once, you have a 39 hour profit, as you can now do the job in 1 hour.

Obviously, things don’t work like that in the real world. Automating a process usually takes longer than actually manually doing the work. In fact, when you want to automate something, that usually means you first have to do it by hand at least once, so you know what you’re automating.

But it’s also unlikely that you will only repeat a process just once. Often tasks are done many times and then it becomes worth the effort of automation. Let’s look at a current example.

A common case

As some of you may know, I’m working on a line of business application with a Silverlight UI and a WCF service layer (although the used technology doesn’t really matter for this case). Part of writing most line of business applications is writing business objects that represent your model and writing logic to retrieve data from the database, either directly or through some ORM.

Another part is writing lot’s of “forms” if you will, to input data into. Usually these forms follow a common pattern. Both of these parts, the business objects and the forms, contain a lot of repetitive work to do manually. Some of the code can be moved into generic super classes, but most of the code is of a nature that doesn’t allow for generalization. This is where automation comes into play. In this case, why write all that code yourself if you can also generate it?

When we first started out with this application, we did set out to build some code generation. Unfortunately most of us didn’t have a lot of experience with code generation as such, so wrong choices were made (C# generation with a StringBuilder is not the way to go and the same goes for Code DOM). After the initial versions of our code generation engine, we were forced to spend all our time on getting a first release out to our customers and no more work on automation in any form was a predictable result.

So, are you a good developer?

We all fall into this trap at some point. I know I have several times. You let a deadline pressure you to much, or you go back to your old naïve thinking saying that this will just be some demo code and it won’t end up in production anyway. Whatever the reason, we all end up breaking our own rules from time to time.

Don’t get me wrong, that doesn’t make you a bad developer. In fact it means you are focusing on how to providing value to your customers, which is key for a good developer. Also, breaking these rules from time to time keeps us alert on why they are rules in the first place.

However, we should also take time every now and then to take a step back and have a look at what we are doing. In my case I found we should put a lot more effort in making our development process more efficient. This means making more code generic, refactoring existing generic code to fit our needs better, and generating more code.

Generating code, a daunting task?

To some developers generating code may seem like a daunting task. Don’t worry. That’s a good thing, because you won’t just dive in without giving it some thought. As you can see from our example, having a developer that things “Oh, this is easy. I’ll just take a StringBuilder and push out lots of .cs files" is far from ideal.

Actually the first step in code generation is not writing a code generation process. It all starts with meta data. If you don’t have something to base your code generation process on, then how are you going to generate code in the first place?

In case of a line of business application, often your data model can provide you with some meta data. In fact we started out with a stored procedure that would take meta data from our data model in SQL Server and put it in another database. Then we would generate source from that data.

Another source for meta data may actually come from people entering that data into an application. We use that too, to allow our functional specialists to provide us with information to generate the forms in the GUI.

Once you have your meta data in some form or the other, you can write your generation process. Actually, that part doesn’t have to be hard. In the end it’s just creating text from data, which most web developers have been doing for years. The main problem is integrating your process with your IDE. That is where T4 comes in.

As some of you may know T4 is already used in Visual Studio to generate source code. It also provides you with a simple button to trigger the execution of all T4 templates within your solution. This is obviously a lot easier than adding files generated by some external program.

As for editing these templates, Visual Studio doesn’t come with standard support for this (it’s treated as text by default, so no IntelliSense). If you are serious about T4, you should check out Tangible. They have an editor for T4, integrated in Visual Studio, both in a free edition as well as a pro edition with more features. Both features do come with IntelliSense and Highlighting, but the free edition only supports limited namespaces and assemblies. You can find them here.

But what about those risks?

I hear you. Starting with this it can be scary. The trick is to start small and then keep on growing your code generation process to support more and more scenario’s. This way you can work on your code generation in iterations and have a working end result often. This reduces the risk, because you can, at any point, decide that you will no longer extend the code generation process, but extend the generated code by hand.

Also, before generating any code, you should have at least written and tested it once, to make sure you know the structure of your code and what meta data is needed in order to generate it. This way you also know you will end up with code that works.

Conclusion

I hope you realize the potential of code generation and how this can really make you more productive. I can tell you from experience that it also makes software development more fun, even if you have to build a lot of the same. With the right tools I’m confident that any development team could benefit from automating their work.

“A developer is only as good as his tools”

noreply@blogger.com (Jonathan van de Veen) — Mon, 09 May 2011 14:12:00 +0000

In this post I’d like to share some tools with you, that I think may be of use for any developer.

You may have come across the expression that is the title of this post. Some think this is not true and others swear by it. I’m sort of a middle of the road kind of guy when it comes to tools, as I do think that a tool is only useful when used effectively. However, our lives would be a lot harder without proper tools for development. For example, development becomes a lot easier with a highly tuned IDE like Visual Studio 2010, instead of typing code in notepad and compiling it directly with msbuild.exe.

As it turns out, I’m migrating to a new laptop for my work, so it seems like a good time to write about some of the tools I install on my new laptop.

As people tend to get very emotional on this subject:

DISCLAIMERS: This post is by no means sponsored in any way, except for the Google ads already in place. Also, most of these tools are NOT specific for developers. Some are useful for most PC users, some are useful for IT people and some are developer specific. This post is also by no means exhaustive. Finally, I will not talk about the obvious things, like Visual Studio, SQL Server, a Virus Scanner, etc..

LiberKey

One of the very first things I always install on a PC I use is LiberKey. This is a free tool to manage your tools. It comes with a massive (at the moment of writing this) 309 applications in 10 main categories, ranging from office tools to system utilities and anything in between. Applications like Notepad++, Paint.NET and HxD are all included.

The cool thing about LiberKey is that you don’t have to actually install anything to your hard disk. All the software just works, whenever you have LiberKey active, including full OS integration things, like explorer integration. It also deals with keeping all these applications up to date and new tools are added often.

Check it out at www.liberkey.com.

NuGet

If you haven’t heard about NuGet, you should check it out. This is basically an extension for Visual Studio 2010, that allows you to search for open source libraries and then immediately install them and add them to your Visual Studio projects.

What makes this so useful, is that you no longer have to and find a download source for these libraries, find out how to install them and then add them to your projects. You need a framework for dependency injection? Right-click your project file and click “Add Library Package Reference…”, type dependency injection in the search box and it will come up with open source libraries for that. Choose one, click install and you’re ready to go.

Check it out at nuget.codeplex.com.

Fiddler

If you’re going to do some work involving HTTP, this tool rocks. It allows you to follow HTTP and HTTPS traffic as long as the client runs on your machine. You can inspect messages going back and forward, look at time lines and even create your own messages.

Check it out at www.fiddler2.com.

XMind

I’m a fan of mind mapping. I use the technique for organizing my thoughts when working on complex problems and also for certain types of meetings to make notes. XMind is a great tool for making digital mind maps. I has great keyboard support, which makes me very productive without distracting from the though process.

Check it out at www.xmind.net.

smtp4dev

As a developer I frequently need to write some code which sends out emails. As you may be aware, testing code like that against an actual SMTP server in production is a hazardous operation. Not only that, it also tends to clutter your inbox with test emails. In some environments it can actually present security issues as well, where virus scanners block unknown processes which send emails, in order to prevent malware attacks.

To actually have a way of testing code that sends emails, I use this great open source tool, called smtp4dev. It runs like an actual application, but it also functions as an SMTP server on your machine. It has a UI that allows you to easily configure and manage it. The UI also allows you to inspect any emails received by it’s SMTP server. Further more, it doesn’t relay any emails, so you can’t accidentally send out emails.

Check it out at smtp4dev.codeplex.com.

The Regulator and Regulazy

Two great tools (by the same developer) for people who need to do work with regular expressions. Regulazy allows you to quickly create a regular expression from a piece of text through a UI. Although it is limited in what you can do, it does usually provide you with a good basis to extend on.

The Regulator is a more comprehensive tool, which allows for better analysis and testing of regular expressions.

Check both of the out at osherove.com/tools/.

LinqPad

If you work on a project that uses Linq and/or Entity Framework, LinqPad is the tool for you. This tool allows you to run Linq queries against your own Entity Framework model on the fly. This makes it a great tool for testing Linq queries, without actually having to compile and run your application every time. For the TDD evangelists out there who will tell you, you should never have to run your application to test such a thing, not having to compile can be a great advantage as well.

Check it out at www.linqpad.net.

Conclusion

Those are just some of the tools I use. Do you have any other tools you think people should at least know about? Leave a comment.

Recruiting a new developer

noreply@blogger.com (Jonathan van de Veen) — Wed, 23 Mar 2011 08:53:00 +0000

In this post I’d like to reflect on our recruitment efforts for a new developer. We’ll have a look at how to go about matching people to roles in a team and how to actually find people.

Disclaimer: I should point out that this post was written in the Netherlands and focuses on how things are done in the Dutch culture. This may very well differ from your culture. I do not claim to be an expert either. Anything in this post is based on my experience so far.

Disclaimer: Although you can always feel free to send me your resume in order to apply for a job, I can not guarantee that we actually have an opening.

We need someone, but who?

Usually it’s obvious there is a need for someone extra on a team. Everyone is busy, yet the back log keeps growing. The problem is, getting the right person for the job is not as simple. There are all kinds of things to think about when hiring (or looking for a job, for that matter). Some of the questions that I think need answering:

What is the level of experience and knowledge needed?
How would I describe the role we’re trying to fill?
What is the type of person we are looking for?
What will the job look like in, lets say, three years?

These are obviously generic questions and there are also some more specific questions to be answered, but lets have a closer look at each of these questions.

What is the level of experience and knowledge needed?

This is obviously the single most basic question one can ask about a job. However the level of detail and the correctness of the answer is imperative for a successful cooperation between employer and employee. Obviously the employee needs to have the experience and knowledge needed to actually do his or her job. But the world doesn’t end there.

Hire someone who can barely do the job and you risk ending up with someone who’s frustrated, or spending loads of time and money to get the person in question to actually do their job.

Hire someone who over qualifies, though, and you risk boredom and a very short lived (and therefore expensive) cooperation.

So judging the level needed for a job is crucial.

How would I describe the role we’re trying to fill?

Having an accurate description of the role you’re trying to fill is again very important. Especially when hiring a developer this description goes a lot further then just saying: “we need a dev who can do tech A, B and C”. You should also know what you expect in terms of involvement in the team and the organization. Obviously this also depends on the kind of organization you’re in and on how the team is organized.

Getting this information across during an interview seems trivial, but without preparation it’s nearly impossible to do right. This becomes even more tricky when the demand for certain people goes up. At that point it becomes important to ‘sell’ the opening you have to candidates. Obviously you shouldn’t over do this. If it doesn’t fit, it doesn’t fit and you should not do business. It’s in the interest of both parties to determine if the candidate is suitable for the role. However, when doing an interview you should be prepared to move away from the ideal and look at options for making a role fit for a candidate.

Knowing what you’re looking for and knowing the options you have to adjust to a candidates needs is important, especially if you have a limited amount of candidates.

What is the kind of person we are looking for?

Off course things don’t end with can and will the candidate do what you need him or her to do. Making sure the candidate will feel at place in the organization and, more importantly, in the team is very important. An important point here, is to be aware of the culture, size and maturity of the organization and the team.

Look at what type of companies a candidate has been working and ask for an opinion on some of them. This should give you a picture as to what the candidate is looking for. It is also a good idea to describe the ‘soft’ part of the company to the candidate.

The point here is that having, for example, a very formal character in an informal organization is going to create friction and both parties will suffer.

What will the job look like in, let’s say, three years?

Let’s face it, you’re not going to hire someone to let them go again in a year. It would be costly to do so. So you should be thinking ahead. Where will this person go after a year, two years, or three years maybe?I picked three years, as this is, in my opinion, as long as you can somewhat realistically make any plans for, especially in IT.

It is important to have a plan as this will make a difference in who you hire. Also, a good candidate will ask about opportunities for him or her to grow. Make sure you have a story, but also be prepared to be creative with it. You might need to make adjustments based on the candidates possibilities and preferences.

Conclusion

Obviously there is a lot more to think about when hiring a new developer. The most important thing I think you should take away from all this, is that you should spend time planning and making sure you know what you’re looking for.

If you have any thoughts or questions on this, please leave a comment below. If you live in the Netherlands and are looking for a job, leave a comment with your email address (I won’t publish it), contact me at LinkedIn, or send me an email: jvandeveen at gmail dot com.

Adventures while building a Silverlight Enterprise application part #41

noreply@blogger.com (Jonathan van de Veen) — Mon, 21 Feb 2011 10:18:00 +0000

In this post we dive into this tiny change that caused a weird bug in the WCF communication between our Silverlight client and one of our web services.

In the last post I mentioned we were about to go live with a new product. It’s my pleasure to report we have gone live since then and everything is going well, except for some minor issues. I also described a process where we would do to improve our development process. At this point we are in full swing with doing that.

While we are doing that, we have also planned a new release to fix the minor issues I mentioned before and add some small features for which we didn’t have the time earlier. As I was working on a bug fix for one of these issues, I ran into a bug that puzzled me at first, but once I figured it out was trivial. Because it took a fair amount of debugging and analyzing to get to it and it highlights a problem with refactoring some parts of your code, I figured I’d share it with all of you.

Sypmtoms

As I was testing this bug fix I had been working on, I ran into some weird behavior. Some of the data in the Silverlight client would not load, or so it seemed. At first I thought this might have something to do with the fix I did.

I started debugging the service I had changed and found out that, for some reason an id we send to the service in order to get the right data in, was empty. This was weird, because I didn’t change anything to that part of the code. I debugged the Silverlight client and found out that it did send the id correctly! That was weird. The only thing I could conclude was that the id must have been getting lost somewhere along the line between the code on the client and the code in the web service.

Debugging the issue

As this appeared to be some kind of a communication issue, I figured I’d break out my old friend Fiddler. I reconfigured the client to access the web service involved through http://ipv4.fiddler, instead of http://localhost and I started reproducing the problem. I browsed through the requests and there it was, the parameter named Id was correctly being transferred and had the right value. For some reason though that value was gone as soon as it reached my web service.

The only conclusion I could make at this point was that it had to have something to do with the service contract. I took a look at the the interface for my service and immediately I spotted a potential problem. The parameter was not called Id anymore, but it was renamed to id! This was done as a result of a warning produced by FxCop. It marked the casing of the parameter name as incorrect, so I corrected it, but then completely forgot about it.

An easy fix

In order to get this fixed, I made sure the casing in my .svc code behind was corrected as well. Then it was all a matter of rebuilding and publishing the service and refreshing the service reference in the Silverlight client and the problem was gone.

As it turns out it was a good idea to do a release on the cleaned up code, because this way we run into some of the consequences early on. It also shows that there is always a danger to changing service contracts, because it creates incompatibilities that might not be so easy to spot.

Adventures while building a Silverlight Enterprise application part #40

noreply@blogger.com (Jonathan van de Veen) — Thu, 06 Jan 2011 12:18:00 +0000

As we are close to releasing our first version and we will have some time on our hands to do clean up on our sources, today we want to look at some of the things we would like to do in order to improve our codebase.

Well, I never thought this series of posts would last this long and have this many parts. We might even make it up to part 42 :-).

The situation

As I wrote above, we are close to releasing our first version to the client. If you’ve followed this series of posts, you may know that we’ve been working under the gun for the past eighteen months or so to actually get to this point. To actually have our team meet deadline after deadline, we had to take some shortcuts here and there.

“What?!”, I hear you scream, “Deadlines are never an excuse for shortcuts!”. Sure, in the academic world that’s true, but in the real world business of software, this doesn’t hold up. Even large software companies like Microsoft constantly make choices about if they can do without a certain workload before releasing the next version. We make these choices on a daily basis as well.

But now we are faced with a rare (or dare I say unique) opportunity. We are actually allowed to reserve time to go back and improve the quality of our codebase. We also are allowed to invest in improving our software development platform in order to improve quality or increase productivity.

The approach

This begs the question, what are we going to do with this time? I’ve composed a list of things we want to do and I’d like to dive into each of these points in this post. Please keep in mind that the following points are not in order of importance. They are in this order for different reasons.

Remove any compiler warnings
Do a complete code review, dealing with the following
1. Make sure all code adheres to our coding standards
2. Make sure all code can deal with multi language and culture differences
3. Decide on what to do with TODO’s
4. Check if workarounds for bugs in external code can be removed
5. Deal with obvious performance killers
6. Mark unclear code for refactoring
Migrate our main web services to .NET 4.0
Migrate to Entity Framework 4
Analyze what code can be made more generic and refactor
Improve our code generators
Get automated testing
Get nightly builds

Remove any compiler warnings

One might argue we should have removed compiler warnings as we went along. And when we started out we did, however as we were confronted with deadlines, this was one of the things we decided we could drop. The reason for this is that we don’t need this for our application to work correctly.

So why fix it? Well, some of these warnings can prevent bugs and some warn about inefficient code. In order to get all our code up to a higher standard it therefore is useful to adhere to these warnings and fix them as necessary.

Do a complete code review

We do this after we have fixed the compiler warnings as we then end up with code as it should be. We may have fixed things we otherwise would have to fix in the review. It also works the other way around. We also now have a codebase without compiler warnings, so we can actually keep it that way as we change code.

Make sure all code adheres to our coding standards

This is and obvious advantage to create. We want all code to follow some rules so we can make assumptions about it. This does not only make it easier to make changes or write new code. It also makes it easier to read, which makes up for around 80% of any developers workload.

Make sure all code can deal with multi language and culture differences

We have international customers who want to use our software in a different language (the default is Dutch). Also their culture settings may differ. We therefore need to make sure we check our existing code base to see if this is taken in account and if not, fix it.

Decide on what to do with TODO’s

During the development we have had times where we though “we should change this part of the code to work so and so.” As we where up against deadlines and most of these changes would have serious impact, we marked these spots with TODO comments.

Again, I can here you say, “well, execute them!”, however we will find TODO’s that are outdated, either because they ended up being implemented directly, because we had too, or because they are no longer relevant because of some decision or insight we had.

Check if workarounds for bugs in external code can be removed

As we work with some third party code, either written by Microsoft or bought from a third party, we encounter bugs in these parts every now and then. We found it good practice to mark any code that we use as a workaround with a comment, describing what we did to work around the bug. Over time these parts might have been updated and chances are we can remove some of these workarounds, benefitting both performance and maintainability of the source.

Deal with obvious performance killers

By this I mean we do a quick analysis of what is happening in a piece of code and see if we can find any obvious inefficiencies. If we do, we remove them on the spot.

We won’t do any performance measurements, so the inefficient code has to be obvious. For example, we might encounter code that uses two queries to retrieve data from the database, where we could have used only one. You don’t need a masters degree to figure out, that you should make this into a single query.

Migrate our web services to .NET 4.0

We’ve already migrated our client to Silverlight 4.0 and Visual Studio 2010. However, because we didn’t want to take any risks with our web services, we didn’t migrate them at the same time. Now that we have the change bringing our web services to .NET 4.0 (and obviously moving them to Visual Studio 2010 in the process) will make our lives a lot easier.

Migrate to Entity Framework 4.0

Right now we are still on Entity Framework 2.0, which was the latest version as we started development late 2008. We haven’t upgraded since, but because there are several improvements in both performance and functionality, we would like to move to Entity Framework 4.0.

Analyze what code can be made more generic and refactor

Right now we are not keeping it DRY all throughout the code base. Especially when it comes to implementing specific functionality we ended up solving the same problems more then once and sometimes even in different ways.

Also, although we tried to take as much care as possible to prevent repeating code, we still had some cases where there was no time to implement a proper generic solution for a recurring problem.

Now we have time to analyze these cases and refactor code as needed.

Improving our code generators

Some of you may know that we use code generators for several parts of our code. Currently we generate our business classes, their collections, XAML for our UI (including an empty code behind) and resource files to go with the XAML.

To create this code, we use metadata from our database models and metadata that’s supplied by our functional specialists. They supply us with information to generate the UI and the resource files.

In order to make this process more efficient we would like to redesign the user interface for the tool that the functional specialists use and change the way we bring all the metadata together. Then we can actually use the combination of this data to generate better and more source code.

Get automated testing

I know, we should have done this from the beginning. However, everyone knows this is one of the first things management will eliminate if time becomes an issue, especially if the team is not experienced in the field of TTD and needs some time to get used to it.

This does pose a problem right now, because it seems almost impossible to introduce actual unit testing after such a long time of development, because we would need to introduce dependency injection in such a large code base. We’ll have some research to do in this area and we’ll have to make some though choices.

Get nightly builds

This one will be a though one to realize, considering all the parts we have and the way we work together, but it does have some important value in integrating source and making sure all the parts keep working together.

A mistake or a consequence?

As you can see there is still a lot of improvement to make. So where did this come from? Well, most of the issues I addressed were results from conscious choices we made. We knew this would have consequences later down the line.

Other things, like the quality of the code generation and a need to rethink our approach was due to new insights and working with some of the concepts in our organization for the first time.

And some things, like code reviews have been a process to get implemented in the culture of our organization. Changes like these tend to take time (and lots of it).

Because of this I have to conclude that the fact we have to do all this work is a consequence of choices we made. And because we knew most of the consequences as we made those choices, I feel it’s a good thing, especially if you consider a lot of development teams do not get this chance.

Have you ever been in a similar situation? Let us know your experience by leaving a comment.

Adventures while building a Silverlight Enterprise application part #39

noreply@blogger.com (Jonathan van de Veen) — Thu, 09 Dec 2010 15:35:00 +0000

In this post we look at an issue we had while implementing a specific form of data entry for our application. Basically there was an oversight in our reasoning about CRUD operations that needed fixing. It also shows the power of GUID’s.

The situation

We are currently in the final sprint towards delivering our first new product release to a group of customers. As one might expect, we are in the process of testing the functionality that was built so far and we ran into some issues. One issue we encountered was hidden deep down in the framework and hidden behind some other bug in the Silverlight UI.

To get a better understanding of the issue we found, it’s important to have some basic knowledge about how are framework works, especially in terms of creating and updating records. We decided early on that creating a record is basically the same as updating one. The only difference in our service call is that we provide an empty GUID instead of an existing GUID. The framework should then detect that there is a new record, create a GUID for it and insert it into the Entity Framework context and all works well (except for the incidental problem with insert).

Recently we added two modules to our UI, which consist of a data grid. The data grid is filled with a default set of rows from one table and once the user fills in certain values, records are created in another table. In order to make this work there is a view which is added to the entity framework to provide the correct rows. We then have some code in the service layer to make sure we don’t get double rows when there are records available in the second table.

The bug

A bug occurred when we tried to enter a value into a record that was not yet available in the second table, saved the modules data and then change that value. Now we ended up with two records in the second table instead of one. A short debugging session revealed the cause. Both update actions where send with an empty GUID, so both we’re interpreted as an insert.

The solution

Solving it was not as easy. In order to prevent these kind of bugs, we already return an updated (or new for that matter) business object from the service. However, for new records, there is no id known (it’s empty), so we have nothing to match to, meaning we can’t update the record in the data grid, which leads to our problem.

In order to fix it, we would want to create a new GUID in the Silverlight client, send it to the server and use it there as well, however then our service would not interpret this as an insert. What we can do is send this new GUID in the actual property of the business object which we want to update. This did require an update of the framework where, in case of an insert (the passed GUID is empty) we now check to see if the business object has a GUID as it’s key already available then we use that GUID instead of generating a new one.

So now, we can generate a GUID in the client, send it with the insert and know which object was updated as soon as it is received in the service call back. This allows us to match the rows in our data grid to the business object and update accordingly, meaning no more double records.

In the end, this his the code we created to make sure we get a GUID if one is available:

   1:              idPropertyName = string.Format("{0}{1}", idPropertyName, IdSuffix);

   2:              PropertyInfo idProperty = dataType.GetProperty(idPropertyName);

   3:              Guid id = Guid.NewGuid();

   4:              if (idProperty != null)

   5:              {

   6:                  if (changedData.PropertyDictionary.ContainsKey(idPropertyName))

   7:                  {

   8:                      PropertyObject idPropertyObject = (PropertyObject)changedData.PropertyDictionary[idPropertyName];

   9:                      Guid requestedId = (Guid)idPropertyObject.CurrentValue;

  10:                      if (!requestedId.Equals(Guid.Empty))

  11:                      {

  12:                          id = requestedId;

  13:                      }

  14:                  }

  15:                  idProperty.SetValue(data, id, null);

  16:              }

  17:              this.Id = id;

  18:              SetProperty(this.IdProperty, id);

Adventures while building a Silverlight Enterprise application part #38

noreply@blogger.com (Jonathan van de Veen) — Tue, 16 Nov 2010 14:13:00 +0000

In this post I want to look into a problem most of us will encounter while building a Line Of Business application in Silverlight. It involves UI paradigms combined with the asynchronous communication model.

The problem

We have all run into it at some point. You have a form that a user needs to fill out, which contains a couple of fields and something like a save button. When the user clicks the save button you validate their input and if the validation fails, you block the save action.

All is well. The user can correct the input and try again at will. However, validation has a downside. It’s limited to either true or false and no way in between.

Let’s consider a scenario where you would like to have confirmation from the user. Now you could hack this into the validation system by using a flag and if the user doesn’t change the value and saves a second time it must be right. However I don’t think this is a very good solution and it would confuse users.

I need a dialog!

So we want some other way of asking the user for input. The usual way of doing this would be to present the user with a dialog where the user can press a button to choose whether or not it’s ok to continue. We worked with them for many years and they are intuitive to use.

In Silverlight however they pose a problem. Of course you can use the dialog included in the framework. However this uses the browser to present the dialog and it’s not a good looking thing. More importantly it’s very limited in it’s functionality. Fortunately it’s not that big of a deal to just build a user control that provides the functionality we want. We can even make it modal by just placing a Canvas over the entire UI and then putting the user control in a Popup (or use a childwindow for that matter).

Then the issues start. If you are to build a user control to display dialogs, like we have done, you most likely implemented it using a callback to signal the application that the user has clicked a button in your dialog and you’re done. Although this is a classic asynchronous model, working with it can become problematic.

Splitting code

Let’s go back to our scenario. We have a form with some fields. Some fields get validated and some can lead to a dialog. The steps we would take to save our data before introducing the dialog look like this:

Validate the input
If all fields are correct start saving the data
Once the save is completed, refresh the current data

Now if we introduce a single dialog that allows the user to choose between stop and continue the steps are like this:

Validate the input
If all fields are correct, check if the dialog is needed
If the dialog is needed, show the dialog
Once the user has made a choice and wants to continue, start saving the data. If the user does not want to continue, stop.
Once the save is completed, refresh the current data

Looking at these steps, we go from one event handler for the save completion, to adding a callback for the dialog. So instead of having two methods with code for saving, we now have three methods. And with every dialog we add, we add another callback, thus another method containing code for our save action. You can see how this gets out of hand quickly.

So make it synchronous…

Well, that would solve our problem, however… Silverlight only has one dispatcher for the UI-thread. This means that as soon as you block the UI-thread, your application becomes unresponsive (basically it just hangs until you unblock). While displaying a dialog that is not a very practical thing to do, as the user would not be able to click a button (defeating the whole purpose, right?).

In effect this means there is no straight forward to provide synchronous UI interaction. It’s going to be event based and thus asynchronous. There is a way around this, however. You can’t block the UI thread, but you can block any other thread, so if we move all the code involved in saving to a background thread we can eliminate a callback for the dialog. Here is some example code to show you how calling such a dialog would look:

   1: private void saveButton_Click(object sender, RoutedEventArgs e)

   2: {

   3:     // Make sure to create your dialog while still running on the UI thread

   4:     _dialog = new DialogWindow();

5:

   6:     // Pass off the save logic to another thread

   7:     ThreadPool.QueueUserWorkItem(SaveData, this);

   8: }

9:

  10: private void SaveData(object stateInfo)

  11: {

  12:     if (DialogWindow.ShowDialog(_dialog, stateInfo))

  13:     {

  14:         // Actually save the data

  15:     }

  16: }

So basically you run anything involved with the save action (except for validation, perhaps) on a background thread. I also pass a reference to the calling UI control. The reason for that will become clear soon.

While running in the background thread, you can call my DialogWindow.ShowDialog method and it will block your thread. This means you can use its result right away.

Run code on the UI thread

In order for this to work we obviously need to be able to run some code on the UI thread and then block our thread while waiting for a result. Here is the code for the ShowDialog mehtod:

   1: public static bool ShowDialog(DialogWindow window, object stateInfo)

   2: {

   3:     bool result = false;

   4:     DependencyObject dependencyObject = stateInfo as DependencyObject;

5:

   6:     // If we got passed a dependency object and we have access

   7:     // to it's dispatcher then we are on the UI thread and we can't block

   8:     if (dependencyObject == null  dependencyObject.Dispatcher.CheckAccess())

   9:     {

  10:         window.Show();

  11:         return false;

  12:     }

13:

  14:     Action action = new Action(window.Show);

  15:     // Run the Show method on the UI thread

  16:     dependencyObject.Dispatcher.BeginInvoke(action);

  17:     // Block this thread until we have a result

  18:     while (!window.DialogResult.HasValue)

  19:     {

  20:         Thread.Sleep(50);

  21:     }

  22:     result = window.DialogResult.Value;

23:

  24:     return result;

  25: }

First thing we do is check if we are actually on the UI thread. If we are, we don’t want to block. Be aware that the CheckAccess method is not shown by IntelliSense for some reason. It compiles without any problems though.

If we are not on the UI thread, we can actually use the dispatcher of the dependency object passed to us, to actually run the Show method on the UI thread. Then we simply wait for the result and pass it back.

Conclusion

In order to have a synchronous dialog we simply introduced a thread to run our code on which runs some code back on the UI thread and waits for it to complete. This should make it easier for us to compose more complex flows in our code.

You can download the code here.

What’s the fuss around Silverlight about?

noreply@blogger.com (Jonathan van de Veen) — Thu, 04 Nov 2010 08:03:00 +0000

Since day one of PDC10 there has been a lot of press attention for what appears to be a change in strategy for Microsoft around Silverlight. As I’ve gotten several questions from people concerning the stories published by several respectable sources and how this would impact the business, I decided to write down my take on this.

The PDC10 debacle

So what are people on about? It basically comes down to two things. One is the “lack” of attention for Silverlight on the PDC10. The other is Microsoft announcing that they completely back HTML5. It was elaborated on by saying that there has been a shift in the strategy around platform independent web technology. Also, because there has not been an announcement on Silverlight 5, people are scared into believing there will not be a Silverlight 5.

Most people interpreted this by concluding that Microsoft will limit their investments in Silverlight to the Windows Phone platform. Let’s assume they are right in their conclusion. If that is true, then it’s devastating for all those developers and companies who have invested heavily in Silverlight in the past years.

My analysis on Silverlight

So how do I look at this? Well, there is a number of things I see differently from most people. Let’s have a look at some of those.

HTML5 vs Silverlight

A lot of people see HTML5 and think it replaces Silverlight for the browser completely. Having worked with both HTML4 and Silverlight and having looked into what HTML5 brings to the table so far (it’s not done yet), I have to disagree.

First of all most people that make this statement think of Silverlight as a platform for media and games, similar to Flash. However there is a lot more to Silverlight. For one it is a lot better suited for Line Of Business (LOB) applications. Another major advantage is that everything works and looks exactly the same in different browsers and on the desktop.

Another major advantage about using Silverlight for web developemt is that you can use one programming language across different tiers, complimented by one markup language. Compare that to HTML and you immediately spot the problem. Not only do you need HTML for markup, you need CSS for styling and Javascript for client interaction. To top it all off you need some server side language to work with data. That’s four languages for something that Silverlight does with two languages. Are the decision makers of IT-world paying attention? Less is definitely more in this case.

Because of all this, and a great IDE in the Visual Studio / Blend combination, productivity in Silverlight is much higher and more business logic focused than the classic web application. This allows developers to provide more value more quickly, making more money for the companies and their customers.

But Microsoft has given up on Silverlight, right??

Well, did they? Let’s look at some facts here.

First of all this statement is all based on Microsoft communicating their support for HTML as the only true platform independent technology. People think that this is change in strategy, but in fact it’s not. Let’s face it, that is just reality. The downside on Silverlight is that there is no complete support for all platforms. I’m not only talking about Linux here. It’s also about a lot of mobile platforms. And it’s hardly realistic to expect to get that support for all these platforms. It’s not going to be cost effective to do so. In my book, this statement is stating the obvious and it would be stupid to say anything else.

But will Microsoft stop investing heavily Silverlight because of HTML5? I don’t think so. First of all HTML5 is far from complete. It will take at least another ten years to make it to the main stream in it’s full glory. Second of all HTML5 will have trouble adapting, because it is a global standard and everyone wants to have their say about it. Silverlight doesn’t have this issue.

And then there is the business side of things. Microsoft has invested so heavily in Silverlight, it would not make sense for them to stop now. They have cranked out three major releases of Silverlight in the past two years, including tooling for Visual Studio and support in Blend. They have invested in the Silverlight Toolkit. They brought Silverlight to the Windows Phone platform. And recently they introduced Visual Studio LightSwitch, which can generate complete Silverlight applications. Really, they have hundreds of millions of dollars invested in Silverlight, with great success as a result. And frankly, there is no reason for them to stop doing so, for all of the reasons mentioned before. As long as it keeps selling software for them, they’ll keep working on it.

Conclusion

So we can conclude that Microsoft will not stop pushing Silverlight forward in the foreseeable future. However, it is up to decision makers to stick with the facts and not go with the press buzz, or they will be investing in the wrong technologies, loosing a lot of money in the process. And it’s up to developers to stand by their choice of technology. There really is no need to suddenly change anything.

Just remind yourselves, what made you choose Silverlight as a platform in the first place? Exactly.

Adventures while building a Silverlight Enterprise application part #37

noreply@blogger.com (Jonathan van de Veen) — Fri, 15 Oct 2010 09:57:00 +0000

In this post we are looking at Entity Framework and specifically at some performance quirks.

The story

Recently we were testing part of our new software. One thing we noticed is that, besides the fact that the first time we load data, things are slow because of services starting up and Entity Framework loading metadata, also the first time we save some data, it is also slow.

We decided that is was worth investigating why this was happening. The goal was to at least come up with an explanation and some more insight into when this was occurring.

Analysis

Some preliminary analysis I made:

This was not the same thing as starting up the service and Entity Framework loading metadata because of that. I knew that because I had just loaded data from the same service and model that I was now saving to.
It was caused by something in the service. I could tell because we have a busy indicator active whenever a service request is pending completion.
It was not caused by any specific code for one of our business objects as it didn’t matter which object I would save. It was always slow the first time.

As you may know, if you are following this series for some time, we have a bunch of complex generic code that’s involved when either loading or saving data. I decided my first task was to identify exactly what line or section of code was causing the delay the first time around. To accomplish this I placed some debug code which would log the exact time to the debug output. This way I could easily calculate where a delay would occur.

By following this strategy I quickly found out that the delay for the first save occurred when calling SaveChanges on our model. However, reflecting the code for the method did not reveal any obvious cause for this. I searched on the internet to find out if anyone had already cracked this problem, to find out that only a handful of people had seen the same issue, but non of them actually resolved it.

As I still did not pinpoint the exact cause, I decided to build a test application to demonstrate the problem and hopefully get a better understanding. As we are currently still using version 2.0 of Entity Framework and running in .NET framework 3.5, I used Visual Studio 2008 to build my test application on top of the AdventureWorks database. I needed it to do basically two things:

Load a single object from the model to initialize the Entity Framework model
Save a change to a single object from the model and time the operation

To make my tests more flexible I decided on a WPF UI that allows me to control the above steps and it presents me with the results instantly. Check out the screenshot below:

The times displayed are in milliseconds. Let me explain what you see here. On top are two rows of buttons, one for the full AdventureWorks model and one for a model with only the Customers table (both models do connect to the same database). Below that is a listbox with performance results for each save action. At the bottom there is a general log to see what is happening.

Basically what you see in the performance logs is this. The first save action is done after loading a customer from the complete model. It takes a whopping 177ms. The next save action, after loading the same record again, takes only 2ms! This reproduces the issue we’ve been having.

Now the next step is doing exactly the same thing with a model that contains only the Customers table, so there is a lot less metadata to be loaded. It should be noted that both models are placed in separate assemblies to make sure metadata of one model is not interfering with the other. The time differences are a lot smaller between the first save (28ms) and the second save (4ms). Note that as performance is reaching lower limits times can fluctuate because of other processes on the system, which can account for the 2ms increase between the large model and the small model.

Because of this we can now conclude that Entity Framework loads some metadata as the first save occurs. Unfortunately we can not do much about that, but at least we now have an explanation.

Taking the next step

But the research didn’t end there. One of my colleagues suggested testing this with .NET 4.0 and Entity Framework 4.0 to see if there would be an improvement there. So I converted the project to Visual Studio 2010 and recreated both models using Entity Framework 4.0. The times are like this:

Saving the first time on the complete model: 82ms
Saving the second time on the complete model: 7ms
Saving the first time on the single table model: 36ms
Saving the second time on the single table model: 3ms

So work has been done in Entity Framework 4.0 to improve performance while loading metadata. Please note that my measurements were made on a system that is not exactly high performance and thus the measurements are not very accurate, however the differences between EF 2.0 and EF 4.0 with the first save are still significant enough.

Downloads

You can download the source of both versions of my test application:

Please be aware that the first project is a Visual Studio 2008 project and the second one is a Visual Studio 2010 project. You can however upgrade the first project, without changing the results.

Conclusion

So, although we can’t improve the performance of the first save action on an entity framework model right away, at least we can now explain what happens and what is the actual impact. As this only happens when IIS recycles the Application Pool for our web service, the impact is minimal. Also we can soften the pain as soon as we migrate our service to Entity framework 4.0.

I hope you have learned as much as I have writing this post. If you have any questions, please leave a comment below and I’ll be happy to help you out.

Developers 42 around for two years!

noreply@blogger.com (Jonathan van de Veen) — Fri, 08 Oct 2010 07:30:00 +0000

Today we celebrate the second anniversary of the Developers 42 Blog. In this article we look back at the past years and we look forward into the future.

Looking back

Wow, it’s been two years. Part of me feels like it’s been forever and part of me feels like I started yesterday. If I think about what I started this blog for, I guess it comes down to sharing my experiences as a developer with anyone who’s interested. However, if I now look back at what I got from this, I feel I got a lot more from building and maintaining Developers 42.

A learning experience

I guess I was a little ignorant when I first started out. I figured it would mostly come down to discipline to maintain a reasonable blog, but I found that it takes knowledge and skill to promote a blog and to keep everything running smoothly.

But also I found out that writing for a technical blog like this one, brings with it that you have to adapt your workflow to facilitate blog posts. One thing I find myself doing over and over, is generalizing scenario’s I run into in my day to day work. It can be very helpful in getting a better feel for the problems you work on, but it can also be a time consuming challenge to come up with good examples of something that is very specific to the field I work in.

Another thing I learned that surprised me, is the way referrals work. I never thought that putting a link in a signature for a forum would generate the results it did. Nor did I think republishing to CodeProject would bring me much hits but it does bring reasonable results.

Opportunity

Another thing I got from running Developers 42 is opportunity. Because of running a blog I get the occasional job offer, including offers to join interesting startups (unfortunately all kinds of reasons prevented me from jumping on to any of these so far). I also got invited to write for SilverlightShow.com, which I still enjoy doing from time to time.

I’ve also met new people and got the opportunity to help people from around the globe with issues they encountered, which is rewarding in itself.

Setbacks

Obviously is wasn’t all a big party. I’ve had to take a step back in posting at more then one occasion. Personal life and work life both interfered with my goals for Developers 42. This resulted in me only posting 18 times (not including this post) over the past year.

Defeat was also part of the deal. I decided to scrap the CodeEmbed4Web project, as it was clearly not going to a success. In fact I have lots of unfinished projects laying around, which I’m not likely to continue.

But enough of that. Let’s move on to more positive things.

The numbers

In fact the numbers might not all seem so positive. There was a reduction in both visitors and page views of around 27,5%. But if you compare that to the reduction of posts (around 35% less content), this suddenly becomes promising. If I would have posted the same amount as last year, hits could have doubled!

Looking at the number of subscribers reflects this positive image. There was actually an increase in subscribers of the past year. Check out the graph below!

I’m really happy with the progress made in this area. It it really motivating to know that people are interested enough to actually subscribe to my RSS feed.

Looking to the future

So what about the future? Well, if there is one thing I learned in the past year is that you can be to ambitious. I do have some plans for new things, but overall my main goal is to keep posting regularly. However if there is a topic you would like me to write about, even if it is only remotely connected to software development, I’d love to hear about it.

Further more, I hope to welcome more new readers in the coming year, both as subscribers, but also as loose visitors.

Thank you!

The last thing that remains is to thank you, the user of this blog, for sticking with me here. Without you this wouldn’t be much fun. I hope you have enjoyed yourself the past year as much as I have and that you have learned as much as I have.

A special thank you goes out to Dave Campbell from Wynapse, for so patiently following and linking to my blog. Another thanks goes out to Svetla from SilverlightShow.net for helping me out with getting published on their website.

I really value your feedback, especially if there is anything you would like to see different or added, or if you have any suggestions about topics to discuss. I hope to see you around this coming year and I hope we can share some insights.

Adventures while building a Silverlight Enterprise application part #36

noreply@blogger.com (Jonathan van de Veen) — Tue, 21 Sep 2010 13:26:00 +0000

This article is about appreciation. It may sound a bit out there, but please hang on. It is about my job of being a lead developer.

The story

Roughly two months ago we where confronted with a deadline, where one of the product managers told us he scheduled a demo for a large group of our customers. He included a list of the functionality he needed to demo in order for it to be successful. He also mentioned that some of the functionality currently planned would not be needed.

I immediately started planning towards this demo, to see if it would be manageable to reach the deadline. I ended up with a plan so tight, I felt it was doomed from the start. I discussed this with management and we agreed that in order to get this to work, the whole team would have to make structural overtime for four weeks in order to have a change of making it. All in all I was worried, but figured that working hard and keeping the right focus would actually be crucial in achieving the goal set for us.

The next thing I did was take a week off right after the deadline (including the day of the demo). You might think I’m crazy for doing this, but let me explain. The demo would take place on Tuesday. I needed to make sure that by Friday we would have a working demo. If not we should fall back to a PowerPoint presentation. To force that GO/NO GO moment, I figured simply not being there would be a good way.

After working completely focused on this deadline for six weeks and the whole team doing overtime for at least four of them, we managed to deploy a working demo on Friday late in the afternoon. It was a complex piece of work which integrated our Silverlight application with various services, Excel and Reporting Services using a complex flow of events.

So what’s wrong, then?

So far so good, but as I checked my email after a week of vacation on Sunday night, I missed something. No email reporting on the presentation, no note to the team telling them they did a good job, or a horrible one for that matter. I was disappointed. The whole team worked so hard to achieve something great and managed to do so against the odds and it appears to go unnoticed.

Obviously something must be wrong here. I talked to several people and found out that the product manager became ill the day after the demo. My manager and my managers manager (yes, I know it sounds silly, but it does work) both didn’t consider sending an email. I also found out that the total presentation, of which the demo was only a small part, didn’t go as planned, which greatly reduced the impact of the demo unexpectedly. It makes it understandable that non of the managers involved figured to send out a note as they were disappointed about the presentation, however it doesn’t form an excuse in my book and they should have send out an email.

Why is that important?

So what’s the big deal? We all have to work overtime every now and then? Well, the reason for doing overtime is important. If you failed to deliver on schedule because you misjudged the amount of work, or you messed up something, it’s fine to spend some of your private time fixing that, so you don’t get behind. However, if someone imposes some nearly impossible deadline, things should not be taken for granted.

Sending out a simple email stating your appreciation for the effort and flexibility of a team of people can make all the difference. This is especially true the next time around (yes, there is always a next time), but it also keeps people motivated to keep at it under normal circumstances.

What are your thoughts on this? Let us know by leaving a comment.

Microsoft introduces LightSwitch: Game changer or gadget showoff?

noreply@blogger.com (Jonathan van de Veen) — Fri, 20 Aug 2010 06:43:00 +0000

On VSLive 2010 Microsoft announced Visual Studio LightSwitch. In this article I want to take a short look at what it is, based on the information provided by Microsoft so far. Then I’ll share some of my thoughts on this with you about how this might affect us as developers.

So what is this LightSwitch thingy about?

Well, basically it is a new edition of Visual Studio 2010 that enables people to quickly build data oriented applications (are there any other types?). ‘Oh,’ I hear you say, ‘another Oracle Forms clone. That’s not going to be any good.’ Well, first of all I’d say go and take a look at the VSLive 2010 Keynote. Also there is a great introduction video here. And for more of a deep dive you can check out this video.

‘Ok, so it’s just the next Microsoft Access?’, I hear you ask. Well, not quite either. The big difference with the previously mentioned platforms is the fact that you get an application with a multi-tier architecture and you get to choose what your client is and what your data source is. You can even have multiple data sources and link them together. And if you need to take your application to the next level, there are all kinds of extensibility points in both the underlying framework and the IDE itself.

On the UI end they have figured a way to do better as well. In the IDE you don’t have a WYSIWYG editor, but a sort of a tree view so you can better see and understand what you’re doing. The generated UI is also based on templates, so you can easily update the look and feel of your application at one point. And then when you do want a WYSIWYG editor you just start up your application, click a button and your on your way.

All in all it seems that Microsoft has succeeded in building a tool attempted and failed by a lot of companies.

The holy grail of database oriented systems?

For decades a lot of companies have attempted to build a system where you would simply define a data model, press a button and presto, there is your new software. Most companies failed and some partly succeeded, but no one was able to build a system that would actually work in a broad spectrum of cases and still provide enough customizability to implement the very specific features needed in very specific applications.

Most of the time failure occurred because the software would be to generic and the resulting applications would not meet enough of the users needs. And when teams would attempt to implement code to satisfy these needs, they would either become to specific or to expensive.

But is it really that simple? I don’t think so. Most database oriented systems I’ve worked on/with always have some form of extra processing, connect to external systems in special ways or need some very specific UI. In practice I think that most application that professional developers work on will not be built with LightSwitch all of a sudden. There may be some, but not many. There will be many reasons not to build an application in LightSwitch, but the main reason will be that developers feel they might not have enough control to actually build was is required. Another important reason is that you don’t get to choose what technology is used to build your application.

‘So it’s not the holy grail, then? It must fail miserably’ I hear you say. Well, no.

LightSwitch as a game changer

LightSwitch will be a game changer. We will see less complicated systems being build in LightSwitch all over the place. For Microsoft I guess it is all about getting the smaller companies to go with them, instead of going open source. And they do give them an important advantage in allowing them to cheaply and easily build their own software and still keep that software scalable. So in that respect it is a game changer.

Another effect will not be seen until we have moved on a couple of years. Right now we see a lot of applications build in Microsoft Access, which have reached their limits and need a ‘professional’ rebuild. Usually that means bringing in a team of developers to get the job done. Changes are these applications will not be rebuild in LightSwitch. However, what will happen with the LightSwitch applications in a couple of years, when they have reached their limits? Exactly, developers will need to be extending those systems. Mind you, I do think this will greatly reduce the number of rebuilds of applications, as I expect Microsoft to come up with a good upgrade strategy for future versions of LightSwitch, making sure they are always running on the latest and greatest technology.

And finally I think we will be seeing more and more ‘secondary’ applications. What I mean by that is applications which are used to fill in some gaps in our current processes. For example, if you are building a LOB application and you have some tables in your databases, that your company needs to maintain for your customers, why would you not build the UI for that in LightSwitch? Or if you need to collect metadata for generating code, why not do that in LightSwitch? And there will be many other uses like this for LightSwitch where it can add to existing systems as well.

Conclusion

As with a lot of the new technologies that we’ve seen in the past decade, LightSwitch does provide us with another useful tool in our toolbox. It again takes away some of the repeating work we as developers do, so we can focus on what makes our software unique. This again will be a game changer, just not as a lot of people expect.

So what are your thoughts on LightSwitch? Let us know by leaving a comment.

Adventures while building a Silverlight Enterprise application part #35

noreply@blogger.com (Jonathan van de Veen) — Fri, 06 Aug 2010 10:41:00 +0000

In this post we’ll look into something I ran into while doing rework on our framework. It involves the order in which code gets executed when using events and such.

The situation

We’re currently in the process of getting our first deliverable out the door on time. One of the things that needed to be addressed before we can actually start testing this code properly, is to get a backlog of issues out of the way. I decided to do that myself as it gives me a better feel for what are common problems and what is the quality of work we as a team put out. Also I realized that issues may had arisen because of wrinkles in the framework architecture, where we’ve had to take shortcuts to get to our first deliverable and I wanted to be able to address these as needed.

Some architecture insight

For you to make sense of what I did to solve one of the problems I encountered, it is important to have some understanding of how our application works (at least at the client side). We host several ‘applications’ insight our Silverlight client. In the end we will have probably about six of these applications. Each application follows the same design. On the left there is navigation, consisting of a ‘menu’ and some UI to select records. Depending on the type of record the user selects and whatever authorization the user has we load modules to work with the data.

To create new records, we use a concept we call a task. This is basically a wizard like UI, which runs as a popup on top of the other UI. It consists of some of the same modules as used when the user navigates to a record, only in a different visual state. This makes for some interesting scenario’s as it is likely that there are two instances of the same module using and updating resources that only have a single instance.

One of these scenario’s is updating the navigation after creating a record through a task. As the task does not have any knowledge on what the module does, it is the modules responsibility to update navigation after creating a new record. To do that, it updates something we named ViewState, which in turn updates both the navigation and the modules that are needed to present this new record to the user.

However, creating the record is usually done in the first module of the task, before other modules can save their data (because they might need to create a relationship with the new record). This means that whenever the modules get the signal to load new data, that most of them can’t because the data is not there yet. It has everything to do with the order in which things are executed, especially because we use events to directly update our modules.

A design oversight

The fix is actually simple once you realize we had a design oversight in our ViewState. Why should you update navigation and load data into modules, when you have a popup presented to the user and the user can’t interact with the newly loaded data? It doesn’t make any sense. So the fix was to get the ViewState to hold any updates to the rest of the application if a popup is being presented and trigger the events once the popup is closed.

The first step to do that was to create a generic way to trigger the events in the ViewState. This is what I came up with:

   1: private void InvokeEvent<T>(MulticastDelegate target, T args) where T:EventArgs

   2: {

   3:     if (target != null)

   4:     {

   5:         if (PopupBridge.Bridge.IsPopupOpen)

   6:         {

   7:             EventHandlerQueueItem<EventArgs> queueItem = new EventHandlerQueueItem<EventArgs>();

   8:             queueItem.EventHandler = target;

   9:             queueItem.EventHandlerArguments = args;

  10:             _eventQueue.Enqueue(queueItem);

  11:         }

  12:         else

  13:         {

  14:             target.DynamicInvoke(new object[] { this, args });

  15:         }

  16:     }

  17: }

The InvokeEvent<T> method takes a MulticastDelegate (which represents the actual event instance) and an args argument based on T where T should always be an EventArgs or derive from it. To make this as easy to use as possible I first check to see if there is actually a valid target (which is null if no handlers are available). Next I check to see if there is any popup open. In our case there is a PopupBridge that has that information for us. Now if there is no poup open I simply call DynamicInvoke on the target and pass in a reference to the ViewState and the args argument.

If there is a popup open, though, I create a EventHandlerQueueItem. This is how that struct looks:

   1: internal struct EventHandlerQueueItem<T> where T:EventArgs

   2: {

   3:     public MulticastDelegate EventHandler;

   4:     public T EventHandlerArguments;

   5: }

As you can see it is a simple struct that has both a MulticastDelegate and can contain arguments to pass off to any handlers. Once we had that, we could now declare a Queue<T> to hold these instances. As we can’t write something like Queue<EventHandlerQueueItem<T>> I decided to use covariance and go with Queue<EventHandlerQueueItem<EventArgs>>.

Now all that remains is to trigger the events once the popup is closed. In our case the PopupBridge knows about this and triggers an event for it. The handler looks like this:

   1: void Bridge_PopupClosing(object sender, PopupClosingEventArgs e)

   2: {

   3:     if (!PopupBridge.Bridge.IsPopupOpen)

   4:     {

   5:         while (_eventQueue.Count > 0)

   6:         {

   7:             EventHandlerQueueItem<EventArgs> item = _eventQueue.Dequeue();

   8:             if (item.EventHandler != null)

   9:             {

  10:                 item.EventHandler.DynamicInvoke(new object[] { this, item.EventHandlerArguments });

  11:             }

  12:         }

  13:     }

  14: }

As there can be multiple popups open I still need to check if there is a popup open at that point. If they are all closed, we can just keep calling Dequeue and invoke the handlers until the queue is empty.

Now that we have solved this design oversight, modules that need to load data that was created in a task, will always be able to get that data from the server and everything works as expected.

I hope you find these pieces of code useful. If you have any questions or comments, please leave them below.

Adventures while building a Silverlight Enterprise application part #34

noreply@blogger.com (Jonathan van de Veen) — Mon, 26 Jul 2010 15:04:00 +0000

In this post we’ll look into analyzing performance in a Silverlight Line Of Business application, including multiple service layers and we come across a common problem using ADO.NET Entity Framework including a not so obvious solution.

The story

At the moment we are working on getting out a deliverable to a group of customers as soon as possible. However, one of the issues that keeps coming up is performance. A general response by a lot of developers is to just scour the web for performance tips on different technologies used in the stack and apply as many of them as they possibly can. However I find it much more useful to analyze the problem first.

Getting some data

To get a better feel for the problem, I first needed to get some numbers. When starting out with a performance issue the first question to get answered is “where is most time being spend?” This usually leads to the point where most time is being wasted as well.

At first I started out with profiling our application with Visual Studio 2010, but this wasn’t very useful to us as it didn’t show us any data on communicating with the service. I do feel compelled to link to some useful information on profiling your Silverlight application as it isn’t available through the IDE just yet and it requires you to jump through some hoops to get it to actually show you any performance data on your code. You can find a decent post by Oren Nachman on this here.

I should mention that profiling our application revealed one small performance issue with creating dynamic objects, which we use to instantiate modules in our application based on authorization and application settings. However, looking at the big picture it is hardly relevant to us, while hard to remedy it in a good way for our application.

The next step for me was to actually get data on communication with our services. As we designed our application in a way that we abstracted our service connections in seperate generic classes and we use generic service calls to get to our data, the easiest way for me was to simply record the DateTime.Now.Ticks. To keep the impact of this as low as possible on any production scenarios, I decided to write this information to the VS debug window, through Debug.WriteLine. Also to make sure this can be turned off easily to allow for other verbose debugging information, I introduced a conditional compilation symbol called DEBUGGINGPERFORMANCE and wrapped all my recording code with it. The code for me looked something like this:

   1: #if DEBUGGINGPERFORMANCE

   2:             Debug.WriteLine("BusinessLayerConnection.GetDataAsync<{0}>({1}): {2}", typeof(T).Name, id, DateTime.Now.Ticks);

   3: #endif

Running the application with this debugging code gave me some interesting data pointing me towards a specific operation, which gets all the data for a particular module at once. Within that set of modules one stood out that was taking far longer then the other modules, so I decided to investigate that further. I decided to use Fiddler to make sure none of this is actually caused by bandwidth issues. The statistics clearly showed my that the problem was with the service taking up a lot of time before actually starting to send a response.

This led me to put similar code in the webservice. Note that Debug.WriteLine doesn’t support the format approach by default and you should wrap that in string.Format. To actually make for a better performance test, I decided to build a small test tool to just make the same service calls several times in a row. This makes sure we’re working with correct numbers instead of chasing some random external factor.

As I started testing like this, I found out there where a lot of messages in the debug output stating that ‘A first change exception of type ‘ had occurred. In this case a large part of these exceptions where related to another service we use, which couldn’t access one of our databases. This was obviously an easy fix and it did improve performance considerably.

Fixing a common Entity Framework problem

After getting one exception out of the way, I found there where some more exceptions being thrown. The exact message that was displayed was: A first chance exception of type 'System.NotSupportedException' occurred in mscorlib.dll. Performance data indicated that this took roughly 3 seconds for six exceptions in each request!

I found it weird to see an exception like this. A web search didn’t return anything useful, except that it could be related to Entity Framework somehow. A forum thread pointed to this bug report about dynamic assemblies, which in itself wasn’t very useful information either.

I also found people asking questions about this message all over the internet, but without any real solutions. I turned on the System.NotSupportedException in Visual Studio, attached it to the service for debugging and ran my test again. As expected it halted in the debugger on the point where the exception was thrown. As expected it halted in mscorlib.dll, in this case in the System.Reflection.Emit.AssemblyBuilder class in the method GetManifestResourceNames(). This does confirm that it is actually caused by the bug I mentioned earlier, however it doesn’t really help us with a solution yet. The question now is, “why is someone calling this method on a dynamic assembly?”.

Browsing up the call stack makes it very clear that this code is being called from the Entity Framework. In fact it is called from the System.Data.Objects.ObjectContext constructor, but why? Taking a closer look at the call stack there is some indication that it must have something to do with initializing the metadata needed for the Entity Framework. It also reveals that the exception occurred in the System.Data.EntityClient.EntityConnection class in a method called SplitPaths. Looking at the code in that method and the state it was in as the exception occurred actually showed me what caused the problem. In the SplitPaths method there is a string[] called results (what’s in a name, right?) which contained three strings: res://*/Model.csdl, res://*/Model.ssdl and res://*/Model.mdl.

Looks familiar? That’s because these url’s are found in the default connection string for Entity Framework and they should point to the three files that make up the Entity Framework metadata. For some more documentation on that look here.

What was actually happening here was that a dynamic assembly was asked for it’s resources as the Entity Framework code was looking for a metadata file in all the loaded resources, as indicated by it’s connection string. The dynamic assembly that was loaded in this case was the binary for the WCF web service that hosts our DAL. The fix was simple. Just point to the exact assembly by it’s full name in the connection string for each metadata file and it’s fixed. This is how the metadata entry in our connection string looks now (obviously with some name changes):

   1: metadata=res://OurApp.Data, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null/OurModel.csdlres://OurApp.Data, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null/OurModel.ssdlres://OurApp.Data, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null/OurModel.msl;

I hope you found this search for this fix as useful as I did and it helps you out. Please leave me a comment if you have any questions.

30-07-2010 EDIT: Added the metadata part of the connection string to make the solution more clear. Thanks to greg-joyal for pointing that out to me.

We’re getting it done

noreply@blogger.com (Jonathan van de Veen) — Mon, 19 Jul 2010 06:33:00 +0000

You might wonder what I mean by that. I’m referring to us moving into our new house. After about a month of hard labor, we’re finally seeing the results. We’re down to the details indoors and outdoors will be progressing in the coming week. I now have a decent place to work with a large desk and my PC right there.

That means it’s time to start going back to normal routine and blog a little. Last week I saw this keynote recording where Scott Hanselman talks about why all developers need a blog and how to make a blog suck less. Although I can come up with reasons why not all developers should have a blog, I do agree with him on the fact that you should engage in some form of social networking over the internet in order to be successful as a developer these days. And the pointers he gives on improving your blog inspired me to finally handle the spam issue I’ve been having here at Developers 42.

I already announced this earlier, but today I have finally turned on comment moderation. This will surely prevent any spam from reaching you as a reader. Unfortunately this also means having a delay between you posting an awesome comment and someone else being able to read that comment. I guess that’s just the way things are, the good having to suffer because of the evil spammers.

Still I hope this doesn’t prevent you from leaving your comments here so we can all interact and learn from one another.