Random Acts of Coding

Iowa Code Camp: Session Overview

James.R.Eggers@gmail.com (J. Eggers) — Tue, 10 Nov 2009 14:00:00 +0000

The Iowa Code Camp was held on November 7th, 2009 in Des Moines, IA. With a capacity crowd and a lot of great sessions, it ended up being an awesome time with a lot of great presentations and conversations. Like conference of this type, you never have time to hit all of the sessions you may want to; however, the ones I was able to attend were still awesome. Below is an overview on each one.

Getting Started With Behavior Driven Development by Lee Brandt

In his presentation, Lee provided a good introduction to the concepts of BDD. The presentation focused on the origin of BDD, where it fits, and how it's an evolution of TDD from the beginning which helped to truly set the stage. For those that have done TDD for a bit, it provided a great introduction into bridging the gap between the business case and the code by using tests as a spec. Through this and Lee's stressing of an Ubiquitous Language as described in various DDD circles, he showed how BDD can be used in order to provide a technique that helps to show the client what the application will do once done as well as a pseudo-burndown chart of what hasn't been finished yet. I'm definitely looking forward to exploring BDD again in the near future.

Twitter: @LeeBrandt
Blog: http://www.geekswithblogs.net/leesblog/Default.aspx

Going From 0 to 100 Dollars an Hour with .Net you Didn't Know by Mitchel Sellers

This session was a good session; however, it wasn't as advanced as I was hoping for. The abstract of the session focused around using advanced features of .net in order to make you more productive; however, I was hoping for some additional tidbits beyond that of the abstract. The primary items focused on this presentation were new tips and tricks introduced by .Net 3.5 such as Lambdas, Self Containing Properties, and LINQ. All of these items I already knew and have used them in the past which is why I was hoping for a few tidbits beyond such. That being said though it WAS a good session with a lot of good code samples to convey each concept and technology. I still learned at least one new thing and was reminded about a few other things as well. While it didn't meet my (probably unrealistic) hopes, I cannot deny that Mitchel's presentation definitely educated a number of people who were also in his session that was overfilled.

Twitter: @MitchelSellers
Blog: http://www.MitchelSellers.com

Open Spaces

I've been to a few open spaces discussion and have came to the conclusion that some will be good, some will be bad, and others will be between or outside of those labels. The open spaces session at the Iowa Code Camp was an in between session. The size of the group was smaller and comprised of a diverse set of primary skills which isn't always a bad thing though the topics in discussion didn't flow like I have experienced in other places. We had a good conversation about how to get the information to the people that don't come to conferences or how to encourage them to come. We also talked a little bit about MEF, coding war stories, and some of the side effects of completely rewriting/redesigning an externally facing website. Lastly we talked a little bit about Kanban vs Scrum and the paradox of software estimation. All in all there was some good things that came up during the conversation; however, part of me was hoping that more people would have been involved in the discussion and the topics would have been more discussion/problem solving based to an extent.

Intro to ASP.Net MVC by Chris Sutton

I've dabbled in ASP.Net MVC and have wanted to learn more about it; however, ~~finding~~ making the time for such is not always easy. Everything that I've learned about such has either been about the MVC pattern itself, or about other frameworks in general like Rails or Django. With only a few MVC Videos watched and about 3hrs worth of coding so far, I decided checking out another intro session wouldn't be a bad idea for me. Chris Sutton did a great job with the presentation. He explained the MVC pattern very well as well as describing how the ASP.Net MVC framework relates to it and ASP.Net in general. He showed a couple of the typical demos for this level of session and afterwards answered all of the questions I had about the next step. All in all it was a very good session and can't wait until I can make time for using such.

Twitter: @ChrisSutton
Blog: http://subjunctive.wordpress.com

Silverlight for WPF Developers by Kirstin Juhl

In my opinion, none of the last block of sessions really looked appealing to me. It wasn't the fact that I didn't think any were going to be good, it was just that I wasn't interested in the topics at hand. I ultimately decided to check out Kirstin's presentation on what WPF Developers would need to know to transition into the Silverlight space. While I have only dabbled in WPF a little bit, I knew a number of differences and knew the power of XAML was greater in WPF than Silverlight. The decision to go here was my current adventures in Silverlight and see what I may be able to learn. All in all, the session was pretty good for the purpose Kirstin was targeting. She did a good job at explaining the differences as well as provide information on how to share components between Silverlight and WPF applications (something I knew was possible but hadn't seen an example of up to this point). Good presentation overall.

Twitter: @KirstinJ
Blog: http://www.geekswithblogs.net/Kirstinj/Default.aspx

Overall:

Overall, I had a great time at the Iowa Code Camp. For being a free conference, the presentations, facilities, and of course the prizes were all top notch. It was a great conference with some amazingly talented speakers and attendees. All in all I can't wait to head to the next one in the April/May/June timeframe. Hopefully some of the presentations I'm working on will be ready by then since I'd love to present one.

Reviewing This Year's Goals (2009)

James.R.Eggers@gmail.com (J. Eggers) — Fri, 06 Nov 2009 16:25:00 +0000

Back in January, I created this post focusing around what I was hoping to accomplish this year. I was going through my old posts and stumbled across it. I figured it would be a good time to look over them once again to gauge my progress along with what else I may have done so far this year.

Learn F#:

While I do not claim to be an authority of F# by any means, I do feel comfortable in saying that I learned the bulk of the language and be able to determine where it makes sense to use it. If time allows, I can see myself using it more and more; however, for right now, I've definitely learned a lot from such and can focus on learning other things.

Learn DDD:

This one I am really only about 1/2 done with. While I have went through the information and can understand the concepts involved, I haven't had a good way of applying such yet. There's existing applications that could benefit from the practices of DDD; but I haven't pushed myself into applying such on a new project yet.

Learn Silverlight 2:

I got sidetracked and ended up procrastinated on this one. While I wanted to learn SL2, I still was a bit apathetic towards it since SL1 wasn't very impressive to me. Thankfully, I've seen a lot of what Silverlight 3 can do and am going all out on learning more about it. Except some posts on Silverlight 3 soon.

Other Things:

So what else have I been up to so far this year? Looking back I have worked on some open source projects (namely my SchemaSpy Task for NAnt and contributing to .Net Migrations over on codeplex). I've also focused a lot of time on keeping up to date on .Net 3.5 and other technologies that I wasn't able to fully take advantage of at a previous employer. Lastly, I've really been focusing on TDD and just writing better code in general. There's always room for improvements but being able to learn from what's being talked about over the past year is what's important.

Things on the Horizon...

Learning MEF:

The Microsoft Extensibility Framework (MEF) provides some great opportunities to make very extensible applications. With a need to make a project extensible, I'm definitely going to be focusing on understanding MEF more.

Learning Silverlight 3:

I'm already in the process of working in Silverlight 3 but I have only started to write applications with such. Just like a person who has finished their first true ASP.Net application, there's always room for improvements and things that one can learn to make things better. I'm at that stage where there's a lot more I can learn about and I'm all for it since Silverlight 3 is awesome.

More SchemaSpy:

My SchemaSpy post earlier this year is one of my highest trafficked posts as well as one that I refer back to all of the time. I'm planning on answering a few outstanding questions associated with that post and also see what else I can do with the tool to make things easier for everyone.

<Unknown> Things:

I always leave myself open to the unknown and unforeseen opportunities. While November and December are usually swamped for me, I never know when inspiration will hit me and I learn something new.

Reviewing UppercuT - A Build Framework Based On NAnt

James.R.Eggers@gmail.com (J. Eggers) — Fri, 30 Oct 2009 12:55:00 +0000

Earlier this week I went to the Kansas City .Net User Group meeting where the topic of discussion was UppercuT, a build management framework created by Rob Reynolds. The discussion was pretty good and Rob did a good job at showing a number of demonstrations on how to use the various aspects of UppercuT. All in all, I feel like the presentation got the point across as to the typical why/when/how/etc. questions.

What Is UppercuT?

UppercuT is a build management framework based on NAnt. It was created in order to establish a consistent way to build .Net applications with a minimum amount of configuration while providing areas for extensibility. In addition to just compiling, UppercuT provides a number of options to manage application versioning, automated testing, and code packaging. While it can be extended to include such steps through NAnt, UppercuT doesn't deploy code...it only stages it since every company/project's deployment strategies may be different. One nice thing that UppercuT does do in terms of deployments though is provide you instructions on how to wire it up to CruiseControl.net.

When Should UppercuT Be Used?

UppercuT provides a way for people who have not started to take advantage of automated builds to do so quickly. Because of this, it's best suited for projects that don't have automated build systems integrated into them yet. With some configuration changes, it is possible to integrate with your current scripts; however, changes would need to be made on both side to the point where it would honestly be easier to convert your current scripts to custom steps for UppercuT to take advantage on. If you currently use some build framework like MSBuild or NAnt, I would probably recommend at least checking out UppercuT to see if it's a good fit; however, if you or your company already have some system that works for you in place, it may be better to just keep with what you have.

Where Can I Download and Learn More?

You can find more information on Uppercut by going to it's project website at http://ProjectUppercuT.org. In addition, you can check out Rob's blog over at http://ferventcoder.com/ as well as on Twitter by following @ferventcoder and @ProjectUppercuT.

Is Learning though a UI worth it?

James.R.Eggers@gmail.com (J. Eggers) — Mon, 26 Oct 2009 03:13:00 +0000

Do you remember the last time you learned a new, back-end technology like Linq-to-Sql, Entity Framework, NHibernate, the Twitter API, or some other 3rd party API? How did you first go about learning it? If you used some tutorial of some sort (be it a blog, video, or some other medium), how did you try it out? Did they have create a basic web page to present the data from the API that you were consuming? If you DID create any form or UI to consume this API, here's two questions for you...

How long did it take you to create that UI compared to writing the code that uses the API?
Did you learn more about the API from the UI or from writing the code that actually interacted with the API?

Now, let me restate that I'm aiming these questions specifically at back-end technologies. Any technology that deals with any aspect of a UI (i.e. JavaScript, Silverlight, etc.) obviously depends on the UI and really does not apply. However, if you're NOT focused on a UI-dependent technology, why do you create a UI to learn something that doesn't require it?

Don't get me wrong. I was guilty of this as well and what I found is that I got sick and tired and bored spending so much time creating a UI and it would take me longer to get to the subject of my studies. Throw in a dash of OCD or ADHD because I couldn't just make a simple UI but needed some form of basic layout and color contrasts at the very least and it was a recipe of boredom. It wasn't really until something clicked that I began to really realize that I learned more from the tests or the debugger while stepping through the code consuming these APIs than how the information was presented. Sure it felt good when I could apply what I learned to a UI but applying the UI layer did not directly provide any value for me towards learning the API.

But again, if it takes so much time and provides a greater opportunity for distractions, is it worth it to learn a back end process by means of applying such to a UI?

My Barriers to Learning TDD

James.R.Eggers@gmail.com (J. Eggers) — Wed, 02 Sep 2009 13:42:00 +0000

As a follow up to my previous post declaring that I finally "get" TDD to some extent, I figured I'd reflect on the barriers that I have had up to this point which made it difficult for me to learn how to unit test in some effective manner as well as truly understanding the benefits of TDD practices. Now, I am by no means claiming to be an expert in unit testing, mocking, or TDD. I didn't open the refrigerator, drink some of the TDD-flavored Kool-Aid, and threw on a subsequent Mortarboard to illustrate that I've somehow graduated into this new, higher level of software development. I'm still learning from others as well as my own ~~errors~~ experiences just like everyone else. The purpose of this is to reveal to others some of the issues that I had and hopefully provide some insight on how to overcome such.

The Examples Are Too Simple

This barrier was by far the most difficult for me to overcome. I would read up on all of the buzz about how to add unit tests to your code and how to apply unit testing into TDD. I would follow the examples and understand it and be fine; however, I would always stall out whenever I tried to do it in a real world scenario. One day, I got extremely frustrated trying to fit all of this this together even if it looked like a square peg for a round hole to me at that time. I began to wonder how unit testing in general would work on such complex real world objects and that's when it hit me. Around this same time, I was diving into the S.O.L.I.D. principles of Object Oriented Design and began to count the dependencies I had in the code. The more I broke out the code into less complex objects that followed the principles, the easier it was for me to map the unit test examples that previously were "too simple" to my code. The barrier wasn't so much that the examples were too simple as it was that my code that I was trying to directly apply the lesson to were too complex. On the last project I had, I didn't apply unit testing to it; however, I tried to adhere to the S.O.L.I.D. principles as much as possible. When I was trying to retrofit the tests into it, I was able to do it as well as find places where I didn't follow the S.O.L.I.D. principles as much as I thought I did. Looking back, I probably wouldn't have had any of the refactoring issues I had after the fact if I had used TDD to provide a test-first basis for the object designs.

But...This Class Uses the Database

This was another barrier for me that partially stems from the previous example. Looking over the majority of Unit Testing and TDD books and blog posts out there, very few of them actually touch dependencies as common as a database. Looking at my own code at the time, I had repository classes that handled all of the CRUD operations to the database as well as spit out the records as strongly typed collections (where necessary). How do I test this class when there is such a deep dependency with the database? I was truly perplexed by this issue and decided to ask the question on StackOverflow. The responses led me to realize that my class was doing 2 things; calling the database AND transforming the returned data in to a manner I needed. By pushing the direct database calls to a separate class, I was able to mock away the database and make sure these particular class could be tested.

Now, that only solved half of the issue. I abstracted the dependency of those classes but I still had data provider classes that still interacted with the database. This is where I truly learned the difference between Unit Testing and Integration Testing. With such an outside dependency as a database or a web service, I began to see some examples that created separate, controlled instances of those dependencies in order to test the queries and CRUD functions. The technique I use now for database integration is to use NHibernate, even if the primary project doesn't use it. NHibernate has the ability to recreate table schemas for each tests which helps automate the state of the database for each test. An example of this can be shown over at NHForge.org's How-To section under Your First NHibernate Based Application. After setting this up, it provides me an easy way to test the barrier of the database.

Am I Just Testing the Mocking Framework?

I had a class the called into a 3rd party library that I wanted to test. I abstracted the 3rd party library into a proxy and called such from my class. The 3rd party library sent messages over a TCP socket connection and returned back an "OK" or an "ERR" message. The class that I wanted to test created and sent the messages and then reported on what it got back. For example, I would call a method in the class and it would send a message "FOO" to the proxy and get back "OK". That's a little simplified (see barrier #1 above) but the class itself did pretty much that. When I setup the test, I mocked out the proxy dependency and had it return "OK" whenever my class sent it "FOO". This is not that meaningful of a test since there is no logic in that 1 method. Other methods had logic based on the parameters; however, this one method was just this simple. Was I just testing the mocking framework with this particular test? Arguably, yes...I was; however, in the future, if that ever had to change, I now have a test that I can use to ensure that I get the information correctly.

But Unit Testing = More Code & Time

This wasn't so much a barrier for me mentally but was a barrier to adopt it at work. From what I've experienced now, writing unit tests AFTER the code has already been written DOES lead to more code being written, more refactoring of existing code to make it testable, and more time to do all of this. However, I have seen that writing your unit tests BEFORE you write any code (as advocated in TDD), you tend to code at the same speed if not a little faster. I found that I wrote code faster when I did tests before since the production code was already designed for me through the tests I had just written. There was less thought at the time of coding since the design was setup while I was writing the tests. I also found myself near-instantly defaulting into the S.O.L.I.D. principles instead of looking at them sometimes as a refactoring step. Lastly, I wrote less code that would have been for future use (i.e. YAGNI).

All of this came with a paradigm shift in my object design style. While some of the basic directory structure and such was the same, I looked at my classes more at a "how do I want to call this class" as opposed to "what methods do I want to show in this class". The shift in my thinking was more of a consumer than a provider or retailer. If you only code what you need, you tend to not code things you don't need. It's as if you go to the grocery store for 1 item and that's the only item that the store has. Nothing else to distract you in that point in time. No other products around that you may need in the future. You are in that 1 moment in time and in that moment, you only have 1 need as a consumer.

Summary

While there were other barriers I came across, the rest were either very minor or more about basic implementation like how to return an IDataReader object. I'm sure that I'll stumble across more barriers and moments of enlightenment the more I practice TDD or just unit testing in general. Until then, I hope some of the items described and discussed here can help others who have struggled to get their head around unit testing in general and some of the principles of TDD.

Finally Understanding the Merits of TDD

James.R.Eggers@gmail.com (J. Eggers) — Wed, 26 Aug 2009 15:32:00 +0000

I'm not a TDD person...or at least I wasn't until last week. Up until then, I had read the blogs and looked at the examples to try to understand TDD and unit testing (with mocking) in general. Almost all of the examples I was shown demonstrated very basic scenarios that, in most cases, were too trivial to show value. I would ask people who would speak about unit testing in general how you'd do a specific scenario and would get mix responses ranging from "just try it" to "you should be using this tool and it'll just write the tests for you".

Last week, I had some free time to where I could truly evaluate a code base I finished up with the prior week and see if I could apply some tests to it. The code I wanted to add tests to were simple repositories where I had abstracted the database calls to separate classes. Being told to mock the databases out in the past, I started to do that and focus solely on these repositories. In some instances, it felt like I was testing the abilities of the mocking framework instead of my code but these methods were simple by design (i.e. "Get Customer by Id 'X'"). It was when I started to test more of the methods that I saw that some of the code I wrote had some issues.

The code was functional; however, it wasn't easily testable. When I would attempt to describe the test, I found that the code did more than one things (breaking the Single Responsibility Principle). Another incident showed that the name of 1 whole class didn't make sense. A third incident showed me an issue with the number of complexities that come from multiple dependencies and object inheritance. After each time that I finished refactoring these issues out of the code and eventually got to working tests, I began to see how the questions that TDD (and BDD) cause you to ask while designing an application really shine.

One example of this was a duplication of code. I had a class called, for the sake of this post, UserRepository. The UserRepository had a method to get all users in the system and returned a List<User> back to the calling code. This class also had a second method that did the same thing but the List<User> only had the FirstName and LastName properties populated. Why did I write such months ago? Short answer - I have no idea. Looking through the code, I found that a different feature needed just the FirstName and LastName properties and nothing else...it was then sending the List<User> to the client for an AJAX call. While I can understand this now, I could have better implemented the solution by if I looked at the scenario and behavior better initially.

If you ever find yourself wondering how writing more code for TDD or just basic unit tests can actually help the code, my recommendation is to just try it. Take a simple class that has only a few pieces of logic in it and test it. Next, take a simple class that has a dependency that you'll need to mock out and test that too. After you understand how to write these basic types of tests, you can move into the self-reflective questions that really shine in TDD. Questions like "What type of data do I want to work with when I call method X?", "What will this class need (a.k.a. dependencies) to get me data Y?", and "Is this functionality already present?" are all questions that get forgotten while we're sometimes blindly coding.

Looking ahead, I can only imagine how my code will be looking.

ParseWiki.js - A JavaScript Wiki Parsing Engine

James.R.Eggers@gmail.com (J. Eggers) — Mon, 24 Aug 2009 00:21:00 +0000

Like most developers, I have a list of applications or pet project that address various scenarios I would like to develop or find solutions for. While I have to admit my pet project list seems to grow more than shrink, I do knock things out on occasion. One of these items was a wiki parsing engine. Now there are plenty of wiki-based solutions out there; however, I wasn't able to find one that allowed me to bring just the parsing engine into my own solution. Most of the solutions that are available are full solutions that include the parsing engine, data store, and full UI. Sure they can be tweaked a bit but having a simple tool that will allow me to mark up some text in a <textarea> field and then preview it is usually too simple for these solutions. WikiPlex is a great solution to fill this gap on the server side and recommend everyone to check it out; however, it got me wondering if a JavaScript solution could be done for client side parsing. This is when I started to work on ParseWiki.js.

An Overview of the Wiki Parsing Engine:

After looking at a lot of wiki solutions and engines, almost all of them use Regular Expressions to parse the raw text. I decided to do a similar approach since JavaScript has a pretty good RegEx system. Since the focus is to just build the engine and not a full solution like Luminotes or Tiddlywiki, I needed to look at the bare minimum items to parse in my engine. Below is a list of what version 1 of ParseWiki.js has:

Headings 1-6
bold/italic/underline
horizontal rule
Named Anchors and Internal Links
External Links
Ordered and Unordered Lists (1 level only right now)

The Syntax:

I've used a lot of wiki platforms from Wikia to CodePlex to Luminotes and SharePoint. Of the differences in wiki markup each provide, I found it that CodePlex's engine provided the easiest/fastest to use for me. Because of this, I based a lot of my syntax on such as well.

Headings:
! Heading1 text
!! Heading2 test
!!! Heading3 text
ect.
Bold:
*Bolded Text*
Italic:
_Italicized Text_
Underline:
+Underlined Text+
Horizontal Rule:
---- (4 hyphens)
Named Anchors:
[a:Anchor Name]
Internal Link:
[goto:Anchor Name|Link Text]
External Link:
[url:External Url|Link Text]
Unordered Lists:
* Item 1
* Item 2
* Item 3
* etc.
Ordered Lists:
# Item 1
# Item 2
# Item 3
# etc.

Using ParseWiki.js:

Since I wanted a simple wiki engine, using ParseWiki is very simple.

Download ParseWiki.js
Add it to your Html page
Have your own JavaScript Code pass the marked up text into the ParseWiki() Method
Do with the outputted text as wish.

In the example files provided with the ParseWiki.js download, a full implementation has been provided where I pass the contents of a <textarea> field into the ParseWiki() method and then output it into a <div> to preview.

Download the Code with Example:

Download ParseWiki.js and Examples: Link

Next Steps:

What's next on this pet project? Well, there are a few things that I need to work on in order to make it better and provide more features. There's a number of syntax features that I still need to add to the engine (obvious ones are multi-level lists and escape syntax). Until that time, feel free to play around and with it and let me know what you think.

SchemaSpy Task for NAnt v1.0 Released

James.R.Eggers@gmail.com (J. Eggers) — Tue, 04 Aug 2009 14:47:00 +0000

Over the past couple of months, I've been working on creating a custom task for NAnt to provide a simplified way of using SchemaSpy, the open source Java application for database documentation and analysis. While the code as been feature complete for about a month now, I have finally finished the documentation and cleaning up the code itself. Please feel free to hop out to CodePlex and give it a try.

SchemaSpy Task for NAnt

Writing a Custom NAnt Task (Part 3)

James.R.Eggers@gmail.com (J. Eggers) — Thu, 30 Jul 2009 00:00:00 +0000

Over the past couple of months, I have been working on writing the SchemaSpy Task for NAnt located on CodePlex. When I began to create a few custom tasks for NAnt in the past, I found very little documentation about how to create one. Much of the documentation surrounding the process how to create a custom task is based solely on examples without any supporting information on how to expand such. While this works very well for basic things, there were some experiences I had to work through while writing the task for SchemaSpy that I would like to talk about here. In this, the third of four parts, we'll be diverting away from the example created by part 1 and 2 and dive straight into examples from the SchemaSpy Task for NAnt code in order to examine how to create a custom element collection for your task. In this post, we'll be reviewing the Schemas property of the SchemaSpy Task for NAnt.

Why Do I Need a Custom Element Collection?

For a very large number of tasks, basic attribute-based task properties should be enough. However, if you find yourself where you need to allow a list of inputs into your task, you are stuck with two different options. The first option is to simply create a basic property just like those described in the first two parts in this series in order to set a delimited string from the build file. While this is easy to code and implement, it is not the most user friendly after the list grows in size. The second option is to create a child element that represents a collection of elements. This is a little bit more complex to develop; however, it provides a much better experience for the people who have to maintain the build file. Unlike a simple decorated class property, a custom element collection requires things to be implemented in the code.

Creating Collection Items

The first item that we need to create is a simple object that will represent the children nodes of your custom element collections. Looking at the code, this is represented by the Schema class in the Schema.cs file. This class has to inherit from the NAnt.Core.Element class and be decorated with the ElementName attribute, the attribute is used to identify the name of the XML node in the build file. In the example (see the code below), the name of the element will be "schema". After the class is established, we are able to add decorated properties that will represent the different attributes for the element.

   1:  [ElementName("schema")]

   2:  public class Schema : Element

   3:  {

   4:      /// <summary>

   5:      /// The name of the schema to analyze and document.

   6:      /// </summary>

   7:      [TaskAttribute("schemaName")]

   8:      [StringValidator(AllowEmpty = false)]

   9:      public string SchemaName { get; set; }

  10:  }

Creating the Strongly-Typed Collection

The second item we need to add to our task is a strongly-typed collection to store instances of our Schema object we just created. Looking at the code, this is represented by the SchemaCollection class in the SchemaCollection.cs file. Like many strongly-type collections created in .Net, this class inherits from the System.Collections.CollectionBase class. Simply by implementing the abstract base class and filling in the code, the custom, strongly-typed collection of Schema objects will be created and ready to use.

Implementing the Collection

The third and fourth tasks left to add our collection of custom elements to our NAnt task is to update the task code itself. Now that we have the collection and child elements created, we have to first create a private variable to hold a new instance of our collection object. If the collection is not created and instantiated, NAnt will not be able to add items to the collection and throw an error.

The last thing to do is to create a new decorated property to tell NAnt to access and expect the collection. Unlike the other properties that we declared that describe node elements, we need to decorate a property with the BuildElementCollection attribute. The attribute requires three parameters. The first describes the name of the collection node. The second describes the name of the children nodes. The last parameter describes the task options. Once all of these items are set, the new code should look like the below:

   1:  /// <summary>

   2:  /// Instantiates a default collection to add elements to.

   3:  /// </summary>

   4:  private SchemaCollection _schemaCollection = new SchemaCollection();

5:

   6:  /// <summary>

   7:  /// Gets or sets the collection of schemas to use.

   8:  /// </summary>

   9:  [BuildElementCollection("schemas", "schema", Required = false)]

  10:  public SchemaCollection Schemas

  11:  {

  12:       get { return _schemaCollection; }

  13:       set { _schemaCollection = value; }

  14:  }

A Look at the Build File

With everything completed, the code can be compiled and installed for NAnt to begin using it. Once all of that is setup, you can access and update the build file to take advantage of the customer collection similar to the example below:

   1:  <example prop1="test">

   2:      <schemas>

   3:          <schema schemaName="dbo" />

   4:          <schema schemaName="sys" />

   5:      </schemas>

   6:  </example>

Summary

In this section we reviewed the code implemented by the SchemaSpy Task for NAnt I wrote and placed out on CodePlex. We looked at what it takes to implement a custom collection. In the next and final installment of this series, we'll be looking at writing a custom task that is used to call an external application and look at what differences there are between it and the task we have been working with in the first two segments of this series.

Writing a Custom NAnt Task (Part 4)

James.R.Eggers@gmail.com (J. Eggers) — Thu, 23 Jul 2009 19:04:00 +0000

In this final installment of a series of posts looking into how to create a custom NAnt task, we'll dive into how to create a task that executes an external application. While NAnt has the built in <exec> task to handling command line programs, there may come a time where the possible arguments or the command itself is just too much for such a generic task. This was the case that I ran into while trying to integrate SchemaSpy, the Java-based database analysis and documentation tool, into my build scripts. With the number of arguments and complexities of the program, I decided to create my own custom task. We'll be diving into this task while exploring the differences between a basic task and an executable task. The source code for this project will refer to the SchemaSpy Task for NAnt located out on CodePlex.

A Look at the Default <exec> Task.

NAnt comes bundled with a task that can be used to call external programs as a way to extend the functionality of the build script. For some applications that take only a few arguments, this is really easy to manage. This can be seen using a command line program for the DotNetMigrations project also located out on CodePlex

   1:  <exec program="db">

   2:      <arg value="migrate dev 2" />

   3:  </exec>

However, if you are using an item like SchemaSpy, the cleanliness of the solution begins to deteriorate fairly quickly with only a partial set of its argument.

   1:  <exec program="java">

   2:      <arg value="-jar SchemaSpy.jar -t mssql05 -host serverName -db myDb -port 1433 -sso -all -o ..\output" />

   3:  </exec>

When there is a need to integrate a program like SchemaSpy into a tool like NAnt, creating a custom task to streamline the arguments into a more user-friendly manner can greatly ease script authoring and debugging.

Task vs. ExternalProgramBase

In previous examples in this series, we would create our custom tasks using the NAnt.Core.Task abstract base class. This base class gave us everything we needed to do to handle properties, children elements, and similar items. One item that the Task base class did not handle though was running external processes though. If you were to attempt to start a new process, an error would occur or the script would hang since the Task base class isn't thread-safe by default. If you wanted to create the multi-threading code in order to make the process run successfully and safely, then you can definitely do such while inheriting from the Task base class; however, NAnt has already done this for you by providing the ExternalProgramBase abstract base class. The ExternalProgramBase class inherits from Task and enhances it by providing specific methods and properties for working with external programs. In addition, it has all of the threading logic established for you.

Diving Deeper into ExternalProgramBase

Since ExternalProgramBase inherits from Task, every thing we can do when we inherited from Task previously can still be done when we inherit from ExternalProgramBase. Where the differences come into play is when we overwrite the ExecuteTask() method. This method is called by the NAnt engine to start the task's specific function. In the ExternalProgramBase base class implementation, this method executes the actual external application we want to start. It does so by looking at the values of 2 properties; ExeName and ProgramArguments. The ExeName property provides the string name (with extension) of the executable program to run. The ProgramArguments property provides a string representing the fully list of arguments. We can still provide our own custom code inside of the ExecuteTask() method, and once we're ready for the external program to start, we can simply set the ExeName property and call base.ExecuteTask(). The base class's ExecuteTask() method will read both properties, create the necessary threads, and execute the program. Below is a snippet from the SchemaSpy task that illustrates this process.

   1:  /// <summary>

   2:  /// The Program Arguments listing used by the ExternalProgramBase class.

   3:  /// </summary>

   4:  public override string ProgramArguments

   5:  {

   6:      get { return BuildArgumentList(); }

   7:  }

8:

   9:  /// <summary>

  10:  /// Executes the task's operation for NAnt.

  11:  /// </summary>

  12:  protected override void ExecuteTask()

  13:  {

  14:      bool isValid = ValidateAttributes();

15:

  16:      if (!isValid)

  17:      {

  18:          Log(Level.Error, "Task Attributes are not valid.");

  19:          return;

  20:      }

21:

  22:      this.ExeName = "java.exe";

  23:      base.ExecuteTask();

  24:  }

Notice that in the SchemaSpy task for NAnt, I did not directly set the ProgramArguments property prior to calling base.ExecuteTask(). Instead, I had the property execute a method that would do it when the base class is ready. This method evaluates the various properties/attributes of the task in order to create all of the arguments required by SchemaSpy to run properly.

The Updated Build Script

With the custom task created, we can now update the build script to use it instead of relying on the <exec> task. Below is an example of the SchemaSpy call previously shown.

   1:  <schemaSpy

   2:      jarPath="..\SchemaSpy.jar"

   3:      dbType="mssql-jtds"

   4:      host="MyDatabaseServer"

   5:      port="1433"

   6:      dbName="MyDatabase"

   7:      schemaName="dbo"

   8:      outputDirectory="..\MyDatabaseDocumentation"

   9:      singleSignOn="true" />

Summary

Throughout this series of blog posts, we've covered how to make very simple custom tasks for NAnt up through the ability to create complex tasks that contain child elements and/or simplifies running external programs. Through the steps listed in this series, hopefully all of information you need to create a custom NAnt task will be available to you. If a scenario comes up that you need assistance with, feel free to post a comment and I'll see what I can do to assist.

Writing a Custom NAnt Task (Part 2)

James.R.Eggers@gmail.com (J. Eggers) — Wed, 01 Jul 2009 02:22:00 +0000

Over the past couple of months, I have been working on writing the SchemaSpy Task for NAnt located on CodePlex. When I began to create a few custom tasks for NAnt in the past, I found very little documentation about how to create one. Much of the documentation surrounding how to create a custom task is based solely on examples without any supporting information. While this works very well for basic things, there were some experiences I had to work through while writing the task for SchemaSpy that I would like to talk about here. In this, the second of four parts, we'll be extending what we created on Part 1 by adding a FileSet collection to add some flexibility to our application. In addition, the accompanying source code will remain in both C# and VB.Net similar to the first section in this series.

Changing the Input Directory Attribute:

In Part 1 of this series, our Log Combining task contained an attribute to identify the directory that held all files that were to be combined. What we want to do first is remove this attribute and provide an alternative solution using the NAnt FileSet object. The reason to change this is to allow us to not be tied to a single directory for combining files, as well as provide a ways to be more selective of the files to combine. For more general information on the FileSet Type of NAnt, make sure to check out the official documentation for FileSet over at the NAnt website.

In order to use a FileSet object, we'll need to import the NAnt.Core.Types namespace. In addition, we have to instantiate a private FileSet object in order to be exposed by the new property. The reason for this is that the NAnt engine does not look to ensure that the collection-based objects are instantiated. On collection-based elements that contains children elements, the NAnt engine ultimately just adds items into the collection. If the collection is not instantiated, a null-reference error will be thrown.

After the property has been setup and the private object has been instantiated, the code for such should look similar to the following:

   1:  private FileSet _logFileSet = new FileSet();

2:

   3:  /// <summary>

   4:  /// The files that will be combined by this task.

   5:  /// </summary>

   6:  [BuildElement("fileset")]

   7:  [StringValidator(AllowEmpty = false)]

   8:  public FileSet LogFileSet

   9:  {

  10:      get { return _logFileSet; }

  11:      set { _logFileSet = value; }

  12:  }

Once this is done, we can update the project's build file to look something like the below to mimic the same functionality as we had with the previous attribute.

   1:  <logCombiner outputFile="combined.txt" >

   2:      <fileset basedir="..\logs\">

   3:          <include name="**/*" />

   4:      </fileset>

   5:  </logCombiner>

Updating the Combining Method to use the FileSet:

Now that we have the task attribute and build file setup, we need to update the code to use the collection. In order to keep things simple, I'm not going to touch the current CombineFiles() method. Instead, I want to ensure the information I will be getting from the FileSet object will be able to be valid and fit into that method call.

To do this, we need to change the name of the ValidateInputDirectory() method to ValidateInputFile(). This will ensure the name of the method will not confuse other developers. In addition, we want to change the logic of this method slightly by changing Directory.Exists() to File.Exists(). We will be working with files at this point instead of directories so testing we want to make sure we test for the proper item.

   1:  /// <summary>

   2:  /// Validates the Input Directory Value

   3:  /// </summary>

   4:  /// <param name="path">The directory location to validate</param>

   5:  private void ValidateInputFile(string path)

   6:  {

   7:      if (!File.Exists(path))

   8:      {

   9:          throw new BuildException("The input directory of " + path + " does not exist.");

  10:      }

  11:  }

Next, we need to update the GetInputFiles() method. This method returns an array of strings representing the file paths. These paths are then passed to the CombineFiles() method. We want to keep this signature. Inside of the method, we need to instantiate an item to hold our string array; like a List<string>. Next, we need to loop through the FileSet.FileNames collection and add each entry into our string list. Lastly, we want to make sure the files are validated through our ValidateInputFile() method before returning the string array. Once all of this is done, the method should look like the following:

   1:  /// <summary>

   2:  /// Retrieves a listing of files from a provided directory

   3:  /// </summary>

   4:  /// <returns>A string array of all file paths.</returns>

   5:  private string[] GetInputFiles()

   6:  {

   7:      List<string> output = new List<string>();

8:

   9:      foreach (string path in _logFileSet.FileNames)

  10:      {

  11:          ValidateInputFile(path);

  12:          output.Add(path);

  13:      }

14:

  15:      return output.ToArray();

  16:  }

Looking Ahead:

At this point, we have updated our NAnt task from using a single input directory to combine all files in the directory to a more robust solution that allows us to specify which files to include and exclude. We can compile the code and install the task as described in the previous segment for testing. In the next post in this series, I'll be diving into writing a custom element collection that can be added to our NAnt task in order to provide more opportunities.

Source Code:

Writing a Custom NAnt Task (Part 1)

James.R.Eggers@gmail.com (J. Eggers) — Thu, 04 Jun 2009 21:11:00 +0000

Over the past month, I have been working on writing the SchemaSpy Task for NAnt located on CodePlex. When I have began to create a few custom tasks for NAnt in the past, I found very little documentation about how to create one. Much of the documentation surrounding how to create a custom task is based solely on examples without any supporting information. While this works very well for basic things, there were some experiences I had to work through while writing the task for SchemaSpy that I would talk about them here. In this, the first of four parts, we'll be going over how to write your first task and add some simple attributes to it.

To make the example a bit more practical than "Hello World", we'll be creating a log combiner task. In addition to attempting to make a more practical example, the accompanying source code contains both a C# and VB examples and has been refactored to assist in readability.

Getting Started Creating the Log Combining Task:

In order to get started we'll need to make sure we have downloaded NAnt. Once that is downloaded, we can open up Visual Studio and create a new Class Library project. In this example, we'll call the project NAntLogCombiner.

Once the project is created we need to do 4 things:

Rename Class1.cs to LogCombinerTask.cs
Add a reference to the NAnt.Core.dll
Add the NAnt.Core and NAnt.Core.Attributes namespaces to our file
Have our LogCombinerTask class inherit from the NAnt.Core.Task class and implement its abstract members.

After these 4 steps are completed, we'll be left with code that looks similar to the code below.

using System.IO;
using NAnt.Core;
using NAnt.Core.Attributes;


namespace NAntLogCombiner
{
    public class LogCombinerTask : Task
    {
        protected override void ExecuteTask()
        {
            throw new System.NotImplementedException();
        }
    }
}

The NAnt.Core.Task abstract class only requires 1 method to be overridden. The ExecuteTask() method is called by the NAnt build engine and is where we will need to place our code. Since we are going to be combining text files, we need 2 things; a directory that holds the files we want to read, and the output file. For the time being, we'll hard code these variables. After writing the code that consumes the input files and writes their contents to the output file, we have the following in our LogCombinerTask:

protected override void ExecuteTask()
{
    string inputDirectory = @"..\logs\";
    string outputFile = "combined.txt";

    string[] inputFiles = Directory.GetFiles(inputDirectory);

    using (StreamWriter outputStream = File.CreateText(outputFile))
    {
        foreach (string file in inputFiles)
        {
            outputStream.Write(ReadFile(file));
        }
    }
}

private string ReadFile(string fileToRead)
{
    StringBuilder returnString = new StringBuilder();

    using (StreamReader inputStream = File.OpenText(fileToRead))
    {
        while (inputStream.Peek() > 0)
        {
            returnString.AppendLine(inputStream.ReadLine());
        }
    }

    return returnString.ToString();
}

While we're missing a few things (like validating the directory exists, making the paths configurable, etc.), the task builds and will work under its current constraints. Before we can test this inside of NAnt though, we have to configure the class and project a bit more.

Decorating the Class for NAnt:

While we have a working class for combining text files located in a single directory, it is not something that can be consumed by NAnt. NAnt provides an assortment of attributes that can be used to decorate class files in order to inform NAnt how each class (and it's members as we'll see) maps to the markup located in a build file.

In our simple class that we have so far, we only have to add the TaskName attribute onto the class declaration. This attribute tasks just the string name of our task that will represent the element inside of a build file. I'm going to name the task "logCombiner" to be consistent with the casing of other tasks as well as the name and purpose of our project. With the attribute added, the class declaration part of our code now looks like the following:

[TaskName("logCombiner")]
public class LogCombinerTask : Task

Naming the Assembly:

With our class properly decorated, we're ready to build our task in preparation for testing. The way that NAnt imports tasks is through dynamic loading of assemblies and then using reflection. This is a fairly common practice; however, NAnt only imports assemblies that meet the following pattern; "NAnt.*.Tasks.dll". Our project, by default, does not output it's assembly into this naming convention. We can easily remedy this by opening the project properties and setting the Assembly Name to "NAnt.LogCombiner.Tasks.dll" as shown in the image below.

Testing the Assembly:

Now that the Assembly is named correctly for NAnt, we can build the project and test is in NAnt. We can import our task into NAnt by dropping the DLL into NAnt's bin directory or using NAnt's loadtasks task. For simplicity sake, I'm going to assume our NAnt.LogCombiner.Tasks.dll has been copied to NAnt's bin directory.

Once the task has been imported, we can call our task in a build file. Below is an example build file that only calls our new task.

<?xml version="1.0" encoding="utf-8"?>
<project name="Log Combiner" default="default" basedir=".">
    <description>This is an example build file.</description>
    
    <target name="default" description="Default Task">
        <logCombiner />
    </target>
</project>

Before we run this build file, we need to make sure that we have a directory above the current directory called "logs" that contain our text files. Once our log files are in place, we can run our build file where it will sequentially read the files from the directory and merge them into the combined.txt file.

Congratulations on creating your first NAnt task; however, it's pretty generic and not very configurable. Let's enhance our task now by moving where we declare our variables for the input directory and output file into task variables instead of being hard coded.

Adding Task Attributes:

In order to add attributes to our task so we can declare where the input folder or output file is located, we simply have to add a couple of properties to our class file and decorate them accordingly. Let's update our LogCombiner class by adding an InputDirectory property as well as an OutputFile property and update the code to use them appropriately. In addition to implementing the properties, we also need to decorate them with the appropriate attributes so NAnt will know how to map the XML-based attributes to our object.

[TaskAttribute("inputDirectory", Required = true)]
[StringValidator(AllowEmpty = false)]
public string InputDirectory { get; set; }

[TaskAttribute("outputFile", Required = true)]
[StringValidator(AllowEmpty = false)]
public string OutputFile { get; set; }

Each of the two properties have 2 attributes associated with it. The first, TaskName, defines what the task attribute is called inside of the build file and if it's required or not. This name does not have to match the name of the property. The second, is a StringValidator to help ensure the integrity of the data that is passed into it. For the example, the validation on both properties is to just ensure that an empty string is not passed into each attribute.

With these updates made, the only things we have left to do is recompile and deploy the dll, update the build file, and test. Below is the updated build file:

<?xml version="1.0" encoding="utf-8"?>
<project name="Log Combiner" default="default" basedir=".">
    <description>This is an example build file.</description>
    
    <target name="default" description="Default Task">
        <logCombiner inputDirectory="..\logs\" outputFile="combined.txt" />
    </target>
</project>

Looking Ahead:

In this part of this series, we created a simple task with a few attributes to enhance the customization. In the next segments, we'll look into what it takes to add our own types for children nodes similar to the NAnt fileset tags to add file separators as well as allow for including and excluding different files.

Source Code:

VS Project for the Log Combiner Task for NAnt.

SchemaSpy for NAnt

James.R.Eggers@gmail.com (J. Eggers) — Thu, 28 May 2009 01:31:00 +0000

Earlier this year, I talked about a couple of tools to help automate pieces of the development process in some fashion. One such tool was SchemaSpy, a Java-based database documentation tool. SchemaSpy is a tool that I love working with; however, always was annoyed of using it in a manual method. I streamlined this a little bit using a batch script but it still didn't feel quite right when I hooked it up to my automated deployments. So, I did what any process driven developer would do; I made it work more naturally into NAnt.

I started a custom NAnt task for SchemaSpy. Given the complexity of using SchemaSpy to begin with, I decided to not seek to have it added to the NAnt contrib or similar libraries around the web. However, even though I wanted it separate, I did decide to open source it. So, I create a CodePlex project to host my NAnt task for SchemaSpy. In it's current version, it only has the bare minimum of options that I have required; however, I am hoping to continue working on it until it's feature complete. If you are looking for a great way to document your databases in associated with your automated builds with NAnt, take a look at it since it hopefully will provide you options for what you need.

I'm always open for feedback and recommendations on the task. Hopefully I'll be posting the next release with in the week.

Expecting More From Example Code

James.R.Eggers@gmail.com (J. Eggers) — Wed, 13 May 2009 14:17:00 +0000

While I'm typing this post, I'm actually on vacation. A road trip to a destination in hopes to relax and unwind from my daily routines. However, I had to bring the laptop for informational purposes (of course). On the road, I was thinking about some of the posts that I have created and also of those that I refer back to. I also compared a lot of these posts against the increased visibility toward the S.O.L.I.D. principles of Object Oriented design. The more I thought about it, the more I became inspired on the question of should people who read blogs and go to presentations expect more from the authors?

In my opinion, the presentations and the traditional style of technical blog posts are fine. I feel like they teach a concentrated concept in a, hopefully, straight forward manner. Some topics provide more practical implementation; however, how many times have you ran across a demo where the variables were only named Foo and Bar? Presenting a concept is great and effective; however, should more be provided in the examples of in a supplemental post on how to apply the information "properly"?

For example, I wrote a post on using jQuery for different types of visual effects of your elements last year. It was a very simple post with a moderate amount of code that focused on the topic and didn't deviate. I, as the author, left it up to the reader to apply this knowledge into their own works. What if I was to go 1 step further and do a live demo similar to blogs like Beckelman.net and JankoAtWarpSpeed.com? Would that help push the concept better? What if I supplied a zip file with files that apply the concepts learn in a more real-world scenario? A lot of my more recent posts include VS projects; however, it is usually a direct reflection of the code inside of the post. In that jQuery effects posts, I could have easily created a simple directory structure containing the proper files in a traditional/standard method (i.e. separating the CSS and JavaScript into their own directories, etc.). With regards to my .Net posts, should I make sure that all of my code is commented? Refactored properly to meet the SOLID principles (sans my F# code due to it being functional and not oo)? Include Unit Tests? I know a lot of people, including myself, who learned some skill by taking someone else's code and then tinkering and tweaking it to learn how things work based on their changes. I'm wondering if more people would used Unit Testing or "proper" OO design if more of the example code in the blogsphere came with such.

I said earlier that presentations and the traditional blog post is fine as is. The reason why I say this yet question the example code is that the extra stuff (i.e. Unit Tests, Full Comments, properly design class structures) can sometimes sidetrack the post or presentation. Questions often rise about these ancillary items as opposed to the concept you are attempting to teach. So, with that said, I still believe that the posts and presentations are fine.

Should we expect more from the Example Code? In my opinion, yes. I know I have a lot of work to do since adding such polish is time consuming; however, if it can assist in improving the overall quality of the code someone else uses, then it'd be worth it.

What do you think? Should those of us that read blog posts and presentations expect more from the example code tied to such? Should those of us that author posts provide a more comprehensive look at their code in order to encourage better practices?

What do you think?

Automating Features vs. Fundamentals

James.R.Eggers@gmail.com (J. Eggers) — Wed, 08 Apr 2009 16:50:00 +0000

I have been in a lot of meetings over the past few years where inefficiencies in a few practices and policies were being discussed. In terms of software development, there's a lot of things that can be done to improve how the code is written, deployed, and tested. The meetings began talking about these possible options; however, after about a 10 minute discussion in the first meeting, everyone seemed intent on designing the tools specific to their domain and not the groundwork that is common across all domains. Now, nothing is wrong for discussing features to standardize an approach and not reinvent the wheel every time; however, I have learned that the features are domain specific (albeit some may be more common than others) and it's the recreation of the fundamental code that saps a developer's energy, focus, and motivation usually.

Years have went by and still the same people talk about the features without giving any thought into the remedial code that is common and takes the longest usually. Between debating the aspects of the features and evaluating tools, such turned into analysis paralysis very quickly with very few realizing it. Soon, the feature advocates turned more into idea zealots; preaching their views while not provided any acknowledgement to other opinions.

Because of all of this talk and no action, I began to run a few tests with some of my colleagues. There were 3 projects that shared the same core project foundation in it's design but each developer was going to create their own. For the sake of this story, we'll say that this was going to be a "professional" Hello, World console application that had to handle industry standard command line arguments (i.e. -? and a few others). Each person took about 5-6hrs writing the basis of their console application; setting the proper output color schemes and the console output for the help information and other fundamental requirements. Finally it took them about 1hr to do the actual project specific logic. Now, in this example, one could say that Developer-A creates it and then passes it along to the others in a copy/paste sort of way; however, the test needed to be done not only for time purposes but for implementation purposes since all 3 application bases were not consistent.

From these examples, we refined the foundations and eventually abstracted it into a Visual Studio project template. The next time they they had an opportunity to use the template, it saved them that much time yet again. This template was only the fundamentals and didn't have any actual features (no consistent way to do things like emailing or FTP services or error handling). Later versions have added these features into it (or provided a model to make it easy to do a custom implementation) but the original versions were just the fundamentals.

In the end of this experience, the idea zealots were still preaching their features without recommending any solution to their implementation and we became more efficient by automating the fundamentals. If you're looking to improve efficiency, try to find remedial, repetitive areas in your process and see about automating them. Start with the fundamentals and then add features to it. There'll be changes; however, if you Develop > Refactor > and then Abstract what works, you'll be in a much better place than if you stayed in a meeting room and debated about what features should be in the initial version of the script or template. My Rule of Thumb from experience: Version 0 of any automation script or template should only have the minimum amount of features possible and focus on the fundamentals.

How I'm Learning F# - Working with .Net Objects and Properties

James.R.Eggers@gmail.com (J. Eggers) — Wed, 01 Apr 2009 11:00:00 +0000

Over the past few months, I've been hearing more and more about the use of functional programming concepts and also languages like Haskell and F#. While some of the initial musings that I've read revolved around how the concepts have been around for decades and that it makes financial and scientific applications easier to read and write, I couldn't find a good reason to start learning it for my typical line-of-business application design and development job or even some of my basic hobby projects. Nonetheless, I kept getting drawn to the concept and have decided to focus on learning it.

This post marks the fourth entry of a series that I'll be writing to discuss how I'm going about learning F#. While I'm not saying that my method of learning is ideal and should be followed by others, I'm just reporting how I'm going about doing it. Over the course of this series, my goal is to provide other .Net developers a(nother) resource for learning the F# language as well as apply the language into some non-financial or scientific scenarios.

Series Table of Contents:

Finding Resources
Writing the First Application
Interacting with the .Net Framework
Working with .Net Objects and Properties

Project Overview:

In this post, I'm going to interact with libraries in the .Net framework to illustrate how to you'll be able to apply F# to functionality that normally may do in another language. Unlike the last project in this series that just brushed on using some static methods of the System.IO.File class, this post will be focusing on the various steps it takes to send an email message and working with the properties of the System.Net.Mail.MailMessage class. In this example, we'll be doing the following:

Instantiating a new MailMessage object.
Set the To, From, Body, and Subject properties of the object
Attach a text file to the message.
Send the message via a SMTP server.

Throughout the course of this project, we'll be dealing with a large amount of F# syntax as well as a couple of assemblies from the .Net Framework. This post will hopefully be pretty straight forward in the steps outlined above.

Starting With The Values:

To keep this example simple, I'm going to be declaring our To Address, From Address, Subject, and SMTP Server values first thing. We will be using these values as we begin to set the object properties of our MailMessage object shortly; however, to allow for easier visibility, I'm setting these up prior.

let ToAddress = "MyToAddress@domain.com"
let FromAddress = "MyFromAddress@domain.com"
let Subject = "F# Email Example"
let SMTPServer = "127.0.0.1"

Instantiating Objects in F#:

Now that we have our values established, let's open up the System.Net.Mail namespace and instantiate our own MailMessage object. To do this, we do the following snippet of code:

open System.Net.Mail

let mm = new MailMessage()
mm.To.Add(ToAddress)
mm.From <- new MailAddress(FromAddress)
mm.Subject <- Subject
mm.Body <- "Hello, F# Email"

Here we begin to see some new syntax in F#. The syntax to instantiating a new MailMessage object is very similar to that of C# or VB.Net in that we are setting a value using the new keyword. Next we are calling the MailMessage's To collection proptery's method of Add() to add our address we assigned initially. After that, we are assigning the MailMessage.From property to a new instance of a MailAddress Object. Since object properties that are not read-only are mutable (or can change), we use the <- operator to assign a value to the property. This action is repeated to assign the Subject and Body properties on the last lines of the snippet.

Attaching a File:

The last item we have to do with our MailMessage object is to add the attachment. The file we're going to use is a local text file. For simplicity sake, we'll use the relative path to the file and assign it to a value first. Then we'll add the file as a new attachment to our MailMessage object.

let filePath = @".\commadelimitedfile.txt"
mm.Attachments.Add(new Attachment(filePath))

Sending the Email:

We now have a perfectly good MailMessage object sitting in memory. The only thing that's left to do is to send it out.

let client = new SmtpClient()
client.Host <- SMTPServer
client.Send(mm)

The Full Project Source:

       1: #light       2:         3: let ToAddress = "MyToAddress@domain.com"       4: let FromAddress = "MyFromAddress@domain.com"       5: let Subject = "F# Email Example"       6: let SMTPServer = "127.0.0.1"       7:         8: open System.Net.Mail       9:        10: let mm = new MailMessage()      11: mm.To.Add(ToAddress)      12: mm.From <- new MailAddress(FromAddress)      13: mm.Subject <- Subject      14: mm.Body <- "Hello, F# Email"      15:        16: let filePath = @".\CommaDelimitedFile.txt"      17: let att = new Attachment(filePath)      18: mm.Attachments.Add(att)      19:        20: let client = new SmtpClient()      21: client.Host <- SMTPServer      22: client.Send(mm)  

Summary

This has been a very short and simple example on how to interact with the .Net framework objects and their properties. At this point, some basic and typical tasks that are used in applications could be transposed to F# if you so desired. With F# being a concise language, it has some benefits over traditional object-oriented languages.

jQuery, JSON, and ASMX 2.0 Services

James.R.Eggers@gmail.com (J. Eggers) — Thu, 26 Mar 2009 02:15:00 +0000

A few weeks ago, I had a project that was grounded in the .Net Framework v2.0 and Visual Studio 2005. The requirements were very focused on usability and speed so AJAX and jQuery were high on my list to use. Through the project, I learned that there isn't a large amount of information in one place that tells you how to setup a ASP.Net solution that uses jQuery and ASMX services to effectively transmit json data back and forth from the client. Because of this, I'll attempt to fill this void since there are still many developers and companies out there that have not been able to upgrade to Visual Studio 2008.

In this post, I'll discuss the process of building a plain ASP.Net 2.0 web application project (not web site project) , setting up the necessary entries in the web.config file to utilize the ASP.Net 2.0 AJAX Extensions v1.0, and use jQuery at the client side of the transfers. In addition, I'll also show to to use the JavaScriptSerializer class and how to write your own custom converter for your objects.

A Note About This Post:

This post is focused on Visual Studio 2005 with the ASP.Net 2.0 AJAX Extensions v1.0. The v3.5 of the extensions that came with ASP.Net v3.5 and Visual Studio 2008 have a few changes. While I will try to point out the differences, the core of this post is going to be focused on Visual Studio 2005 web application projects and v1.0 of the AJAX extensions. While we all love focusing on the latest and greatest, I'm aware that there are a large number of companies and professionals out there that are locked into using the older version of the software for a number of reasons.

Additions to VS2005 Used in This Post:

This post will be using the following additions to VS2005. Below are the items and links to their respective installers:

Code Downloads for This Post:

Visual Studio 2005 Example Project

Creating and Configuring a New WAP:

Now that we have the required additions established, we're ready to create a new ASP.Net 2.0 Web Application Project. I'm not going to get into the details of how to do this; however, I want to stress that I'm NOT choosing an ASP.Net AJAX Enabled Web Application. I'm just choosing to create a new, basic ASP.Net Web Application Project. Now that the project has been created, we need to add a few references and add some information into the Web.Config file.

In the Solution Explorer, we'll need to add a reference to the AJAX Extensions v1.0 library, System.Web.Extensions.dll. By default, this library is located at C:\Program Files\Microsoft ASP.Net\ASP.Net 2.0 AJAX Extensions\v1.0.61025\ directory. After adding this reference, we can update our Web.Config file by including a reference to the ScriptHandlerFactory HttpHandler using the following snippet inside of the <System.Web> config section:

       1: <remove verb="*" path="*.asmx"/>       2: <add verb="*"        3:      path="*.asmx"        4:      validate="false"        5:      type="System.Web.Script.Services.ScriptHandlerFactory,        6:            System.Web.Extensions,        7:            Version=1.0.61025.0,        8:            Culture=neutral,        9:            PublicKeyToken=31bf3856ad364e35"/>  

By adding the AJAX Extensions reference and updating the web.config file, we're now ready to enable our ASP.Net Web Services (ASMX) to be called from JavaScript.

Setting Up an ASMX Web Service:

We have our new WAP setup and configured, it's time to write a simple ASMX web service and configure it so that it will be able to return JSON. To do this, let's add a web service to our project called JsonService.asmx and add the following code snippet to replace some of the defaults Visual Studio gives you:

       1: using System.Collections.Generic;       2: using System.Web.Script.Services;       3: using System.Web.Script.Serialization;       4:         5: [WebService(Namespace = "http://YourNamespaceHere.com")]       6: [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]       7: [ScriptService()]       8: public class JsonService : System.Web.Services.WebService       9: {      10:     [WebMethod]      11:     [ScriptMethod(ResponseFormat = ResponseFormat.Json)]      12:     public string GetCustomer()      13:     {      14:         // Method Body      15:     }      16: }  

Lines 1-3, I imported 3 additional namespaces to use in the code. System.Collections.Generics will be used in the next section. System.Web.Script.Services allow us to decorate the service and it's methods as Script methods for the ScriptHandlerFactory to use when making AJAX calls to and from the client.

Line 7 decorates the web service with the [ScriptService()] attribute.

Line 11 decorates the GetCustomer() web method with the [ScriptMethod()] attribute. This attribute tells the ScriptHandlerFactory that this method is allowed to be called from an Ajax Client. The properties inside of the attribute, ResponseFormat = ResponseFormat.Json, tells the ScriptHandlerFactory to send the response stream as a json string and not XML or Soap. If a response to the web service that is not formatted as json, the response will be returned as XML.

At this point, we can create the body of our web method in any fashion as long as it returns a string. If you are only passing base types, you can skip down to the Talking to the Server Using jQuery section; however, if you want to pass something a bit more complex, I recommend you continue to the next section.

Using the JavaScriptSerializer and a Custom Converter:

While passing base types is easy enough, it can be important to pass objects back and forth from the client. In order to assist with this, the ASP.Net AJAX Extensions v1.0 comes with the JavaScriptSerializer object. This object has the ability to serialize certain objects into strings representing json objects. This sounds great and the answer to all of our problem! Too bad it is very limiting in it's natural state. It CAN convert Arrays of base types and (from what I can tell) any framework classes that implements IEnumerable<T>. I haven't experimented with some of the more obscure generic collections; however, I do know that it does concern List<T> and Dictionary<S,V> just fine.

In order to use the JavaScriptSerializer, you simply instantiate it and call its Serialize() method, passing the object that you wish to serialize. If this is a string array or an object of type Dictionary<string,string>, it will do all of the heavy conversion for you and give you a nice little string to return to the client as shown below:

       1: // Method Body       2: Dictionary<string, string> customerInfo = new Dictionary<string, string>();       3: customerInfo.Add("FirstName", "John");       4: customerInfo.Add("LastName", "Doe");       5: customerInfo.Add("EmailAddress", "JohnDoe@Domain.Com");       6: customerInfo.Add("PhoneNumber", "555-555-1212");       7:         8: return new JavaScriptSerializer().Serialize(customerInfo);  

In this code snippet, I have instantiated a new Dictionary<string,string> generic object and populated with the property information for a customer. Lastly, I instantiate a new JavaScriptSerializer object and call it's Serialize method, passing our customer information into it. The JavaScriptSerializer will create following string from our dictionary:

       1: {"FirstName":"John","LastName":"Doe","EmailAddress":"JohnDoe@Domain.Com","PhoneNumber":"555-555-1212"}  

This is just a simple string that, technically, we could have concatenated ourselves; however, you see how it converts the name-value pairs of the Dictionary object and turns them into a json Object with properties and string values.

Seems pretty simple. Now, let's turn our customer Dictionary into a CustomerInfo object with the same four properties. Below is the class definition:

       1: public class CustomerInfo       2: {       3:     public string FirstName { get; set; }       4:     public string LastName { get; set; }       5:     public string EmailAddress { get; set; }       6:     public string PhoneNumber { get; set;}       7: }  

Now, if we replace our Dictionary object with our CustomerInfo object we get code that looks similar to the following, easier to read, snippet:

       1: // Method body       2: CustomerInfo custInfo = new CustomerInfo();       3: custInfo.FirstName = "John";       4: custInfo.LastName = "Doe";       5: custInfo.EmailAddress = "JohnDoe@Domain.Com";       6: custInfo.PhoneNumber = "555-555-1212";       7:         8: return new JavaScriptSerializer().Serialize(custInfo);  

Sadly, running the above code will give you the following error when you attempt to run it:

Since the JavaScriptSerializer doesn't know the definition of our CustomerInfo object, we get this Circular Reference error. This causes us to go down one of two roads. We can either turn the object manually back into our Dictionary object, or we can write a custom JavaScriptConverter for our CustomerInfo Class. The nice thing about a custom converter is that once it's setup to our JavaScriptSerializer, we can then have it convert any number of CustomerInfo classes (or collections of CustomerInfo objects) we may need.

Writing A Custom JavaScriptConverter

Writing our own custom JavaScriptConverter is not as difficult as one may first assume. Inside the System.Web.Script.Serialization namespace, we are provided with an abstract base class that helps us outline the definition and gets us started quickly. In our project, let's add another class called CustomerInfoConverter. Have the class inherit from the JavaScriptConverter class and right-click on the class name and select "Implement Abstract Class". What you get is the following code snippet:

       1: public class CustomerInfoConverter : JavaScriptConverter       2: {       3:     public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)       4:     {       5:         throw new Exception("The method or operation is not implemented.");       6:     }       7:         8:     public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)       9:     {      10:         throw new Exception("The method or operation is not implemented.");      11:     }      12:        13:     public override IEnumerable<Type> SupportedTypes      14:     {      15:         get { throw new Exception("The method or operation is not implemented."); }      16:     }      17: }  

The first method the JavaScriptConverter makes us define is the Deserialize method. This method is used by the JavaScriptSerializer to convert a json object from a client into the expected type of the web method uses as a parameter. The JavaScriptSerializer automatically converts the json object into a Dictionary<string, object> collection. Inside this method, you could add our mappings between the Dictionary keys to a new CustomerInfo object's properties and finally return that new instance of the CustomerInfo object. Below is the method body:

       1: CustomerInfo cust = new CustInfo();       2: cust.FirstName = dictionary["FirstName"].ToString();       3: cust.LastName = dictionary["LastName"].ToString();       4: cust.EmailAddress = dictionary["EmailAddress"].ToString();       5: cust.PhoneNumber = dictionary["PhoneNumber"].ToString();       6:         7: return cust;  

The second method the JavaScriptConverter makes us define is the Serialize method. This method is used during the actual serialize process. Here, we're creating a new Dictionary<string, object> collection for our CustomerInfo class. Since our CustomerInfo class will be boxed through the method's parameter, we'll need to properly cast it before we can start adding the values to the Dictionary object. Below is the method body:

       1: // Cast the obj parameter       2: CustomerInfo cust = obj as CustomerInfo;       3:         4: if (cust != null)       5: {       6:     Dictionary<string, object> result = new Dictionary<string, object>();       7:     result.Add("FirstName", cust.FirstName);       8:     result.Add("LastName", cust.LastName);       9:     result.Add("EmailAddress", cust.EmailAddress);      10:     result.Add("PhoneNumber", cust.PhoneNumber);      11:        12:     return result;      13: }      14:        15: // If the obj doesn't convert for some reason, return an empty dictionary.      16: return new Dictionary<string, object>();  

The last item the JavaScriptConverter makes us define is the SupportedTypes property of type IEnumerable<Type>. This property should be a collection of types that this converter supports. Since we are using this converter only for our CustomerInfo class, the property can be simplified to the following line of code:

       1: get { return new Type[] { typeof(CustomerInfo) }; }  

Now that we have our converter built, we can attach it to our serializer to get our json string as shown below:

       1: CustomerInfo cust = new CustomerInfo();       2: cust.FirstName = "John";       3: cust.LastName = "Doe";       4: cust.EmailAddress = "JohnDoe@Domain.com";       5: cust.PhoneNumber = "555-555-1212";       6:         7: JavaScriptSerializer jss = new JavaScriptSerializer();       8: jss.RegisterConverters(new CustomerInfoConverter[] { new CustomerInfoConverter() });       9: return jss.Serialize(cust);  

Talking to the Server using jQuery:

Now that we have our server-side infrastructure setup, we can begin to write our AJAX client-side code using jQuery. Compared to all of the code we've written, this is the easier part. Below is the JavaScript/jQuery code that can be used to call our web service:

       1: function GetCustomerFromServer()       2: {       3:     $.ajax({       4:         type: "POST",       5:         url: "/JsonService.asmx/GetCustomer",       6:         dataType: "json",       7:         data: "{}",       8:         contentType: "application/json; charset=utf-8",       9:         success: function(msg){      10:             var custInfo = eval("(" + msg + ")");      11:             alert(custInfo.FirstName);      12:         }      13:     });      14: }  

In this client-side code, we're making a HTTP Post call to our web service by calling the web method of it directly. We are stating that we are sending and receiving data in json format. Despite the fact that our GetCustomer() web method does not take any parameters, we still need to send an empty json object in order to have json returned.

Lastly, we write an anonymous method to handle the successful message returned to us. The message is evaluated in order to be turned into a json object on the client side. We then validate the object by echoing the FirstName property in an alert box.

One thing to remind, this is for ASP.Net 2.0 Web Services. The response from ASP.Net 3.5 Web Services IS different in that the response message (msg) has it's content in a property only called "d". So instead of eval("(" + msg + ")"), it would be eval("(" + msg.d + ")").

Summary:

This post has covered a large amount of steps to get a json-based web service infrastructure setup using jQuery and ASP.Net 2.0 web services. After a lot of low level research and asking questions to people, I realized that there was not a single location for this information. Hopefully, this post will help fill that gap.

How I'm Learning F# - Interacting with the .Net Framework

James.R.Eggers@gmail.com (J. Eggers) — Wed, 18 Mar 2009 21:57:00 +0000

This post marks the third entry of a series that I'll be writing to discuss how I'm going about learning F#. While I'm not saying that my method of learning is ideal and should be followed by others, I'm just reporting how I'm going about doing it. Over the course of this series, my goal is to provide other .Net developers a(nother) resource for learning the F# language as well as apply the language into some non-financial or scientific scenarios.

Series Table of Contents:

Finding Resources
Writing the First Application
Interacting with the .Net Framework

Downloadable Resources:

F# Email Example VS2008 Project

Project Overview:

In this post, I'm going to interact with libraries in the .Net framework to illustrate how to you'll be able to apply F# to functionality that normally may do in another language. The basis of this project will incorporate 2 activities:

Read from a comma delimited text file
Create a new Fixed-Width based text file

Throughout the course of this project, we'll be dealing with a large amount of F# syntax as well as a couple of assemblies from the .Net Framework. This post is a bit of a step from the previous post; however, I am hoping that a lot of the concepts will be applied through a little more comprehensive example.

What Does It Take to Read a Text File?:

While I was writing this example, I began to think of how a person traditionally learns a language. While in our day to day jobs we may gloss over some of the granular steps of what it takes to read from a text file, I began to look for that level of detail in this program. To read a text file in any .Net language the follow basic steps must be done:

Open the System.IO namespace
Call the File.ReadAllLines(string) static function, passing the path to the file as a parameter.

What's nice about F# is that when you think about the detailed steps of a task, you begin to see the lines that you need to write. Here is a function in F# that does the above steps:

open System.IO
let readFile =
File.ReadAllLines(@"C:\...\myFile.txt")

By writing the above code, we have just opened a .Net Assembly (System.IO in this case) and declared a value that returns an array of strings representing each line of the text file. In the above code, we could have fully qualified the ReadAllLines() function instead of opening the assembly; however, we will be creating a new file here in a moment so this works out better.

How Do I Manipulate This Array of Lines?

So, we have an array of strings representing the lines of delimited words. Now what? If we want to take these lines and break them into a fixed lines, we'll need to do the following things:

Identify each word in each line (a.k.a. split the delimited string of words)
Pad each word with spaces until its total length is 25
Concatenate the words into one long string per line

Once again, we can directly map each of these lines to a line or function of code. Let's see how these steps would look when we translate them into just functions:

let obtainWords (line : string) =
    line.Split(",".toCharArray())

let padWord (word : string) =
    word.PadRight(25, ' ')

let joinWords (words : string []) =
    System.String.Concat(words)

Our 3 steps translate fairly easily into single lined functions thanks to some build in String functions of the .Net Framework. Our obtainWords function takes a string and splits it using a comma being the delimiter. Next, our padWord function takes a string an pads it to the right to ensure that it's 25 characters in length. Lastly, we call the System.String.Concat function to take a string array and turn it into a single string.

We have our "what" to do to the lines but we haven't really answered the "how". In traditional C# or VB.net, we would probably use a for loop against each read line and then call each function to update the variables in those languages. We would end up with something that looks like the following in C# (using our function names from above):

for(int x = 0; x<readFile.Length; x++)
{
    string[] words = obtainWords(readFile[x]);

    for(int i = 0; i<words.Length; i++)
    {
        words[i] = padWord(words[i]);
    }

    readFile[x] = joinWords(words);
}

Here we have a loop that iterates through each line read by the readFile function. Then we split each line into another array variable. Next we iterate over the words array and update the values of the array to the padded versions. Finally, we update the line with the combined strings of our padded words.

The code is pretty straightforward but I don't know many people who like inner loops. Also, some developers may fall into a trap and attempt to use a foreach loop instead of a for loop. If you are not aware of the difference here, the variable generated by a foreach loop is readonly. I could iterate over the arrays in a foreach loop;however I would have to add the values into a different variable altogether in order to "update" the values like I did above. Thankfully, F# has a special function that cleans this up for us.

Understanding Array.map()

One of the built in functions that I have really enjoyed in learning is the Array.map() function. The Array.map() function takes a function value and an array as parameters. It returns a new array that is comprised of values where the function was applied to each element of the provided array. An example of this functionality is line C# example where we took the words array of strings and updated all values of the array with the padWord() function. By using the Array.map() function in F#, we get code that does the same thing but looks like the following:

let newWordsArray =
Array.map (padWord) words

This creates a new array (newWordsArray) where the values are the same as if each string in the words array was applied to the padWord() function. One nice thing about the Array.map() function is that it also allows lambda expressions to be used in place of the function value also. Lambdas will be covered at a later time though. By using Array.map(), we can begin to chain our values together and be able to take our delimited file and turn it into a fixed with file. However, before we get into that, let's look at a technique in F# that allows us to chain these together even easier called Pipelining.

Pipelining with |>

Pipelining is a technique where you pass the returned value of 1 item and pass it as the parameter of another function. This is very similar to the a technique in shell scripting using the pipe (|), greater than (>), and double greater than (>>) operators. For example, in a command window you can type in type in the dir command and see a list of directories; however, if you wanted to apply paging to the list you can type in dir | more. Likewise, if you wanted to send the directory listing from the dir command to a file you could type in the dir > file.txt to create/overwrite a new file with the redirected output or dir >> file.txt to append the directory listing to the contents of file.txt if the file already exists.

In F#, we can take a function and send it's returned function to another function using the |> operator. In our last examples, we can illustrate this by pipelining the return value of the obtainWords function (which returns a string array) into our applyPadding function. The resulting function (see below) returns a string array that already has every string in the array padded to 25 characters.

let obtainWords (line : string) =
line.Split(",".toCharArray())
|> applyPadding

At first glance, this may seem a bit confusing; however, remember what I said in that the output of the first expression (in this case line.Split()) is used as the parameter of the second (applyPadding). In essence we are just reordering how we are just reordering a chain of events. This simple example is just to show how we can remove the need for one additional value to hold the output of obtainWords before passing it to applyPadding. Below is a more applicable example where we do multiple chains in order to give our initial readFile function the ability to output an already transformed array of strings.

open System.IO

let obtainWords (line : string) =
line.Split(",".toCharArray())

let padWord (word : string) =
    word.PadRight(25, ' ')

let applyPadding (words : string []) =
    Array.map (padWord) words

let joinWords (words : string []) =
    System.String.Concat(words)

let readFile =
    File.ReadAllLines(@"C:\...\myFile.txt")
    |> Array.map(obtainWords)
    |> Array.map(applyPadding)
    |> Array.map(joinWords)

Here, we open our System.IO namespace. Next we define our functions that we'll be used to transform the data in the file. Lastly, we create our function, readFile, that reads the lines into a string array, pipes the array into the Array.map function that maps the obtainWords function to the array and returns an array of words. Those words are then padded and subsquently joined. Through all of those steps, the readFile now contains an array of strings that represent the fixed width version of the comma delimited file that it read. The final step is to now write those lines to our output file.

File.WriteAllLines(@"C:\...\myOutputFile.txt", readFile)

Summary

At this point, another function can be established to do anything with the output file that you wish. You could use the System.Net namespace to gain access to the mail message object and email the new file to another process or possibly even FTP/copy it somewhere. This is just a simple file transformation example to illustrate some advanced functionality and how to interact with the .Net framework using System.IO and System. If you know any other .Net language, it works exactly the same in F# as in VB or C# from what I can tell so far.

This project was one where things began to click for me inside of F#. I knew the basics and seen examples through euler problems and such; however, seeing an example like this saw how easy and straight forward F# can make things.

Tackling Anxiety Against Automation

James.R.Eggers@gmail.com (J. Eggers) — Thu, 12 Mar 2009 17:26:00 +0000

A few years ago, the company that I worked for was preparing to upgrade from MS Sql Server 2000 to Sql Server 2005. While there wasn't too much abnormal concern about upgrading the databases themselves, there was a lot of concern about the large number of DTS packages that the company created for the majority of their B2B processes. Their solution until they could come up and complete a test strategy to ensure the DTS packages would run under Sql 2005's DTS runtime, just incase the packages couldn't be converted to SSIS in some fashion, was to stop writing new DTS packages and push the processes into .Net console applications that would be scheduled through a batch processing system. Personally, I liked this solution since it allowed me to write .net code in VS and not VBS code inside of the DTS Package designer in Enterprise Manager. Actually abstracting out processes into a more reusable and testable state was a great benefit over VBS and DTS as a whole.

One of the first console applications written under this initiative was to automate a series of manually ran Sql Scripts that people were running against production to accompany a file import process. It was a pretty simple project and ran, still to my knowledge, bug free, even when tested by my QA team.

Now fast forward to today. The need for the process is still in place and the code hasn't been updated since no changes have been required; however, I found out that the application was only ran once in the past few years and then the person who used to manually go through the steps and scripts didn't want to automate it and has been doing the manual steps ever since. Even today, this person has continued to manually spend a few hours every other day going through the same ceremony. I think my brain gave me a BSOD from what I find highly illogical thinking.

So I approached my coworker who has been doing this and start asking the number of "why" questions I had. Here are some of the responses I was treated to:

"I don't trust what I cannot see or watch."
"I like being able to watch and correct where necessary."
"The process had a bug."
"I won't ever know when it breaks or succeeds."
"What else would I work on?"

From the discussion around these themes, there are a few design and development themes that can/should be implemented when faced with such an anxious coworker.

Establishing Trust through Quality

While I see James Bach's point in his "Quality Is Dead" blog post, I disagree with the blanket of the statement. There are always constraints when you're talking about software development (usually Time/deadlines being the largest constraint), it doesn't mean that every piece of code that you churn out will be of poor quality. Practices such as TDD or just plain Unit Testing and reporting can be implemented in order to demonstrate and achieve a higher level of quality to someone who doubts quality. Now, I believe that any developer can write code with and without unit tests and still achieve the same level of quality; however, it's easier to validate that level of quality through unit tests and TDD practices. This level of validation is essential to people who do not trust something to be automated.

Establishing Awareness through Instrumentation

The bulk of the other themes that I received surrounded the concept of being able to step through the process and watch/correct the data as necessary. This is a nice thing for debugging; however, if an automated process has to be stepped through, then it's not automated. One way to handle this issue is to ensure you have a high level of coverage of instrumentation in the code/process. This means making sure the proper types of notification is provided, auditing is turned on to the degree needed, and any errors (be it bugs or bad data-related) are captured and reported appropriately. I cannot confirm my coworker's claim that there was a bug in the code since I was unable to find one in the bug database. If an email to capture the error existed, at the very least, I would begin having my own doubts about the quality of the code; however, even that was not found. If instrumentation was implemented to a better degree than what I put into it for error logging and tracking, I would be able to verify the claim. As far as bad-data and catastrophic scenarios (i.e. server loses network connection mid-process), those items can (and was) implemented through logging or some for of process auditing infrastructure. The more data that can be provided, the more confidence the person will have when automated.

Establishing Automation Can Assist with Workload

The last theme that came from the conversation dealt with how the person's routine became. This person was a very routine-oriented individual who knew exactly when to do when and how long it would take. This person was a person of ceremony and possibly a bit fearful of being automated out of a job. After being in the industry for a number of years, I have came to the conclusions that those nightmare stories of people being replaced by programs rarely happen to the developers or analysts that implement the program or automated process. The automation allows for new, different tasks to be done. Even if the automated process is kicked off manually, it still frees up time to do other things. This means you can do more easily or in general transition onto the next big thing. I do not really know any company out there that would say they have a truly shortage of work. Even if it's wish-list, lowest priority, internal cost projects, there's always something to do.

Summary

Automation is a beautiful thing that I have grown more and more fond with. I have grown into a developer where I look for things to automation. I have seen the themes I talked about here at a couple of different companies I've been with and people that I've talked with. Each time, I find more and more ways to automate certain elements of a work day and also make the work flow easier. Hopefully a few things here will help others as well. Now if only I could find a way to automate my chores at home. :-)

How I'm Learning F# - Writing the First Application

James.R.Eggers@gmail.com (J. Eggers) — Fri, 13 Feb 2009 02:34:00 +0000

This post marks the second entry of a series that I'll be writing to discuss how I'm going about learning F#. While I'm not saying that my method of learning is ideal and should be followed by others, I'm just reporting how I'm going about doing it. Over the course of this series, my goal is to provide other .Net developers a(nother) resource for learning the F# language as well as apply the language into some non-financial or scientific scenarios.

Series Table of Contents:

Finding Resources
Writing the First Application

Downloadable Resources:

F# Hello, World Visual Studio 2008 Project

Setting Up Visual Studio 2008 with F#:

As of this writing, F# is at CTP status since September 2008 (v1.9.6.2). For this example, and possibly for the remainder of the series, I'll be using this September 2008 CTP of Microsoft F#. This CTP installer only works with Visual Studio 2008. The CTP version does not work with Visual Studio 2005 (which previous versions did) nor the Visual Studio Express editions. If you would like to use F# using Visual Studio 2005, please refer to this StackOverflow question about how to set up F# in Visual Studio 2005 using an older version. NOTE: If you are using an older version of F# for use in Visual Studio 2005, I cannot guarantee that the examples presented in this series will work. After you have downloaded and installed F# (either the CTP or a previous version), we are ready to begin writing our first F# application.

Writing the "Hello,World!":

Like many great computer books, this series will be starting with a simple example which we'll dissect afterwards. In order to write the example, we'll need to start a new F# project in Visual Studio. If we go to File > New Project..., we are treated to the following dialog window:

The projects listed under Visual F# include a F# Application, a F# Library, and a F# Tutorial. For the sake of this example, we'll be creating an F# Application named HelloWorldFSharp. After clicking on the OK button, Visual Studio creates the new F# project containing a single Program.fs file.

When the project is created, the Program.fs file contains a single line #light. This tells the F# compiler to use the lightweight syntax of F# which removes some of the verbosity of the language and applies rules around white-space. In the majority of examples on the Internet and books, the lightweight syntax is used and will be used going forward here as well.

Under the #light line, add the following code snippet:

let question = "How are you?"

let sayHello name =
"Hello, " + name + "! " + question

In this code, we are doing two things. First, we are declaring a variable named question. Second, we are declaring a method called sayHello and takes a parameter called name. The sayHello method concatenates some strings together and returns the composed string. When using the #light syntax, every indented line is considered part of the function or action with the last line being the return value.

One thing to mention about F# when compared to other languages is that variables are immutable. Once a variable is declared (and given a value), that value can never change. I believe there are some exceptions to this when interacting with object properties; however, at the core of the language, basic variables cannot be re-declared nor their values redefined after they have been initially declared. Because of this, many F# documents and tutorials will use "values" instead of "variables" to prevent the confusion. I'll attempt to follow this pattern.

One thing that a developer coming from VB.net or C# may notice is that there are no types defined here. F# infers typing based on the value and how they are used. If we wanted to declare a type for our question and name values, the code would like like the following:

let (question : string) = "How are you?"

let sayHello (name : string) =
"Hello, " + name + "! " + question

In F#, you can declare the type by using (variablename : type) syntax.

Now that we've seen how to declare a value and a method, we can output a value to a console window using the following line:

printfn "%s " (sayHello "James")

Here, we are using the printfn method which prints formatted text to the console window. The %s is a placeholder for a string value which we are supplying by calling our sayHello method passing in the name "James". In this example, we're treating the method like a value.

Here is the full code of Program.fs.

       1: #light       2:         3: let question = "How are you?"       4:         5: let sayHello name =        6:     "Hello, " + name + "! " + question       7:         8: printfn "%s " (sayHello "James")  

Running the Program using F# Interactive:

At this point, we can either go the Build menu and build the project like many other console applications or we can use a feature unique to F# called the F# Interactive window. F# Interactive (fsi.exe) allows you to run sections of your F# program as well as write ad-hoc F# statements. To use F# Interactive from within Visual Studio, highlight the code you wish to run and press Alt+Enter. A new Visual Studio panel opens for F# Interactive and reports information about the code (or the error it generates).

For example, if you were to highlight the line where we declare our value question and press Alt+Enter, the following will appear:

val question : string

What this line explains is that we have a value called question that is of type string. If we do the same to our sayHello method, the following appears:

val sayHello : string -> string

This line states that sayHello is a value that accepts a parameter of type string and returns (signified by the -> ) a value of type string. If we highlight the all of the code we've written so far, we are shown the following:

val question : string
val sayHello : string -> string

Hello, James! How are you?

This shows us the same output as we had previously for question and sayHello. In addition, it executes the last line and displays the output.

Conclusion:

While this wasn't the largest demo nor the most unique, it is a simple step forward. The premises of immutable variables (values) being declared and understanding the lightweight syntax are important to learning F#. Over the next post of the series I'll be making some different applications in order to continue to explore F#'s syntax and it's power. F# can do pretty much everything that VB.net or C# can do (with a few exceptions I'm now finding out) and hopefully the next posts will help us begin to find examples of where to apply it.

How I'm Learning F# - Finding Resources

James.R.Eggers@gmail.com (J. Eggers) — Wed, 04 Feb 2009 13:00:00 +0000

This post marks the beginning of a new series that I'll be writing to discuss how I'm going about learning F#. While I'm not saying that my method of learning is ideal and should be followed by others, I'm just reporting how I'm going about doing it. Over the course of this series, my goal is to provide other .Net developers a(nother) resource for learning the F# language as well as apply the language into some non-financial or scientific scenarios.

Series Table of Contents:

Finding Resources

Where To Start:

One of the challenges of learning anything is to find where to start gathering the necessary information on the subject matter. Thankfully, there's already a great number of sources that can be used to learn F#. Here are some that I've been using to get started in F#:

StackOverflow.com's F# tagged Questions
Foundations of F# by Robert Pickering
Expert F# by Don Syme, Adam Granicz, and Antonio Cisternino
MSDN F# Developer's Portal

Of these 4 resources, I've so far leaned on StackOverflow and Expert F# the most. I have examined only a little bit of the MSDN site and have been using it if I cannot find my answer on StackOverflow. I initially began reading Foundations of F#; however, I was having a hard time building on each chapter. Expert F# flows a lot better and also build on each previous chapter which makes it ideal for learning.

Why am I learning F#:

I'm learning F# in order to broaden my skill set into functional programming. Initially, there wasn't any other reason for that. I looked at OCaml, Haskell, and F# as all possibilities and eventually settled on F#. Since then, I have began seeking a better reason for learning F#. Many of reasons presented in interviews and blogs and books talked about it's use in financial and scientific fields of study. While this is fine, I was still looking for more since I, like many other developers, am not in those fields of application development often. After diving deeper, I found that F# is a very concise language that allows developers to write some code faster and in a more logical way. Where as there is a large amount of code used to write a program in OO for the sake of being OO, F# allows a very simple and straight forward syntax to write programs or pieces of .Net applications in a functional manner.

What F# can do:

One other thing that I was fooling myself with once I began learning F# was this idea that it could or couldn't do something that a different .Net language could. It wasn't until I began reading Expert F# that I truly understood how narrow minded I was being. In Expert F#, the authors do a fantastic job at illustrating not only the language in a general sense, but also how to apply F# as a replacement to VB.net or C# in applications like ASP.Net, Winforms, and even Domain Specific Languages. While, from what I can tell, many people agree that replacing C# or VB.net with F# in the ASP.Net realm isn't the best or most recommended application for the language, it does illustrate the point that F# can do anything that C# or VB.Net can do today.

Where does F# fit:

I have a feeling that I may be skewered a bit for this; however, this is just my perception as a novice F# developer at this point. I've been developing in Microsoft technologies for years now and have also done work in other languages. Where I see F# fits is where you have functional steps of processing of data. I can see F# being a great language for writing the data engines for custom ETL applications and processes, as well as similar business logic processors. I am certain I'll find additional locations that would assist me in my daily development job as I learn more of F#.

What's Next:

Next up in the series we'll dive into setting up VS2008 with the F# CTP and write the ever so popular "Hello, World!" or something like that :-). Until then, I highly recommend checking out the F# resources mentioned above and also read a lot of the questions currently out there on Stack Overflow.

Saving Time Through the Command Prompt Explorer Bar

James.R.Eggers@gmail.com (J. Eggers) — Wed, 28 Jan 2009 12:00:00 +0000

In the last couple of posts, I have been mentioning different command line tools that I've been using (specifically Nant and SchemaSpy). While some people love command line applications and using the command prompt, there are some that cherish the nice Windows GUI and the mouse. The issue with the GUI is that applications like Nant and SchemaSpy don't really work well out of the box if you double click on them. This means that in order to have double click functionality that does something, you will need to script it out using tools like shell or PowerShell scripting that executes the configured command for the desired results. If you don't create (and subsequently debug) the script wrapper, you will find yourself opening up the command prompt, navigating to the directory, and then typing in the command manually. Thankfully, there's a tool out there that makes things easier.

The Command Prompt Explorer Bar

The Command Prompt Explorer Bar is an open source tool on the Code Project web site developed by Pavel Zolnikov that allows you to place a command prompt window inside of your windows explorer window (see image below). While this immediately doesn't sound useful, it offers some powerful productivity features that a typical command window doesn't. First, when you open the Command Prompt Explorer Bar, it immediately takes you to the directory you're in. So, if you are in the directory that contains the Nant script you want to manually execute, you can open the command prompt and immediately type in your command. In addition, as you move around windows in the explorer window, your command prompt automatically synchronizes with it. Last but not least, the Command Prompt Explorer Bar offers the ability to have macros.

Figure 1: The Command Prompt Explorer Bar for a user's (named llama in this case) home folder.

Where to Download the Command Prompt Explorer Bar

As stated previously, the Command Prompt Explorer Bar is an open source tool on the Code Project web site. You can download the installation binaries and/or the source code from this location. After you have downloaded the installer for it, you will be able to begin using it.

How Do I Open it?

The Command Prompt Explorer Bar is very simple to use since it's just an embedded command prompt at it's core. It works in both XP and Vista and there's no special window style needed to have it open (folder bar is supported as is Vista's Aero interface). There are two ways to open the Command Prompt Explorer Bar. First, you can go to the View Menu > Explorer Bar > Command Prompt menu option. The second way is to just press Ctrl+M in the folder. NOTE: Ctrl+M does not work if you are literally on your windows desktop, a file system/explorer window must be open for the Command Prompt Explorer Bar to open.

What Are Those Icons For?

As you may have noticed from the above image or if you've installed the Command Prompt Explorer Bar already, there are a series of buttons/icons on the left sidebar. These buttons help provide some quick additional functionality. Going from top to bottom, the buttons are as follows:

Command Menu - Provides the command prompt context menu, similar to right clicking inside of the prompt. Clicking directly on the button shows an "About" dialog.
Synchronize - Turns on and off the ability for the command prompt to synchronize as you move through folders above it.
CLS - Sends the CLS command to the command prompt; clearing the screen.
Get Selected Folder Items - Enters all items you have selected in the folder view above in a <space> separated list.
Enter Key - For those who don't like to press the enter key on the keyboard...I think.
Esc Key - Used to clear a partially typed line; same as the Escape Key on the keyboard.
Macro - A drop down menu containing a list commands that you can use for quick access. (more on this in the next section)
Grow - Grows the constraints of the Command Prompt Explorer Bar.
Shrink - Shrinks the constraints of the Command Prompt Explorer Bar.

Using Macros

When the Command Prompt Explorer Bar was created, Pavel Zolnikov entered in the ability to create and use macros. The macros that he supplied with it were very much .Net Command Prompt tools. Browsing the list, you'll find a number of macros for generating a type library, register a COM compliant assembly, and registering an assembly in the GAC. There are also some macros for traditional command prompt commands like attrib and cd. There is also a few macros that contain a {1} in their name. What this syntax represents is the first selected item in folder list above. For example, if you highlight a folder and then call the "cd {1}" macro, it executes the change directory command into the selected folder in the command prompt. It does not sync back to the folder view however.

Adding New Macros

While the default macros are useful, Pavel also provided us the ability to create our own macros. This is where it shines. To create a new macro, Click on the Macro menu button directly. This will open the C:\Program Files\Command Prompt Explorer Bar\macro.xml file. Inside of this file you'll see some very basic XML syntax which is described below.

<group> - The document element as well as elements that denote a grouping/submenu list of macros
- text - Sets the text to be shown as the submenu item.
<macro> - Denotes a macro-ed command.
- text - Sets the menu text to be shown for the macro
- execute - a boolean value telling the command prompt to execute the macro when selected or not.
- command - the command the macro will execute upon selected if different than the text.(optional)
{0} - types out all currently selected files in the folder view
{1} - types out the first selected file in the folder view (subsequent numbers represent that element of selected items; i.e. {2} is the 2nd selected item).
{1:n'} - types out the last selected file in the folder view (n' represents the inverse order of the files selected)

Now that we've looked at the syntax, let's add a new macro group with a Nant command. This will assume that you have Nant installed and configured with the Nant.bat file in your system path as described by the Nant installation and configuration steps.

       1: <group text="Nant">       2:     <macro text="completeBuild {1}"        3:            execute="true"        4:            command="Nant -buildfile:{1} completeBuild" />       5: </group>  

Adding this snippet to the file (inside of the parent <group> xml document element), the new macro group and command is instantly added to the macro menu (no close and reopen needed). This command allows us to select our build file in the folder view and then executes the macro to call it's "completeBuild" task that it presumably contains. Quick and easy for executing command line tools and applications without having to navigate and type in the command in a command prompt nor mess with scripted wrappers.

In Summary

While this tool isn't the most amazing one out there, it is very useful if you find yourself working in a command prompt a lot. Great tools and utilities like Nant and SchemaSpy can be easily reduced to a simple click of a macro while you never have to leave the window you are browsing the file system with.

Database Documentation using SchemaSpy

James.R.Eggers@gmail.com (J. Eggers) — Wed, 21 Jan 2009 12:56:00 +0000

One of the largest challenges that developers and software project managers have a difficult time with is how to keep documentation in sync with the code elements. While there are a large amount of tools that allows for documentation to be created based on code comments (i.e. nDoc, JavaDoc, SandCastle, etc.), there are not a lot of tools that do the same for Databases. Now, there are tools that will document a database after the fact (or will create the database based on the documentation you're creating), many are very expensive or appear to be no longer support. One tool that I have started using recently for database documentation was recommended in the book The Productive Programmer by Neal Ford. The tool is called SchemaSpy.

For the sake of brevity, this post will cover how to generate database documentation for only Microsoft SqlServer 2000 and 2005 databases using SchemaSpy.

What is SchemaSpy?

SchemaSpy is a Java command-line application that will connect to your database and generate information about the tables and their relationship. SchemaSpy does does not address any database programming elements (i.e. Functions, Triggers, Stored Procedures, etc.); however, supplies a large amount of information on the tables and views themselves along with their relationships. It also evaluates your indexes and keys in order to build relational diagrams. Furthermore, it retrieves information from the database tables in order to give you a view of the number of records, the types of columns, and a large amount of other information for it. In addition to large amount of information, it also provides notification if there are any anomalies in the schema; such as implied relationships between columns (no foreign key constraints), tables without indexes, columns marked as both "null" and "must be unique", single column tables, and more.

What does SchemaSpy Produce?

Below are a few screen shots of the documentation that SchemaSpy creates for the Northwind database.

Figure 1: SchemaSpy's Index page for the Northwind Database.

Figure 2: An example of the the diagramed relationships SchemaSpy produces

What do I need to start using SchemaSpy?

SchemaSpy requires 4 separate components to be installed in order to work properly:

The SchemaSpy application itself (version 4.1.1 as of this post) - Download
The Graphviz Diagram Engine - Download
Java Runtime Environment (v1.5 or higher) - Download
The Database Driver(s) required for a Java Application to connect to your database.
- MS SqlServer 2000 / 2005 - JTDS v1.2 - Download

Once the 4 components are downloaded and the JRE and Graphviz are installed, the next thing we have to do is install/configure the JTDS drivers (more on why JTDS instead of traditional JDBC in a moment). As of this post, SchemaSpy only works with the 1.2 version of the JTDS driver and requires the jtds-1.2.jar file to be in the C:\Program Files\Java\Shared\jtds directory. The driver jar file can be stored in other directories; however, an additional command parameter has to be passed to SchemaSpy if you do (see below). At this point, the only thing that's left is to execute SchemaSpy with the proper command line arguments.

Running SchemaSpy

After all of the above components are installed, you are able to execute SchemaSpy from a command prompt using the following (general) syntax:

java -jar SchemaSpy_4.1.1.jar -t databaseType -host serverName -port portNumber -db dbName -u userName -p password -s schemaName -o outputDirectory

Let's dissect these command parameters a bit:

-t : This identifies the type of the database we're using. Since this post is focusing on MS SqlServer using the JTDS driver, the database type will be either mssql-jtds (for SqlServer 2000) or mssql05-jtds (for SqlServer 2005).
-host : This is required by our type. It signifies what database server to connect to.
-port : This is required by our type. This signifies what port to connect to the database. This will not always be 1433 (see below).
-db : This tells SchemaSpy what database to diagram for us.
-u : This tells SchemaSpy the database user name to login with.
-p : This tells SchemaSpy the database user's password to login with.
-s : This tells SchemaSpy what schema to read and diagram. The user credentials provided must have access to at least read this schema.
-o : This tells SchemaSpy the output directory to generate the documentation.
-dp : This tells SchemaSpy the path to locate the database driver file (not used in this example)

An example of a fully completed command:

java -jar SchemaSpy_4.1.1.jar -t mssql-jtds -host myDBServer -port 1433 -db Northwind -u myDbUser -p password -s dbo -o NorthwindDocumentation

If your projects use Sql Server's integration authentication option, you can use the -sso option. The -sso option allows you to remove both the username (-u) and password (-p) parameters and use the identity of the account used to called the Java Runtime Environment. Modifying the example above to use integrated authentication, we get the following:

java -jar SchemaSpy_4.1.1.jar -t mssql-jtds -host myDBServer -port 1433 -bb Northwind -sso -s dbo -o NorthwindDocumentation

Why JTDS instead of the JDBC drivers?

As I mentioned above, I recommend using the JTDS Java Database drivers instead of the traditional JDBC drivers that Microsoft offers off of their web site. While the fact that JTDS is a single driver as opposed to the 2 separate JDBC drivers is a plus, the main reason is the trouble I had getting the JDBC drivers to work with SqlServer 2005 databases. If I was working on documenting a database I'm creating for a project on my local system using SQL Server 2005 Express Edition, it would always give me an error stating that the "variant" data type is not supported. After a lot of research and trial and error, the only result I could only get this working by using the JTDS driver.

How to use SchemaSpy to Diagram a Named Instance

When installing Microsoft Sql Server, you have the option of installing it as a stand alone service or a named instance. In the event of a named instance, you would normally connect to it in your .Net code (or via Management Studio/Query Analyzer) as ServerName\InstanceName. For example, the default instance of a Sql Server 2005 Express Edition is localhost\SQLExpress. In order to connect to a named instance database with SchemaSpy, you do not need the Instance Name of the database (just the server name as if it wasn't an instance), and you have to use the database's TCP Dynamic Port instead of the typical 1433.

You can locate a named instance's dynamic port through the following steps:

Open the Sql Server Configuration Manager application (either through the Programs Menu or a MMC Snap In)
Expand the Sql Server 2005 Network Configuration element
Click on the Protocols for "INSTANCE" where "INSTANCE" is the name of your named instance
In the right pane, double click on TCP/IP
In the TCP/IP Properties dialog that appears, click on the IP Addresses tab.
Towards the bottom of the IP Addresses tab, you'll find the TCP Dynamic Ports value to use for SchemaSpy

The above steps were taken based on a local installation of SQL Server 2005 Express Edition. Your steps may vary depending upon configuration.

Conclusion

SchemaSpy is a tool that I have been using to help with some of the documentation pains that I have experienced on some of my personal and professional projects. Being able to add the command into a build script to automatically keep the documentation up to date is a huge time saver and is invaluable as people migrate onto the project(s). I hope you find it as useful as I have, and to assist further, I have added a simple step by step guide to setting up and begin using SchemaSpy below.

The Step-By-Step

Install the Java Runtime Environment version 1.5 or higher if it is not already installed.
Install Graphviz Diagramming Engine.
Download the SchemaSpy jar file.
Download the JTDS v1.2 driver and place the jar file in the C:\Program Files\Java\Shared\JTDS directory.
Obtain User Login Credentials with the proper permissions to read the schema for the database you will be documenting if integrated authentication is not available.
Obtain the TCP Dynamic Port if the database you are accessing is a named instance.
Open a command window and navigate to the location of the SchemaSpy jar file.
Execute the SchemaSpy jar file using the proper command line parameters.

Filtering Table Results with jQuery

James.R.Eggers@gmail.com (J. Eggers) — Wed, 14 Jan 2009 14:00:00 +0000

A few weeks ago, I was looking for a way to filter records in a table. The screen that I was going to use this one was a composite/advanced search screen to where after the results are populated, the user can continue typing in search results into other fields and it would auto-filter.

There are a large amount of jQuery plug-ins that offer table filtering features. I reviewed the TableSorter plug-in; however, I didn't want a filtering system where 1 text box would filter all columns. That plug-in works extremely well and I am using it in other areas, but I'm looking for a text box-to-column filtering mechanism instead. Bill Beckelmen, who has a lot of great posts and examples on the TableSoter plug-in, suggested trying the jQuery Column Filter or tablefilter plug-ins. Both were better; however, also auto-created the text boxes in another header row for the table. After looking a bit more, I came to the conclusion that I would need to write my own functionality. If I continue working on refining this code, I will probably see about turning it into a plug-in for easier use.

Post Downloads:

Table Filtering with jQuery Example - TableFilter.zip

The filterColumn Function:

At the heart of the filtering process is a JavaScript function called filterColumn. This function takes 2 parameters. The first parameter takes the text name of the table column that is being evaluated. The second parameter is the text value to filter on. For the sake of the example, the filterColumn function will be bound to each text box's onKeyUp event; passing it's current value for the filtering text.

Inside of the filterColumn function, 3 things occur to make the filtering mechanism work. First, the two input parameters are inserted into regular expression patterns that will be used to locate the table column header as well as the records in that column once found. The second thing it does is loop through the column header (first table row in the thead tag that contains th tags...or "#resultsTable thead tr:first th" in jQuery selector syntax) to locate the column's index. Lastly, it loop through the table's records using the column's index and sets the row's display style to "none" if it doesn't match the filtering criteria.

       1: function filterColumn(columnHeader, value){       2:         3:     // set regex objects with the appropriate patterns.       4:     var headerPattern = new RegExp('^' + columnHeader + '$', 'i');       5:     var filterPattern = new RegExp('^' + value + '', 'i');       6:         7:     // initialize the column identifier       8:     var colIndex = -1;       9:           10:     // loop through the header's first row of cells      11:     var headerRow = $("#resultsTable thead tr:first th");      12:     headerRow.each(function(n){      13:         if (headerPattern.test($.trim(this.innerHTML))){      14:             colIndex = n;              15:         }          16:     });      17:           18:     // check to see if the column was found.      19:     if (colIndex > -1){      20:               21:         // loop through each table body row and evaulate the proper cell.      22:         var bodyRows = $("#resultsTable tbody tr");      23:         bodyRows.each(function(n){      24:             if(!filterPattern.test($.trim(this.childNodes[colIndex].innerHTML))){      25:                 this.style.display = 'none';      26:             }      27:             else {      28:                 this.style.display = '';      29:             }      30:         });      31:     }      32: }  

Next Steps:

I have a lot of refinement to go on this function if I am thinking about converting it into a plug-in at some point. I need to make the table's id not hard coded and bind the onKeyUp events when the search button is pressed instead of having it attached all the time in the HTML. In addition, I need to look into incorporating a zebra-striping plug-in as well to have it address the row styling after the filtering.

The code is a great starting point and it's very useful as is. With a little refinement, it can provide a very robust mechanism for simple table filtering without having to learn and use a very complex plug-in.

Goals for the New Year

James.R.Eggers@gmail.com (J. Eggers) — Wed, 07 Jan 2009 18:33:00 +0000

Happy New Year! Ok, so I'm a week late. With the start of each year comes a tradition that some view as a farse while others hold as an event of refocusing. New Year's Resolutions are are great time to reflect and focus what someone wants to achieve. While I'm not big on making my personal goals known like some other bloggers (it's just not my cup of tea), below are a few goals I've set for myself that relate (directly or indirectly) to this site.

Blog More

This is a given. 2008 was a huge year of transition for me in my personal and professional life. While I was able to churn out more post than I had before, I struggled on many weeks. I'm hoping to be able to hit my goal of once per week this year. Hopefully I'll be able to keep it up while also providing interesting topics for everyone to follow.

Learn F#

One of the things that I've wanted to do in recent months is expand my professional arsenal. Coming from a Java background and transitioned into the .Net world almost exclusively now, I've somewhat been stuck trying to stay afloat with just C# and VB.Net (ASP.Net included). While I am still working on keeping up with the times and changes in those languages, I'm wanting to expand into F# as well for a truly different perspective.

Learn Silverlight 2

I dabbled in Silverlight 1 while I was also learning XAML in general for WPF. It was fun but I didn't get too deep into it due to the JavaScript model that it used and the obvious changes Silverlight 2 (then 1.1 alpha) had. Now that Silverlight 2 is out. It's time for me to get back to work on it and begin getting a subtle level of proficiency in it.

Finish the Books I Start Reading

For those that know me, I love reading; however, I tend to read books in parallel and rarely finish technical books. This works well for some books since they are great for references just as much as reading cover-to-cover (i.e. Wrox's Professional C# 2008, etc.). The issue comes in when I'm reading a book to clarify a topic or to truly learn, I tend to lose attention on the material. This goal is to simply push myself to finish the books that I feel need to be finished. For example, here are all of the books I'm currently reading:

Website Optimization by Andrew B. King
jQuery in Action by Bear Bibeault and Yehuda Katz
The Design of Sites by Van Duyne, Landay, and Hong
Introducing Microsoft Silverlight 2 by Laurence Moroney
Alice in Wonderland by Lewis Carroll (ok not an IT book but is a great way to stimulate the imagination and unwind)

Learn DDD

One concept that is being shown a lot of attention recently is Domain Driven Design. It's got some great concepts for managing complexity and the research I've done on it so far has yielded some interesting ideas on my application designs. I just recently picked up a copy of Eric Evans' book and will hopefully get to it very soon to clarify a lot of the items I've had some trouble with.

Use jQuery More

I love jQuery. Of all of the JavaScript frameworks that I've used on both personal and professional projects, it is the one I prefer hands down. With that being said, I haven't had much spare time to work on any personal projects that require new ways of using it so such posts have been reduced a bit. I have projects queued up so hopefully there'll be more jQuery related posts to come up in the future.

In Summary

I'm sure that I'll have more goals as the year progresses; however, for right now I feel like I have a good focus for the next handful of months. Next week I'll start up the normal posts again. This year will have a more diverse mix of posts with more tools and different technologies and techniques coming. Hopefully I'll be able to stick to my goals :-).