Hector Correa's blog

Compiling CoffeeScript from Sublime Text 2

hector@hectorcorrea.com

A few days ago I configured Sublime Text 2 editor to use three different hotkeys for compiling and running CoffeeScript files. I am new to Sublime so I consider somewhat of an achievement that I was able to pull this off with help from the interwebz and a bit of luck.

Run a CoffeeScript file in the Sublime console

The first hotkey that I configured runs a CoffeeScript file and displays its output to the Sublime’s console window. This is useful when I want to quickly run a small CoffeeScript program that outputs to the console. The following screenshot shows this:

To wire up this hotkey I created a new Build System for CoffeeScript files. You can do this via the Tools + Build System + New Build System menu option. Copy the following code inside the editor and save it as CoffeeScriptRun.sublime-build in the default directory (in my Mac the path was ~/Library/Application Support/Sublime Text 2/Packages/User)

 
{
    "cmd": ["coffee", "$file"],
    "selector" : "source.coffee",
    "path" : "/usr/local/bin"
}

From now on, when I hit command+B ( +B) while editing a CoffeeScript file, Sublime automatically calls the CoffeeScript compiler with the current opened file. Command+B is the default key in Sublime Text to run the build command. You can also run this command from the menu by going to Tools + Build. The output of the CoffeeScript will be displayed in the Sublime Text console at the bottom of the screen.

Generate the JavaScript file for a CoffeeScript file

The second command that I configured was to re-generate the JavaScript file associated with a CoffeeScript file.

One easy way to do this is to create another build file like I did above but use a slightly different command to tell the CoffeeScript compiler to produce a JavaScript file. The disadvantage of this approach is that then I have to toggle between build modes via the Tools + Build System and then use command+B to produce the JavaScript file. This is just too many steps for something that I need to do quite often.

What I really wanted was a hotkey to re-generate the JavaScript file in one shot. To do this I had to create a new key binding (Sublime Text wording for hotkey) by going to the Sublime Text 2 + Preferences + Key bindings - User and entering the following text:

[
    { "keys": ["super+j"], "command": "coffeescript_to_javascript" }
]

This tells Sublime to wire the command+J key to a command called “coffeescript_to_javascript”. The next step is to create the code associated with “coffeescript_to_javascript”. To do this I went to Tools + New Plugin... menu and replaced the default code with the following:

import sublime, sublime_plugin

class CoffeescriptToJavascript(sublime_plugin.TextCommand):
  def run(self, edit):
    self.view.window().run_command('exec', {'cmd': ["coffee", "-c", self.view.file_name()]})

Two things to notice in this code. One is that the name of the class (CoffeescriptToJavascript) is the CamelCase version of the name of the command (“coffeescript_to_javascript”) that I defined in the key bindings. This is very important as Sublime will automatically look for a class with the CamelCase version for the value defined in the key bindings.

I wish I could remember where I stole the idea for this code, I know it was somewhere in the Sublime message board but I didn’t save the link.

The second thing to notice is that the run command in this code is calling the CoffeeScript compiler (coffee) and passing both the -c flag to produce the JavaScript and the name of the file to compile (which will be the name of the current file being edited.)

I saved this plugin file as CoffeeScriptCompile.py in the same location where you saved the build file that we created before. I restarted Sublime to make sure the new plugin was loaded.

From now on, when I use command+J Sublime runs “coffee -c somefile.coffee” and generates the corresponding somefile.js file. This command does not output the result to the console, though.

Outputting the JavaScript code to the Sublime Console

Lastly, using the steps defined in the previous section I created another plugin to output the JavaScript to Sublime’s console rather than to a file. This is very useful when I want to dig and see how a particular CoffeeScript feature is translated to its JavaScript equivalent.

I wired this command to the command+M key and pass the “-p” flag to the CoffeeScript compiler to get the JavaScript to the standard output.

Drawing a Binary Tree in CoffeeScript

hector@hectorcorrea.com

A few months ago I wrote a small CoffeeScript program to draw binary trees on a web page page using the HTML 5 Canvas element and was very pleased with clarity and structure of the resulting code. This blog post elaborates on the code structure and how it compares to previous implementations of the same idea that I have done in other languages.

I’ve written programs to draw binary trees before in C# and Ruby and I wasn’t sure how a CoffeeScript implementation would compare against implementations in more traditional object oriented programming languages. Thanks to CoffeeScript’s support for classes the structure of this implementation is pretty much as you would expect: a BinaryNode class to hold node data, a BinaryTree class to implement the basic binary tree operations (add, walk), and a BinaryTreeDrawer to calculate the coordinates where each of the nodes should be drawn.

In addition, since this project requires only client-side code (I don’t store the binary tree data in a server database) it’s was very pleasant not having to deal with a context switching between a “client side” mental model and a “server side” one. This is uncommon in real life projects but it was a nice bonus for a pet project like this.

You can see a running version of this project here.

The full CoffeeScript (and JavaScript) source code is available on github.

Basic Structure of the Code

The main classes that I used to implement the binary tree drawer functionality are BinaryNode, a BinaryTree, and a BinaryTreeDrawer.

The BinaryNode class is just a small “data class” that stores the information about each node, namely the value of the node and pointers to the node to the left and right of it.

The BinaryTree class implements the functionality to add nodes to the tree and walk the tree. The add method takes care of adding each new value to the tree and updating the other nodes accordingly. The walk method goes through each node of the tree in-order and calls another function to allow you to decide what to do on each node (e.g. print it to the screen.)

The code for this class is shown below. Notice that this is just a partial implementation of a binary tree since it only implements the add and walk methods but is missing functionality to remove nodes, search for values, find the min and max values in the tree, et cetera.

The BinaryTreeDrawer class walks through the nodes of the tree and calculates the coordinates where each of them should be drawn. This class does not draws the tree per-se, rather it calls a function of your choosing with the coordinates where the node should be drawn.

Console Application

File binary_console.coffee shows a very sample of usage of the BinaryTree and BinaryTreeDrawer classes. This console application creates a binary tree, adds a few nodes to it, and outputs to the console the coordinates where each of these nodes should be drawn.

If you have CoffeeScript installed on your box you can just run “coffee binary_console” and you’ll see an output similar to this:

The Demo Web Page

Once I implemented the BinaryTreeDrawer it was easy to hook this to a web page and actually draw a binary tree using the HTML 5 Canvas element. The callback passed to the drawer code in this case actually draws circles and lines for each node rather than just outputting the result to the console. Below is a snippet of the code found in binary_web.coffee that shows the basic code to draw the tree:

Final Thoughts

As I said at the beginning of this blog post, I was very pleased with how succinct the code in CoffeeScript to implement a binary tree drawer ended. The clear and concise CoffeeScript syntax aided with CoffeeScript support for a traditional way of defining classes (while behind the scenes using JavaScript prototypes) resulted in code that is both easy to write, test, and maintain.

Related posts

Drawing a Binary Tree in Ruby (2011)
Binary Tree in C#(2006)

JavaScript Async Programming for Sync Heads

hector@hectorcorrea.com

As I start learning Node.js one of the things that have puzzled me the most is having to write JavaScript code to run in an asynchronous fashion. Since Node.js uses an event-driven, non-blocking I/O model pretty much everything has to be written to run asynchronously. For somebody like me, that has always written code to be executed synchronously, the asynchronous paradigm takes a while to get used to.

This blog post compares how a particular synchronous piece of JavaScript code looks and feels against against its asynchronous counterpart. My gut feeling was that the asynchronous code would be much longer (in lines of code) and more complicated than the synchronous version but I wanted to see proof of it, well, at least as much proof as one single code example could provide.

I should clarify that I am, by no means, an expert on asynchronous programming or Node.js and the approach that I am showing in this blog post might not be optimal. I think it is a decent implementation but you should factor my newbie status on this kind of programming as you read this post.

The code that I describe in this blog is a function called getTopicByUrl which retrieves information for a blog entry. The steps to achieve this are as follows:

Read a list of topics from a text file
Find if the requested topic in the list
Read the contents of the topic (from another file on disk)
Create an object with the all topic data
Return the topic data

In a typical synchronous implementation in JavaScript the pseudo-code for this function would look something like this:

function getTopicByUrl(url) {
  topics = getAllTopics();
  topic = findTopicByUrl(url, topics);
  text = readTopicContent(topic.id);
  data = {id: topic.id, title: topic.title, content: text};
  return data;
}

To write this code in an asynchronous fashion the idea is to execute the first step but we don't wait for a return value. Instead we let the first step know what to do when it's done (i.e. what to function to execute to perform the second step.) Below is a pseudo-code implementation for this function:

function getTopicByUrl(url, callback) {
  getAllTopics(step2);
  
  function step2(topics) {
    topic = findTopicByUrl(url, topics); // this is sync operation
    step3(topic);
  }

  function step3(topic) {
    readTopicContent(topic.id, step4);
  }

  function step4(topic, text) {
    data = {id: topic.id, title: topic.title, content = text};
    callback(data);
  }
}

By looking at these two examples we can easily see that the asynchronous code is larger than the synchronous version and, at least for the async newbie, much more complicated.

Let's address these two issues separately.

On the number of lines of code

What I found in a full blown implementation of this code (one with variables declaration, error handling and other areas of the code fully fleshed out) is that the asynchronous code is not much larger than the synchronous version. The following picture shows a complete implementation of this code. On the left side we have the synchronous version and on the right side we have the asynchronous one.

The asynchronous code is barely four lines of code larger than the synchronous version (or about 10% longer.) A 10% code increase in lines of code is nothing to be proud of, but the asynchronous code was not 300% larger as the two pseudo code samples at the top of this blog post might have lead you to believe. This is something that I've noticed more than once. The asynchronous implementation tends to look larger than the sync version initially but once all the functionality has been implemented (error handling, variable declaration, conditional branching) the difference in the number lines of code is not that much.

Although I have yet to find an example in which the asynchronous code is shorter than the synchronous counterpart I've started to be less concerned about this since in the end the difference in the number of lines of code is not significant.

On the complexity of the code

The second and more important part of the story has to do with code complexity. In the previous example the synchronous version of the code is definitively much simpler to read than the asynchronous counterpart. Regardless of how much experience you might have with sync or async programming I think is pretty clear that the synchronous version is easier to read since its sequential and there is no need to introduce the concept of callbacks.

Having said that, after writing asynchronous code for a little while the code does not look as daunting as it does the first time you read or write asynchronous code. The asynchronous code is not as simple as the synchronous version but it's not rocket science either.

The code shown in the picture has numbers indicating where each step is implemented on both versions.

Except for the very first step you can see on the picture how each of the steps on the synchronous version (step 2-5) lined up with its asynchronous counterpart. The very first step is defined at the bottom in the asynchronous version since it depends on the getTopicDetailsCallback being already defined. This looks and feel odd and I wish that wasn't the case but I could not find a way around it.

A few observations on the code

The asynchronous version relies on callbacks. This is something that you don't have to deal with in synchronous programming but that you must work with when doing asynchronous programming.

One of the callbacks that I defined (getTopicDetailsCallback) is a function inside another function. This is normal and common in JavaScript, but if you have never done this it might look strange initially. By having getTopicDetailsCallback defined inside getTopicByUrl it has access to variables defined in getTopicUrl which made sense since I wanted to re-use some values (like the dataPath) across multiple steps.

In the synchronous version I have a single exit point (line 78) regardless of whether the blog topic was found or not or whether there was an error reading the text of the blog topic. Since the code in this version will run sequentially I can setup stuff to be picked up in the following lines with no additional effort. In this case I initialized the data object with the appropriate information depending on whether there were any errors or not and then just return it to the called in line 78.

In the asynchronous version however I have two exit points (line 61 and line 77.) Since the code does not execute sequentially it was easier for me to bail out immediately in line 61 if a topic was not found and have another exit point in line 77 for the case where the topic was found and the call to fs.readFile was executed.

The thought of having more than one exit point for this case made me uncomfortable initially but I wonder if this is just because I am bringing "best practices" from the synchronous world into the asynchronous world.

The asynchronous version shows two ways to handle going from one step to another. To go from step 1 to step 2 we pass a function name to step 1 so that it knows what to call when it's done. However, in step 3 we define an anonymous function (line 65) that it will be executed when the process of reading the file has completed.

Most of the techniques used in the asynchronous code are regular JavaScript code that most seasoned developers use on a daily basis but that would be new for casual JavaScript developers. Having to do asynchronous programming in JavaScript will definitively push you to learn more about the language - and that's a good thing.

Source code

The entire code for the project that I am using as my sandbox can be found on github at: https://github.com/hectorcorrea/simple-blog. File /models/blogsync.js has the synchronous implementation and models/blog.js has the asynchronous implementation. The synchronous version is only for demonstration purposes and will not be maintained as the project evolves.

Where to find more

Garann Means has a really nice (and short) book on Node.js targeted to front end developers that it's worth to take a look at if you are new to Node.js.

If you are interested in learning more about asynchronous programming in JavaScript I highly recommend Trevor Burnham's Async JavaScript: Recipes for Event-Driven Code. The first chapter alone on "The JavaScript Event Model" is worth the price of the book.

Conclusions

As I make my way through the world of asynchronous programming in JavaScript and Node.js I am finding that the asynchronous code is usually slightly longer than it would be in a synchronous environment and a bit harder to understand.

However, as I become more used to this new style the initial shock seems to fade away and the code starts to make sense. I think there are a lot of best practices that I have yet to figure out in the asynchronous world (just like I did over the years on synchronous programming) but that's just part of the learning process. The good thing is that there is a lot of information (books, web sites) that are available to help in this process as well as new tools and libraries are being developed to simplify the programming of asynchronous applications.

You might be wondering why bother? Why go through all this effort just to use an event-driven tool like Node.js. The truth is that I believe that asynchronous programming is here to stay and will become the main way of developing applications. You can see this today if you are developing applications with JavaScript, jQuery and using AJAX. If you are using Microsoft Silverlight you also have noticed that pretty much everything is done asynchronously. The same is true if you are looking into WebSockets (aka the next AJAX) since those calls are also asynchronous.

In my case since Node.js pretty much forces you to work in an asynchronous environment it helps me to make sure I stay on track as I jump head first into this brave new world of asynchronous programming.

Web Development on the Mac OS X (part II)

hector@hectorcorrea.com

Last year I blogged about my first impressions using a Mac for web development from the point of view of somebody with a strong Windows/C# background. This blog post is a long overdue continuation to that post.

I bought a Mac Air a year and a half ago with the intention of experiencing what is like to develop web applications in a non-Windows environment and it has been a great learning experience. I am amazed how much I've learned in those 18 months and the breadth of technologies and tools that I've played with in that time.

Hardware and built-in software

The Mac Air is undoubly a fantastic piece of hardware, it's slim, lightweight, powerful, and overall a beautiful piece of hardware. It's not cheap, but it's worth every penny.

Learning OS X for somebody that has always lived in a Microsoft Windows world was not hard. I really thought I was going to struggle more with OS X but I didn't. The transition to OS X was pretty easy for me. Having said that, I don't find OS X incredibly better than Windows 7. There are a few differences here and there but nothing earth shattering.

Some of the software that comes bundled with OS X is certainly better than its Windows counterpart. For example iPhoto is a much better photo management tool than the standard photo software that has always come with Windows. iTunes (not surprisingly) seems to work better on the Mac than on Windows. The operating system updates also seem to work smoother than Windows updates and be less frequent and pesky. Support for viewing and creating PDFs is native (it even supports annotating PDFs.) Other tools like Finder are no better (or worse) than Windows Explorer to browse the file system. I did find surprising the lack a a built-in Paintbrush application, though.

Software Development

As far as software development for web applications is concerned I have enjoyed learning what's available on the Mac much more than I had anticipated. Since I got my Mac I've gotten my hands on Ruby, Sinatra, Ruby on Rails, CoffeeScript, Node.js, Apache, MySQL, Heroku, git, github, xCode, TextMate, Sublime, and a bunch of other tools that I had never used before.

As I blogged a year ago, the language that I decided to learn first was Ruby and the popular web development framework Ruby on Rails. That was certainly a great and fun experience.

I found the Ruby language easy to learn (although I don't call myself an expert on it by any means) and there are a lot good books and web sites with information on how to get started.

I also enjoyed learning Ruby on Rails and gained appreciation for an "MVC framework" that truly provides support for models (the M in MVC.) In my web development experience with ASP.NET MVC I knew that there were other frameworks that did a more complete job but I didn't quite get it until I understood how things like ActiveRecord and database migrations work in Rails.

Rails was not as easy to understand as I had heard before. Rails is actually quite complex to get started since it does so much and there are so many pieces that work together. In addition, people seem to use many different options and configurations when working on Rails that baffled me for a while. For example, it is possible (and common) for people to use different web servers to host Rails applications and having to learn the pros and cons for Apache, Passenger, Mongrel, WEBrick, and when they are a good fit was a bit overwhelming for somebody new to the OS X environment. This is in contrast to ASP.NET MVC web development in which the web server is almost always IIS. The same goes for view rendering engines, in Rails people use Embedded Ruby (erb), HAML, Mustache, and a bunch of other tools that can make the process of learning Rails confusing once you get pass the "hello world" web page. I don't mean to indicate that these options are a knock against Rails, but they can make it hard for the beginner to understand what's the best way for him/her to take.

I also enjoyed a lot learning Sinatra. Sinatra is more similar to ASP.NET MVC than Rails because it only provides support for views and controllers (the V and C in MVC.) I found Sinatra's learning curve much sorter than Rails but be aware that you get much less out of the box with Sinatra than you do with Rails. The binary tree drawer that I put together in a previous blog post uses Sinatra.

I would recommend learning Sinatra before learning Rails because the learning curve is much shorter and less complicated. However, if you are planning on doing a traditional web applications with a relational database Rails is definitively a great fit. All the "extra stuff" that Rails provides are things that you will need once your application grows and it's deployed to production. For those kinds of applications (which are very common) Rails is definitively a more mature and complete product.

During the process of learning Ruby and Rails I started hearing about CoffeeScript and got interested on it.

CoffeeScript is a language that compiles to JavaScript and provides a very pleasant way of writing good JavaScript applications. I rewrote in CoffeeScript the program that I had written in Ruby a few months prior to draw binary trees and I was amazed how closely the CoffeeScript code is to the Ruby code. You can check out the running version here and the source code in github.

Learning CoffeeScript itself lead me to read and become more aware of Node.js and decided to take the plunge and start learning that too. There is a lot of stuff going on with Node.js in the community and new libraries and utilities are coming out at a fast pace.

I am not sure if CoffeeScript and Node.js are going to become the next Ruby on Rails as some people are suggesting, but I do agree with Giles Bowkett when he says "you should learn Node.js, CoffeeScript, and Backbone because the work going on in those communities changes paradigms."

In my blog post last year I mentioned that the lack of integrated development environments (IDE) for Ruby dumbfounded me. As I asked around it turns out that there are several good IDEs but most people are comfortable using text editors with some advanced features.

The same is true for CoffeeScript and Node.js. There is no "standard IDE" that everybody uses. Quite the opposite, people use a variety of text editors with some development features like auto-completion, color syntax, and ways to kick of the application but nothing remotely compared to the features provided by Visual Studio. This seems to be the norm in the Mac (and Linux) world. This diversity of tools is very different from the Windows world where pretty much all the web development happens in .NET languages and using Visual Studio.

For my part I ended settling on Sublime Text as my editor of choice and bought a license for it.

Conclusions

As I said at the beginning of this post, I am really happy with the large amount of technologies and tools for web development that I've got to experiment and learn on the Mac in the last 18 months. Although I am pretty sure that I could have learned and played with most of these technologies on a Windows machine, I am glad I decided to do it on a non-Windows machine to be able to compare and contrast first hand.

Macs are very popular among web developers that are not using the Microsoft stack and I enjoyed being able to experience how things are done in this environment.

In the end I think that both Macs and Windows environments are very powerful and fun to work with, each of them with their own strengths and weaknesses.

A few days ago I noticed that I had not turned on my Windows machine at home in over 4 months, that's probably a telling sign of how much I am enjoying the Mac as a whole.

Functional Asynchronicity Explained

hector@hectorcorrea.com

Over at the Pragmatic Bookshelf Trevor Burnham has a great post titled A CoffeeScript Intervention in which he shows several code snippets that demonstrate how CoffeeScript code is typically smaller and cleaner than the equivalent JavaScript code.

In particular I love the example under the Functional Asynchronicity section. In this section he shows a piece of code that trips most new JavaScript developers.

for (var i = 1; i <= 3; i++) {
  setTimeout(function() { console.log(i); }, 0);
}

When this code is executed it will output 4, 4, 4 to the console rather than 1, 2, 3 as most people guess the first time they work with JavaScript.

Although in his original blog post Trevor explains the reason for this behavior, in this blog I would like to expand a little bit more on the reasons why we get 4, 4, 4 rather than 1, 2, 3 since I believe they are fundamental to both understand and appreciate CoffeeScript.

There are two reasons why we get 4, 4, 4 in the previous code. The first reason is related to the way JavaScript’s setTimeout function works. The second reason has to do with the way JavaScript picks the value of i when the console.log(i) line is finally executed.

Understanding setTimeout in JavaScript

In JavaScript the setTimeout() function is used to execute a piece of code sometime in the future. In his book CoffeeScript: Accelerated JavaScript Development (p. 108) Trevor sums it up like this:

setTimeout always adds its target to the “event queue,” which isn’t invoked until after all the other code has run.

In our particular example this means that the function() { console.log(i); } (aka the target) will be queued up and executed when the for loop has ended -- not on each iteration.

This is true regardless of the timeout value used. Since we used a timeout value of 0 ms the target functions would be executed immediately after the loop has ended. If we had used a value of 1000 ms they would be executed 1000 ms after the loop ended. The important thing to recognize is that the targets won’t be executed until after the loop has ended.

This thread in stackoverflow.com gives a great overview on deferred execution in JavaScript and it’s worth a look if you are not very familiar with this concept.

The value of "i"

Now that we know that the calls to console.log won’t happen until after the loop has ended, we need to understand how does JavaScript determine what value of "i" would be used at that time.

We know that at the end of the loop the value of "i" would be 4. That’s easy enough to understand. However, what most JavaScript newcomers fail to grasp is that when we create an inner function that references an existing variable, the inner function will actually have access to the original variable when is eventually executed. Let’s spell this out.

The first thing to notice is that we are not passing the value of “i” to the setTimeout function. Instead we are creating and passing a function to setTimeout.

Secondly, the function that we are creating and passing to setTimeout references a variable “i” which exists in the context in which we are creating it. This is a key concept called closure and not found in most programming languages.

In his book JavaScript: The Good Parts (p. 38) Douglas Crockford defines closure as

the ability of a function to access the context in which it was created

In our case the new function (which just outputs the value of “i” to the console) was created in a context that has an “i” variable defined and therefore the new function will have access to the value of “i” whenever it is executed. Notice that this is very different than assuming that the new function will have the value of “i” at the time the new function was created!

We know from our review of setTimeout that the target (i.e. the new function that outputs "i" to the console) won’t be executed until after the for loop ends and that time the value of “i” will be 4. This explains why we got 4, 4, 4.

On his book Crockford also has a great example showing how a variable lingers after the original function has completed if the variable is referenced by any of the inner functions. In fact the whole context in which the inner function was created is what lingers. Definitively worth reading if you are not familiar with closures in JavaScript.

Yeah, but that’s still not what I wanted...

So now that you understand why JavaScript outputs 4, 4, 4 in our example rather than 1, 2, 3 you are probably wondering but how can I change this behavior.

Basically what you need to do is create a new context for the function() { console.log(i); } and make sure that in this new context the value of “i” is locked to the value of the iteration (1, 2, 3.)

Here is how Trevor does this on his blog post:

for (var i = 1; i <= 3; i++) {
  (function(i) {
    setTimeout(function() { console.log(i); }, 0);
  })(i);   // execute it!
}

In this example a new anonymous function with an “i” parameter is created and executed immediately on each iteration of the loop. Notice the “(i);” in line number 4 means we are executing it.

When this new anonymous function is executed on each iteration it calls setTimeout and creates a new function to output the value of “i” to the console - just as we did before. However, in this case the context in which the console.log will be executed will see the “i” defined as a parameter to this anonymous function -- not the “i” in the for loop!

In other words, the context in which the function() { console.log(i) } is created is now function(i) { … } rather than the original for loop. Therefore when console.log(i) runs it will pick the value of “i” from this new context and this new context receives a single value on each iteration.

Below is a slightly modified version of Trevor's code. In this new version I’ve renamed the variable “i” in the for loop to “x” to make it easier to see what we are trying to do:

for (var x = 1; x <= 3; x++) {
  (function(i) {
    setTimeout(function() { console.log(i); }, 0);
  })(x);
}

From this example it should be clearer that the we are creating a new variable “i” in the declaration of the anonymous function. The variable of the for loop is irrelevant because setTimeout and its target function will have a brand new context on each iteration and in this new context “i” will have the expected value (1, 2, 3) on each iteration.

The CoffeeScript Way

Trevor makes a great point on his post when he shows how much simpler the code to achieve the same result is in CoffeeScript:

for i in [1..3]
  do (i) ->
    setTimeout (-> console.log i), 0

In this case “do(i) ->” is doing the dirty work of creating a new function and executing it. Gotta love the simplicity of CoffeeScript syntax!

However, I doubt many CoffeeScript newcomers will really understand how this work unless they understand how JavaScript works behind the scenes. I hope this blog post helps to clarify what it’s really going on behind the scenes.

Sliding-in a Full Page Panel with CSS and jQuery

hector@hectorcorrea.com

In this blog post I show how to use CSS and jQuery to slide-in a panel full page on top of another and then slide back in the original content.

You can see a running version of the code presented in this blog post here. This demo page starts with a panel showing some part of the Lorem Ipsum text. When you click on the text a new panel displaying the Bacon Ipsum text slides-in on top of the previous text. Clicking on the new text slides the Lorem Ipsum back again. Nothing fancy but enough to show the slide-in functionality in a full page panel.

The HTML

The HTML used in this example is basically composed of two HTML divs. The first div (called mainDiv) includes the Lorem Ipsum text while the second div (called galleryDiv) includes the Bacon Ipsum.

<div id="mainDiv">Lorem Impsum text goes here</div>

<div id="galleryDiv">Bacon ipsum text goes here</div>

The CSS

The CSS to style these two divs is shown below. As you can see they are just plain divs using fixed positioning. The main difference between both styles is the background color (the mainDiv is beige and the galleryDiv is a tone of red.) Notice that initially the mainDiv is set to use all the real state (height and width are set to 100%).

The jQuery Code

The basics of the jQuery code to slide-in the one panel on top of another were taken from the excellent
Sliding Panel Photo Wall Gallery with jQuery example by Mary Lou at Codrops. I’ve simplified the code to make it easier to see what it takes to slide-in content on top of another back and forth.

When the page first loads we force the mainDiv to be visible and the galleryDiv invisible.

The next thing we do is to wire the mainDiv so that when the user clicks on it we slide-in the galleryDiv. The code for this is very simple too:

This code does the following:

Pushes the mainDiv to the back by setting its z-index to zero.
Pops the galleryDiv to the front by setting its z-index to something greater than zero, in our case to 100.
Sets the height of the galleryDiv to 100% using an animation (so that the slide-in effect is noticeable.)
Finally we collapse the mainDiv by setting its height to zero. This allows us to animate it when we slide-in back again.

The code to wire the galleryDiv to do the reverse is also very simple:

And that’s it

With just a few of lines of code we can easily slide-in content in our web pages. You can see this code running in this link. The page is self contained. View source and grab the code if you like it.

Update: A previous version of this code used the wrong kind of positioning on the mainDiv and galleryDiv styles and that resulted in weird behaviors when sliding content back and forth if the text was longer than what fits on the page (e.g. if the browser window was resized to be smaller than the text.) The current version that you are reading addresses this issue.

Future Directions for C# and Visual Basic

hector@hectorcorrea.com

These are my notes from Anders Hejlsberg session on Future Directions for C# and Visual Basic.

It's been more than 10 years since managed code was introduced back in PDC 2000. Every new release of C# has had one or two major themes:

Version 1: Managed Code was introduced
Version 2 2: Generics
Version 3: LINQ
Version 4: Dynamic features and language parity

The two major themes in the next release (version 5) are support for WinRT in Windows 8 and asynchronous programming.

Asynchronous Programming is becoming the norm in modern, connected applications.

About 10-15% of the WinRT API supports asynchronous operations. Sounds like a small percent but this percent is probably the most used stuff.

There are different asynchronous programming models in Windows 8 and .NET

WinRT uses AsyncOperation<T>
.NET uses Task<T>
JavaScript uses Promises

All of these models are objects representing "ongoing operations".

Callbacks turn your code inside out and are a big challenge but they have figured out how to automatically handle this so that asynchronous code "feel just like good old synchronous code".

The general pattern is:

public async SomeMethodAsync()
{
  // code before firing async operation
  await code to execute async
  // code to execute after async operation has completed
}

Use await operator to cooperatively yield control.

You can use Task.WhenAll() to run multiple tasks asynchronous in parallel!

It turns out that asynchronous method don't start new threads! I need to do more reading on this. Threading is only need when you run CPU intensive operations like a loop to perform math calculations.

Anders showed some JavaScript in which he consumed some C# code from within C# but he couldn't just use Task as-is and he had to bridge the two through IAsynchOperation (didn't quite get why)

Anders is obviously a big proponent of static languages and took several jabs at the dynamic nature of JavaScript. The funniest line was when he said:

“Sometimes you get lucky in JavaScript, you always get lucky in C#”

Another feature that is coming in C# is what is called Caller Info Attributes. These are built-in attributes that you can add to parameters to a method to receive the name of the caller file path, the caller line number and member name.

[CallerFilePath]
[CallerLineNumber]
[Caller/MemberName]

This is very nice for logging code.

Roslyn Project

Towards the end of the session Anders talked about what’s coming in C# 6.0 and a project named “The Roslyn project.” The basics of this project are:

Reimagine what compilers do
Compiler as a service
Open the compiler, make its information available to others (formatters, highlighters, refactoring, DSL)

With feature the syntax tree produced by the C# compiler becomes available to other tools so that they can peek inside the code with a true semantic knowledge of what the code does.

Anders shows a C# interactive prompt in which he was able to create a C# program on the fly, line by line, and execute each line as it went. This is nothing new to people that use JavaScript or Ruby but for a static language like C# this is huge! I was really impress on how clean the syntax to do this was.

The following picture (blurry) shows an example of what he did. In this code he declares a square function in a string and executes is in the program via a delegate. Wicked stuff.

Another example of using Roslyn that we showed was a feature to paste C# code as VB.NET code and vice-versa. This showed how the semantic of the original code was preserved.

There is a CTP available of Roslyn already.

The C# and VB.NET compilers are being rewritten in managed code.

8 Traits of Great Metro Style Apps

hector@hectorcorrea.com

These are my notes from Jensen Harris session at Build Windows conference on Tuesday September 13th, 2011. Jensen is the Director of Program Management for the Microsoft Windows User Experience Team.

Metro style design

There are no apps yet. Opportunity.
Sidebars are for the system, top and bottom are for the app
Win32 vs Metro apps
Content before Chrome
Apps re-imagined

Fast and Fluid

Touch faster than GUI, GUI faster than command line
Touch first approach
Beautiful, fast, and fluid vs crap
Most controls support animation
Touch has its own language, don’t mimic the mouse
Semantic zoom: not only does the content change, but also the meaning (e.g. show categories rather than details)
Don’t separate touch and mouse interfaces
A screen without touch is a broken screen - in a few years

Snap and Scale beautifully (form factor)

minimal: 1024x768
widescreen: 1366x768+
snap view (required)
portrait (optional)

Use the right contracts

Binds metro style apps together
Share, Search, and Picker
Share content with other apps (e-mail, social network)
Search: each app implements its own view
Picker: select a picture from file system or the web (sample: change user profile picture from a picture in Facebook without having to do the intermediate step of saving picture in my hard drive first)

Invest in a great Tile

Icons are yesterday’s way of representing apps
The Tile should be an extension of the app
You can also create tiles from content of the app (e.g. tile your favorite photos album or weather from SCE)

Feel connected

“Windows is alive with activity”
Live tiles
We don’t have folders, we have groups
Folders put stuff down one level, it’s hard to name them
People do love to make groups
Notifications are displayed for a small period of time, after that update the tile

Roam to the cloud

Every app gets per-user cloud storage for settings, state, and small amount of user content (via Live ID)
Applications preserve state as they go, do not require explicit save
Roam settings (favorite cities, last level on game played)

Metro style design principles

Other sessions to check out: 207, 211, 391, 395, 396, 405, 406

Simple Branching Strategies for Team Foundation Server

hector@hectorcorrea.com

In this blog post I describe some branching strategies that I've found useful when using traditional source control systems like Team Foundation Server (TFS).

Team Foundation Server has very good branching and merging capabilities that can help teams manage source code in a team environment. However, despite the fact that TFS provides excellent branching and merging capabilities, I usually recommend a simple approach to branching since the process of branching and merging can easily get convoluted and out of control even with the best of tools.

The basic idea

The way I approach branching in TFS is to have one main branch for all on-going development on the project. This is the branch where developers do their work on a daily basis. Once the team reaches the point where the code will be released to production then a second branch (a release branch) is created. This release branch is a copy of the code that will be pushed to production. The main branch stays active and most of the team will continue working of the main branch as they add new features to be released in a future version of the system. A few of the developers will make changes to the release branch to fix bugs that must be ironed out before the release goes out or even after it has been promoted to production.

The following picture shows an example of these concepts and how the branches are used over time.

In the previous picture we can see how the team worked on the main branch from week 1 through week 4 (the green bars.) At the end of week 4 the team decided this code had all the features needed for a version 1.0 release and created a release branch with these features.

During the next two weeks the team made a few changes to the code in the release branch, perhaps fixed a few show-stopper bugs before promoting to production. Meanwhile, the rest of the team continued working on the main branch on long term features to be released in version 2.0 (the blue bars.)

The team worked from week 5 through week 12 on the main branch (the blue bars.) During this time there were two hot fixes pushed to production from the release branch (green bars on week 9 and week 11.) The nice thing about the release branch concept is that it gives us total isolation from the on-going long term development on the main branch. Since the release branch is an exact copy of what's in production updating this branch when a hot-fix is needed is very safe and easy.

On week 12, the team decided that they were ready to promote the next major version of the system to production. A that time the main branch was branched out again to the release branch. From this point the cycle is repeated: hot fixes for version 2 are made on the release branch while all new development for version 3.0 is done on the main development branch (now the purple bars.)

Variations to this approach

In some teams we have made a small variation to this pattern in which a totally new branch is created every time a new major version of the system is deployed to production. For example, we could create a totally new branch on week 12 for release 2 rather than sticking the version 2.0 code on the existing release branch. The advantage of this variation is that, if at the end of week 12 we need to do a hot fix to version 1 (while version 2 is still in QA) we can easily make the fix on the original release 1 branch since that branch is still an exact copy of what's in production. This could be a life saver and worth consideration.

I first read about the concepts of the main branch and a release branch in the book Software Configuration Management Patterns: Effective Teamwork, Practical Integration by Stephen P. Berczuk and Brad Appleton. This book is a good read if you want more details on the approach that I describe in this post. Chapter 4 and 5 describe the Mainline and Active Development Line patterns while chapter 17 talks about the Release Line.

Merging - the dark side of branching

Regardless of how neat your branching strategy might be the real issues are found in the merging process. For example, in the previous picture I didn't indicate how changes made on weeks 5 and 6 on the release branch will be merged back to the main branch. The same is true on weeks 9 and 11 when hot fixes were applied to the release branch. If these changes are also needed in the main branch somebody will need to go perform this process.

There is nothing inherent to the branching strategy that I describe in this blog post that makes the merge process easier. Depending on the nature of the changes merging can be a messy process. The best that you can do with merging is to implement a workflow in which merging conflicting changes is minimized. However, since inevitably you will have conflicting changes to merge I find Jennifer Donnelly's advise on the subject rather valuable: merging should happen frequently and within context, in other words, by the person who made the change and as soon as possible. On small teams this is usually easy to do and prevents the merging process from becoming a disaster.

In closing...

The ideas that I've described in this blog are neither new nor the only options for branching. However, I've found them very useful for small teams working with centralized source control systems like Team Foundation Server. The fact that these ideas are simple to implement allow development teams to gain the most of branching (i.e. code isolation) without having to have a dedicated staff in charge of source control and merging.

Although I suspect these ideas apply to most traditional client-server source control systems I have only apply them with TFS. I am not sure however how well they fit with a distributed source control system like Git.

Running Visual Studio inside Mac OS X

hector@hectorcorrea.com

Lookie here...I am running Microsoft Visual Web Developer Express on my MacAir!

When I bought my MacAir last year to start learning Ruby and Ruby on Rails I wasn't sure how much I was going to really use it. Overtime, I've gotten familiar with it and little by little it has become my main machine at home to do pretty much everything including Ruby programming, managing my music, pictures, and documents, and web browsing. It's gotten to the point that the only reason I power up my Windows laptop at home is when I need to update one of my personal ASP.NET projects or research some C# topic in particular.

My Windows laptop at home is a nice small Lenovo 3000 v200 that is about 2 or 3 years old. It's a very nice laptop but compared with the MacAir it feels like a clunker. There isn't anything wrong it per-se, it just feels like an old machine.

As time went on I started pondering how to run Visual Studio inside my Mac so that I don' have to rely on two machines. There are many options to host virtual machines inside OS X including the built-in Boot Camp software that comes with the Mac, VMWare, and VirtualBox.

In my case I decided to use VirtualBox to create a Windows 7 virtual machine where I can install Microsoft Visual Web Developer Express 2010.

The following screenshot shows Visual Web Developer Express running inside a Windows 7 virtual machine hosted on OS X.

I decided to use VirtualBox over the other virtualization options for two main reasons. One is that it's free (that's hard to beat) but also because is multi-platform which allowed me to create the virtual machine on my Windows machine (where I have plenty of disk space) and once it was ready just transfer VM to my Mac and power it up there.

Although I configured the virtual machine that emulate a hard drive with 20 GB the VM itself only uses 12.5 GB with Windows 7 and Visual Web Developer Express installed. That was perfect for me as my MacAir has a very small solid state drive to begin with. Being able to create the virtual machine on a different machine with plenty of disk space was a big plus for me.

Now I am able to work on all my home projects on the same machine regardless of whether I want to program in Ruby or in ASP.NET. I am not quite ready to sell my Windows laptop yet, though. We'll see if that changes in the future.

Finder Error -36 on a NAS

hector@hectorcorrea.com

Ever since I got my MacAir late last year I've been having issues copying files from my Mac to my external storage device. My external storage device is a Buffalo HD-CELU2 DriveStation that can be accessed as a standard USB drive or as network attached storage (NAS.) I've been using this device as a NAS from my Windows computers for a couple of years with no problems but my Mac refused to work smoothly with it.

On my Mac I was able to read files from the NAS but I couldn't write files to it. When I tried to copy files to the NAS I got the dreaded and cryptic "Finder Error -36". There are several people reporting similar issues when using a NAS from their Macs and it took me several months before I found a solution that worked for me. Below is an example of the error that Finder will throw when I tried to copy a file (SamplePDF.pdf) to my NAS (hd-celu2-fce7).

The clues

One of the things that I noticed in dealing with this issue was that the Buffalo DriveStation worked smoothly when I connected it directly to my Mac's USB port (rather than accessing it through the network.) When I used the device through the USB port I was able to read and write files with no problems. This lead me to believe that the problem was somehow related to the way the Mac was accessing the device through the network rather than the drive itself. If the drive was formatted in a way that was incompatible with the Mac I would have the same problems no matter how I accessed it.

Here is a description of the issues that I noticed when I connected to the Buffalo DriveStation as a network drive:

My Mac computer can read files from the NAS with no problem.
My Mac cannot write files from the NAS. I get the infamous "Finder Error -36".
My Windows computer can read from the NAS and write files to it with no problem

Although I was able to read and write with no problems if I connected to the device through my USB port, using it this way defeated the purpose of having a NAS in the first place. I wanted to be able to connect to the device through my wireless network at home rather than having to drag the darn thing with me around the house.

I also noticed that the problem was not exclusive to Finder, I would also get an error when I tried to copy files to my NAS through the Terminal as shown in the following screenshot. In Terminal I was able to work around this problem by using -X switch on the copy command as shown in the following screenshot. Notice how the first copy failed but the second succeeded. The -X switch prevents the copy command from copying Extended Attributes or resource forks.

The solution

I found the solution to my problem at macwindows.com under TIP: Another take on smb.config fix for -36 file sharing error: edit nsmb.conf for "streams=no". You basically need to edit a configuration file to tell the Mac not to use streams when connecting to the NAS. In particular you need to add the following two lines to the /etc/nsmb.conf file:

[default]
[streams=no]

If the file does not exist on your machine (it didn't on mine) you will need to create it. Editing/creating this file is bit tricky since you need elevated permissions. The macwindows.com link has detailed steps on how to do this.

The aforementioned lines will tell the Mac OS X to turn off the use of NTFS streams when connecting via SMB to a device.

As it turns out, when the Mac connects to the Buffalo DriveStation through the network it uses a protocol called SMB. That's the reason you see "SMB://somename/somepath" in Finder when you connect to an external storage device. One of the things that I learned on the macwindows.com page is that SMB is a protocol to access drive and other devices (just like FTP or HTTP) and file nsmb.conf file configures the SMB client that comes with the Mac (see SMB vs. Samba, and what Apple uses for file sharing.)

An even simpler solution

I found an even simpler way to turn off streams in this article on Apple's web site: Mac OS X v10.5, v10.6: About named streams on SMB-mounted NAS, Mac OS X, and Windows servers; "-36" or "-50" alerts may appear. It turns out you can just create file on the device where you want to turn streams off by using a command like this:

touch /Volumes/SharedNAS/.com.apple.smb.streams.off

where SharedNAS is the name of your shared device. The Apple article also includes a great explanation of what the heck it means to "turn off streams" when connecting to an external device through SMB.

Are there any side effects?

I've been using my NAS device from my Mac with no problems for almost two weeks since I turned off streams with the solutions in this blog post. I can now read from and write to the NAS from both my Mac and Windows computers. The only side effect that I've seen so far is that, since streams are turned off, the Mac OS X creates two files on the NAS for every file that I copy to it. These double files are only visible when I access the NAS device from my Windows computer.

The aforementioned article on the Apple web site indicates that "named streams are used to store Mac OS X extended attributes and can be leveraged to avoid using AppleDouble files to store the data fork and the resource fork of legacy Mac files" so it makes sense that (1) the files are now being created since we turned off streams and (2) these double files are visible from a non OS X operating system.

The following screenshots show how the folder with the SamplePDF file is displayed in Finder and in Windows Explorer

This double file nuance is not a big deal for me specially given that for many months I couldn't even write files to my NAS from my Mac.

Drawing a Binary Tree in Ruby

hector@hectorcorrea.com

When I started learning Ruby last year I decided to implement a binary tree and some of its basic operations (insert, delete, walk, and search) just to get my feet wet on the language. Binary trees are a good exercise because you need to use several features of the language like conditional statements, loops, and classes while at the same time you are solving an interesting problem. The algorithm for binary trees is well documented in most introductory books on algorithms and on the web.

To make things a bit more challenging I also implemented a program to draw the binary tree so that people can see how a binary tree looks like when it has more than just a handful of nodes. In this particular case I wrote small a web application in Ruby that draws the binary tree using the new canvas element supported in HTML 5. You can see a live demo of the Ruby web site which allows you to draw binary trees with our own set of values or you can just let it draw binary trees with random values. For demo purposes trees on the demo site are limited to 250 nodes but you can draw much larger trees if you download the code and host it on your own.

This blog post gives short overview of the main methods to implement the binary tree, the algorithm to draw the binary tree, and the web site to demo the drawing algorithm.

Binary Tree or Binary Search Tree

Properly speaking the code in this blog post implements a Binary Search Tree (BST) which is a more specific version of a Binary Tree. Cormen et al. give a good explanation of this on their book Introduction to Algorithms. Binary trees are trees in which each node has at most two children, one of them called "left subtree" and the other called "right subtree" but there is no order enforced per-se (pp. 1178). Binary Search Trees on the other hand are always stored in a way as to satisfy the binary-search-tree property (pp. 287):

Let x be a node in a binary search tree. If y is a node in the left subtree of x, then y.key ≤ x.key. If y is a node in the right subtree of x, then y.key ≥ x.key.

I took a bit of a liberty on this blog post (and its related source code) and use both terms interchangeably. Please be aware that all of the references to binary search in this blog post are in fact references to binary search trees.

Implementing a Binary Search Tree

The procedures to implement the basic binary search tree operations like insert, delete, search, and walk are well documented in most introductory books on data structures and algorithms and on the web (e.g. www.algolist.net and wikipedia) and they are fairly easy to implement in Ruby.

The class that I implemented can be used as follows:

Out of the box we can use this BinaryTree class with numbers or strings or any other native Ruby type. We can also use it with other types as long as they implement comparable operations (i.e. def <=>).

Adding Nodes to a Binary Tree

The code to add nodes to the binary tree is shown below. This code first evaluates if we already have a root node, if we don't then we create a root node with the new value and we are done. If we already have a root node then scan the nodes of the tree (using the binary-search-tree property aforementioned, smaller values go to the left, equal or greater than values go to the right) until we reach a leaf node. Once we've reached a leaf node we update it to point to the new node.

Walking the Tree

One of the advantages of a binary search tree is that it's very easy to retrieve in order the values stored on it. This process is called "walking the tree in order". For example, if we have a tree with the following values: 100, 50, 200, and 150 walking the tree in order will give us: 50, 100, 150, and 200. Walking the tree can be implemented easily with a recursive function. However the recursive method, although elegant, it's not very efficient when the tree is expected to have a large number of nodes. The code that I implemented to walk the tree does not use recursion but instead uses an array as a stack to keep track of the nodes that it visits. This version is not as elegant as the recursive version found in most introductory books but it works well even when the tree has thousands of nodes.

Below is the code for walking the tree in order that I implemented.

While implementing this method I got to experience first hand the beauty and simplicity of one of Ruby's most touted features: blocks. You can see the use of blocks in the line yield node of the previous code. As the algorithm finds the next node to process it simple yields it to the caller program. This allows the code to walk the tree to be completely independent from the action that we want to perform on each node. For example, we can print each node to the console with the following code:

tree.walk { |node| puts "#{node.value}" }

In this example the block is a simple "puts" statement but we'll see a more sophisticated example of this when we review the code to draw the binary tree. This kind of decoupling between the iteration of the elements in the tree and the action to execute on each of them can be implemented in other programming languages via delegates or pointers but I was amazed on how simple and elegant it was to do this in Ruby.

How to Draw a Binary Tree

The algorithm to draw a binary tree that I implemented is pretty much a variation of the algorithm to walk the tree with the exception that we need to calculate the coordinates where each node needs to be placed as we traverse the nodes in order. Calculating the coordinates turned out to be an interesting challenge but a final simple solution.

The basics of this algorithm are as follow:

Draw the root node in the coordinates indicated.
Draw the left subtree to the left of the root node.
Draw the right subtree to the right of the root node.

To calculate the coordinate where the left subtree should be drawn we count how many children the right side of the left subtree has. We use this number to calculate the x-coordinate as a negative offset from the root. The y-coordinate is simple a positive offset from the root. Then we recurse this process for the left subtree and right subtrees. The following code snippet shows this code.

The procedure to calculate the coordinate where the right subtree should be drawn is just the reverse: we count how many children the left side of the right subtree has. We use this number to calculate the x-coordinate as a positive offset from the root. The y-coordinate is simple a positive offset from the root. Then we recurse this process for the left subtree and right subtrees. The following code snippet shows this code.

Like in the walk method, the three draw methods presented here only calculate where the node should be drawn but they don't actually draw the node. Instead they call a block with the appropriate parameters to perform the actual operation. You can see this in the line block.call(node.value, x, y, parent_x, parent_y) in these three routines.

The following code shows an example of calling the draw method on a simple tree and displaying on the console the coordinate where each of the nodes should be drawn

There is plenty of research on the best way to draw binary trees with lots of data. The book Graph Drawing: Algorithms for the Visualization of Graphs by Tollis et al. seems to be the reference that most research papers and presentations cite. On pages 43-45 they provide a more sophisticated version of the algorithm that I used plus several advanced algorithms for different scenarios.

Perhaps in the future I'll update my code to implement the enhancements suggested by Tollis et al. and also to remove the recursive calls from my implementation.

A Real Example of Drawing a Binary Tree on a Web Page

Once we have the coordinates where each node can be drawn we could use any graphic program to perform the actual drawing. Since I wrote this program as learning exercise I decided to write a small Ruby web application to do the actual drawing. In this web application I use the new HTML 5 canvas element as the drawing surface and a series of JavaScript calls to draw lines and circles on the appropriate coordinates.

The following code shows a simplified version of how I generate the JavaScript calls to draw the nodes (circles and labels) and lines connecting the nodes.

Notice that this code merely creates three arrays (circles, lines, labels) with some JavaScript calls. These arrays are later inserted on a web page so that when the user renders it the tree is drawn on their browser.

Source Code and Live Demo

If you want to the see this code running you can visit http://binarytree.heroku.com and generate a few binary trees with random data or draw a binary tree with your own set of data.

The demo site is limited to 250 nodes but on my local environment I've drawn trees with up to 10,000 nodes. One of the disadvantages of the drawing algorithm that I implemented is that it's not an area-efficient algorithm which leads to very wide drawings. You can see this effect in this tree with 500 nodes. A more dramatic example is a tree with 5,000 nodes that I generated with this code too. If I were to print this tree it would be 115 feet long by 1 foot tall (as a reference, a basketball court is 94 feet long!) You can download this tree from http://hectorcorrea.com/downloads/tree5000.png but be aware that it's almost 4MB in size (which is why I am not making it a hyperlink.)

You can find the entire Ruby source code to implement the binary tree and the binary tree drawer, as well as the demo web site on github at https://github.com/hectorcorrea/binary-tree. Feel free to check it out and let me know what you think.

Related Post

Drawing a Binary Tree in CoffeeScript

Git Basics

hector@hectorcorrea.com

This blog post is a beginners' guide (written by a git beginner) on how to perform basic version control operations with git including initializing a repository, adding files to it, and using branches.

The goal of this post is to describe how to use git in your local machine for version control. In a future post I'll cover the basics of using git and github for collaboration with other developers.

Everything that I show on this blog is done through git's command line via the Mac OS X terminal. The steps should be very similar in other operating systems but I have not tested them outside of the Mac.

Is Git Installed on my Mac?

We can issue the following commands from the terminal to find out if git is installed and what version we have installed:

which git 
git --version

The command which git tells us if git is in our path and should return something like "/usr/local/git/bin/git". If it returns nothing it means that git is not in our path and probably not installed in our machine.

If git is installed, we can issue git --version to figure out what version we are using. The following screenshot shows what I got on my machine:

If git is not installed on your Mac you can use the pre-compiled installer to install it on your computer.

Creating a Git Repository

Once we have git installed on our machine creating a repository for version control is pretty simple. Let's start by creating a brand new test folder and asking git for its status.

mkdir testgit
cd testgit
git status

The previous call to git status will return something along the lines of "Not a git repository (or any of the parent directories)" which is OK since we haven't told git anything about this folder.

To put this folder under version control, or in git terms, to create a local repository, we can simply issue the git init command. This command will create several hidden folders and files inside our testgit folder. Git uses these folders and files to keep track of changes to files. If we issue git status now, we will see that git reports a status rather than an error like it did before we issued git init. The following screenshot shows the results of running these two commands on my machine:

Notice how git reports that a branch called master has been created and that there is nothing to commit at this point. This folder is now officially under version control and everything that we do inside of it will be monitored by git.

Adding Files to a Git Repository

Let's start by saving a text file named "file1.txt" into this folder and asking git again for the status of the folder. This time git will report something like the screenshot below

Notice that git is now reporting that the new file (file1.txt) exist in the folder but has not been added to the repository, or in other words, the file is not under version control yet.

Let's go ahead and issue the git add file1.txt command as suggested in the previous screenshot and then check the status of the repository again. The following screenshot shows what git reports now. Notice how git now reports that there is one new file to be committed.

By adding a file we let git know that this file now needs to be tracked for version control but it's content hasn't been added to the repository. We need to issue another git command in order to actually commit the file to the repository.

Committing Files to a Git Repository

To commit to the repository new files or changes to existing files we simply need to issue a command like git commit -a -m "first file added". The following screenshot shows the result of committing this file and getting the status of the repository again.

At this point we can add more files to the testgit folder, add them to the repository with git add and commit them with git commit and we are pretty much up and running with git.

Likewise, we can make changes to files already in the repository (e.g. to file1.txt) and commit them with git commit. We don't need to do anything special to be able to make changes to files already in the repository. We just edit the file and when we are ready we commit the changes. The following screenshot shows how an example of this.

A note for Microsoft Visual Source Safe (VSS) and Team Foundation System (TFS) users: In git, the term commit is similar to the concept of checking in a file in VSS and TFS. However the term checkout in git refers to the process of switching to another branch which is totally different from the meaning of in VSS and TFS. Keep this in mind.

View History of Changes

Now that we have committed a few changes to our repository we can use several git commands to review what has changed, when each change took place, and who made each change. For example, the git log command gives us a list of commits that have been performed on the current branch. From there we can use the git show command to find out what commits affected a particular file or the git whatchanged command to review what were the specific changes made to a file on a particular commit. We can also use the git blame command to find out who made changes to a file (you've got to love a version control system that has a blame command.)

There are a lot of options that can be used with any of these commands and an entire blog post could be written about any one of them. The best place to get more information is probably the git docs web site. Below is a screenshot of what git log displays in our sample after a few commits.

It's important to notice that aforementioned commands are command line tools and their results, although comprehensive, are not exactly easy to navigate or review since they are just plain text.

However, on the Mac OS X git comes with a graphic user interface to review the changes made to a particular branch. You can invoke this tool by issuing gitk on the terminal. Gitk displays the history of changes in a format that it's easier to navigate than plain text but in my opinion is still very rudimentary compared with what you can do in other version control systems like TFS. There are perhaps better tools out there that might be worth researching. Below is a screenshot of how gitk displays the history of changes in our repository. In this screenshot we can see that (1) on the second commit (2) file1.txt was changed and (3) the specific changes that were made to this file.

Using Branches

One of the biggest advantages of using a version control system for source code is that it allows you to make changes to your projects to test things out without having to worry about messing up a good working version of your system. By using branches you can create "sandboxes" from a good version of your source code, play on your sandbox until you are happy with your changes and then merge the changes back to the good version of your system...or you can also discard the sandbox altogether if you end up not being happy with the changes. As the git tutorial page says "branches are cheap and easy, so this is a good way to try something out."

When we first added a file to our repository git created a branch called master. You can see this by issuing the git branch command from the terminal.

If we want to create a new branch called "sandbox" from the current master branch we just need to issue git branch sandbox. This will create a new branch called "sandbox" with an exact copy of the code in the current master branch. After this if we issue the git branch command again we should see both branches listed as in the following screenshot. The asterisk next to the master branch means that we are currently on the master branch.

To switch to the sandbox branch we issue the git checkout sandbox command. If we issue the git branch again the asterisk will be next to the sandbox branch.

Any changes that we make to the files from this point forward will be localized to the sandbox branch until we switch back to the master branch. Let's say for example that we edit file1.txt and commit these changes using the regular git commit command. The following screenshot shows how the changes to file1.txt on the sandbox branch have been committed but the file still shows the original content on the master branch.

We can use the git merge command to merge the content of one branch with another. For example, let's assume that we are happy with the changes that we made on the sandbox branch and we want to bring them to the master branch. The following commands will merge the contents of the sandbox branch back into the master branch:

The merge command will automatically update on the master branch any files that were updated on the sandbox branch, it will also add any news files to the master branch that might have been added to the sandbox branch and delete any files that were deleted on the sandbox. You can use some additional arguments with the merge command to configure exactly what should be merged.

If the merge command detects that some of the files to be merged changed in both branches, git will do an automatic merge and ask you to verify the changes and perform a manual commit with git commit. Take a look at the git merge documentation for more details on this.

Summary

In this blog post we've seen how to get started with git. We went from creating a brand new repository, committing changes to it, reviewing the history of changes that we made, and created a separate branch to test changes before merging them back to the master branch.

As I mentioned at the beginning of this blog in a future post we'll explore how to use git to collaborate with others and work on remote repositories like github.

References

There are plenty of good references to find more information about git, the basic commands that we've reviewed in this blog entry, and advanced git topics. Below is a selected list that I've found useful.

This Git Tutorial is a great place to start. I learned a lot of what I posted on this blog in this tutorial.
More comprehensive documentation can be found on Git User's Manual.
The official git web site has also a ton of documentation that I've found useful.

Update:Lo and behold, today I found by accident the Git Community Book. This is a fantastic reference that explains in great detail some of the beginner and intermediate workflows. I highly recommend you take a look at it. It's too bad this book (although it lives under the http://git-scm.com/ domain) is not listed on the "documentation" section. Sigh.

Ruby Development on the Mac OS X

hector@hectorcorrea.com

A little more than a month ago I got myself a MacAir. Even though I’ve own computers since the mid 80s this is the first time that I buy a Machintosh. The reason I bough a Mac rather than a PC is because I wanted to start experimenting with the web development tools that exist for non-Windows environments. Although I’ve been a software developer for more than 15 years all my experience is with Microsoft Windows and I thought is about time I start experimenting with the tools that “the “other half of the world” is using.

Since I got a Mac most people would probably thing that I want to start writing applications for the iPhone or the Mac but in reality I want to experiment with the development tools for web applications. I will probably write an app or two for the iPhone or the Mac just to see what it’s like but my real motivation is web development using non-Windows tools.

This is not to say that I am abandoning development with Microsoft’s tools (or the Windows ecosystem) but rather I want to experience what else is out there. There are plenty of successful web sites (e.g. Google and Facebook) that are developed using non-Windows tools and I want to see what the development experience on those tools is like.

The programming language that I decided to learn first is Ruby along with the popular web development framework Ruby on Rails. Over the following months I will be posting my experiences developing applications on the Mac OS X using Ruby and Ruby on Rails from the perspective of somebody that is very familiar and comfortable with Microsoft/Windows development.

Why Ruby?

When I decided to learn a new language I wasn’t too thrilled to learn Perl, PHP, or Phyton. I don’t really know why, the thought of learning any of those languages just didn’t excite me despite the fact that they are the ones used some of the most successful sites including Google and Facebook.

But Ruby gave me a different vibe, people seem to be very passionate and excited about Ruby and Ruby on Rails. Both of them seem to be quite popular to build web applications which is what I am most interested in. In addition a lot of the ideas for Microsoft’s ASP.NET MVC (which is my main development platform these days) were taken from Rails which made me think that there is definitively a lot to be gained from learning both Ruby and Rails.

In his presentation 10 Things Every Java Programmer Should Know About Ruby Jim Weirich has this great quote:

“A language that doesn’t affect the way you think about programming is not worth knowing — Alan Perlis”

So What Does Ruby Offer? (or How I Learned to Stop Worrying and Love the Bomb)

The Ruby programming language is introduced like this in the ruby-lang.org web site:

“A dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write.”

In digging deeper I found that Ruby is object oriented, provides garbage collection, borrows some interesting features from functional languages (which is something I’ve been meaning to look into), and it has indeed a clean syntax.

However, there were also a few things in Ruby that I wasn’t too excited about. In particular the fact that Ruby is an interpreted language and that it is not statically typed (although it is strongly typed.) After writing code in C# for so many years I had a hard time imaging myself using a programming language that didn't offer these features.

With my background in C# I found the slides for 10 Things Every Java Programmer Should Know About Ruby by Jim Weirich and Ruby for Java Programmers by Mike Bowler very interesting. Both of these presentations are packed with a lot information on what makes the Ruby language different from Java/C# and highlight its strengths. Some of the topics that they cover include:

The Dynamic nature of Ruby makes factories and mock objects trivial
Reflection is also taken to a new level (compared with what you can do in a statically typed language like Java or C#)
Nil is an object which means you never get a null pointer
Closures: an object that is a block of code

Jim’s slides also address my biggest concern (namely that Ruby is not statically typed) with a great quote from Bob Martin in which he mentions how the safety net provided by static typed languages is not as important when your code is covered with unit tests as those test would catch type errors for you. I am not sure I agree with this statement 100%. It is certainly an interesting point, though.

You can find these and other interesting articles at the ruby-doc.org web site.

Ruby on the Mac OS X

One of the nice things about OS X is that out of the box it comes with the runtime libraries for several programming languages including Perl, PHP, and Ruby.

This is really nice feature for me since that means I can start playing with Ruby without having too worry too much about where to get the binaries or how to compile the source code. In my case my Mac came with Ruby 1.8.7.

Beginning Ruby

I have found the book Beginning Ruby: From Novice to Professional by Peter Cooper a great resource to learn how to program in Ruby and get going.

Peter does a great job of covering everything from how to make sure Ruby is installed on your machine, how to install Ruby, very basic programming topics (e.g. what is a class) to more advanced topics like exception handling, database access, web development or Ruby-specific techniques like how to implement enumerators and the yield operator, mix-ins, and code blocks, development frameworks for Ruby like Rails and Sinatra.

I highly recommend this book to anyone starting with Ruby.

Development Tools for Ruby

The fact that the Ruby interpreter comes built in with the Mac OS X is great, but what about editors, debuggers, IDE, source control, and other development tools. For somebody like me used to a great IDE like Microsoft’s Visual Studio I was shocked to find that there is no “standard IDE” for Ruby on the Mac or any other operating system.

From what I’ve been able to gather, most Ruby developers are comfortable editing their code in a variety of text editors that provide color syntax, basic statement completion, and indentation but not much more. There does not seem to be a “visual debugger” like what Visual Studio provides for C#, there isn’t the concept of IntelliSense either. Despite this Ruby developers seem to be able to crank up code pretty fast.

Editors

You can download for free the Mac OS X development tool (called XCode) that is a full blown IDE and supports color syntax for Ruby but not visual debugger or much more integration.

I’ve found two text editors that seem to be both popular and powerful for Ruby development:

TextMate seems to be one of the favorite editors for Ruby (and Rails) development on the Mac as it provides really nice keyboard shortcuts for code completion, running unit tests, integration with source control (both SubVersion and Git) and other goodies. This has been my favorite editor so far. It is not free but I will probably end up buying a license in the next few days.
TextWrangler is a free text editor and supports color syntax for Ruby. Somehow I found the workflow for Ruby development not as good with this editor as with TextMate but you cannot beat the price.

Here is an screenshot of TextMate

There are plenty of other editors that people use on the OS X to edit Ruby code that are common among *nix users including Vim and Emacs. Yet, for somebody like me, with a Windows background, those editors look and feel like something from the stone age. They can be very powerful once you learn them but the learning curve might be very steep.

Visual Debugger

There is not a visual debugger like the one in Microsoft Visual Studio for Ruby on the OS X or other *nix environments. This has been one of the biggest shocks to me as I’ve gotten used to have one and still miss it.

One can make the argument that if your code is testable and properly structured (small methods, classes with single responsibility) the need for an advanced debugger is less critical. That might be true but I am still dumbfounded by the lack of one. I really hope that Ted Neward’s prediction for 2011 that “Apple starts feeling the pressure to deliver a developer experience that isn’t mired in mid-90’s metaphor” comes true.

I should clarify that there is a debugger in Ruby that allows you to step through your code but it is nothing like what I was used to in Visual Studio.

Source Control

Although more than two years old the article 7 Version Control Systems Reviewed in Smashing Magazine is a great resource to get an idea of what’s available and popular on the Mac OS X.

Of the source control systems mentioned in the article so far I have only experimented with Git and I am very pleased with it. As with the text editors I was a shocked with the lack of visual tools to do basic operations like check in and check out but this probably due in part to the lack of a standard IDE for Ruby development.

However, Git’s installation process on the OS X was incredible painless. I was pleasantly surprised on how well documented the process to install it and get started are. I did notice that Git seems very light-weight compared to a traditional source control system in Windows (e.g. Team Foundation or SourceSafe.)

It has taken me a bit to get used to depend on command line tools to check in and check out my code but at the same time I am amazed how simple it becomes after a few days of using it. I am even using GitHub to push my code “to the cloud” and keep a copy of it outside my machine.

In Summary

So after little bit more than a month of playing with Ruby on the OS X on my spare time, I am starting to get familiar with the language and the development tools. All of a sudden I find myself writing code without looking at my book (or the web), running my unit tests, making changes, and checking my changes to my local and remote source control repositories. I am not nearly as proficient with Ruby or the development tools on OS X as I am with C# or Visual Studio but I am starting to get to the point where I feel comfortable.

I am sure it will be a fun and interesting ride. I’ll keep blogging about my experiences of learning Ruby (and eventually Rails) and the development tools for OS X as I learn more.

Update: Check out Web Development on the Mac OS X for an update on my experience 18 months later.

Returning HTTP 404 in ASP.NET MVC

hector@hectorcorrea.com

A few weeks ago I noticed that when users of my site request a blog topic that does not exist although I was displaying a user friendly message indicating that the topic does not exist I was not returning the proper HTTP status code (404) to indicate to the user that the page was not found. This approach is typically not a problem if the user visiting my site is a human (users don’t really care about HTTP status codes) but it is important if the “user” visiting the site is a web crawler like the Googlebot or the Yahoo Slurp because for these non-human users the HTTP code is very important as it means something concrete to them.

In digging deeper into this issue I realized that I was doing what is known as a “soft 404” which is not good for web crawlers because it fools them into thinking that there is a valid page for an invalid URL. In my case my site was returning a status code 200 (which means OK) rather than a status code 404 that means “Not Found.” This is in fact so common that both, the Googlebot and the Yahoo Slurp, web sites talk about this problem here and here.

The Root of the Problem

The code that was causing this issue in my controller looked similar to this:

   1:  public class BlogController : Controller

   2:  {

   3:      public ActionResult Sample1(string topicName)

   4:      {

   5:          bool isTopicFound = SomeLogic(topicName);

   6:          if (!isTopicFound)

   7:          {

   8:              var route = new RouteValueDictionary(new { controller = "Error", action = "NotFound" });

   9:              return new RedirectToRouteResult(route);

  10:          }

  11:          // regular code to display topic

  12:      }

  13:  }

Although this code displays a nice user friendly page to the user (via the NotFound action of the Error controller) because I am using RedirectToRouteResult the browser gets an HTTP code 302 (Found) followed by a 200 (OK) which is not what I wanted. Below is a trace of a request like this with Fiddler. Notice how the original request to /Blog/Sample1 got a 302 response code indicating that something was found at a new URL and then the new URL (/Error/NotFound) in turn got an HTTP status code of 200 (OK) because this second URL indeed returned OK.

Throwing an HttpException

When I decided to update my ASP.NET MVC site to handle this issue correctly my first thought was to throw an HttpException from my controller, for example something along these lines:

   1:  public ActionResult Sample2(string topic)

   2:  {

   3:      bool isTopicFound = false; // SomeLogic(topicName);

   4:      if (!isTopicFound)

   5:      {

   6:          throw new HttpException(404, "Blog topic not found");

   7:      }

   8:      // regular code to display topic

   9:  }

The HttpException can either be handled in the controller’s OnException method or on the Application_Error (in Global.asax.) Whatever code we use we need to be sure we don't rely on RedirectToRouteResult like in the previous example or otherwise we'll be back on the HTTP 302 + HTTP 200 trap.

If the HttpException is not handled at all in the code then the customErrors section in the web.config could redirect the user to a default error page so they see a user friendly error message. In this case however the returned HTTP code is an HTTP 302 followed by an HTTP 200 just like in the previous example! This actually makes sense giving that the parameter in the customErrors section where the page is configured is clearly named “defaultRedirect”. However this does not address the problem that I am trying to solve.

The ASP.NET MVC way

As it turns in ASP.NET MVC you can directly change the status code of the response and still display whatever view you want to. The code is actually very simple as shown in the following example. Notice how the status code is forced to 404 (Not Found) and the “Error” view is selected.

   1:  public ActionResult Sample3(string topic)

   2:  {

   3:      bool isTopicFound = false; // SomeLogic(topicName);

   4:      if (!isTopicFound)

   5:      {

   6:          HttpContext.Response.StatusCode = 404;

   7:          HttpContext.Response.Clear();

   8:          return new ViewResult() { ViewName = "Error" };

   9:      }

  10:      // regular code to display topic

  11:  }

This code accomplishes both goals: the user sees a human friendly error (via the Error view) and the HTTP status code returned is 404 as show in the following Fiddler trace. Notice how the original request to /Blog/Sample3 returns a 404 code directly without doing a redirect.

Now when somebody tries to access a topic that does not exist on my blog they get both a nice error message and the correct HTTP status code. You can try this by hitting this link to a non-existing blog topic and looking with Fiddler the HTTP status code returned.

A Couple of Gotchas

While looking into this problem I noticed that throwing an HttpException from a controller is slightly different than throwing any other exception. For example, although you can decorate your controllers with the HandleError attribute and configure what view will be rendered in case of exceptions, HandleError does not handle HttpExceptions. This post on StackOverflow has more information on this issue.

The second gotcha that I ran into is that if you return an HTTP status code 404 and your view is smaller than 513 bytes Google's Chrome browser will not render your view but rather Chrome will render its own view. This issue has been documented before and it only happens on Chrome. Most likely this won't be an issue for you in production as most views are in fact larger than 512 bytes that by the time all the content of your page is considered but you should keep it in mind while testing as it might throw you off.

Log4net Thread-Safe but not Process-Safe

hector@hectorcorrea.com

A few days ago I was struggling with a problem with log4net in a web application. I have used log4net in many applications before with little or no problem but in this case even with a single user hitting the web site I was seeing strange logging behaviors. In my case some of the log entries were being logged but others were not.

The application in question is a C# ASP.NET MVC web application running under IIS. I am using the true and tried RollingFileAppender. It couldn't have been a more typical use of log4net and yet I was seeing a weird things.

Here are the symptoms that I was seeing:

Log4net was logging some entries but not others.
The issue was happening even when I was the only user on the site.
The issue was only happening in one of our development servers but not on my local box.

Since some entries were being logged but not others I was able to rule out the possibility that IIS might not have enough rights to write to the folder configured for the RollingFileAppender. This is a typical issue but not applicable in my case since if IIS wouldn't have enough rights nothing would be logged at all. In addition since the issue was happening even when I was the only user on the site I was sure it was not due to heavy utilization or some sort of race condition (which log4net typically handles nicely anyway.)

The fact that the issue was happening only on the development server with IIS 6 and not on my local machine with IIS 7 made me suspect that the different versions of IIS might the culprit but wasn't sure about that.

Turning log4net internal diagnostic

To begin troubleshooting I turned on log4net's diagnostic option to see if log4net was itself reporting problems that would give me a hint. If you are not familiar with this log4net feature check out the "How do I enable log4net internal debugging?" section in the log4net FAQ.

In my case I configured log4net to dump all of its diagnostic information to a file called "LogDiag.txt". After running my application and hitting the site a few times I went to see what was being logged in the LogDiag.txt file and lo and behold I noticed that there were three LogDiag.txt files being generated. The following picture shows how this looked on my machine:

I opened the "LogDiag.txt" file and there were no errors indicated logged in there. However, the other two files (the ones prefixed with a GUID) had the following error:

log4net:ERROR [RollingFileAppender] 
   Unable to acquire lock on file c:\dev\SandboxMvc\SandboxMvc\App_Data\LogFile. 
   The process cannot access the file 'c:\dev\SandboxMvc\SandboxMvc\App_Data\LogFile' 
   because it is being used by another process.
log4net:ERROR [RollingFileAppender] 
   OpenFile(c:\dev\SandboxMvc\SandboxMvc\App_Data\LogFile,True) call failed.

Houston, we have a problem...

OK so we indeed have a problem. Log4net is clearly telling me that it is having problems writing to the log file. That's good. At least I am not going crazy. But how come? How come am I having this problem now but not in other applications? How come it happens only on the web server but not in my local box?

The answer turned out to be the way IIS was configured on the server where the problem was happening. It turns out that IIS on that particular box is configured to use what is known as a web garden which means that IIS will launch multiple worker processes to handle requests. That was different from the configuration on my local box where IIS was setup to have a single worker process running.

The following picture shows how to configure IIS to use one or many worker processes. In this picture you can see that the "ASP.NET 4.0 Application Pool" is configured to use a maximum of three worker processes.

When the IIS application pool is configured to use more than one worker process (i.e. a web garden) what happens is that multiple processes will be spun by IIS to serve requests for your application. If you configured log4net to use a single file to log requests (as I did) then this means that multiple processes will attempt to write to a single file and locking issues will arise. Once I realized what the problem was finding a solution was quite easy.

The fact that there were three worker process running also explained why I saw three "LogDiag.txt" files on my machine. Log4net was smart enough to name each file differently (remember that two of them were prefixed with a GUID) so that it's internal logging didn't experience the same problem that I was having.

Before I discuss how you can work around this issue let me clarify that there is nothing wrong per-se with having a web garden in IIS. This configuration is actually good on a server-class machine. The discussion on when and how to configure a web garden is beyond the scope of this blog post however the lesson learned is that if you have a web garden you need to take this into account when configuring the RollingFileAppender.

How to address the issue

There are two ways that we can configure the RollingFileAppender to work in a web garden. One option is to configure it to use what is known as "minimal lock" on the log file. With this option log4net will only lock the file for the duration of each log operation. This option reduces the chances that more than one process will collide as they keep the file locked to a minimum. However, there is no guarantee that under load they might actually collide. Additionally there is a performance penalty in opening and closing the file on each log entry. I am not sure if the penalty is too high but it's worth testing if you decide to use this option. Below is an example of how to configure the RollingFileAppender to use minimal locking (notice the lockingModel setting in line 2.)

   1:  <appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender,log4net">

   2:      <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />

   3:      <param name="File" value=".\\App_Data\\LogFile" />

   4:      <param name="AppendToFile" value="true" />

   5:      <datePattern value=".yyyy-MM-dd'.txt'" />

   6:      <rollingStyle value="Composite" />

   7:      <maxSizeRollBackups value="5" />

   8:      <maximumFileSize value="100KB" />

   9:      <staticLogFileName value="true" />

  10:      <layout type="log4net.Layout.PatternLayout">

  11:          <conversionPattern value="[LOGENTRY] %date %-5level %logger{2} - %message %newline" />

  12:      </layout>

  13:  </appender>

The second option to take care of the problem is to make sure that each IIS worker process writes to its own file so that there are no collisions at all. With this option if the web garden is configured to use three worker processes then you'll have three simultaneous log files (one for each active worker process.) Each of them can use an exclusive lock on the file as these files are not shared across processes. This method is guaranteed not to have collisions and allows you to keep using the fast exclusive lock method. The disadvantage is that now you have multiple log files to look at rather than a single consolidated one. Below is an example on how to configure the RollingFileAppender to create one file for each process. Notice how line 2 now has been configured to use a PatternString to name the file and the inclusion of pattern %processid to make each file unique by each process ID.

   1:  <appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender,log4net">

   2:      <file type="log4net.Util.PatternString" value=".\\App_Data\\Log[%processid]" />

   3:      <!--<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />

   4:      <param name="File" value=".\\App_Data\\LogFile" />-->

   5:      <param name="AppendToFile" value="true" />

   6:      <datePattern value=".yyyy-MM-dd'.txt'" />

   7:      <rollingStyle value="Composite" />

   8:      <maxSizeRollBackups value="5" />

   9:      <maximumFileSize value="100KB" />

  10:      <staticLogFileName value="true" />

  11:      <layout type="log4net.Layout.PatternLayout">

  12:          <conversionPattern value="[LOGENTRY] %date %-5level %logger{2} - %message %newline" />

  13:      </layout>

  14:  </appender>

But I thought log4net was thread-safe?

Log4net is indeed thread-safe as it is clearly indicated on log4net's FAQ. A lot of people get confused on this because the documentation for the RollingFileAppender says that the appender is not thread safe. In reality the log4net code takes this into account ([1][2]) and makes sure logging is thread-safe. Some people have even run tests to prove this.

However, the fact that log4net is thread-safe does not mean that is process-safe to write to a single file. This is not log4net's fault, there is no such thing as the ability to have multiple processes write concurrently to the same text file. We use relational databases (not text files) when we need to have multiple processes write to a single repository of data.

In summary

If you are using log4net's RollingFileAppender in an environment where IIS is configured to use multiple worker processes (i.e. a web garden) you must ensure that the RollingFileAppender uses one file per worker process rather than a single file.

Encrypt and Decrypt a string in C#

hector@hectorcorrea.com

This blog posts presents an easy to use C# class to encrypt and decrypt strings in .NET.

Although the System.Security.Cryptography namespace in the .NET Framework provides a wealth of classes to encrypt and decrypt values, it seems that MSDN fails short of providing a good and simple example on how to use these classes for the most common request: encrypt a string.

The class presented in this blog allows you to encrypt and decrypt a string with the following lines of code:

   1:  string encryptionPassword = "supersecret";

   2:  string textToEncrypt = "the quick brown fox jumps over the lazy dog";

3:

   4:  string encrypted = Crypto.Encrypt(textToEncrypt, encryptionPassword);

   5:  string original = Crypto.Decrypt(encrypted, encryptionPassword);

Disclaimer: The code presented in this blog post is just one of the many ways to encrypt and decrypt strings in .NET. If you need to encrypt incredible confidential information (like the nuke launch codes) you should check specific books on Cryptography. Having said that, you probably arrived to this post because you are in search of a simple example on encryption with .NET. If that's the case then this post is for you.

Below is the code of a simple C# class to encrypt and decrypt strings. All in all it's 61 lines of code including curly braces. You can just copy and paste this code to your project and voilà you are ready to encrypt and decrypt strings. If you are bit curious the remainder of this post explains how the code works.

// Code based on the book "C# 3.0 in a nutshell by Joseph Albahari" (pages 630-632) // and from this StackOverflow post by somebody called Brett // http://stackoverflow.com/questions/202011/encrypt-decrypt-string-in-net/2791259#2791259 static public class Crypto { private static readonly byte[] salt = Encoding.ASCII.GetBytes("Ent3r your oWn S@lt v@lu# h#r3"); public static string Encrypt(string textToEncrypt, string encryptionPassword) { var algorithm = GetAlgorithm(encryptionPassword); byte[] encryptedBytes; using (ICryptoTransform encryptor = algorithm.CreateEncryptor(algorithm.Key, algorithm.IV)) { byte[] bytesToEncrypt = Encoding.UTF8.GetBytes(textToEncrypt); encryptedBytes = InMemoryCrypt(bytesToEncrypt, encryptor); } return Convert.ToBase64String(encryptedBytes); } public static string Decrypt(string encryptedText, string encryptionPassword) { var algorithm = GetAlgorithm(encryptionPassword); byte[] descryptedBytes; using (ICryptoTransform decryptor = algorithm.CreateDecryptor(algorithm.Key, algorithm.IV)) { byte[] encryptedBytes = Convert.FromBase64String(encryptedText); descryptedBytes = InMemoryCrypt(encryptedBytes, decryptor); } return Encoding.UTF8.GetString(descryptedBytes); } // Performs an in-memory encrypt/decrypt transformation on a byte array. private static byte[] InMemoryCrypt(byte[] data, ICryptoTransform transform) { MemoryStream memory = new MemoryStream(); using (Stream stream = new CryptoStream(memory, transform, CryptoStreamMode.Write)) { stream.Write(data, 0, data.Length); } return memory.ToArray(); } // Defines a RijndaelManaged algorithm and sets its key and Initialization Vector (IV) // values based on the encryptionPassword received. private static RijndaelManaged GetAlgorithm(string encryptionPassword) { // Create an encryption key from the encryptionPassword and salt. var key = new Rfc2898DeriveBytes(encryptionPassword, salt); // Declare that we are going to use the Rijndael algorithm with the key that we've just got. var algorithm = new RijndaelManaged(); int bytesForKey = algorithm.KeySize / 8; int bytesForIV = algorithm.BlockSize / 8; algorithm.Key = key.GetBytes(bytesForKey); algorithm.IV = key.GetBytes(bytesForIV); return algorithm; } }

The Crypto class

The Crypto class presented in this post has four methods. Methods Crypt and Decrypt are public and are the ones that your code calls to encrypt and decrypt values. The other two methods (InMemoryCrypt and GetAlgorithm) are used internally by the class.

Encrypt()

The process of encrypting values in .NET requires a few steps that are not evident to first comers. In a nutshell this process involves setting up the algorithm that will be used for encryption and pushing the data to encrypt through it. In our case since we want to encrypt strings and the .NET cryptography classes don't provide overloads for strings out of the box we'll need to take a few extra steps to account for this. The Encrypt() method in our Crypto class takes care of these steps. Let's review this method in detail.

   8:      public static string Encrypt(string textToEncrypt, string encryptionPassword)

   9:      {

  10:          var algorithm = GetAlgorithm(encryptionPassword);

11:

  12:          byte[] encryptedBytes;

  13:          using (ICryptoTransform encryptor = algorithm.CreateEncryptor(algorithm.Key, algorithm.IV))

  14:          {

  15:              byte[] bytesToEncrypt = Encoding.UTF8.GetBytes(textToEncrypt);

  16:              encryptedBytes = InMemoryCrypt(bytesToEncrypt, encryptor);

  17:          }

  18:          return Convert.ToBase64String(encryptedBytes);

  19:      }

The first thing we do in the Encrypt() method is define what algorithm we want to use for encryption (see line 10.) In our case the we will use the Rijndael algorithm (more on this later.) This algorithm class has the ability to provide either an encryptor or a decryptor. For the Encrypt() method we obviously need an encryptor (see line 13.)

The second thing we do is convert to bytes the string that we received with the value to encrypt (line 15.) This is an important step as the .NET encryption classes work with bytes, not strings.

As you probably already know strings in .NET are encoded in Unicode and as such they can hold characters from pretty much any language in the world. Some of these characters need more than one byte to be stored and therefore the number of bytes that you get on line 15 might be larger than the number of characters in your string.

If you only deal with ASCII values you probably don't give this much thought as most of the characters in your strings translate to a single byte (for example character "A" can be represented by one byte with value of 65 - its ASCII value.) However, strings in .NET can hold characters that need multiple bytes to be stored. For example, the Japanese character maps to three bytes with values 227, 130, and 166.

So now that we have defined an algorithm to use (along with its corresponding encryptor) and converted to bytes the information to encrypt we are ready to pass this information to the InMemoryCrypt method (line 16) to finally perform the encryption. We'll talk about the details of the InMemoryCrypt method in the section below but for now let's just say that it encrypts the bytes that we pass to it and gives us back the corresponding encrypted bytes.

Finally, on line 18 we convert the encrypted bytes back to a .NET string since that's what we want as an end result. This is the reverse process of what we did in line 15.

As you can see, although the Encrypt method is only a few lines long there is a lot going on. This is not exactly what I would call a self evident process and it is no wonder this is one of the most commonly asked questions in .NET message boards!

GetAlgorithm()

The GetAlgorithm() method declares what algorithm will be used for encryption and sets up the corresponding encryption keys. The .NET framework supports several algorithms and each of them has different strengths and weaknesses. On this particular example we are using the Rijndael algorithm. You can find a lot of information about this algorithm on the web but for the purpose of this blog post let's just say that it uses a pair of values (known as key and initialization vector) to encrypt a set of bytes. In our case both, the key and the initialization vector (IV), will be calculated from an "encryptionPassword" that we provide.

Let's look in detail at the code of this method.

  47:      private static RijndaelManaged GetAlgorithm(string encryptionPassword)

  48:      {

  49:          // Create an encryption key from the encryptionPassword and salt.

  50:          var key = new Rfc2898DeriveBytes(encryptionPassword, salt);

51:

  52:          // Declare that we are going to use the Rijndael algorithm with the key that we've just got.

  53:          var algorithm = new RijndaelManaged();

  54:          int bytesForKey = algorithm.KeySize / 8;

  55:          int bytesForIV = algorithm.BlockSize / 8;

  56:          algorithm.Key = key.GetBytes(bytesForKey);

  57:          algorithm.IV = key.GetBytes(bytesForIV);

  58:          return algorithm;

  59:      }

Line 50 shows the call to Rfc2898DeriveBytes to get a generator of pseudo-random bytes based on our "encryptionPassword" and "salt" values.

Line 53 declares an instance of the Rijndael algorithm and then on lines lines 56 and 57 we set the key and initialization vector values needed by this algorithm. Notice how we set these two values with as many bytes required depending on the KeySize and BlockSize of the algorithm.

Technically you could skip the call to Rfc2898DeriveBytes and manually set the values used for key and initialization vector. However if you do this you need to make sure that you have enough bytes to fill the key and initialization vector. This can be a problem if the "encryptionPassword" value was too short (say it's only 10 bytes long but the algorithm expects 32 bytes values.) The Rfc2898DeriveBytes takes care of this problem as it can generate enough bytes even if the "encryptionPassword" and "salt" are too short.

InMemoryCrypt()

The classes on the .NET Framework use streams to perform the encryption process. This allows the encryption of very large blocks of text while consuming very little memory. This is a nice feature of the framework as it allows you to encrypt entire documents without having to load the entire document in memory in one chunk. The drawback is that you must go through these streams even when you encrypt small sets of data like in our example.

The InMemoryCrypt() method (lines 35-43) hides the process of setting up the streams to read through the bytes to encrypt and dump the resulting bytes into another stream.

Encryption password and Salt

The "encryptionPassword" value that the Encrypt() and Decrypt() methods receive should be a value that only you know. This "encryptionPassword" alone is enough to decrypt a value encrypted with this code.

On the other hand the "salt" value defined on the class (line 6) and used in GetAlgorithm() on line 50 does not need to be kept secret as it is useless without the encryption password. It is important however that you use the exact same "salt" value to decrypt as you used to encrypt which is why I declared it as a static member of the class rather than make it a parameter to Encrypt and Decrypt. Feel free to update this value or make it configurable on your own implementation.

A few final words

The Decrypt() method (lines 21-32) is pretty much the reverse of the Encrypt() method so I won't elaborate on it.

There are many things to consider when choosing an encryption algorithm:

Some algorithms are one-way only meaning that they can encrypt a value but you cannot decrypt it. These algorithms are known as hashing algorithms and are perfect for storing password in a database.
Other algorithms do allow for encryption and decryption. Within these algorithms some of them are symmetrical (like Rijndael) meaning they use the same key for encryption and decryption whereas others are asymmetrical and use different keys known as private and public keys.

Keep this in mind when deciding what algorithm to use in your application.

References

As indicated at the top of the Crypto class most of the code for this blog post was taken from the excellent book C# 3.0 in a Nutshell by Joseph Albahari and Ben Albahari (pages 630-632) and from this post in StackOverflow by somebody called Brett.

There is a good summary of the different encryption options in the book "C# 3.0 in a Nutshell" (see pages 627-638)

AJAX calls with jQuery in ASP.NET MVC

hector@hectorcorrea.com

One of the nice things about ASP.NET MVC is the ability to easily integrate standard web libraries like jQuery in .NET web applications. In this blog post I am going to show how to execute AJAX calls with jQuery from an ASP.NET MVC application. The first example shows how to do it with an HTTP GET call and the second one uses an HTTP POST call. Both examples use JSON in the return type.

HTTP GET Example

This is probably the simplest example on how to execute an AJAX call with jQuery but it should be OK to show the general concept. In this page when the user clicks on the "click here" hyperlink a request is made to the server to retrieve the time on the server and display to the user. The AJAX part of this example is that the request to the server is executed without making a full page reload as it can be seen in the screenshot below (notice that the time the page was originally loaded does not change after I clicked the hyperlink and retrieved the server time.) Again, not terribly exciting but good enough for our purposes.

The MVC View

The MVC view that makes this possible is rather simple and it's composed of the following elements

A SCRIPT tag on the HEAD to include the jQuery library
A SCRIPT block to declare the code to execute the jQuery call
The HTML to render the page

To include jQuery in an ASP.NET MVC you just need to a add a SCRIPT tag to the head that looks like this (line 3):

   1:  <head id="Head1" runat="server">

   2:      <title>AJAX Sample 1</title>

   3:      <script src="<%: Url.Content("~/Scripts/jquery-1.4.1.js") %>" type="text/javascript"></script>

   4:  </head>

The HTML code for this page is also very simple as we are only displaying some text coming from the server (line 2), rendering a hyperlink to let the user invoke the AJAX call (line 4), and rendering a placeholder to display the value returned by the AJAX call (line 5). Notice that there is no JavaScript code in this HTML block. This is by design and I'll describe the reason for this in the next paragraphs (see Hijax section below.)

   1:      <h2>Ajax Sample 1</h2>

   2:      <p>Page loaded at: <%: System.DateTime.Now.ToString() %></p>

3:

   4:      <p><a href="GetServerTime" id="AjaxLink">Click here</a> to get current time.</p>

   5:      <div id="CurrentServerTimeDiv"></div>

Of course the interesting part of this example is the actual JavaScript block that wires the HTML anchor tag to JavaScript and makes the AJAX call.

   1:  <script type="text/javascript">

2:

   3:      $(document).ready(function () {

   4:          //Setup the AJAX call

   5:          $("#AjaxLink").click(function (event) {

   6:              event.preventDefault();

   7:              GetCurrentServerTime();

   8:          });

   9:      });

10:

  11:      function GetCurrentServerTime() {

  12:          // Make the call to the server

  13:          $.getJSON("GetServerTime", null, UpdateServerTime);

  14:      }

15:

  16:      function UpdateServerTime(result) {

  17:          // Update the page with the result

  18:          var message = "Time on the server is: " + result.ServerTime;

  19:          $("#CurrentServerTimeDiv").html(message);

  20:      }

21:

  22:  </script>

This JavaScript block has three main sections. The first section (lines 3-9) wires the HTML anchor tag with name "AjaxLink" to a JavaScript function called "GetCurrentServerTime()". It is customary to add this setup code to the ready event to make sure it's executed when the DOM of the page has been fully constructed even if not all of the resources for the page (e.g. images) have been downloaded. You can read more about this here. In this case we are replacing the "click" event of the anchor tag "AjaxLink" with a call to our own "GetCurrentServerTime()" JavaScript function.

The second section of the JavaScript block (lines 11-14) is the actual JavaScript function that will be executed when the user clicks on the link. This code executes a jQuery's getJSON call to URL "GetServerTime", passing no arguments (null), and executing function "UpdateServerTime" upon completion. When this code is executed our MVC controller will be hit. More on this later.

The third section of the JavaScript code (lines 16-19) is executed when our controller finishes its execution. In our case the controller returns an object with one property "ServerTime" that we use to construct a message to display on the placeholder "CurrentServerTimeDiv" that was defined in the HTML code.

The MVC Controller

When the user clicks on the hyperlink a request is sent through an HTTP GET to our controller. The controller will perform an operation (in this case just merely calculating the time on the server) and return this value to the view. The code for this controller is shown below:

   1:  // GET /AjaxSample/GetServerTime

   2:  public ActionResult GetServerTime()

   3:  {

   4:      var viewModel = new Models.AjaxSampleViewModel();

   5:      viewModel.ServerTime = DateTime.Now.ToString();

   6:      if (Request.IsAjaxRequest())

   7:      {

   8:          return Json(viewModel, JsonRequestBehavior.AllowGet);

   9:      }

  10:      // Fallback view if JavaScript is not supported by the browser

  11:      return View("ServerTime", viewModel);

  12:  }

When this code executes through an AJAX call (as in our case) it will return a JSON object to the client with the results. This is why our JavaScript code was able to use "result.ServerTime" to display the result. The following screenshot shows how the resulting object looks in JavaScript:

Hijax

You might be wondering why do we need a fallback clause in our controller (line 10-11) if the view explicitly makes an AJAX call. The reason for this precaution is in case the client browser does not support JavaScript or JavaScript is disabled. Although JavaScript is almost always enabled in most desktop and laptop computers that's not always the case in mobile devices.

This is also the reason we wired the AJAX call on the document ready event rather than directly on the HTML anchor tag in our view. For example, although we could have just as easily wired the AjaxLink anchor tag directly to the "GetCurrentServerTime" function that wouldn't have worked if JavaScript was disabled. Instead we wired the JavaScript call on the document ready event which is only executed if JavaScript is enabled. If JavaScript is disabled (and therefore the document ready event is not fired) the HTML anchor tag in our view is already wired to hit the "GetCurrentServer" URL which maps to the "GetServerTime" method in our controller.

This approach to JavaScript is known as Hijax and is a good way to provide progressive enhancement to your web pages.

HTTP POST Example

You can also execute AJAX calls using an HTTP POST request. For example, in the following screenshot the information in the textboxes (e-mail and comment) is sent through an asynchronous HTTP POST to the server.

The MVC View

Like in our previous example, the MVC view that makes this possible is very simple and it's composed of the following elements

A SCRIPT tag on the HEAD to include the jQuery library
A SCRIPT block to declare the code to execute the jQuery call
The HTML to render the page

The HTML code for this page is shown below. Notice how we declare an HTML form named "SaveCommentForm" (line 3) with two textbox controls (lines 4-6) and a submit button (line 7). Below the form there is a placeholder section called "NewComment" where we will display the results from the server after the user has added a new comment. Like in our previous example there is no JavaScript on this HTML code, we will be wire the JavaScript call when the web page is rendered on the browser using the Hijax method that we described before.

   1:  <p>Page loaded at: <%: System.DateTime.Now.ToString() %></p>

2:

   3:  <% using (Html.BeginForm("SaveComment", "AjaxSample", FormMethod.Post, new { id = "SaveCommentForm" })) { %>

   4:  Your e-mail address:<%: Html.TextBox("emailAddress")%><br />

   5:  Enter a comment:<br />

   6:  <%: Html.TextArea("comment")%><br />

   7:  <input type="submit" value="Save" />

   8:  <% } %>

9:

  10:  <div id="NewComment"> </div>

The JavaScript that powers this web page is very similar to the one that we used in our previous example and is shown below.

   1:  <script type="text/javascript">

2:

   3:      $(document).ready(function () {

   4:          //Setup the AJAX call

   5:          $("#SaveCommentForm").submit(function (event) {

   6:              event.preventDefault();

   7:              SaveComment(this);

   8:          });

   9:      });

10:

  11:      function SaveComment(form) {

  12:          // Make an AJAX call to the server

  13:          // notice that we request a JSON response

  14:          $.ajax({

  15:              url: form.action,

  16:              type: form.method,

  17:              dataType: "json",

  18:              data: $(form).serialize(),

  19:              success: CommentSaved

  20:          });

21:

  22:      }

23:

  24:      function CommentSaved(result) {

  25:          // Update the page with the result

  26:          var newComment = "Thank you for you comment " + result.emailAddress + "<br/><pre>" + result.comment + "</pre>";

  27:          var comments = newComment + "<br/>" + $("#NewComment").html();

  28:          $("#NewComment").html(comments);

  29:      }

30:

  31:  </script>

The first section of this JavaScript block (lines 3-9) uses the Hijax technique aforementioned to redirect the submit event of the "SaveCommentForm" to call a JavaScript function named "SaveComment".

The "SaveComment" function is defined on the second section of the JavaScript block (lines 11-20.) In this example we are issuing an HTTP POST (line 16) to the original location (line 15) that the HTML form would have on its own but we are making the call asynchronous (by using the AJAX function) and requesting that the result is delivered to us in JSON notation (line 17). When the asynchronous call completes we want jQuery to fire the "CommentSaved" function (line 19.) The following picture shows the values that are passed at runtime to the url, type, and data parameters in this AJAX call. Notice that the form.method is POST and form.action points to the "SaveComment" URL.

The third section of the JavaScript code (lines 24-30) is executed when our controller finishes its execution. In our case the controller returns a JSON object (similar to the HTTP GET example) that we use to update the page.

The MVC Controller

The controller to process our HTTP POST request is shown below. In essence this code is very similar to the one that we used before in the HTTP GET example. We process the request (although in this code I just indicate that we would need to do something with it) and then return a JSON object back to the called. Like in our previous example we return a full view as a fallback plan.

   1:  // POST /AjaxSample/SaveComment

   2:  [HttpPost]

   3:  public ActionResult SaveComment(string comment, string emailAddress)

   4:  {

   5:      //[Code to actually save the comment would go here]

   6:      if (Request.IsAjaxRequest())

   7:      {

   8:          return Json(new { comment = comment, emailAddress = emailAddress, savedAt = DateTime.Now.ToString() });

   9:      }

  10:      // Fallback view if JavaScript is not supported by the browser

  11:      return View("CommentSaved");

  12:  }

...and that's it

As you can see it is relatively straightforward to make AJAX calls using jQuery within ASP.NET MVC views to your controllers.

References

I took the idea of using Hijax for the jQuery calls from the book ASP.NET MVC in Action by Jeffrey Palermo et al. (pages 195-214)

In addition to the online documentation for jQuery, Joe Brinkman's e-book jQuery for ASP.NET Developers is a great place for beginners. If you are new to jQuery this is probably the best way to spend 7 bucks (other than buying a latte and a croissant at your local coffee shop!)

Steve Sanderson's book Pro ASP.NET MVC 2 Framework also has a good explanation on how to use jQuery in ASP.NET MVC applications (pages 517-559) that is worth checking out.

Password Recovery in an ASP.NET MVC Project

hector@hectorcorrea.com

While rewriting my personal web site with ASP.NET I noticed that although support for the ASP.NET Membership Provider comes included out of the box in a ASP.NET MVC project not all the options are fully implemented to the same extend that they are in a brand new ASP.NET WebForms project. For example, the option to reset your own password if you forgot your old one is not available out of the box in an ASP.NET MVC project.

Out of the box the following options are fully implemented in a ASP.NET MVC project:

Login in
Login out
Change your password
Create new user

Adding support for Password Recovery to an ASP.NET MVC project turned out to be pretty easy as the core functionality already exists in the Membership Provider and it's just a matter of calling it from your application.

The process that I implemented goes like this:

From the LogOn view users can go to the PasswordReset view
In the PasswordReset the user indicates his/her username and then they are sent to the QuestionAndAnswer view
In the QuestionAndAnswer view the user enters the answer to their own security question
Finally, the user is sent to the PasswordResetFinal view with a message indicating that their password has been reset and e-mailed to them.

The rest of this blog post describes the details on how this works. The source code is also available in the link at the bottom of this post.

New Views and Controllers

LogOn

The first thing that I did was update the LogOn view that comes with ASP.NET and added a link to start the Password Reset process. I wired this link to the a new method called PasswordReset in the AccountController.

Password Reset

Secondly I created the HTTP-GET PasswordReset method in the AccountController and a very simple view (PasswordReset.aspx) to allow the user to enter his/her username so that we can reset their password. The PasswordReset.aspx view is extremely simple as it only has a textbox where the user enter their user name.

public ActionResult PasswordReset()
{
if (!MembershipService.PasswordResetEnabled) throw new Exception("Password reset is not allowed");
return View();
}

I also implemented an HTTP-POST PasswordReset method in the AccountController to pick up the data and continue the process. This controller method decides whether the next step is to reset the password for this user or if we need to ask him/her a Password Recovery question before we reset their password. This step is required to honor the requiresQuestionAndAnswer configuration setting in the ASP.NET Membership Provider.

[HttpPost]
public ActionResult PasswordReset(string userName)
{
    if (!MembershipService.PasswordResetEnabled) throw new Exception("Password reset is not allowed");

    if (MembershipService.RequiresQuestionAndAnswer)
    {
        return RedirectToAction("QuestionAndAnswer", new { userName = userName } );
    }
    else
    {
        MembershipService.ResetPassword(userName, GetLoginUrl());
        return RedirectToAction("PasswordResetFinal", new { userName = userName });
    }
}

Password Question and Answer

If the Membership Provider is configured to require a question and answer before resetting a user's password then we route users to the QuestionAndAnswer view. This view is also very simple as it merely has two labels (one with the username and another with password question for the user) and a textbox where the user will enter the answer to their password question.

To support this QuestionAndAnswerView I implemented an HTTP-GET controller method that fetches the security question for the username entered in the PasswordReset view.

public ActionResult QuestionAndAnswer(string userName)
{
    if (!MembershipService.PasswordResetEnabled) throw new Exception("Password reset is not allowed");
    ViewData["UserName"] = userName;
    ViewData["Question"] = MembershipService.GetUserQuestion(userName);
    return View();
}

Finally I added an HTTP-POST method to support the QuestionAndAnswer. By the time we get to this HTTP-POST method we have all the information that we need to reset a user's password (namely the user name and the answer to the security question.) Hence this method is where the call to actually reset the user's password actually happens.

[HttpPost]
public ActionResult QuestionAndAnswer(string userName, string answer)
{
    if (!MembershipService.PasswordResetEnabled) throw new Exception("Password reset is not allowed");
    MembershipService.ResetPassword(userName, answer, GetLoginUrl());
    return RedirectToAction("PasswordResetFinal", new { userName = userName });
}

Password Reset Final

The last step in the process if a new view called PasswordResetFinal that just displays a message to the user telling him/her that a new password has been generated and e-mailed to them.

Changes to the Model

Every ASP.NET MVC project comes with a default model called AccountMembershipService to support the Views and Controllers that handle membership information. This AccountMembershipService is not much more than a wrapper for the ASP.NET MembershipProvider. Adding functionality to this model to support the password reset operation was very simple as the MembershipProvider already provides the core functions. The MembershipService referenced in the controller actions in the code actually point to an instance of this AccountMembershipService.

For example, the QuestionAndAnswer view calls the following method to to retrieve the security question for a user. Notice how this method's only job is to forward the calls to the Membership provider.

public string GetUserQuestion(string userName)
{
    MembershipUser user = _provider.GetUser(userName, false);
    if (user == null)
    {
        throw new Exception("User name not found");
    }
    else
    {
    return user.PasswordQuestion;
    }
}

All in all I added two methods to the AccountMembershipService (one to retrieve a user's security question and one to actually do the password reset) plus a few properties to expose a couple of features of the Membership provider (like the need for a security question) that were not exposed on the default implementation.

In Summary...

As I indicated at the beginning of this blog post, adding support for Password Recovery to an ASP.NET MVC project turned out to be pretty easy as the core of the functionality already exists in the Membership Provider.

You can download the source code of a brand new working ASP.NET MVC web site updated to support the password recovery functionality from here. Keep in mind that this code assumes that you already have the ASP.NET membership provider and its database configured in your system. The readme.txt file inside the download provides further instructions on how to run this sample.

Yield Return

hector@hectorcorrea.com

Even though the yield return statement was added to C# since version 2.0 (around 2004) I never quite understood how it really works. A few days ago while going through some tutorials I saw a sample of code that used yield return and decided to finally look under the covers and learn how it works.

How well do you understand yield return? Look at the code below and answer these two questions:

What is the output of this code?
Can you explain why?

class YieldSample

{

static void Main(string[] args)

{

foreach (int i in YieldSample.GetSomeIntegers())

{

Console.WriteLine("{0}", i);

}

Console.ReadLine();

}

public static IEnumerable<int> GetSomeIntegers()

{

yield return 100;

yield return 200;

yield return 300;

}

The answer to the first question is pretty simple: 100, 200, and 300.

The tricky part is explaining how come C# arrives to this output. How come C# knows that it needs to pick up the first value (100) in the first loop, 200 in the second, and 300 in the third one?

The reason for this behavior is that at compile time C# rewrites the code with the yield return into a state machine that has the logic to return the first value and prepare the code so that in the second call it picks up the second value, and so on and so forth. The pseudo-code for the generated code basically looks like this:

private bool GetNextElement()

{

switch ( state )

{

case 0: // first state of the machine

currentValue = 100;

state = 1; // prepare machine for next state

return true;

case 1: // second state of the machine

currentValue = 200

state = 2; // prepare machine for next state

return true;

case 2: // third state of the machine

currentValue = 300;

state = 3; // prepare machine for next state

return true;

case 3:

state = -1

return false;

}

While trying to understand how the yield return statement works I found two great articles that explain this in more detail. In particularly I liked Yield, and the C# state machine by Erik Forbes and C#: Dynamic Iterators with Yield Return by Paul Kimmel. These two articles actually gave me the clue to use Reflector to look at the code generated by the compiler.

If you are like me you are probably wondering how well this code generation works when the method is not as straightforward as this little example that just returns three hard coded values. The answer is pretty darn good. State machines have been studied by a lot of people and very well understood.

However, just for fun I wrote a few examples of my own with if and nested if statements around the yield return and reviewed the code that the compiler generated. The result was always the same: the resulting code returned the values as expected.