Motivation to write this article came from a colleague on one of the projects I am busy with these days. I am mentoring a couple of developers who have joined the project straight out of university. They are here to learn the ropes of Microsoft .NET centric software development. Yesterday during a design discussion we talked about serialization of objects in our system. They are well familiar with the concepts of serialization, I showed them how it is done in .NET world.

Serialization is a technique we use to convert the state of an object to a format which can be persisted for later retrieval. Last sentence perhaps sounded very bookish. Let’s try some plain English and understand the concept using an example. We will use an example which involves pizza.

We will write a class to represent a Pizza. The class has a property Slices to hold the number of slices. When pizza object is initialized, the number of slices are set to 8. Pizza class also has methods for eating a slice, saving the pizza for later consumption and a method to get leftover pizza. Here are the stubs for properties and methods.

public int Slices
public bool IsThereAnyPizzaLeftForMe

public void EatSlice()
public void SaveForLater()
public static Pizza GetLeftOverPizza()

We will now fill in the stubs for SaveForLater() and GetLeftOverPizza(). This is where we will serialize and deserialize our pizza object.

public void SaveForLater()
{
	XmlSerializer serializer = new XmlSerializer(typeof(Pizza));

	using (StreamWriter writer = new StreamWriter(@"c:\temp\pizza.xml"))
	{
		serializer.Serialize(writer,this);
	}
}

In the code above we are saying that serialize the pizza object to XML file. XmlSerializer comes with .NET framework and can be found in System.Xml.Serialization namespace. Objects can also be serialized to binary format. We can look at that some other day. For now let’s go with XML.

Here is the code from a console application which creates an instance of Pizza class, eats a slice and calls SaveForLater() to serialize the object state to XML.

static void Main(string[] args)
{
	Pizza pizza = new Pizza();
	Console.WriteLine("{0} slices left in the pizza", pizza.Slices);
	
	pizza.EatSlice();
	Console.WriteLine("{0} slices left in the pizza", pizza.Slices);

	pizza.SaveForLater();
}

Let’s look at the XML to which our pizza object was serialized.

<?xml version="1.0" encoding="utf-8"?>
<Pizza xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Slices>7</Slices>
</Pizza>

Note that it only saves Slices property. There is no need to save/serialize the IsThereAnyPizzaLeftForMe property because it’s value can be determined by checking if Slices is equal to zero. XmlSerializer is smart enough to know which data is absolutely required to deserialize an object. Also methods are behaviors which act upon data so there is no need to retain them.

GetLefOverPizza() method creates an instance of Pizza object from XML. To be technically correct, it deserializes a pizza object from XML.

public static Pizza GetLeftOverPizza()
{
	Pizza pizza = null;
	XmlSerializer serializer = new XmlSerializer(typeof(Pizza));

	using (FileStream fileStream = new FileStream(@"c:\temp\pizza.xml", FileMode.Open))
	{
	pizza = (Pizza)serializer.Deserialize(fileStream);
	}

	return pizza;
}

Another console application shows retrieving a left over pizza, eating a slice and then again saving it for later.

static void Main(string[] args)
{
	Pizza pizza = Pizza.GetLeftOverPizza();
	Console.WriteLine("{0} slices left in the pizza", pizza.Slices);

	pizza.EatSlice();
	Console.WriteLine("{0} slices left in the pizza", pizza.Slices);

	pizza.SaveForLater();
	
}

So there you have it. A simple example which shows how to serialize and deserialize an object in .NET Framework.

Thanks also to Andrew Coates and Dave Glover for their support and feedback.

Link to my app: Visuals

It turns out that all the rules can be disabled easily by right clicking the project –> StyleCop Settings. Then in the Rules tab by unchecking Enabled rules.

The solution:

Right click on the solution and click Enable NuGet Package Restore.

This option saved many hours of cannot-compile frustration.

Code for this post is available on Github.

The UI for this app is very simple. It contains just one Image control and a button

<StackPanel x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">
  <Image x:Name="ImageChosen" />
  <Button x:Name="ButtonChoosePhoto"
    Content="choose photo"
    Click="ButtonChoosePhoto_OnClick"/>
</StackPanel

In the OnClick event handler I instantiate a PhotoChooserTask object, hook it with an event handler when Completed event is fired and finally call Show method on PhotoChooserTask.

private void ButtonChoosePhoto_OnClick(object sender, RoutedEventArgs e)
{
  PhotoChooserTask photoChooser = new PhotoChooserTask();
  photoChooser.Completed += photoChooser_Completed;
  photoChooser.Show();
}

Within the event handler I place this code. The code checks if the TaskResult is OK and then creates a new BitmapImage object from the stream in PhotoResult object which is supplied as an argument to the handler.

void photoChooser_Completed(object sender, PhotoResult e)
{
  if (e.TaskResult == TaskResult.OK)
  {
    BitmapImage bitmap = new BitmapImage();
    bitmap.SetSource(e.ChosenPhoto);
    ImageChosen.Source = bitmap;
  }
}

Here is the app in working.

What about the camera?

What if the photo I am interested in is not on the phone and I’d like to use the camera. That’s very easy. PhotoChooserTask has an boolean property ShowCamera. I can set it to true to get an option to use camera.

private void ButtonChoosePhoto_OnClick(object sender, RoutedEventArgs e)
{
  PhotoChooserTask photoChooser = new PhotoChooserTask();
  photoChooser.Completed += photoChooser_Completed;
  photoChooser.ShowCamera = true;
  photoChooser.Show();
}

Easy, isn’t it?

Link

Rather than right clicking the shortcut every time and another click to run as administrator, I found a permanent solution. For the lazy me, it saves two extra clicks.

Open properties of the shortcut to Visual Studio 2012, go to Shortcut tab.

Click on Advanced and then check Run as administrator checkbox.

My first reaction was to execute aspnet_regiis –i command. That did not work.

This was a WTF moment. It turns out that I should be running Visual Studio 2012 as an administrator. After running Visual Studio 2012 as admin, the error went away and life once again seemed a bit normal.

As you saw we implemented our map and reduce methods in their own classes by extending Mapper and Reducer class from Hadoop framework. It turns out that there is a way to implement a word count example without doing that. Hadoop framework already ships with two classes which can be used as our mapper and reducer. They are TokenCounterMapper from org.apache.hadoop.mapreduce.lib.map package and IntSumReducer from org.apache.hadoop.mapreduce.lib.reduce package.

Here is a revised word count which uses built-in TokenCounterMapper and IntSumReducer classes.

package com.thereforesystems.hadoop;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.map.TokenCounterMapper;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.reduce.IntSumReducer;
import org.apache.hadoop.util.GenericOptionsParser;

public class HadoopWordCountRevised {

    public static void main(String[] args) throws Exception {
        
        Configuration config = new Configuration();
        String[] otherArgs = 
            new GenericOptionsParser(config, args).getRemainingArgs();
        
        Job job = new Job(config, "Word Count Tutorial");
        job.setJarByClass(HadoopWordCountRevised.class);
        job.setMapperClass(TokenCounterMapper.class);
        job.setReducerClass(IntSumReducer.class);
        
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);
        
        FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
        FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
        
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}

For this walkthrough if you want to call it that, I have Hadoop running on a single node setup on Ubuntu 11.10. My preferred IDE is Netbeans.

Here it goes.

Create a project

First of all create a Java project in Netbeans. Call it HadoopWordCountTutorial. I also like to use proper package names so my class HadoopWordCountTutorial is in package com.thereforesystems.hadoop.

Add Libraries

Next thing we need to do is add some libraries. Here is a list of libraries required to compile our Hadoop project.

These jars can be found in your Hadoop folders. An easy way to find where things are is by using locate command. For example to locate hadoop-core-0.20.2-cdh3u4.jar execute the command in terminal.

locate hadoop-core-0.20.2-cdh3u4.jar

On my machine the file is located in

/usr/lib/hadoop-0.20/

Once we have added required libraries, we are all set to write some code.

Writing code

Hadoop is a framework which provides us plumbing to write MapReduce operations (This is such an understatement). Here is a good tutorial on MapReduce. If you are not familiar with MapReduce then I suggest that you read it before continuing with this tutorial.

There are two operations we will write. One is the mapper and the other is reducer. Our objective is to count words in a file or many files and write the results to an output location. We will start with our mapper.

Mapper

Mapper in Hadoop is implemented by extending Mapper class found in org.apache.hadoop.mapreduce. This class implements a map method in which we will write our logic. Here is the code for our class.

public static class WordCountMapper extends Mapper