Laserbrain — a software blog

Async singleton initialization

Sun, 22 Nov 2015 13:00:00 +0200

Still in these days you once in a while have use for a singleton object. But how do you achieve a thread-safe singleton that requires async initialization?

First, let's start off with a good ol' singleton class that can be constructed synchronously.

Synchronous Singleton

Assuming your singleton class needs some initialization data that it can construct itself (synchronously). To make it thread-safe, we rely on locking on a static read-only object.

public class MySingleton
{
    private static readonly object SyncObj = new object();   
    private static MySingleton _singleton;

    /// <summary>
    /// The public available singleton instance.
    /// </summary>
    public static MySingleton Singleton
    {
        get
        {
            if (_singleton == null)
            {
                lock (SyncObj)
                {
                    if (_singleton == null)
                    {
                        _singleton = CreateSingleton();
                    }
                }
            }

            return _singleton;
        }
    }

    /// <summary>
    /// The private instance-constructor, taking some data.
    /// </summary>
    private MySingleton(SomeData someData)
    {
        SomeData = someData;
    }

    /// <summary>
    /// Some data that the singleton exposes.
    /// </summary>
    public SomeData SomeData
    { get; }

    /// <summary>
    /// Create the singleton instance.
    /// </summary>
    private static MySingleton CreateSingleton()
    {
        SomeData someData = CreateSomeData();
        return new MySingleton(someData);
    }

    /// <summary>
    /// A private method that can create some data, called once.
    /// </summary>
    private static SomeData CreateSomeData()
    {
        // ...some way to create some data...
    }
}

In the version above the Singleton property is designed to work as quick as possible, only entering the lock if there is a chance that the class hasn’t yet been initialized. Within the lock a method that creates the MySingleton object is called. This way it's ensured that only one instance of the MySingleton is ever created.

Asynchronous Singleton

But what to do if the CreateSomeData method would work asynchronous, returning a Task<SomeData> instead, using the await keyword? Let's modify the code in this way:

private static async Task<SomeData> CreateSomeData()
{
    // ...some way to create some data, using the await keyword...
}

Making this change, we should play nice and propagate the async’ness to the CreateSingleton method:

private static async Task<MySingleton> CreateSingleton()
{
    SomeData someData = await CreateSomeData();
    return new MySingleton(someData);
}

That would naturally imply that the Singleton property should also be asynchronous, returning a Task<MySingleton> instead.

So, is that possibly?

First of all, it turns out that it’s not allowed to use the async keyword with a property. But this is no biggie, we could afford a method instead.

But more importantly, we are not allowed to use a lock around an await keyword. (This is actually a good thing, since the lock would really work against us here – so we should thank the C# team for not allowing this!)

So how do we do this then? As a starter, a naïve, non thread-safe solution would look like this:

public static Task<MySingleton> Singleton
{
    get { return CreateSingleton(); } // WOHAA! NOT SINGLETON AT ALL!
}

Requesting the Singleton property would of course lead to a call to the CreateSingleton method every time, and would hardly be a singleton...

But the solution to make it both a singleton and thread-safe turns out to be ridiculously simple – we let the inner mechanisms of the Task class work for us!

So, how does a task work?

Let’s say you have an instance of a Task<T> and you await it once. Now the task is executed, and a value of T is produced and returned to you. Now what if you await the same task instance again? In this case the task just returns the previously produced value immediately in a completely synchronous manner.

And what if you await the same task instance simultaneously from multiple threads (where you would normally get a race condition)? Well, the first one (since there will be one that gets there first) will execute the task code while the others will wait for the result to be processed. Then when the result has been produced, all the await’s will finish (virtually) simultaneously and return the value.

So, a Task is thread-safe, and it looks as if we could use this power in our advantage here!

In the end, all we need is to replace the old Singleton property with this one:

public static Task<MySingleton> Singleton { get; } = CreateSingleton();

Or, rewritten in a more classic C# way (without making use of the fancy new C# 6 “read-only properties” and “property initializers”):

private static readonly Task<AsyncSingleton> CreateSingletonTask = CreateSingleton();

public static Task<AsyncSingleton> Singleton
{
    get { return CreateSingletonTask; }
}

So the dead simple solution is to assign the async method to a read-only (static) Task field (or read-only property in C# 6). This gives you both a singleton and thread-safety for free!

Lazy Synchronous Singleton

Being empowered with the super-simple async version, you might wonder if there really is no similar way to achieve this in the synchronous version?

Of course there is!

Revisiting the synchronous first version of the MySingleton in this blog post, you may replace the _SyncObj and _singleton fields and the Singleton property with these two lines:

private static readonly Lazy<MySingleton> LazySingleton = new Lazy<MySingleton>(CreateSingleton);

public static MySingleton Singleton => LazySingleton.Value;

Or, if you prefer the pre-C# 6 code:

private static readonly Lazy<MySingleton> LazySingleton  = new Lazy<MySingleton>(CreateSingleton);

public static MySingleton Singleton
{
    get { return MySingleton.Value; }
}

Wow, magic! No need for locks or anything! So how does this work?

Well, the Lazy<T> can be seen as the synchronous counterpart to the Task<T> in some aspects. At least it has the same singleton and thread-safety properties, and keeps its results for latecomers.

As you can see, the constructor of the Lazy<MySingleton> is given the means of producing a MySingleton (i.e. the CreateSingleton method) – not an instance of the type directly. Then, when someone is requesting the Singleton property, the Value of the Lazy-class is accessed. And in the same manner as the Task, the first one reaching for the Value will make the actual call to the CreateSingleton method. Any other thread asking at the same time will simply be hanging in the Value getter, and continues once the CreateSingleton method is done and the Value is produced. Further on, any consecutive calls to the Value getter will return immediately with the same instance of the MySingleton. So, again, the Lazy gives us both a singleton and thread-safety.

And there was much rejoicing!

Exporting a certificate without its private key and password-protect the output? Beware, there is a serious trap!

Tue, 10 Nov 2015 19:00:00 +0200

So you have an instance of an X509Certificate2 (or X509Certificate) that you want to export as a byte array – and you want to exclude the private key – and encrypt the output using a password.

You have found the Export method of the certificate class which takes one of the X509ContentType enum values and an optional password. The MSDN help informs that you must choose between X509ContentType.Cert, X509ContentType.SerializedCert and X509ContentType.Pkcs12 for the export to work. You also find out (by experimenting or googling) that exporting using X509ContentType.Cert produces a serialized certificate without the private key – just what you want! Hooray!

Now you specify a password and think you’re OK.

Fail. You are not. The password is actually ignored.

If you try this:

byte[] a = cert.Export(X509ContentType.Cert);
byte[] b = cert.Export(X509ContentType.Cert, "P@ssw0rd #1");
byte[] c = cert.Export(X509ContentType.Cert, "P@ssw0rd #2");

The resulting byte arrays a, b and c will have the exact same content, even though you specified different passwords! (And it behaves the same if you use the SecureString class.)

Personally I would expect the Export method to throw an exception when specifying X509ContentType.Cert together with a password (other than null). That would give me, as a developer, a clear sign that I am trying to use an unsupported parameter combination which gives me a chance to try to figure out a work-around. As it is now I am lead to believe that the output content is in fact encrypted.

It is also possible to recreate the certificate again from the byte array giving any password:

var certX = new X509Certificate2(a, "P@ssw0rd #2");
var certY = new X509Certificate2(b, "correct horse battery staple");

Both certX and certY above will be correctly reconstructed.

Here is a simple solution you can use to export a certificate without its private key and encrypt the exported bytes:

byte[] ExportCertificate(X509Certificate certificate, string password, bool includePrivateKey)
{
     if (!includePrivateKey)
     {
           // Export the certificate (temporarily) using the content type "Cert".
           // The exported cert will NOT include the private key -- but it will not
           // be encrypted (the given password will be ignored). So we are not there yet.
           byte[] exportedWithoutPrivateKey = certificate.Export(X509ContentType.Cert, "");
 
           // Now recreate the certificate again from the exported bytes.
           // The recreated certificate will not contain a private key.
           certificate = new X509Certificate2(
                exportedWithoutPrivateKey,
                "", // The (ignored) password
                X509KeyStorageFlags.Exportable | X509KeyStorageFlags.PersistKeySet);
     }
 
     // Export the certificate using the PKCS #12 format.
     // This content type will include the private key among the exported bytes (if there is one).
     return certificate.Export(X509ContentType.Pkcs12, password);
}

Now calling this method, specifying two different passwords and asking not to include the private key...

byte[] d = ExportCertificate(cert, "P@ssw0rd #1", false);
byte[] e = ExportCertificate(cert, "P@ssw0rd #2", false);

...generates two byte arrays d and e that are different. Further on, if you try to recreate it you must specify the correct password.

var certZ = new X509Certificate2(d, "P@ssw0rd #1");
var fails = new X509Certificate2(d, "correct horse battery staple");

The certZ will be correctly reconstructed, but the second try (with wrong password) will throw a CryptographicException with the message “The specified network password is not correct.”

Best practices of the Dictionary class

Sun, 09 Feb 2014 15:50:00 +0200

It’s important to know how the standard classes behave so that your code gets optimized, not performing unnecessary checks or making unnecessary calls.

Here's a number of issues I've seen time after time:

Fetching a value from the dictionary only if it exists

In the first version (bad) there are two dictionary lookups performed. It’s like walk in to a public library asking the librarian whether a certain book is in – and when the librarian comes back and says “yes, it’s in”, you ask again saying "Good, could you please get it for me?". In real life it's pretty obvious that this is a silly behavior, but in the computer world it's not that apparent.

if (myDict.ContainsKey(key)) // First lookup.
{
	ValueType value = myDict[key]; // Second lookup.
	// ... use the "value" here ...
}

:-(

In the second version (good), only one lookup is performed:

ValueType value;
if (myDict.TryGetValue(key, out value))
{
	// ... use the "value" here ...
}

:-)

Accessing all the keys and values

I've seen many real-life cases where access to both the key and value is needed in a foreach-loop, and the iteration is performed over the Keys-properties while the value is accessed by a lookup:

foreach (TKey key in myDict.Keys)
{
	TValue value = myDict[key];
	// ... use the "key" and "value" here ...
}

:-(

If you want both the key and the value simultaneously it’s better to iterate over the dictionary as KeyValuePairs:

foreach (KeyValuePair<TKey, TValue> keyValuePair in myDict)
{
	TKey key = keyValuePair.Key;
	TValue value = keyValuePair.Value;
	// ... use the "key" and "value" here ...
}

:-)

(Obviously; if you want only the keys you should use the Keys-property, and if you want only the values use the Values-property)

Overwriting a value

if (myDict.ContainsKey(key)) // First lookup.
{
	myDict.Remove(key); // Second lookup.
}
myDict.Add(key, value); // Third lookup.

:-(

There is really no need to remove a key before assigning a new value to it:

myDict[key] = value;

:-)

Removing a value

Once again, the following implementation leads to two lookups:

bool entryExists = myDict.ContainsKey(key); // First lookup.
if (entryExists)
{
	myDict.Remove(key); // Second lookup.
}
return entryExists;

:-(

The Remove-method is actually very kind. It doesn't crash if you try to remove something that is not there but returns a bool telling the result of the operation:

return myDict.Remove(key);

:-)

When may my call fail?

The call...	...fails when
myDict[key] = value;	Never ()* The value will be added if it's not already in the dictionary, and it will be replaced if it does.

var value = myDict[key];	When the entry “key” does not exist.

myDict.Add(key, value);	When the entry “key” already exists.

myDict.TryGetValue(key, out value);	Never ()*

myDict.Remove(key);	Never ()* (And the call returns false if the entry “key” does not exist.)

(*) Well, you get an ArgumentNullException if the key is null...

Fixing the assembly hopping between WP7 and WP8

Sun, 28 Apr 2013 16:47:00 +0200

If you want to develop an Windows Phone app that can run on both WP7 and WP8 you can simply submit one built for Windows Phone 7. But if you want to take advantage of some of the new feature of Windows Phone 8 you will need to submit two versions of it (one for 7 and one for 8). This means that you will have to maintain two copies of the project file -- but you obviously don't want two copies of each source code file.

This can be solved by putting the code in a Portable Class Library and then reference it by the two projects. There are however some limitations using a Portable Class Library, and you might find yourself stuck between a rock and a hard [coded] place trying to use it.

Each time I was facing the dual project problem, I ended up with using linked source code files instead, i.e. making one of the projects the "main" project, and then simply linking all the source code files into the other project (i.e. "Add As Link" rather than "Add" when adding "Existing Item..."). An annoying problem with this solution is however that each time you rename a source file you will need to remove and add the linked file again. Aaarrgghh! :[ But for me this actually was less painful than using the Portable Class Library.

Anyway, this way you will only have to maintain one copy of the code files that are identical on both platforms. For the ones that differs a bit, you either use compiler switches (e.g. "#if WP7 [...] #endif") -- or totally different classes (i.e. non-linked code, if there are "too much" differences to make use of #if's).

But when it comes to XAML you are in more trouble. For instance, if you are sharing the same implementation of a page in both WP7 and WP8, you will notice that the home of the Pivot (and Panorama) has changed between the two platforms. Ouch.

In WP7 your page declaration looks like this:

xmlns:xxx="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone.Controls"

but your WP8 projects want this line:

xmlns:xxx="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"

As you can see above, the assembly name has changed.

This makes you yearn for the following (illegal) construction:

<phone:PhoneApplicationPage 
    x:Class="Wp7App.PivotPage1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
#if WP7
    xmlns:xxx="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone.Controls"
#elif WP8
    xmlns:xxx="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"
#endif
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

But unfortunately compiler switches are not allowed in XAML... So the above will NOT compile. :(

There is however a pretty simple solution to this. Create the following two classes:

namespace MyShared
{
	using Microsoft.Phone.Controls;

	public class MyPivot : Pivot
	{
	}

	public class MyPivotItem : PivotItem
	{
	}
}

This file can be linked across the two platforms intact without any need of #if's (since it wasn't the namespace that was changed, "just" the assembly).

Now you simply use the MyPivot wrapper classes (instead of the Pivot directly) in your XAML, and you're good to go:

<Phone:PhoneApplicationPage 
    x:Class="MyShared.PivotPage1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
	xmlns:MyShared="clr-namespace:MyShared"
    xmlns:Phone="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone">

    <Grid>
        <MyShared:MyPivot Title="MY APPLICATION">
            <MyShared:MyPivotItem Header="item1">
                <Grid/>
            </MyShared:MyPivotItem>
            <MyShared:MyPivotItem Header="item2">
                <Grid/>
            </MyShared:MyPivotItem>
        </MyShared:MyPivot>
    </Grid>
    
</Phone:PhoneApplicationPage>

Running a "Hello World" written in C# on a Raspberry Pi for Linux n00bs like me

Wed, 21 Nov 2012 20:57:00 +0200

A while ago I found myself in the possession of a Raspberry Pi.

O the joy of a new toy! But what to do with it?

Well, being a C# developer of course I would like to be able to install and run my own applications on it — and I wanted to do that remotely from my regular laptop.

The biggest problem was that I have no Linux skills what so ever. I've been living a comfortable (well, hm... you know...) Windows life ever since the days of the Amiga and Commodore 64. But how hard could it be?

The following are the steps I had to go through to make it work. It's a hodgepodge of wisdoms found all over the internet. So; disclaimer: It worked for me, but I can't promise it will work for you!

Here we go...

Step 1. Formatting the SD card

To do this properly you'll need to use the SD Formatter 3.1 application.

Install it and launch it having your SD card inserted. Simply press the Format button. It's enough to just do a quick format.

Important note!

If you are repeating this step later on, your SD card might appear to have lost a few gigabytes, only showing something like 56MB! This is due to that Windows cannot see the Linux partitions, and the 56MB is just some left-overs that Linux didn't claim.

You have at least a couple of options to reclaim the full space in Windows:

Find yourself a [video]camera or any other non-windows device and try to format it in there. I first tried my age-old compact camera, but it just said that the card was bad. Then I tried my somewhat newer video camera, and it formatted it nicely and I got back my gigabytes. Phew...
Use the dreaded (?) Flashnul (blog post) tool. I haven't tried this one since my first option worked for me. Be careful...

Step 2. Installing a Linux distribution on the freshly formatted SD card

I used the Raspbian “wheezy” image. (I was somehow drawn to its tag-line "If you’re just starting out, this is the image we recommend you use".)

To write the image to the card you'll need the Image Writer for Windows application. There is no installation required; just download and launch the Win32DiskImager.exe. Launch it, click the blue little folder-button and point out the image file. Then press the Write button to start writing. This will take a few moments, depending on the size and speed of your SD card.

Step 3. First launch

After preparing the SD-card it's time for the first launch.

Insert the SD card into the Pi and plug in an ethernet cable (with internet access), a USB keyboard and a monitor/tv (hdmi). Finally plug in the micro USB power adapter to boot it up. You will see a whole lot of text burst out on the screen. You do not need to read it all. :)

After a while a classic ASCII-artish gray popup appears, entitled Raspi-config.

Here you can for instance set your keyboard layout. When you're done, use the tab key to move to the Finish button, then press enter to access the console.

This popup is shown only the very first launch time. The following times you start the Pi you will need to enter a user name and password to access the console. The default user name is "pi" and the password is "raspberry".

Step 4. Optional: Setting the keyboard locale

If you didn't change the keyboard layout in that first popup (which only will show up upon the first launch), you can do it using the following command:

sudo nano /etc/default/keyboard

Where 'sudo' means 'run this command as super user', 'nano' is a text editor and '/etc/default/keyboard' is the file to edit.

Now the text editor appears, and you may change the 'XKBLAYOUT' to whatever you like. I chose 'se' for Sweden.

Save with Ctrl + O (!) and exit with Ctrl + X.

Reboot. This can be done by pulling the power cord or by issuing the command

sudo reboot

Step 5. Set a fixed IP-address

Since you want to remotely connect to your Raspberry Pi you might want to give it a fixed IP-address. This step is of course optional.

If you want to know the current network settings on your Pi, execute the following command:

ifconfig

(No, that is not a typo. It should be an 'f', not a 'p'...) You will see the current IP-address under the section 'eth0', on the line starting with 'inet'. (The MAC address is found after the word 'HWaddr'.)

Edit the network settings by executing:

sudo nano /etc/network/interfaces

When the editor appears, replace the line

iface eth0 inet dhcp

with these lines (the #-mark means that the line is commented out.)

#iface eth0 inet dhcp
iface eth0 inet static
address 192.168.1.200
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

Obviously you will need to fill in your preferred numbers here...

After saving the file you'll need to reboot. The best thing is of course to check your new settings by executing the ifconfig-command again, after rebooting.

Step 6. Ensure the SSH service is started at boot time

This is necessary for connecting remotely. It was however already active on the image I used, but I'll keep this step here anyway.

The command to execute is

sudo mv /boot/boot_enable_ssh.rc /boot/boot.rc

which means to rename/move the file/directory specified. I got the following error message:

mv: cannot stat `/boot/boot_enable_ssh.rc': No such file or directory

which I interpreted that the SSH service was already active for me.

Step 7. Connect remotely from your favorite Windows PC

Download PuTTY.

There's no installation here, just an exe-file. Launch it and fill in the IP-address to your Pi (and the port, which defaults to 22) and connect with the Open button.

You will get a warning of a potential security breach the first time you connect, because your PuTTY instance (obviously) does not recognize your Pi.

I guess it is quite harmless to press the Yes button here.

Now the Pi console window greets you by asking for your username (pi) and password (raspberry). You're in!

Step 8. FTP

In addition to execute command on your Pi you will also need to transfer files to and from it. Use your favorite FTP client, for instance FileZilla. Simply connect to the correct IP address, port 22, user "pi" and password "raspberry".

Step 9. Install the C# compiler and .net exe-executer

To be able to compile and execute C#-code on the Raspberry Pi, you will need to install Mono.

This turns out to be pretty simple. Just execute the following command to install the dmcs (4.0 mscorlib) C# compiler:

sudo apt-get install mono-dmcs

This will take a while to process, the files requested will be downloaded from the internet and installed by some magic Linux fairy. Please note that in the beginning you will get a "Do you want to continue"-question. After saying Y + enter you might want to get a quick cup of coffee (or a beer since this is most probably done off-business hours). It will take a couple of minutes.

Step 10. Compile and run your program.

Now, I wanted the full monty and even compile the C# code on the raspberry. So, in order to make it as simple as possible, I put all the code into one file (HelloPi.cs):

namespace Test
{
	using System;

	public class HelloPi
	{
		public static void Main(string[] args)
		{
			Console.WriteLine("Hello World!");
			Console.ReadKey();
			Console.WriteLine("End.");
		}
	}
}

Using the FTP client, transfer your 'HelloPi.cs' source file to a suitable folder. Compile the source file with the dmcs-command:

dmcs HelloPi.cs

Since it's not a very large application, it will be compiled pretty much instantly. Launch it with the mono-command:

mono MyApp.exe

Joy to the world, my app has launched!

Important note!

You might as well do the compilation on your PC (which is Much. More. Convenient.) and just transfer the exe-file to the Pi instead.

Some useful links

An A-Z Index of the Bash command line for Linux.
Penguin Tutor - Some Linux Tutorials.
Accessing the GPIO on the Pi board using C#.
The MagPi - A Magazine for Raspberry Pi Users.

Good luck!

Announcing the Diversify app for Øredev 2012

Wed, 31 Oct 2012 21:02:00 +0200

A collegue of mine, Fredrik Mörk, noticed the announcement of the Øredev 2012 mobile app contest, and soon a team was formed that happily started hacking away (hosting the code in a Mercurial repository on BitBucket, and also using Trello to have some sort of control of who was what, and what needed to be done). The team consisted of Micael Carlstedt, Niclas Carlstedt, Fredrik Mörk, Markus Wallén, Sebastian Johansson and myself.

The app is supposed to function as a companion before and during the conference. The main functionality is that it offers you to easily navigate and explore in the conference program, and build your own personalized conference program by “favoriting” sessions. It also features a twitter feed, listening to stuff related to the conference and the app. One focus of the design has been to allow you to explore. From almost every view, there is a way to move on and get information about related stuff. One example is when you are looking at a session, from this view you can in one touch navigate to

a page highlighting the room in which the session is on a map
a page showing all sessions in that room during the conference
a page showing details about the speaker (OK, this typically requires two interactions; scroll to the bottom of the page and then tap the speaker)
a page showing details about each other session running in other rooms at the same time
a page showing all sessions for a topic that this session also has

This may be the most extreme example, but it’s not unique in its concept.

One other thing that the app does is that it invites to sharing information about the conference. You can (again with very few interactions) share info about a session or speaker to social networks.

There is an update submitted to the market place which will add a couple of features (tell the app what days you will attend and it will filter the data throughout the app accordingly and, as a personal touch, the app authors’ suggestions for some sessions we find extra interesting). If you happen to have an unlocked developer phone, you can grab the XAP file for that update here. Otherwise, go ahead and get the app from the Windows Phone Marketplace.

(Thank you Fredrik, for allowing me to steal your words from your blog post. )

Some screen shots (click for full size):

Rendering the audio captured by a Windows Phone device

Sun, 11 Mar 2012 17:11:00 +0200

This small app displays the audio captured by the microphone of a Windows Phone device and displays it as a continous waveform on the screen using the XNA framework.

A slightly modified version of the app (changing color when touching the screen) can be found in the Windows Phone Marketplace.

Background

In order to capture the audio on a Windows Phone device, you need an instance to the default microphone (Microphone.Default), decide how often you want samples using the BufferDuration-property and hook up the BufferReady-event. Then you control the capturing with the Start() and Stop() methods.

The microphone is giving you samples at a fixed rate of 16 000 Hz, i.e. 16 000 samples per second. There is a property SampleRate that will tell this value. This means that you won't be able to capture audio of higher frequency than 8000 Hz (without distortion) according to the sampling theorem.

You are also limited when it comes to choose the value for the BufferDuration-property; it must be between 0.1 and 1 seconds (100 - 1000 ms) in 10ms-steps. This means that you must choose a value of 100, 110, 120, ..., 990, 1000 milliseconds.

When the microphone event BufferReady is fired, you should call the microphone.GetData(myBuffer)-method, in order to copy the samples from the microphone's internal buffer to a buffer that belongs to you. The recorded audio comes in the form of a byte-array, but since the samples are actually signed 16-bits integers (i.e. an integer in the range of -32'768 ... 32'767), you will probably need to do some convertion before you can process them.

How this application works

The way this application works is keeping a fixed number of narrow images, here called "(image) slices", arranged in a linked list. The images are rendered on the screen and smoothly moved from the right to the left. When the left-most slice has gone off the screen, it is moved to the far right (still outside the screen) in order to create the illusion of an unlimited number of images.

Each slice holds the rendered samples from the content of one microphone buffer. When the buffer is filled by the microphone mechanism, the rightmost slice (outside of the screen) is rendered with these new samples and started to be moved inwards the screen.

The speed of how fast the slices are moving across the screen is correlated to the duration of the buffer in such a way that the slices are moved a total of "one slice width" during the time the microphone is capturing the next buffer.

Since the buffer of captured audio is rendered as graphic on a texture as soon it is received, there is no reason to keep any old buffer data. Therefore the application only keeps one buffer in memory which is reused over and over.

A flag is set each time the microphone buffer is ready. Since the BufferReady event is fired on the main thread, there is no need for any lock-mechanism.

In the Update()-method of the XNA app, the flag is checked whether new data has arrived, and if so, the slice in line is drawn. In the Draw()-method, the slices are drawn on the screen and slightly moved as time goes by.

The complete Visual Studio solution file can be downloaded from here.

Here's a description of the structure of the main "Game"-class.

Some constants:

// The size of the screen.
private const int LandscapeWidth = 800;
private const int LandscapeHeight = 480;
 
// The number of milliseconds per time slice.
private const int SliceMilliseconds = 100;

Fields regarding the microphone and the captured data:

// The microphone and the sample data.
private readonly Microphone microphone;
private readonly byte[] microphoneData;

// The time it takes for a sample to pass over the screen.
private readonly TimeSpan screenMilliseconds = TimeSpan.FromSeconds(5);

Choose a color that is almost transparent (the last of the four parameters; it's the red, green, blue and alpha-component of the color). The reason is that many samples are drawn on top of each other, and keeping each individual sample almost see-through makes an interesting visual effect.

// The color of the samples.
private readonly Color sampleColor = new Color(0.4f, 0.9f, 0.2f, 0.07f);

The drawing classes. The white pixel texture is doing all the drawing.

// The sprite batch and a white single-pixel texture for drawing.
private SpriteBatch spriteBatch;
private Texture2D whitePixelTexture;

The size of each image slice.

// The size in pixels of one time slice.
private int imageSliceWidth;
private int imageSliceHeight;

There's no need to keep a reference to the linked list itself; just the first and last link. These links keeps references to their neighbors. The currentImageSlice is the one to draw on the next time.

// The first, last and current image slices.
private LinkedListNode<RenderTarget2D> firstImageSlice;
private LinkedListNode<RenderTarget2D> lastImageSlice;
private LinkedListNode<RenderTarget2D> currentImageSlice;

The speed of the slices moving across the screen.

// The current speed of the samples.
private float pixelsPerSeconds;

In order to know how far the current samples should be moved, the application must keep track of when they appeared.

// The time in seconds when the current microphone data appeared.
private float microphoneDataAppearedAtSeconds;

The signal that tells the Update()-method that there is new data to handle.

// A flag telling whether new data has been read.
private bool hasNewMicrophoneData;

The density of samples per pixel.

// The number of samples squeezed into the width of one pixel.
private int samplesPerPixel;

Here's the constructor. In it the graphics mode is set and the microphone is wired up and asked to start listening.

public Waveform()
{
    // Set the screen mode.
    new GraphicsDeviceManager(this)
    {
        PreferredBackBufferWidth = LandscapeWidth,
        PreferredBackBufferHeight = LandscapeHeight,
        IsFullScreen = true,
        SupportedOrientations =
            DisplayOrientation.Portrait |
            DisplayOrientation.LandscapeLeft |
            DisplayOrientation.LandscapeRight
    };

    // Standard setup.
    Content.RootDirectory = "Content";
    TargetElapsedTime = TimeSpan.FromTicks(333333);
    InactiveSleepTime = TimeSpan.FromSeconds(1);

    // Refer to the default microphone and hook the BufferReady-event.
    microphone = Microphone.Default;
    microphone.BufferReady += MicrophoneBufferReady;

    // Set the buffer duration to the length of one slice.
    microphone.BufferDuration = TimeSpan.FromMilliseconds(SliceMilliseconds);

    // Calculate the size in bytes of the sound buffer and create the byte array.
    var microphoneDataLength = microphone.GetSampleSizeInBytes(microphone.BufferDuration);
    microphoneData = new byte[microphoneDataLength];

    // Start listening.
    microphone.Start();
}

In the XNA's LoadContent nothing is actually loaded since the app is not dependent on any predrawn images. The SpriteBatch is created, the white pixel texture is generated and the image slices are initialized (as black images).

protected override void LoadContent()
{
    // Create a SpriteBatch for drawing.
    spriteBatch = new SpriteBatch(GraphicsDevice);

    // Create a 1x1 texture containing a white pixel.
    whitePixelTexture = new Texture2D(GraphicsDevice, 1, 1);
    var white = new[] { Color.White };
    whitePixelTexture.SetData(white);

    // Create the image slices.
    CreateSliceImages();
}

The CreateSliceImages is calculating how many slices that are needed to cover the entire screen (plus two so there's room for movement). In the end of the method the regular RenderSamples-method is called in order to initial all the images. Since there is no data yet (all samples are zero) it will generate black images.

private void CreateSliceImages()
{
    // Calculate how many slices that fits the screen (rounding upwards).
    var imageSlicesOnScreenCount = (int)Math.Ceiling(screenMilliseconds.TotalMilliseconds / SliceMilliseconds);

    // Calculate the width of each slice.
    imageSliceWidth = (int)Math.Ceiling((float)LandscapeWidth / imageSlicesOnScreenCount);

    // Set the height of each slice to the largest screen size
    // (this way the full height is utilized in Portrait mode without stretching)
    imageSliceHeight = LandscapeWidth;

    // Create a linked list with the required number of slices, plus two
    // so that there's room for scrolling off-screen a bit.
    var imageSlices = new LinkedList<RenderTarget2D>();
    for (var i = 0; i < imageSlicesOnScreenCount + 2; i++)
    {
        var imageSlice = new RenderTarget2D(GraphicsDevice, imageSliceWidth, imageSliceHeight);
        imageSlices.AddLast(imageSlice);
    }

    // Reference the first, last and current slice.
    firstImageSlice = imageSlices.First;
    lastImageSlice = imageSlices.Last;
    currentImageSlice = imageSlices.Last;

    // Calculate the speed of the pixels for an image slice.
    pixelsPerSeconds = imageSliceWidth / (SliceMilliseconds / 1000f);

    // Since the byte-array buffer really holds 16-bit samples, the actual
    // number of samples in one buffer is the number of bytes divided by two.
    var sampleCount = microphoneData.Length / 2;

    // Calculate how many samples that should be squeezed in per pixel (width).
    samplesPerPixel = (int)Math.Ceiling((float)sampleCount / imageSliceWidth);

    // Iterate through all the image slices and render with the empty microphone buffer.
    var slice = firstImageSlice;
    while (slice != null)
    {
        RenderSamples(slice.Value);
        slice = slice.Next;
    }
}

The XNA's UnloadContent is just cleaning up what the LoadContent created.

protected override void UnloadContent()
{
    // Dispose the SpriteBatch.
    spriteBatch.Dispose();

    // Dispose the white pixel.
    whitePixelTexture.Dispose();

    // Dispose all the image slices.
    var slice = firstImageSlice;
    while (slice != null)
    {
        slice.Value.Dispose();
        slice = slice.Next;
    }
}

The event handler to the microphone's BufferReady-event. It copies the data from the microphone buffer and raises the flag that new data has arrived.

private void MicrophoneBufferReady(object sender, EventArgs e)
{
    // New microphone data can now be fetched from its buffer.

    // Copy the samples from the microphone buffer to our buffer.
    microphone.GetData(microphoneData);

    // Raise the flag that new data has come.
    hasNewMicrophoneData = true;
}

The XNA's Update method checks the phone's Back-button to see if it's time to quit. After that it checks the flag to see if new data has been recorded. If so, the new samples are rendered by calling the RenderSamles-method.

 protected override void Update(GameTime gameTime)
{
    // Exit the app if the user presses the back-button.
    if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
    {
        Exit();
    }

    // If new data has been captured, a new slice should be drawn.
    if (hasNewMicrophoneData)
    {
        // Reset the flag.
        hasNewMicrophoneData = false;

        // Express the current point in time as "seconds passed since start of app".
        var currentSeconds = (float)gameTime.TotalGameTime.TotalSeconds;

        // Remember the current time as "when the mic data appeared".
        microphoneDataAppearedAtSeconds = currentSeconds;

        // Render the new samples on the current image slice.
        RenderSamples(currentImageSlice.Value);

        // Select the next image slice as the new "current".
        currentImageSlice = currentImageSlice.Next ?? firstImageSlice;
    }

    base.Update(gameTime);
}

The XNA's Draw-method takes care of drawing the rendered slices. It handles the two screen orientation modes; landscape and portrait, by scaling the images accordingly. If it is landscape mode the height of the images are squeezed and if it is portrait mode the width of the images are squeezed.

When all is setup, the method iterates through the images and render them one-by-one on the screen, adjusted a bit along the X-axis to make up for the time that has passed.

protected override void Draw(GameTime gameTime)
{
    // Clear the device. (Actually unnecessary since the whole screen will be painted below.)
    GraphicsDevice.Clear(Color.Black);

    // Calculate the "screen width-scale", to allow the app to be drawn both in
    // Landscape and Portrait mode.
    // In Landscape mode, the screenWidthScale will be 1.0 (i.e. original scale)
    // In Portrait mode, the screen must be squeezed.
    var screenWidthScale = (float)GraphicsDevice.Viewport.Width / LandscapeWidth;

    // Calculate the scaled width of one slice.
    var scaledWidth = (int)Math.Ceiling(imageSliceWidth * screenWidthScale);

    // Express the current point in time as "seconds passed since start of app".
    var currentSeconds = (float)gameTime.TotalGameTime.TotalSeconds;

    // Calculate how many seconds that has passed since the current microphone data was captured.
    var secondsPassed = currentSeconds - microphoneDataAppearedAtSeconds;

    // For a smooth move of the pixels, calculate the offset of the current microphone data
    // (where the offset is zero at the time of the new data arrived, and then growing up
    // one full width of a slice.
    var drawOffsetX = secondsPassed * pixelsPerSeconds;

    // Since it is not certain that the next microphone data will come before the current
    // slice has moved its full distance, the offset needs to be truncated so it doesn't
    // move too far.
    if (drawOffsetX > scaledWidth)
    {
        drawOffsetX = scaledWidth;
    }

    try
    {
        // Start draw the slices
        spriteBatch.Begin();

        // Start with one slice before the current one, wrap if necessary.
        var imageSlice = currentImageSlice.Previous ?? lastImageSlice;

        // Prepare the rectangle to draw within, starting with the newest
        // slice far to the right of the screen (a bit outside, even).
        var destinationRectangle = new Rectangle(
            (int)(GraphicsDevice.Viewport.Width + scaledWidth - drawOffsetX),
            0,
            scaledWidth,
            GraphicsDevice.Viewport.Height);

        // Draw the slices in the linked list one by one from the right
        // to the left (from the newest sample to the oldest)
        // and move the destinationRectangle one slice-width at a time
        // until the full screen is covered.
        while (destinationRectangle.X > -scaledWidth)
        {
            // Draw the current image slice.
            spriteBatch.Draw(imageSlice.Value, destinationRectangle, Color.White);

            // Move the destinationRectangle one step to the left.
            destinationRectangle.X -= scaledWidth;

            // Select the previous image slice to draw next time, wrap if necessary.
            imageSlice = imageSlice.Previous ?? lastImageSlice;
        }
    }
    finally
    {
        // Drawing done.
        spriteBatch.End();
    }

    base.Draw(gameTime);
}

The RenderSamples is taking a RenderTarget2D as an argument, which is the texture to be drawn on. The routine iterates through the samples and render them one by one.

private void RenderSamples(RenderTarget2D target)
{
    try
    {
        // Redirect the drawing to the given target.
        GraphicsDevice.SetRenderTarget(target);

        // Clear the target slice.
        GraphicsDevice.Clear(Color.Black);

        // Begin to draw. Use Additive for an interesting effect.
        spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.Additive);

        // The X-variable points out to which column of pixels to
        // draw on.
        var x = 0;

        // Since the byte-array buffer really holds 16-bit samples, the actual
        // number of samples in one buffer is the number of bytes divided by two.
        var sampleCount = microphoneData.Length / 2;

        // The index of the current sample in the microphone buffer.
        var sampleIndex = 0;

        // The vertical mid point of the image (the Y-position
        // of a zero-sample and the height of the loudest sample).
        var halfHeight = imageSliceHeight / 2;

        // The maximum number of a 16-bit signed integer.
        // Dividing a signed 16-bit integer (the range -32768..32767)
        // by this value will give a value in the range of -1 (inclusive) to 1 (exclusive).
        const float SampleFactor = 32768f;

        // Iterate through the samples and render them on the image.
        for (var i = 0; i < sampleCount; i++)
        {
            // Increment the X-coordinate each time 'samplesPerPixel' pixels
            // has been drawn.
            if ((i > 0) && ((i % samplesPerPixel) == 0))
            {
                x++;
            }

            // Convert the current sample (16-bit value) from the byte-array to a
            // floating point value in the range of -1 (inclusive) to 1 (exclusive).
            var sampleValue = BitConverter.ToInt16(microphoneData, sampleIndex) / SampleFactor;

            // Scale the sampleValue to its corresponding height in pixels.
            var sampleHeight = (int)Math.Abs(sampleValue * halfHeight);

            // The top of the column of pixels.
            // A positive sample should be drawn from the center and upwards,
            // and a negative sample from the center and downwards.
            // Since a rectangle is used to describe the "pixel column", the
            // top must be modified depending on the sign of the sample (positive/negative).
            var y = (sampleValue < 0)
                ? halfHeight
                : halfHeight - sampleHeight;

            // Create the 1 pixel wide rectangle corresponding to the sample.
            var destinationRectangle = new Rectangle(x, y, 1, sampleHeight);

            // Draw using the white pixel (stretching it to fill the rectangle).
            spriteBatch.Draw(
                whitePixelTexture,
                destinationRectangle,
                sampleColor);

            // Step the two bytes-sample.
            sampleIndex += 2;
        }
    }
    finally
    {
        // Drawing done.
        spriteBatch.End();

        // Restore the normal rendering target (the screen).
        GraphicsDevice.SetRenderTarget(null);
    }
}

The Challenge of the Day

Sat, 10 Mar 2012 16:30:00 +0200

Dear fellow developers,

Please create an instance of the following class (using any type as T):

class Foo<T>
    where T : Foo<T>
{
}

You may use any technique you like; plain "new MyClass...", using reflection, hacking MSIL, whatever.

Good luck! :)

Follow the rules when developing your Windows Phone app

Thu, 09 Feb 2012 16:25:00 +0200

Roughly one month after the first version of Badgers Rock had passed the certification cycle and happily thrived in the Windows Phone Marketplace, I received the following e-mail from one of the app testers:

Dear Windows Phone Marketplace Developer,

It has been determined through certification testing that the application indicated in the subject line has the following problem(s):

5.2.4.4
Pass with Notes: Pressing the device back button has no effect on the application.

Steps to Reproduce:
1. Launch the application.
2. Select "PLAY!".
3. Select a stage.
4. Press the device Back button.
5. A pause menu is displayed.
6. Press the device Back button again.
7. Observe the application the application does not respond.

The application passes certification, however the reported issues need to be resolved in the next update to the application.

ACTIONS REQUIRED: Please address the issues listed above, review the Windows Phone Application Certification requirements (http://go.microsoft.com/?linkid=9730556) and resubmit your updated application for certification testing. Future application update submissions will fail certification testing if the same issue(s) are not corrected.
PLEASE DO NOT REPLY TO THIS MESSAGE. For further assistance, please visit Developing for Windows Phone 7 forum.

Thanks,

Sam
Windows Phone Marketplace Certification

Sam was absolutely right about his finding, but it was in fact a design decision from my side. In the described scenario you are in an ongoing game and then you press the physical Back button on the phone. The game is now paused and you are presented with a couple of options; basically "do you want to end the game or resume?". So, at this point, how should I handle the Back-button? It was of course out of the question to end the game; it's way too simple to accidentally touch this button (depending on how you are holding the device). So making a double miss-tap ending the game would not make any hardcore Badgers Rock-gamer happy. So should the game resume then? This was the only obvious option left (making the game respond to the click), but I didn't really like that anyway because it would not be 100% clear to the player what the Back button would actually do in this case. So I decided to ignore the Back button in this screen and force the player to take an active decision choosing from the spelled-out options on screen -- again, to make it perfectly clear for the player what will happen.

Then this message came from which a couple of things can be noticed:

First of all, Microsoft is (for the first time?) really anxious that all the applications are behaving in a consistent way, not only giving guidelines. This is a good thing.
Secondly, it's not only automatic testing being performed on your submitted app. This is also a good thing. Although it doesn't seem as every release is tested manually since I had maybe four versions successfully passing the certification cycle when I got this message -- and the behavior in question had been there from the start.
Also, at the point in time when I received this message there was no certification phase currently going on; my latest release had already been passed successfully some time ago. Does this mean that the testers are doing the manual testing as a parallel task on top of the (more or less) automatic certification cycle? My app was presumably put in a queue and rested there for a month until someone (Sam!) had time to perform the manual testing.
A side note is the not-so-professional link they've included in the e-mail (http://go.microsoft.com/?linkid=9730556). This points to a PDF-file you'll have to download that's only telling you that the link has been moved (and of course where to). Suprising? Not really. Elegant? Well, not really...

Of course I should follow the requirements of always having a responsive Back-button!
So now during game-play, the Back-button will basically toggle pause mode on and off.

Badgers Rock

Wed, 08 Feb 2012 17:54:00 +0200

Badgers Rock is a small, completely free (no ads) Windows Phone game I created using the XNA framework during the autumn of 2011.
You can find it on the Windows Phone Marketplace and on Facebook.

The latest released version (V1.4) has both an online high score and cool retro sound.

There was a contest arranged by the "Windows Phone Sweden" Facebook page, that ended on February 3, 2012, and the Winner Niklas Dahlman successfully completed all thirty levels with "three stars" on each levels. Not an easy task considering some levels are really hard and time-demanding to complete. He won a Nokia Bluetooth Stereo Headset BH-111. Congratulations!

The online highscore is provided and hosted by http://mogade.com/. It is free, open source and have a very easy API to integrate. You can choose to host your data yourself or let them do it for you. Really nice.

All the sounds in the game were taken from the http://www.freesound.org/. Here is a complete list of the sounds that I used in the game.