ὅπερ ἔδει ποιῆσαι (hoper edei poiēsai)

Blog has moved

2009-09-24T09:12:00.004-05:00

I have moved my blog to a new host, and there are a number of added and updated posts. I apologize for not making this post sooner. Please go to http://innovatian.com/blog for all of my new posts.

Ruby First Impressions

2008-12-28T18:28:00.001-05:00

I decided to try out Ruby today. I have wanted to try it out for quite a while. Some of my impressions may be inaccurate, but I have done what I can. I have Zero interest in DB and web ‘programming’ so I am going to skip rails and related material.

I tried to use JetBrains’ RubyMine, but I couldn’t get anything to run. I know it is a public preview, but for the time I am willing to put into Ruby, there was just too much friction for now. I look forward to trying out the full product when it is RTM.

I installed Ruby In Steel Personal Edition and that was a great way to get started. I ran the installers and I was up and running in a familiar environment. I created a Ruby project and I started with hello world. After that I started going through the guide at Techtopia and the RDocs. I love the product for a free version, but for someone trying to learn Ruby, the intellisence feature would be handy. I am never going to pay $200 for an IDE that I would use as a hobby; that being said, if I ever were to program Ruby as a job, $200 is a great price.

What I like

very straightforward language
- I loved that when I wanted to shift the elements of an array I found it was built-in
Class library
being able to have a line of code like this; I wish I knew what to call it

return SolveUsingClosedFormExpression( n ) if ( 0..1474 ).include? n
return SolveUsingClosedFormExpression( n ) if (0..1474) === n

I will complain about this as well, but in writing the closed form Fibonacci solution, the data type changes allowing for huge results

def SolveUsingClosedFormExpression(n)
    left = GOLDENRATIO ** n
    right = (-GOLDENRATIOINV) ** n
    return ( (left - right) / ROOT5 ).to_i
end

What I didn’t like

Trying to figure out how to use gems (packages, not the tool) and files in the class library
Lack of type information

Yes, yes, I know, I know, but for a c, c++/cli, c# programmer, it feels just wrong
Declaring a variable feels like I am introducing a magic variable - *poof* it exists. At least in TI-Basic I had to declare my dynamically typed variables. \
Lack of method return types

At least four ways to return a value from a method, see below.
Inconsistent API

I was very frustrated when trying to use .power! only to find that it isn’t defined for Float types – I have to use ** everywhere.

Ruby is supposed to be super OO, but what object contains puts/print?

What I really didn’t like

def multiplyWithReturn(val1, val2 )
    result = val1 * val2
    return result
end

def multiplyLocalVariable(val1, val2 )
    result = val1 * val2
end

def multiplyNoVariables(val1, val2 )
    val1 * val2
end

def multiplyAssignToMethodName(val1, val2 )
    multiplyAssignToMethodName = val1 * val2
end

puts multiplyWithReturn(5,6)
puts multiplyLocalVariable(5,6)
puts multiplyNoVariables(5,6)
puts multiplyAssignToMethodName(5,6)

Running this code prints

I don’t know if I am missing a key ‘feature’ of the language, but at least four ways to return a value from a method just really irks me. Given that there is no return type of a method, reading code to determine what is returning a value and what methods are void seems ridiculous.

I have never programmed in a dynamic language, so it has been a bit of a ride. There a so many nuances to the language, it will take a while to get them down, but they allow for very concise code.

Continuous Build Systems

2008-12-15T11:36:00.001-05:00

I have gone through so many tools trying to create the right build environment that it is a bit disheartening. I have set up multiple continuous integration systems and it keeps getting easier, but there is still a bit too much friction.

I tried to look into using Team Foundation Server and Team Build, but the cost is very high for a startup (even with the top msdn subscription). This is compounded when you have external developers that must also hook into the system.

I have fallen for TeamCity but with one deal breaker for me: no FinalBuilder support. I submitted some feedback and I received a detailed response from JetBrains; I have always received great customer support from JetBrains:

There are several ways to support FinalBuilder:
1. You may start using CommandLine runner to start the process.
2. To Add custom FinalBuilder reporting you may use TeamCity service messages.
   Please refer to
http://www.jetbrains.net/confluence/display/TCD4/Build+Script+Interaction+with+TeamCity
   documentation article on service messages for details
3. You may start FinalBuilder from any other build scripts,
   for example NAnt, MSBuild, Ant, Maven.
   If you need custom logging, you may consider using TeamCity
   Service messages as well
4. Write a build runner plugin for TeamCity. There are two
   public available examples on the build runners at:
http://www.jetbrains.net/confluence/display/TW/Rake+Runner
   and
http://svn.jetbrains.org/teamcity/plugins/fxcop/
   Please feel free asking any questions on the integration.
5. Post an issue to our tracker at
   http://www.jetbrains.net/tracker

I checked out the code for the build runner plugins, but it is too much work. It is hard not to sound lazy in saying that, but I haven’t used java since my sophomore year in college, and it doesn’t sit high on my list to spend hours getting a build runner going (I am looking for less friction, not more). I chose the 5th option and submitted a feature request TW-6442. With FinalBuilder support I don’t need any other build runners/tools as FinalBuilder most likely wraps them up in its UI.

One other feature that I would like to have, but am hesitant to request right now, is to be able to selected a successful build, and run a publish script (through the web interface (see Cruise)) that can tag the build and use the build artifacts to deploy that build to Dev/Stage/Production environment.

Ninject MessageBroker Update

2008-11-29T11:49:00.001-05:00

It has been a long time (10 months) since I worked with the Ninject MessageBroker and a couple things have changed since my last post. In the old version you had to connect the MessageBroker as a kernel component by hand. Since then the extension environment has been flushed out a lot more. Now you can register the MessageBrokerModule when constructing your kernel object.

using System;
using System.Diagnostics;
using System.Net;
using System.Text.RegularExpressions;
using Ninject.Core;
using Ninject.Extensions.MessageBroker;

namespace NinjectMessageBroker
{
    internal class Program
    {
        private static void Main()
        {
            // Intialize our injection kernel adding message broker functionality.
            using (var kernel = new StandardKernel(new[] { new MessageBrokerModule() }))
            {
                // Get the event publisher. It reads the current time and fires an event
                var pub = kernel.Get<TimeReader>();
                Debug.Assert(pub != null);

                // Get the subscriber, it waits to get the current time and writes it to stdout
                var sub = kernel.Get<TimeWriter>();
                Debug.Assert(sub != null);

                // Verify that they were wired together
                Debug.Assert(pub.HasListeners);
                Debug.Assert(sub.LastMessage == null);

                // Get the current time. It should automatically let the TimeWriter know
                // without either of them ever knowing of one another.
                pub.GetCurrentTime();

                // Wait to exit.
                Console.ReadLine();
            }
        }
    }

    internal class TimeWriter
    {
        public string LastMessage { get; set; }

        [Subscribe("message://Time/MessageReceived")]
        public void OnMessageReceived(object sender, EventArgs<string> args)
        {
            LastMessage = args.EventData;
            Console.WriteLine(LastMessage);
        }
    }

    internal class TimeReader
    {
        public bool HasListeners
        {
            get { return (MessageReceived != null); }
        }

        [Publish("message://Time/MessageReceived")]
        public event EventHandler<EventArgs<string>> MessageReceived;

        /// <summary>
        /// Gets the current time and updates all subscribers.
        /// </summary>
        public virtual void GetCurrentTime()
        {
            string text = GetWebPage();
            var regex = new Regex(@"\d\d:\d\d:\d\d");
            MatchCollection matches = regex.Matches(text);
            string time = ((matches.Count == 2) ? matches[1] : matches[0]).Value;
            SendMessage(time);
        }

        /// <summary>
        /// Gets the contents of a web page as a string.
        /// </summary>
        /// <returns></returns>
        private static string GetWebPage()
        {
            const string url = "http://www.time.gov/timezone.cgi?Eastern/d/-5";
            var webClient = new WebClient();
            return webClient.DownloadString(url);
        }

        /// <summary>
        /// Sends the message to all subscribers in a threadsafe manner.
        /// </summary>
        /// <param name="message">The message.</param>
        public void SendMessage(string message)
        {
            EventHandler<EventArgs<string>> messageReceived = MessageReceived;

            if (messageReceived != null)
            {
                messageReceived(this, new EventArgs<string>(message));
            }
        }
    }

    public class EventArgs<TData> : EventArgs
    {
        public new static readonly EventArgs<TData> Empty;

        static EventArgs()
        {
            Empty = new EventArgs<TData>();
        }

        private EventArgs()
        {
        }

        public EventArgs(TData eventData)
        {
            EventData = eventData;
        }

        public TData EventData { get; private set; }
    }
}

C++ if statement oddity

2008-11-28T13:52:00.001-05:00

I found that you can do work such as assigning a variable and executing a method outside of the main expression of an if statement. For example:

    if(hRes = GetApplicationState(), NT_SUCCESS(hRes))
    {
        // Do something
    }

Yes, that is a comma inside of the if statement. To the left of the comma is executed first and then the if statement uses the right side as its evaluation expression.

Open Source Projects

2008-11-28T13:34:00.001-05:00

I had two open source projects: Ensurance, and IMAPI.Net.

Ensurance did not really fit into any particular group. It was a tool for me while Spec# was still a research project at MS. I wanted to be able to use pre/post conditions, parameter constraints, and tie them into logging and debugging. I don’t think that Spec# will do the last bit, but I am really looking forward to its release.
IMAPI.Net is a C# cd authoring library that wraps the IMAPI system in windows to allow programmers of any application to write to CD. It supports data and audio discs. It was expensive to test the library and the COM issues were a huge problem for a while.

PInvoke.net was a good help for the most part, but the marshalling took a long time to get right. I started the project back in .net 1.1 while working on my BS - and it worked. With each .NET release the project would have been easier to write with the COM interop support increasing. I used to be the lead of the XPBurn component project on GotDotNet before it shut down; in truth though, I had stopped supporting it a couple months before that. No one would read directions, read the forum, follow any kind of guidelines and would essentially expect me to figure out why they are using the code wrong. It became too much hassle in the end for what I got out of the project. It was a great learning experience though.

Both projects are essentially dead and I will be closing down the Ensurance project on CodePlex. I will leave IMAPI.Net up on google code in case anyone ever finds use for the code. If I get the time I will try to add my PInvoke code to the wiki so others don’t have to work as hard as I did.

Yet Another Video Update (YAVU)

2008-06-13T22:01:00.001-05:00

Here is the latest PLAYXPERT video demo. While playing the game the user:

Starts playing a song.
Searches the web for a video.

Web browser
AJAX
FLASH

Opens the Friends widget and initiates an IM session
Continues playing the game the whole time.

STAY IN GAME!

PLAYXPERT in Action

2008-05-02T18:41:00.002-05:00

Although this blog is in no way associated with my employer, I feel I need to post this. It is a video of PLAYXPERT running in game. For many years I had wished there was a product like this, and now I get to be part of the team that brings it to life :)

Ninject Message Broker

2008-01-16T15:54:00.001-05:00

I updated my local copy of Ninject today and decided to try out the message broker. I opened the test fixture to see how it is used and wrote up this little test. Overall I am very happy with how this simple demo worked out. I am going to try to set aside some time to try something more in depth. This simple example has one object getting the current time from the Internet and then publishes the new time it received. The second object listens for the time to be updated and then writes it to the console. What I really dig is that the two objects never know of one another.

internal class Program
{
    private static IKernel _kernel;

    private static void Main(string[] args)
    {
        // Intialize our injection kernel
        _kernel = new StandardKernel();

        // Add message broker functionality.
        _kernel.Connect<IMessageBroker>(new StandardMessageBroker());

        // Get the event publisher. It reads the current time and fires an event
        TimeReader pub = _kernel.Get<TimeReader>();
        Debug.Assert(pub != null);

        // Get the subscriber, it waits to get the current time and writes it to stdout
        TimeWriter sub = _kernel.Get<TimeWriter>();
        Debug.Assert(sub != null);

        // Verify that they were wired together
        Debug.Assert(pub.HasListeners);
        Debug.Assert(sub.LastMessage == null);

        // Get the current time. It should automatically let the TimeWriter know
        // without either of them ever knowing of one another.
        pub.GetCurrentTime();

        // Wait to exit.
        Console.ReadLine();
    }
}

internal class TimeWriter
{
    private string _lastMessage;

    public string LastMessage
    {
        get { return _lastMessage; }
        set { _lastMessage = value; }
    }

    [Subscribe("message://Time/MessageReceived")]
    public void OnMessageReceived(object sender, EventArgs<string> args)
    {
        _lastMessage = args.EventData;
        Console.WriteLine(_lastMessage);
    }
}

internal class TimeReader
{
    public bool HasListeners
    {
        get { return (MessageReceived != null); }
    }

    [Publish("message://Time/MessageReceived")]
    public event EventHandler<EventArgs<string>> MessageReceived;


    /// <summary>
    /// Gets the current time and updates all subscribers.
    /// </summary>
    public virtual void GetCurrentTime()
    {
        string text = GetWebPage();
        Regex regex = new Regex(@"<b>\d\d:\d\d:\d\d<br>");
        Match match = regex.Match(text);
        string time = match.Value.TrimStart("<b>".ToCharArray()).TrimEnd("<br>".ToCharArray());
        SendMessage(time);
    }

    /// <summary>
    /// Gets the contents of a web page as a string.
    /// </summary>
    /// <returns></returns>
    private static string GetWebPage()
    {
        string url = "http://www.time.gov/timezone.cgi?Eastern/d/-5";
        HttpWebRequest webRequest = WebRequest.Create(url) as HttpWebRequest;
        webRequest.Method = "GET";
        using (WebResponse webResponse = webRequest.GetResponse())
        {
            using (StreamReader sr = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
            {
                return sr.ReadToEnd();
            }
        }
    }

    /// <summary>
    /// Sends the message to all subscribers in a threadsafe manner.
    /// </summary>
    /// <param name="message">The message.</param>
    public void SendMessage(string message)
    {
        EventHandler<EventArgs<string>> messageReceived = MessageReceived;

        if (messageReceived != null)
        {
            messageReceived(this, new EventArgs<string>(message));
        }
    }
}

Visual Studio 2008 InstallFest

2007-12-11T00:59:00.001-05:00

I will be attending the Visual Studio 2008 InstallFest in Spokane Valley on Wednesday the 12th with a few other deve lopers from PLAYXPERT. It will be nice to meet some other .NET coders in the Spokane area. Hopefully we can pick something up soon (better SIGS). I was really looking forward to attending the Agile Philly and Philly ALT.NET group meetings before the move to Spokane.

Settling In

2007-12-11T00:16:00.001-05:00

Well, I have been in Spokane for a week now. I still feel like I am just settling in. My car finally shipped (thanks to MN for helping) and should arrive this week. The contents of my apt may not be here until Christmas, so it will be a little rough until then. The weather the last two weekends has been as bad as the worst I saw in Philadelphia (and it is only December). I wish I could find reliable information on what are the best winter tires.

SC07: It Begins

2007-11-12T20:32:00.001-05:00

All around you will find Intel trying to make waves

Celebrating the 45nm Launch - 45 steps to Technology Leadership

Ok. So they made 45nm, what's next? With the physical limit a few nm away (from what I understand, 30nm is the limit), what are they going to shock and awe us with next?

Everyone is trying to get the final touches finished on their booths while eagerly awaiting the food coming in just 5 minutes. After an hour of eating, we will have a half hour break before the VIPs get their privileged two hour private run on the show floor. From what I hear, D-Wave is claiming to have the first quantum computer presented here, but we will have to see about that.

News

2007-11-10T11:53:00.001-05:00

It has been too long since my last post, for which there are a number of reasons.

Super Computing 2007
Vacation Preparation
New Job
Moving

I head to Reno, Nevada early tomorrow morning for SC07. I have been trying to get my research ready for presentation. Hopefully this year will be as fun and interesting as last.

The day after I get back from Reno, my girlfriend and I are flying to Aruba for ten days.

There is a lot of preparation when I will essentially be gone for fifteen days. This should be a nice break compared to the bedlam to come.

I resigned. When I get back from Aruba, I will be working my last week as a Software Developer for NAVTEQ. I will be moving 2529 miles from Philadelphia to Spokane, WA to take the role of Software Quality Assurance Director for PlayXPert.

I will be doing software architecture, software development, developer management, project management, system administration, continuous integration, QA management, and other tasks.

Along with the new job comes moving. I work for one week, during which I need to pack every morning and night ( as I will be gone for the two weeks prior ) which leaves no room for anything. I start Monday, December 3rd.

This come as a stumbling block for my Ensurance project on CodePlex. It is stable, fully featured, and usable, but I will not be able to work on it for a month which really pains me. The only pieces missing are the distribution tasks such as strong naming, finishing internationalization, and such. I was really looking forward to implementing the full set of fluent parameter attributes, but there is a minimal implementation right now.

I will try to post from SC07 with anything that I find.

New Ensurance Release

2007-10-11T17:43:00.001-05:00

Yesterday I released version 0.4 on Codeplex. All of the functionality for the 1.0 release is finished. I am trying to wrap up all of the tasks needed for a distribution. When all of that is done I will mark the release as 1.0. I am still looking for functionality ideas.

Long time, no posts, new OS project Ensurance

2007-10-05T14:35:00.001-05:00

I have been incredibly busy lately and thus there have been no posts. I have finally published the first release of my project Ensurance on CodePlex. It is based on the NUnit 2.4.3 framework and is intended to be used for production code to verify particular constraints in an application. Check out the CodePlex page for more information. I will be posting sample code soon.

Portable 'supercomputer'

2007-09-02T10:16:00.001-05:00

The story of the Calvin College portable supercomputer really irks me. It is getting a lot of attention for doing nothing. You can see it on engadget, Slashdot, and others. Anyone can build a cluster like what they have. The only credit I will give them is that not everyone would go through the benchmark and analysis as they did. On the other hand, these people are still on the same old trail of HPC. They ignore the entire issue of programming -- they used HPL code. Ask if they can write their own code for the microwolf. What kind of real application performance can they achieve? The engadget article says "Talk about a solid price-to-performance ratio." - what load of crap. How about asking real questions and reporting real HPC news?

Ninject Dependency Injection Framework

2007-09-02T07:39:00.001-05:00

I have been playing with Ninject for a couple days now and I am having a lot of fun. One feature I have been really digging is Contextual Binding. The kernel allows multiple modules to be loaded that define the bindings of your types. So depending on which modules and bindings are set, I can change the way other binding resolve. Here is an example:

Say I want to resolve a different type of car dependent on a selected engine. I have an interface IEngine and two classes V4, V6 that inherit from IEngine. If V4 is resolved by the kernel then the car returned would be a Civic; an Accord if a V6 is returned. Here is the code in custom modules:

    internal class EngineModule : StandardModule
    {
        public override void Load()
        {
            Bind<IEngine>().To( typeof ( V4 ) );
        }
    }

    internal class CarModule : StandardModule
    {
        public override void Load()
        {
            IContext context = new StandardContext( Kernel, typeof ( IEngine ) );
            IBinding binding = Kernel.GetBinding<IEngine>( context );
            Type implementationType = binding.Provider.GetImplementationType( context, false );

            Bind<ICar>().To( typeof ( Accord ) ).OnlyIf(
                delegate
                {
                    if ( implementationType == typeof ( V6 ) )
                    {
                        return true;
                    }

                    return false;
                } );

            Bind<ICar>().To( typeof ( Civic ) ).OnlyIf(
                delegate
                {
                    if ( implementationType == typeof ( V4 ) )
                    {
                        return true;
                    }

                    return false;
                } );
        }
    }

I can then load the modules into my kernel and use it to resolve cars:

    internal class Program
    {
        private static void Main( string[] args )
        {
            IModule[] modules = new IModule[] { new SettingsModule(), new EngineModule(), new CarModule() };
            IKernel kernel = new StandardKernel( modules );

            ICar car = kernel.Get<ICar>();
            Console.WriteLine( "I just got a brand new {0} {1}.", car.Manufacturer, car.Model );
            Console.WriteLine( "It has a {0} cylinder engine with {1} horsepower and {2} ft-lb torqe.", car.Engine.Cylinders, car.Engine.Horsepower, car.Engine.Torque );
            Console.WriteLine( "Time to start the Engine!" );
            car.Engine.Start();
            Console.WriteLine( "Sound good, but I am going to turn it off for now." );
            car.Engine.Stop();
            Console.ReadLine();
        }
    }

Nate Kohari just wrote a first cut user guide that is very helpful when getting started.

Running Mocked Unit Tests in FinalBuilder With Coverage

2007-08-27T12:22:00.001-05:00

I just read an article by Craig Murphy on running unit tests in FinalBuilder. I liked the article and it gives some good pointers. I wish he had written it a while ago, it would have been helpful to me. We use TypeMock which will not allow us to make our unit tests as simply. When I wrote our build scripts I took the same general approach he did.

Starting in the Main ActionList we defined a few variables that give the paths to the applications we use. I also created an ActionList call RunTest which I will go into detail below. For each DLL being tested a call to RunTest is made passing in the name of the DLL to be tested.

In each call we set the CURRENTDLL ActionList Parameter with the name of the DLL (Animation,Audio,Core,..).

NCOVEROPTIONS holds the command-line options for ncover.console.exe. The same follows for NUNITOPTIONS and TMOCKRUNNEROPTIONS. The rest of the ActionList Parameters are used as temporary variables allowing me to put together very complex command argument strings and parse XML into variables.

The RunTest ActionList is separated into two try...catch blocks. The first to run the tests and the second to process the results.

Here are what the commands are doing:

Set Variable NCOVEROPTIONS
//x "%BUILDDIR%\%CURRENTDLL%Coverage.Xml" //l "%BUILDDIR%\%CURRENTDLL%Coverage.Log"

Set Variable TESTDLLS
"%ROOTDIR%\bin\%CURRENTDLL%Tests\%CURRENTDLL%.Tests.dll"

Set Variable NUNITOPTIONS
%TESTDLLS% /xml:"%BUILDDIR%\%CURRENTDLL%TestResult.xml" /noshadow

Set Variable PARAMETERS
-first -link NCover %NCOVER% %NCOVEROPTIONS% %NUNIT% %NUNITOPTIONS%

Execute Program
%TMOCKRUNNER% %PARAMETERS%
-Wait For Completion, Log Output, and Hide Window are checked. I do not check the exit code. The second try block handles it.

The first sets up all of our variables and launches the TMockRunner application with all of our parameters. The second tries to read the test results into the TESTFAILURES variable. If the TMockRunner failed then the file will not exist and the Catch block will append our error message to indicate this. If there are test failures we read the failed test cases into a temporary error variable using XPath. This temporary variable is appended to the error message variable which is sent via email to our team. This allows us to find the exact tests that failed directly from our email.

That is basically it. Our CI server runs the FinalBuilder script with a few extra options and processes the NCover output to give us a nice web page.

Parallel Functionals

2007-08-18T10:36:00.001-05:00

I have used the loop tiling code with scheduling algorithms to parallelize Dustin Campbell's code from his post A Higher Calling. The parallelized funtionals can be found here. It uses the scheduling algorithm code from the last post. The parallel reduction is actually based on Joe Duffy's serial reduction.

Scheduling Algorithms

2007-08-18T10:11:00.001-05:00

I have uploaded my current version of the scheduling algorithms code that I referenced in my previous post. You can view the file here. I no longer have access to a multi core/processor machine so I cannot benchmark them properly right now.

Correctness of Parallel Code

2007-08-12T16:58:00.001-05:00

One of the big problems in writing parallel applications is code correctness. How do we know that our parallel implementations are correct? While multi-core correctness has been researched and documented a great deal, correctness for clustering code appears to still be in its infancy. Writing, debugging, and verifying the correctness of your applications remains incredibly difficult. Now, I am not talking about grid computing or web services. I am specifically referring to MPI and the like doing high performance computing.

When debugging a parallel application, a user must set breakpoints, attach to multiple processes, and then try to step through the applications in parallel. This becomes even more unwieldy if you are also trying to write the library/service and test your applications. I am not a fan of debugging, and parallel debugging just makes my skin crawl. So, what can one do?

When designing the implementation of SPPM I wanted to approach it in a way that would allow flexibility never seen in a parallel programming system. Starting at the highest level, I abstracted the service to a well defined interface. Given this interface, I can actually implement several versions of my service that can be swapped out at runtime. But why would I want multiple versions? Using a framework like Castle, I can specify implementations of my service in a configuration file for the WindsorContainer to resolve. Depending on how I want to test, I can change my service implementation. I have two implementations right now: the real service, and a local virtual service. The real service tells my library that when calls are made, try to talk to an external service which runs on every node in the cluster. The local virtual service is used when you want to test a master and worker application without needing a cluster or external service. From the applications' point of view nothing changes. They make the same library calls, but the service implementation has changed allowing the programmer to focus on their application and not its dependencies.

Another set of work dealt with the library implementation. All calls were implemented with a composite pattern partnering the command pattern and template method pattern. All commands would request a pointer from a communication interface factory which would return a pointer to come kind of network transmission class. This class is switched out when the service implementation changes - it may get a TCP socket communicator, a remoting communicator, or, just a pointer to an object that routes calls into the local virtual service.

Up to this point, all of the features has been done with design patterns and object containers. To push things one step further, I had to solve a problem that has really bothered me. Why must I actually execute all of my parallel applications together in order to test them? Why must I run a service, master, and worker just to verify that the simplest case works? I have already shown how to test using just the master and worker by abstracting the service, so what about verifying the correctness of the master and worker independently of one another? Impossible? Why? External Dependencies? What about the service layer? Is it at all possible to apply TDD to HPC and clusters? Of coarse, but it is far from obvious.

Aspect-oriented programming will allow us to simulate our dependencies without changing our code. The TypeMock web site does a good job at illustrating this. Our problem breaks down to the need to simulate external dependencies. Write unit tests for your parallel applications, but where external library calls would be made, like a parallel library which talks to the cluster, mock the calls! Below is a small sample trying to mock the worker of a parallel matrix multiplication application. The worker application is supposed to get the B matrix, and a series of rows Ai from A, and compute a sub-result Ci which is sent back to the master. The Check method will be called when the Put call is made and intercepted, and the Ci matrix will be verified for correct contents. The term tuple tells the worker application to exit when it tries to get the next piece of A.

[Test]
public void WorkerTest() {
    // We use this call to tell TypeMock that the next creation of
    // the InjectionLibrary needs to be faked. TypeMock will replace
    // the instance that should have been created with its own
    // empty implementation. The mocks we set up below tell
    // TypeMock that we are going to have calls made by another component
    // and we need to insert fake results.
    Mock mock = MockManager.Mock(typeof(InjectionLibrary));
    
    // Fake the B call. When the Worker application calls
    // Slm.Get("B")
    mock.ExpectAndReturn("Get", 0).Args(new Assign("B"), new Assign(B));
    
    // fake the Ai call
    string tupleName = String.Format("A0:A{0}", N - 1);
    mock.ExpectAndReturn("Get", 0).Args(new Assign(tupleName, new Assign(A)));
    
    // Grab the put of Ci and validate the result
    // We tell TypeMock to call the Check method below to
    // validate the argument (Ci matrix).
    mock.ExpectCall("Put").Args(Check.CustomChecker(new ParameterCheckerEx(Check)));
    
    // Fake the term tuple
    mock.ExpectAndReturn("Get", 0).Args(new Assign("A0:A0"), new Assign(term));
    
    // This next line is mocked, the library is static
    // and the faked instance is shared.
    InjectionLibrary Slm = InjectionLibrary.Instance;
    
    // Call our worker's main method.
    // All of the calls that were mocked above will
    // have there calls hijacked.
    OCMMWorker.Main(null);
}

Here we have referenced a .NET application like an external DLL and invoked its Main method. Everything is run from the comfort of your TestFixture and you have verified one case in your worker application without the need of a cluster, master, or external/virtual service. Using these methods, we can write parallel cluster code while minimizing our need to debug. Why debug when you know it works?

Overhead of Parallel Applications and Scheduling Algorithms

2007-08-11T18:20:00.001-05:00

When we typically try to measure the speedup of a parallelized algorithm, many people only calculate the serial vs. parallel running time (Sp = Tseq/Tpar). The effective speedup can more closely be approximated with the following equation to further define Tpar:

Tpar = Tcomp + Tcomm + Tmem + Tsync
Tpar is the total parallel running time.
Tcomp is the time spent in parallel computation on all machines / processors.
Tmem is the time spent in main memory or disk.
Tsync is the time required to synchronize.

All of these are easily quantifiable with the exception of Tsync. In an ideal world Tsync is zero, but this is never the case. A single slow processor/machine/data link can make a huge difference with Tsync. So, how can we manage it?

The most straightforward way to try to minimize Tsync it it to apply a fixed chunk load. MPI uses this method. This is fine unless you have a heterogeneous cluster or other applications are running on the processors/machines being used.

An extension of the fixed chunking is to create a normalized value for all of your computing resources. Then, let the computing resources get jobs whose size is comparable to their ability. Another way, say we have three machines whose computing powers are A:1, B:2, C:3 (C is three times as powerful as A). If we have twelve pieces of work to distribute, we can group them into six groups of two (total / sum of power). In the time C will compute three of the chunks, B will have done two and A will have done one. The entire job should wrap up with a minimal Tsync. This kind of work load balancing works very well for heterogeneous clusters.

Other scheduling methods had been tried in clusters; factoring and guided self-scheduling being two, but they seem to add too much overhead for the Tcomm. There has to be a balance in creating chunky communication and minimizing Tsync. They may however work as valid scheduling algorithms for multi-core processing. The Tcomm is greatly reduced compared to cluster computing.

Working again from Joe Duffy's MSDN article "Reusable Parallel Data Structures and Algorithms: Reusable Parallel Data Structures and Algorithms", I have reworked the loop tiling code to support scheduling algorithms.

[ThreadStatic]
private static int partitions = ProcessorCount;

[ThreadStatic]
private static SchedulingAlgorithms algorithm = SchedulingAlgorithms.Factoring;

[ThreadStatic]
private static double factor = 0.5;

private static void For<TInput>( IList<TInput> data, int from, int to, Action<TInput> dataBoundClause, Action<int> indexBoundClause )
{
    Debug.Assert( from < to );
    Debug.Assert( ( dataBoundClause != null ) || ( indexBoundClause != null ) );
    Debug.Assert( ( data != null && dataBoundClause != null ) ||
                  ( data == null && dataBoundClause == null ) );

    int size = to - from;
    int offset = 0;
    List<int> gp = GetPartitionList( size );

    int parts = gp.Count;
    CountdownLatch latch = new CountdownLatch( parts );
    for (int i = 0; i < parts; i++) {
        int start = offset + from;
        int partitionSize = gp[0];
        gp.RemoveAt( 0 );
        int end = Math.Min( to, start + partitionSize );
        offset += partitionSize;
        ThreadPool.QueueUserWorkItem( delegate
                                      {
                                          for (int j = start; j < end; j++) {
                                              if ( data != null ) {
                                                  dataBoundClause( data[j] );
                                              }
                                              else {
                                                  indexBoundClause( j );
                                              }
                                          }

                                          latch.Signal();
                                      } );
    }
    latch.Wait();
}

private static List<int> GetPartitionList( int size ) {
    ISchedulingAlgorithm schedulingAlgorith = null;

    switch ( Algorithm ) {
        case SchedulingAlgorithms.FixedChunking:
            schedulingAlgorith = new FixedChunking( size, Math.Min( size, partitions ) );
            break;
        case SchedulingAlgorithms.GuidedSelfScheduling:
            schedulingAlgorith = new GuidedSelfScheduling( size );
            break;
        case SchedulingAlgorithms.Factoring:
            schedulingAlgorith = new Factoring( size, factor, factoringThreshold );
            break;
    }

    Debug.Assert( schedulingAlgorith != null );
    return schedulingAlgorith.GetPartitionSizes();
}

With this setup, all you have to do is set the algorithm that you would like to use along with the scheduling algorithm. The variables are thread static so that many threads can have their own algorithms, but since the for loop is blocking you only need one copy per thread.

It will take a little bit of tweaking for your application's algorithm, but these should produce reasonable results. My next goal is to get them benchmarked under different scenarios.

Follow Up: Frustration Caused by Amdahl's 'Law'

2007-08-11T05:04:00.001-05:00

Michael Suess from Thinking Parallel was kind enough to point me to a paper he coauthored "Implementing Irregular Parallel Algorithms with OpenMP". In it he goes on to describe a way to implement the early thread sync that I mentioned using OpenMP. If you are interested in more of his papers, here is his publications page.

RE: Why are int[,], double[,], single[,], et alia so slow?

2007-08-11T04:53:00.001-05:00

I actually wrote the previous post in October of 2006. Since then, someone was nice enough to look up the reasons for me. Wesner Moise posted this article on CodeProject that I think explains a lot.

Jamie Cansdale - TestDriven.Net - Silently Posts a Resolution with Microsoft

2007-08-08T10:34:00.001-05:00

From the 2.8.2130 release notes

Release Notes - TestDriven.NET: 2.8

993: Retire Visual Studio Express support

Joint statement: "Jamie Cansdale and Microsoft Corporation have agreed to concentrate on working together on future releases of TestDriven.Net for Microsoft's officially extensible editions of Visual Studio, as opposed to spending time litigating their differences."