Yet Another C# Blog

.NET 2.0 Extension Methods

2008-11-16T19:03:00.004-05:00

I regularly write programs in C# on Visual Studio 2008. The default Framework for the environment is 3.5. But sometimes I must re-target 2008 to Framework 2.0. Is is possible to take advantage of the new "Extension Method" language feature of 2008 when required to target 2.0? This is a good question and applies to more than the extension methods. But for now, let's answer the question for extensions.

Yes.

But, you need to do some of the work yourself. The compiler's support for extension methods is really unconnected with Framework 3.5. But maybe you've seen the error...

Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll?

To write an extension, you need the ExtensionAttribute somewhere, anywhere. The compiler is looking for it in the System.Runtime.CompilerServices namespace which is in System.Core.dll. The 2.0 and 3.5 versions of this DLL are different. The good news is you don't need the whole 3.5 DLL, you just need the ExtensionAttribute class and you can actually write it yourself.

So, here I have written the class you need in C#.


namespace System.Runtime.CompilerServices
{
    [AttributeUsage(
        AttributeTargets.Assembly
        |AttributeTargets.Class
        |AttributeTargets.Method,
        Inherited=false,
        AllowMultiple=false)
    ]
    public class ExtensionAttribute : Attribute
    {
    }
}

Add this to a new class file or an existing file. Make sure you add it outside of all other namespace definitions since you will be adding to the System.Runtime.CompileServices namespace.

That's all it should take. Good luck with your projects and using extension methods in C# with Framework 2.0.

An Interesting Delegate Usage

2008-11-09T19:40:00.001-05:00

An interesting thing happened while I was surfing the MSDN C# forums. I was completely thrown for a loop when I came across some code posted by a regular contributor. It went something like this...


    private void ACallBackFunction()
    {
        if(this.InvokeRequired)
            this.Invoke(new ThreadStart(ACallBackFunction));
        else
            this.label1.Text = "Something or other";
    }

The programming pattern should be very familiar to those using threads, because this is how a child thread updates the GUI (only the thread on which the GUI is created can update the UI). But, what seemed very strange was the use of "new ThreadStart(...)". I thought, what does a ThreadStart() delegate have to do with this operation?

The answer is ... nothing. But the application is none the less interesting. Here, the programmer decided to draw from a handy delegate in the C# tool chest just to get that required delegate to call "Invoke()" with. "ThreadStart" is a delegate type for a void method taking no arguments. It saved having to write a delegate type for the operation that might have looked like this...


    delegate void ACallBackFunctionDelegate();

...just so that the program could later write...


    private void ACallBackFunction()
    {
        if(this.InvokeRequired)
            this.Invoke(new ACallBackFunctionDelegate(ACallBackFunction));
        else
            this.label1.Text = "Something or other";
    }

So the by borrowing the pre-existing delegate declaration, namely ThreadStart, a few lines of coding were avoided. Now, I don't particularly like the implementation (it might lead a newbie astray). But, I do appreciate the awakening it caused in my thinking about delegates. Sometimes I would get all hung up in the declaration of the delegate and forget that the delegate need not be so tightly coupled to the method I'm intending it for. The delegate describes a well-typed variable for safely calling any number of methods which adhere to the prescribed signature.

Okay, so now I am more open minded about the re-use of my delegates as well. In fact, with the .NET Framework 3.x, there are a few new delegates that address the common signatures of many of our functions. The delegate types are "Action" and "Func". Both, are available as generic types making it very easy to construct a delegate variable on the fly similar to the example above. For example, one can write...


    private void ACallBackFunction()
    {
        if(this.InvokeRequired)
            this.Invoke(new Action(ACallBackFunction));
        else
            this.label1.Text = "Something or other";
    }

The benefit of using "Action" instead of "ThreadStart", in my opinion, is it's a little less confusing.

Regular Expressions in C# - Password Validator Revisited

2008-10-25T19:07:00.001-04:00

Sometimes we make life more difficult than it needs to be.

Lately, I've learned a little more about Regular Expressions and "Negative Look-Around". I've used it a lot, but it only recently dawned upon me that I was not making full use of it.

Case in point, take a look at my earlier regular expression article where I explain how to validate a password with multiple requirements (Regular Expression Alternations). This article discusses the absense of the boolean AND in regular expressions and provides a complex IF-THEN-ELSE approach to test a string for conforming to multiple "password" constraints. And my subsequent article (Regular Expression Double Negatives) gets more complex with an "inside-out" approach to compensate for the lack of an AND operator.

But, let's go back even further to the article where I "explain" Negative Look-ahead. In this article, I provide the simple Negative Look-ahead pattern...


    “(?!pattern)”

... and proceed to show the unexpected behavior and put it in perspective so that you can understand the behavior better. Though the pattern looks simple, its behavior is not. And frankly, I'm beginning to consider it a mis-use of look-around. Even my standard pattern for any sort of look-around goes something like this...


    [look-behind]pattern[look-ahead]

... where either "look-around" pattern is optional. This is certainly a valid use of the look-around as it creates required bounds around the pattern to be found. Or, to put it another way, the positive look-behind and look-ahead here describes the "context" of the pattern to match.

But in my earlier articles, I suggested that multiple constraints on a pattern was a lot more difficult. Yes, it's more difficult, but not really as difficult as I first believed. Once you understand the look-around pattern better than I did, in particular, once you understand the construction...


    [look-ahead]pattern[look-behind]

... you can begin writing more understandable multiple-constraint patterns.

Let's dig in. I've introduced two terms that need explanation. Context and Constraint are the terms I used for describing two important aspects of the pattern matching requirements.

When requirements dictate that we are looking for occurances of a particular pattern in a string, very often the match will be dependent upon the non-matched terms around it. This is the context of the match. For example, if we are looking for the word "an" in a paragraph, the implied context is that the two characters "an" are only a word if it is preceded by whitespace or starts the paragraph, and it is followed by whitespace or ends the paragraph. The simple pattern for the word "an" would be...


    \b[Aa]n\b

... but look-around syntax can also be used...


    [look-behind]pattern[look-ahead]
    (?<=^|\s+)[Aa]n(?=$|\s)

... thus expressing the context of the pattern that makes it a word. (Actually, word boundary must take into account words at the end of phrases and sentences which are followed with punctuation. For now, we'll keep it simple.)

Expanding the context, let's say we want to find the word "an" where ever it is mis-used. That would be every occurance of "an" that is not followed by a word beginning with a vowel. (Again, there are some exceptions. When an acronym begins with an F, H, L, M, N, R, S, or X, and we vocalize the letters as in FTP, we precede the acronym with the article "an" rather than "a". But when vocalizing the acronym as a word as in SCSI (pronounced scuzzy), we use the article "a". Again, let's keep it simple.) So, the pattern would look like this...


    \b[Aa]n\b(?=\s*[^AaEeIiOoUu])

... finding all occurances of the word "an" followed by a word that should use the article "a" instead. And a simple Regex.Replace() will fix such occurances as in this example...


            string test = 
@"This is a test of a string that mis-uses the word ""an"".  
An helicopter is incorrect in American english.  
But an elephant is correct usage.";
            string pattern = @"\b[Aa]n\b(?=\s*[^AaEeIiOoUu])";
            Console.WriteLine("{0}", Regex.Replace(test, pattern, "A"));

... Yes, it will replace "An" or "an" with the uppercase "A". An "Evaluator" parameter would make great sense here. But rather than getting bogged down in the details, I want to look at constraints right now.

Constraints generally provide the exceptions to the rule. Your search requirements may be to find words, but not all words. Constraints limit your matches to more specific values or exclude certain values. By their very nature, constraints are often best described with the boolean AND operator. According to my previous articles, you might conclude that there is no hope but to write a very obscure pattern using double negative and OR pattern matching. I'm happy to say, you have a few more options based upon the constraining construction using look-around. That construction would be...


    [look-ahead]pattern[look-behind]

... In this construction, when implemented correctly, the look-ahead and look-behind will both examine the same characters that pattern examines. But, look-around is non-consumptive. That means a look-around expression that matches does not appear in the Match.Value property of a Regex.Match() operation. For example...


    string test = "the end.";
    string pattern = "(?<=the )end";  //[look-behind]pattern
    Console.WriteLine("what'd I find ({0})",Regex.Match(test,pattern).Value);
    // Outputs:  what'd I find (end)

... So, the pattern "the " was found but was not considered part of the match.

The non-consumptive behavior comes in very handy when placing a look-ahead in front of a pattern. That's because the look-ahead AND the pattern must both match to have a match. Let's say we want to validate a string contains upper and lowercase letters and has at least one lowercase letter. You could write this pattern without look-around as follows...


    string pattern = "^[a-zA-Z]*[a-z]+[a-zA-Z]*$";

... or with look-ahead as ...


    string pattern = "^(?=[a-zA-Z]*$).*[a-z].*$";

... Notice that the look-ahead makes a general check that it only contains letters. But, because it does not consume the letters, the pattern following examines the same characters to make sure that the string contains a lowercase letter. The judicious placement of ^ and $ guarantee that both patterns examine the same set of characters. When arranged like this, the construction says that the look-ahead pattern AND the capture pattern must match or nothing matches.

Now, if you want to validate that the letter string has at least one uppercase and one lowercase letter, you cannot do it without look-around. The single pattern cannot be constructed in a way that does not impose an unwanted ordering. But with look-around, you only need to add additional look-around patterns like so...


    string pattern = "^(?=.*[A-Z].*$)(?=[a-zA-Z]*$).*[a-z].*$";

... Here's some code to demonstrate...


           string [] tests = {
                                  "abcdEFG",
                                  "ABCDEFG",
                                  "Abcd$fg",
                                  "abcdefg",
                              };
            string pattern1 = "^[a-zA-Z]*[a-z]+[a-zA-Z]*$";
            string pattern2 = "^(?=[a-zA-Z]*$).*[a-z].*$";
            string pattern3 = "^(?=.*[A-Z].*$)(?=[a-zA-Z]*$).*[a-z].*$";
            foreach (string test in tests)
            {
                Console.WriteLine("pattern1 {0} {1}", Regex.IsMatch(test, pattern1).ToString(), test);
                Console.WriteLine("pattern2 {0} {1}", Regex.IsMatch(test, pattern2).ToString(), test);
                Console.WriteLine("pattern3 {0} {1}", Regex.IsMatch(test, pattern3).ToString(), test);
            }

So then, we do have an AND operation implied in the way we combine the look-ahead and pattern. Similarly, the AND operation is implied when combining a pattern followed by the look-behind. But, in order for these to operate like an AND operation, the patterns in each look-ahead and the final pattern must overlap the string space exactly. I accomplished the overlap by placing the "^" start-of-string and "$" end-of-string in such a way that each individual pattern match refered to the same characters. Of course, you don't have to make them overlap precisely if you are looking for more exotic behavior than AND.

Here is another example of a pattern that's very difficult (if not impossible) to write without look-around. Consider a string with words in it. You want to capture all words except a few choice words, let's say "and", "but" and "or". You need a pattern that finds all words AND a pattern that rejects specific words. You can only reject specific words using negative look-around. Here is the pattern and some code to demonstrate...


            string pattern = @"\b\w+(?<!\band|\bbut|\bor)\b";
            string test =
                 "This is a test string, and it is good for finding words, but "
                 +"it will not find all words.  If you want to find all words, "
                 +"modify this or write your own.  Band is a word to make sure "
                 +"words are words, butter is another and so is order.";
            MatchCollection mx = Regex.Matches(test, pattern);
            foreach (Match m in mx)
                Console.WriteLine("{0}", m.Value);

... The pattern first finds the words, then the negative look-behind tosses out the ones we don't want.

Finally, let's look at that one more pattern. This would be the password validator pattern I covered in previous articles. But, this time we shall use a straight forward AND construction with look-ahead. A valid password will be only letters, and digits , will have at least one uppercase, one lowercase and one digit, and be at least 6 characters long. Here's the pattern...


    string pattern = "^(?=[a-zA-Z0-9]{6,}$)(?=.*[a-z].*$)(?=.*[A-Z].*$).*[0-9].*$";

That it for now. I hope you've found this useful.

Callbacks And Delegates

2008-10-15T21:59:00.007-04:00

It's been a couple of weeks since last writing. Besides being very busy, I've had a touch of "writer's block". Nothing too serious, I should get over it soon.

Lately, I've been intrigued by the abundant use of callback functions and delegates in the .NET Framework. If you want to do anything interesting, you gotta know callbacks. I'll try to describe callbacks and delegates in simplistic terms. Maybe it will help you over the hump.

Consider the following code...


    List list = new List();
    //Populate list with some MyClass objects
    for(int i=0; i<10; ++i)
        list.Add(new MyClass(){ ID=i, strValue=i.ToString() });
    //Sort the list
    list.Sort();
    foreach(MyClass c in list)
        Console.WriteLine("{0}",c.strValue);

    ...

    public class MyClass
    {
        public int ID;
        public string strValue;
        public override string ToString() { return strValue; }
    }

This code throws an exception at the line that says Sort(). That's because the default List.Sort() expects the objects it contains to implement IComparable. Now, we could quickly add IComparable to the class and implement the interface. But, if you don't have control of the class, what would you do then? Or if the built-in IComparable interface didn't sort on the field you want, what then?

You're in luck, because Sort() is overloaded and will accept as a parameter an IComparer or a Comparison object.


    public class CompareMyClass : IComparer
    {
        public int Compare(MyClass x, MyClass y)
        {
            return x.strValue.CompareTo(y.strValue);
        }
    }

With this piece of code, you can now call Sort() with a new CompareMyClass object and the exception goes away and the list comes out sorted. This is pretty cool, but it's also pretty static. If I wanted to sort by the ID field instead, I would have to change my class or write a new class. We can reduce some of that by going to the Comparison class. This is a generic class whose constructor takes as an argument a callback function. The Comparison constructor has the calling signature...



    Comparison.Comparison(int (T,T) target)

The generic parameter T can be anything. But when we see the signature in the constructor's parameter list, one might start scratching their heads. The best way to read the parameter signature is to start with the word "target". It could have been anything, but "target" suggests that it is the target of some operation. In fact, it is. It is the target function that Sort() will call over and over to determine the correct order of the list elements. The "int (T,T)" is the "type" of the parameter. This means that "target" is a function that takes two parameters of type T and returns an "int" value. Since T is a generic parameter we can replace it with MyClass as we do below. We no longer need the CompareMyClass class, but we do need a function to call.


    int MyCompare(MyClass x, MyClass y)
    { return x.strValue.CompareTo(y.strValue); }  

    // and Sort looks like this...
    list.Sort(new Comparison(MyCompare));

You could have two different functions to compare MyClass objects two different ways.



    int MyCompare2(MyClass x, MyClass y)
    { return x.ID.CompareTo(y.ID); }

But, then you have to plug in the correct function when you want a different sort behavior. If the comparison type might change at runtime based upon user input, you can set a "delegate" variable and provide that to Comparison() instead.


    // at class scope
    delegate int MyCompareDelegate(MyClass x, MyClass y); // declares a delegate type

...
    // at method scope
    MyCompareDelegate dlgt;  // declares a delegate variable
    if(radioButton1.Checked == true)
        dlgt = MyCompare;
    else
        dlgt = MyCompare2;

    list.Sort(new Comparison(dlgt));

The delegate lets us treat the functions as objects. Since a delegate for the callback will work just as well as the callback, we can write the comparison code right at the place where we use it with "anonymous delegates"...


            MyCompareDelegate dlgt;
            if (radioButton1.Checked == true)
                dlgt = delegate(MyClass x, MyClass y)
                {
                    return x.ID.CompareTo(y.ID);
                };
            else
                dlgt = delegate(MyClass x, MyClass y)
                {
                    return x.strValue.CompareTo(y.strValue);
                };
            //Sort the list
            list.Sort(new Comparison(dlgt));

Here we have set the "dlgt" delegate variable to one of two "anonymous" functions. Creating and anonymous function returns a delegate that can be assigned to a variable like any other function, as long as the signatures match. Well if that's the case, then the Lambda syntax should also work, shouldn't it?


            MyCompareDelegate dlgt;
            if (radioButton1.Checked == true)
                dlgt = (x,y) => x.ID.CompareTo(y.ID);
            else
                dlgt = (x,y) => x.strValue.CompareTo(y.strValue);
            //Sort the list
            list.Sort(new Comparison(dlgt));

Yes. The Lamda Expression implicitly determines the types of x, y and the return value from the delegate assignment and context. Pretty cool!

I stop there. This is starting to be too much fun. The power of the delegate and callback is huge. Keep playing with these and you'll get the hang of it.

A Simple Task Queue

2008-09-22T07:00:00.001-04:00

I’ve spent the last 4 posts talking about Regular Expressions and some difficult patterns. But, this is a C# blog, so I really want to be talking about C#. Today, I hope to provide you with a nice little start on a multi-threading "Task Queue" application. A Task Queue will place task requests in a queue that will be serviced Asynchronously and in the order received.

Rather than keep you in suspense, here’s the code up front. If you'd like an explanation, I've attempted that below. (Update 10/6/2008: Sorry folks about the bug below. The Enqueue method must set the the Busy field to true when queing the first task in order to avoid the thread race. It's now been fixed.)


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace TaskQueuePOC
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        int counter = 0;
        private void button1_Click(object sender, EventArgs e)
        {
            new MyTaskQueue(counter++).Enqueue();
        }
    }

    public class MyTaskQueue : TaskQueue
    {
        public MyTaskQueue(object UserData)
            : base(UserData)
        {
        }
        protected override void Task()
        {
            Thread.Sleep(2000);
            Console.WriteLine(UserData.ToString());
        }
    }

    public abstract class TaskQueue
    {
        public object UserData { get; private set; }
        public TaskDelegate TaskDlgt { get; private set; }
        protected abstract void Task();

        public void Enqueue()
        {
            TaskDlgt = new TaskDelegate(Task);
            lock(lockObject)
            {
                if(Busy)
                    _q.Enqueue(TaskDlgt);
                else {
                    Busy = true;
                    TaskDlgt.BeginInvoke(new AsyncCallback(this.TaskCallback), TaskDlgt);
                }
            }
        }

        private static Queue _q = new Queue();
        private static bool Busy = false;
        private static object lockObject = new object();

        public TaskQueue(object Data)
        {
            UserData = Data;
        }

        public delegate void TaskDelegate();

        private void TaskCallback(IAsyncResult ar)
        {
            TaskDelegate dlgt = ar.AsyncState as TaskDelegate;
            if(dlgt.Equals(TaskDlgt))
                dlgt.EndInvoke(ar);
            NextTask();
        }
        private void NextTask()
        {
            TaskDelegate dlgt;
            lock(lockObject)
            {
                if(_q.Count > 0)
                {
                    dlgt = _q.Dequeue();
                    dlgt.BeginInvoke(TaskCallback,dlgt);
                }
                else
                    Busy = false;
            }
        }
    }
}

I've titled the article "A Simple Task Queue", but simple is a little mis-leading. That's because it's difficult to tell by looking (for the beginner) where the threads are or even how it works. The key to understanding this implementation is to understand the "delegate BeginInvoke" call. I've covered that in other articles on threading, in particular, I refer you to "Threading with .NET ThreadPool Part 4" for a deeper discussion.

Overall, the work horse of this application is the Abstract Class TaskQueue. The class implements all that's needed to queue tasks and execute them in order. All the user need supply is a derived class that overrides "Task()" and a Constructor that passes in some UserData. The application would instantiate a new derived TaskQueue object with the data needed for the task and then call the Enqueue() method.

The TaskQueue works by creating 3 private static control members. There is the queue which holds delegates to run. There's also a Busy member to say when there is an active thread running. There is also a LockObject used internally to synchronize thread access to the the queue and the Busy indicator. This is needed because as one thread completes and tries to update the Busy indicator or take a new delegate off of the queue for execution, the "producer" thread (our UI in this case) may be trying to enqueue another delegate. Since two threads could be accessing these variables simultaneously, we synchronize them with a Mutex on the LockObject.

The multi-threading comes into play when a delegate's BeginInvoke(...) method is called. This method will allocate a thread from the ThreadPool and execute the Task in that thread. BeginInvoke is provided a TaskCallback() function and some state information. In this case, the state information is a reference to the delegate.

The Callback function is responsible for checking the queue and launching the next Task or setting Busy to false.

The magic in all of this is that the delegate is a reference to a specific instance of a TaskQueue object's Task() method. That way, the delegate's Task() method has access to local information about that specific task. So, each delegate will operate with its own version of UserData. Notice, this is also the reason that NextTask() dequeue's a delegate and places the reference in a local variable rather than the TaskDlgt member. TaskDlgt is a reference to one's self, while the delegate taken from the queue is a different delegate. So, each task's completion callback is responsible to start the next task (if any).

Also, notice that the Tasks are highly encapsulated. Once the task is on the queue, all that is available is the delegate. The rest of the task is somewhat hidden, though reflection can be used to make tweaks to the tasks and task data if necessary.

There are many improvements that can be made. Most notably, one could add a way to stop the queue and cancel remaining tasks. One could add controls to their derived class and overridden Task() member to allow individual task control or gross control over the entire queue.

There you have it, a "simple" Task Queue for your amusement and edification. Hopefully, you find it as useful as I have. Also, I have written several other articles on threads and threading. Please feel free to poke around my archives, you may find these articles useful as well.

10/6/2008: Recently, a kind reader pointed out a bug in my code (since fixed) where "Busy" was not being set to "true" anywhere. The obvious behavior was that all tasks ran immediately, no queing was done at all.

Regular Expression Double Negatives

2008-09-20T07:49:00.007-04:00

In my last article titled "Regular Expression Alternation" we discussed Alternation as one means of performing the AND operation in a regular expression pattern.


        (?(expression)yes|no)

But, what if you don't have this pattern in your version of RegEx? Some of you may be reading to gain knowledge that you can apply in other programming languages besides C# or the .NET environment. The pattern I described in the last article does not work in all versions of .NET Regular Expressions. In particular, it does not work in JScript Regular Expression Syntax. And, when you use tools like ASP.NET’s RegularExpressionValidator, you are implicitly using JScript. So then, what do you do?

To restate the original requirement, we want to validate a string that has at least one digit, at least one uppercase letter, and at least 6 characters. The problem statement is clearly an AND problem, so it begs to be written with the Alternation pattern. But in situations where we can’t use the Alternation, we can opt for my second approach. We can use the Negative Look-Ahead pattern.


 (?!expression)

The negative look-ahead pattern is “non-consuming” and matches strings (or parts of string) that are not followed with the “expression”. You may wish to brush up on negative look-ahead with my article “Regular Expressions in C# - Negative Look-Ahead”. With negative look-ahead and using the OR pattern, we can use double-negative logic to express the original problem. This approach works based upon the logic axiom…


    ( A  AND  B )  ==  NOT ( (NOT A) OR (NOT B) )

Using this rule, the problem can be stated differently. We can state our problem in terms of ORs instead of ANDs. In other words...

If it is not the case that our string is devoid of digits or devoid of uppercase letters or short of 6 characters, then it is a valid string.

Notice I chose terms that suggest negative tests. Take a moment and think about it.


 IF ( NOT ( NOT(Has Digit) OR NOT(Has Uppercase) OR NOT(6 or more chars) ) )

This has the same logic result as…


 IF ( (Has Digit) AND (Has Uppercase) AND (6 or more chars) )

If you can’t get your mind around this basic programming concept, then re-writing your pattern requirement in terms of ORs will be difficult. So, experiment on paper and with generic logic variables to prove to yourself that your re-written pattern logic means the same thing as the original.

Now let’s look at the actual Regular Expression pattern. The component patterns from which we build look like this...


    (^[^\d]*$)  string is devoid of digits
    (^[^A-Z]*$) string is devoid of uppercase letters
    (^.{0,5}$)  string has 5 or fewer characters

String them together to get the inner test ...


    (^[^\d]*$)|(^[^A-Z]*$)|(^.{0,5}$)

Sometimes you can pause here and simply test for your string to match. If you supply an invalid string, it should MATCH. If you provide a valid string, it should NOT match. So, at this point, we have the opposite of what we really want. (A validator may not allow you to test this way, you would have to write a C# program to use as a “test jig”). So, we now negate the inner pattern with the negative look-ahead syntax...


    (?!(^[^\d]*$)|(^[^A-Z]*$)|(^.{0,5}$))

This would work for the IsMatch() approach, but it won't quite work for the Match() approach. We have to add some bounds...


    ^.*(?!(^[^\d]*$)|(^[^A-Z]*$)|(^.{0,5}$)).*$

This pattern is much harder to decipher than the AND pattern using Alternation. But now, armed with the knowledge from these last two articles, you should be able to discern the meaning of such patterns. We should be able to plug this pattern in to all of the algorithms we covered in previous article and have it validate strings according to the requirement.

That’s all for now, I hope you enjoyed the discussion and find these examples useful.

Regular Expression Alternations

2008-09-14T20:25:00.012-04:00

I promised in my last article that I'd have more to say about regular expression. So here you have it. Today, we look at another tough problem for Regular Expressions. Let’s consider writing a pattern to validate that a string contains at least one digit, at least one uppercase letter, and at least 6 characters. One would think that this would be easy. But here we’re faced with an AND situation and the regular expression syntax doesn't provide an AND operator. Take, for example, any validation problem that has the form…

A and B and C

Without an AND operation in Regex, you are almost forced to go outside of the pattern and implement the test in multiple patterns and multiple passes of your validator. That may be the best approach, or it may be impossible if you are working with a blackbox validator and must provide a single Regex pattern.

I've found two approaches that can be applied to solve such problems. We'll discuss one of those approaches today. The approach is to use the “Alternation” pattern. MSDN documentation lists 3 Alternations, the simple OR, or vertical bar, the "expression" and the "name" alternation. Our problem requires the "expression" version.

(?(expression)yes|no)

This pattern is used like an IF-THEN-ELSE programming pattern. In the IF-THEN-ELSE pattern, the THEN portion can function like an AND. We could rewrite our test above as…

IF A THEN IF B THEN IF C THEN MATCH

But the alternation pattern isn’t as flexible as a programming language like C# or VB.Net. We can’t easily drop the ELSE as we did above. So, our pseudo code above would have to take the form…

IF A THEN
IF B THEN
IF C THEN
MATCH
ELSE
FAIL MATCH
ELSE
FAIL MATCH
ELSE
FAIL MATCH

The psuedo regex pattern using our original requirements looks something like this...


   (? (does string have an uppercase letter)
             ( ?(does string have a digit)
                     (? does string have at least 6 characters)
                            (match the whole string)
                          | (fail the match))
                   | (fail the match))
           | ( fail the match))

This should look very similar to the IF-THEN-ELSE statement above. All we have left now is to write the individual patterns. We need a pattern for each test, a pattern to match the whole string and a pattern that will never match any string.

(.*[A-Z].*) matches if there's at least one upper case letter in the string
(.*[0-9].*) matches if there's at lease one digit in the string
(.{6,}) matches if there are 6 or more characters in the string
(.*) matches the entire string
([^\W\w]) won't match anything

Notice how each test is written in such a way that the whole string is selected. That way all tests are operating on the exact same input. Now piece it together and it looks like this...

(?(.*[A-Z].*)(?(.*[0-9].*)(?(.{6,})(.*)|([^\W\w]))|([^\W\w]))|([^\W\w]))

It's not very pretty, but it's conceptually straight forward. It follows the pseudo-regex precisely. See if you can format it nicely like the pseudo-pattern above.

The final step is to verify our pattern with some code. We’ll use C# for that. Just create a new Windows Forms Application project in C#. On the form, add a reference to the Regular Expression name space.


    using System.Text.RegularExpressions;

Next drop a Label and a Textbox on the form. Clear the text in label1. Then double click the textbox and insert into the textBox1_TextChanged handler the following code…


    string pattern = @"(?(.*[A-Z].*)(?(.*[0-9].*)(?(.{6,})(.*)|([^\W\w]))|([^\W\w]))|([^\W\w]))";
    if(Regex.IsMatch(textBox1.Text,pattern))
    {
        label1.Text = "Valid";
        label1.ForeColor = Color.Black;
    }
    else
    {
        label1.Text = "Invalid Entry";
        label1.ForeColor = Color.Red;
    }

The program will display “Valid” or “Invalid Entry” depending upon the input in the text box. This is a fairly simple password validator, but now you have the tools to expand upon it if you like. There is one caveat though. This pattern will not work in the ASP.Net RegularExpressionValidator component. In my next article on Regular Expressions, I will explain why. I will also explain my second approach mentioned above which will work in the RegularExpressionValidator situation.

Regular Expressions in C# - Negative Look-ahead

2008-09-12T21:03:00.005-04:00

In my last article on Regular Expressions, we looked at a couple of simple expressions and 3 algorithms to use the expressions for “validating” strings. The main point of the article is that Regular Expression behavior will be confusing if not considered in the context of the algorithm being used. Please take a look at Regular Expressions. What’s that got to do with C#? to get the background for this article.

In this article, we will discuss one of the more difficult Regular Expression problems. How do you use “Negative Look-ahead”? To use Regular Expression “Negative Look-Ahead” or “Negative Look-Behind”, you have to change the way you think about pattern matching. First, the negative look-ahead takes the syntax…


 “(?!pattern)”

In the words of the MSDN documentation…

(Zero-width negative lookahead assertion.) Continues match only if the subexpression does not match at this position on the right. For example, \b(?!un)\w+\b matches words that do not begin with un.

This expression will “find” parts of the supplied string that are not followed by the “pattern”. One might erroneously think that this pattern will refuse to match strings that have the “pattern” in them. It does not, in fact, the following example actually “finds” some strange results.


        string pattern = @"(?!invalid)"; //negative lookahead
        string test = "invalid";
        Regex rx = new Regex(pattern);
        Console.WriteLine("Match:\t\t\t{0}\t({1})", rx.Match(test).Success.ToString(), test);
        Match mx = rx.Match(test);
        while (mx.Success)
        {
            foreach (Group g in mx.Groups)
            {
                Console.WriteLine("\t\t\t\t({0}) ({1}): {2}", mx.Value, g.Value, g.Index);
            }
            mx = mx.NextMatch();
        }

The result is that there are actually 7 matches in the test string. The confusing matter is that each of the matches is an empty string, and the index for each match increments, from 1 to the number of characters in the test string. To understand this behavior, let’s think about the pattern differently. Let’s break it apart into some simple components. You could consider the pattern to be equivalent to…


 “” + “(?!invalid)” + “”

This is a pattern to match an empty string, followed by a pattern to reject the string “invalid”, followed by a pattern to match an empty string. In short, without the negative look-ahead, this is a pattern to match an empty string. If you were to use just the simple empty string pattern on the test string, there would be 8 matches with indices for each match incrementing from 0 to the number of characters in the test string. There are 7 matches with the negative look-ahead and 8 matches without. The indices start at 1 with the negative look-ahead, and they start at 0 without. So, the negative look-ahead is causing one of the potential matches to fail. It’s rejecting the empty string match that occurs just prior to the first character in the test string. I.e. it is rejecting the only empty string match that is followed by “invalid” and matching all the others.

Oh my head is spinning! Why did I eliminate the negative look-ahead above? I did so to understand its effect on the entire pattern matching process. And I learned that the negative is actually finding a pattern to eliminate. In other words, it’s doing its job. It is eliminating from the set of matches the one match that is followed by the string “invalid”. But, because there are many other “empty-string” matches, IsMatch() returns “true” and Match() returns 7 matches. I also eliminated the negative look-ahead because look-around” patterns do not “consume” characters. They will never show up in the match result, so by temporarily removing the non-consuming pattern, I can see the real pattern that the look-around pattern constraints.

The Regular Expression philosophy is to find things, not to eliminate things. Since Regex works so hard at finding matches, it becomes difficult to write patterns whose job is to exclude strings. One could just write the pattern to find the strings that are unwanted and negate the match in program logic. Negative matching syntax is available, but Regex treats such syntax as a means of reducing the number of matches while trying to find ANYTHING that would otherwise match.

So how do I make this work? Consider the requirement to match only those strings that do not have the sequence “invalid” in it. First, you must define a pattern to match any otherwise valid string. You must define the pattern in such a way that it will match the string using the most restrictive form of the 3 programming approaches described in my last article. In particular, your pattern must pass the following test…


 Match mx = Regex.Match(test, pattern);
 if(mx.Success && (mx.Groups[0].Value == test))

The test above makes sure that not only does the test string “have” a match but the test string “is” the match. The empty string pattern will find many matches, but the whole match will not equal the test string for any test string other than an empty string. You have to watch out for the asterisk (*) and plus (+) which will consume as much as they can, or as little as necessary to achieve a match. Such patterns will work in the logic above, but they can change behavior as you start adding look-around patterns. In the case of our requirement, the all encompassing pattern would be “^.*”. Leave off the “$” because we will be using look-ahead. When looking ahead, we do not want to anchor the end of the string unless absolutely necessary.

Next, define a pattern to find whole strings that you’d like to exclude. “.*invalid.*” works. This pattern matches any string containing the sequence “invalid”. Next, wrap this pattern with the negative look-ahead syntax “(?.*invalid.*)”. And finally, insert it into the first pattern after the first anchor (^ in our case). Our resulting pattern would be “^(?!.*invalid.*).*”. Use the following test jig to prove this pattern to yourself.


        string pattern = @"^(?!.*invalid.*).*";//negative look-ahead
        string[] tests = {
                 "invalid",
                 "",
                 "this is also invalid",
                 "but this is okay",
                 "but this invalid string is not",
                          };

            Regex rx = new Regex(pattern);
            foreach (string test in tests)
            {
                Console.WriteLine("Match:\t\t\t{0}\t({1})", rx.Match(test).Success.ToString(), test);
                Match mx = rx.Match(test);
                while (mx.Success)
                {
                    foreach (Group g in mx.Groups)
                    {
                        Console.WriteLine("\t\t\t\t({0}) ({1}): {2}", mx.Value, g.Value, g.Index);
                    }
                    mx = mx.NextMatch();
                }
            }

I’ll stop here for today and let you digest what’s going on. Certainly, the pattern can be optimized and some characters reduced. Experiment with changes and observe the effects. I am not through with the subject of Regular Expressions and Negative type matching. Check back later as I hope to post on the subject again soon.

Regular Expressions? What's That Got To Do With C#?

2008-09-05T13:25:00.007-04:00

...Only that I often need to know Regular Expressions for my C# work. However, the online help and resources seem to come up a little short. So, today I diverge a little and discuss this cryptic yet valuable ancillary topic to try to help you through your next Regex dilema.

I'm not going to waste time and internet bandwidth explaining what a Regular Expression is, there are plenty of sites for those. But, I will give special thanks here to OmegaMan who compiled and posted the following on the MSDN Regex Forum...

OmegaMan's .Net Regex Resources Reference

I refer to it often.

So let's dive right in. Whenever you are considering using regular expressions, you need to determine what kind of pattern matching problem are you trying to solve.

1) Do I want a regular expression to check a string for validity?
2) Do I want a regular expression to find certain things in my string?
3) Do I want a regular expression so that I can replace patterns in my string?

There can be some overlap in #1 and #2 since validity may depend upon the string containing "certain things". Certainly, if you want to replace something in #3, you need to find it in the string with #2. But, many of the problems people run into with regular expressions can be traced to not having identified the problem properly.

Take the problem, "make sure my string contains only letters and digits". Sounds simple enough, and one may write the pattern ...


   "[\da-zA-Z]*"
\d = digit
a-z = lowercase letter
A-Z = uppercase letter
[]* = zero or more of the characters in the brackets

This pattern might "find" letters and digits in a string, but it doesn't say that there are ONLY letters and digits in that string. Regex.IsMatch(...) tests a string with a pattern and returns true if the string contains a match. Regex.Match(...) tests a string with a pattern and returns a Match object indicating if the string contains matches and details, if any, about each match.

So if you are doing a validity check with the pattern above, your results will depend on which tools you use and how you use them. Given that pattern, both IsMatch() and Match() will find matches in a string, even if it contains undesirable characters. In fact, because of the asterisk (*), the string doesn't have to contain any of the pattern characters for there to be "a match" (it matches the empty string). These functions, given this pattern, are simply indicating whether or not any part of the string matches the pattern. Here's some code to demonstrate...

Example 1:


        string pattern = @"[\da-zA-Z]*"; //use the @ to tell c# to leave \ alone
        string[] tests = {
            "containsOnlyLettersAnd01234",
            "contains letters And 01234, but also spaces",
            "!@#$%", // contains none of the desired characters
            "", // a completely empty string
                       };
        foreach(string test1 in tests)
            Console.WriteLine("IsMatch:\t{0}\t({1})",Regex.IsMatch(test1,pattern).ToString(),test1);
        foreach(string test2 in tests)
            Console.WriteLine("Match:\t{0}\t({1})",Regex.Match(test2,pattern).Success.ToString(), test2);

...outputting...


IsMatch: True (containsOnlyLettersAnd01234)
IsMatch: True (contains letters And 01234, but also spaces)
IsMatch: True (!@#$%)
IsMatch: True ()
Match: True (containsOnlyLettersAnd01234)
Match: True (contains letters And 01234, but also spaces)
Match: True (!@#$%)
Match: True ()

Obviously, several of these strings are not valid by our requirements. So, what went wrong, and how would you "validate" the string? In order to do the desired validity check, one must consider both the pattern and the Regex method that will be used. For instance, the pattern as written will validate if you write the supporting code to accommodate it. For example...

Example 2:


    Regex rx = new Regex(pattern);
    foreach(string test1 in tests)
        Console.WriteLine("Modified Match:\t{0}\t({1})",
               (rx.Match(test1).Success && (rx.Match(test1).Value == test1)).ToString(),
                test1);

...outputting...


Modified Match: True (containsOnlyLettersAnd01234)
Modified Match: False (contains letters And 01234, but also spaces)
Modified Match: False (!@#$%)
Modified Match: True ()

Now that looks much better. Our pattern match now "works" except for the empty string. The requirement might be considered vague and allow for such a match. Many applications will have an input text box that starts out blank. When the user enters characters, the text is then validated. These apps usually explicitly test for empty text. Case in point, the ASP.NET RegularExpressionValidator states that it will not validate the empty string, i.e., empty strings will PASS. It is up to the programmer to require some input. By the way, RegularExpressionValidator does pattern matching validation on both the client and the server. On the client, it uses JScript Regular Expression syntax, which has a smaller feature set and syntax than the server uses. It also uses the same program construction as the second example.


     if(match != null && (match[0] == value))  // valid

If you use other tools, you must know how the validity test is done. For, in the last example, patterns that have look-ahead or look-behind will often fail. They will look perfectly valid, and they will match using IsMatch(), but they require that you add some additional pattern to consume those characters that look-around does not consume. We'll get into that in a future article.

Now the pattern could have been written differently. Using the next pattern example, you can use any of the three methods to validate that the string contians only letters and digits. This time, I also require that the input string not be blank.

"^[\da-zA-Z]+$"
^ = match the beginning of string/line, zero width pattern
$ = match the end of the string/line, zero width pattern
[]+ = 1 or more of the characters in the brackets

Example 3:


        string pattern2 = @"^[\da-zA-Z]+$";
        Regex rx = new Regex(pattern2);
        foreach (string test in tests)
        {
            Console.WriteLine("Modified Match:\t{0}\t({1})",
                   (rx.Match(test).Success && (rx.Match(test).Value == test)).ToString(),
                    test);
            Console.WriteLine("IsMatch:\t\t{0}\t({1})", rx.IsMatch(test).ToString(), test);
            Console.WriteLine("Match:\t\t\t{0}\t({1})", rx.Match(test).Success.ToString(), test);
        }

...outputting...


Modified Match: True (containsOnlyLettersAnd01234)
IsMatch:  True (containsOnlyLettersAnd01234)
Match:   True (containsOnlyLettersAnd01234)
Modified Match: False (contains letters And 01234, but also spaces)
IsMatch:  False (contains letters And 01234, but also spaces)
Match:   False (contains letters And 01234, but also spaces)
Modified Match: False (!@#$%)
IsMatch:  False (!@#$%)
Match:   False (!@#$%)
Modified Match: False ()
IsMatch:  False ()
Match:   False ()

In otherwords, the pattern and the approach must work together. When you have control of both, then solving the problem is easier. But as you can see in the last example, all 3 approaches can validate according to our requirements by tweaking the pattern. The trick is often to find the tweak that works in all cases. When you can't change the underlying programming, like in the case of RegularExpressionValidator, you have to be able to write your pattern to "match" the underlying approach.

Now these are simple examples and I would have liked to get into some real meaty Regex expressions, but I've run out of time, and this posting is late. I'll be back though with more on Regular Expressions in the next article. For now, I hope this is of some use to you.

Enumerations and Strings

2008-09-01T10:50:00.008-04:00

It has been a while since my last post. Things get crazy at times. But, here is another bit of sample code for you to muse over. We will look at enumerations and strings.

In C#, an enumeration is NOT a string, nor can you define it to be one. An enumeration can be defined as any of a number of integer types. You can leave it as its default type...


    public enum unspecifiedTypeEnum
    {
       one = 1, two, three,
    }

In which case you get named 32 bit values. Or you can specify the type of the underlying value as byte, sbyte, ushort, short, uint, int, ulong or long...


    public enum byteTypeEnum : byte
    {
        one = 1, two, three,
    }
    // or...
    public enum ushortTypeEnum : ushort
    {
        one = 1, two, three,
    }
    // etc...

... But, you can't declare it as a string.

That doesn't mean you can't use strings at all. When I want to save data to a file for future reference, I like to make my enumerations human readable and store them in the file as human-readable. It would be a shame to use enumerations to write clear code only to store them as very cryptic values in my maintenance and configuration files. Getting the string representation given the enumeration name is easy. We just use the object.ToString() overloaded method. For enumerations, this returns the name of the value as a string...


    public enum Animals { cat, mouse, bird, dog, }
    //....

    Animals myAnimal = Animals.cat;
    Console.WriteLine("CurrentAnimal={0}",myAnimal.ToString());

When I load a configuration value back in to my program, I would want to work with the value as the original enumeration type. That's a little trickier, but with the help of "reflection", it can be done. This little routine shows how...


    using System.Reflection;
    // ...

        Animals FromString(string animal)
        {
            Type t = typeof(Animals);
            FieldInfo[] fi = t.GetFields();

            try
            {
                foreach (FieldInfo f in fi)
                {
                    if (f.Name.Equals(animal))
                    {
                        return (Animals)f.GetRawConstantValue();
                    }
                }
            }
            catch
            {
            }
            throw new Exception("Not an Animal");
        }

Actually, it's not so tricky. A reader (thankyou Paul) pointed me to the "static" Enum functions which do much of the hard work for you. The function above can be more simply written as...


    Animals FromString(string animal)
    {
        try
        {
           return (Animals)Enum.Parse(typeof(Animals), animal);
        }
        catch (ArgumentException ex)
        {
            throw new Exception("(" + animal + ")" is not in the Animals enumeration.",ex);
        }
    }

This gives me an enumeration value for my string, assuming the string matches one of the enumeration names. Enhancements could be made to this code to store and retrieve the fully qualified name. I'll leave that as an exercise for the reader.

You may be wordering about the try and catch above. The GetRawConstantValue() returns a value that we cast to our enumeration type. If that value is invalid for our enumeration, then the cast throws an exception. Also, understand that there are other values in the FieldInfo array besides just enumeration fields. Enumerations have hidden fields that are accessible through reflection. Under contrived circumstances, the caller may pass in the name of one of these hidden fields. The "animal" name will match a field in the FieldInfo array, but the cast with throw an exception as we want it to.

Now, with the ability to convert between string and enumeration, you can write your code so that everything internal is performed on the enumeration type, while persistance and other external representation can be strings. But, what if we want a list of all possible values. We might let the user choose from a list, how might we get that? Here's a way to create a string list for the enumeration...


    // also requires 
    using System.Reflection;
    ...

        IEnumerable AnimalsList()
        {
            Type t = typeof(Animals);
            foreach (FieldInfo f in t.GetFields())
            {
                try
                {
                    if (f.GetRawConstantValue() is Animals)
                        ;
                }
                catch
                {
                    continue;
                }
                yield return f.Name;
            }
        }

...or simply ...


        IEnumerable AnimalList()
        {
            return Enum.GetNames(typeof(Animals)).ToList();
        }

Now in your Form constructor you can write the following to show your enumerations as strings.


    listBox1.Items.AddRange(AnimalsList().ToArray());

Someone will invariably want to associate a different string to their enumeration than the name. Maybe they want to obscure the meaning. More likely, the programmer wants to change the enumeration after there are already configuration files in production using older enumeration names. Changing the enumeration could break existing configuration files. But you have to deal with that regardless of how you store the data. I won't go into code examples, but a Dictionary object could be used internally. Another possibility is the string name and the enumeration name may differ by case only, but it should be a trivial modification to deal with that condition, so I leave that also as an exercise for the reader.

Finally, these enumeration functions are perfect for making Generic. They could be used on several enumerations in our code, and we wouldn't want to re-implement it each time. So, I will leave you with the following generic example of Enumerations and Strings.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Reflection;

namespace EnumExample
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            listBox1.Items.AddRange(EnumList().ToArray());
        }

        public enum Animals { dog = 1, cat, mouse, bird, }

        private void button1_Click(object sender, EventArgs e)
        {
            Animals myAnimal = Animals.bird;
            Console.WriteLine("CurrentAnimal={0}", myAnimal.ToString());

            myAnimal = FromString("cat");
            try {
                myAnimal = FromString("value__");
            } catch (Exception ex) {
                Console.WriteLine("error: {0}", ex.Message);
            }
        }

        T FromString(string animal) where T : struct
        {
            Type t = typeof(T);
            FieldInfo[] fi = t.GetFields();

            try
            {
                foreach (FieldInfo f in fi)
                {
                    if (f.Name.Equals(animal))
                    {
                        return (T)f.GetRawConstantValue();
                    }
                }
            }
            catch
            {
            }
            throw new Exception("Not a Type " + typeof(T).Name);
        }

        IEnumerable EnumList()
        {
            Type t = typeof(T);
            foreach (FieldInfo f in t.GetFields())
            {
                try
                {
                    if (f.GetRawConstantValue() is T)
                        ;
                }
                catch
                {
                    continue;
                }
                yield return f.Name;
            }
        }
    }
}

Threading with .NET ThreadPool Part 4

2008-08-22T20:50:00.013-04:00

Suppose you've been given the task to write a function that copies the contents of one folder to another. So you set off on your merry way and come up with something like the following.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO; 

namespace ThreadPoolPart4
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        
        private void button1_Click(object sender, EventArgs e)
        {
            // Get the Folder names, copy contents from one to the other
            FolderBrowserDialog fb = new FolderBrowserDialog();
            fb.ShowDialog();
            string src = fb.SelectedPath;
            fb.ShowDialog();
            string dst = fb.SelectedPath;
            // no error checking on the names, this is an example only

            if (dst != src)
            {
                if (!Directory.Exists(dst))
                    Directory.CreateDirectory(dst);
                RecurseCopyFolder(new DirectoryInfo(src), new DirectoryInfo(dst), MyCopyCallback);
            }
        }
        private void RecurseCopyFolder(DirectoryInfo src, DirectoryInfo dst, CopyCallbackDelegate cb)
        {
            bool cancelled = false;
            CopyArgs ca;
            try
            {
                foreach (FileInfo fi in src.GetFiles())
                {
                    string newfile = Path.Combine(dst.FullName, fi.Name);

                    ca = new CopyArgs() { IsDir = false, CurrentObject = newfile };
                    cb(ref cancelled, ca);
                    if (cancelled)
                        break;
                    if (!ca.Skip)
                    {
                        if (Directory.Exists(ca.CurrentObject))
                            Directory.Delete(ca.CurrentObject);
                        fi.CopyTo(ca.CurrentObject, true);
                    }

                }
                if (!cancelled)
                    foreach (DirectoryInfo subsrc in src.GetDirectories())
                    {
                        ca = new CopyArgs() { IsDir = true, CurrentObject = Path.Combine(dst.FullName, subsrc.Name) };
                        cb(ref cancelled, ca);

                        if (cancelled)
                            break;
                        if (ca.Skip)
                            continue;
                        DirectoryInfo subdst;
                        if (!Directory.Exists(ca.CurrentObject))
                            subdst = dst.CreateSubdirectory(subsrc.Name);
                        else
                            subdst = new DirectoryInfo(ca.CurrentObject);

                        RecurseCopyFolder(subsrc, subdst, cb);
                    }
            }
            catch (Exception e)
            {
                throw e;
            }
        }

        public class CopyArgs : EventArgs
        {
            public bool IsDir;
            public bool Skip;
            public string CurrentObject;
        }

        private delegate void CopyCallbackDelegate(ref bool Cancel, CopyArgs args);
        private void MyCopyCallback(ref bool Cancel, CopyArgs args)
        {
                if (args.IsDir)
                    label1.Text = args.CurrentObject;
                else
                    label2.Text = args.CurrentObject;
        }

    }
}

But when you test this on a somewhat larger folder, your application appears to hang. What gives? Well, it's very common for programmers to hang up their GUI by waiting, sleeping, reading the network, or in this case, doing intensive file I/O operations on the GUI thread. It's been said hundreds if not thousands of times on the forums, "don't Sleep() in the GUI thread". Because, if your GUI gets stuck in a Sleep() or anywhere else, it does not get a chance to pump its message loop. A GUI that does not service its message queue looks like it is dead. That's why sending your GUI down a recursive file copy excursion like we just did is a really bad idea.

That is also why threading has become a best practice for C# application development. And today, we take another look at the System.Threading.ThreadPool. In my previous articles on the ThreadPool ("Threading with .NET ThreadPool, Part 1, Part 2 and Part 3"), I spent the entire time talking about ThreadPool.QueueUserWorkItem(). But, there are other ways within your power to put the ThreadPool to work for you. Specifically, I am talking about the delegate.BeginInvoke() call. Let us see how it is used.

First off, to solve our dead looking GUI application (if your's doesn't look dead, you didn't try copying a big enough folder), we need to put the "heavy lifting" into a thread. All the hard work occurs in RecurseCopyFolder(), so we will put that call into a thread using BeginInvoke(). We do it by declaring a delegate for the function, then calling the delegate's BeginInvoke() like so...


        private delegate void RecurseCopyFolderDelegate(DirectoryInfo src, DirectoryInfo dst, CopyCallbackDelegate cb);
        private void RecurseCopyFolder(DirectoryInfo src, DirectoryInfo dst, CopyCallbackDelegate cb)
        { //...
        }

You should be familiar with delegates, but if not, just note how the delegate call signature is exactly the same as the function it will delegate for. I name them similarly for the sake of self-documenting code.

Next, we create an instance of the delegate. I create a class scope variable to hold the delegate instance because I need it later. I also create an IAsyncResult variable because I will be needing that, too when the thread completes.


        RecurseCopyTreeDelegate dlgt;
        IAsyncResult  asyncResult;

Now, we instantiate the delegate and get a new thread started with BeginInvoke(). Notice that BeginInvoke() takes the same parameters as the original function (plus two, we'll come back to those later). The "delegate" is implemented by the compiler, so the compiler also does us the favor of letting us call the BeginInvoke() almost like we would call the delegate itself.


                // RecurseCopyFolder(new DirectoryInfo(src), new DirectoryInfo(dst), MyCopyCallback);
                dlgt = new RecurseCopyFolderDelegate(RecurseCopyFolder);
                asyncResult = dlgt.BeginInvoke(new DirectoryInfo(src), new DirectoryInfo(dst), MyCopyCallback, null, null);

We are almost ready to run, but notice we are using a callback function to update our GUI. You never update the GUI directly from a different thread, its not thread-safe. So we will add the code to make the callback check the form's InvokeRequired property and Invoke() the call if necessary. It is common to have one function check the InvokeRequired and call itself again with Invoke(), so I've modified MyCopyCallback() to do just that.


        private delegate void CopyCallbackDelegate(ref bool Cancel, CopyArgs args);
        private void MyCopyCallback(ref bool Cancel, CopyArgs args)
        {
            if (this.InvokeRequired)
            {
                this.Invoke(new CopyCallbackDelegate(MyCopyCallback), new object[] { Cancel, args });
            }
            else
            {
                if (args.IsDir)
                    label1.Text = args.CurrentObject;
                else
                    label2.Text = args.CurrentObject;
            }
        }

There's still one more requirement for using BeginInvoke(). When using the "delegate" BeginInvoke(), you are required to call the delegate's EndInvoke() when the thread completes. So, in order to know when the function completes, you can use a timer (remember to enable it) and check the asyncResult variable...


        private void timer1_Tick(object sender, EventArgs e)
        {
            if(asyncResult != null)
                if (asyncResult.IsCompleted)
                {
                    dlgt.EndInvoke(asyncResult);
                    asyncResult = null;
                    timer1.Enabled = false;
                }
        }

But, even better than a timer would be to use those two nice little parameters at the end of BeginInvoke(). They are the "Thread Completion Callback" and the "User Data Object". You supply a callback that will get called when the thread completes. It is called with the user data object passed in inside of the IAsyncResult parameter. We normally pass the delegate itself as the user data object so that we can use it to call the EndInvoke() with. This allows us to eliminate those class scope variables we added earlier (though I don't, I'll leave it as an exercise). So we still have to write a completion routine.


        public void RecurseCopyDone(IAsyncResult result)
        {
            if (this.InvokeRequired)
            {
                this.Invoke(new AsyncCallback(RecurseCopyDone), new object[] { result });
            }
            else
            {
                RecurseCopyFolderDelegate d = result.AsyncState as RecurseCopyFolderDelegate;
                d.EndInvoke(result);
                label1.Text = "Done";
                label2.Text = "Done";
            }
        }

The Invoke() above is not necessary to call the delegate's EndInvoke(), but it is needed to update my labels, so I opt to keep all the code together. EndInvoke() can be run from either the GUI thread or the callback thread.

So, there you have it. The complete source that follows of the Form1.cs and the Form1.Designer.cs has an additional button allowing us to cancel the operation. Copying files the way I am doing it cannot be interrupted, so cancellation occurs between files. The final version uses the IAsyncCallback instead of a timer. I hope this has been of use to you. Here's the code...

Form1.cs


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO; 

namespace ThreadPoolPart4
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private bool CancelCopyFlag = false;
        private IAsyncResult asyncResult = null;
        private RecurseCopyFolderDelegate dlgt;
        private void button1_Click(object sender, EventArgs e)
        {
            CancelCopyFlag = false;

            // Get the Folder names, copy contents from one to the other
            FolderBrowserDialog fb = new FolderBrowserDialog();
            fb.ShowDialog();
            string src = fb.SelectedPath;
            fb.ShowDialog();
            string dst = fb.SelectedPath;
            // no error checking on the names, this is an example only

            if (dst != src)
            {
                if (!Directory.Exists(dst))
                    Directory.CreateDirectory(dst);
                // RecurseCopyFolder(new DirectoryInfo(src), new DirectoryInfo(dst), MyCopyCallback);
                dlgt = new RecurseCopyFolderDelegate(RecurseCopyFolder);
                asyncResult = dlgt.BeginInvoke(new DirectoryInfo(src), new DirectoryInfo(dst), MyCopyCallback, new AsyncCallback(RecurseCopyDone), dlgt);
            }
        }
        private delegate void RecurseCopyFolderDelegate(DirectoryInfo src, DirectoryInfo dst, CopyCallbackDelegate cb);
        private void RecurseCopyFolder(DirectoryInfo src, DirectoryInfo dst, CopyCallbackDelegate cb)
        {
            bool cancelled = false;
            CopyArgs ca;
            try
            {
                foreach (FileInfo fi in src.GetFiles())
                {
                    string newfile = Path.Combine(dst.FullName, fi.Name);

                    ca = new CopyArgs() { IsDir = false, CurrentObject = newfile };
                    cb(ref cancelled, ca);
                    if (cancelled)
                        break;
                    if (!ca.Skip)
                    {
                        if (Directory.Exists(ca.CurrentObject))
                            Directory.Delete(ca.CurrentObject);
                        fi.CopyTo(ca.CurrentObject, true);
                    }

                }
                if (!cancelled)
                    foreach (DirectoryInfo subsrc in src.GetDirectories())
                    {
                        ca = new CopyArgs() { IsDir = true, CurrentObject = Path.Combine(dst.FullName, subsrc.Name) };
                        cb(ref cancelled, ca);

                        if (cancelled)
                            break;
                        if (ca.Skip)
                            continue;
                        DirectoryInfo subdst;
                        if (!Directory.Exists(ca.CurrentObject))
                            subdst = dst.CreateSubdirectory(subsrc.Name);
                        else
                            subdst = new DirectoryInfo(ca.CurrentObject);

                        RecurseCopyFolder(subsrc, subdst, cb);
                    }
            }
            catch (Exception e)
            {
                throw e;
            }
        }

        public void RecurseCopyDone(IAsyncResult result)
        {
            if (this.InvokeRequired)
            {
                this.Invoke(new AsyncCallback(RecurseCopyDone), new object[] { result });
            }
            else
            {
                RecurseCopyFolderDelegate d = result.AsyncState as RecurseCopyFolderDelegate;
                d.EndInvoke(result);
                label1.Text = "Done";
                label2.Text = "Done";
            }
        }

        public class CopyArgs : EventArgs
        {
            public bool IsDir;
            public bool Skip;
            public string CurrentObject;
        }

        private delegate void CopyCallbackDelegate(ref bool Cancel, CopyArgs args);
        private void MyCopyCallback(ref bool Cancel, CopyArgs args)
        {
            if (this.InvokeRequired)
            {
                this.Invoke(new CopyCallbackDelegate(MyCopyCallback), new object[] { Cancel, args });
            }
            else
            {
                if (args.IsDir)
                    label1.Text = args.CurrentObject;
                else
                    label2.Text = args.CurrentObject;
                Cancel = CancelCopyFlag;
                if (Cancel)
                {
                    label1.Text = "Cancelled";
                    label2.Text = "Cancelled";
                }
            }
        }

        private void button2_Click(object sender, EventArgs e)
        {
            CancelCopyFlag = true;
        }
    }
}

Form1.Designer.cs


namespace ThreadPoolPart4
{
    partial class Form1
    {
        private System.ComponentModel.IContainer components = null;

        protected override void Dispose(bool disposing)
        {
            if (disposing && (components != null))
            {
                components.Dispose();
            }
            base.Dispose(disposing);
        }

        #region Windows Form Designer generated code

        private void InitializeComponent()
        {
            this.components = new System.ComponentModel.Container();
            this.button1 = new System.Windows.Forms.Button();
            this.label1 = new System.Windows.Forms.Label();
            this.label2 = new System.Windows.Forms.Label();
            this.button2 = new System.Windows.Forms.Button();
            this.SuspendLayout();
            // 
            // button1
            // 
            this.button1.Location = new System.Drawing.Point(18, 17);
            this.button1.Name = "button1";
            this.button1.Size = new System.Drawing.Size(75, 23);
            this.button1.TabIndex = 0;
            this.button1.Text = "button1";
            this.button1.UseVisualStyleBackColor = true;
            this.button1.Click += new System.EventHandler(this.button1_Click);
            // 
            // label1
            // 
            this.label1.AutoSize = true;
            this.label1.Location = new System.Drawing.Point(22, 71);
            this.label1.Name = "label1";
            this.label1.Size = new System.Drawing.Size(35, 13);
            this.label1.TabIndex = 1;
            this.label1.Text = "label1";
            // 
            // label2
            // 
            this.label2.AutoSize = true;
            this.label2.Location = new System.Drawing.Point(22, 113);
            this.label2.Name = "label2";
            this.label2.Size = new System.Drawing.Size(35, 13);
            this.label2.TabIndex = 2;
            this.label2.Text = "label2";
            // 
            // button2
            // 
            this.button2.Location = new System.Drawing.Point(169, 22);
            this.button2.Name = "button2";
            this.button2.Size = new System.Drawing.Size(75, 23);
            this.button2.TabIndex = 3;
            this.button2.Text = "button2";
            this.button2.UseVisualStyleBackColor = true;
            this.button2.Click += new System.EventHandler(this.button2_Click);
            // 
            // Form1
            // 
            this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
            this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
            this.ClientSize = new System.Drawing.Size(292, 266);
            this.Controls.Add(this.button2);
            this.Controls.Add(this.label2);
            this.Controls.Add(this.label1);
            this.Controls.Add(this.button1);
            this.Name = "Form1";
            this.Text = "Form1";
            this.ResumeLayout(false);
            this.PerformLayout();

        }

        #endregion

        private System.Windows.Forms.Button button1;
        private System.Windows.Forms.Label label1;
        private System.Windows.Forms.Label label2;
        private System.Windows.Forms.Button button2;
    }
}

Generics Fast

2008-08-18T22:22:00.014-04:00

You are trying to come up to speed in C# fast. Then all of a sudden, you come across the C# Generic Type and wonder how do I master that.

Well, I've got some good news for you. You are in good company. You thought I was going to say I'll help you master generics. No, that's beyond a simple blog like mine. But, maybe I can help you become proficient. We just have to get the main learning hurdles out of the way.

Hurdle #1 - Rationale
Generic Types were added to C# in version 2.0. Now, I am not going to go into any detail on the history of C# or generics. I just mention this because the C# team decided they had a serious enough short coming in 1.1 to add this major feature. That is, many, many programmers were opting to ditch type-safety and decent performance in order to write more versatile data structures. How's that? Well, folks (MS included) were writing things like List and Queue and Stack to take "objects". Then their particular data structure could work on anything, and that would maximize code reuse (and likewise productivity).

The problem is, though re-use is greatly increased, productivity may be closer to a "wash" than a gain. That's because in implementing a "generic" structure that takes only "object" types, type-safety is lost. With lost type-safety, there's an increase in coding errors and run-time bugs and thus more time is spent testing and fixing the code. Okay, reusability is a win, productivity is a tie maybe, and performance is a what? Well performance is a loser. To use such a data structure with integers will require the compiler to "box" the type to make it an object. For reference types, the compiler spends extra time with cast operations. One article states the performance of the object data structure is about 200% slower than if it had been implemented as a specific value type.

Generic types solve the performance issue while giving the programmer back the type safety. And, clearly generic types provide for substantial code reuse.

Hurdle #2 - How to Use a Generic Type
Alright. It's good to know why, but it's better to know how. Let's start by using a generic type defined by the framework. Let's start with the List.


    // instantiate a List of Strings
    List strList = new List();
    strList.Add("One");
    strList.Add("Two");
    foreach(string s in strList)
        Console.WriteLine(s);

Rather than cut and paste this code into your IDE, go ahead and type it out. Intellisense will help you. In declaring your "strList" as a "List", everything about the list is now "string" oriented. The constructor, the Add(), the enumeration, etc. It's as if a wholesale find and replace has be done on the type place holder. Maybe you saw <T> in the intellisense, T is the place holder. More precisely, T is the "generic parameter" to the List<> type. And though, a macro like replacement is done, it's very different from the C++ Template where the compiler does the replacement. In C#, generics are implemented in the CLR, so its the JIT that is doing the final replacement. It will be good to keep this in mind when you "type-cast" your way around compiler errors only to have your code throw an exception at run-time on a bad conversion.

Hurdle #3 - How to Write a Generic Type
Which brings us to writing our own generic types. This next bit of example code makes use of several applications of the generic type syntax.


    public class GenericClass
    {
        private T genProp; // a generic field
        public T GenProp { // a generic property
            get { return genProp; }
            set { genProp = value; }
        }
        public GenericClass() // a generic constructor
        {
        }
        public T GenMethod() // a generic method
        {
            return genProp;
        }
        public string GenMethodT(T val) // you get the picture
        {
            return val.ToString();
        }
        // and so on...
    }
    // ...
        GenericClass gc = new GenericClass();
        Console.WriteLine(gc.GenMethodT(5));

You can see that the generic parameter T is all over the place and each occurance, if you can imagine, will be filled in with whatever type you end up instantiating this class with. It's behavior will be approriate for whatever "generic argument" you pass.

So, we've got the "generic type", the "generic parameter" and the "generic argument". What are each of these? In the example above, the "generic type" is "GenericClass<T>", the "generic parameter" is "T", and the "generic argument" is "int". There can be multiple generic parameters, and the name of the variable is arbitrary (within naming rule limits, of course). To specify multiple parameters, simply separate them with commas.


    Dictionary< int, string> dict = new Dictionary< int, string>();
    dict.Add(1,"One");
    dict.Add(2,"Two"); // and so on...

    //...
    public class MyNextGeneric
    {
        public T myMethod(U arg)
        {
           //...
        }
     }

Hurdle #4 - Using Generic Constraints
You are now well on your way to writing generic classes and methods. It won't take long before you start seeing some interesting compiler errors.


    public bool  GenMethodCompare(T val1, T val2)
    {
        return (val1 == val2) ? true : false;
    }

This bit of code produces the error "Operator '==' cannot be applied to operands of type 'T' and 'T'". You may ask why?! That might be because you are thinking like C++ instead of C#. In C++, maybe you have instantiated the class with "int" which is very easy to compare. The C++ compiler can make the final call. But with C#, the JIT will make the final call, and though the compiler knows what generic arguments you are passing, it does not know what might be passed by an external assembly. The C# team could probably have figured something out, but with deadlines looming, who knows. Regardless, it's situations like these for which "Generic Constraints" were invented.

There are three kinds of Generic Constraint, "Derivation Constraints", "Construction Contraints", and "Reference/Value Type Constraints".

In the last example, the compiler "could" compile the generic method if instead of "==", we used object.Compare() and if the compiler could just be assured that any generic argument supplied for T will be an IComparable type object. This would be a "Derivation Constraint". Notice in this next example that I use a generic IComparable interface as the constraint. Any class or interface can be used as a constraint, even other generic parameters.


    public MyClass where T : IComparable
    {
         public bool GenMethodCompare(T val1, T val2)
         {
             return (val1.Equals(val2)) ? true : false;
         }
    }

All the generic constraints use the syntax "where T : constraint, constraint" at the end of the type declaration (after any base class and/or interfaces) and before the open curly brace. If there are multiple generic parameters that each need constraints, the syntax would be


     public class GClass : SomeBaseClass, ISomeInterface
                    where T : IComparable
                    where U : IEnumerable
     { ... }

"Construction Constraints" are used when your generic class must constuct objects of type T. You may get the error "Cannot create an instance of the variable type 'T' because it does not have the new() constraint". This is very self explanatory message in terms of what you've got to do. But, it occurs because your code is trying to instantiate an object of type "T". Many types are implemented which do not have default (i.e., parameter-less) constructors. Without the "new()" constraint, the compiler will not allow code that instantiates a generic parameter to compile. You can combine new() with other constraints by separating with a comma and placing it after the others.


    public class GClass where T : IComparable, new()
    {
        public T factory() { return new T(); }
    }

"Reference/Value Type Constraints" tell the compiler that only reference types or only value types are allowed as the generic argument. "class" indicates a reference type constraint, while "struct" indicates a value type constraint. Notice that it doesn't make any sense to combine the "struct" generic constraint with the "Derivation Constaints", they are incompatible. It doesn't make sense to combine the "class" generic constraint either, it is redundant.


    public class RTypeClass where T : class { ... }
    public class VTypeClass where T : struct { ... }

Finish Line
That's about all I can cover in an evening. You should have everything you need to start using and creating your own generic types. My article "Thread Synchronized Queing" is reasonable working example of a generic Queue class if you think that would be of any value to you. Just don't try writing anything too complex right away, I wouldn't want you to hurt yourself.

Useful Resources
An Introduction to C# Generics
Generics (C#)
Using C# Generics on Static Classes - Wes' Puzzling Blog
C# Frequently Asked Quetions - How do C# generics compare to C++ templates?

Sleep

2008-08-12T08:33:00.008-04:00

When I first started programming, I thought "Sleep" to be one of the most useless calls I could make. No, I'm not talking about personal habits, I'm talking about the Sleep() function.


using System.Threading;
Thread.Sleep(int milliSeconds)

We find this little "do nothing" function in C, C++, C#, VB, VB.Net, WScipt, and similar functions in other languages. And, I have since come to understand how useful and confusing "doing nothing" can be. So, let's take a look at it; I'll be focusing on the C# Sleep() function.

Sleep() simply suspends the execution of your thread for a specific period of time. It relinquishes the remainder of its time-slice and becomes "unrunnable" for a period of time. If the time period is zero, it relinquish its time-slice only if another thread is ready and waiting to run. If there are no waiting threads, and the time span is zero, the calling thread remains ready to run and therefore returns from Sleep() immediately.

So, Sleep() is useful for delaying a thread, or for getting a thread out of the way of other threads. Let's say a user enters a password incorrectly, a delay at this point is useful to help dissuade a rapid fire brute force attack on your password input algorithm. Keep in mind that Sleep() also takes away the thread's responsiveness. So, delaying operations should only be implemented with Sleep() if you can tolerate or mitigate the total lack of responsiveness from the sleeping thread.

Sleep() is also useful for making a thread behave nicely by not hogging the CPU. The Operating System will preempt threads for you and let other threads run. But you may still see your thread taking up way too much processor resources and not allowing other threads enough time to run. A Sleep() for some very short interval could be useful in improving overall application responsiveness. Here again, calling Sleep(0) on a thread that is very busy processing messages is about useless because each call to the message queue already allows other waiting threads to run.

You can use Sleep() for very simple timed delays. A pattern where there is a producer and several consumers might use a Sleep() to throttle the producer to let the consumers keep up or catch up. That is, the producer might fill a queue and then stop and Sleep() when the queue gets too full. The amount of time the producer thread sleeps determine how much the consumers have a chance to process. But even so, a more robust implementation would have the producer "wait" for space available on the queue or even the queue reaching a "low water mark", rather than blindly sleeping.

The single most often misuse of Sleep() in C# is when a thread that has created a window then calls Sleep(). Threads that create windows, whether directly or indirectly, must remain awake to process messages. When a thread sleeps, it cannot process those messages and the application appears to hang. The C/C++ documentation for SleepEx() and Sleep() in fact warn that should a thread sleep indefinitely, when it should be processing messages, then the system will deadlock because message broadcasts are sent to all windows in the system.

We are not discussing C/C++ and we certainly are not going to sleep indefinitely. So, C# should keep us out of such a catastrophic situation, but the C# programmer must still avoid putting "GUI" threads to sleep to avoid hanging up his application. The C# documentation for Sleep() simply says "This method does not perform standard COM and SendMessage pumping". This hardly suggests how ugly the application behavior can be if you use Sleep() on your GUI thread. In short, just keep your Sleep() off of the GUI threads, and keep your COM and DDE calls (unless absolutely sure there is no GUI involved) on the GUI threads.

What are the alternatives to calling Sleep() on a GUI thread? First, and foremost, your best alternative is a good architecture starting out. GUI threads never "need" to sleep because they should normally be waiting for messages. If a GUI thread is so busy that it can't service the messages as they arrive, the GUI should offload the work to a separate thread. But, if your GUI is so busy, it's probably not going to sleep anyway. If a GUI needs to wait for something to happen that won't occur on its message/event loop, it should delegate that wait to a thread as well, and then use state variables to modify its behavior until the desired event occurs.

For example, reading from a network could wait for a very long time. A GUI would act like it's hung if it got stuck at a network read for any length of time. The same goes for reading streams. Often a stream may resolve to a network resource. Or, if you are reading a lot of data from a file, the application will similarly appear hung. By delegating the operation to a separate thread, the GUI is free to respond to other events. Its tempting at this point for the GUI to Sleep(), but don't do it. Instead, set a state variable that indicates an operation is in progress and prevent your other dependant GUI operations from proceeding until the operation finishes. Better yet, try to eliminate the GUI dependancy on the completion of the read altogether. Architect your application with threads well chosen for the task, especially if you must wait for various kinds of system events.

Another alternative to Sleep() is a "Timer" function (see my articles on timers). With a timer function, a thread simply schedules a time when it will get back to the item its waiting for. In the meantime, the GUI continues to receive messages and events. In fact, the timer expiration is just an event which you write a handler for.

If your thread must wait for something, your best performance will be achieved through the proper use of "Wait" type calls. WaitOne() and WaitAny() on an appropriate "WaitHandle" is always better performance wise than Sleep() because your thread waits only as long as it takes for the event to fire and the context switch to occur. Furthermore, there is no risk of the thread returning earlier than the event (unless you provide a timeout to "Wait").

How accurate is Sleep()? The Sleep time interval is specified to the millisecond. But Sleep() time expiration is serviced by the Operating System based upon its "tick" size. CPU ticks occur at a constant rate larger than a 1 millisecond granularity. So, threads will not wake at exactly the time span specified due to CPU tick granularity and scheduling priorities. They wake at their first scheduled opportunity after becoming "ready to run". They become ready to run at a time determined by how divisible the sleep time interval is by the CPU tick value. Sleeping threads become runnable on CPU Tick intervals. You can control the "tick" size (and thus the accuracy of Sleep) in C/C++ with timeGetDevCaps, timeBeginPeriod and timeEndPeriod.

By the way, while we are on the topic of C/C++, the call to SleepEx() is worth mentioning. SleepEx() is an alertable sleep function that awakes prior the elaspe of the time period when the same thread's I/O completion callback is called or when an APC function is called. The lines between "Sleep" and "Wait" begin to blur with this function.

In summary, use Sleep() as the "poor man's" Wait() function. Setting up a WaitHandle, synchronizing correctly, and handling timeouts with a "Wait" can be complex. Certainly, the "Wait" type calls would provide much lower latency, but if a few seconds of latency does not adversely affect your application, you don't need that added complexity. Sleep() is simple, use it simply. Never sleep on the GUI, or on the job, the consequences are not pretty.

That about wraps it up. There is no code today. Enjoy.

Threading with .NET ThreadPool Part 3

2008-08-08T20:25:00.007-04:00

To wrap up this series of articles, I am going to make the application we've been working on do something a little more interesting. If you will recall in my first article on the subject of ThreadPool, I go over several ways to call the primary thread pool function QueueUserWorkItem and demonstrate how simple it is to multi-thread an application. In my second article, I look at a skeletal application that doesn't really do anything; it just displays messages on the console. However, and more importantly, it demonstrates some of the aspects of synchronization. You must synchronize if you are to do anything of real consequence with threads.

So, what sort of interesting things will this application do? My application will paint a pop-art picture. Actually, it will place random pixels on a PictureBox. Okay, its not that useful (or argueably that interesting), but it will demonstrate a means by which threads report progress to the GUI thread.

Let's look at the work item classes again and consider my updates. First, there are the changes to "WorkItem". I've added a public event handler called "Done" that will be fired by both of the derived classes. I've also added two public static properties called "Width" and "Height" which simply serve as a convenient place to store some global parameters for the work items to use. As a bonus, I've added some code to paint a background message that should materialize as the dots are painted.


    public abstract class WorkItem
    {
        public abstract void ThreadPoolCallback(object context);
        public event EventHandler Done;
        public static int Width { get; set; }
        public static int Height { get; set; }
        
        private static bool _interrupted = false;
        private static long _numWorkItems = 0;
        protected static Bitmap _secretMessage;
        protected static void InitMessage()
        {
            _secretMessage = new Bitmap(Width, Height);
            Graphics g = Graphics.FromImage(_secretMessage);
            g.FillRectangle(Brushes.White, 0, 0, Width, Height);
            g.DrawString("Far Out!", new Font("Courier New",(float)18.0), Brushes.Black, (Width / 4), (Height / 2) - 15);
        }

        protected WorkItem()
        {
            Interlocked.Increment(ref WorkItem._numWorkItems);
        }
        protected void WorkItemDone()
        {
            Interlocked.Decrement(ref WorkItem._numWorkItems);
        }
        public static bool Interrupted
        {
            get { return _interrupted; }
            set { _interrupted = value; }
        }

        public long NumWorkItems
        {
            get { return _numWorkItems; }
        }

        protected void OnDone()
        {
            if (Done != null)
                Done(this, new EventArgs());
        }
    }

Next, the MasterWorkItem is modified to provide its "context" variable to each new ServiceWorkItems in the call to QueueUserWorkItem(). The "context" is declared as an object and is designed to let the programmer pass any reference he likes. We will be passing a reference to the GUI form. Furthermore, MasterWorkItem will call OnDone when completing normally to signal the Done event. The number of active work items has been increased to 1000. This increases the speed at which the painting is displayed. But, if you increase it too much, your CPU(s) will saturate and your GUI may not display the image (for me this was between 10-50 thousand work items. The painting will continually update until the GUI interrupts the master work item.


    public class MasterWorkItem : WorkItem
    {
        public override void ThreadPoolCallback(object context)
        {
            try
            {
                WorkItem.InitMessage();
                while (!Interrupted)
                {
                    if (NumWorkItems < 1000)
                    {
                        ThreadPool.QueueUserWorkItem(new ServiceWorkItem().ThreadPoolCallback, context);
                    }
                    else
                    {
                        Console.WriteLine("Letting pool drain some");
                        Thread.Sleep(500);
                    }
                }
                while (NumWorkItems > 1)
                {
                    Console.WriteLine("Waiting for child tasks {0}", NumWorkItems);
                    Thread.Sleep(1000);
                }
                Console.WriteLine("All tasks have completed.");
                OnDone();
            }
            finally
            {
                WorkItemDone();
            }
        }
    }

The ServiceWorkItem gets a new static method called "MyRand()" and a private static Random object. All the service threads use the same random number generator to determine the location and color of their pixel. Each service thread determines its pixel location and color and "tells" the GUI to paint it. Locks are used around structures that are not safe. Each work item calls OnDone when complete (but we don't implement a handler for it, so it goes ignored). Since threads cannot interact directly with the GUI, the service work item calls a GUI function that tests whether Invoke is required. BeginInvoke() is therefore our primary means of communication back to the GUI of our progress.


    public class ServiceWorkItem : WorkItem
    {
        public override void ThreadPoolCallback(object context)
        {
            try
            {
                if (!Interrupted)
                {
                    int X = ServiceWorkItem.MyRand(WorkItem.Width);
                    int Y = ServiceWorkItem.MyRand(WorkItem.Height);
                    int R = ServiceWorkItem.MyRand(255);
                    int G = ServiceWorkItem.MyRand(255);
                    int B = ServiceWorkItem.MyRand(255);
                    if (X > ((WorkItem.Width / 2) / 2)
                        && X < ((WorkItem.Width / 2) / 2 + (WorkItem.Width / 2))
                        && Y > ((WorkItem.Height / 2) / 2)
                        && Y < ((WorkItem.Height / 2) / 2 + (WorkItem.Height / 2)))
                    {
                        R += 50; R = (R > 255) ? 255 : R;
                        //B += 50; B %= 256;
                    }
                    lock (WorkItem._secretMessage)
                    {
                        if (!WorkItem._secretMessage.GetPixel(X, Y).Name.Equals("ffffffff"))
                        {
                            R = 255;
                            B = 255;
                            G = 255;
                        }
                    }
                    Color clr = Color.FromArgb(R, G, B);

                    int myThread = Thread.CurrentThread.GetHashCode();
                    ((Form1)context).SetPixelCallback(X,Y,clr);
                }
                else
                {
                    Console.WriteLine("Thread {0}, WorkItem {1}, Interrupted",
                        Thread.CurrentThread.GetHashCode().ToString(),
                        this.GetHashCode().ToString());
                }
                OnDone();
            }
            finally
            {
                WorkItemDone();
            }
        }
        private static Random rnd = new Random();
        public static int MyRand(int limit)
        {
            return rnd.Next(0, limit);
        }
    }

Finally, the main form (Form1) is changed to provide a Bitmap, a function to update the bitmap, and a PictureBox to display it. The Form1_Load handler initializes the bitmap, paints it white and hooks it up to the pictureBox1. The Load handler also initializes the work item globals, creates a new master work item, wires up the work item's "Done" handler, and starts the master thread.


    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        private Bitmap bitmap;
        private Random rnd;

        void m_Done(object sender, EventArgs e)
        {
            if (this.InvokeRequired)
            {
                this.BeginInvoke(new MethodInvoker(() => this.label1.Text = "Done"));
            }
            else
            {
                this.label1.Text = "Done";
            }
        }
        private void button1_Click(object sender, EventArgs e)
        {
            WorkItem.Interrupted = true;
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            bitmap = new Bitmap(pictureBox1.Width, pictureBox1.Height);
            Graphics g = Graphics.FromImage(bitmap);
            g.FillRectangle(Brushes.White, 0, 0, bitmap.Width, bitmap.Height);
            pictureBox1.Image = bitmap;

            rnd = new Random();

            WorkItem.Width = bitmap.Width;
            WorkItem.Height = bitmap.Height;
            MasterWorkItem m = new MasterWorkItem();
            m.Done += new EventHandler(m_Done);
            ThreadPool.QueueUserWorkItem(m.ThreadPoolCallback,this);
        }

        internal delegate void SetPixelDelegate(int x, int y, Color clr);
        internal void SetPixelCallback(int X, int Y, Color clr)
        {
            if (this.InvokeRequired)
            {
                this.BeginInvoke(new SetPixelDelegate(SetPixelCallback), new object[] { X, Y, clr } );
            }
            else
            {
                bitmap.SetPixel(X, Y, clr);
            }
        }
        private void timer1_Tick(object sender, EventArgs e)
        {
            pictureBox1.Image = bitmap;
        }
    }

And, here is the code for Form1's code-behind (Form.Designer.cs)...


    partial class Form1
    {
        private System.ComponentModel.IContainer components = null;

        protected override void Dispose(bool disposing)
        {
            if (disposing && (components != null))
            {
                components.Dispose();
            }
            base.Dispose(disposing);
        }

        private void InitializeComponent()
        {
            this.components = new System.ComponentModel.Container();
            this.button1 = new System.Windows.Forms.Button();
            this.label1 = new System.Windows.Forms.Label();
            this.pictureBox1 = new System.Windows.Forms.PictureBox();
            this.timer1 = new System.Windows.Forms.Timer(this.components);
            ((System.ComponentModel.ISupportInitialize)(this.pictureBox1)).BeginInit();
            this.SuspendLayout();
            // 
            // button1
            // 
            this.button1.Location = new System.Drawing.Point(29, 217);
            this.button1.Name = "button1";
            this.button1.Size = new System.Drawing.Size(75, 26);
            this.button1.TabIndex = 0;
            this.button1.Text = "button1";
            this.button1.UseVisualStyleBackColor = true;
            this.button1.Click += new System.EventHandler(this.button1_Click);
            // 
            // label1
            // 
            this.label1.AutoSize = true;
            this.label1.Location = new System.Drawing.Point(124, 224);
            this.label1.Name = "label1";
            this.label1.Size = new System.Drawing.Size(35, 13);
            this.label1.TabIndex = 1;
            this.label1.Text = "label1";
            // 
            // pictureBox1
            // 
            this.pictureBox1.Location = new System.Drawing.Point(25, 28);
            this.pictureBox1.Name = "pictureBox1";
            this.pictureBox1.Size = new System.Drawing.Size(239, 162);
            this.pictureBox1.TabIndex = 2;
            this.pictureBox1.TabStop = false;
            // 
            // timer1
            // 
            this.timer1.Enabled = true;
            this.timer1.Tick += new System.EventHandler(this.timer1_Tick);
            // 
            // Form1
            // 
            this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
            this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
            this.ClientSize = new System.Drawing.Size(292, 266);
            this.Controls.Add(this.pictureBox1);
            this.Controls.Add(this.label1);
            this.Controls.Add(this.button1);
            this.Name = "Form1";
            this.Text = "Form1";
            this.Load += new System.EventHandler(this.Form1_Load);
            ((System.ComponentModel.ISupportInitialize)(this.pictureBox1)).EndInit();
            this.ResumeLayout(false);
            this.PerformLayout();

        }


        private System.Windows.Forms.Button button1;
        private System.Windows.Forms.Label label1;
        private System.Windows.Forms.PictureBox pictureBox1;
        private System.Windows.Forms.Timer timer1;
    }

I call my namespace "ThreadPoolApp" and here are my "using" statements as well...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace ThreadPoolApp
{
    //...
}

To review, there are several aspects of a multi-threaded that in my opinion should be present to be a robust and stable app. These are...

1. Enqueuing work.
2. Detecting queue overload and throttling.
3. Detecting work start.
4. Reporting work progress.
5. Detecting work completion.
6. Stopping work-in-progress.
7. Cancelling un-started work.

But, I've done nothing in my code to detect the starting of work (#3). I leave it to the reader as an exercise, but for something this simple, I would probably use an Invoke to change a visible label since the startup is obvious by observing the updating of the picture. Of course, others may have their own list, and depending upon the application, I may not implement all of these suggestions myself.

To wrap up, let's consider what kind of applications benefit from such an architecture. An application benefits where one or a few threads need to listen for incoming network requests and then pass off a (short lived) work item for processing while it goes back to listening. If you are considering writing a loop for a thread to wait for work and effectively do the same thing or kind of things over and over, your application could probably benefit from this approach. There are several architectural choices when it comes to multi-threading, and that choice is yours. I hope these articles can help your deliberations.

Threading with .NET ThreadPool Part 2

2008-08-07T07:05:00.011-04:00

Let's continue looking at System.Threading.ThreadPool. In the last article, we went over some of the basic features a thread pool should have. We looked specifically at System.Threading.ThreadPool to see how to get your App threading quickly. And, we touched on some aspects missing from Microsoft's thread pool implementation that are needed for an "industrial strength" application. So today, we will examine some of those missing features.

Let's start with a skeleton application. This application starts a master work item that is responsible for feeding service work items to the thread pool. The service work items do some work (right now, they display a message with Console.WriteLine(...) and then sleep a random amount of time in the sub-second range). We get a taste of synchronization with the shared random number generator where I surround it with a lock(...){ } statement to make it thread safe. When service work items are done, they exit. "Exit" is a term you should be careful with. Often, it means an explicit call to some function to exit a program. Here we mean, the thread returns from the callback function. So, it's our task that's exiting, not the thread.

This skeletal application also has a button to interrupt the master work item's thread, which in turn interrupts the service work item threads and waits for their completion which should occur within 1 second or so. To facilitate a clean shutdown, I employ a little more thread synchronization using a simple task counter.

Here's the code. To use it, you can create a new Windows Forms project named ThreadPoolApp, drop a button on it, then view the source code for Form1.cs. Copy the code below and replace the code in Form1.cs. Last, set your button's event handler to use the one in my code.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace ThreadPoolApp
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            ThreadPool.QueueUserWorkItem(new MasterWorkItem().ThreadPoolCallback);
        }
        private void button1_Click(object sender, EventArgs e)
        {
            WorkItem.Interrupted = true;
        }
    }
    public abstract class WorkItem
    {
        public abstract void ThreadPoolCallback(object context);
        
        private static bool _interrupted = false;
        public static bool Interrupted {
            get { return _interrupted; }
            set { _interrupted = value; }
        }
        private static long _numWorkItems = 0;
        protected WorkItem() 
        {
            Interlocked.Increment(ref WorkItem._numWorkItems);
        }
        protected void WorkItemDone()
        {
            Interlocked.Decrement(ref WorkItem._numWorkItems);
        }
        protected long NumWorkItems
        {
            get { return _numWorkItems; }
        }
    }
    public class MasterWorkItem : WorkItem
    {
        public override void ThreadPoolCallback(object context)
        {
            try
            {
                while (!Interrupted)
                {
                    if (NumWorkItems < 500)
                    {
                        ThreadPool.QueueUserWorkItem(new ServiceWorkItem().ThreadPoolCallback);
                    }
                    else
                    {
                        Console.WriteLine("Letting pool drain some");
                        Thread.Sleep(500);
                    }
                }
                while (NumWorkItems > 1)
                {
                    Console.WriteLine("Waiting for child tasks {0}", NumWorkItems);
                    Thread.Sleep(1000);
                }
            }
            finally
            {
                WorkItemDone();
            }
        }
    }
    public class ServiceWorkItem : WorkItem
    {
        public override void ThreadPoolCallback(object context)
        {
            try
            {
                if (!Interrupted)
                {
                    int val = ServiceWorkItem.MyRand();
                    int myThread = Thread.CurrentThread.GetHashCode();
                    Console.WriteLine("Thread {0}, WorkItem {1}, Sleeping {2}",
                        myThread.ToString(),
                        this.GetHashCode().ToString(),
                        val.ToString());
                    Thread.Sleep(val);
                    Console.WriteLine("Thread {0}, WorkItem {1}, done.", myThread.ToString(), this.GetHashCode().ToString());
                }
                else
                {
                    Console.WriteLine("Thread {0}, WorkItem {1}, Interrupted",
                        Thread.CurrentThread.GetHashCode().ToString(),
                        this.GetHashCode().ToString());
                }
            }
            finally
            {
                WorkItemDone();
            }
        }
        private static Random rnd = new Random();
        public static int MyRand()
        {
            lock (rnd)
            {
                return rnd.Next(100, 1000);
            }
        }
    }
}

I've created three classes, an abstract WorkItem class, the MasterWorkItem class and the ServiceWorkItem class. The MasterWorkItem and the ServiceWorkItem classes derive from the WorkItem class which enforces that the derived classes implement the CallBack according to the function signature required by the the static ThreadPool method QueueUserWorkItem(). The WorkItem class implements a control member whereby I can stop the Master and Service work items. Each work item has its own instance data that the callback can refer to. Each can also refer to some global information in the WorkItem static class.

Now, if you run the code, you will start seeing text information in the debugger output window. The master is limited to having only 500 tasks active. It goes to sleep and lets the queue "drain" a little bit when it hits this limit.

When you click the button after a few seconds, you will see output from the tasks being terminated. The threads are not necessarily terminating, they are just pulling tasks from the queue and the callbacks are returning immediately after reporting that they have been interrupted. When the task count drops to 1, the master work item (task) can return from it's callback.

Maybe you see "thread xxx has exited" messages. I have said that with the thread pool, tasks exit and not threads. So, why is it then that we see messages from the debugger that such-and-such thread has exited? That is because the thread pool is managing it's resources. If it doesn't need as many active threads, some of the threads exit. But that is not a concern of our application, we are only concerned about our task exiting. Whether the thread pool starts or stops threads to service our app is completely under the hood.

As I mentioned in my last article, getting multiple threads working in your application is almost trivial with the ThreadPool, but you must deal with several aspects of thread synchronization to have a useful app. Again, these aspects are...

1. Enqueuing work.
2. Detecting queue overload and throttling.
3. Detecting work start.
4. Reporting work progress.
5. Detecting work completion.
6. Stopping work-in-progress.
7. Cancelling un-started work.

This sample application demonstrates several of these aspects. Of course, it enqueues work, the Master work item self-throttles with a simple count and sleep mechanism. Work completion is detected at the high level by the Master task with a task counter, but it is a weak method in that we cannot really tell which work item completes. Cancelling un-started work and stopping work-in-progress is accomplished with a simple state variable shared by all tasks called "Interrupted". This application does not really report progress or indicate when a task starts (I don't consider debug messages to the console to be accomplishing that). I will try to flesh out the progress communication and startup notification in my next article.

Before I leave, I have just a quick word about thread synchronization. You will notice that I am not using anything fancy like AutoResetEvent, or Mutex, or Monitor, or the Wait{...} type calls. I use Interlock.Increment() and Interlock.Decrement() to manage the changing of the task count, and I use lock(){...} to protect the Random number generator. Even there, the lock() may not be necessary, but the documentation does not say the Random.Next() call is thread safe, so I add it. I can get away with my Interrupted flag being a simple variable because I know there is only one thread that will ever write to it. Threads that read from Interrupted are not harmed by rogue timings or simultaneous access. Similarly, reading the task count is not protected because timing of reads and writes do not significantly affect the result.

In my next article, I will provide more on fancier synchronization, and demonstrate some progress reporting and startup detection.

Threading with .NET ThreadPool

2008-08-05T06:24:00.010-04:00

The System.Threading.ThreadPool is a great little feature for programmers wishing to add instant threading to their applications. You just enqueue callback functions, what could be more simple than that? I agree, so we'll look now at some simple coding experiments with the ThreadPool. By the way, here's the MSDN version of "How to: Use a Thread Pool" which makes for some good pre-requisit reading. And though I risk repeating much of what may be already elsewhere on the web, I hope that I can add something of use in your estimation.

So first, let's get a basic understanding of thread pooling. Maybe you've already read my article, Thread Synchronized Queing. If so, you have a good start. A thread pool, like my SynchQueue class, will involve some thread(s) writing to a queue, and some set number of threads reading from the queue. Similarly, if there isn't enough work to do, the reader threads wait or block on the queue. And, if there is too much work to do, the queue grows indefinitely or until you throttle your writer threads somehow.

Let's start with a simple Windows Forms application. Later, I'll come back and discuss some of the caveats. I create a new application called ThreadPoolExample. To Form1 I add a couple of methods that I will use as callback functions. One is an instance method and the other a static class method. I also add a new class to Form1.cs called TestClass. TestClass also has a callback function defined.


        private void CallBack(object Context)
        {
            Console.WriteLine("Thread called");
        }
        private static void SCallBack(object Context)
        {
            Console.WriteLine("Thread static callback");
        }
...
    public class TestClass
    {
        public void CallBack(object context)
        {
            Console.WriteLine("TestClass #{0}",_mynum);
        }
        int _mynum;
        public TestClass(int n) { _mynum = n; }
    }

Now, I add a button to the form and add the following click handler for the button.


        private void button1_Click(object sender, EventArgs e)
        {
            ThreadPool.QueueUserWorkItem(this.CallBack);
            ThreadPool.QueueUserWorkItem(new WaitCallback(this.CallBack));
            ThreadPool.QueueUserWorkItem(SCallBack);
            ThreadPool.QueueUserWorkItem((o) => { Console.WriteLine("lamda callback"); });
            ThreadPool.QueueUserWorkItem(delegate(object o) { Console.WriteLine("delegate callback"); });

            int max, dummy;
            ThreadPool.GetMaxThreads(out max, out dummy);
            for (int i = 0; i < max + 20; ++i)
                ThreadPool.QueueUserWorkItem(new TestClass(i).CallBack);
        }

With this click handler, I am demonstrating several aspects of starting work with a ThreadPool. In all cases, I start a thread working with the static ThreadPool method QueueUserWorkItem(). This method gives the task to the first available thread, so most of the calls start (or can start) immediately. I say, "start a thread working" rather than "start a thread" because I am doing nothing to start threads. All the threads are started by the underlying framework. It is the framework that controls the maximum number of threads to start and the minimum number of threads to have running in standby. I simply enqueue work to perform and let the framework manage the threads.

The first two calls show the ThreadPool being given an instance method. One call just provides the name of the method, while the other call uses the familiar "new Delegate(function)" construction. I added both of these because I wanted to see if there are any real differences in the two calling methods. I can't see any except I presume the second may not perform as well as the first due to the additional memory allocation. By using the "new" operator, you get the help of Intellisense in figuring out how to write the callback signature. Other than that, I think it's purely up to programmer style. The third line provides a static class method as the callback.

The fourth line demonstrates passing a Lamda Expression to the ThreadPool. Lamda Expressions are new to Visual Studio 2008. Comment it out if you are using 2005 or earlier. The fifth line demonstrates passing an anonymous function to the ThreadPool. Lastly, I use the ThreadPool static method GetMaxThreads() to figure out how many threads the pool will use, and then I exceed that number by 20. The last set of calls also demonstrates passing instance methods of different object instances. This is handy because the instance method has direct access to the object's encapsulated data. In other words, you can implement a class to represent small units of work and hand these off to the ThreadPool by their callback method thus maintaining separate states for each.

You might also note, the ThreadPool.QueueUserWorkItem() is given methods to different classes. The ThreadPool does not care about the type of the underlying data, only the signature of the callback you pass to it. This makes the tool quite versatile. But, similar behavior is gotten from the SynchQueue generic class by declaring it to use a delegate type instead of a data type.

Now, run the code and click the button. You will see in Visual Studio's output window the various strings the threads are supposed to write. Everything works fine. Take a closer look at the output and you will see that things mostly run in the order that they were enqueued. But some of the tasks appear out of order. In fact, the callbacks are assigned a thread in the order that they were enqueued, but there is no guarantee that the threads will finish in the same order as the work was queued. This is fairly intuitive, tasks can take varying amounts of time to complete.

But also consider, though the work is dequeued by threads in FIFO order, it is entirely possible that some tasks will actually start out of order. Picture there being multiple tasks on the queue ready to go, and picture one thread grabbing a task and then immediately being preempted. Another thread grabs the next task and begins work before the first thread comes back.

Thus, we have our first caveat. That is, don't expect the ThreadPool to start your tasks in a fixed order. If you need them started in a precise, fixed order, you will need to synchronize your tasks in some other way.

Like I said earlier, getting multiple threads working in your application is almost trivial with the ThreadPool. But alas, multi-threading is never trivial. There are several aspect of thread pooling that should be addressed if you wish to have an industrial strength application. These are...

1. Enqueuing work.
2. Detecting queue overload and throttling.
3. Detecting work start.
4. Reporting work progress.
5. Detecting work completion.
6. Stopping work-in-progress.
7. Cancelling un-started work.

We've only looked at enqueuing work. But, without the ability to know for sure that the work has been fully completed by the thread, it makes any amount of interdepenancy risky and a clean shutdown nearly impossible. You may have need to end your application or server while tasks are still running. To do this cleanly, you need a way to stop adding tasks to the queue, to cancel any un-started tasks on the ThreadPool queue, and to stop tasks currently being processed by a ThreadPool thread.

I'll look at each of these in my next article. Until then, here is the full source for Form1.cs (you'll need to add your own button and hook up the button click handler yourself).


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace ThreadPoolExample
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        private void CallBack(object Context)
        {
            Console.WriteLine("Thread called");
        }
        private static void SCallBack(object Context)
        {
            Console.WriteLine("Thread static callback");
        }
        private void button1_Click(object sender, EventArgs e)
        {
            ThreadPool.QueueUserWorkItem(this.CallBack);
            ThreadPool.QueueUserWorkItem(new WaitCallback(this.CallBack));
            ThreadPool.QueueUserWorkItem(SCallBack);
            ThreadPool.QueueUserWorkItem((o) => { Console.WriteLine("lamda callback"); });
            ThreadPool.QueueUserWorkItem(delegate(object o) { Console.WriteLine("delegate callback"); });

            int max, dummy;
            ThreadPool.GetMaxThreads(out max, out dummy);
            for (int i = 0; i < max + 20; ++i)
                ThreadPool.QueueUserWorkItem(new TestClass(i).CallBack);
        }
    }

    public class TestClass
    {
        public void CallBack(object context)
        {
            Console.WriteLine("TestClass #{0}",_mynum);
        }
        int _mynum;
        public TestClass(int n) { _mynum = n; }
    }
}

Learning LINQ to SQL (Part 2)

2008-07-31T21:09:00.007-04:00

In my last post, I had worked through a few section of the white paper "LINQ to SQL: .NET Language-Integrated Query for Relational Data" and had gotten a working LINQ program going.

Everything was going just fine... until now. At the end of the Defining Relationships section, the article leaves me kind'a hanging. It says "there is a tool (described later) that can be used to generate all the necessary definitions..." So, I've got to go back to the TOC to see what this "special tool" is for generating the Entities.

I jump to "The Entity Class Generator Tool" section and start reading. My first order of business is to find the program called SqlMetal. I launch a COMMAND window and type "sqlmetal" and receive the expected "'sqlmetal' is not recognized as an internal or external command." error. Hey, it was worth a try! So, I next use the old stand-by "cd \" then "dir /s sqlmetal.*". Yup, there it is under the Microsoft SDKs (C:\Program Files\Microsoft SDKs\WIndows\v6.0A\bin).

So, I add the path of the tool to my User PATH variable. I change directories to my project directory and run the command...

SqlMetal /server:.\SQLExpress /database:Northwind /pluralize /namespace:nwind /code:Northwind.cs

SqlMetal reports an error...

Error : Cannot open database "Northwind" requested by the login. The login failed. Login failed for user XALNIX-DEVBOX\Les

To solve this problem, I launch SQL Server Management Studio Express and see why the database won't log me in given my program is working and accessing the database. Once in Management Studio, I note that there is no Northwind database, of course. My program accesses the database by filename. So I attach the Northwind database file and change the database name from the filename to Northwind. Now the SqlMetal runs and produces a file called Northwind.cs.

By the way, if you need the SQL Server Management Studio Express, just follow the link to Microsoft's download page.

Now I can get back to the Learning some LINQ. Before I made the excursion into SqlMetal domain, I was about to enter the section, Querying Across Relationships. Getting back on track, I load the new Northwind.cs into my project. Next, I lift the code snippet from the article and plug it into a new Button2 click handler. Since the snippet from the article does not contain the DataContext, I add a line to create one. And since, the snippet does not have an iteration to look at the results, I use the earlier iterator (with a few tweaks). Now my code looks something like this...


        private void button2_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            var q =
               from c in db.Customers
               from o in c.Orders
               where c.City == "London"
               select new { c, o };
            foreach (var cust in q)
                Console.WriteLine("Customer id = {0}, City = {1}, OrderID {2}", cust.c.CustomerID, cust.c.City, cust.o.OrderID);
        }
    ...

    [Table(Name = "Customers")]
    public class Customer
    {
        [Column(IsPrimaryKey = true)]
        public string CustomerID;
        [Column]
        public string City;

        private EntitySet _Orders;
        [Association(Storage = "_Orders", OtherKey = "CustomerID")]
        public EntitySet Orders
        {
            get { return this._Orders; }
            set { this._Orders.Assign(value); }
        }
    }
    [Table(Name = "Orders")]
    public class Order
    {
        [Column(IsPrimaryKey = true)]
        public int OrderID;
        [Column]
        public string CustomerID;
        private EntityRef _Customer;
        [Association(Storage = "_Customer", ThisKey = "CustomerID")]
        public Customer Customer
        {
            get { return this._Customer.Entity; }
            set { this._Customer.Entity = value; }
        }
    }
    public partial class Northwind : DataContext
    {
        public Table Customers;
        public Table Orders;
        public Northwind(string connection)
            : base(connection)
        {
        }
    }

The code would not run though, because SQL Server could not log me into the database. I had attached the Northwnd.mdf in order to get SqlMetal to run. If I were to take the time to figure out the new connection string, I could just change it in my code. But, I cheat and just detach the database. But WATCH OUT! If you create a SQL login for yourself and set the default database to Northwind, and then detach Northwind, you have locked yourself out of the database (if I sound like I have experience with this, [blush] I do).

With the code back to running, I experiment a little. The new query above lets me now get at the members of the Orders with the member "cust.o". The "cust" variable is an anonymous object of type { Customer c, Order o }. So, "intellisense" works great showing the members of the Order class as I type. But, I've just spent a lot of time figuring out SqlMetal, I should be using the Northwind.cs definition it created.

To switch to the auto-generated Entities, I open the Northwind.cs file and change the namespace from "nwind" to LingPOC (to be compatible with my project). I use the handy refactor tool to change all occurances of "nwind". I really don't expect there to be any other occurances, but the tool is very useful in that it reports errors about my existing definitions for Customer and Order not being marked partial. That's okay because I just comment out my three classes (Northwind, Customer, Order). After all that work, gone in a moment.

I run again, everything is still working, and the intellisense now has more information to report about Order and Customer.

The final section of the Quick Tour gets into updating the database. In Modifying and Saving Entities, I note a few points. First, there is a nice little call on the Customers object called "Single()" which gets a single Customer object. The previous examples get a list of objects. So, even if the select statement narrows the results to one row, I would have to use a "foreach" statement to get at the row. The "Single()" statement reduces the code and complexity. Also, I note that creating a new Order appears very database independent, nice. Last, I note that the update to the database is performed by the "SubmitChanges()" member of the DataContext. Hmm, if I do a query for a customer and a query for an order on the same context, it appears separate changes to each cannot be separately updated to the database. I'll have to check this out.

Yes, it seems so. A single DataContext will update multiple separate selections. But, if I use separate DataContexts, I can independantly control the updates. There is a note that this behavior can be overridden. Anyway, that's enough for now. In a future post, I will embark upon the next section of the article, Queries in Depth. Here's my updated code. You'll need to generate your own Northwind.cs. And if you use this code, be sure to add the buttons to your form and link up the button click handlers. Enjoy...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Data.Linq;
using System.Data.Linq.Mapping;

namespace LinqPOC
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            // Query for customers from London
            var q =
               from c in db.Customers
               where c.City == "London"
               select c;
            foreach (var cust in q)
                Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
        }

        private void button2_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            var q =
               from c in db.Customers
               from o in c.Orders
               where c.City == "London"
               select new { c, o };
            foreach (var cust in q)
                Console.WriteLine("Customer id = {0}, City = {1}, OrderID {2}", cust.c.CustomerID, cust.c.City, cust.o.OrderID);
        }

        private void button3_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            var cust = db.Customers.Single( c => c.CustomerID == "ALFKI");
            var ord = db.Orders.Single( o => o.OrderID ==  10269);// customer WHITC

            ord.OrderDate = DateTime.Now;

            Order order = new Order();
            order.OrderDate = DateTime.Now;
            cust.Orders.Add(order);

            db.SubmitChanges();
        }

        private void button4_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            Northwind db2 = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            var cust = db.Customers.Single(c => c.CustomerID == "ALFKI");
            var ord = db2.Orders.Single(o => o.OrderID == 10269);// customer WHITC

            ord.OrderDate = DateTime.Now;

            Order order = new Order();
            order.OrderDate = DateTime.Now;
            cust.Orders.Add(order);

            db.SubmitChanges();
            db2.SubmitChanges();
        }
    }
}

(To be continued...)

Learning LINQ to SQL

2008-07-31T18:59:00.008-04:00

I've decided to spend a little time these next few evenings coming up to speed on LINQ (Language-Integrated Query) and writing about my experience with it. Let's see what sorts of problems I run into. I'm using as reference the 119 page LINQ to SQL: .NET Language-Integrated Query for Relational Data in order to jump start my learning. I hope that my SQL and C# experience will give me a head start and keep me moving. But, I have done nothing yet with the Northwind sample database (the database used in the white paper) in any of my projects or tests. I am not even sure where to get it yet. Ha, ha, Noobie on the loose, let me at it...

Let's Create some Entity Classes. The white paper starts out with creating entity classes. Granted, I'm starting in the "Quick Tour" section of the white paper, so I expect to find some detail ommisions. I'll take my chances. So, I jump right into the declaration of the Customer class.


    [Table(Name="Customers")]
    public class Customer
    {
        public string CustomerID;
        public string City;
    }

Well, by looking at it, I can see I am probably missing a few things already. I compile and sure enough, the errors tell me I need to reference something. My hunch is I need something like "using System.Data.Linq;", but since intellisense doesn't show it, I look under References instead and find "System.Data.Linq". I add it and now the intellisense is happy. I attempt to compile again.

The next stumbling block is the error that System.Data.Linq.Table<TEntity> requires a type argument. Hmm, let's hit MSDN and search for TableAttribute. Why not "Table<TEntity>" you ask? That's because the only "Table" reference in my program is [Table...] and I know this syntax is shorthand for [TableAttribute...]. Now, I could go looking for "Table<TEntity>", but the real problem is TableAttribute is not being found in one of my references, so the compiler is finding something else. I'll figure out the something else later. Right now, I'm on a mission. The search brings up "TableAttribute Class (System.Data.Linq.Mapping)". As I thought, I was missing another reference. I add the "using System.Data.Linq.Mapping;" and I'm back on track.

After adding column attributes and checking out my compile again, I next come to "The DataContext". This part of the white paper calls for "northwnd.mdf". Well, I've never used the Northwinds example database, so I have no clue where to find the file. I start searching the hard drive, and of course, I hit the search engines again searching for "northwnd.mdf sample file". Look's like someone else was having trouble finding it, too. The search rendered "Can't find Northwnd.mdf example database", so I read up. But my search of the hard drive also turns up a copy in the LinqSamples/Data directory. So, I use that (actually, I extract it from the CSharpSamples.zip file in c:\<install-location>\Samples\1033 ).

With database file in place and all code adjusted for the correct path, I compile and run. The next error I hit is an exception when I interate through my query results. The code up to this point looks like this...


        private void button1_Click(object sender, EventArgs e)
        {
            // DataContext takes a connection string 
            DataContext db = new DataContext(@"c:\LinqSamples\data\northwnd.mdf");
            // Get a typed table to run queries
            Table Customers = db.GetTable();
            // Query for customers from London
            var q =
               from c in Customers
               where c.City == "London"
               select c;
            foreach (var cust in q)
                Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
        }
    ...

    [Table(Name="Customers")]
    public class Customer
    {
        [Column(IsPrimaryKey=true)]
        public string CustomerID;
        [Column]
        public string City;
    }

The exception is occuring when I step through the code and land on "q" in...


foreach(var cust in q)

... That clues me in that database connections are not made until later than I expect, and some of this new LINQ syntax does not run immediately. Interesting. The exception says effectively that I don't have a SqlServer running, and upon checking my configuration, I find my SqlServer Express edition is stopped and set to start up manually. So, I turn it on and run my first attempt at a LINQ program. My output is ...

id = AROUT, City = London
id = BSBEV, City = London
id = CONSH, City = London
id = EASTC, City = London
id = NORTS, City = London
id = SEVES, City = London

Wow, that wasn't too painful. I have a working LINQ program. I recognize the white paper was not intended to be a tutorial, so I'm pleased with how few are the problems I've run into thus far. Maybe I should have started with a tutorial, but the level of detail here is perfect for someone like myself who is after a really rapid ramp-up, and who has enough experience to solve some of the basic problems on his own.

The DataContext is a little too loose for the author of the article, so I follow along by deriving the Northwind class from the DataContext and updating my code.


    public partial class Northwind : DataContext
    {
        public Table Customers;
        public Table Orders;
        public Northwind(string connection)
            : base(connection) {}
    }

    ...

        private void button1_Click(object sender, EventArgs e)
        {
            Northwind db = new Northwind(@"c:\LinqSamples\data\northwnd.mdf");
            // Query for customers from London
            var q =
               from c in db.Customers
               where c.City == "London"
               select c;
            foreach (var cust in q)
                Console.WriteLine("id = {0}, City = {1}", cust.CustomerID, cust.City);
        }
    ...

    [Table(Name="Customers")]
    public class Customer
    {
        [Column(IsPrimaryKey=true)]
        public string CustomerID;
        [Column]
        public string City;
    }
    [Table(Name = "Orders")]
    public class Order
    {
    }

So, with this declaration (and having to define the Order class on my own), I can now do without the DataContext and the GetTable<Customer>() call. I compile again and run it successfully. So, I proceed to "Defining Relationships".

Here the example code has an error. It applies the ColumnAttribute with an invalid parameter "Id". I believe the correct parameter to be "IsPrimaryKey", so I use that instead. I also find that here is where the Order class is defined, so I use this definition and fix the ColumnAttribute on OrderID as well (change "Id" to "IsPrimaryKey"). It all compiles, but I am concerned that neither "_Orders" nor "_Customer" are ever instantiated. Well, I don't know how "Storage=" works, so I look at it once more cautiously before moving on.

(to be continued...)

Timers are a Changin' (Part 3)

2008-07-30T19:20:00.007-04:00

In my last couple of articles, we saw the System.Windows.Forms.Timer and the System.Timers.Timer. Today, I shall look at the 3rd form of Timer in the .NET library, the System.Threading.Timer. I refer you again to the that very good article I mentioned a while back, Comparing the Timer Classes in the .NET Framework Class Library by Alex Calvo. Take a moment to read his section on System.Threading.Timer then come back and we will jump right into some code...


    using System.Threading;
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            timer1 = new System.Threading.Timer(new TimerCallback(timer_elapsed), null, 2000, 1000);
        }

        System.Threading.Timer timer1;
        private void timer_elapsed(object sender)
        {
            Console.WriteLine(DateTime.Now.ToString());
            TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
        }

        private delegate void TimerEventDelegate(string label, string name);
        private void TimerEvent(string label, string name)
        {
            if (label1.InvokeRequired)
            {
                BeginInvoke(new TimerEventDelegate(TimerEvent), new object[] { label, name });
            }
            else
            {
                label1.Text = "Thread:" + name + " - Time:" + label;
            }
        }
    }

To use this code above, create a new project named TimerExample3 and drop a label on the form. Then copy the code into your form1.cs replacing everything inside of the Namespace. It's ready to run!

The extra features gained with this timer are the ability to tell it to start at a future time, and you can pass it a state object. But there is another feature that is not so obvious. That is, the timer actually runs on the thread pool. If the timer_elapsed handler were to get hung in a sleep or something, that's okay because the next timer event will occur on time and fire the timer_elapsed again on a different thread. But, watch out! If you do not program for re-entrancy, you'll likely have problems.

At the moment, I have no guards for re-entrancy because the Console.WriteLine and the BeginInvoke should finish well within 1 second interval. So then, let's look at the behavior. First, notice that it takes 2 seconds before anything happens. That's in accordance with the 2000 millisecond startup time I gave the timer. Then you see the label updated with the thread number and time once every second along with console output. This is roughly the same as the last projects, only it took some very different code to get there.

Now, lets add a modal dialog to tie up the GUI thread and a button to tell the GUI to sleep as we did before. Here are the handlers, just drop two buttons on the form and wire up the handlers.


        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog f = new FolderBrowserDialog();
            f.ShowDialog();
        }
        private void button2_Click(object sender, EventArgs e)
        {
            Thread.Sleep(5000);
        }

Run the application and experiment with the buttons. The console output does not miss a beat regardless of which button you click. The form continues to update the label with the time while the modal dialog is visible, but not when clicking the button to sleep. So far, so good. No real difference from our last timer.

Next, modify the handler by adding a Sleep() call after the TimerEvent() call. This will simulate some long running timer event processing for the sake of making the event handler exceed the programmed interval. Then run and look at the thread number and time.


        private void timer_elapsed(object sender)
        {
            Console.WriteLine(DateTime.Now.ToString());
            TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
            Thread.Sleep(2000);
        }

Sleep(2000) ties up the thread running timer_elapsed() callback. So, the Timer simply allocates another thread to handle the next timer interval. You can see this in the thread number in the form label, there are different thread numbers for each update, probably alternating between two values as my version does. If you recall then, the System.Windows.Forms.Timer runs in the same thread as the form. The System.Timers.Timer runs in a thread different than the GUI and has to be synchronized to the GUI of choice. Now we see that the System.Threading.Timer runs in a separate thread allocated from the thread pool.

This got me to thinking. Is the System.Timers.Timer also allocated from the thread pool? With a few quick modifications to my TimerExample2 application, I found the answer is yes!

I thought the System.Timers.Timer it would be on its own dedicated thread. Finding that it's not, we need to realize that we must deal with re-entrancy in both System.Timers.Timer and System.Threading.Timer. This was not mentioned or wasn't clear in the article I referenced. The good news is that you can treat re-entrancy the same way for both handlers.

So, the only real new features of the System.Threading.Timer are the delayed start and being able to pass in a state variable. So I will be revisiting my last article to correct any dis-information I may have provided about single-threadedness. But, now I wonder, will the System.Timers.Timer "elapsed" handler provides useful information in the "object sender" parameter? Does this serve as nearly a state variable? Maybe, I will post again soon on these and other questions I find.

For now, here's the code for TimerExample3. I figure I should just let you have what we've got until more research has been done.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

namespace TimerExample3
{
    using System.Threading;
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            timer1 = new System.Threading.Timer(new TimerCallback(timer_elapsed), null, 5000, 1000);
        }

        System.Threading.Timer timer1;
        private void timer_elapsed(object sender)
        {
            Console.WriteLine(DateTime.Now.ToString());
            TimerEvent(DateTime.Now.ToString(), Thread.CurrentThread.GetHashCode().ToString());
        }

        private delegate void TimerEventDelegate(string label, string name);
        private void TimerEvent(string label, string name)
        {
            if (label1.InvokeRequired)
            {
                BeginInvoke(new TimerEventDelegate(TimerEvent), new object[] { label, name });
            }
            else
            {
                label1.Text = "Thread:" + name + " - Time:" + label;
            }
        }
        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog f = new FolderBrowserDialog();
            f.ShowDialog();
        }
        private void button2_Click(object sender, EventArgs e)
        {
            Thread.Sleep(5000);
        }
    }
}

... and here is my updated version of TimerExamples2, notice that I added a Button6 ...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Threading;

namespace TimerExample2
{
    public partial class Form1 : Form
    {
        System.Timers.Timer timer1;
        System.Timers.Timer timer2;
        public Form1()
        {
            InitializeComponent();
            timer1 = new System.Timers.Timer(1000);
            timer1.SynchronizingObject = this;
            timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
            timer1.Start();
        }

        private string theText(string signal)
        {
            string name = "Thread: " + Thread.CurrentThread.GetHashCode().ToString();
            string stime = "Time: " + DateTime.Now.ToString("hh:mm:ss");
            string s = name + " - " + stime + " - SignalTime: " + signal;
            return s;
        }
        void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {

            Console.WriteLine("Timer1 {0}",theText(e.SignalTime.ToString()));
            label1.Text = theText(e.SignalTime.ToString());
        }
        void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = theText(e.SignalTime.ToString());
            Console.WriteLine("Timer2 {0}",s);
            label2.Invoke(new MethodInvoker(() => label2.Text = s));
        }
        void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = theText(e.SignalTime.ToString());
            Console.WriteLine("Timer2 {0}", s);
            label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
        }
        void timer2_ElapsedAsynchSleep(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = theText(e.SignalTime.ToString());
            Console.WriteLine("Timer2 {0}", s);
            label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
            Thread.Sleep(2000);
        }

        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog fb = new FolderBrowserDialog();
            fb.ShowDialog();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            Form2 frm = new Form2();
            frm.ShowDialog();
        }

        private void button3_Click(object sender, EventArgs e)
        {
            Thread.Sleep(5000);
        }

        private void button4_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
                timer2.Start();
            }
        }

        private void button5_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
                timer2.Start();
            }
        }

        private void button6_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynchSleep);
                timer2.Start();
            }
        }
    }
}

Timers are a Changin' (Part 2)

2008-07-25T07:00:00.007-04:00

Today, I take a look at System.Timers.Timer. I suggested an article in my last discussion of timers. The article was written back in 2004, and it says that both the System.Windows.Forms.Timer and the System.Timers.Timer are found in the IDE Toolbox, one on the Windows Forms tab and the other on the Components tab. Well in my stock installation of Visual Studio 2008, both of those timers are System.Windows.Forms.Timer components. This may be due to some install problem I didn't know I had, or it may be the article is correct for Visual Studio .Net 2003 but not 2008.

That's okay because you can just add System.Timers.Timer to your project the old fashioned way. I simply start a new Windows Forms project and name it TimerExample2. After dropping a label on my form, I add the following code to my class file (I include a full listing at the end).


        System.Timers.Timer timer1; //added
        public Form1()
        {
            InitializeComponent();
            // added the following
            timer1 = new System.Timers.Timer(1000);
            timer1.SynchronizingObject = this;
            timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
            timer1.Start();
        }
        void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            label1.Text = DateTime.Now.ToString("hh:mm:ss");
            Console.WriteLine(e.SignalTime.ToString());
        }

When I compile and run this application, I have a simple clock ticking away the seconds. Behaviorally speaking, its not much different from the last timer project. By keeping things simple, I can take a closer look at the differences. Besides the different constructor and handler, the thing that most stands out is the "SynchronizingObject" property. This property clues us in to one of the most fundamental differences between the standard Timer and the System.Timers.Timer. The System.Timers.Timer will actually run on its own dedicated thread. If I ran without setting the SynchronizingObject to "this" (my Form1), the timer would not have direct to my main GUI label1. Try it yourself, comment out the line that sets the SynchronizingObject and try running the application. It will throw an exception when it reaches the line that sets label1.Text. The SynchronizingObject in effect determines the thread on which the event handler will run, the timer still runs in a separate thread, but its event handler is run on the thread of the form you choose (in my case I chose Form1). Or, as I said, if you don't set the SynchronizingObject, the timer's event runs on its own thread and any interaction with the GUI will require Invoke type calls.

So let's test what I just said with some code. Let's go ahead and add some buttons and code like we did in our last discussion. I have a Button1 to launch a modal dialog box, Button2 to launch a modal form (Form2), and Button3 to "sleep" for 5 seconds. The modal operations stop the main event loop and start a sub-event loop, while the "Sleep" pauses the thread and all its event loops. I've wired these all up using the IDE, and I created a Form2 to be called. The handlers are below.


        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog fb = new FolderBrowserDialog();
            fb.ShowDialog();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            Form2 frm = new Form2();
            frm.ShowDialog();
        }

        private void button3_Click(object sender, EventArgs e)
        {
            Thread.Sleep(5000);
        }

When you run the application and click the first button, you will notice the timer continues to run even though we have a modal dialog. But, that's no surprise because that's what the last Timer we looked at did. We determined last time that the modal dialog's event loop was picking up the events and dispatching the parent's timer events though the other events were being eaten while the modal dialog was open. Similarly, clicking on the second button shows a form with the ShowDialog() call, a modal call. The timer still ticks away for the same reasons. And, when you click the third button, the clock stops ticking for 5 seconds. It doesn't appear to be any different until you take a moment to review the console output. When the Sleep returns, there are actually 5 timer events processed immediately following. Herein lies the difference. During the sleep the System.Timers.Timer continued to operate and post tick events. The last timer we looked at, System.Windows.Forms.Timer, does not post these events, its ticks are lost. By operating in its own thread, System.Timers.Timer is not effected by the GUI thread's call to Sleep().

To further demonstrate the threaded-ness of this Timer, let's look as some more code. I add another label (label2) and another Timer (timer2) to Form1. Then I add a handler for timer2 and a Button4 which instantiates and starts the timer, and some additional information to the Console.WriteLine to distinguish the two event handlers' output. The added code basically looks like this...


        System.Timers.Timer timer2;
        void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = DateTime.Now.ToString("hh:mm:ss");
            label2.Invoke(new MethodInvoker(() => label2.Text = s ));
            Console.WriteLine("Timer2 {0}",e.SignalTime.ToString());
        }

        private void button4_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
                timer2.Start();
            }
        }

If you run the application now, when you click Button4, the second timer starts ticking away, and when you click it again, it stops. When you click Button3, the main GUI sleeps for 5 seconds and both timers stop, but when the GUI awakes, notice that only timer1 continued to post events. What happened to timer2's events? Timer2 posts its events to the GUI through Invoke(). Invoke() is a blocking/synchronous call, so though timer2 is on its own thread, the thread was stopped waiting for Invoke to return which in turn was waiting for the GUI to service the invoke request, which couldn't happen while the GUI was asleep. So, I add Button5 and another event handler for timer2.


        void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = DateTime.Now.ToString("hh:mm:ss");
            label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
            Console.WriteLine("Timer2 {0}", e.SignalTime.ToString());
        }
        private void button5_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
                timer2.Start();
            }
        }

Now, when you click Button5 to start the timer2 and then click Button3 to put the GUI to sleep, both labels stop updating, but the console output shows that timer2 continues to fire events. When the sleep completes, the timer1 events appear in the output. This is still not quite the same behavior as using the SynchronizingObject which is actually post a message to the GUI to fire the event handler. Our little example fires the event handler on the Timer thread and posts an asynchronous message to Invoke the update on "Text". But we don't need the exact same behavior. In order to make the best use of a timer on a dedicated thread, its best to let the tick event handler process rather than blocking it through Invoke() or by synchronizing it through the GUI event loop. This way, should the GUI sleep (which it should not) or get bogged down with lots of events (which it very well could), the timer on its own thread will still get its event on time (roughly). And if the CPU isn't too consumed, the timer thread should get enough cycles to perform your custom timer operations.

Well, I really wanted to get this article out much sooner than this. But I still would like to see how the System.Timers.Timer works in conjunction with an event loop on a different thread than the main GUI. For example, put Form1 on one thread, Form2 on another thread and then play around with the SynchronizingObject variable and see what happens. But alas, I need to post this and move on to the last Timer in my next article. Maybe I'll come back to these, or you can post your findings.

Anyway, here's the code for the article. You need to ...
1) start a project called TimerExample2,
2) drop a couple of labels on Form1,
3) drop 5 buttons on Form1,
4) add another Form (Form2),
5) replace the code in your form1.cs with the code below, and
6) connect each of your buttons' click handlers to the appropriate handler below


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Threading;

namespace TimerExample2
{
    public partial class Form1 : Form
    {
        System.Timers.Timer timer1;
        System.Timers.Timer timer2;
        public Form1()
        {
            InitializeComponent();
            timer1 = new System.Timers.Timer(1000);
            timer1.SynchronizingObject = this;
            timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Elapsed);
            timer1.Start();
        }

        void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            label1.Text = DateTime.Now.ToString("hh:mm:ss");
            Console.WriteLine("Timer1 {0}",e.SignalTime.ToString());
        }
        void timer2_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = DateTime.Now.ToString("hh:mm:ss");
            label2.Invoke(new MethodInvoker(() => label2.Text = s ));
            Console.WriteLine("Timer2 {0}",e.SignalTime.ToString());
        }
        void timer2_ElapsedAsynch(object sender, System.Timers.ElapsedEventArgs e)
        {
            string s = DateTime.Now.ToString("hh:mm:ss");
            label2.BeginInvoke(new MethodInvoker(() => label2.Text = s));
            Console.WriteLine("Timer2 {0}", e.SignalTime.ToString());
        }

        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog fb = new FolderBrowserDialog();
            fb.ShowDialog();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            Form2 frm = new Form2();
            frm.ShowDialog();
        }

        private void button3_Click(object sender, EventArgs e)
        {
            Thread.Sleep(5000);
        }

        private void button4_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_Elapsed);
                timer2.Start();
            }
        }

        private void button5_Click(object sender, EventArgs e)
        {
            if (timer2 != null)
            {
                timer2.Stop();
                timer2.Dispose();
                timer2 = null;
            }
            else
            {
                timer2 = new System.Timers.Timer(1000);
                timer2.Elapsed += new System.Timers.ElapsedEventHandler(timer2_ElapsedAsynch);
                timer2.Start();
            }
        }
    }
}

Timers are a Changin' (Part 1)

2008-07-24T09:28:00.002-04:00

Sometimes mundane things turn out to be really nice after all. No, I'm not talking about one of your last dates. I'm talking about timers, yes, the Timer classes in C#. These little tools are found everywhere in programming, and it seemed for the longest time, there was only one stock timer to choose from. Well, now there are three.

Before I go any further, let me recommend to you Alex Calvo's MSDN magazine article Timers: Comparing the Timer Classes in the .NET Framework Class Library. Here, you will find some good background for applying timers to your application.

But, in this little series, I want to look at how the timers interact with the event loop. This first article in the series will look at the System.Windows.Form.Timer.

The System.Windows.Form.Timer is supposed to be synchronous with respect to the rest of your Windows Forms app. That means, if you "sleep" or block in some other way, the timer will stop working (while sleeping). However, programmers forget the one caveat. That is, the timer stops working unless "DoEvents" is called (or it's equivalents, let me add). Take a look a the following example. Create a new Forms project. From the toolbox, drop a timer, a button, and a label on the form. Replace the code inside your namespace with the code below. Add a "using System.IO;" to the top. Then link the click and timer events to the appropriate functions below.


    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            timer1.Start();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog f = new FolderBrowserDialog();
            f.ShowDialog(this);
        }

        private void timer1_Tick(object sender, EventArgs e)
        {
            label1.Text = DateTime.Now.ToString("hh:mm:ss");
        }
    }

When you run the example, the time is displayed and updated ever 1/10 of a second (the default 100ms tick). And when you click the button, a FolderBrowserDialog is launched, and the time keeps ticking? Doesn't the article we just read say the timer is synchronous with the rest of our Windows Form app. It says it won't preempt our code as long as we don't call DoEvents(). Yet the modal FolderBrowserDialog has blocked all our other events from being processed. What's going on?

When ever things don't behave as you expect, you have an opportunity to learn something. So, I made a few changes to the code above so that I can analyse further. I added a button to launch a modal MessageBox and a button to call System.Threading.Thread.Sleep(). Now I have...


    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            timer1.Start();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog f = new FolderBrowserDialog();
            f.ShowDialog(this);
        }

        private void timer1_Tick(object sender, EventArgs e)
        {
            label1.Text = DateTime.Now.ToString("hh:mm:ss");
        }

        private void button2_Click(object sender, EventArgs e)
        {
            MessageBox.Show("hello");
        }

        private void button3_Click(object sender, EventArgs e)
        {
            System.Threading.Thread.Sleep(5000);
        }
    }

Now, when you click on button2, a MessageBox is displayed, and the timer keeps ticking. But if you click on button3, the timer pauses for 5 seconds. So, some operations will stop the timer and some won't. Sleep() will suspend the timer events. But, FolderBrowserDialog and MessageBox and other file dialogs will not. What operations will suspend the timer events? And which operations are modal but will not interfere with the timer? It's important to know because the operations that interfer with the event loop (like Sleep) will interfer with more than just the timer, they will make your application appear to hang.

To answer this question, run the code again, but while its running with no buttons having been clicked, insert a breakpoint in the timer1_tick callback. When the code breaks at this breakpoint, look at the call stack. You should see some external code, above that the main appliction, above that some more external code, and above that the timer1_tick at the top of the stack. Right click on the external code stack element and select the "show external code" context menu item. Reading the stack you can see that above Main() we have the Application.Run() call and some additional calls. These calls implement the main event loop. A timer tick triggers the callback and you find the debuggable callback and further calls to the timer1_tick callback.

Clear the break point and continue your program. Now click button one so that the FolderBrowserDialog opens. Next, set your breakpoint in timer1_Tick() again. Looking at the stack trace this time, we see external code, above that Main(), above that external code, above that button1_Click(), above that somemore external code, and finally above that timer1_Tick(). We deduce a couple important things based upon this experiment. 1) All the controls, including the timer and the FolderBrowserDialog, run in the same thread. That is why sleeping your thread makes everything stop responding. 2) Modal windows do not block the thread (nor do they call DoEvents), they are windows forms in and of their own right and are not required to know or care that they have a parent window. In fact, if you add another Form to the project (say Form2.cs), place a button on it to close the form, and add a button to Form1 to create and show the form using ShowDialog(), then you will see the same behavior.

But, why then are the timer events getting through to Form1 at all while mouse events and other events are not? If the call stack shows the timer1_Tick() getting called deep in the call stack well away from the parent's event loop. The answer lies in the fact that ShowDialog has its own internal event loop. This event loop receives the timer_Tick event, but since it does not belong to this loop, its dispatcher processes the event on the parent window. Some events are eaten by the dispatcher in order to implement "modality", others are not.

The following has button4 added to show the new form. You would have to add a button to your Form1 and link in the button4_Click() handler. And you would need to implement a simple Form2.cs, mine just has an exit button.


    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            timer1.Start();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            FolderBrowserDialog f = new FolderBrowserDialog();
            f.ShowDialog(this);
        }

        private void timer1_Tick(object sender, EventArgs e)
        {
            label1.Text = DateTime.Now.ToString("hh:mm:ss");
        }

        private void button2_Click(object sender, EventArgs e)
        {
            MessageBox.Show(this, "hello");
        }

        private void button3_Click(object sender, EventArgs e)
        {
            System.Threading.Thread.Sleep(5000);
        }

        private void button4_Click(object sender, EventArgs e)
        {
            Form2 frm = new Form2();
            frm.ShowDialog(this);
        }
    }

We have just shown that opening a form with ShowDialog() puts the forms new event loop on the same thread as the main GUI event loop. Form2 is shown modally, so Form1 stops responding to all events except the timer event. But, what if I start a new thread for the second form and ShowDialog() on the new thread, will the event loop run on the new thread separate from the main GUI thread? Yes, but this is well off topic, maybe I'll discuss that in another article.

Well we learned almost nothing about configuring a timer control. But, you should already know how to do that anyway. I simply hope this discussion gives you a little more insight into how the stock System.Windows.Form.Timer will operate under different situations.

In my next article in this series, I will look at the System.Timers.Timer in more detail.

Why Try? Finally!

2008-07-18T13:45:00.006-04:00

Today we talk about the "try/finally" construct. This article should be very simple for most of you with any C# or C++ experience to understand. I've written a sample that you will find at the end of this article to let you see the various affects of the "finally" statement under different scenarios.

But why use "finally" at all? I like to think of "finally" as being to the flow of logic what the "destructor" or "dispose" is to the life of an object. It is a common place to put all the necessary logic for shutting down the logic context. With the ubiquitous "execption" lurking behind every corner, you never know when you will be forced to leave the context, and it's nice to have a place for common cleanup code. You say you could just write the cleanup in the common exception handler. But then you have to repeat the code, line for line, in every other named exception handler as well as the error free path of execution (not to mention, you may just want the exception to go unhandled). The "finally" construct allows you to avoid duplication of logic or insertion of unnecessary logic. One of my favorite examples of the try/finally construct is the use of "Monitor".


    Monitor.Enter(lockObject);
    try {
        // critical section
    }
    finally {
        Monitor.Exit(lockObject);
    }

Here, if the Monitor.Enter() is successful, it MUST be followed by a Monitor.Exit() or else everthing could come to a halt. This example has been replaced by the built-in compiler directive "lock(lockObject){...}" but that isn't always the best choice.

Let me repeat. The main usefulness of "finally" is to avoid duplication of cleanup logic whether normal or exceptional. It provides one place to write the logic (and maintain it).

Here is a break-down of the how the this features behaves. First, we start with one of the simplest examples of try/finally.


    try {
        // do your thing
        Console.WriteLine("Example 1: In Try.");
    }
    finally {
        // run some code that must alway follow our "try"
        Console.WriteLine("Example 1: In Finally");
    }
    // stuff to run after the try/finally
    Console.WriteLine("Example 1: After the Try/Finally");

The compiler guarantees that the "finally" block of code will run upon the exit of the "try" block, regardless of how the block exist. In Example 1, the try block exits according to the normal flow of code falling out of the block. Example 2 demonstrates some more normal flow, but this time it is through the "return" statement.


    try
    {
        Console.WriteLine("Example 2: Try called");
        return;
    }
    finally
    {
        Console.WriteLine("Example 2: Finally called");
    }
    Console.WriteLine("Example 2: More code called");

Here you can see that even though we "return" within the "try", the "finally" is still run. But, the stuff following the "finally" block does not run. That code is effectively unreachable now. But what happens if an exception is thrown in the "try" block. The "finally" should still be called as the next example demonstrates. I have included a "catch" to absorb the exception for the sake of the example.


    try
    {
        Console.WriteLine("Example 3: Try called");
        throw new Exception("Example 3: Exception thrown");
    }
    catch (Exception ex)
    {
        Console.WriteLine("Example 3: Exception Caught: ErrMsg={0}", ex.Message);
    }
    finally
    {
        Console.WriteLine("Example 3: Finally called");
    }
    Console.WriteLine("Example 3: More code called");

You may have noticed that the last line of code following the finally block executes this time, but note the order. First the "try" executes, then the "catch" executes, then the "finally" and then the code following the "finally" block. Again, "finally" works as advertised. We could also "return" from the "catch" block and "finally" will still run.


    try
    {
        Console.WriteLine("Example 4: Try called");
        throw new Exception("Example 4: Exception thrown");
    }
    catch (Exception ex)
    {
        Console.WriteLine("Example 4: Exception Caught: ErrMsg={0}", ex.Message);
        return;
    }
    finally
    {
        Console.WriteLine("Example 4: Finally called");
    }
    Console.WriteLine("Example 4: More code called");

... and of course, the line following the "finally" block does not execute because of the return. You say, "yeh, but you are catching the exception, what if you don't catch the exception?" I'm glad you asked. This next example makes use of a helper function to demonstrate just that. In the helper function, there is a "try/finally" that throws an exception in the "try". The exception is not caught in that scope and context. I've added a catch higher up in the stack to allow you to prove these examples in a debugger. You could do without the higher catch if you want to compile and run the application outside of debug mode. But, everything is eventually caught, even if it is by the run-time. Regardless, here is the example.


    try
    {
        button5helper();
    }
    catch (Exception ex)
    {
        Console.WriteLine("Example 5: Outer catch: ErrMsg={0}",ex.Message);
        return;
    }
    ...
    private void button5helper()
    {
        try
        {
            Console.WriteLine("Example 5: Try called");
            throw new Exception("Example 5: Exception thrown");
        }
        finally
        {
            Console.WriteLine("Example 5: Finally called");
        }
        Console.WriteLine("Example 5: More code called");
    }

The example will run "try", then "finally", then the "catch" in the outer context. The last line after the "finally" block is not run. But you ask, "what happenens when the "finally" throws an exception?" This next example will show you.


    try
    {
        button6helper();
    }
    catch (Exception ex)
    {
        Console.WriteLine("Example 6: Outer catch: ErrMsg={0}",ex.Message);
        return;
    }
    ...
    private void button6helper()
    {
        try
        {
            Console.WriteLine("Example 6: Try called");
            throw new Exception("Example 6: Exception thrown");
        }
        finally
        {
            Console.WriteLine("Example 6: Finally called");
            throw new Exception("Example 6: Exception thrown by finally");
        }
        Console.WriteLine("Example 6: More code called");
    }

Here, the "try" is called wich throws an exception, which results in the finally being called which throws an exception, followed by the "catch" in the outer context being called. But, ooh! I guess nothing's perfect. Look a the printout from the "catch". It says that it caught the exception thrown by "finally", so what happened to the original exception? Well, it's lost, just like if your "catch" logic were to throw an unintended exception. The only difference is that if your catch logic intends to throw a new exception, it would have the original exception to work with, the "finally" does not. Suffice it to say, "finally" should never "intend" to throw an exception, nor should it be used to contain the bulk of your logic (where exceptions are more likely to lurk).

Our next example shows "finally" being called after "try" throws an exception and "catch" rethrows the exception. Then "finally" also throws an exception.


    try
    {
        button7helper();
    }
    catch (Exception ex)
    {
        Console.WriteLine("Example 7: Outer catch: ErrMsg={0}",ex.Message);
        return;
    }
    ...
    private void button7helper()
    {
        try
        {
            Console.WriteLine("Example 7: Try called");
            throw new Exception("Example 7: Exception thrown");
        }
        catch (Exception ex)
        {
            Console.WriteLine("Example 7: Catch caught: ErrMsg={0}",ex.Message);
            Console.WriteLine("Example 7: Catch rethrows exception");
            throw ex;
        }
        finally
        {
            Console.WriteLine("Example 7: Finally called");
            throw new Exception("Example 7: Exception thrown by finally");
        }
    }

As expected, the try runs, then the catch runs, then the finally runs, then the outer context catch runs. The outer context can only recover the exception info from the "finally".

We haven't touched on threads here. But if its a child thread you are worried about, don't. The try/finally construct is implemented in the compiler and is thread safe. If your thread catches any exceptions, the behavior should be the same as in these examples. But, if your thread lets the exception go un-caught, then the parent thread gets the exception and will have a hard time dealing with it. But, the child thread will at least have cleaned up in the "finally" before the parent dies an ungraceful death.

Oh, and what about the C++ "try/except" construct? This is a C# blog, but I will answer that one, too. The "except" construct is only implemented in C++ and VB (under the name "catch"). It is called a User-Filtered Exception and allows higher level contexts to insert specialized handling into lower level exceptions. In so doing, they insert extra programming from outside of the try/finally context that your code is probably not aware of. It kind of violates the whole purpose of "finally". But that's a C++ and VB problem that C# doesn't have (yet). And the issue and solution are well documented.

Anyhow, here's the example listing. Just create a new Window Forms application, drop 7 buttons on the form, then open the form's code and replace it all with the following...


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

namespace TryCatchFinallyExample
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            // A simple try finally, the try will run, then the finally, then the remaining code
            try
            {
                Console.WriteLine("Example 1: Try called");
            }
            finally
            {
                Console.WriteLine("Example 1: Finally called");
            }
            Console.WriteLine("Example 1: More code called");
        }

        private void button2_Click(object sender, EventArgs e)
        {
            // Now the try returns, never reaching the extra code, but finally still runs
            try
            {
                Console.WriteLine("Example 2: Try called");
                return;
            }
            finally
            {
                Console.WriteLine("Example 2: Finally called");
            }
            Console.WriteLine("Example 2: More code called");
        }

        private void button3_Click(object sender, EventArgs e)
        {
            // Now the try throws an exception, the exception is caught, finally 
            // runs, additional code runs
            try
            {
                Console.WriteLine("Example 3: Try called");
                throw new Exception("Example 3: Exception thrown");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 3: Exception Caught: ErrMsg={0}", ex.Message);
            }
            finally
            {
                Console.WriteLine("Example 3: Finally called");
            }
            Console.WriteLine("Example 3: More code called");

        }

        private void button4_Click(object sender, EventArgs e)
        {
            // Now the try throws an exception, the exception is caught and return called,
            // but finally still runs, however additional code does not run
            try
            {
                Console.WriteLine("Example 4: Try called");
                throw new Exception("Example 4: Exception thrown");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 4: Exception Caught: ErrMsg={0}", ex.Message);
                return;
            }
            finally
            {
                Console.WriteLine("Example 4: Finally called");
            }
            Console.WriteLine("Example 4: More code called");
        }

        private void button5_Click(object sender, EventArgs e)
        {
            try
            {
                button5helper();
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 5: Outer catch: ErrMsg={0}",ex.Message);
                return;
            }
        }
        private void button5helper()
        {
            // Now the try throws an exception, but the exception is not caught, finally 
            // runs, additional code does not
            try
            {
                Console.WriteLine("Example 5: Try called");
                throw new Exception("Example 5: Exception thrown");
            }
            finally
            {
                Console.WriteLine("Example 5: Finally called");
            }
            Console.WriteLine("Example 5: More code called");
        }

        private void button6_Click(object sender, EventArgs e)
        {
            try
            {
                button6helper();
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 6: Outer catch: ErrMsg={0}", ex.Message);
                return;
            }
        }
        private void button6helper()
        {
            // Now the try throws an exception, but the exception is not caught, finally 
            // runs amd also throws and exceptoin, additional code does not run
            try
            {
                Console.WriteLine("Example 6: Try called");
                throw new Exception("Example 6: Exception thrown");
            }
            finally
            {
                Console.WriteLine("Example 6: Finally called");
                throw new Exception("Example 6: Exception thrown by finally");
            }
            Console.WriteLine("Example 6: More code called");

        }

        private void button7_Click(object sender, EventArgs e)
        {
            try
            {
                button7helper();
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 7: Outer catch: ErrMsg={0}",ex.Message);
                return;
            }
        }
        private void button7helper()
        {
            // Now the try throws an exception, the exception is caught but catch
            // throws an exception, finally runs and also throws and exception,
            // additional code does not run
            try
            {
                Console.WriteLine("Example 7: Try called");
                throw new Exception("Example 7: Exception thrown");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Example 7: Catch caught: ErrMsg={0}",ex.Message);
                Console.WriteLine("Example 7: Catch rethrows exception");
                throw ex;
            }
            finally
            {
                Console.WriteLine("Example 7: Finally called");
                throw new Exception("Example 7: Exception thrown by finally");
            }
            
        }
    }
}

Thread Synchronized Queing

2008-07-17T10:03:00.019-04:00

Today's topic gets back to some multi-threading. I find it very useful to have threads pull work from a queue and have one master assignment manager feed the queue. However the standard System.Collections.Generic.Queue is a little inadequate for me because it is not fully thread-safe. I found it a little disheartening that the Queue class members are not overridable either. So we write our own, here are my requirements...

The queue must synchronize multiple writer threads writing elements to the queue.

The queue must synchronize multiple reader threads reading from the queue.

If there is nothing in the queue, a reader thread must wait for something or timeout.

Those waiting on the queue can be “interrupted”.

Of course, no two readers will ever get the same element off the queue.

So let's start coding...

I start with a new class called SyncQueue in my own name space and I will make it generic.


namespace  QueueExample
{
 public class SyncQueue
 {
  public void Enqueue(T element)
  {
   // we'll get to it
  }
  public T Dequeue()
  {
   return Dequeue(-1);  // wait forever
  }
  public T Dequeue(int timeout_milliseconds)
  {
   // we'll get to this later
   return default(T); // effectively null
  }
  public void Interrupt()
  {
  }
  public T Peek()
  {
   return default(T);
  }
  public int Count { get { return 0; } }
 }
}

So there's the basic skeleton of the queue. You can add elements with Enqueue(), remove elements with Dequeue() and even look at the next element with Peek(). There is an operation for interrupting watiting threads called Interrupt(), and for good measures, I will be able to check how many elements are on the queue with a Count property.

But, how will we synchronize the threads you ask. We will make use of a couple of synchronization tools at our disposal. After adding the name space System.Threading, we have two WaitHandles available to us, AutoResetEvent and ManualResetEvent. So I add them to my class in a private array of WaitHandles called "handles";



 private WaitHandle[] handles = {
   new AutoResetEvent(false),
   new ManualResetEvent(false),
     };

The other item we will need is a lock so that we can protect the queue from multiple threads tampering with its critical members at the same time. Oh! and lest we forget, we need to add a queue. So I add a System.Collections.Generic.Queue to my class and call it _q.


 private Queue _q = new Queue();

And now when I need to access a member of _q, I will wrap the critical section with a lock(_q) { } block. So, let's start with something real easy. Let's start with Count.


 public int Count { get { lock(_q){ return _q.Count; } } }

Count get the _q.Count value after first gaining exclusive access to the queue. The fact is, it really shouldn't matter because as soon as the lock(_q) is released, the count could be changed. This value can be very dynamic so its instantaneous value is of limited use. The lock could be omitted entirely, since interlocking the value won't affect our results.

Next, lets look at Peek(). It returns the value of the next item on the queue. But, unless there is some form of thread synchronization external to the queue class, peeking at the queue is just for fun because by the time you actually try to dequeue something, the one you peeked will probably be gone. But, here it is anyway, in case you have some additional synchronization you'd like to add.


 public T Peek()
 {
  lock(_q)
  {
   if(_q.Count > 0)
    return _q.Peek();
  }
  return default(T);
 }

I haven't mentioned it yet, but notice that there is no unlock() that one must call. The lock() is convenient because it effectively implements a try/finally block with a “Monitor” that looks like this...


 Monitor.Enter(_q)
 try {
  //do your stuff
 }
 finally {
  Monitor.Exit(_q);
 }

Thus it doesn't matter if you return() or throw() an exception, the “finally” will always be called to release the lock. Monitors have some other nice features, but we'll leave that for another day.

We still haven't used that “handles” array I create nor have we implemented the Enqueue, Dequeue or Interrupt, so let's move on. Next, let's look at Enqueue.


 public void Enqueue(T element)
 {
  lock(_q)
  {
   _q.Enqueue(element);
   ((AutoResetEvent)handles[0]).Set();
  }
 }

Enqueue locks the queue, adds the element and then signals any thread waiting for a queue element with the AutoResetEvent we setup above. The AutoResetEvent was created in the Reset state so that any thread attempting to remove an item before anything is added will have to wait. By calling the “Set()” method, the Enqueue method is effectively releasing one waiting thread to attempt to lock and read the queue and immediately resets so that only one waiting thread is released.

Now for Dequeue. It needs to lock the queue, check if there are any elements, and if not, unlock the queue and wait for something to come available. Note that Dequeue cannot lock the queue and then while locked wait for something to come available. With the queue locked, nothing would ever come available.


 public T Dequeue(int timeout_milliseconds)
 {
  T element;
  try
  {
   while(true)
   {
    if(WaitHandle.WaitAny(handles, timeout_milliseconds, true) == 0)
    {
     lock(_q)
     {
      if(_q.Count > 0)
      {
       element = _q.Dequeue();
       if(_q.Count >  0)
        ((AutoResetEvent)handles[0]).Set();
       return element;
      }
     }
    }
    else
    {
     return default(T);
    }
   }
  }
  catch (Exception e)
  {
   return default(T);
  }
 }

That's a lot more complex than the rest of the methods, so let's look at it in detail starting from the outer most block and working in. We start with a “try” block because we are going to be doing things that are known to cause exceptions from time to time. Specifically, we will be calling WaitAny() which throws an exception if a thread aborts while holding a synchronization handle we need. WaitAny can throw an exception for several other reasons, too. So, we need to deal with the possibility (just returning a null is not the best response, you should expand this section).

Next step down is a loop where threads compete for the queue and its elements. The first step is to wait. When ever something is added to the queue, the AutoResetEvent is Set, so we wait for something to be added. If something has already been added, the event will be set when the thread gets to this point. The WaitAny call will return 0 if the AutoResetEvent is set or it will return 1 if the ManualResetEvent is Set. It returns a value equal to WaitHandle.WaitTimeout if the operation times out. Therefore we are looking for 0 if we are to possibly take an element from the queue. The other values will result in the else condition being met and the default return value of null being returned. This will be the basics to the operation of the Interrupt() method.

Now if a 0 is returned, there may be something on the queue and the thread is released to try and lock the queue. It is possible that other items were enqueued as well resulting in multiple threads trying to get to the queue, so the lock(_q) is again necessary. Upon obtaining the lock, there is a possibility that there are no elements on the queue, so we first check the count and only dequeue if the count is greater than 0. If nothing is there, we go back to the WaitAny and wait again. After taking an element off the queue, we again check the count and if there are still elements, we Set the AutoResetEvent so the next waiting thread can proceed.

Last, we have the Interrupt() method. The Interrupt() method is now very easy.


 public void Interrupt()
 {
  ((ManualResetEvent)handles[1]).Set();
 }

The Interrupt() method uses the ManualResetEvent to signal waiting threads. This type of event stays signalled until manually reset. So all threads waiting and andy threads coming to the WaitAny will immediately release and fall into the else condition. Note that Interrupt and Timeout are indistinguishable based upon return value. Your thread architecture will need to deal with the higher issues of interrupting threads.

I moved through the dequeue code pretty quickly, and you may be wondering why the dequeue fires the AutoResetEvent event at all. Maybe you think if it didn't fire the event then there wouldn't be any competition for the queue and then there wouldn't be the need to check if there were anything on the queue. You may ask isn't it guaranteed that there is something on the queue if you get the event signal? No, its not.

First, consider the very simple and likely case that several items will be added to the queue before any thread gets an element from the queue. The AutoResetEvent will be Set() multiple times, but only allow one thread through, It does not keep track of how many elements there are, it is only a signal to say that there is at least one element on the queue. So, if there are multiple elements on the queue, even though the AutoResetEvent has been Set() several times, the first thread that dequeues results in the AutoResetEvent being Reset(), therefore, it must signal again, or the queue will stop working.

Now, consider the condition where an item is added to the queue while 3 threads already wait for an element. One thread releases on the event being signaled, but before it can obtain the lock, yet another thread writes another element to the queue and sets the event. This releases the second thread. The two free threads now compete for the lock and only one gets through while the other waits at the lock. Now when the winner takes an element off the queue, it finds another element still there, so it signals the event again according to our first scenario. This releases the third thread to compete with the second thread over the single remaining item on the thread. Therefore, they must check the quantity before they try to take an element. One of the threads will lose and have to go back and wait.

This queue is rather simple. It does not limit capacity, or seriously block any writers from adding to the queue. The error checking in weak, but all in all, you should be able to gleen a better understanding of the complexities of queue synchonization in a multi-threaded application. And you get some working code to start with to boot! Here is the full listing ...


using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace QueueExample
{
    public class SyncQueue
    {
        private WaitHandle[] handles = {
       new AutoResetEvent(false),
       new ManualResetEvent(false),
                                       };
        private Queue _q = new Queue();

        public int Count { get { lock (_q) { return _q.Count; } } }
        public T Peek()
        {
            lock (_q)
            {
                if (_q.Count > 0)
                    return _q.Peek();
            }
            return default(T);
        }

        public void Enqueue(T element)
        {
            lock (_q)
            {
                _q.Enqueue(element);
                ((AutoResetEvent)handles[0]).Set();
            }
        }

        public T Dequeue(int timeout_milliseconds)
        {
            T element;
            try
            {
                while (true)
                {
                    if (WaitHandle.WaitAny(handles, timeout_milliseconds, true) == 0)
                    {
                        lock (_q)
                        {
                            if (_q.Count > 0)
                            {
                                element = _q.Dequeue();
                                if (_q.Count > 0)
                                    ((AutoResetEvent)handles[0]).Set();
                                return element;
                            }
                        }
                    }
                    else
                    {
                        return default(T);
                    }
                }
            }
            catch (Exception e)
            {
                return default(T);
            }
        }

        public T Dequeue()
        {
            return Dequeue(-1);
        }

        public void Interrupt()
        {
            ((ManualResetEvent)handles[1]).Set();
        }
        public void Uninterrupt()
        {
            // for completeness, lets the queue be used again
            ((ManualResetEvent)handles[1]).Reset();
        }
    }
}

For more on queing, take a look at my article A Simple Task Queue.

Locking Guidelines

2008-07-13T22:30:00.006-04:00

In my last post (Lock Up Unlocked), I gave you some basic guidelines for locking resources in multi-threaded programs.

Basic guidelines for locking.

Avoid simultaneously locking multiple shared resources in a single thread.

If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.

If they can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.

If you can't follow these guidelines, then timeout.

In today's post, I am going to elaborate a little on each principle. But first, let's discuss "resources". Many of you can quickly name the commonly understood resources. Things like memory, cpu, network jump right out, but that's because we are so used to looking at Task Manager and Resource Monitor. Besides, they are a little esoteric, programmers are concerned about specific area of memory when it comes to synchronization and locking, not "memory" as that big fuzzy thing that we don't even manage ourselve anymore. So, "share memory" locations are one resource, a communications port is another.

But look at it another way. ANYTHING that would cause the thread to wait suggests a resource. Yes, we insert our own locks to make the thread wait for our resources, but there are many operations the thread will perform that could cause the thread to wait. Even a "sleep" operation is a wait for the resource of elapsed time. Or maybe you put your thread to sleep to yield the processor resource for another thread. Reading a file, allocating memory, reading a communications port, all could result in a wait. Writing a file, sending an event, throwing an exception, could result in a wait as well. With C# it has become easy to hook just about everything and with our hooks, we could feasibly insert waits, or blocking calls to obtain resources our event handler needs. In the last post, I pointed out that one of the locked resources in the program was the GUI's event loop. Since every wait is a kind of "lock", it behooves us to know when we are likely to block if we are to accurately apply the "basic guidelines for locking. Now, let's take a look at them.

Avoid simultaneously locking multiple shared resources in a single thread.
This one is almost too easy. If a thread only locks one resource, it is never holding a lock some other thread needs while waiting for that lock. Therefore, in is never in the way of another thread. It may compete with other threads, but that's really the name of the game in thread synchronization.

If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.
It can be easily shown that if all threads honor an agreed upon sequence of resource locking, then deadlock is prevented, since no thread can lock resource RB without first locking RA, then the holder of the lock on RA can be sure he's not in the way of some thread holding a lock on RB. This principle becomes particularly important when working with semaphores that allow several threads to access the resource (RA). This is typically done with read-only access to a resource. If write access is needed (RB), read access must first be obtained, preserving the order of locks.

If they (the resources) can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.
This approach really only works for applications that need to move through a predefined set of states. If the thread never moves backwards in the states, it can unlock previous locks. Think about a person moving from station to station in a sequence of operations. If the thread must backup, it can't release earlier locks, it must unlock in reverse order.

If you can't follow these guidelines, then timeout.
One of the important skills of multi-threaded programming is writing your wait call in such a way as to handle a timeout gracefully. It's even more important when you know you've violated one of the guidelines and therefore will likely deadlock without a timeout. Examine Microsoft's Reader-Writer class and example to see how an attempt is made to gain a write lock, but a timeout is coded for just in case the write lock can't be gotten.

We started this series with a simple deadlock on what looked like a critical sections. "Locking" and its twin brother "waiting" can do a whole lot more than critical sections. But, how then was the deadlock solved? Yes, we could have just removed the locks, but that would have been no fun. The deadlock was prevented when the child thread chose not to lock RB while holding the lock on RA. BeginInvoke() is analogous to the Post Event call of the old days. It does not wait for the GUI thread to service the event. The child thread then releases the lock (RA), then tries to lock only RB by calling EndInvoke() to wait for completion of the event. Since the first guideline is not violated, there is no deadlock.

No code today.

Lock Up Unlocked

2008-07-12T07:14:00.009-04:00

In my last post (Locked Up!), I gave an example of a simple deadlocking application. If you run it and click the button a few times rapidly, the application stops responding. Then, if you pause the program and examine the stack trace of each thread, you find the GUI thread waiting for the lock, and the child thread waiting for Invoke() to return. The child thread is holding the lock, so the application is deadlocked. This is also known as a "deadly embrace".

So, I affectionately referred to the two resources as Resource RA and Resource RB and asked you to identify them in the program. One of the resources is pretty obvious. That would be the lblResult.Text. It doesn't matter whether you call this RA or RB, so I'll call it Resource RA. Regardless, for some odd reason, the programmer has decided that the lblResult.Text property needs to be protected from simultaneous read and write. This protection is not really necessary, but pretend for a moment a valid sort of resource did indeed need this kind of protection. Thus, we have the lock on Resource RA as lock(lockObject) { ... }. Were you going to say lockObject is RA? That's okay, that can work, too. But, as I'll try to show, its preferable to think of the lockObject as part of the mechanism for locking a resource, and consider the code or resource being protected as the true resource.

Determining Resource RB is more difficult, since there are no other locks in the code to make it stand out. There is only the lock around the reading and the lock around the writing of the text field. The key to figuring out what Resource RB is is to look for where the program is waiting or blocking.

The child thread is stuck waiting for Invoke() to complete. Invoke() is not the resource and technically, its not the lock either. But, none the less, that is the point at which the child appears blocked. The child is waiting for a resource to come free. The Resource RB is then the GUI thread's processing of the child thread's request. But, the GUI thread did not lock a resources, or did it?

The GUI thread is regularly returning to the event loop to process the next event on the queue or wait until there are events (it's more complex than this, but we'll keep it simple for the sake of the discussion). A queue such as this is a locking mechanism. While the GUI is busy, servicing an event, anyone waiting for a particular event to complete is blocked until the GUI pulls the event and processes the event being waited for. You can think of the GUI thread as having locked RB by pulling an event and working on it. In this simple case, the GUI thread actually pulled the child thread's event (Invoke request) and tried to work on it. But, the child had locked the resource the GUI needed to continue. Consider the more complex case where the GUI pulls some other event for some other thread waiting for service and the problem could become daunting.

Now, the whole purpose in thinking about this problem as dealing with just two resources RA and RB is to present a simple analytic approach. You may have hit a deadlock or two in your novice foray into multi-threaded applications. Having hit a deadlock or related race condition, you may not have had the analytic tools to diagnose the problem. So start with the first tool called "understanding". The basic or classic deadlock or deadly-embrace occurs when one thread holds a resource (RA) and needs another resource (RB) while another thread holds the second resource (RB) and needs the first resource (RA). If you can understand this, and if you can boil the problem down to this simple formula, you can begin solving the problem with basic locking principles. As you advance in multi-threaded programming, you'll learn more and more of the locking patterns and how to apply them. But, in the mean time, if you boil everything down to simply "locking a resource", some basic guidelines will help you correct your logic or design.

Basic guidelines for locking.

Avoid simultaneously locking multiple shared resources in a single thread.

If multiple shared resources must be locked simultaneously, endeavor that all threads lock the resources in the same order and unlock them in reverse order.

If they can't be unlocked in reverse order, then endeavor to unlock in the same order and do not lock an earlier resource till all later locks are released.

If you can't follow these guidelines, then timeout.

There's more to be said about these basic locking principles, and I'd like to touch on the touchy subject of knowing one's resources. But, I'll save that for the next post and instead leave you with some code that breaks the deadlock and hopefully provokes some more thought about the often overlooked event loop "resource". Maybe you can tell why this breaks the deadlock based upon the RA/RB discussion.

Here's the code.


using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;
// Now let's see how we can avoid locking ourselves up
//
namespace LockupExample
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            _t = new Thread(new ParameterizedThreadStart(WorkerThread));
        }

        private Thread _t;
        private object lockObject = new object();
        private bool StopThread = false;  // to let the thread quit gracefully
        private void WorkerThread(object sender)
        {
            Thread.Sleep(1000);
            while (!StopThread)
            {
                string result = "This is a Test";
                IAsyncResult aResult;

                lock (lockObject)
                {
                    Thread.Sleep(25);
                    aResult = lblResult.BeginInvoke(new MethodInvoker(delegate { lblResult.Text = result; }));
                }
                lblResult.EndInvoke(aResult);
                Thread.Sleep(500);
            }
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            _t.Start();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            lock (lockObject)
            {
                lblResult.Text = "Override the Test";
            }
        }

        private void Form1_FormClosing(object sender, FormClosingEventArgs e)
        {
            StopThread = true;
        }
    }
}