// Wouldn't it be great
// if we could do something
// like this?

Memo1.OnKeyPress :=
  procedure(Sender: TObject; var Key: Char)
  begin
    Key := Chr(Ord(Key) + 1);
  end;

One thing I wasn’t aware of when I wrote that previous post is that while method pointers can’t be used to store anonymous methods, the opposite is true: Method references can indeed store both anonymous methods and instance methods. Many thanks to Barry Kelly who pointed it out to me in a comment.

Could this be the way to go? Shall we all start using method references instead of method pointers when we declare the events of our components from now on? Let’s take a look shall we? But just so that we’re clear with what we mean – there are many ways to say the same thing - here are the definitions I’m using in this post.

First of all, when I say method I mean either a function or a procedure, and it can be any of the types below. This is different from the common definition stating that methods are subroutines associated with either a class or an object.

Plain methods
These are functions and procedures declared outside the context of a class.

procedure PlainProcedure;
begin
  // Has no Self pointer
end;

Instance methods
Functions or procedures associated to an object instance

Procedure TSomeClass.InstanceProcedure;
begin
  // Self is an instance of TSomeClass
end;

Class methods
Functions or procedures associated to a class. In other languages also called static methods.

class procedure TSomeClass.ClassProcedure;
begin
  // Self is TSomeClass
end;

Anonymous methods
Functions or procedures declared as they are assigned in an execution context. These methods capture the surrounding context and may use local variables and arguments even if they’re out of execution scope.

SomeItems.Sort(
  function(Item1, Item2: TSomeItem): Integer;
  begin
    if Item1.Value > Item2.Value then
      Result := 1
    else if Item1.Value < Item2.Value then
      Result :=  -1
    else
      Result := 0;
    end
  );

For these four types of methods there are three reference types to store pointers for later invocation of the methods.

Plain method pointer

type TPlainMethodPointer =
  procedure(ASender: TObject);

Method pointers

type TMethodPointer =
  procedure(ASender: TObject) of object;

Method references

type TMethodReference =
  reference to procedure(ASender: TObject);

This compatibility graph shows what method types can be stored with the respective reference type.

From this graph we can see that the Method reference type is the unifying type that can store all types of method references. It seems like the way to go is to embrace Method references and render the other reference types obsolete. Is it possible? The answer is yes but no.

Take the event example of my last post. The VCL can not be changed to utilize Method pointers for backward compatibility reasons (like the widespread use of TMethod casts) so there’s nothing to do about it there. But what about our own components, and components created from now on?

type
  TNotifyMethod = reference to procedure(Sender: TObject);

  TMyComponent = class(TComponent)
    ...
  published
    property OnChange: TNotifyMethod read FOnChange write FOnChange;
  end;

Well, it works fine as long as you set up the event handlers (OnChange in the above example) dynamically, but the method reference type events do not play well with the Delphi 2009 IDE. The object inspector doesn’t recognize possible event handlers and won’t create one automatically I you double click the event property. If you try to force the assignment by giving the name of an existing event handler method explicitly, the IDE throws an ugly Invalid Property Error Dialog.

As much as I’d like to be able to assign anonymous methods as event handlers to my components’, I’m not prepared to sacrifice the IDE integration. Hopefully CodeGear will fix this issue in future releases but until then anonymous methods will remain less useful than they could be.

Cheers!

I am still not convinced that this convenience is worth getting all that excited about - It has not enabled anything that was previously impossible, nor even especially difficult either to implement or to understand

[...]

anonymous methods should be used only when you absolutely have to, and so far I have not yet seen an example where anonymous methods are the only way to achieve anything

I agree that more often than not a problem can be solved with equal elegance using a pure object orientated approach, but there are situations where anonymous methods may actually be the better alternative. One situation that comes to mind is the setting up of test fixtures.
Say, for instance, that we want to test the event firing mechanism of a component. An easy way to set this up could be like the following.

procedure TestSomeComponent.TestOnChange;
var
  SUT: TSomeComponent;
  OnChangeFired: Boolean;
begin
  OnChangeFired := False;
  // Set up the fixture
  SUT := CreateTheComponentToTest;
  SUT.OnChange :=
    procedure(ASender: TObject)
    begin
      OnChangeFired := True;
      CheckSame(SUT, ASender, 'The Sender event argument should be set');
    end;
  // Perform operation
  SUT.Text := 'Some text';
  // Check the result
  CheckTrue(OnChangeFired, 'Setting the Text property should fire the OnChange event');
end;

The above code checks that the component’s OnChange event is fired when the Text property is set, and that the component is set as the Sender reference to the event handler. Except for being more compact, the biggest advantage to using anonymous methods in this scenario is that we avoid the obscure test smell and that we don’t have to clutter our test class with an instance variable (like FOnChangeFired) to identify if the event was really fired or not.

The only problem is: it doesn’t work. Why? Well, because the OnChange property is most likely of an instance method reference type (i.e. TNotifyEvent), meaning it doesn’t accept a references to an anonymous method even if it has the same signature.

type TNotifyEvent = procedure(ASender: TObject) of object;

For our code to compile we need to redeclare TNotifyEvent and remove the “of object” keywords and instead use the method reference type.

type TNotifyEvent = reference to procedure(ASender: TObject);

But of course, that’s not an option. It would mean that the traditional way of setting up an event handler (by using an instance method) would not work.

I see a definite problem with how the Delphi language explicitly forces you to distinguish between instance methods (and class methods for that matter) and anonymous method references, even though they share the same signature.
This is most unfortunate since I feel that the situations where we’d like to support both kinds of event handlers are quite common. And with the current semantics we have to use code duplication in order to achieve that. Like the two Synchronize methods of the built in TThread class.

In Delphi 2009 an overloaded variant of the TThread.Synchronize method was introduced, one that make use of anonymous methods. Here are snippets from that code:

type
  TThreadMethod = procedure of object;
  TThreadProcedure = reference to procedure;
...
  class TThread = class
    …
    procedure Synchronize(AMethod: TThreadMethod); overload;
    procedure Synchronize(AThreadProc: TThreadProcedure); overload;
    ...
  end;

The two methods have almost identical implementations. The only real difference is the type of the argument.

procedure TThread.Synchronize(AMethod: TThreadMethod);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := AMethod;
  FSynchronize.FProcedure := nil;
  Synchronize(@FSynchronize);
end;

procedure TThread.Synchronize(AThreadProc: TThreadProcedure);
begin
  FSynchronize.FThread := Self;
  FSynchronize.FSynchronizeException := nil;
  FSynchronize.FMethod := nil;
  FSynchronize.FProcedure := AThreadProc;
  Synchronize(@FSynchronize);
end;

I may be overly sensitive, but code like that really disturbs me. Unfortunately it gets worse. If we follow the execution chain down into the Synchronize class method that is invoked by the two, we find this.

class procedure TThread.Synchronize(ASyncRec: PSynchronizeRecord; QueueEvent: Boolean = False);
var
  SyncProc: TSyncProc;
  SyncProcPtr: PSyncProc;
begin
  if GetCurrentThreadID = MainThreadID then
  begin
    if Assigned(ASyncRec.FMethod) then
      ASyncRec.FMethod()
    else if Assigned(ASyncRec.FProcedure) then
      ASyncRec.FProcedure();
    end else
      …
end;

It would be a lot nicer if the two reference types were joined under a common reference type. And I can’t see why it couldn’t be done. When I look at the “of object” keywords I get a feeling that the language is leaking compiler implementation through the language interface; information that is indifferent to the developer. What matters from a callers’ perspective is the method signature, not whether the method has a self pointer or not.

I hope CodeGear recognizes this problem and find a way to clean this assymetry from the language. Anonymous methods would be so much more useful if they do.

Cheers!

A sort of homecoming

Firing up the new IDE from CodeGear with Delphi for Win32 felt a lot like coming home. The outstanding graphical editor was there as was the lightning fast compiler. Visual Studio is great in many ways but it can’t compete with CodeGear’s RAD Studio when it comes to speed and rapid application development.

Not only that, the language (Object Pascal) has finally woken from its stagnated state and is getting increasingly modern. It now implements, for instance, closures (anonymous methods) which is quite rare for a non garbage collected language.

type
  TClosureProc = reference to procedure(AMsg: string);

function CreatePrefixedWriter(APrefix: string): TClosureProc;
begin
  Result := procedure(AMsg: string)
  begin
    WriteLn(APrefix + AMsg);
  end;
end;

var
  Log: TClosureProc;

begin
  Log := CreatePrefixedWriter('Closure Test: ');
  Log('Write this');
  Log('And this');
end.

The above program should produce the following output.

Closure Test: Write this
Closure Test: And this


I gotta wear shades

I am not the only one being thrilled with the rebirth of Delphi. A look at the TIOBE Programming Community Index for February shows that Delphi is the 9th most popular programming language, with a steady increase. That is a lot better than the 15th position a year and a half ago. At that time Delphi felt like a marginalized language with little promises for the future; Now it feels alive and vibrant.

Since the magic is back, I think I’ll stick around for a while.

Cheers

Anyway, HTML-Kit was supposedly an excellent HTML editor and at a first glance it seemed good enough. That was until I tried to hit the ” (double quotation marks) key. Instead of inserting the expected character into the html document I was currently editing, HTML-Kit threw me a new MDI window containing a fresh HTML document. As if I’d pressed Ctrl-N(ew) or something.

What was going on? Well, it turned out that the default shortcuts of HTML-Kit was interfering with my custom keyboard layout. (This has happened to me with other editors as well.) My layout takes advantage of the Alt Gr button to move common special characters, like the ” character, from the top row to the base row, all for the sake of speed and ergonomy.

On my keyboard, to strike the ” key I have to press Alt Gr + L, which is actually Alt Gr + N since I base my layout on Dvorak, and since pressing Alt Gr is the same as pressing Alt + Ctrl this combination was coming in the way of the built in Alt + Ctrl + N, which means New HTML document i HTML-Kit. (Do you follow?)

The fact that HTML-Kit didn’t allow me to type a common HTML character made the editor seemingly useless. But, that happened to not be the case. HTML-Kit has a great feature that allows you to define your own shortcuts. So, I did just that: define a new Alt + Ctrl + N to just output my double-quote. (I had to do the same for right brackets, } )

I’d much prefer it if application creators wouldn’t define Alt + Ctrl shortcuts at all, for the reasons given above, but if they absolutely have to, HTML-Kit’s level of customizability is the next best thing.

The future of programming will be multicore, multithreaded. Languages that make it easy to program them will supplant languages that don’t. Transitive const is key to bringing D into this paradigm. [...] C++ cannot be retrofitted to supporting multiprogramming in a manner that makes it accessible. D isn’t there yet, but it will be, and transitive const will be absolutely fundamental to making it work.

[Quote from the D website]

What is transitive const?
Just a quick explanation for those of us who doesn’t have academic terms in close memory. Transitivity is a property of some binary relations, for example equality:

if A = B and B = C, then A = C

Applied to the concept of const it means, simply put, that anything reachable from a const type is also a const. So, for a declaration const int **p, p is const, as well as *p and **p.
The same is true in the case of composite types:

class A {
  int f;

  void set_f(int a_value) {
    f = a_value;
  }
}

const A a = new A();
a = new A(); // error
a.a = 2; // error
a.set_a(2); // error

All three reassignments above result in compiler errors due to the fact that a is const, and anything reachable from it is also const.

Why does it matter?
So, in what way is transitive const fundamental to concurrent programming? Well, it isn’t. What Walter Bright and his companions refer to is the fact that pure functional programming is thread-safe by design. That is, in a pure functional language the result of a function is solely dependent on its arguments. Thus, in a code like this:

val = some_func( a(), b(), c() );

functions a(), b() and c() can be safely executed concurrently in a multi-core architecture; Nothing a(), b() or c() does can affect each others results. This is not the case for imperative languages that builds on the notion of mutable and global state. With mutable state comes hidden side-effects (a(), b() or c() could change common data and cause raise conditions) that complicates multi-thread programming so much.

What the people behind D tries to do is to create a pure functional subset within the language. I like to refer to it as Functional D. Such a subset would allow us to write code that is thread-safe by default, all you have to do is to write Functional D code. The compiler would then be able to chisel out the functional code and fully utilize the advantages of functional programming.

Immutable data and pure functions are fundamental
To make this work we need a way to make data immutable and a way to shut down access to the global state. In D you use the invariant keyword to create immutable data. The pure keyword is used to mark functions that may only take invariant arguments, no access to the global state, and that may only invoke other pure functions. (As of this writing, the semantics of the pure keyword is not yet implemented).

int g = 0;

pure int pure_func(invariant int a) {
  a = 0; // error, a is invariant
  g = 1; // error, can't write to global g
  writefln(a); // error, writefln is not pure
  return g + a; // error, can't read global g
}

How does transitive const fit into all of this? To use the intuitive definitions from Andrei Alexandrescu’s slides on the functional subset of D:

• const(T) x: I can’t modify x or anything reachable from it
• invariant(T) x: Nobody can modify x or anything reachable from it

Const is not strong enough to be used in the functional subset (which depends on truly immutable data), but it has one application that could be important. From Walter Bright at the D newsgroup:

Const allows you to reuse code that works on invariants to work with
mutables, too.

How usable is const to the functional subset?
Const can be used to write code that works with data from both the imperative (mutable) and the functional (immutable) subsets. For example, the print function below.

void print(const int a) {
  writefln(a);
}

const int a = 1;
print(a); // ok

invariant int b = 2;
print(b); // ok

print(3); // ok

The reason this works is that invariants and immutable data is implicitly converted to const when necessary. One can question how useful this would be in practice though, the print function above would not be invokable from the functional subset (which would require it to be pure).

My conclusion is that although it may very well be important, transitive const is not “absolutely fundamental to making it work.” Transitive invariant, on the other hand, is.

Cheers!

Time after time I see requirements gathered and presented in a way that is totally disconnected from the business problem that’s being addressed. There are two ways to handle the situation.

1. Write the best code you can.
2. Talk to the business.

Hackers generally go for the first choice, which doesn’t guarantee failure. In fact, a good hacker can still deliver code quickly enough that the business will be happy. Often, he gets it right on the 2nd or 3rd try.

But, a developer can deliver what the business is looking for the first time. A(n often) quick conversation with the business ensures that the developer knows what to work on and how it will benefit the business. This dialog often leads to a superior solution implementation where the business gets what it wants and the developer is able to do it in the most efficient way (e.g. I don’t need a beautiful website, I need a command line application).

It’s essentially the same distinction I tried to make a while ago, although I used the term Programmer instead of Hacker. Maybe if I’d used the term Hacker as well, I might have avoided some of the heat I took for those definitions

Interestingly Jay too suggests — although he doesn’t say it outright — a definition where human interaction and taking the Customer’s view is what distinguishes a developer, rather than coding skills.

Cheers!

As you may know I keep personal logs for almost everything, including notes on the books I’ve read. Now I decided to take that habit and make it public, something I’ve been meaning to do for a very long time. For that reason I’ve set up a new sub domain, reviews.hans-eric.com, where I’ll publish such notes. It won’t be fully fledged reviews, but short posts that primarily reflects how I found the book (or the movie, or whatever.)

Here are four of the books I read during my holiday:

• Agile Software Development with Scrum
• The Art of Happiness
• My Job Went to India
• Maya Cosmogenesis 2012

If you’re interested in following what books I read (and what movies I watch,) subscribe to my reviews feed.

Cheers!

Somehow the ModRewrite rules got screwed up, no idea how. Anyway, it has been taken care of now and the site should be up and running as usual. Many thank’s to Robert Ames who pointed it out for me.

Now, back to vacationing (I still have a week left).

Cheers!

Let’s take a closer look at that functional subset that is taking form in the experimental 2.0 version of D.

Immutable data

The most fundamental difference between a purely functional language and an imperative one is how they treat data. Many of us are used to think of data in terms of state, where variables can be changed through assignments. But in a functional program there are no states. There are only constant values and functions that operate on them.

In D, immutability is achieved with the invariant keyword, either as a storage class:

invariant int i = 3;

or as a type constructor:

invariant(int) i = 3;

Transitive invariance

The side-effect free nature that comes with immutable data has some great advantages. For one thing it simplifies testing since the result of a function only depends on its input. There are also some optimizations that can be done by the compiler, but the biggest advantage is that programs written in this way are thread-safe by default.

To take advantage of these things the compiler needs to be able to trust the immutability of our data. This is where transitivity comes in. In D, invariance is transitive, which basically means that no data reachable from an invariant reference can be changed. Here’s an example.

int ga = 2; // mutable

struct A {
  int *p = &ga; // pointer to mutable
}

invariant(A) a; // a is immutable
A b; // b is mutable

// invariant is transitive
a = b; // ERROR, a is immutable
a.p = null; // ERROR, a.p is immutable
*a.p = 3; // ERROR, data pointed to by a.p is also immutable

Garbage collection

Since data must never change in a functional program, and consequently must not be destroyed while the data is in use, it’s usually a good idea for a functional language to utilize automatic memory management. Like most functional languages, D features garbage collection (alongside with explicit memory management.)

Higher class functions

In order to do anything interesting in a purely functional language you need higher order functions, or - in other words - the ability to send functions as arguments to other functions. For this we can use the function pointers (or delegates for methods and nested functions).

As an example, let’s say that we want to create a function that calculates the sum of two adjacent Fibonacci numbers. Here’s one way to do that.

int nth_fib(int n) {
  if(n == 0) return 0;
  if(n == 1) return 1;
  return nth_fib(n-1) + nth_fib(n-2);
}

int add_next_fib(int n) {
  return nth_fib(n) + nth_fib(n+1);
}

Now, let’s say that we want to do the same operation on a different sequence, for example natural numbers. Well, we could use the good old copy and paste but that isn’t very DRY. Let’s make add_next a higher order function instead so that it could be used with any sequence function.

int add_next(int n, int function(int) nth) {
  return nth(n) + nth(n+1);
}

int i = add_next(3, &nth_fib);
// i is 8 (3+5)

Now, we can write any sequence function we want and have add_next apply it.

// Sequence function for natural numbers
int nth_nat(int n) {
  return n;
}

int i = add_next(3, &nth_nat);
// i is 7 (3+4)

Note: For methods and nested functions, the keyword function is replaced with the keyword delegate, otherwise it’s the same syntax.

Closures

Closures is another indispensable feature of functional languages. In short, it’s the ability to extract a function pointer for later use, and when invoked the function will still have access to the context in which it was created, even though that context has gone out of scope.

In D, closures are created with the delegate keyword.

int delegate() create_closure() {
  int x = 3;

  int f() {
    return x;
  }

  return &f;
}

int delegate() a_closure = create_closure();
int i = a_closure();
// i is 3

Note that the extracted function f (referenced by the a_closure variable) accesses the local variable x, although it has gone out of scope at the time of execution. D got this ability with the 2.07 version, before that it didn’t have real closures.

Currying

Closures provide an easy way to do currying, which is common in functional languages. Simply put, currying is a technique where functions take a general function and return a new, more specialized one.

For instance, we could curry our add_next function in our previous example and create a specialized version of it, say add_next_fib (and thus get back to where we started).

int delegate(int) curry_add_next(int function(int) nth) {
  int curry_f(int n) {
    return add_next(n, nth);
  }
  return &curry_f;
}

int delegate(int) add_next_fib = curry_add_next(&nth_fib);
int i = add_next_fib(5);
// i is still 8

Pure functions

These features are all we need to write purely functional code, but in order to take full advantage of the functional programming paradigm some major things remain unsolved.

For one thing, the compiler needs to know whether or not our code is functional in order to apply possible optimizations. The easiest way to do this is to give the programmer a keyword to tell the compiler she wishes purity, and then have the compiler enforce it. In D this is the purpose of the pure storage class.

pure int a_pure_square_function(invariant(int) x) {
  return x * x;
}

A pure function must not access non-invariant data, and may not invoke other non-pure functions. As per D2.13, the pure storage class has not had its semantics implemented and are therefore not enforcing purity. I sense this is not a trivial matter, so it may take some time before we have it.

Cheers!

Update: Added a link and rewrote the section on garbage collection.

Well, there are plenty of good reasons for doing Open-Source pro bono work; it’s a great way to get experience and widen perspectives for instance, but still, Alan has a point. Many of us are not as content as we could be with our regular work. Instead we’re seeking satisfaction elsewhere.

So, what should our employers do to get our full attention? Here’s my list.

• Creative freedom
  Give me a chance to contribute, to be innovative and creative. Let me spend a part of my time doing research and follow paths that interest and inspires me. Google is a great example of a company that understands the importance of this.
• Personal Development
  As Ron Jeffries has said, “the river is moving, and if we don’t keep rowing, we are going to drift back downstream.” I think self improvement is a spice of life. If my company provides me with all the books I need (and want), and lets me attend courses and conferences of my choice, chances are I’ll stay with it for life.
• Ownership
  People usually do a better job, are more careful and thorough, if they own the thing they’re working on. This is true for software as well. Make me a business partner and I’ll optimize my work according to your business goals.
• Appreciation
  The human race seems to be immensely better at criticizing than at giving appreciation. Yet, this is what we all crave, and - it has a great impact on how we see our employers. A rewarding salary is one form of showing appreciation, but I also need the outspoken forms.
• Closures
  No one can go on for ever without reaching a finish line. I want to work in a team that is long-term efficient and gets to get done often. Help me divide and I’ll conquer for you.

That was my list, what’s on yours?

Cheers!