Stephen Cleary (the blog)

C# Advent: No one loves the .NET Core name anymore

Tue, 03 Dec 2024 00:00:00 +0000

This post is part of C# Advent organized by @mgroves.

A Brief History of “.NET Core”

‘Twas a holiday season, so bright and so clear,
When .NET Core was a name we held dear.
Once, it meant Silverlight, in the browser’s embrace,
A vision of apps that could run with great grace.

But as time moved on, so too did the trend,
Silverlight’s reign began to descend.
Then WinRT arrived, all shiny and new,
With Windows Store apps, the promise it grew.

Next came UWP, a universal dream,
For apps that could run on each Windows stream.
Yet, despite all the progress, something was amiss,
The tech world was changing, and something we’d miss.

Then .NET Core rose, its name now aglow,
A shift to the future, a new path to go.
It wasn’t just Windows, it reached far and wide,
Cross-platform magic, no longer confined.

Gone were the days of Silverlight’s reign,
No more WinRT to tether and chain.
UWP too, was a chapter now closed,
But the legacy of .NET still brightly glowed.

Now .NET Core means something profound,
A modern framework that’s world-renowned.
It’s cloud, it’s mobile, it’s web all combined,
A platform so nimble, for all to find.

So here’s to the future, to the journey ahead,
With .NET Core, new dreams are now spread.
We celebrate Christmas, not just for the cheer,
But for how far .NET has come through the years!

ChatGPT, of course. I couldn't write this.

I thought today would be a fun time to write up something I’ve been meaning to write for years: the history of “.NET Core” terminology.

More specifically, this is how I’ve seen the term “.NET Core” used in the past. It’s not guaranteed to be correct, and certainly not guaranteed to be complete! Am I missing anything? Feel free to shout out in the comments!

.NET Compact Framework

I believe I first heard the term “.NET Core” referring to the .NET Compact Framework. This was way back when .NET Framework 3.5 was the new hotness. Actually, that was “.NET Framework 3.5 SP1” - as in “Service Pack 1”. Yes, .NET used to have service packs.

I’m not sure if “.NET Core” was the proper term to refer to NetCF or not. AFAIK, it was an unofficial term that was applied because .NET Compact Framework contained a small subset of the (massive) .NET Framework, intended to run on embedded devices. Hence, it was a “core” (minimal) framework.

Silverlight, WinRT, Windows Phone, Windows Store, and Universal Windows Platform

Perhaps I should have put a trigger warning on this article… Sorry for anyone who winced in pain just reading the title for this section…

All of these technologies had something or other to do with “.NET Core” as a term. They were all minimal sub-frameworks of the full .NET Framework. And at least some of them had similar terminology for their runtimes (e.g., Silverlight ran on “Core CLR”). My brain has mercifully purged the sordid details. I only vaguely remember Windows Phone being different from Windows Phone Apps, and Silverlight getting even more stripped-down to run on Windows Phone.

This was not a fun time to be a library developer.

netcore as a Target Framework Moniker

This is what I actually want to focus on. Target framework monikers (TFMs) are very important to library developers.

The term “.NET Core” officially entered the NuGet space as a TFM, but it wasn’t for what we call .NET Core today. It was for Windows Store apps (and WinRT apps), and briefly extending into UWP apps.

Yes, Windows Store projects way back in Windows 8.0 (as well as WinRT) used a TFM identifier of netcore45. Windows 8.1 used netcore451. When Windows Store apps were replaced by Universal Windows Platform (UWP) apps, they briefly used netcore50 before changing to uap10.0.

So, when the (modern) .NET Core team needed a TFM, they found netcoreNN wasn’t available. I assume this is why they went with netcoreappNN instead (after a brief foray into dnxcore50 and aspnetcore50). So .NET Core TFMs for older versions (1.0-3.1) are netcoreapp1.0 - netcoreapp3.1. It was just an unfortunate accident of history.

No More .NET Core

The .NET team has dropped “Core”. There is now one .NET moving forward - a great relief to library authors!

Since v5, the TFMs have even changed: modern .NET uses net9.0 and similar, dropping the “core” moniker even from the TFM.

I still refer to .NET as “.NET Core” from time to time, especially in conversations that have to do with both .NET Framework and the new .NET. E.g., I’ll refer to a “.NET Core migration” rather than just “.NET migration” for a project currently targeting .NET Framework. The proper term, though, is no longer “.NET Core” but just “.NET”.

“.NET Core” is gone. And good riddance!

May the future be One .NET, and I wish you all joy in this Christmas season!

Cancellation, Part 6: Linking

Thu, 10 Oct 2024 00:00:00 +0000

So far we’ve covered how cancellation is requested by one piece of code, and responded to by another piece of code. The requesting code has a standard way of requesting cancellation, as well as a standard way of detecting whether the code was canceled or not. Meanwhile, the responding code can observe cancellation either by polling or (more commonly) by registering a cancellation callback. So far, so good; and we’re ready for the next step!

In this article, we’ll look at how linked cancellation tokens work.

Linked Cancellation Tokens

Linked cancellation tokens allow your code to create a CancellationTokenSource that cancels whenever any other cancellation tokens are canceled, in addition to a manual cancellation request.

The following code creates a linked cancellation token source:

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
    var task = DoSomethingElseAsync(cts.Token);
    ... // Do something while operation is in progress, possibly calling `cts.Cancel()`
    await task;
}

The DoSomethingAsync method above takes a cancellationToken - I’ll call this the “outer” cancellation token. It then creates a CTS that is linked to that outer token. Then, when it calls DoSomethingElseAsync, it passes the token from that linked CTS, which I’ll call the “inner” cancellation token.

If the outer cancellation token (cancellationToken) is ever canceled, then the linked CTS (cts) and its inner cancellation token (cts.Token) are also canceled. Furthermore, the DoSomethingAsync method has the option of explicitly cancelling the linked CTS - in this case, only the inner cancellation token would be canceled, leaving the outer cancellation token unchanged.

Sharp observers may have noticed that the same thing can be done using registrations:

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var cts = new CancellationTokenSource();
    using var registration = cancellationToken.Register(cts.Cancel);
    var task = DoSomethingElseAsync(cts.Token);
    ... // Do something while operation is in progress, possibly calling `cts.Cancel()`
    await task;
}

Indeed, logically this is pretty much what is happening: you can think of a linked cancellation token source as a perfectly ordinary cancellation token source along with a registration that cancels it when some other token is canceled.

Multiple Links

The inner cancellation token above is canceled when the outer token is canceled or when its source is explicitly canceled. Similarly, we can pass any number of cancellation tokens into CreateLinkedTokenSource, and the cancellation token source it returns will be canceled when any of the outer tokens are canceled.

Use Cases

The outer token and the inner cancellation source can really represent anything; linked cancellation tokens are useful whenever you need code to be canceled if “A or B”.

But I suspect the most common use case is when the outer token represents an end-user cancellation request, and the inner token represents a timeout. E.g., this can happen when the business logic includes a timeout-and-retry kind of code pattern, while also allowing the end-user to cancel all the retries with a single button click.

One natural place where this kind of code is used is Polly. Polly will allow you to pass in a cancellation token - an outer token that is under your control. Then it passes a potentially different cancellation token to your execution delegate; this inner token is controlled by Polly. Polly’s pipelines (e.g., timeout) may trigger the inner token to cancel your delegate. Naturally, if your code cancels the outer token passed to Polly, that would flow to the inner token as well. I.e., they are linked.

Taking a simplified code example right from the Polly homepage:

async Task ExecuteWithTimeoutAsync(CancellationToken cancellationToken)
{
    ResiliencePipeline pipeline = new ResiliencePipelineBuilder()
        .AddTimeout(TimeSpan.FromSeconds(10))
        .Build();

    await pipeline.ExecuteAsync(async token =>
    {
        /* Your custom logic goes here */
    }, cancellationToken);
}

The ExecuteWithRetryAndTimeoutAsync takes an outer token cancellationToken and passes it to Polly. Polly then creates a linked inner token (which includes pipeline behaviors such as the timeout), and passes the inner token (token) to your delegate.

Delegates you pass to Polly should observe the token they get from Polly, not any other tokens!

This is particularly a pitfall when you’re adding Polly pipelines to existing code, e.g., when adding timeouts to this code:

async Task ExecuteAsync(CancellationToken cancellationToken)
{
    for (int i = 0; i != 10; ++i)
        await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
}

A common mistake is to forget to update the token usage:

// BAD CODE!!! DO NOT USE!!!
async Task ExecuteWithTimeoutAsync(CancellationToken cancellationToken)
{
    ResiliencePipeline pipeline = new ResiliencePipelineBuilder()
        .AddTimeout(TimeSpan.FromSeconds(10))
        .Build();

    await pipeline.ExecuteAsync(async token =>
    {
        // BAD CODE!!! DO NOT USE!!!
        for (int i = 0; i != 10; ++i)
            await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
    }, cancellationToken);
}

In this case, the delegate is still observing cancellationToken, when it should be observing token instead:

async Task ExecuteWithTimeoutAsync(CancellationToken cancellationToken)
{
    ResiliencePipeline pipeline = new ResiliencePipelineBuilder()
        .AddTimeout(TimeSpan.FromSeconds(10))
        .Build();

    await pipeline.ExecuteAsync(async token =>
    {
        for (int i = 0; i != 10; ++i)
            await Task.Delay(TimeSpan.FromSeconds(1), token);
    }, cancellationToken);
}

Sharp Corner: Don’t use OperationCanceledException.CancellationToken

Consider the original example code again:

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
    var task = DoSomethingElseAsync(cts.Token);
    ... // Do something while operation is in progress, possibly calling `cts.Cancel()`
    await task;
}

Now consider some code that may call DoSomethingAsync and respond to cancellation:

// BAD CODE!!! DO NOT USE!!!
async Task MainAsync()
{
    using var cts = new CancellationTokenSource();
    cts.CancelAfter(TimeSpan.FromSeconds(2000));

    try
    {
        await DoSomethingAsync(cts.Token);
    }
    catch (OperationCanceledException ex) when (ex.CancellationToken == cts.Token) // BAD CODE!!! DO NOT USE!!!
    {
        Console.WriteLine("Timeout!");
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
    }
}

The intent of the handling code is to do something different if the code is canceled due to this particular cancellation source. Unfortunately, this code is problematic in the real world; DoSomethingAsync may be using a linked cancellation token source, in which case the OperationCanceledException.CancellationToken would not match cts.Token, even if that was the source of the cancellation!

This is why I always recommend not using OperationCanceledException.CancellationToken. A proper solution is to check whether that source has been triggered:

async Task MainAsync()
{
    using var cts = new CancellationTokenSource();
    cts.CancelAfter(TimeSpan.FromSeconds(2000));

    try
    {
        await DoSomethingAsync(cts.Token);
    }
    catch (OperationCanceledException) when (cts.IsCancellationRequested)
    {
        Console.WriteLine("Timeout!");
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex);
    }
}

Checking Inner Tokens: Still don’t use OperationCanceledException.CancellationToken

You might be tempted to do this kind of test when using linked cancellation tokens, again to determine what the cancellation source is:

// BAD CODE!!! DO NOT USE!!!
async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
    cts.CancelAfter(TimeSpan.FromSeconds(10));

    try
    {
        await DoSomethingElseAsync(cts.Token);
    }
    catch (OperationCanceledException ex) when (ex.CancellationToken == cts.Token) // BAD CODE!!! DO NOT USE!!!
    {
        ... // Do some recovery specific to the timeout.
        throw;
    }
}

However, this code has the same issue! It’s possible that DoSomethingElseAsync may itself use a linked cancellation token (or may be changed to use one in the future)!

The solution - again - is to not use OperationCanceledException.CancellationToken:

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
    cts.CancelAfter(TimeSpan.FromSeconds(10));

    try
    {
        await DoSomethingElseAsync(cts.Token);
    }
    catch (OperationCanceledException ex) when (cts.IsCancellationRequested)
    {
        ... // Do some recovery specific to the timeout.
        throw;
    }
}

Summary

Most of the time you won’t need to use linked cancellation tokens in your code, but linked cancellation tokens are useful when you need them! Some points to remember:

Dispose your cancellation token sources - including linked cancellation token sources.
Don’t use OperationCanceledException.CancellationToken; use IsCancellationRequested instead.
For any code that has multiple tokens in scope, be mindful about which one you are observing.

Cancellation, Part 5: Registration

Thu, 08 Aug 2024 00:00:00 +0000

Last time in this series I talked about how to respond to cancellation requests by polling for them. That’s a common approach for synchronous or CPU-bound code. In this post, I’m covering a pattern more common for asynchronous code: registration.

Registration is a way for your code to get a callback immediately when cancellation is requested. This callback can then perform some operation (often calling a different API) to cancel the asynchronous operation. Due to the multiple ways callbacks may be invoked, it’s generally recommended that cancellation callbacks should not throw exceptions.

Your callback should not throw exceptions.

How to Register

Your code can register a callback with any CancellationToken by calling CancellationToken.Register. This callback is invoked when (if) the cancellation token is cancelled. The Register method returns a cancellation token registration, which is essentially just an IDisposable.

Pretty much every asynchronous API already has cancellation support, so the example code below is somewhat contrived. The API in this example code provides a StartSomething() method to start some asynchronous operation, a StopSomething() method to cancel that operation, and a SomethingCompletedTask property to detect when the operation completed. There are very few APIs like this in the .NET ecosystem, but you may run into a design similar to this if you’re wrapping code from another language that doesn’t have a promise-based async system.

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var registration = cancellationToken.Register(() => StopSomething());
    StartSomething();
    await SomethingCompletedTask;
}

Note that callbacks might never be invoked! Tasks always complete (that is, they always should complete), but that doesn’t hold for cancellation tokens. There are some cancellation tokens that will never be cancelled, so they will never invoke their callbacks.

Your callback might never be called.

A Race Condition

What happens if a cancellation token is cancelled at approximately the same time the callback is registered? This situation is properly handled by a simple rule: if a callback is ever added to a cancellation token that is already canceled, then it is immediately and synchronously invoked.

Your callback might be called immediately on the same thread before Register returns.

Cleanup Is Important!

The lifetime of cancellation tokens can vary greatly. Some cancellation tokens are used for short, individual operations. Other cancellation tokens are used for application shutdown. When writing your cancelable code, ensure that your code disposes of the registration; this will prevent resource leaks in your application.

The using var registration in the example code above (repeated below) is one common way of handling cleanup: the registration is disposed once the asynchronous work completes.

async Task DoSomethingAsync(CancellationToken cancellationToken)
{
    using var registration = cancellationToken.Register(() => StopSomething());
    StartSomething();
    await SomethingCompletedTask;
} // The registration is cleaned up here if the operation completed without being cancelled.

Be sure to dispose your cancellation token registrations!

Sharp Corner: Synchronous Cancellation Callbacks

As discussed in requesting cancellation, token sources may be cancelled by calling the Cancel method. It’s important to note that any registered callbacks are immediately (and synchronously) run by the Cancel method before it returns. This can be a source of deadlocks or other unexpected behavior if your code is written assuming that callbacks are invoked after the Cancel method. Specifically, callbacks shouldn’t perform any blocking operation.

Your callback is invoked synchronously in most cases.

This is awkward often enough that .NET 8.0 added a CancellationTokenSource.CancelAsync method which invokes the cancellation callbacks on a thread pool thread. Technically, it immediately and synchronously transitions the cancellation token source to the canceled state, and then queues the callback invocations on a thread pool thread. The returned task completes when the callbacks have completed.

One more wrinkle, actually: it’s possible that CancelAsync will return before the callbacks have completed if it’s already been called. As soon as the CancellationTokenSource transitions to the canceled state, any future calls to Cancel or CancelAsync will return immediately, even if a previous call to CancelAsync hasn’t finished running its callbacks yet. But you probably don’t need to worry about that; it’s just a side note.

Summary

This post has a bunch of scary warnings, but really, registering callbacks is the natural way to implement cancellation at the lowest levels. Polling is commonly used in sample code because it’s simpler, but registration allows your code to react immediately.

Don’t let the warnings dissuade you from using cancellation registrations! They’re more like guidelines for proper usage:

Your callback should not throw exceptions.
Your callback might never be called.
Your callback might be called immediately on the same thread before Register returns.
Be sure to dispose your cancellation token registrations.
Your callback is invoked synchronously in most cases.

If you follow these guidelines, you should be able to use cancellation token registrations successfully!

ICYMI: Video Series on TCP/IP Application Protocol Design

Thu, 01 Aug 2024 00:00:00 +0000

I speak regularly at various programming conferences, and one of my favorite talks is actually on TCP/IP protocol design. This is a skill that I learned almost by accident (literally because someone had to, and no one else in the company wanted to). It’s always been one of my favorite talks, and I submit it all over the place. Most of the time it’s rejected. Which I get - protocol design is esoteric and just plain useless for most developers. But I think it’s still fun!

A bit ago I decided to take everything I had ever learned about TCP/IP protcols and live-code a server and client. So, all the knowledge that goes into my talk is also in this video series. I chose a basic chat server, since the requirements are simple and easily understood. I also wanted to try out some of the newer .NET APIs. Specifically, the final solution uses:

async and await - naturally!
Socket - the Berkeley-ish API for socket communication.
System.IO.Pipelines - for low-level buffer management.
System.Threading.Channels and IAsyncEnumerable - for asynchronous streams of messages.
A dictionary of TaskCompletionSource instances - to implement higher-level request/response APIs.

Someday I’d like to update this to use QUIC (System.Net.Quic). Let me know if that’s something you’d be interested in!

The full video series (over 16 hours!) is available for free on YouTube, and the code is on GitHub. Enjoy!

MetaPost 2024-04-24

Thu, 25 Apr 2024 00:00:00 +0000

I don’t do meta-posts that often, but this is one. Feel free to skip this if you don’t care! :D

ICYMI (In Case You Missed It)

I’m planning to write several rather small blog posts that will mostly be pointers to other things. I’ve been publishing more GitHub repositories, Gists, and other things (occasional videos and podcasts). It has occurred to me recently that most of my followers are not aware of these; I apologize, and I’ll try to be better about publishing a note here about them.

These blog posts will have an ICYMI prefix. Unless I forget.

On a completely different note, I’ve always had a tolerate-hate relationship with advertisements on my site. Before jumping into the latest change, I’m going to briefly describe some history.

Google Ads: The Early Days

Google Ads have always been on my site. Except recently (more on that below). Google Ads give you a great way to very easily set up ads and also give you a good deal of control on what kinds of ads are on your site. This is very important to me since I have personal objections to advertisements that encourage addictions. Specifically, gambling and pornography ads have never been allowed on my site.

Perhaps non-coincidentally, gambling and pornography ads are the highest paying ad categories (or at least they were last time I checked). I’ve never made a lot of money off my ads, though, and cutting my ad income by a third or half has always been an acceptable tradeoff to me.

Disqus

Back in the day, I used Disqus for comments on my blog. It was good (enough), and the price was right: free. Sooner or later, though, they decided they should actually make some money.

I was first made aware of this by several visitors to my site informing me that my site was showing ads of… let’s say “unsavory content”. Yes, Disqus - without warning - had started showing ads for pornographic sites on my site.

Not only were they showing ads I didn’t want on my site, but they also deliberately hid them from me. It turns out Disqus would not show ads if they could tell you were potentially the owner of your site.

I tried to control the ad categories, but Disqus (at least at that time) did not have any such controls or options. It was either pay them money or put up with their ad choices.

Of course, there was a third option, too. I spent a weekend hacking together my own comment system. Goodbye forever, Disqus.

Google Ads: Now Also Gone

Recently, I removed Google Ads from my site. They’ve been getting more annoying, doing sneaky full-page takeovers if users switch to another tab and then back. I’ve tried to restrain them, but at the end of the day, there’s only so much I can do.

So, I’ve removed Google Ads. I’m not against companies making money, and I’m not against advertising. But I am against annoying ads (Google), or sneaky uncontrollable ads (Disqus).

The Future of My Ads

All that to say this: I’m open to considering sponsor messages on my blog post(s). If you’re interested in sponsoring an existing post, contact me!

C# Advent: The Joy of Immutable Update Patterns

Sun, 24 Dec 2023 00:00:00 +0000

This is my first-ever post that is part of C# Advent organized by @mgroves. This year there’s a video, too, including yours-truly singing while wearing my favorite Christmas shirt!

Joy to the World!

Glory to God in the highest, and on earth peace, good will toward men.

Luke 2:14

I love Christmas! It’s easily my favorite holiday.

In spite of difficulties and upheaval in the world (it is 2023 right now), Christmas still stands as a time of refection and remembrance and expectation.

I do approach Christmas from a Christian perspective, and I enjoy meditating on Jesus’ birth during this time. In particular this year, I’ve been focusing on peace and joy, two words commonly associated with Christmas and the coming of the Christ.

So, when I was considering the topic for my C# Advent article, I particularly wanted one that invoked Peace or Joy. And, when working with C#, there is one aspect of the language that truly does cause feelings of joy whenever I use it. It’s not a single language feature, but rather a collection of language features that all work together in a beatiful way.

Hence the title of this blog post: The Joy of Immutable Update Patterns. Wow, that sounds nerdy…

Immutability

An immutable type is one whose value can’t change. Immutable types have several advantages, not the least of which is that they’re just easier to reason about. Some languages push immutability very strongly; C# takes a relatively pragmatic approach.

Immutability varies across the C# ecosystem (and it’s currently seeing a gradual rise in popularity). Most value types are usually immutable (e.g., int, decimal, and Guid); most reference types are not immutable (e.g., List<int>). However, there are lots of exceptions to that general rule; mutable value types are common in performance-sensitive scenarios, and some reference types such as string are immutable.

Modern code has a few additional options for immutable types: you can use record class (C# 9) for immutable reference types and readonly record struct (C# 10) for immutable value types. There’s also the collections in System.Collections.Immutable for more complex data structures such as stacks, queues, and dictionaries. Of course, with your own immutable types and collections, their members/elements must be immutable in order for the composite value to be immutable.

Updating Immutable Data

Immutable data makes local functions easier to reason about - you know the data can’t change - but of course every program has to model modifications in some way. One approach is to have a variable that is mutable, referring to some data that is immutable. To change the immutable data, you can write code that transitions one immutable value to another.

It is in this area that C# has been slowly adding enhancements over many years, and is now approaching beautiful code. Code that makes me smile when I write it!

`switch` Expressions

switch expressions (C# 8) are at the core of immutable update patterns. At their simplest, they provide a mapping from one constant to another:

static Category Map(AdventThing t) => t switch
{
  AdventThing.Mary => Category.Person,
  AdventThing.Sheep => Category.Animal,
  AdventThing.Camel => Category.Animal,
  AdventThing.Bethlehem => Category.Place,
  _ => throw new ArgumentOutOfRangeException(nameof(t)),
};

switch expressions are built on pattern matching, which started in C# 8 (alongside the switch expression) and have received enhancements in C# 9, C# 10, and C# 11. They’re practically a separate mini-language at this point!

Switch expressions on their own are powerful, but they’re especially useful alongside with expressions.

`with` Expressions

with expressions are a shorthand way of copying a composite value (i.e., a record class or readonly record struct) and changing only the specified properties. A simple example should suffice:

record class Inn(string Name, int RoomsAvailable);
Inn Full(Inn inn) => inn with
{
  RoomsAvailable = 0,
};

Inn myInn = new("Bethlehem Getaway", 50);
Inn fullInn = Full(myInn);

In the example above, Full does not modify the (immutable) inn; it returns a new inn that is full.

Combining switch expressions with with expressions is where you start to see the beauty of this kind of immutable update pattern:

Inn ReserveRoom(Inn inn) => inn switch
{
  { RoomsAvailable: >0 } => inn with { RoomsAvailable = inn.RoomsAvailable - 1 },
  _ => throw new InvalidOperationException("No rooms available."),
};

Here’s a method that decrements the available rooms in an Inn. It is a pure method; it depends only on its inputs and produces only its outputs, with no mutation. This is the point at which we’re starting to do immutable state transitions.

Collections

I tend to use System.Collections.Immutable whenever I need something stack- or queue- or dictionary-like in an immutable context. These types all have methods like Add that return a new collection rather than modifying one in place. Internally, the immutable collections share internal data structures, so this isn’t as inefficient as copying the entire collection; immutable collections can never be as memory-efficient as mutable collections, but they’re usually efficient enough to not be an issue. I find immutable collections satisfy my needs quite well.

However, I would be remiss if I didn’t mention that C# has added a new way to create collections (including ImmutableArray<T>). It’s very reminiscient of JavaScript’s spread operator:

ImmutableArray<int> list = ImmutableArray.Create(3, 5, 6, 11, 13);
int index = 2;

ImmutableArray<int> result = [..list[..index], 7, ..list[^index..]];
// result: [3, 5, 7, 11, 13]
// Equivalent to:
//   ImmutableArray<int> result = list.SetItem(index, 7);

As of this writing, though, the implementation iterates over all the elements and builds an entirely new collection. This is quite inefficient for immutable collections, so I do not use C# 12’s collection expressions when working with immutable data. (But for mutable code, they rock!)

Application: Unidirectional Data Flow

Let’s build this up into something a bit more complex! We can give each inn an actual collection of rooms, and just acquire an available one when requested.

record class Room(int Id, bool Available);
record class Inn(string Name, ImmutableHashSet<Room> Rooms);

Inn ReserveRoom(Inn inn)
{
  var room = inn.Rooms.FirstOrDefault(r => r.Available)
      ?? throw new InvalidOperationException("No rooms available.");
  return inn with
  {
    Rooms = inn.Rooms.Remove(room).Add(room with { Available = false }),
  };
}

Inn myInn = new("Bethlehem Getaway",
    Enumerable.Range(0, 50)
    .Select(x => new Room(x + 100, Available: true))
    .ToImmutableHashSet());
Inn resultInn = ReserveRoom(myInn);

Now we have some more complex state, and our ReserveRoom method now finds a room, reserves it, and returns the new composite state of the inn. I often find it useful to have these modifier methods also return some indication of what they did - in this case, it can return the room that was reserved. Tuples are convenient for multiple return values:

(Inn Inn, int RoomId) ReserveRoom(Inn inn)
{
  var room = inn.Rooms.FirstOrDefault(r => r.Available)
      ?? throw new InvalidOperationException("No rooms available.");
  return (
      inn with
      {
        Rooms = inn.Rooms.Remove(room).Add(room with { Available = false }),
      },
      room.Id
  );
}

If you squint a bit, you can see ReserveRoom as being like a Redux reducer (for a single action: reserving a room). Many years ago, React and Redux took the world by storm. Although Redux has fallen out of favor in some circles, it made some ideas popular, and those continue to live on.

Specifically, the idea of Unidirectional Data Flow is one that has taken hold, particularly in UI applications. The core idea is that the application has a single instance of composite immutable state, and that this state is only changed by applying pure functions to it (commonly called “reducers”). Other parts of the application (including the UI) listen for and respond to state changes. UDF is an architecture that is overkill for extremely simple applications, but is an absolute lifesaver when there is significant complexity.

Unidirectional Data Flow (UDF) can go by several names. Model-View-Intent (MVI) is an architecture common on mobile platforms that is based on UDF. Another fairly common architecture name is The Elm Architecture (TEA, or sometimes just Elm). Today most C# UI applications still use a basic MVVM style of architecture, but I expect with the language changes that better support immutable update patterns, we’ll start to see more adoption of UDF architectures in C#. At least, I hope so!

Conclusion

I hope this post has been interesting to you! Personally, I do enjoy writing code combining switch and with expressions. I think the resulting code is really elegant, and I hope you got some joy out of this! Merry Christmas!

ConfigureAwait in .NET 8

Thu, 09 Nov 2023 00:00:00 +0000

I don’t often write “what’s new in .NET” posts, but .NET 8.0 has an interesting addition that I haven’t seen a lot of people talk about. ConfigureAwait is getting a pretty good overhaul/enhancement; let’s take a look!

ConfigureAwait(true) and ConfigureAwait(false)

First, let’s review the semantics and history of the original ConfigureAwait, which takes a boolean argument named continueOnCapturedContext.

When await acts on a task (Task, Task<T>, ValueTask, or ValueTask<T>), its default behavior is to capture a “context”; later, when the task completes, the async method resumes executing in that context. The “context” is SynchronizationContext.Current or TaskScheduler.Current (falling back on the thread pool context if none is provided). This default behavior of continuing on the captured context can be made explicit by using ConfigureAwait(continueOnCapturedContext: true).

ConfigureAwait(continueOnCapturedContext: false) is useful if you don’t want to resume on that context. When using ConfigureAwait(false), the async method resumes on any available thread pool thread.

The history of ConfigureAwait(false) is interesting (at least to me). Originally, the community recommended using ConfigureAwait(false) everywhere you could, unless you needed the context. This is the position I recommended in my Async Best Practices article. There were several discussions during that time frame over why the default was true, especially from frustrated library developers who had to use ConfigureAwait(false) a lot.

Over the years, though, the recommendation of “use ConfigureAwait(false) whenever you can” has been modified. The first (albeit minor) shift was instead of “use ConfigureAwait(false) whenever you can”, a simpler guideline arose: use ConfigureAwait(false) in library code and don’t use it in application code. This is an easier guideline to understand and follow. Still, the complaints about having to use ConfigureAwait(false) continued, with periodic requests to change the default on a project-wide level. These requests have always been rejected by the C# team for language consistency reasons.

More recently (specifically, since ASP.NET dropped their SynchronizationContext with ASP.NET Core and fixed all the places where sync-over-async was necessary), there has been a move away from ConfigureAwait(false). As a library author, I fully understand how annoying it is to have ConfigureAwait(false) litter your codebase! Some library authors have just decided not to bother with ConfigureAwait(false). For myself, I still use ConfigureAwait(false) in my libraries, but I understand the frustration.

An earlier version of this post incorrectly claimed that the Entity Framework Core team had decided not to use ConfigureAwait(false). This was only true in early versions of Entity Framework Core. Entity Framework Core added ConfigureAwait(false) in version 5.0.0 and continues to use ConfigureAwait(false) as of this writing (2023-11-11).

Since we’re on the topic of ConfigureAwait(false), I’d like to note a few common misconceptions:

ConfigureAwait(false) is not a good way to avoid deadlocks. That’s not its purpose, and it’s a questionable solution at best. In order to avoid deadlocks when doing direct blocking, you’d have to make sure all the asynchronous code uses ConfigureAwait(false), including code in libraries and the runtime. It’s just not a very maintainable solution. There are better solutions available.
ConfigureAwait configures the await, not the task. E.g., the ConfigureAwait(false) in SomethingAsync().ConfigureAwait(false).GetAwaiter().GetResult() does exactly nothing. Similarly, the await in var task = SomethingAsync(); task.ConfigureAwait(false); await task; still continues on the captured context, completely ignoring the ConfigureAwait(false). I’ve seen both of these mistakes over the years.
ConfigureAwait(false) does not mean “run the rest of this method on a thread pool thread” or “run the rest of this method on a different thread”. It only takes effect if the await yields control and then later resumes the async method. Specifically, await will not yield control if its task is already complete; in that case, the ConfigureAwait has no effect because the await continues synchronously.

OK, now that we’ve refreshed our understanding of ConfigureAwait(false), let’s take a look at how ConfigureAwait is getting some enhancements in .NET 8. None of the existing behavior is changed; await without any ConfigureAwait at all still has the default behavior of ConfigureAwait(true), and ConfigureAwait(false) still has the same behavior, too. But there’s a new ConfigureAwait coming into town!

ConfigureAwait(ConfigureAwaitOptions)

There are several new options available for ConfigureAwait. ConfigureAwaitOptions is a new type that provides all the different ways to configure awaitables:

namespace System.Threading.Tasks;
[Flags]
public enum ConfigureAwaitOptions
{
    None = 0x0,
    ContinueOnCapturedContext = 0x1,
    SuppressThrowing = 0x2,
    ForceYielding = 0x4,
}

First, a quick note: this is a Flags enum; any combination of these options can be used together.

The next thing I want to point out is that ConfigureAwait(ConfigureAwaitOptions) is only available on Task and Task<T>, at least for .NET 8. It wasn’t added to ValueTask / ValueTask<T> yet. It’s possible that a future release of .NET may add ConfigureAwait(ConfigureAwaitOptions) for value tasks, but as of now it’s only available on reference tasks, so you’ll need to call AsTask if you want to use these new options on value tasks.

Now, let’s consider each of these options in turn.

ConfigureAwaitOptions.None and ConfigureAwaitOptions.ContinueOnCapturedContext

These two are going to be pretty familiar, except with one twist.

ConfigureAwaitOptions.ContinueOnCapturedContext - as you might guess from the name - is the same as ConfigureAwait(continueOnCapturedContext: true). In other words, the await will capture the context and resume executing the async method on that context.

Task task = ...;

// These all do the same thing
await task;
await task.ConfigureAwait(continueOnCapturedContext: true);
await task.ConfigureAwait(ConfigureAwaitOptions.ContinueOnCapturedContext);

ConfigureAwaitOptions.None is the same as ConfigureAwait(continueOnCapturedContext: false). In other words, await will behave perfectly normally, except that it will not capture the context; assuming the await does yield (i.e, the task is not already complete), then the async method will resume executing on any available thread pool thread.

Task task = ...;

// These do the same thing
await task.ConfigureAwait(continueOnCapturedContext: false);
await task.ConfigureAwait(ConfigureAwaitOptions.None);

Here’s the twist: with the new options, the default is to not capture the context! Unless you explicitly include ContinueOnCapturedContext in your flags, the context will not be captured. Of course, the default behavior of await itself is unchanged: without any ConfigureAwait at all, await will behave as though ConfigureAwait(true) or ConfigureAwait(ConfigureAwaitOptions.ContinueOnCapturedContext) was used.

Task task = ...;

// Default behavior (no ConfigureAwait): continue on the captured context.
await task;

// Default flag option (None): do not continue on the captured context.
await task.ConfigureAwait(ConfigureAwaitOptions.None);

So, that’s something to keep in mind as you start using this new ConfigureAwaitOptions enum.

ConfigureAwaitOptions.SuppressThrowing

The SuppressThrowing flag suppresses exceptions that would otherwise occur when awaiting a task. Under normal conditions, await will observe task exceptions by re-raising them at the point of the await. Normally, this is exactly the behavior you want, but there are some situations where you just want to wait for the task to complete and you don’t care whether it completes successfully or with an exception. SuppressThrowing allows you to wait for the completion of a task without observing its result.

Task task = ...;

// These do the same thing
await task.ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
try { await task.ConfigureAwait(false); } catch { }

I expect this will be most useful alongside cancellation. There are some cases where some code needs to cancel a task and then wait for the existing task to complete before starting a replacement task. SuppressThrowing would be useful in that scenario: the code can await with SuppressThrowing, and the method will continue when the task completes, whether it was successful, canceled, or finished with an exception.

// Cancel the old task and wait for it to complete, ignoring exceptions.
_cts.Cancel();
await _task.ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);

// Start the new task.
_cts = new CancellationTokenSource();
_task = SomethingAsync(_cts.Token);

If you await with the SuppressThrowing flag, then the exception is considered “observed”, so TaskScheduler.UnobservedTaskException is not raised. The assumption is that you are awaiting the task and deliberately discarding the exception, so it’s not considered unobserved.

TaskScheduler.UnobservedTaskException += (_, __) => { Console.WriteLine("never printed"); };

Task task = Task.FromException(new InvalidOperationException());
await task.ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);
task = null;

GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();

Console.ReadKey();

There’s another consideration for this flag as well. When used with a plain Task, the semantics are clear: if the task faults, the exception is just ignored. However, the same semantics don’t quite work for Task<T>, because in that case the await expression needs to return a value (of type T). It’s not clear what value of T would be appropriate to return in the case of an ignored exception, so the current behavior is to throw an ArgumentOutOfRangeException at runtime. To help catch this at compile time, a new warning was added: CA2261 The ConfigureAwaitOptions.SuppressThrowing is only supported with the non-generic Task. This rule defaults to a warning, but I’d suggest making it an error, since it will always fail at runtime.

Task<int> task = Task.FromResult(13);

// Causes CA2261 warning at build time and ArgumentOutOfRangeException at runtime.
await task.ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing);

As a final note, this is one flag that also affects synchronous blocking in addition to await. Specifically, you can call .GetAwaiter().GetResult() to block on the awaiter returned from ConfigureAwait. The SuppressThrowing flag will cause exceptions to be ignored whether using await or GetAwaiter().GetResult(). Previously, when ConfigureAwait only took a boolean parameter, you could say “ConfigureAwait configures the await”; but now you have to be more specific: “ConfigureAwait returns a configured awaitable”. And it is now possible that the configured awaitable modifies the behavior of blocking code in addition to the behavior of the await. ConfigureAwait is perhaps a slight misnomer now, but it is still primarily intended for configuring await. Of course, blocking on asynchronous code still isn’t recommended.

Task task = Task.Run(() => throw new InvalidOperationException());

// Synchronously blocks on the task (not recommended). Does not throw an exception.
task.ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing).GetAwaiter().GetResult();

ConfigureAwaitOptions.ForceYielding

The final flag is the ForceYielding flag. I expect this flag will be rarely used, but when you need it, you need it!

ForceYielding is similar to Task.Yield. Yield returns a special awaitable that always claims to be not completed, but schedules its continuations immediately. What this means is that the await always acts asynchronously, yielding to its caller, and then the async method continues executing as soon as possible. The normal behavior for await is to check if its awaitable is complete, and if it is, then continue executing synchronously; ForceYielding prevents that synchronous behavior, forcing the await to behave asynchronously.

For myself, I find forcing asynchronous behavior most useful in unit testing. It can also be used to avoid stack dives in some cases. It may also be useful when implementing asynchronous coordination primitives, such as the ones in my AsyncEx library. Essentially, anywhere where you want to force await to behave asynchronously, you can use ForceYielding to accomplish that.

One point that I find interesting is that await with ForceYielding makes the await behave like it does in JavaScript. In JavaScript, await always yields, even if you pass it a resolved promise. In C#, you can now await a completed task with ForceYielding, and await will behave as though it’s not completed, just like JavaScript’s await.

static async Task Main()
{
  Console.WriteLine(Environment.CurrentManagedThreadId); // main thread
  await Task.CompletedTask;
  Console.WriteLine(Environment.CurrentManagedThreadId); // main thread
  await Task.CompletedTask.ConfigureAwait(ConfigureAwaitOptions.ForceYielding);
  Console.WriteLine(Environment.CurrentManagedThreadId); // thread pool thread
}

Note that ForceYielding by itself also implies not continuing on the captured context, so it is the same as saying “schedule the rest of this method to the thread pool” or “switch to a thread pool thread”.

// ForceYielding forces await to behave asynchronously.
// Lack of ContinueOnCapturedContext means the method continues on a thread pool thread.
// Therefore, code after this statement will *always* run on a thread pool thread.
await task.ConfigureAwait(ConfigureAwaitOptions.ForceYielding);

Task.Yield will resume on the captured context, so it’s not exactly like ForceYielding by itself. It’s actually like ForceYielding with ContinueOnCapturedContext.

// These do the same thing
await Task.Yield();
await Task.CompletedTask.ConfigureAwait(ConfigureAwaitOptions.ForceYielding | ConfigureAwaitOptions.ContinueOnCapturedContext);

Of course, the real value of ForceYielding is that it can be applied to any task at all. Previously, in the situations where yielding was required, you had to either add a separate await Task.Yield(); statement or create a custom awaitable. That’s no longer necessary now that ForceYielding can be applied to any task.

Padding for Overlaid Structs

Thu, 05 Oct 2023 00:00:00 +0000

Last time we covered the basics of memory-mapped files and how to overlay structs onto the in-memory view of the file. This time we’ll take a look at different techniques to add “padding” or “holes” in our overlaid structs. Sometimes your overlaid struct is a header or container for another struct, which may be one of several different structure types. For example, a binary file may be composed of records, each with an identical header, and one field of that header is the record type, which defines how the remainder of that record should be interpreted.

For this post, we’ll use the same Data struct we were working with last time, but this time we want to add some padding between the first and second data fields:

public struct Data
{
  private int _first;
  /* TODO: forty bytes of padding goes here */
  private int _second;
}

Bonus points if our solution allows accessing that padding as another overlaid struct type.

The Ideal Solution (Not Supported): Safe Fixed-Size Buffers

Ideally, we could just define a block of memory in our struct. This is similar to how it’s done in unamanged languages:

// The code below currently causes these compiler erorrs.
// Error CS0650 Bad array declarator: To declare a managed array the rank specifier precedes the variable's identifier. To declare a fixed size buffer field, use the fixed keyword before the field type.
// Error CS0270 Array size cannot be specified in a variable declaration (try initializing with a 'new' expression)
public struct Data
{
  private int _first;
  private byte _padding[40];
  private int _second;
}

There’s actually been some discussion about adding this to C#; the feature is called “safe fixed-size buffers” (a.k.a., “anonymous inline arrays”). It didn’t make it into C# 11. The syntax above was considered for C# 12 but rejected earlier this year.

Inline Arrays (.NET 8.0 / C# 12)

Even though the nicer syntax above was rejected, inline arrays themselves have been accepted. Indeed, it is possible that a future version of C# may give us the nice syntax above, implemented using inline arrays.

For now, we can just deconstruct that ourselves and write by hand what we wish the compiler would write for us:

public struct Data
{
  private int _first;
  private Padding40 _padding;
  private int _second;

  [InlineArray(40)]
  private struct Padding40
  {
    private byte _start;
  }
}

The InlineArrayAttribute is a bit odd; what it’s actually doing is telling the runtime to repeat the single field in that struct that many times. So Padding40 is actually 40 bytes long.

This works fine, as long as you’re on .NET 8.0; the InlineArrayAttribute requires runtime support. If you define your own InlineArrayAttribute and try to run this on earlier runtimes, the Padding40 struct will be the wrong size, and Data will not get the correct amount of padding.

Bonus: we can access the padding as another overlaid struct type by adding this member to the Data struct:

[UnscopedRef] public ref T PaddingAs<T>() where T : struct => ref Unsafe.As<Padding40, T>(ref _padding);

Unsafe Fixed-Size Buffers

The nicer syntax above is all about taking an existing feature - unsafe fixed-size buffers - and allowing them in a safe context. If you’re not on .NET 8.0 yet, you can still use the old-school unsafe fixed-size buffers:

public unsafe struct Data
{
  private int _first;
  private fixed byte _padding[40];
  private int _second;
}

This also works fine, but has the drawback of requiring an unsafe context. The Overlay helper from the last post is also unsafe, but it would be nice if that was the only unsafe thing and all my overlay structures don’t have to be unsafe just to add padding.

Bonus: we can access the padding as another overlaid struct type by adding this member to the Data struct:

public unsafe ref T PaddingAs<T>()
{
  fixed (byte* p = _padding)
    return ref Unsafe.AsRef<T>(p);
}

It does seem a bit awkward to me, though. The fixed statement is informing the GC that _padding can’t be moved… but since this is an overlaid structure (at the address of a memory-mapped view), it can’t be moved anyway. So it seems superfluous. Probably not a lot of overhead; it’s just that the code seems awkward: “pin this thing in memory, read the pointer value, and then unpin it”.

Explicit Struct Layout

Let’s try an old-school, p/Invoke-style approach:

[StructLayout(LayoutKind.Explicit)]
private struct Data
{
  [FieldOffset(0)]
  private int _first;
  [FieldOffset(44)]
  private int _second;
}

I was curious to know if this approach worked, and it does. I don’t really recommend it, since you have to explicitly lay out your entire struct. Also, there isn’t a good way of referencing the padding.

Marshalling (Doesn’t Work)

Just as a side note, marshalling directives don’t work. For example:

// Does not work!
[StructLayout(LayoutKind.Sequential)]
private struct Data
{
  private int _first;
  [MarshalAs(UnmanagedType.ByValArray, SizeConst = 40)]
  private byte[] _padding;
  private int _second;
}

This works if we’re doing p/Invoke, because it’s marshalling (copying) the structure to/from unmanaged code. Since we’re overlaying the structure directly in memory, marshalling directives like this don’t work.

Explicit Fields

Of course, you can always define padding using multiple explicit fields. The resulting code is ugly (and IMO more awkward to maintain), but it works fine:

public struct Data
{
  private int _first;
  private int _padding0, _padding1, _padding2, _padding3, _padding4, _padding5, _padding6, _padding7, _padding8, _padding9;
  private int _second;
}

I’m using int fields above so I only have to type 10 of them, as opposed to 40 byte-sized fields.

You can even do a bonus round with this approach by referencing the first padding member:

[UnscopedRef] public ref T PaddingAs<T>() where T : struct => ref Unsafe.As<int, T>(ref _padding0);

Of course, if you have lots of padding (or multiple padding sections), this can get tedious.

Conclusion

Since I’m working on a greenfield project, I’ve chosen to use the .NET 8.0-style InlineArrayAttribute approach, with the hope that the syntax becomes nicer in future versions of C#. If I had to support older .NET versions, I’d probably take the “Unsafe Fixed-Size Buffers” approach, even though it requires unsafe contexts for all those overlaid structs.

I hope this has been helpful to you during your memory-mapping adventures!

Memory-Mapped Files and Overlaid Structs

Thu, 28 Sep 2023 00:00:00 +0000

It has been a long, long time since I’ve used memory-mapped files - I think the last time was before .NET existed (!). Recently, I had a need to work with memory-mapped files in C#, and I gathered together a few resources that explain how to do it - specifically, how to map a file into memory and then “overlay” a structure on top of that memory. Since it took me a while to figure this out (and I learned about some cool upcoming features along the way), I thought I’d write this up into a proper post or two.

Memory-Mapped Files

Memory-mapped files are a pretty cool technique, where instead of reading disk data into memory directly, you can map it into the memory space of your process very quickly. Once it’s mapped into your process memory, reading from that memory will read from the disk (as necessary), and writing to that memory will write out to the file (eventually). You can do cool things like create a huge file mapping (way larger than your memory), and it will Just Work, paging memory in and out of your process behind the scenes. There’s a ton of information about memory-mapped files out there; if you’re on Windows, I like Windows Internals - Part 1 covers the memory manager (including memory-mapped files), and Part 2 has a few additional details on how memory-mapped files interact with the cache manager.

In C#, mapping a file into memory isn’t terribly complex. First, you open the file (i.e., create a FileStream object). Then, you create a file mapping. Tip on the file mapping: if you’re mapping an existing file, you can pass 0 for the file length to just map the entire file. Finally, you create a view on that file mapping - and this is the step that actually maps the file into the memory space for your process. You can create a view over the entire file, but if you’re dealing with a very large file mapping, it’s common to create partial views as you need them.

This code will create a new file, a file mapping (specifying 1000 bytes as the length of the file; the file is immediately grown to this size), and a single view over the entire file:

using FileStream file = new FileStream(@"tmp.dat", FileMode.Create, FileAccess.ReadWrite,
    FileShare.None, 4096, FileOptions.RandomAccess);
using MemoryMappedFile mapping = MemoryMappedFile.CreateFromFile(file, null, 1000,
    MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, leaveOpen: true);
using MemoryMappedViewAccessor view = mapping.CreateViewAccessor();

At this point, you have a view, which is a handle (actually a pointer) to the part of your process’ memory that actually represents the file contents. What’s really nice about this code is that it’s portable; the same code works on Linux and Windows (and presumably Mac and mobile platforms, though I haven’t tried those). However, pointers aren’t a great interface, especially in a managed language like C#. MemoryMappedViewAccessor has a bunch of… well… awkward methods that are essentially “read a signed 16-bit integer at this offset”, “write an unsigned 32-bit integer at this offset”, etc. You can also copy a struct into and out of the view, but I don’t want to go through the trouble of doing a file mapping just to turn around and serialize a struct anyway.

For convenience, unmanaged languages commonly overlay a structure onto the mapped memory. This approach allows you to define the file structure as an actual struct and then read/write fields in that struct instead of serializing values to memory or view offsets. “Overlapped structures” might be a more common term than “overlaid structures”, but I want to avoid any confusion with OVERLAPPED, so I’m using the term “overlaid structures” in these posts.

If you’re in an unmanaged language like C++, you can just reinterpret_cast your file mapping view pointer to a structure pointer, and that’s it: you’ve got a struct at the same memory address as your file view! I found that there was much less information about overlaying structs in C#, though. So, let’s see how to do the same thing in C#!

Overlaid Structs

After a bit of experimentation, this is what I ended up with:

public sealed unsafe class Overlay : IDisposable
{
  private readonly MemoryMappedViewAccessor _view;
  private readonly byte* _pointer;

  public Overlay(MemoryMappedViewAccessor view)
  {
    _view = view;
    view.SafeMemoryMappedViewHandle.AcquirePointer(ref _pointer);
  }

  public void Dispose() => _view.SafeMemoryMappedViewHandle.ReleasePointer();

  public ref T As<T>() where T : struct => ref Unsafe.AsRef<T>(_pointer);
}

This is an unsafe type, but ideally this is the only place where unsafe is necessary.

Overlay is mainly just a pointer - the pointer to the view of the file that has been mapped into your process’ memory. It also has a MemoryMappedViewAccessor member, but that’s just used to free the pointer when the Overlay instance is disposed.

Overlay has a single notable member: As<T>(), which allows you to get a reference to a struct that overlays the mapped memory view.

On Windows (at least), the SafeMemoryMappedViewHandle handle actually is a pointer, and the AcquirePointer and ReleasePointer calls increment and decrement a reference counter for that handle. Overlay could be designed very differently (and more efficiently) if it cast the SafeMemoryMappedViewHandle handle value to a pointer.

However, on other platforms, I’m not sure if SafeMemoryMappedViewHandle is actually a pointer or not, so I’ve stuck with this safer implementation just to make sure the code is portable.

If you are OK with assuming SafeMemoryMappedViewHandle is a pointer, you can use this instead of Overlay:

public static class MemoryMappedViewAccessorExtensions
{
  public static unsafe ref T As<T>(this MemoryMappedViewAccessor accessor) where T : struct =>
    ref Unsafe.AsRef<T>(accessor.SafeMemoryMappedViewHandle.DangerousGetHandle().ToPointer());
}

There’s a fair amount of “unsafe” and “dangerous” in that code, though, and it also makes some implementation assumptions (specifically, that SafeMemoryMappedViewHandle’s handle is an actual pointer to memory). So, for safety, I’m just sticking with Overlay with its explicit AcquirePointer and ReleasePointer calls.

Using Overlay

First, define your struct type, keeping in mind that the in-memory layout (including packing/padding) must reflect the on-disk file structure. Then, you can map a file just like the above code, create an Overlay type, and acquire a struct reference. At that point, you can read or write the struct as desired.

public struct Data
{
  public int First;
  public int Second;
}

using FileStream file = new FileStream(@"tmp.dat", FileMode.Create, FileAccess.ReadWrite,
    FileShare.None, 4096, FileOptions.RandomAccess);
using MemoryMappedFile mapping = MemoryMappedFile.CreateFromFile(file, null, 1000,
    MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, leaveOpen: true);
using MemoryMappedViewAccessor view = mapping.CreateViewAccessor();
using Overlay overlay = new Overlay(view);
ref Data data = ref overlay.As<Data>();
data.First = 1;
data.Second = 2;

Run the code above (works in LINQPad!), and you’ll end up with a tmp.dat file 1000 bytes long, with the first four bytes having the value of First (1) and the second four bytes having the value of Second (2). Note that since you’re reading/writing structures in memory, whatever endianness your machine is will determine the endianness of the binary file. Go ahead and pop it open in a hex editor (there’s an online one called HexEd.it), and take a look at the binary file itself.

Endianness

If you’re working with portable file formats, handling endianness is a necessity. Values in files on disk must be little-endian or big-endian, regardless of what processor happens to be reading or writing them. I recommend handling the differences in code with helpers, like this:

public static class OverlayHelpers
{
  public static int ReadBigEndian(int bigEndian) =>
      BitConverter.IsLittleEndian ? BinaryPrimitives.ReverseEndianness(bigEndian) : bigEndian;
  public static void WriteBigEndian(out int bigEndian, int value) =>
      bigEndian = BitConverter.IsLittleEndian ? BinaryPrimitives.ReverseEndianness(value) : value;
  public static int ReadLittleEndian(int littleEndian) =>
      BitConverter.IsLittleEndian ? littleEndian : BinaryPrimitives.ReverseEndianness(littleEndian);
  public static void WriteLittleEndian(out int littleEndian, int value) =>
      littleEndian = BitConverter.IsLittleEndian ? value : BinaryPrimitives.ReverseEndianness(value);
}

The helpers above let you read/write big- or little-endian values, regardless of the endianness of the current machine. They can be used in your structure definitions as such:

public struct Data
{
  // Layout
  private int _first;
  private int _second;

  // Convenience accessors
  public int First
  {
    readonly get => OverlayHelpers.ReadBigEndian(_first);
    set => OverlayHelpers.WriteBigEndian(out _first, value);
  }
  public int Second
  {
    readonly get => OverlayHelpers.ReadBigEndian(_second);
    set => OverlayHelpers.WriteBigEndian(out _second, value);
  }
}

Now the same program as above will always write the “first” and “second” fields as 32-bit signed big-endian values:

// (this is the same code as above)
using FileStream file = new FileStream(@"tmp.dat", FileMode.Create, FileAccess.ReadWrite,
    FileShare.None, 4096, FileOptions.RandomAccess);
using MemoryMappedFile mapping = MemoryMappedFile.CreateFromFile(file, null, 1000,
    MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, leaveOpen: true);
using MemoryMappedViewAccessor view = mapping.CreateViewAccessor();
using Overlay overlay = new Overlay(view);
ref Data data = ref overlay.As<Data>();
data.First = 1;
data.Second = 2;

Now, the code is completely portable: any .NET runtime that supports memory-mapped files (which AFAIK is all of them) will run this code, giving you the ability to define portable binary file formats using overlaid structures.

A Word of Warning: Alignment

Since you’re overlaying structures directly into memory addresses, you have to handle all the alignment requirements yourself. Some more common architectures such as x86/x64 don’t care about alignment and allow you to, e.g., define an int field at an offset of 1. Other architectures do not allow unaligned access at all.

As a general guideline, align your structure members by their own size. E.g., an int is 4 bytes, so it should be aligned on a 4-byte boundary. Put another way, the offset of an int field from the beginning of the struct should be evenly divisible by 4. Same for other types: long should be aligned on an 8-byte boundary, while byte should be aligned on a 1-byte boundary (i.e., anywhere).

A Word of Warning: Exceptions

Memory mapped files give you one kind of convenience by mapping files into memory, but the counterpoint is that I/O exceptions may not happen exactly when you expect them to.

When reading a file using normal I/O calls, if the read fails, then it fails right at that time. When using memory-mapped files, reads from memory may cause an I/O exception. This is true even if a previous read from that same memory succeeded.

Similarly, if you write to a file using normal I/O calls, any failures are reported immediately. With memory-mapped files, memory writes may cause an I/O exception. And since memory-mapped files are lazily flushed to disk, I/O exceptions may be delayed until the view is flushed (during disposal).

Next Time

I hope this has been helpful! If anyone out there knows a way to eliminate the unsafe code in Overlay, I’d love to hear it!

Next time I’m planning to write a bit about overlaying structures with holes in them, which is a useful technique when you have “header” or “container” structures that wrap other structures possibly of different types.

Grounded ChatGPT

Thu, 18 May 2023 00:00:00 +0000

So, there’s this thing you may have heard of called ChatGPT. A lot of people (myself included) have thought “OK, nice toy. It’s pretty good at producing human-sounding text. But I want to run it on my own data without becoming a data scientist and spending a few hundred thousand dollars in training costs.”

Then someone pointed out to me there’s already a technique for this called Retrieval Augmented Generation, and in fact there’s some sample code right there showing how to do it.

Retrieval Augmented Generation

To save you a Google search (or ChatGPT query?), here’s my super-simple description of this technique: when the user asks a question, instead of just giving it to ChatGPT directly, first do a search for that question over your own data, and combine the search results along with the user’s question as the ChatGPT input.

This technique “grounds” ChatGPT, giving it your own data alongside the user’s question. If you structure your input properly, you can influence ChatGPT to produce relevant results, even including source references. With this technique, ChatGPT is able to produce much better results, without the need for training or even fine-tuning the model itself.

Sample Code

The official sample referenced above is in Python. And I love Python. As a language, I mean. But it’s been… um… 25 years or so since I’ve used it. Definitely rusty. So I decided to write my own sample (heavily influenced by the official one) in C#. And using local Docker containers as much as possible instead of creating a bunch of Azure resources.

You can find my C# Retrieval Augmented Generation code on GitHub. It’s not production-ready, but it gets the general point across. You can use it pretty easily to “teach” ChatGPT about modern events or your own custom data. It uses Elasticsearch and Seq (both in local Docker containers), preserving its data in local Docker volumes. And it has exhaustive logging out of the box, so you can always review what APIs were called and how exactly they work. My code does use the Azure OpenAI API to talk to ChatGPT, but everything else is in local Docker containers.

More Implementation Details

When you use this sample code to do a retrieval-augmented generation, what actually happens is this:

The user’s question is sent to ChatGPT to extract search keywords, using this template:

Below is a question asked by the user that needs to be answered by searching.
Generate a search query based on names and concepts extracted from the question.

### Question:
{question}

### Search query:

ChatGPT is pretty good at generating a search query from a user question; I set the temperature to zero to ensure there’s no randomness in this call.

Next, this ChatGPT response is sent to Elasticsearch (just as a simple query string).

The results of the Elasticsearch search are then formatted and injected into a ChatGPT prompt that looks like this:

Answer the following question. You may include multiple answers, but each answer may only use the data provided in the References below.
Each Reference has a number followed by tab and then its data.
Use square brakets to indicate which Reference was used, e.g. [7]
Don't combine References; list each Reference separately, e.g. [1][2]
If you cannot answer using the References below, say you don't know. Only provide answers that include at least one Reference name.
If asking a clarifying question to the user would help, ask the question.
Do not comment on unused References.

### References:
{sources}

The result is then post-processed to extract the quoted references and change them to hyperlinks.

Have Fun!

I’ve been pretty pleased with the results, even though I’m using very simplistic source processing, and a lexical search instead of a more proper semantic/vector search. Even with those limitations, the results are pretty impressive!

That’s all I have to say for now. Have fun!

Stephen Cleary (the blog)

C# Advent: No one loves the .NET Core name anymore

A Brief History of “.NET Core”

.NET Compact Framework

Silverlight, WinRT, Windows Phone, Windows Store, and Universal Windows Platform

netcore as a Target Framework Moniker

No More .NET Core

Cancellation, Part 6: Linking

Linked Cancellation Tokens

Multiple Links

Use Cases

Sharp Corner: Don’t use OperationCanceledException.CancellationToken

Checking Inner Tokens: Still don’t use OperationCanceledException.CancellationToken

Summary

Cancellation, Part 5: Registration

How to Register

A Race Condition

Cleanup Is Important!

Sharp Corner: Synchronous Cancellation Callbacks

Summary

ICYMI: Video Series on TCP/IP Application Protocol Design

MetaPost 2024-04-24

ICYMI (In Case You Missed It)

Advertisements

Google Ads: The Early Days

Disqus

Google Ads: Now Also Gone

The Future of My Ads

C# Advent: The Joy of Immutable Update Patterns

Joy to the World!

Immutability

Updating Immutable Data

switch Expressions

with Expressions

Collections

Application: Unidirectional Data Flow

Conclusion

ConfigureAwait in .NET 8

ConfigureAwait(true) and ConfigureAwait(false)

ConfigureAwait(ConfigureAwaitOptions)

ConfigureAwaitOptions.None and ConfigureAwaitOptions.ContinueOnCapturedContext

ConfigureAwaitOptions.SuppressThrowing

ConfigureAwaitOptions.ForceYielding

Further Reading

Padding for Overlaid Structs

The Ideal Solution (Not Supported): Safe Fixed-Size Buffers

Inline Arrays (.NET 8.0 / C# 12)

Unsafe Fixed-Size Buffers

Explicit Struct Layout

Marshalling (Doesn’t Work)

Explicit Fields

Conclusion

Memory-Mapped Files and Overlaid Structs

Memory-Mapped Files

Overlaid Structs

Using Overlay

Endianness

A Word of Warning: Alignment

A Word of Warning: Exceptions

Next Time

Grounded ChatGPT

Retrieval Augmented Generation

Sample Code

More Implementation Details

Have Fun!

`switch` Expressions

`with` Expressions