Honest Illusion

Inside a Where() - Understanding IEnumerables

2024-12-05T00:00:00+00:00

Inside a Where() - Understanding IEnumerables

This post is for day 5 of the 2024 C# Advent Calendar operated by Matthew Groves. Thanks for letting me participate! (I’m filling in for someone who dropped out at the last minute.)

IEnumerable<T> is one of the most basic elements of the .NET framework (and really Computer Engineering itself), so I’m often surprised when I see some commentator giving a rather confused description of it.

I found this especially true when IEnumerables are chained together, say in

 var newList = 
     myCollection
         .Where(x => x > 5)
         .Where(x => x % 3 == 0)
         .ToList();

(Yes, that’s silly, but I kept it simple). I’ve heard people claim that the first Where() is building a whole new list, which it passes to the second Where(), which passes it on the the ToList(), which apparently they think builds a new list out of the list….

That’s not at all what’s happening.

At it’s most basic level, an IEnumerable is simple a object generator, which, when asked, will give you the next item in a sequence. How exactly “the next item” is defined depends on the IEnumerable.

The important thing to remember, is that it is not limited to getting the next item from a collection. The basic case of this is the Enumerable.Range() method, which is essentially this (simplified for the example):

 IEnumerable<int> Range(int start, int count)
 {
    while(count-- > 0)
        yield return start++;
 }

No collection behind it. Just “the next item”, which in this case is the next higher number.

Now, with the case of Where(), and other LINQ methods like Select, which take a IEnumerable as a parameter, and returns an IEnumerable, the method really just creates a new IEnumerable object which wraps the given IEnumerable.

Essentially what is happening in the example I wrote above, is that the ToList() says “I need to build a list. Give me the first item to start”, and asks its input IEnumerable for some item. This input IEnumerable is the object created by the second Where() method.

This IEnumerable says “I have no object to complete that request, so I must ask my input IEnumerable for one”. That input IEnumerable is, of course, the object created by the first Where().

And so on. That IEnumerable similarly says “I have no object to complete that request, so I must ask my input IEnumerable for one”. That input IEnumerable is myCollection, which we’ll assume is a List<int> holding [4, 7, 2, 6, etc] for this example.

myCollection (which, here, is just another IEnumerable) cordially offers up its first element, 4, when asked. To which the first Where() says “Nope, not what I was looking for”, and asks for another. myCollection then gives up 7, which the first Where() likes, so it passes it onto the second Where().

But, the second Where rejects the 7, and thus must ask the first Where() for another int. For this, the first Where() must go back to myCollection, first for the 2 (which the first Where rejects), and then for 6, which it likes and passes onto the second Where().

The second Where() also like the 6, and therefore passes it onto the ToList() which finally can place a value into the list it’s building. But then, it immediately starts the whole process over — asking for an int from the second Where(), which then asks the first Where, which asked myCollection and so on, until myCollection says it has no more, and that information is passed down the chain.

We can see this by taking our example from before and augmenting it with some logging in the predicate lambdas.

int[] myCollection = {4,7,2,6};

var newList =
	myCollection
		.Where(x =>
		{
			Console.WriteLine($"is {x} greater than 5?");
			return x > 5;
		})
		.Where(x => 
		{
			Console.WriteLine($"is {x} divisible by 3?");
			return x % 3 == 0;
		})
		.ToList();

And we’ll get the output:

Is 4 greater than 5?
Is 7 greater than 5?
Is 7 divisible by 3?
Is 2 greater than 5?
Is 6 greater than 5?
Is 6 divisible by 3?

Percentage --- Simplifying Progress Reporting

2024-11-18T00:00:00+00:00

Percentage — Simplifying Progress Reporting

A few years, I wrote a post about a simple class a wrote called Every (Every – Doing Something Occasionally). It’s purpose was to ease handling, if you had, say, 50,000 records to process, and you wanted some notification after every 1000 were complete.

That’s all well and good, but sometimes, rather than knowing some fixed number of records is complete, you prefer knowing what percentage is complete.

I came upon that need recently at work, so I created this class, which builds on Every.

Use is simple:

   var progress = new Percentage(numRecords, 
        (count, percentage) => Console.WriteLine($"We're at {percentage}% done.");
   foreach(var item in MyLongList)
   {
          Process(item);
          ++progress;
   }

It’s rather straightforward, much like it predecessor, except for one tricky part, which causes it to have two basic modes.

If the total number of items being processed is greater than 100, then the provided Action<int, double> will be called 100 times, once every time another percent of the total is complete. The two parameters are the number of items processed, and the percentage complete, which will increase by one each time.

If the total is less than 100, then then action is called every time the operator++ is called, in this case, with the percentage increased appropriately. For example, if you only have 17 steps, then the percentage will step about 6 each time.

Of course, that is handled internally, so you don’t have to worry about it.

Source code is available here: https://github.com/jamescurran/Every

The Oregon Trail Project - Part 5 - Shooting

2023-12-27T00:00:00+00:00

Part 0 : Intro
Part 1 : Markdown to Spectre Console
Part 2 : Dates
Part 3 : Data
Part 4 : Trail Events
Part 5 : Shooting (this one)

At various time in the course of the game, you’ll have to shoot things, either to hunt for food, or to defend yourself against bandits or wild animals. This is handled by asking you to type a word, and timing how long it takes you. This is scaled by a rating you gave yourself. At the start of the game, you are asked how good a shot you are;

PRINT "HOW GOOD A SHOT ARE YOU WITH YOUR RIFLE?"
PRINT "  (1) ACE MARKSMAN,  (2) GOOD SHOT,  (3) FAIR TO MIDDLIN'"
PRINT "         (4) NEED MORE PRACTICE,  (5) SHAKY KNEES"
PRINT "ENTER ONE OF THE ABOVE . THE BETTER YOU CLAIM YOU ARE, THE"
PRINT "FASTER YOU'LL HAVE TO BE WITH YOUR GUN TO BE SUCCESSFUL."
INPUT D9

Due to the way it is handled, the better a shot you claim to be, the worst shot you actually are. And this rating it used nowhere else in the code, so there is zero advantage to claiming you are a good shot. But, regardless….

Now, the actual code handling shooting take advantage of a specific feature of HP Timeshare Basic.

S$="WHAM"
PRINT "TYPE ";S$
ENTER 255,B1,C$
B1=B1-(D9-1)
PRINT
IF B1>0 THEN 6260
B1=0
IF C$=S$ THEN 6280
B1=100
RETURN

Fortunately, I was able to track down a manual for the version of Basic online:

TSB includes ENTER, a variation on the standard INPUT statement that continues after a time limit is reached. ENTER has three inputs, a time limit in seconds, a return variable containing the actual time elapsed (or a status code), and then finally the user input. For instance, ENTER 15,T,A$[1,1] will wait 15 seconds for the user to type in a single character. T will contain the actual time they took, -256 if the timer expired, or -257 or -258 to indicate problems with the terminal.

So, on line 6210, the ENTER 255,B1, C$ means wait up to 255 seconds (4+ minutes) for the player to type a line, returning the line typed in C$ and the length of time (presumably in seconds) in B1.

The timing in fairly simple in .NET. But, we have to be concerned about making it device independent. Which give us this:

And this bring us to the rather mysterious IO property in the context object, which will be the subject of our next update.

All Purpose Object Updater

2023-12-15T00:00:00+00:00

This post is for day 15 of the 2023 C# Advent Calendar operated by Matthew Groves. Thanks for letting me participate!

In past years, when I’ve written for the Advent Calender (this is my sixth year participating), I’ve noted how many weeks, months or …um… years, since my last blog post. But not this year! I actually did SIX last month! In fact, I’m in the midst of a series where I refactor the classic game Oregon Trail into modern C#. Check it out!

So, now back to the matter at hand, a one-off bit coding for those just popping up for the holidays.

In my day job, we had a need to test a object for some condition, and if true, make some change to the object. For example, if in an order record, a credit card number is given, then the PaymentType property must be set to CreditCard. The trick is that both the condition (there are many of them) and the changes to be made are stored in a database as JSON.

Testing for the condition is handled by the MicroRulesEngine, an open source project I’ve contributed heavily to.

The method to handle updating the object had to be home-grown, and now is time to share it with the world.

Perhaps you are familiar with the “non-destructive mutation” syntax added in C# v 9:

var me = new Person { FirstName = "James", LastName ="Curran" };
var sister = me with { FirstName = "Jean" };

This is conceptional similar, except it’s a destructive, in-place mutation, which actually means it’s quite different.

var me = new Person { FirstName = "James", LastName ="Curran" };
me.Update( new { FirstName = "Jean" });

The process is quite simple: Iterate over the properties in the changes object, look up the same-named property in the target object, and update it, with some sanity checks along the way.

Now that’s fine if you have an object with the changes (or can deserialize a bit of JSON), but sometimes it’s easier to have the changes in a Dictionary<string,object>:

This runs basically the same way, except now we are iterating over the items in the dictionary. It has a couple special features: You can use the word “null” instead of a null value to set the property to null. (null itself also works) And you can use “Y” or “N” (or “Yes” or “no”) to set a boolean to true or false.

Which brings us to another special case we had. Sometimes there was only one property to change, but it had to be set to one of two values depending on condition. Again, the changes were stored as text in the database, so we settled on the format: “{propertyname}={truevalue} | {falsevalue}, e.g., Mode=3|5.

The Oregon Trail Project - Part 4 - Trail Events

2023-11-30T00:00:00+00:00

Oregon Trail Project

Part 0 : Intro
Part 1 : Markdown to Spectre Console
Part 2 : Dates
Part 3 : Data
Part 4 : Trail Events (this one)

Virtual Coffee’s Blogging Challenge is on it’s last day! And we’re still quite a ways short of the 100K words goals, so we’re gonna work this down to the last minute….

Ya’know, I think it about time that we actually get to some real code for this project. An important element of gameplay are the occurrences that happen to you along th trail, which the source code refers to as “events”, but which I’ll call “trail events”, to avoid confusion with delegates and the event keyword.

There are sixteen different things that could happen to you each turn, such as “WAGON BREAKS DOWN–LOSE TIME AND SUPPLIES FIXING IT” and “OX INJURES LEG—SLOWS YOU DOWN REST OF TRIP”. The selection of which trail event happens on any given turn is handled by this bit of code:

LET D1=0
RESTORE
R1=100*RND(0)
LET D1=D1+1
IF D1=16 THEN 4670
READ D
IF R1>D THEN 3580
DATA 6,11,13,15,17,22,32,35,37,42,44,54,64,69,95
IF D1>10 THEN 3650
GOTO D1 OF 3660,3700,3740,3790,3820,3850,3880,3960,4130,4190
GOTO D1-10 OF 4220,4290,4340,4560,4610,4670

If you’re not familiar with Basic code, let me explain. RND(0) return a random floating point value between 0 and 1, so R1 is between 0 and 100. The DATA value on line 3620 correspond to the percentage chance for each trail event being selected. Each time the READ is executed, the next DATA value is assigned to the variable D. RESTORE reset the DATA so the next READ would start again with 6.

So, first pass thru, D1 is 1, D is 6, and R1 is a random number [0..100). If that random number is less than or equal to 6, the GOTO .. OF lines will direct execution to the first trail event. If it is greater than 6, it tries again with D1 as 2, D as 11.

It tries this up to 16 times, with the value of D increasing each time until one hits. On the 16th iteration, if the random number less than 100, is also less than 95, we go to trail event at line 4610 (line 3650). On the next time thru the loop, we go directly to line 4670 (“HELPFUL INDIANS SHOW YOU WHERE TO FIND MORE FOOD”).

So, the percentage chance of any particular trail event happening is the difference between it’s corresponding value in the DATA line, and the value below it. Which means we can just use the difference, and the order doesn’t matter. This makes the next step possible.

I want to isolate each of those trail events into individual classes (and then organize them using .NET’s Managed Extensibility Framework (MEF)). This is major over-engineering, but that pretty much describes this entire project, so we’re not going to let that stop is.

A typical trail event looks like this:

PRINT "HAIL STORM---SUPPLIES DAMAGED"
M=M-5-RND(0)*10
B=B-200
M1=M1-4-RND(0)*3
GOTO 4710

To translate this into C# with MEF, first we define an Interface to be used by each of the trail events:

public interface ITrailEvent
{
    string Occasion(GameContext context);
}

(I couldn’t stand calling the method Invoke, which led to a trip to a thesaurus, hence Occasion. That interface may change as this project develops. I think particularly that return value will change.)

This will make the code:

public string Occasion(GameContext context)
{
    context.Miles -= Random.Shared.Next(5, 15);
    context.Resources.Bullets -= 200;
    context.Resources.Misc -= Random.Shared.Next(1, 4);

    return "Hail Storm---Supplies damaged";
}

Next, we have to mark this as part of our MEF system. This involve adding a [Export] attribute giving the interface it implements. Later we’ll ask MEF to gather up all classes that export that interface. But, also want to know what percent of time that trail event should occur – before we actually load it. For that we can use the [ExportMetadata] attribute.

[Export(typeof(ITrailEvent))]
[ExportMetadata("Weight", 10)]
public class HailStorm : ITrailEvent
{
    ...
}

MEF will create an object on the fly with the metadata, but we’ll need to define an interface that represents it:

public interface ITrailEventData
{
    int  Weight { get; }
}

Ok, we’ve get the piece there, now we have to work on loading and invoking them. First we’ll need a place to store them:

[ImportMany]
List<Lazy<ITrailEvent, ITrailEventData>> trailEventsImport;

And then you need to load it up. Here we just take all trail event defined in the same assembly as the GameContext class, which is where I’ll all the trail events defined in the original game. But the advantage here is that we can add another catalog.Catalogs.Add( ) line, pointing to a folder, and it will scan all the assemblies in the folder for more components, so we can extend this with many events, without ever having to recompile the game.

var catalog = new AggregateCatalog();
catalog.Catalogs.Add(new AssemblyCatalog(typeof(GameContext).Assembly));
var container = new CompositionContainer(catalog);
container.ComposeParts(this);

Next build a new list, replacing the individual weights, with a running total, so each gets it own piece of the total range.

var trailEvents = new List<(int weight, ITrailEvent)>(trailEventsImport.Count);
int totalWeight = 0;
foreach (var item in trailEventsImport)
{
    totalWeight += item.Metadata.Weight;
    trailEvents.Add(ValueTuple.Create(totalWeight, item.Value));
}

Then to randomly select one, at the same frequency at defined in the original code:

var val = Random.Shared.Next(totalWeight);
var todaysEvent = trailEvents
    .SkipWhile(te => te.weight < val)
    .Select(te => te.evtTrail)
    .FirstOrDefault();

Then we just call that

GameContext context = new GameContext();
// :
// :
todaysEvent.Occasion(context);

The Oregon Trail Project - Part 3 - Data

2023-11-29T00:00:00+00:00

Oregon Trail Project

Part 0 : Intro
Part 1 : Markdown to Spectre Console
Part 2 : Dates
Part 3 : Data (this one)

Virtual Coffee’s Blogging Challenge has just a couple days left, and we’re still quite a ways short of the 100K words goals, so lets see how many I can get out before time runs out…

One of the four principles of Object-Oriented Programming is Encapsulation, also known as (well, closely related to) Data Hiding. Basically, bundling all the data used by one section of the application together, and hiding it away from the rest.

Programs written for computer languages like the HP TIME-SHARED BASIC used for this version of Oregon Trail are pretty much why that tenet was added.

In Basic compilers of that era (actually, most were interpreters) ALL variables were global. (In C# these days it’s actually impossible to have an actual global, and even simulating one is difficult — and very much frowned on). And variable names could only be a single letter, or a single letter followed by a single digit. Numeric variable were all floating point (no true integers). String variables were available, indicated by a trailing dollar sign (e.g. A$).

This led to such a mess in most programs, that the Oregon Trail source code helpfully include a map in the comments at the end.

REM ***IDENTIFICATION OF VARIABLES IN THE PROGRAM***
REM A = AMOUNT SPENT ON ANIMALS
REM B = AMOUNT SPENT ON AMMUNITION
REM B1 = ACTUAL RESPONSE TIME FOR INPUTTING "BANG"
REM B3 = CLOCK TIME START OF INPUTTING "BANG"
REM C = AMOUNT SPENT ON CLOTHING
REM C1 = FLAG FOR INSUFFICIENT CLOTHING IN COLD WEATHER
REM C$ = YES/NO RESPONSE TO QUESTIONS

(that continues on for another 25 lines)

So, one of our primary tasks as we port the code is fixing all that. I’ll give each variable a real name, and limit it’s scope to just the locality where it’s used.

The first thing done when playing the game, is that you are given some money ($700), and you allocate that to supplies you will need along the trip (Oxen, Food, Clothes, Bullets, and Miscellany plus remaining Cash). These fluctuate over the course of the game, and their level determine if you win or lose, so I figure they belong in an class of their own.

public class Resources : IFormattable
{
	public int Oxen { get; set; } // 200-300
	public int Food { get; set; } // 0 -
	public int Bullets { get; set; } // 0-
	public int Clothes { get; set; } // 0-
	public int Misc { get; set; } // 0-

Each turn begins by running through this block of code:

IF F >= 0 THEN 1770
F=0
IF B >= 0 THEN 1790
B=0
IF C >= 0 THEN 1810
C=0
IF M1 >= 0 THEN 1830
M1=0
F=INT(F)
B=INT(B)
C=INT(C)
M1=INT(M1)
T=INT(T)
M=INT(M)

If it’s not obvious to you, that ensuring that the levels of Food (F), Bullets (B), Clothing (C) and Miscellany (M1) are at least zero, and then truncates each of them as well as the leftover Cash (T) and distance travels in Miles (M) to whole numbers. Since C# allows us to use true integers, we don’t need the second part, but the first section needs to be moved to a member method of our Resources class:

	public void Clean()
	{
		if (Oxen < 0) Oxen = 0;
		if (Food < 0) Food = 0;
		if (Bullets < 0) Bullets = 0;
		if (Clothes < 0) Clothes = 0;
		if (Misc < 0) Misc = 0;
	}

Now, before we move on, I must tell a story from programming assignments past. In the early 1990s, I was working for a company that was very much mainframe-centered, and very reluctant to change. They had started introducing PCs, and had a PC development team, but you can tell they weren’t happy about it. Their IT department had just approved Windows 2.0 for use on office machines, despite Windows 3.0 having been out for over a year.

We (myself and another new hire) were supporting one application and developing another application in C. The first app was written by a few of the existing developers, where it was their first C program after years of working in Fortran for a mainframe. It was a paragon of monolithic spaghetti code. At one spot, I found a 100-line if() block, inside a 200-line for() loop, inside a 300-line while loop, inside a 400-line case statement, inside a 1000-line switch block. The idea of a simple function to do one small task was completely alien to them – partly because any code “entry-point” (which in Fortran would be the program start, but in C was a function), require literally seven pages of documentation.

The point of my reliving nightmares of the past, is this: The project leader on writing the new application was one of those hold-overs from Fortran. But, he’d been trying to keep up with software development trends, and knew global variables were bad, and he wanted to minimize them as much as possible in the new code – but he didn’t quite get it.

In his design, each section of the application would define a struct to hold all the data needed for that section. These structs would hold dozens of unrelated variables, which were essentially just a bunch of globals for that module. But, at least they were localized.

But it gets worse. All of these structs were allocated at program startup, and pointers to them were stored in another struct, which we called the SAAB, which was officially the “Structured A{something} Allocation Block”, but was really just named after my car. The we had one global variable, pSAAB, which held a pointer the that struct.

The project leader was overjoyed. He’d had reduced the application to just a single global variable.

But that was nonsense. The program had hundred of global variables, just like it’s predecessor; we had just renamed them to something long & complicated. But whether you call something WidgetThingee or pSAAB -> pWidgetBlock -> pThingeePtr, if it’s globally accessible, it’s a global variable.

Which brings us back to the present (where we are dealing with code even older than we were in the above). We now have to create the classes used by the rest of the code, and in the original, everything was a global, so every variable was used all over the place. We have to resist the urge to create a class that just holds every variable, and pass that object around like a global.

The heart of the game are 16 “events” (Wagon breaks down, bandits attack, etc) which could happen to you along the trail. In our final design, each of those will get its own class (actually, a whole module, but that’s getting ahead of ourselves), so we’ll need a game context object which can be passed to and from any of the Event classes, holding all the information that class would need about the status of the game, but no more, lest we fall into the pSAAB problem of yore.

After a bit of analysis, I came up with this:

public class GameContext
{
    public Resources Resources { get; set; }
    public int TurnCounter { get; set; }
    public int Miles { get; set; }

    public bool ClearedSouthPass { get; set; }       //  REM F1 = FLAG FOR CLEARING SOUTH PASS
    public bool ClearedBlueMountains { get; set; }   //  REM F2 = FLAG FOR CLEARING BLUE MOUNTAINS
    public bool  ClearSouthPassSettingMileage  { get; set;}   //  REM M9 = FLAG FOR CLEARING SOUTH PASS IN SETTING MILEAGE

    public bool AreInjured  { get; set;}   //  REM K8 = FLAG FOR INJURY
    public bool AreIll  { get; set;}       //  REM S4 = FLAG FOR ILLNESS
    public bool HadBlizzard  { get; set;}  //  REM L1 = FLAG FOR BLIZZARD
}

And we start on those Event modules (using MEF !) next time.

The Oregon Trail Project - Part 2 - Dates

2023-11-27T00:00:00+00:00

Oregon Trail Project

Part 0 : Intro
Part 1 : Markdown to Spectre Console
Part 2 : Dates (this one)

After spending nearly three weeks getting installment #1 out, I figured I should get the next one out quickly, especially since Virtual Coffee’s Blogging Challenge have only a few days left.

Fortunately, the next topic is one I can crank out fast: Fixing how the BASIC code handled dates.

It goes like this: given an integer number of weeks, return a string of the date that many weeks from the start of the trek (27-Mar-1847), in the form MONDAY MARCH 29 1847.

In the original, it took quite a bit on rather monotonous code to handle this:

D3=D3+1
PRINT 
PRINT "MONDAY ";
IF D3>10 THEN 1300
GOTO D3 OF 1310,1330,1350,1370,1390,1410,1430,1450,1470,1490
GOTO D3-10 OF 1510,1530,1550,1570,1590,1610,1630,1650,1670,1690
PRINT "APRIL 12 ";
GOTO 1720
PRINT "APRIL 26 ";
GOTO 1720
PRINT "MAY 10 ";
GOTO 1720
PRINT "MAY 24 ";

That goes on for 50 lines — in a program that’s only 695 lines total — that’s over 7%.

Fortunately, .NET build date handling and format right into the framework. All of that can be replaced by:

public static string CurrentDate(int fortnights, int extraDays = 0) 
    => new DateTime(1847, 3, 29)
        .AddDays(fortnights * 14 + extraDays)
        .ToString("dddd MMMM d yyyy");

The Oregon Trail Project - Part 1 - Text (Markdig.SpectreConsole)

2023-11-26T00:00:00+00:00

Oregon Trail Project

Part 0 : Intro
Part 1 : Markdown to Spectre Console (this one)

Continuing my attempt to port the 1978 BASIC code for the Oregon Trail game to C#, the first step is quite simple. We need a way to display text.

The original game just had some basic text. Mostly just a sentence or less at a time. The largest block was the instructions and it’s not even 300 words. But I wanted to added some formatting to it – in a way that could be adapted to whatever output media this was eventually going to be used. That was going to be console text in this first version, but HTML or WPF versions might be in the future. I figured the best way would be to use MarkDown..

And Then I went down a rabbit hole

I started looking for a markdown component to output to the console. The problem here is that Markdown was intended to translate to Html, so a parser to do that is easy to find. It was also intended for formatting which just can’t be done on a text console. So, no one tried.

I did, however, discover SpectreConsole, which makes it easier to create beautiful console output.

So, I could get a really nice output, if I was willing to give up the possibility of non-console output – or I could keep looking for a markdown solution.

But then I discovered MarkDig, which is a parser for markdown, which has a number of renderers for output media (Html, Wpf, Xml), but more importantly, it’s essentially a toolkit for building a renderer for other media.

So, I just used Kryptos-FR’s Wpf renderer for Markdig as a model – really, I stole a lot from that project – and created a renderer for SpectreConsole.

For the most part, the translation was straightforward and simple. For example, a simple bit of markdown like This is *emphasized* becomes This is <em>emphasized</em> in Html (or, if you prefer, This is <span style="font-style:italic">emphasized</style>), in Spectre, in needs to be This is [italic]emphasized[/].

The tricky part was ordered and unordered lists. In Html (and Markdown), you just say “This is a list, and these are the items”, and the browser takes care of the details of indenting and bullets and sequential numbering and all that – all of which I had to handle directly in the renderer. Which led to another problem. The parser treats all the text of an item as a block, or, more specifically, as a paragraph. And paragraphs start and end with a newline. Now, browsers know to strip this whitespace off. But consoles are much more literal when it comes to spaces and new lines. And these line breaks were messing out the flow of the list:

    1.

This is the text for bullet #1

    2.

This is the text for bullet #2

It took me a while to figure out that the solution wasn’t in the list handling code, but involved changing the paragraph handler. Once I came to that realization, the fix was easy, which just shows how robust the MarkDig parser is.

And those pretty much handled everything I needed for Oregon Trail.

And the rabbit hole got deeper

While that was enough, I wasn’t quite satisfied. I really should to something for headers, but we can’t change the font size in text mode, but SpectreConsole did have a feature called Figlet which could be used. And both MarkDown and Spectre had Tables, and Code Samples.

But, the problem here is that those features didn’t use the markup style the other features did. Using the advanced features entails creating a object and rendering it, which didn’t really play well with the markup style. After playing around for much too long, I decided to give up of those, and come back to them later.

The code for this is available at: https://github.com/jamescurran/Markdig.SpectreConsole

The Oregon Trail Project - Part 0 - Intro

2023-11-07T00:00:00+00:00

You might notice that I don’t write much on this blog. My last entry was the written last Decemeber for the 2022 C# Advent Calendar, and I’ve already signed up for the 2023 version, and I had a idea for that.

Side note: While not writing to this blog, I missed the 20th anniversary of my starting it in August of 2003

But then, Virtual Coffee announced that this month’s Challenge, inspired by the NaNoWriMo (National Novel Writing Month) Challenge, would be a Blogging Challenge, where the members, collectively, try to write 100,000 words worth of blog articles.

Side note II: I’m writing this in VS Code, which has some ChatGPT based extension loaded. I paused for a second, and it provided me with this bit of text:

The Oregon Trial Project is a collection of Python scripts that I’ve written to help me understand the trial process in the state of Oregon.

This is not even close to what this is going to be about (But then, it is based on my original misspelling of “Trail” in the title.)

I came up with one idea for that (which I think I’ll write up after this – but post it first – just to keep you guessing), and was just going to leave it at that, when I recalled this project, which I had been thinking about for a few months, but hadn’t spend any actual time working on it. It seemed like it could make a nice serial, which would rack up the word count.

The Oregon Trail : Background

The Oregon trail, of course, was an actual historical route real American settlers used in the 1800s, but no one cares about that. The important one was a computer game used in schools in the 1970s & ’80s, to teach history (or something like that). It was apparently written in 1971, but didn’t go nationwide until 1978, when grammar schools were being given Apple II computers, and Oregon Trail was pretty much the only thing they had that ran on them.

Or at least, that’s what I’ve been told. By 1978, I was already in high school, and I missed all that. I didn’t hear about it until the 2000s, when I was quite surprised to learn it was some sort of cultural phenomenon. And it mostly involved dying of dysentery.

So, I was a bit delighted to find copy of the original BASIC source code on the Interwebs a few months ago. (OK, not the “original-original” – The one I found claims it was from 1978 (thanks to Jimmy Maher for posting it)).

The Project

The bad news is that I don’t have access to a computer running a HP Timed-Shared BASIC from the 1970s. The good news is that I learned BASIC back in the ’70s, and it’s simple enough to understand. So, I can read this source, but can’t run it.

So, the plan is to port it into a language/environment that I do have access to, specifically C#.

Now, I’ve been involved in a project very much like this before. Back in 2008, (could it really be 15 years already?) I was part of a contest on CodeProject to port a Basic version of a Star Trek game. Usually, when someone attempts something like this, they usually just work on getting a modern compiler to accept old code, by tricking it into forgetting 40 years of software development techniques.

You see, back then Basic didn’t have a concept of functions/methods, let alone objects. (Neither did Fortran or COBOL or most other languages at the time) Basic had a keyword GOSUB for calling “subroutines” but they had only the barest resemblance to what we now call a function. Particular, there was no way to pass arguments to a subroutine, nor a way of return a value. The work-around for this that we all accepted was using global variable to handle those tasks. Further, most Basic interpreters only allowed one statement per line. These limitation, plus a few others, led to a code-style known as “spaghetti code”.

So, when people wanted to port a basic program to modern language, they just made a single class with a single method with hundreds of local variables, and overuse the goto command which most languages still have (but don’t talk about). An example of this can be seen in one of the other entries in the Star Trek contest (Star Trek 1971 Text Game).

This always impressed me as a big waste of time. If I was going to port a program to a new language, I was going to take advances of all that language had to offer. For the Start Trek contest, I completely rewrote the code, giving it a modern Object-oriented design — but, retaining the exact same look and game play. You can see that here: The Object-Oriented Text Star Trek Game in C++.

That’s my goal for this project: Oregon Trail, exactly as people remember it, but a completely new (and better) design under the covers. Further, I want to separate the actual processing in its own assembly, apart from the UI, so that it can be uses to port the game different clients. (There may be a Blazer version in the future.)

As the series progresses, I’ll be taking you through this process…

What is (was) Beta Software?

2023-11-05T00:00:00+00:00

For Virtual Coffee’s November Blogging Challenge

I just wrote another blog article which I’ll post tomorrow, which explains a bunch about this, but since I decided to slip this one in first, you’ll just have to wait for it.

What is Beta Software, and Why Do We Call It That?

These days, when we refer to “beta” software, it’s generally means some half-baked, buggy, pre-release version of the code, where the developers are still working on it. This is a bit different from it’s original meaning.

It started in the dim dates of computing, in the 1960s & ’70s. Back then we didn’t have the fancy graphical user interfaces on our software as we do now. (I use “we” and “our” to refer to our collective consciousness – I’m not quite that old.)

In those days, programs were more like what we’d call a “console” or “command-line” program today – although that still implies the existence of a video display, which there wasn’t. These would be loaded from a stack of punch cards.

Lacking interactivity, programs tended to be single-purpose. You run it, it does something, it reports what it did (which came via a printout, often came off the printer hours after the program was actually run). Also, there wasn’t much retail software. Most was either included as part of the operating system, or written in-house.

In-house software didn’t have the formal QA process we have today, as the users were generally the authors of the code. But it’s always good to have a second set of eyes looking at something, so if someone else needed that functionality, they could often get a copy of the code, with the promise that they would report back any problems. This became known as the “Beta site” for the code — the second place the code was used. The author’s workplace would, in theory, be the “Alpha site” but no one called it that.

The point is that the code that went to the beta site was the complete, final version of the program. If someone at the beta site find and reported a bug, it might be fixed in version 2 – if there was a version 2. (We hadn’t got to point-versions yet.)

It was only is the packaged software era, which basically started when we moved from mainframes to PCs in the ’80s & ’90s, and particular as we moved to graphical interfaces – where the user interface, separate from the basic function of the application, grew in importance, requiring design input from many people, and being a source of many bugs, did the idea of release unfinished software to a select group of people rise. And this is when we coined the back-formation “beta software” for this half-baked code that went to the beta site.

A Literary Clock in Blazor

2022-12-13T00:00:00+00:00

This post is for day 13 of the 2022 C# Advent Calendar operated by Matthew Groves. Thanks for letting me participate!

When I last blogged (um.. four months ago…), we discussed putting time-related quotes into your PowerShell prompt (Click the left arrow at the top of the page to see that one), and I mentioned a Blazor version. It’s time to present that.

Some background: back in 2011, The Guardian built a list of literary quotes containing a time of day, from reader contributions, and made the lists publically available. They been re-used on a number of projects.

One such project is https://github.com/JohannesNE/literature-clock, which uses Javascript to present the appropriate quotes on a webpage. The repository also maintains a pipe delimited list of all the quotes, and has a program written in R to read that file and generate individual json files for each minute.

That’s all well & good, but I do C#, so I figured I’d port that in Blazer (while making the R program into a C# console app).

After that app runs, for each minute (well, for 958 of the 1440 minutes in a day), there’s a json file with a name in the form “HH_MM.json”, which holds an array of objects which look like this:

Not sure how fast the R version is, but on my PC, the C# version takes 13 seconds to produce the 958 files.

So, let’s look at the Blazor page, a piece at a time. We need to do something once every minute, so on start, we set up a timer, which calls the method which does the real work:

protected override  Task OnInitializedAsync()
{
	_timer = new Timer(obj => OnTimerCallback(), null, TimeSpan.Zero, TimeSpan.FromMinutes(1));
	return Task.CompletedTask;
}
private void OnTimerCallback()
{
	_ = InvokeAsync(() =>
	{
		UpdateText(DateTime.Now);
	});
}

The reason we pass in the time will become clear in a minute. In UpdateText, we read the quote file for the passed time:

protected async  void UpdateText(DateTime time) 
{
	var strTime =time.ToString("HH_mm");
	Quote[] quotes = await Http.GetFromJsonAsync<Quote[]>($"times/{strTime}.json") ?? Array.Empty<Quote>();

But, since not every minute has a quote, if we don’t find any for the current minute, we’ll hold on to and use the previous minutes – or, if we don’t have any already (i.e., we just started), we just go backwards in time until we find one:

	if (quotes.Any())
		_quotes = quotes;
	else if (_quotes == null)
	{
		UpdateText(time.AddMinutes(-1));
		return;
	}

Then we just randomly pick a quote from the array

	_quote = _quotes[Rnd.Next(quotes.Length)];

And bind it to the page:

	<div id="main_text">
		<blockquote id="lit_quote">@_quote.Prefix<em>@_quote.Quote_time_case</em>@_quote.Suffix</blockquote>
		<cite>
			-
			<span id="book">@_quote.Title</span>,
			<span id="author">@_quote.Author</span>
		</cite>
	</div>

The javascript version includes a formula to adjust the font size to accomodate the widely differing quote lengths. This is easily transaled into C#:

	var quote_len = _quote.Quote_first.Length +
	                 _quote.Quote_time_case.Length +
	                 _quote.Quote_last.Length;

	_fontSize = 7.000864 - 0.01211676 * quote_len + 0.00001176814 * quote_len * quote_len - 1.969435e-9 * quote_len * quote_len * quote_len;

And Blazor:

    <style>
    	#lit_quote {
    		font-size: @(_fontSize)vw
    	}
    </style>

You can see the clock in action (along with two other online clocks of mine) here: https://noveltheory.com/clock/

The full project repository (which includes a folk of the original Javascript version) is here https://github.com/jamescurran/literature-clock.

Putting quotes in your Powershell prompt.

2022-08-13T00:00:00+00:00

I just discovered my friend Meg Gutshall has a blog, and was reading her latest entry (from last year – she updates hers about as often as I update this one): A Random Quote Generator for Your Terminal.

It displays a quote as part of the prompt in a BASH shell. I thought this was a cool idea, but I don’t use BASH – I use PowerShell, so I figured I port hers to PS. I’d also found this project on Github (Literature Clock) which collects quotes based on the time (e.g., “At 7:32, he suffered a fatal stroke,” IT, Stephen King), and I thought it would be cool to use those.

But, I’m getting a bit ahead. Let’s first talk of changing the PowerShell prompt. The prompt is produced by a script which is held in a file named Microsoft.PowerShell_profile.ps1, which, on my system is located in the WindowsPowerShell\ subfolder of my Documents folder. I’m not sure how standard that is, but try looking there for it.

The default prompt is the current folder name followed by a “>”. This is fine if you have short folder names, but no one has short path names anymore. So now it looks something like:

PS C:\Users\James\Source\repos\Tests\MyBlazorApp1>

And you’ve already used half the line before you typed anything. The simplest command wraps to the next line. That’s too messy for me. So, I first put the current directory on it’s own line, and then threw in the current date & time for good measure:

C:\Users\James\Source\repos\Tests\MyBlazorApp1
12-Aug-2022 18:52:29
PS>

(The path is in magenta and the date in green, but I don’t seem to be able to color text in markdown.)

Ok, now we move to adding the quotes. The Literature Clock project has a large collection of time-based quotes from books, organized by minute. They have at least one quote for 950 of the 1440 minutes in a day. Some are duplicated if it’s unclear if the time refers to AM or PM. And while many minutes have no quote, others (like midnight or 3 o’clock) have dozens. The quotes are kept in a CSV (actually pipe-separated) file. The project includes a javascript program to display the clock on a webpage, and a script written in the R language to convert the CSV file into a collection of JSON files. It’s the json files that the javascript and this project will use.

I’ve also ported the clock webpage into Blazor (as well as port the R program into C#) so that might be a future blog story.

For each minute, there’s a json file with a name in the form “HH_MM.json”, which holds an array of objects which look like this:

So, our steps will be: Get the time, format it like the filename, read the file, deserialize it into an array of objects, pick one randomly, and display it. Except for the last step, the PowerShell is just as straightforward as the description:

$quote = get-content "C:\Users\zames\OneDrive\Documents\times\$(Get-Date -Format "HH_mm").json"  | ConvertFrom-Json  | Get-random

The rest just is formatting and coloring it to look nice. Full script here:

TaskCollection - Making your async code parallel

2021-12-17T00:00:00+00:00

When I wrote an article for the 2021 C# Advent Calendar on Tuesday, I promised to post a couple days in a row.

That didn’t work out.

But at least I eventually got to it.

The async\await keyword added to C# a few years ago greatly simplified use asynchronous code in our applications. However, asynchronous code is inheriently tricky, and in trying to make it simple, async/await sometimes undermines itself.

Consider the code:

_cacheCustomers = await LoadTableAsync("Customers");
_cacheProducts = await LoadTableAsync("Products");
_cacheOrders = await LoadTableAsync("Orders");

Fairly typical code to preload three tables from a database that we are going to use. Three separate round-trips to the data server with much waiting for the response from that server. So the perfect situation for those actions to be run in parallel, which is why we used the ...Async version of the method, and used await.

Note that LoadTableAsync is just generic example representative of long-running non-CPU bound method. Could be a database, could be a Web API, etc.

But they aren’t going to be run in parallel, because await means, well, await. Wait for the first call to LoadTableAsync to complete before we start the second call. Now, granted, something will be run at the same time as each of those calls, but since you probably have awaits going all the way up the call chain back to the Main() function, it’s probably going to be the operating system. But, importantly, it won’t be the other calls to LoadTableAsync.

What we need to do is start them all first, and then wait for them to finish. Something like:

var taskCustomers = LoadTableAsync("Customers");
var taskProducts = LoadTableAsync("Products");
var taskOrders = LoadTableAsync("Orders");

And then, at some point later,

_cacheCustomers = await taskCustomers;
_cacheProducts = await taskProducts;
_cacheOrders = await taskOrders;

or alternately,

Task.WaitAll(taskCustomers, taskProducts, taskOrders);
_cacheCustomers = taskCustomers.Result;
_cacheProducts = taskProducts.Result; 
_cacheOrders = taskOrders.Result;

The advantage of this is that between the LoadTableAsync()s, and the waits, you can put some cpu-bound actions, so that when you get to the waits, your data is already waiting for you. The database calls become free!

Now, this is all well & good, but we’ve added a bunch of lines of code, and that gets messy, particularly if the return value of the method isn’t being used. Consider:

await FillListFromTableAsync("Customers", _cacheCustomers);
await FillListFromTableAsync("Products", _cacheProducts);
await FillListFromTableAsync("Orders", _cacheOrders);

To make these parallel, now we have to create the local task variables, just to use the the Task.WaitAll. And if you have more than three, it really starts to get out of hand.

Which brings us to TaskCollection, a simple class to manage that for you.

var tasks = new TaskCollection();
tasks += FillListFromTableAsync("Customers", _cacheCustomers);
tasks += FillListFromTableAsync("Products", _cacheProducts);
tasks += FillListFromTableAsync("Orders", _cacheOrders);

and then,

_logger.LogInformation($"{tasks.Count} tasks queued, waiting on {tasks.Running}");
tasks.WaitAll();

It includes the properties: Count - total number of tasks in the collection (running and completed) Running - number of tasks still running.

And the methods: RemoveCompleted - Removes completed tasks from internal list. Afterward, Count == Running. WhenAll - Returns a task which completes when all tasks on the list has completed.

Full code given here:

TimeThis! - Simple timings for code blocks (Fun with IDisposable)

2021-12-14T00:00:00+00:00

This post is for day 14 of the 2021 C# Advent Calendar operated by Matthew Groves. Thanks for letting me participate!

After I reserving my spot for this year, I realized two things:

I forgot to sign up last year, after participating the first three years and
Largely due to that, I haven’t written anything to my blog in two years!

So, to make up for that, I figured, “Let’s post a couple in a row”. So, if you like this, check back here tomorrow and Thursdays for more!

Sometimes, you want to know long a bit of code it taking to run. Maybe not a full code analysis, but a read-out of how many second something is taking, particular if your not sure which section is the problem, or if one section varies a lot from run to run.

Doing this in C# is rather straight-forward : Just create and Start() a Stopwatch object before the code you want to time:

var sw = new StopWatch();
sw.Start();

And then afterwards, Stop() the timer, and report the results:

sw.Stop();
Console.WriteLine($"It took {sw.Elapsed}");

(Plus you have to remember to include “using System.Diagnostics” in your code.)

Two simple steps, but that’s the problem – Two steps, with your code in between, which makes it tricky to package into a neat library function.

One could make it into a method which takes a lambda, which would be the code you’re trying to time, but that messy – all local variables used need to be passed in, and all outputs need to be passed out. And now you’re changing the code you want to test.

Now, bak when I was wrting C++, this was simple. Just create a class, which did the setup in its constructor, and the reporting in its destructor. We create an object of that class where we want to start it, and it is automatically destroyed (and reports the results) when it goes out of scope.

Unfortunately, in C#, we don’t have destructors that work like C++’s. But we do have the IDisposable.

using (var tt = new TimeThis(_logger, "My Complex code"))
{
    // my complex code here
}

This will add to your logs something like this:

Complete:My Complex code: Elapsed: 00:00:01.3521282

The message (“My Complex code” in the example), is optional. That’s nice, but we can make it a bit neater with an extension method.

using (_logger.TimeThis())
{
    // my complex code here
}

Here’s the full code.

Cleanup Scaffolded Code with modelbuild

2019-12-12T00:00:00+00:00

This post is for day 12 of the 2019 C# Advent Calendar operated by Matthew Groves. Thanks for letting me participate!

Now, this isn’t specifically about C#; it’s more directly about .NET Core and Visual Studio tooling, but we use C# in it, so it counts….

EntityFramework would really like you to use the “code first” approach to defining your database. In fact, EntityFramework.Core seems to have removed support for the “Database First” method. Unfortunately, in my experience, in at least 90% of projects, the database already exists, and we have to work with that, and reverse engineer Code First classes from our database.

Fortunately, .NET Core offers us the Scaffold-DbContext command of the EF Core Package Manager Console to do that reverse engineering for us. It will generate a DbContext-derived class for the database, plus one file (with one class) for each table, defining the fields. But this article is not about that, you can read more about it here.

Unfortunately, the code for all the other details about each table — Keys, indexes, relationships – are put entirely in the OnModelCreating() method of the DbContext class.

There are two problems with this: First, any reasonable database will probably have a couple dozen tables, meaning that method will be several hundred lines long — I never want to have a method that long. More importantly, the properties and the other data are both equal parts of the schema, and they should be recorded in the same place (or at least the same file).

So I devised a plan to do just that. For each table/class, I’d create a secondary static class holding just a single extension method. I’d move the schema defining code for that table into the extension method, and then call it from the dbcontext’s OnCreate. That leaves the OnCreate with essentially just a list of the tables in the database.

However, writing the boilerplate code for this got rather tedious, especially since the rest of the work was mostly just cut’n’paste. So I figured I’d create a code snippet to provide the framework.

Do use, crate a file, modelbuild.snippet, out of the gist, then install it as a code snippet: Tools/Code Snippet Manager…/Add.

Then, in a source file for one of the classes for a table, just before the final closing brace (closing the namespace), type modelbuild, [tab] tablename [enter], and Visual Studio will fill in the boring part, then you just have to cut’n’paste from the DbContext class to this class, and replace the code with modelBuilder.Configuretablename()