Tales from the Evil Empire

Secrets of the Grey Beards: drawing circles with additions and shifts

Sat, 30 Dec 2023 20:47:05 GMT

A long time ago, before GPUs, before multicore processors and gigabytes of RAM, clever algorithms were the only way to render complex graphics in real-time. Those algorithms are now mostly forgotten as nobody needs them or is interested in them. Nobody? Well, almost. I like to reverse-engineer the clever tricks programmers of that era were using (in fact, I was one of them), and I hope you'll follow me on this exploration.

In this post, I'm going to show you algorithms that solve the seemingly simple task of drawing a circle, and we'll do so with very strong constraints on the operations we're allowed to use. In the early days of video games, the late 70s and early 80s, the processors in home computers and game consoles were mostly 8-bit processors such as the 6502 and the Z80. Those processors were not only slow by today's standards with frequencies around 1-2MHz and no parallel execution of any kind, they had very limited instruction sets, without multiplication and division operations, and with only support for integer numbers. That doesn't mean that multiplications were impossible, but just that they were very, very expensive. As a consequence, algorithms avoiding them were preferable.

Operations that 8-bit processors were very good at running fast, because they can be implemented with few simple logic gates, include additions and bit shifting.

The way mathematicians would describe circles is often with equations such as $x^{2} + y^{2} = r^{2}$ or ${\begin{cases} x = r \cos θ \\ y = r \sin θ \end{cases}$ .

Both of those involve multiplications, or worse, trigonometric function evaluation. So what are we to do?

The first thing we can notice is that the first equation is quadratic, so everything in there, once differentiated, becomes linear, and as such, easy to compute using additions and multiplications by constants. Let's keep that differentiation idea but it's the second system of equations that I'm going to differentiate, to come up with the first algorithm:

${\begin{cases} ⅆ x = - r \sin θ ⅆ θ \\ ⅆ y = r \cos θ ⅆ θ \end{cases}$

We can then notice that we can spot the expressions for $y$ and $x$ in those differential equations and reformulate them as follows.

${\begin{cases} ⅆ x = - y ⅆ θ \\ ⅆ y = x ⅆ θ \end{cases}$

What that tells us is that when the angle $θ$ changes by an infinitesimally small amount $ⅆ θ$ , $x$ changes by $- y ⅆ θ$ and $y$ changes by $x ⅆ θ$ . This still contains a multiplication, but only by the constant amount we want to vary the angle between two steps of the rendering. If we choose that amount to be the inverse of a power of two $\frac{1}{2^{n}}$ so that the corresponding change in coordinate is always under one pixel, thus ensuring continuity, we can avoid the multiplication because multiplication by the inverse of a power of two is the same as shifting bits to the right by a number of bits that is that power: x/2^n is the same as x >> n.

We still have the problem of determining the value of $n$ , but we only have to do it once per circle, so this is less of a hot path. It turns out, all you have to do is shift 1 left (which is the same as enumerating powers of two) until you run over $r$ :

And this is it, we now have a working algorithm:

// Now we can start from (r, 0) and apply the differential equations at each step let x = r << bdp, y = 0, t = 0; while (x >= y) { // We only need draw 1/8th of the circle, symmetries will do the rest const [screenx, screeny] = [x >> bdp, y >> bdp]; drawPointAndSymmetries(ctx, xc, screenx, yc, screeny); // Side effect: it's trivial to derive a table of sine and cosine values // from this algorithm while we're here... sine[t] = y; cosine[t] = x; t++; // Apply the differential equations [x, y] = [x - (y >> n), y + (x >> n)]; }

The call to drawPointAndSymmetries is just using the axial symmetries of a circle to only compute one eights of the circle:

function drawPointAndSymmetries(ctx, xc, screenx, yc, screeny) { // Draw the current pixel and all its easily computable symmetries ctx.fillRect(xc + screenx, yc + screeny, 1, 1); ctx.fillRect(xc - screenx, yc + screeny, 1, 1); ctx.fillRect(xc + screenx, yc - screeny, 1, 1); ctx.fillRect(xc - screenx, yc - screeny, 1, 1); ctx.fillRect(xc + screeny, yc + screenx, 1, 1); ctx.fillRect(xc - screeny, yc + screenx, 1, 1); ctx.fillRect(xc + screeny, yc - screenx, 1, 1); ctx.fillRect(xc - screeny, yc - screenx, 1, 1); }

If you're wondering about bdp, this is the fixed point bit depth that we use to make decimal calculations using only integers. It's 8 in the sample below, meaning eight bits of precision under the decimal point.

This works fine, but we can do better and faster.

Imagine we wanted to walk the circle in the dark. If we could erect a wall around the circle, we could just walk straight ahead and whenever we hit the wall, take a lateral step away from it. This strategy produces a trajectory that is very close to the circle. It also works on any concave shape, not just circles.

The wall in question is called a discriminating function. It's a function of the coordinates that evaluates as positive outside of the shape and as negative inside. It's called discriminating because by just evaluating it on a point, you can discriminate whether you are inside or outside.

For a circle, it's easy to build a discriminating function from the circle's equation: $f (x, y) = x^{2} + y^{2} - r^{2}$ .

It may seem like computing this function for every point is going to be challenging because of all those squarings. In this case again, variations of the function from pixel to pixel are going to be sufficient if we know the value at any point.

When we move one pixel to the right, here's how the function varies:

$f (x + 1, y) - f (x, y) = 2 x + 1$

This is of course very easy to compute: shift $x$ left and add one.

Whenever the new value of the discriminating function goes positive, we need to make a side step, for which it is also easy to compute the discriminating function variation:

$f (x, y - 1) - f (x, y) = 1 - 2 y$

Subtract left-shifted $y$ from one.

You may also notice that this second algorithm has no need for fixed-point decimal numbers, it can work on regular integer coordinates, which further simplifies it and makes it more efficeint.

Here's the implementation:

// Start with a value half a pixel inside the circle: f(0, r - 1/2) = 1/4 - r and 1/4 falls below precision. let x = 0, y = r, f = -r; drawPointAndSymmetries(ctx, xc, x, yc, y); while (x <= y) { // We only need draw 1/8th of the circle, symmetries will do the rest // Draw the current pixel and all its easily computable symmetries f += 1 + x << 1; x++; drawPointAndSymmetries(ctx, xc, x, yc, y); if (f > 0) { f += 1 - y << 1; y--; drawPointAndSymmetries(ctx, xc, x, yc, y); } }

You can see both of those algorithms in action below:

The source code can be found here: https://github.com/bleroy/3d-junkyard/tree/main/AddAndShift

But of course, this is javascript running on a modern computer, and I sold this as a way to draw a circle on vintage 8-bit computers. Does it work on old computers? Yes, of course it does. Here's the algorithm implemented and running on an Atari 8-bit computer in 6502 assembly (what you see below is an actual emulator in your browser, hosted by the excellent https://8bitworkshop.com; if you see the circle as a little oval, that's normal, vintage computers often have pixels that are not exactly square):

Here's the source code for this:

Use explicit Lambdas with LINQ

Wed, 30 Aug 2023 22:05:00 GMT

Here's an interesting bug... What's wrong with this code?

In principle, nothing. Your IDE might even encourage you to write this instead of the longer (but also more expressive):

Let me give you a couple of hints... First, the second version, the explicit one, works perfectly well, while the first one behaves very oddly in some contexts. Second, here's the signature for the Clone method:

Here's the key: LINQ's select method can take basically two signatures for its argument function. The first one is a simple mapping that takes a single parameter. The second takes the source item as its first argument, but has a second argument that is the index of the current element being processed in the source enumeration.

Now surely you can see how this causes a subtle bug in the first version of our code. Seeing that our method has an overload that takes two argument, the second being an integer, the compiler thinks our function needs the index as its second argument. Essentially, the compiler interprets our code as this:

As the code runs, our Clone method will get called with 0, 1, 2, 3, etc. as its max depth parameter, resulting, presumably, in very shallow, then deeper and deeper clones being made. Very strange behavior that will be extremely difficult to track down...

For those reasons, here's my recommendation: just be explicit, use the second version. Never pass the raw method as the argument to Select (and other LINQ methods), explicitly write a Lambda instead.

A breaking change in .NET argument exception message formatting

Wed, 01 Mar 2023 00:55:42 GMT

A quick reference post about an interesting change in the way .NET formats argument exception messages. This tripped me up when debugging a test that was failing on .NET Core / 6.0 whereas it had been passing forever and still passes on .NET Framework / 4.7. The test was expecting a specific error message. Because .NET Core apparently has changed the format string used for argument exception messages, that expectation was broken.

Technically a breaking change, but then again expecting a specific error message is usually not the best idea.

Anyway, here's how .NET 4.7 formats the exception:

And here's how .NET 6.0 does:

The fix for the test of course is to test for the presence of the message that was passed in when creating the exception as a substring of the exception's Message property, instead of expecting the whole formatted message.

3D before GPUs Part 1: Dungeon Master

Thu, 29 Jul 2021 01:49:00 GMT

In this series, I'll reverse-engineer algorithms from video games dating back to that time when the CPU was all you had. Today, we're looking at Dungeon Master, a fantastic game by FTL that set the gold standard for RPGs for the years to follow. It looked amazing, and still does to this day. It changed everything.

Eye of the Beholder and Ultima Underworld are examples of later games that owe almost everything to Dungeon Master.

Before Dungeon Master, dungeon crawlers existed of course, but were often limited to crude wireframe graphics.

The way Dungeon Master rendered the walls of its dungeons was by cleverly overlaying and clipping bitmaps for the floor, ceiling and walls. There were of course many other objects to compose, such as monsters, doors, fountains or scrolls, but for the purposes of this post, we'll limit ourselves to the corridors and leave the rest to your imagination.

For this, we need only one tile for the background, ceiling and floor, that will be the background that we'll fall back to when no other object is in front:

We also need three tiles for the front wall at the three distances where the wall is still visible: the dark corridors of Dungeon Master are dimly lit, and everything farther than 3 units conveniently fades into darkness...

Also note that these wall textures don't need to be the full width of the screen, they just need to tile, so repeating the texture appears seamless.

Then we have one tile for the left wall and one tile for the right wall, that we'll clip as needed:

And finally we need two tiles for the second row of walls on the left and right that are sometimes barely visible in the distance:

Here's a drawing of what those panes correspond to, when viewed from above:

The HTML for the whole setup is extremely simple:

Each potentially visible pane of wall is represented by a div with classes enabling their precise targeting by CSS. The divs are ordered from back to front so we don't have to worry about depth and closer walls naturally occlude the ones behind them.

CSS is then used to dimension, position, texture and clip each pane. Here's for example the style for the middle-left wall:

JavaScript is used to make panes visible or hidden based on the map data for the level:

We first peek at the map locations in view of the party, then set the visibility of the corresponding wall panes.

There's one last trick that Dungeon Master uses to make its levels feel more real at a very low cost: every time the party moves, all textures are flipped horizontally to their mirror image. This is important because it gives the illusion that something changed without costing anything since the images stored in memory are exactly the same, just rendered flipped. The textures have actually been designed so that what you see one square away looks like the mirror image of what's immediately around you. So flipping the textures gives a semi-realistic illusion of movement despite the fact that everything moves square by square rather than progressively and smoothly.

In my reverse-engineered web version, I was able to reproduce this very easily by toggling an additional CSS class, "alternate", on the parent viewport element, every time the party moves or rotates:

The CSS for the alternate viewport just applies a transform that applies a horizontal scale of "-1". Tile elements have a reversed version that adjusts the clipping of the texture if necessary:

That's pretty much all you need to reproduce the rendering of Dungeon Master walls and get a result that is very close to the original.

In the end, Dungeon Master's rendering of its levels is about as simple and low-cost as it gets, but still manages to create an immersive and great-looking environment.

Here's the final result, that you can use from your browser, right here (the level is that Hall of Champions, the first level of the game):

Go ahead, click on the arrows and play, it really works from this post!

If you want to see some gameplay from the original, here's a video I recorded a few weeks ago where I go through the first couple of levels of the game, and proceed to die a horrible death:

Like and subscribe!

In the next post, I'll look at Rescue on Fractalus, the first game from LucasFilm that introduced a very innovative fractal rendering technique for mountain ranges.

The full code for this demo is on GitHub: 3d-junkyard/DungeonMaster at main · bleroy/3d-junkyard (github.com)

LunrCore, a lightweight search library for .NET

Sun, 26 Jul 2020 09:02:08 GMT

I'm pretty much convinced almost all applications need search. No matter what you're building, you'll likely handle data, and no matter how well you organize it, a good text search is often the fastest way for your users to find what they're looking for. As such, search should be a commodity, a feature that should be as easy as possible to integrate. I'm so convinced of that in fact that my day job is on Azure Cognitive Search, a Microsoft product that provides search as a service and makes indexing smart by adding a customizable pipeline of AI and machine learning enrichments.

A month ago, I saw this tweet:

The .NET library ecosystem is sometimes so unequipped. Every 5 years, I'm looking for a .NET library that could build a search index (inverted index, with stems, fuzzy search...etc.) that I could plug into a JS frontend library (e.g lunr.js), and it's still the same no lib's land
— Alexandre Mutel (@xoofx) June 24, 2020

Being a user of lunr.js myself (I used it as the search engine in my Node CMS project, Decent CMS), I agreed totally, and felt I had to do something. Of course, .NET has Lucene.NET, a port of Java's Lucene, and that's a fantastic library. We've been using it forever in Orchard, and it's a high-performance and mature project. It is however not that simple and it can in some situations lack flexibility. It's what your choice should be if you're building anything at scale, but if all you're trying to do is to plug search into your app with minimal headache and a very gentle learning curve, now you have LunrCore as an option...

LunrCore is a direct port of lunr.js. If you don't know it, lunr.js is a very nice JavaScript library that implements a simple search engine with a focus on simplicity. I really love it because it hits that sweet spot of commodity that I was talking about at the beginning of this post. Easy to integrate, easy to use, and still has almost all the features you'd expect:

Index anything: just feed it a dictionary of strings to objects
Stemmers
Fuzzy search with edit distance and wildcards
Boost
Term presence or absence
Scoring
Stop words
Search over fields or the entire document
Pluggable indexing and search pipelines

Creating an index is as simple as this:

Then, searching it can be done through expressive search query strings or through an object model of combinable clauses.

Being a port of an existing JavaScript library, we also have an additional trick up our sleeves: an index can be prepared on the server, then sent down into the browser for super-fast client-side searches. The JSON that LunrCore produces when serializing an index can be consumed by the original lunr.js library. I think that is pretty cool if I may say so myself.

The library is of course open source under the MIT license, which means everybody should be able to use it to build commercial and non-commercial applications with no strings attached. It has no runtime dependency beyond the SDK, BCL AsyncInterfaces and System.Text.Json, making it even more of a no-brainer to integrate anywhere. I'd love to hear what you think and what you built with it, so don't hesitate to drop me a note (or even contribute something on GitHub).

Package: https://www.nuget.org/packages/LunrCore

Repository: https://github.com/bleroy/lunr-core

The original JavaScript library: https://lunrjs.com/guides/getting_started.html

Why I dislike tuple return types

Sun, 16 Feb 2020 23:53:00 GMT

Tuples are great additions to C#. They're simple immutable structures made of a small number of members organized in a specific order. For example, a point on a plane could be represented as a pair of X and Y coordinates, or a person's name could be represented as a title, first, middle and last names. If the types are that simple, and have no custom logic associated with them, using a tuple looks like a lightweight and simple implementation that requires a lot less effort than a simple class or struct. Dynamic and functional languages have long enjoyed tuples, of course, but I like how the C# implementation manages to stay true to the core tenets of the language, in particular type safety.

However, I personally cringe a little every time I see a public API returning a tuple. I really, really dislike them as return types. Let me explain why.

As a user of a method, you will use what you know of the return type, and use the data contained therein according to your needs. There's a whole category of backwards-compatible changes to a class, such as adding a new member, that are trivial with classes and require no change or even recompilation of client code, that become very difficult with tuples.

This comes from the fact that with tuples, you need to copy the very definition of the type everywhere you use it, whereas with a class, you only use its name as a reference to the type. Almost no refactoring is possible with tuples without finding all of the code in the world that uses your tuple return type, and modifying it as well. This is at best impractical. Tooling can't even help you much beyond CTRL+SHIFT+F and finding all references to the method. I know from experience that tracking down those types in a large codebase can be far from trivial.

It's actually less effort in the end to write a simple class. I have a lot of hope for records (vote) as a better option to use wherever you need lightweight types, but in the meantime, I prefer in most cases to write a simple class rather than to return a tuple.

There's also the fact that a named type can in most cases better convey the semantics and intentions of your code than a tuple, which is pure structure. Finally, the temptation to write Cthulhuesque abominations such as IEnumerable<(string, IDictionary<string, (string, string)>)> exists. When you have to put the name of your method on a different line than its return type, you know you've gone too far.

So at this point, you may ask, what do I still think tuples are good for? That is a fair question... I have several answers to that.

First, I don't think tuples should always be avoided as return types. When the semantics of a method are to manipulate or produce a tuple, they are more than acceptable, in fact nothing else makes sense:

I think tuples are also fantastic at simplifying code that produces and processes intermediary results. I particularly like to use them as return types of Lambdas in LINQ expressions:

This totally non-optimized, allocation-happy and non parameter validating piece of code conveys the idea: it uses a simple tuple as an intermediary structure between the Zip call and the SelectMany. The tuple brings real value here, as doing the same thing without it would be considerably more tedious and less expressive (let's put aside the existence of a Pair class that could be used in this simple case, the general point remains). The proximity of the code that produces the tuple to the code that consumes it, associated with a tight scope means none of the objections I have to using tuples as public API return types apply here. It's pure goodness.

Another very compelling feature of C# tuples is deconstruction, and I wish we also had it for anonymous objects like JavaScript does. Deconstruction allows you to directly assign values inside of a tuple that only lives as part of that assignment. It is a very convenient way to access and use the components in the tuple:

This is really lovely, and the good news is that tuples are not the only types this works on: you can add type-safe deconstructing capabilities to your own classes by implementing something like this:

The GetMiddlePoint deconstruction sample above will work exactly the same whether the method returns a (double, double) tuple or an instance of a Point class that implements a Deconstruct method. This is a great way to refactor a tuple-returning method to return an instance of a class instead, while keeping client code untouched (recompiling may be necessary). To me this is the best of both worlds: a deconstruct-aware class remains easy to refactor cleanly, and can still present the greatly expressive deconstruction capabilities of a tuple.

The case of the defined undefined property

Wed, 26 Dec 2018 16:00:00 GMT

I like JavaScript, for some reason, I really do, and I still write and maintain a few open source JavaScript projects. It’s undeniable that it has bad parts though, that remain today, even in strict ES2017. In this post, I want to show you one that builds an interesting bug farm.

Being a dynamic language, JavaScript allows you to add properties to pretty much any object, on the fly. This can be done using the dot notation (a.b = 1), or the indexer notation (a['b'] = 1). The indexer notation is more verbose, but it also allows for more flexible scenarios. For example, you can use the value of a variable or function parameter to set a value on an object:

That looks innocuous, but what would happen if we put JavaScript’s careless tendency to convert anything on-the-fly into the mix? What would happen if we used an undefined variable as the index? Let’s find out.

Ouch! That looks bad. So what happened here? Well, the value of b here is undefined, but JavaScript needs a number or a string in this context, so it converts your undefined value to the string ‘undefined’ and then uses that as the index value. When we try to retrieve the value of the property indexed by c, the same conversion happens, and we retrieve the same value.

Seems far-fetched? Check this one out:

Notice the spelling on lines 15 and 16. This is a much more likely case where a typo in the program goes undetected and caused unexpected results. The consequences can be very serious, depending on the context. In normal conditions, everything seems to work fine. You set a property, retrieve it, and the value is the one you set. This is an illusion however, because you’re creating that bucket of undefined properties that all leak one another’s value. You might set a value for one user, for example, in some cache in a Node application, only to have it propagate to another user. That is bad™.

Any bug is bad, but a bug that can go undetected until it causes mayhem is worse. So what can you do to detect something like this? Well, maybe JavaScript’s strict mode should error on implicit conversion of undefined to string: I can’t think of a situation where you would intentionally do that, and even if there’s one, being explicit about it would be better. Since it doesn’t, however, one thing we can do is build a little trap.

Here’s a bookmarklet that you can bookmark to set-up the trap (keep in mind your browser will likely not let this run just anywhere, like on https, but you can always run the same code in the F12 console):

undefined trap

Here's the source code for it:

Once you’ve executed the bookmarklet by clicking on the bookmark, the active web page in your browser is now instrumented so that if any object property is set with undefined (or the string ‘undefined’), the browser will break execution into the debugger. Go up one level in the stack trace, and you’ll see the responsible code. It may not be the ultimate answer to this problem, but it may help find a few hidden and nasty issues in your client-side code.

Quantum computing and topological qubits explained clearly

Mon, 30 Apr 2018 18:16:00 GMT

Don't let yourself be intimidated by all the quantum jargon. The bases of quantum computing are not that complicated, and I can explain them to anyone who understands programming, classical logic gates, the bare minimum about complex numbers and linear algebra… I'll do so in the light of Microsoft's recent announcement of a new discovery that could bring us much more stable quantum computers.

But first, a disclaimer: I do work at Microsoft, but I don't work on the quantum computing team. I'm just an enthusiast developer who happens to be trained in quantum mechanics.

What did Microsoft just announce?

Microsoft's quantum computing group announced in Nature they had observed “quantized Majorana conductance”. Unless you're well-versed in quantum mechanics, I don't recommend you check it out, as even the abstract is quite technical. Let me explain what it's essentially saying.

There's a type of particle called a “Majorana fermion” (after Ettore Majorana, who predicted their existence in 1937) that is its own anti-particle. Don't worry if you don't know exactly what that entails, that's not the important part. What's important is that specially prepared pairs of these particles can exhibit a very interesting property: it is possible to entangle the states of those particles, in the same way that you'd braid two pieces of string together.

Imagine two rubber bands attached at both of their ends to some common support that cannot move. The bands may for example be parallel to each other. That's one state. Now exchange one of the extremities of each rubber band. That's another state. Notice that even though the rubber bands can be deformed, there is no way to go from one state to the other, except by detaching the extremities and reattaching them. The bands will continue to have, in one case an even numbers of overlaps, and in the other an odd number of overlaps. This sort of property that can resist continuous deformation of the system is the subject of a branch of mathematics called topology (of which a whole sub branch is the study of braids). Pairs of Majorana fermions, if properly prepared, can exhibit such properties.

Majorana fermions were theoretical until recent years. We think neutrinos (a very elusive type of particle involved in radioactivity) may be Majorana fermions, but we're not sure yet. Last Summer, the first smoking gun of quasi-particles behaving like Majorana fermions has been announced by a group of scientists from Stanford. Quasi-particles are configurations of quantum fields in a material that behave like elementary particles, but aren't. They are typically found in two-dimensional materials like the contact surface between two materials, or single-dimensional materials like a wire.

What the Microsoft quantum computing group announced this first week of April 2018 is that they have strong and consistent evidence for the presence of Majorana quasi-particles in nanowires. In simpler words, they have a device with the elusive Majoranas in it.

Why is it important?

Being immune to deformations is a very interesting property if you want to build a quantum computer. Quantum states tend to be confined to small scales and low temperatures, because interaction with large and warm systems will destroy them. Even a small perturbation from the environment can have catastrophic effects on the “quantumness” of the system, which is precisely what we're interested in with quantum computing. You can deal with it in a few ways: you can keep the system small (but that limits how much you can achieve), you can add redundancy to the system (but that makes it more difficult to scale the system up), or you can find more stable quantum systems (that's the direction the Microsoft team has chosen to take).

Topological quantum states can still be destroyed by the environment, but there are ways to mitigate the problem that other sorts of quantum states don't have access to. Typically, the only thing that can destroy a topologically entangled pair of Majoranas, is the interaction with another pair in a state that can collide with it. This does happen, because in quantum mechanics, such pairs spontaneously pop into existence all the time. You can make that very unlikely however, with a simple trick: move both halves of the pair apart as much as you can. The particles will remain entangled (so their state remains unchanged), but for another pair to be able to destroy the state, it would have to be similarly stretched across space. This is considerably less likely than if the pair was spatially close together.

The Microsoft results, the implementation of Majorana states, are a very important step on the path of stable, reliable, and scalable quantum computing. This is why it matters.

But let's take a step back, and try to understand what quantum computing is, and why we want it.

A quick intro to quantum mechanics

I should probably stress out before I go any further that while I'll teach you rudiments of quantum mechanics, this is not going to give you a deep understanding of it. It will however give you an understanding of the mechanisms at work in quantum computing. In other words, you won't necessarily understand Schrödinger's cat much better, but you'll be able to understand and use basic quantum algorithms.

Quantum states

In any text about quantum mechanics, you'll read a lot about “quantum states”. I've already used the term a lot in this post without properly defining it. A state is the condition a system is in. For example, a bit can have one of two states: 0 and 1. A byte's state can be any integer between 0 and 255. You could also say that a byte's state is described by the states of each of its eight constituent bits. Now that's interesting, because those eight bits can be seen as independent sub-states. They are orthogonal. You could describe a byte as a point in an eight-dimensional space where each dimension is one of the bits, and where the coordinates can be 0 or 1. When you think about it, any classical physical system can be described with a set of coordinates over elementary, orthogonal states.

For example, the state of a ball may be described by its position (three coordinates), and its velocity (three coordinates again).

Quantum states are pretty much the same, with an interesting twist… A quantum state is also described with a set of coordinates, but in a more abstract space than what we've considered so far, and with complex coordinates.

Now hold it right there… Complex numbers? Why? How can complex numbers appear in nature? I can only see real numbers around. Well, in our classical perception of the world, yes, only real numbers ever show up. If you measure the width of your desk, you will never, ever measure $(2 + 3i)$ meters. In the quantum world, however, complex numbers are the norm. Complex numbers are also, in many ways, easier to manipulate and more “natural” than real numbers. All second degree complex equations have solutions, for instance, whereas real ones don't ( $x^{2} + 1 = 0$ for example).

A good way to think about complex numbers in physics is to imagine them as a real amplitude with a phase, rather than a combination of real and imaginary parts. The phase is an angle between 0 and $2π$ (360° in radians). Positive numbers have a 0 phase, negative numbers have a $π$ phase, i has a $π / 2$ phase, and all other values are possible. When you multiply two numbers, the amplitudes get multiplied, and the phases get added (modulo $2π$ ). When you add two numbers, you can imagine them as being drawn one at the end of the other.

Depending on how the phases align or not, adding complex numbers may result in constructive or destructive interference.

This ability for out of phase states to interfere, much like waves do, is key to quantum weirdness. Let's look at the specifics.

Let's get back to our classical bit and try to understand what would be a quantum equivalent, a “qubit”. To build this, we start from two orthogonal states (that's the first difference with the classical bit where 0 and 1 are not orthogonal). We'll arbitrarily label these states $|0⟩$ and $|1⟩$ . This is just a weird physicist's notation, labelling two states. There is no numerical value associated with them, they are just equivalents of the zero and one states of a classical bit.

The state of the qubit can be described by complex coordinates along those two orthogonal states:

$|q⟩ = a \cdot |0⟩ + b \cdot |1⟩$

So far, we've just thrown some definitions, without explaining where they're coming from, what they represent, or what they mean. $|0⟩$ and $|1⟩$ may be any possible pair of orthogonal values that we may measure for the same quantity. For example, they may represent the spin of an electron being up or down, or they may be the untangled and tangled states of the braid we were talking about earlier. The calculations will be the same no matter what the physical substrate actually is. This is good: we wouldn't want our Q# code to have to look different whether we run it on a Google qubit, or on a Microsoft Majorana topological qubit.

Our qubit already is a superset of a classical bit: {a: 1, b: 0} and {a: 0, b: 1} correspond to classical bit values 0 and 1. We do have however other possible states: other complex values for a and b, which physicists call superpositions. There is just one constraint on these coordinates, and to understand it, we'll introduce some physical meaning with what happens when we measure a quantum state.

Measurement

In order to extract information from the system, we need to measure it. Measuring has the effect of collapsing the state of the system along one, and one only, of the possible classical states, $|0⟩$ and $|1⟩$ . The probability of getting one or the other is the square of the amplitude of the state's component along it (if the states are normalized, more on this in a second). Applying this to the two base states, we can see that for $|0⟩ = 1 \cdot |0⟩ + 0 \cdot |1⟩$ , the probability of measuring $|0⟩$ is 1, and the probability of measuring $|1⟩$ is zero. For $|1⟩$ , those are reversed. We thus get confirmation that those two states are measured as themselves. They are classical states, and have no uncertainty associated with them.

Now let's look at a state that is a superposition of both states, the $|q⟩$ state above. The probability to get $|0⟩$ is ${|a|}^{2}$ , and the probability to get $|1⟩$ is ${|b|}^{2}$ . No other result is possible, which means that the sum of those probabilities should be 1. The probabilities of a complete set of possible outcomes always sum up to 1. Numerically, this translates as ${|a|}^{2} + {|b|}^{2} = 1$ . This is what we mean by “normalized state”, and it is a constraint that we'll enforce to keep the formulas simple: not enforcing it would add a scale factor to the probability formula. Not a big deal, but it's fine to see it as a useful convention.

Can you guess what the probabilities will be to measure either value from a qubit built as a superposition of equal amounts of each state?

$|p⟩ = \frac{1}{\sqrt{2}} \cdot |0⟩ + \frac{1}{\sqrt{2}} \cdot |1⟩$

The squared amplitude for each value is the same: $\frac{1}{2}$ . This means that, as expected, each value is equally likely, and that the probabilities add up to 1.

Once a state has been measured, its state becomes whichever base state happens to have been randomly selected. It is no longer a superposition. So in the previous example, a measure will give one or the other value half the time. Let's say we measured $|1⟩$ . The state is now $|1⟩$ , so if we measure it again, because it is now this state, and we've seen $|1⟩$ measured as $|1⟩$ 100% of the time, we will always measure $|1⟩$ .

We now have the basic understanding of a qubit, and of what happens when we attempt to measure it. Let's now look at what operations we can perform on qubits, in which we'll introduce the quantum building blocks that are analogous to classical logic gates.

Quantum operators

It is possible to transform quantum states while keeping their quantum properties (unlike what happens when measuring them). Those transformations must obey a number of rules in order to be valid, and one of those rules is that the transformed state must still be a valid quantum state, for which the probabilities of observing the various possible states have a sum of 1. Such transformations are called “unitary”, and they can be described with matrices. Applying the transformation is the same thing as multiplying the matrix by the vector representing the state.

Let's start with one of the simplest operations we can do: NOT. A NOT gate, both in classical and quantum computing, transforms 0 into 1, and 1 into 0. The matrix for this is easy to build:

$[\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix}]$

It's easy to see that this works as expected:

$[\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix}] [\begin{matrix} 1 \\ 0 \end{matrix}] = [\begin{matrix} 0 \\ 1 \end{matrix}] [\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix}] [\begin{matrix} 0 \\ 1 \end{matrix}] = [\begin{matrix} 1 \\ 0 \end{matrix}]$

A NOT gate is usually represented with the following symbol:

Another interesting gate is the Hadamard gate, or H gate:

$\frac{1}{\sqrt{2}} [\begin{matrix} 1 & 1 \\ 1 & -1 \end{matrix}]$

This gate transforms $|0⟩$ into $\frac{|0⟩ + |1⟩}{\sqrt{2}}$ , and $|1⟩$ into $\frac{|0⟩ - |1⟩}{\sqrt{2}}$ . This is interesting because if you started with one of the base states, you can use this to transform it into a superposition.

Let's move on to gates that involve more than one qubit. Things are going to get a lot more exciting and spooky, because this will open the door to quantum entanglement.

To represent two qubits, we just use a tensor product. The individual base states for our dual-qubit quantum states are all tensor products of the base states of both qubits. There's 4 of them, that we'll use as the bases for the space of 2-qubit states:

$|00⟩ |01⟩ |10⟩ |11⟩$

Extrapolating from this, you should start to see how this will will scale: a 8 qubit system will be represented by vectors in a space whose base are the tensor products of eight individual states, each of which have two base states. That's a $2^{8} = 256$ dimensional space. As you can see, the number of degrees of freedom of the system literally grows exponentially with the number of qubits, and yet, the number of gates you need to process them grows only linearly. That is what makes quantum computers more powerful (for algorithms that can take advantage of them) than classical ones. I cannot stress enough that this is one case where the word “exponential” is actually justified, because it is so rarely the case. It also shows how the benefits of quantum computing are ones of scalability: it can solve problems where the complexity grows too fast for classical computers, by growing the computing power non-linearly.

If we build a state from two qubits

$a |0⟩ + b |1⟩$ and $c |0⟩ + d |1⟩$

That same state can be represented in terms of tensor products as:

$ac |00⟩ + ad |01⟩ + bc |10⟩ + bd |11⟩$

Remember however that any linear combination of the four base states is possible. Is it always possible to express such a state as the tensor product of two qubits? In other words, given a state $A |00⟩ + B |01⟩ + C |10⟩ + D |11⟩$ , is it always possible to find $a$ , $b$ , $c$ , and $d$ such that:

$\{\begin{matrix} A = ac \\ B = ad \\ C = bc \\ D = bd \end{matrix}$

The answer is no. For example, $\frac{1}{\sqrt{2}} |00⟩ + \frac{1}{\sqrt{2}} |11⟩$ cannot be expressed as a tensor product of states of individual qubits. It's easy to see that since $B = 0$ , $a$ or $d$ would have to be zero, but that would make $D$ zero, which is false.

Such a state, that can be realized in a two-qubit system, but isn't a simple combination of two qubits, is called an entangled state. This name is justified by an interesting property, which is that the measurement of the two bits is tightly coupled.

In the example above (known as a Bell state), the two states that can be measured are $|00⟩$ and $|11⟩$ . All other states have a zero probability of coming up. Another way to say this is that if the first bit is measured as a 1, the second bit will also be measured as a 1, and the same goes for measuring a zero. There is still chance involved in the measurement, but the system's bits are strangely coupled. It looks like the first bit is communicating its own measure to the second one, instantly (and even potentially, over huge distances that couldn't be covered by a physical signal that would have to be faster than light, which is impossible).

From a quantum perspective, nothing really weird is going on: there is only one state, that spans what we as classical observers identify as two separate entities. There is only one measurement being performed, over a single system, even if that system is delocalized. When we choose to measure the individual classical bits, we're slicing into that state in "diagonal", we're looking at the physical system from a point of view that isn't particularly natural. The weirdness is in our perspective.

There's more to quantum entanglement (for instance it's what enables quantum teleportation), but you should know enough from that brief introduction to understand its usefulness in the context of quantum computing. Entangled states are those states that cannot be described as the combination of its qubits. They highlight those cases where the system is more than the sum of its parts.

Another consequence of the existence of entangled states is that you can't take shortcuts when simulating qubits on classical hardware: you need to consider the whole exponentially growing set of base states, and can't consider the qubits in isolation. This means that the simulation will become impractical very fast, as the number of qubits grows.

So how do we build one of those states? For that, we'll need new multiple-qubit gates. The first we need is the controlled NOT gate, or CNOT. With this gate, the first qubit acts as a control (it's unchanged by the operation), and the second bit is inverted only if the first qubit is $|1⟩$ (otherwise it's left alone):

$CNOT = [\begin{matrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \end{matrix}]$

It's easy to verify that, as announced, we have the following mapping on the base states:

$|00⟩ \to |00⟩ |01⟩ \to |01⟩ |10⟩ \to |11⟩ |11⟩ \to |10⟩$

If we first apply a Hadamard gate on the first qubit, then CNOT on the resulting state, the resulting combination has the interesting property that it maps $|00⟩$ onto the Bell state $\frac{1}{\sqrt{2}} |00⟩ + \frac{1}{\sqrt{2}} |11⟩$ .

Let's verify this. The Hadamard gate on the first qubit maps $|00⟩$ onto $\frac{1}{\sqrt{2}} |00⟩ + \frac{1}{\sqrt{2}} |10⟩$ . Then, the CNOT gate leaves $|00⟩$ unchanged while it maps $|10⟩$ onto $|11⟩$ . QED, the resulting state is $\frac{1}{\sqrt{2}} |00⟩ + \frac{1}{\sqrt{2}} |11⟩$ , and we have successfully prepared an entangled state. It's left as an exercise to the reader that the other three base states also get mapped to entangled states.

Of course, there are many other possible gates. The point being that like for classical computing, there are minimal universal sets of gates that are enough, when combined, to perform almost any calculations. Also like in classical computing, implementing an algorithm when all you have is the lowest level of gates is not exactly the easiest thing in the world. With classical computers, we have several layers of abstraction that enables us to express computations with high-level languages, but we have no such luxury (yet) with quantum computers. Even something like Q# only enables developers to mix high-level classical constructs with low-level qubit manipulation.

The trouble with quantum computers: the inaccessible state

With this, we've in principle set-up all of the basic concepts needed by quantum computing. It even looks like we have a way to simultaneously perform arbitrary calculations on all possible inputs, in the time it would take a classical computer to run the same calculations on just one input. Prepare a state such that each qubit is a perfect superposition of $|0⟩$ and $|1⟩$ , run it through gates that implement your calculation, and presto!, you've got a result that is the superposition of the results for all possible input values. If you had access to that result state, there are simple tricks that would enable you to actually extract each individual result.

That is precisely the issue, though: we do not have access to the quantum state, not directly at least. The quantum state may be the true physical state of the system, but we can never get all the information about it. All we can do is measure it, and as we've seen, measurement is a destructive, stochastic, and lossy process. In effect, it is a general principle that you cannot extract more than $n$ bits of information out of $n$ qubits. Does this mean that we've just lost all the potential benefits of quantum computing? Not exactly, but it does mean that we'd better make those measurements count...

The simplest win: guess the function

Imagine that we have an unknown function that takes one classical bit as an input, and outputs one classical bit. There are only four such functions:

$\begin{matrix} f (0) & f (1) \\ f_{0} & 0 & 0 \\ f_{1} & 0 & 1 \\ f_{2} & 1 & 0 \\ f_{3} & 1 & 1 \end{matrix}$

Two of those ( $f_{0}$ and $f_{3}$ ) are constants. If you wanted to determine if the function is one of those two constant functions with a classical computer, you have to perform exactly two operations: feed it a 0, then feed it a 1. If both results are equal, the function is constant.

A quantum computer can do that in only one operation. The trick to do it is to sandwich between two Hadamard gates a special function called an oracle, that is built from the one we're trying to guess. We'll then feed a $|0⟩$ into the process. The oracle to use is the following:

$|x⟩ \to {(-1)}^{f (x)} |x⟩$

Whether the result of the evaluation of $f$ is 0 or 1, if the function is constant, the same phase will be applied to both of the base states. This means that any quantum state will come out unchanged, except for the occasional $(-1)$ phase. And remember from what we said about measurement that phase cannot change the probability of the outcomes, only the amplitudes count.

Now if the function is not constant, each of the base states will get opposite phase shifts. All we have to do now is add the Hadamard gates around the oracle, and feed the system the $|0⟩$ state.

Let's break the whole process down from beginning to end. The first Hadamard gate will change the state into a superposition of the base states. If the function is constant, the oracle will give back the same state, with maybe a phase change. The second Hadamard gate will transform that back into $|0⟩$ , with maybe a phase change. If the function is not constant, a different state comes out, with a phase shift on one of the base states, and not on the other. In this case, the second Hadamard gate is going to transform that into a pure $|1⟩$ . All we have to do is measure the output. If the function is constant, we'll get $|0⟩$ 100% of the time, and if it's not, we'll get $|1⟩$ 100% of the time. We have successfully obtained information from the system, without any quantum uncertainty, faster than a classical system ever could.

Conclusion

We went through one of the simplest examples of a quantum algorithm that outperforms any possible classical system. The principles we've applied are the same you'd apply at a larger scale, but the benefits would then be considerably more substantial. We take a known base state, change it into some superposed quantum state, then apply some transformation, and come back to pure base states that depend on the outcome of the computation, then we measure. We will not gain the exponentially faster parallel computing we may have been led to hope for from the behavior of the quantum state and gates, but we do get very real benefits on appropriate processes. Learning and discovering those processes can be a lot more complicated than what I've shown in this article, but I hope that you've gained an understanding and an appreciation of it. Most of all, I hope things look a little less mysterious, and a lot more fascinating, that you'll be driven to read more about quantum computing. Tell me what you think in the comments, what you'd like to read in future posts, and of course, let me know if I got anything wrong.

Orchard Harvest 2017–Orchard Core CMS

Wed, 22 Feb 2017 21:57:12 GMT

For the last presentation of the day, Sébastien explained what Orchard Core is all about. Orchard Core runs on ASP.NET Core, and as a consequence is leaner, faster, and cross-platform. One big change is that its content is persisted as documents instead of relational tables.

Other differences include:

The admin theme is now a setting.
The content item editor has a new live preview feature, content editor, an flow part that largely replaces the layout feature of Orchard 1. The whole thing is responsive and easy to work with.
A navigation is one single content item, with its hierarchy persisted as a single document. This is very fast, but preserves the extensibility of Orchard 1: menu items are still themselves content items, it’s just that they are persisted all together in the containing menu content item’s document. This also helps versioning the menu as a whole, as well as efficiently caching it.
The order of parts in type definitions is easier to change.
There are named parts.
Blogs are just lists of blog posts.
Tokens are input into Handlebar templates (for example in Autoroute).
The widget editor supports drag and drop between zones, and has an undo button.
Widgets can be used in “flow pages”, in which case their editor is displayed inline in the content item editor.
Recipes can have dynamic values and define variables. They are also JSON documents.
It’s crazy fast. 15ms to render a blog with widgets without caching on a laptop in Debug mode under load (2000 rps). This is way faster than Orchard 1 with caching on.
Creating and setting up a new tenant is instantaneous. So is enabling a module.
Memory usage: tenants are lazily loaded on their first requests, and 500 tenants fit in 2GB on a 64 bit machine.

Migrating data will not be trivial, and will have to go through import/export. Migrating modules should be easy. Modules are like a mini-MVC application, much like in Orchard 1.

You can follow the progress on https://github.com/OrchardCMS/Orchard2/wiki/Roadmap

It’s also a great time to start contributing!

Orchard Harvest 2017–YesSql

Wed, 22 Feb 2017 19:01:00 GMT

Sébastien Ros gave a surprise demo of YesSql, the document database interface over relational databases that powers Orchard Core’s data access. YesSql stores plain objects into documents stored on a relational database. It supports SQL Server, MySql, Postgre, Sqlite, LightningDB, and in-memory. YesSql also allows for querying those objects using queryable indexes. Those indexes are projections of documents into structured tables, that only exist for the purpose of being queried.

YesSql supports map indexes that map one document to one or more index records, and map-reduce indexes that aggregate documents. For all operations, you create a transactional session from a store object. The store object needs to be initialized once before the database can be used for YesSql. The session can save documents using the Save method, which results in a JSON-serialized record in the database. Known classes, as well as anonymous objects can be persisted.

When fetching documents, it’s possible to get a dynamic object when the type is not known. If you know the type, you can GetAsync<KnownType>(id).

An index is built by deriving from IndexProvider<T>. The Describe method defines the mapping from the type T to another type that derives from MapIndex, and has properties that will get serialized as individual columns of the index table in the database. That table can then be queried in rich manners by calling QueryAsync<T, TIndex>(), that can then be chained with Where, OrderBy, and other Linq extensions.

When building an index, it’s possible to pre-filter documents, which means that there is no runtime cost to filtering when the conditions are known in advance.

It’s also possible to perform joined queries between several indexes.

Map/reduce indexes can first do a Map, then usually a Group, and finally compute some aggregate values over the group in Reduce. There is an optional Delete method that describes how to update an index in case of document deletion, to avoid having to re-compute the whole index.

All indexes are automatically maintained.

During questions, Sébastien explained how integer document ids are acquired: the sessions get assigned next ids one by one, and different sessions will be assigned new ids that are guaranteed to be owned by them. The result is not necessarily sequential, but enables clients to know unique ids on creation without round-tripping to the database.

Orchard Harvest 2017–Client-side components in Orchard

Wed, 22 Feb 2017 17:32:32 GMT

A client-side component is defined as a component that implements its rendering and behavior on the client, with server interactions going through some APIs.

Such components can be implemented in Orchard using a range of technologies from simple shape templates to full-blown parts and elements. A good compromise is layout snippets. Layout snippets can be implemented with a simple SomethingSnippet.cshtml file that can simply be dropped into the Views folder of a module, and is then available as a regular element in layouts. Snippets can also be associated with a manifest that specifies metadata such as toolbox icons, and also configurable fields.

Daniel showed how to use a shape table provider in his module to add an API URL onto his shape from site settings.

To manage client-side assets, he’s using Gulp. There’s a choice however to put the generated assets in source control or not. His preference is to keep them out: they don’t get stale, don’t need to be pushed on all modifications, it reduces noise and footprint. The downside is of course that developers wanting to modify the module need to install the tooling.

The component client assets are Handlebars templates, TypeScript, and scss, and they are bundled using Browserify. The corresponding tooling is brought in by a package.json file.

Things get a little more complicated when building Angular components, because of Angular routing, which happens purely on the client and integrates into browser navigation. That requires integration with Orchard server-side routing, which doesn’t work out of the box. The way to make it work is to use the routing features from the IDeliverable.Bits module to add a base URL tag to the page and tweak routing so Orchard can recognize that base URL as an Orchard route. The rest of the URL will be handled client-side by the Angular component to display the proper thing.

One last thing that is required is to prepare some data for the scripts to use, such as base paths. This is done by rendering those pieces of data into data-* attributes in the markup.

Orchard Harvest 2017–Localization

Wed, 22 Feb 2017 16:29:49 GMT

Benedek is one of the founders of Lombiq, and the caretaker of Orchard localization. Application localization requires taking into account cultural differences such as gender, formal vs informal, pluralization, right to left, verbosity, accents, etc. In Orchard, all localizable strings should be wrapped in T() calls. That is enough to make the string localizable in PO files. Some strings can contain placeholders, such as T("Hello, I’m from {0}."). You can use the Vandelay Industries module’s translation extraction feature to extract all the T strings from a module into po files. The module produces a zip archive that contains the layout of localization files that you’d unzip into the site in order to install it. To localize, you can copy any po file from its en-us directory to a new directory for the new culture, then translate the strings inside.

Manual translation of po files is not the only option, however. The Orchard community is crowd-sourcing and centralizing translation of the core platform and of the most common modules on Crowdin. Benedek uploads po files for all new releases of Orchard onto the site, then anybody can contribute translations, with machine translation suggestions from Crowdin, and proofread existing translations. The resulting translated strings can be downloaded as an archive that can be unzipped over the web site. You can then add cultures in Orchard settings, switch the default culture and start using the new translations.

Now that we know how to localize static strings defined in modules, we can look at how to translate contents. Translating contents can be challenging, especially in cases where relationships between items exist, such as taxonomies, or media library picker fields. All the tools are in place in Orchard for this to work however, with the possibility to specify items as culture neutral, cloning capabilities, and culture matching of associated items. Navigation is still work in progress but is following the same path.

A guide to localization is available from http://orchardproject.net/localization

In questions, bulk translation was mentioned, for which the q42 module is very useful.

Jorge also mentioned how the en-us translation can be used to hack the site into displaying a different string in English than what’s specified in code.

Orchard Harvest 2017–Writing a theme for Orchard Core

Wed, 22 Feb 2017 15:02:46 GMT

In the first session of the second day of Orchard Harvest, Steve Taylor showed how to build a new theme for Orchard Core. All the pieces are already in place for building themed sites, and the work is similar to Orchard 1.x themes, except for some json file editing because of missing admin UI in places. New Razor Pages features can be used, such as tag helpers, @inject directives, etc. The tag helpers in particular, coupled with Orchard’s shapes, make for very clean markup in view files. The video for the talk, when available, will be a valuable reference for people who want to get started building sites with Orchard Core: the CMS now looks feature-complete enough to do some serious work. Widgets are there, the shape system is there, search, navigation, all work. That Steve was able to build a complete site and theme under an hour (with some pre-built css and views, of course) shows how far Orchard Core has gone already.

Orchard Harvest 2017–What is Orchard Core SaaS Framework?

Tue, 21 Feb 2017 21:52:01 GMT

Nick Mayne is one of the main developers working on Orchard 2.0, a.k.a. Orchard Core.

Orchard Core started as an MVC application, but this had several challenges with multi-tenancy and Glimpse integration. To solve those problems, Nick moved things around so MVC actually is a module instead. I found that really interesting as I had implemented something really similar when I built DecentCMS: Express is loaded as a module, for each tenant.

In his first demo, he started with a simple ASP.NET Core application, and transformed it into an Orchard Core application where the application code lives in a module instead of the startup class where it began. In the module, a simple class derives from StartupBase and overrides Configure and ConfigureServices.

The second demo is improving on the first one by adding actual MVC to the application. The existing module is modified to register MVC. Nick then added a new module, gave it a controller with an Index action. The result is MVC in an Orchard Core site with a module registering MVC, and another using MVC. In a full Orchard stack, the module registering MVC would be “the MVC module”, and the other would just take a dependency on it.

Next, Nick showed how to point the site’s host to an alternate module location. This is how themes are added: services.AddExtensionLocation(“Themes”). You can also modify manifest definitions.

For the moment, we haven’t created any tenants. In other words, everything has been running on the default tenant. Using services.AddSitesfolder(“App_data”, “Sites”), it’s possible to point the application to locations where tenants are defined. Nick created two sites with their own manifests, which creates two tenants with all features enabled. To differentiate them, he injected the ShellSettings into the controller, whose action can now output properties coming from each tenant manifest.

Finally, Nick showed how to swap MVC for Nancy. Yes, one interesting consequence of having MVC as a module is that you may not use it at all, and you may even use something else altogether, such as Nancy. Moreover, you may even have one module built on Nancy, and another on MVC, all running in the same host. This is really very cool and shows just how low footprint Orchard Core is, and how its services, such as multi-tenancy and modules, come at a very low price and impose very few constraints on the whole system. In other words, you get exactly what you want, and nothing more.

Orchard Harvest 2017–Scaling Orchard

Tue, 21 Feb 2017 20:35:47 GMT

Rob King works for Bede Gaming, which specializes in providing a platform for gambling web sites. The company moved to Orchard in 2013, and has strong scalability requirements, with sites serving millions of requests per day.

One of the problems they are facing is dependency management. One of the ways they deal with that problem is that they avoid features depending on each other, but instead try to have dependencies on abstractions. Instead of having feature A depending on feature B because they have common interfaces, those interfaces are extracted into a common module, and both features are made to depend on it.

They cache layer rules to avoid going to the content manager to get get them.

The Bede web sites are typically entirely dealing with authenticated users, so caching has to implement donut caching techniques. They also use Redis for distributed output caching, plus in-memory caching.

They had an incident when they had a deadlock on scheduled tasks that caused the scheduled tasks table to fill up with millions of records. It was solved with a node-aware version of the scheduled task manager.

Being multi-node aware everywhere was key to fixing many of the problems faced.

Orchard Harvest 2017–What’s new in ASP.NET MVC Core 2.0

Tue, 21 Feb 2017 19:43:55 GMT

Taylor Mullen from the ASP.NET team is a developer working on MVC Core 2.0. In this session, he went over the design of the new Razor Pages feature. He carefully explained what is difficult with MVC currently, in order to justify the feature. He actually started by showing what it’s not: it’s not PHP-like, and it’s not a new take on previous “ASP.NET Pages” features.

Razor Pages enable you to get started rendering views without having to build a controller. As the page is growing more complex, the code can be moved into a “page model”, that is starting to look more like a controller, mixed with a view model, but that is actually closer in spirit to what you’d have in a MVVM pattern. This all creates a smoother “grow up” story.

The last piece in the puzzle is routing, which is dealt with using a nice model binding syntax on the @page directive: “@page foo/{id:int}” would setup the route so “foo/12” would hit the page with the id int parameter set to the value 12.

Taylor then showed how al this is wired up together, and how the same plumbing could be used with your own custom providers, in particular for directives.

Orchard Harvest 2017: when output cache just isn’t enough

Tue, 21 Feb 2017 16:49:02 GMT

Daniel Stolt and Chris Payne from IDeliverable presented some new output caching techniques that they’ve developed. The problem that they’re solving is what happens when you have to output user-specific data into rendered contents. In those cases, you want to cache the constant parts of the output, while keeping holes dynamic. This is known as donut caching.

The holes in the donuts can be puched into rich text using a placeholder with the %%{name} notation. Those placeholders are not exactly tokens, but behave similarly, and are also extensible.

They also have an item-level cache part that can be added for instance to widgets, and used to set the output cache policy for that specific item, or to exclude it from the cache altogether. The item-level cache configuration is very rich, and allows for very fine-grained control. This can even be extended with your own “vary by” providers.

Cached output is stored into the same place as Orchard’s built-in output cache. This means that any provider that already exists can be used without changes.

The module also introduces some new options to evict pages that use a given widget, that gives a lot more control than the built-in feature. For instance, if you have a menu widget on all your pages, you don’t want just any change on the menu to cause the eviction of every single page in the site from output cache.

It’s possible to cache summary versions of items, such as the ones in search results. In this case, individual items in a results list, if they are in the cache, do not need to be re-rendered, but can come from the cache. This is quite neat as the caching structure of the page in this case is nested.

Finally, it is possible to trigger pre-rendering of an item into the output cache whenever it gets evicted (on a change, typically).

IDeliverable’s donut caching module is a premium module that can be tried for free on localhost, bought, or subscribed to.

Orchard Harvest 2017 – Using external data with Orchard

Tue, 21 Feb 2017 16:08:39 GMT

Jorge Agraz is opening the conference after Sébastien Ros’ keynote with a talk about using external data in Orchard. Jorge work for Onestop, a company that builds e-commerce sites. Their web sites get their commerce data from APIs and then used to display that through their controllers with Orchard shapes. They attempted to use widgets instead of controllers, but that came with some significant maintenance problems.

With the latest version, commerce pages are actual Orchard content types: category, product and shopping cart. They have elements, fields, and parts to represent everything. A Product driver lazily gets data from the commerce API and instantiates more than 20 shapes from it, that can be used to display the contents. This provides a lot of flexibility, and is easier to move around between dev, staging, and production servers. The number of shape alternates in typical themes has been drastically reduced from 50 to 90 down to 6.

Interestingly, themes have a shape table provider that can be used to turn shapes on or off, and configure them, going beyond what placement.info can do.

Unimportant egocentric rant: don’t .ToUpper() your titles

Sat, 09 Apr 2016 16:23:00 GMT

Here’s a minor inconvenience that I’m going to put out there in the hope that one or two of the horrible people responsible for it may realize the errors of their ways and make the world a marginally better place.

So you like your titles to be all caps. Fine. I’m not sure why you insist on screaming like that, but that’s your choice, and it’s your web site/blog. If you are going to do this, however, please don’t do this:

@post.Title.ToUpper()

And not just because you neglected to specify a culture (you monster). No, this is wrong because next time the guy who builds the week in .NET wants to mention your site and copies the title over, he’s going to copy this:

HOW TO USE ENTITY FRAMEWORK TO KILL PUPPIES!!!

instead of this:

How to use Entity Framework to make a rainbow

He’s going to curse loudly, his kids may hear him, and they’ll grow up to become as obnoxious as he is, all because of you. Well done.

This is particularly infuriating because there is a perfectly good way to do this bad thing that you insist on doing. In your CSS, just do something like this:

h1 { text-transform: uppercase; }

There. Was that so hard?

Except that no. When you copy text that has been styled this way, you’ll still get the all caps text instead of the unstyled underlying text. F@$#!

OK.

Fine.

Just don’t .ToUpper your titles. Please. It looks like crap and it’s less readable. And you sound like a Facebook comment from that uncle who ruins Thanksgiving every year.

On dependencies, dependency injection, and sanity

Thu, 24 Mar 2016 23:50:29 GMT

What's interesting about debacles such as the recent left-padding madness is how it can get you to re-think seemingly obvious concepts, and challenge some basic assumptions. Here's in particular a little reflection that just occurred to me on the way between the bathroom and my office, and that may seem super-obvious to some of you. It wasn't to me, so here goes, I'm sharing…

Whether you go all theoretical, call it fancy and talk about "inversion of control", or you just pragmatically do constructor injection like God intended, at this point almost everyone agrees that dependency injection is a Good Idea. You inject a contract, and the black magic inside your IoC container finds the right implementation and gives you an instance, with the right scope, lifetime, etc. Coupling is low, applications are more robust, more modular, more flexible, and unicorns flutter around happily.

Now when you look at the wonderful world of package managers (a clear progress over what the hell we thought we were doing before), what are we doing exactly? Well, we need a component that performs a certain task (in other words, it fulfils a specific contract). We then go on to ask The One True Package Manager to hand us a very specific version of a very specific package. And we do so USING A STRONG NAME. Pardon me for screaming, but can you see the parallel here?

We are "injecting" a "dependency" into our application, but it doesn't for one second occur to us that the concept is almost exactly the same. Let's go over the comparison one more time.

Dependency Injection	"dependency" "injection"
Ask for a contract	Ask for a strong and versioned package name
Get a managed and scoped instance with members that you can call into	Get a set of namespaces with stuff in them that you can call into

So what is good practice on one side (resolution by contract) is completely out of the picture on the other. Why can't we ask for packages by contract? I don't know exactly what that would be looking like, but I have the nagging feeling that there's something to invent here. Or does something like that exist? What do you think?