We represent floating-point numbers using the IEEE standard. For very small numbers, the standard uses special subnormal numbers. Unfortunately, they have a reputation of making operations slow. Thus video game programmers and machine learning specialists sometimes avoid computing with subnormal numbers for performance.

How slow are they? Let me measure. I wrote a small C++ benchmark with a few kernels over arrays of 16384 values (small enough to fit in cache):

• multiply each value by 0.75,
• add two arrays,
• divide each value by 3,
• multiply normal values by a tiny constant (2^-1030) so that the inputs are normal but the outputs are subnormal,
• a dependent chain x *= 0.9999 repeated 16384 times.

For each kernel, I feed either normal values (in [0.5, 1)), subnormal values, or normal values where one value in a hundred is subnormal. The compiler is allowed to autovectorize the array computations. I use GCC 15 with -O3 -march=native on Linux and Apple clang 17 with the same flags on macOS. I also checked with clang 21 on Linux to make sure.

I ran the benchmark on five processors:

• Intel Xeon 6975P-C (Granite Rapids), on an AWS c8i.xlarge instance,
• Intel Xeon Gold 6548N (Emerald Rapids), a server in my lab,
• AMD EPYC 9R45 (Zen 5), on an AWS c8a.xlarge instance,
• AWS Graviton 5 (Arm Neoverse V3), on a c9g.xlarge instance,
• Apple M4 Max.

Here are the results for double values, in nanoseconds per element (or per step).

Intel Granite Rapids

Intel Emerald Rapids

AMD Zen 5

AWS Graviton 5

Apple M4 Max

On Intel processors, a multiplication involving a subnormal number is about 45 to 50 times slower than a multiplication over normal numbers. A division is 18 times slower. The dependent chain, where each multiplication waits for the previous one, goes from about 1 ns to over 30 ns per step. A normal multiplication in the dependent chain has a latency of 4 cycles. With a subnormal, it has a latency of 128 cycles. It does not matter whether the subnormal is an input or an output: multiplying normal numbers into a subnormal result is just as slow as multiplying subnormal numbers. The exception is additions and subtractions: they run at full speed. Even if subnormals are rare (1%), the cost on Intel can be significant because when the compiler vectorizes the computation, a single subnormal can slow down a whole block of computations.

AMD does much better. On Zen 5, multiplications and additions run at full speed regardless of the inputs. The dependent multiplication chain is a third slower (0.66 ns to 0.88 ns per step): the multiplier needs an extra cycle or so to handle a subnormal. Divisions are about twice as slow. Interestingly, with divisions, having one subnormal in a hundred is almost as slow as having all subnormals. The two Arm processors, the Graviton 5 and the Apple M4 Max, do not care at all. Subnormal numbers are handled at full speed.

Thus it appears that on the latest AMD and ARM processors subnormals might not be a concern. But they remain very much a performance issue under Intel processors.

My source code is available.

Mathematicians are unhappy about OpenAI. Several influential mathematicians wrote an open letter. The gist of their argument is that they form a community that trains young people.

When AI started producing breakthroughs on hard mathematical problems, I asked what a very smart 17-year-old would feel. Do you still choose a math major and train yourself to prove difficult results by hand?

This crisis has been coming for a long time. When I was a kid, the four-colour theorem was proved by a computer, in 1976. An intense debate followed. Does it count as a proof?

I wrote my PhD thesis using symbolic algebra software. To my knowledge, nobody then would publish their scripts. I tried to include mine. I was told it would make me look bad.

The letter says:

“In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas.”

They worry that kids will not choose to become old-school mathematicians. That is a reasonable fear. Yet they do not seem to consider that some kids might still do mathematics, just in a very different way.

The famous mathematician Doron Zeilberger announced the problem in 2009:

“Teaching computers how to discover and prove mathematical results is certainly the way to go, and I believe that mathematicians who continue to do pure human, pencil-and-paper, computer-less, research, are wasting their time.”

The letter implies that people, because of AI, will stop having ideas, or will stop taking the time to understand the issues. If that were true, mathematics would continue only on computers, with no humans interested, or it would stop. Both scenarios assume that people care about mathematics only when they can claim credit.

I am not sure why I cannot study a proof generated by AI if I want to. I can give talks about it. What becomes less likely is the reward of having been the one who proved the result.

The rest of the letter makes a big deal of credit. What if the AI builds on what it read and does not give proper credit? Where is the evidence that AI is worse than human beings at citing sources?

The letter notes that mathematics contributes to society. It never considers that faster progress might increase those contributions.

My stance is simple.

Mathematicians, software developers, engineers, lawyers, physicians will all learn to work with AI. You cannot put the genie back in the bottle. Only a totalitarian world government could try, and some of us would rather avoid that outcome.

Difficult proofs will now be built with AI, just as most code will be written with AI. People who insist on pen and paper should view themselves as artists.

Other mathematicians will work with AI. They will have no trouble finding interested kids. They will contribute to society.

We are told that AI entities much like ChatGPT might soon kill us all. The statement is vague and unfalsifiable. It might be true, it might be false. People with credentials (e.g., Turing Award recipient Yoshua Bengio) believe it.

Many still remember the Year-2000 bug. Our computers used two-digit coding for dates, and some software could get confused. At the time, experts worried that a bug in dates might trigger nuclear Armageddon or an infrastructure collapse. At the very least, planes could fall.

Americans had a moral panic over alcohol that lasted more than a century. It began with pledges of moderation. From 1920 the Volstead Act banned nearly all legal drink. It collapsed in 1933, at great expense.

The Club of Rome predicted mass starvation. As an answer, we sterilized by force millions of Indians, and introduced the devastating one-child policy in China. The authors were never held accountable. The projections were purely mathematical, unescapable. They said. But also totally wrong and silly. Yet, we listened to them and caused great harm.

End-of-the-world scenarios are nothing new. Pretty much all civilizations have lived with various such predictions.

Some people are offended by my comparisons. I truly do not mean to offend. But the fact that disagreeing can lead to deep offense is, by itself, a sign that we face a moral issue. There is a sense in which you must agree that these intense fears are warranted.

Many will remember that when OpenAI first developed GPT-2, they told the world that it was too dangerous to release. Year after year, we were warned that the next iteration of it would doom us all.

A large language model takes tokens (words) in and outputs tokens (words). The big models can take many, many tokens in. And they do much compute. And they are based on clever ideas like vector embeddings. But, ultimately, no large language model can do anything but output tokens. You can build a better model, but the model itself does not ‘learn’. It is a fixed set of weights. If you take a model that has been used for months, and always feed the same tokens, you will get the same results (up to some randomness).

I fear that some people exploit the fact that people cannot grasp how conceptually simple a language model is. In any case, most people don’t understand how things work.

Things become interesting because these models can be hooked up to tools. So you can tell your language model that whenever it outputs ‘boom’, then a nuclear weapon will be launched. And if you hook up a nuclear weapon, then, certainly, you may start a nuclear war. So don’t do that.

One unproven thesis is that the models (that take in tokens and output tokens) will ‘decide’ to acquire access to these nuclear weapons, maybe through a subterfuge. What does that mean? It is always conveniently vague. In the movie WarGames (1983), a teenager uses his computer to access a computer in charge of nuclear weapons. He almost wipes out humanity. Could this happen by accident with a kid using a language model hooked up to the Internet? But if it happens, we should blame whoever hooked up a deadly computer to the Internet.

Of course, any technology is inherently dangerous. Invent the bow to go hunting, and someone might soon turn the bow against you. Invent the engine, and one might soon build tanks and destroy nations. Develop nuclear technology, and one might soon raze your cities.
 
Yet that is not what is at stake in these discussions. The concrete threats are not ascertained and addressed. No doubt, there are some people doing this work, hopefully in the US military. What if an adversary can take control of the economy or military installations? What if an AI agent goes rogue? It is worth investing time in designing defenses.

What we have instead is something of the sort:

1. A vague but global threat. It could be a fatal virus engineered in a lab, a climate catastrophe, a fatal bug affecting all our software, an alien invasion, a rogue AI, Jews taking over our institutions.
2. A few people come forward and they offer to save us. Importantly we must give them resources and influence. Ultimately, they seek a totalitarian solution: everyone must be made to agree so that we can be saved.
3. As the process unfolds, people with an opposing viewpoint are described as a danger. They must be silenced and discredited. Eventually, it can become moral to exaggerate the threat or to rewrite counterpoints. People must be made to understand one way or another.

In this instance, I refer to people who advocate that AI will doom us as AI Doomers. These people tend to carry a totalitarian ideology. Their ideas will only work if everyone is made to agree. And it would severely restrict the freedom of billions of people, although they usually present it differently.

Doomers do not have bad intentions. On the contrary, they are often really out there to save the world. But good intentions do not, in any way, justify the means nor guarantee a good outcome.

Human beings reason based on cultural knowledge. For centuries or more, totalitarian ideas have led to ruin or pain. We ignore the warnings at our peril. It is one after the other: alcohol, the need to restrict the number of children, and so on.

But shouldn’t we just be prudent and adopt their views, just in case? It is a fallacious argument. Members of the intellectual elite have a tendency to fall for the kind of hubris where they think that, if only they were given more power, the world would be better off. It is rarely true. Thomas Sowell has an excellent book on the topic, Intellectuals and Society. He makes the case that intellectuals often promote harmful ideas, at no cost to themselves. Rationally, we should therefore be cautious.

You are not safer without technology. In fact, the risk of human extinction is assuredly higher if we are poorer and have less technology.

Is this unprecedented? The printing press was unprecedented. Arabic numbers were unprecedented. Maybe we should go back to Roman numerals, to be safe. Fear of what is without precedent soon becomes indistinguishable from an anti-innovation stance.

What if you do not like the people who lead the big AI companies like OpenAI and Anthropic. Maybe you think that these billionaires are a danger. And you might be right. But consider the history of humanity. Wealthy people have primarily caused harm through the promotion of bad ideas. The mass murders are almost invariably derived from politics. Stalin, Hitler, Mao.

Throughout much of the world, we are facing demographic collapse and an inverted age pyramid. Soon there may be just one worker per retiree. Choosing to have fewer kids has consequences and we are about to face them. AI might be a way out of significant problems. If you are otherwise wealthy, that might not be a significant concern to you. But for the least fortunate, it might turn out to be quite a problem. Who will take care of you when you are sick? AI and robotics might prove useful to reduce suffering, don’t we think?

Further, we need to consider how powerful people might use the fear of AI for their own purposes. It is entirely credible that the owners of large companies could promote fear so that they get to write the regulations that will keep out their competitors, or merely as a form of cheap marketing.

How do I know that it is moral? Because there cannot be reasoned debate about a moral question. The facts are obvious or you are a bad person. Whenever there is a complex question, one that involves predicting the future, that cannot be discussed, unless it is in agreement with the side of fear, then you are very likely in a moral question. « Don’t you see, AI will soon kill all of us, it is obvious. » No explanation can be demanded.

You might accuse me of, in turn, promoting fear. But it should be obvious that I am doing no such thing. What I am encouraging rather is the use of reason. I am forced to give examples where inciting fear has led to disastrous effects, but my hope is that it will lead my reader to sit and reflect.

To my friends who fear AI, I urge you. Use reason. Do the work. Do not rely on hasty thought experiments. Work out the details. Think. Think about the countermeasures. And, please, do not include abstract thinking machines. A language model is a box that takes in token and produces tokens. Nothing more.

And for the rest of us. Let us build. Let us bring prosperity. Let us hasten the cure for cancer. Let us dream of exploring our solar system.

Further reading.

• • Nirit Weiss-Blatt, “The Weak Foundations of AI Doomsday” — AI Panic, May 15, 2026.
  • Cal Newport, “Dear A.I. Companies, the Doom Trolling Needs to Stop” — New York Times, June 17, 2026
  • Andrew Orlowski, “Anthropic’s doom predictions are merely hype intended to make AI look important” — The Telegraph, June 5, 2026.

If you write in C, by default, you use a single thread. Extra cores do not help until you create more threads. However, if you include the header <threads.h>, you can pass a function to thrd_create, and wait for it with thrd_join.

#include <threads.h>
#include <stdio.h>
int worker(void *arg) {
    printf("hello from thread %d\n", *(int *)arg);
    return 0;
}
int main(void) {
    thrd_t t;
    int id = 1;
    thrd_create(&t, worker, &id);
    thrd_join(t, NULL);
}

Be warned that C11 threads are an optional feature. If the macro __STDC_NO_THREADS__ is defined, you do not have them. Apple’s C library has never shipped <threads.h>, so the program above does not compile on macOS, and glibc only added it in version 2.28 (2018). On such systems you fall back on POSIX threads (pthread_create, pthread_join).

Once you have more than one thread, they may share memory. If two threads access the same non-atomic variable with no ordering between them, and at least one of them writes, the C language calls that a data race. In other words, it is unsafe.

If you have a variable and it is effectively constant, then it is fine to share it. But as soon as anyone changes it, then it might get corrupted. If it is not guarded somewhat, you are in trouble.

To be clear, that is what the C programming language says. I don’t mean that it will happen on your machine.

To get a better behavior, we can use atomic variables. In C, you have the <stdatomic.h> header.

An atomic integer is never garbage. You always read a value that was once written.

In practice, on most computers you might use today, aligned 8-, 16-, 32- and 64-bit loads and stores are atomic. The C language does not care about that, so if you don’t specifically require atomicity, you might get in trouble with your C compiler.

The next funny problem is that instructions can be reordered. When you write:

x = 2
y = 3

This may not happen in this sequence. The variable y might be set before the variable x. You may wonder why this is allowed at all. The fundamental reason is that our processors are quite complex. They have layers of buffers and they can execute multiple instructions at once. They can issue several memory loads or stores at once.

By default, in C, atomic accesses are all ordered. It is as if there is an oracle that watches all threads and comes up with a consistent story where everything is in order. This can be expensive, so we prefer not to do it that way.

At the other extreme is the relaxed model: your reads and stores are not garbage, and a given atomic still has one modification order (you will not see 1 and then 0 if the counter only went from 0 to 1), but there is no ordering with respect to other memory.

So we use something intermediate, the release and acquire semantics. They are ordering barriers. A strict barrier would be ‘everything before me really happens before me, and everything after me really happens after me’. (Where ‘really happens’ refers to visible effects, the hardware and compiler are allowed to cheat as long as you don’t catch them.) It is a bit too strong. So we split it in two parts: release and acquire. Intuitively, release means ‘if you see me, you see all the stuff before me’. Acquire means ‘I take that package, and everything I do after this load really happens after it’.

Consider the case where you have a resource (such as a block of allocated memory). You share this resource, but count how many people have access to it. When the counter goes to zero, you free the resource.

One thread could do…

access resource
decrement counter // I won't need it anymore

You see these operations happening one after the other, but they may not execute this way. It is possible that they overlap, or even that the decrement occurs before the access. It is entirely safe in a single threaded context.

Anyhow, so the following could happen

decrement counter // I won't need it anymore
access resource

But what if you have a second thread that does:

access resource
decrement counter // I won' need it anymore
if (counter is zero)
  free(resource)

You could have this interplay:

[thread2] access resource
[thread2] decrement counter
[thread1] decrement counter
[thread2] free(resource)
[thread1] access resource

That would be a bug.

So what you first do is make the decrement a ‘release access’ which means that operations that come before it cannot be reordered after it. So if you do it this way…

access resource
decrement counter using release

Then it is not possible that we ‘see’ the operations as if they happened in the reverse order.

But then we have a second problem. Release is enough for this thread: we cannot still be using the resource after we drop it. It does not tell the last owner that everyone else is finished. The last decrement is itself a release, so it does not observe the other threads’ releases. Without an acquire, that last thread can call free while another thread’s earlier access is not yet done.

[thread2] access resource
[thread2] decrement counter using release
[thread1] decrement counter using release
[thread1] free(resource)

Thread 2 did its access before its release decrement, but thread 1 never acquired, so it is not required to see that access as finished before free.

So we need the counterpart to a release, an acquire. The last owner acquires before it frees, and that pairs with everyone else’s release:

access resource
decrement counter with release
if (counter is zero)
  acquire barrier // see that everyone else is done
  free(resource)

Alternatively, you could do this.

access resource
decrement counter with release and acquire
if (counter is zero)
  free(resource)

The two are equivalent, but they are not necessarily equally cheap.

So let us consider a nice example. Let us build a small array that several threads can share. If you are the only owner, you overwrite an element in place. If not, you copy, then you update the copy. That is called copy-on-write. It is a really nice idea that you will find in many important systems.

We start with the type.

#include <assert.h>
#include <stdatomic.h>
#include <stdlib.h>
#define STR_SIZE 16
typedef struct {
    atomic_int refs;
    int values[STR_SIZE];
} shared_array;

The payload is a plain int array. Only refs is atomic. That is deliberate. We never write values while another thread might be reading them.

We create an instance like so.

shared_array *str_new(void) {
    shared_array *o = malloc(sizeof *o);
    if (o == NULL) {
        return NULL;
    }
    atomic_init(&o->refs, 1);
    for (int i = 0; i < STR_SIZE; i++) {
        o->values[i] = 0;
    }
    return o;
}

The atomic_init is not an atomic access in the memory-model sense. Nobody else has the pointer yet, so there is no other thread to race with. The caller owns one reference. It is just how we initialize an atomic_int.

Here is how we might naively release an instance.

// not real code
void obj_release(shared_array *o) {
    auto ref = o->refs;
    o->refs -= 1;
    if (ref != 1)
        return;
    // we are the last copy
    free(o);
}

What is the problem with this code?

The load and the decrement are two operations. Two threads can both read 2, both subtract, the counter hits zero, and nobody frees: the resource leaks. Write the check the other way around, decrementing first and then testing whether the counter is zero, as in the pseudocode above, and you get the mirror-image bug instead: with refs at 2, one thread decrements to 1, the other decrements to 0, both then read 0, and both call free. You need one atomic subtract that hands you the previous value: only the thread that saw 1 was last.

So you could try

void obj_release(shared_array *o) {
    if (atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed) != 1)
        return;
    free(o);
}

But suppose you have two owners, so that refs is 2. And you have two threads doing

(void)o->values[4];
obj_release(o);

One of them will call free, but the order could be

...
[thread1] o->values[4];
[thread2] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread1] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread1] free(o);
[thread2] o->values[4];

It is a bit confusing because things are not happening in order within thread 2:

[thread2] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread2] o->values[4];

But this is allowed.

So what we can do is put a release on the atomic_fetch_sub_explicit and then an acquire right before the free.

void obj_release(shared_array *o) {
    if (atomic_fetch_sub_explicit(&o->refs, 1, memory_order_release) != 1)
        return;
    atomic_thread_fence(memory_order_acquire);
    free(o);
}

The release on every decrement means “I am done with the payload.” The acquire fence, only on the last owner, means “I have seen that everyone else is done.” Then free is safe.

That release does double duty, as we are about to see. It is also what lets the last remaining owner write to the payload in place.

If a thread wants another reference to the same instance, it only needs a relaxed access.

shared_array *str_retain(shared_array *o) {
    atomic_fetch_add_explicit(&o->refs, 1, memory_order_relaxed);
    return o;
}

Why relaxed? Because the caller already holds a reference, so the object cannot be freed under us: the last owner would need our reference to be gone first.

We can now write update. It consumes the caller’s reference and returns a reference to the array that contains the new value, which may or may not be the same object. After you call it, you must not touch the pointer you passed in. There is one exception: if a copy was needed and the allocation failed, it returns NULL and leaves the caller’s reference to o untouched, so you still own it and must still release it.

shared_array *update(size_t idx, int value, shared_array *o) {
    assert(idx < STR_SIZE);
    if (atomic_load_explicit(&o->refs, memory_order_acquire) == 1) {
        o->values[idx] = value;
        return o;
    }
    shared_array *new_o = str_new();
    if (new_o == NULL) {
        return NULL;
    }
    for (int i = 0; i < STR_SIZE; i++) {
        new_o->values[i] = o->values[i];
    }
    new_o->values[idx] = value;
    obj_release(o);
    return new_o;
}

If the load reads 1, we are the only owner. No other thread holds a reference, so we can write values[idx] in place.

The load is an acquire. When the load reads 1, it may read the value written by the release decrement of the last other owner to drop out. Everything that thread did with values happens before our write. Nothing in our code appearing after such as o->values[idx] = value may move before it. No other thread still holds a reference, so the write does not race with a concurrent reader. Later, after a retain, other threads can see it.

On x64, acquire and release are effectively free at the CPU: ordinary loads already behave like acquire, ordinary stores like release. You still have to write them in C, or the compiler may reorder the payload accesses. ARM has a weaker memory model so the acquire/release require different instructions (ldapr, ldaddl) which may incur a small perforamnce hit.

The code is available.

In Python, the dict data structure is the conventional key-value structure. E.g., you might store a list of names as keys and have their phone numbers as values. Valentin Ignatev wrote this amusing post on X:

It is indeed widely believed that, in the strict sense, the dict data structure and its companion, the set data structure, are O(1), meaning that as you increase the size of the data structure, the time to insert or query a key remains constant.

Let us examine the claim.

A hash function is a function from objects (like strings, integers, etc.) to integer values. We typically expect hash functions to be random-like, although they should always map the same object to the same integer within the current program execution. From hash functions, we construct hash tables:

1. Create an array of buckets.
2. Given an object, apply the hash function to map it to a bucket.
3. Store the object in the bucket. When the bucket is already occupied, use some other trick (such as using a nearby bucket).

If everything goes well, access and insertion in a hash table take nearly constant time, meaning that the time they take is independent of the size of the hash table.

This can be almost true in many instances. However, it is not formally true. There are many reasons why it is false. For example, if your data structure grows, it might be necessary to reallocate, which will typically take time proportional to the size of the data structure. But we also have the issue of collisions. A collision is what happens when two objects have the same hash value. When we use hash tables, we assume that collisions are uncommon. But it is not difficult to create many of them by picking our objects carefully.

In Python, set and dict are hash tables. I can ‘easily’ make my version of Python crumble:

M = (1 << 61) - 1
values = [i * M for i in range(1, n + 1)]
s = set(values)                       # insertions
count = sum(v in s for v in values)   # checks

If the insertions and the checks are constant-time operations, then the whole construction and the entire check should take linear time. I ran this on an Apple M4 Max with Python 3.14, reporting the median of three runs.

The time roughly quadruples each time n doubles. That is quadratic time, not linear time. The membership checks behave the same way: 1066 ms at n = 16000. At a hundred thousand elements, building the set takes 45 seconds.

But could we create a hash table that would be truly constant-time? No. As the size of your data structure grows, it requires progressively slower memory. If you have a small hash table, it can reside in the CPU cache and be fast. Once it reaches megabytes in size, the data structure tends to live in RAM, which is much slower. And then, eventually, you have to store it on disk, which is even slower. And so forth.

To put it differently, saying that a hash table is O(1) or constant time is a model. It can be true, maybe even often, but it is not reality. Models are great teaching tools: they present a simplified model that you can quickly learn. But models can also introduce biases in how we think.

For example, even though you have read my paragraph that says that the dict data structure gets slower, you may not believe it. You may also believe that it is typically going to be the fastest approach you can use.

Let us consider another practical case. Suppose that you have a large map from strings to integers, that you build once and then only query. That is a common situation: a dictionary of words to identifiers, a lookup table of country codes, a table of feature names.

The fastconstmap library builds an immutable map from a dict[str, int]. It is suitable when your keys are known in advance.

I build a map from a million random sixteen-character strings to integers, and then look up every key in a shuffled order. With a dict, I write the obvious loop:

total = 0
for k in probes:
    total += d[k]

With fastconstmap, I ask for all the keys at once, writing the values into a buffer that I own, so that no Python object is allocated per key:

out = array("Q", bytes(8 * n))
cm.get_many_into(probes, out)

I am being generous to the dict. I reuse the same string objects for the lookups, and a Python string caches its hash value the first time it is computed. So the dict does not pay for hashing at all, while fastconstmap hashes every key every time. Here are the results, in nanoseconds per key.

The dict is not constant time. It goes from 22 ns to 202 ns per key as the map grows, a factor of nine, and it is not because the algorithm changed or because of collisions. It is because a million keys, their string objects, and their integer objects occupy about 116 bytes per key, so the lookups miss in the cache. The fastconstmap version needs 9 bytes per key: it stays in the cache much longer. Pay attention to how the numbers scale: the dict becomes 10 times slower as the size grows.

The lesson is always the same. Some models are useful but none of them is reality. Be mindful of cognitive biases.

The code is available.

JSON is a standard format for data interchange. It is effectively a tiny subset of JavaScript made of objects and arrays. It looks as follows {"key":1, "text":[1.0,2.0]}.

Many programming languages include a JSON library in their standard libraries: C#, Go, Java (soon), Python, JavaScript, etc. The Go implementation is convenient, but not especially fast.

Go 1.27 makes a new JSON package (encoding/json/v2) available by default in its standard library. The two APIs look almost the same:

import (
    json    "encoding/json"
    jsonv2  "encoding/json/v2"
)
b, err := json.Marshal(v)
b, err  = jsonv2.Marshal(v)
err = json.Unmarshal(b, &v)
err = jsonv2.Unmarshal(b, &v)

The two are not directly comparable as they differ with respect to Unicode validation, case sensitivity, etc. So it is not a drop-in replacement.

However, the legacy API (encoding/json) has also been reimplemented on top of the new engine. You can use the legacy API with either the new engine or the old one (GOEXPERIMENT=nojsonv2) through a flag. So we have three possibilities.

1. json (legacy) — encoding/json built with GOEXPERIMENT=nojsonv2, the original implementation
2. json (Go 1.27) — encoding/json as of 1.27, v1 API on the v2 backend
3. json/v2 — encoding/json/v2

I used the usual simdjson documents: twitter.json (632 kB, nested objects with short string keys), canada.json (2.25 MB, one large array of coordinates), and citm_catalog.json (1.73 MB, nested objects with numeric keys). I parse them into any (interface{}), which is the general-purpose path.

I ran this on an Apple M4 Max and on an Intel Xeon Gold 6548N (Emerald Rapids) using Go 1.27.0, on a single core (GOMAXPROCS=1), reporting the median of eight runs.

When unmarshalling, the legacy API on the new backend is faster than the original on twitter.json (172 MB/s to 203 MB/s) and on citm_catalog.json (186 MB/s to 241 MB/s), but slower on canada.json (128 MB/s down to 106 MB/s). When marshalling, it is up to twice as fast: 198 MB/s to 374 MB/s on twitter.json. So merely upgrading to Go 1.27, without changing a line of code, should make marshalling faster.

Switching to the new API helps more. Compared to the original implementation, encoding/json/v2 unmarshals 1.5x to 2.3x faster and marshals 1.2x to 3x faster. Compared to the Go 1.27 legacy API, unmarshalling gains another 1.8x to 2x, while marshalling gains much less (1.0x to 1.7x): part of the remaining difference is that json/v2 does less work during marshalling.

Thus far, I was unmarshalling into any, meaning that I assumed that I did not know the structure of the document. I also round-trip a slice of 10,000 small structs:

type Record struct {
    ID     int      `json:"id"`
    Name   string   `json:"name"`
    Email  string   `json:"email"`
    Active bool     `json:"active"`
    Score  float64  `json:"score"`
    Tags   []string `json:"tags"`
}

The schema is specified: the JSON must be [{"id":..., "name":...}, {"id":..., "name":...}...]. I still get faster unmarshalling with the new API, but the legacy API with the legacy engine is faster when marshalling.

The original implementation is faster. The Go 1.27 release notes said that marshal performance is broadly at parity with the previous implementation. For my test, it is not the case.

So unmarshalling gets faster across the board with encoding/json/v2, and marshalling gets faster for any, but it is about 1.5x slower for typed structs in my tests.

The code is available.

In Java, you find the location of a substring using indexOf.

String haystack = "The quick brown fox jumps over the lazy dog";
String needle = "fox";
int index = haystack.indexOf(needle);

Naively, you might implement indexOf by a loop inside a loop, like so.

int naiveIndexOf(String haystack, String needle) {
    for (int i = 0; i <= haystack.length() - needle.length(); i++) {
        int j = 0;
        for (; j < needle.length() 
          && haystack.charAt(i + j) == needle.charAt(j); j++) {}
        if (j == needle.length()) { return i; }
    }
    return -1;
}

The Java implementation is much more sophisticated, and it is highly accelerated.

However, there are pathological cases where the Java implementation can be slow. What do I mean? Well, you do expect that the search will be more and more expensive as the size of the string grows. Right? So if you search through a 1 kilobyte string and then search through a 10 kilobyte search, you would not be surprised if the latter takes ten times slower.

But what of the substring? If you search for short substrings (fox in my example), the everything is fine. But what if you search longer and longer substrings (fox jumps or fox jumps over)? If it gets more expensive when both the string and the substring get longer, then you have what we call a quadratic complexity. In other words, it is slow.

In Java, if n is the length of your string and m is the length of the substring, then the complexity of indexOf is O(n·m). And if you look at my naive implementation (naiveIndexOf) then you see that in the worst case, it might do up to close to haystack.length() * needle.length() comparisons, that is, it is O(n·m).

The exact implementation of the indexOf function depends on your CPU and Java version. I am using OpenJDK 25 on Apple Silicon (ARM). For my purposes, I will use as a haystack of n copies of a and for the needle, the same thing, but ending with a different letter.

// n > m
String haystack = "a".repeat(n);
String needle = "a".repeat(m - 1) + "b";

I measured OpenJDK 25 on an Apple M4 Max. The haystack is one megabyte. Numbers are nanoseconds per haystack character.

At m = 4096, a single indexOf over one megabyte takes 1.1 seconds.

Can you do better against such adversarial inputs? The textbook solution is the Two-Way algorithm of Crochemore and Perrin (1991). The implementation is simple and your favourite AI can code it for you in any programming language.

Two-Way stays at about 0.3 ns/character no matter how long the needle is. At m = 4096 it is about 3500 times faster than indexOf on the first-character adversary.

So, should you switch to Two-Way for everything? No. On random text, the indexOf function is much faster than Two-Way.

And Two-Way has to do non-trivial work before the search begins. So it has additional fixed overhead. It would lose most of the time in the real world, sometimes by a wide margin.

Should you worry about this? No. The indexOf function in Java is fine.

If an adversary can control the needle (substring), then make sure to reject long needles. Most of the time, we search for short sequences (say, less than 80 characters). If you are worried about your system crashing, you will put bounds on inputs in any case.

The Java source is available.

Further reading: Crochemore, M., & Perrin, D. (1991). Two-way string-matching. Journal of the ACM, 38(3), 650–674.

We are all familiar with IP addresses such as 192.168.0.1. They are typically written as four numbers in the range 0 to 255 inclusive, separated by dots. In C#, you can parse them with the standard library using IPAddress.TryParse.

Pedantic people are quick to point out that IP addresses can take different forms: they can be IPv6 or IPv4 and there are many weird ways to write an IPv4 address. But for the purpose of performance optimization, we care about the common case. The common case is strings such as 192.168.0.1 or 12.121.244.111.

Our processors are capable of data parallelism, meaning that they have instructions (called SIMD) that can process several bytes at once, at least 16 bytes, sometimes more. A few years ago, I showed that you can parse IPv4 addresses with SIMD. I have been revisiting this idea with AVX-512, the instruction set that recent x64 (AMD/Intel) processors support. I expect that all Intel and AMD processors made in the near future will have great support for AVX-512, and it is already the case for server processors and recent AMD processors.

So I wondered, could we do it in C#? People are sometimes surprised that I care about C#. Isn’t that more Microsoft slop? No. Not at all. C# and .NET are very reasonable, portable systems.

Plus you can write fast code in C#. I have two optimized libraries that I hope the Microsoft .NET team will one day adopt in the standard .NET library: an optimized Utf8Utility.GetPointerToFirstInvalidByte function used internally to validate Unicode strings (in the SimdUnicode library) and a fast base64 decoding library. I love working with .NET C#.

As of .NET 10, we have AVX-512 support, including masked loads. What are masked loads and why do they matter? Suppose that I give you a string that is no longer than 16 bytes, but could be shorter. If you load data in a SIMD register, you normally have to load the full register width (so 8, 16, 32, 64 bytes). So what do you do when it is not possible? You can pad the input string or pull other tricks, but it gets dirty. A nice approach is to have masked loads where you, say, load the full register (say 16 bytes), but you indicate which bytes you want to be loaded from memory with a mask. So if you use 0b10011 as a mask, then only the first, second, and fifth bytes are loaded from memory. This makes it possible to initialize a 16-byte register with a string that has between 0 and 16 bytes, while never reading beyond the string. I have an article entitled Modern vector programming with masked loads and stores if you want to know more.

To make things trickier, C#, like Java and JavaScript, defaults to UTF-16, meaning that each character, even if it is an ASCII character like A or 1, uses two bytes. The ASCII codepoint value occupies the least significant bits of a 16-bit word.

So what we need to do is to selectively load from a 32-byte input, and then drop the unnecessary zero bytes. The gist of it looks as follows in C#.

unsafe bool TryParseAvx512(ReadOnlySpan<char> s, out uint ip) {
        int len = s.Length;
        fixed (char* cp = s)
        {
            // next two lines are a trick to load just the first len characters
            Vector256<ushort> charMask = Vector256.LessThan(CharLaneIndex, Vector256.Create((ushort)len));
            Vector256<ushort> chars = Avx512BW.VL.MaskLoad((ushort*)cp, charMask, Vector256.Create((ushort)'0'));
            // check that everything is ASCII otherwise, it is not an IP!
            if (Avx512BW.VL.CompareGreaterThan(chars, Vector256.Create((ushort)0x7F)).ExtractMostSignificantBits() != 0)
            {
                return false;
            }
            // There we go, we have the address as ASCII
            // in a 16-byte register.
            Vector128<byte> str = Avx512BW.VL.ConvertToVector128Byte(chars);
            // ...
        }
}

This looks a bit difficult to read, but that’s fine. Most people never need to worry about such code.

Then we use a somewhat fancy trick where we locate the dots, and use the fact that there are only 81 ways to position the dots. We then move the bytes, do a dot product and validate. It is the same routine as the C++ code. It is not trivial, but I am working on a formal paper to document the tricks used.

The pedantic people will say: wait, there are other ways to write IP addresses !!! Ok fine. We handle them with a fallback, like so.

if (TryParseAvx512(s, out uint ip))
{
    address = new IPAddress(ip);
    return true;
}
return IPAddress.TryParse(s, out address);

What about the cases where your processor does not support AVX-512? C# makes this dead easy. You can just guard it with one if:

if (Avx512BW.VL.IsSupported) { ... }

To benchmark this, I generated 10,000 random 32-bit addresses and parsed the resulting strings 20 million times, constructing an IPAddress each time. On a relatively recent Intel processor (Intel Xeon Gold 6548N, Emerald Rapids) running .NET 10, I get the following.

So the AVX-512 approach is about three times faster than the standard library. My routine itself does not take fourteen nanoseconds; there is other overhead.

As usual, the C# source is available.

Like most programming languages, Go has both stack allocations, whose lifetime is limited to the current function, and dynamic (or heap) allocations.

The name stack comes from the fact that the memory management is somewhat trivial. There is typically one stack per thread (or goroutine in Go). When a function needs memory, it simply appends data to the stack. When the function returns, the memory is dropped from the end of the stack. So the memory last allocated is deallocated first.

Heap memory is potentially considerably more complex. For one thing, it is meant to be accessible by several threads (or goroutines). An object can be allocated by one function and later reclaimed after an entirely different function, possibly running on a different thread (or goroutine), has dropped the last reference to it. Unlike the stack, there is no prescribed order for allocating and reclaiming heap memory. In Go, the garbage collector does the reclaiming.

Typically, stack allocations have a size known at compile time. Many systems give each thread a fixed-size stack, although Go grows goroutine stacks as needed.

There are many ways in Go to do a heap allocation. A common one is when you allocate a slice, as in this instance where you allocate memory for 100 integers:

x := make([]int, 100)

If the slice x is not entirely local to a function, Go will typically just allocate it on the heap. It will do so similarly when a function returns a pointer. For example, in the following instance, I assign the value 1 to a local integer variable, but I return a pointer to it.

func f() *int {
  x := 1
  return &x
}

In C/C++, this would be quite bad. You should get a warning such as address of local variable 'x' returned. In Go, the variable x will typically get allocated on the heap.

In many Go programs, we end up doing a lot of heap allocations of small objects. It can become a bottleneck in some cases. Think about when you are maintaining a tree or a linked list where each value (node) is an object that must live on the heap. If the data structure is highly dynamic, you will be constantly allocating these small objects.

Memory allocation on the heap is usually not done at arbitrary sizes. You often cannot get exactly, say, 13 bytes. In Go, small allocations are rounded up to a size class: 8 bytes, 16 bytes, 24 bytes, 32 bytes, and so forth. There is also some overhead to each heap allocation, from rounding and from allocator metadata.

The compiler knows the size of the object, but prior to Go 1.27, Go would call a generic function when doing a heap allocation. This generic function would then look up the size class and take the corresponding path. Starting with 1.27, for small objects (under 80 bytes), Go relies on dedicated functions.

It is easy to benchmark in Go. A basic benchmark might look as follows.

type Node struct {
    value int64
    next  *Node
}
var sink any
func BenchmarkAllocNode16(b *testing.B) {
    for b.Loop() {
        sink = &Node{}
    }
}

On my MacBook, the results are quite telling. Go 1.27 is nearly twice as fast!

This will not help all software, just the components that do many small allocations.

The code is available.

AI-assisted programming is fast evolving and there is a tension between ‘we no longer need to understand the code’ and ‘what is my purpose as a programmer’. I recorded a short video on this topic with how I think the tension can result in conflicts.

When a compiler optimizes your program, it has to guess. Which functions are worth inlining? Which side of a branch is the common one? Which method does this interface call actually reach? At compile time it cannot know, so it uses heuristics. Profile-guided optimization (PGO) replaces the guessing with measurement: you run your program, record where it spends its time, and hand that recording back to the compiler for a second build.

PGO is a common feature of compiler systems. Google applied PGO to Chrome under Windows in 2016, reporting gains of up to 15%. I expect all mainstream Web browsers to be built with PGO.

There are now fancier techniques than mere heuristics with PGO. You can use AI to recognize patterns and so forth. But they are not always widely available.

Go has supported PGO since version 1.20. You collect a profile, and pass it to the compiler.

A CPU profile is a statistical record of where a program spends its time. While the program runs, the Go runtime interrupts it about a hundred times a second and writes down the call stack at that instant. After a few seconds you have thousands of such samples, and counting them tells you which functions were executing and who called them. In Go you produce one by wrapping the work you care about:

f, _ := os.Create("cpu.pprof")
pprof.StartCPUProfile(f)   // from runtime/pprof
defer pprof.StopCPUProfile()

The compiler reads the call-stack counts and uses them for two things above all: inlining call sites that turn out to be hot, and devirtualizing interface calls whose target is nearly always the same concrete type.

I took three JSON documents that I wanted to parse:

• twitter.json (632 kB), a nest of small objects with short string keys
• canada.json (2.25 MB), essentially one enormous array of floating-point coordinates
• citm_catalog.json (1.73 MB), deeply nested objects with numeric keys

I parse each of them with the standard library’s encoding/json into an interface{}. The baseline, with no profile, parses at 112 MB/s for twitter.json, 74 MB/s for canada.json and 116 MB/s for citm_catalog.json.

The procedure is three commands:

go build -o bench .                        # ordinary build
./bench -profile cpu.pprof -train twitter.json   # collect a CPU profile
go build -pgo=cpu.pprof -o bench_pgo .     # build again, with the profile

I did it three times, profiling each document on its own, and then measured all three documents against each of the three builds.

Each panel of the figure is one document being parsed, and the three bars inside it are the three PGO builds: the binary trained on twitter.json, the one trained on canada.json, and the one trained on citm_catalog.json. Bar height is the speed gain over the ordinary, profile-free build of that same document, in percent, so zero means PGO changed nothing and a bar below the axis means the PGO build was slower. The green bar in each panel is the matched case, where the profile was collected on the very document being measured.

The gains are modest. The best result is canada.json at +4.7%, and most differences are in the 2–3% range. Profiling one document usually helps the others, but not reliably. Profiling twitter.json gave a decent improvement everywhere: +3.1%, +2.0%, +2.8%. But profiling canada.json bought 4.7% on canada.json and essentially nothing anywhere else. Interestingly, profiling citm_catalog.json produced a mere +0.8% on its own document while helping twitter.json more.

A 3% speedup is not exciting in isolation, but it may come nearly for free. Observe how you may get slightly negative results for cases you did not train for. That’s expected generally, but the effect is modest in the case of Go because its optimizations are themselves modest in the first pace. That is, you are not getting a much an effect, but the process is less likely to backfire for other workloads.

The code is available.

C++26 adds a new container to the standard library: std::hive. It is meant to occupy the ground between std::vector and std::list. Like a vector, it keeps its elements in contiguous blocks of memory, so scanning it does not require you to chase a pointer for every element. Like a list, it never moves an element once it has been inserted: your pointers, references and iterators stay valid, and you may erase any element in constant time without disturbing the others.

Internally, a hive is a linked list of blocks. Each block carries a skipfield: a small integer per slot that tells the iterator how many erased slots to jump over.

No standard library ships std::hive yet to my knowledge. Fortunately there is an implementation (plf::hive by Matt Bentley) as a single header file that you can use today.

I use elements of type uint64_t, GCC 16.1 with -O3 -march=native, on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core. Numbers are nanoseconds per element, along with the cycles and instructions retired per element.

We start from an empty container and append a million values. The container is then destroyed.

A std::list needs one allocation per element, and glibc’s malloc and free together cost over 200 instructions per element. It is an order of magnitude behind everyone else. That is not news.

The interesting comparison is vector against hive. A hive is about twice the cost of a vector, and it needs twice the instructions. This is the price of the skipfield: every insertion writes an element and a skipfield entry, and maintains the block bookkeeping. Note that calling reserve on a hive does not help in my experiments.

Next we iterate over the the container and sum the values.

A hive iterates no faster than a linked list here, slightly slower, in fact, and about eight times slower than a vector. (Update: Joseph Garvin points out that I measure the happy case for the std::list in this instance where all the entries were allocated in sequence. The worst case scenario for std::list when the nodes are all over the heap can be much slower.)

The vector loop retires one instruction per element and finishes in 0.78 cycles: the processor is executing several elements at once. This is possible because the std::vector implementation benefits from autovectorization: the compiler recognizes that it can load several words at once in wide (SIMD). Further, it does not have to check the bitfield like the std::hive data structure.

We can check this. Walk the same container with two independent iterators, one starting halfway in, and count the cost per element visited:

The vector does not care: it was already throughput-bound. The hive and the list get nearly twice as fast per element, because two independent chains can be in flight at once. Hive iteration is latency-bound, exactly like list iteration. It merely has better locality.

That locality does show up when the data gets big. At ten million elements the list falls apart while the hive holds steady:

Erasing is what a hive is for, so it would be unfair not to look. I erase half the elements at scattered positions using std::remove_if:

The hive wins, but by less than you might expect, and at ten million elements the ordering reverses (1.3 ns for the vector against 2.5 for the hive). std::remove_if is a single streaming pass, and streaming passes are cheap. Of course the vector moved every surviving element and invalidated every pointer into it, which is precisely what a hive promises not to do.

Memory, measured by asking glibc how many bytes it has handed out, per live element:

A hive costs about a byte per element over a vector, for a payload of eight bytes, when the vector is packed tight. A list costs more due to the overhead of the linked list.

A vector built by push_back has a capacity that typically exceeds its size. Thus even if you have 8-byte entries, you will use, on average, more than 8 bytes per entry even for large vectors. You can recover the excess capacity with the shrink_to_fit method.

What should we conclude?

The std::hive data structure is not a faster vector. But it is a much better std::list. It gives you the same guarantees that make people reach for a list, stable references, cheap erasure anywhere, while using less memory.

My source code is available.

When your program asks for memory that is not in cache, the processor has to go to RAM. That trip costs on the order of 100 nanoseconds. On a 3 GHz core, that is about 300 cycles of doing nothing.

Memory latency has not improved in ten years. The 2016 Broadwell answers a random access in 100 ns. The 2025 Turin, with DDR5-6400 and every advantage of a decade of progress, takes 140 ns. It got worse.

The good news is that a modern core does not have to sit still. It can issue a second request before the first one comes back, and a third, and a tenth. The number of requests a single core can keep in flight is its memory-level parallelism. It is one of the most important numbers in software performance, and one of the least advertised: you will not find it on a spec sheet.

Thankfully, memory-level parallelism has improved a lot. To measure it, I use my testingmlp benchmark. The idea is a pointer chase. We build a 1 GiB array containing a single random cycle covering every element: each element holds the index of the next. Following the cycle is inherently serial. Each load has to complete before you know the address of the next one, so a single chase measures pure memory latency and nothing else. Then we run several such chases at once, from different starting points on the same cycle. We call these lanes. With two lanes, the core has two independent loads to work on. With twenty, twenty. We increase the number of lanes and watch the throughput. When adding a lane stops helping, we have found the limit. As my metric, I use the total estimated bandwidth.

I ran experiments on the Amazon cloud (AWS). The bandwidth shape is the same everywhere: a steep, nearly linear climb as we add lanes, then a knee, then a plateau. 

How did it evolve over time? Intel went from 10 to 30, meaning that a single Intel core can sustain 30 memory requests at once in practice. AMD went from 15 to 58. Graviton went from 6 to 19.

Intel was flat for a long time. Broadwell and Cascade Lake both sit at 10 concurrent misses. Ice Lake doubled it to 20. Granite Rapids is at 30. Intel has roughly tripled in a decade, with all the gain arriving in the last two generations.

AMD started ahead and stayed ahead, then jumped. Naples was already at 15 in 2018, when Intel was at 10. Milan reached 22. And then Turin does something different in kind: 58 concurrent cache lines from a single core.

Graviton 1 was a toy: 6 concurrent misses. Graviton 2 doubled it, Graviton 3 went to 17, and then Graviton 4 essentially stood still at 18. Graviton 5 only reaches 19. But look at the latency panel: since 2017, Graviton 5 is the only chip in this entire collection that made a random access faster than its predecessor. AWS advertised better DRAM latency for Graviton 5, and that claim holds up.

So who wins? On bandwidth and memory-level parallelism, it is AMD, and it is not close. The Zen 5 core in the m8a instances sustains 58 concurrent cache-line fetches and 24.5 GiB/s of random-access throughput from one core. AMD is roughly twice as fast as Intel.

The raw output, the system information from each machine, and the scripts are in the usual place.

Note that Apple Silicon does even better, but it is another category.

Every week, I discuss with people who want to get a PhD. For years, I have been advising people not to pursue a PhD. It may come as a surprise to some.

You would expect people with a PhD to earn more money. Individuals who complete doctorates tend to have higher cognitive abilities and greater motivation. But smarter people tend to earn more, period.

So do people with a PhD earn more?

Historically, PhD holders earn more, but the bulk of the observed advantage is concentrated among those who get a professorship after the PhD. And there is no certain path from the PhD to a professorship. We have been producing many more PhDs than we have professorship, for decades. And the disparity is ever growing.

When I entered university at the beginning of the 1990s, about 0.5% of the Canadian population had a PhD. This has nearly tripled today, and it is fast increasing. Something of the order of one person out of 80 has a PhD. Comparatively, there is roughly one professor or university-level instructor per 900 people. With a fast aging population, we simply do not need many more professors and instructors than we already have.

There are specific fields where some jobs are difficult to get without a PhD. Machine learning is one such example. Many people in the industry have a PhD, and they tend to select those who also have a PhD. Further, there is a somewhat direct relationship between the work you might do during your PhD, if you are any good, and the actual work you might do later. It is much less clear in a lot of other disciplines.

The most significant economic cost of a PhD is not tuition but the years of delayed full-time earnings and career progression. In the tech industry, it is typical to award half a year of experience for each year spent on a PhD. This means that even though the individual starting with a PhD might earn more starting out, they are not necessarily getting a higher lifetime income.

Benjamin et al. (2025) find that the early-career benefits a PhD can be effectively zero:

In the short run, pursuing a PhD entails substantial opportunity costs. Early-career earnings for PhD graduates are significantly lower than those of individuals with master’s or professional degrees, reflecting prolonged enrolment and delayed entry into the labour market. These costs are especially high for non-completers, particularly those who exit the program after several years without earning a credential. Over the lifecycle, earnings do eventually recover (and surpass those of bachelor’s and master’s graduates) but only under specific conditions. The most favourable long-run outcomes are concentrated among those who secure academic employment and remain in full-time work late into life. This “double premium,” combining higher earnings and longer careers, plays a central role in shaping the average return to a PhD. Outside academia, PhD holders resemble master’s graduates in both earnings and employment patterns.

Thus, the financial case for a PhD is narrower than people assume. If you fail to get a professorship, or you want an early retirement, you may very well end up with a poor outcome. And it is not getting better over time.

References

• Altonji, J. G., & Zhu, Z. (2025). Returns to specific graduate degrees: Estimates using Texas administrative records (NBER Working Paper No. 33530). National Bureau of Economic Research. https://www.nber.org/papers/w33530
• Benjamin, D., Miloucheva, B., & Vigezzi, N. (2025). The opportunity cost of a PhD: Spending your twenties (Working Paper No. 802). University of Toronto, Department of Economics. https://www.economics.utoronto.ca/public/workingPapers/tecipa-802.pdf
• Cooper, P. Is grad school worth it? A comprehensive return on investment analysis. Foundation for Research on Equal Opportunity. https://freopp.org/whitepapers/is-grad-school-worth-it-a-comprehensive-return-on-investment-analysis/

In 1945, Vannevar Bush published a report entitled Science: The Endless Frontier. His thesis was that prosperity follows from basic research. The report was highly influential in the United States and elsewhere. It led to the creation of an entirely new government bureaucracy.

With this report, Bush popularized the linear model of innovation: innovation (such as medical cures) flows sequentially from basic research to applied research to development to production and diffusion. Grow basic research, and the rest will follow.

When Bush wrote his report, basic research was not usually supported directly by the state. We did not have a large basic research infrastructure. And yet, the West had just lived through an unprecedented period of rapid scientific progress: the theory of evolution, electromagnetism, radio communication, special and general relativity, quantum mechanics, nuclear technology, rockets, the combustion engine, and more. We would get the invention of the transistor only two years after Bush’s report. We also did not have today’s peer-review mechanism.

Even though Bush’s report has been viewed as a piece of genius that unlocked a golden era of scientific prosperity, I believe that the linear model of innovation is hopelessly naïve. I believe the thesis that a large bureaucracy delivering funding to other bureaucracies (such as universities) is how we get innovation is absurd. Except perhaps in the domain of computing (“bits”), we have been largely stagnant technologically since about the 1970s. So Bush’s model failed over time. To be clear, it might have worked for a while by encouraging more young people to study engineering and science. It may also have shone a favorable light on a few enterprising professors who got to promote useful ideas.

If you visit a research lab today in a leading university, what you are most likely to see is a boring bureaucracy that caters to whatever is politically favorable at the moment—a bureaucracy that plays it safe and avoids controversy. You see young people seeking well-paid jobs, going through the motions with often little genuine interest in, say, curing cancer. We have never published so many research papers—the volume has been growing exponentially ever since Bush wrote his report—but it is doubtful that this is how technological breakthroughs are achieved.

The evidence is overwhelming that shoddy science is widespread. We have a severe reproducibility crisis: if you redo an experiment (even a highly cited one), you are likely to fail to reproduce the results. This affects psychology, medicine, and many other fields. The system does not particularly care because the incentives to get things right are not there. As long as the work is politically aligned, solidity of the results seems secondary.

There was a TV show (The Big Bang Theory) where the main character, Sheldon Cooper—an awkward genius—gets to work on crazy ideas. That is how Bush imagined it: fund young people like Sheldon Cooper, and you will get extraordinary breakthroughs. In the real world, Sheldon would not get very far on campus. I have met misfits like him. When they are incapable of playing the political game, the system crushes them. But even if that were not the case, extraordinary intelligence needs to be applied to the right problems to be of value. You could have a ChatGPT that is brighter than any of us in every possible way, and it could still be deployed simply to fill out forms faster and better than we do—it may not cure cancer.

The American government has just released what might be considered an update to Bush’s report. Michael Kratsios wrote a report entitled Science, A New Golden Age. The report states outright that the linear model no longer holds. It states what I have argued for vehemently: innovation is not a linear process. Take large-language models, for example, which can be used by engineers and scientists to further their research. I have also argued that the success of large-language models today has as much to do with the users as with the researchers.

At this point, some people engage in the following type of rhetoric: if we had not invented calculus, we would not have AI today; therefore, calculus caused AI. But you could also say that the subsidized nail factory in the Soviet Union, which made overpriced and bad nails, was necessary to hold Landau’s house together, and that without those nails we would not have the theory of Landau levels. The causality argument goes in all directions.

Innovation is the result of a complex system. We see that the United States and, more recently, China are innovative countries. In 2026, you do not go to France for the latest advances. The evidence is overwhelming that scientific and technological progress depends as much on culture as on anything else. It is not something to be managed by bureaucrats.

One of the cultural ingredients that seems essential is meritocracy. You must put the people who are good at building on top of your hierarchy. This does not happen magically. You need a set of incentives in which rewarding the wrong people is costly.

What does Kratsios propose? Many interesting ideas that, I expect, could renew our culture. He proposes to break out of the Cold War–era funding model. Today, the research funding mechanism is centered around the government giving money to the university bureaucracy. The grant might be in the name of one professor, but the recipient is still the university. In the new model, instead of funding universities, the government would assign money directly to individuals in various ways (short grants, prizes, and so forth). This would shift power away from administrators toward individuals who know how to get things done. It would also neutralize some of the political power of the current mandarin class of scientists who control access to the top positions.

The report recommends restoring permissionless innovation. It is sometimes poorly understood how limited the system has become. I once had a graduate student undertake interviews with practitioners. This required an ethics approval which, in her case, took a few months to obtain. Again, the system has built up political structures that seek to block innovation it does not like. They need to be torn down, the sooner the better.

The report has many other interesting recommendations. One that I particularly like is an AI-guided agenda. We need to hook up our brand-new AIs to experimental devices. We are not going to cure aging with chatbots. We need experiments on a massive scale.

Will Kratsios’s vision move from report to reality? History shows that cultural and institutional change is never easy. Yet the stakes could not be higher. By embracing meritocracy, permissionless innovation, and ambitious AI-augmented experimentation, we have a genuine chance to escape decades of stagnation and rekindle the spirit of discovery that once defined the West. The opportunity is before us. It must not be squandered.

When my kids were young, instead of helping them study algebra and grammar, I wrote small JavaScript apps for them to use. I built a small collection of educational tools.

The great success story of AI for me is exactly this: AI helps you write your own tools, faster and better.

Last night, I was struggling with videos I had to process. I wanted to add nice subtitles to them. There are software applications for that, but they require manual labor and don’t always work the way I want them to. After a long night, I had an insight: why don’t I ask my AI to help build the automated tool I need? So I did—and it worked really well, very quickly. So what’s the lesson here? Maybe that we should spend more time building our own software for our own personal use than we used to.

I have been on X for a long time. Like most people who post regularly, I have a gut feeling for what might interest people. I post in the morning. Longer posts seem to do better.

But gut feelings are not measurements. And until recently, digging into your own posting data meant either clicking around the web UI or writing custom scripts. Neither is particularly friendly when you want to ask ad hoc questions with an AI assistant.

X recently launched hosted MCP servers: official endpoints that AI tools can connect to. MCP is a protocol for plugging tools into language models: the model can search posts, manage bookmarks, fetch trends, and so on. In practice, I connected an AI coding agent to the X MCP server and simply started asking questions about my account.

I spent a session exploring about two months of my own activity. Here is what I found interesting.

Over roughly sixty days (mid-May through mid-July 2026), I published on the order of 435 posts that were not pure retweets of other people—mostly a mix of original posts, replies, and a few X Articles. The agent pulled them through the MCP tools, kept the public metrics (likes, views, reposts), and ran simple analyses.

I asked for every post to be binned by local hour of day (America/Toronto, Eastern time), and for each hour: how many posts, and the min / median / max view count.

My posting is heavily skewed toward the morning:

The 9 a.m. hour is both my busiest and, among busy hours, my strongest by median views. The overall median across all hours was only about 188 views, so most of what I write is quiet. The distribution is heavy-tailed: a few posts get tens or hundreds of thousands of impressions; the rest are background noise.

I then binned posts by character length in steps of 25 characters (using the text as returned by the API, including short t.co URLs).

The bulk of my writing is short, often a reply of a few dozen characters:

Under about 175 characters, the median stays at zero or one like. Around full-length posts (roughly the old 280-character regime and a bit beyond), engagement jumps. The 300–325 character band is where a large fraction of my “serious” posts live, and the median likes there are an order of magnitude higher than for short replies.

I also asked the AI to identify the posts that had the most likes, the following types of posts were liked:

• AI vs. “experts” claiming models are nowhere near human intelligence
• Go adding SIMD-style data-parallelism to the standard library
• SIMD-accelerated data processing talks and library notes (JSON, string→integer maps, vulnerability-report fatigue)
• Nvidia hardware, university AI-cheating, C++ contracts

The interesting part is the workflow. I did not export a CSV by hand and open Excel. I asked an agent, connected to X’s MCP server. If AI agents can do this for one account’s metrics, they can do it for bug trackers, logs, paper drafts, and codebases.

When I was a kid, most people did not know how to type. We took typing class. The final exam was a speed test: words per minute. Today, you will not impress anyone by saying you can type. In fact, cursive writing is fading. Kids increasingly cannot read or write it. We type constantly. We forget how many skills are learned, and how often some of these skills have faded.

But not everything fades. Socrates would be immensely popular today as a teacher. I still buy and recommend paper books.

Is reading and writing code more like Socrates, or more like cursive writing? There are clear signs that code could become like cursive writing. This year, I have met more than one student who could use AI to build an application but could not read or write code. It is not new. Software has long had non-technical people who describe what they built or designed. In fact, in much of the industry, the standard view was that once you had a university degree, you no longer coded. Coding was for monkeys or low-status employees. Top engineers paid a million dollars a year at Google or Meta know how to write code. They often read and write assembly and TypeScript. They know it all.

Why the discrepancy?

We pay an engineer a million dollars because he understands concepts few others grasp. He outruns others because he sees the problems more deeply. Reading and writing large amounts of code is part of how you gain those insights. Chatting with an AI will not make you a top 1% programmer. In the future, top engineers might read more code than anyone could in the past. These engineers will not be everywhere, but they will pack a punch. “But Daniel, people say programming is solved. Why read or write code?” Be careful with your models. When television arrived, some predicted it would replace the university lecturer. In some respects the model was correct, yet it did not happen. The lecturer’s job was never to deliver a TV show. The Google engineer paid a million dollars was never a machine that produces code. Nobody actually wants code, any more than they want raw text.

In fact, I predict a bifurcation in the tooling. The best engineers will work with tools that maximize their understanding of the code. I believe that reading and writing code, at a high level, is more like studying Socrates than like cursive writing. It is a necessary mental labor that does not become obsolete just because we have better tools for generating output.

Suppose that you have a configuration file in JSON. Something like this:

{ "width": 1920, "height": 1080, "fullscreen": true,
  "title": "My Game", "volume": 0.8 }

Normally you ship this file alongside your program, open it at startup, read it, and parse it. That is a lot of work for data that never changes. What if the file is fixed at build time? Could the compiler read it, parse it, and bake the result directly into the executable as a constant?

With C++26, the answer is yes. We need two new ingredients, all of which are usable right now with the latest version of the GCC compiler (16).

1. #embed to pull the file into the program at compile time,
2. A software library supporting static reflection like simdjson.

Let me show you how far we can take this.

The new #embed directive reads a file and expands it into a comma-separated list of byte values. To read the file data.json at compile time and keep it around as a constant, we write:

constexpr const char json_data[] = {
#embed "data.json"
    , 0
};

I use constexpr because I want the compiler to be allowed to inspect these bytes during constant evaluation. The trailing , 0 simply appends a null terminator, so the array can be treated as an ordinary C string.

There is no run-time input/output of any kind. The bytes are part of the program.

But embedded bytes are not yet useful by themselves. What I really want is a typed C++ object. In my example, the target type is this configuration struct:

struct Window {
  int         width;
  int         height;
  bool        fullscreen;
  std::string title;
  double      volume;
};

The traditional way to populate such a struct from JSON is to write, by hand, one line per field: read "width", store it into width, read "height", store it into height, and so on. It is tedious. And because it runs at startup, a malformed file becomes a run-time error, discovered by your users rather than by you.

Recent versions of simdjson can parse JSON at compile time using C++26 static reflection. The entry point is simdjson::compile_time::parse_json, and it does something I still find slightly magical: it reads the JSON and, from the keys it finds, and synthesises the struct type for you.

#define SIMDJSON_STATIC_REFLECTION 1
#include "simdjson.h"
constexpr const char json_data[] = {
#embed "data.json"
    , 0
};
constexpr auto window = simdjson::compile_time::parse_json<json_data>();

The variable window is a value computed entirely by the compiler. Its type is generated from the document: it has a width and a height (both 64-bit integers), a bool fullscreen, a double volume, and a title. From here on I write window.width and it behaves like any ordinary field.

How do I know the parsing really happened at compile time? Because I can assert things about the result that the compiler must check before the program even exists:

static_assert(window.width      == 1920);
static_assert(window.height     == 1080);
static_assert(window.fullscreen == true);

If I corrupt the JSON — delete a brace, misspell true, leave a trailing comma — the program no longer compiles, and the error points at the parse_json line. The broken file is caught at build time, on my machine, instead of at startup on someone else’s.

Because window is a genuine compile-time constant, any computation over it is a constant too. Consider this function:

int  screen_area()   { return window.width * window.height; }

Compiled with -O3, there is no multiplication, no field access, and certainly no parsing left — only the answers, as immediate values (here on my macBook):

screen_area:    mov  w0, #0xa400        // 0x1fa400 = 2073600
                movk w0, #0x1f, lsl #16
                ret

The JSON has vanished from the binary. It was read and parsed exactly once, by the compiler, and all that survives is the number 2073600.

Because static reflection is so new, when building with GCC 16, you need to pass the flags -std=c++26 -freflection: the -freflection flag is necessary to activate compile-time reflection You must also set the simdjson macro SIMDJSON_STATIC_REFLECTION=1 before importing the simdjson.h. It is a temporary safeguard.

The source code to reproduce these examples is available.

Reference: P2996 — Reflection for C++26 and the simdjson library.

Credit: The simdjson implementation is joint work with Francisco Geiman Thiesen.

The keyword in politics these days is ‘sovereign’.

What few will admit is that it is effectively the adoption of the American strategy: Make America Great Again. In other words, reindustrialization of key sectors of the economy. The UK used to be a computing champion. Our chip designs (ARM) originated from the UK. Canada had BlackBerry, everyone was using Canadian phones.

Like Canada, many countries have progressively slid into financialization. Huge banks and bank-related businesses, surrounded by emptied factories.

Part of it was the doing of economists who promoted globalization. We are going to make our best CPUs in Taiwan, because they have a comparative advantage (whatever that means).

Another part is the rise of the managerial class, or our version of the technocracy: the summum of the status game is to make PowerPoint presentations in a nice office. Everyone has 2 or 3 university degrees. And if you don’t have many degrees, what is wrong with you?

I think that this is breaking apart for a few reasons.

One of them is Trump. And I don’t mean bombastic statements, bad hair color or overly long ties… But rather the realization that globalization might leave your people economically better off for a time because they have cheap stuff… But it also leaves your people with few skills. You can fund a robotics factory near Montreal, and you’ll find 2000 people with robotics PhDs, but nobody actually knows how to build robots. The analogy is thus: salary is not everything (to the great chagrin of economists). I have left a much better paying job. My previous job meant that I had to sit in an office and do little concrete. I would have had a great and early retirement… but I would never have developed my skills nor would I have built anything. And that’s what we did at the country level. Great total compensation, but a dead-end skill-wise.

Another factor, I believe, is the COVID era (2020-2023) and its final outcome: empty offices, closed coffee shops. What happened at work was illegible. Lots of people in offices. Certainly, something important was happening. Entire businesses and government organizations have now migrated partially or entirely to a pajama party of some kind. Netflix in the middle of the workday is no longer a dream, but a reality.

Another element is educational misalignment. A country like Canada has the most schooled population in history. You cannot throw a rock without hitting someone with a PhD. Meanwhile, we are not making robots or microchips. You can’t even pay with your phone in the Montreal subway. It is a project for another decade, maybe. We are using a push strategy: push more people with degrees into the economy and you are going to get a fancier economy. Won’t work.

Finally, the AI breakthrough of 2022 is the final nail in the proverbial coffin. My country (Canada) claimed for decades that we were the AI powerhouse. All these PDFs online can’t lie, can they? Canada basically invited modern AI, didn’t it? We did. On paper. On paper we did a great many things. In practice? Few know how to build anything.

In a country like Canada, the population has not yet caught up. They blame the orange man for whatever trouble they see. And the politicians promise to do what they must: shower money to build tech sovereignty. It won’t work. They will try again. It won’t work again. The cycle makes things worse because it sustains a managerial class that is great at politics but terrible at building.

Meanwhile, you can’t escape preference falsification. People will use ChatGPT, Claude, Grok, Gemini. And if Elon produces robots, they’ll want them.

Trump will leave office in two years… Canada and the UK will still be flat-lined economically. The USA will still surge ahead.

Countries like Canada and the UK will have to realize that it is industry and know-how first. Build stuff and the wealth will follow. Stop the virtue signaling. Stop the credentialism. Build.

Our 64-bit Intel and AMD processors have evolved over decades. When you compile a Go program for a 64-bit Intel or AMD processor, the compiler targets, by default, a nearly 20-year-old instruction set. The binary that comes out runs on essentially any x64 chip, but it also leaves on the table every instruction that was added since 2003.

We often refer to microarchitecture levels. Each level bundles a set of instruction-set extensions that you can assume are present:

In my view, this ladder is already slightly obsolete. It was frozen around 2020, and the hardware has moved on. We would need to add the latest AVX-512 sub-extensions (VBMI, VBMI2, VNNI, BF16, FP16, VPOPCNTDQ, and so on), which recent server and consumer chips support but which v4 does not require. While v1 through v4 are a useful common language, a realistic “use everything this CPU offers” target today would need at least a v5, and arguably the whole scheme should be replaced by finer-grained feature detection.

In any case, the Go toolchain exposes this v1 through v4 ladder via the GOAMD64 environment variable. Setting GOAMD64=v3 tells the compiler it may use everything up to and including AVX2. The default is v1, the lowest common denominator.

This raises an obvious question. If I take a real, performance-sensitive library and recompile it at each level, how much do I actually gain? I picked Roaring Bitmaps, a compressed bitset data structure used in databases and search engines.

A Roaring Bitmap stores a set of 32-bit integers. It splits the 32-bit space into chunks of 65,536 values, keyed by the high 16 bits, and stores each chunk in a container that holds only the low 16 bits. A container comes in one of three shapes, and the library always keeps whichever is smallest:

• an array container: a sorted list of 16-bit values, used when the chunk is sparse (a few thousand elements at most);
• a bitmap container: a flat 8 KB bit vector (65,536 bits, one per possible value), used when the chunk is dense;
• a run container: a list of [start, length] intervals, used when the set bits cluster into consecutive runs.

I fetched the latest release of the library, then ran its own benchmark suite four times, once per level, collecting eight samples each. I did this on a single Intel Xeon Gold 6548N (Emerald Rapids, which supports all four levels, including AVX-512) under Go 1.26.2 and Roaring v2.18.2.

A population count (or popcount, also called the Hamming weight) is simply the number of bits set to 1 in a machine word. Roaring leans on it constantly: the cardinality of a bitmap container, how many values it holds, is the sum of the population counts of its 1024 64-bit words. Modern x86 chips have a dedicated popcnt instruction that does this in a single operation, but it only became available at the v2 level (SSE4.2, 2008). Without it, the compiler has to fall back to a multi-instruction bit-twiddling sequence.

The clearest single result is population count: counting the number of set bits in a bitmap container. The v1 baseline cannot use the popcnt instruction, so Go emits a software fallback. The moment we move to v2, popcnt becomes available and the time is cut almost in half:

That is a 43% reduction, and it is free: no source change, just a compiler flag. Notice, though, that v3 and v4 do nothing more. A single popcnt instruction is already optimal; as far as the Go compiler is concerned, AVX2 and AVX-512 have nothing to add.

Population count is the easy win. What about the rest of the library?

Another clear win is building a container from a dense bitmap. The FromDense array benchmark takes a raw 8 KB bit vector and constructs the most compact container for it: it popcounts every word to learn the cardinality, then scans out the positions of the set bits. That word-at-a-time popcount-and-scan loop is exactly what the compiler can auto-vectorize once 256-bit registers are available, so the gains keep coming past v2:

v2 already cuts 21% by using scalar popcnt/tzcnt instructions, and v3 (AVX2) nearly doubles that to a 38% reduction. As with popcount, v4 adds nothing.

Set operations show the same pattern. The IntersectionCardinality benchmark counts how many values two bitmaps have in common: for bitmap containers, it ANDs the words pairwise and population-counts the result, without ever materializing the intersection. Here v2 does essentially nothing (the scalar popcnt is already in the inner loop), but v3 lets the compiler widen the AND-and-count loop to 256-bit registers, cutting the time by 22%:

Takeaways:

1. On modern hardware, everyone should be using v2 or better. The resulting binary will run in any data center and on any non-ancient laptop.
2. The v3 level might be worth investigating.
3. The v4 level should have helped in some of my benchmarks, but it did not. I suspect that the Go compiler is just not great at it.

(Obviously: run your own benchmarks.)

Where is your intelligence located? In your brain?

It is a simplistic answer. A better model is that your intelligence is embodied.

Consider a cook working at an expensive restaurant. He has all his favorite knives and cooking instructions, placed exactly where he wants them. His kitchen is part of his intelligence, of his skills. The same cook working in your kitchen can probably cook better than you do, but he can’t reproduce the same meals he would prepare in his favorite kitchen.

We often assess computer programmers using whiteboard tests. It is an endless source of complaints. Programmers rightly point out that it forces them out of their element. They are just not as good when you take away their laptop. It is not an excuse, it is a real issue: you are cutting them off from part of what makes them so intelligent.

To sum it up, the model of intelligence as a brain in a jar, disconnected from anything else, is ridiculous.

If you accept the idea of embodied intelligence, then many actions that we view as a consequence of our intelligence are actually part of our intelligence. First and foremost, language. Our ability to talk or write to each other means that I am not limited by my own person. Have you ever heard of human beings isolated in small tribes making technological breakthroughs? Nah. Progress requires lots of people communicating together. Up until a few decades ago, progress required cities. Today I am less certain than it does, as I can more and more communicate with anyone in the world from anywhere. But language is still critical, we have not invented anything better. Similarly, having hands and the ability to build sophisticated tools (like laptops) allows us to extend our intelligence.

At the end of 2022, we got a breakthrough technology: ChatGPT. It built on several pre-existing ideas such as (large) language models, neural networks, and so forth. That’s the ‘GPT’ part. But an important, if underappreciated, part of the breakthrough was the ‘Chat’ component. Someone had the idea of connecting a large language model with a chat interface. Maybe this came naturally and obviously to people building this system, but it should not be assumed to be trivial or unimportant.

Language is a key component of our intelligence, and, thus, it makes sense that it would be pivotal for machine intelligence.

We embodied the AI software in a chat box.

The next step was what we call today ‘agentic AI’. We keep the chat box, but we add the ability for the AI software to interact with tools, and to make plans to use them. In effect, we give the AI more agency: it can do stuff and learn from the results as they happen. It is starting to resemble a human being with hands and tools.

I was talking with a colleague this week. My colleague is all in on the AI revolution. He uses his AI to help him write better and faster, and to get his data analysis done faster, without so much help from technical experts.

But my colleague was not aware of the agentic AI approach. I tried to explain on the phone. What does it mean to give the AI access to tools? Is this only about saving the effort of copying and pasting the AI’s response?

I ended up making a video where I start an AI in a shell within something called RStudio. It is an environment people use to program in R, to do data analysis. I don’t use R or RStudio, but thanks to the AI, I was able to build an entire climate research project in a few minutes, complete with the retrieval of the data from the web.

How did the AI do it? I recorded it. It tried a few things, initially struggling to download the data. At some point, it finds out that it needs new R packages, so it installs them, and once they are installed, it can proceed to generate figures, verifying that it works.

Agentic AI greatly extends machine intelligence by improving the embodiment of AI.

I believe that it is not yet understood as it should be.

In Montreal, the most established professor in the field of AI is Yoshua Bengio. He started his own non-trivial enterprise a few years ago (Element AI). His latest venture is Law Zero, which aims to create a Scientist AI. The first goal of this project is to build AI without the agentic component. It should be a disembodied AI that has no goal of its own, no agency.

I fear that Bengio suffers from what Kevin Kelly called Thinkism. Let me quote from Kelly’s 2008 essay.

No intelligence, no matter how super duper, can figure out how human body works simply by reading all the known scientific literature in the world and then contemplating it. No super AI can simply think about all the current and past nuclear fission experiments and then come up with working nuclear fusion in a day. Between not knowing how things work and knowing how they work is a lot more than thinkism. There are tons of experiments in the real world which yields tons and tons of data that will be required to form the correct working hypothesis. Thinking about the potential data will not yield the correct data. Thinking is only part of science; maybe even a small part. (…) Thinkism is not enough. Without conducting experiments, building prototypes, having failures, and engaging in reality, an intelligence can have thoughts but not results. It cannot think its way to solving the world’s problems. (…) The Singularity is an illusion that will be constantly retreating — always “near” but never arriving. We’ll wonder why it never came after we got AI. Then one day in the future, we’ll realize it already happened. The super AI came, and all the things we thought it would bring instantly — personal nanotechnology, brain upgrades, immortality — did not come. Instead other benefits accrued, which we did not anticipate, and took long to appreciate. Since we did not see them coming, we look back and say, yes, that was the Singularity.

I believe that University professors are especially prone to thinkism. They view intelligence as being centered on what is happening in their brain. When you live in an ivory tower, it is easy to dismiss the real world as the core source of intelligence. Further, they are often people who did quite well in school where thinkism is naturally prevalent.

I have been a professor most of my life. However, I tire quickly of talking with other professors. What I most enjoy is working with people who have new tools that they apply in the real world. Unsurprisingly, I spent most of my time working with software that people deploy in the real world.

What Kelly is saying is that a high degree of intelligence is not enough to do much of anything. The real world is not the final stage of your thinking process. It is maybe the most important part of it.

And thus, when you connect your AI with the real world, giving it the ability of running experiments (as virtually all software developers do today), you get impressive results that go much beyond what AI software can do on its own.

Agency is not a feature. Agency is primary.

Every machine connected to the Internet has an address called an IP address. Originally, these addresses were 32-bit integers (IPv4), giving a theoretical maximum of about four billion distinct addresses. We are all familiar with these addresses (e.g., 192.168.0.0). There was a big fuss about how we would run out of addresses. It never happened because we don’t actually need every device to have its own unique address. Your home router needs an address, but every device in your home does not need a worldwide unique address.

Nevertheless, the range was extended to cover 128 bits (IPv6). An IPv6 address is conventionally written as eight groups of four hexadecimal digits separated by colons. For example:

2001:0db8:85a3:0000:0000:8a2e:0370:7334

Because addresses often contain runs of zeros, the format allows two shortcuts:

• Leading zeroes within a group may be omitted: 2001:db8:85a3:0:0:8a2e:370:7334.
• A single run of all-zero groups may be replaced by ::: 2001:db8:85a3::8a2e:370:7334.

The double-colon trick can appear only once in an address, and can match one or more zero groups. Hence ::1 is the loopback address (all zeros except the last group), and :: is the unspecified address (all zeros).

IPv6 also accepts an embedded IPv4 address in the last 32 bits, written in the usual dotted-decimal form. This is mostly used for IPv4-mapped IPv6 addresses such as ::ffff:192.168.1.1. The longest possible textual form is 45 characters:

0000:0000:0000:0000:0000:ffff:255.255.255.255

So a parser must accept hexadecimal groups, the compressed form, and an optional IPv4 tail. It is more involved than parsing IPv4.

The standard C function for the job is inet_pton, available on essentially every system.

Can we do better?

A few years ago, I showed that you could parse IPv4 addresses really fast. Can we do the same with IPv6?

The trick is to use data parallelism: we invoke the so-called SIMD instructions that all our processors support. These instructions can process potentially dozens of bytes at once.

Shreesh Adiga gave it a try with AVX-512, the powerful instruction set supported by recent Intel server processors and all new AMD CPUs. The idea is to load the entire string into a 512-bit register, find the colons with a single comparison, compute the spacing between them to drive a byte-level expand, translate hex digits via a permute, and finish with a multiply-accumulate that combines the hex digits into bytes. Almost the whole parser is branch-free, meaning that there are few if clauses.

I put together a small benchmark that generates random IPv6 addresses with inet_ntop (so the addresses are written in their canonical, compressed form) and parses each one with both inet_pton and the AVX-512 routine. The benchmark runs on an Intel Xeon Gold 6548N CPU @ 2.8 GHz (Emerald Rapids) with GCC, compiled with -march=native -O3.

The AVX-512 routine is about 12 times faster than inet_pton, parsing more than 70 million addresses per second on a single core. It uses eight times fewer instructions, and runs them at a higher throughput (2.45 instructions per cycle versus 1.56).

The source code used for this benchmark is available on my blog repository.

Update. Peter Fors points out that the step in my benchmark, where I sum
up the bytes, adds some overhead especially under GCC. Thus I underestimate the speed slightly.

In software programming, the product between two integers is often computed to a fixed number of bits with overflow. Consider 8-bit integers. If you multiply 127 by 127, you get back the number 1 as an 8-bit unsigned integer, with an overflow. The actual full product is 16129. To represent 16129, you typically use 16 bits of precision.

Thus we have the notion of the full product. The full product of two 32-bit integers is typically represented using 64 bits. The question that preoccupied me is what fraction of all 64-bit integers can be written as the product of two 32-bit integers.

You might wonder why you would care?

We often design hash functions: they are special functions that take an input and generate a random-looking output. Several years ago I designed a very fast hash function called clhash. It is a super-fast hash function for strings having a few hundred bytes or more. If you don’t know about clhash, check it out. It is interesting in its own right.

This clhash hash function uses a type of multiplication typical of cryptographic applications. I was trying to argue that our approach had benefits compared with techniques based on standard multiplications. Let me illustrate. A simple hash function for 32-bit integers could take the least significant bits and multiply them with the most significant bits.

// simpleHighLowHash is a simple (and weak) 32-bit hash
// that multiplies the high 16 bits by the low 16 bits.
func simpleHighLowHash(x uint32) uint32 {
    high := uint16(x >> 16)
    low := uint16(x & 0xFFFF)
    return uint32(high) * uint32(low)
}

Maybe you’d want the hash function to be uniform: all possible 32-bit hash values should be equally probable. It is only possible in this instance if the hash function can produce all 32-bit hash values, which is not the case.

The great mathematician Erdös showed that the proportion of all 2n-bit values that can be generated by the product of two n-bit values goes to zero as n becomes large. This means that if you have, say, 10000000-bit integers multiplying 10000000-bit integers, you’d expect relatively few 20000000-bit integers to be produced. But what about practical cases like 32-bit integers or 64-bit integers?

You can just brute-force the problem easily up to the multiplication of 16-bit integers into 32-bit products. At that point, slightly one out of five 32-bit numbers is a product between two 16-bit integers. About 80% of all 32-bit integers are never produced by this hash. However, the running time grows exponentially, and brute force won’t scale all the way to 32 bits.

So what do we do about the 32-bit case? That is, what do you do when you multiply two 32-bit integers to produce a 64-bit product? What fraction of 64-bit values can the following function produce?

func simpleHighLowHash(x uint64) uint64 {
    high := uint32(x >> 32)
    low := uint32(x & 0xFFFFFFFF)
    return uint64(high) * uint64(low)
}

Can we get an exact result?

Yes!!!

Webster and his colleagues built the math to allow us to scale up the exact computation. He was kind enough to publish his code.

There are 3,215,709,724,700,470,902 64-bit (unsigned) integers that can be written as a product of two 32-bit integers. That’s about 17% of all possible values.

What about actually computing a pair of integers given their product? One approach consists of computing its full prime factorization, and then using those factors to build all possible divisors that are strictly less than 2^32, starting with a set of candidates containing only 1 and iteratively multiplying existing candidates by each prime factor (only keeping products that stay below 2^32). We can avoid adding duplicates to our set by processing unique prime factors with their multiplicity. Finally, we select the maximum such candidate m as the largest divisor under 2^32, compute the corresponding leftover n / m, and report whether a valid split into two 32-bit factors exists. In general, the answer (if it exists) is not unique: this returns the pair where one value is maximized. In Python, the code might look as follows.

for p in factor_multiplicities:
    new_candidates = []
    for c in candidates:
        for i in range(factor_multiplicities[p] + 1):
            if c * (p ** i) < 2**32:
                new_candidates.append(c * (p ** i))
    for new_c in new_candidates:
        candidates.append(new_c)
m = max(candidates)
print(f"Maximum candidate: {m}")
leftover = n // m
print(f"Leftover: {leftover}")
if leftover >= 2**32:
    print("Leftover is too large, cannot find a suitable candidate.")

You might be able to come up with a more efficient algorithm. I find it interesting to consider that if you pick a value at random, it will usually fail! That is, most 64-bit integers cannot be written as the product of two 32-bit integers.

Converting a 64-bit integer to its decimal string representation is a mundane task that shows up everywhere: logging, JSON serialization, CSV output, debug prints, etc. In C++, you might use std::to_chars, sprintf, or some library routine.

How do these functions work? At a high level, they repeatedly divide by ten. Start with your integer k. Divide it by ten, use the remainder as the last digit (it is between 0 and 9 inclusively). You then add the code point value of the character 0 to get the ASCII digit. To go faster, you can divide by 100 and use a lookup table so that the value between 0 and 99 inclusively is mapped to a string.

So far so good. Unfortunately, even with all these optimizations, this string generation may become a performance bottleneck. Can you do better?

Let us assume that you have a recent AMD processor or an Intel server. Then you have powerful data-parallel instructions (AVX-512) that can multiply eight 64-bit integers at once. We often refer to these instructions as SIMD (single instruction multiple data). My colleague Jaël Champagne Gareau and I recently published a new paper on exactly this problem. The title says it all: Converting an Integer to a Decimal String in Under Two Nanoseconds.

When you write n / 100 in code, an optimizing compiler converts the operation to a multiplication followed by a shift. It is often described as a multiplicative inverse. Generally, you can replace the division of n by d with the division of c * n or c * n + c by m for convenient integers c and m chosen so that they approximate the reciprocal: c/m ~= 1/d. We often call c * n + c a fused multiply-add. Picking m to be a power of two means that the division by m is just a shift. Then you can get the remainder of the division by using the remainder of the division by m, multiplied by d and divided again by m, which is essentially a multiplication followed by a shift (Lemire et al., 2021).

We can put this to good use with the Integer Fused Multiply-Add (IFMA) instructions available on recent Intel and AMD processors. They essentially allow you to compute eight instances of (c * n + c)/m in one instruction. The expression (c * n + c)/m gives you the division, but we need the remainder, so instead we pick (c * n + c)%m which we need to multiply by the divisor.

The fun thing with AVX-512 instructions is that they can use a different c and a different divisor for each of the eight operations. Using Intel intrinsic functions, our core routine which converts a value smaller than 10^8 to eight digits looks as follows:

__m512i to_string_avx512ifma_8digits(uint64_t n) {
  __m512i bcstq_l   = _mm512_set1_epi64(n);
  constexpr uint64_t twoto52 = 0x10000000000000ULL; // 2^52
  __m512i ifma_const = _mm512_setr_epi64(
    twoto52 / 100000000, twoto52 / 10000000,
    twoto52 / 1000000, twoto52 / 100000,
    twoto52 / 10000, twoto52 / 1000, 
    twoto52 / 100, twoto52 / 10
  );
  __m512i zmmTen    = _mm512_set1_epi64(10);
  __m512i asciiZero = _mm512_set1_epi64('0');
  __m512i lowbits_l  = _mm512_madd52lo_epu64(ifma_const, 
    bcstq_l, ifma_const); // ifma_const * bcstq_l + ifma_const
  __m512i highbits_l = _mm512_madd52hi_epu64(asciiZero, 
    zmmTen, lowbits_l);
  return highbits_l;
}

It compiles down to two multiplication-add instructions: vpmadd52huq. That’s it. Two instructions to generate eight digits.

It works by broadcasting n across all eight 64-bit lanes of a __m512i vector (bcstq_l). It then prepares a vector of carefully chosen multiplicative inverses (ifma_const) that represent the reciprocals of 10^8, 10^7, … The magic happens in the single _mm512_madd52lo_epu64 instruction, which simultaneously performs eight fused multiply-adds: each lane computes (ifma_const[i] * n + ifma_const[i]) using 52-bit low-half multiplication, effectively extracting the quotient when dividing by the corresponding power of ten. A second _mm512_madd52hi_epu64 instruction (with a vector of ten and a vector of '0') then isolates the digit values and adds the ASCII '0' offset in the high 52 bits, producing eight packed digit characters in a single 512-bit register.

If all your integers require eight digits, you are done. But in the general case, putting this to good use requires a bit of effort.

Thankfully, even if you, say, need only six digits, you can do the full 8-digit computation and then use a masked store if you want to store only six digits, ignoring the two leftovers. That is, instruction sets like AVX-512 allow you to write only some of the data to memory, which is quite convenient.

We have two variants. One is branch-heavy and does well on homogeneous data (numbers with similar digit lengths). The other is branch-light and better for mixed workloads. A quick profiling step can pick the right one for your dataset.

Our implementation is consistently 1.4–2× faster than the best competitors and 2–4× faster than std::to_chars across a wide range of inputs. What I find interesting is that even if the std::to_chars implementation is not at all naive, you can do significantly better in many ways. James Anhalt’s approach (jeaiii) is also quite fast on modern hardware.

Further reading
– The paper: doi:10.1002/spe.70079
– The benchmarks are on GitHub (fully reproducible)
– Shortly after our paper came online, Barend Erasmus created a software library implementing our proposed approach. I am not certain that Barend includes both the homogeneous and heterogeneous approaches.

Suppose that x is a variable of an unsigned type. In C/C++, it could be of type size_t for example.

You have an expression like 6 * x and you want to know whether 6 * x overflows. That is, you want to know if 6 * x exceeds the range of values that can be represented by the type. In most cases, a variable of type size_t will be about to represent all values in the range [0, 2^64-1]. Instead of 64, let me use a variable for the number of bits: [0, 2^L-1].

The easiest approach is to compare x with (2^L-1) // 6 where I use the symbol // to denote the integer division (as opposed to /).

But can you do otherwise ?

If the value does not overflow, we know for sure that (6 * x)//6 == x. The interesting question is what happens when it overflows. We can answer this directly for an arbitrary non-zero constant a in the range [1, 2^L-1].

Let k = (a*x)//2^L be the number of times the multiplication wraps around. The effective (wrapped) value computed by the machine is r = a*x - k*2^L, with 0 <= r < 2^L. Overflow happens precisely when k >= 1. We have that k <= a − 1 because x<2^L.

Performing the integer division of r = a*x - k*2^L by a, we get x plus -k*2^L//a. When k is non-zero, this last value (-k*2^L//a) is one of -2^L//a, -2* 2^L//a, …, -(a-1) * 2^L//a.

• When k = 0 (no overflow): r // a = x.
• When k ≥ 1: r // a = x + (negative integer) ≠ x.

Hence we have the following result.

Theorem If x is of an unsigned type and a is a non-zero constant, then a * x overflows if and only if (a * x)//a != x.

In practice, a simple comparison x with (2^L-1) // a  is likely more efficient. Optimizing compilers might be able to convert (a * x)//a != x to a simple comparison. Unfortunately, the Go compiler (for example) cannot.

An open question is whether there is a more mathematically elegant check.

A common pattern in modern software is to map a string key to a small array of floating-point numbers. Word embeddings, feature vectors, lookup tables for physical constants: all variations on the same theme. In Go, the obvious way to write this is a map[string][]float32. But how fast is it, really, and can we do better?

I have been working on constmap, a Go library that builds an immutable map from strings to uint64 values using the binary fuse filter construction. A lookup amounts to one hash, three array reads, and two XORs. There is no comparison, no chaining, no probing. The whole table fits in roughly 9 bytes per key, which often means it fits in cache where a Go map does not.

Go has fast maps, you cannot easily beat them in performance. But if you build a smaller data structure that causes fewer cache misses, you can definitively go faster.

By default, the constmap returns a uint64. But what if your value is an array of eight float32 numbers? You have at least two options:

1. Keep the arrays in a separate slice [][]float32. The constmap returns the index.
2. Store a pointer to the float array directly inside the constmap’s uint64.

The second option requires the unsafe package because we are smuggling a pointer through an integer field. It has some limitations.

• You cannot and should not deserialize the data structure to disk.
• You must make sure that a reference remains to your float array, or else the garbage collector could collect it and you’d be left with a dangling pointer. It is trickier than it sounds because Go can collect your memory if it sees that it is no longer used. And it cannot see through your unsafe calls converting an integer to a pointer value. Thankfully, you can just put all your arrays of floats in an array and call runtime.KeepAlive(mybigarray) at a strategic location: this will prevent Go from collecting mybigarray. The call to runtime.KeepAlive is not free but also quite cheap so you can possibly use a lot of such calls. benchmark

I built three lookups over 100,000 keys, each mapping to an 8-element []float32. We always access the first element of the array, to make sure that the bencmark is a bit fair. We have a large set of random queries (a query is a string).

We compare map[string][]float32, the standard constmap coupled with an array (so that the constmap constains indexes), and the constmap that contains what is effectively a pointer to the location of the []float32.

Run on an Apple M4 Max with Go’s standard benchmark harness:

The constmap with an index is already a twice as fast as the Go map. Replacing the index by a raw pointer shaves another 2 ns by skipping the indirection through the [][]float32 slice header. It is a speedup of about 20% in my case.

The result is interesting on its own: a constmap lookup is fast enough that the next memory load, the slice header read, becomes a measurable fraction of the work.

The benchmark and code are in github.com/lemire/constmap. Run them with:

go test -bench 'FloatArray' -benchtime=1s

No, rising house prices are not the driver of sharp fertility declines. The evidence shows only modest, mixed effects that cannot explain the large drops observed in places like Canada.

What the Research Actually Shows:

A well-known study by Dettling and Kearney (2014) found that rising house prices have opposing effects: they slightly increase fertility among homeowners (via a “home equity” or wealth effect) and slightly decrease it among renters (via a price effect). At average U.S. homeownership rates, the net effect was a small increase in fertility.

This pattern has held up in other countries. For example, Daysal et al. (2021) and related work confirm similar homeowner/renter dynamics in Denmark and elsewhere.

Clark (2012) found that expensive housing markets are associated with a modest delay in age at first birth (roughly 3–4 years after controls), but the overall impact on completed fertility remains limited.

Canadian evidence aligns with this. Clark and Ferrer (2019) analyzed longitudinal data and found that higher lagged house prices were positively associated with the probability of an additional birth among homeowners in some specifications, but the effects were small and did not drive large-scale declines. They noted that falling school-aged children in high-price cities like Vancouver or Toronto are better explained by selective migration of people with preferences for fewer children into urban centers, rather than existing residents having fewer kids due to prices.

These effects are too modest to explain Canada’s fertility rate of about 1.25–1.3 children per woman (a record low, far below the 2.1 replacement level).

Expensive cities do have fewer children on average, but this largely reflects self-sorting: higher-income, higher-educated people (who tend to have fewer children) cluster in costly urban areas. Correlation is not causation.

Fertility is declining across most developed countries, regardless of housing costs:

– Israel maintains a high fertility rate (~2.9 children per woman, highest in the OECD), despite expensive housing; Tel Aviv is pricier than Montreal.
– Japan has long had very low fertility (~1.2) despite more affordable housing in many areas compared to Canada.

As Richard Florida summarized in “Don’t Blame Expensive Housing for Falling Fertility”: intuition suggests high costs deter families, but demographic research points more strongly to higher education and related lifestyle shifts as drivers of lower birth rates. Richer societies and cities tend to have lower fertility.

Fertility choices are heavily shaped early in life. A 16-year-old planning her future is unlikely to base major decisions on distant future house prices in her 20s or 30s. Models ignoring how teenagers view family, career, and life priorities miss the bigger picture.

Housing costs matter at the margins (with offsetting effects by tenure), but they are not the story behind broad fertility collapses. Policies focused on housing affordability are unlikely to reverse fertility trends.

We have no reason whatsoever to believe that a collapse in housing prices in a country like Canada would drive fertility upward. It is motivated reasoning.

High house prices are a terrible thing in my opinion. Investing all your capital in houses is an odd way to get prosperity. But it does not drive our fertility collapse.

References

Clark, Jeremy. 2012. “Do Women Delay Family Formation in Expensive Housing Markets?” Demographic Research 27(1): 1–24. https://doi.org/10.4054/DemRes.2012.27.1

Clark, Jeremy, and Ana Ferrer. 2019. “The Effect of House Prices on Fertility: Evidence from Canada.” Economics: The Open-Access, Open-Assessment E-Journal 13(2019-38): 1–32. https://doi.org/10.5018/economics-ejournal.ja.2019-38

Daysal, N. Meltem, Michael F. Lovenheim, Nikolaj Siersbæk, and David N. Wasser. 2021. “Home Prices, Fertility, and Early-Life Health Outcomes.” Journal of Public Economics 198: 104366. https://doi.org/10.1016/j.jpubeco.2021.104366

Dettling, Lisa J., and Melissa S. Kearney. 2014. “House Prices and Birth Rates: The Impact of the Real Estate Market on the Decision to Have a Baby.” Journal of Public Economics 110: 82–100. https://doi.org/10.1016/j.jpubeco.2013.09.009 (NBER Working Paper 17485, 2011/2014 version)

Florida, Richard. 2018. “Don’t Blame Expensive Housing for Falling Fertility.” CityLab (Bloomberg), June 14, 2018. https://www.bloomberg.com/news/articles/2018-06-14/the-complex-relationship-between-house-prices-and-fertility

We sometimes have to look for a value in a sorted array. The simplest algorithm consists in just going through the values one by one, until we encounter the value, or exhaust the array. We sometimes call this algorithm a linear search. In C++, you can get the desired effect with the std::find function.

For large arrays, you can do better with a binary search. Binary search is a classic algorithm that efficiently locates a target value in a sorted array by repeatedly dividing the search interval in half. Starting with the entire array, it compares the target to the middle element: if the target is smaller, it discards the upper half; if larger, it discards the lower half. This process continues until the target is found or the interval is empty. It is much faster than linear search for large datasets. In C++, this is implemented by the std::binary_search function, which returns a boolean indicating whether the value is present.

The popular Roaring Bitmap format uses arrays of 16-bit integers of size ranging from 1 to 4096. We sometimes have to check whether a value is present. We use a binary search.

I wanted a faster approach. I had two insights.

1. Virtually all processors today have data parallel instructions (sometimes called SIMD) that can check several values at once. Both 64-bit ARM and x64 processors (Intel/AMD) always support comparing eight 16-bit integers with a target value using a single instruction. This suggests that you should not bother going down in the binary search to blocks that are smaller than eight elements. And you may also want to cheaply compare sixteen elements or more.
2. The binary search checks one value at a time. However, recent processors can load and check more than one value at once. They have excellent memory-level parllelism. This suggest that instead of a binary search, we might want to try a quaternary search: instead of splitting arrays in halves, we might split them in quarters. The net result might generate a few more instructions but the number of instructions is likely not the limiting factor.

Thus, I created something I call the SIMD Quad algorithm. It is an efficient search algorithm for sorted arrays of 16-bit unsigned integers, combining a quaternary interpolation search with SIMD (Single Instruction, Multiple Data). The algorithm divides the array into fixed-size blocks of 16 elements (except maybe for the last block) and uses the last element of each block as interpolation keys to quickly narrow down the search to a single block, then employs SIMD instructions to check all 16 elements in that block simultaneously.

The core idea is to perform a hierarchical search: first, use interpolation search on a coarser level (block boundaries) to find the likely block containing the target value, then switch to SIMD for fine-grained parallel checking within the block. This hybrid approach leverages the strengths of both algorithmic optimization (interpolation search reduces comparisons logarithmically) and hardware acceleration (SIMD checks multiple elements at once).

1. Initial Check: If the array has fewer than 16 elements, perform a simple linear search through all elements.
2. Block Division: Divide the array into blocks of 16 consecutive elements. For an array of size cardinality, there are num_blocks = cardinality / 16 full blocks.
3. Quaternary Interpolation Search: Use the last element of each block (at positions 16-1, 32-1, etc.) as keys for interpolation. The search performs a quaternary (base-4) interpolation to find the block where the target pos is likely located. This involves comparing the target against quarter-points of the current search range and adjusting the base accordingly.
4. Block Selection: After narrowing down, select the appropriate block index lo based on the interpolation results.
5. SIMD Check: If a valid block is found, load the 16 elements into SIMD registers (using NEON on ARM or SSE2 on x64) and perform parallel equality comparisons with the target value. If any match is found, return true.
6. Remainder Check: For any elements not in full blocks (remainder), perform a linear search.

How does it do? I wrote a benchmark. The benchmark works as follows. For each array size from 2 to 4096 elements, it generates 100,000 sorted arrays of 16-bit unsigned integers. For each size, it performs 10 million membership queries in “cold” mode (each query searches a different array, simulating cache misses) and 10 million queries in “warm” mode (queries are grouped by array, with each array being searched 100 times consecutively, simulating cache hits). The benchmark measures the average time per query for three algorithms: linear search (std::find), binary search (std::binary_search), and the new SIMD Quad algorithm.

I use two systems. An Apple M4 with Apple LLVM and an Intel Emeral Rapids processor with GCC.

Firstly, let us compare the linear search with the binary search.

Intel/GCC:

Apple/LLVM

The result is clear. The binary search beats the linear search as soon as the arrays get large. That is to be expected.

On a cold cache, the linear search is relatively worse. That is to be expected because it accesses more data, causing more cache faults.

We have established that the binary search is the net winner over the linear search. Let us now compare with the SIMD Quad algorithm.

Intel/GCC:

Apple/LLVM

The results differ markedly between the Intel and Apple platform. On the Intel platform the SIMD Quad is more than twice as fast as the binary search on the warm cache. The benefits are lesser on the cold cache. On the Apple platform, the reverse is true, it is with the cold cache that the SIMD Quad is more than twice as fast, whereas the benefits are more marginal on the warm cache.

But the important point is that, in all instances, SIMD Quad is faster than the binary search.

The SIMD component of the algorithm is rather straightforward: we use specialized instructions that save work. So it is easy to see why it might make things faster. There are few instructions, fewer branches.

But what about the ‘quad’ part. Does it matter? So I tried a binary version of the same algorithm. It has the same SIMD optimization, but I am dropping the quaternary interpolation search and replacing it with a standard binary search.

Intel/GCC:

Apple/LLVM

To put it in simple terms, the quad approach has little effect on the Apple platform, but it is a decent optimization on the Intel platform for large arrays in the cold case. The quaternary search better exploits the memory-level parallelism on my Intel server.

My source code is available.

Conclusion. What my results suggest is that while a textbook binary search is a decent algorithm, you can do better in ways that matter. Standard algorithms were often not designed for computers that have so much parallelism. The SIMD Quad algorithm tries to leverage both the memory-level and data parallelism. Further, I suspect that we can do even better than my algorithm. Let us get creative!

Further reading: Faster intersections between sorted arrays with shotgun

Appendix (source code)

bool simd_quad(const uint16_t *carr, int32_t cardinality, 
            uint16_t pos) {
    constexpr int32_t gap = 16;
    if (cardinality < gap) {
      for (int32_t j = 0; j < cardinality; j++) {
          if (carr[j] == pos) return true;
        }
        return false;
    }
    int32_t num_blocks = cardinality / gap;
    int32_t base = 0;
    int32_t n = num_blocks;
    while (n > 3) {
      int32_t quarter = n >> 2;

      int32_t k1 = carr[(base + quarter + 1) * gap - 1];
      int32_t k2 = carr[(base + 2 * quarter + 1) * gap - 1];
      int32_t k3 = carr[(base + 3 * quarter + 1) * gap - 1];

      int32_t c1 = (k1 < pos);
      int32_t c2 = (k2 < pos);
      int32_t c3 = (k3 < pos);

      base += (c1 + c2 + c3) * quarter;
      n -= 3 * quarter;
    }
    while (n > 1) {
        int32_t half = n >> 1;
        base = (carr[(base + half + 1) * gap - 1] < pos) 
                 ? base + half : base;
        n -= half;
    }
    int32_t lo = (carr[(base + 1) * gap - 1] < pos) 
                ? base + 1 : base;

    if (lo < num_blocks) {
        const uint16_t *blk = carr + lo * gap;
#ifdef __ARM_NEON
        uint16x8_t needle = vdupq_n_u16(pos);
        uint16x8_t v0 = vld1q_u16(blk);
        uint16x8_t v1 = vld1q_u16(blk + 8);
        uint16x8_t hit = vorrq_u16(vceqq_u16(v0, needle), 
                  vceqq_u16(v1, needle));
        return vmaxvq_u16(hit) != 0;
#else
        __m128i needle = _mm_set1_epi16((short)pos);
        __m128i v0 = _mm_loadu_si128((const __m128i *)blk);
        __m128i v1 = _mm_loadu_si128((const __m128i *)(blk + 8));
        __m128i hit = _mm_or_si128(_mm_cmpeq_epi16(v0, needle),
                                   _mm_cmpeq_epi16(v1, needle));
        return _mm_movemask_epi8(hit) != 0;
#endif
    }

    for (int32_t j = num_blocks * gap; j < cardinality; j++) {
        uint16_t v = carr[j];
        if (v >= pos) return (v == pos);
    }
    return false;
}

Consider the following problem. Given a string, you must match all of the ASCII white-space characters (\t, \n, \r, and the space) and some characters important in JSON (:, ,, [, ], {, }). JSON is a text-based data format used for web services. A toy JSON document looks as follows.

{
  "name": "Alice",
  "age": 30,
  "email": "alice@example.com",
  "tags": ["developer", "python", "open-source"],
  "active": true
}

We want to solve this problem using SIMD (single-instruction-multiple-data) instructions. With these instructions, you can compare a block of 16 bytes with another block of 16 bytes in one instruction.

It is a subproblem in the fast simdjson JSON library when we index a JSON document. We call this task vectorized classification. We also use the same technique when parsing DNS records, and so forth. In the actual simdjson library, we must also handle strings and quotes, and it gets more complicated.

I need to define what I mean by ‘matching’ the characters. In my case, it is enough to get, for each block of 64 bytes, two 64-bit masks: one for spaces and one for important characters. To illustrate, let me consider a 16-byte variant:

{"name": "Ali" }
1000000100000001 // important characters
0000000010000010 // spaces

Thus, I want to get back the numbers 0b1000000100000001 and 0b0000000010000010 in binary format (they are 33025 and 130 in decimal).

I refer you to Langdale and Lemire (2019) for how to do it using the conventional SIMD instructions available on ARM processors (NEON). Their key idea is a table-driven, branch-free classifier: for each byte, use SIMD table lookups to map each nibble to a bitmask, and compare to decide whether the byte belongs to a target set (whitespace or structural JSON characters). This avoids doing many separate equality comparisons per character.

There is now a better way on recent ARM processors.

The 128-bit version of NEON was introduced in 2011 with the ARMv8-A architecture (AArch64). Apple played an important role and it was first used by the Apple A7 chip in the iPhone 5S. You can count on all 64-bit ARM processors to support NEON, which is convenient. (There are 32-bit ARM processors but they are mostly used for embedded systems, not mainstream computing.)

ARM NEON is good but getting old. It is no match for the AVX-512 instruction set available on x64 (AMD and Intel) processors. Not only do the AVX-512 instructions support wider registers (64 bytes as opposed to ARM NEON’s 16 bytes), but they also have more powerful instructions.

But ARM has something else to offer: Scalable Vector Extension (SVE) and its successor, SVE2. Though SVE was first introduced in 2016, it took until 2022 before we had actual access. The Neoverse V1 architecture used by the Amazon Graviton 3 is the first one I had access to. Soon after, we got SVE2 with the Neoverse V2 and N2 architectures. Today it is readily available: the Graviton4 on AWS, the Microsoft Cobalt 100 on Azure, the Google Axion on Google Cloud (and newer Google Cloud ARM CPUs), the NVIDIA Grace CPU, as well as several chips from Qualcomm, MediaTek, and Samsung. Notice who I am not including? Apple. For unclear reasons, Apple has not yet adopted SVE2.

I have mixed feelings about SVE/SVE2. Like RISC-V, it breaks with the approach from ARM NEON and x64 SIMD that uses fixed-length register sizes (16 bytes, 32 bytes, 64 bytes). This means that you are expected to code without knowing how wide the registers are.

This is convenient for chip makers because it gives them the option of adjusting the register size to better suit their market. Yet it seems to have failed. While the Graviton 3 processor from Amazon had 256-bit registers… all commodity chips have had 128-bit registers after that.

On the plus side, SVE/SVE2 has masks a bit like AVX-512, so you can load and process data only in a subset of the registers. It solves a long-standing problem with earlier SIMD instruction sets where the input is not a multiple of the register size. Both SVE/SVE2 and AVX-512 might make tail handling nicer. Being able to operate on only part of the register allows clever optimizations. Sadly, SVE/SVE2 does not allow you to move masks to and from a general-purpose register efficiently, unlike AVX-512. And that’s a direct consequence of their design with variable-length registers. Thus, even though your registers might always be 128-bit and contain 16 bytes, the instruction set is not allowed to assume that a mask fits in a 16-bit word.

I was pessimistic regarding SVE/SVE2 until I learned that it is designed to be interoperable with ARM NEON. Thus you can use the SVE/SVE2 instructions with your ARM NEON code. This works especially well if you know that the SVE/SVE2 registers match the ARM NEON registers (16 bytes).

For the work I do, there are two SVE2 instructions that are important: match and nmatch. In their 8-bit versions, what they do is the following: given two vectors a and b, each containing up to 16 bytes, match sets a predicate bit to true for each position i where a[i] equals any of the bytes in b. In other words, b acts as a small lookup set, and match tests set membership for every byte of a simultaneously. The nmatch instruction is the logical complement: it sets a predicate bit to true wherever a[i] does not match any byte in b. A single instruction thus replaces a series of equality comparisons and OR-reductions that would otherwise be needed. In the code below, op_chars holds the 8 structural JSON characters and ws_chars holds the 4 whitespace characters; calling svmatch_u8 once on a 16-byte chunk d0 produces a predicate that has a true bit exactly where that input byte is a structural character. The code uses SVE2 intrinsics: compiler-provided C/C++ functions that map almost one-to-one to CPU SIMD instructions, so you get near-assembly control without writing assembly.

// : , [ ] { }
uint8_t op_chars_data[16] = {
    0x3a, 0x2c, 0x5b, 0x5d, 0x7b, 0x7d, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0
};
// \t \n \r ' '
uint8_t ws_chars_data[16] = {
    0x09, 0x0a, 0x0d, 0x20, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0
};

// load the characters in SIMD registers
svuint8_t op_chars = svld1_u8(svptrue_b8(), op_chars_data);
svuint8_t ws_chars = svld1_u8(svptrue_b8(), ws_chars_data);

// load data
// const char * input = ...
svbool_t pg = svptrue_pat_b8(SV_VL16);
svuint8_t d = svld1_u8(pg, input);

// matching
svbool_t op = svmatch_u8(pg, d, op_chars);
svbool_t ws = svmatch_u8(pg, d, ws_chars);

In this code snippet, svuint8_t is an SVE vector type containing unsigned 8-bit lanes (bytes). svbool_t is an SVE predicate (mask) type. svptrue_b8() builds a predicate where all 8-bit lanes are active, and svld1_u8(pg, ptr) loads bytes from memory into an SVE vector, using predicate pg to decide which lanes are actually read.

If you paid attention thus far, you might have noticed that my code is slightly wrong since I am including 0 in the character sets. But it is fine as long as I assume that the zero byte is not present in the input. In practice, I could just repeat one of the characters, or use a bogus character that I do not expect to see in my inputs (such as the byte value 0xFF, which cannot appear in a valid UTF-8 string).

In standard SVE/SVE2, op and ws are predicates, not integer masks. A practical trick is to materialize each predicate as bytes (0xFF for true, 0x00 for false), for example with svdup_n_u8_z.

svuint8_t opm = svdup_n_u8_z(op, 0xFF);
svuint8_t wsm = svdup_n_u8_z(ws, 0xFF);

When SVE vectors are 128 bits, this byte vector maps naturally to a NEON uint8x16_t via svget_neonq_u8, and from there we can build scalar bitmasks efficiently with NEON operations (masking plus pairwise additions). Repeating this over four 16-byte chunks gives the two 64-bit masks needed for a 64-byte block.

I wanted to quickly run my benchmarks on an AWS Graviton 4. I used LLVM clang 20 which was readily available in the images that AWS makes available (I picked RedHat 10).

The AWS Graviton 4 processor is a Neoverse V2 processor. Google has its own Neoverse V2 processors in its cloud. In my tests, it ran at 2.8 GHz.

My benchmark generates a random string of 1 MiB and computes the bitmaps indicating the positions of the characters. It is available on GitHub. My results are as follow.

So the SVE/SVE2 approach is faster than the NEON equivalent and uses 25% fewer instructions, and that’s without any kind of fancy optimization. Importantly, the code is relatively simple thanks to the match instruction.

It might be that the SVE2 function match is the fastest way to match characters on ARM processors.

Credit: This post was motivated by a sketch by user liuyang-664 on GitHub.

References

Langdale, G., & Lemire, D. (2019). Parsing gigabytes of JSON per second. The VLDB Journal, 28(6), 941-960.

Koekkoek, J., & Lemire, D. (2025). Parsing millions of DNS records per second. Software: Practice and Experience, 55(4), 778-788.

Lemire, D. (2025). Scanning HTML at Tens of Gigabytes Per Second on ARM Processors. Software: Practice and Experience, 55(7), 1256-1265.

Suppose you have several strings and you want to count the number of instances of the character ! in your strings. In C++, you might solve the problem as follows if you are an old-school programmer.

size_t c = 0;
for (const auto &str : strings) {
    c += std::count(str.begin(), str.end(), '!');
}

You can also get fancier with ranges.

for (const auto &str : strings) {
    c += std::ranges::count(str, '!');
}

And so forth.

But what if you want to go faster? Maybe you’d want to rewrite this function in assembly. I decided to do so, and to have fun using both Grok and Claude as my AIs, setting up a friendly competition.

I started with my function and then I asked AIs to optimize it in assembly. Importantly, they knew which machine I was on, so they started to write ARM assembly.

By repeated prompting, I got the following functions.

• count_classic: Uses C++ standard library std::count for reference.
• count_assembly: A basic ARM64 assembly loop (byte-by-byte comparison). Written by Grok.
• count_assembly_claude: Claude’s SIMD-optimized version using NEON instructions (16-byte chunks).
• count_assembly_grok: Grok’s optimized version (32-byte chunks).
• count_assembly_claude_2: Claude’s further optimized version (64-byte chunks with multiple accumulators).
• count_assembly_grok_2: Grok’s latest version (64-byte chunks with improved accumulator handling).
• count_assembly_claude_3: Claude’s most advanced version with additional optimizations.

You get the idea.

So, how is the performance? I use random strings of up to 1 kilobyte. In all cases, I test that the functions provide the correct count. I did not closely examine the code, so it is possible that mistakes could be hiding in the code.

I record the average number of instructions per string.

By repeated optimization, I reduced the number of instructions by a factor of eight. The running time decreases similarly.

Can we get the AIs to rewrite the best option in C? Yes, although you need SIMD intrinsics. So there is no benefit to leaving the code in assembly in this instance.

An open question is whether the AIs could find optimizations that are not possible if we use a higher-level language like C or C++. It is an intriguing question that I will seek to answer later. For the time being, the AIs can beat my C++ compiler!

My source code is available.

Consider the following problem. You have a large set of strings, maybe millions. You need to map these strings to 8-byte integers (uint64). These integers are given to you.

If you are working in Go, the standard solution is to create a map. The construction is trivial, something like the following loop.

m := make(map[string]uint64, N)
for i, k := range keys {
    m[k] = values[i]
}

One downside is that the map may use over 50 bytes per entry.

In important scenarios, we might have the following conditions. The map is large (a million of entries or more), you do not need to modify it dynamically (it is immutable), and all queried keys are in the set. In such conditions, you can reduce the memory usage down to almost the size of the keys, so about 8 bytes per entry. One fast technique is the binary fuse filters.

I implemented it as a Go library called constmap that provides an immutable map from strings to uint64 values using binary fuse filters. This data structure is ideal when you have a fixed set of keys at construction time and need fast, memory-efficient lookups afterward. You can even construct the map once, save it to disk so you do not pay the cost of constructing the map each time you need it.

The usage is just as simple.

package main

import (
    "fmt"
    "log"

    "github.com/lemire/constmap"
)

func main() {
    keys := []string{"apple", "banana", "cherry"}
    values := []uint64{100, 200, 300}

    cm, err := constmap.New(keys, values)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(cm.Map("banana")) // 200
}

The construction time is higher (as expected for any compact data structure), but lookups are optimized for speed. I ran benchmarks on my Apple M4 Max processor to compare constmap lookups against Go’s built-in map[string]uint64. The test uses 1 million keys.

ConstMap is nearly 3 times faster than Go’s standard map for lookups! And we reduced the memory usage by a factor of 6.

The ConstMap may not always be faster, but it should always use significantly less memory. If it can reside in CPU cache while the map cannot, then it will be significantly faster.

Source Code The implementation is available on GitHub: github.com/lemire/constmap.

The next C++ standard (C++26) is getting exciting new features. One of these features is compile-time reflection. It is ideally suited to serialize and deserialize data at high speed. To test it out, we extended our fast JSON library (simdjson) and we gave a talk at CppCon 2025. The video is out on YouTube.

Our slides are also available.

Modern processors have the ability to execute many instructions per cycle, on a single core. To be able to execute many instructions per cycle in practice, processors predict branches. I have made the point over the years that modern CPUs have an incredible ability to predict branches.

It makes benchmarking difficult because if you test on small datasets, you can get surprising results that might not work on real data.

My go-to benchmark is a function like so:

while (howmany != 0) {
    val = generate_random_value()
    if(val is odd) write to buffer
    decrement howmany
}

The processor tries to predict the branch (if clause). Because we use random values, the processor should mispredict one time out of two.

However, if we repeat multiple times the benchmark, always using the same random values, the processor learns the branches. How many can processors learn? I test using three recent processors.

• The AMD Zen 5 processor can predict perfectly 30,000 branches.
• The Apple M4 processor can predict perfectly 10,000 branches.
• Intel Emerald Rapids can predict perfectly 5,000 branches.

Once more I am disappointed by Intel. AMD is doing wonderfully well on this benchmark.

My source code is available.

Suppose that you have a record of your sales per day. You might want to get a running record where, for each day, you are told how many sales you have made since the start of the year.

Such an operation is called a prefix sum or a scan.

Implementing it in C is not difficult. It is a simple loop.

  for (size_t i = 1; i < length; i++) {
    data[i] += data[i - 1];
  }

How fast can this function be? We can derive a speed limit rather simply: to compute the current value, you must have computed the previous one, and so forth.

data[0] -> data[1] -> data[2] -> ...

At best, you require one CPU cycle per entry in your table. Thus, on a 4 GHz processor, you might process 4 billion integer values per second. It is an upper bound but you might be able to reach close to it in practice on many modern systems. Of course, there are other instructions involved such as loads, stores and branching, but our processors can execute many instructions per cycle and they can predict branches effectively. So you should be able to process billions of integers per second on most processors today.

Not bad! But can we do better?

We can use SIMD instructions. SIMD instructions are special instructions that process several values at once. All 64-bit ARM processors support NEON instructions. NEON instructions can process four integers at once, if they are packed in one SIMD register.

But how do you do the prefix sum on a 4-value register? You can do it with two shifts and two additions. In theory, it scales as log(N) where N is the number elements in a vector register.

input   = [A B   C     D]
shift1  = [0 A   B     C]
sum1    = [A A+B B+C   C+D]
shift2  = [0 0   A     B+A]
result  = [A A+B A+B+C A+B+C+D]

You can then extract the last value (A+B+C+D) and broadcast it to all positions so that you can add it to the next value.

Is this faster than the scalar approach? We have 4 instructions in sequence, plus at least one instruction if you want to use the total sum in the next block of four values.

Thus the SIMD approach might be worse. It is disappointing.

A solution might be the scale up over many more integer values.

Consider ARM NEON which has interleaved load and store instructions. If you can load 16 values at once, and get all of the first values together, all of the second values together, and so forth.

original data : ABCD EFGH IJKL MNOP
loaded data   : AEIM BFJN CGKO DHLP

Then I can do a prefix sum over the four blocks in parallel. It takes three instructions. At the end of the three instructions, we have one register which contains the local sums:

A+B+C+D E+F+G+H I+J+K+L M+N+O+P

And then we can apply our prefix sum recipe on this register (4 instructions). You might end up with something like 8 sequential instructions per block of 16 values.

It is theoretically twice as fast as the scalar approach.

In C with instrinsics, you might code it as follows.

void neon_prefixsum_fast(uint32_t *data, size_t length) {
  uint32x4_t zero = {0, 0, 0, 0};
  uint32x4_t prev = {0, 0, 0, 0};

  for (size_t i = 0; i < length / 16; i++) {
    uint32x4x4_t vals = vld4q_u32(data + 16 * i);

    // Prefix sum inside each transposed ("vertical") lane
    vals.val[1] = vaddq_u32(vals.val[1], vals.val[0]);
    vals.val[2] = vaddq_u32(vals.val[2], vals.val[1]);
    vals.val[3] = vaddq_u32(vals.val[3], vals.val[2]);

    // Now vals.val[3] contains the four local prefix sums:
    //   vals.val[3] = [s0=A+B+C+D, s1=E+F+G+H, 
    //                  s2=I+J+K+L, s3=M+N+O+P]

    // Compute prefix sum across the four local sums 
    uint32x4_t off = vextq_u32(zero, vals.val[3], 3);
    uint32x4_t ps = vaddq_u32(vals.val[3], off);       
    off = vextq_u32(zero, ps, 2);                      
    ps = vaddq_u32(ps, off);

    // Now ps contains cumulative sums across the four groups
    // Add the incoming carry from the previous 16-element block
    ps = vaddq_u32(ps, prev);

    // Prepare carry for next block: broadcast the last lane of ps
    prev = vdupq_laneq_u32(ps, 3);

    // The add vector to apply to the original lanes is the 
    // prefix up to previous group
    uint32x4_t add = vextq_u32(prev, ps, 3);  

    // Apply carry/offset to each of the four transposed lanes
    vals.val[0] = vaddq_u32(vals.val[0], add);
    vals.val[1] = vaddq_u32(vals.val[1], add);
    vals.val[2] = vaddq_u32(vals.val[2], add);
    vals.val[3] = vaddq_u32(vals.val[3], add);

    // Store back the four lanes (interleaved)
    vst4q_u32(data + 16 * i, vals);
  }

  scalar_prefixsum_leftover(data, length, 16);
}

Let us try it out on an Apple M4 processor (4.5 GHz).

So the SIMD approach is about 2.3 times faster than the scalar approach. Not bad.

My source code is available on GitHub.

Appendix. Instrinsics

The Internet relies on text formats. Thus, we spend a lot of time producing and consuming data encoded in text.

Your web pages are HTML. The code running in them is JavaScript, sent as text (JavaScript source), not as already-parsed code. Your emails, including their attachments, are sent as text (your binary files are sent as text).

It does not stop there. The Python code that runs your server is stored as text. It queries data by sending text queries. It often gets back the answer as text that must then be decoded.

JSON is the universal data interchange format online today. We share maps as JSON (GeoJSON).

Not everything is text, of course. There is no common video or image format that is shared as text. Transmissions over the Internet are routinely compressed to binary formats. There are popular binary formats that compete with JSON.
But why is text dominant?

It is not because, back in the 1970s, programmers did not know about binary formats.

In fact, we did not start with text formats. Initially, we worked with raw binary data. Those of us old enough will remember programming in assembly using raw byte values.

Why text won?

1.Text is efficient.

In the XML era, when everything had to be XML, there were countless proposals for binary formats. People were sometimes surprised to find that the binary approach was not much faster in practice. Remember that many text formats date back to an era when computers were much slower. Had text been a performance bottleneck, it would not have spread. Of course, there are cases where text makes things slower. You then have a choice: optimize your code further or transition to another format. Often, both are viable.

It is easy to make wrong assumptions about binary formats, such as that you can consume them without any parsing or validation. If you pick up data from the Internet, you must assume that it could have been sent by an adversary or someone who does not follow your conventions.

2.Text is easy to work with.

If you receive text from a remote source, you can often transform it, index it, search it, quote it, version it… with little effort and without in-depth knowledge of the format. Text is often self-documenting.

In an open world, when you will never speak with the person producing the data, text often makes everything easier and smoother.

If there is an issue to report and the data is in text, you can usually copy-paste the relevant section into a message. Things are much harder with a binary format.

We locate web content using special addresses called URLs. We are all familiar with addresses like https://google.com. Sometimes, URLs can get long and they can become difficult to read. Thus, we might be tempted to format them
like so in HTML using newline and tab characters, like so:

<a href="https://lemire.me/blog/2026/02/21/
        how-fast-do-browsers-correct-utf-16-strings/">my blog post</a>

It will work.

Let us refer to the WHATWG URL specification that browsers follow. It makes two statements in sequence.

1. If input contains any ASCII tab or newline, invalid-URL-unit validation error.
2. Remove all ASCII tab or newline from input.

Notice how it reports an error if there is a tab or newline character, but continues anyway? The specification says that A validation error does not mean that the parser terminates and it encourages systems to report errors somewhere. Effectively, the error is ignored although it might be logged. Thus our HTML is fine in practice.

The following is also fine:

<a href="https://go
ogle.c
om" class="button">Visit Google</a>

You can also use tabs. But you cannot arbitrarily insert any other whitespace.

Yet there are cases when you can use any ASCII whitespace character: data URLs. Data URLs (also called data URIs) embed small files—like images, text, or other content—directly inside a URL string, instead of linking to an external resource. Data URLs are a special kind of URL and they follow different rules.

A typical data URL might look like data:image/png;base64,iVBORw0KGgoAAAANSUhEUg... where the string iVBORw0KGgoAAAANSUhEUg... is the binary data of the image that has been encoded with base64. Base64 is a text format that can represent any binary content: we use 64 ASCII characters so that each character encodes 6 bits. Your binary email attachments are base64 encoded.

On the web, when decoding a base64 string, you ignore all ASCII whitespaces (including the space character itself). Thus you can embed a PNG image in HTML as follows.

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA
                                 QAAAAECAIAAAAmkwkpAAAAEUl
                                 EQVR4nGP8z4AATEhsPBwAM9EB
                                 BzDn4UwAAAAASUVORK5CYII=" />

This HTML code is valid and will insert a tiny image in your page.

But there is more. A data URL can also be used to insert an SVG image. SVG (Scalable Vector Graphics) is an XML-based vector image format that describes 2D graphics using mathematical paths, shapes, and text instead of pixels.
The following should draw a very simple sunset:

<img src='data:image/svg+xml,
<svg width="200" height="200" 
     xmlns="http://www.w3.org/2000/svg">
  <rect width="100%" height="100%" fill="blue" /> 
  <!-- the sky -->
  <circle cx="100" cy="110" r="50" fill="yellow" />  
  <!-- the sun -->
  <rect x="0" y="120" width="200" height="80" fill="brown" />  
  <!-- the ground -->
</svg>' />

Observe how I was able to format the SVG code so that it is readable.

Further reading: Nizipli, Y., & Lemire, D. (2024). Parsing millions of URLs per second. Software: Practice and Experience, 54(5), 744-758.

JavaScript represents strings using Unicode, like most programming languages today. Each character in a JavaScript string is stored using one or two 16-bit words. The following JavaScript code might surprise some programmers because a single character becomes two 16-bit words.

> t="🧰"
'🧰'
> t.length
2
> t[0]
'\ud83e'
> t[1]
'\uddf0'

The convention is that \uddf0 is the 16-bit value 0xDDF0 also written U+DDF0.

The UTF-16 standard is relatively simple. There are three types of values. high surrogates (the range U+D800 to U+DBFF), low surrogates (U+DC00 to U+DFFF), and all other code units (U+0000–U+D7FF together with U+E000–U+FFFF). A high surrogate must always be followed by a low surrogate, and a low surrogate must always be preceded by a high surrogate.

What happens if you break the rules and have a high surrogate followed by a high surrogate? Then you have an invalid string. We can correct the strings by patching them: we replace the bad values by the replacement character (\ufffd). The replacement character sometimes appears as a question mark.

To correct a broken string in JavaScript, you can call the toWellFormed method.

> t = '\uddf0\uddf0'
'\uddf0\uddf0'
> t.toWellFormed()
'��'

How fast is it?

I wrote a small benchmark that you can test online to measure its speed. I use broken strings of various sizes up to a few kilobytes. I run the benchmarks on my Apple M4 processor using different browsers.

Quite a range of performance! The speed of other chromium-based browsers (Brave and Edge) is much the same as Chrome.

I also tested with JavaScript runtimes.

Usually Bun is faster than Node, but in this instance, Node is twice as far as Bun.

Thus, we can correct strings in JavaScript at over ten gigabytes per second if you use Chromium-based browsers.