I write a lot about data parallelism. About how our processors have special instructions that can do several operations at once. They are often critical to get the best performance out of a CPU.

Compilers are sometimes able to take advantage of these instructions. But it is hard to tell when it will work.

For Java programmers, you can write directly for data parallelism since JDK 16. With enough skill, you might multiply the performance of a few key functions.

The Vector API is in the incubator module jdk.incubator.vector. It still not officially supported, but you can enable it by passing --add-modules=jdk.incubator.vector. I expect that it will soon become a mainstream feature in Java.

Roman Snytsar wrote a book about it: Mastering SIMD with Java Vector API, and the subtitle is Unlocking Single-Core Performance Through SIMD Optimization. I had the honor of being the technical reviewer. I learned a few things while reading the book and I enjoyed it.

Who has time for books, especially technical books?

What a book offers is time to reflect. Reading technical books today is probably just as relevant as it ever was. The author takes you on a story… and gets you to think. 

Snytsar’s book works from  problems like sum an array, compute a mean and a standard deviation, remove duplicates, merge two sorted arrays, and so forth. A lot of them are similar to the problems you will encounter as a programmer.  He starts from the ordinary solution and rebuilds it with the Java Vector API. He then runs benchmarks. He looks at the assembly code.

The book covers performance issues such as unrolling, asymmetric loops, structural hazards, dependencies. Snytsar shows us that, in some instances, data parallelism can fail to bear fruits. The negative lessons are just as important as the positive ones.

The Java Vector API is a thick layer of abstractions, but Snytsar shows that some tricks work better on some hardware than others.  I liked the chapter on removing duplicates, which is built on compress. It is one instance where the specific hardware matters. And you are unlikely to just find out about it yourself.

It is a book worth buying if you are a Java programmer with a focus on performance. 

The book: Roman Snytsar, Mastering SIMD with Java Vector API, Apress, 2026.

When you build a program, the compiler turns each source file into an object file. Then a linker stitches all the object files and libraries into one executable.

On Linux, the default linker is usually GNU ld (also called bfd). The GNU binutils also ship gold, an alternative linker that was designed to be faster. Gold was built a Google and first made available in 2008.

The mold linker is a newer linker written by Rui Ueyama, it was first released in 2021. It is meant as a drop-in replacement for GNU ld, and its main selling point is speed. The latest release is version 3.

How much faster is it on a large project? I built Node.js from its main branch on an Intel Xeon Gold 6548N server (Emerald Rapids, two sockets, 64 cores and 128 threads) running Linux with GCC 14.3. The final node executable weighs about 160 MB.

I captured the command that links the node executable and ran it six times with each linker. I report the median.

With multithreading, mold is about 24 times faster than GNU ld and about 14 times faster than gold.

Even if I restrict mold to a single thread, it links Node.js in half a second, five times faster than GNU ld. With only 8 threads, you get nearly all of the benefit.

Using mold is easy. You can pass -fuse-ld=mold to GCC or clang. Or you can wrap your whole build:

mold -run make -j128

The -run option intercepts every call to the default linker and redirects it to mold. You do not need to change the build scripts.

Of course, two seconds saved on a link does not matter much if you build Node.js once. But a developer who edits a file and rebuilds dozens of times a day pays the link cost each time. With mold, the link step becomes nearly free.

My scripts and raw results are available.

Much of my research is on arXiv. I was one of the early adopters. It is simply a repository of research papers with PDF and metadata.

It makes it convenient to find research without any paywall.

Physicists started it in 1991. Math and computer science followed gradually.

It relies on volunteers to moderate it, because we don’t want garbage or spam.

On October 1, they capped the submissions to two per person per month.

Why?

arXiv got over 40,000 submissions in September.

Two years ago, it was about 20,000.

Doubling every two years. It is unsustainable for human beings.

In related news, Google stopped taking new bug reports in its open-source bounty program. They could not cope with the influx.

The writing is on the wall.

Google’s first orbital data center is in orbit (October 1st). They say it is about the size of a fridge.

It has one kilowatt of solar power. It is about the power of a household.

It was launched on a Falcon 9 rocket (SpaceX).

It should run for about a year. Google plans to launch two more orbital data centres next year. The plan is to use laser communications.

For people complaining that the latency is going to be a problem with orbital data centres…

An orbital data centre can be effectively in your line of sight and it can be relatively close (much closer than the diameter of the USA).

Further if you have a cluster of space data centres, they can remain close and in a line of sight.

So you are not going to run your video games or do high frequency trading from space, but for compute, latency is unlikely to be the bottleneck.

The great features are space (no need to wait five years for a permit) and power (no need to negotiate with a local government for electricity). Solar power in space is great because it runs 24h a day.

A common way to store JSON data is to write one document per line. We call it NDJSON or JSON Lines. Log files, database exports and machine-learning datasets often come in this format. The files can be large, so we may compress them.

{"id":1,"active":true,"user":{"name":"user_1","tags":["guest"]},"score":-3862,"note":"..."}
{"id":2,"active":false,"user":{"name":"user_2","tags":["staff","admin"]},"score":8123,"note":"..."}

How fast can you read such a file? I wrote a small demo with simdjson. My test file has 5 million records: 812 MB of NDJSON. For each record, I read a few fields. I count the active records, and I sum the scores of the active records whose user has the admin tag.

You could decompress the whole file, but it might be better to process the compressed file. So you decompress a chunk, you parse the complete lines in that chunk, and you move the incomplete last line to the front of the buffer. In NDJSON, a newline can never appear inside a document: a newline in a string must be escaped as n. So you can always cut the buffer right after its last newline character. The simdjson library has a function for many documents in one buffer (iterate_many):

simdjson::ondemand::parser parser;
while (true) {
  // fill the buffer with decompressed bytes...
  size_t cut = eof ? len : last_newline(buf, len) + 1;
  simdjson::ondemand::document_stream stream;
  parser.iterate_many(buf, cut, cut).get(stream);
  for (auto doc : stream) {
    accumulate(doc.value_unsafe(), result);
  }
  if (eof) { break; }
  std::memmove(buf, buf + cut, len - cut);
  len -= cut;
}

I ran my benchmarks on an Intel Xeon Gold 6548N server (Emerald Rapids) with two sockets, 64 cores and 128 threads. I use GCC 14.

A gzip file is one long compressed stream. The decompressor needs the previous 32 KiB of output to decode what comes next. So you must decompress the file from the start, with one thread. I can still parse in other threads: one thread decompresses chunks and puts them in a queue, and the other threads parse them. It doubles the speed to 2.5 GB/s.

Other formats do better. A zstd or an lz4 file can be made of many independent frames, one after the other. It is still a regular .zst or .lz4 file: the usual command-line tools decompress it as usual.

My program writes a new frame every 256 KiB of JSON, always after a newline. Each frame stores its decompressed size and a checksum. To find where a frame ends, you only need to read a few block headers. You do not need to decompress anything. So the threads can take frames one by one, and each thread decompresses and parses its own frames.

With one thread, zstd and lz4 are no faster than gzip: about 1.1 GB/s. With 64 threads, I get 40 GB/s with zstd and 34 GB/s with lz4. That is 16 times faster than the best I can do with gzip.

The files are not much larger. Small frames compress a bit worse, since each frame starts from scratch, but the zstd file is only 6% larger than the gzip file.

My data is synthetic. The records are very repetitive, which makes decompression fast. Your data might decompress more slowly.

If you control how your JSON files are written, consider using zstd with many frames. You get files about as small as with gzip, and you can read them many times faster with multiple threads.

Source code: https://github.com/simdjson/simdjson_compressed_demo. The benchmark numbers and the plotting script are in my blog repository.

Our software represents strings using the UTF-16 or the UTF-8 formats. Most text on the web is UTF-8, but Java, C# or JavaScript represents the strings as UTF-16 to the programmer.

Sometimes we need to transcode (convert) strings. Your browser probably uses the simdutf library for validating or transcoding. It is part of the widely used V8 JavaScript engine.

Up until a few days ago, the simdutf library missed one key feature: if the UTF-8 input is invalid, it did not know how to transcode it to UTF-16. The objective is to replace ill-formed UTF-8 sequences with the replacement character U+FFFD. It is the character that shows up as a weird question mark sometimes. You may have seen it in a broken web site.

The new function convert_utf8_to_utf16_with_replacement always succeeds. It must be used in conjunction with the function utf16_length_from_utf8_with_replacement, which scans and validates the input, recording the offsets of errors if there are some. The usage is as follows.

const char source[] = {'c', 'a', 'f', '\xff'};
size_t length = 4;
simdutf::utf8_to_utf16_result res =
    simdutf::utf16_length_from_utf8_with_replacement(source, length);
std::unique_ptr<char16_t[]> utf16{new char16_t[res.count]};
size_t written = simdutf::convert_utf8_to_utf16_with_replacement(
    source, length, utf16.get(), res);

Because we have a list of errors before even beginning the transcoding, the bytes between the recorded offsets are valid UTF-8, so we can transcode faster. Each recorded error becomes one U+FFFD.

I timed a scalar decoder against the library on Japanese Wikipedia. The file is 531 KB. I repeated it so the timed buffer is 1.1 MB. I use one core of a Xeon Gold 6548N. GCC 14.3.1 at -O3, best of eight runs.

With no errors, simdutf reaches 7.0 GB/s and the scalar loop 1.7 GB/s.

On valid input, the new approach may even be marginally faster than the previous method, because our initial validation pass allows us to transcode faster afterward.

Compiler: GCC 14.3.1, -O3, one core (taskset -c 2), simdutf icelake kernel. The functions are on the branch utf8-to-utf16-with-replacement.

Credit: The most non-trivial part of this routine was coded by Benjamin Bucher over the summer.

The simdjson library is a C++ library to parse and generate JSON. It is used in Node.js, ClickHouse, Meta Velox, StarRocks, Apache Doris, the Ladybird browser and many other systems. We released version 4.0 a year ago, in September 2025. Its headline feature was C++26 static reflection: you could turn a C++ structure into JSON, and back, without writing any glue code.

Today we are releasing version 5.0.

1. Static reflection is no longer guarded and is an officially supported feature. When your compiler has reflection enabled (e.g., g++ -std=c++26 -freflection with GCC 16), simdjson detects it by itself and the reflection-based functions become available.

2. When deserializing a C++ structure through reflection, simdjson now uses key selectors (see below) by default: it reads the object in a single pass, whatever the order of the keys. You can return to the previous approach (one lookup per member) with -DSIMDJSON_DISABLE_KEY_SELECTOR_REFLECTION=1.

3. A positive integer in [2^64, 10^20) is now reported as a big integer (BIGINT_NUMBER), like other overflowing integers, instead of a malformed number. When big integers are parsed as strings, a token such as 123456789123456789123x is now rejected.

One major change is the key selectors. A common task is to extract a few fields from a JSON object. In simdjson 5.0 (C++20 or better), you can name the keys at compile time and visit the object once:

using namespace simdjson;
auto json = R"({ "name": "Daniel", "age": 42, "city": "Montreal" })"_padded;
ondemand::parser parser;
auto doc = parser.iterate(json);
std::string_view name, city;
uint64_t age = 0;
auto result = doc.get_object().for_each<"name", "city", "age">(name, city, age);
// name == "Daniel", city == "Montreal", age == 42

The keys can appear in any order. At compile time, simdjson builds a perfect hash function for your set of keys. At run time, recognizing a key takes a hash computed from a couple of bytes and one comparison. You can also pass one callback per key instead of variables. The iteration stops as soon as all keys have been found.

We added annotations for the data structures for automated (C++26) serialization and deserialization: rename, rename_all, alias, skip, default_value, flatten, deny_unknown_fields, transparent, and so forth.

struct [[= simdjson::rename_all<simdjson::case_style::camel_case>]] User {
  std::string first_name;
  int64_t user_id;
  [[= simdjson::rename<"KEY">]] int api_key;
};
// {"firstName":"Ann","userId":7,"KEY":8}

We often get many JSON documents in one file or one network message. simdjson has long supported streams of documents separated by white space (NDJSON). In 5.0 we added:

• RFC 7464 JSON text sequences (each document preceded by the record separator character) and comma-separated documents;
• stream_format::newline_delimited: you promise that each document sits on its own line. When you only read part of a document, simdjson jumps to the next line instead of walking over the rest of the document.
• simdjson::slice_at, which cuts a stream into blocks at document boundaries so you can parse the blocks on as many threads as you like. The built-in threaded mode uses at most two threads.

We also fixed several bugs in document_stream, found in an audit by Francisco Geiman Thiesen.

There are many other smaller features.

• NaN and infinity: JSON does not allow NaN or Infinity, but many systems produce them anyway. If you define SIMDJSON_ENABLE_NAN_INF, simdjson parses them, and serializes them.
• Narrow types: get_uint8(), get_int8(), get_uint16(), get_int16() check the range for you. With C++23, get_float32() and get_float64() return std::float32_t and std::float64_t. The binary32 value is rounded once, directly from the decimal string, not through a double.
• The DOM API can parse a buffer that has no padding (parser.parse_unpadded(...)). It is slower than the regular function, but it never reads past the end of your buffer and it does not copy your data.
• With C++17, simdjson::padded_input adds padding only when it is needed: when your string ends near a page boundary.
• C++20 ranges: you can pipe On-Demand arrays and objects into std::views::transform and other adaptors.
• DOM arrays support reverse iteration (rbegin(), rend()) with no allocation.
• On-Demand objects offer get_current_position() and revert_position(): if you miss an optional field, you can go back to where you were instead of rescanning the whole object.
• char8_t (u8) variants of the string accessors in C++20.
• Better pretty printing with the FracturedJson style, including tables.
• Memory-mapped files under Windows.
• We support the memory-safe compiler Fil-C.

The simdjson 5.0 release improved performance compared to simdjson 4.0 in some key cases. Let me review some of them.

I built both versions with GCC 16.1 (-O3, CMake Release) and ran them on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core.

Let us start with DOM parsing of our standard files (GB/s):

Files full of numbers (canada, marine_ik, mesh, numbers) are 8% to 25% faster. And a file full of escaped Unicode characters (twitterescaped) is almost twice as fast: among other changes, we now decode consecutive uXXXX sequences without going back to the string scanner between them.

We also serialize faster. Printing floating-point numbers used to be a bottleneck. We replaced the ancient Grisu2 by Dragonbox, and removed calls to memcpy and memmove from the hot path.

The simdjson library is a community project. Since version 4.6, contributions came from fior512, 吴杨帆, Alecto Irene Perez, Francisco Geiman Thiesen, Max Bachmann, MoonFlowww, Advit Arora, Jaël Champagne Gareau, Taimoor Kiani, Vasily Pelikh, jmestwa-coder, liyinlong, AlbertoFVisconti, Aylin Dmello, Cuda Chen, Ezra Li, Madhurendra Purbay, Makkar, Pastoray, Paul Dreik, Pavel Kruglov, Piotr Kubaj, Yusuf İhsan Görgel, metsw24-max, neil, pratap singh, Vladimir Saraikin, wankun, xaldarof, Riyane El Qoqui, Justin Li and others. Thank you!

C# strings are UTF-16. Most characters are one 16-bit code unit. Characters outside the basic multilingual plane, emoji included, take two: a high surrogate (U+D800 to U+DBFF) followed by a low surrogate (U+DC00 to U+DFFF). A surrogate with the wrong neighbor, or with none, is ill-formed.

You should never send an ill-formed string to disk or to the network. It is a bad practice.

In JavaScript, we have fast functions to fix strings or check whether they need fixing:

• String.prototype.toWellFormed() replaces every lone surrogate with U+FFFD.
• isWellFormed() reports whether any replacement is needed.

I added both functions to my C# library SimdUnicode, in pull request 54. The algorithm is the same that we contributed to the JavaScript engine V8, so Chrome already fixes strings this way.

string s = UTF16.ToWellFormed(input); // same instance, when the input is already well formed
bool ok = UTF16.IsWellFormed(span);

When the input is well formed, ToWellFormed returns it as is. No allocation.

How are strings fixed? Basically, you replace bad inputs by the replacement character U+FFFD.

Our processors have special instructions called SIMD that allow data parallelism: you can compare multiple values at once. Recent x64 processors from AMD and Intel have better data parallelism than ARM chips, although both have powerful instructions.

The conventional approach in C# to repair a string is a function such as the following.

static void Repair(ReadOnlySpan<char> input, Span<char> output)
{
    input.CopyTo(output);
    int i = NextError(output, 0);
    while (i >= 0)
    {
        output[i] = 'uFFFD';
        i = NextError(output, i + 1);
    }
}
// Index of the next lone surrogate at or after 'start', or -1 if none.
static int NextError(ReadOnlySpan<char> s, int start)
{
    int i = start;
    while (true)
    {
        int k = s.Slice(i).IndexOfAnyInRange('uD800', 'uDFFF');
        if (k < 0) return -1;
        i += k;
        if (char.IsHighSurrogate(s[i]) && i + 1 < s.Length && char.IsLowSurrogate(s[i + 1]))
            i += 2; // valid pair, skip it
        else
            return i;
    }
}

In SimdUnicode, I also use data parallelism.

Let me measure.

On the Xeon, with AVX-512, Latin validates at 69 GB/s against 33 GB/s for IndexOfAnyInRange. The Emoji input is well formed, and it is nothing but surrogate pairs. The runtime search drops to 0.4 GB/s. Our check holds 53 GB/s.

Our results are similar on the M4 Max, although a bit less impressive compared to the Intel results.

Validation can return at the first lone surrogate. The buffer form of ToWellFormed writes every code unit, a copy of the input or U+FFFD. When the input is well formed, it is effectively a memory copy. Thus we can compare the performance against a copy.

Roughly speaking, we are consistently about as fast as a copy.

Versions used: .NET SDK 10.0.400 on Linux, 10.0.103 on macOS. Intel Xeon Gold 6548N (Emerald Rapids). Apple M4 Max.

Clausecker, R., & Lemire, D. (2026). Fixing ill-formed UTF-16 strings with SIMD instructions. Software: Practice and Experience. (arXiv)

Source code.

We create new strings all the time. How quickly can you produce short strings in your programming languge? To create a meaningful string, we convert an integer to a string. In Python, that is str(i). The loop stores each new string in a small ring buffer of 1024 slots, so that the engine cannot simply discard the work.

def from_int(n):
    b = buf
    for i in range(n):
        b[i & 1023] = str(i)

In JavaScript (Node.js and Bun), I use String(i):

function from_int(n) {
  for (let i = 0; i < n; i++) buf[i & 1023] = String(i);
}

In C++, I use std::to_string:

void from_int(uint64_t n) {
  for (uint64_t i = 0; i < n; i++) buf[i & 1023] = std::to_string(i);
}

In Rust, I use the standard to_string() and, as an alternative, the popular itoa crate, which is designed for fast integer formatting:

fn std_to_string(buf: &mut [String], n: u64) {
    for i in 0..n {
        buf[(i & 1023) as usize] = i.to_string();
    }
}
fn itoa_to_string(buf: &mut [String], n: u64) {
    let mut b = itoa::Buffer::new();
    for i in 0..n {
        buf[(i & 1023) as usize] = b.format(i).to_owned();
    }
}

In Go, I use strconv.Itoa:

func fromInt(n int) {
    for i := 0; i < n; i++ {
        buf[i&1023] = strconv.Itoa(i)
    }
}

In Nim, I use the $ operator:

proc fromInt(n: int) =
  for i in 0 ..< n:
    buf[i and 1023] = $i

The integers go from 0 to 100 million (10 million in Python), so each string has up to eight digits. I report the best of five runs on an Apple M4 Max. C++ is compiled with -O3, Rust in release mode, Nim with -d:danger.

Millions of strings per second:

Python is the slowest at 44 ns per string, about three times slower than JavaScript and eight times slower than C++.

The compiled languages that allocate each string on the heap (Rust, Go, Nim) end up in the same range as JavaScript: 12 to 16 ns per string. Rust with its standard to_string() is even a bit slower than Node.js and Bun. Garbage-collected runtimes like Go and JavaScript are very good at allocating many small, short-lived objects.

C++ wins by a wide margin at 5.4 ns per string. The trick is the small string optimization: a std::string stores short strings, directly inside the object. Our strings have at most eight digits, so C++ never calls the memory allocator.

Versions used: macOS 15.7.7, CPython 3.14.5, Node.js 25.9.0, Bun 1.4.2, Apple clang 17.0.0 (clang-1700.6.4.2) with libc++, rustc 1.94.1 with itoa 1.0.18, Go 1.24.3, Nim 2.2.12.

Source code.

Mainstream economists have been advocating for the marvellous effects of global trade for decades. Who needs these factory jobs anyway? We’ll be designing the robots and the nuclear rockets.

Except that, no. It does not work like that.

If you have the factories, sooner or later, you get the designers. If you don’t have the factories, you lose the expertise. Or you never get it.

There aren’t that many people left in the Silicon Valley capable of working on silicon. You can design a chip from California. Making one is done where the fab is.

You can get a PhD in robotics in Quebec City, but you won’t be designing robots unless you fly over to where they are made. Zoom calls won’t cut it at scale.

Jensen Huang’s account, as Joseph Steinberg reported it, is that American manufacturing jobs declined because the work was outsourced. Steinberg says this is flatly wrong, and that technology accounts for the vast majority of the decline in manufacturing’s share of employment.

In 2000 there were 17.3 million manufacturing jobs in the United States. The peak was 19.6 million, in June 1979. From the early 1980s to 2000 the count stayed high, apart from the recessions. China joined the WTO in December 2001. By 2010 manufacturing employment was 11.5 million. In August 2026 it was 12.6 million.

That is 5.8 million jobs gone in a decade. About a million have come back since the bottom. The rest have not.

If technology were the main driver, output should have kept rising while the jobs fell. The fifteen years before 2000 are what that looks like. Manufacturing output nearly doubled, from an index of 51 to an index of 93 (2017 = 100). Employment went from 17.8 million to 17.3 million. The robots were already here in 1985. Employment did not collapse.

After 2000, production stalled. Manufacturing output peaked just under 107 in December 2007. In August 2026 the index was 99.1. Total industrial production was 103.1. American factories are not turning out a flood of extra goods. They are turning out roughly what they turned out twenty years ago.

Output per hour did rise while the jobs were disappearing. The BLS index of manufacturing labor productivity went from about 70 in 2000 to about 100 in 2010. Then it stopped. In early 2026 it was still about 100. Flat output, fewer workers, a higher ratio. The ratio has been flat for fifteen years.

The problem after 2000 was not that American factories became too productive. What changed after 2000 was where the goods were made.

And now, often, we don’t know anymore how to make things. Human expertise matters, and you maintain it by building stuff locally. You are not going to design microprocessors in Maine. It just won’t happen.

I maintain and comaintain several open-source libraries. Some of them are widely used: ada parses URLs in Node.js, fast_float parses numbers in GCC’s standard library and in Chromium, simdjson parses JSON in Node.js, simdutf validates and transcodes Unicode in Node.js, and the Roaring bitmap libraries sit inside many database engines.

These libraries are mature. They have been optimized for years, by me and by others. For a long time, their performance was flat. Not because nobody cared, but because the remaining gains were expensive: each one required a few days of careful work, and nobody had the days.

Then, in 2026, six of them got much faster, most of it in a few weeks of summer.

To formalize my feeling, I rebuilt every commit of each library from scratch and benchmarked it on one machine (an Intel Xeon Gold 6548N). I track the speedup over time relative to August 2024. Thus the value 1.0 means no speedup. Whereas 2.0 means that the performance doubled. The lines are steps because performance only changes at a commit.

I should say that I cannot know how much AI was involved in each instance. I don’t ask how people arrived at their code. All I ask is that it be good. As for myself, I code with Claude (Opus 5), Grok and DeepSeek (V4 Pro). I was an early adopter of Grok for coding, and it got really good over time.

1. roaring (compressed bitmaps, Go)

The roaring library is the Go version of the Roaring index data structure. Decoding to an array got 2.5 times faster, the multi-way union FastOr got 3.1 times faster on one data set, the many-value iterator got 4.5 to 5.9 times faster, and the intersection cardinality gained 10%.

One of the contributors is an AI, actually. It is perfloop. (Disclosure: I am an advisor for perfloop.)

I did a lot of work. We also got help from Philipp Klose who declared using Claude.

2. ada (URL parsing)

The ada library is a standard compliant URL parser. From August 2024 to July 2026, about 550 commits went in and the throughput on a corpus of 100,000 URLs stayed at 0.54 GB/s. Then, in six weeks, it went to 1.28 GB/s: 2.4 times faster, about 15 million URLs per second on one core.

Most of the optimizations were done by Yagiz Nizipli, my long-time co-author. Yagiz works at SpaceX and uses Cursor (presumably with a grok model). Abdul Rawoof Khan and Dillon Mulroy also contributed an optimization each. I worked at optimizing IP address parsing, but it won’t show in this particular benchmark.

3. fast_float (number parsing)

The fast_float library parses floating-point numbers from text. It is part of GCC and most browsers. Performance was flat for fifteen months. Then, from March to July 2026, it gained 43% on one file (canada.txt, long coordinates) and 70% on another (mesh.txt, short coordinates). The optimizations should be credited to Koleman Nix and Filipe Oliveira.

4. simdjson (JSON serialization and deserialization with C++26 reflection)

The simdjson library recently gained support for C++26 static reflection: you serialize and parse your own structs directly, with no glue code. Since February 2026, serialization is 1.6 times faster on twitter.json and 2.1 times faster on citm_catalog.json. Deserialization, JSON straight into a struct, gained a more modest 10% and 14% (the second panel). (The reflection code only exists since early 2026.) The number of instructions per byte fell by almost exactly the same ratio as the throughput rose: from 6.1 to 3.1 instructions per byte on citm_catalog.json serialization.

Francisco Geiman Thiesen (Microsoft) did most of the work on the serialization side while I mostly helped improve our parsing. Francisco uses Claude.

5. simdutf (Unicode validation and transcoding)

The simdutf library validates and transcodes UTF-8, UTF-16 and UTF-32, and encodes and decodes base64. ASCII validation went from 83 GB/s to 160 GB/s. UTF-16 validation went from 62 GB/s to 102 GB/s. Base64 decoding gained 17%.

The work was done by Yagiz Nizipli (again) and myself.

The library got other amazing optimizations that do not show up on this benchmark by Gaspard Petit and Shreesh Adiga.

6. CRoaring (compressed bitmaps, C)

CRoaring implements Roaring bitmaps in C. On the real data sets from the repository, membership tests (contains) got 2.4 times faster, the cardinality of 64-bit bitmaps got 4.9 times faster, iterating over a 64-bit bitmap got 1.9 times faster, decoding a dense bitmap to an array got 2.2 times faster. Unions gained a more modest 13% to 16%.

The authors were Andrei Gudkov and myself.

What happened

The techniques used are all well-known. So why all these optimizations all of a sudden? Simply put, in my view, because it got cheap to try new ideas.

There is a lot of talk about the risks of AI in software. Human beings tend to be susceptible to the one-sided bet fallacy: when we see the downsides, we tend to ignore the benefits. Cars kill people, but ambulances save them.

In this instance, the benefits are concrete. Millions of people run these libraries, and this summer, they got faster.

Our processors can execute many instructions per cycle; they are superscalar. But not all instructions are equal.

Branches are particularly tricky. A branch occurs often when you use an if-then clause or a loop. We distinguish between a taken branch and a not taken branch.

A not-taken branch is often cheap. The processor just keeps going.

A taken branch jumps to a new location. A taken branch can be more expensive.

You will often hear that processors are limited to one taken branch per cycle.

I decided to test it out with a loop with an if inside it. Here is the function I tested in Go.

func lastHit(p []byte, thresh byte, last *byte) {
    n := len(p)
    if n == 0 {
        return
    }
    i := 0
    for {
        v := p[i]
        if v > thresh {
            *last = v
        }
        i++
        if i == n {
            break
        }
    }
}

Look at the main loop. We load a value from an array, we compare it with a threshold. If it is greater than the threshold, then we assign it to the last pointer. So the function effectively records the last seen value that is greater than the threshold. That’s pretty reasonable code.

Consider the case where you always miss. The values are always smaller than or equal to the threshold. In these cases, we get two taken branches in close proximity, but no store. (It is a bit confusing but that’s how the Go compiler does it.)

The processor that struggles the most is the AMD Zen 4 processor. But AMD Zen 5 is much better.

So two processors are able to take two branches in less than 2 cycles on average in this test: the Apple processor (M4 Max) and the Granite Rapids processor.

This means that, yes, modern processors can execute more than one taken branch per cycle under some conditions.

The code is under benchmark/experiments/ifloop in the GitHub repo.

I see the mathematicians panicking at what we can do with agentic AI.

Let us be clear on what we are talking about. In a few weeks, someone who is not a high-level mathematician can, with AI, produce the equivalent of a good PhD thesis in math. I could see a top 1% high school student, given enough of an AI token budget, just write the equivalent of a PhD thesis as a hobby.

If Joe Smith completed a PhD thesis in math back in 2022, it is now possible for a really smart high schooler to generate the same output while playing video games. Joe must not be too happy about it, especially if Joe is still looking for a prestigious faculty position.

Notice how we are not so excited. I mean, why aren’t we celebrating this incredible breakthrough? Finally, all the math problems we have can be solved faster! It is worth reflecting on why we don’t care.

Part of the issue is that most of academic research lost its customers years ago.

In part thanks to the arrival of “peer review” in the 1970s, we have long ago closed the research world into siloed communities. The purpose of the academic output is to sort people out for jobs. Write papers that are impressive and you may get a good professorship. If you don’t, then you will be flipping burgers. There is an incredible glut of people with academic credentials, but only so many jobs at the top. The glut has been continuously, and somewhat deliberately, increasing.

If you hold a PhD today, your chance of having a tenure-track or tenured position is about 10% and falling. Let me be clear. Any academic who shows worry for what the PhDs might do now should look in a mirror because you have been training too many for decades. And, also, if fewer people decide to go for the PhD that might be more than fine.

AI disrupts the sorting mechanism in mathematics… But do you think for a minute that it does not apply to mechanical engineering, sociology, etc.? Is that bad news? No. I think that it is excellent news and I have been saying so for years.

Here is what I wrote in 2024…

« AI’s ability to generate vast amounts of text raises concerns about a potential flood of irrelevant theoretical papers, further straining the evaluation system. Stonebraker’s (2018) call for rewarding problem-solving over publication needs revisiting. Perhaps the emphasis should be on the impact and significance of research, not just its passage through peer review—a skill replicable by AI. AI can pave the way for a “golden age” of scientific progress if we can develop new evaluation methods focused on problem-solving and real-world impact. The scientific community must adapt to the evolving landscape. By recognizing the limitations of peer review and prioritizing the pursuit of meaningful solutions, we can ensure that AI becomes a catalyst for scientific advancement, not a detriment. »

I predict that, over time, the focus will move away from “papers as the final output.”

Nobody wants a paper about cancer, we want to eradicate cancer. We should reward people who get results, no matter which tools they use. And solving a problem because it is impressive to do so is not enough.

« But Daniel, Mathematicians can’t cure cancer or give us antigravity. »

Maybe it is time they try. Frankly, the AI disruption might be precisely what we needed.

« But Daniel, won’t AI just replace all of us? »

I wish. But thus far, it is not happening. I have more AI accounts than most people have pencils and I am working 50 hours a week. Intelligence is not a scalar quantity. Once an AI can do something, I somehow always find more work to do.

Further reading. Daniel Lemire, Will AI Flood Us with Irrelevant Papers? Communications of the ACM, Vol. 67, No. 9 (September 2024).

Recently, I looked at how AMD made its chips 50% faster within a span of two years, in How did AMD Ryzen get 50% faster in two years? Many people asked me to cover Apple processors. So let us go.

Consider the base Apple Silicon chips: M2 (2022), M3 (2023), M4 (2024) and M5 (2025). I am skipping the original one (M1) for simplicity and I am not considering the Pro and Max chips. The M6 is out there too, just released, but I am going to cover it at the end.

On Geekbench 6, performance went up by about 50% in three years. The increase is rather gradual. No big jump.

The 2025 chip is 52% faster on a single core than the 2022 chip, and 83% faster with all cores.

Unlike AMD, Apple improved the CPU frequency quite a bit. The P-core went from 3.5 GHz to 4.6 GHz, a 30% increase. On its own, it can explain the bulk of the single-core performance increase.

But still, even if we normalized the clock speed, an Apple M5 would be noticeably faster than an Apple M2.

Let us look at the transistor count. For the M5, I could not find any information. But there is otherwise a nice gradual increase. From the M2 to the M4, the number went up by about 40%, from 20 billion to 28 billion.

Where did they go?

The base M4 processor has two extra efficiency cores. This will not help with single-core performance, but it will help multicore performance.

But what about each individual core?

An important variable at the core level is the number of instructions per cycle. And this went up. Decode width went from 8 instructions per cycle on the M2 to 10 on the M4 and the M5, which is higher than what most competitors can do. So you can get more compute per cycle.

Thus far, we do not see much difference between the M4 and the M5, and yet the M5 can be significantly faster. Let us look next at memory bandwidth.

And yeah, the M5 has much more memory bandwidth: 154 GB/s against 100 GB/s on the M2 and the M3. That helps on multicore processing tasks, as it is when you are most likely to run out of bandwidth.

What about data parallelism? AMD improved data parallelism (SIMD) considerably. The story is much less interesting with Apple processors. They have four 128-bit execution units, all of them. Meanwhile the Zen 5 has four 512-bit units. It is four times more. What is new with the M4 and the M5 is the new 512-bit SME matrix unit.

But there is a new iteration. The M6.

The number of cores goes from 10 to 12. They added two ‘super’ cores. The bandwidth is up a bit. My expectation is that Apple will still again deliver a boost in performance.

In effect, what I am demonstrating is that CPUs are not boring. They are improving fast.

I already made the point that processors are getting wider. See also Memory-level parallelism: Apple M2 vs Apple M4.

ARM processors, like those in your phone, have instructions capable of processing several elements at once (SIMD). These instructions are called NEON. But many newer processors have a different SIMD extension called SVE. The latest ARM processors have SVE2. Unfortunately, Apple has not yet adopted SVE, but SVE processors are available in the cloud.

In April, I wrote that the SVE2 match instruction might be the fastest way to match characters on ARM processors. At the time, my benchmark was a toy. The question was whether the idea survives contact with a real parser. Madhurendra Purbay, an engineer at ARM, answered the question with a pull request to the simdjson library. Let me go through what it does and what it buys us.

The simdjson library includes a fast JSON parser. JSON is a ubiquitous data format online; everyone uses it. It is made of strings, numbers, arrays ([1,2,3]) and objects. An object is a key-value map where keys are strings, written as {"key1": 1, "key2": 2}. You can combine arrays and objects (e.g., an object can be in an array).

When the simdjson library indexes a JSON document, it first computes, for each block of 64 bytes, a few 64-bit masks. One of them marks the JSON structural characters (,, :, [, ], {, }). From these masks and a few others, we derive the positions of all the JSON tokens.

The ARM NEON version of the classifier, which I designed with Geoff Langdale years ago, uses a table lookup (tbl). Take the byte, add 3, keep the high nibble, and look it up in a 16-byte table that returns the one structural character with that nibble (or 0xff). If the looked-up byte is equal to the input, the input is structural. With the vaddq/vshrq/vqtbl1q/vceqq NEON intrinsics, it is four instructions per 16 bytes:

const uint8x16_t op_table = simd8<uint8_t>(
  0xff, 0, ',', ':', 0, '[', ']', '{', '}', 0, 0, 0, 0, 0, 0, 0
);
const uint8x16_t match_op_0 = vceqq_u8(
  vqtbl1q_u8(op_table, vshrq_n_u8(vaddq_u8(d0_0, vdupq_n_u8(3)), 4)),
  d0_0);

We get a vector of 16 bytes that are either 0x00 or 0xff. To turn four such vectors into one 64-bit mask, we AND each byte with a bit weight (1, 2, 4, …, 128) and sum adjacent bytes three times with addp. That is another eight instructions or so per 64-byte block, and it is shared with the white-space mask.

SVE2 has an instruction, match, that takes a vector of bytes and a second vector that acts as a small set: it produces a predicate (a mask) with a bit set at each position where the input byte belongs to the set. Because it works within 128-bit segments, the set is at most 16 bytes, which is plenty for our six structural characters. NEON has nothing like it; on x64, the closest thing is the SSE4.2 string-comparison instructions (pcmpistrm), which are slow.

In C++, using intrinsics, the code might look as follows:

// input is a set of 16 ASCII bytes we want to classify
// the whole thing compiles to little more than the match instruction
svbool_t match_operators_sve2(uint8x16_t input) {
  // The characters we care about. We use `0xff` as a
  // filler (it is an impossible byte value within a JSON document)
  const uint8x16_t operators = {
    0xff, ',', ':', '[', ']', '{', '}', 0xff,
    ',', ':', '[', ']', '{', '}', ',', ':'
  };
  // pg is a mask over the first 16 values
  const svbool_t pg = svptrue_pat_b8(SV_VL16);
  // 'move' the NEON register to SVE
  const svuint8_t data = svset_neonq_u8(svundef_u8(), input);
  // 'move' the table to SVE
  const svuint8_t table = svset_neonq_u8(svundef_u8(), operators);
  // call the match instruction
  return svmatch_u8(pg, data, table);
}

This function classifies 16 ASCII bytes with maybe just one instruction (match).

We use svset_neonq_u8, which is part of the NEON-SVE bridge. It allows you to mix and match NEON and SVE. The uint8x16_t type is a NEON type (16 8-bit integers). The type svuint8_t is an SVE type (a vector of 8-bit integers). As a convention, the first 16 bytes of SVE types are shared with NEON. Thus I expect that svuint8_t data = svset_neonq_u8(svundef_u8(), input) might compile to nothing.

The catch, as I explained in April, is that a predicate lives in a predicate register. My function returns svbool_t. SVE gives you no cheap way to move it to a general-purpose register: the architecture does not want to assume that a mask fits in 16 bits, since the registers might be wider. So we materialize the predicate as bytes instead, with a predicated select (svsel). The whole thing is a bit complicated (see Lemire (2025) for an explanation of the trick).

// We have four masks, p0, p1, p2, p3
// and we want to convert them each to a 16-bit value and then combine them to
// form a 64-bit mask.
uint64_t operator_predicates_to_bytes(
    svbool_t p0, svbool_t p1, svbool_t p2, svbool_t p3) {
  uint8x16_t bit_mask = {0x01, 0x02, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80,
                         0x01, 0x02, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80};
  // map the NEON register bit_mask to an SVE register
  const svuint8_t weights = svset_neonq_u8(svundef_u8(), bit_mask);
  // create a zero register
  const svuint8_t zero = svdup_n_u8(0);
  // where p0 is set, put the value from bit_mask, otherwise zero
  // The `svget_neonq_u8` function is part of the NEON-SVE bridge.
  const uint8x16_t b0 = svget_neonq_u8(svsel_u8(p0, weights, zero));
  const uint8x16_t b1 = svget_neonq_u8(svsel_u8(p1, weights, zero));
  const uint8x16_t b2 = svget_neonq_u8(svsel_u8(p2, weights, zero));
  const uint8x16_t b3 = svget_neonq_u8(svsel_u8(p3, weights, zero));
  const uint8x16_t sum = vpaddq_u8(vpaddq_u8(b0, b1), vpaddq_u8(b2, b3));
  return vgetq_lane_u64(vreinterpretq_u64_u8(vpaddq_u8(sum, sum)), 0);
}

Does it help?

I benchmarked the match classifier against the NEON classifier, using the parse benchmark that comes with simdjson, over the 22 JSON files that we use as our standard corpus (about 24 MB in total). The benchmark parses each file 300 times and keeps the best time. I ran each binary three times, interleaved with its counterpart, pinned to one core, and I kept the best. The run-to-run variation is under 1%. I used GCC 15 and LLVM clang 21 on Ubuntu 26.04, with -mcpu=native.

The match instruction is part of SVE2, not the original SVE. Among the AWS Graviton processors, the Graviton 3 (Neoverse V1) has SVE but not SVE2: it cannot run this code. So I used the two processors that can:

• Graviton 4 (Neoverse V2) on a c8g.2xlarge instance,
• Graviton 5 (Neoverse V3) on a c9g.2xlarge instance.

Both have 128-bit SVE registers.

Here is the gain in the indexing stage (stage 1), file by file, as the percentage increase in throughput with match over NEON. The dashed line in each panel is the geometric mean over the 22 files. First with GCC:

And with clang:

The files that gain the least (canada, mesh, marine_ik) are mostly numbers, where the indexing stage is cheap to begin with. The files that gain the most (gsoc-2018, random, github_events) are the ones with a lot of structure. No file gets slower, except canada and mesh on the Graviton 5 with GCC (by 2% to 3%, at the edge of what I can measure).

In absolute terms, the indexing stage goes from 4.8 GB/s to 5.3 GB/s on the Graviton 4 with clang (5.5 GB/s to 5.8 GB/s with GCC), and from 6.3 GB/s to 6.6 GB/s on the Graviton 5 with clang (7.1 GB/s to 7.3 GB/s with GCC). The Graviton 4 benefits more than the Graviton 5.

The second stage of the parser is untouched, so the gain on the whole parse is smaller: 2% to 4% on the Graviton 4 and 1% to 2% on the Graviton 5. It is a modest gain, but it comes from replacing four NEON instructions with one, in a routine that we had already tuned carefully.

In a real parser, match gives 3% to 9% faster indexing on Graviton 4 and Graviton 5, with a handful of intrinsics and no assembly. The code is in simdjson pull request 2866. It requires SVE2, which Apple processors and the older Graviton processors do not have, so simdjson falls back on NEON when SVE2 is not available at compile time.

The limitation today is that the code is compiled in only if you build with -mcpu=native or the equivalent: a default build gets the NEON code everywhere. The next step for simdjson is to select the SVE2 code at runtime, as we do with the various x64 instruction sets, so that a default build uses it on processors that have the instruction. I am working on it.

Credit: The match classifier is the work of Madhurendra Purbay (ARM), from his pull request 2863, where he used a different (and slightly faster) technique, with inline assembly, to extract the predicates. My benchmark results and scripts are available.

References

Keiser, J., & Lemire, D. (2024). On-demand JSON: A better way to parse documents?. Software: Practice and Experience, 54(6), 1074-1086.

Langdale, G., & Lemire, D. (2019). Parsing gigabytes of JSON per second. The VLDB Journal, 28(6), 941-960. (arXiv)

Lemire, D. (2025). Mixing ARM NEON with SVE code for fun and profit.

Lemire, D. (2025). Scanning HTML at tens of gigabytes per second on ARM processors. Software: Practice and Experience, 55(7), 1256-1265.

People still tell me that CPUs are boring. That nothing much happens anymore.

Let us look at AMD Ryzen 7 processors from 2022 to 2024: the 5800X3D (Zen 3), the 7800X3D (Zen 4) and the 9800X3D (Zen 5). They are comparable 8-core chips with 3D V-Cache. I have written about them before, in How stagnant is CPU technology?

On Geekbench 6, performance went up by about 50% in two years.

The 2024 chip is 47% faster on a single core than the 2022 chip, and 58% faster with all cores.

The clock did not do that. Max boost went from 4.5 GHz to 5.2 GHz, a 15% increase.

The number of transistors is way up, by about 50%, from roughly 11 billion to 16 billion. Most of the extra transistors went into the core, not the cache.

How do you turn extra transistors into extra performance?

You make the core wider, and you give it more to work with. Dispatch width went from a maximum of 6 instructions per cycle to 8. The L2 cache per core doubled, from 512 KB to 1 MB. The L1 data cache went from 32 KB to 48 KB. Integer ALUs went from 4 to 6. The reorder buffer grew from 256 to 448 entries, so the processor can keep more instructions in flight and schedule them better.

For data parallelism (SIMD), Zen 5 is a different machine. Zen 3 and Zen 4 had four 256-bit SIMD arithmetic units. Zen 5 has four 512-bit units. Loads and stores widened the same way: two 512-bit loads per cycle, one 512-bit store.

I already made the point that processors are getting wider. This is what that looks like on a desktop chip you can buy.

What about the next step? Zen 6 is arriving. AMD is talking about a 256-core Epyc part (Venice) with a gigabyte of L3 cache. We do not yet know what the desktop cores will look like. It could be wild.

Further reading: AMD’s 256-core Epyc 9996 ‘Venice’ (Tom’s Hardware).

C++23 added a new type to the standard library: std::flat_map. There is also a std::flat_set and other variants, but let me focus on std::flat_map.

A flat map is a sorted vector of keys next to a vector of values. A query is a binary search over the sorted keys.

You need a recent standard library: GCC 15’s libstdc++ has std::flat_map, as does LLVM’s libc++ since LLVM 20 (Apple’s clang 17 has it too).

Because the container is just two arrays, we can save it by copying the two arrays. Take a map from 64-bit keys to a fixed-size value:

struct point {
  double x;
  double y;
};
std::flat_map<uint64_t, point> m;
m[42] = {1.0, 2.0};
m[7] = {3.0, 4.0};
m[1000] = {5.0, 6.0};
std::vector<uint64_t> keys(m.keys().begin(), m.keys().end());
std::vector<point> values(m.values().begin(), m.values().end());
// keys   = {7, 42, 1000}
// values = {{3, 4}, {1, 2}, {5, 6}}

The keys array is 8 * keys.size() bytes and the values array is 16 * values.size() bytes: you can write them to disk with two memcpy or write calls, as they are.

If you loaded both arrays from a network or the disk, you can then move them straight into your std::flat_map like so.

std::flat_map<uint64_t, point> back(std::sorted_unique,
                                    std::move(keys), std::move(values));
// back == m

With the std::sorted_unique tag, you promise that the keys are sorted and distinct. In practice, if the data comes from the network or some untrusted source, you should do some sanity testing.

Like the good old std::map, a flat map keeps its keys in sorted order, so you can iterate over it in key order. But the std::map is a red-black tree so you have significant storage overhead and possibly poor memory locality.

The downside of a std::flat_map is that it might be slower if you need to mutate it.

Let us examine the speed.

I use random 64-bit keys mapped to 64-bit values, GCC 16.1 with -O3
-march=native, on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core. I report nanoseconds per operation

Let us start with inserting elements one at a time, in random order, into an initially empty container.

Up to maybe a thousand elements or so, the std::flat_map is fine and maybe faster than the std::map. But as the size grows, the time goes up quadratically. Thus, do not use an std::flat_map if you need to insert millions of keys in random order. It is bad.

If the keys arrive in increasing order, then it is entirely different. The std::flat_map is much faster.

You can also do bulk inserts. Given a batch of new pairs, in any order, the map sorts the batch and merges it with its arrays in one pass, instead of shifting the arrays once per element:

std::vector<std::pair<uint64_t, point>> more = {{500, {7.0, 8.0}},
                                                {3, {9.0, 10.0}}};
m.insert_range(more);
// keys = {3, 7, 42, 500, 1000}

Constructing a map from a range of random key-value pairs works the same way, and it is much faster with a std::flat_map:

Random lookups are also much faster for large maps in part because the std::flat_map uses less memory.

In many practical cases, the new std::flat_map is a better alternative to the std::map. It is somewhat amusing considering that you are replacing a fancy textbook data structure (red-black tree) with a trivial one.

My source code is available.

We represent floating-point numbers using the IEEE standard. For very small numbers, the standard uses special subnormal numbers. Unfortunately, they have a reputation of making operations slow. Thus video game programmers and machine learning specialists sometimes avoid computing with subnormal numbers for performance.

How slow are they? Let me measure. I wrote a small C++ benchmark with a few kernels over arrays of 16384 values (small enough to fit in cache):

• multiply each value by 0.75,
• add two arrays,
• divide each value by 3,
• multiply normal values by a tiny constant (2^-1030) so that the inputs are normal but the outputs are subnormal,
• a dependent chain x *= 0.9999 repeated 16384 times.

For each kernel, I feed either normal values (in [0.5, 1)), subnormal values, or normal values where one value in a hundred is subnormal. The compiler is allowed to autovectorize the array computations. I use GCC 15 with -O3 -march=native on Linux and Apple clang 17 with the same flags on macOS. I also checked with clang 21 on Linux to make sure.

I ran the benchmark on five processors:

• Intel Xeon 6975P-C (Granite Rapids), on an AWS c8i.xlarge instance,
• Intel Xeon Gold 6548N (Emerald Rapids), a server in my lab,
• AMD EPYC 9R45 (Zen 5), on an AWS c8a.xlarge instance,
• AWS Graviton 5 (Arm Neoverse V3), on a c9g.xlarge instance,
• Apple M4 Max.

Here are the results for double values, in nanoseconds per element (or per step).

Intel Granite Rapids

Intel Emerald Rapids

AMD Zen 5

AWS Graviton 5

Apple M4 Max

On Intel processors, a multiplication involving a subnormal number is about 45 to 50 times slower than a multiplication over normal numbers. A division is 18 times slower. The dependent chain, where each multiplication waits for the previous one, goes from about 1 ns to over 30 ns per step. A normal multiplication in the dependent chain has a latency of 4 cycles. With a subnormal, it has a latency of 128 cycles. It does not matter whether the subnormal is an input or an output: multiplying normal numbers into a subnormal result is just as slow as multiplying subnormal numbers. The exception is additions and subtractions: they run at full speed. Even if subnormals are rare (1%), the cost on Intel can be significant because when the compiler vectorizes the computation, a single subnormal can slow down a whole block of computations.

AMD does much better. On Zen 5, multiplications and additions run at full speed regardless of the inputs. The dependent multiplication chain is a third slower (0.66 ns to 0.88 ns per step): the multiplier needs an extra cycle or so to handle a subnormal. Divisions are about twice as slow. Interestingly, with divisions, having one subnormal in a hundred is almost as slow as having all subnormals. The two Arm processors, the Graviton 5 and the Apple M4 Max, do not care at all. Subnormal numbers are handled at full speed.

Thus it appears that on the latest AMD and ARM processors subnormals might not be a concern. But they remain very much a performance issue under Intel processors.

My source code is available.

Mathematicians are unhappy about OpenAI. Several influential mathematicians wrote an open letter. The gist of their argument is that they form a community that trains young people.

When AI started producing breakthroughs on hard mathematical problems, I asked what a very smart 17-year-old would feel. Do you still choose a math major and train yourself to prove difficult results by hand?

This crisis has been coming for a long time. When I was a kid, the four-colour theorem was proved by a computer, in 1976. An intense debate followed. Does it count as a proof?

I wrote my PhD thesis using symbolic algebra software. To my knowledge, nobody then would publish their scripts. I tried to include mine. I was told it would make me look bad.

The letter says:

“In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas.”

They worry that kids will not choose to become old-school mathematicians. That is a reasonable fear. Yet they do not seem to consider that some kids might still do mathematics, just in a very different way.

The famous mathematician Doron Zeilberger announced the problem in 2009:

“Teaching computers how to discover and prove mathematical results is certainly the way to go, and I believe that mathematicians who continue to do pure human, pencil-and-paper, computer-less, research, are wasting their time.”

The letter implies that people, because of AI, will stop having ideas, or will stop taking the time to understand the issues. If that were true, mathematics would continue only on computers, with no humans interested, or it would stop. Both scenarios assume that people care about mathematics only when they can claim credit.

I am not sure why I cannot study a proof generated by AI if I want to. I can give talks about it. What becomes less likely is the reward of having been the one who proved the result.

The rest of the letter makes a big deal of credit. What if the AI builds on what it read and does not give proper credit? Where is the evidence that AI is worse than human beings at citing sources?

The letter notes that mathematics contributes to society. It never considers that faster progress might increase those contributions.

My stance is simple.

Mathematicians, software developers, engineers, lawyers, physicians will all learn to work with AI. You cannot put the genie back in the bottle. Only a totalitarian world government could try, and some of us would rather avoid that outcome.

Difficult proofs will now be built with AI, just as most code will be written with AI. People who insist on pen and paper should view themselves as artists.

Other mathematicians will work with AI. They will have no trouble finding interested kids. They will contribute to society.

We are told that AI entities much like ChatGPT might soon kill us all. The statement is vague and unfalsifiable. It might be true, it might be false. People with credentials (e.g., Turing Award recipient Yoshua Bengio) believe it.

Many still remember the Year-2000 bug. Our computers used two-digit coding for dates, and some software could get confused. At the time, experts worried that a bug in dates might trigger nuclear Armageddon or an infrastructure collapse. At the very least, planes could fall.

Americans had a moral panic over alcohol that lasted more than a century. It began with pledges of moderation. From 1920 the Volstead Act banned nearly all legal drink. It collapsed in 1933, at great expense.

The Club of Rome predicted mass starvation. As an answer, we sterilized by force millions of Indians, and introduced the devastating one-child policy in China. The authors were never held accountable. The projections were purely mathematical, unescapable. They said. But also totally wrong and silly. Yet, we listened to them and caused great harm.

End-of-the-world scenarios are nothing new. Pretty much all civilizations have lived with various such predictions.

Some people are offended by my comparisons. I truly do not mean to offend. But the fact that disagreeing can lead to deep offense is, by itself, a sign that we face a moral issue. There is a sense in which you must agree that these intense fears are warranted.

Many will remember that when OpenAI first developed GPT-2, they told the world that it was too dangerous to release. Year after year, we were warned that the next iteration of it would doom us all.

A large language model takes tokens (words) in and outputs tokens (words). The big models can take many, many tokens in. And they do much compute. And they are based on clever ideas like vector embeddings. But, ultimately, no large language model can do anything but output tokens. You can build a better model, but the model itself does not ‘learn’. It is a fixed set of weights. If you take a model that has been used for months, and always feed the same tokens, you will get the same results (up to some randomness).

I fear that some people exploit the fact that people cannot grasp how conceptually simple a language model is. In any case, most people don’t understand how things work.

Things become interesting because these models can be hooked up to tools. So you can tell your language model that whenever it outputs ‘boom’, then a nuclear weapon will be launched. And if you hook up a nuclear weapon, then, certainly, you may start a nuclear war. So don’t do that.

One unproven thesis is that the models (that take in tokens and output tokens) will ‘decide’ to acquire access to these nuclear weapons, maybe through a subterfuge. What does that mean? It is always conveniently vague. In the movie WarGames (1983), a teenager uses his computer to access a computer in charge of nuclear weapons. He almost wipes out humanity. Could this happen by accident with a kid using a language model hooked up to the Internet? But if it happens, we should blame whoever hooked up a deadly computer to the Internet.

Of course, any technology is inherently dangerous. Invent the bow to go hunting, and someone might soon turn the bow against you. Invent the engine, and one might soon build tanks and destroy nations. Develop nuclear technology, and one might soon raze your cities.
 
Yet that is not what is at stake in these discussions. The concrete threats are not ascertained and addressed. No doubt, there are some people doing this work, hopefully in the US military. What if an adversary can take control of the economy or military installations? What if an AI agent goes rogue? It is worth investing time in designing defenses.

What we have instead is something of the sort:

1. A vague but global threat. It could be a fatal virus engineered in a lab, a climate catastrophe, a fatal bug affecting all our software, an alien invasion, a rogue AI, Jews taking over our institutions.
2. A few people come forward and they offer to save us. Importantly we must give them resources and influence. Ultimately, they seek a totalitarian solution: everyone must be made to agree so that we can be saved.
3. As the process unfolds, people with an opposing viewpoint are described as a danger. They must be silenced and discredited. Eventually, it can become moral to exaggerate the threat or to rewrite counterpoints. People must be made to understand one way or another.

In this instance, I refer to people who advocate that AI will doom us as AI Doomers. These people tend to carry a totalitarian ideology. Their ideas will only work if everyone is made to agree. And it would severely restrict the freedom of billions of people, although they usually present it differently.

Doomers do not have bad intentions. On the contrary, they are often really out there to save the world. But good intentions do not, in any way, justify the means nor guarantee a good outcome.

Human beings reason based on cultural knowledge. For centuries or more, totalitarian ideas have led to ruin or pain. We ignore the warnings at our peril. It is one after the other: alcohol, the need to restrict the number of children, and so on.

But shouldn’t we just be prudent and adopt their views, just in case? It is a fallacious argument. Members of the intellectual elite have a tendency to fall for the kind of hubris where they think that, if only they were given more power, the world would be better off. It is rarely true. Thomas Sowell has an excellent book on the topic, Intellectuals and Society. He makes the case that intellectuals often promote harmful ideas, at no cost to themselves. Rationally, we should therefore be cautious.

You are not safer without technology. In fact, the risk of human extinction is assuredly higher if we are poorer and have less technology.

Is this unprecedented? The printing press was unprecedented. Arabic numbers were unprecedented. Maybe we should go back to Roman numerals, to be safe. Fear of what is without precedent soon becomes indistinguishable from an anti-innovation stance.

What if you do not like the people who lead the big AI companies like OpenAI and Anthropic. Maybe you think that these billionaires are a danger. And you might be right. But consider the history of humanity. Wealthy people have primarily caused harm through the promotion of bad ideas. The mass murders are almost invariably derived from politics. Stalin, Hitler, Mao.

Throughout much of the world, we are facing demographic collapse and an inverted age pyramid. Soon there may be just one worker per retiree. Choosing to have fewer kids has consequences and we are about to face them. AI might be a way out of significant problems. If you are otherwise wealthy, that might not be a significant concern to you. But for the least fortunate, it might turn out to be quite a problem. Who will take care of you when you are sick? AI and robotics might prove useful to reduce suffering, don’t we think?

Further, we need to consider how powerful people might use the fear of AI for their own purposes. It is entirely credible that the owners of large companies could promote fear so that they get to write the regulations that will keep out their competitors, or merely as a form of cheap marketing.

How do I know that it is moral? Because there cannot be reasoned debate about a moral question. The facts are obvious or you are a bad person. Whenever there is a complex question, one that involves predicting the future, that cannot be discussed, unless it is in agreement with the side of fear, then you are very likely in a moral question. « Don’t you see, AI will soon kill all of us, it is obvious. » No explanation can be demanded.

You might accuse me of, in turn, promoting fear. But it should be obvious that I am doing no such thing. What I am encouraging rather is the use of reason. I am forced to give examples where inciting fear has led to disastrous effects, but my hope is that it will lead my reader to sit and reflect.

To my friends who fear AI, I urge you. Use reason. Do the work. Do not rely on hasty thought experiments. Work out the details. Think. Think about the countermeasures. And, please, do not include abstract thinking machines. A language model is a box that takes in token and produces tokens. Nothing more.

And for the rest of us. Let us build. Let us bring prosperity. Let us hasten the cure for cancer. Let us dream of exploring our solar system.

Further reading.

• • Nirit Weiss-Blatt, “The Weak Foundations of AI Doomsday” — AI Panic, May 15, 2026.
  • Cal Newport, “Dear A.I. Companies, the Doom Trolling Needs to Stop” — New York Times, June 17, 2026
  • Andrew Orlowski, “Anthropic’s doom predictions are merely hype intended to make AI look important” — The Telegraph, June 5, 2026.

If you write in C, by default, you use a single thread. Extra cores do not help until you create more threads. However, if you include the header <threads.h>, you can pass a function to thrd_create, and wait for it with thrd_join.

#include <threads.h>
#include <stdio.h>
int worker(void *arg) {
    printf("hello from thread %d\n", *(int *)arg);
    return 0;
}
int main(void) {
    thrd_t t;
    int id = 1;
    thrd_create(&t, worker, &id);
    thrd_join(t, NULL);
}

Be warned that C11 threads are an optional feature. If the macro __STDC_NO_THREADS__ is defined, you do not have them. Apple’s C library has never shipped <threads.h>, so the program above does not compile on macOS, and glibc only added it in version 2.28 (2018). On such systems you fall back on POSIX threads (pthread_create, pthread_join).

Once you have more than one thread, they may share memory. If two threads access the same non-atomic variable with no ordering between them, and at least one of them writes, the C language calls that a data race. In other words, it is unsafe.

If you have a variable and it is effectively constant, then it is fine to share it. But as soon as anyone changes it, then it might get corrupted. If it is not guarded somewhat, you are in trouble.

To be clear, that is what the C programming language says. I don’t mean that it will happen on your machine.

To get a better behavior, we can use atomic variables. In C, you have the <stdatomic.h> header.

An atomic integer is never garbage. You always read a value that was once written.

In practice, on most computers you might use today, aligned 8-, 16-, 32- and 64-bit loads and stores are atomic. The C language does not care about that, so if you don’t specifically require atomicity, you might get in trouble with your C compiler.

The next funny problem is that instructions can be reordered. When you write:

x = 2
y = 3

This may not happen in this sequence. The variable y might be set before the variable x. You may wonder why this is allowed at all. The fundamental reason is that our processors are quite complex. They have layers of buffers and they can execute multiple instructions at once. They can issue several memory loads or stores at once.

By default, in C, atomic accesses are all ordered. It is as if there is an oracle that watches all threads and comes up with a consistent story where everything is in order. This can be expensive, so we prefer not to do it that way.

At the other extreme is the relaxed model: your reads and stores are not garbage, and a given atomic still has one modification order (you will not see 1 and then 0 if the counter only went from 0 to 1), but there is no ordering with respect to other memory.

So we use something intermediate, the release and acquire semantics. They are ordering barriers. A strict barrier would be ‘everything before me really happens before me, and everything after me really happens after me’. (Where ‘really happens’ refers to visible effects, the hardware and compiler are allowed to cheat as long as you don’t catch them.) It is a bit too strong. So we split it in two parts: release and acquire. Intuitively, release means ‘if you see me, you see all the stuff before me’. Acquire means ‘I take that package, and everything I do after this load really happens after it’.

Consider the case where you have a resource (such as a block of allocated memory). You share this resource, but count how many people have access to it. When the counter goes to zero, you free the resource.

One thread could do…

access resource
decrement counter // I won't need it anymore

You see these operations happening one after the other, but they may not execute this way. It is possible that they overlap, or even that the decrement occurs before the access. It is entirely safe in a single threaded context.

Anyhow, so the following could happen

decrement counter // I won't need it anymore
access resource

But what if you have a second thread that does:

access resource
decrement counter // I won' need it anymore
if (counter is zero)
  free(resource)

You could have this interplay:

[thread2] access resource
[thread2] decrement counter
[thread1] decrement counter
[thread2] free(resource)
[thread1] access resource

That would be a bug.

So what you first do is make the decrement a ‘release access’ which means that operations that come before it cannot be reordered after it. So if you do it this way…

access resource
decrement counter using release

Then it is not possible that we ‘see’ the operations as if they happened in the reverse order.

But then we have a second problem. Release is enough for this thread: we cannot still be using the resource after we drop it. It does not tell the last owner that everyone else is finished. The last decrement is itself a release, so it does not observe the other threads’ releases. Without an acquire, that last thread can call free while another thread’s earlier access is not yet done.

[thread2] access resource
[thread2] decrement counter using release
[thread1] decrement counter using release
[thread1] free(resource)

Thread 2 did its access before its release decrement, but thread 1 never acquired, so it is not required to see that access as finished before free.

So we need the counterpart to a release, an acquire. The last owner acquires before it frees, and that pairs with everyone else’s release:

access resource
decrement counter with release
if (counter is zero)
  acquire barrier // see that everyone else is done
  free(resource)

Alternatively, you could do this.

access resource
decrement counter with release and acquire
if (counter is zero)
  free(resource)

The two are equivalent, but they are not necessarily equally cheap.

So let us consider a nice example. Let us build a small array that several threads can share. If you are the only owner, you overwrite an element in place. If not, you copy, then you update the copy. That is called copy-on-write. It is a really nice idea that you will find in many important systems.

We start with the type.

#include <assert.h>
#include <stdatomic.h>
#include <stdlib.h>
#define STR_SIZE 16
typedef struct {
    atomic_int refs;
    int values[STR_SIZE];
} shared_array;

The payload is a plain int array. Only refs is atomic. That is deliberate. We never write values while another thread might be reading them.

We create an instance like so.

shared_array *str_new(void) {
    shared_array *o = malloc(sizeof *o);
    if (o == NULL) {
        return NULL;
    }
    atomic_init(&o->refs, 1);
    for (int i = 0; i < STR_SIZE; i++) {
        o->values[i] = 0;
    }
    return o;
}

The atomic_init is not an atomic access in the memory-model sense. Nobody else has the pointer yet, so there is no other thread to race with. The caller owns one reference. It is just how we initialize an atomic_int.

Here is how we might naively release an instance.

// not real code
void obj_release(shared_array *o) {
    auto ref = o->refs;
    o->refs -= 1;
    if (ref != 1)
        return;
    // we are the last copy
    free(o);
}

What is the problem with this code?

The load and the decrement are two operations. Two threads can both read 2, both subtract, the counter hits zero, and nobody frees: the resource leaks. Write the check the other way around, decrementing first and then testing whether the counter is zero, as in the pseudocode above, and you get the mirror-image bug instead: with refs at 2, one thread decrements to 1, the other decrements to 0, both then read 0, and both call free. You need one atomic subtract that hands you the previous value: only the thread that saw 1 was last.

So you could try

void obj_release(shared_array *o) {
    if (atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed) != 1)
        return;
    free(o);
}

But suppose you have two owners, so that refs is 2. And you have two threads doing

(void)o->values[4];
obj_release(o);

One of them will call free, but the order could be

...
[thread1] o->values[4];
[thread2] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread1] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread1] free(o);
[thread2] o->values[4];

It is a bit confusing because things are not happening in order within thread 2:

[thread2] atomic_fetch_sub_explicit(&o->refs, 1, memory_order_relaxed)
[thread2] o->values[4];

But this is allowed.

So what we can do is put a release on the atomic_fetch_sub_explicit and then an acquire right before the free.

void obj_release(shared_array *o) {
    if (atomic_fetch_sub_explicit(&o->refs, 1, memory_order_release) != 1)
        return;
    atomic_thread_fence(memory_order_acquire);
    free(o);
}

The release on every decrement means “I am done with the payload.” The acquire fence, only on the last owner, means “I have seen that everyone else is done.” Then free is safe.

That release does double duty, as we are about to see. It is also what lets the last remaining owner write to the payload in place.

If a thread wants another reference to the same instance, it only needs a relaxed access.

shared_array *str_retain(shared_array *o) {
    atomic_fetch_add_explicit(&o->refs, 1, memory_order_relaxed);
    return o;
}

Why relaxed? Because the caller already holds a reference, so the object cannot be freed under us: the last owner would need our reference to be gone first.

We can now write update. It consumes the caller’s reference and returns a reference to the array that contains the new value, which may or may not be the same object. After you call it, you must not touch the pointer you passed in. There is one exception: if a copy was needed and the allocation failed, it returns NULL and leaves the caller’s reference to o untouched, so you still own it and must still release it.

shared_array *update(size_t idx, int value, shared_array *o) {
    assert(idx < STR_SIZE);
    if (atomic_load_explicit(&o->refs, memory_order_acquire) == 1) {
        o->values[idx] = value;
        return o;
    }
    shared_array *new_o = str_new();
    if (new_o == NULL) {
        return NULL;
    }
    for (int i = 0; i < STR_SIZE; i++) {
        new_o->values[i] = o->values[i];
    }
    new_o->values[idx] = value;
    obj_release(o);
    return new_o;
}

If the load reads 1, we are the only owner. No other thread holds a reference, so we can write values[idx] in place.

The load is an acquire. When the load reads 1, it may read the value written by the release decrement of the last other owner to drop out. Everything that thread did with values happens before our write. Nothing in our code appearing after such as o->values[idx] = value may move before it. No other thread still holds a reference, so the write does not race with a concurrent reader. Later, after a retain, other threads can see it.

On x64, acquire and release are effectively free at the CPU: ordinary loads already behave like acquire, ordinary stores like release. You still have to write them in C, or the compiler may reorder the payload accesses. ARM has a weaker memory model so the acquire/release require different instructions (ldapr, ldaddl) which may incur a small perforamnce hit.

The code is available.

In Python, the dict data structure is the conventional key-value structure. E.g., you might store a list of names as keys and have their phone numbers as values. Valentin Ignatev wrote this amusing post on X:

It is indeed widely believed that, in the strict sense, the dict data structure and its companion, the set data structure, are O(1), meaning that as you increase the size of the data structure, the time to insert or query a key remains constant.

Let us examine the claim.

A hash function is a function from objects (like strings, integers, etc.) to integer values. We typically expect hash functions to be random-like, although they should always map the same object to the same integer within the current program execution. From hash functions, we construct hash tables:

1. Create an array of buckets.
2. Given an object, apply the hash function to map it to a bucket.
3. Store the object in the bucket. When the bucket is already occupied, use some other trick (such as using a nearby bucket).

If everything goes well, access and insertion in a hash table take nearly constant time, meaning that the time they take is independent of the size of the hash table.

This can be almost true in many instances. However, it is not formally true. There are many reasons why it is false. For example, if your data structure grows, it might be necessary to reallocate, which will typically take time proportional to the size of the data structure. But we also have the issue of collisions. A collision is what happens when two objects have the same hash value. When we use hash tables, we assume that collisions are uncommon. But it is not difficult to create many of them by picking our objects carefully.

In Python, set and dict are hash tables. I can ‘easily’ make my version of Python crumble:

M = (1 << 61) - 1
values = [i * M for i in range(1, n + 1)]
s = set(values)                       # insertions
count = sum(v in s for v in values)   # checks

If the insertions and the checks are constant-time operations, then the whole construction and the entire check should take linear time. I ran this on an Apple M4 Max with Python 3.14, reporting the median of three runs.

The time roughly quadruples each time n doubles. That is quadratic time, not linear time. The membership checks behave the same way: 1066 ms at n = 16000. At a hundred thousand elements, building the set takes 45 seconds.

But could we create a hash table that would be truly constant-time? No. As the size of your data structure grows, it requires progressively slower memory. If you have a small hash table, it can reside in the CPU cache and be fast. Once it reaches megabytes in size, the data structure tends to live in RAM, which is much slower. And then, eventually, you have to store it on disk, which is even slower. And so forth.

To put it differently, saying that a hash table is O(1) or constant time is a model. It can be true, maybe even often, but it is not reality. Models are great teaching tools: they present a simplified model that you can quickly learn. But models can also introduce biases in how we think.

For example, even though you have read my paragraph that says that the dict data structure gets slower, you may not believe it. You may also believe that it is typically going to be the fastest approach you can use.

Let us consider another practical case. Suppose that you have a large map from strings to integers, that you build once and then only query. That is a common situation: a dictionary of words to identifiers, a lookup table of country codes, a table of feature names.

The fastconstmap library builds an immutable map from a dict[str, int]. It is suitable when your keys are known in advance.

I build a map from a million random sixteen-character strings to integers, and then look up every key in a shuffled order. With a dict, I write the obvious loop:

total = 0
for k in probes:
    total += d[k]

With fastconstmap, I ask for all the keys at once, writing the values into a buffer that I own, so that no Python object is allocated per key:

out = array("Q", bytes(8 * n))
cm.get_many_into(probes, out)

I am being generous to the dict. I reuse the same string objects for the lookups, and a Python string caches its hash value the first time it is computed. So the dict does not pay for hashing at all, while fastconstmap hashes every key every time. Here are the results, in nanoseconds per key.

The dict is not constant time. It goes from 22 ns to 202 ns per key as the map grows, a factor of nine, and it is not because the algorithm changed or because of collisions. It is because a million keys, their string objects, and their integer objects occupy about 116 bytes per key, so the lookups miss in the cache. The fastconstmap version needs 9 bytes per key: it stays in the cache much longer. Pay attention to how the numbers scale: the dict becomes 10 times slower as the size grows.

The lesson is always the same. Some models are useful but none of them is reality. Be mindful of cognitive biases.

The code is available.

JSON is a standard format for data interchange. It is effectively a tiny subset of JavaScript made of objects and arrays. It looks as follows {"key":1, "text":[1.0,2.0]}.

Many programming languages include a JSON library in their standard libraries: C#, Go, Java (soon), Python, JavaScript, etc. The Go implementation is convenient, but not especially fast.

Go 1.27 makes a new JSON package (encoding/json/v2) available by default in its standard library. The two APIs look almost the same:

import (
    json    "encoding/json"
    jsonv2  "encoding/json/v2"
)
b, err := json.Marshal(v)
b, err  = jsonv2.Marshal(v)
err = json.Unmarshal(b, &v)
err = jsonv2.Unmarshal(b, &v)

The two are not directly comparable as they differ with respect to Unicode validation, case sensitivity, etc. So it is not a drop-in replacement.

However, the legacy API (encoding/json) has also been reimplemented on top of the new engine. You can use the legacy API with either the new engine or the old one (GOEXPERIMENT=nojsonv2) through a flag. So we have three possibilities.

1. json (legacy) — encoding/json built with GOEXPERIMENT=nojsonv2, the original implementation
2. json (Go 1.27) — encoding/json as of 1.27, v1 API on the v2 backend
3. json/v2 — encoding/json/v2

I used the usual simdjson documents: twitter.json (632 kB, nested objects with short string keys), canada.json (2.25 MB, one large array of coordinates), and citm_catalog.json (1.73 MB, nested objects with numeric keys). I parse them into any (interface{}), which is the general-purpose path.

I ran this on an Apple M4 Max and on an Intel Xeon Gold 6548N (Emerald Rapids) using Go 1.27.0, on a single core (GOMAXPROCS=1), reporting the median of eight runs.

When unmarshalling, the legacy API on the new backend is faster than the original on twitter.json (172 MB/s to 203 MB/s) and on citm_catalog.json (186 MB/s to 241 MB/s), but slower on canada.json (128 MB/s down to 106 MB/s). When marshalling, it is up to twice as fast: 198 MB/s to 374 MB/s on twitter.json. So merely upgrading to Go 1.27, without changing a line of code, should make marshalling faster.

Switching to the new API helps more. Compared to the original implementation, encoding/json/v2 unmarshals 1.5x to 2.3x faster and marshals 1.2x to 3x faster. Compared to the Go 1.27 legacy API, unmarshalling gains another 1.8x to 2x, while marshalling gains much less (1.0x to 1.7x): part of the remaining difference is that json/v2 does less work during marshalling.

Thus far, I was unmarshalling into any, meaning that I assumed that I did not know the structure of the document. I also round-trip a slice of 10,000 small structs:

type Record struct {
    ID     int      `json:"id"`
    Name   string   `json:"name"`
    Email  string   `json:"email"`
    Active bool     `json:"active"`
    Score  float64  `json:"score"`
    Tags   []string `json:"tags"`
}

The schema is specified: the JSON must be [{"id":..., "name":...}, {"id":..., "name":...}...]. I still get faster unmarshalling with the new API, but the legacy API with the legacy engine is faster when marshalling.

The original implementation is faster. The Go 1.27 release notes said that marshal performance is broadly at parity with the previous implementation. For my test, it is not the case.

So unmarshalling gets faster across the board with encoding/json/v2, and marshalling gets faster for any, but it is about 1.5x slower for typed structs in my tests.

The code is available.

In Java, you find the location of a substring using indexOf.

String haystack = "The quick brown fox jumps over the lazy dog";
String needle = "fox";
int index = haystack.indexOf(needle);

Naively, you might implement indexOf by a loop inside a loop, like so.

int naiveIndexOf(String haystack, String needle) {
    for (int i = 0; i <= haystack.length() - needle.length(); i++) {
        int j = 0;
        for (; j < needle.length() 
          && haystack.charAt(i + j) == needle.charAt(j); j++) {}
        if (j == needle.length()) { return i; }
    }
    return -1;
}

The Java implementation is much more sophisticated, and it is highly accelerated.

However, there are pathological cases where the Java implementation can be slow. What do I mean? Well, you do expect that the search will be more and more expensive as the size of the string grows. Right? So if you search through a 1 kilobyte string and then search through a 10 kilobyte search, you would not be surprised if the latter takes ten times slower.

But what of the substring? If you search for short substrings (fox in my example), the everything is fine. But what if you search longer and longer substrings (fox jumps or fox jumps over)? If it gets more expensive when both the string and the substring get longer, then you have what we call a quadratic complexity. In other words, it is slow.

In Java, if n is the length of your string and m is the length of the substring, then the complexity of indexOf is O(n·m). And if you look at my naive implementation (naiveIndexOf) then you see that in the worst case, it might do up to close to haystack.length() * needle.length() comparisons, that is, it is O(n·m).

The exact implementation of the indexOf function depends on your CPU and Java version. I am using OpenJDK 25 on Apple Silicon (ARM). For my purposes, I will use as a haystack of n copies of a and for the needle, the same thing, but ending with a different letter.

// n > m
String haystack = "a".repeat(n);
String needle = "a".repeat(m - 1) + "b";

I measured OpenJDK 25 on an Apple M4 Max. The haystack is one megabyte. Numbers are nanoseconds per haystack character.

At m = 4096, a single indexOf over one megabyte takes 1.1 seconds.

Can you do better against such adversarial inputs? The textbook solution is the Two-Way algorithm of Crochemore and Perrin (1991). The implementation is simple and your favourite AI can code it for you in any programming language.

Two-Way stays at about 0.3 ns/character no matter how long the needle is. At m = 4096 it is about 3500 times faster than indexOf on the first-character adversary.

So, should you switch to Two-Way for everything? No. On random text, the indexOf function is much faster than Two-Way.

And Two-Way has to do non-trivial work before the search begins. So it has additional fixed overhead. It would lose most of the time in the real world, sometimes by a wide margin.

Should you worry about this? No. The indexOf function in Java is fine.

If an adversary can control the needle (substring), then make sure to reject long needles. Most of the time, we search for short sequences (say, less than 80 characters). If you are worried about your system crashing, you will put bounds on inputs in any case.

The Java source is available.

Further reading: Crochemore, M., & Perrin, D. (1991). Two-way string-matching. Journal of the ACM, 38(3), 650–674.

We are all familiar with IP addresses such as 192.168.0.1. They are typically written as four numbers in the range 0 to 255 inclusive, separated by dots. In C#, you can parse them with the standard library using IPAddress.TryParse.

Pedantic people are quick to point out that IP addresses can take different forms: they can be IPv6 or IPv4 and there are many weird ways to write an IPv4 address. But for the purpose of performance optimization, we care about the common case. The common case is strings such as 192.168.0.1 or 12.121.244.111.

Our processors are capable of data parallelism, meaning that they have instructions (called SIMD) that can process several bytes at once, at least 16 bytes, sometimes more. A few years ago, I showed that you can parse IPv4 addresses with SIMD. I have been revisiting this idea with AVX-512, the instruction set that recent x64 (AMD/Intel) processors support. I expect that all Intel and AMD processors made in the near future will have great support for AVX-512, and it is already the case for server processors and recent AMD processors.

So I wondered, could we do it in C#? People are sometimes surprised that I care about C#. Isn’t that more Microsoft slop? No. Not at all. C# and .NET are very reasonable, portable systems.

Plus you can write fast code in C#. I have two optimized libraries that I hope the Microsoft .NET team will one day adopt in the standard .NET library: an optimized Utf8Utility.GetPointerToFirstInvalidByte function used internally to validate Unicode strings (in the SimdUnicode library) and a fast base64 decoding library. I love working with .NET C#.

As of .NET 10, we have AVX-512 support, including masked loads. What are masked loads and why do they matter? Suppose that I give you a string that is no longer than 16 bytes, but could be shorter. If you load data in a SIMD register, you normally have to load the full register width (so 8, 16, 32, 64 bytes). So what do you do when it is not possible? You can pad the input string or pull other tricks, but it gets dirty. A nice approach is to have masked loads where you, say, load the full register (say 16 bytes), but you indicate which bytes you want to be loaded from memory with a mask. So if you use 0b10011 as a mask, then only the first, second, and fifth bytes are loaded from memory. This makes it possible to initialize a 16-byte register with a string that has between 0 and 16 bytes, while never reading beyond the string. I have an article entitled Modern vector programming with masked loads and stores if you want to know more.

To make things trickier, C#, like Java and JavaScript, defaults to UTF-16, meaning that each character, even if it is an ASCII character like A or 1, uses two bytes. The ASCII codepoint value occupies the least significant bits of a 16-bit word.

So what we need to do is to selectively load from a 32-byte input, and then drop the unnecessary zero bytes. The gist of it looks as follows in C#.

unsafe bool TryParseAvx512(ReadOnlySpan<char> s, out uint ip) {
        int len = s.Length;
        fixed (char* cp = s)
        {
            // next two lines are a trick to load just the first len characters
            Vector256<ushort> charMask = Vector256.LessThan(CharLaneIndex, Vector256.Create((ushort)len));
            Vector256<ushort> chars = Avx512BW.VL.MaskLoad((ushort*)cp, charMask, Vector256.Create((ushort)'0'));
            // check that everything is ASCII otherwise, it is not an IP!
            if (Avx512BW.VL.CompareGreaterThan(chars, Vector256.Create((ushort)0x7F)).ExtractMostSignificantBits() != 0)
            {
                return false;
            }
            // There we go, we have the address as ASCII
            // in a 16-byte register.
            Vector128<byte> str = Avx512BW.VL.ConvertToVector128Byte(chars);
            // ...
        }
}

This looks a bit difficult to read, but that’s fine. Most people never need to worry about such code.

Then we use a somewhat fancy trick where we locate the dots, and use the fact that there are only 81 ways to position the dots. We then move the bytes, do a dot product and validate. It is the same routine as the C++ code. It is not trivial, but I am working on a formal paper to document the tricks used.

The pedantic people will say: wait, there are other ways to write IP addresses !!! Ok fine. We handle them with a fallback, like so.

if (TryParseAvx512(s, out uint ip))
{
    address = new IPAddress(ip);
    return true;
}
return IPAddress.TryParse(s, out address);

What about the cases where your processor does not support AVX-512? C# makes this dead easy. You can just guard it with one if:

if (Avx512BW.VL.IsSupported) { ... }

To benchmark this, I generated 10,000 random 32-bit addresses and parsed the resulting strings 20 million times, constructing an IPAddress each time. On a relatively recent Intel processor (Intel Xeon Gold 6548N, Emerald Rapids) running .NET 10, I get the following.

So the AVX-512 approach is about three times faster than the standard library. My routine itself does not take fourteen nanoseconds; there is other overhead.

As usual, the C# source is available.

Like most programming languages, Go has both stack allocations, whose lifetime is limited to the current function, and dynamic (or heap) allocations.

The name stack comes from the fact that the memory management is somewhat trivial. There is typically one stack per thread (or goroutine in Go). When a function needs memory, it simply appends data to the stack. When the function returns, the memory is dropped from the end of the stack. So the memory last allocated is deallocated first.

Heap memory is potentially considerably more complex. For one thing, it is meant to be accessible by several threads (or goroutines). An object can be allocated by one function and later reclaimed after an entirely different function, possibly running on a different thread (or goroutine), has dropped the last reference to it. Unlike the stack, there is no prescribed order for allocating and reclaiming heap memory. In Go, the garbage collector does the reclaiming.

Typically, stack allocations have a size known at compile time. Many systems give each thread a fixed-size stack, although Go grows goroutine stacks as needed.

There are many ways in Go to do a heap allocation. A common one is when you allocate a slice, as in this instance where you allocate memory for 100 integers:

x := make([]int, 100)

If the slice x is not entirely local to a function, Go will typically just allocate it on the heap. It will do so similarly when a function returns a pointer. For example, in the following instance, I assign the value 1 to a local integer variable, but I return a pointer to it.

func f() *int {
  x := 1
  return &x
}

In C/C++, this would be quite bad. You should get a warning such as address of local variable 'x' returned. In Go, the variable x will typically get allocated on the heap.

In many Go programs, we end up doing a lot of heap allocations of small objects. It can become a bottleneck in some cases. Think about when you are maintaining a tree or a linked list where each value (node) is an object that must live on the heap. If the data structure is highly dynamic, you will be constantly allocating these small objects.

Memory allocation on the heap is usually not done at arbitrary sizes. You often cannot get exactly, say, 13 bytes. In Go, small allocations are rounded up to a size class: 8 bytes, 16 bytes, 24 bytes, 32 bytes, and so forth. There is also some overhead to each heap allocation, from rounding and from allocator metadata.

The compiler knows the size of the object, but prior to Go 1.27, Go would call a generic function when doing a heap allocation. This generic function would then look up the size class and take the corresponding path. Starting with 1.27, for small objects (under 80 bytes), Go relies on dedicated functions.

It is easy to benchmark in Go. A basic benchmark might look as follows.

type Node struct {
    value int64
    next  *Node
}
var sink any
func BenchmarkAllocNode16(b *testing.B) {
    for b.Loop() {
        sink = &Node{}
    }
}

On my MacBook, the results are quite telling. Go 1.27 is nearly twice as fast!

This will not help all software, just the components that do many small allocations.

The code is available.

AI-assisted programming is fast evolving and there is a tension between ‘we no longer need to understand the code’ and ‘what is my purpose as a programmer’. I recorded a short video on this topic with how I think the tension can result in conflicts.

When a compiler optimizes your program, it has to guess. Which functions are worth inlining? Which side of a branch is the common one? Which method does this interface call actually reach? At compile time it cannot know, so it uses heuristics. Profile-guided optimization (PGO) replaces the guessing with measurement: you run your program, record where it spends its time, and hand that recording back to the compiler for a second build.

PGO is a common feature of compiler systems. Google applied PGO to Chrome under Windows in 2016, reporting gains of up to 15%. I expect all mainstream Web browsers to be built with PGO.

There are now fancier techniques than mere heuristics with PGO. You can use AI to recognize patterns and so forth. But they are not always widely available.

Go has supported PGO since version 1.20. You collect a profile, and pass it to the compiler.

A CPU profile is a statistical record of where a program spends its time. While the program runs, the Go runtime interrupts it about a hundred times a second and writes down the call stack at that instant. After a few seconds you have thousands of such samples, and counting them tells you which functions were executing and who called them. In Go you produce one by wrapping the work you care about:

f, _ := os.Create("cpu.pprof")
pprof.StartCPUProfile(f)   // from runtime/pprof
defer pprof.StopCPUProfile()

The compiler reads the call-stack counts and uses them for two things above all: inlining call sites that turn out to be hot, and devirtualizing interface calls whose target is nearly always the same concrete type.

I took three JSON documents that I wanted to parse:

• twitter.json (632 kB), a nest of small objects with short string keys
• canada.json (2.25 MB), essentially one enormous array of floating-point coordinates
• citm_catalog.json (1.73 MB), deeply nested objects with numeric keys

I parse each of them with the standard library’s encoding/json into an interface{}. The baseline, with no profile, parses at 112 MB/s for twitter.json, 74 MB/s for canada.json and 116 MB/s for citm_catalog.json.

The procedure is three commands:

go build -o bench .                        # ordinary build
./bench -profile cpu.pprof -train twitter.json   # collect a CPU profile
go build -pgo=cpu.pprof -o bench_pgo .     # build again, with the profile

I did it three times, profiling each document on its own, and then measured all three documents against each of the three builds.

Each panel of the figure is one document being parsed, and the three bars inside it are the three PGO builds: the binary trained on twitter.json, the one trained on canada.json, and the one trained on citm_catalog.json. Bar height is the speed gain over the ordinary, profile-free build of that same document, in percent, so zero means PGO changed nothing and a bar below the axis means the PGO build was slower. The green bar in each panel is the matched case, where the profile was collected on the very document being measured.

The gains are modest. The best result is canada.json at +4.7%, and most differences are in the 2–3% range. Profiling one document usually helps the others, but not reliably. Profiling twitter.json gave a decent improvement everywhere: +3.1%, +2.0%, +2.8%. But profiling canada.json bought 4.7% on canada.json and essentially nothing anywhere else. Interestingly, profiling citm_catalog.json produced a mere +0.8% on its own document while helping twitter.json more.

A 3% speedup is not exciting in isolation, but it may come nearly for free. Observe how you may get slightly negative results for cases you did not train for. That’s expected generally, but the effect is modest in the case of Go because its optimizations are themselves modest in the first pace. That is, you are not getting a much an effect, but the process is less likely to backfire for other workloads.

The code is available.

C++26 adds a new container to the standard library: std::hive. It is meant to occupy the ground between std::vector and std::list. Like a vector, it keeps its elements in contiguous blocks of memory, so scanning it does not require you to chase a pointer for every element. Like a list, it never moves an element once it has been inserted: your pointers, references and iterators stay valid, and you may erase any element in constant time without disturbing the others.

Internally, a hive is a linked list of blocks. Each block carries a skipfield: a small integer per slot that tells the iterator how many erased slots to jump over.

No standard library ships std::hive yet to my knowledge. Fortunately there is an implementation (plf::hive by Matt Bentley) as a single header file that you can use today.

I use elements of type uint64_t, GCC 16.1 with -O3 -march=native, on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core. Numbers are nanoseconds per element, along with the cycles and instructions retired per element.

We start from an empty container and append a million values. The container is then destroyed.

A std::list needs one allocation per element, and glibc’s malloc and free together cost over 200 instructions per element. It is an order of magnitude behind everyone else. That is not news.

The interesting comparison is vector against hive. A hive is about twice the cost of a vector, and it needs twice the instructions. This is the price of the skipfield: every insertion writes an element and a skipfield entry, and maintains the block bookkeeping. Note that calling reserve on a hive does not help in my experiments.

Next we iterate over the the container and sum the values.

A hive iterates no faster than a linked list here, slightly slower, in fact, and about eight times slower than a vector. (Update: Joseph Garvin points out that I measure the happy case for the std::list in this instance where all the entries were allocated in sequence. The worst case scenario for std::list when the nodes are all over the heap can be much slower.)

The vector loop retires one instruction per element and finishes in 0.78 cycles: the processor is executing several elements at once. This is possible because the std::vector implementation benefits from autovectorization: the compiler recognizes that it can load several words at once in wide (SIMD). Further, it does not have to check the bitfield like the std::hive data structure.

We can check this. Walk the same container with two independent iterators, one starting halfway in, and count the cost per element visited:

The vector does not care: it was already throughput-bound. The hive and the list get nearly twice as fast per element, because two independent chains can be in flight at once. Hive iteration is latency-bound, exactly like list iteration. It merely has better locality.

That locality does show up when the data gets big. At ten million elements the list falls apart while the hive holds steady:

Erasing is what a hive is for, so it would be unfair not to look. I erase half the elements at scattered positions using std::remove_if:

The hive wins, but by less than you might expect, and at ten million elements the ordering reverses (1.3 ns for the vector against 2.5 for the hive). std::remove_if is a single streaming pass, and streaming passes are cheap. Of course the vector moved every surviving element and invalidated every pointer into it, which is precisely what a hive promises not to do.

Memory, measured by asking glibc how many bytes it has handed out, per live element:

A hive costs about a byte per element over a vector, for a payload of eight bytes, when the vector is packed tight. A list costs more due to the overhead of the linked list.

A vector built by push_back has a capacity that typically exceeds its size. Thus even if you have 8-byte entries, you will use, on average, more than 8 bytes per entry even for large vectors. You can recover the excess capacity with the shrink_to_fit method.

What should we conclude?

The std::hive data structure is not a faster vector. But it is a much better std::list. It gives you the same guarantees that make people reach for a list, stable references, cheap erasure anywhere, while using less memory.

My source code is available.

When your program asks for memory that is not in cache, the processor has to go to RAM. That trip costs on the order of 100 nanoseconds. On a 3 GHz core, that is about 300 cycles of doing nothing.

Memory latency has not improved in ten years. The 2016 Broadwell answers a random access in 100 ns. The 2025 Turin, with DDR5-6400 and every advantage of a decade of progress, takes 140 ns. It got worse.

The good news is that a modern core does not have to sit still. It can issue a second request before the first one comes back, and a third, and a tenth. The number of requests a single core can keep in flight is its memory-level parallelism. It is one of the most important numbers in software performance, and one of the least advertised: you will not find it on a spec sheet.

Thankfully, memory-level parallelism has improved a lot. To measure it, I use my testingmlp benchmark. The idea is a pointer chase. We build a 1 GiB array containing a single random cycle covering every element: each element holds the index of the next. Following the cycle is inherently serial. Each load has to complete before you know the address of the next one, so a single chase measures pure memory latency and nothing else. Then we run several such chases at once, from different starting points on the same cycle. We call these lanes. With two lanes, the core has two independent loads to work on. With twenty, twenty. We increase the number of lanes and watch the throughput. When adding a lane stops helping, we have found the limit. As my metric, I use the total estimated bandwidth.

I ran experiments on the Amazon cloud (AWS). The bandwidth shape is the same everywhere: a steep, nearly linear climb as we add lanes, then a knee, then a plateau. 

How did it evolve over time? Intel went from 10 to 30, meaning that a single Intel core can sustain 30 memory requests at once in practice. AMD went from 15 to 58. Graviton went from 6 to 19.

Intel was flat for a long time. Broadwell and Cascade Lake both sit at 10 concurrent misses. Ice Lake doubled it to 20. Granite Rapids is at 30. Intel has roughly tripled in a decade, with all the gain arriving in the last two generations.

AMD started ahead and stayed ahead, then jumped. Naples was already at 15 in 2018, when Intel was at 10. Milan reached 22. And then Turin does something different in kind: 58 concurrent cache lines from a single core.

Graviton 1 was a toy: 6 concurrent misses. Graviton 2 doubled it, Graviton 3 went to 17, and then Graviton 4 essentially stood still at 18. Graviton 5 only reaches 19. But look at the latency panel: since 2017, Graviton 5 is the only chip in this entire collection that made a random access faster than its predecessor. AWS advertised better DRAM latency for Graviton 5, and that claim holds up.

So who wins? On bandwidth and memory-level parallelism, it is AMD, and it is not close. The Zen 5 core in the m8a instances sustains 58 concurrent cache-line fetches and 24.5 GiB/s of random-access throughput from one core. AMD is roughly twice as fast as Intel.

The raw output, the system information from each machine, and the scripts are in the usual place.

Note that Apple Silicon does even better, but it is another category.

Every week, I discuss with people who want to get a PhD. For years, I have been advising people not to pursue a PhD. It may come as a surprise to some.

You would expect people with a PhD to earn more money. Individuals who complete doctorates tend to have higher cognitive abilities and greater motivation. But smarter people tend to earn more, period.

So do people with a PhD earn more?

Historically, PhD holders earn more, but the bulk of the observed advantage is concentrated among those who get a professorship after the PhD. And there is no certain path from the PhD to a professorship. We have been producing many more PhDs than we have professorship, for decades. And the disparity is ever growing.

When I entered university at the beginning of the 1990s, about 0.5% of the Canadian population had a PhD. This has nearly tripled today, and it is fast increasing. Something of the order of one person out of 80 has a PhD. Comparatively, there is roughly one professor or university-level instructor per 900 people. With a fast aging population, we simply do not need many more professors and instructors than we already have.

There are specific fields where some jobs are difficult to get without a PhD. Machine learning is one such example. Many people in the industry have a PhD, and they tend to select those who also have a PhD. Further, there is a somewhat direct relationship between the work you might do during your PhD, if you are any good, and the actual work you might do later. It is much less clear in a lot of other disciplines.

The most significant economic cost of a PhD is not tuition but the years of delayed full-time earnings and career progression. In the tech industry, it is typical to award half a year of experience for each year spent on a PhD. This means that even though the individual starting with a PhD might earn more starting out, they are not necessarily getting a higher lifetime income.

Benjamin et al. (2025) find that the early-career benefits a PhD can be effectively zero:

In the short run, pursuing a PhD entails substantial opportunity costs. Early-career earnings for PhD graduates are significantly lower than those of individuals with master’s or professional degrees, reflecting prolonged enrolment and delayed entry into the labour market. These costs are especially high for non-completers, particularly those who exit the program after several years without earning a credential. Over the lifecycle, earnings do eventually recover (and surpass those of bachelor’s and master’s graduates) but only under specific conditions. The most favourable long-run outcomes are concentrated among those who secure academic employment and remain in full-time work late into life. This “double premium,” combining higher earnings and longer careers, plays a central role in shaping the average return to a PhD. Outside academia, PhD holders resemble master’s graduates in both earnings and employment patterns.

Thus, the financial case for a PhD is narrower than people assume. If you fail to get a professorship, or you want an early retirement, you may very well end up with a poor outcome. And it is not getting better over time.

References

• Altonji, J. G., & Zhu, Z. (2025). Returns to specific graduate degrees: Estimates using Texas administrative records (NBER Working Paper No. 33530). National Bureau of Economic Research. https://www.nber.org/papers/w33530
• Benjamin, D., Miloucheva, B., & Vigezzi, N. (2025). The opportunity cost of a PhD: Spending your twenties (Working Paper No. 802). University of Toronto, Department of Economics. https://www.economics.utoronto.ca/public/workingPapers/tecipa-802.pdf
• Cooper, P. Is grad school worth it? A comprehensive return on investment analysis. Foundation for Research on Equal Opportunity. https://freopp.org/whitepapers/is-grad-school-worth-it-a-comprehensive-return-on-investment-analysis/

In 1945, Vannevar Bush published a report entitled Science: The Endless Frontier. His thesis was that prosperity follows from basic research. The report was highly influential in the United States and elsewhere. It led to the creation of an entirely new government bureaucracy.

With this report, Bush popularized the linear model of innovation: innovation (such as medical cures) flows sequentially from basic research to applied research to development to production and diffusion. Grow basic research, and the rest will follow.

When Bush wrote his report, basic research was not usually supported directly by the state. We did not have a large basic research infrastructure. And yet, the West had just lived through an unprecedented period of rapid scientific progress: the theory of evolution, electromagnetism, radio communication, special and general relativity, quantum mechanics, nuclear technology, rockets, the combustion engine, and more. We would get the invention of the transistor only two years after Bush’s report. We also did not have today’s peer-review mechanism.

Even though Bush’s report has been viewed as a piece of genius that unlocked a golden era of scientific prosperity, I believe that the linear model of innovation is hopelessly naïve. I believe the thesis that a large bureaucracy delivering funding to other bureaucracies (such as universities) is how we get innovation is absurd. Except perhaps in the domain of computing (“bits”), we have been largely stagnant technologically since about the 1970s. So Bush’s model failed over time. To be clear, it might have worked for a while by encouraging more young people to study engineering and science. It may also have shone a favorable light on a few enterprising professors who got to promote useful ideas.

If you visit a research lab today in a leading university, what you are most likely to see is a boring bureaucracy that caters to whatever is politically favorable at the moment—a bureaucracy that plays it safe and avoids controversy. You see young people seeking well-paid jobs, going through the motions with often little genuine interest in, say, curing cancer. We have never published so many research papers—the volume has been growing exponentially ever since Bush wrote his report—but it is doubtful that this is how technological breakthroughs are achieved.

The evidence is overwhelming that shoddy science is widespread. We have a severe reproducibility crisis: if you redo an experiment (even a highly cited one), you are likely to fail to reproduce the results. This affects psychology, medicine, and many other fields. The system does not particularly care because the incentives to get things right are not there. As long as the work is politically aligned, solidity of the results seems secondary.

There was a TV show (The Big Bang Theory) where the main character, Sheldon Cooper—an awkward genius—gets to work on crazy ideas. That is how Bush imagined it: fund young people like Sheldon Cooper, and you will get extraordinary breakthroughs. In the real world, Sheldon would not get very far on campus. I have met misfits like him. When they are incapable of playing the political game, the system crushes them. But even if that were not the case, extraordinary intelligence needs to be applied to the right problems to be of value. You could have a ChatGPT that is brighter than any of us in every possible way, and it could still be deployed simply to fill out forms faster and better than we do—it may not cure cancer.

The American government has just released what might be considered an update to Bush’s report. Michael Kratsios wrote a report entitled Science, A New Golden Age. The report states outright that the linear model no longer holds. It states what I have argued for vehemently: innovation is not a linear process. Take large-language models, for example, which can be used by engineers and scientists to further their research. I have also argued that the success of large-language models today has as much to do with the users as with the researchers.

At this point, some people engage in the following type of rhetoric: if we had not invented calculus, we would not have AI today; therefore, calculus caused AI. But you could also say that the subsidized nail factory in the Soviet Union, which made overpriced and bad nails, was necessary to hold Landau’s house together, and that without those nails we would not have the theory of Landau levels. The causality argument goes in all directions.

Innovation is the result of a complex system. We see that the United States and, more recently, China are innovative countries. In 2026, you do not go to France for the latest advances. The evidence is overwhelming that scientific and technological progress depends as much on culture as on anything else. It is not something to be managed by bureaucrats.

One of the cultural ingredients that seems essential is meritocracy. You must put the people who are good at building on top of your hierarchy. This does not happen magically. You need a set of incentives in which rewarding the wrong people is costly.

What does Kratsios propose? Many interesting ideas that, I expect, could renew our culture. He proposes to break out of the Cold War–era funding model. Today, the research funding mechanism is centered around the government giving money to the university bureaucracy. The grant might be in the name of one professor, but the recipient is still the university. In the new model, instead of funding universities, the government would assign money directly to individuals in various ways (short grants, prizes, and so forth). This would shift power away from administrators toward individuals who know how to get things done. It would also neutralize some of the political power of the current mandarin class of scientists who control access to the top positions.

The report recommends restoring permissionless innovation. It is sometimes poorly understood how limited the system has become. I once had a graduate student undertake interviews with practitioners. This required an ethics approval which, in her case, took a few months to obtain. Again, the system has built up political structures that seek to block innovation it does not like. They need to be torn down, the sooner the better.

The report has many other interesting recommendations. One that I particularly like is an AI-guided agenda. We need to hook up our brand-new AIs to experimental devices. We are not going to cure aging with chatbots. We need experiments on a massive scale.

Will Kratsios’s vision move from report to reality? History shows that cultural and institutional change is never easy. Yet the stakes could not be higher. By embracing meritocracy, permissionless innovation, and ambitious AI-augmented experimentation, we have a genuine chance to escape decades of stagnation and rekindle the spirit of discovery that once defined the West. The opportunity is before us. It must not be squandered.

When my kids were young, instead of helping them study algebra and grammar, I wrote small JavaScript apps for them to use. I built a small collection of educational tools.

The great success story of AI for me is exactly this: AI helps you write your own tools, faster and better.

Last night, I was struggling with videos I had to process. I wanted to add nice subtitles to them. There are software applications for that, but they require manual labor and don’t always work the way I want them to. After a long night, I had an insight: why don’t I ask my AI to help build the automated tool I need? So I did—and it worked really well, very quickly. So what’s the lesson here? Maybe that we should spend more time building our own software for our own personal use than we used to.

I have been on X for a long time. Like most people who post regularly, I have a gut feeling for what might interest people. I post in the morning. Longer posts seem to do better.

But gut feelings are not measurements. And until recently, digging into your own posting data meant either clicking around the web UI or writing custom scripts. Neither is particularly friendly when you want to ask ad hoc questions with an AI assistant.

X recently launched hosted MCP servers: official endpoints that AI tools can connect to. MCP is a protocol for plugging tools into language models: the model can search posts, manage bookmarks, fetch trends, and so on. In practice, I connected an AI coding agent to the X MCP server and simply started asking questions about my account.

I spent a session exploring about two months of my own activity. Here is what I found interesting.

Over roughly sixty days (mid-May through mid-July 2026), I published on the order of 435 posts that were not pure retweets of other people—mostly a mix of original posts, replies, and a few X Articles. The agent pulled them through the MCP tools, kept the public metrics (likes, views, reposts), and ran simple analyses.

I asked for every post to be binned by local hour of day (America/Toronto, Eastern time), and for each hour: how many posts, and the min / median / max view count.

My posting is heavily skewed toward the morning:

The 9 a.m. hour is both my busiest and, among busy hours, my strongest by median views. The overall median across all hours was only about 188 views, so most of what I write is quiet. The distribution is heavy-tailed: a few posts get tens or hundreds of thousands of impressions; the rest are background noise.

I then binned posts by character length in steps of 25 characters (using the text as returned by the API, including short t.co URLs).

The bulk of my writing is short, often a reply of a few dozen characters:

Under about 175 characters, the median stays at zero or one like. Around full-length posts (roughly the old 280-character regime and a bit beyond), engagement jumps. The 300–325 character band is where a large fraction of my “serious” posts live, and the median likes there are an order of magnitude higher than for short replies.

I also asked the AI to identify the posts that had the most likes, the following types of posts were liked:

• AI vs. “experts” claiming models are nowhere near human intelligence
• Go adding SIMD-style data-parallelism to the standard library
• SIMD-accelerated data processing talks and library notes (JSON, string→integer maps, vulnerability-report fatigue)
• Nvidia hardware, university AI-cheating, C++ contracts

The interesting part is the workflow. I did not export a CSV by hand and open Excel. I asked an agent, connected to X’s MCP server. If AI agents can do this for one account’s metrics, they can do it for bug trackers, logs, paper drafts, and codebases.

When I was a kid, most people did not know how to type. We took typing class. The final exam was a speed test: words per minute. Today, you will not impress anyone by saying you can type. In fact, cursive writing is fading. Kids increasingly cannot read or write it. We type constantly. We forget how many skills are learned, and how often some of these skills have faded.

But not everything fades. Socrates would be immensely popular today as a teacher. I still buy and recommend paper books.

Is reading and writing code more like Socrates, or more like cursive writing? There are clear signs that code could become like cursive writing. This year, I have met more than one student who could use AI to build an application but could not read or write code. It is not new. Software has long had non-technical people who describe what they built or designed. In fact, in much of the industry, the standard view was that once you had a university degree, you no longer coded. Coding was for monkeys or low-status employees. Top engineers paid a million dollars a year at Google or Meta know how to write code. They often read and write assembly and TypeScript. They know it all.

Why the discrepancy?

We pay an engineer a million dollars because he understands concepts few others grasp. He outruns others because he sees the problems more deeply. Reading and writing large amounts of code is part of how you gain those insights. Chatting with an AI will not make you a top 1% programmer. In the future, top engineers might read more code than anyone could in the past. These engineers will not be everywhere, but they will pack a punch. “But Daniel, people say programming is solved. Why read or write code?” Be careful with your models. When television arrived, some predicted it would replace the university lecturer. In some respects the model was correct, yet it did not happen. The lecturer’s job was never to deliver a TV show. The Google engineer paid a million dollars was never a machine that produces code. Nobody actually wants code, any more than they want raw text.

In fact, I predict a bifurcation in the tooling. The best engineers will work with tools that maximize their understanding of the code. I believe that reading and writing code, at a high level, is more like studying Socrates than like cursive writing. It is a necessary mental labor that does not become obsolete just because we have better tools for generating output.