<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Daniel Lemire&#039;s blog</title>
	<atom:link href="https://lemire.me/blog/feed/" rel="self" type="application/rss+xml" />
	<link>https://lemire.me/blog</link>
	<description>Daniel Lemire is a software performance expert. He ranks among the top 2% of scientists globally (Stanford/Elsevier 2025) and is one of GitHub&#039;s top 1000 most followed developers.</description>
	<lastBuildDate>Tue, 17 Nov 2026 22:35:35 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.2</generator>

<image>
	<url>https://lemire.me/blog/wp-content/uploads/2026/09/cropped-IMG_9858-2-32x32.jpeg</url>
	<title>Daniel Lemire&#039;s blog</title>
	<link>https://lemire.me/blog</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Cognitive radius</title>
		<link>https://lemire.me/blog/2026/11/16/cognitive-radius/</link>
					<comments>https://lemire.me/blog/2026/11/16/cognitive-radius/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 16 Nov 2026 16:51:07 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23152</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/cognitive-radius-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" />The most interesting change AI brings to software work is not faster coding. It is the expansion of context or range&#8230; what I call radius. For decades the job was defined by the limits of what one person could hold in their head. You tended to specialize. You got good at one or two systems, &#8230; <a href="https://lemire.me/blog/2026/11/16/cognitive-radius/" class="more-link">Continue reading <span class="screen-reader-text">Cognitive radius</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/cognitive-radius-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" /><p>The most interesting change AI brings to software work is not faster coding. It is the expansion of context or range&#8230; what I call radius.</p>
<p>For decades the job was defined by the limits of what one person could hold in their head. You tended to specialize. You got good at one or two systems, one or two types of problems because keeping more than that coherent was expensive. Context switching hurt. Onboarding to a new project took days or weeks.</p>
<p>Simply issuing a pull request to a new open-source repository took days. You had to learn the conventions. Learn the code base. Find out how to test it. Find out how to run benchmarks.</p>
<p>With a good model and the right tools, you can load a large amount of relevant information into the session: the structure of several repositories, the conventions of different teams, recent issues, design decisions, performance constraints, the history of related libraries. You can ask questions across them, generate changes that stay consistent with local style, and keep more projects alive at once without the usual collapse in quality.</p>
<p>The unit of work stops being &#8220;the project I am currently deep in&#8221; and becomes something closer to &#8220;the set of projects I can keep coherent right now.&#8221;</p>
<p>This is not the same as having an assistant write code for you. It is more. The gain comes from the ability to hold and operate on a larger working set. You still decide what matters. You still check the results. What changes is how much of the surrounding system you can bring into focus at the same time.</p>
<p>The practical consequences are already visible for people who treat this as a skill rather than a novelty:</p>
<ul>
<li>You can enter a codebase you have not touched in months and become useful again in an hour instead of a day, because the model can reconstruct the local idioms and the reasons behind them from the history and tests.</li>
<li>You can maintain consistency across languages or frameworks. The cost of switching is lower when the model carries the surface details.</li>
</ul>
<p>None of this is automatic. The model does not magically expand your capacity. You have to learn how to feed it the right context, how to structure the questions so the answers stay grounded, and how to verify the parts that matter. The people who get the most out of it treat the model as a high-bandwidth interface to a larger set of information.</p>
<p>Most discussion of AI in programming still focuses on the speed of generating individual functions or on the fear that juniors will not learn. Those are real topics, but they miss the larger shift. The constraint that shaped the profession for a long time, the limited amount of context a single developer could keep active, has loosened. What you can usefully own has grown. Learning to operate at that new scale is the part that is still under-discussed and under-practiced.</p>
<p>The cognitive radius of an engineer is the distance (in projects, technologies, or time) over which they can still produce consistent, well-informed work. AI has increased that radius.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/16/cognitive-radius/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Reading JSON at 277 GB/s in Python</title>
		<link>https://lemire.me/blog/2026/11/14/reading-json-at-277-gb-s-in-python/</link>
					<comments>https://lemire.me/blog/2026/11/14/reading-json-at-277-gb-s-in-python/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 14 Nov 2026 14:53:50 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23149</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" />Many systems log their data as JSON, one record per line (NDJSON). GitHub, for example, publishes every public event on its site that way through GH Archive: about a gigabyte per hour. Suppose you want to know which repositories received the most stars over eight hours. That is 7.6 GB of JSON, 2.2 million events. &#8230; <a href="https://lemire.me/blog/2026/11/14/reading-json-at-277-gb-s-in-python/" class="more-link">Continue reading <span class="screen-reader-text">Reading JSON at 277 GB/s in Python</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Many systems log their data as JSON, one record per line (NDJSON). GitHub, for example, publishes every public event on its site that way through <a href="https://www.gharchive.org">GH Archive</a>: about a gigabyte per hour. Suppose you want to know which repositories received the most stars over eight hours. That is 7.6 GB of JSON, 2.2 million events.</p>
<p>In Python, the obvious code reads each line with the <code>json</code> module:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">line</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #204a87;">open</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">"events.ndjson"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #4e9a06;">"rb"</span><span style="color: #000; font-weight: bold;">):</span>
    <span style="color: #000;">event</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">json</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">loads</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">line</span><span style="color: #000; font-weight: bold;">)</span>
    <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: #000;">event</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"type"</span><span style="color: #000; font-weight: bold;">]</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: #4e9a06;">"WatchEvent"</span><span style="color: #000; font-weight: bold;">:</span>
        <span style="color: #000;">stars</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">event</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"repo"</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #4e9a06;">"name"</span><span style="color: #000; font-weight: bold;">]]</span> <span style="color: #ce5c00; font-weight: bold;">+=</span> <span style="color: #0000cf; font-weight: bold;">1</span>
</code></pre>
</div>
<p>It runs at 0.28 GB/s: 27 seconds for this file. The orjson package is twice as fast. The msgspec package, which decodes only the fields you declare, is eight times as fast.</p>
<p>A server has many cores, but a Python process with the GIL runs Python code on one core at a time. The usual answer is a pool of processes: cut the file in pieces, give a piece to each process, and add up their counts. Each process returns a small table, so little has to come back.</p>
<p>Python 3.14 has a free-threaded build (<code>python3.14t</code>) without the GIL: threads run Python code at the same time. The <a href="https://github.com/simdjson/fastpysimdjson">fastsimdjson</a> library is threaded. You give it the data and a function. It parses the records on many threads, and calls your function on the thread that parsed the record. You name the fields you want. It reads only what leads to them, and your function receives those values, not the whole record.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">collections</span>
<span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fastsimdjson</span>
<span style="color: #204a87; font-weight: bold;">def</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">starred</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">event</span><span style="color: #000; font-weight: bold;">):</span>
    <span style="color: #000;">kind</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">repo</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">event</span>
    <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: #000;">kind</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: #4e9a06;">"WatchEvent"</span><span style="color: #000; font-weight: bold;">:</span>
        <span style="color: #204a87; font-weight: bold;">return</span> <span style="color: #000;">repo</span>
<span style="color: #000;">names</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">fastsimdjson</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">map_many</span><span style="color: #000; font-weight: bold;">(</span>
    <span style="color: #000;">starred</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">data</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">threads</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #0000cf; font-weight: bold;">192</span><span style="color: #000; font-weight: bold;">,</span>
    <span style="color: #000;">fields</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"/type"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #4e9a06;">"/repo/name"</span><span style="color: #000; font-weight: bold;">],</span> <span style="color: #000;">drop_none</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #204a87; font-weight: bold;">True</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000;">stars</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">Counter</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">names</span><span style="color: #000; font-weight: bold;">)</span>
</code></pre>
</div>
<p>The variable <code>data</code> is the file, mapped in memory. The two paths are the event type and the repository name. The <code>drop_none</code> parameter leaves out the records that are not stars.</p>
<p>I ran this on an AWS instance (<code>c8a.metal-48xl</code>) with two AMD EPYC 9R45 processors, 192 cores, using Python 3.14.10, orjson 3.13 and msgspec 0.22. The file is mapped before the clock starts. On one core, fastsimdjson reads 8 GB/s, three and a half times as fast as msgspec.</p>
<p><img decoding="async" alt="Counting stars on one core" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-stars-one.webp" class="alignnone size-large" /></p>
<p>With 192 threads, fastsimdjson counts the stars in the 7.6 GB in 55 milliseconds: 138 GB/s. The best pool of msgspec processes reaches 116 GB/s, at 96 processes. More processes make it slower.</p>
<p><img decoding="async" alt="Counting stars on all cores" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-stars-all.webp" class="alignnone size-large" /></p>
<p>fastsimdjson is ahead at every worker count. The lead shrinks from three and a half times on one core to about 20%.</p>
<p><img decoding="async" alt="By number of workers" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-stars-workers.webp" class="alignnone size-large" /></p>
<p>Counting is the best case for processes, because little comes back. Often you want the values themselves. I took six files of about a gigabyte from an existing benchmark (Pison), and pulled a field or two from every record. On the twitter file, fastsimdjson reaches 277 GB/s. Four of the six files pass 100 GB/s. The other two are slower: one stays near 20 GB/s, still about four times as fast as msgspec, and one is a tie.</p>
<p><img decoding="async" alt="Six queries" src="https://lemire.me/blog/wp-content/uploads/2026/11/superfastjson-queries.webp" class="alignnone size-large" /></p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/11/superfastjson">My source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/14/reading-json-at-277-gb-s-in-python/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Telling your number parser what to expect</title>
		<link>https://lemire.me/blog/2026/11/10/telling-your-number-parser-what-to-expect/</link>
					<comments>https://lemire.me/blog/2026/11/10/telling-your-number-parser-what-to-expect/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Tue, 10 Nov 2026 12:44:09 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23140</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/tunefastfloat-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Our fast_float library parses numbers such as -731.51 into binary floating-point values. It is part of GCC&#8217;s C++ standard library, among others. It is fast: often a few nanoseconds per number. Parsing a number means making decisions. Is there a sign? How many digits are there? Where is the decimal point? Is the processor rounding &#8230; <a href="https://lemire.me/blog/2026/11/10/telling-your-number-parser-what-to-expect/" class="more-link">Continue reading <span class="screen-reader-text">Telling your number parser what to expect</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/tunefastfloat-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Our <a href="https://github.com/fastfloat/fast_float">fast_float</a> library parses numbers such as <code>-731.51</code> into binary floating-point values. It is part of GCC&#8217;s C++ standard library, among others. It is fast: often a few nanoseconds per number.</p>
<p>Parsing a number means making decisions. Is there a sign? How many digits are there? Where is the decimal point? Is the processor rounding in the default mode? Each decision is a branch in the code. Modern processors do not wait to know the outcome of a branch: they guess, and they keep working. When they guess right, the branch is nearly free. When they guess wrong, they throw away the work done since the branch, and that costs 15 to 20 cycles.</p>
<p>When all your numbers look alike, the processor guesses right. When they do not, it guesses wrong often. Consider a column of gains and losses: the sign is random. The processor cannot guess it, and it pays for its mistakes on about half of the numbers.</p>
<p>You often know things about your data that the parser cannot know. So we are adding <em>performance hints</em> to fast_float. You pass them as a template argument:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">using</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fast_float</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">performance_hints</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">double</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">value</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">answer</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fast_float</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">from_chars</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #000;">performance_hints</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">unpredictable_sign</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">(</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">data</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">data</span><span style="color: #000; font-weight: bold;">()</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">size</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">value</span><span style="color: #000; font-weight: bold;">);</span>
</code></pre>
</div>
<p>There are three hints, and you may combine them with <code>|</code>:</p>
<ul>
<li><code>unpredictable_sign</code>: the signs come in random order. The parser then handles the sign without a branch.</li>
<li><code>short_numbers</code>: each number is given alone (the range holds just the number) and most numbers have at most 8 characters, such as <code>42.5</code> or <code>-3.75</code>, in varying shapes. The parser then reads all 8 characters at once with a few bitwise operations, without a branch on the number of digits or the position of the decimal point.</li>
<li><code>assume_round_to_nearest</code>: your program never changes the rounding mode, so the parser does not check it.</li>
</ul>
<p>The hints never change the results, with one exception. If you promise <code>assume_round_to_nearest</code> and some code in your program changes the rounding mode, some results will be wrong.</p>
<p>The hints work with <code>fast_float::from_chars</code>, <code>fast_float::from_chars_advanced</code> and <code>fast_float::integer_times_pow10</code>. Without hints, nothing changes.</p>
<p>How much do they help? I generated four kinds of data, 100,000 numbers each:</p>
<ul>
<li>price changes, two decimals and random signs, such as <code>-731.51</code> or <code>42.07</code>;</li>
<li>account balances, two decimals and random signs, but longer, such as <code>-731271.51</code>;</li>
<li>sensor readings, all positive, but in varying shapes, such as <code>7</code>, <code>1042</code>, <code>42.5</code> and <code>0.375</code>;</li>
<li>regular prices, always the same shape, such as <code>12.34</code>.</li>
</ul>
<p>I parse them on an Intel Xeon Gold 6338 (Ice Lake) with GCC 15, and on an Apple M4 Max with Apple clang 17. I report millions of numbers parsed per second: higher is better.</p>
<p><img decoding="async" alt="Millions of numbers per second, Intel Ice Lake, GCC 15" src="https://lemire.me/blog/wp-content/uploads/2026/11/tunefastfloat-icelake-scaled.webp" class="alignnone size-large" /></p>
<p><img decoding="async" alt="Millions of numbers per second, Apple M4 Max, Apple clang 17" src="https://lemire.me/blog/wp-content/uploads/2026/11/tunefastfloat-m4-scaled.webp" class="alignnone size-large" /></p>
<p>Each kind of data has its own best combination.</p>
<p>Price changes have random signs and short numbers. On the Intel processor, adding <code>unpredictable_sign</code> and <code>assume_round_to_nearest</code> goes from 73 to 105 million numbers per second. All three hints do as well (104 million), and they are best on the Apple processor, going from 153 to 345 million numbers per second: more than twice as fast. Without hints, the Intel processor mispredicts a branch on 60% of the numbers. With all three hints, the mispredictions disappear.</p>
<p>Account balances also have random signs, but the numbers are too long for <code>short_numbers</code>. Here <code>unpredictable_sign</code> with <code>assume_round_to_nearest</code> is the right choice: from 61 to 82 million numbers per second on the Intel processor, and from 132 to 199 million on the Apple processor. Adding <code>short_numbers</code> makes things worse on the Intel processor (70 million): the hint does not apply to these numbers, yet we pay for it.</p>
<p>Sensor readings have no sign, but their shapes vary. Here <code>short_numbers</code> is the key: from 67 to 92 million numbers per second on the Intel processor, and from 123 to 250 million on the Apple processor. The hint <code>unpredictable_sign</code> is harmful: there is no sign to predict, and the branch-free code is more work. On the Intel processor, it goes down to 57 million.</p>
<p>Regular prices are the easy case: all positive, always the same shape. The processor guesses everything right. On the Intel processor, the best choice is no hint at all (152 million numbers per second), and <code>short_numbers</code> makes things much slower (104 million). The Apple processor is different: <code>short_numbers</code> still helps, from 251 to 355 million numbers per second.</p>
<p>With Clang 21 on the Intel processor, the results are similar: the same hints win on the same data.</p>
<p>So the hints are not free. They trade branches, which are cheap when the processor guesses right, for more instructions. They pay off when the processor would guess wrong.</p>
<p>The hints are currently in a <a href="https://github.com/fastfloat/fast_float/pull/434">pull request</a>.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/11/tunefastfloat">My source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/10/telling-your-number-parser-what-to-expect/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title>A performance trap: when store forwarding fails</title>
		<link>https://lemire.me/blog/2026/11/05/a-performance-trap-when-store-forwarding-fails/</link>
					<comments>https://lemire.me/blog/2026/11/05/a-performance-trap-when-store-forwarding-fails/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 05 Nov 2026 13:06:20 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23118</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/storeforwarding-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="A store sits in the store buffer. A later load can take those bytes from the buffer instead of waiting for the data cache." style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />A store does not go straight into the cache. Waiting for that write would stall the processor. It sits in a store buffer, and a later load can take its bytes from that buffer. That is store forwarding. Forwarding works when the load is the same size as the store, or smaller and completely inside &#8230; <a href="https://lemire.me/blog/2026/11/05/a-performance-trap-when-store-forwarding-fails/" class="more-link">Continue reading <span class="screen-reader-text">A performance trap: when store forwarding fails</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/storeforwarding-feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="A store sits in the store buffer. A later load can take those bytes from the buffer instead of waiting for the data cache." style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>A store does not go straight into the cache. Waiting for that write would stall the processor. It sits in a store buffer, and a later load can take its bytes from that buffer. That is store forwarding.</p>
<p>Forwarding works when the load is the same size as the store, or smaller and completely inside it. It fails when the load is wider than the store, or when the two ranges only partly overlap. The load then waits until the store has reached the cache.</p>
<p>Here is an example in C. We have a buffer, that might have received data previously. Yet one byte is overwritten, and we try to load 8 bytes. It will typically cause a store forwarding failure.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">reload_byte</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">unsigned</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">v</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #8f5902; font-style: italic;">// Copy one byte from v into the buffer.</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">memcpy</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">v</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #8f5902; font-style: italic;">// Copy eight bytes from the buffer into w.</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">memcpy</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">w</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">8</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>A 4-byte store followed by an 8-byte load would cause the same issue. So would an 8-byte store followed by an 8-byte load that starts four bytes later: the ranges overlap.</p>
<p>You can sidestep the issue by working in a register, like the following example.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">update_byte</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">unsigned</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">v</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #8f5902; font-style: italic;">// load from p to x</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">memcpy</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">8</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">x</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">x</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">~</span><span style="color: #0000cf; font-weight: bold;">0xffull</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">|</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">v</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #8f5902; font-style: italic;">// store the modified result to p</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">memcpy</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">8</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>Further, if you store eight bytes and read those same eight back, a compiler might delete the reload.</p>
<p>I ran some experiments to see how many cycles various store/load operations take.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">What the iteration does</th>
<th style="text-align: right;">Xeon Gold 6548N</th>
<th style="text-align: right;">Xeon 6975P-C</th>
<th style="text-align: right;">EPYC 9R45</th>
<th style="text-align: right;">Graviton 5</th>
<th style="text-align: right;">Apple M4 Max</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">store 8, load 8, same address</td>
<td style="text-align: right;">1</td>
<td style="text-align: right;">1</td>
<td style="text-align: right;">1</td>
<td style="text-align: right;">5</td>
<td style="text-align: right;">5</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">store 8, load 4 at +4</td>
<td style="text-align: right;">5</td>
<td style="text-align: right;">5</td>
<td style="text-align: right;">7</td>
<td style="text-align: right;">6</td>
<td style="text-align: right;">5</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">store 4, load 8</td>
<td style="text-align: right;">19</td>
<td style="text-align: right;">19</td>
<td style="text-align: right;">14</td>
<td style="text-align: right;">5</td>
<td style="text-align: right;">5</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">store 4, then store 4, load 8</td>
<td style="text-align: right;">20</td>
<td style="text-align: right;">20</td>
<td style="text-align: right;">15</td>
<td style="text-align: right;">5</td>
<td style="text-align: right;">8</td>
</tr>
</tbody>
</table>
<p><img decoding="async" alt="Cycles for a dependent store followed by a load" src="https://lemire.me/blog/wp-content/uploads/2026/11/storeforwarding-latency-3.webp" class="alignnone size-large" /></p>
<p>On the Xeons and the EPYC, a load that matches the store takes one cycle. A wider load takes 19 cycles on the Xeons and 14 on the EPYC.</p>
<p>On the M4 and Graviton 5, a wider load takes 5 cycles. Two stores bring the M4 to 8.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/11/storeforwarding">My source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/05/a-performance-trap-when-store-forwarding-fails/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title>Open source as a research method</title>
		<link>https://lemire.me/blog/2026/11/04/open-source-as-a-research-method/</link>
					<comments>https://lemire.me/blog/2026/11/04/open-source-as-a-research-method/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 04 Nov 2026 12:48:24 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23113</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/seriq-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Many people still picture innovation as a line. The professor thinks and publishes. The engineer reads the paper and designs a product. The customer consumes it. It rarely works that way. The industrial revolution was not set off by professors writing papers. It was driven by demand for cheaper and better clothing for the masses. &#8230; <a href="https://lemire.me/blog/2026/11/04/open-source-as-a-research-method/" class="more-link">Continue reading <span class="screen-reader-text">Open source as a research method</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/seriq-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Many people still picture innovation as a line. The professor thinks and publishes. The engineer reads the paper and designs a product. The customer consumes it.</p>
<p>It rarely works that way. The industrial revolution was not set off by professors writing papers. It was driven by demand for cheaper and better clothing for the masses. Demand came first.</p>
<p>Michael Stonebraker made a related point in &#8220;Where We Have Failed&#8221; (2018). He argued that the database research community has lost its customer: &#8220;there are no &#8216;real world&#8217; clients to keep us focused.&#8221; Without a customer, research drifts.</p>
<p>In 2019, JSON was everywhere, and parsing it was slow, at a few hundred megabytes per second. So we built <a href="https://github.com/simdjson/simdjson">simdjson</a>. Today it parses JSON at gigabytes per second; I measured about 4.8 GB/s on twitter.json on a recent Intel Xeon.</p>
<p>We did not write the paper first. We wrote the code, then a blog post. We put it on GitHub and posted about it on social networks. Industry adopted it. Our first paper was published by the VLDB Journal, but it was more of an advertisement for our ideas and designs than the endpoint of the research.</p>
<p>Fred Brooks wrote that &#8220;the scientist builds in order to study; the engineer studies in order to build.&#8221; In my work, building is how I study.</p>
<p>Once simdjson parsed JSON quickly, a new bottleneck appeared: floating-point numbers. If everything else is fast, converting strings like <code>3.1416</code> into binary floats dominates the running time. That led to what is now called the Eisel-Lemire algorithm, which is exact and fast (<a href="https://arxiv.org/abs/2101.11408">Number Parsing at a Gigabyte per Second</a>). It now sits inside the standard libraries of Rust, Go, .NET and C++ (through <a href="https://github.com/fastfloat/fast_float">fast_float</a>).</p>
<p>Then came the next bottleneck. Parsing JSON also means validating UTF-8, and security depends on getting that right. We got validation down to less than one instruction per byte. That code broke off into its own library, <a href="https://github.com/simdutf/simdutf">simdutf</a>, which is now used by Node.js, Bun, Deno and web browsers for Unicode validation and transcoding.</p>
<p>I did not plan any of this from a mountaintop. Stonebraker said it well: &#8220;my ideas come from two sources: talking to real users with real problems and then trying to solve them.&#8221; Open source gives me those users. They show up with bug reports, adversarial inputs, exotic hardware and workloads I would never have imagined. Pull requests and code reviews are often a tougher form of peer review than most program committees. Reviewers who depend on your code read it more carefully than referees who skim your paper.</p>
<p>David Patterson, in &#8220;How to Build a Bad Research Center&#8221; (2014), wrote that for his successful projects, &#8220;the early indicators were not papers or awards, but non-researchers trying to use our ideas.&#8221; I agree.</p>
<p>Is open source still as important now that we have AI? If a model can write the code for you, why bother publishing yours?</p>
<p>What I see points the other way. More people than ever contribute to my projects, and more people than ever use my code and my ideas. AI tools seem to make it easier, not harder, for people to read an unfamiliar codebase, find a problem and send a fix.</p>
<h2>When I delivered this message&#8230;</h2>
<p>Last week (October 29, 2026), I was one of four invited speakers at the launch of <a href="https://seriq.ca/">SERIQ</a>, the new <em>Centre interuniversitaire de recherche en génie logiciel pour la société numérique</em>. SERIQ continues the SEMTL (Software Engineering at Montreal) meetings that have brought together Montreal software engineering researchers for more than a decade. The new centre is backed by the Université de Montréal, Polytechnique Montréal, McGill University and the École de technologie supérieure, and it aims to cover all of Quebec. About a hundred people came to the Pavillon Jean-Coutu at the Université de Montréal for the <a href="https://seriq.ca/en/events/2026-10-29-rentree-seriq/">event</a>.</p>
<p>The other three guests were:</p>
<ul>
<li><a href="https://www.lbriand.info/">Lionel Briand</a> (University of Ottawa and Lero, the Research Ireland centre for software at the University of Limerick), who spoke about how AI disrupts software testing strategies, and how it can also help fix them.</li>
<li><a href="https://discover.research.utoronto.ca/24922-marsha-chechik">Marsha Chechik</a> (University of Toronto), who spoke about designing the digital society, from human intent to reliable systems.</li>
<li>Yves Daoust (Ferme d&#8217;Hiver), an engineer who worked in telecommunications before founding a farming company. He looked back on forty years of software, from punched cards to AI agents, and asked why this moment is different.</li>
</ul>
<p>Each of us had fifteen minutes. Afterwards, the four of us sat down for a panel on software research for society, moderated by Martin Robillard (McGill).</p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/11/seriq-panel.jpg" alt="The panel at the SERIQ launch" class="alignnone size-large" /></p>
<p>My <a href="http://lemire.github.io/talks/2026/seriq/seriq_en.html">slides are online</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/04/open-source-as-a-research-method/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>In C++, = default and {} are not the same constructor</title>
		<link>https://lemire.me/blog/2026/11/02/default-and-braces-are-not-the-same-constructor/</link>
					<comments>https://lemire.me/blog/2026/11/02/default-and-braces-are-not-the-same-constructor/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 02 Nov 2026 12:48:59 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23107</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/cppinit-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />You can write an empty default constructor as span() = default or as span() {}. It can have remarkable effects on performance. struct span { const char *p; int len; const char *end; span() = default; // or span() {} span(const char *s, int n) : p(s), len(n), end(s + n) {} }; The constructor &#8230; <a href="https://lemire.me/blog/2026/11/02/default-and-braces-are-not-the-same-constructor/" class="more-link">Continue reading <span class="screen-reader-text">In C++, = default and {} are not the same constructor</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/11/cppinit-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>You can write an empty default constructor as <code>span() = default</code> or as <code>span() {}</code>. It can have remarkable effects on performance.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">struct</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">span</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">len</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">end</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">span</span><span style="color: #000; font-weight: bold;">()</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">default</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">// or span() {}</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">span</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">len</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">end</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">s</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{}</span>
<span style="color: #000; font-weight: bold;">};</span>
</code></pre>
</div>
<p>The constructor <code>span() = default</code> on its first declaration makes <code>span</code> trivial. The constructor <code>span() {}</code> is user-provided, so <code>span</code> is not trivial.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">Constructor</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">Declaration</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">What happens</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">Members</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span() = default</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span s;</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">nothing</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">uninitialized</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span() = default</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span s{}</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">the whole object is zeroed, padding included; the constructor does not run</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">zero</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span() {}</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span s;</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">the empty constructor is called; the compiler can delete the call</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">uninitialized</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span() {}</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>span s{}</code></td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">the empty constructor is called; the object is not zeroed</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">uninitialized</td>
</tr>
</tbody>
</table>
<h2>An example</h2>
<p>This <code>span</code> is a start pointer, a length, and an end pointer. A <code>box</code> holds one span. The expression <code>box()</code> value-initializes that span. The loop assigns a value-initialized box when <code>i</code> reaches <code>n</code>, and otherwise a span of length <code>i + 1</code>:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">struct</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">box</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">span</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">box</span><span style="color: #000; font-weight: bold;">()</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">()</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{}</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">box</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">span</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">in</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">in</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{}</span>
<span style="color: #000; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">consume</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">box</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">walk</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">box</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">b</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">?</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">box</span><span style="color: #000; font-weight: bold;">()</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">box</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">span</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">consume</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">b</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>The <code>box()</code> path runs once, as the loop exits, and that span is not passed to <code>consume</code>.</p>
<p>With <code>span() {}</code>, GCC 16 (<code>-O2 -march=x86-64</code>) writes the three fields directly where <code>consume</code> expects them:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f57900;">.L4:</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ebx</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">QWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">16</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rdi</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rsp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sub</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ebp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">QWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">DWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">8</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">call</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">consume</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">box</span><span style="color: #a40000; border: 1px solid #EF2929;">&amp;</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">cmp</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">r12</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">jne</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">.L4</span>
</code></pre>
</div>
<p>With <code>span() = default</code>, GCC 16 stores the end pointer, <code>rbx</code>, straight into the box. The start pointer and the length go through a temporary, and the first 16 bytes of that temporary are copied back.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f57900;">.L3:</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ebx</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">QWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">32</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rdi</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rsp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sub</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ebp</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">QWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">16</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">DWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">40</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">movdqa</span><span style="color: #f8f8f8;">  </span><span style="color: #000;">xmm0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">XMMWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #a40000; border: 1px solid #EF2929;">+</span><span style="color: #0000cf; font-weight: bold;">32</span><span style="color: #000; font-weight: bold;">]</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">movaps</span><span style="color: #f8f8f8;">  </span><span style="color: #000;">XMMWORD</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PTR</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">rsp</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">xmm0</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">call</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">consume</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">box</span><span style="color: #a40000; border: 1px solid #EF2929;">&amp;</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">cmp</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rbx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">r12</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">jne</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">.L3</span>
</code></pre>
</div>
<p>I time the best of five runs of 100 million spans, on one core of an Intel Xeon Gold 6548N (Emerald Rapids), with <code>-O2 -march=x86-64</code>. <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/cppinit">My code is available</a>.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">compiler</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;"><code>span() = default</code></th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;"><code>span() {}</code></th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">GCC 14.3</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.73 ns/span</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.74 ns/span</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">GCC 15.2</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">7.75 ns/span</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.73 ns/span</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">GCC 16.2</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">8.04 ns/span</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.73 ns/span</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">clang 21</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.75 ns/span</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.74 ns/span</td>
</tr>
</tbody>
</table>
<p><img decoding="async" class="alignnone size-large" alt="GCC 15 and GCC 16 take about 8 ns per span with span() = default. The braces constructor stays near 1.7 ns." src="https://lemire.me/blog/wp-content/uploads/2026/11/cppinit-times.webp" /></p>
<p>With GCC 15 and GCC 16, the <code>= default</code> version is over four times slower.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/11/02/default-and-braces-are-not-the-same-constructor/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Beating the compiler at signed 32-bit division</title>
		<link>https://lemire.me/blog/2026/10/31/beating-the-compiler-at-signed-32-bit-division/</link>
					<comments>https://lemire.me/blog/2026/10/31/beating-the-compiler-at-signed-32-bit-division/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 31 Oct 2026 13:05:35 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23088</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Dividing integers is slow on our processors. A 32-bit division costs: processor latency throughput Emerald Rapids about 12 cycles one every 6 cycles Zen 4 9 to 12 cycles one every 6 cycles Zen 5 10 to 13 cycles one every 6 cycles Arrow Lake performance core 12 cycles one every 6 cycles Apple M4 &#8230; <a href="https://lemire.me/blog/2026/10/31/beating-the-compiler-at-signed-32-bit-division/" class="more-link">Continue reading <span class="screen-reader-text">Beating the compiler at signed 32-bit division</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/feature-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Dividing integers is slow on our processors. A 32-bit division costs:</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">processor</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">latency</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">throughput</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Emerald Rapids</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">about 12 cycles</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">one every 6 cycles</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Zen 4</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">9 to 12 cycles</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">one every 6 cycles</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Zen 5</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">10 to 13 cycles</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">one every 6 cycles</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Arrow Lake performance core</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">12 cycles</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">one every 6 cycles</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Apple M4 performance core</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">7 cycles</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">one every 2 cycles</td>
</tr>
</tbody>
</table>
<p>On Zen 4 and Zen 5, the latency depends on the inputs. The x86 rows are <code>idiv</code> (<a href="https://uops.info/html-instr/IDIV_R32.html">uops.info</a>). The Apple row is <code>sdiv</code> (<a href="https://jia.je/hardware/2025/05/21/apple-m4/">measurements</a>).</p>
<p>A multiplication has a latency of 3 cycles. Emerald Rapids can do one every cycle. Zen 5 and Arrow Lake do three 64-bit multiplications per cycle (<a href="https://uops.info/html-instr/IMUL_R64_R64.html">uops.info</a>).</p>
<p>Thankfully, when the divisor is a constant, compilers replace the division by a multiplication and a shift (Granlund and Montgomery, 1994). Dividing by 7 is like multiplying by 1/7. In integer arithmetic, we multiply by an integer <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>c</mi></math> close to <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mfrac><msup><mn>2</mn><mi>s</mi></msup><mn>7</mn></mfrac></math> and we divide by <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mi>s</mi></msup></math> with a shift.</p>
<p>Can we do better than the compiler? With signed 32-bit integers on a 64-bit system, often yes.</p>
<h2>The compiler&#8217;s approach</h2>
<p>Let us divide a signed 32-bit integer by 7:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">div7</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">/</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">7</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>GCC 14 produces the following code on an x64 processor:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f8f8f8;">    </span><span style="color: #000;">movsx</span><span style="color: #f8f8f8;">   </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">imul</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">-1840700269</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">shr</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">32</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sar</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">31</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sar</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">2</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sub</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">ret</span>
</code></pre>
</div>
<p>It multiplies by a 32-bit constant, keeps the most significant 32 bits, and adds <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> back (<code>add eax, edi</code>). Then it shifts. The instructions <code>sar edi, 31</code> and <code>sub eax, edi</code> add one when <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> is negative, so that the result is rounded toward zero, as C requires.</p>
<p>Why add <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> back? The ideal multiplier for 7 is 2454267027 which needs 33 bits. It does not fit in a signed 32-bit integer. So the compiler stores <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mn>2454267027</mn><mo>−</mo><msup><mn>2</mn><mn>32</mn></msup><mo>=</mo><mo>−</mo><mn>1840700269</mn></math> and it corrects the result by adding <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math>. This is not rare: among the divisors between 3 and 1000 (excluding powers of two), 38% require this correction (376 out of 990). Over that range, GCC and LLVM pick the same multipliers, following <em>Hacker&#8217;s Delight</em>, Henry S. Warren, Jr.&#8217;s book of small algorithms for integers and bits.</p>
<h2>A 33-bit multiplier</h2>
<p>But we are on a 64-bit processor! Multiplying a 32-bit integer by a 33-bit integer gives at most a 65-bit integer… but not with <em>signed</em> integers. The absolute value of a signed 32-bit integer is at most <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mn>31</mn></msup></math>. Our multiplier needs 33 bits as a signed integer, but it is less than <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mn>32</mn></msup></math>: it fits in 32 bits as an unsigned integer. So the absolute value of the product is less than <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mn>63</mn></msup></math>, and it fits in a signed 64-bit integer.</p>
<p>We need to pick the multiplier. In a paper with Colin Bartlett and Owen Kaser (<a href="https://arxiv.org/abs/2012.12369">Integer division by constants: optimal bounds</a>), we show (Theorem 1) that <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>⌊</mo><mfrac><mrow><mi>c</mi><mo>⋅</mo><mi>n</mi></mrow><mi>m</mi></mfrac><mo>⌋</mo></mrow><mo>=</mo><mrow><mo>⌊</mo><mfrac><mi>n</mi><mi>d</mi></mfrac><mo>⌋</mo></mrow></math> for all integers <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>n</mi></math> in <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mo>[</mo><mn>0</mn><mo>,</mo><mi>N</mi><mo>]</mo></math> if and only if</p>
<p style="text-align: center;">
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mfrac><mn>1</mn><mi>d</mi></mfrac><mo>≤</mo><mfrac><mi>c</mi><mi>m</mi></mfrac><mo>&lt;</mo><mrow><mo>(</mo><mn>1</mn><mo>+</mo><mfrac><mn>1</mn><mrow><mi>N</mi><mo>−</mo><mo>(</mo><mrow><mo>(</mo><mi>N</mi><mo>+</mo><mn>1</mn><mo>)</mo><mo>mod</mo><mi>d</mi></mrow><mo>)</mo></mrow></mfrac><mo>)</mo><mo>⋅</mo><mfrac><mn>1</mn><mi>d</mi></mfrac></mrow><mo>.</mo></math> </p>
<p>We pick <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>m</mi><mo>=</mo><msup><mn>2</mn><mi>s</mi></msup></math>, the smallest power of two such that <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>m</mi><mo>≥</mo><mrow><mi>d</mi><mo>⋅</mo><msup><mn>2</mn><mn>31</mn></msup></mrow></math>, and we round up: <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>c</mi><mo>=</mo><mrow><mo>⌈</mo><mfrac><mi>m</mi><mi>d</mi></mfrac><mo>⌉</mo></mrow></math>. Then <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi>c</mi><mo>⋅</mo><mi>d</mi></mrow><mo>−</mo><mi>m</mi><mo>&lt;</mo><mi>d</mi></math> and both inequalities hold for <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>N</mi><mo>=</mo><msup><mn>2</mn><mn>31</mn></msup></math>. We get <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>⌊</mo><mfrac><mrow><mo>|</mo><mi>x</mi><mo>|</mo></mrow><mi>d</mi></mfrac><mo>⌋</mo></mrow></math> for all signed 32-bit integers <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math>. For negative values of <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math>, we compute <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>⌊</mo><mfrac><mrow><mi>c</mi><mo>⋅</mo><mi>x</mi></mrow><mi>m</mi></mfrac><mo>⌋</mo></mrow></math> and add one, as the compiler does. It works because <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mfrac><mrow><mi>c</mi><mo>⋅</mo><mrow><mo>|</mo><mi>x</mi><mo>|</mo></mrow></mrow><mi>m</mi></mfrac></math> is never an integer when <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> is not zero: if it were an integer <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>k</mi></math>, the bounds would force <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>|</mo><mi>x</mi><mo>|</mo></mrow><mo>=</mo><mrow><mi>k</mi><mo>⋅</mo><mi>d</mi></mrow></math>, and thus <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi>c</mi><mo>⋅</mo><mi>d</mi></mrow><mo>=</mo><mi>m</mi></math>, which is impossible since <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>d</mi></math> is not a power of two.</p>
<p>The code is short:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">template</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">d</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">div32</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">constexpr</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">31</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">bit_width</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">uint32_t</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">d</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">));</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">// 31 + ceiling(log2(d))</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">constexpr</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">c</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">/</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">d</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;">        </span><span style="color: #8f5902; font-style: italic;">// ceiling(2^s / d)</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// floor(c * x / 2^s), plus one when x is negative</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #000; font-weight: bold;">((</span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">c</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">x</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">31</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>I assume that the divisor is positive and not a power of two.</p>
<p>GCC 14 compiles <code>div32&lt;7&gt;</code> to:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f8f8f8;">    </span><span style="color: #000;">movsx</span><span style="color: #f8f8f8;">   </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">2454267027</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">shr</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">31</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">imul</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rdx</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sar</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">rax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">34</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eax</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">edi</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">ret</span>
</code></pre>
</div>
<p>There is no correction step after the multiplication. The longest chain of dependent instructions is now <code>movsx</code>, <code>imul</code>, <code>sar</code>, <code>add</code> instead of <code>movsx</code>, <code>imul</code>, <code>shr</code>, <code>add</code>, <code>sar</code>, <code>sub</code>. The constant fits in 32 bits as an <em>unsigned</em> value, so loading it is cheap.</p>
<p>On ARM (Apple M4), the compiler gives us:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f57900;">_compiler_div7:</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#9363</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">movk</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#37449</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">lsl</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#16</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">smull</span><span style="color: #f8f8f8;">   </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">lsr</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#32</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">asr</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w9</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#2</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w9</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">lsr</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#31</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">ret</span>
</code></pre>
</div>
<p>and the new approach gives us:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #f57900;">_div32_7:</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">sxtw</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mov</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w9</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#9363</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">movk</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">w9</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#37449</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">lsl</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#16</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">mul</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x9</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">asr</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#34</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">add</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">w0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">lsr</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">#31</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">ret</span>
</code></pre>
</div>
<p>Why don&#8217;t compilers do it? I don&#8217;t know.</p>
<h2>Performance</h2>
<p>I divide 16384 random signed integers. In the <em>throughput</em> test, the divisions are independent. In the <em>latency</em> test, each division depends on the previous one. The latency includes two cycles of overhead (to make each division depend on the previous one). I disable autovectorization. I use an Intel Xeon Gold 6548N (Emerald Rapids) with GCC 14 and clang 21, and an Apple M4 Max with Apple clang 17.</p>
<p><em>Intel, cycles per division: throughput (latency)</em></p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/10/signeddiv-intel-latency-v2-scaled.webp" alt="Latency in cycles on Intel" class="alignnone size-large" /></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: right;">divisor</th>
<th style="text-align: right;">GCC: x / d</th>
<th style="text-align: right;">GCC: div32</th>
<th style="text-align: right;">clang: x / d</th>
<th style="text-align: right;">clang: div32</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="text-align: right;">7</td>
<td style="text-align: right;">2.0 (10.0)</td>
<td style="text-align: right;">2.0 (8.0)</td>
<td style="text-align: right;">1.76 (10.4)</td>
<td style="text-align: right;">1.38 (8.0)</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">60</td>
<td style="text-align: right;">2.0 (10.0)</td>
<td style="text-align: right;">2.0 (8.0)</td>
<td style="text-align: right;">1.76 (10.4)</td>
<td style="text-align: right;">1.38 (8.0)</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">365</td>
<td style="text-align: right;">2.0 (10.0)</td>
<td style="text-align: right;">2.0 (8.0)</td>
<td style="text-align: right;">1.76 (10.6)</td>
<td style="text-align: right;">1.38 (8.0)</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">1000</td>
<td style="text-align: right;">2.0 (8.0)</td>
<td style="text-align: right;">2.0 (8.0)</td>
<td style="text-align: right;">1.35 (8.0)</td>
<td style="text-align: right;">1.38 (8.0)</td>
</tr>
</tbody>
</table>
<p><em>Apple M4, nanoseconds per division: throughput (latency)</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: right;">divisor</th>
<th style="text-align: right;">x / d</th>
<th style="text-align: right;">div32</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="text-align: right;">7</td>
<td style="text-align: right;">0.34 (2.01)</td>
<td style="text-align: right;">0.29 (1.77)</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">60</td>
<td style="text-align: right;">0.31 (2.00)</td>
<td style="text-align: right;">0.27 (1.77)</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">365</td>
<td style="text-align: right;">0.34 (2.00)</td>
<td style="text-align: right;">0.28 (1.77)</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">1000</td>
<td style="text-align: right;">0.29 (1.57)</td>
<td style="text-align: right;">0.27 (1.77)</td>
</tr>
</tbody>
</table>
<p>For 7, 60 and 365, the latency goes down by 2 cycles with GCC (10 to 8), and from 10.4 or 10.6 to 8 with clang. On the M4 it goes down by 12%. With clang on the Intel processor, the throughput improves by more than 20%. With GCC, the throughput does not change in my benchmark.</p>
<p>The divisor 1000 is a control. The compiler&#8217;s multiplier is 274877907, which fits in a signed 32-bit register, so there is nothing to add back. On the Intel processor both versions take 8 cycles. Each one starts with <code>movsx</code>, then a multiplication, a shift, and an add or a subtract.</p>
<p>On the M4, <code>div32</code> is slightly slower: 1.77 ns against 1.57 ns. The compiler&#8217;s multiplier fits in a signed 32-bit register, so it uses <code>smull</code>, which sign-extends <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> as part of the multiplication. Our multiplier (2199023256) does not fit in a signed 32-bit register, so we need a separate <code>sxtw</code> instruction before the multiplication. Thus you should only use <code>div32</code> when the compiler needs the correction step.</p>
<h2>When the divisor is not a constant</h2>
<p>When the divisor is only known at runtime, the compiler cannot help and it emits the division instruction. Years ago, I wrote a small library, <a href="https://github.com/lemire/fastmod">fastmod</a>, that precomputes a 64-bit multiplier <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>M</mi></math>, about <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mfrac><msup><mn>2</mn><mn>64</mn></msup><mrow><mo>|</mo><mi>d</mi><mo>|</mo></mrow></mfrac></math>, once, and then divides with a multiplication. It is described in <a href="https://arxiv.org/abs/1902.01961">Faster Remainder by Direct Computation</a> (Software: Practice and Experience, 2019).</p>
<p>Its signed division function treated <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>M</mi></math> as an unsigned integer. Multiplying an unsigned 64-bit integer by a signed integer requires a correction when the signed integer is negative. The 0.1.0 code branches on the sign of <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math>. On x64 the compiler leaves the branch in place. With random signs, it is often mispredicted: in my tests, that happens in 36% to 48% of the divisions.</p>
<p>The same observation applies: the absolute value of <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> is at most <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mn>31</mn></msup></math>, and as long as <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>|</mo><mi>d</mi><mo>|</mo></mrow><mo>&gt;</mo><mn>2</mn></math>, <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>M</mi></math> is less than <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msup><mn>2</mn><mn>63</mn></msup></math>. So a <em>signed</em> 64-bit multiplication gives the right answer directly. When <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>|</mo><mi>d</mi><mo>|</mo></mrow><mo>=</mo><mn>2</mn></math>, we add <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>x</mi></math> back. The new function has no branch:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fastdiv_s32</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">a</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">M</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">d</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">signed_M</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #000;">M</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">highbits</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">mul128_s64_hi</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">a</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">signed_M</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">a</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">signed_M</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">63</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// Round toward zero: add one when a &lt; 0.</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">q</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #000;">highbits</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">a</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">?</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// Negate when d &lt; 0.</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int32_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">sign_d</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">d</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">31</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">q</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">^</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">sign_d</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">sign_d</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>The function <code>mul128_s64_hi</code> returns the most significant 64 bits of the 128-bit product of two signed 64-bit integers: it compiles to a single instruction (<code>imul</code> on x64, <code>smulh</code> on ARM). The value <code>signed_M &gt;&gt; 63</code> is zero unless <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mo>|</mo><mi>d</mi><mo>|</mo></mrow><mo>=</mo><mn>2</mn></math>; it only depends on the divisor, so the compiler can compute it once, outside the loop. I released this change as <a href="https://github.com/lemire/fastmod/releases/tag/v0.2.0">fastmod 0.2.0</a>.</p>
<p>Here are the results for a divisor (7) passed on the command line, with random signs. I report the time per division in nanoseconds: throughput (latency).</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">system</th>
<th style="text-align: right;">division instruction</th>
<th style="text-align: right;">fastmod 0.1.0</th>
<th style="text-align: right;">fastmod 0.2.0</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Intel, GCC 14</td>
<td style="text-align: right;">1.72 (4.0)</td>
<td style="text-align: right;">3.07 (4.2)</td>
<td style="text-align: right;">0.77 (3.2)</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Intel, clang 21</td>
<td style="text-align: right;">1.72 (4.0)</td>
<td style="text-align: right;">3.42 (3.4)</td>
<td style="text-align: right;">0.78 (2.9)</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Apple M4</td>
<td style="text-align: right;">0.44 (2.2)</td>
<td style="text-align: right;">0.39 (2.0)</td>
<td style="text-align: right;">0.38 (2.0)</td>
</tr>
</tbody>
</table>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/10/signeddiv-runtime-latency-v2.webp" alt="Latency in nanoseconds when the divisor is known only at runtime" class="alignnone size-large" /></p>
<p>On the Intel processor, the previous version of the library was <em>slower</em> than the division instruction with random signs because of the mispredicted branches. The new version is four times faster than the previous one, and more than twice as fast as the division instruction. On the Apple M4, the previous version did not branch (the compiler computed both cases and selected one), so there is little difference in scalar code. However, without the branch, clang can partially vectorize a loop calling the new function: if I allow autovectorization, the new version takes 0.22 ns on the M4.</p>
<h2>Conclusion</h2>
<p>When dividing signed 32-bit integers on a 64-bit processor, there is a free bit. Compilers do not use it. For more than a third of the divisors, we can avoid two instructions on the critical path, and in a library that divides by a runtime divisor, we can avoid a branch.</p>
<p>This investigation was prompted by Ben Joffe&#8217;s recent post on <a href="https://www.benjoffe.com/fast-floordiv">fast floor division by constants</a>, which you should read.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/signeddiv">My source code is available</a>.</p>
<p><strong>References</strong></p>
<ul>
<li>Daniel Lemire, Colin Bartlett, Owen Kaser, <a href="https://arxiv.org/abs/2012.12369">Integer division by constants: optimal bounds</a>, Heliyon 7 (6), 2021.</li>
<li>Daniel Lemire, Owen Kaser, Nathan Kurz, <a href="https://arxiv.org/abs/1902.01961">Faster Remainder by Direct Computation: Applications to Compilers and Software Libraries</a>, Software: Practice and Experience 49 (6), 2019.</li>
<li>Torbjörn Granlund, Peter L. Montgomery, Division by invariant integers using multiplication, PLDI 1994.</li>
<li>Henry S. Warren, Jr., Hacker&#8217;s Delight, 2nd edition, 2013.</li>
</ul>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/31/beating-the-compiler-at-signed-32-bit-division/feed/</wfw:commentRss>
			<slash:comments>4</slash:comments>
		
		
			</item>
		<item>
		<title>Billion-query-per-second lookup tables in C++</title>
		<link>https://lemire.me/blog/2026/10/26/billion-query-per-second-lookup-tables-in-c/</link>
					<comments>https://lemire.me/blog/2026/10/26/billion-query-per-second-lookup-tables-in-c/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 26 Oct 2026 18:23:08 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23082</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/perfect-hash-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />A lot of software spends its time mapping strings to values. A web server must identify the HTTP method (GET, POST, &#8230;). It must recognize header names. A URL parser must recognize the protocol (http, https, ftp, &#8230;). A compiler must recognize keywords. In C++, the default approach is to put the strings in a &#8230; <a href="https://lemire.me/blog/2026/10/26/billion-query-per-second-lookup-tables-in-c/" class="more-link">Continue reading <span class="screen-reader-text">Billion-query-per-second lookup tables in C++</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/perfect-hash-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>A lot of software spends its time mapping strings to values. A web server must identify the HTTP method (<code>GET</code>, <code>POST</code>, &#8230;). It must recognize header names. A URL parser must recognize the protocol (<code>http</code>, <code>https</code>, <code>ftp</code>, &#8230;). A compiler must recognize keywords.</p>
<p>In C++, the default approach is to put the strings in a <code>std::unordered_map</code>.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">unordered_map</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">string_view</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">methods</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">"GET"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">GET</span><span style="color: #000; font-weight: bold;">},</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">"POST"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">POST</span><span style="color: #000; font-weight: bold;">},</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">"PUT"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">PUT</span><span style="color: #000; font-weight: bold;">},</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">"DELETE"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">DELETE</span><span style="color: #000; font-weight: bold;">},</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">"PATCH"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">PATCH</span><span style="color: #000; font-weight: bold;">}};</span>
</code></pre>
</div>
<p>It works. But notice something: the keys are known when you write the code. They never change. And yet we pay at runtime as if they could change at any time: we hash the whole string, we pick a bucket, we follow a pointer, we compare, maybe we walk a list.</p>
<h2>Perfect hashing</h2>
<p>A hash function maps keys to slots. When two keys land in the same slot, we have a collision, and we must deal with it. Collisions are what make hash tables complicated and slow.</p>
<p>If you know all of your keys ahead of time, you can construct a hash function designed for your keys, a function that never produces a collision. It is called a perfect hash function. The idea is old: the GNU <code>gperf</code> tool has been doing it since 1989. You give it a list of keywords, and it generates C code.</p>
<p>We follow the same design as <code>gperf</code>. The slot for a key is computed as follows.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">slot</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">length</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">asso</span><span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">key[i</span><span style="color: #ce5c00; font-weight: bold;">]]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">asso</span><span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">key[j</span><span style="color: #ce5c00; font-weight: bold;">]]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">...)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">%</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">table_size</span>
</code></pre>
</div>
<p>We add the length of the key to a few small values taken from an association table (<code>asso</code>) indexed by the characters at a few selected positions. Usually, one or two positions are enough. The table size is a power of two, so the modulo is a single bitwise AND.</p>
<p>Consider the four HTTP methods <code>GET</code>, <code>PUT</code>, <code>POST</code> and <code>DELETE</code>. We use only the first character. Set <code>asso['G'] = 0</code>, <code>asso['P'] = 1</code> and <code>asso['D'] = 0</code>. Then <code>GET</code> goes in slot 3 + 0 = 3, <code>PUT</code> goes in slot 3 + 1 = 4, <code>POST</code> goes in slot 4 + 1 = 5 and <code>DELETE</code> goes in slot 6 + 0 = 6. No collision.</p>
<p>To look up <code>PUT</code>, we compute the slot (4), and we compare <code>PUT</code> with the one key stored in slot 4. One hash, one comparison. If you look up <code>PATCH</code>, which is not in the set, you land in slot 5 + 1 = 6, you compare with <code>DELETE</code>, and you are done: not found. There is no probing, no chain to walk.</p>
<p>How do we find the association values? We sort the characters by how many keys they touch. We then take them one at a time and try 0, 1, 2, &#8230; until we find a value that does not cause a collision. We never revise a value once it is set. It is a greedy algorithm with bounded work. The bound matters because we do all of this at compile time, and C++ compilers limit how much work a <code>constexpr</code> function may do.</p>
<h2>Doing it in C++</h2>
<p>With <a href="https://github.com/ConstexprCore/perfect_hash">our library</a>, you write ordinary C++23. There is no external tool to run.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;ConstexprCore/perfect_hash.h&gt;</span>
<span style="color: #204a87; font-weight: bold;">using</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">namespace</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ConstexprCore</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">enum</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">class</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">GET</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">POST</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PUT</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">DELETE</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">PATCH</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">constexpr</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">methods</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">make_perfect_map</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">kv</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"GET"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">GET</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">kv</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"POST"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;">   </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">POST</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">kv</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"PUT"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;">    </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">PUT</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">kv</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"DELETE"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">DELETE</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">kv</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"PATCH"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;">  </span><span style="color: #000;">Method</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">PATCH</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span>
<span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">();</span>
</code></pre>
</div>
<p>The map is a compile-time constant. The compiler searches for the perfect hash function when it compiles your code. At runtime, computing the hash can take as few as four instructions: load one byte of the key, load one byte from the association table, add the length, apply a mask.</p>
<h2>Comparing strings quickly</h2>
<p>Computing the hash is cheap. But we still need to check that the string we are looking up matches the key stored in the slot. And comparing strings can be expensive: the textbook approach is a loop over the bytes with a branch for each byte. You can easily end up with dozens of instructions and several branches, and the branches are hard to predict when the strings vary.</p>
<p>Instead, we treat the key as a vector. Our processors, whether ARM or x64, have 16-byte registers. So we load 16 bytes unconditionally, even if the key is only three bytes long. We then use a shuffle instruction (<code>tbl</code> on ARM NEON) to zero the bytes past the end of the key. The stored keys are zero-padded at compile time. A single vector comparison handles the whole key. We go from about forty instructions and eight branches to five instructions and no branch.</p>
<p>But wait: is it safe to read 16 bytes when you only have three? Reading past the end of a buffer is fine as long as you do not cross into another memory page, and pages are at least 4 kilobytes. So we check whether the address is within 16 bytes of the end of its page. If it is, we take a slower path. Otherwise, we do the wide load. We traded several unpredictable branches for a single branch that is almost always taken the same way.</p>
<h2>How fast is it?</h2>
<p>I ran the benchmarks on an Apple M3 Max with AppleClang 15. We look up the keys in random order and we report the time per lookup. Let us first consider the 20 standard HTTP header names.</p>
<p><img decoding="async" alt="HTTP header names, nanoseconds per lookup" class="alignnone size-large" src="https://lemire.me/blog/wp-content/uploads/2026/10/perfect-hash-http-headers.webp" /></p>
<p>We need about 1.3 ns per lookup, or about 5 cycles. The classic <code>gperf</code> needs 7.7 ns. Abseil&#8217;s <code>flat_hash_map</code>, an excellent hash table, needs 10.9 ns. And <code>std::unordered_map</code> needs 16.6 ns. We are thirteen times faster than the standard library.</p>
<p>It is not one lucky key set. Here are the results over several sets of keys, from 6 URL protocols to the 100 tickers of the S&amp;P 100.</p>
<p><img decoding="async" alt="Seven key sets, nanoseconds per lookup" class="alignnone size-large" src="https://lemire.me/blog/wp-content/uploads/2026/10/perfect-hash-keysets.webp" /></p>
<p>We stay between 1.2 ns and 2 ns per lookup. We are 3 to 6 times faster than <code>gperf</code>, and 7 to 13 times faster than <code>std::unordered_map</code>. Notice how <code>gperf</code> struggles with the JavaScript reserved words: there we are 17 times faster.</p>
<p>Why is <code>std::unordered_map</code> so much slower? The C++ standard essentially requires a chained hash table. A lookup hashes the whole string by calling a function, follows a pointer to a heap-allocated node and walks a list. Our lookup does a few loads and has no function call. The processor can retire over 8.6 instructions per cycle with our code, which is about as good as it gets on this chip, and it almost never mispredicts a branch.</p>
<p><img decoding="async" alt="Cost of one lookup" class="alignnone size-large" src="https://lemire.me/blog/wp-content/uploads/2026/10/perfect-hash-lookup-cost.webp" /></p>
<p>On an Apple M4, with the URL protocols, we exceed one billion lookups per second.</p>
<h2>Conclusion</h2>
<p>If your keys are known at compile time, you should not pay for collision handling at runtime. A perfect hash function gets you down to one comparison. With some care, a lookup costs about five cycles. The library is a single header you can drop in your project: <a href="https://github.com/ConstexprCore/perfect_hash">github.com/ConstexprCore/perfect_hash</a>.</p>
<p>This is joint work with Francisco Geiman Thiesen (Microsoft). We presented it at P99 CONF 2026. You can <a href="https://www.youtube.com/watch?v=_LG7ZeS5j5A">watch the talk on YouTube</a>.</p>
<p>The <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/perfect_hash">benchmark numbers and the plotting script</a> are available.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/26/billion-query-per-second-lookup-tables-in-c/feed/</wfw:commentRss>
			<slash:comments>16</slash:comments>
		
		
			</item>
		<item>
		<title>Decoding base64 in Python at 10 gigabytes per second</title>
		<link>https://lemire.me/blog/2026/10/17/decoding-base64-in-python-at-10-gigabytes-per-second/</link>
					<comments>https://lemire.me/blog/2026/10/17/decoding-base64-in-python-at-10-gigabytes-per-second/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 17 Oct 2026 04:40:34 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23075</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/pybase64-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />We sometimes need to convert binary data to text. For example, your emails must be stored as text files. Naïvely, you could just use a string of 0 and 1, but it would mean using one character per bit. We can do better by using 64 ASCII characters: all letters, all numbers, plus a few &#8230; <a href="https://lemire.me/blog/2026/10/17/decoding-base64-in-python-at-10-gigabytes-per-second/" class="more-link">Continue reading <span class="screen-reader-text">Decoding base64 in Python at 10 gigabytes per second</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/pybase64-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>We sometimes need to convert binary data to text. For example, your emails must be stored as text files. Naïvely, you could just use a string of <code>0</code> and <code>1</code>, but it would mean using one character per bit. We can do better by using 64 ASCII characters: all letters, all numbers, plus a few extra characters. Base64 represents three bytes with four ASCII characters.</p>
<p>Base64 is how email carries attachments, how certificates are stored, how images get embedded in web pages (data URLs), and how binary data is encoded inside JSON.</p>
<p>Python comes with a <code>base64</code> module. It is written in C, but it processes the data one character at a time. There is a popular alternative, <a href="https://github.com/mayeut/pybase64">pybase64</a>.</p>
<p>We have been working on base64 for years. With Wojciech Muła, we showed how to encode and decode base64 with SIMD instructions. More recently, I worked on <em>forgiving</em> base64 decoding: the variant used by web browsers, which skips spaces and line breaks. Our <a href="https://github.com/simdutf/simdutf">simdutf</a> library implements these ideas. Mainstream browsers rely on it.</p>
<p>But simdutf had no Python binding. So we wrote one: <a href="https://github.com/simdutf/fastpybase64">fastpybase64</a>.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>pip<span style="color: #f8f8f8;"> </span>install<span style="color: #f8f8f8;"> </span>fastpybase64
</code></pre>
</div>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fastpybase64</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">as</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">base64</span>
<span style="color: #000;">base64</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">b64encode</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">b"hello"</span><span style="color: #000; font-weight: bold;">)</span>      <span style="color: #8f5902; font-style: italic;"># b'aGVsbG8='</span>
<span style="color: #000;">base64</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">b64decode</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">"aGVsbG8="</span><span style="color: #000; font-weight: bold;">)</span>    <span style="color: #8f5902; font-style: italic;"># b'hello'</span>
</code></pre>
</div>
<p>It is meant as a drop-in replacement for the standard <code>base64</code> module.</p>
<p>How fast is it? I use an Intel Xeon Gold 6548N (Emerald Rapids) pinned to one core, with Python 3.14.6. I installed fastpybase64 0.1.0 and pybase64 1.5.1 from PyPI. I decode the base64 encoding of random bytes, and I report the speed in gigabytes of decoded data per second. A lot of real base64 has line breaks. MIME, the format of email attachments, breaks the base64 text into lines of 76 characters. So that&#8217;s what I use as an example. We use a single thread.</p>
<p><img decoding="async" class="alignnone size-large" alt="Decoding base64 with line breaks" src="https://lemire.me/blog/wp-content/uploads/2026/10/pybase64-mime.webp" /></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: right;">size</th>
<th style="text-align: right;">base64</th>
<th style="text-align: right;">pybase64</th>
<th style="text-align: right;">fastpybase64</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="text-align: right;">1 KiB</td>
<td style="text-align: right;">0.44</td>
<td style="text-align: right;">1.5</td>
<td style="text-align: right;">4.4</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">64 KiB</td>
<td style="text-align: right;">0.49</td>
<td style="text-align: right;">3.1</td>
<td style="text-align: right;">13.3</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">1 MiB</td>
<td style="text-align: right;">0.49</td>
<td style="text-align: right;">3.2</td>
<td style="text-align: right;">13.1</td>
</tr>
</tbody>
</table>
<p>The fastpybase64 module is four times faster than pybase64 and 27 times faster than the standard library.</p>
<p>The fastpybase64 module is available on <a href="https://pypi.org/project/fastpybase64/">PyPI</a> for Linux, macOS and Windows, for Python 3.10 to 3.14, including free-threaded Python 3.14. My <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/pybase64">benchmark is available</a>.</p>
<p><strong>Further reading</strong></p>
<ul>
<li>Daniel Lemire, <a href="https://dl.acm.org/doi/10.1145/3859286">Fast Forgiving Base64 Decoding with SIMD Instructions</a>, ACM Transactions on Architecture and Code Optimization, 2026.</li>
<li>Wojciech Muła, Daniel Lemire, <a href="https://arxiv.org/abs/1910.05109">Base64 encoding and decoding at almost the speed of a memory copy</a>, Software: Practice and Experience 50 (2), 2020.</li>
<li>Wojciech Muła, Daniel Lemire, <a href="https://arxiv.org/abs/1704.00605">Faster Base64 Encoding and Decoding using AVX2 Instructions</a>, ACM Transactions on the Web 12 (3), 2018.</li>
</ul>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/17/decoding-base64-in-python-at-10-gigabytes-per-second/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title>Redis now has compressed bitmaps (Roaring)</title>
		<link>https://lemire.me/blog/2026/10/15/redis-now-has-compressed-bitmaps-roaring/</link>
					<comments>https://lemire.me/blog/2026/10/15/redis-now-has-compressed-bitmaps-roaring/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 15 Oct 2026 22:46:54 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23066</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-roaring-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Redis is an in-memory database. Salvatore Sanfilippo started it in 2009, and it is now one of the most widely deployed pieces of server software. It is often used as a cache in front of a slower database. You talk to it over the network with short text commands such as SET user:42 "Alice" and &#8230; <a href="https://lemire.me/blog/2026/10/15/redis-now-has-compressed-bitmaps-roaring/" class="more-link">Continue reading <span class="screen-reader-text">Redis now has compressed bitmaps (Roaring)</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-roaring-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p><a href="https://redis.io">Redis</a> is an in-memory database. Salvatore Sanfilippo started it in 2009, and it is now one of the most widely deployed pieces of server software. It is often used as a cache in front of a slower database. </p>
<p>You talk to it over the network with short text commands such as <code>SET user:42 "Alice"</code> and <code>GET user:42</code>. Each key maps to a value, and the value can be more than a string: Redis has lists, hashes, sets, sorted sets, streams and more, each with its own commands. </p>
<p>We often represent sets of integers as bitmaps (or bitsets). The simplest bitmap is a single machine word of, say, 8 bits. It can be used to represent all sets of integers smaller than 8. By adding more words, you can represent sets of integers smaller than 16, or any value you like. You can then do fast operations over bitmaps such as intersections and unions. They translate into a few bit-wise operations such as AND, OR, XOR. </p>
<p>Redis has supported bitmaps for a long time. You call <code>SETBIT key offset 1</code>, and Redis stores a plain bitmap inside an ordinary Redis string: bit <em>i</em> of the bitmap is bit <em>i</em> of the string. It is simple and fast. But the string takes one byte per eight bit positions, up to the largest offset you ever set, however many bits are actually set. Setting a single bit at offset four billion allocates 512 MB, and every <code>BITCOUNT</code> or <code>BITOP</code> has to scan all of it.</p>
<p>A compressed bitmap avoids this cost: it stores a set of integers in a form that takes little room when the set is sparse, and it computes counts, intersections and unions without decompressing. <a href="https://roaringbitmap.org">Roaring</a> is probably the most widely used compressed bitmap format. Rather than compressing a long string of bits, Roaring adapts its storage to the data: sparse regions are stored as short lists of integers, dense regions as plain bitsets, and long stretches of consecutive values as runs. It is often both smaller and faster than an uncompressed bitset. Roaring is used by <a href="https://roaringbitmap.org">many systems you already know</a>: Elasticsearch, Apache Spark, ClickHouse &#8230;</p>
<p>Antonio Viggiano has brought Roaring into Redis itself. It should ship in Redis 8.12. There is a new type: <code>TYPE key</code> returns <code>bitmap</code> and <code>OBJECT ENCODING key</code> returns <code>bitmap-roaring</code>. It is opt-in: <code>CONFIG SET bitmap-default-roaring yes</code>.</p>
<p>I built Redis at the merge commit and ran the server on one core of an Intel Xeon Gold 6548N. For each workload, I create two bitmaps, <code>a</code> and <code>b</code>, once as strings and once as Roaring bitmaps, with the same content, and I time the commands. My workloads:</p>
<ul>
<li><em>0.1% random</em>: 100,000 random ids out of 100 million (think: the users who used a given feature today);</li>
<li><em>50% random</em>: 50 million random ids out of 100 million;</li>
<li><em>runs</em>: ten runs of one million consecutive ids in a range of 100 million (think: cohorts of users who signed up in the same week);</li>
<li><em>hashed 32-bit ids</em>: 100,000 random 32-bit values, so the string is 512 MB.</li>
</ul>
<p><img decoding="async" class="alignnone size-large" alt="BITCOUNT" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-bitcount.webp" /></p>
<p><img decoding="async" class="alignnone size-large" alt="BITOP AND" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-bitop-and.webp" /></p>
<p><img decoding="async" class="alignnone size-large" alt="BITOP OR" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-bitop-or.webp" /></p>
<p>You can see that Roaring can be much faster than a standard uncompressed bitmap in some cases. The hashed ids are even more compelling. The string must span all four billion bit positions, and each <code>BITOP</code> scans 512 MB. </p>
<p><img decoding="async" class="alignnone size-large" alt="BITOP AND, hashed ids" src="https://lemire.me/blog/wp-content/uploads/2026/10/redis-bitop-and-hashed.webp" /></p>
<p>Compressed bitmaps like Roaring are not always better. There are many tradeoffs. But turning them on will soon take one configuration setting. <a href="https://roaringbitmap.org">Check out the Roaring bitmap website for more information</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/15/redis-now-has-compressed-bitmaps-roaring/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Parsing JSON at tens of gigabytes per second&#8230; in Python</title>
		<link>https://lemire.me/blog/2026/10/12/parsing-json-at-tens-of-gigabytes-per-second-in-python/</link>
					<comments>https://lemire.me/blog/2026/10/12/parsing-json-at-tens-of-gigabytes-per-second-in-python/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 12 Oct 2026 13:06:39 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/2026/10/12/parsing-json-at-tens-of-gigabytes-per-second-in-python/</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/pyjson-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Python used to have limited parallelism, due to its global interpreter lock (GIL). Only one thread could run Python code at a time. Python 3.14 comes in a free-threaded version (python3.14t), without the global interpreter lock. Several threads can now run Python code at the same time. Let us pick a realistic problem. GitHub publishes &#8230; <a href="https://lemire.me/blog/2026/10/12/parsing-json-at-tens-of-gigabytes-per-second-in-python/" class="more-link">Continue reading <span class="screen-reader-text">Parsing JSON at tens of gigabytes per second&#8230; in Python</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/pyjson-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Python used to have limited parallelism, due to its global interpreter lock (GIL). Only one thread could run Python code at a time. Python 3.14 comes in a free-threaded version (<code>python3.14t</code>), without the global interpreter lock. Several threads can now run Python code at the same time.</p>
<p>Let us pick a realistic problem. GitHub publishes everything that happens on its public repositories: pushes, issues, pull requests, stars, and so forth. The <a href="https://www.gharchive.org/">GH Archive</a> project collects these events as JSON, one event per line (NDJSON), one file per hour. I took eight hours of March 13, 2024: 7.6 GB of JSON once decompressed, 2.2 million events. I want to know which programming languages get the most new pull requests. So I count the events by type, and for each pull request that was opened, I look up the language of the repository. Each event is a large JSON object, about 3.4 kB on average: I need only a few of its fields.</p>
<p>The answer, for these eight hours: 65,269 pull requests were opened, mostly toward TypeScript (11,565), JavaScript (7,778), Python (7,057) and Java (4,668) repositories. About 5,857 targeted repositories without a language.</p>
<p>With <a href="https://github.com/simdjson/fastpysimdjson">fastsimdjson</a>, our Python binding for simdjson, the code looks as follows. The function <code>map_many</code> calls my function on each event, using as many threads as I ask for.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">threading</span>
<span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fastsimdjson</span>
<span style="color: #000;">local</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">threading</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">local</span><span style="color: #000; font-weight: bold;">()</span>
<span style="color: #000;">parts</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000; font-weight: bold;">[]</span>
<span style="color: #204a87; font-weight: bold;">def</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">one_event</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">e</span><span style="color: #000; font-weight: bold;">):</span>
    <span style="color: #000;">acc</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">getattr</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">local</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #4e9a06;">"acc"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #204a87; font-weight: bold;">None</span><span style="color: #000; font-weight: bold;">)</span>
    <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: #000;">acc</span> <span style="color: #204a87; font-weight: bold;">is</span> <span style="color: #204a87; font-weight: bold;">None</span><span style="color: #000; font-weight: bold;">:</span>  <span style="color: #8f5902; font-style: italic;"># first event seen by this thread</span>
        <span style="color: #000;">acc</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">local</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">acc</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">Counter</span><span style="color: #000; font-weight: bold;">(),</span> <span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">Counter</span><span style="color: #000; font-weight: bold;">())</span>
        <span style="color: #000;">parts</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">append</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">acc</span><span style="color: #000; font-weight: bold;">)</span>
    <span style="color: #000;">t</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">e</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"type"</span><span style="color: #000; font-weight: bold;">]</span>
    <span style="color: #000;">acc</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">]</span> <span style="color: #ce5c00; font-weight: bold;">+=</span> <span style="color: #0000cf; font-weight: bold;">1</span>
    <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: #000;">t</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: #4e9a06;">"PullRequestEvent"</span><span style="color: #000; font-weight: bold;">:</span>
        <span style="color: #000;">p</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">e</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"payload"</span><span style="color: #000; font-weight: bold;">]</span>
        <span style="color: #204a87; font-weight: bold;">if</span> <span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"action"</span><span style="color: #000; font-weight: bold;">]</span> <span style="color: #ce5c00; font-weight: bold;">==</span> <span style="color: #4e9a06;">"opened"</span><span style="color: #000; font-weight: bold;">:</span>
            <span style="color: #000;">acc</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #4e9a06;">"pull_request"</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #4e9a06;">"base"</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #4e9a06;">"repo"</span><span style="color: #000; font-weight: bold;">][</span><span style="color: #4e9a06;">"language"</span><span style="color: #000; font-weight: bold;">]]</span> <span style="color: #ce5c00; font-weight: bold;">+=</span> <span style="color: #0000cf; font-weight: bold;">1</span>
<span style="color: #000;">data</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">open</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">"events.json"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #4e9a06;">"rb"</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">read</span><span style="color: #000; font-weight: bold;">()</span>
<span style="color: #000;">fastsimdjson</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">map_many</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">one_event</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">data</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #204a87;">format</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #4e9a06;">"lines"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">threads</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #0000cf; font-weight: bold;">64</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">lazy</span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #204a87; font-weight: bold;">True</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000;">types</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">languages</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">Counter</span><span style="color: #000; font-weight: bold;">(),</span> <span style="color: #000;">collections</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">Counter</span><span style="color: #000; font-weight: bold;">()</span>
<span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">lang</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #000;">parts</span><span style="color: #000; font-weight: bold;">:</span>
    <span style="color: #000;">types</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">update</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">)</span>
    <span style="color: #000;">languages</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">update</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">lang</span><span style="color: #000; font-weight: bold;">)</span>
</code></pre>
</div>
<p>Each thread keeps its own counters. The first event a thread sees creates the pair and appends it to <code>parts</code>. Later events on that thread update the same pair. Nothing in the counters is shared, so the threads do not wait for each other. <code>map_many</code> returns only once every event has been counted. The main thread then walks <code>parts</code>, one pair per thread, and <code>Counter.update</code> adds the counts. A few dozen event types and about a hundred languages: the merge is tiny next to the parsing.</p>
<p>With <code>lazy=True</code>, my function receives a lazy view of the event instead of a dictionary: the event is parsed, but Python objects are created only for the values I access. The lazy approach was first developed by <a href="https://github.com/TkTech">Tyler Kennedy</a> in another simdjson binding, <a href="https://github.com/TkTech/pysimdjson">pysimdjson</a>.</p>
<p>How does it compare? I tried the usual suspects: the standard library (<code>json</code>), <a href="https://github.com/ijl/orjson">orjson</a> and <a href="https://github.com/jcrist/msgspec">msgspec</a>. msgspec deserves special mention: it can decode JSON into typed structures that declare only the fields you need, skipping the rest. It is a lot like our lazy views, except that you must declare a schema beforehand. I gave msgspec this advantage. In each case, I verify that we get exactly the same counts.</p>
<p>I use a server with two Intel Xeon Gold 6548N processors (64 cores), Python 3.14.6, and the current versions of the libraries (fastsimdjson 0.6.0, orjson 3.13.0, msgspec 0.22.0). The data is decompressed in memory before the timer starts. I report the best of seven runs, and one run for the slow methods: the single-thread parsers, and the standard library on 64 threads. Let us first use a single core.</p>
<p><img decoding="async" class="alignnone size-large" alt="One core" src="https://lemire.me/blog/wp-content/uploads/2026/10/pyjson-one-core.webp" /></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">method</th>
<th style="text-align: right; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">GB/s</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">json (standard library)</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.18</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">orjson</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.42</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">fastsimdjson <code>map_many</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.63</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">msgspec (typed structures)</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.29</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">fastsimdjson <code>map_many</code> (lazy)</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">2.13</td>
</tr>
</tbody>
</table>
<p>The standard library needs 42 seconds to go through the eight hours. With fastsimdjson and lazy views, it takes 3.6 seconds. Notice how the methods that materialize everything (json, orjson and fastsimdjson without lazy views) are slow: most of the time goes into creating Python objects that we discard right away.</p>
<p>Now let us use all 64 cores, still in one process. On free-threaded Python, I use a thread pool for json and msgspec, and <code>map_many(..., threads=64)</code> for fastsimdjson. The threads share the input.</p>
<p><img decoding="async" class="alignnone size-large" alt="64 cores" src="https://lemire.me/blog/wp-content/uploads/2026/10/pyjson-64-cores.webp" /></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">method</th>
<th style="text-align: right; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">GB/s</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">json (standard library), 64 threads</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">3.17</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">msgspec (typed structures), 64 threads</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">24.6</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">fastsimdjson <code>map_many</code>, 64 threads</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">29.8</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">fastsimdjson <code>map_many</code> (lazy), 64 threads</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">65.7</td>
</tr>
</tbody>
</table>
<p>We go through 7.6 GB of JSON in about 0.12 seconds, at 66 GB/s, in one Python process. That is 31 times faster than with one core. It is faster than msgspec using 64 threads (25 GB/s).</p>
<p>So, can you parse JSON at tens of gigabytes per second in Python? Yes.</p>
<p>fastsimdjson 0.6.0 is available on <a href="https://pypi.org/project/fastsimdjson/">PyPI</a>. My <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/pyjson">benchmark is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/12/parsing-json-at-tens-of-gigabytes-per-second-in-python/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>WebSocket servers in JavaScript: Node.js, Bun and Deno in 2026</title>
		<link>https://lemire.me/blog/2026/10/10/websocket-servers-in-javascript-node-js-bun-and-deno-in-2026/</link>
					<comments>https://lemire.me/blog/2026/10/10/websocket-servers-in-javascript-node-js-bun-and-deno-in-2026/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 10 Oct 2026 08:56:11 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23043</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/websocket-bin-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />WebSocket is a standard protocol to hook up a client and a server in a persistent manner. It is commonly used for live chat, notifications, multiplayer games, collaborative editing, and streaming dashboards. Many servers are written in JavaScript and use Node.js, Bun or Deno as an engine. A few years ago, I wrote a simple &#8230; <a href="https://lemire.me/blog/2026/10/10/websocket-servers-in-javascript-node-js-bun-and-deno-in-2026/" class="more-link">Continue reading <span class="screen-reader-text">WebSocket servers in JavaScript: Node.js, Bun and Deno in 2026</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/websocket-bin-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>WebSocket is a standard protocol to hook up a client and a server in a persistent manner. It is commonly used for live chat, notifications, multiplayer games, collaborative editing, and streaming dashboards. Many servers are written in JavaScript and use Node.js, Bun or Deno as an engine.</p>
<p>A few years ago, I wrote <a href="https://lemire.me/blog/2023/11/25/a-simple-websocket-benchmark-in-javascript-node-js-versus-bun/">a simple WebSocket benchmark in JavaScript</a>. Two clients exchange messages through a server: client 1 sends a message to the server, the server forwards it to client 2, client 2 answers, and the server forwards the answer back to client 1. I found that Bun could be twice as fast as Node.js. That result was wrong. The Bun server sent each message back to the sender, so a round trip was two messages instead of four. Since then, Anthropic bought Bun. Also, Cloudflare bought Deno.</p>
<p>My benchmark had an issue: my clients were written in JavaScript, and they were slow. So I wrote a client in C. Doing so by hand in 2023 was expensive, but in 2026, my AI friends can write anything in C.</p>
<p>I run a single <em>ping-pong</em> test: a single pair of clients chats through the server.</p>
<p>I compare three servers against a small C relay:</p>
<ul>
<li>Node.js 26 with <a href="https://github.com/uNetworking/uWebSockets.js">uWebSockets.js</a>, a native addon written in C++;</li>
<li>Bun 1.4.2 with its built-in server (<code>Bun.serve</code>);</li>
<li>Deno 2.9.7 with its built-in server (<code>Deno.serve</code>).</li>
</ul>
<p>My test machine is a Linux server with an Intel Xeon Gold 6548N (Emerald Rapids). Everything runs locally over the loopback interface. The server is restricted to a single core, and the client runs on another core. I repeated the whole benchmark three times: the results are stable, within a few percent.</p>
<p>I report the number of round trips per second and the time per round trip (median and 99th percentile).</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">server</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">round trips/s</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">median (µs)</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">99th percentile (µs)</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">C relay (reference)</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">54,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">18.4</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">23.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Node.js 26 + uWebSockets.js</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">49,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">20.4</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">24.8</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Deno 2.9.7 + Deno.serve</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">40,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">25.0</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">32.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Bun 1.4.2 + Bun.serve</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">37,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">26.8</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">46.4</td>
</tr>
</tbody>
</table>
<p><img decoding="async" class="alignnone size-large" alt="Round trips per second" src="https://lemire.me/blog/wp-content/uploads/2026/10/websocket-roundtrips.webp" /></p>
<p><img decoding="async" class="alignnone size-large" alt="Time per round trip" src="https://lemire.me/blog/wp-content/uploads/2026/10/websocket-latency.webp" /></p>
<p>A JavaScript WebSocket server can get close to a simple C server in this test. Node.js 26 with uWebSockets.js is quite a bit faster than Bun and Deno: 49,000 round trips per second, against 40,000 for Deno and 37,000 for Bun. The 99th percentile is 24.8 µs, against 32.0 µs and 46.4 µs.</p>
<p>I repeated the same ping-pong test with the JavaScript <a href="https://github.com/websockets/ws"><code>ws</code></a> package, version 8.22.0, on all three runtimes. </p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">server</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">round trips/s</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">median (µs)</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">99th percentile (µs)</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Node.js 26 + ws</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">45,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">21.7</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">31.5</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Bun 1.4.2 + ws</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">41,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">22.5</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">36.9</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Deno 2.9.7 + ws</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">34,000</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">29.6</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">45.9</td>
</tr>
</tbody>
</table>
<p><img decoding="async" class="alignnone size-large" alt="Round trips per second with the ws package" src="https://lemire.me/blog/wp-content/uploads/2026/10/ws-roundtrips.webp" /></p>
<p><img decoding="async" class="alignnone size-large" alt="Time per round trip with the ws package" src="https://lemire.me/blog/wp-content/uploads/2026/10/ws-latency.webp" /></p>
<p>Node.js is ahead, then Bun, then Deno. On Node.js, <code>ws</code> is close to uWebSockets.js (45,000 against 49,000). On Bun, <code>ws</code> beats <code>Bun.serve</code> (41,000 against 37,000). On Deno, <code>ws</code> is behind <code>Deno.serve</code> (34,000 against 40,000).</p>
<p><strong>My source code is <a href="https://github.com/lemire/jswebsocket_bench">available</a>.</strong></p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/10/websocket-servers-in-javascript-node-js-bun-and-deno-in-2026/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Faster Python startup with lazy imports</title>
		<link>https://lemire.me/blog/2026/10/09/faster-python-startup-with-lazy-imports/</link>
					<comments>https://lemire.me/blog/2026/10/09/faster-python-startup-with-lazy-imports/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Fri, 09 Oct 2026 03:15:06 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23032</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/lazy-import-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />When a Python program starts, it needs to load all its dependencies (import). With the upcoming version of Python, it is possible to use lazy imports instead. lazy import json lazy from decimal import Decimal In this instance, the name json is no longer the module, but a mere placeholder object. The actual import only &#8230; <a href="https://lemire.me/blog/2026/10/09/faster-python-startup-with-lazy-imports/" class="more-link">Continue reading <span class="screen-reader-text">Faster Python startup with lazy imports</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/lazy-import-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>When a Python program starts, it needs to load all its dependencies (<code>import</code>). With the upcoming version of Python, it is possible to use <code>lazy</code> imports instead.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">lazy</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">import</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">json</span>
<span style="color: #204a87; font-weight: bold;">lazy</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">from</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">decimal</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">import</span> <span style="color: #000;">Decimal</span>
</code></pre>
</div>
<p>In this instance, the name <code>json</code> is no longer the module, but a mere placeholder object. The actual import only happens when you use <code>json</code>. If you never use <code>json</code>, the module is never loaded.</p>
<p>How much does it help? Let us measure.</p>
<p>I wrote a toy command-line tool. It imports sixteen modules: <code>json</code>, <code>csv</code>, <code>decimal</code>, <code>sqlite3</code>, <code>asyncio</code>, <code>email.parser</code>, <code>http.client</code>, <code>urllib.request</code>, <code>xml.etree.ElementTree</code>, <code>zipfile</code>, <code>tarfile</code>, <code>statistics</code> and some popular third-party packages (<code>numpy</code>, <code>pandas</code>, <code>requests</code>, <code>rich</code>). The tool has three paths:</p>
<ul>
<li><code>--version</code> prints a string and needs nothing;</li>
<li><code>mean</code> reads a small CSV file with the <code>csv</code> module and calls <code>statistics.fmean</code>;</li>
<li><code>stats</code> loads the same file with pandas and prints a summary.</li>
</ul>
<p>The lazy version of the tool is identical, except that I prefix the imports with <code>lazy</code>. It is a one-word change per line.</p>
<p>I use Python 3.15 (3.15.0b4) on an Intel Xeon Gold 6548N (Emerald Rapids), with numpy 2.5, pandas 3.0 and requests 2.34.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">command</th>
<th style="text-align: right;">eager imports</th>
<th style="text-align: right;">lazy imports</th>
<th style="text-align: right;">speedup</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>--version</code></td>
<td style="text-align: right;">295 ms</td>
<td style="text-align: right;">20 ms</td>
<td style="text-align: right;">15×</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>mean</code></td>
<td style="text-align: right;">296 ms</td>
<td style="text-align: right;">24 ms</td>
<td style="text-align: right;">12×</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>stats</code></td>
<td style="text-align: right;">299 ms</td>
<td style="text-align: right;">224 ms</td>
<td style="text-align: right;">1.3×</td>
</tr>
</tbody>
</table>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/10/lazy-startup.webp" alt="Eager imports take about 300 ms. Lazy imports take 20 ms for --version, 24 ms for mean, and 224 ms for stats." class="alignnone size-large" /></p>
<p>Printing the version number goes from 295 ms to 20 ms. If you subtract the interpreter startup (11.7 ms), the cost of the imports goes from 283 ms to about 8 ms: a 35-fold reduction. The <code>mean</code> command, which needs a few standard modules, is twelve times faster.</p>
<p>There is a downside: errors move. With an eager import, a missing module fails at startup. With a lazy import, it fails at the first use, perhaps deep inside a function, perhaps hours later in a long-running server.</p>
<p>My <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/lazy">source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/09/faster-python-startup-with-lazy-imports/feed/</wfw:commentRss>
			<slash:comments>4</slash:comments>
		
		
			</item>
		<item>
		<title>Mastering SIMD with Java Vector API</title>
		<link>https://lemire.me/blog/2026/10/07/mastering-simd-with-java-vector-api/</link>
					<comments>https://lemire.me/blog/2026/10/07/mastering-simd-with-java-vector-api/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 07 Oct 2026 02:07:05 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23008</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/mastering-simd-java-vector-api-150x150.webp" class="webfeedsFeaturedVisual wp-post-image" alt="Cover of Mastering SIMD with Java Vector API by Roman Snytsar" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />I write a lot about data parallelism. About how our processors have special instructions that can do several operations at once. They are often critical to get the best performance out of a CPU. Compilers are sometimes able to take advantage of these instructions. But it is hard to tell when it will work. For &#8230; <a href="https://lemire.me/blog/2026/10/07/mastering-simd-with-java-vector-api/" class="more-link">Continue reading <span class="screen-reader-text">Mastering SIMD with Java Vector API</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/mastering-simd-java-vector-api-150x150.webp" class="webfeedsFeaturedVisual wp-post-image" alt="Cover of Mastering SIMD with Java Vector API by Roman Snytsar" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>I write a lot about data parallelism. About how our processors have special instructions that can do several operations at once. They are often critical to get the best performance out of a CPU.</p>
<p>Compilers are sometimes able to take advantage of these instructions. But it is hard to tell when it will work.</p>
<p>For Java programmers, you can write directly for data parallelism since JDK 16. With enough skill, you might multiply the performance of a few key functions. </p>
<p>The Vector API is in the incubator module <code>jdk.incubator.vector</code>. It is still not officially supported, but you can enable it by passing <code>--add-modules=jdk.incubator.vector</code>. I expect that it will soon become a mainstream feature in Java.</p>
<p>Roman Snytsar wrote a book about it: <em>Mastering SIMD with Java Vector API</em>, and the subtitle is <a href="https://www.amazon.com/dp/B0GPBQNTJ5"><em>Unlocking Single-Core Performance Through SIMD Optimization</em></a>. I had the honor of being the technical reviewer. I learned a few things while reading the book and I enjoyed it.</p>
<p>Who has time for books, especially technical books?</p>
<p>What a book offers is time to reflect. Reading technical books today is probably just as relevant as it ever was. The author takes you on a story&#8230; and gets you to think. </p>
<p><img decoding="async" class="alignnone size-large" alt="Cover of Mastering SIMD with Java Vector API by Roman Snytsar" src="https://lemire.me/blog/wp-content/uploads/2026/10/mastering-simd-java-vector-api.webp" /></p>
<p>Snytsar&#8217;s book works from  problems like sum an array, compute a mean and a standard deviation, remove duplicates, merge two sorted arrays, and so forth. A lot of them are similar to the problems you will encounter as a programmer.  He starts from the ordinary solution and rebuilds it with the Java Vector API. He then runs benchmarks. He looks at the assembly code.</p>
<p>The book covers performance issues such as unrolling, asymmetric loops, structural hazards, dependencies. Snytsar shows us that, in some instances, data parallelism can fail to bear fruits. The negative lessons are just as important as the positive ones.</p>
<p>The Java Vector API is a thick layer of abstractions, but Snytsar shows that some tricks work better on some hardware than others.  I liked the chapter on removing duplicates, which is built on <code>compress</code>. It is one instance where the specific hardware matters. And you are unlikely to just find out about it yourself.</p>
<p>It is a book worth buying if you are a Java programmer with a focus on performance. </p>
<p>The book: Roman Snytsar, <a href="https://www.amazon.com/dp/B0GPBQNTJ5"><em>Mastering SIMD with Java Vector API</em>,</a> Apress, 2026.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/07/mastering-simd-with-java-vector-api/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>Faster software linking with mold</title>
		<link>https://lemire.me/blog/2026/10/06/linking-node-js-with-mold/</link>
					<comments>https://lemire.me/blog/2026/10/06/linking-node-js-with-mold/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Tue, 06 Oct 2026 08:00:20 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23017</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8cr2v98cr2v98cr2-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />When you build a program, the compiler turns each source file into an object file. Then a linker stitches all the object files and libraries into one executable. On Linux, the default linker is usually GNU ld (also called bfd). The GNU binutils also ship gold, an alternative linker that was designed to be faster. &#8230; <a href="https://lemire.me/blog/2026/10/06/linking-node-js-with-mold/" class="more-link">Continue reading <span class="screen-reader-text">Faster software linking with mold</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8cr2v98cr2v98cr2-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>When you build a program, the compiler turns each source file into an object file. Then a linker stitches all the object files and libraries into one executable.</p>
<p>On Linux, the default linker is usually GNU ld (also called bfd). The GNU binutils also ship gold, an alternative linker that was designed to be faster. Gold was built at Google and first made available in 2008.</p>
<p>The <a href="https://github.com/rui314/mold">mold</a> linker is a newer linker written by Rui Ueyama. It was first released in 2021. It is meant as a drop-in replacement for GNU ld, and its main selling point is speed. The latest release is version 3.</p>
<p>How much faster is it on a large project? I built <a href="https://github.com/nodejs/node">Node.js</a> from its main branch on an Intel Xeon Gold 6548N server (Emerald Rapids, two sockets, 64 cores and 128 threads) running Linux with GCC 14.3. The final <code>node</code> executable weighs about 160 MB.</p>
<p>I captured the command that links the <code>node</code> executable and ran it six times with each linker. I report the median.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">linker</th>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">time to link node</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">GNU ld (bfd) 2.41</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">2.52 s</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">GNU gold 2.41</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">1.46 s</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">mold 3.0.0, 1 thread</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.49 s</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">mold 3.0.0, 8 threads</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.13 s</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">mold 3.0.0, 128 threads</td>
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">0.11 s</td>
</tr>
</tbody>
</table>
<p>
<a href="https://lemire.me/blog/wp-content/uploads/2026/10/link_big4.png"><img loading="lazy" decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/10/link_big4-1024x614.png" alt="" width="660" height="396" class="alignnone size-large wp-image-23023" srcset="https://lemire.me/blog/wp-content/uploads/2026/10/link_big4-1024x614.png 1024w, https://lemire.me/blog/wp-content/uploads/2026/10/link_big4-300x180.png 300w, https://lemire.me/blog/wp-content/uploads/2026/10/link_big4-768x461.png 768w, https://lemire.me/blog/wp-content/uploads/2026/10/link_big4.png 1050w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>With multithreading, mold is about 24 times faster than GNU ld and about 14 times faster than gold.</p>
<p>Even if I restrict mold to a single thread, it links Node.js in half a second, five times faster than GNU ld. With only 8 threads, you get nearly all of the benefit.</p>
<p>Using mold is easy. You can pass <code>-fuse-ld=mold</code> to GCC or clang. Or you can wrap your whole build:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>mold<span style="color: #f8f8f8;"> </span>-run<span style="color: #f8f8f8;"> </span>make<span style="color: #f8f8f8;"> </span>-j128
</code></pre>
</div>
<p>The <code>-run</code> option intercepts every call to the default linker and redirects it to mold. You do not need to change the build scripts.</p>
<p>Of course, two seconds saved on a link does not matter much if you build Node.js once. But a developer who edits a file and rebuilds dozens of times a day pays the link cost each time. With mold, the link step becomes nearly free.</p>
<p>My scripts and raw results are <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/10/mold">available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/06/linking-node-js-with-mold/feed/</wfw:commentRss>
			<slash:comments>6</slash:comments>
		
		
			</item>
		<item>
		<title>Ephemeral testing</title>
		<link>https://lemire.me/blog/2026/10/05/ephemeral-testing/</link>
					<comments>https://lemire.me/blog/2026/10/05/ephemeral-testing/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 05 Oct 2026 08:00:51 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=23013</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" srcset="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-150x150.jpg 150w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-300x300.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-768x768.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9.jpg 1024w" sizes="auto, (max-width: 150px) 100vw, 150px" />We have many ways to ensure software quality. Unit testing. Fuzz testing. Integration testing. And so forth.   I&#8217;d like to propose a method that was unthinkable before: ephemeral testing. (Ephemeral is a fancy word for &#8216;throw away&#8217; or &#8216;temporary&#8217;.)   You write your code. You build your software component. Or the AI agent does &#8230; <a href="https://lemire.me/blog/2026/10/05/ephemeral-testing/" class="more-link">Continue reading <span class="screen-reader-text">Ephemeral testing</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" srcset="https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-150x150.jpg 150w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-300x300.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-768x768.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9.jpg 1024w" sizes="auto, (max-width: 150px) 100vw, 150px" /><div class="" data-block="true" data-editor="51ceu" data-offset-key="9ps6d-0-0">
<div data-offset-key="9ps6d-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9ps6d-0-0">We have many ways to ensure software quality. Unit testing. Fuzz testing. Integration testing. And so forth.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="bkkb0-0-0">
<div data-offset-key="bkkb0-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="bkkb0-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="n0f7-0-0">
<div data-offset-key="n0f7-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="n0f7-0-0">I&#8217;d like to propose a method that was unthinkable before: </span><span data-offset-key="n0f7-0-1">ephemeral testing</span><span data-offset-key="n0f7-0-2">. (Ephemeral is a fancy word for &#8216;throw away&#8217; or &#8216;temporary&#8217;.)</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="6f3i2-0-0">
<div data-offset-key="6f3i2-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="6f3i2-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="5q75-0-0">
<div data-offset-key="5q75-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5q75-0-0">You write your code. You build your software component. Or the AI agent does it for you, it does not matter.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="43dkq-0-0">
<div data-offset-key="43dkq-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="43dkq-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="f23fg-0-0">
<div data-offset-key="f23fg-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="f23fg-0-0">Then you ask an AI agent to build on it: an application, another layer, maybe several. You have it test what it built. You do not assess the original work directly. You assess how good the software built on top of it is.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="9k6bp-0-0">
<div data-offset-key="9k6bp-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9k6bp-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="9m917-0-0">
<div data-offset-key="9m917-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9m917-0-0">It is a form of integration testing. The difference is that the software on top is entirely ephemeral. You throw it away when you are done.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="n0mf-0-0">
<div data-offset-key="n0mf-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="n0mf-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="5r99t-0-0">
<div data-offset-key="5r99t-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5r99t-0-0">A library with a clean API, stable invariants, and useful errors lets the agent produce something that works quickly. A library with hidden state, surprising defaults, or incomplete docs produces a pile of patches and failures. The failures are evidence about your code, not about the agent.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="5klhj-0-0">
<div data-offset-key="5klhj-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5klhj-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="ac4qd-0-0">
<div data-offset-key="ac4qd-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="ac4qd-0-0">You can repeat it. Different agents, different tasks, same foundation.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="3bk3f-0-0">
<div data-offset-key="3bk3f-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="3bk3f-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="7uv4n-0-0">
<div data-offset-key="7uv4n-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="7uv4n-0-0">In effect, instead of building the core while trying to anticipate what might be needed at the other layers, you just simulate the other layers by actually building them.</span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="3t8kv-0-0">
<div data-offset-key="3t8kv-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="3t8kv-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="2hmec-0-0">
<div data-offset-key="2hmec-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="2hmec-0-0">Of course, you could argue that with AI, you can rebuild everything whenever you need to. But that&#8217;s not practical. You need some form of stability. </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="c1lk0-0-0">
<div data-offset-key="c1lk0-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="c1lk0-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="51ceu" data-offset-key="a1c0t-0-0">
<div data-offset-key="a1c0t-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="a1c0t-0-0">I have been applying this trick to various projects. As I consider a new feature, I ask my AI to prototype quickly what I might later build based on what I am doing. Ephemeral testing works for me thus far.</span></div>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/05/ephemeral-testing/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>Research paper overload: submissions capped at two a month</title>
		<link>https://lemire.me/blog/2026/10/04/arxiv-capped-submissions-at-two-a-month/</link>
					<comments>https://lemire.me/blog/2026/10/04/arxiv-capped-submissions-at-two-a-month/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sun, 04 Oct 2026 17:07:27 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23004</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/arxiv-limit-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Much of my research is on arXiv. I was one of the early adopters. It is simply a repository of research papers with PDF and metadata. It makes it convenient to find research without any paywall. Physicists started it in 1991. Math and computer science followed gradually. It relies on volunteers to moderate it, because &#8230; <a href="https://lemire.me/blog/2026/10/04/arxiv-capped-submissions-at-two-a-month/" class="more-link">Continue reading <span class="screen-reader-text">Research paper overload: submissions capped at two a month</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/arxiv-limit-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Much of my research is on <a href="https://arxiv.org">arXiv</a>. I was one of the early adopters. It is simply a repository of research papers with PDF and metadata.</p>
<p>It makes it convenient to find research without any paywall.</p>
<p>Physicists started it in 1991. Math and computer science followed gradually.</p>
<p>It relies on volunteers to moderate it, because we don&#8217;t want garbage or spam.</p>
<p>On October 1, they <a href="https://blog.arxiv.org/2026/10/01/updated-rate-limit-policy/">capped the submissions to two per person per month</a>.</p>
<p>Why?</p>
<p>arXiv got over 40,000 submissions in September.</p>
<p>Two years ago, it was about 20,000.</p>
<p>Doubling every two years. It is unsustainable for human beings.</p>
<p><img decoding="async" class="alignnone size-large" alt="Papers posted to arXiv each month" src="https://lemire.me/blog/wp-content/uploads/2026/10/arxiv-monthly.webp" /></p>
<p>In related news, <a href="https://x.com/GoogleVRP/status/2105689195180179605">Google stopped taking new bug reports in its open-source bounty program</a>. They could not cope with the influx.</p>
<p>The writing is on the wall.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/04/arxiv-capped-submissions-at-two-a-month/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>Google&#8217;s first orbital data center is in orbit</title>
		<link>https://lemire.me/blog/2026/10/04/googles-first-orbital-data-center-is-in-orbit/</link>
					<comments>https://lemire.me/blog/2026/10/04/googles-first-orbital-data-center-is-in-orbit/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sun, 04 Oct 2026 16:49:29 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=23002</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/orbital-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Google&#8217;s first orbital data center is in orbit (October 1st). They say it is about the size of a fridge. It has one kilowatt of solar power. It is about the power of a household. It was launched on a Falcon 9 rocket (SpaceX). It should run for about a year. Google plans to launch &#8230; <a href="https://lemire.me/blog/2026/10/04/googles-first-orbital-data-center-is-in-orbit/" class="more-link">Continue reading <span class="screen-reader-text">Google&#8217;s first orbital data center is in orbit</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/orbital-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p><a href="https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-prototype/">Google&#8217;s first orbital data center</a> is in orbit (October 1st). They say it is about the size of a fridge.</p>
<p>It has one kilowatt of solar power. It is about the power of a household.</p>
<p>It was launched on a Falcon 9 rocket (SpaceX).</p>
<p>It should run for about a year. Google plans to launch two more orbital data centres next year. The plan is to use laser communications.</p>
<p>For people complaining that the latency is going to be a problem with orbital data centres…</p>
<p>An orbital data centre can be effectively in your line of sight and it can be relatively close (much closer than the diameter of the USA).</p>
<p><img decoding="async" class="alignnone size-large" alt="Orbital latency compared with California to New York" src="https://lemire.me/blog/wp-content/uploads/2026/10/orbital-latency.webp" /></p>
<p><a href="https://lemire.me/blog/wp-content/uploads/2026/10/image.gif"><img loading="lazy" decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/10/image.gif" alt="" width="800" height="450" class="alignnone size-full wp-image-23028" /></a><br />
Further, if you have a cluster of space data centres, they can remain close and in a line of sight.</p>
<p>So you are not going to run your video games or do high frequency trading from space, but for compute, latency is unlikely to be the bottleneck.</p>
<p>The great features are space (no need to wait five years for a permit) and power (no need to negotiate with a local government for electricity). Solar power in space is great because it runs 24h a day.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/04/googles-first-orbital-data-center-is-in-orbit/feed/</wfw:commentRss>
			<slash:comments>7</slash:comments>
		
		
			</item>
		<item>
		<title>Parsing compressed JSON at 40 GB/s</title>
		<link>https://lemire.me/blog/2026/10/01/parsing-compressed-json-at-40-gb-s/</link>
					<comments>https://lemire.me/blog/2026/10/01/parsing-compressed-json-at-40-gb-s/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 01 Oct 2026 03:10:29 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22991</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/jsonstream-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Parsing compressed JSON at 40 GB/s" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />A common way to store JSON data is to write one document per line. We call it NDJSON or JSON Lines. Log files, database exports and machine-learning datasets often come in this format. The files can be large, so we may compress them. {"id":1,"active":true,"user":{"name":"user_1","tags":["guest"]},"score":-3862,"note":"..."} {"id":2,"active":false,"user":{"name":"user_2","tags":["staff","admin"]},"score":8123,"note":"..."} How fast can you read such a file? I wrote &#8230; <a href="https://lemire.me/blog/2026/10/01/parsing-compressed-json-at-40-gb-s/" class="more-link">Continue reading <span class="screen-reader-text">Parsing compressed JSON at 40 GB/s</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/10/jsonstream-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Parsing compressed JSON at 40 GB/s" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>A common way to store JSON data is to write one document per line. We call it NDJSON or JSON Lines. Log files, database exports and machine-learning datasets often come in this format. The files can be large, so we may compress them.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000; font-weight: bold;">{</span><span style="color: #204a87; font-weight: bold;">"id"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"active"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #204a87; font-weight: bold;">true</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"user"</span><span style="color: #000; font-weight: bold;">:{</span><span style="color: #204a87; font-weight: bold;">"name"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #4e9a06;">"user_1"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"tags"</span><span style="color: #000; font-weight: bold;">:[</span><span style="color: #4e9a06;">"guest"</span><span style="color: #000; font-weight: bold;">]},</span><span style="color: #204a87; font-weight: bold;">"score"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #0000cf; font-weight: bold;">-3862</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"note"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #4e9a06;">"..."</span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">{</span><span style="color: #204a87; font-weight: bold;">"id"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #0000cf; font-weight: bold;">2</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"active"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #204a87; font-weight: bold;">false</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"user"</span><span style="color: #000; font-weight: bold;">:{</span><span style="color: #204a87; font-weight: bold;">"name"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #4e9a06;">"user_2"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"tags"</span><span style="color: #000; font-weight: bold;">:[</span><span style="color: #4e9a06;">"staff"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #4e9a06;">"admin"</span><span style="color: #000; font-weight: bold;">]},</span><span style="color: #204a87; font-weight: bold;">"score"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #0000cf; font-weight: bold;">8123</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #204a87; font-weight: bold;">"note"</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #4e9a06;">"..."</span><span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>How fast can you read such a file? I wrote a small demo with <a href="https://github.com/simdjson/simdjson">simdjson</a>. My test file has 5 million records: 812 MB of NDJSON. For each record, I read a few fields. I count the active records, and I sum the scores of the active records whose user has the <code>admin</code> tag.</p>
<p>You could decompress the whole file, but it might be better to process the compressed file. So you decompress a chunk, you parse the complete lines in that chunk, and you move the incomplete last line to the front of the buffer. In NDJSON, a newline can never appear inside a document: a newline in a string must be escaped as <code>n</code>. So you can always cut the buffer right after its last newline character. The simdjson library has a function for many documents in one buffer (<code>iterate_many</code>):</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">simdjson</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">ondemand</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">parser</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">parser</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">while</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87;">true</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// fill the buffer with decompressed bytes...</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">size_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">eof</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">?</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">len</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">last_newline</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">len</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">simdjson</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">ondemand</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">document_stream</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">stream</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">parser</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">iterate_many</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #000; font-weight: bold;">).</span><span style="color: #000;">get</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">stream</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">doc</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">stream</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">accumulate</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">doc</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">value_unsafe</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">result</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">eof</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">break</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">memmove</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">buf</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">len</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">len</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">cut</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>I ran my benchmarks on an Intel Xeon Gold 6548N server (Emerald Rapids) with two sockets, 64 cores and 128 threads. I use GCC 14.</p>
<p>A gzip file is one long compressed stream. The decompressor needs the previous 32 KiB of output to decode what comes next. So you must decompress the file from the start, with one thread. I can still parse in other threads: one thread decompresses chunks and puts them in a queue, and the other threads parse them. It doubles the speed to 2.5 GB/s.</p>
<p>Other formats do better. A zstd or an lz4 file can be made of many independent <em>frames</em>, one after the other. It is still a regular <code>.zst</code> or <code>.lz4</code> file: the usual command-line tools decompress it as usual.</p>
<p>My program writes a new frame every 256 KiB of JSON, always after a newline. Each frame stores its decompressed size and a checksum. To find where a frame ends, you only need to read a few block headers. You do not need to decompress anything. So the threads can take frames one by one, and each thread decompresses and parses its own frames.</p>
<p><img decoding="async" alt="Decompressing and parsing NDJSON" src="https://lemire.me/blog/wp-content/uploads/2026/10/jsonstream-formats.webp" class="alignnone size-large" /></p>
<p>With one thread, zstd and lz4 are no faster than gzip: about 1.1 GB/s. With 64 threads, I get 40 GB/s with zstd and 34 GB/s with lz4. That is 16 times faster than the best I can do with gzip.</p>
<p>The files are not much larger. Small frames compress a bit worse, since each frame starts from scratch, but the zstd file is only 6% larger than the gzip file.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">file</th>
<th style="text-align: right;">size</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">NDJSON</td>
<td style="text-align: right;">812.0 MB</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">gzip (one stream)</td>
<td style="text-align: right;">56.9 MB</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">zstd (256 KiB frames)</td>
<td style="text-align: right;">60.6 MB</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">lz4 (256 KiB frames)</td>
<td style="text-align: right;">108.5 MB</td>
</tr>
</tbody>
</table>
<p>My data is synthetic. The records are very repetitive, which makes decompression fast. Your data might decompress more slowly.</p>
<p>If you control how your JSON files are written, consider using zstd with many frames. You get files about as small as with gzip, and you can read them many times faster with multiple threads.</p>
<p><strong>Source code</strong>: <a href="https://github.com/simdjson/simdjson_compressed_demo">https://github.com/simdjson/simdjson_compressed_demo</a>. The benchmark numbers and the plotting script are in <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/jsonstream">my blog repository</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/10/01/parsing-compressed-json-at-40-gb-s/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Transcoding UTF-8 to UTF-16 with replacement at gigabytes per second</title>
		<link>https://lemire.me/blog/2026/09/30/transcoding-utf-8-to-utf-16-with-replacement-at-gigabytes-per-second/</link>
					<comments>https://lemire.me/blog/2026/09/30/transcoding-utf-8-to-utf-16-with-replacement-at-gigabytes-per-second/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 30 Sep 2026 12:33:27 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22987</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/withreplacement-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Transcoding UTF-8 to UTF-16 with replacement" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Our software represents strings using the UTF-16 or the UTF-8 formats. Most text on the web is UTF-8, but Java, C# or JavaScript represents the strings as UTF-16 to the programmer. Sometimes we need to transcode (convert) strings. Your browser probably uses the simdutf library for validating or transcoding. It is part of the widely &#8230; <a href="https://lemire.me/blog/2026/09/30/transcoding-utf-8-to-utf-16-with-replacement-at-gigabytes-per-second/" class="more-link">Continue reading <span class="screen-reader-text">Transcoding UTF-8 to UTF-16 with replacement at gigabytes per second</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/withreplacement-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Transcoding UTF-8 to UTF-16 with replacement" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Our software represents strings using the UTF-16 or the UTF-8 formats. Most text on the web is UTF-8, but Java, C# or JavaScript represents the strings as UTF-16 to the programmer.</p>
<p>Sometimes we need to transcode (convert) strings. Your browser probably uses the simdutf library for validating or transcoding. It is part of the widely used V8 JavaScript engine.</p>
<p>Up until a few days ago, the simdutf library missed one key feature: if the UTF-8 input is invalid, it did not know how to transcode it to UTF-16. The objective is to replace ill-formed UTF-8 sequences with the replacement character U+FFFD. It is the character that shows up as a weird question mark sometimes. You may have seen it in a broken web site.</p>
<p>The new function <code>convert_utf8_to_utf16_with_replacement</code> always succeeds. It must be used in conjunction with the function <code>utf16_length_from_utf8_with_replacement</code>, which scans and validates the input, recording the offsets of errors if there are some. The usage is as follows.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">source</span><span style="color: #000; font-weight: bold;">[]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #4e9a06;">'c'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'a'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'f'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'\xff'</span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #204a87; font-weight: bold;">size_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">length</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000;">simdutf</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">utf8_to_utf16_result</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">res</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">simdutf</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">utf16_length_from_utf8_with_replacement</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">source</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">length</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">unique_ptr</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">char16_t</span><span style="color: #000; font-weight: bold;">[]</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">utf16</span><span style="color: #000; font-weight: bold;">{</span><span style="color: #204a87; font-weight: bold;">new</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char16_t</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">res</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">count</span><span style="color: #000; font-weight: bold;">]};</span>
<span style="color: #204a87; font-weight: bold;">size_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">written</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">simdutf</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">convert_utf8_to_utf16_with_replacement</span><span style="color: #000; font-weight: bold;">(</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">source</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">length</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">utf16</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">get</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">res</span><span style="color: #000; font-weight: bold;">);</span>
</code></pre>
</div>
<p>Because we have a list of errors before even beginning the transcoding, the bytes between the recorded offsets are valid UTF-8, so we can transcode faster. Each recorded error becomes one U+FFFD.</p>
<p>I timed a scalar decoder against the library on Japanese Wikipedia. The file is 531 KB. I repeated it so the timed buffer is 1.1 MB. I use one core of a Xeon Gold 6548N. GCC 14.3.1 at <code>-O3</code>, best of eight runs.</p>
<p><img decoding="async" alt="UTF-8 to UTF-16 with replacement against a scalar decoder, Japanese Wikipedia" src="https://lemire.me/blog/wp-content/uploads/2026/09/withreplacement-japanese.webp" class="alignnone size-large" /></p>
<p>With no errors, simdutf reaches 7.0 GB/s and the scalar loop 1.7 GB/s.</p>
<p>On valid input, the new approach may even be marginally faster than the previous method, because our initial validation pass allows us to transcode faster afterward.</p>
<p><img decoding="async" alt="Valid UTF-8 to UTF-16, ordinary conversion and conversion with replacement" src="https://lemire.me/blog/wp-content/uploads/2026/09/withreplacement-valid.webp" class="alignnone size-large" /></p>
<p><strong>Compiler</strong>: GCC 14.3.1, <code>-O3</code>, one core (<code>taskset -c 2</code>), simdutf icelake kernel. The functions are on the branch <a href="https://github.com/simdutf/simdutf/tree/utf8-to-utf16-with-replacement"><code>utf8-to-utf16-with-replacement</code></a>.</p>
<p><strong>Credit</strong>: The most non-trivial part of this routine was coded by Benjamin Bucher over the summer.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/30/transcoding-utf-8-to-utf-16-with-replacement-at-gigabytes-per-second/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How do you deal with people who strongly disagree with your views?</title>
		<link>https://lemire.me/blog/2026/09/30/how-do-you-deal-with-people-who-strongly-disagree-with-your-views/</link>
					<comments>https://lemire.me/blog/2026/09/30/how-do-you-deal-with-people-who-strongly-disagree-with-your-views/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 30 Sep 2026 01:19:54 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22978</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/ohs7r-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />  1. Begin by genuinely listening to them. Make sure that you understand them. It is often useful to ask for confirmation: restate what you think they are saying and make sure that you understand. Physically, it helps to turn your shoulders so that you face them. Don&#8217;t look at your screen while they talk. &#8230; <a href="https://lemire.me/blog/2026/09/30/how-do-you-deal-with-people-who-strongly-disagree-with-your-views/" class="more-link">Continue reading <span class="screen-reader-text">How do you deal with people who strongly disagree with your views?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/ohs7r-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><div class="" data-block="true" data-editor="3lgi9" data-offset-key="37v46-0-0">
<div data-offset-key="37v46-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="9ph0p-0-0">
<div data-offset-key="9ph0p-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9ph0p-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="98jvh-0-0">
<div data-offset-key="98jvh-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="98jvh-0-0">1. Begin by genuinely listening to them. Make sure that you understand them. It is often useful to ask for confirmation: restate what you think they are saying and make sure that you understand. Physically, it helps to turn your shoulders so that you face them. Don&#8217;t look at your screen while they talk.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="4jbfc-0-0">
<div data-offset-key="4jbfc-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="4jbfc-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="dq3hi-0-0">
<div data-offset-key="dq3hi-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="dq3hi-0-0">2. Ask more questions. You can often effortlessly weaken the stance of someone who has not given much thought to an issue just by asking them questions. Don&#8217;t &#8216;challenge them&#8217;. Ask genuine questions. The objective is not to break their views with questions, but to establish the playing ground. Many people are unclear about what they think they believe. They may simply be repeating what they heard.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="5ahpf-0-0">
<div data-offset-key="5ahpf-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5ahpf-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="3aesg-0-0">
<div data-offset-key="3aesg-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="3aesg-0-0">3. Focus on what you share in common. Often, the disagreement is smaller than you think. Furthermore, you can sometimes entirely avoid a conflict by establishing strong agreement on many points.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="b6leb-0-0">
<div data-offset-key="b6leb-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="b6leb-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="22jv9-0-0">
<div data-offset-key="22jv9-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="22jv9-0-0">4. Unless you are clearly a knowledgeable person, do not immediately express your opinion. Begin by establishing your credibility. Refer to hard facts, data. Refer to people or documents you have consulted. This, in turn, should have been done ahead of time. Never engage in a debate without having done at least a bit of research and reflection.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="1vnie-0-0">
<div data-offset-key="1vnie-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="1vnie-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="bpsvg-0-0">
<div data-offset-key="bpsvg-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="bpsvg-0-0">5. Use language that fits the issue. You need to learn the vocabulary. Learn the technical terms and use them, although you may want to remain accessible, so explain them as well.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="5jluv-0-0">
<div data-offset-key="5jluv-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5jluv-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="ef9md-0-0">
<div data-offset-key="ef9md-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="ef9md-0-0">6. Tell stories. People understand by emotions as much as by their reason. Ideally, the story should be true and it should be backed by hard evidence.</span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="1bhuq-0-0">
<div data-offset-key="1bhuq-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="1bhuq-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="3lgi9" data-offset-key="3pr8k-0-0">
<div data-offset-key="3pr8k-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="3pr8k-0-0">7. Avoid personal attacks and do not tolerate them against you and others. If someone is insulting you, it is fair to say so. You may say that you feel offended. It is the one case where you can and maybe should go on the offensive. Don&#8217;t answer by an insult, but attack the fact that they insulted you. You may ask for an apology.</span></div>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/30/how-do-you-deal-with-people-who-strongly-disagree-with-your-views/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>simdjson 5.0 is out</title>
		<link>https://lemire.me/blog/2026/09/28/simdjson-5-0-is-out/</link>
					<comments>https://lemire.me/blog/2026/09/28/simdjson-5-0-is-out/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 28 Sep 2026 12:38:25 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22975</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/simdjson5-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="simdjson 5.0 is out" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />The simdjson library is a C++ library to parse and generate JSON. It is used in Node.js, ClickHouse, Meta Velox, StarRocks, Apache Doris, the Ladybird browser and many other systems. We released version 4.0 a year ago, in September 2025. Its headline feature was C++26 static reflection: you could turn a C++ structure into JSON, &#8230; <a href="https://lemire.me/blog/2026/09/28/simdjson-5-0-is-out/" class="more-link">Continue reading <span class="screen-reader-text">simdjson 5.0 is out</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/simdjson5-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="simdjson 5.0 is out" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>The <a href="https://github.com/simdjson/simdjson">simdjson</a> library is a C++ library to parse and generate JSON. It is used in Node.js, ClickHouse, Meta Velox, StarRocks, Apache Doris, the Ladybird browser and many other systems. We released version 4.0 a year ago, in September 2025. Its headline feature was C++26 static reflection: you could turn a C++ structure into JSON, and back, without writing any glue code.</p>
<p>Today we are releasing version 5.0.</p>
<ol>
<li>
<p>Static reflection is no longer guarded and is an officially supported feature. When your compiler has reflection enabled (e.g., <code>g++ -std=c++26 -freflection</code> with GCC 16), simdjson detects it by itself and the reflection-based functions become available.</p>
</li>
<li>
<p>When deserializing a C++ structure through reflection, simdjson now uses <em>key selectors</em> (see below) by default: it reads the object in a single pass, whatever the order of the keys. You can return to the previous approach (one lookup per member) with <code>-DSIMDJSON_DISABLE_KEY_SELECTOR_REFLECTION=1</code>.</p>
</li>
<li>
<p>A positive integer in <code>[2^64, 10^20)</code> is now reported as a big integer (<code>BIGINT_NUMBER</code>), like other overflowing integers, instead of a malformed number. When big integers are parsed as strings, a token such as <code>123456789123456789123x</code> is now rejected.</p>
</li>
</ol>
<p>One major change is the key selectors. A common task is to extract a few fields from a JSON object. In simdjson 5.0 (C++20 or better), you can name the keys at compile time and visit the object once:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">using</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">namespace</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">simdjson</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">json</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">R"({ "name": "Daniel", "age": 42, "city": "Montreal" })"</span><span style="color: #000;">_padded</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000;">ondemand</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">parser</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">parser</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">doc</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">parser</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">iterate</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">json</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">string_view</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">name</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">city</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">age</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">result</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">doc</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">get_object</span><span style="color: #000; font-weight: bold;">().</span><span style="color: #000;">for_each</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #4e9a06;">"name"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">"city"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">"age"</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">name</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">city</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">age</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #8f5902; font-style: italic;">// name == "Daniel", city == "Montreal", age == 42</span>
</code></pre>
</div>
<p>The keys can appear in any order. At compile time, simdjson builds a perfect hash function for your set of keys. At run time, recognizing a key takes a hash computed from a couple of bytes and one comparison. You can also pass one callback per key instead of variables. The iteration stops as soon as all keys have been found.</p>
<p>We added annotations for the data structures for automated (C++26) serialization and deserialization: <code>rename</code>, <code>rename_all</code>, <code>alias</code>, <code>skip</code>, <code>default_value</code>, <code>flatten</code>, <code>deny_unknown_fields</code>, <code>transparent</code>, and so forth.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">struct</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[[</span><span style="color: #a40000; border: 1px solid #EF2929;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #c4a000;">simdjson</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #c4a000;">rename_all</span><span style="color: #a40000; border: 1px solid #EF2929;">&lt;</span><span style="color: #c4a000;">simdjson</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #c4a000;">case_style</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #c4a000;">camel_case</span><span style="color: #a40000; border: 1px solid #EF2929;">&gt;</span><span style="color: #000; font-weight: bold;">]]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">User</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">string</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">first_name</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">int64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">user_id</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000; font-weight: bold;">[[</span><span style="color: #a40000; border: 1px solid #EF2929;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #c4a000;">simdjson</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #c4a000;">rename</span><span style="color: #a40000; border: 1px solid #EF2929;">&lt;"</span><span style="color: #c4a000;">KEY</span><span style="color: #a40000; border: 1px solid #EF2929;">"&gt;</span><span style="color: #000; font-weight: bold;">]]</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">api_key</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">};</span>
<span style="color: #8f5902; font-style: italic;">// {"firstName":"Ann","userId":7,"KEY":8}</span>
</code></pre>
</div>
<p>We often get many JSON documents in one file or one network message. simdjson has long supported streams of documents separated by white space (NDJSON). In 5.0 we added:</p>
<ul>
<li><a href="https://www.rfc-editor.org/rfc/rfc7464">RFC 7464</a> JSON text sequences (each document preceded by the record separator character) and comma-separated documents;</li>
<li><code>stream_format::newline_delimited</code>: you promise that each document sits on its own line. When you only read part of a document, simdjson jumps to the next line instead of walking over the rest of the document.</li>
<li><code>simdjson::slice_at</code>, which cuts a stream into blocks at document boundaries so you can parse the blocks on as many threads as you like. The built-in threaded mode uses at most two threads.</li>
</ul>
<p>We also fixed several bugs in <code>document_stream</code>, found in an audit by Francisco Geiman Thiesen.</p>
<p>There are many other smaller features.</p>
<ul>
<li>NaN and infinity: JSON does not allow <code>NaN</code> or <code>Infinity</code>, but many systems produce them anyway. If you define <code>SIMDJSON_ENABLE_NAN_INF</code>, simdjson parses them, and serializes them.</li>
<li>Narrow types: <code>get_uint8()</code>, <code>get_int8()</code>, <code>get_uint16()</code>, <code>get_int16()</code> check the range for you. With C++23, <code>get_float32()</code> and <code>get_float64()</code> return <code>std::float32_t</code> and <code>std::float64_t</code>. The binary32 value is rounded once, directly from the decimal string, not through a double.</li>
<li>The DOM API can parse a buffer that has no padding (<code>parser.parse_unpadded(...)</code>). It is slower than the regular function, but it never reads past the end of your buffer and it does not copy your data.</li>
<li>With C++17, <code>simdjson::padded_input</code> adds padding only when it is needed: when your string ends near a page boundary.</li>
<li>C++20 ranges: you can pipe On-Demand arrays and objects into <code>std::views::transform</code> and other adaptors.</li>
<li>DOM arrays support reverse iteration (<code>rbegin()</code>, <code>rend()</code>) with no allocation.</li>
<li>On-Demand objects offer <code>get_current_position()</code> and <code>revert_position()</code>: if you miss an optional field, you can go back to where you were instead of rescanning the whole object.</li>
<li><code>char8_t</code> (<code>u8</code>) variants of the string accessors in C++20.</li>
<li>Better pretty printing with the <a href="https://github.com/j-brooke/FracturedJson">FracturedJson</a> style, including tables.</li>
<li>Memory-mapped files under Windows.</li>
<li>We support the memory-safe compiler Fil-C.</li>
</ul>
<p>The simdjson 5.0 release improved performance compared to simdjson 4.0 in some key cases. Let me review some of them.</p>
<p>I built both versions with GCC 16.1 (<code>-O3</code>, CMake Release) and ran them on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core.</p>
<p>Let us start with DOM parsing of our standard files (GB/s):</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">file</th>
<th style="text-align: right;">4.0</th>
<th style="text-align: right;">5.0</th>
<th style="text-align: right;">speedup</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">twitter</td>
<td style="text-align: right;">4.78</td>
<td style="text-align: right;">4.82</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">citm_catalog</td>
<td style="text-align: right;">4.82</td>
<td style="text-align: right;">4.79</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">github_events</td>
<td style="text-align: right;">5.33</td>
<td style="text-align: right;">5.30</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">canada</td>
<td style="text-align: right;">1.10</td>
<td style="text-align: right;">1.21</td>
<td style="text-align: right;">1.1</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">marine_ik</td>
<td style="text-align: right;">1.25</td>
<td style="text-align: right;">1.38</td>
<td style="text-align: right;">1.1</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">mesh</td>
<td style="text-align: right;">1.17</td>
<td style="text-align: right;">1.26</td>
<td style="text-align: right;">1.1</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">numbers</td>
<td style="text-align: right;">1.13</td>
<td style="text-align: right;">1.41</td>
<td style="text-align: right;">1.3</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">twitterescaped</td>
<td style="text-align: right;">1.59</td>
<td style="text-align: right;">2.85</td>
<td style="text-align: right;">1.8</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">update-center</td>
<td style="text-align: right;">3.96</td>
<td style="text-align: right;">3.84</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">apache_builds</td>
<td style="text-align: right;">4.95</td>
<td style="text-align: right;">4.78</td>
<td style="text-align: right;">1.0</td>
</tr>
</tbody>
</table>
<p>Files full of numbers (canada, marine_ik, mesh, numbers) are 8% to 25% faster. And a file full of escaped Unicode characters (twitterescaped) is almost twice as fast: among other changes, we now decode consecutive <code>uXXXX</code> sequences without going back to the string scanner between them.</p>
<p>We also serialize faster. Printing floating-point numbers used to be a bottleneck. We replaced the ancient Grisu2 by Dragonbox, and removed calls to <code>memcpy</code> and <code>memmove</code> from the hot path.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">file</th>
<th style="text-align: right;">4.0</th>
<th style="text-align: right;">5.0</th>
<th style="text-align: right;">speedup</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">twitter</td>
<td style="text-align: right;">0.94</td>
<td style="text-align: right;">0.96</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">citm_catalog</td>
<td style="text-align: right;">1.07</td>
<td style="text-align: right;">1.08</td>
<td style="text-align: right;">1.0</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">gsoc-2018</td>
<td style="text-align: right;">1.10</td>
<td style="text-align: right;">1.25</td>
<td style="text-align: right;">1.1</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">canada</td>
<td style="text-align: right;">0.31</td>
<td style="text-align: right;">0.52</td>
<td style="text-align: right;">1.7</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">marine_ik</td>
<td style="text-align: right;">0.28</td>
<td style="text-align: right;">0.37</td>
<td style="text-align: right;">1.3</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">mesh</td>
<td style="text-align: right;">0.34</td>
<td style="text-align: right;">0.47</td>
<td style="text-align: right;">1.4</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">numbers</td>
<td style="text-align: right;">0.32</td>
<td style="text-align: right;">0.49</td>
<td style="text-align: right;">1.6</td>
</tr>
</tbody>
</table>
<p>The simdjson library is a community project. Since version 4.6, contributions came from fior512, 吴杨帆, Alecto Irene Perez, Francisco Geiman Thiesen, Max Bachmann, MoonFlowww, Advit Arora, Jaël Champagne Gareau, Taimoor Kiani, Vasily Pelikh, jmestwa-coder, liyinlong, AlbertoFVisconti, Aylin Dmello, Cuda Chen, Ezra Li, Madhurendra Purbay, Makkar, Pastoray, Paul Dreik, Pavel Kruglov, Piotr Kubaj, Yusuf İhsan Görgel, metsw24-max, neil, pratap singh, Vladimir Saraikin, wankun, xaldarof, Riyane El Qoqui, Justin Li and others. Thank you!</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/28/simdjson-5-0-is-out/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How fast can you fix a UTF-16 string in C#?</title>
		<link>https://lemire.me/blog/2026/09/26/how-fast-can-you-fix-a-utf-16-string-in-c/</link>
					<comments>https://lemire.me/blog/2026/09/26/how-fast-can-you-fix-a-utf-16-string-in-c/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 26 Sep 2026 16:53:53 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22970</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="How fast can you fix a UTF-16 string in C#" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />C# strings are UTF-16. Most characters are one 16-bit code unit. Characters outside the basic multilingual plane, emoji included, take two: a high surrogate (U+D800 to U+DBFF) followed by a low surrogate (U+DC00 to U+DFFF). A surrogate with the wrong neighbor, or with none, is ill-formed. You should never send an ill-formed string to disk &#8230; <a href="https://lemire.me/blog/2026/09/26/how-fast-can-you-fix-a-utf-16-string-in-c/" class="more-link">Continue reading <span class="screen-reader-text">How fast can you fix a UTF-16 string in C#?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-cover-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="How fast can you fix a UTF-16 string in C#" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>C# strings are UTF-16. Most characters are one 16-bit code unit. Characters outside the basic multilingual plane, emoji included, take two: a <em>high surrogate</em> (U+D800 to U+DBFF) followed by a <em>low surrogate</em> (U+DC00 to U+DFFF). A surrogate with the wrong neighbor, or with none, is ill-formed.</p>
<p>You should never send an ill-formed string to disk or to the network. It is a bad practice.</p>
<p>In JavaScript, we have fast functions to fix strings or check whether they need fixing:</p>
<ul>
<li><code>String.prototype.toWellFormed()</code> replaces every lone surrogate with U+FFFD.</li>
<li><code>isWellFormed()</code> reports whether any replacement is needed.</li>
</ul>
<p>I added both functions to my C# library <a href="https://github.com/simdutf/SimdUnicode">SimdUnicode</a>, in <a href="https://github.com/simdutf/SimdUnicode/pull/54">pull request 54</a>. The algorithm is the same one that we contributed to the JavaScript engine V8, so Chrome already fixes strings this way.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">string</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">UTF16</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">ToWellFormed</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">);</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">// same instance, when the input is already well formed</span>
<span style="color: #204a87; font-weight: bold;">bool</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ok</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">UTF16</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">IsWellFormed</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">span</span><span style="color: #000; font-weight: bold;">);</span>
</code></pre>
</div>
<p>When the input is well formed, <code>ToWellFormed</code> returns it as is. No allocation.</p>
<p>How are strings fixed? Basically, you replace bad inputs by the replacement character <code>U+FFFD</code>.</p>
<p>Our processors have special instructions called SIMD that allow data parallelism: you can compare multiple values at once. Recent x64 processors from AMD and Intel have better data parallelism than ARM chips, although both have powerful instructions.</p>
<p>The conventional approach in C# to repair a string is a function such as the following.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">static</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Repair</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">ReadOnlySpan</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">Span</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">output</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">CopyTo</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">output</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">NextError</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">output</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">while</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">output</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000;">uFFFD</span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">NextError</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">output</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
<span style="color: #8f5902; font-style: italic;">// Index of the next lone surrogate at or after 'start', or -1 if none.</span>
<span style="color: #204a87; font-weight: bold;">static</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">NextError</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">ReadOnlySpan</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">start</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">start</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">while</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">true</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">k</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">Slice</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">).</span><span style="color: #000;">IndexOfAnyInRange</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000;">uD800</span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000;">uDFFF</span><span style="color: #a40000; border: 1px solid #EF2929;">'</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">k</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">k</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">IsHighSurrogate</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">])</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">Length</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">char</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">IsLowSurrogate</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">s</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">]))</span>
<span style="color: #f8f8f8;">            </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">+=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">2</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">// valid pair, skip it</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">else</span>
<span style="color: #f8f8f8;">            </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>In SimdUnicode, I also use data parallelism.</p>
<p>Let me measure.</p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-validate-xeon.webp" alt="Is the UTF-16 string well formed? Intel Xeon Gold 6548N" class="alignnone size-large" /></p>
<p>On the Xeon, with AVX-512, Latin validates at 69 GB/s against 33 GB/s for <code>IndexOfAnyInRange</code>. The Emoji input is well formed, and it is nothing but surrogate pairs. The runtime search drops to 0.4 GB/s. Our check holds 53 GB/s.</p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-validate-m4.webp" alt="Is the UTF-16 string well formed? Apple M4 Max" class="alignnone size-large" /></p>
<p>Our results are similar on the M4 Max, although a bit less impressive compared to the Intel results.</p>
<p>Validation can return at the first lone surrogate. The buffer form of <code>ToWellFormed</code> writes every code unit, a copy of the input or U+FFFD. When the input is well formed, it is effectively a memory copy. Thus we can compare the performance against a copy.</p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-buffer-xeon.webp" alt="Copy the string, replace lone surrogates. Intel Xeon Gold 6548N" class="alignnone size-large" /></p>
<p><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/utf16-buffer-m4.webp" alt="Copy the string, replace lone surrogates. Apple M4 Max" class="alignnone size-large" /></p>
<p>Roughly speaking, we are consistently about as fast as a copy.</p>
<p><strong>Versions used</strong>: .NET SDK 10.0.400 on Linux, 10.0.103 on macOS. Intel Xeon Gold 6548N (Emerald Rapids). Apple M4 Max.</p>
<p>Clausecker, R., &amp; Lemire, D. (2026). <a href="https://doi.org/10.1002/spe.70105">Fixing ill-formed UTF-16 strings with SIMD instructions</a>. Software: Practice and Experience. (<a href="https://arxiv.org/abs/2601.06349">arXiv</a>)</p>
<p><a href="https://github.com/simdutf/SimdUnicode/pull/54">Source code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/26/how-fast-can-you-fix-a-utf-16-string-in-c/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>A thesis isn&#8217;t enough for a PhD</title>
		<link>https://lemire.me/blog/2026/09/26/a-thesis-isnt-enough-for-a-phd/</link>
					<comments>https://lemire.me/blog/2026/09/26/a-thesis-isnt-enough-for-a-phd/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 26 Sep 2026 02:48:13 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22961</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_ehvp0eehvp0eehvp-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />To get a PhD, you typically have to enroll in a graduate program. Then you complete a few relatively easy courses.   You might need to pass a comprehensive examination that checks whether you have a basic understanding of the field. A few students fail at this point, but not many.Then you write a thesis &#8230; <a href="https://lemire.me/blog/2026/09/26/a-thesis-isnt-enough-for-a-phd/" class="more-link">Continue reading <span class="screen-reader-text">A thesis isn&#8217;t enough for a PhD</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_ehvp0eehvp0eehvp-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><div class="" data-block="true" data-editor="f3cv" data-offset-key="98c3d-0-0">
<div data-offset-key="98c3d-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="98c3d-0-0">To get a PhD, you typically have to enroll in a graduate program. Then you complete a few relatively easy courses.</span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="2niva-0-0">
<div data-offset-key="2niva-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="2niva-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="49c49-0-0">
<div data-offset-key="49c49-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="49c49-0-0">You might need to pass a comprehensive examination that checks whether you have a basic understanding of the field. A few students fail at this point, but not many.Then you write a thesis and defend it.</span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="crf9l-0-0">
<div data-offset-key="crf9l-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="crf9l-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="5bcv3-0-0">
<div data-offset-key="5bcv3-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="5bcv3-0-0">In theory, you could write a strong thesis and still fail the oral defense. That is uncommon. The defense is typically a public event, and there may be guests. Failing a student at that stage would be a public humiliation. I have seen students who could not answer basic questions still receive their PhDs because the thesis itself was good enough.</span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="e7303-0-0">
<div data-offset-key="e7303-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="e7303-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="1d8fi-0-0">
<div data-offset-key="1d8fi-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="1d8fi-0-0">Thus, to a very good approximation, getting a PhD amounted to writing a thesis.But we have a problem. AI can write something that looks quite a bit like a thesis. With a bit of prompting, you can produce something that looks like a PhD thesis.I have been telling everyone I can that we have a big problem.Apparently other people have realized this as well. People at Harvard are now saying that a thesis is insufficient for a PhD.</span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="4o4ed-0-0">
<div data-offset-key="4o4ed-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="4o4ed-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="fogll-0-0">
<div data-offset-key="fogll-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><a href="https://cmsa.fas.harvard.edu/media/2026/09/Summit-on-PhD-Math-Education-in-the-Age-of-AI.pdf"><span data-offset-key="fogll-0-0">Here is the key passage:</span></a></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="7u1rh-0-0">
<div data-offset-key="7u1rh-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="7u1rh-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="bodfc-0-0">
<div data-offset-key="bodfc-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="bodfc-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="f3cv" data-offset-key="cc61t-0-0">
<div data-offset-key="cc61t-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="cc61t-0-0">« The dissertation has in the past frequently been used as a proxy for the kind of mathematical development of a student that we expect: acquiring mathematical knowledge and demonstrating independent achievement. However, with modern AI systems, dissertations are not (&#8230;) a reliable tool for evaluating students, and PhDs should not be awarded primarily on the basis of the text of the dissertation. We thus recommend regular, multi-faceted evaluations of students in person, by multiple faculty, as the central component of assessment. These evaluations must be rigorous, not pro-forma, and should produce reports that might become part of the student’s dossier for future employment. » </span></div>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/26/a-thesis-isnt-enough-for-a-phd/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How many strings can you create per second?</title>
		<link>https://lemire.me/blog/2026/09/25/how-many-strings-can-you-create-per-second/</link>
					<comments>https://lemire.me/blog/2026/09/25/how-many-strings-can-you-create-per-second/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Fri, 25 Sep 2026 01:30:49 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22958</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/strings-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="How many strings can you create per second?" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />We create new strings all the time. How quickly can you produce short strings in your programming language? To create a meaningful string, we convert an integer to a string. In Python, that is str(i). The loop stores each new string in a small ring buffer of 1024 slots, so that the engine cannot simply &#8230; <a href="https://lemire.me/blog/2026/09/25/how-many-strings-can-you-create-per-second/" class="more-link">Continue reading <span class="screen-reader-text">How many strings can you create per second?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/strings-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="How many strings can you create per second?" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>We create new strings all the time. How quickly can you produce short strings in your programming language? To create a meaningful string, we convert an integer to a string. In Python, that is <code>str(i)</code>. The loop stores each new string in a small ring buffer of 1024 slots, so that the engine cannot simply discard the work.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">def</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">from_int</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">):</span>
    <span style="color: #000;">b</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">buf</span>
    <span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">i</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #204a87;">range</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">):</span>
        <span style="color: #000;">b</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span> <span style="color: #ce5c00; font-weight: bold;">&amp;</span> <span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">]</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">str</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">)</span>
</code></pre>
</div>
<p>In JavaScript (Node.js and Bun), I use <code>String(i)</code>:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">function</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">from_int</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">let</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">String</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>In C++, I use <code>std::to_string</code>:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">from_int</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">to_string</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>In Rust, I use the standard <code>to_string()</code> and, as an alternative, the popular <a href="https://crates.io/crates/itoa">itoa</a> crate, which is designed for fast integer formatting:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">fn</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">std_to_string</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">&amp;</span><span style="color: #000;">mut</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #204a87;">String</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">u64</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">in</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #ce5c00; font-weight: bold;">..</span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">[(</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">as</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">usize</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">to_string</span><span style="color: #000; font-weight: bold;">();</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">fn</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">itoa_to_string</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">&amp;</span><span style="color: #000;">mut</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[</span><span style="color: #204a87;">String</span><span style="color: #000; font-weight: bold;">],</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">u64</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">let</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">mut</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">itoa</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #000;">Buffer</span><span style="color: #000; font-weight: bold;">::</span><span style="color: #000;">new</span><span style="color: #000; font-weight: bold;">();</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">in</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #ce5c00; font-weight: bold;">..</span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">[(</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">as</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">usize</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">format</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">).</span><span style="color: #000;">to_owned</span><span style="color: #000; font-weight: bold;">();</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>In Go, I use <code>strconv.Itoa</code>:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">func</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fromInt</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">buf</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">strconv</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">Itoa</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>In Nim, I use the <code>$</code> operator:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">proc</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">fromInt</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">:</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">int</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">in</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">..</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">:</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">buf</span><span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">and</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1023</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">$</span><span style="color: #000;">i</span>
</code></pre>
</div>
<p>The integers go from 0 to 100 million (10 million in Python), so each string has up to eight digits. I report the best of five runs on an Apple M4 Max. C++ is compiled with <code>-O3</code>, Rust in release mode, Nim with <code>-d:danger</code>.</p>
<p><img decoding="async" alt="Time to convert an integer to a new string on an Apple M4 Max" src="https://lemire.me/blog/wp-content/uploads/2026/09/strings.webp" class="alignnone size-large" /></p>
<p>Millions of strings per second:</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;"></th>
<th style="text-align: right; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">million strings per second</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">C++ <code>std::to_string</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">183.8</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Nim <code>$i</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">85.9</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Go <code>strconv.Itoa</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">84.3</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Node.js <code>String(i)</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">73.1</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Rust <code>itoa</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">71.0</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Bun <code>String(i)</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">68.5</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Rust <code>to_string()</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">63.6</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">Python <code>str(i)</code></td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">22.9</td>
</tr>
</tbody>
</table>
<p>Python is the slowest at 44 ns per string, about three times slower than JavaScript and eight times slower than C++.</p>
<p>The compiled languages that allocate each string on the heap (Rust, Go, Nim) end up in the same range as JavaScript: 12 to 16 ns per string. Rust with its standard <code>to_string()</code> is even a bit slower than Node.js and Bun. Garbage-collected runtimes like Go and JavaScript are very good at allocating many small, short-lived objects.</p>
<p>C++ wins by a wide margin at 5.4 ns per string. The trick is the <em>small string optimization</em>: a <code>std::string</code> stores short strings, directly inside the object. Our strings have at most eight digits, so C++ never calls the memory allocator.</p>
<p><strong>Versions used</strong>: macOS 15.7.7, CPython 3.14.5, Node.js 25.9.0, Bun 1.4.2, Apple clang 17.0.0 (clang-1700.6.4.2) with libc++, rustc 1.94.1 with itoa 1.0.18, Go 1.24.3, Nim 2.2.12.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/24">Source code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/25/how-many-strings-can-you-create-per-second/feed/</wfw:commentRss>
			<slash:comments>4</slash:comments>
		
		
			</item>
		<item>
		<title>If you don&#8217;t have the factories, you lose the expertise</title>
		<link>https://lemire.me/blog/2026/09/24/if-you-dont-have-the-factories-you-lose-the-expertise/</link>
					<comments>https://lemire.me/blog/2026/09/24/if-you-dont-have-the-factories-you-lose-the-expertise/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 24 Sep 2026 14:09:39 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22952</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/factories-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Mainstream economists have been advocating for the marvellous effects of global trade for decades. Who needs these factory jobs anyway? We&#8217;ll be designing the robots and the nuclear rockets. Except that, no. It does not work like that. If you have the factories, sooner or later, you get the designers. If you don&#8217;t have the &#8230; <a href="https://lemire.me/blog/2026/09/24/if-you-dont-have-the-factories-you-lose-the-expertise/" class="more-link">Continue reading <span class="screen-reader-text">If you don&#8217;t have the factories, you lose the expertise</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/factories-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Mainstream economists have been advocating for the marvellous effects of global trade for decades. Who needs these factory jobs anyway? We&#8217;ll be designing the robots and the nuclear rockets.</p>
<p>Except that, no. It does not work like that.</p>
<p>If you have the factories, sooner or later, you get the designers. If you don&#8217;t have the factories, you lose the expertise. Or you never get it.</p>
<p>There aren&#8217;t that many people left in the Silicon Valley capable of working on silicon. You can design a chip from California. Making one is done where the fab is.</p>
<p>You can get a PhD in robotics in Quebec City, but you won&#8217;t be designing robots unless you fly over to where they are made. Zoom calls won&#8217;t cut it at scale.<a href="https://lemire.me/blog/wp-content/uploads/2026/09/factories.jpg"></a></p>
<p>Jensen Huang&#8217;s account, as Joseph Steinberg <a href="https://x.com/jbsteinberg/status/2102912376852914190">reported it</a>, is that American manufacturing jobs declined because the work was outsourced. Steinberg says this is flatly wrong, and that technology accounts for the vast majority of the decline in manufacturing&#8217;s share of employment.</p>
<p><a href="https://lemire.me/blog/wp-content/uploads/2026/09/jobs.jpg"><img loading="lazy" decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/jobs-1024x570.jpg" alt="U.S. manufacturing employment, millions of jobs" width="660" height="367" class="alignnone size-large wp-image-22950" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/jobs-1024x570.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/jobs-300x167.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/jobs-768x428.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/jobs-1536x855.jpg 1536w, https://lemire.me/blog/wp-content/uploads/2026/09/jobs.jpg 2048w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>In 2000 there were 17.3 million manufacturing jobs in the United States. The peak was 19.6 million, in June 1979. From the early 1980s to 2000 the count stayed high, apart from the recessions. China joined the WTO in December 2001. By 2010 manufacturing employment was 11.5 million. In August 2026 it was 12.6 million.</p>
<p>That is 5.8 million jobs gone in a decade. About a million have come back since the bottom. The rest have not.</p>
<p>If technology were the main driver, output should have kept rising while the jobs fell. The fifteen years before 2000 are what that looks like. Manufacturing output nearly doubled, from an index of 51 to an index of 93 (2017 = 100). Employment went from 17.8 million to 17.3 million. The robots were already here in 1985. Employment did not collapse.</p>
<p><a href="https://lemire.me/blog/wp-content/uploads/2026/09/output.jpg"><img loading="lazy" decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/output-1024x562.jpg" alt="U.S. industrial production, index 2017 = 100" width="660" height="362" class="alignnone size-large wp-image-22951" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/output-1024x562.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/output-300x165.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/output-768x421.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/output-1536x842.jpg 1536w, https://lemire.me/blog/wp-content/uploads/2026/09/output.jpg 2048w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>After 2000, production stalled. Manufacturing output peaked just under 107 in December 2007. In August 2026 the index was 99.1. Total industrial production was 103.1. American factories are not turning out a flood of extra goods. They are turning out roughly what they turned out twenty years ago.</p>
<p>Output per hour did rise while the jobs were disappearing. The BLS index of manufacturing labor productivity went from about 70 in 2000 to about 100 in 2010. Then it stopped. In early 2026 it was still about 100. Flat output, fewer workers, a higher ratio. The ratio has been flat for fifteen years.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden;">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem;">Year</th>
<th style="text-align: right; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem;">Jobs (millions)</th>
<th style="text-align: right; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem;">Manufacturing output (2017 = 100)</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">1985</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">17.8</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">51</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">2000</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">17.3</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">93</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">2007</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">13.9</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">105</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">2010</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">11.5</td>
<td style="text-align: right; padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2;">93</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem;">August 2026</td>
<td style="text-align: right; padding: 0.65rem 0.8rem;">12.6</td>
<td style="text-align: right; padding: 0.65rem 0.8rem;">99.1</td>
</tr>
</tbody>
</table>
<p>The problem after 2000 was not that American factories became too productive. What changed after 2000 was where the goods were made.</p>
<p>And now, often, we don&#8217;t know anymore how to make things. Human expertise matters, and you maintain it by building stuff locally. You are not going to design microprocessors in Maine. It just won&#8217;t happen.</p>
<p>&nbsp;</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/24/if-you-dont-have-the-factories-you-lose-the-expertise/feed/</wfw:commentRss>
			<slash:comments>3</slash:comments>
		
		
			</item>
		<item>
		<title>A summer of AI optimization</title>
		<link>https://lemire.me/blog/2026/09/22/a-summer-of-ai-optimization/</link>
					<comments>https://lemire.me/blog/2026/09/22/a-summer-of-ai-optimization/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Tue, 22 Sep 2026 02:27:44 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22946</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="A summer of AI optimization" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />I maintain and co-maintain several open-source libraries. Some of them are widely used: ada parses URLs in Node.js, fast_float parses numbers in GCC&#8217;s standard library and in Chromium, simdjson parses JSON in Node.js, simdutf validates and transcodes Unicode in Node.js, and the Roaring bitmap libraries sit inside many database engines. These libraries are mature. They &#8230; <a href="https://lemire.me/blog/2026/09/22/a-summer-of-ai-optimization/" class="more-link">Continue reading <span class="screen-reader-text">A summer of AI optimization</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="A summer of AI optimization" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>I maintain and co-maintain several open-source libraries. Some of them are widely used: ada parses URLs in Node.js, fast_float parses numbers in GCC&#8217;s standard library and in Chromium, simdjson parses JSON in Node.js, simdutf validates and transcodes Unicode in Node.js, and the Roaring bitmap libraries sit inside many database engines.</p>
<p>These libraries are mature. They have been optimized for years, by me and by others. For a long time, their performance was flat. Not because nobody cared, but because the remaining gains were expensive: each one required a few days of careful work, and nobody had the days.</p>
<p>Then, in 2026, six of them got much faster, most of it in a few weeks of summer.</p>
<p>To formalize my feeling, I rebuilt every commit of each library from scratch and benchmarked it on one machine (an Intel Xeon Gold 6548N). I track the speedup over time relative to August 2024. Thus the value 1.0 means no speedup. Whereas 2.0 means that the performance doubled. The lines are steps because performance only changes at a commit.</p>
<p>I should say that I cannot know how much AI was involved in each instance. I don&#8217;t ask how people arrived at their code. All I ask is that it be good. As for myself, I code with Claude (Opus 5), Grok and DeepSeek (V4 Pro). I was an early adopter of Grok for coding, and it got really good over time.</p>
<h2>1. roaring (compressed bitmaps, Go)</h2>
<p><img decoding="async" alt="roaring: speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-roaring.webp" class="alignnone size-large" /></p>
<p>The roaring library is the Go version of the Roaring index data structure. Decoding to an array got 2.5 times faster, the multi-way union <code>FastOr</code> got 3.1 times faster on one data set, the many-value iterator got 4.5 to 5.9 times faster, and the intersection cardinality gained 10%.</p>
<p>One of the contributors is an AI, actually. It is <a href="https://www.perfloop.com">perfloop</a>. (Disclosure: I am an advisor for perfloop.)</p>
<p>I did a lot of work. We also got help from Philipp Klose who declared using Claude.</p>
<h2>2. ada (URL parsing)</h2>
<p><img decoding="async" alt="ada: speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-ada.webp" class="alignnone size-large" /></p>
<p>The ada library is a standards-compliant URL parser. From August 2024 to July 2026, about 550 commits went in and the throughput on a corpus of 100,000 URLs stayed at 0.54 GB/s. Then, in six weeks, it went to 1.28 GB/s: 2.4 times faster, about 15 million URLs per second on one core.</p>
<p>Most of the optimizations were done by Yagiz Nizipli, my long-time co-author. Yagiz works at SpaceX and uses Cursor (presumably with a grok model). Abdul Rawoof Khan and Dillon Mulroy also contributed an optimization each. I worked at optimizing IP address parsing, but it won&#8217;t show in this particular benchmark.</p>
<h2>3. fast_float (number parsing)</h2>
<p><img decoding="async" alt="fast_float: speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-fast-float.webp" class="alignnone size-large" /></p>
<p>The fast_float library parses floating-point numbers from text. It is part of GCC and most browsers. Performance was flat for fifteen months. Then, from March to July 2026, it gained 43% on one file (<code>canada.txt</code>, long coordinates) and 70% on another (<code>mesh.txt</code>, short coordinates). The optimizations should be credited to Koleman Nix and Filipe Oliveira.</p>
<h2>4. simdjson (JSON serialization and deserialization with C++26 reflection)</h2>
<p><img decoding="async" alt="simdjson: serialization and deserialization speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-simdjson.webp" class="alignnone size-large" /></p>
<p>The simdjson library recently gained support for C++26 static reflection: you serialize and parse your own structs directly, with no glue code. Since February 2026, serialization is 1.6 times faster on <code>twitter.json</code> and 2.1 times faster on <code>citm_catalog.json</code>. Deserialization, JSON straight into a struct, gained a more modest 10% and 14% (the second panel). (The reflection code only exists since early 2026.) The number of instructions per byte fell by almost exactly the same ratio as the throughput rose: from 6.1 to 3.1 instructions per byte on <code>citm_catalog.json</code> serialization.</p>
<p>Francisco Geiman Thiesen (Microsoft) did most of the work on the serialization side while I mostly helped improve our parsing. Francisco uses Claude.</p>
<h2>5. simdutf (Unicode validation and transcoding)</h2>
<p><img decoding="async" alt="simdutf: speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-simdutf.webp" class="alignnone size-large" /></p>
<p>The simdutf library validates and transcodes UTF-8, UTF-16 and UTF-32, and encodes and decodes base64. ASCII validation went from 83 GB/s to 160 GB/s. UTF-16 validation went from 62 GB/s to 102 GB/s. Base64 decoding gained 17%.</p>
<p>The work was done by Yagiz Nizipli (again) and myself.</p>
<p>The library got other amazing optimizations that do not show up on this benchmark by Gaspard Petit and Shreesh Adiga.</p>
<h2>6. CRoaring (compressed bitmaps, C)</h2>
<p><img decoding="async" alt="CRoaring: speedup over time" src="https://lemire.me/blog/wp-content/uploads/2026/09/summer-ai-croaring.webp" class="alignnone size-large" /></p>
<p>CRoaring implements Roaring bitmaps in C. On the real data sets from the repository, membership tests (<code>contains</code>) got 2.4 times faster, the cardinality of 64-bit bitmaps got 4.9 times faster, iterating over a 64-bit bitmap got 1.9 times faster, decoding a dense bitmap to an array got 2.2 times faster. Unions gained a more modest 13% to 16%.</p>
<p>The authors were Andrei Gudkov and myself.</p>
<h2>What happened</h2>
<p>The techniques used are all well-known. So why all these optimizations all of a sudden? Simply put, in my view, because it got cheap to try new ideas.</p>
<p>There is a lot of talk about the risks of AI in software. Human beings tend to be susceptible to the one-sided bet fallacy: when we see the downsides, we tend to ignore the benefits. Cars kill people, but ambulances save them.</p>
<p>In this instance, the benefits are concrete. Millions of people run these libraries, and this summer, they got faster.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/22/a-summer-of-ai-optimization/feed/</wfw:commentRss>
			<slash:comments>5</slash:comments>
		
		
			</item>
		<item>
		<title>More than a taken branch per cycle?</title>
		<link>https://lemire.me/blog/2026/09/21/more-than-a-taken-branch-per-cycle/</link>
					<comments>https://lemire.me/blog/2026/09/21/more-than-a-taken-branch-per-cycle/#comments</comments>
		
		<dc:creator><![CDATA[Antonio Badia]]></dc:creator>
		<pubDate>Mon, 21 Sep 2026 18:36:05 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22921</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/branching_branch_art-150x150.png" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Our processors can execute many instructions per cycle; they are superscalar. But not all instructions are equal. Branches are particularly tricky. A branch occurs often when you use an if-then clause or a loop. We distinguish between a taken branch and a not taken branch. A not-taken branch is often cheap. The processor just keeps &#8230; <a href="https://lemire.me/blog/2026/09/21/more-than-a-taken-branch-per-cycle/" class="more-link">Continue reading <span class="screen-reader-text">More than a taken branch per cycle?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/branching_branch_art-150x150.png" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Our processors can execute many instructions per cycle; they are <em>superscalar</em>. But not all instructions are equal.</p>
<p>Branches are particularly tricky. A branch occurs often when you use an if-then clause or a loop. We distinguish between a taken branch and a not taken branch.</p>
<p>A not-taken branch is often cheap. The processor just keeps going.</p>
<p>A taken branch jumps to a new location. A taken branch can be more expensive.</p>
<p>You will often hear that processors are limited to one taken branch per cycle.</p>
<p>I decided to test it out with a loop with an <code>if</code> inside it. Here is the function I tested in Go.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">func</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">lastHit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">[]</span><span style="color: #204a87; font-weight: bold;">byte</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">thresh</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">byte</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">last</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #204a87; font-weight: bold;">byte</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:=</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">len</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">v</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">:=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">]</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">v</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">thresh</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">            </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">last</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">v</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">n</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">            </span><span style="color: #204a87; font-weight: bold;">break</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>Look at the main loop. We load a value from an array, we compare it with a threshold. If it is greater than the threshold, then we assign it to the <code>last</code> pointer. So the function effectively records the last seen value that is greater than the threshold. That&#8217;s pretty reasonable code.</p>
<p>Consider the case where you always miss. The values are always smaller than or equal to the threshold. In these cases, we get two taken branches in close proximity, but no store. (It is a bit confusing but that&#8217;s how the Go compiler does it.)</p>
<p><a href="https://lemire.me/blog/wp-content/uploads/2026/09/misses.webp"><img decoding="async" src="https://lemire.me/blog/wp-content/uploads/2026/09/misses.webp" alt="Cycles per iteration when the if always misses" class="alignnone size-large" /></a></p>
<p>The processor that struggles the most is the AMD Zen 4 processor. But AMD Zen 5 is much better.</p>
<p>So two processors are able to take two branches in less than 2 cycles on average in this test: the Apple processor (M4 Max) and the Granite Rapids processor.</p>
<p>This means that, yes, modern processors can execute more than one taken branch per cycle under some conditions.</p>
<p>The code is under <code>benchmark/experiments/ifloop</code> in the <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/21">GitHub repo</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/21/more-than-a-taken-branch-per-cycle/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>The data does not show mass unemployment</title>
		<link>https://lemire.me/blog/2026/09/21/nomassunemployment/</link>
					<comments>https://lemire.me/blog/2026/09/21/nomassunemployment/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Mon, 21 Sep 2026 15:04:08 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22933</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/P9wow-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />If AI is causing mass unemployment among software developers, it is not showing up in the data yet.   The USA had more software developers as a percentage of the population in 2025 than in 2021.   On average salaries are slightly up (average of 148k$ a year). The 90th percentile is up from 2021 &#8230; <a href="https://lemire.me/blog/2026/09/21/nomassunemployment/" class="more-link">Continue reading <span class="screen-reader-text">The data does not show mass unemployment</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/P9wow-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><div class="" data-block="true" data-editor="1lfk9" data-offset-key="2ql3n-0-0">
<div data-offset-key="2ql3n-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="2ql3n-0-0">If AI is causing mass unemployment among software developers, it is not showing up in the data yet.</span></div>
</div>
<div class="" data-block="true" data-editor="1lfk9" data-offset-key="8rfph-0-0">
<div data-offset-key="8rfph-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="8rfph-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="1lfk9" data-offset-key="bsf6a-0-0">
<div data-offset-key="bsf6a-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="bsf6a-0-0">The USA had more software developers as a percentage of the population in 2025 than in 2021.</span></div>
</div>
<div class="" data-block="true" data-editor="1lfk9" data-offset-key="1algn-0-0">
<div data-offset-key="1algn-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="1algn-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="1lfk9" data-offset-key="2pvg6-0-0">
<div data-offset-key="2pvg6-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="2pvg6-0-0">On average salaries are slightly up (average of 148k$ a year). The 90th percentile is up from 2021 (215k$US a year).</span><a href="http://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-scaled.png"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-1024x475.png" alt="" width="660" height="306" class="alignnone size-large wp-image-22934" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-1024x475.png 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-300x139.png 300w, https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-768x356.png 768w, https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-1536x712.png 1536w, https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-21-a-10.59.39-2048x950.png 2048w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></div>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/21/nomassunemployment/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>AI is breaking the academic sorting machine</title>
		<link>https://lemire.me/blog/2026/09/20/ai-is-breaking-the-academic-sorting-machine/</link>
					<comments>https://lemire.me/blog/2026/09/20/ai-is-breaking-the-academic-sorting-machine/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sun, 20 Sep 2026 21:47:46 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22928</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/INeo8-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />I see the mathematicians panicking at what we can do with agentic AI. Let us be clear on what we are talking about. In a few weeks, someone who is not a high-level mathematician can, with AI, produce the equivalent of a good PhD thesis in math. I could see a top 1% high school &#8230; <a href="https://lemire.me/blog/2026/09/20/ai-is-breaking-the-academic-sorting-machine/" class="more-link">Continue reading <span class="screen-reader-text">AI is breaking the academic sorting machine</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/INeo8-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p><span>I see the mathematicians panicking at what we can do with agentic AI. </span></p>
<p>Let us be clear on what we are talking about. In a few weeks, someone who is not a high-level mathematician can, with AI, produce the equivalent of a good PhD thesis in math. I could see a top 1% high school student, given enough of an AI token budget, just write the equivalent of a PhD thesis as a hobby.</p>
<p>If Joe Smith completed a PhD thesis in math back in 2022, it is now possible for a really smart high schooler to generate the same output while playing video games. Joe must not be too happy about it, especially if Joe is still looking for a prestigious faculty position.</p>
<p>Notice how we are not so excited. I mean, why aren’t we celebrating this incredible breakthrough? Finally, all the math problems we have can be solved faster! It is worth reflecting on why we don’t care.</p>
<p>Part of the issue is that most of academic research lost its customers years ago.</p>
<p>In part thanks to the arrival of “peer review” in the 1970s, we have long ago closed the research world into siloed communities. The purpose of the academic output is to sort people out for jobs. Write papers that are impressive and you may get a good professorship. If you don’t, then you will be flipping burgers. There is an incredible glut of people with academic credentials, but only so many jobs at the top. The glut has been continuously, and somewhat deliberately, increasing.</p>
<p>If you hold a PhD today, your chance of having a tenure-track or tenured position is about 10% and falling. Let me be clear. Any academic who shows worry for what the PhDs might do now should look in a mirror because you have been training too many for decades. And, also, if fewer people decide to go for the PhD that might be more than fine.<br />
<a href="http://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq.jpeg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq-1024x822.jpeg" alt="" width="660" height="530" class="alignnone size-large wp-image-22931" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq-1024x822.jpeg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq-300x241.jpeg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq-768x616.jpeg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/HSqvM7IXQAAL7iq.jpeg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>AI disrupts the sorting mechanism in mathematics… But do you think for a minute that it does not apply to mechanical engineering, sociology, etc.? Is that bad news? No. I think that it is excellent news and I have been saying so for years.</p>
<p>Here is what I wrote in 2024…</p>
<p>« AI’s ability to generate vast amounts of text raises concerns about a potential flood of irrelevant theoretical papers, further straining the evaluation system. Stonebraker’s (2018) call for rewarding problem-solving over publication needs revisiting. Perhaps the emphasis should be on the impact and significance of research, not just its passage through peer review—a skill replicable by AI. AI can pave the way for a “golden age” of scientific progress if we can develop new evaluation methods focused on problem-solving and real-world impact. The scientific community must adapt to the evolving landscape. By recognizing the limitations of peer review and prioritizing the pursuit of meaningful solutions, we can ensure that AI becomes a catalyst for scientific advancement, not a detriment. »</p>
<p>I predict that, over time, the focus will move away from “papers as the final output.”</p>
<p>Nobody wants a paper about cancer, we want to eradicate cancer. We should reward people who get results, no matter which tools they use. And solving a problem because it is impressive to do so is not enough.</p>
<p>« But Daniel, Mathematicians can&#8217;t cure cancer or give us antigravity. »</p>
<p>Maybe it is time they try. Frankly, the AI disruption might be precisely what we needed.</p>
<p>« But Daniel, won’t AI just replace all of us? »</p>
<p>I wish. But thus far, it is not happening. I have more AI accounts than most people have pencils and I am working 50 hours a week. Intelligence is not a scalar quantity. Once an AI can do something, I somehow always find more work to do.</p>
<p><strong>Further reading</strong>. Daniel Lemire, <a href="https://dl.acm.org/doi/full/10.1145/3673649">Will AI Flood Us with Irrelevant Papers?</a> Communications of the ACM, Vol. 67, No. 9 (September 2024).</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/20/ai-is-breaking-the-academic-sorting-machine/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How did Apple Silicon get 50% faster in three years?</title>
		<link>https://lemire.me/blog/2026/09/19/how-did-apple-silicon-get-50-faster-in-three-years/</link>
					<comments>https://lemire.me/blog/2026/09/19/how-did-apple-silicon-get-50-faster-in-three-years/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 19 Sep 2026 16:13:41 +0000</pubDate>
				<category><![CDATA[Science and Technology]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22914</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/apple-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Four processor packages in a row on a desk" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Recently, I looked at how AMD made its chips 50% faster within a span of two years, in How did AMD Ryzen get 50% faster in two years? Many people asked me to cover Apple processors. So let us go. Consider the base Apple Silicon chips: M2 (2022), M3 (2023), M4 (2024) and M5 (2025). &#8230; <a href="https://lemire.me/blog/2026/09/19/how-did-apple-silicon-get-50-faster-in-three-years/" class="more-link">Continue reading <span class="screen-reader-text">How did Apple Silicon get 50% faster in three years?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/apple-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Four processor packages in a row on a desk" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>Recently, I looked at how AMD made its chips 50% faster within a span of two years, in <a href="https://lemire.me/blog/2026/09/18/how-did-amd-ryzen-get-50-faster-in-two-years/">How did AMD Ryzen get 50% faster in two years?</a> Many people asked me to cover Apple processors. So let us go.</p>
<p>Consider the base Apple Silicon chips: M2 (2022), M3 (2023), M4 (2024) and M5 (2025). I am skipping the original one (M1) for simplicity and I am not considering the Pro and Max chips. The M6 is out there too, just released, but I am going to cover it at the end.</p>
<p>On Geekbench 6, performance went up by about 50% in three years. The increase is rather gradual. No big jump.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/geekbench.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/geekbench-1024x424.jpg" alt="Geekbench 6 single-core and multi-core scores for Apple M2, M3, M4 and M5" width="660" height="273" class="alignnone size-large wp-image-22906" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/geekbench-1024x424.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/geekbench-300x124.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/geekbench-768x318.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/geekbench.jpg 1199w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
M2<br />
4P+4E</td>
<td>2023<br />
M3<br />
4P+4E</td>
<td>2024<br />
M4<br />
4P+6E</td>
<td>2025<br />
M5<br />
4P+6E</td>
</tr>
<tr>
<td>Single-core</td>
<td>2,401</td>
<td>2,767</td>
<td>3,278</td>
<td>3,642</td>
</tr>
<tr>
<td>Multi-core</td>
<td>9,814</td>
<td>11,544</td>
<td>15,345</td>
<td>17,955</td>
</tr>
</tbody>
</table>
<p>The 2025 chip is 52% faster on a single core than the 2022 chip, and 83% faster with all cores.</p>
<p>Unlike AMD, Apple improved the CPU frequency quite a bit. The P-core went from 3.5 GHz to 4.6 GHz, a 30% increase. On its own, it can explain the bulk of the single-core performance increase.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/frequency.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/frequency-1024x753.jpg" alt="P-core and E-core max frequencies for Apple M2, M3, M4 and M5" width="660" height="485" class="alignnone size-large wp-image-22907" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/frequency-1024x753.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/frequency-300x221.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/frequency-768x564.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/frequency.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
M2</td>
<td>2023<br />
M3</td>
<td>2024<br />
M4</td>
<td>2025<br />
M5</td>
</tr>
<tr>
<td>P-core max</td>
<td>3.5 GHz</td>
<td>4.0 GHz</td>
<td>4.4 GHz</td>
<td>4.6 GHz</td>
</tr>
<tr>
<td>E-core max</td>
<td>2.4 GHz</td>
<td>2.8 GHz</td>
<td>2.9 GHz</td>
<td>3.0 GHz</td>
</tr>
</tbody>
</table>
<p>But still, even if we normalized the clock speed, an Apple M5 would be noticeably faster than an Apple M2.</p>
<p>Let us look at the transistor count. For the M5, I could not find any information. But there is otherwise a nice gradual increase. From the M2 to the M4, the number went up by about 40%, from 20 billion to 28 billion.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/transistors.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/transistors-1024x753.jpg" alt="Transistor counts for Apple M2, M3 and M4; M5 not disclosed" width="660" height="485" class="alignnone size-large wp-image-22908" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/transistors-1024x753.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/transistors-300x221.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/transistors-768x564.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/transistors.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>Where did they go?</p>
<p>The base M4 processor has two extra efficiency cores. This will not help with single-core performance, but it will help multicore performance.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/cores.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/cores-1024x753.jpg" alt="Performance and efficiency core counts for Apple M2, M3, M4 and M5" width="660" height="485" class="alignnone size-large wp-image-22909" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/cores-1024x753.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/cores-300x221.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/cores-768x564.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/cores.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
M2</td>
<td>2023<br />
M3</td>
<td>2024<br />
M4</td>
<td>2025<br />
M5</td>
</tr>
<tr>
<td>Performance cores</td>
<td>4</td>
<td>4</td>
<td>4</td>
<td>4</td>
</tr>
<tr>
<td>Efficiency cores</td>
<td>4</td>
<td>4</td>
<td>6</td>
<td>6</td>
</tr>
</tbody>
</table>
<p>But what about each individual core?</p>
<p>An important variable at the core level is the number of instructions per cycle. And this went up. Decode width went from 8 instructions per cycle on the M2 to 10 on the M4 and the M5, which is higher than what most competitors can do. So you can get more compute per cycle.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ipc.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ipc-1024x753.jpg" alt="Decode width in instructions per cycle for Apple M2, M3, M4 and M5" width="660" height="485" class="alignnone size-large wp-image-22910" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ipc-1024x753.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ipc-300x221.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ipc-768x564.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ipc.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>Thus far, we do not see much difference between the M4 and the M5, and yet the M5 can be significantly faster. Let us look next at memory bandwidth.</p>
<p>And yeah, the M5 has much more memory bandwidth: 154 GB/s against 100 GB/s on the M2 and the M3. That helps on multicore processing tasks, as it is when you are most likely to run out of bandwidth.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/bandwidth.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/bandwidth-1024x751.jpg" alt="Memory bandwidth for Apple M2, M3, M4 and M5" width="660" height="484" class="alignnone size-large wp-image-22911" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/bandwidth-1024x751.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/bandwidth-300x220.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/bandwidth-768x563.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/bandwidth.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
M2</td>
<td>2023<br />
M3</td>
<td>2024<br />
M4</td>
<td>2025<br />
M5</td>
</tr>
<tr>
<td>Bandwidth</td>
<td>100 GB/s</td>
<td>100 GB/s</td>
<td>120 GB/s</td>
<td>154 GB/s</td>
</tr>
<tr>
<td>Memory</td>
<td>LPDDR5-6400</td>
<td>LPDDR5-6400</td>
<td>LPDDR5X-7500</td>
<td>LPDDR5X-9600</td>
</tr>
</tbody>
</table>
<p>What about data parallelism? AMD improved data parallelism (SIMD) considerably. The story is much less interesting with Apple processors. They have four 128-bit execution units, all of them. Meanwhile the Zen 5 has four 512-bit units. It is four times more. What is new with the M4 and the M5 is the new 512-bit SME matrix unit.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/simd.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/simd-1024x445.jpg" alt="Apple M-series four 128-bit NEON units versus AMD Zen 5 four 512-bit AVX-512 units" width="660" height="287" class="alignnone size-large wp-image-22912" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/simd-1024x445.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/simd-300x131.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/simd-768x334.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/simd.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>But there is a new iteration. The M6.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/m6.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/m6-1024x834.jpg" alt="Apple M6 processor" width="660" height="538" class="alignnone size-large wp-image-22913" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/m6-1024x834.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/m6-300x244.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/m6-768x625.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/m6.jpg 1086w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>The number of cores goes from 10 to 12. They added two &#8216;super&#8217; cores. The bandwidth is up a bit. My expectation is that Apple will again deliver a boost in performance.</p>
<p>In effect, what I am demonstrating is that CPUs are not boring. They are improving fast.</p>
<p>I already made the point that <a href="https://lemire.me/blog/2025/09/01/processors-are-getting-wider/">processors are getting wider</a>. See also <a href="https://lemire.me/blog/2025/07/09/memory-level-parallelism-apple-m2-vs-apple-m4/">Memory-level parallelism: Apple M2 vs Apple M4</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/19/how-did-apple-silicon-get-50-faster-in-three-years/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>Faster JSON parsing with SVE2 on ARM processors</title>
		<link>https://lemire.me/blog/2026/09/18/faster-json-parsing-with-sve2-on-arm-processors/</link>
					<comments>https://lemire.me/blog/2026/09/18/faster-json-parsing-with-sve2-on-arm-processors/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Fri, 18 Sep 2026 21:38:09 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22899</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_co7959co7959co79-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />ARM processors, like those in your phone, have instructions capable of processing several elements at once (SIMD). These instructions are called NEON. But many newer processors have a different SIMD extension called SVE. The latest ARM processors have SVE2. Unfortunately, Apple has not yet adopted SVE, but SVE processors are available in the cloud. In &#8230; <a href="https://lemire.me/blog/2026/09/18/faster-json-parsing-with-sve2-on-arm-processors/" class="more-link">Continue reading <span class="screen-reader-text">Faster JSON parsing with SVE2 on ARM processors</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_co7959co7959co79-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>ARM processors, like those in your phone, have instructions capable of processing several elements at once (SIMD). These instructions are called NEON. But many newer processors have a different SIMD extension called SVE. The latest ARM processors have SVE2. Unfortunately, Apple has not yet adopted SVE, but SVE processors are available in the cloud.</p>
<p>In April, I wrote that the SVE2 <code>match</code> instruction might be <a href="https://lemire.me/blog/2026/04/19/the-fastest-way-to-match-characters-on-arm-processors/">the fastest way to match characters on ARM processors</a>. At the time, my benchmark was a toy. The question was whether the idea survives contact with a real parser. Madhurendra Purbay, an engineer at ARM, answered the question with a <a href="https://github.com/simdjson/simdjson/pull/2863">pull request to the simdjson library</a>. Let me go through what it does and what it buys us.</p>
<p>The simdjson library includes a fast JSON parser. JSON is a ubiquitous data format online; everyone uses it. It is made of strings, numbers, arrays (<code>[1,2,3]</code>) and <em>objects</em>. An object is a key-value map where keys are strings, written as <code>{"key1": 1, "key2": 2}</code>. You can combine arrays and objects (e.g., an object can be in an array).</p>
<p>When the simdjson library indexes a JSON document, it first computes, for each block of 64 bytes, a few 64-bit masks. One of them marks the JSON <em>structural</em> characters (<code>,</code>, <code>:</code>, <code>[</code>, <code>]</code>, <code>{</code>, <code>}</code>). From these masks and a few others, we derive the positions of all the JSON tokens.</p>
<p>The ARM NEON version of the classifier, which I designed with Geoff Langdale years ago, uses a table lookup (<code>tbl</code>). Take the byte, add 3, keep the high nibble, and look it up in a 16-byte table that returns the one structural character with that nibble (or <code>0xff</code>). If the looked-up byte is equal to the input, the input is structural. With the <code>vaddq</code>/<code>vshrq</code>/<code>vqtbl1q</code>/<code>vceqq</code> NEON intrinsics, it is four instructions per 16 bytes:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">op_table</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">simd8</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">uint8_t</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #000; font-weight: bold;">(</span>
<span style="color: #f8f8f8;">  </span><span style="color: #0000cf; font-weight: bold;">0xff</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">','</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">':'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'['</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">']'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'{'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'}'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span>
<span style="color: #000; font-weight: bold;">);</span>
<span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">match_op_0</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vceqq_u8</span><span style="color: #000; font-weight: bold;">(</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">vqtbl1q_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">op_table</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vshrq_n_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">vaddq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">d0_0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vdupq_n_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">3</span><span style="color: #000; font-weight: bold;">)),</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">)),</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">d0_0</span><span style="color: #000; font-weight: bold;">);</span>
</code></pre>
</div>
<p>We get a vector of 16 bytes that are either <code>0x00</code> or <code>0xff</code>. To turn four such vectors into one 64-bit mask, we AND each byte with a bit weight (1, 2, 4, &#8230;, 128) and sum adjacent bytes three times with <code>addp</code>. That is another eight instructions or so per 64-byte block, and it is shared with the white-space mask.</p>
<p>SVE2 has an instruction, <code>match</code>, that takes a vector of bytes and a second vector that acts as a small set: it produces a predicate (a mask) with a bit set at each position where the input byte belongs to the set. Because it works within 128-bit segments, the set is at most 16 bytes, which is plenty for our six structural characters. NEON has nothing like it; on x64, the closest thing is the SSE4.2 string-comparison instructions (<code>pcmpistrm</code>), which are slow.</p>
<p>In C++, using intrinsics, the code might look as follows:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">// input is a set of 16 ASCII bytes we want to classify</span>
<span style="color: #8f5902; font-style: italic;">// the whole thing compiles to little more than the match instruction</span>
<span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">match_operators_sve2</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// The characters we care about. We use `0xff` as a</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// filler (it is an impossible byte value within a JSON document)</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">operators</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #0000cf; font-weight: bold;">0xff</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">','</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">':'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'['</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">']'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'{'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'}'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0xff</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">    </span><span style="color: #4e9a06;">','</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">':'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'['</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">']'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'{'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">'}'</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">','</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">':'</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// pg is a mask over the first 16 values</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">pg</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svptrue_pat_b8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">SV_VL16</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// 'move' the NEON register to SVE</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svuint8_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">data</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svset_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svundef_u8</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">input</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// 'move' the table to SVE</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svuint8_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">table</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svset_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svundef_u8</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">operators</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// call the match instruction</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svmatch_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">pg</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">data</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">table</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>This function <em>classifies</em> 16 ASCII bytes with maybe just one instruction (<code>match</code>).</p>
<p>We use <code>svset_neonq_u8</code>, which is part of the NEON-SVE bridge. It allows you to mix and match NEON and SVE. The <code>uint8x16_t</code> type is a NEON type (16 8-bit integers). The type <code>svuint8_t</code> is an SVE type (a vector of 8-bit integers). As a convention, the first 16 bytes of SVE types are shared with NEON. Thus I expect that <code>svuint8_t data = svset_neonq_u8(svundef_u8(), input)</code> might compile to nothing.</p>
<p>The catch, as I explained in April, is that a predicate lives in a predicate register. My function returns <code>svbool_t</code>. SVE gives you no cheap way to move it to a general-purpose register: the architecture does not want to assume that a mask fits in 16 bits, since the registers might be wider. So we materialize the predicate as bytes instead, with a predicated select (<code>svsel</code>). The whole thing is a bit complicated (see <a href="https://doi.org/10.1002/spe.3420">Lemire (2025)</a> for an explanation of the trick).</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">// We have four masks, p0, p1, p2, p3</span>
<span style="color: #8f5902; font-style: italic;">// and we want to convert them each to a 16-bit value and then combine them to</span>
<span style="color: #8f5902; font-style: italic;">// form a 64-bit mask.</span>
<span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">operator_predicates_to_bytes</span><span style="color: #000; font-weight: bold;">(</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p2</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svbool_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">p3</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">bit_mask</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">0x01</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x02</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x4</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x10</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x20</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x40</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x80</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">                         </span><span style="color: #0000cf; font-weight: bold;">0x01</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x02</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x4</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x8</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x10</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x20</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x40</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0x80</span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// map the NEON register bit_mask to an SVE register</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svuint8_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">weights</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svset_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svundef_u8</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">bit_mask</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// create a zero register</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svuint8_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svdup_n_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// where p0 is set, put the value from bit_mask, otherwise zero</span>
<span style="color: #f8f8f8;">  </span><span style="color: #8f5902; font-style: italic;">// The `svget_neonq_u8` function is part of the NEON-SVE bridge.</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b0</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svget_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svsel_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">weights</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b1</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svget_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svsel_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">weights</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b2</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svget_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svsel_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p2</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">weights</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b3</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">svget_neonq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">svsel_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">p3</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">weights</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">const</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">uint8x16_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">sum</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vpaddq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">vpaddq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">b0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b1</span><span style="color: #000; font-weight: bold;">),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vpaddq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">b2</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">b3</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">vgetq_lane_u64</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">vreinterpretq_u64_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">vpaddq_u8</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">sum</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">sum</span><span style="color: #000; font-weight: bold;">)),</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>Does it help?</p>
<p>I benchmarked the <code>match</code> classifier against the NEON classifier, using the <code>parse</code> benchmark that comes with simdjson, over the 22 JSON files that we use as our standard corpus (about 24 MB in total). The benchmark parses each file 300 times and keeps the best time. I ran each binary three times, interleaved with its counterpart, pinned to one core, and I kept the best. The run-to-run variation is under 1%. I used GCC 15 and LLVM clang 21 on Ubuntu 26.04, with <code>-mcpu=native</code>.</p>
<p>The <code>match</code> instruction is part of SVE2, not the original SVE. Among the AWS Graviton processors, the Graviton 3 (Neoverse V1) has SVE but not SVE2: it cannot run this code. So I used the two processors that can:</p>
<ul>
<li>Graviton 4 (Neoverse V2) on a <code>c8g.2xlarge</code> instance,</li>
<li>Graviton 5 (Neoverse V3) on a <code>c9g.2xlarge</code> instance.</li>
</ul>
<p>Both have 128-bit SVE registers.</p>
<p>Here is the gain in the indexing stage (stage 1), file by file, as the percentage increase in throughput with <code>match</code> over NEON. The dashed line in each panel is the geometric mean over the 22 files. First with GCC:<br />
<a href="http://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc.webp"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-1024x768.webp" alt="" width="660" height="495" class="alignnone size-large wp-image-22902" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-1024x768.webp 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-300x225.webp 300w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-768x576.webp 768w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-1536x1152.webp 1536w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_gcc-2048x1536.webp 2048w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>And with clang:</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang.webp"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-1024x768.webp" alt="" width="660" height="495" class="alignnone size-large wp-image-22903" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-1024x768.webp 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-300x225.webp 300w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-768x576.webp 768w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-1536x1152.webp 1536w, https://lemire.me/blog/wp-content/uploads/2026/09/speedup_clang-2048x1536.webp 2048w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>The files that gain the least (<code>canada</code>, <code>mesh</code>, <code>marine_ik</code>) are mostly numbers, where the indexing stage is cheap to begin with. The files that gain the most (<code>gsoc-2018</code>, <code>random</code>, <code>github_events</code>) are the ones with a lot of structure. No file gets slower, except <code>canada</code> and <code>mesh</code> on the Graviton 5 with GCC (by 2% to 3%, at the edge of what I can measure).</p>
<p>In absolute terms, the indexing stage goes from 4.8 GB/s to 5.3 GB/s on the Graviton 4 with clang (5.5 GB/s to 5.8 GB/s with GCC), and from 6.3 GB/s to 6.6 GB/s on the Graviton 5 with clang (7.1 GB/s to 7.3 GB/s with GCC). The Graviton 4 benefits more than the Graviton 5.</p>
<p>The second stage of the parser is untouched, so the gain on the whole parse is smaller: 2% to 4% on the Graviton 4 and 1% to 2% on the Graviton 5. It is a modest gain, but it comes from replacing four NEON instructions with one, in a routine that we had already tuned carefully.</p>
<p>In a real parser, <code>match</code> gives 3% to 9% faster indexing on Graviton 4 and Graviton 5, with a handful of intrinsics and no assembly. The code is in <a href="https://github.com/simdjson/simdjson/pull/2866">simdjson pull request 2866</a>. It requires SVE2, which Apple processors and the older Graviton processors do not have, so simdjson falls back on NEON when SVE2 is not available at compile time.</p>
<p>The limitation today is that the code is compiled in only if you build with <code>-mcpu=native</code> or the equivalent: a default build gets the NEON code everywhere. The next step for simdjson is to select the SVE2 code at runtime, as we do with the various x64 instruction sets, so that a default build uses it on processors that have the instruction. I am working on it.</p>
<p><em>Credit</em>: The <code>match</code> classifier is the work of <a href="https://github.com/mpurbay-arm">Madhurendra Purbay</a> (ARM), from his <a href="https://github.com/simdjson/simdjson/pull/2863">pull request 2863</a>, where he used a different (and slightly faster) technique, with inline assembly, to extract the predicates. <a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/svehelp">My benchmark results and scripts are available</a>.</p>
<p><em>References</em></p>
<p>Keiser, J., &amp; Lemire, D. (2024). <a href="https://doi.org/10.1002/spe.3313">On-demand JSON: A better way to parse documents?</a>. Software: Practice and Experience, 54(6), 1074-1086.</p>
<p>Langdale, G., &amp; Lemire, D. (2019). <a href="https://doi.org/10.1007/s00778-019-00578-5">Parsing gigabytes of JSON per second</a>. The VLDB Journal, 28(6), 941-960. (<a href="https://arxiv.org/abs/1902.08318">arXiv</a>)</p>
<p>Lemire, D. (2025). <a href="https://lemire.me/blog/2025/03/29/mixing-arm-neon-with-sve-code-for-fun-and-profit/">Mixing ARM NEON with SVE code for fun and profit</a>.</p>
<p>Lemire, D. (2025). <a href="https://doi.org/10.1002/spe.3420">Scanning HTML at tens of gigabytes per second on ARM processors</a>. Software: Practice and Experience, 55(7), 1256-1265.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/18/faster-json-parsing-with-sve2-on-arm-processors/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>How did AMD Ryzen get 50% faster in two years?</title>
		<link>https://lemire.me/blog/2026/09/18/how-did-amd-ryzen-get-50-faster-in-two-years/</link>
					<comments>https://lemire.me/blog/2026/09/18/how-did-amd-ryzen-get-50-faster-in-two-years/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Fri, 18 Sep 2026 17:26:04 +0000</pubDate>
				<category><![CDATA[Science and Technology]]></category>
		<guid isPermaLink="false">http://lemire.me/blog/?p=22896</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Three desktop processors in a row, from nearest to farthest" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />People still tell me that CPUs are boring. That nothing much happens anymore. Let us look at AMD Ryzen 7 processors from 2022 to 2024: the 5800X3D (Zen 3), the 7800X3D (Zen 4) and the 9800X3D (Zen 5). They are comparable 8-core chips with 3D V-Cache. I have written about them before, in How stagnant &#8230; <a href="https://lemire.me/blog/2026/09/18/how-did-amd-ryzen-get-50-faster-in-two-years/" class="more-link">Continue reading <span class="screen-reader-text">How did AMD Ryzen get 50% faster in two years?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-featured-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="Three desktop processors in a row, from nearest to farthest" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>People still tell me that CPUs are boring. That nothing much happens anymore.</p>
<p>Let us look at AMD Ryzen 7 processors from 2022 to 2024: the 5800X3D (Zen 3), the 7800X3D (Zen 4) and the 9800X3D (Zen 5). They are comparable 8-core chips with 3D V-Cache. I have written about them before, in <a href="https://lemire.me/blog/2026/01/14/how-stagnant-is-cpu-technology/">How stagnant is CPU technology?</a></p>
<p>On Geekbench 6, performance went up by about 50% in two years.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6-1024x424.jpg" alt="Geekbench 6 single-core and multi-core scores for Ryzen 7 5800X3D, 7800X3D and 9800X3D" width="660" height="273" class="alignnone size-large wp-image-22891" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6-1024x424.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6-300x124.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6-768x318.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-geekbench6.jpg 1199w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
5800X3D<br />
Zen 3</td>
<td>2023<br />
7800X3D<br />
Zen 4</td>
<td>2024<br />
9800X3D<br />
Zen 5</td>
</tr>
<tr>
<td>Single-core</td>
<td>2,016</td>
<td>2,426</td>
<td>2,969</td>
</tr>
<tr>
<td>Multi-core</td>
<td>11,832</td>
<td>15,508</td>
<td>18,751</td>
</tr>
</tbody>
</table>
<p>The 2024 chip is 47% faster on a single core than the 2022 chip, and 58% faster with all cores.</p>
<p>The clock did not do that. Max boost went from 4.5 GHz to 5.2 GHz, a 15% increase.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency-1024x753.jpg" alt="Base and max boost frequencies for Ryzen 7 5800X3D, 7800X3D and 9800X3D" width="660" height="485" class="alignnone size-large wp-image-22892" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency-1024x753.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency-300x221.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency-768x564.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-frequency.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>2022<br />
Zen 3</td>
<td>2023<br />
Zen 4</td>
<td>2024<br />
Zen 5</td>
</tr>
<tr>
<td>Base</td>
<td>3.4 GHz</td>
<td>4.2 GHz</td>
<td>4.7 GHz</td>
</tr>
<tr>
<td>Max boost</td>
<td>4.5 GHz</td>
<td>5.0 GHz</td>
<td>5.2 GHz</td>
</tr>
</tbody>
</table>
<p>The number of transistors is way up, by about 50%, from roughly 11 billion to 16 billion. Most of the extra transistors went into the core, not the cache.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors-1024x564.jpg" alt="Transistor counts for Ryzen 7 5800X3D, 7800X3D and 9800X3D, split into core, I/O and cache" width="660" height="364" class="alignnone size-large wp-image-22893" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors-1024x564.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors-300x165.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors-768x423.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-transistors.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<p>How do you turn extra transistors into extra performance?</p>
<p>You make the core wider, and you give it more to work with. Dispatch width went from a maximum of 6 instructions per cycle to 8. The L2 cache per core doubled, from 512 KB to 1 MB. The L1 data cache went from 32 KB to 48 KB. Integer ALUs went from 4 to 6. The reorder buffer grew from 256 to 448 entries, so the processor can keep more instructions in flight and schedule them better.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes-1024x596.jpg" alt="Cache sizes, dispatch width, integer ALUs and reorder buffer from Zen 3 to Zen 5" width="660" height="384" class="alignnone size-large wp-image-22894" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes-1024x596.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes-300x175.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes-768x447.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-core-changes.jpg 1200w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>Zen 3<br />
2022</td>
<td>Zen 4<br />
2023</td>
<td>Zen 5<br />
2024</td>
</tr>
<tr>
<td>L2 cache per core</td>
<td>512 KB</td>
<td>1 MB</td>
<td>1 MB</td>
</tr>
<tr>
<td>L1 data cache</td>
<td>32 KB</td>
<td>32 KB</td>
<td>48 KB</td>
</tr>
<tr>
<td>Dispatch width</td>
<td>6</td>
<td>6</td>
<td>8</td>
</tr>
<tr>
<td>Integer ALUs</td>
<td>4</td>
<td>4</td>
<td>6</td>
</tr>
<tr>
<td>Reorder buffer</td>
<td>256</td>
<td>320</td>
<td>448</td>
</tr>
</tbody>
</table>
<p>For data parallelism (SIMD), Zen 5 is a different machine. Zen 3 and Zen 4 had four 256-bit SIMD arithmetic units. Zen 5 has four 512-bit units. Loads and stores widened the same way: two 512-bit loads per cycle, one 512-bit store.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd.jpg"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd-1024x341.jpg" alt="SIMD arithmetic units, loads per cycle and stores per cycle from Zen 3 to Zen 5" width="660" height="220" class="alignnone size-large wp-image-22895" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd-1024x341.jpg 1024w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd-300x100.jpg 300w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd-768x256.jpg 768w, https://lemire.me/blog/wp-content/uploads/2026/09/ryzen-simd.jpg 1199w" sizes="auto, (max-width: 660px) 100vw, 660px" /></a></p>
<table>
<tbody>
<tr>
<td></td>
<td>Zen 3<br />
2022</td>
<td>Zen 4<br />
2023</td>
<td>Zen 5<br />
2024</td>
</tr>
<tr>
<td>SIMD arithmetic units</td>
<td>4 × 256-bit</td>
<td>4 × 256-bit</td>
<td>4 × 512-bit</td>
</tr>
<tr>
<td>Loads per cycle</td>
<td>2 × 256-bit</td>
<td>2 × 256-bit</td>
<td>2 × 512-bit</td>
</tr>
<tr>
<td>Stores per cycle</td>
<td>1 × 256-bit</td>
<td>1 × 256-bit</td>
<td>1 × 512-bit</td>
</tr>
</tbody>
</table>
<p>I already made the point that <a href="https://lemire.me/blog/2025/09/01/processors-are-getting-wider/">processors are getting wider</a>. This is what that looks like on a desktop chip you can buy.</p>
<p>What about the next step? Zen 6 is arriving. AMD is talking about a 256-core Epyc part (Venice) with a gigabyte of L3 cache. We do not yet know what the desktop cores will look like. It could be wild.</p>
<p>Further reading: <a href="https://www.tomshardware.com/pc-components/cpus/amds-256-core-epyc-9996-venice-claims-up-to-a-3-4x-jump-over-intel-xeon-competition-20-percent-over-nvidia-vera-zen-6-comes-with-up-to-1024mb-of-l3-16-channel-memory-and-5ghz-clock-speeds">AMD’s 256-core Epyc 9996 ‘Venice’</a> (Tom&#8217;s Hardware).</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/18/how-did-amd-ryzen-get-50-faster-in-two-years/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>How fast is C++23&#8217;s std::flat_map?</title>
		<link>https://lemire.me/blog/2026/09/16/how-fast-is-c23s-stdflat_map/</link>
					<comments>https://lemire.me/blog/2026/09/16/how-fast-is-c23s-stdflat_map/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 16 Sep 2026 20:26:36 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22881</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-16-a-16.20.19-e1789590380973-150x150.png" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />C++23 added a new type to the standard library: std::flat_map. There is also a std::flat_set and other variants, but let me focus on std::flat_map. A flat map is a sorted vector of keys next to a vector of values. A query is a binary search over the sorted keys. You need a recent standard library: &#8230; <a href="https://lemire.me/blog/2026/09/16/how-fast-is-c23s-stdflat_map/" class="more-link">Continue reading <span class="screen-reader-text">How fast is C++23&#8217;s std::flat_map?</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Capture-decran-le-2026-09-16-a-16.20.19-e1789590380973-150x150.png" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>C++23 added a new type to the standard library: <code>std::flat_map</code>. There is also a <code>std::flat_set</code> and other variants, but let me focus on <code>std::flat_map</code>.</p>
<p>A flat map is a sorted vector of keys next to a vector of values. A query is a binary search over the sorted keys.</p>
<p>You need a recent standard library: GCC 15&#8217;s libstdc++ has <code>std::flat_map</code>, as does LLVM&#8217;s libc++ since LLVM 20 (Apple&#8217;s clang 17 has it too).</p>
<p>Because the container is just two arrays, we can save it by copying the two arrays. Take a map from 64-bit keys to a fixed-size value:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">struct</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">point</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">double</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">x</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">  </span><span style="color: #204a87; font-weight: bold;">double</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">y</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">};</span>
<span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">flat_map</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">point</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">42</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">1.0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">2.0</span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">7</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">3.0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">4.0</span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">1000</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">5.0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">6.0</span><span style="color: #000; font-weight: bold;">};</span>
<span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">vector</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">keys</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">keys</span><span style="color: #000; font-weight: bold;">().</span><span style="color: #000;">begin</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">keys</span><span style="color: #000; font-weight: bold;">().</span><span style="color: #000;">end</span><span style="color: #000; font-weight: bold;">());</span>
<span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">vector</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #000;">point</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">().</span><span style="color: #000;">begin</span><span style="color: #000; font-weight: bold;">(),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">().</span><span style="color: #000;">end</span><span style="color: #000; font-weight: bold;">());</span>
<span style="color: #8f5902; font-style: italic;">// keys   = {7, 42, 1000}</span>
<span style="color: #8f5902; font-style: italic;">// values = {{3, 4}, {1, 2}, {5, 6}}</span>
</code></pre>
</div>
<p>The <code>keys</code> array is <code>8 * keys.size()</code> bytes and the <code>values</code> array is <code>16 * values.size()</code> bytes: you can write them to disk with two <code>memcpy</code> or <code>write</code> calls, as they are.</p>
<p>If you loaded both arrays from a network or the disk, you can then move them straight into your <code>std::flat_map</code> like so.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">flat_map</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">point</span><span style="color: #ce5c00; font-weight: bold;">&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">back</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">sorted_unique</span><span style="color: #000; font-weight: bold;">,</span>
<span style="color: #f8f8f8;">                                    </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">move</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">keys</span><span style="color: #000; font-weight: bold;">),</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">move</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">));</span>
<span style="color: #8f5902; font-style: italic;">// back == m</span>
</code></pre>
</div>
<p>With the <code>std::sorted_unique</code> tag, you promise that the keys are sorted and distinct. In practice, if the data comes from the network or some untrusted source, you should do some sanity testing.</p>
<p>Like the good old <code>std::map</code>, a flat map keeps its keys in sorted order, so you can iterate over it in key order. But the <code>std::map</code> is a red-black tree so you have significant storage overhead and possibly poor memory locality.</p>
<p>The downside of a <code>std::flat_map</code> is that it might be slower if you need to mutate it.</p>
<p>Let us examine the speed.</p>
<p>I use random 64-bit keys mapped to 64-bit values, GCC 16.1 with <code>-O3<br />
-march=native</code>, on an Intel Xeon Gold 6548N (Emerald Rapids), pinned to one core. I report nanoseconds per operation.</p>
<p>Let us start with inserting elements one at a time, in random order, into an initially empty container.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">container</th>
<th style="text-align: right;">1K</th>
<th style="text-align: right;">100K</th>
<th style="text-align: right;">1M</th>
<th style="text-align: right;">10M</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::map</code></td>
<td style="text-align: right;">93</td>
<td style="text-align: right;">253</td>
<td style="text-align: right;">465</td>
<td style="text-align: right;">1118</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::flat_map</code></td>
<td style="text-align: right;">83</td>
<td style="text-align: right;">7149</td>
<td style="text-align: right;"></td>
<td style="text-align: right;"></td>
</tr>
</tbody>
</table>
<p>Up to maybe a thousand elements or so, the <code>std::flat_map</code> is fine and maybe faster than the <code>std::map</code>. But as the size grows, the time goes up quadratically. Thus, do not use an <code>std::flat_map</code> if you need to insert millions of keys in random order. It is bad.</p>
<p>If the keys arrive in increasing order, then it is entirely different. The <code>std::flat_map</code> is much faster.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">container</th>
<th style="text-align: right;">1K</th>
<th style="text-align: right;">100K</th>
<th style="text-align: right;">1M</th>
<th style="text-align: right;">10M</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::map</code></td>
<td style="text-align: right;">31</td>
<td style="text-align: right;">67</td>
<td style="text-align: right;">125</td>
<td style="text-align: right;">202</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::flat_map</code></td>
<td style="text-align: right;">6.8</td>
<td style="text-align: right;">12</td>
<td style="text-align: right;">15</td>
<td style="text-align: right;">19</td>
</tr>
</tbody>
</table>
<p>You can also do bulk inserts. Given a batch of new pairs, in any order, the map sorts the batch and merges it with its arrays in one pass, instead of shifting the arrays once per element:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">vector</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #000;">std</span><span style="color: #ce5c00; font-weight: bold;">::</span><span style="color: #000;">pair</span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #204a87; font-weight: bold;">uint64_t</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">point</span><span style="color: #ce5c00; font-weight: bold;">&gt;&gt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">more</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{{</span><span style="color: #0000cf; font-weight: bold;">500</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">7.0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">8.0</span><span style="color: #000; font-weight: bold;">}},</span>
<span style="color: #f8f8f8;">                                                </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">3</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span><span style="color: #0000cf; font-weight: bold;">9.0</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">10.0</span><span style="color: #000; font-weight: bold;">}}};</span>
<span style="color: #000;">m</span><span style="color: #000; font-weight: bold;">.</span><span style="color: #000;">insert_range</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">more</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #8f5902; font-style: italic;">// keys = {3, 7, 42, 500, 1000}</span>
</code></pre>
</div>
<p>Constructing a map from a range of random key-value pairs works the same way, and it is much faster with a <code>std::flat_map</code>:</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">container</th>
<th style="text-align: right;">1K</th>
<th style="text-align: right;">100K</th>
<th style="text-align: right;">1M</th>
<th style="text-align: right;">10M</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::map</code></td>
<td style="text-align: right;">69</td>
<td style="text-align: right;">176</td>
<td style="text-align: right;">316</td>
<td style="text-align: right;">825</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::flat_map</code></td>
<td style="text-align: right;">23</td>
<td style="text-align: right;">62</td>
<td style="text-align: right;">72</td>
<td style="text-align: right;">87</td>
</tr>
</tbody>
</table>
<p>Random lookups are also much faster for large maps in part because the <code>std::flat_map</code> uses less memory.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">container</th>
<th style="text-align: right;">1K</th>
<th style="text-align: right;">100K</th>
<th style="text-align: right;">1M</th>
<th style="text-align: right;">10M</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::map</code></td>
<td style="text-align: right;">52</td>
<td style="text-align: right;">192</td>
<td style="text-align: right;">386</td>
<td style="text-align: right;">980</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;"><code>std::flat_map</code></td>
<td style="text-align: right;">54</td>
<td style="text-align: right;">96</td>
<td style="text-align: right;">152</td>
<td style="text-align: right;">239</td>
</tr>
</tbody>
</table>
<p>In many practical cases, the new <code>std::flat_map</code> is a better alternative to the <code>std::map</code>. It is somewhat amusing considering that you are replacing a fancy textbook data structure (red-black tree) with a trivial one.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/17">My source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/16/how-fast-is-c23s-stdflat_map/feed/</wfw:commentRss>
			<slash:comments>4</slash:comments>
		
		
			</item>
		<item>
		<title>Subnormal floating-point numbers are expensive&#8230; on Intel processors</title>
		<link>https://lemire.me/blog/2026/09/15/subnormal-floating-point-numbers-are-expensive-on-intel-processors/</link>
					<comments>https://lemire.me/blog/2026/09/15/subnormal-floating-point-numbers-are-expensive-on-intel-processors/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Tue, 15 Sep 2026 12:54:32 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22876</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_gh1asmgh1asmgh1a-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />We represent floating-point numbers using the IEEE standard. For very small numbers, the standard uses special subnormal numbers. Unfortunately, they have a reputation of making operations slow. Thus video game programmers and machine learning specialists sometimes avoid computing with subnormal numbers for performance. How slow are they? Let me measure. I wrote a small C++ &#8230; <a href="https://lemire.me/blog/2026/09/15/subnormal-floating-point-numbers-are-expensive-on-intel-processors/" class="more-link">Continue reading <span class="screen-reader-text">Subnormal floating-point numbers are expensive&#8230; on Intel processors</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_gh1asmgh1asmgh1a-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>We represent floating-point numbers using the IEEE standard. For very small numbers, the standard uses special <em>subnormal</em> numbers. Unfortunately, they have a reputation of making operations slow. Thus video game programmers and machine learning specialists sometimes avoid computing with subnormal numbers for performance.</p>
<p>How slow are they? Let me measure. I wrote a small C++ benchmark with a few kernels over arrays of 16384 values (small enough to fit in cache):</p>
<ul>
<li>multiply each value by 0.75,</li>
<li>add two arrays,</li>
<li>divide each value by 3,</li>
<li>multiply normal values by a tiny constant (2<sup>-1030</sup>) so that the <em>inputs</em> are normal but the <em>outputs</em> are subnormal,</li>
<li>a dependent chain <code>x *= 0.9999</code> repeated 16384 times.</li>
</ul>
<p>For each kernel, I feed either normal values (in <code>[0.5, 1)</code>), subnormal values, or normal values where one value in a hundred is subnormal. The compiler is allowed to autovectorize the array computations. I use GCC 15 with <code>-O3 -march=native</code> on Linux and Apple clang 17 with the same flags on macOS. I also checked with clang 21 on Linux to make sure.</p>
<p>I ran the benchmark on five processors:</p>
<ul>
<li>Intel Xeon 6975P-C (Granite Rapids), on an AWS <code>c8i.xlarge</code> instance,</li>
<li>Intel Xeon Gold 6548N (Emerald Rapids), a server in my lab,</li>
<li>AMD EPYC 9R45 (Zen 5), on an AWS <code>c8a.xlarge</code> instance,</li>
<li>AWS Graviton 5 (Arm Neoverse V3), on a <code>c9g.xlarge</code> instance,</li>
<li>Apple M4 Max.</li>
</ul>
<p>Here are the results for <code>double</code> values, in nanoseconds per element (or per step).</p>
<p><em>Intel Granite Rapids</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">kernel</th>
<th style="text-align: right;">normal</th>
<th style="text-align: right;">1% subnormal</th>
<th style="text-align: right;">subnormal</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">multiply by 0.75</td>
<td style="text-align: right;">0.17</td>
<td style="text-align: right;">0.44</td>
<td style="text-align: right;">8.35</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">add two arrays</td>
<td style="text-align: right;">0.20</td>
<td style="text-align: right;">0.18</td>
<td style="text-align: right;">0.18</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">divide by 3</td>
<td style="text-align: right;">0.51</td>
<td style="text-align: right;">0.85</td>
<td style="text-align: right;">9.38</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">normal in, subnormal out</td>
<td style="text-align: right;">0.17</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">8.58</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">dependent chain</td>
<td style="text-align: right;">0.77</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">32.69</td>
</tr>
</tbody>
</table>
<p><em>Intel Emerald Rapids</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">kernel</th>
<th style="text-align: right;">normal</th>
<th style="text-align: right;">1% subnormal</th>
<th style="text-align: right;">subnormal</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">multiply by 0.75</td>
<td style="text-align: right;">0.21</td>
<td style="text-align: right;">0.49</td>
<td style="text-align: right;">9.25</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">add two arrays</td>
<td style="text-align: right;">0.23</td>
<td style="text-align: right;">0.25</td>
<td style="text-align: right;">0.25</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">divide by 3</td>
<td style="text-align: right;">0.57</td>
<td style="text-align: right;">0.94</td>
<td style="text-align: right;">10.40</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">normal in, subnormal out</td>
<td style="text-align: right;">0.21</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">9.27</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">dependent chain</td>
<td style="text-align: right;">1.14</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">36.51</td>
</tr>
</tbody>
</table>
<p><em>AMD Zen 5</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">kernel</th>
<th style="text-align: right;">normal</th>
<th style="text-align: right;">1% subnormal</th>
<th style="text-align: right;">subnormal</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">multiply by 0.75</td>
<td style="text-align: right;">0.07</td>
<td style="text-align: right;">0.10</td>
<td style="text-align: right;">0.08</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">add two arrays</td>
<td style="text-align: right;">0.09</td>
<td style="text-align: right;">0.09</td>
<td style="text-align: right;">0.09</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">divide by 3</td>
<td style="text-align: right;">0.11</td>
<td style="text-align: right;">0.24</td>
<td style="text-align: right;">0.25</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">normal in, subnormal out</td>
<td style="text-align: right;">0.07</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.07</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">dependent chain</td>
<td style="text-align: right;">0.66</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.88</td>
</tr>
</tbody>
</table>
<p><em>AWS Graviton 5</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">kernel</th>
<th style="text-align: right;">normal</th>
<th style="text-align: right;">1% subnormal</th>
<th style="text-align: right;">subnormal</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">multiply by 0.75</td>
<td style="text-align: right;">0.17</td>
<td style="text-align: right;">0.17</td>
<td style="text-align: right;">0.16</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">add two arrays</td>
<td style="text-align: right;">0.19</td>
<td style="text-align: right;">0.20</td>
<td style="text-align: right;">0.20</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">divide by 3</td>
<td style="text-align: right;">0.30</td>
<td style="text-align: right;">0.30</td>
<td style="text-align: right;">0.30</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">normal in, subnormal out</td>
<td style="text-align: right;">0.17</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.17</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">dependent chain</td>
<td style="text-align: right;">0.91</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.91</td>
</tr>
</tbody>
</table>
<p><em>Apple M4 Max</em></p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: left; font-weight: 600; border-bottom: 2px solid #d0d7de; padding: 0.65rem 0.8rem; vertical-align: top;">kernel</th>
<th style="text-align: right;">normal</th>
<th style="text-align: right;">1% subnormal</th>
<th style="text-align: right;">subnormal</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">multiply by 0.75</td>
<td style="text-align: right;">0.06</td>
<td style="text-align: right;">0.06</td>
<td style="text-align: right;">0.06</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">add two arrays</td>
<td style="text-align: right;">0.12</td>
<td style="text-align: right;">0.12</td>
<td style="text-align: right;">0.12</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">divide by 3</td>
<td style="text-align: right;">0.11</td>
<td style="text-align: right;">0.11</td>
<td style="text-align: right;">0.11</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">normal in, subnormal out</td>
<td style="text-align: right;">0.06</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.06</td>
</tr>
<tr style="background: #ffffff;">
<td style="padding: 0.65rem 0.8rem; border-bottom: 1px solid #eaeef2; vertical-align: top;">dependent chain</td>
<td style="text-align: right;">0.72</td>
<td style="text-align: right;"></td>
<td style="text-align: right;">0.75</td>
</tr>
</tbody>
</table>
<p>On Intel processors, a multiplication involving a subnormal number is about 45 to 50 times slower than a multiplication over normal numbers. A division is 18 times slower. The dependent chain, where each multiplication waits for the previous one, goes from about 1 ns to over 30 ns per step. A normal multiplication in the dependent chain has a latency of 4 cycles. With a subnormal, it has a latency of 128 cycles. It does not matter whether the subnormal is an input or an output: multiplying normal numbers into a subnormal result is just as slow as multiplying subnormal numbers. The exception is additions and subtractions: they run at full speed. Even if subnormals are rare (1%), the cost on Intel can be significant because when the compiler vectorizes the computation, a single subnormal can slow down a whole block of computations.</p>
<p>AMD does much better. On Zen 5, multiplications and additions run at full speed regardless of the inputs. The dependent multiplication chain is a third slower (0.66 ns to 0.88 ns per step): the multiplier needs an extra cycle or so to handle a subnormal. Divisions are about twice as slow. Interestingly, with divisions, having one subnormal in a hundred is almost as slow as having all subnormals. The two Arm processors, the Graviton 5 and the Apple M4 Max, do not care at all. Subnormal numbers are handled at full speed.</p>
<p>Thus it appears that on the latest AMD and ARM processors subnormals might not be a concern. But they remain very much a performance issue on Intel processors.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/15">My source code is available</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/15/subnormal-floating-point-numbers-are-expensive-on-intel-processors/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>The four-colour theorem was only the start</title>
		<link>https://lemire.me/blog/2026/09/11/the-four-colour-theorem-was-only-the-start/</link>
					<comments>https://lemire.me/blog/2026/09/11/the-four-colour-theorem-was-only-the-start/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Fri, 11 Sep 2026 19:53:31 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22869</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/kALnB-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />Mathematicians are unhappy about OpenAI. Several influential mathematicians wrote an open letter. The gist of their argument is that they form a community that trains young people. When AI started producing breakthroughs on hard mathematical problems, I asked what a very smart 17-year-old would feel. Do you still choose a math major and train yourself &#8230; <a href="https://lemire.me/blog/2026/09/11/the-four-colour-theorem-was-only-the-start/" class="more-link">Continue reading <span class="screen-reader-text">The four-colour theorem was only the start</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/kALnB-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p><iframe loading="lazy" width="560" height="315" src="https://www.youtube.com/embed/exa7EX5YYeA?si=n58hi6V7oVusOB27" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen="allowfullscreen"></iframe></p>
<p><span class="css-1jxf684 r-bcqeeo r-1ttztb7 r-qvutc0 r-poiln3"><br />
Mathematicians are unhappy about OpenAI. <a href="https://mathandai.org">Several influential mathematicians wrote an open letter.</a> The gist of their argument is that they form a community that trains young people. </span></p>
<p>When AI started producing breakthroughs on hard mathematical problems, I asked what a very smart 17-year-old would feel. Do you still choose a math major and train yourself to prove difficult results by hand?</p>
<p>This crisis has been coming for a long time. When I was a kid, the four-colour theorem was proved by a computer, in 1976. An intense debate followed. Does it count as a proof?</p>
<p>I wrote my PhD thesis using symbolic algebra software. To my knowledge, nobody then would publish their scripts. I tried to include mine. I was told it would make me look bad.</p>
<p>The letter says:</p>
<blockquote><p>&#8220;In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas.&#8221;</p></blockquote>
<p>They worry that kids will not choose to become old-school mathematicians. That is a reasonable fear. Yet they do not seem to consider that some kids might still do mathematics, just in a very different way.</p>
<p><a href="https://sites.math.rutgers.edu/~zeilberg/Opinion94.html">The famous mathematician Doron Zeilberger announced the problem in 2009</a>:</p>
<blockquote><p>&#8220;Teaching computers how to discover and prove mathematical results is certainly the way to go, and I believe that mathematicians who continue to do pure human, pencil-and-paper, computer-less, research, are wasting their time.&#8221;</p></blockquote>
<p>The letter implies that people, because of AI, will stop having ideas, or will stop taking the time to understand the issues. If that were true, mathematics would continue only on computers, with no humans interested, or it would stop. Both scenarios assume that people care about mathematics only when they can claim credit. </p>
<p>I am not sure why I cannot study a proof generated by AI if I want to. I can give talks about it. What becomes less likely is the reward of having been the one who proved the result. </p>
<p>The rest of the letter makes a big deal of credit. What if the AI builds on what it read and does not give proper credit? Where is the evidence that AI is worse than human beings at citing sources? </p>
<p>The letter notes that mathematics contributes to society. It never considers that faster progress might increase those contributions. </p>
<p>My stance is simple. </p>
<p>Mathematicians, software developers, engineers, lawyers, physicians will all learn to work with AI. You cannot put the genie back in the bottle. Only a totalitarian world government could try, and some of us would rather avoid that outcome. </p>
<p>Difficult proofs will now be built with AI, just as most code will be written with AI. People who insist on pen and paper should view themselves as artists. </p>
<p>Other mathematicians will work with AI. They will have no trouble finding interested kids. They will contribute to society.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/11/the-four-colour-theorem-was-only-the-start/feed/</wfw:commentRss>
			<slash:comments>2</slash:comments>
		
		
			</item>
		<item>
		<title>Fear Is Not an Argument</title>
		<link>https://lemire.me/blog/2026/09/10/fear-is-not-an-argument/</link>
					<comments>https://lemire.me/blog/2026/09/10/fear-is-not-an-argument/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 10 Sep 2026 18:23:42 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22845</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/wm0AF-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />We are told that AI entities much like ChatGPT might soon kill us all. The statement is vague and unfalsifiable. It might be true, it might be false. People with credentials (e.g., Turing Award recipient Yoshua Bengio) believe it. Many still remember the Year-2000 bug. Our computers used two-digit coding for dates, and some software &#8230; <a href="https://lemire.me/blog/2026/09/10/fear-is-not-an-argument/" class="more-link">Continue reading <span class="screen-reader-text">Fear Is Not an Argument</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/wm0AF-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>We are told that AI entities much like ChatGPT might soon kill us all. The statement is vague and unfalsifiable. It might be true, it might be false. People with credentials (e.g., Turing Award recipient Yoshua Bengio) believe it.</p>
<p>Many still remember the Year-2000 bug. Our computers used two-digit coding for dates, and some software could get confused. At the time, experts worried that a bug in dates might trigger nuclear Armageddon or an infrastructure collapse. At the very least, planes could fall.</p>
<p dir="auto">Americans had a moral panic over alcohol that lasted more than a century. It began with pledges of moderation. From 1920 the Volstead Act banned nearly all legal drink. It collapsed in 1933, at great expense.</p>
<p id="ember371" class="ember-view reader-text-block__paragraph">The Club of Rome predicted mass starvation. As an answer, we sterilized by force millions of Indians, and introduced the devastating one-child policy in China. The authors were never held accountable. The projections were purely mathematical, unescapable. They said. But also totally wrong and silly. Yet, we listened to them and caused great harm.</p>
<p>End-of-the-world scenarios are nothing new. Pretty much all civilizations have lived with various such predictions.</p>
<p>Some people are offended by my comparisons. I truly do not mean to offend. But the fact that disagreeing can lead to deep offense is, by itself, a sign that we face a moral issue. There is a sense in which you must agree that these intense fears are warranted.</p>
<p>Many will remember that when OpenAI first developed GPT-2, they told the world that it was too dangerous to release. Year after year, we were warned that the next iteration of it would doom us all.</p>
<p>A large language model takes tokens (words) in and outputs tokens (words). The big models can take many, many tokens in. And they do much compute. And they are based on clever ideas like vector embeddings. But, ultimately, no large language model can do anything but output tokens. You can build a better model, but the model itself does not &#8216;learn&#8217;. It is a fixed set of weights. If you take a model that has been used for months, and always feed the same tokens, you will get the same results (up to some randomness).</p>
<p>I fear that some people exploit the fact that people cannot grasp how conceptually simple a language model is. In any case, most people don&#8217;t understand how things work.</p>
<p>Things become interesting because these models can be hooked up to tools. So you can tell your language model that whenever it outputs &#8216;boom&#8217;, then a nuclear weapon will be launched. And if you hook up a nuclear weapon, then, certainly, you may start a nuclear war. So don&#8217;t do that.</p>
<p>One unproven thesis is that the models (that take in tokens and output tokens) will &#8216;decide&#8217; to acquire access to these nuclear weapons, maybe through a subterfuge. What does that mean? It is always conveniently vague. In the movie WarGames (1983), a teenager uses his computer to access a computer in charge of nuclear weapons. He almost wipes out humanity. Could this happen by accident with a kid using a language model hooked up to the Internet? But if it happens, we should blame whoever hooked up a deadly computer to the Internet.</p>
<p>Of course, any technology is inherently dangerous. Invent the bow to go hunting, and someone might soon turn the bow against you. Invent the engine, and one might soon build tanks and destroy nations. Develop nuclear technology, and one might soon raze your cities.<br />
 <br />
Yet that is not what is at stake in these discussions. The concrete threats are not ascertained and addressed. No doubt, there are some people doing this work, hopefully in the US military. What if an adversary can take control of the economy or military installations? What if an AI agent goes rogue? It is worth investing time in designing defenses.</p>
<p>What we have instead is something of the sort:</p>
<ol>
<li>A vague but global threat. It could be a fatal virus engineered in a lab, a climate catastrophe, a fatal bug affecting all our software, an alien invasion, a rogue AI, Jews taking over our institutions.</li>
<li>A few people come forward and they offer to save us. Importantly we must give them resources and influence. Ultimately, they seek a totalitarian solution: everyone must be made to agree so that we can be saved.</li>
<li>As the process unfolds, people with an opposing viewpoint are described as a danger. They must be silenced and discredited. Eventually, it can become moral to exaggerate the threat or to rewrite counterpoints. People must be made to understand one way or another.</li>
</ol>
<p>In this instance, I refer to people who advocate that AI will doom us as AI Doomers. These people tend to carry a totalitarian ideology. Their ideas will only work if everyone is made to agree. And it would severely restrict the freedom of billions of people, although they usually present it differently.</p>
<p>Doomers do not have bad intentions. On the contrary, they are often really out there to save the world. But good intentions do not, in any way, justify the means nor guarantee a good outcome.</p>
<p><span>Human beings reason based on cultural knowledge. For centuries or more, totalitarian ideas have led to ruin or pain. We ignore the warnings at our peril. It is one after the other: alcohol, the need to restrict the number of children, and so on.</span></p>
<p>But shouldn’t we just be prudent and adopt their views, just in case? It is a fallacious argument. Members of the intellectual elite have a tendency to fall for the kind of hubris where they think that, if only they were given more power, the world would be better off. It is rarely true. Thomas Sowell has an excellent book on the topic, Intellectuals and Society. He makes the case that intellectuals often promote harmful ideas, at no cost to themselves. Rationally, we should therefore be cautious.</p>
<p><span>You are not safer without technology. In fact, the risk of human extinction is assuredly higher if we are poorer and have less technology.</span></p>
<p>Is this unprecedented? The printing press was unprecedented. Arabic numbers were unprecedented. Maybe we should go back to Roman numerals, to be safe. Fear of what is without precedent soon becomes indistinguishable from an anti-innovation stance.</p>
<p>What if you do not like the people who lead the big AI companies like OpenAI and Anthropic. Maybe you think that these billionaires are a danger. And you might be right. But consider the history of humanity. Wealthy people have primarily caused harm through the promotion of bad ideas. The mass murders are almost invariably derived from politics. Stalin, Hitler, Mao.</p>
<div class="" data-block="true" data-editor="3bk51" data-offset-key="8gbvc-0-0">
<div data-offset-key="8gbvc-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="8gbvc-0-0">Are the fears grounded in reason or is some of it signaling? We have been deploying AI-enhanced drones in Ukraine for two years. Once we designate the target, they engage, autonomously. </span>At a strategic level, Palantir’s Maven Smart System is used to pick targets. It has been deployed against Iran. I have not seen much opposition to drone attacks by Ukraine against Russia, at least in the West. I cannot recall any AI Doomer denouncing Ukraine&#8217;s drones and some even endorsed them. Yet it is largely the West that is funding these drones. If you fear rogue AIs, it seems that drones able to engage a target on their own would cause enormous worry&#8230; But it would be morally inconvenient in the West to criticize the use of AI against Russia&#8230; and so, the AI Doomers are largely silent. They do not lobby their governments to require Ukraine to abstain from building AI-driven weapons. That is another sign that they do not act on reason, but, rather, on moral grounds. You might argue that these drones are not entirely autonomous, since, as far as we know, they do not pick their own targets. But ChatGPT also does not pick the prompts. The hypocrisy is par for the course <span>for many</span>. It is akin to the governor of California dining at a fancy restaurant while a stay-at-home order is in effect. Or the prime minister of Canada ranking in the top air travelers of all time, while advocating for a carbon-neutral lifestyle. You can be quite sure that many of the AI Doomers are heavy users of AI services and, in some cases, investors. It is telling you that their stance is primarily moral. Expressing fear of AI can become a form of virtue signaling. It is a convenient stance, but you would not go so far as to stop using AI, and shut down Ukraine&#8217;s drones. </p>
<p><span> In some sense, there is also a form of luxury beliefs involved. A luxury belief is a belief that makes you look good and cost you nothing, while it might harm people who are not so well-off. AI in the form of ChatGPT is proving to be a great equalizer. My plumber has access to AI that is comparable to that of a billionaire. It has the potential to serve as a superior tutor to all these kids who are left out. Many of the people engaging in the promotion of fear are either upper middle class or better. Many of them pay little attention to the fact that many of the beneficiaries of the huge investments in AI have been the men building the data centres. Thus far, AI has been great at creating jobs for blue collars. That&#8217;s not nothing.</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/HRjc6QSXYAIlS2k.png"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/HRjc6QSXYAIlS2k.png" alt="" width="600" height="731" class="alignnone size-full wp-image-22863" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/HRjc6QSXYAIlS2k.png 600w, https://lemire.me/blog/wp-content/uploads/2026/09/HRjc6QSXYAIlS2k-246x300.png 246w" sizes="auto, (max-width: 600px) 100vw, 600px" /></a><br />
</span></div>
</div>
<p>
Throughout much of the world, we are facing demographic collapse and an inverted age pyramid. Soon there may be just one worker per retiree. Choosing to have fewer kids has consequences and we are about to face them. AI might be a way out of significant problems. If you are otherwise wealthy, that might not be a significant concern to you. But for the least fortunate, it might turn out to be quite a problem. Who will take care of you when you are sick? AI and robotics might prove useful to reduce suffering, don&#8217;t we think?</p>
<p>Further, we need to consider how powerful people might use the fear of AI for their own purposes. It is entirely credible that the owners of large companies could promote fear so that they get to write the regulations that will keep out their competitors, or merely as a form of cheap marketing.</p>
<p>How do I know that it is moral? Because there cannot be reasoned debate about a moral question. The facts are obvious or you are a bad person. Whenever there is a complex question, one that involves predicting the future, that cannot be discussed, unless it is in agreement with the side of fear, then you are very likely in a moral question. « Don&#8217;t you see, AI will soon kill all of us, it is obvious. » No explanation can be demanded.</p>
<p><span>You might accuse me of, in turn, promoting fear. But it should be obvious that I am doing no such thing. What I am encouraging rather is the use of reason. I am forced to give examples where inciting fear has led to disastrous effects, but my hope is that it will lead my reader to sit and reflect.</span></p>
<p><span>To my friends who fear AI, I urge you. Use reason. Do the work. Do not rely on hasty thought experiments. Work out the details. Think. Think about the countermeasures. And, please, do not include abstract thinking machines. A language model is a box that takes in token and produces tokens. Nothing more.</span></p>
<p>And for the rest of us. Let us build. Let us bring prosperity. Let us hasten the cure for cancer. Let us dream of exploring our solar system.</p>
<p><strong>Further reading</strong>.</p>
<ul>
<li style="list-style-type: none;">
<ul>
<li><span>Nirit Weiss-Blatt, “<a href="https://www.aipanic.news/p/the-weak-foundations-of-ai-doomsday">The Weak Foundations of AI Doomsday”</a> — AI Panic, May 15, 2026.</span></li>
<li><span></span>Cal Newport, “<a href="https://www.nytimes.com/2026/06/17/opinion/ai-dangerous-openai-anthropic.html">Dear A.I. Companies, the Doom Trolling Needs to Stop</a>” — New York Times, June 17, 2026</li>
<li>Andrew Orlowski, “<a href="https://www.telegraph.co.uk/news/2026/06/05/anthropics-ai-doom-predictions-hype-share-price/">Anthropic’s doom predictions are merely hype intended to make AI look important</a>” — The Telegraph, June 5, 2026.</li>
</ul>
</li>
</ul>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/10/fear-is-not-an-argument/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>A quick overview of atomics in C</title>
		<link>https://lemire.me/blog/2026/09/09/a-quick-overview-of-atomics-in-c/</link>
					<comments>https://lemire.me/blog/2026/09/09/a-quick-overview-of-atomics-in-c/#respond</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Wed, 09 Sep 2026 20:41:53 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22835</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/image-4-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />If you write in C, by default, you use a single thread. Extra cores do not help until you create more threads. However, if you include the header &#60;threads.h&#62;, you can pass a function to thrd_create, and wait for it with thrd_join. #include &#60;threads.h&#62; #include &#60;stdio.h&#62; int worker(void *arg) { printf("hello from thread %d\n", *(int &#8230; <a href="https://lemire.me/blog/2026/09/09/a-quick-overview-of-atomics-in-c/" class="more-link">Continue reading <span class="screen-reader-text">A quick overview of atomics in C</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/image-4-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>If you write in C, by default, you use a single thread. Extra cores do not help until you create more threads. However, if you include the header <code>&lt;threads.h&gt;</code>, you can pass a function to <code>thrd_create</code>, and wait for it with <code>thrd_join</code>.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;threads.h&gt;</span>
<span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;stdio.h&gt;</span>
<span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">worker</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">arg</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">printf</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">"hello from thread %d\n"</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #000;">arg</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
<span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">main</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">thrd_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">id</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">thrd_create</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">worker</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">id</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">thrd_join</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">t</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">NULL</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>Be warned that C11 threads are an optional feature. If the macro <code>__STDC_NO_THREADS__</code> is defined, you do not have them. Apple&#8217;s C library has never shipped <code>&lt;threads.h&gt;</code>, so the program above does not compile on macOS, and glibc only added it in version 2.28 (2018). On such systems you fall back on POSIX threads (<code>pthread_create</code>, <code>pthread_join</code>).</p>
<p>Once you have more than one thread, they may share memory. If two threads access the same non-atomic variable with no ordering between them, and at least one of them writes, the C language calls that a data race. In other words, it is unsafe.</p>
<p>If you have a variable and it is effectively constant, then it is fine to share it. But as soon as anyone changes it, then it might get corrupted. If it is not guarded somewhat, you are in trouble.</p>
<p>To be clear, that is what the C programming language says. I don&#8217;t mean that it will happen on your machine.</p>
<p>To get a better behavior, we can use atomic variables. In C, you have the <code>&lt;stdatomic.h&gt;</code> header.</p>
<p>An atomic integer is never garbage. You always read a value that was once written.</p>
<p>In practice, on most computers you might use today, aligned 8-, 16-, 32- and 64-bit loads and stores are atomic. The C language does not care about that, so if you don&#8217;t specifically require atomicity, you might get in trouble with your C compiler.</p>
<p>The next funny problem is that instructions can be reordered. When you write:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>x = 2
y = 3
</code></pre>
</div>
<p>This may not happen in this sequence. The variable <code>y</code> might be set before the variable <code>x</code>. You may wonder why this is allowed at all. The fundamental reason is that our processors are quite complex. They have layers of buffers and they can execute multiple instructions at once. They can issue several memory loads or stores at once.</p>
<p>By default, in C, atomic accesses are all ordered. It is as if there is an oracle that watches all threads and comes up with a consistent story where everything is in order. This can be expensive, so we prefer not to do it that way.</p>
<p>At the other extreme is the relaxed model: your reads and stores are not garbage, and a given atomic still has one modification order (you will not see 1 and then 0 if the counter only went from 0 to 1), but there is no ordering with respect to other memory.</p>
<p>So we use something intermediate, the release and acquire semantics. They are ordering barriers. A strict barrier would be &#8216;everything before me really happens before me, and everything after me really happens after me&#8217;. (Where &#8216;really happens&#8217; refers to visible effects, the hardware and compiler are allowed to cheat as long as you don&#8217;t catch them.) It is a bit too strong. So we split it in two parts: release and acquire. Intuitively, release means &#8216;if you see me, you see all the stuff before me&#8217;. Acquire means &#8216;I take that package, and everything I do after this load really happens after it&#8217;.</p>
<p>Consider the case where you have a resource (such as a block of allocated memory). You share this resource, but count how many people have access to it. When the counter goes to zero, you free the resource.</p>
<p>One thread could do&#8230;</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>access resource
decrement counter // I won't need it anymore
</code></pre>
</div>
<p>You see these operations happening one after the other, but they may not execute this way. It is possible that they overlap, or even that the decrement occurs before the access. It is entirely safe in a single threaded context.</p>
<p>Anyhow, so the following could happen</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>decrement counter // I won't need it anymore
access resource
</code></pre>
</div>
<p>But what if you have a second thread that does:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
<span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">//</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">I</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">won' need it anymore</span>
<span style="color: #a40000; border: 1px solid #EF2929;">if (counter is zero)</span>
<span style="color: #a40000; border: 1px solid #EF2929;">  free(resource)</span>
</code></pre>
</div>
<p>You could have this interplay:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">resource</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
</code></pre>
</div>
<p>That would be a bug.</p>
<p>So what you first do is make the decrement a &#8216;release access&#8217; which means that operations that come before it cannot be reordered after it. So if you do it this way&#8230;</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code>access resource
decrement counter using release
</code></pre>
</div>
<p>Then it is not possible that we &#8216;see&#8217; the operations as if they happened in the reverse order.</p>
<p>But then we have a second problem. Release is enough for this thread: we cannot still be using the resource after we drop it. It does not tell the last owner that everyone else is finished. The last decrement is itself a release, so it does not observe the other threads&#8217; releases. Without an acquire, that last thread can call <code>free</code> while another thread&#8217;s earlier access is not yet done.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">using</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">release</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">using</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">release</span>
<span style="color: #ce5c00; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #ce5c00; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">resource</span><span style="color: #000; font-weight: bold;">)</span>
</code></pre>
</div>
<p>Thread 2 did its access before its release decrement, but thread 1 never acquired, so it is not required to see that access as finished before <code>free</code>.</p>
<p>So we need the counterpart to a release, an acquire. The last owner acquires before it frees, and that pairs with everyone else&#8217;s release:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
<span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">with</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">release</span>
<span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">(</span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">is</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #4e9a06;">)</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">acquire</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">barrier</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">//</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">see</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">that</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">everyone</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">else</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">is</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">done</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">free</span><span style="color: #4e9a06;">(</span><span style="color: #000;">resource</span><span style="color: #4e9a06;">)</span>
</code></pre>
</div>
<p>Alternatively, you could do this.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">access</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">resource</span>
<span style="color: #000;">decrement</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">with</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">release</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">and</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">acquire</span>
<span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #4e9a06;">(</span><span style="color: #000;">counter</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">is</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">zero</span><span style="color: #4e9a06;">)</span>
<span style="color: #f8f8f8;">  </span><span style="color: #000;">free</span><span style="color: #4e9a06;">(</span><span style="color: #000;">resource</span><span style="color: #4e9a06;">)</span>
</code></pre>
</div>
<p>The two are equivalent, but they are not necessarily equally cheap.</p>
<p>So let us consider a nice example. Let us build a small array that several threads can share. If you are the only owner, you overwrite an element in place. If not, you copy, then you update the copy. That is called copy-on-write. It is a really nice idea that you will find in many important systems.</p>
<p>We start with the type.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;assert.h&gt;</span>
<span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;stdatomic.h&gt;</span>
<span style="color: #8f5902; font-style: italic;">#include</span><span style="color: #f8f8f8;"> </span><span style="color: #8f5902; font-style: italic;">&lt;stdlib.h&gt;</span>
<span style="color: #8f5902; font-style: italic;">#define STR_SIZE 16</span>
<span style="color: #204a87; font-weight: bold;">typedef</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">struct</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">atomic_int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">STR_SIZE</span><span style="color: #000; font-weight: bold;">];</span>
<span style="color: #000; font-weight: bold;">}</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">shared_array</span><span style="color: #000; font-weight: bold;">;</span>
</code></pre>
</div>
<p>The payload is a plain <code>int</code> array. Only <code>refs</code> is atomic. That is deliberate. We never write <code>values</code> while another thread might be reading them.</p>
<p>We create an instance like so.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">str_new</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">malloc</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">sizeof</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">NULL</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">NULL</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">atomic_init</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">STR_SIZE</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>The <code>atomic_init</code> is not an atomic access in the memory-model sense. Nobody else has the pointer yet, so there is no other thread to race with. The caller owns one reference. It is just how we initialize an <code>atomic_int</code>.</p>
<p>Here is how we might naively release an instance.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #8f5902; font-style: italic;">// not real code</span>
<span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">obj_release</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">auto</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">ref</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">-=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">ref</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">!=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #8f5902; font-style: italic;">// we are the last copy</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>What is the problem with this code?</p>
<p>The load and the decrement are two operations. Two threads can both read 2, both subtract, the counter hits zero, and nobody frees: the resource leaks. Write the check the other way around, decrementing first and then testing whether the counter is zero, as in the pseudocode above, and you get the mirror-image bug instead: with <code>refs</code> at 2, one thread decrements to 1, the other decrements to 0, both then read 0, and both call <code>free</code>. You need one atomic subtract that hands you the previous value: only the thread that saw 1 was last.</p>
<p>So you could try</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">obj_release</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">atomic_fetch_sub_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_relaxed</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">!=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>But suppose you have two owners, so that <code>refs</code> is 2. And you have two threads doing</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">];</span>
<span style="color: #000;">obj_release</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
</code></pre>
</div>
<p>One of them will call <code>free</code>, but the order could be</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000; font-weight: bold;">...
[</span><span style="color: #000;">thread1</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">];</span>
<span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">atomic_fetch_sub_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_relaxed</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">atomic_fetch_sub_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_relaxed</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread1</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">];</span>
</code></pre>
</div>
<p>It is a bit confusing because things are not happening in order within thread 2:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">atomic_fetch_sub_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_relaxed</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">thread2</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #0000cf; font-weight: bold;">4</span><span style="color: #000; font-weight: bold;">];</span>
</code></pre>
</div>
<p>But this is allowed.</p>
<p>So what we can do is put a release on the <code>atomic_fetch_sub_explicit</code> and then an acquire right before the <code>free</code>.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #204a87; font-weight: bold;">void</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">obj_release</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">atomic_fetch_sub_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_release</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">!=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">atomic_thread_fence</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">memory_order_acquire</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">free</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>The release on every decrement means &#8220;I am done with the payload.&#8221; The acquire fence, only on the last owner, means &#8220;I have seen that everyone else is done.&#8221; Then <code>free</code> is safe.</p>
<p>That release does double duty, as we are about to see. It is also what lets the last remaining owner write to the payload in place.</p>
<p>If a thread wants another reference to the same instance, it only needs a relaxed access.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">str_retain</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">atomic_fetch_add_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_relaxed</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>Why relaxed? Because the caller already holds a reference, so the object cannot be freed under us: the last owner would need our reference to be gone first.</p>
<p>We can now write <code>update</code>. It consumes the caller&#8217;s reference and returns a reference to the array that contains the new value, which may or may not be the same object. After you call it, you must not touch the pointer you passed in. There is one exception: if a copy was needed and the allocation failed, it returns <code>NULL</code> and leaves the caller&#8217;s reference to <code>o</code> untouched, so you still own it and must still release it.</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">update</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">size_t</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">idx</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">value</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">assert</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">idx</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">STR_SIZE</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">atomic_load_explicit</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #ce5c00; font-weight: bold;">&amp;</span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">refs</span><span style="color: #000; font-weight: bold;">,</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">memory_order_acquire</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">idx</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">value</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">shared_array</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">*</span><span style="color: #000;">new_o</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">str_new</span><span style="color: #000; font-weight: bold;">();</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">if</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">new_o</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">==</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">NULL</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #204a87;">NULL</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">for</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">(</span><span style="color: #204a87; font-weight: bold;">int</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #0000cf; font-weight: bold;">0</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">&lt;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">STR_SIZE</span><span style="color: #000; font-weight: bold;">;</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">i</span><span style="color: #ce5c00; font-weight: bold;">++</span><span style="color: #000; font-weight: bold;">)</span><span style="color: #f8f8f8;"> </span><span style="color: #000; font-weight: bold;">{</span>
<span style="color: #f8f8f8;">        </span><span style="color: #000;">new_o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span><span style="color: #000; font-weight: bold;">];</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000; font-weight: bold;">}</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">new_o</span><span style="color: #ce5c00; font-weight: bold;">-&gt;</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">idx</span><span style="color: #000; font-weight: bold;">]</span><span style="color: #f8f8f8;"> </span><span style="color: #ce5c00; font-weight: bold;">=</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">value</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #f8f8f8;">    </span><span style="color: #000;">obj_release</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">o</span><span style="color: #000; font-weight: bold;">);</span>
<span style="color: #f8f8f8;">    </span><span style="color: #204a87; font-weight: bold;">return</span><span style="color: #f8f8f8;"> </span><span style="color: #000;">new_o</span><span style="color: #000; font-weight: bold;">;</span>
<span style="color: #000; font-weight: bold;">}</span>
</code></pre>
</div>
<p>If the load reads 1, we are the only owner. No other thread holds a reference, so we can write <code>values[idx]</code> in place.</p>
<p>The load is an acquire. When the load reads 1, it may read the value written by the release decrement of the last <em>other</em> owner to drop out. Everything that thread did with <code>values</code> happens before our write. Nothing in our code appearing after it, such as <code>o-&gt;values[idx] = value</code>, may move before it. No other thread still holds a reference, so the write does not race with a concurrent reader. Later, after a retain, other threads can see it.</p>
<p>On x64, acquire and release are effectively free at the CPU: ordinary loads already behave like acquire, ordinary stores like release. You still have to write them in C, or the compiler may reorder the payload accesses. ARM has a weaker memory model so the acquire/release require different instructions (<code>ldapr</code>, <code>ldaddl</code>) which may incur a small performance hit.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/09">The code is available.</a></p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/09/a-quick-overview-of-atomics-in-c/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>AI programming: a layered model</title>
		<link>https://lemire.me/blog/2026/09/05/ai-programming-a-layered-model/</link>
					<comments>https://lemire.me/blog/2026/09/05/ai-programming-a-layered-model/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Sat, 05 Sep 2026 14:02:12 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22831</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_km32u4km32u4km32-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />In the late 1960s and 1970s, people like David Parnas faced a problem. A decade earlier there were almost no programmers. Suddenly there were hordes of inexperienced ones. What could have been a golden era was turning into a mess: far more software, much of it falling apart.   It sent Edsger Dijkstra into a &#8230; <a href="https://lemire.me/blog/2026/09/05/ai-programming-a-layered-model/" class="more-link">Continue reading <span class="screen-reader-text">AI programming: a layered model</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/Gemini_Generated_Image_km32u4km32u4km32-150x150.jpg" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><div class="" data-block="true" data-editor="egmet" data-offset-key="9b314-0-0">
<div data-offset-key="9b314-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9b314-0-0">In the late 1960s and 1970s, people like David Parnas faced a problem. A decade earlier there were almost no programmers. Suddenly there were hordes of inexperienced ones. What could have been a golden era was turning into a mess: far more software, much of it falling apart. </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="d2c0s-0-0">
<div data-offset-key="d2c0s-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="d2c0s-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="9e8td-0-0">
<div data-offset-key="9e8td-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9e8td-0-0">It sent Edsger Dijkstra into a depression. Does this sound familiar? </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="fqomm-0-0">
<div data-offset-key="fqomm-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="fqomm-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="8rvod-0-0">
<div data-offset-key="8rvod-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="8rvod-0-0">AI-assisted coding is producing far more code. Whether the projects will work or crumble remains to be seen. There is a danger. </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="djbp6-0-0">
<div data-offset-key="djbp6-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="djbp6-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="4tm2v-0-0">
<div data-offset-key="4tm2v-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="4tm2v-0-0">I&#8217;d like to propose the layered model. </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="lqkp-0-0">
<div data-offset-key="lqkp-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="lqkp-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="beh5r-0-0">
<div data-offset-key="beh5r-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="beh5r-0-0">Keep a small core that changes slowly and on purpose. For that part you actually read the code. You insist on tests. You can use AI assistance, but there is no vibe coding allowed. </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="e0m4-0-0">
<div data-offset-key="e0m4-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="e0m4-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="9umb1-0-0">
<div data-offset-key="9umb1-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="9umb1-0-0">Everything else can move fast. There will be bugs, but the AI fixes them quickly. </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="elnou-0-0">
<div data-offset-key="elnou-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="elnou-0-0"> </span></div>
</div>
<div class="" data-block="true" data-editor="egmet" data-offset-key="1u4p7-0-0">
<div data-offset-key="1u4p7-0-0" class="public-DraftStyleDefault-block public-DraftStyleDefault-ltr"><span data-offset-key="1u4p7-0-0">Dependencies should be one way: the outer layers depend on the core. The core cannot depend on the outer layers.</span></div>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/05/ai-programming-a-layered-model/feed/</wfw:commentRss>
			<slash:comments>1</slash:comments>
		
		
			</item>
		<item>
		<title>Python sets and dictionaries can have quadratic-time performance</title>
		<link>https://lemire.me/blog/2026/09/03/python-sets-and-dictionaries-can-have-quadratic-time-performance/</link>
					<comments>https://lemire.me/blog/2026/09/03/python-sets-and-dictionaries-can-have-quadratic-time-performance/#comments</comments>
		
		<dc:creator><![CDATA[Daniel Lemire]]></dc:creator>
		<pubDate>Thu, 03 Sep 2026 14:01:45 +0000</pubDate>
				<category><![CDATA[]]></category>
		<guid isPermaLink="false">https://lemire.me/blog/?p=22821</guid>

					<description><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/quadratic-150x150.webp" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" />In Python, the dict data structure is the conventional key-value structure. E.g., you might store a list of names as keys and have their phone numbers as values. Valentin Ignatev wrote this amusing post on X: It is indeed widely believed that, in the strict sense, the dict data structure and its companion, the set &#8230; <a href="https://lemire.me/blog/2026/09/03/python-sets-and-dictionaries-can-have-quadratic-time-performance/" class="more-link">Continue reading <span class="screen-reader-text">Python sets and dictionaries can have quadratic-time performance</span></a>]]></description>
										<content:encoded><![CDATA[<img width="150" height="150" src="https://lemire.me/blog/wp-content/uploads/2026/09/quadratic-150x150.webp" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" /><p>In Python, the <code>dict</code> data structure is the conventional key-value structure. E.g., you might store a list of names as keys and have their phone numbers as values. Valentin Ignatev wrote this amusing post on X:</p>
<p><a href="http://lemire.me/blog/wp-content/uploads/2026/09/xpost.webp"><img loading="lazy" decoding="async" src="http://lemire.me/blog/wp-content/uploads/2026/09/xpost-612x1024.webp" alt="" width="612" height="1024" class="alignnone size-large wp-image-22824" srcset="https://lemire.me/blog/wp-content/uploads/2026/09/xpost-612x1024.webp 612w, https://lemire.me/blog/wp-content/uploads/2026/09/xpost-179x300.webp 179w, https://lemire.me/blog/wp-content/uploads/2026/09/xpost-768x1286.webp 768w, https://lemire.me/blog/wp-content/uploads/2026/09/xpost.webp 908w" sizes="auto, (max-width: 612px) 100vw, 612px" /></a><img decoding="async" alt="" src="xpost.webp" /></p>
<p>It is indeed widely believed that, in the strict sense, the <code>dict</code> data structure and its companion, the set data structure, are O(1), meaning that as you increase the size of the data structure, the time to insert or query a key remains constant.</p>
<p>Let us examine the claim.</p>
<p>A hash function is a function from objects (like strings, integers, etc.) to integer values. We typically expect hash functions to be random-like, although they should always map the same object to the same integer within the current program execution. From hash functions, we construct hash tables:</p>
<ol>
<li>Create an array of buckets.</li>
<li>Given an object, apply the hash function to map it to a bucket.</li>
<li>Store the object in the bucket. When the bucket is already occupied, use some other trick (such as using a nearby bucket).</li>
</ol>
<p>If everything goes well, access and insertion in a hash table take nearly constant time, meaning that the time they take is independent of the size of the hash table.</p>
<p>This can be almost true in many instances. However, it is not formally true. There are many reasons why it is false. For example, if your data structure grows, it might be necessary to reallocate, which will typically take time proportional to the size of the data structure. But we also have the issue of collisions. A collision is what happens when two objects have the same hash value. When we use hash tables, we assume that collisions are uncommon. But it is not difficult to create many of them by picking our objects carefully.</p>
<p>In Python, <code>set</code> and <code>dict</code> are hash tables. I can &#8216;easily&#8217; make my version of Python crumble:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">M</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">1</span> <span style="color: #ce5c00; font-weight: bold;">&lt;&lt;</span> <span style="color: #0000cf; font-weight: bold;">61</span><span style="color: #000; font-weight: bold;">)</span> <span style="color: #ce5c00; font-weight: bold;">-</span> <span style="color: #0000cf; font-weight: bold;">1</span>
<span style="color: #000;">values</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">i</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #000;">M</span> <span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">i</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #204a87;">range</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">n</span> <span style="color: #ce5c00; font-weight: bold;">+</span> <span style="color: #0000cf; font-weight: bold;">1</span><span style="color: #000; font-weight: bold;">)]</span>
<span style="color: #000;">s</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">set</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">)</span>                       <span style="color: #8f5902; font-style: italic;"># insertions</span>
<span style="color: #000;">count</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #204a87;">sum</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">v</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #000;">s</span> <span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">v</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #000;">values</span><span style="color: #000; font-weight: bold;">)</span>   <span style="color: #8f5902; font-style: italic;"># checks</span>
</code></pre>
</div>
<p>If the insertions and the checks are constant-time operations, then the whole construction and the entire check should take linear time. I ran this on an Apple M4 Max with Python 3.14, reporting the median of three runs.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: right;">n</th>
<th style="text-align: right;">time</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="text-align: right;">1000</td>
<td style="text-align: right;">4.8 ms</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">2000</td>
<td style="text-align: right;">15.5 ms</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">4000</td>
<td style="text-align: right;">65.5 ms</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">8000</td>
<td style="text-align: right;">257 ms</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">16000</td>
<td style="text-align: right;">1072 ms</td>
</tr>
</tbody>
</table>
<p>The time roughly quadruples each time <em>n</em> doubles. That is quadratic time, not linear time. The membership checks behave the same way: 1066 ms at <em>n</em> = 16000. At a hundred thousand elements, building the set takes 45 seconds.</p>
<p>But could we create a hash table that would be truly constant-time? No. As the size of your data structure grows, it requires progressively slower memory. If you have a small hash table, it can reside in the CPU cache and be fast. Once it reaches megabytes in size, the data structure tends to live in RAM, which is much slower. And then, eventually, you have to store it on disk, which is even slower. And so forth.</p>
<p>To put it differently, saying that a hash table is O(1) or constant time is a model. It can be true, maybe even often, but it is not reality. Models are great teaching tools: they present a simplified model that you can quickly learn. But models can also introduce biases in how we think.</p>
<p>For example, even though you have read my paragraph that says that the dict data structure gets slower, you may not believe it. You may also believe that it is typically going to be the fastest approach you can use.</p>
<p>Let us consider another practical case. Suppose that you have a large map from strings to integers, that you build once and then only query. That is a common situation: a dictionary of words to identifiers, a lookup table of country codes, a table of feature names.</p>
<p>The <a href="https://pypi.org/project/fastconstmap/">fastconstmap</a> library builds an immutable map from a <code>dict[str, int]</code>. It is suitable when your keys are known in advance.</p>
<p>I build a map from a million random sixteen-character strings to integers, and then look up every key in a shuffled order. With a <code>dict</code>, I write the obvious loop:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">total</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #0000cf; font-weight: bold;">0</span>
<span style="color: #204a87; font-weight: bold;">for</span> <span style="color: #000;">k</span> <span style="color: #204a87; font-weight: bold;">in</span> <span style="color: #000;">probes</span><span style="color: #000; font-weight: bold;">:</span>
    <span style="color: #000;">total</span> <span style="color: #ce5c00; font-weight: bold;">+=</span> <span style="color: #000;">d</span><span style="color: #000; font-weight: bold;">[</span><span style="color: #000;">k</span><span style="color: #000; font-weight: bold;">]</span>
</code></pre>
</div>
<p>With fastconstmap, I ask for all the keys at once, writing the values into a buffer that I own, so that no Python object is allocated per key:</p>
<div class="highlight" style="background: #f8f8f8;">
<pre style="line-height: 125%;"><span></span><code><span style="color: #000;">out</span> <span style="color: #ce5c00; font-weight: bold;">=</span> <span style="color: #000;">array</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #4e9a06;">"Q"</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #204a87;">bytes</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #0000cf; font-weight: bold;">8</span> <span style="color: #ce5c00; font-weight: bold;">*</span> <span style="color: #000;">n</span><span style="color: #000; font-weight: bold;">))</span>
<span style="color: #000;">cm</span><span style="color: #ce5c00; font-weight: bold;">.</span><span style="color: #000;">get_many_into</span><span style="color: #000; font-weight: bold;">(</span><span style="color: #000;">probes</span><span style="color: #000; font-weight: bold;">,</span> <span style="color: #000;">out</span><span style="color: #000; font-weight: bold;">)</span>
</code></pre>
</div>
<p>I am being generous to the <code>dict</code>. I reuse the same string objects for the lookups, and a Python string caches its hash value the first time it is computed. So the <code>dict</code> does not pay for hashing at all, while fastconstmap hashes every key every time. Here are the results, in nanoseconds per key.</p>
<table style="width: 100%; border-collapse: collapse; margin: 1.2rem 0; border: 1px solid #d0d7de; border-radius: 8px; overflow: hidden; box-shadow: 0 1px 2px rgba(0,0,0,0.04);">
<thead style="background: #f6f8fa; color: #24292f;">
<tr>
<th style="text-align: right;">n</th>
<th style="text-align: right;">dict</th>
<th style="text-align: right;">get_many_into</th>
</tr>
</thead>
<tbody>
<tr style="background: #ffffff;">
<td style="text-align: right;">1000</td>
<td style="text-align: right;">21.8</td>
<td style="text-align: right;">4.3</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">10000</td>
<td style="text-align: right;">31.9</td>
<td style="text-align: right;">4.8</td>
</tr>
<tr style="background: #ffffff;">
<td style="text-align: right;">100000</td>
<td style="text-align: right;">48.1</td>
<td style="text-align: right;">5.2</td>
</tr>
<tr style="background: #fbfcfd;">
<td style="text-align: right;">1000000</td>
<td style="text-align: right;">201.9</td>
<td style="text-align: right;">11.8</td>
</tr>
</tbody>
</table>
<p>The <code>dict</code> is not constant time. It goes from 22 ns to 202 ns per key as the map grows, a factor of nine, and it is not because the algorithm changed or because of collisions. It is because a million keys, their string objects, and their integer objects occupy about 116 bytes per key, so the lookups miss in the cache. The fastconstmap version needs 9 bytes per key: it stays in the cache much longer. Pay attention to how the numbers scale: the dict becomes 10 times slower as the size grows.</p>
<p>The lesson is always the same. Some models are useful but none of them is reality. Be mindful of cognitive biases.</p>
<p><a href="https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2026/09/upcoming">The code is available.</a></p>
]]></content:encoded>
					
					<wfw:commentRss>https://lemire.me/blog/2026/09/03/python-sets-and-dictionaries-can-have-quadratic-time-performance/feed/</wfw:commentRss>
			<slash:comments>8</slash:comments>
		
		
			</item>
	</channel>
</rss>
