CSS Wizardry

Correctly Configure (Pre) Connections

Sat, 09 Dec 2023 19:17:04 +0000

A trivial performance optimisation to help speed up third-party or other-origin requests is to preconnect them: hint that the browser should preemptively open a full connection (DNS, TCP, TLS) to the origin in question, for example:

<link rel=preconnect href=https://fonts.googleapis.com>

In the right circumstances, this simple, single line of HTML can make pages hundreds of milliseconds faster! But time and again, I see developers misconfiguring even this most basic of features. Because, as is often the case, there’s much more to this ‘basic feature’ than meets the eye. Let’s dive in…

Learn by Example

At the time of writing, the BBC News homepage (in the UK, at least) has these four preconnects defined early in the <head>:

<link rel=preconnect href=//static.bbc.co.uk crossorigin>
<link rel=preconnect href=//m.files.bbci.co.uk crossorigin>
<link rel=preconnect href=//nav.files.bbci.co.uk crossorigin>
<link rel=preconnect href=//ichef.bbci.co.uk crossorigin>

Readers on narrow screens should know that each of these preconnects also carries a crossorigin attribute—scroll along to see for yourself!

Note that the BBC use schemeless URLs (i.e. href=//…). I would not recommend doing this. Always force HTTPS when it’s available.

Having consulted for the BBC a number of times, I know that they make heavy use of internal subdomains to share resources across teams. While this suits developer ergonomics, it’s not great for performance, particularly in cases where the subdomain in question is on the critical path. Warming up connections to important origins is a must for the BBC.

However, a look at a waterfall tells me that none of these preconnects worked!

Above, you can see that the browser discovered references to each of these origins in the first chunk of HTML, before the 1-second mark. This is evidenced by the light white bars that denote ‘waiting’ time—the browser knows it needs the files, but is waiting to dispatch the requests. However, we can also see that the browser didn’t begin network negotiation until closer to the 1.5-second mark, when we begin seeing a tiny slither of green—DNS—followed by the much more costly TCP and TLS. What went wrong?!

Working Out Which Origins to `preconnect`

In the example above, we have five connections to the following four domains (more on that later):

nav.files.bbci.co.uk: On the critical path with render-blocking CSS.
static.files.bbci.co.uk: On the critical path with render-blocking CSS and JS.
m.files.bbci.co.uk: On the critical path with render-blocking CSS.
- The screenshot above marks the CSS as non-blocking because of the way it’s fetched—it’s preloaded, which is non-blocking, but it’s then conditionally applied to the page using document.write() (which is its own performance faux pas in itself).
ichef.bbci.co.uk: Not on the critical path, but does host the homepage’s LCP element.

N.B. For neatness, I am omitting the https:// from written prose, but it is vital that you include the relevant scheme in your href attribute. All code examples are complete and correct.

Each of these four origins is vital to the page, so all four would be candidates for preconnect. However, the BBC aren’t attempting to preconnect static.files.bbci.co.uk at all; instead, they’re preconnecting static.bbc.co.uk, which is also used, but isn’t on the critical path. This feels more like a simple oversight or a typo than anything else.

As a rule, if the origin is important to the page and is used within the first five seconds of the page-load lifecycle, preconnect it. If the origin is not important, don’t preconnect it; if it is important but is used more than five seconds into the page load lifecycle, your priority should be moving it sooner.

Note that important is very subjective. Your analytics isn’t important; your chat client isn’t important. Your consent management platform is important; your image CDN is important.

One easy way to get an overview of early and important origins—and the method I use when advising clients—is to use WebPageTest. Once you’ve run a test, you can head to a Connection View of the waterfall which shows a diagram comprising entries per origin, not per response:

Note that some connections are actually shared across more than one domain: this is HTTP2’s connection coalescence, available when origins share the same IP address and certificates.

As easy as that—that’s your list of potential origins!

Don’t `preconnect` Too Many Origins

preconnect should be used sparingly. Connection overhead isn’t huge, but too many preconnects that either a) aren’t critical, or b) don’t get used at all, is definitely wasteful.

Flooding the network with unnecessary preconnects early in the page load lifecycle can steal valuable bandwidth that could have been given to more important resources—the overhead of certificates alone can exceed 3KB. Further, opening and persisting connections has a CPU overhead on both the client and the server. Lastly, Chrome will close a connection if it isn’t used within the first 10 seconds of being opened, so if you act too soon, you might end up doing it all over again anyway.

With preconnect, you should strive for as few as possible but as many as necessary. In fact, I would consider too many preconnects a code smell, and you probably ought to solve larger issues like self-hosting your static assets and reducing reliance on third parties in general.

When to Use `crossorigin`

Okay. Now it’s time to learn why the BBC’s preconnects weren’t working!

This is the third time I’ve seen this problem this month (and we’re only nine days in…). It stems from a misunderstanding around when to use crossorigin. I get the impression that developers think ‘this request is going to another origin, so it must need the crossorigin attribute’. But that’s not what the attribute is for—crossorigin is used to define the CORS policy for the request. crossorigin=anonymous (or a bare crossorigin attribute) will never exchange any user credentials (e.g. cookies); crossorigin=use-credentials will always exchange credentials. Unless you know that you need it, you almost never need the latter. But when do we use the former?

If the resulting request for a file would be CORS-enabled, you would need crossorigin on the corresponding preconnect. Unfortunately, CORS isn’t the most straightforward thing in the world. Fortunately, I have a shortcut…

Firstly, identify a file on the origin that you’re considering preconnecting. For example, let’s take a look at the BBC’s box.css. In DevTools (or WebPageTest if you already have one available—you don’t need to run one just for this task), look at the resource’s request headers:

There it is right there: Sec-Fetch-Mode: no-cors.

The preconnect for nav.files.bbci.co.uk doesn’t currently (I’ll come back to that shortly) need a crossorigin attribute:

<link rel=preconnect href=https://nav.files.bbci.co.uk>

Let’s look at another request. orbit-v5-ltr.min.css from static.files.bbci.co.uk also carries a Sec-Fetch-Mode: no-cors request header, so that won’t need crossorigin either:

<link rel=preconnect href=https://nav.files.bbci.co.uk>
<link rel=preconnect href=https://static.files.bbci.co.uk>

Let’s keep looking.

How about the font BBCReithSans_W_Rg.woff2 also from static.files.bbci.co.uk?

Hmm. This does need crossorigin as it’s marked Sec-Fetch-Mode: cors. What do we do here?

Simple!

<link rel=preconnect href=https://nav.files.bbci.co.uk>
<link rel=preconnect href=https://static.files.bbci.co.uk>
<link rel=preconnect href=https://static.files.bbci.co.uk crossorigin>

We just add a second preconnect to open an additional CORS-enabled connection to static.files.bbci.co.uk. (Remember earlier when the browser had opened five connections to four origins? One of them was CORS-enabled!)

Let’s keep going and see where we end up…

As it stands, the very specific example of the homepage right now, needs the following preconnects. Notice that all origins didn’t need crossorigin, except static.files.bbci.co.uk which needed both:

<link rel=preconnect href=https://nav.files.bbci.co.uk>
<link rel=preconnect href=https://static.files.bbci.co.uk>
<link rel=preconnect href=https://static.files.bbci.co.uk crossorigin>
<link rel=preconnect href=https://m.files.bbci.co.uk>
<link rel=preconnect href=https://ichef.bbci.co.uk>

This feels comfortable! The browser naturally opened five connections, so I’m happy to see that we’ve also landed on five preconnects; nothing is unaccounted for.

`Sec-*` Request Headers

I’d recommend familiarising yourself with the entire suite of Sec-* headers—they’re incredibly useful debugging tools.

`preconnect` and DNS

Because DNS is simply IP resolution, it is unaffected by anything CORS-related. This means that:

If you have mistakenly configured your preconnects to use or omit crossorigin when you should have actually omitted or used crossorigin, the DNS step can still be reused—only the TCP and TLS need discarding and doing again. That said, DNS is usually—by far—the fastest part of the process anyway, so speeding it up while missing out on TCP and TLS isn’t much of an optimisation to celebrate.
If you have everything configured correctly, or you aren’t using preconnect at all, you’ll actually see the browser reusing the DNS resolution for a subsequent request that needs a different CORS mode. If you zoom right in on this abridged waterfall, you’ll see that the second CORS-enabled request to static.files.bbci.co.uk doesn’t incur any DNS at all:

The Three Cs: 🤝 Concatenate, 🗜️ Compress, 🗳️ Cache

Tue, 17 Oct 2023 00:00:00 +0000

I began writing this article in early July 2023 but began to feel a little underwhelmed by it and so left it unfinished. However, after recent and renewed discussions around the relevance and usefulness of build steps, I decided to dust it off and get it finished.

Let’s go!

When serving and storing files on the web, there are a number of different things we need to take into consideration in order to balance ergonomics, performance, and effectiveness. In this post, I’m going to break these processes down into each of:

🤝 Concatenating our files on the server: Are we going to send many smaller files, or are we going to send one monolithic file? The former makes for a simpler build step, but is it faster?
🗜️ Compressing them over the network: Which compression algorithm, if any, will we use? What is the availability, configurability, and efficacy of each?
🗳️ Caching them at the other end: How long should we cache files on a user’s device? And do any of our previous decisions dictate our options?

🤝 Concatenate

Concatenation is probably the trickiest bit to get right because, even though the three Cs happen in order, decisions we make later will influence decisions we make back here. We need to think in both directions right now.

Back in the HTTP/1.1 world, we were only able to fetch six resources at a time from a given origin. Given this limitation, it was advantageous to have fewer files: if we needed to download 18 files, that’s three separate chunks of work; if we could somehow bring that number down to six, it’s only one discrete chunk of work. This gave rise to heavy bundling and concatenation—why download three CSS files (half of our budget) if we could compress them into one?

Given that 66% of all websites (and 77% of all requests) are running HTTP/2, I will not discuss concatenation strategies for HTTP/1.1 in this article. If you are still running HTTP/1.1, my only advice is to upgrade to HTTP/2.

With the introduction of HTTP/2, things changed. Instead of being limited to only six parallel requests to a given origin, we were given the ability to open a connection that could be reused infinitely. Suddenly, we could make far more than six requests at a time, so bundling and concatenation became far less relevant. An anti pattern, even.

Or did it?

It turns out H/2 acts more like H/1.1 than you might think…

As an experiment, I took the CSS Wizardry homepage and crudely added Bootstrap. In one test, I concatenated it all into one big file, and the other had the library split into 12 files. I’m measuring when the last stylesheet arrives¹, which is denoted by the vertical purple line. This will be referred to as css_time.

Read the complete test methodology.

Plotted on the same horizontal axis of 1.6s, the waterfalls speak for themselves:

201ms of cumulative latency; 109ms of cumulative download. (View full size.)

With one huge file, we got a 1,094ms css_time and transferred 18.4KB of CSS.

4,362ms of cumulative latency; 240ms of cumulative download. (View full size.)

With many small files, as ‘recommended’ in HTTP/2-world, we got a 1,524ms css_time and transferred 60KB of CSS. Put another way, the HTTP/2 way was about 1.4× slower and about 3.3× heavier.

What might explain this phenomenon?

When we talk about downloading files, we—generally speaking—have two things to consider: latency and bandwidth. In the waterfall charts above, we notice we have both light and dark green in the CSS responses: the light green can be considered latency, while the dark green is when we’re actually downloading data. As a rule, latency stays constant while download time is proportional to filesize. Notice just how much more light green (especially compared to dark) we see in the many-files version of Bootstrap compared to the one-big-file.

This is not a new phenomenon—a client of mine suffered the same problem in July, and the Khan Academy ran into the same issue in 2015!

If we take some very simple figures, we can soon model the point with numbers…

Say we have one file that takes 1,000ms to download with 100ms of latency. Downloading this one file takes:

(1 × 1000ms) + (1 × 100ms) = 1,100ms

Let’s say we chunk that file into 10 files, thus 10 requests each taking a tenth of a second, now we have:

(10 × 100ms) + (10 × 100ms) = 2,000ms

Because we added ‘nine more instances of latency’, we’ve pushed the overall time from 1.1s to 2s.

In our specific examples above, the one-big-file pattern incurred 201ms of latency, whereas the many-files approach accumulated 4,362ms by comparison. That’s almost 22× more!

It’s worth noting that, for the most part, the increase is parallelised, so while it amounts to 22× more overall latency, it wasn’t back-to-back.

It gets worse. As compression favours larger files, the overall size of the 10 smaller files will be greater than the original one file. Add to that the browser’s scheduling mechanisms, we’re unlikely to dispatch all 10 requests at the same time.

So, it looks like one huge file is the fastest option, right? What more do we need to know? We should just bundle everything into one, no?

As I said before, we have a few more things to juggle all at once here. We need to learn a little bit more about the rest of our setup before we can make a final decision about our concatenation strategy.

🗜️ Compress

The above tests were run with Brotli compression². What happens when we adjust our compression strategy?

As of 2022, roughly:

28% of compressible responses were Brotli encoded;
46% were Gzipped;
25% were, worryingly, not compressed at all.

What might each of these approaches mean for us?

Compression	Bundling	`css_time` (ms)
None	One file	4,204
None	Many files	3,663
Gzip	One file	1,190
Gzip	Many files	1,485
Brotli	One file	1,094
Brotli	Many files	1,524

If you can’t compress your files, splitting them out is faster.

Viewed a little more visually:

(View full size.)

These numbers tell us that:

at low (or no) compression, many smaller files is faster than one large one;
at medium compression, one large file is marginally faster than many smaller ones;
at higher compression, one large file is markedly faster than many smaller ones.

Basically, the more aggressive your ability to compress, the better you’ll fare with larger files. This is because, at present, algorithms like Gzip and Brotli become more effective the more historical data they have to play with. In other words, larger files compress more than smaller ones.

This shows us the sheer power and importance of compression, so ensure you have the best setup possible for your infrastructure. If you’re not currently compressing your text assets, that is a bug and needs addressing. Don’t optimise to a sub-optimal scenario.

This looks like another point in favour of serving one-big-file, right?

🗳️ Cache

Caching is something I’ve been obsessed with lately, but for the static assets we’re discussing today, we don’t need to know much other than: cache everything as aggressively as possible.

Each of your bundles requires a unique fingerprint, e.g. main.af8a22.css. Once you’ve done this, caching is a simple case of storing the file forever, immutably:

Cache-Control: max-age=2147483648, immutable

max-age=2147483648: This directive instructs all caches to store the response for the maximum possible time. We’re all used to seeing max-age=31536000, which is one year. This is perfectly reasonable and practical for almost any static content, but if the file really is immutable, we might as well shoot for forever. In the 32-bit world, forever is 2,147,483,648 seconds, or 68 years.
immutable: This directive instructs caches that the file’s content will never change, and therefore to never bother revalidating the file once its max-age is met. You can only add this directive to responses that are fingerprinted (e.g. main.af8a22.css)

All static assets—provided they are fingerprinted—can safely carry such an aggressive Cache-Control header as they’re very easy to cache bust. Which brings me nicely on to…

The important part of this section is cache busting.

We’ve seen how heavily-concatenated files compress better, thus download faster, but how does that affect our caching strategy?

While monolithic bundles might be faster overall for first-time visits, they suffer one huge downfall: even a tiny, one-character change to the bundle would require that a user redownload the entire file just to access one trivial change. Imagine having to fetch a several-hundred kilobyte CSS file all over again for the sake of changing one hex code:

  .c-btn {
-   background-color: #C0FFEE;
+   background-color: #BADA55;
  }

This is the risk with monolithic bundles: discrete updates can carry a lot of redundancy. This is further exacerbated if you release very frequently: while caching for 68 years and releasing 10+ times a day is perfectly safe, it’s a lot of churn, and we don’t want to retransmit the same unchanged bytes over and over again.

Therefore, the most effective bundling strategy would err on the side of as few bundles as possible to make the most of compression and scheduling, but enough bundles to split out high- and low-rate of change parts of your codebase so as to hit the most optimum caching strategy. It’s a balancing act for sure.

📡 Connection

One thing we haven’t looked at is the impact of network speeds on these outcomes. Let’s introduce a fourth C—Connection.

I ran all of the tests over the following connection types:

3G: 1.6 Mbps downlink, 768 Kbps uplink, 150ms RTT
4G: 9 Mbps downlink, 9 Mbps uplink, 170ms RTT
Cable: 5 Mbps downlink, 1 Mbps uplink, 28ms RTT
Fibre: 20 Mbps downlink, 5 Mbps uplink, 4ms RTT

All variants begin to converge on a similar timing as network speed improves. (View full size.)

This data shows us that:

the difference between no-compression and any compression is vast, especially at slower connection speeds;
- the helpfulness of compression decreases as connection speed increases;
many smaller files is faster at all connection speeds if compression is unavailable;
one big file is faster at all connection speeds as long as it is compressed;
- one big file is only marginally faster than many small files over Gzip, but faster nonetheless, and;
- one big file over Brotli is markedly faster than many small files.

Again, no compression is not a viable option and should be considered a bug—please don’t design your bundling strategy around the absence of compression.

This is another nod in the direction of preferring fewer, larger files.

📱 Client

There’s a fifth C! The Client.

Everything we’ve looked at so far has concerned itself with network performance. What about what happens in the browser?

When we run JavaScript, we have three main steps:

Parse: the browser parses the JavaScript to create an AST.
Compile: the parsed code is compiled into optimised bytecode.
Execute: the code is now executed, and does whatever we wanted it to do.

Larger files will inherently have higher parse and compile times, but aren’t necessarily slower to execute. It’s more about what your JavaScript is doing rather than the size of the file itself: it’s possible to write a tiny file that has a far higher runtime cost than a file a hundred times larger.

The issue here is more about shipping an appropriate amount of code full-stop, and less about how it’s bundled.

As an example, I have a client with a 2.4MB main bundle (unfortunately that isn’t a typo) which takes less than 10ms to compile on a mid-tier mobile device.

My Advice

Ship as little as you can get away with in the first place.
- It’s better to send no code than it is to compress 1MB down to 50KB.
If you’re running HTTP/1.1, try upgrade to HTTP/2 or 3.
If you have no compression, get that fixed before you do anything else.
If you’re using Gzip, try upgrade to Brotli.
Once you’re on Brotli, it seems that larger files fare better over the network.
- Opt for fewer and larger bundles.
The bundles you do end up with should, ideally, be based loosely on rate or likelihood of change.

If you have everything in place, then:

Bundle infrequently-changing aspects of your app into fewer, larger bundles.
As you encounter components that appear less globally, or change more frequently, begin splitting out into smaller files.
Fingerprint all of them and cache them forever.
Overall, err on the side of fewer bundles.

For example:

<head>

  <script src=vendor.1a3f5b7d.js   type=module></script>
  <script src=app.8e2c4a6f.js      type=module></script>
  <script src=home.d6b9f0c7.js     type=module></script>
  <script src=carousel.5fac239e.js type=module></script>

</head>

vendor.js is needed by every page and probably updates very infrequently: we shouldn’t force users to redownload it any time we make a change to any first-party JS.
app.js is also needed by every page but probably updates more often than vendor.js: we should probably cache these two separately.
home.js is only needed on the home page: there’s no point bundling it into app.js which would be fetched on every page.
carousel.js might be needed a few pages, but not enough to warrant bundling it into app.js: discrete changes to components shouldn’t require fetching all of app.js again.

The Future Is Brighter

The reason we’re erring on the side of fewer, larger bundles is that currently-available compression algorithms work by compressing a file against itself. The larger a file is, the more historical data there is to compress subsequent chunks of the file against, and as compression favours repetition, the chance of recurring phrases increases the larger the file gets. It’s kind of self-fulfilling.

Understanding why things work this way is easier to visualise with a simple model. Below (and unless you want to count them, you’ll just have to believe me), we have one-thousand a characters:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

This takes up 1,000 bytes of data. We could represent these one-thousand as as 1000(a), which takes up just seven bytes of data, but can be multiplied back out to restore the original thousand-character string with no loss of data. This is lossless compression.

If we were to split this string out into 10 files each containing 100 as, we’d only be able to store those as:

100(a)

100(a)

100(a)

100(a)

100(a)

100(a)

100(a)

100(a)

100(a)

100(a)

That’s ten lots of 100(a), which comes in at 60 bytes as opposed to the seven bytes achieved with 1000(a). While 60 is still much smaller than 1,000, it’s much less effective than one large file as before.

If we were to go even further, one-thousand files with a lone a character in each, we’d find that things actually get larger! Look:

harryroberts in ~/Sites/compression on (main)
» ls -lhFG
total 15608
-rw-r--r--    1 harryroberts  staff   1.0K 23 Oct 09:29 1000a.txt
-rw-r--r--    1 harryroberts  staff    40B 23 Oct 09:29 1000a.txt.gz
-rw-r--r--    1 harryroberts  staff     2B 23 Oct 09:29 1a.txt
-rw-r--r--    1 harryroberts  staff    29B 23 Oct 09:29 1a.txt.gz

Attempting to compress a single a character increases the file size from two bytes to 29. One mega-file compresses from 1,000 bytes down to 40 bytes; the same data across 1,000 files would cumulatively come in at 29,000 bytes—that’s 725 times larger.

Although an extreme example, in the right (wrong?) circumstances, things can get worse with many smaller bundles.

Shared Dictionary Compression for HTTP

There was an attempt at compressing files against predefined, external dictionaries so that even small files would have a much larger dataset available to be compressed against. Shared Dictionary Compression for HTTP (SDHC) was pioneered by Google, and it worked by:

…using pre-negotiated dictionaries to ‘warm up’ its internal state prior to encoding or decoding. These may either be already stored locally, or uploaded from a source and then cached.
— SDHC

Unfortunately, SDHC was removed in Chrome 59 in 2017. Had it worked out, we’d have been able to forgo bundling years ago.

Compression Dictionaries

Friends Patrick Meenan and Yoav Weiss have restarted work on implementing an SDCH-like external dictionary mechanism, but with far more robust implementation to avoid the issues encountered with previous attempts.

While work is very much in its infancy, it is incredibly exciting. You can read the explainer, or the Internet-Draft already. We can expect Origin Trials as we speak.

The early outcomes of this work show great promise, so this is something to look forward to, but widespread and ubiquitous support a way off yet…

tl;dr

In the current landscape, bundling is still a very effective strategy. Larger files compress much more effectively and thus download faster at all connection speeds. Further, queueing, scheduling, and latency work against us in a many-file setup.

However, one huge bundle would limit our ability to employ an effective caching strategy, so begin to conservatively split out into bundles that are governed largely by how often they’re likely to change. Avoid resending unchanged bytes.

Future platform features will pave the way for simplified build steps, but even the best compression in the world won’t sidestep the way HTTP’s scheduling mechanisms work.

Bundling is here to stay for a while.

Appendix: Test Methodology

To begin with, I as attempting to proxy the performance of each by taking the First Contentful Paint milestone. However, in the spirit of measuring what I impact, not what I influence, I decided to lean on the User Timing API and drop a performance.mark() after the last stylesheet:

<link rel=stylesheet href=...>

<script>
  const css_time = performance.mark('css_time');
</script>

I can then pick this up in WebPageTest using their Custom Metrics:

[css_time]
return css_time.startTime

Now, I can append ?medianMetric=css_time to the WebPageTest result URL and automatically view the most representative of the test runs. You can also see this data in WebPageTest’s Plot Full Results view:

For the one-big-file version, outliers were pushing 1.5s. (View full size.)

More or less. It’s accurate enough for this experiment. To be super-thorough, I should really grab the latest single responseEnd value of all of the CSS files, but we’d still arrive at the same conclusions. ↩
All compression modes were Cloudflare’s default settings and applied to all resources, including the host HTML document. ↩

What Is the Maximum max-age?

Mon, 16 Oct 2023 14:18:39 +0000

If you wanted to cache a file ‘forever’, you’d probably use a Cache-Control header like this:

Cache-Control: max-age=31536000

This instructs any cache that it may store and reuse a response for one year (60 seconds × 60 minutes × 24 hours × 365 days = 31,536,000 seconds). But why one year? Why not 10 years? Why not max-age=forever? Why not max-age=∞?!

I wondered the same. Let’s find out together.

Like spoilers? See the answer.

It’s 2147483648 seconds, or 68 years. To find out why, read on!

`max-age`

max-age is a Cache-Control directive that instructs a cache that it may store and reuse a response for n seconds from the point at which it entered the cache in question. Once that time has elapsed, the cache should either revalidate the file with the origin server, or do whatever any additional directives may have instructed it to do. But why might we want to have a max-age that equates to forever?

`immutable`

If we’re confident that we can cache a file for a year, we must be also quite confident that it never really changes. After all, a year is a very long time in internet timescales. If we have this degree of confidence that a file won’t change, we can cache the file immutably.

immutable is a relatively new directive that effectively makes a contract with the browser in which we as developers tell the browser: this file will never, ever change, ever; please don’t bother coming back to the server to check for updates.

Let’s say we have a simple source CSS file called button.css. Its content is as follows:

.c-btn {
  background-color: #C0FFEE;
}

Once our build system has completed, it will fingerprint the file and export it with a unique hash, or fingerprint, in its filename. The MD5 checksum for this file is 7fda1016c4f1eaafc5a4e50a58308b79, so we’d probably end up with a file named button.7fda1016.css.

If we change the colour of the button, the next time we roll a release, the build step will do its thing and now, the following content:

.c-btn {
  background-color: #BADA55;
}

…would have a checksum of 6bb70b2a68a0e28913a05fb3656639b6. In that case, we’d call the new file button.6bb70b2a.css.

Notice how the content of the original file button.7fda1016.css hasn’t changed; button.7fda1016.css has ceased to exist entirely, and is replaced by a whole new file called button.6bb70b2a.css.

Fingerprinted files never change—they get replaced. This means we can safely cache any fingerprinted file for, well, forever.

But how long is forever?!

`31536000` Seconds

Traditionally, developers have set ‘forever’ max-age values at 31536000 seconds, which is a year. Why a year, though? A year isn’t forever. Was 31536000 arrived at by agreement? Or is it specified somewhere? RFC 2616 says of the Expires header:

To mark a response as “never expires,” an origin server sends an Expires date approximately one year from the time the response is sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one year in the future.

Historically—very historically—caching was bound to approximately one year from the time the response is sent. This restriction was introduced by the long defunct Expires header, and we’re talking about max-age, which is a Cache-Control directive. Does Cache-Control say anything different?

`2147483648` Seconds

It turns out there is a maximum value for max-age, and it’s defined in RFC 9111’s delta-seconds:

A recipient parsing a delta-seconds value and converting it to binary form ought to use an arithmetic type of at least 31 bits of non-negative integer range. If a cache receives a delta-seconds value greater than the greatest integer it can represent, or if any of its subsequent calculations overflows, the cache MUST consider the value to be 2147483648 (2³¹) or the greatest positive integer it can conveniently represent.

The spec says caches should accept a maximum max-age value of whatever-it’s-been-told, falling back to 2,147,483,648 seconds (which is 68 years), or failing that, falling back to as-long-as-it-possibly-can. This wording means that, technically, there isn’t a maximum as long as the cache understands the value you passed it. Theoretically, you could set a max-age=9999999999 (that’s 317 years!) or higher. If the cache can work with it, that’s how long it will store it. If it can’t handle 317 years, it should fall back to 2,147,483,648 seconds, and if it can’t handle that, whatever the biggest value it can handle.

And why 2,147,483,648 seconds?

In a 32-bit system, the largest possible integer that can be represented in binary form is 01111111111111111111111111111111: a zero followed by 31 ones (the first zero is reserved for switching between positive and negative values, so 11111111111111111111111111111111 would be equal to -2,147,483,648).

Does It Matter?

Honestly, no.

It’s unlikely that a year would ever be insufficient, and it’s also unlikely that any cache would store a file for that long anyway: browsers periodically empty their cache as part of their general housekeeping, so even files that have been stored for a year might not actually make it that long.

This post was mostly an exercise in curiosity. But, if you wanted to, you could go ahead and swap all of your 31536000s for 2147483648s. It works in all major browsers.

How to Clear Cache and Cookies on a Customer’s Device

Mon, 02 Oct 2023 15:30:49 +0000

If you work in customer support for any kind of tech firm, you’re probably all too used to talking people through the intricate, tedious steps of clearing their cache and clearing their cookies. Well, there’s an easier way!

Getting Someone to Clear Their Own Cache

Trying to talk a non-technical customer through the steps of clearing their own cache is not an easy task—not at all! From identifying their operating system, platform, and browser, to trying to guide them—invisibly!—through different screens, menus, and dropdowns is a big ask.

Thankfully, any company that has folk in customer support can make use of a new web platform feature to make the entire process a breeze: Clear-Site-Data.

`Clear-Site-Data`

A relatively new HTTP header, available in most modern browsers, allows developers to declaratively clear data associated with a given origin¹ via one simple response header: Clear-Site-Data.

Clear-Site-Data: "cache", "cookies"

Any response carrying this header will clear the caches² associated with that origin, so all your customer support team needs now is a simple URL that they can send customers to that will clear all of their caches for them.

Preventing Malicious Clears

While it probably wouldn’t be disastrous, it is possible that a bad actor could link someone directly to https://www.example.com/clear and force an unsuspecting victim into clearing their cache or cookies.

Instead, I would recommend that your /clear page contains links to URLs like /clear/cache, /clear/cookies, /clear/all, each of which check and ensure that the referer request header is equal to https://www.example.com/clear. This way, the only way the clearing works is if the user initiated it themselves. Something maybe a little like this:

const referer = req.get('Referer');

if (referer === 'https://www.example.com/clear') {
  res.set('Clear-Site-Data', 'cache');
} else {
  res.status(403).send('Forbidden: Invalid Referer');
}

`Clear-Site-Data` for Developers

This isn’t the first time I’ve written about Clear-Site-Data—I mentioned it briefly in my 2019 article all about setting the correct caching headers. However, this is the first time I’ve focused on Clear-Site-Data in its own right.

Naturally, the use case isn’t just limited to customer support. As developers, we may have messed something up and need to clear all visitors’ caches right away. We could attach the Clear-Site-Data header to all HTML responses for a short period of time until we think the issue has passed.

Note that this will prevent anything from going into cache while active, so you will notice performance degradations. While ever the header is live, you will be constantly evicting users’ caches, effectively disabling caching for your site the whole time. Tread carefully!

Clearing Cache on iOS

Unfortunately, Clear-Site-Data is not supported by Safari, and as all browsers on iOS are just Safari under the hood, there is no quick way to achieve this for any of your iPhone users. Therefore, my advice to you is to immediately ask your customer Are you using an iPhone?. If the answer is no, direct them to your /clear page; if yes, then, well, I’m sorry. It’s back to the old fashioned way.

It’s also worth noting that Firefox doesn’t support the specific "cache" directive, it was removed in 94, but I can’t imagine the average Firefox user would need assistance clearing their cache.

https://www.bar.com and https://foo.bar.com are different origins: an origin is scoped to scheme, domain, and port. ↩
The spec dictates that any sort of cache associated with the given origin should be cleared, and not just the HTTP cache ↩

The Ultimate Low-Quality Image Placeholder Technique

Thu, 28 Sep 2023 18:59:20 +0000

At the time of writing, 99.9% of pages on the web include at least one image. The median image-weight per page landed at 881KB in 2022, which is more than HTML, CSS, JS, and fonts combined! And while images do not block rendering (unless you do something silly), it’s important to consider how we offer a reasonably pleasant experience while users are waiting for images to load. One solution to that problem is Low-Quality Image Placeholders.

Low-Quality Image Placeholders

Low-Quality Image Placeholders are nothing new. Guy Podjarny is responsible, I think, for coining the term over a decade ago! And before that, we even had the lowsrc attribute for <img> elements:

<img lowsrc=lo-res.jpg src=hi-res.jpg alt>

I wish we’d never deprecated lowsrc—it would have saved us so much hassle in the long run.

The technique is simple: as images are typically heavier and slower resources, and they don’t block rendering, we should attempt to give users something to look at while they wait for the image to arrive. The solution? Show them a low-quality image placeholder, or LQIP.

The upshot is that the user knows that something is happening, and, ideally, they should have roughly some idea what is happening—after all, we want our LQIP to somewhat resemble the final image.

Core Web Vitals and Largest Contentful Paint

While LQIP isn’t a new subject at all, Core Web Vitals and Largest Contentful Paint are, and unfortunately, they don’t necessarily get along so well…

If your LCP candidate is an image (whether that’s a background-image or an <img> element), it’s going to be somewhat slower than if your LCP candidate was a text node, and while making image-based LCPs fast isn’t impossible, it is harder.

Using an LQIP while we wait for our full-res LCP candidate certainly fills a user-experience gap, but, owing to certain rules and restrictions with LCP as a spec (more on that later), it’s unlikely to help our LCP scores.

When the full resolution image eventually arrives, it’s likely that that image will be counted as your LCP, and not your LQIP:

It would be nice to have a scenario whereby your LQIP does meet requirements for consideration as LCP, leading to sub-2.5s scores, but also load in a high resolution soon after, thus improving the user experience. The best of both worlds:

Is that even possible? Let’s find out…

Largest Contentful Paint Caveats

There is some important nuance that we should be aware of before we go any further. There are quite a few moving parts when it comes to how and when your LCP candidates are captured, when they’re updated, and which candidate is ultimately used.

Chrome keeps taking new LCP candidates right up until a user interacts with the page. This means that if an <h1> is visible immediately, a user scrolls, then a larger <img> arrives moments after, the <h1> is your LCP element for that page. If a user doesn’t interact in that short window, a new entry is captured, and now the <img> is your LCP element. Notice below how our <h1> is momentarily considered our LCP candidate at 1.0s, before ultimately being replaced by the <img> at 2.5s:

Blue shading shows an LCP candidate; green shading and/or a red border shows the actual LCP element and event. (View full size)

The key takeaway here is that Chrome keeps looking for new LCP candidates, and the moment it finds anything larger, it uses that.

What if Chrome finds a later element of the same size? Thankfully, Chrome will not consider new elements of the same size as the previously reported LCP candidate. That protects us in situations like this:

Things like image grids are measured on their first image, not their last. This is great news. (View full size)

Note that at 1.4s we get our LCP event in full. When the other eight images arrive at 2.0s, they make no difference to our score.

This all seems straightforward enough—Chrome keeps on looking for the largest element and then uses that, right? And it doesn’t necessarily spell bad news for our LQIP either. As long as our final image is the same dimensions as the LQIP was…?

Not quite. There’s some subtle complexity designed to prevent people gaming the system, which is exactly what we’re trying to do.

Warning: It is imperative that you still provide a great user experience. Passing LCP for metrics’ sake is unwise and against the spirit of web performance. Ensure that your LQIP is still of sufficient quality to be useful, and follow it up immediately with your full-quality image. Poor quality images, particularly where ecommerce is concerned, are especially harmful.

Don’t Upscale Your LQIP

Each image in the tests so far has been a 200×200px <img> displayed at 200×200px:

<img src=https://dummyimage.com/200/000/fff.png?2&text=200@200
     width=200 height=200 alt>

Which is this, coming at 2KB:

What if we change the <img> to 100×100px displayed at 200×200px, or upscaled?

<img src=https://dummyimage.com/100/000/fff.png?4&text=100@200
     width=200 height=200 alt>

Which comes in at 1.4KB:

Already, you can see the loss in quality associated with upscaling this image.

Upscaled images will be discounted against higher-resolution ones. (View full size)

Above, we see that we log a candidate at 1.5s, but the second image at 2.0s becomes our LCP despite being rendered at the exact same size!

And there is the nuance. Chrome doesn’t want to reward a poor experience, so simply serving a tiny image and displaying it much larger will not help your LCP scores if a denser image turns up later on. And I agree with this decision, for the most part.

The first takeaway is: don’t upscale your LQIP.

Calculating the Upscaling Penalty

Let’s get a bit more detailed about upscaling and penalties. Some close reading of the spec tells us exactly how this works. It’s not the easiest thing to digest, but I’ll do my best to distil it for you here. The reported area of your LCP element is calculated as:

area = size × penaltyFactor

Where:

size is the area of the LCP candidate currently in the viewport and not cropped or off-screen.
penaltyFactor is the factor by which upscaling will count against us, given by min(displaySize, naturalSize) / displaySize, where:
- naturalSize is the pixel area of the image file in question.
- displaySize is the pixel area that the image will be rendered, regardless of how much of it is currently on-screen.

In full:

area = size × min(displaySize, naturalSize) / displaySize

Imagine we took a large landscape image, downscaled it to a predetermined height, and then displayed it, cropped, as a square:

In the above diagram, a is naturalSize, b is displaySize, and c is size.

For the sake of ease, let’s assume your LCP candidate is always fully on-screen, in-viewport, and not cropped (if you have known and predictable cropping or off-screen image data, you can adjust your maths accordingly). This means that size and displaySize are now synonymous.

Let’s say we have a 400×400px image that is downscaled to 200×200px. Its area would be calculated as:

200 × 200 × min(200 × 200, 400 × 400) / (200 × 200) = 40,000

Thus the LCP’s reported size would be 40,000px²:

(View full size)

If we were to use a 100×100px image and upscale it to 200×200px, our equation looks a little different:

200 × 200 × min(200 × 200, 100 × 100) / (200 × 200) = 10,000

(View full size)

This image’s reported area is significantly smaller, despite being rendered at the exact same size! This means that any subsequent images of a higher quality may well steal our LCP score away from this one and to a much later time.

Even if we used a 199×199px LQIP, we’d still register a new LCP the moment our full quality image arrives:

200 × 200 × min(200 × 200, 199 × 199) / (200 × 200) = 39,601

That all got pretty academic, but my advice is basically: if you want your LQIP to be considered as your LCP, do not upscale it. If you do upscale it, your reported area will come back smaller than you might expect, and thus the later, high resolution image is likely to ‘steal’ the LCP score.

N.B. Thankfully, none of the specs take device pixels or pixel densities into account. It’s CSS pixels all the way down.

Aim for a Minimum of 0.05BPP

The second restriction we need to get around is the recently announced bits per pixel (BPP) threshold. Again, to stop people gaming the system, Chrome decided that only images of a certain quality (or entropy) will be considered as your LCP element. This prevents people using incredibly low quality images in order to register a fast LCP time:

That heuristic discounts paints which are not contentful, but just serve as backgrounds or placeholders for other content.

This change extends that heuristic to other images as well, when those images have very little content, when compared to the size at which they are displayed.
— Largest Contentful Paint change in Chrome 112 to ignore low-entropy images

This one is much simpler to make sense of. In order for an image to be counted as an LCP candidate, it needs to contain at least 0.05 bits of data per pixel displayed.

Note that this applies to the image’s displayed size and not its natural size:

Controls whether LCP calculations should exclude low-entropy images. If enabled, then the associated parameter sets the cutoff, expressed as the minimum number of bits of encoded image data used to encode each rendered pixel. Note that this is not just pixels of decoded image data; the rendered size includes any scaling applied by the rendering engine to display the content.
— features.cc

A 200×200px image has 40,000 pixels. If we need 0.05 bits of data for each pixel, the image needs to be at least 2,000 bits in size. To get that figure in Kilobytes, we simply need to divide it by 8,000: 0.25KB. That’s tiny!

A 1024×768px image?

(1024 × 768 × 0.05) / 8000 = 4.9152KB

720×360px?

(720 × 360 × 0.05) / 8000 = 1.62KB

That was much less academic, but my advice is basically: if you want your LQIP to ever be considered as your LCP, make sure it contains enough data.

To err on the side of caution, I go by a BPP figure of 0.055. Honestly, the filesizes you’re aiming for are so small at this point that you’ll probably struggle to get as low as 0.055BPP anyway, but it just seems wise to build in 10% of buffer in case any intermediaries attempt to compress your images further. (This should actually be impossible because you’re serving your images over HTTPS, right?)

LQIP and BPP Calculator

That’s a lot of specs and numbers. Let’s try make it all a little easier. I’ve built this simplified calculator to help you work out the mathematically smallest possible LCP candidate. It is this image that becomes your LQIP.

Your LQIP should be 1,440×810px (1,166,400px²), and should have a filesize no smaller than 8.019KB.

Using the exact same calculator you’re playing with right now, I plugged in my homepage’s numbers and rebuilt my LCP. I managed to get my LQIP–LCP down to just 1.1s on a 3G connection.

Note that my <h1> and a <p> are initially flagged as candidates before Chrome finally settles on the image. (View full size)

And from a cold cache, over train wifi as I was writing this post, I got a 2.1s LCP score on desktop!

(View full size)

Implementing Low-Quality Image Placeholders

My implementation becomes incredibly simple as I’m using a background image. This means I can simply layer up the progressively higher-resolution files using CSS’ multiple backgrounds:

<head>

  ...

  <link rel=preload as=image href=lo-res.jpg fetchpriority=high>

  ...

</head>
<body>

  <header style="background-image: url(hi-res.jpg),
                                   url(lo-res.jpg)">
    ...
  </header>

</body>

As background-image is hidden from the preload scanner, I’m preloading the LQIP (lo-res.jpg) so that it’s already on its way before the parser encounters the <header>.
- Note that I’m not preloading hi-res.jpg—we don’t want the two images to race each other, we want them to arrive one after the other.
Once the parser reaches the <header>, the request for hi-res.jpg is dispatched.
- At this point, if it’s fully fetched, we can render lo-res.jpg as the <header>’s background.
- If lo-res.jpg isn’t ready yet, we’d fall back to a background-color or similar while we wait.
As lo-res.jpg is guaranteed to arrive first (it was requested much earlier and is much smaller in file-size), it gets displayed first.
Once hi-res.jpg arrives, whenever that may be, it takes the place of lo-res.jpg, switching out automatically.

My very specific implementation is more complex and nuanced than that (it’s responsive and I also use a super low-resolution Base64 placeholder that’s far too small to be considered an LCP candidate), but that’s the main technique in a few lines. My layers look like this:

(View full size)

The very first frame is 810 bytes of 16×11px Base64 image, far too small to qualify for LCP, and massively upscaled:
The second frame is a 24KB image that is both my LQIP and my LCP.
The third and final frame is the full-resolution, 160KB image.

The background-image method only works if images are decorational. If your image is content (e.g. it’s a product image), then semantically, a background-image won’t be good enough. In this case, you’ll probably end up absolutely positioning some <img> elements, but it’s also worth noting that you can apply background-images to <img>s, so the technique I use will be more or less identical. Something like this:

<head>

  ...

  <link rel=preload as=image href=lo-res.jpg fetchpriority=high>

  ...

</head>
<body>

  <img src=hi-res.jpg
       alt="Appropriate alternate text"
       width=360
       height=360
       style="background-image: url(lo-res.jpg)">

</body>

In fact, I do exactly that with the photo of me in the sidebar.

Use an Image Transformation Service

Being so tightly bound to these figures isn’t very redesign-friendly—you’d have to reprocess your entire image library if you made your LCP candidate any bigger. With this in mind, I wouldn’t recommend attempting this manually, or batch-processing your entire back catalogue.

Instead, use a service like Cloudinary to size and compress images on the fly. This way, you only need to redesign a handful of components and let Cloudinary do the rest on demand. They make available a quality parameter that takes a number which is a value between 1 (smallest file size possible) and 100 (best visual quality). E.g. q_80. Note that this number is not a percentage.

To get your BPP down to roughly 0.05, you’re going to want to experiment with a really small number. Play around with numerous different images from your site to ensure whatever quality setting you choose doesn’t ever take you below 0.05BPP.

Use Your Judgement

If you do manage to get your image all the way down to your target filesize, there’s every chance it will be too low quality to be visually acceptable, even if it does satisfy LCP’s technical requirements.

Here’s a current client’s product image compressed down to 4KB (their target was actually 3.015KB, but even the most aggressive settings couldn’t get me all the way):

This is visually unacceptable as an LCP candidate, even though it ticks every box in the spec. My advice here—and it’s very subjective—is that you shouldn’t accept an LQIP–LCP that you wouldn’t be happy for a user to look at for any period of time.

In this particular instance, I bumped the quality up to 10, which came in at 12KB, was still super fast, but was visually much more acceptable.

Summary

In their attempts to prevent people gaming the system, spec writers have had to define exactly what that system is. Ironically, codifying these constraints makes gaming the system so much easier, as long as you can be bothered to read the specifications (which, luckily for you, I have).

Largest Contentful Paint candidates are penalised for upscaling and also for low entropy. By understanding how the upscaling algorithm works, and how to calculate target filesizes from input dimensions, we can generate the smallest possible legitimate LCP image which can be used as a low-quality placeholder while we wait for our full-resolution image to arrive. The best of both worlds.

Core Web Vitals for Search Engine Optimisation: What Do We Need to Know?

Mon, 24 Jul 2023 00:00:00 +0000

Updates

Stay updated by following this article’s Twitter thread. I will post amendments and updates there.

26 July, 2023: iOS (and Other) Traffic Doesn’t Count

Core Web Vitals

Google’s Core Web Vitals initiative was launched in May of 2020 and, since then, its role in Search has morphed and evolved as roll-outs have been made and feedback has been received.

However, to this day, messaging from Google can seem somewhat unclear and, in places, even contradictory. In this post, I am going to distil everything that you actually need to know using fully referenced and cited Google sources.

Don’t have time to read 5,500+ words? Need to get this message across to your entire company? Hire me to deliver this talk internally.

If you’re happy just to trust me, then this is all you need to know right now:

Google takes URL-level Core Web Vitals data from CrUX into account when deciding where to rank you in a search results page. They do not use Lighthouse or PageSpeed Insights scores. That said, it is just one of many different factors (or signals) they use to determine your placement—the best content still always wins.

To get a ranking boost, you need to pass all relevant Core Web Vitals and everything else in the Page Experience report. Google do strongly encourage you to focus on site speed for better performance in Search, but, if you don’t pass all relevant Core Web Vitals (and the applicable factors from the Page Experience report) they will not push you down the rankings.

All Core Web Vitals data used to rank you is taken from actual Chrome-based traffic to your site. This means your rankings are reliant on your performance in Chrome, even if the majority of your customers are in non-Chrome browsers. However, the search results pages themselves are browser-agnostic: you’ll place the same for a search made in Chrome as you would in Safari as you would in Firefox.

Conversely, search results on desktop and mobile may appear different as desktop searches will use desktop Core Web Vitals data and mobile searches will use mobile data. This means that your placement on each device type is based on your performance on each device type. Interestingly, Google have decided to keep the Core Web Vitals thresholds the same on both device classifications. However, this is the full extent of the segmentation that they make; slow experiences in, say, Australia, will negatively impact search results in, say, the UK.

If you’re a Single-Page Application (SPA), you’re out of luck. While Google have made adjustments to not overly penalise you, your SPA is never really going to make much of a positive impact where Core Web Vitals are concerned. In short, Google will treat a user’s landing page as the source of its data, and any subsequent route change contributes nothing. Therefore, optimise every SPA page for a first-time visit.

The best place to find the data that Google holds on your site is Search Console. While sourced from CrUX, it’s here that is distilled into actionable, Search-facing data.

The true impact of Core Web Vitals on ranking is not fully understood, but investing in faster pages is still a sensible endeavour for almost any reason you care to name.

Now would be a good time to mention: I am an independent web performance consultant—one of the best. I am available to help you find and fix your site-speed issues through performance audits, training and workshops, consultancy, and more. You should get in touch.

For citations, quotes, proof, and evidence, read on…

Site-Speed Is More Than SEO

While this article is an objective look at the role of Core Web Vitals in SEO, I want to take one section to add my own thoughts to the mix. While Core Web Vitals can help with SEO, there’s so much more to site-speed than that.

Yes, SEO helps get people to your site, but their experience while they’re there is a far bigger predictor of whether they are likely to convert or not. Improving Core Web Vitals is likely to improve your rankings, but there are myriad other reasons to focus on site-speed outside of SEO.

I’m happy that Google’s Core Web Vitals initiative has put site-speed on the radar of so many individuals and organisations, but I’m keen to stress that optimising for SEO is only really the start of your web performance journey.

With that said, everything from this point on is talking purely about optimising Core Web Vitals for SEO, and does not take the user experience into account. Ultimately, everything is all, always about the user experience, so improving Core Web Vitals irrespective of SEO efforts should be assumed a good decision.

The Core Web Vitals Metrics

Generally, I approve of the Core Web Vitals metrics themselves (Largest Contentful Paint, First Input Delay, Cumulative Layout Shift, and the nascent Interaction to Next Paint). I think they do a decent job of quantifying the user experience in a broadly applicable manner and I’m happy that the Core Web Vitals team constantly evolve or even replace the metrics in response to changes in the landscape.

— Web Vitals

I still feel that site owners who are serious about web performance should augment Core Web Vitals with their own custom metrics (e.g. ‘largest content’ is not the same as ‘most important content’), but as off-the-shelf metrics go, Core Web Vitals are the best user-facing metrics since Patrick Meenan’s work on SpeedIndex.

N.B. In March 2024, First Input Delay (FID) will be removed, and Interaction to Next Paint (INP) will take its place. – Advancing Interaction to Next Paint

Some History

Google has actually used Page Speed in rankings in some form or another since as early as 2010:

As part of that effort, today we’re including a new signal in our search ranking algorithms: site speed.
— Using site speed in web search ranking

And in 2018, that was rolled out to mobile:

Although speed has been used in ranking for some time, that signal was focused on desktop searches. Today we’re announcing that starting in July 2018, page speed will be a ranking factor for mobile searches.
— Using page speed in mobile search ranking

The criteria was undefined, and we were offered little more than it applies the same standard to all pages, regardless of the technology used to build the page.

Interestingly, even back then, Google made it clear that the best content would always win, and that relevance was still the strongest signal. From 2010:

While site speed is a new signal, it doesn’t carry as much weight as the relevance of a page.
— Using site speed in web search ranking

And again in 2018:

The intent of the search query is still a very strong signal, so a slow page may still rank highly if it has great, relevant content.
— Using page speed in mobile search ranking

In that case, let’s talk about relevance and content…

The Best Content Always Wins

Google’s mission is to surface the best possible response to a user’s query, which means they prioritise relevant content above all else. Even if a site is slow, insecure, and not mobile friendly, it will rank first if it is exactly what a user is looking for.

In the event that there are a number of possible matches, Google will begin to look at other ranking signals to further arrange the hierarchy of results. To this end, Core Web Vitals (and all other ranking signals) should be thought of as tie-breakers:

Google Search always seeks to show the most relevant content, even if the page experience is sub-par. But for many queries, there is lots of helpful content available. Having a great page experience can contribute to success in Search, in such cases.
— Understanding page experience in Google Search results

The latter half of that paragraph is of particular interest to us, though: Core Web Vitals do still matter…

Get in touch

Need Some Help?

I help companies find and fix site-speed issues. Performance audits, training, consultancy, and more.

Core Web Vitals Are Important

Though it’s true we have to prioritise the best and most relevant content, Google still stresses the importance of site speed if you care about rankings:

We highly recommend site owners achieve good Core Web Vitals for success with Search…
— Understanding Core Web Vitals and Google search results

That in itself is a strong indicator that Google favours faster websites. Furthermore, they add:

Google’s core ranking systems look to reward content that provides a good page experience.
— Understanding page experience in Google Search results

Which brings me nicely onto…

It’s Not Just About Core Web Vitals

What’s this phrase page experience that we keep hearing about?

It turns out that Core Web Vitals on their own are not enough. Core Web Vitals are a subset of the Page Experience report, and it’s actually this that you need to pass in order to get a boost in rankings.

In May 2020, Google announced the Page Experience report, and, a year later, from June to August 2021, they rolled it out for mobile. Also in August 2021, they removed Safe Browsing and Ad Experience from the report, and in February 2022, they rolled Page Experience out for desktop.

The simplified Page Experience report contains:

Core Web Vitals
- Largest Contentful Paint
- First Input Delay
- Cumulative Layout Shift
Mobile Friendly (mobile only, naturally)
HTTPS
No Intrusive Interstitials

— Simplifying the Page Experience report

From Google:

…great page experience involves more than Core Web Vitals. Good stats within the Core Web Vitals report in Search Console or third-party Core Web Vitals reports don’t guarantee good rankings.
— Understanding page experience in Google Search results

What this means is we shouldn’t be focusing only on Core Web Vitals, but on the whole suite of Page Experience signals. That said, Core Web Vitals are quite a lot more difficult to achieve than being mobile friendly, which is usually baked in from the beginning of a project.

You Don’t Need to Pass FID

You don’t need to pass First Input Delay. This is because—while all pages will have a Largest Contentful Paint event at some point, and the ideal Cumulative Layout Shift score is none at all—not all pages will incur a user interaction. While rare, it is possible that a URL’s FID data will read Not enough data. To this end, passing Core Web Vitals means Good LCP and CLS, and Good or Not enough data FID.

The URL has Good status in the Core Web Vitals in both CLS and LCP, and Good (or not enough data) in FID
— Page Experience report

Interaction to Next Paint Doesn’t Matter Yet

Search Console, and other tools, are surfacing INP already, but it won’t become a Core Web Vital (and therefore part of Page Experience (and therefore part of the ranking signal)) until March 2024:

INP (Interaction to Next Paint) is a new metric that will replace FID (First Input Delay) as a Core Web Vital in March 2024. Until then, INP is not a part of Core Web Vitals. Search Console reports INP data to help you prepare.
— Core Web Vitals report

Incidentally, although INP isn’t yet a Core Web Vital, Search Console has started sending emails warning site owners about INP issues:

Search Console emails have begun warning people about INP issues. Credit: Ryan Townsend.

You don’t need to worry about it yet, but do make sure it’s on your roadmap.

You’re Ranked on Individual URLs

This has been one of the most persistently confusing aspect of Core Web Vitals: are pages ranked on their individual URL status, or the status of the URL Group they live in (or something else entirely)?

It’s done on a per-URL basis:

— Page Experience report

Google evaluates page experience metrics for individual URLs on your site and will use them as a ranking signal for a URL in Google Search results.
— Page Experience report

There are also URL Groups and larger groupings of URL data:

Our core ranking systems generally evaluate content on a page-specific basis […] However, we do have some site-wide assessments.
— Understanding page experience in Google Search results

If there isn’t enough data for a specific URL Group, Google will fall back to an origin-level assessment:

If a URL group doesn’t have enough information to display in the report, Search Console creates a higher-level origin group…
— Core Web Vitals report

This doesn’t tell us why we have URL Groups in the first place. How do they tie into SEO and rankings if we work on a URL- or site-level basis?

My feeling is that it’s less about rankings and more about helping developers troubleshoot issues in bulk:

URLs in the report are grouped [and] it is assumed that these groups have a common framework and the reasons for any poor behavior of the group will likely be caused by the same underlying reasons.
— Core Web Vitals report

URLs are judged on the three Core Web Vitals, which means they could be Good, Needs Improvement, and Poor in each Vital respectively. Unfortunately, URLs are ranked on their lowest common denominator: if a URL is Good, Good, Poor, it’s marked Poor. If it’s Needs Improvement, Good, Needs Improvement, it’s marked Needs Improvement:

The status for a URL group defaults to the slowest status assigned to it for that device type…
— Core Web Vitals report

The URLs that appear in Search Console are non-canonical. This means that https://shop.com/products/red-bicycle and https://shop.com/bikes/red-bicycle may both be listed in the report even if their rel=canonical both point to the same location.

Data is assigned to the actual URL, not the canonical URL, as it is in most other reports.
— Core Web Vitals report

Note that this only discusses the report and not rankings—it is my understanding that this is to help developers find variations of pages that are slower, and not to rank multiple variants of the same URL. The latter would contravene their own rules on canonicalisation:

Google can only index the canonical URL from a set of duplicate pages.
— Canonical

Or, expressed a little more logically, canonical alternative (and noindex) pages can’t appear in Search in the first place, so there’s little point worrying about Core Web Vitals for SEO in this case anyway.

Get in touch

Need Some Help?

I help companies find and fix site-speed issues. Performance audits, training, consultancy, and more.

Interestingly:

Core Web Vitals URLs include URL parameters when distinguishing the page; PageSpeed Insights strips all parameter data from the URL, and then assigns all results to the bare URL.
— Core Web Vitals report

This means that if we were to drop https://shop.com/products?sort=descending into pagespeed.web.dev, the Core Web Vitals it presents back would be the data for https://shop.com/products.

Search Console Is Gospel

When looking into Core Web Vitals for SEO purposes, the only real place to consult is Search Console. Core Web Vitals information is surfaced in a number of different Google properties, and is underpinned by data sourced from the Chrome User Experience Report, or CrUX:

CrUX is the official dataset of the Web Vitals program. All user-centric Core Web Vitals metrics will be represented in the dataset.
— About CrUX

And:

The data for the Core Web Vitals report comes from the CrUX report. The CrUX report gathers anonymized metrics about performance times from actual users visiting your URL (called field data). The CrUX database gathers information about URLs whether or not the URL is part of a Search Console property.
— Core Web Vitals report

This is the data that is then used in Search to influence rankings:

The data collected by CrUX is available publicly through a number of tools and is used by Google Search to inform the page experience ranking factor.
— About CrUX

The data is then surfaced to us in Search Console.

Search Console shows how CrUX data influences the page experience ranking factor by URL and URL group.
— CrUX methodology

Basically, the data originates in CrUX, so it’s CrUX all the way down, but it’s in Search Console that Google kindly aggregates, segments, and otherwise visualises and displays the data to make it actionable. Google expects you to look to Search Console to find and fix your Core Web Vitals issues:

Google Search Console provides a dedicated report to help site owners quickly identify opportunities for improvement.
— Evaluating page experience for a better web

Ignore Lighthouse and PageSpeed Insights Scores

This is one of the most pervasive and definitely the most common misunderstandings I see surrounding site-speed and SEO. Your Lighthouse Performance scores have absolutely no bearing on your rankings. None whatsoever. As before, the data Google use to influence rankings is stored in Search Console, and you won’t find a single Lighthouse score in there.

Frustratingly, there is no black-and-white statement from Google that tells us we do not use Lighthouse scores in ranking, but we can prove the equivalent quite quickly:

The Core Web Vitals report shows how your pages perform, based on real world usage data (sometimes called field data).
– Core Web Vitals report

And:

The data for the Core Web Vitals report comes from the CrUX report. The CrUX report gathers anonymized metrics about performance times from actual users visiting your URL (called field data).
– Core Web Vitals report

That’s two definitive statements saying where the data does come from: the field. So any data that doesn’t come from the field is not counted.

PSI provides both lab and field data about a page. Lab data is useful for debugging issues, as it is collected in a controlled environment. However, it may not capture real-world bottlenecks. Field data is useful for capturing true, real-world user experience – but has a more limited set of metrics.
— About PageSpeed Insights

In the past—and I can’t determine the exact date of the following screenshot—Google used to clearly mark lab and field data in PageSpeed Insights:

— What is Google PageSpeed Insights? – SISTRIX

Nowadays, the same data and layout exists, but with much less deliberate wording. Field data is still presented first:

Here we can see that this data came from CrUX and is based on real, aggregated data.

And lab data, from the Lighthouse test we just initiated, beneath that:

Here we can clearly see that this was run from a predetermined location, on a predetermined device, over a predetermined connection speed. This was one page load run by us, for us.

So for all there is no definitive warning from Google that we shouldn’t factor Lighthouse Performance scores into SEO, we can quickly piece together the information ourselves. It’s more a case of what they haven’t said, and nowhere have they ever said your Lighthouse/PageSpeed scores impact rankings.

On the subject of things they haven’t said…

Failing Pages Don’t Get Penalised

This is a critical piece of information that is almost impressively-well hidden.

Google tell us that the criteria for a Good page experience are:

Passes all relevant Core Web Vitals
No mobile usability issues on mobile
Served over HTTPS

If a URL achieves Good status, that status will be used as a ranking signal in search results.

— Page Experience report

Note the absence of similar text under the Failed column. Good URLs’ status will be used as a ranking signal, Failed URLs… nothing.

Good URLs’ status will be used as a ranking signal.

All of Google’s wording around Core Web Vitals is about rewarding Good experiences, and never about suppressing Poor ones:

We highly recommend site owners achieve good Core Web Vitals for success with Search…
— Understanding Core Web Vitals and Google search results

Google’s core ranking systems look to reward content that provides a good page experience.
— Understanding page experience in Google Search results

…for many queries, there is lots of helpful content available. Having a great page experience can contribute to success in Search….
— Understanding page experience in Google Search results

Note that this is in contrast to their 2018 announcement which stated that The “Speed Update” […] will only affect pages that deliver the slowest experience to users… – Speed Update was a precursor to Core Web Vitals.

This means that failing URLs will not get pushed down the search results page, which is probably a huge and overdue relief for many of you reading this. However…

If one of your competitors puts in a huge effort to improve their Page Experience and begins moving up the search results pages, that will have the net effect of pushing you down.

Put another way, while you won’t be penalised, you might not get to simply stay where you are. Which means…

Core Web Vitals Are a Tie-Breaker

Core Web Vitals really shine in competitive environments, or when users aren’t searching for something that only you could possibly provide. When Google could rank a number of different URLs highly, it defers to other ranking signals to refine its ordering.

…for many queries, there is lots of helpful content available. Having a great page experience can contribute to success in Search, in such cases.
— Understanding page experience in Google Search results

Get in touch

Need Some Help?

I help companies find and fix site-speed issues. Performance audits, training, consultancy, and more.

There Are No Shades of Good or Failed URLs

Going back to the Good versus Failed columns above, notice that it’s binary—there are no grades of Good or Failed—it’s just one or the other. A URL is considered Failed the moment it doesn’t pass even one of the relevant Core Web Vitals, which means a Largest Contentful Paint of 2.6s is just as bad as a Largest Contentful Paint of 26s.

Put another way, anything other than Good is Failed, so the actual numbers are irrelevant.

Mobile and Desktop Thresholds Are the Same

Interestingly, the thresholds for Good, Needs Improvement, and Poor are the same on both mobile and desktop. Because Google announced Core Web Vitals for mobile first, the same thresholds on desktop should be achieved automatically—it’s very rare that desktop experiences would fare worse than mobile ones. The only exception might be Cumulative Layout Shift in which desktop devices have more screen real estate for things to move around.

For each of the above metrics, to ensure you’re hitting the recommended target for most of your users, a good threshold to measure is the 75th percentile of page loads, segmented across mobile and desktop devices.
— Web Vitals

This does help simplify things a little, with only one set of numbers to remember.

Slow Countries Can Harm Global Rankings

While Google does segment on desktop and mobile—ranking you on each device type proportionate to your performance on each device type—that’s as far at they go. This means that if an experience is Poor on mobile but Good on desktop, any searches for you on desktop will have your fast site taken into consideration.

Treo makes it easy to visualise global CrUX data.

Unfortunately, that’s as far as their segmentation goes, and even though CrUX does capture country-level data:

…we are expanding the existing CrUX dataset […] to also include a collection of separate country-specific datasets!
— Chrome User Experience Report - New country dimension

…it does not make its way into Search Console or any ranking decision:

Remember that data is combined for all requests from all locations. If you have a substantial amount of traffic from a country with, say, slow internet connections, then your performance in general will go down.
— Core Web Vitals report

Unfortunately, for now at least, this means that if the majority of your paying customers are in a region that enjoys Good experiences, but you have a lot of traffic from regions that suffer Poor experiences, those worse data points may be negatively impacting your success elsewhere.

iOS (and Other) Traffic Doesn’t Count

Core Web Vitals is a Chrome initiative—evidenced by Chrome User Experience Report, among other things. The APIs used to capture the three Core Web Vitals are available in Blink, the browser engine that powers Chromium-based browsers such as Chrome, Edge, and Opera. While the APIs are available to these non-Chrome browsers, only Chrome currently captures data themselves, and populates the Chrome User Experience Report from there. So, Blink-based browsers have the Core Web Vitals APIs, but only Chrome captures data for CrUX.

It should be, hopefully, fairly obvious that non-Chrome browsers such as Firefox or Edge would not contribute data to the Chrome User Experience Report, but what about Chrome on iOS? That is called Chrome, after all?

Unfortunately, while Chrome on iOS is a project owned by the Chromium team, the browser itself does not use Blink—the only engine that can currently capture Core Web Vitals data:

Due to constraints of the iOS platform, all browsers must be built on top of the WebKit rendering engine. For Chromium, this means supporting both WebKit as well as Blink, Chrome’s rendering engine for other platforms.
— Open-sourcing Chrome on iOS!

From Apple themselves:

2.5.6 Apps that browse the web must use the appropriate WebKit framework and WebKit JavaScript.
— App Store Review Guidelines

Any browser on the iOS platform—Chrome, Firefox, Edge, Safari, you name it—uses WebKit, and the APIs that power Core Web Vitals aren’t currently available there:

From Google themselves:

There are a few notable exceptions that do not provide data to the CrUX dataset […] Chrome on iOS.
— CrUX methodology

The key takeaway here is that Chrome on iOS is actually WebKit under the hood, so capturing Core Web Vitals is not possible at all, for developers or for the Chrome team.

Core Web Vitals and Single Page Applications

If you’re building a Single-Page Application (SPA), you’re going to have to take a different approach. Core Web Vitals was not designed with SPAs in mind, and while Google have made efforts to mitigate undue penalties for SPAs, they don’t currently provide any way for SPAs to shine.

However, properly optimized MPAs do have some advantages in meeting the Core Web Vitals thresholds that SPAs currently do not.
— How SPA architectures affect Core Web Vitals

Core Web Vitals data is captured for every page load, or navigation. Because SPAs don’t have traditional page loads, and instead have route changes, or soft navigations, they don’t emit a standardised way to tell Google that a page has indeed changed. Because of this, Google has no way of capturing reliable Core Web Vitals data for these non-standard soft navigations on which SPAs are built.

The First Page View Is All That Counts

This is critical for optimising SPA Core Web Vitals for SEO purposes. Chrome only captures data from the first page a user actually lands on:

Each of the Core Web Vitals metrics is measured relative to the current, top-level page navigation. If a page dynamically loads new content and updates the URL of the page in the address bar, it will have no effect on how the Core Web Vitals metrics are measured. Metric values are not reset, and the URL associated with each metric measurement is the URL the user navigated to that initiated the page load.
— How SPA architectures affect Core Web Vitals

Subsequent soft navigations are not registered, so you need to optimise every page for a first-time visit.

What is particularly painful here is that SPAs are notoriously bad at first-time visits due to front-loading the entire application. They front-load this application in order to make subsequent page views much faster, which is the one thing Core Web Vitals will not measure. It’s a lose–lose. Sorry.

The (Near) Future Doesn’t Look Bright

Although Google are experimenting with defining soft navigations, any update or change will not be seen in the CrUX dataset anytime soon:

The Chrome User Experience Report (CrUX) will ignore these additional values… — Experimenting with measuring soft navigations

Chrome Have Done Things to Help Mitigate

As soft navigations are not counted, the user’s landing page appears very long lived: as far as Core Web Vitals sees, the user hasn’t ever left the first page they came to. This means Core Web Vitals scores could grow dramatically out of hand, counting n page views against one unfortunate URL. To help mitigate these blind spots inherent in not-using native web platform features, Chrome have done a couple of things to not overly penalise SPAs.

Firstly, Largest Contentful Paint stops being tracked after user interaction:

The browser will stop reporting new entries as soon as the user interacts with the page.
— Largest Contentful Paint (LCP)

This means that the browser won’t keep looking for new LCP candidates as the user traverses soft navigations—it would be very detrimental if a new route loading at 120 seconds fired a new LCP event against the initial URL.

Similarly, Cumulative Layout Shift was modified to be more sympathetic to long-lived pages (e.g. SPAs):

We (the Chrome Speed Metrics Team) recently outlined our initial research into options for making the CLS metric more fair to pages that are open for a long time.
— Evolving the CLS metric

CLS takes the cumulative shifts in the most extreme five-second window, which means that although CLS will constantly update throughout the whole SPA lifecycle, only the worst five-second slice counts against you.

These Mitigations Don’t Help Us Much

No such mitigations have been made with First Input Delay or Interaction to Next Paint, and none of these mitigations change the fact that you are effectively only measured on the first page in a session, or that all subsequent updates to a metric may count against the first URL a visitor encountered.

Solutions are:

Move to an MPA. It’s probably going to be faster for most use cases anyway.
Optimise heavily for first visits. This is Core Web Vitals-friendly, but you’ll still only capture one URL’s worth of data per session.
Cross your fingers and wait. Work on new APIs is promising, and we can only hope that this eventually gets incorporated into CrUX.

We Don’t Know How Much Core Web Vitals Help

Historically, Google have never typically told us what weighting they give to each of their ranking signals. The most insight we got was back in their 2010 announcement:

While site speed is a new signal, it doesn’t carry as much weight as the relevance of a page. Currently, fewer than 1% of search queries are affected by the site speed signal in our implementation and the signal for site speed only applies for visitors searching in English on Google.com at this point. We launched this change a few weeks back after rigorous testing. If you haven’t seen much change to your site rankings, then this site speed change possibly did not impact your site.
— Using site speed in web search ranking

However, this is completely separate to the Core Web Vitals initiative, where we still have zero insight as to how much impact site-speed will have on rankings.

Measuring the Impact of Core Web Vitals on SEO

If Google won’t tell us, can we work it out ourselves?

To the best of my knowledge, no one has done any meaningful study about just how much Good Page Experience might help organic rankings. The only way to really work it out would be take some very solid baseline measurements of a set of failing URLs, move them all into Good, and then measure the uptick in organic traffic to those pages. We’d also need to be very careful not to make any other SEO-facing changes to those URLs for the duration of the experiment.

Anecdotally, I do have one client that sees more than double average click-through rate—and almost the same improvement in average position—for Good Page Experience over the site’s average. For them, the data suggests that Good Page Experience is highly impactful.

So, What Do We Do?!

Search is complicated and, understandably, quite opaque. Core Web Vitals and SEO is, as we’ve seen, very intricate. But, my official advice, at a very high level is:

Keep focusing on producing high-quality, relevant content and work on site-speed because it’s the right thing to do—everything else will follow.

Faster websites benefit everyone: they convert better, they retain better, they’re cheaper to run, they’re better for the environment, and they rank better. There is no reason not to do it.

If you’d like help getting your Core Web Vitals in order, you can hire me.

Get in touch

Need Some Help?

I help companies find and fix site-speed issues. Performance audits, training, consultancy, and more.

Sources

For this post, I have only taken official Google publications into account. I haven’t included any information from Google employees’ Tweets, personal sites, conference talks, etc. This is because there is no expectation or requirement for non-official sources to edit or update their content as Core Web Vitals information changes.

Using site speed in web search ranking – Google Search Central Blog – 9 April 2010
Open-sourcing Chrome on iOS! – Chromium Blog – 31 January, 2017
Using page speed in mobile search ranking – Google Search Central Blog – 17 January 2018
Chrome User Experience Report - New country dimension – Chrome Developers – 24 January, 2018
Largest Contentful Paint (LCP) – web.dev – 8 August, 2019
Introducing Web Vitals: essential metrics for a healthy site – Chromium Blog – 5 May, 2020
Evaluating page experience for a better web – Google Search Central Blog – 28 May, 2020
Page Experience report – Search Console Help
Core Web Vitals report – Search Console Help
Canonical – Search Console Help
Understanding page experience in Google Search results – Google Search Central
Understanding Core Web Vitals and Google search results – Google Search Central
Timing for bringing page experience to Google Search – Google Search Central Blog – 10 November, 2020
Evolving the CLS metric – web.dev – 7 April, 2021
More time, tools, and details on the page experience update – Google Search Central Blog – 19 April, 2021
Simplifying the Page Experience report – Google Search Central Blog – 4 August, 2021
How SPA architectures affect Core Web Vitals – web.dev – 14 September 2021
Timeline for bringing page experience ranking to desktop – Google Search Central Blog – 4 November, 2021
About CrUX – Chrome Developers 23 June, 2022
CrUX methodology – Chrome Developers – 23 June, 2022
Experimenting with measuring soft navigations – Chrome Developers – 1 February, 2023
The role of page experience in creating helpful content – Google Search Central Blog – 19 April, 2023
Advancing Interaction to Next Paint – web.dev – 10 May, 2023
About PageSpeed Insights
App Store Review Guidelines
LargestContentfulPaint – MDN
PerformanceEventTiming – MDN
LayoutShift – MDN

The HTTP/1-liness of HTTP/2

Tue, 11 Jul 2023 20:30:54 +0000

This article started life as a Twitter thread, but I felt it needed a more permanent spot. You should follow me on Twitter if you don’t already.

I’ve been asked a few times—mostly in workshops—why HTTP/2 (H/2) waterfalls often still look like HTTP/1.x (H/1). Why are things are done in sequence rather than in parallel?

Let’s unpack it!

Fair warning, I am going to oversimplify some terms and concepts. My goal is to illustrate a point rather than explain the protocol in detail.

One of the promises of H/2 was infinite parallel requests (up from the historical six concurrent connections in H/1). So why does this H/2-enabled site have such a staggered waterfall? This doesn’t look like H/2 at all!

This doesn’t look very parallelised!

Things get a little clearer if we add Chrome’s queueing time to the graph. All of these files were discovered at the same time, but their requests were dispatched in sequence.

The white bars show how long the browser queued the request for. All files were discovered around 3.25s, but were all requested sometime after that.

As a performance engineer, one of the first shifts in thought is that we don’t care only about when resources were discovered or requests were dispatched (the leftmost part of each entry). We also care about when responses are finished (the rightmost part of each entry).

When we stop and think about it, ‘when was a file useful?’ is much more important than ‘when was a file discovered?’. Of course, a late-discovered file will also be late-useful, but really the only thing that matters is usefulness.

With H/2, yes, we can make far more requests at a time, but making more requests doesn’t magically make everything faster. We’re still limited by device and network constraints. We still have finite bandwidth, only now it needs sharing among more files—it just gets diluted.

Let’s leave the web and HTTP for a second. Let’s play cards! Taylor, Charlie, Sam, and Alex want to play cards. I am going to deal the cards to the four of them.

These four people and their cards represent downloading four files. Instead of bandwidth, the constant here is that it takes me ONE SECOND to deal one card. No matter how I do it, it will take me 52 seconds to finish the job.

The traditional round-robin approach to dealing cards would be one to Taylor, one to Charlie, one to Sam, one to Alex, and again and again until they’re all dealt. Fifty-two seconds.

This is what that looks like. It took 49 seconds before the first person had all of their cards.

Everything isn’t faster—everything is slower.

Can you see where this is going?

What if I dealt each person all of their cards at once instead? Even with the same overall 52-second timings, folk have a full hand of cards much sooner.

Half a JavaScript file is useless to us, so let’s focus on getting complete responses over the wire as soon as possible.

Thankfully, the (s)lowest common denominator works just fine for a game of cards. You can’t start playing before everyone has all of their cards anyway, so there’s no need to ‘be useful’ much earlier than your friends.

On the web, however, things are different. We don’t want files waiting on the (s)lowest common denominator! We want files to arrive and be useful as soon as possible. We don’t want a file at 49, 50, 51, 52s when we could have 13, 26, 39, 52!

On the web, it turns out that some slightly H/1-like behaviour is still a good idea.

Back to our chart. Each of those files is a deferred JS bundle, meaning they need to run in sequence. Because of how everything is scheduled, requested, and prioritised, we have an elegant pattern whereby files are queued, fetched, and executed in a near-perfect order!

Hopefully it all makes a little more sense now.

Queue, fetch, execute, queue, fetch, execute, queue, fetch, execute, queue, fetch, execute, queue, fetch, execute with almost zero dead time. This is the height of elegance, and I love it.

I fondly refer to this whole process as ‘orchestration’ because, truly, this is artful to me. And that’s why your waterfalls look like that.

In Defence of DOMContentLoaded

Sat, 01 Jul 2023 00:01:19 +0000

Honestly, I started writing this article for no real reason, and somewhat without context, in December 2022—over half a year ago! But, I left it in _drafts/ until today, when a genuinely compelling scenario came up that gives real opportunity for explanation. It no longer feels like trivia-for-the-sake-of-it thanks to a recent client project.

I never thought I’d write an article in defence of DOMContentLoaded, but here it is…

For many, many years now, performance engineers have been making a concerted effort to move away from technical metrics such as Load, and toward more user-facing, UX metrics such as Speed Index or Largest Contentful Paint. However, as an internal benchmark, there are compelling reasons why some of you may actually want to keep tracking these ‘outdated’ metrics…

Measure the User Experience

The problem with using diagnostic metrics like Load or DOMContentLoaded to measure site-speed is that it has no bearing on how a user might actually experience your site. Sure, if you have Load times of 18 seconds, your site probably isn’t very fast, but a good Load time doesn’t mean your site is necessarily very fast, either.

Which do you think provides the better user experience?

In the comparison above, which do you think provides the better user experience? I’m willing to bet you’d all say B, right? But, based on DOMContentLoaded, A is actually over 11s faster!

Load and DOMContentLoaded are internal browser events—your users have no idea what a Load time even is. I bet half of your colleagues don’t either. As metrics themselves, they have little to no reflection on the real user experience, which is exactly why we’ve moved away from them in the first place—they’re a poor proxy for UX as they’re not emitted when anything useful to the user happens.

Or are they…?

Technically Meaningful

Not all metrics need to be user-centric. I’m willing to bet you still monitor TTFB, even though you know your customers will have no concept of a first byte whatsoever. This is because some metrics are still useful to developers. TTFB is a good measure of your server response times and general back-end health, and issues here may have knock-on effects later down the line (namely with Largest Contentful Paint).

Equally, both DOMContentLoaded and Load aren’t just meaningless browser events, and once you understand what they actually signify, you can get some real insights as to your site’s runtime behaviour from each of them. Diagnostic metrics such as these can highlight bottlenecks, and how they might ultimately impact the user experience in other ways, even if not directly.

This is particularly true in the case of DOMContentLoaded.

What Does It Actually Mean?

The DOMContentLoaded event fires once all of your deferred JavaScript has finished running.

Therefore, anyone leaning heavily on defer—or frameworks that utilise it—should immediately see the significance of this metric.

If you aren’t (able to) monitoring custom metrics around your application’s interactivity, hydration state, etc., then DOMContentLoaded immediately becomes a very useful proxy. Knowing when your main bundles have run is great insight in lieu of more forensic runtime data, and it’s something I look at with any client that leans heavily on (frameworks that lean heavily on) defer or type=module.

More accurately, DOMContentLoaded signifies that all blocking and defer and type=module code has finished running. We don’t have any visibility on whether it ran successfully but it has at least finished.

Putting It to Use

I’m working with a client at the moment who is using Nuxt and has their client-side JavaScript split into an eyewatering 121 deferred files:

View unabridged.

Above, the vertical pink line at 12.201s signifies the DOMContentLoaded event. That’s late! This client doesn’t have any RUM or custom monitoring in place (yet), so, other than Core Web Vitals, we don’t have much visibility on how the site performs in the wild. Based on a 12s DOMContentLoaded event, I can’t imagine it’s doing so well.

The problem with Core Web Vitals, though, is that its only real JavaScripty metric, First Input Delay, only deals with user interaction: what I would like to know is with 121 deferred files, when is there something to actually interact with?! Based on the lab-based 12s above, I would love to know what’s happening for real users. And luckily, while DOMContentLoaded is now considered a legacy metric, we can still get field data for it from two pretty decent sources…

Chrome User Experience Report (CrUX)

Things got a lot worse between March and April 2023

CrUX Dashboard is one of very few CrUX resources that surfaces the DOMContentLoaded event to us. Above, we can see that, currently, only 11% of Chrome visitors experience a Good DOMContentLoaded—almost 90% of people are waiting over 1.5s before the app’s key functionality is available, with almost half waiting over 3.5s!

DOMContentLoaded was 4.7s for 75% of Chrome visitors in May 2023.

It would also seem that Treo (which is a truly amazing tool) surfaces DOMContentLoaded data for a given origin.

Google Analytics

Until, well, today, Google Analytics also surfaced DOMContentLoaded information. Only this time, we aren’t limited to just Chrome visits! That said, we aren’t presented with particularly granular data, either:

Huge and non-linear buckets make interrogating the data much more difficult.

After a bit of adding up (2.15 + 10.26 + 45.28 + 25.68 + 13.07 = 96.44), we see that the 95th percentile of DOMContentLoaded events for the same time period (May 2023) is somewhere between five and 10 seconds. Not massively helpful, but an insight nonetheless, and at least shows us that the lab-based 12s is unlikely to be felt by anyone other than extreme outliers in the field.

Takeaways here are:

Only about 10% of Chrome visitors have what Google deem to be a Good DOMContentLoaded. All deferred JavaScript has run within 1.5s for only the vast minority of visitors.
3.56% of all users waited over 10s for DOMContentLoaded. This is a 10 second wait for key deferred JavaScript to run.

Given that the DOMContentLoaded event fires after the last of our deferred files has run, there’s every possibility that key functionality from any preceding files has already become available, but that’s not something we have any visibility over without looking into custom monitoring, which is exactly the situation we’re in here. Remember, this is still a proxy metric—just a much more useful one than you may have realised.

If we want to capture this data more deliberately ourselves, we need to lean on the Navigation Timing API, which gives us access to a suite of milestone timings, many of which you may have heard of before.

The DOMContentLoaded as measured and emitted by the Navigation Timing API is actually referred to as domContentLoadedEventStart—there is no bare domContentLoadedEvent in that spec. Instead, we have:

domContentLoadedEventStart: This is the one we’re interested in, and is equivalent to the concept we’ve been discussing in this article so far. To get the metric we’ve been referring to as DOMContentLoaded, you need window.performance.timing.domContentLoadedEventStart.
- Because deferred JS is guaranteed to run after synchronous JS, this event also marks the point that all synchronous work is complete.
domContentLoadedEventEnd: The end event captures the time at which all JS wrapped in a DOMContentLoaded event listener has finished running:
```
window.addEventListener('DOMContentLoaded', (event) => {
  // Do something
});
```
- This is separate to deferred JavaScript and runs after our DOMContentLoaded event—if we are running a nontrivial amount of code at DOMContentLoaded, we’re also interested in this milestone. That’s not in the scope of this article, though, so we probably won’t come back to that again.

Very, very crudely, with no syntactic sugar whatsoever, you can get the page’s DOMContentLoaded event in milliseconds with the following:

console.log(window.performance.timing.domContentLoadedEventStart - window.performance.timing.navigationStart);

…and the duration (if any) of the DOMContentLoaded event with:

window.performance.timing.domContentLoadedEventEnd - window.performance.timing.domContentLoadedEventStart

And of course, we should be very used to seeing DOMContentLoaded at the bottom of DevTools’ Network panel:

They’re some satisfying numbers.

Even More Insights

While DOMContentLoaded tells us when our deferred code finished running—which is great!—it doesn’t tell us how long it took to run. We might have a DOMContentLoaded at 5s, but did the code start running at 4.8s? 2s? Who knows?!

We do.

View unabridged.

In the above waterfall, which is the same one from earlier, only even shorter, we still have the vertical pink line around 12s, which is DOMContentLoaded, but we also have a vertical sort-of yellow line around 3.5s (actually, it’s at 3.52s exactly). This is domInteractive. domInteractive is the event immediately before domContentLoadedEventStart. This is the moment the browser has finished parsing all synchronous DOM work: your HTML and all blocking scripts it encountered on the way. Basically, the browser is now at the </html> tag. The browser is ready to run your deferred JavaScript.

One very important thing to note is that the domInteractive event fired long, long before the request for file 133 was even dispatched. Immediately this tells us that the delta between domInteractive and DOMContentLoaded includes code execution and any remaining fetch.

Thankfully, the browser wasn’t just idling in this time. Because deferred code runs in sequence, the browser sensibly fetches the files in order and immediately executes them when they arrive. This level of orchestration is very elegant and helps to utilise and conserve resources in the most helpful way. Not flooding the network with responses that can’t yet be used, and also making sure that the main thread is kept busy.

This is the JavaScript we need to measure how long our deferred activity took:

console.log(window.performance.timing.domContentLoadedEventStart - window.performance.timing.domInteractive);

Now, using the Navigation Timing API, we have visibility on when our deferred finished running, and how long it took!

This demo below contains:

A slow-to-load, fast-to-run deferred JavaScript file.
A fast-to-load, slow-to-run inline script set to run at DOMContentLoaded.
Logging that out to the console at the Load event.

<!-- [1] -->
<script src=https://slowfil.es/file?type=js&delay=2000 defer></script>

<!-- [2] -->
<script>
  window.addEventListener('DOMContentLoaded', (event) => {

    // Hang the browser for 1s at the `DOMContentLoaded` event.
    function wait(ms) {
      var start = Date.now(),
      now = start;
      while (now - start < ms) {
        now = Date.now();
      }
    }

    wait(1000);

  });
</script>

<!-- [3] -->
<script>
  window.addEventListener('load', (event) => {

    const timings = window.performance.timing;
    const start   = timings.navigationStart;

    console.log('Ready to start running `defer`ed code: ' + (timings.domInteractive - start + 'ms'));
    console.log('`defer`ed code finished: ' + (timings.domContentLoadedEventEnd - start + 'ms'));
    console.log('`defer`ed code duration: ' + (timings.domContentLoadedEventStart - timings.domInteractive + 'ms'));
    console.log('`DOMContentLoaded`-wrapped code duration: ' + (timings.domContentLoadedEventEnd - timings.domContentLoadedEventStart + 'ms'));

  });
</script>

</body>
</html>

The `defer`ed code finished: 3129ms lines up with DevTools’ own reported 3.13s DOMContentLoaded.

Or take a look at the live demo on Glitch.

A Better Way?

This is all genuinely exciting and interesting to me, but we’re running into issues already:

DOMContentLoaded is a proxy for when all your deferred JavaScript has run, but it doesn’t notify you if things ran successfully, or highlight any key milestones as functionality is constantly becoming available for the duration.
DOMContentLoaded tells us how long everything took, but that could include fetch, and there’s no way of isolating the fetch from pure runtime.
If you’re capturing these technical timings, you might as well use the User Timing API.

I want to expand on the last point.

If we’re going to go to the effort of measuring Navigation Timing events, we might as well use the much more useful User Timing API. With this, we can emit high-resolution timestamps at arbitrary points in our application’s lifecycle, so instead of proxying availability via a Navigation Timing, we can drop, for example, performance.mark('app booted') in our code. In fact, this is what Next.js does to let you know when the app has hydrated, and how long it took. These User Timings automatically appear in the Performance panel:

I use performance.mark() and performance.measure() in a few places on this site, chiefly to monitor how long it takes to parse the <head> and its CSS.

The User Timing API is far more suited to this kind of monitoring than something like DOMContentLoaded—I would only look at DOMContentLoaded if we don’t yet have appropriate metrics in place.

Still, the key and most interesting takeaway for me is that if all we have access to is DOMContentLoaded (or we aren’t already using something more suitable), then we do actually have some visibility on app state and availability. If you are using defer or type=module, then DOMContentLoaded might be more useful to you than you realise.

Back to Work

I mentioned previously that the DOMContentLoaded event fires once all deferred JavaScript has run, which means that we could potentially be trickling functionality throughout the entire time between domInteractive and DOMContentLoaded.

In my client’s case, however, the site is completely nonfunctional until the very last file (response 133 in the waterfall) has successfully executed. In fact, blocking the request for file 133 has the exact same effect as disabling JavaScript entirely. This means the DOMContentLoaded event for them is an almost exact measure of when the app is available. This means that tracking and improving DOMContentLoaded will have a direct correlation to an improved customer experience.

Improving `DOMContentLoaded`

Given that DOMContentLoaded marks the point at which all synchronous HTML and JavaScript has been dealt with, and all deferred JavaScript has been fetched and run, this leaves us many different opportunities to improve the metric: we could reduce the size of our HTML, we could remove or reduce expensive synchronous JavaScript, we could inline small scripts to remove any network cost, and we can reduce the amount of deferred JavaScript.

Further, as DOMContentLoaded is a milestone timing, any time we can shave from preceding timings should be realised later on. For example, all things being equal, a 500ms improvement in TTFB will yield a 500ms improvement in subsequent milestones, such as First Contentful Paint or, in our case, DOMContentLoaded.

However, in our case, the delta between domInteractive and DOMContentLoaded was 8.681s, or about 70%. And while their TTFB certainly does need improvement, I don’t think it would be the most effective place to spend time while tackling this particular problem.

Almost all of that 8.7s was lost to queuing and fetching that sheer number of bundles. Not necessarily the size of the bundles—just the sheer quantity of files that need scheduling, and which each carry their own latency cost.

While we haven’t worked out the sweet spot for this project, as a rule, a smaller number of larger bundles would usually download much faster than many tiny ones:

As a rule, RTT (α) stays constant while download time (𝑥) is proportional to filesize. Therefore, splitting one large bundle into 16 smaller ones goes from 1α + 𝑥 to 16α + 16(0.0625𝑥). Expect things to probably get a little slower. pic.twitter.com/c0hEsIAwKq
— Harry Roberts (@csswizardry) 21 January, 2021

My advice in this case is to tweak their build to output maybe 8–10 bundles and re-test from there. It’s important to balance bundle size, number of bundles, and caching strategies, but it’s clear to me that the issue here is overzealous code-splitting.

With that done, we should be able to improve DOMContentLoaded, thus having a noticeable impact on functionality and therefore customer experience.

DOMContentLoaded has proved to be a very, very useful metric for us.

Site-Speed Topography Remapped

Wed, 07 Jun 2023 17:11:58 +0000

N.B. This is an update to my 2020 article Site-Speed Topography. You will need to catch up with that piece before this one makes sense.

Around two and a half years ago, I debuted my Site-Speed Topography technique for getting broad view of an entire site’s performance from just a handful of key URLs and some readily available metrics.

In that time, I have continued to make extensive use of the methodology (alongside additional processes and workflows), and even other performance monitoring tools have incorporated it into their own products. Also in that time, I have adapted and updated the tools and technique itself…

Get the new spreadsheet!

What Is Site-Speed Topography?

Firstly, let’s recap the methodology itself.

The idea is that by taking a handful of representative page- or template-types from an entire website, we can quickly build the overall landscape—or topography—of the site by comparing and contrasting numerical and milestone timings.

Realistically, you need to read the original post before this article will make sense, but the basic premise is that by taking key metrics from multiple different page types, and analysing the similarities, differences, or variations among them, you can also very quickly realise some very valuable insights about other metrics and optimisations you haven’t even captured.

Pasting a bunch of WebPageTest results into a spreadsheet is where we start:

The old Site-Speed Topography spreadsheet. Plugging milestone timings into a spreadsheet helps us spot patterns and problems.

Similar TTFB across pages suggests that no pages have much more expensive database calls than others; large deltas between TTFB and First Contentful Paint highlight a high proportion of render-blocking resources; gaps between Largest Contentful Paint and SpeedIndex suggest late-loaded content. These insights gained across several representative page types allow us to build a picture of how the entire site might be built, but from observing only a small slice of it.

The backbone of the methodology is—or at least was—viewing the data graphically and spotting patterns in the bar chart:

Viewing the data as a bar chart can help highlight some of the issues.

Above, we can see that the Product Listing Page (PLP) is by far the worst performing of the sample, and would need particular attention. We can also see that First Paint and First Contentful Paint are near identical on all pages except the PLP—is this a webfont issue? In fact, we can see a lot of issues if we look hard enough. But… who wants to look hard? Shouldn’t these things be easier to spot?

Yes. They should.

Remapping

If you read the original post, the section Building the Map explains in a lot of words a way to spot visually a bunch of patterns that live in numbers.

Surely, if we have all of the facts and figures in front of us anyway, manually eyeballing a bar chart to try and spot patterns is much more effort than necessary? We’re already in a spreadsheet—can’t we bring the patterns to us?

Yes. We can.

Here is the new-look Site-Speed Topography spreadsheet:

A much more colourful spreadsheet provides way more information. View full size/quality (182KB)

Get the new spreadsheet!

Now, without having to do any mental gymnastics at all, we can quickly see:

How pages perform overall across all metrics.
Which pages exhibit the best or worst scores for a given metric.
General stability of a specific metric.
Are any metrics over budget? By how much?
- We can also set thresholds for those budgets.
We can begin to infer other issues from metrics already present.

Of course, we can still graph the data, but we soon find that that’s almost entirely redundant now—we solved all of our problems in the numbers.

Graphing the data doesn’t provide as much benefit anymore, but it’s built into the spreadsheet by default.

Visually, patterns do still emerge: the PD- and SR-Pages have more dense clusters of bars, suggesting overall worse health; the Home and Product pages have by far the worst LCP scores; the SRP’s CLS is through the roof. But this is only visual and not exactly persistent. Still, I have included the chart in the new spreadsheet because different people prefer different approaches.

Without looking at a single line of code—without even visiting a single one of these pages in a browser!—we can already work out where our main liabilities lie. We know where to focus our efforts, and our day-one to-do list is already written. No more false starts and dead ends. Optimise the work not done.

So, what are you waiting for? Grab a copy of the new Site-Speed Topography spreadsheet along with a 15-minute screencast explainer!

Get the new spreadsheet!

Why Not document.write()?

Tue, 10 Jan 2023 16:17:11 +0000

If you’ve ever run a Lighthouse test before, there’s a high chance you’ve seen the audit Avoid document.write():

For users on slow connections, external scripts dynamically injected via document.write() can delay page load by tens of seconds.

You may have also seen that there’s very little explanation as to why document.write() is so harmful. Well, the short answer is:

From a purely performance-facing point of view, document.write() itself isn’t that special or unique. In fact, all it does is surfaces potential behaviours already present in any synchronous script—the only main difference is that document.write() guarantees that these negative behaviours will manifest themselves, whereas other synchronous scripts can make use of alternate optimisations to sidestep them.

N.B. This audit and, accordingly, this article, only deals with script injection using document.write()—not its usage in general. The MDN entry for document.write() does a good job of discouraging its use.

What Makes Scripts Slow?

There are a number of things that can make regular, synchronous scripts¹ slow:

Synchronous JS can block DOM construction while the file is downloading.
- The belief that synchronous JS blocks DOM construction is only true in certain scenarios.
Synchronous JS always blocks DOM construction while the file is executing.
- It runs in-situ at the exact point it’s defined, so anything defined after the script has to wait.
Synchronous JS never blocks downloads of subsequent files.
- This has been true for almost 15 years at the time of writing, yet still remains a common misconception among developers. This is closely related to the first point.

The worst case scenario is a script that falls into both (1) and (2), which is more likely to affect scripts defined earlier in your HTML. document.write(), however, forces scripts into both (1) and (2) regardless of when they’re defined.

The Preload Scanner

The reason scripts never block subsequent downloads is because of something called the Preload Scanner. The Preload Scanner is a secondary, inert, download-only parser that’s responsible for running down the HTML and asynchronously requesting any available subresources it might find, chiefly anything contained in src or href attributes, including images, scripts, stylesheets, etc. As a result, files fetched via the Preload Scanner are parallelised, and can be downloaded asynchronously alongside other (potentially synchronous) resources.

The Preload Scanner is decoupled from the primary parser, which is responsible for constructing the DOM, the CSSOM, running scripts, etc. This means that a large majority of files we fetch are done so asynchronously and in a non-blocking manner, including some synchronous scripts. This is why not all blocking scripts block during their download phase—they may have been fetched by the Preload Scanner before they were actually needed, thus in a non-blocking manner.

The Preload Scanner and the primary parser begin processing the HTML at more-or-less the same time, so the Preload Scanner doesn’t really get much of a head start. This is why early scripts are more likely to block DOM construction during their download phase than late scripts: the primary parser is more likely to encounter the relevant <script src> element while the file is downloading if the <script src> element is early in the HTML. Late (e.g. at-</body>) synchronous <script src>s are more likely to be fetched by the Preload Scanner while the primary parser is still hung up doing work earlier in the page.

Put simply, scripts defined earlier in the page are more likely to block on their download than later ones; later scripts are more likely to have been fetched preemptively and asynchronously by the Preload Scanner.

`document.write()` Hides Files From the Preload Scanner

Because the Preload Scanner deals with tokeniseable src and href attributes, anything buried in JavaScript is invisible to it:

<script>
  document.write('<script src=file.js><\/script>')
</script>

This is not a reference to a script; this is a string in JS. This means that the browser can’t request this file until it’s actually run the <script> block that inserts it, which is very much just-in-time (and too late).

document.write() forces scripts to block DOM construction during their download by hiding them from the Preload Scanner.

What About Async Snippets?

Async snippets such as the one below suffer the same fate:

<script>
  var script = document.createElement('script');
  script.src = 'file.js';
  document.head.appendChild(script);
</script>

Again, file.js is not a filepath—it’s a string! It’s not until the browser has run this script that it puts a src attribute into the DOM and can then request it. The primary difference here, though, is that scripts injected this way are asynchronous by default. Despite being hidden from the Preload Scanner, the impact is negligible because the file is implicitly asynchronous anyway.

That said, async snippets are still an anti-pattern—don’t use them.

`document.write()` Executes Synchronously

document.write() is almost exclusively used to conditionally load a synchronous script. If you just need a blocking script, you’d use a simple <script src> element:

<script src=file.js></script>

If you needed to conditionally load an asynchronous script, you’d add some if/else logic to your async snippet.

<script>

  if (condition) {
    var script = document.createElement('script');
    script.src = 'file.js';
    document.head.appendChild(script);
  }

</script>

If you need to conditionally load a synchronous script, you’re kinda stuck…

Scripts injected with, for example, appendChild are, per the spec, asynchronous. If you need to inject a synchronous file, one of the only straightforward options is document.write():

<script>

  if (condition) {
    document.write('<script src=file.js><\/script>')
  }

</script>

This guarantees a synchronous execution, which is what we want, but it also guarantees a synchronous fetch, because this is hidden from the Preload Scanner, which is what we don’t want.

document.write() forces scripts to block DOM construction during their execution by being synchronous by default.

Is It All Bad?

The location of the document.write() in question makes a huge difference.

Because the Preload Scanner works most effectively when it’s dealing with subresources later in the page, document.write() earlier in the HTML is less harmful.

Early `document.write()`

<head>

  ...

  <script>
    document.write('<script src=https://slowfil.es/file?type=js&delay=1000><\/script>')
  </script>

  <link rel=stylesheet href=https://slowfil.es/file?type=css&delay=1000>

  ...

</head>

If you put a document.write() as the very first thing in your <head>, it’s going to behave the exact same as a regular <script src>—the Preload Scanner wouldn’t have had much of a head start anyway, so we’ve already missed out on the chance of an asynchronous fetch:

document.write() as the first thing in the <head>. FCP is at 2.778s.

Above, we see that the browser has managed to parallelise the requests: the primary parser ran and injected the document.write(), while the Preload Scanner fetched the CSS.

Owing to CSS’ Highest priority, it will always be requested before High priority JS, regardless of where each is defined.

If we replace the document.write() with a simple <script src>, we’d see the exact same behaviour, meaning in this specific instance, document.write() is no more harmful than a regular, synchronous script:

<head>

  ...

  <script src=https://slowfil.es/file?type=js&delay=1000></script>

  <link rel=stylesheet href=https://slowfil.es/file?type=css&delay=1000>

  ...

</head>

This yields an identical waterfall:

Using a syncrhonous <script src> instead of document.write(). FCP is at 2.797s.

Because the Preload Scanner was unlikely to find either variant, we don’t notice any real degradation.

Late `document.write()`

<head>

  ...

  <link rel=stylesheet href=https://slowfil.es/file?type=css&delay=1000>

  <script>
    document.write('<script src=https://slowfil.es/file?type=js&delay=1000><\/script>')
  </script>

  ...

</head>

Because JS can write/read to/from the CSSOM, all browsers will halt execution of any synchronous JS if there is any preceding, pending CSS. In effect, CSS blocks JS, and in this example, serves to hide the document.write() from the Preload Scanner.

Thus, document.write() later in the page does become more severe. Hiding a file from the Preload Scanner—and only surfacing it to the browser the exact moment we need it—is going to make its entire fetch a blocking action. And, because the document.write() file is now being fetched by the primary parser (i.e. the main thread), the browser can’t complete any other work while the file is on its way. Blocking on top of blocking.

As soon as we hide the script file from the Preload Scanner, we notice drastically different behaviour. By simply swapping the document.write() and the rel=stylesheet around, we get a much, much slower experience:

document.write() late in the <head>. FCP is at 4.073s.

Now that we’ve hidden the script from the Preload Scanner, we lose all parallelisation and incur a much larger penalty.

It Gets Worse…

The whole reason I’m writing this post is that I have a client at the moment who is using document.write() late in the <head>. As we now know, this pushes both the fetch and the execution on the main thread. Because browsers are single-threaded, this means that not only are we incurring network delays (thanks to a synchronous fetch), we’re also leaving the browser unable to work on anything else for the entire duration of the script’s download!

The main thread goes completely silent during the injected file’s fetch. This doesn’t happen when files are fetched from the Preload Scanner.

Avoid `document.write()`

As well as exhibiting unpredictable and buggy behaviour as keenly stressed in the MDN and Google articles, document.write() is slow. It guarantees both a blocking fetch and a blocking execution, which holds up the parser for far longer than necessary. While it doesn’t introduce any new or unique performance issues per se, it just forces the worst of all worlds.

Avoid document.write() (but at least now you know why).

<script src=></script> ↩

CSS Wizardry

Correctly Configure (Pre) Connections

Learn by Example

Working Out Which Origins to preconnect

Don’t preconnect Too Many Origins

When to Use crossorigin

Sec-* Request Headers

preconnect and DNS

The Three Cs: 🤝 Concatenate, 🗜️ Compress, 🗳️ Cache

🤝 Concatenate

🗜️ Compress

🗳️ Cache

📡 Connection

📱 Client

My Advice

The Future Is Brighter

Shared Dictionary Compression for HTTP

Compression Dictionaries

tl;dr

Appendix: Test Methodology

What Is the Maximum max-age?

max-age

immutable

31536000 Seconds

2147483648 Seconds

Does It Matter?

How to Clear Cache and Cookies on a Customer’s Device

Getting Someone to Clear Their Own Cache

Clear-Site-Data

Preventing Malicious Clears

Clear-Site-Data for Developers

Clearing Cache on iOS

The Ultimate Low-Quality Image Placeholder Technique

Low-Quality Image Placeholders

Core Web Vitals and Largest Contentful Paint

Largest Contentful Paint Caveats

Don’t Upscale Your LQIP

Calculating the Upscaling Penalty

Aim for a Minimum of 0.05BPP

LQIP and BPP Calculator

Implementing Low-Quality Image Placeholders

Use an Image Transformation Service

Use Your Judgement

Summary

Core Web Vitals for Search Engine Optimisation: What Do We Need to Know?

Updates

Core Web Vitals

Site-Speed Is More Than SEO

The Core Web Vitals Metrics

Some History

The Best Content Always Wins

Core Web Vitals Are Important

It’s Not Just About Core Web Vitals

You Don’t Need to Pass FID

Interaction to Next Paint Doesn’t Matter Yet

You’re Ranked on Individual URLs

Search Console Is Gospel

Ignore Lighthouse and PageSpeed Insights Scores

Failing Pages Don’t Get Penalised

Core Web Vitals Are a Tie-Breaker

There Are No Shades of Good or Failed URLs

Mobile and Desktop Thresholds Are the Same

Slow Countries Can Harm Global Rankings

iOS (and Other) Traffic Doesn’t Count

Core Web Vitals and Single Page Applications

The First Page View Is All That Counts

The (Near) Future Doesn’t Look Bright

Chrome Have Done Things to Help Mitigate

These Mitigations Don’t Help Us Much

We Don’t Know How Much Core Web Vitals Help

Measuring the Impact of Core Web Vitals on SEO

So, What Do We Do?!

Sources

The HTTP/1-liness of HTTP/2

In Defence of DOM­Content­Loaded

Measure the User Experience

Technically Meaningful

What Does It Actually Mean?

Putting It to Use

Chrome User Experience Report (CrUX)

Working Out Which Origins to `preconnect`

Don’t `preconnect` Too Many Origins

When to Use `crossorigin`

`Sec-*` Request Headers

`preconnect` and DNS

`max-age`

`immutable`

`31536000` Seconds

`2147483648` Seconds

`Clear-Site-Data`

`Clear-Site-Data` for Developers

In Defence of DOMContentLoaded

Improving `DOMContentLoaded`

`document.write()` Hides Files From the Preload Scanner

`document.write()` Executes Synchronously

Early `document.write()`

Late `document.write()`

Avoid `document.write()`