blog.robino.dev

Fast Response Times with ETags

  • cache
  • etag
  • javascript
  • performance
Oct 8, 2024
How to use ETags in server-side JavaScript to speed up your website.

Caching basics

When creating a web application, a common performance optimization is to save many of the files that are required to run your application on users’ devices so that the next time they visit the page it will load faster. This is called caching, you can open your browser cache and see all of the different assets that have been saved to your computer.

For each Response sent from your application, you can set headers in the Response that tell the browser to cache the request in various ways.

Linked assets

Many assets such as JS and CSS can be cached for a long time since they have a unique hashed filenames that are generated during the build. These assets are requested indirectly by the client—for example via <link> or <script> tags—after the initial HTML response is received.

A linked stylesheet might look like this.

<!doctype html>
<html lang="en">
	<head>
		<meta charset="UTF-8" />
		<meta name="viewport" content="width=device-width, initial-scale=1.0" />
		<title>etag</title>
		<link
			rel="stylesheet"
			crossorigin
			href="/_immutable/assets/main.Cs4aW6Ww.css"
		/>
	</head>
	<body>
		...
	</body>
</html>

Each time the CSS file is updated, a new hash will be generated for the file using its content. The client will always make the request to the correct, newest asset, since the link is provided in the HTML.

These assets are often all output during the build process into an immutable/ directory and served with immutable cache headers.

{ "Cache-Control": "public, immutable, max-age=31536000" }

By doing this, users only ever have to download these assets once, they are saved to their computer for up to a year and loaded from the cache in subsequent requests.

HTML

One response that generally cannot have a hashed URL is the HTML response. Users need to be able to navigate to the same URL and always receive the most up to date HTML page. You wouldn’t want to have to send a new link to your users every time you updated your website!

This response can still be sped up by using ETags (entity tags). Etags allow you to check if the file has been modified since the last time it was requested. If it hasn’t, you can just send a small response saying that it is the same as last time, instead of sending the entire response again.

ETag Tutorial

In this tutorial, I’ll show you how to implement ETags to speed up an HTML response. I’m going to be using domco, but this example can be used in any server-side JavaScript framework that supports setting custom headers on a response.

Run npm create domco@latest to get a similar template as the one in this tutorial. The final example can be found in this repository as well.

Here’s an example of sending a HTML response without an ETag.

import { html } from "client:page";
import type { Handler } from "domco";

export const handler: Handler = (req) => {
	return new Response(html, {
		headers: {
			"Content-Type": "text/html",
		},
	});
};

Currently, this response requires Response.body—the html—to be sent over the network with each request.

Creating a hash

The first step to utilizing ETags is to find a hashing algorithm to use to create the tag and to check if the content has changed.

Since ETags will often be generated on every request, it’s important to choose an efficient method to create a unique, consistent hash. The hash must produce the same output, given the same input, in order to understand if the content has changed since the last time the hash was created. So methods like using the current time will not work if the ETag is generated for each request, since the time will be different each time the function is run.

Framework examples

Here’s how a variety of popular frameworks generate ETags, I’ve linked to the code where the ETag is created. All of these methods have proved to work well in widespread production use.

  • sirv uses a combination of the file size and the last modified time. This works great for static assets, but it won’t work if you are generating a string of HTML dynamically since there is no file to read.
  • Next.js and Astro both use the same FNV-1a algorithm from this fnv-plus project.
  • SvelteKit uses a JavaScript implementation of the DJB2 algorithm.
  • Hono uses the SHA1 algorithm via the Web Crypto API. The etag package (used in Express) also uses SHA1 via the Node Crypto API.

DJB2

I’m going to use the DJB2 algorithm from SvelteKit for this tutorial, I modified it to use TypeScript.

Sending the ETag to the client

Next, we can create an ETag using the HTML string, and send it to the client in a response header along with the HTML.

import { html } from "client:page";
import type { Handler } from "domco";

export const handler: Handler = (req) => {
	// surround the hash with double quotes
	const eTag = `"${djb2(html)}"`;

	return new Response(html, {
		headers: {
			// other headers should remain the same
			"Content-Type": "text/html",
			// send it as the "Etag" header
			ETag: eTag,
		},
	});
};

Now, the client’s browser will automatically send this hash back in subsequent requests in the If-None-Match header.

import { html } from "client:page";
import type { Handler } from "domco";

export const handler: Handler = (req) => {
	// hash is sent back in this header
	console.log(req.headers.get("If-None-Match")); // ex: "1wnhp22"

	const eTag = `"${djb2(html)}"`;

	return new Response(html, {
		headers: {
			"Content-Type": "text/html",
			ETag: eTag,
		},
	});
};

Check for changes

You can check if the content is still the same as what the user has by comparing their hash, to the one you are currently generating.

If the content hasn’t been modified, we can send null instead of sending the content again, with a 304 status code to tell the client that the response is Not Modified.

import { html } from "client:page";
import type { Handler } from "domco";

export const handler: Handler = (req) => {
	const eTag = `"${djb2(html)}"`;

	// Check if the hash sent from the client matches the hash
	// generated. If it does, the content hasn't been modified.
	const notModified = eTag === req.headers.get("If-None-Match");

	// send `null` instead of the html in the body
	return new Response(notModified ? null : html, {
		// change the status to 304 - not modified
		status: notModified ? 304 : 200,
		headers: {
			"Content-Type": "text/html",
			ETag: eTag,
		},
	});
};

Now when the user refreshes the page, a much smaller response will be sent instead of the entire HTML page being sent over the network again.

Conclusion

I hope this provides some insight into how you can speed up your web application with different caching techniques. For large HTML pages or for slow connections, ETags can really speed up the loading time of a website.

Thanks for reading!

Edit