caduh

HTTP Caching Deep‑Dive — headers that actually make things fast

5 min read

Freshness, validators, and directives: Cache-Control, ETag/Last-Modified, Vary, s-maxage, stale-while-revalidate, immutable. Patterns for APIs, HTML, and assets—plus CDN tips.

TL;DR

  • Static assets: fingerprint filenames and serve with Cache-Control: public, max-age=31536000, immutable. Cache hard at CDN + browser; deploy new URLs on change.
  • HTML & APIs: make them revalidatable: send ETag (or Last-Modified) and short freshness (max-age=0 or small), plus stale-while-revalidate/stale-if-error.
  • Shared caches (CDN): use s-maxage (edge TTL) that can be longer than max-age. Optionally use Surrogate-Control headers on CDNs that support them.
  • Personalized/auth responses aren’t shared by default. Use private (browser-only) or explicitly allow with public/s-maxage + Vary: Authorization when safe.
  • Keep Vary tight (e.g., Accept-Encoding, Accept-Language). Over-varying destroys hit rate.
  • Prefer strong ETags (content hash). Use weak ETags (W/) when bytes differ but representation is equivalent (e.g., gzip differences).

1) Freshness model (how caches decide to serve vs ask)

A cache response is fresh if within its freshness lifetime; otherwise it’s stale and needs revalidation.

Explicit freshness (best)

Cache-Control: max-age=60, s-maxage=300, stale-while-revalidate=60, stale-if-error=300
  • max-age: browser (and shared caches) freshness in seconds.
  • s-maxage: overrides max-age for shared caches (CDN/proxies).
  • stale-while-revalidate: serve stale while revalidating in background.
  • stale-if-error: serve stale on 5xx/timeouts.

Heuristic freshness (avoid): if no max-age, caches may derive it from Last-Modified. Prefer explicit headers.

Must revalidate

Cache-Control: max-age=0, must-revalidate
  • Always revalidate when stale; no heuristic freshness.

2) Validators: ETag & Last-Modified

  • ETag (opaque token, ideally a content hash):
ETag: "b3c3b8b6"         # strong (byte-for-byte)
ETag: W/"b3c3b8b6"       # weak (semantically same)
  • Last-Modified:
Last-Modified: Wed, 24 Sep 2025 10:21:00 GMT

Conditional requests
Client sends:

If-None-Match: "b3c3b8b6"
If-Modified-Since: Wed, 24 Sep 2025 10:21:00 GMT

Server replies 304 Not Modified if unchanged, 200 with body if changed. Prefer ETag; time-based validation is coarser.

3) Cache-Control directives that matter

  • public / private — allow shared caches vs browser-only.
  • no-store — don’t write to cache at all (use for highly sensitive).
  • no-cache — may store, but must revalidate before each use.
  • max-age=N — freshness lifetime for all caches.
  • s-maxage=N — freshness for shared caches only.
  • must-revalidate — once stale, no serving without revalidation.
  • immutable — representation won’t change while fresh (great for fingerprinted assets).

4) Vary: pick the minimal key set

Vary tells caches which request headers affect the response. Too many → tiny hit rates; too few → incorrect content.

Common, safe values:

Vary: Accept-Encoding
Vary: Accept-Language

For content negotiation (e.g., API returns JSON vs HTML):

Vary: Accept

Personalization/auth (only if deliberately caching at CDN):

Cache-Control: public, s-maxage=60
Vary: Authorization

Be careful: Vary: * disables reuse in shared caches.

5) Patterns by content type

A) Fingerprinted static assets (JS/CSS/images)

  • Build assets with hashes in filenames (e.g., app.ab12cd34.js).
  • Serve:
Cache-Control: public, max-age=31536000, immutable
ETag: "ab12cd34"   # optional when filename is already content-hashed
  • Deploy: change URL when content changes; no purges needed.

Nginx

location ~* \.(?:js|css|png|jpg|svg)$ {
  add_header Cache-Control "public, max-age=31536000, immutable";
}

B) SSR/HTML documents

  • Revalidatable, short freshness (CDN can be longer):
Cache-Control: public, max-age=0, s-maxage=60, stale-while-revalidate=30
ETag: "page-2025-09-24T10:20Z"
  • Avoid immutable on HTML. If personalized, use private (browser-only) or separate by cookie/auth at the edge.

C) API GET endpoints

  • Shared cache when responses are tenant/anon-safe:
Cache-Control: public, max-age=30, s-maxage=120, stale-while-revalidate=60
ETag: "list-v2-9a7f"
Vary: Accept, Accept-Encoding
  • Personalized responses: use private, max-age=0 (browser can revalidate with ETag).
  • For infrequently changing resources, send longer max-age and rely on ETag revalidation for freshness.

D) Truly sensitive data

Cache-Control: no-store
Pragma: no-cache      # legacy proxies

No ETag, no caching.

6) CDN/Reverse proxy specifics (mental model)

  • Edge TTL: set via s-maxage or Surrogate-Control (Fastly/Akamai).
  • Cache key: define which request parts matter (host, path, query, headers). Ignore noisy query params when safe.
  • Purge/Invalidate: prefer surrogate keys/tags to invalidate groups (e.g., all pages with product 123).
  • Compression: enable Brotli/gzip at the edge; ensure Vary: Accept-Encoding is present.
  • HTTPS: caches should normalize to the same origin; don’t mix schemes inadvertently.

Express example (API)

app.get("/api/products", (req, res) => {
  res.set("Cache-Control", "public, max-age=30, s-maxage=180, stale-while-revalidate=60");
  res.set("ETag", makeStrongEtag(resBody)); // content hash
  res.set("Vary", "Accept, Accept-Encoding");
  res.json(resBody);
});

7) Service Worker (cache + revalidate pattern)

// sw.js
self.addEventListener("fetch", event => {
  const req = event.request;
  event.respondWith((async () => {
    const cache = await caches.open("v1");
    const cached = await cache.match(req);
    const fetchPromise = fetch(req).then(async res => {
      if (res.ok) await cache.put(req, res.clone());
      return res;
    }).catch(() => cached || Response.error());
    return cached || fetchPromise; // cache-first, then update
  })());
});

Use HTTP headers in addition to SW logic; they help CDN + browser when SW is absent/disabled.

8) Debugging & observability

  • In DevTools Network: look for (from disk cache) / (from memory cache) and 304 responses.
  • Log ETag, Age, Cache-Control, Vary at the edge.
  • Track hit rate, revalidation rate, and 5xx served via stale-if-error.

Revalidation flow

  1. Browser has a stale cached response with ETag: "abc"
  2. Sends If-None-Match: "abc"
  3. Server replies 304, small payload, cache updates freshness.

9) Pitfalls & fast fixes

| Pitfall | Why it bites | Fix | |---|---|---| | No explicit freshness | Heuristic caching, inconsistent behavior | Set max-age/s-maxage explicitly | | ETag varies per server | Always miss 304s | Use content hash; avoid inode/mtime-based tags | | Over-broad Vary (e.g., User-Agent) | Zero hit rate | Keep Vary minimal (Accept-*, encoding, auth when needed) | | Authenticated GETs cached by CDN accidentally | Data leakage | Use private or add explicit public,s-maxage + Vary: Authorization | | Mixing no-store with immutable | Contradiction | Choose one; no-store wins | | Relying on cache purge for assets | Complexity & risk | Fingerprint URLs and cache forever | | Forgetting Date header | Freshness math breaks | Ensure server/proxy sets correct Date |

Quick checklist

  • [ ] Fingerprint assets; serve with max-age=31536000, immutable.
  • [ ] HTML/API: ETag + short max-age (and s-maxage for CDNs) + stale-while-revalidate.
  • [ ] Keep Vary minimal and deliberate.
  • [ ] Use public/private correctly; avoid caching sensitive data.
  • [ ] Define CDN cache key and purge by tags/keys.
  • [ ] Monitor hit rate, Age, and 304 rates; fix misses.

One‑minute adoption plan

  1. Turn on filename fingerprinting for static assets and set immutable caching.
  2. Add ETag to HTML/API responses; set max-age=0, s-maxage=60, stale-while-revalidate=30.
  3. Audit and trim Vary headers; ensure Accept-Encoding is present when compressing.
  4. Configure CDN cache key and purge by surrogate key; stop purging assets.
  5. Add dashboards for edge hit rate and 304 ratio; iterate.
← Back to all posts