Serving an index

An index is two JSON documents and a pile of immutable package files, so anything that serves bytes can host one. A server earns its place by doing what static hosting cannot.

If none of those apply, DirectoryIndex plus aws s3 sync is the better answer.

Routes

GET|HEAD  /v1/index.json              the catalog
GET|HEAD  /v1/plugins/<name>.json     one plugin's releases
GET|HEAD  /v1/blobs/sha256:<hex>      the package for that digest

Paths come from CATALOG_PATH and releases_path in stanchion-dist, so the server and the client cannot drift apart when the schema moves to v2.

404 is load-bearing. A client maps 404 to “no such plugin” and every other non-2xx to “the index is unreachable” — the difference between reporting a missing plugin and retrying. An unknown plugin or digest is 404; a malformed name or digest is 400.

Freshness, and why time is quantised

expires is computed per response from a TTL — but from the start of the TTL window, not from the wall clock:

IndexServer::new(DirectorySource::new("/srv/plugins"))
    .ttl(Duration::from_secs(3600))

That detail is what makes conditional requests possible at all. A document that took expires from now would have different bytes on every request, so an ETag over the body could never match and If-None-Match would be dead weight. Quantised, every response inside a window is byte-identical, the ETag is a plain content hash, and Cache-Control: max-age counts down to the window’s end.

It also closes the obvious hole: a 304 lets a client keep the body it has, including the expires it was served with. Because a match can only occur inside the window that issued it, a conditional request can never extend a document past its expiry — at the boundary the bytes change, the validator stops matching, and the client is handed a fresh document.

The TTL is the one real tuning decision, so it is clamped to 60s–24h. Too short and clients with skewed clocks start refusing documents; too long and a yank takes that long to reach someone being fed a withheld snapshot.

Packages are immutable — a package is named by the digest of what it unpacks to — so they are served with immutable and their digest as the ETag.

Framework independence

Nothing in stanchion-index knows about a web framework. IndexServer::serve takes a method, a path and an optional If-None-Match, and returns a Served: a status, headers and bytes. Routing, freshness and conditional requests are all tested without binding a port.

Poem needs its own adapter because it is not Tower-based: it has its own Endpoint trait, and its tower-compat feature only lets it consume tower::Layer, pinned to tower 0.4 while axum is on 0.5. Any other framework is a ten-line match on Served.

The source is synchronous, matching PluginIndex, so adapters run it on a blocking worker rather than an async one.

On-disk layout

<base>/v1/index.json              the catalog, optional
<base>/v1/plugins/<name>.json     one plugin's releases
<base>/blobs/sha256/<hex>         the package for that digest

Packages live under a sha256/ segment rather than a sha256:<hex> filename, because a colon in a filename is legal on Linux and a nuisance everywhere else.

Stored documents are read with Freshness::Ignored: their own expires is irrelevant when the server issues a new one. A directory too stale for a client to accept is still a perfectly good source for a server that keeps it fresh.

Streaming packages

A package is served from a reader, never buffered. IndexSource::blob returns a Blob — a Box<dyn Read + Send> and a length where the source knows one — so building a response reads nothing at all; the bytes move only as an adapter drains them into the socket.

fn blob(&self, digest: &str) -> Result<Blob, IndexError>;

The source stays synchronous, matching PluginIndex: opening a file is not an async problem, but writing it to a socket is. One bridge in stanchion-index::chunks reconciles those — the read runs on a blocking worker and 64 KiB chunks arrive through a channel four deep, so a response holds a few hundred KiB in flight regardless of the package’s size. Both adapters use it, so chunking is decided in one place.

Headers are decided in one place for the same reason. Served::headers assembles Content-Type, Cache-Control, ETag and Content-Length — each when it means something — and an adapter sets what it is handed rather than working out which headers an answer deserves. Both adapters used to assemble them independently, which is how a freshness fix reaches one framework and not the other.

Two consequences worth knowing:

• A HEAD never reads the package. The length comes from the source, so a HEAD on a 2 GiB archive costs a stat.
• A read failure part-way truncates the response. By then the status and headers have gone, so a truncated body is the only signal left — and a client verifying the digest of what it unpacks will reject it, which is the behaviour we want.

If the client hangs up, the channel closes and the reader stops rather than pulling the rest of the file into a queue nobody drains.

Documents are still bytes, because they have to be: the ETag is a hash of them.

What this does not do

• No write path. Publishing is putting an archive somewhere fetchable and adding a release to the index — two file writes. A publish API would make the server authoritative over what gets pinned, which is a different trust model.
• No authority. Everything served is re-checked against a digest the client holds. A compromised bucket reached over flawless TLS is a compromised index.

← Documentation index