Skip to content

The production stack

A single R2/S3 read is cheap. A repo page that renders a file tree, a commit log, and a diff is not — it can mean hundreds of object reads. This guide is the stack a production git-hosting service actually runs, in the order it actually runs in, with the reasoning for each layer.

import {
createCachedStore,
createGitFs,
createRetryStore,
} from 'git-fs-s3';
import { S3ObjectStore } from 'git-fs-s3/s3';
const store = createCachedStore(
createRetryStore(
new S3ObjectStore({
client,
bucket: 'my-git-repos',
})
),
{
maxBytes: 256 * 1024 * 1024,
ttlMs: 3_600_000,
cacheMisses: true,
cacheLists: true,
copy: false, // isomorphic-git never mutates what it reads
}
);
const fs = createGitFs(store, { looseObjectHints: true });
// Once per repository, after confirming it exists (e.g. its database row):
fs.registerGitdir('repos/alice/blog/git', {
absentFiles: ['packed-refs', 'shallow'],
});
  1. S3ObjectStore at the bottom. The real network call.

  2. createRetryStore wraps it directly. Retry belongs as close to the network as possible: retrying at this layer means every consumer above (including the cache) sees a request that either eventually succeeds or fails for real — never a request that failed transiently and got treated as a permanent miss.

  3. createCachedStore wraps the retry store. This is the order that matters most: if the cache were underneath retry, a transient failure during a cached-misses lookup could get memoized as “not found” for the full TTL. With retry underneath, the cache never even sees a transient failure — only a real result or a real (rare, already-retried) error.

  4. createGitFs wraps the cache. The git-aware layer (loose-object hints, registered gitdirs) sits on top so its own bookkeeping benefits from everything below it being fast and resilient.

isomorphic-git probes for a loose object before falling back to searching packs. On a fully packed repository, every one of those probes is a guaranteed miss — pure wasted round trips. looseObjectHints: true tracks, per gitdir, whether any loose object has ever been seen, so a packed repo skips the probe entirely.

The hint is per process: a process that saw “no loose objects” can’t learn that another process later wrote one. That’s why this package’s own writers never write loose objects — commits go through withPackedWrites, pushes and repacks write packs — and why you should wrap any other object-writing code the same way. A repository that only ever receives packs keeps every process’s hint correct.

The first loose-object read under a gitdir detects its hint with one bounded LIST (concurrent reads share it). Call detectLooseObjects(gitdir) to do it up front instead, e.g. alongside other warm-up work:

await fs.detectLooseObjects(gitdir); // one bounded LIST per repo

A loose write flips the hint back on instantly, so it’s impossible for a hint to go stale mid-push and hide a just-written object.

isomorphic-git stats the gitdir on every command, and probes packed-refs and shallow on most of them. In object storage the gitdir stat is a HEAD that always misses plus a LIST, and the probes are guaranteed misses for a backend that never packs refs or serves shallow clones. registerGitdir(gitdir, { absentFiles }) declares both facts, so those calls cost zero store requests:

fs.registerGitdir(gitdir, { absentFiles: ['packed-refs', 'shallow'] });

absentFiles names top-level files of that gitdir only, so a branch called refs/heads/packed-refs is unaffected. invalidate(prefix) forgets registrations under the prefix (e.g. when a repository is deleted).

Before a full-history walk (a commit log, a reachability traversal), warm every .pack/.idx file in parallel (.bitmap/.rev are skipped — isomorphic-git never reads them) instead of paying one round trip per pack as the walk discovers each one sequentially:

await fs.prefetchPacks(gitdir);

Anything that writes to the bucket around this fs — a hydrate/sync pipeline, a bulk upload, another process — must invalidate the affected state, or reads can stay stale for up to ttlMs:

fs.invalidate('repos/alice/blog');

This forwards to the underlying cached store’s own invalidate(prefix) and also clears any loose-object hints under that prefix.