Serving clones and pushes

This wires all three request types from the overview into one route, the shape you’d drop into Next.js, Remix, Hono, a Cloudflare Worker, or any framework with a Fetch-API route handler.

1. Advertise refs. Auth/authorization is entirely the caller’s job — these handlers assume it already happened.

   import { handleInfoRefs } from 'git-fs-s3/http';
   
   export async function infoRefs(repo: Repo, service: 'git-upload-pack' | 'git-receive-pack') {
     const { status, headers, body } = await handleInfoRefs(repo, { service });
     return new Response(body, { status, headers });
   }

2. Serve a clone or fetch.

   import { handleUploadPack } from 'git-fs-s3/http';
   
   export async function uploadPack(repo: Repo, request: Request) {
     const body = new Uint8Array(await request.arrayBuffer());
     const { status, headers, body: respBody } = await handleUploadPack(repo, body);
     return new Response(respBody, { status, headers });
   }

3. Accept a push.

   import { applyReceivePack, parseReceivePackBody, receivePackResponse } from 'git-fs-s3/http';
   
   export async function receivePack(repo: Repo, request: Request) {
     const body = new Uint8Array(await request.arrayBuffer());
     const parsed = parseReceivePackBody(body);
     const { results, stalePackPaths, unpackError } = await applyReceivePack(
       repo,
       parsed,
       { repack: { threshold: 4 } } // or false to never auto-consolidate
     );
   
     // stalePackPaths were removed locally by the repack — delete them from
     // any secondary storage this repo also lives in, same as you'd
     // invalidate a cache. See "Repacking" for the full story.
     for (const path of stalePackPaths) {
       await deleteFromSecondaryStorage(repo, path);
     }
   
     const { status, headers, body: respBody } = receivePackResponse(results, {
       capabilities: parsed.capabilities,
       unpackError,
     });
     return new Response(respBody, { status, headers });
   }

Wiring loose-object detection and pack prefetch into upload-pack

handleUploadPack’s second argument accepts a beforeWalk hook — fire it right before the reachability walk. Two independent optimizations belong here, and it’s easy to wire only the first and miss the second:

1. Loose-object detection, so a fully packed repo’s clone doesn’t pay a doomed loose-object probe per object (see the production stack guide).
2. Pack prefetch, so a repo sitting on more than one pack (the normal state between repacks — the single-pack fast path below only covers an already-consolidated repo) doesn’t pay one R2/S3 round trip per pack, serially, as the walk’s single-threaded early stages happen to discover they need it. The walk itself is a sequential commit chain — you only learn the next object to fetch after reading the current one — so without this, pack downloads end up serialized behind that chain instead of running as one parallel batch upfront. This is the same shape of fix getCommitLog/getLastCommitsForTree/getFileHistory (/ops) already apply via OpsHooks.prefetch — beforeWalk here is the equivalent seam for /http.

await handleUploadPack(
  repo,
  body,
  {
    beforeWalk: () =>
      Promise.all([
        (repo.fs as GitFs).detectLooseObjects(repo.gitdir),
        (repo.fs as GitFs).prefetchPacks(repo.gitdir),
      ]).then(() => undefined),
  },
  hooks
);

Don't wire only one of these

A real production gap: a consumer of this package wired detectLooseObjects into beforeWalk but not prefetchPacks — the code looked correct (the reachability walk itself is fully, correctly parallelized internally), so this was invisible in review and only surfaced under live profiling against a real bucket. Measured on a 2-pack repo: the reachability walk alone dropped from 744.7ms to 20.1ms once prefetchPacks was added.

Tip

The single-pack fast path (a fresh clone of an already-consolidated repo) skips the reachability walk entirely, so it never reaches beforeWalk — you don’t need to special-case that. It only applies when the repository has exactly one pack, no loose objects, and that pack’s index lists every wanted object; anything else takes the normal walk.

What upload-pack will serve

Every want must be a ref tip or an ancestor of one; anything else (say, a commit a force-push removed, still sitting in an old pack) is answered with ERR upload-pack: not our ref <oid>. Object ids that aren’t 40 lowercase hex digits are a protocol error. The pack is side-band-64k framed only when the client asked for side-band-64k (git and isomorphic-git always do); a client that didn’t gets the raw pack after NAK.

How the clone/fetch pack is built

After the walk, upload-pack builds the response pack by reusing stored entries: each wanted object’s compressed bytes are copied straight out of the pack that holds it, deltas included. OFS_DELTA entries are rewritten as REF_DELTA (base named by oid, since offsets don’t survive the move), and every delta base is written before the deltas that use it. Nothing is inflated or re-deflated, so the response is as compact as the stored packs. If any wanted object isn’t packed (loose objects), it falls back to isomorphic-git’s packObjects, which deflates every object without deltas.

Measured on a real 3,368-object repository: building the pack went from ~1.1 s to ~8 ms, and the clone download from 10 MB to 2.6 MB.

For a fetch with have lines, the server walks only the client’s commit ancestry (commit objects, no trees), finds the boundary commits the new history builds on, and treats the full content of those boundary commits as already present. The wants walk then stops at anything in that set — so a fetch costs the new objects plus a few trees, not a walk of every tree the client already has. (Objects only older commits share may be re-sent; that’s git’s own boundary approximation, and never leaves anything missing.) Blobs aren’t read during the upload-pack walk (their tree entries already say what they are); receive-pack and repack still read them, because there the walk is the existence check.

collectReachableOids bounds in-flight object reads with one semaphore per walk (MAX_CONCURRENT_READS = 16): a permit covers the read itself and is released before the walk recurses into that object’s children, so a wide tree can’t fan out into thousands of simultaneous requests and a parent never blocks its own children.

Share one isomorphic-git cache per repository

Pass the same cache object on every repo for a given repository, across requests. isomorphic-git keeps parsed pack indexes (and its one-time checksum of each pack) in it; without one, every object read re-parses every .idx and re-hashes every .pack. On the repository above that was the difference between a 2.2 s and a 0.1 s clone walk.

Routing

None of these handlers care how you route to them — they only need repo (a { fs, gitdir, cache }, however you resolve it per-request) and the raw Request. A typical catch-all route parses the URL (.../<owner>/<repo>.git/info/refs, .../git-upload-pack, .../git-receive-pack), resolves repo for that owner/repo pair, checks auth, and dispatches to the matching handler above.