This repository was archived by the owner on Jul 4, 2026. It is now read-only.
fix(snapshot): bump FetchSnapshotConfig read cap from 1 MiB to 64 MiB - #3
Merged
Merged
Conversation
The previous cap was set with the comment "config blob is tiny", but
the SnapshotConfig embeds a per-file SparseMap for every file in the
snapshot. A 4 GiB Windows VM whose guest has many small live
allocations (e.g. an Electron app with a Firebase WebSocket signed in)
produces sparse maps in the high-hundred-KB range per fragmented file,
and the config blob easily exceeds 1 MiB once memory-ranges and
overlay.qcow2 are both accounted for. Observed live: 580 KB +
612 KB + base fields = 1.37 MB total config blob.
Wake-side symptom was
pull hibernation snapshot ...:
stream snapshot:
fetch snapshot config:
parse snapshot config: unexpected end of JSON input
— `io.ReadAll(io.LimitReader(body, 1<<20))` truncated the JSON exactly
at 1 MiB, leaving an unterminated structure.
Bump the cap to 64 MiB (still defends against pathological / malicious
manifests) and add a descriptor.Size pre-check so we reject oversize
blobs before reading. Add a streaming overflow guard for the case
where the descriptor lies about Size.
Tests:
- TestFetchSnapshotConfigOverOneMiB: builds a real ~1.4 MB
SnapshotConfig with two fragmented sparse maps and asserts parse
succeeds (would have failed under the old 1 MiB cap).
- TestFetchSnapshotConfigRejectsOversizeDescriptor: rejects a
descriptor advertising > 64 MiB up front.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tonicmuroq
force-pushed
the
fix/snapshot-config-limit
branch
from
May 6, 2026 02:17
dc0745d to
0315e8a
Compare
tonicmuroq
added a commit
to cocoonstack/vk-cocoon
that referenced
this pull request
May 6, 2026
Picks up tombstack/epoch#3, which raises the snapshot-config blob read cap from 1 MiB to 64 MiB. Wake-side parse failed with "unexpected end of JSON input" when the SnapshotConfig — including per-file SparseMap fields for memory-ranges and overlay.qcow2 — grew past 1 MiB, which a 4 GiB Windows guest with a Firebase-active agent hits routinely. No vk-cocoon source changes; just the go.mod / go.sum bump and a rebuild. Existing tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tonicmuroq
added a commit
to cocoonstack/vk-cocoon
that referenced
this pull request
May 6, 2026
Picks up tombstack/epoch#3, which raises the snapshot-config blob read cap from 1 MiB to 64 MiB. Wake-side parse failed with "unexpected end of JSON input" when the SnapshotConfig — including per-file SparseMap fields for memory-ranges and overlay.qcow2 — grew past 1 MiB, which a 4 GiB Windows guest with a Firebase-active agent hits routinely. No vk-cocoon source changes; just the go.mod / go.sum bump and a rebuild. Existing tests pass.
tonicmuroq
added a commit
to cocoonstack/vk-cocoon
that referenced
this pull request
May 6, 2026
Picks up tombstack/epoch#3, which raises the snapshot-config blob read cap from 1 MiB to 64 MiB. Wake-side parse failed with "unexpected end of JSON input" when the SnapshotConfig — including per-file SparseMap fields for memory-ranges and overlay.qcow2 — grew past 1 MiB, which a 4 GiB Windows guest with a Firebase-active agent hits routinely. No vk-cocoon source changes; just the go.mod / go.sum bump and a rebuild. Existing tests pass.
CMGS
pushed a commit
to cocoonstack/vk-cocoon
that referenced
this pull request
May 6, 2026
Picks up tombstack/epoch#3, which raises the snapshot-config blob read cap from 1 MiB to 64 MiB. Wake-side parse failed with "unexpected end of JSON input" when the SnapshotConfig — including per-file SparseMap fields for memory-ranges and overlay.qcow2 — grew past 1 MiB, which a 4 GiB Windows guest with a Firebase-active agent hits routinely. No vk-cocoon source changes; just the go.mod / go.sum bump and a rebuild. Existing tests pass.
CMGS
pushed a commit
to cocoonstack/vk-cocoon
that referenced
this pull request
May 6, 2026
Picks up tombstack/epoch#3, which raises the snapshot-config blob read cap from 1 MiB to 64 MiB. Wake-side parse failed with "unexpected end of JSON input" when the SnapshotConfig — including per-file SparseMap fields for memory-ranges and overlay.qcow2 — grew past 1 MiB, which a 4 GiB Windows guest with a Firebase-active agent hits routinely. No vk-cocoon source changes; just the go.mod / go.sum bump and a rebuild. Existing tests pass.
CMGS
added a commit
that referenced
this pull request
Jun 28, 2026
Pin the merged cocoon-common main (cross-node migration API surface, PR #3) via pseudo-version until v0.2.2 is tagged.
CMGS
added a commit
that referenced
this pull request
Jun 28, 2026
Pin the merged cocoon-common main (cross-node migration API surface, PR #3) via pseudo-version until v0.2.2 is tagged.
CMGS
added a commit
that referenced
this pull request
Jun 28, 2026
* feat(server): redirect blob GETs to presigned object-store URLs v2GetBlob proxies every blob byte through the epoch process: it reads from the object store and io.Copy's the body back to the client, paying TLS on both sides for multi-GiB VM disk/memory blobs. A single stream tops out at ~20 MB/s even with 3 CPUs, so a 10.5 GB snapshot pull takes ~9 min, and concurrent transfers saturate the pod. When EPOCH_BLOB_REDIRECT is set, v2GetBlob now responds with a 307 to a presigned object-store URL (minio-go PresignedGetObject), so blob bytes flow client<->storage directly and epoch leaves the data path entirely. GCS honors the SigV4-signed URL and serves Range requests natively. Measured on a same-region GCE VM pulling the 5.7 GB memory-ranges blob: path single-stream 10.5 GB pull proxy (1 CPU, before) 3.4 MB/s ~51 min proxy (3 CPU) 20 MB/s ~9 min presigned direct-to-GCS 275 MB/s ~38 s ~13x over the proxy with zero client changes — any redirect-following OCI client (registryclient/vk-cocoon, oras, crane, docker) benefits transparently, and CopyBlobExact still verifies the digest end to end. Intra-blob Range parallelism (measured 1.28 GB/s at 8 connections) would need a client-side downloader and is out of scope here. The flag defaults off. On any server-side failure (existence check or presign) the handler falls back to streaming. Note there is no fallback once the 307 is sent: every client hitting /v2/blobs must have egress to the object-store host when the flag is enabled. * chore(deploy): enable blob redirect and right-size epoch-server Turn on EPOCH_BLOB_REDIRECT so pulls bypass the proxy. Bump resources from 1 CPU / 512Mi to 3 CPU / 4Gi: pushes still stream through epoch (this PR only redirects GETs) and the redirect fallback path also proxies, so the pod still needs headroom for multi-GiB transfers. * chore(deps): bump cocoon-common to v0.2.2 (merged #3) Pin the merged cocoon-common main (cross-node migration API surface, PR #3) via pseudo-version until v0.2.2 is tagged. * fix(server): /code review hardening of blob redirect - clamp EPOCH_BLOB_REDIRECT_TTL to [1s, 7d]: an out-of-range value fails every presign and silently degrades to proxying, so fall back to the default below 1s (the minio presign floor) and clamp above 7d - read the redirect env via commonk8s.EnvBool / EnvDuration, dropping the hand-rolled strconv/ParseDuration to match the package convention - mark the blobRedirectTTL config field; note why redirectBlob HEADs before presigning; document the EPOCH_BLOB_REDIRECT_TTL knob --------- Co-authored-by: CMGS <ilskdw@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2.
Summary
pull.go:159reads the snapshot config blob viaio.ReadAll(io.LimitReader(body, 1<<20))with the comment "config blob is tiny", but the SnapshotConfig embeds per-file SparseMap fields. A 4 GiB Windows guest running an Electron app with a live Firebase WebSocket signed in produces sparse maps in the high-hundred-KB range per fragmented file, and the config blob easily exceeds 1 MiB oncememory-rangesandoverlay.qcow2are both accounted for.Observed live:
files["memory-ranges"].sparseMapfiles["overlay.qcow2"].sparseMapSymptom on the wake side:
io.LimitReaderreturns EOF after 1,048,576 bytes;io.ReadAllswallows the EOF; the caller hands the truncated buffer tojson.Unmarshal, which fails on the unterminated structure.Fix
desc.Sizeso we reject oversize blobs before any byte is read.Size.The reader path is the only thing that changes; pushers and on-disk format are untouched.
Relationship to cocoonstack/cocoon#23
Independent of #23. #23 addressed the writer-side tar PAX cap (a single file's sparse map > 1 MiB couldn't be encoded). This is the reader-side config cap (aggregate sparse maps + config metadata > 1 MiB couldn't be parsed). Both bugs share the same triggering condition (live Firebase agent → fragmented memory) but live in different layers.
Test plan
TestFetchSnapshotConfigOverOneMiB— builds a real ~1.4 MB SnapshotConfig with two fragmented sparse maps and asserts parse succeeds. Would have failed under the old 1 MiB cap with "unexpected end of JSON input".TestFetchSnapshotConfigRejectsOversizeDescriptor— descriptor advertising > 64 MiB is rejected before any read.go test ./snapshot/— all existing tests still pass alongside the new ones.🤖 Generated with Claude Code