Conversation
Two deploy steps each walked the whole `$web` container. `Set public URL
metadata on blobs` spawned one Azure CLI process per blob, 20 at a time; at
~0.66s of cold start each and ~86k blobs that step alone took 47 of the 70
minutes a push to main spent in CI, and it re-stamped every blob regardless
of what changed. `Set MIME types for all file types` then made 19 more
passes, one `az storage blob update-batch` per file extension.
Fold both into a single listing pass in scripts/set-blob-metadata.mjs.
`listBlobsFlat({ includeMetadata: true })` returns each blob's metadata and
properties inline, so deciding what needs writing costs no extra round trips
and only missing or stale blobs get a PUT, over a bounded pool rather than a
process-per-blob fan-out. Pass --force to rewrite unconditionally, or
--dry-run to list and plan without writing.
Blob names are percent-encoded per path segment before going into the
metadata value. The old loop interpolated them raw, which is not safe: the
site ships names carrying a Cyrillic homoglyph and a curly quote, and both
throw at socket-write time because `x-ms-meta-*` headers are latin-1 on the
wire. Those writes failed silently before, since each `az` call was
backgrounded and its exit code never checked. Unencoded spaces also produced
invalid URLs. Failures are now reported per blob and fail the step.
Content types are set with the other content settings carried through, since
setHTTPHeaders replaces every header it accepts rather than merging.
Also:
- Merge the build and deploy jobs. Handing the 1.5GB build/ directory
between two runners cost ~4.5min in artifact upload plus download for no
benefit now that both halves run in one place. Deploy steps are gated on
the event not being a pull_request, so PR runs build and stop as before.
The build-output artifact upload is kept, now PR-only.
- Drop the Azure CLI install and every use of sudo. The deploy now runs on
the self-hosted gh-runner-large, where sudo cannot be assumed. azcopy is
unpacked into $RUNNER_TEMP and added to $GITHUB_PATH, and the SAS token
azcopy uploads with is minted by scripts/mint-container-sas.mjs using the
storage SDK already in node_modules.
- Fix the compiler cache key. It was keyed on package-lock.json alone, and
actions/cache only saves on a key miss, so after the first run it hit that
key forever and never saved a refreshed cache. Appending github.run_id
makes every run miss-then-save while restore-keys pull the newest prior
cache.
- Drop the .docusaurus cache. Its key hashed docs/**, so any content change
missed it, and the restore-keys fallback only ever restored a stale
directory that Docusaurus regenerates anyway.
- Enable ssgWorkerThreads. Measured on a scoped pingcastle build (1595
pages, two rounds each): 59.7s -> 49.8s mean, with peak heap dropping from
680MB to 313MB since rendering moves off the main thread.
set-blob-metadata.mjs gains --endpoint (or AZURE_BLOB_ENDPOINT) so it can be pointed at the Azurite emulator. The integration test seeds a container with the blob names that broke the old az loop (Cyrillic homoglyph, space), wrong and unmapped content types, and stale/current metadata, then checks dry-run writes nothing, a real run fixes exactly the stale blobs while preserving cache-control, a second run is a no-op, and a SAS minted by mint-container-sas.mjs can list/write/delete while a list-only SAS is refused. Skips cleanly when no emulator is listening.
The test-scripts job ran npm run test:scripts on a bare checkout. That worked while every test was a shell script, but the blob metadata tests import @azure/storage-blob.
perf(ci): cut deploy time from ~55min to a few minutes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.