perf(s3): HEAD-based existence + parallel tile upload (MAPCO-11322)

[shimoncohen]

LOGIC-3 of MAPCO-11317. Scoped S3 I/O wins.

Scope decision

"Async end-to-end" taken literally would convert the whole tile pipeline (IDataUtils/IData/ITileMerger/Process/TaskExecutor) to async — a large rewrite that collides with SVC-1 and the other PRs. This PR delivers the self-contained I/O wins and leaves full pipeline-async as a follow-up.

Changes

• HEAD-based existence — S3Client.GetTileKey (used by TileExists) did a full GetObjectAsync, downloading the entire object body just to read its .Key. Now a GetObjectMetadata (HEAD) per candidate extension (Jpeg, then Png) — an O(1) key lookup that resolves existence + the real extension with no body transfer and no bucket-index scan. A prefix ListObjectsV2 was deliberately rejected: LIST scans the bucket index, which does not scale on the billions-of-objects bucket (MAPCO-7954).
• Parallel upload — S3.InternalUpdateTiles uploaded tiles one blocking PutObject at a time. Now Parallel.ForEach, materializing tiles first so the single-threaded grid/origin projection completes before uploads fan out. (chore: update logs + add parallel.for when putting updating tiles to S3 #112 attempted this via Task.Run(...).Wait(), which isn't actually parallel; this supersedes it.)
• Removed no-op connection cap — dropped a ServicePointManager.DefaultConnectionLimit line that was added alongside the parallel upload. ServicePointManager governs the legacy HttpWebRequest stack; on net6 the AWS SDK's HttpClient/SocketsHttpHandler ignores it, so it never capped anything. Pure dead-code removal.

Deferred (follow-ups) — tracked in MAPCO-11364

• P4 — full end-to-end async of the read/write path (sync-over-async .Result).
• P5r — GetTile still probes Jpeg-then-Png (up to 2 full GETs/tile); the speculative probe was not dropped. (Only GetTileKey/TileExists moved to HEAD here.)
• P10r — download parallelism; only uploads are parallelized in this PR.
• P11 — ctor issues 25 LIST calls (FolderExists loop 0..24) to discover the zoom set → single ListObjectsV2 with Delimiter="/".
• P14 — HTTP/FS clients' own async/existence tuning (kept out to bound this PR to S3).
• P15 — upload-concurrency bound. With the no-op cap gone, the upload Parallel.ForEach has no deliberate ceiling: concurrency tracks ThreadPool sizing (blocking PutObject bodies) and socket count is unbounded (SocketsHttpHandler.MaxConnectionsPerServer default = int.MaxValue). Follow-up adds a config-driven MaxDegreeOfParallelism paired with AmazonS3Config.MaxConnectionsPerServer (the real net6 lever), value from a load test.

Existence uses HEAD (GetObjectMetadataAsync), matching parent-ticket P5 and MAPCO-7954 — no bucket-index LIST on the billions-object bucket. Trade-off: an unknown-extension miss costs up to two HEADs (Jpeg + Png); both are index-free O(1) lookups, cheaper than one prefix LIST on a large bucket. Passing the extension per source to collapse this to a single HEAD is tracked in MAPCO-11382.

Testing

Full suite: 1141 passed, 0 failed. TileExists client test drives GetObjectMetadata (HEAD) per extension and asserts Png is only probed after a Jpeg 404; the S3 UpdateTiles data-type test no longer sequences UpdateTile (uploads are unordered) but still asserts one upload per tile.

Supersedes / relates

• Supersedes chore: update logs + add parallel.for when putting updating tiles to S3 #112 (parallel S3 put).

🤖 Generated with Claude Code

[shimoncohen]

Existence path changed: LIST → HEAD.

GetTileKey/TileExists now issue GetObjectMetadata (HEAD) per candidate extension (Jpeg, then Png) instead of a prefix ListObjectsV2(MaxKeys=1). HEAD is an O(1) key lookup; a prefix LIST scans the bucket index and does not scale on the billions-of-objects bucket (MAPCO-7954). Miss detection uses HEAD 404 (IsKeyNotFound), not the GET NoSuchKey.

Trade-off: an unknown-extension miss costs up to two HEADs (Jpeg + Png) — both index-free, cheaper than one prefix LIST on a large bucket. Collapsing to a single HEAD needs the per-source extension (MAPCO-11382).

Title + description updated. Suite: 1141 passed, 0 failed.