perf(gateway): cut per-request allocations on the inference hot path

[SantiagoDePolonia]

Reduces allocations on the chat-completion hot path without changing public behavior. Measured with the existing `tests/perf` harness:

Benchmark	Before	After
Production shape (auth + audit + usage + session + rate limit)	301 allocs · 25.6 KB · 15.5 µs	228 allocs · 22.3 KB · 14.3 µs
Bare chat completion	105 allocs · 13.8 KB	85 allocs · 12.9 KB
Routed chat completion	125 allocs · 14.4 KB	99 allocs · 13.3 KB

Changes:

• Session auto-detect canonicalizes JSON segments by walking the parsed gjson tree into one buffer instead of decoding to `map[string]any` and re-marshaling. Output is pinned byte-for-byte to the previous path by a randomized oracle test, so auto session ids are unchanged; unsupported inputs fall back to the old path.
• Audit header capture redacts in a single pass; `IsCredentialHeader` folds case in a stack buffer.
• Header keys used on every request (`X-Request-Id`, `X-Ratelimit-*`, `X-Gomodel-Auth-User`) are spelled in canonical form so `Header.Get/Set` skip per-call canonicalization. Wire format is unchanged.
• Request logger uses typed `slog.LogAttrs`; `core.WithWorkflow` is idempotent so the request context is wrapped once.
• Smaller cleanups: `strings.Cut` over `Split`, single backing array in header clone, lazy unknown-field buffer, single-alloc body reader, `gjson.GetBytes` instead of a full-body copy for selector peeking.
• Perf-guard ceilings tightened to the new baselines.

User-visible impact: none intended; lower memory and GC pressure per request.

Summary by CodeRabbit

• Bug Fixes

  • Improved request ID handling across requests, logs, caching, and passthrough operations.
  • Standardized authentication and rate-limit response header names.
  • Improved recognition of audio and event-stream content types despite casing or formatting differences.
  • Enhanced protection of sensitive header and URL information in audit data.
  • Improved consistency when processing JSON session data.

• Performance

  • Reduced overhead in request snapshots, JSON inspection, session handling, and frequently used request paths.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 635b4ae0-14fd-44f0-af2b-35a04178f245

📥 Commits

Reviewing files that changed from the base of the PR and between e826249 and ff6db9c.

📒 Files selected for processing (2)

• internal/responsecache/semantic.go
• internal/responsecache/simple.go

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.

---

📝 Walkthrough

Walkthrough

The PR centralizes HTTP header names, adds JSON session canonicalization with oracle tests, and reduces allocations in request, audit, cache, server, and JSON processing paths.

Changes

Header and runtime processing updates

Layer / File(s)	Summary
Session JSON canonicalization internal/session/canonical.go, internal/session/canonical_oracle_test.go, internal/session/detect.go	Adds canonical JSON session segments, duplicate-key handling, fallback encoding, oracle tests, and fixed-size automatic session ID construction.
Canonical header definitions and propagation ext/auth.go, internal/core/context.go, internal/admin/..., internal/server/..., internal/responsecache/..., internal/auditlog/..., internal/mcpgateway/...	Adds shared request ID and rate-limit header constants and replaces hard-coded header names across request, response, logging, gateway, audit, and cache paths.
Audit header and media processing internal/auditlog/..., internal/responsecache/stream_cache.go	Uses first-segment, case-insensitive media-type checks, extracts redacted header values inline, and encodes API-key hash prefixes directly.
Allocation and repeated-work reductions internal/core/..., internal/server/..., tests/perf/hotpath_test.go	Reduces repeated context and request work, optimizes JSON and map handling, uses typed structured log attributes, and lowers benchmark allocation ceilings.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to ff6db

The PR reduces per-request allocations while preserving request, session, header, logging, and cache behavior; no actionable merge-blocking risk remains beyond normal checks and review.

Poem

A rabbit sorts the headers bright
Canonical names now fit just right
JSON keys hop into a row
Fewer spare allocations flow
Tests watch each segment’s trail
Clean paths carry every detail

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 46.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 50 functions across 29 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary change: reducing per-request allocations on the gateway inference hot path.
Description check	✅ Passed	The description explains the purpose, key implementation changes, measured performance improvements, compatibility intent, and test coverage. It is complete and directly aligned with the pull request …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description explains the purpose, key implementation changes, measured performance improvements, compatibility intent, and test coverage. It is complete and directly aligned with the pull request objectives, although it omits the template's optional headings.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/faster

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 85.04673% with 32 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/server/http.go	0.00%	11 Missing ⚠️
internal/session/canonical.go	91.30%	10 Missing ⚠️
internal/server/ratelimit_support.go	50.00%	6 Missing ⚠️
internal/core/context.go	0.00%	2 Missing ⚠️
internal/core/credential_headers.go	90.00%	1 Missing ⚠️
internal/server/passthrough_support.go	66.66%	1 Missing ⚠️
internal/server/request_snapshot.go	85.71%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

Safe to merge based on the exercised request-body ownership and session canonicalization behaviors.

No final defects remain after focused executable checks contradicted the two investigated failure paths.

Files Needing Attention: No files need corrective changes. The most thoroughly exercised files were internal/core/semantic.go and internal/session/canonical.go.

T-Rex Logs

What T-Rex did

• Ran a focused Go ownership regression test through the deriveSnapshotSelectorHintsGJSON path using a 12 MB streaming JSON body and compared the result to the pre-change implementation; the after-change run reported getbytes_aliases=(model:false provider:false) and mutation_stable=true, while the pre-change run passed with independent selector strings.
• Ran a review-authored differential test comparing the optimized canonicalization walker and derived automatic session IDs against the former reference implementation, covering nested objects, duplicate keys, escaping and control cases, surrogate handling, number formats, malformed segments, and stable versus distinct session-ID pairs; the optimized walker produced byte-for-byte matching canonical output and identical session grouping, and the existing randomized oracle, automatic-session regressions, and full internal/session test suite also passed.
• After-change output reports getbytes_aliases=(model:false provider:false) and mutation_stable=true, while the pre-change output reports prechange_clone_aliases=(model:false provider:false) and prechange_mutation_stable=true; the same focused test passed in both cases.
• Source location assessment notes were captured for relevant internal/session and canonical.go references; the review-authored test was uploaded verbatim and its SHA-256 equality to the uploaded source was recorded, with the executable harness source and all observed outputs saved in artifacts.

_{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "perf(gateway): cut per-request allocatio..." | Re-trigger Greptile}