Skip to content

fix(db): inline RDS CA bundle to bypass Turbopack ignoring outputFileTracingIncludes - #2772

Merged
Marfuen merged 1 commit into
mainfrom
mariano/fix-rds-ca-turbopack
May 6, 2026
Merged

Marfuen merged 1 commit into
mainfrom
mariano/fix-rds-ca-turbopack

Conversation

@Marfuen

@Marfuen Marfuen commented May 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Root cause: All Vercel deployments use Turbopack (`bundler: "turbopack"` in deployment metadata). `next/dist/build/index.js` line ~1537 gates `collectBuildTraces` on `bundler !== Bundler.Turbopack`, so `outputFileTracingIncludes` is silently no-op'd under Turbopack. The cert from PR feat: verified-TLS to RDS from every runtime #2761 never landed at `/var/task/packages/db/certs/rds-global-bundle.pem` for App Router page bundles — every cold start logged `Warning: Ignoring extra certs from … load failed: error:80000002:system library`. Connections still 200'd because Node's default trust store happens to verify the AWS RDS Proxy chain, but the bundle was effectively unused.
  • Fix: Inline the PEM as a TypeScript string constant (`RDS_CA_BUNDLE`) and pass it directly to the Postgres adapter via `ssl.ca`. Bundler-agnostic; no env var, no tracing hacks. Generated by `packages/db/scripts/generate-ca-bundle-ts.mjs` from the existing `packages/db/certs/rds-global-bundle.pem`.
  • Files touched:
    • `packages/db/src/rds-ca-bundle.ts` (generated, committed)
    • `packages/db/src/ssl-config.ts` (uses inline bundle, drops the env-var fallback + throw)
    • `apps/app/prisma/`, `apps/portal/prisma/`, `apps/framework-editor/prisma/` — local rds-ca-bundle.ts + `ssl.ca` wiring (duplicated rather than imported from `@trycompai/db` because the Trigger.dev indexer pins to the npm version)
    • `apps/app/next.config.ts`, `apps/portal/next.config.ts` — drop the now-redundant `outputFileTracingIncludes`
    • `@trycompai/db`: 2.1.1 → 2.2.0
    • Updated deploy checklist
  • apps/api unchanged — the Docker runtime already sets `NODE_EXTRA_CA_CERTS` at the OS level.

Investigation receipts

  • Read `next/dist/build/index.js` line 1537 directly to confirm the Turbopack gate.
  • Verified live picomatch + glob behavior via `node -e ...` against the actual matchers Next.js uses (`/**/*` does match `/[orgId]/policies`, the relative cert path does resolve). The pattern is correct — Turbopack just never runs the include logic.
  • Pulled deployment metadata via Vercel MCP for prj_f5xEePfjwPO3rPhxDrDmVN2BSlWN — every recent deploy lists `bundler: "turbopack"`.
  • Confirmed the warning fires during the build itself (build log line 1778092756529), implying the env var was applied with a path that doesn't exist at build time either.

Post-merge action

Unset `NODE_EXTRA_CA_CERTS` on the Vercel team's shared variables once this rolls out — the path no longer exists on the function (cert is in-bundle now), and leaving the env var set will keep producing the cold-start warning.

Test plan

  • Verify a preview deploy of this branch logs no `Ignoring extra certs` warning at cold start
  • Verify `/api/health` succeeds on the preview deploy
  • Verify a page route (e.g. `/[orgId]/policies`) renders successfully on the preview deploy
  • After merge to release: confirm prod cold starts no longer emit the warning
  • Unset `NODE_EXTRA_CA_CERTS` shared env var on Vercel team

🤖 Generated with Claude Code


Summary by cubic

Inline the AWS RDS CA bundle as RDS_CA_BUNDLE and pass it via Prisma ssl.ca so verified Postgres TLS works under Turbopack and the cold‑start warning disappears. Removes the need for NODE_EXTRA_CA_CERTS and any outputFileTracingIncludes.

  • Bug Fixes

    • Root cause: Turbopack ignores outputFileTracingIncludes, so the PEM never shipped and cold starts logged “Ignoring extra certs…”.
    • Fix: inline bundle (RDS_CA_BUNDLE) used in ssl.ca; removed outputFileTracingIncludes; added packages/db/scripts/generate-ca-bundle-ts.mjs; duplicated rds-ca-bundle.ts in each app’s prisma/ for Trigger.dev.
    • Bumped @trycompai/db to 2.2.0.
  • Migration

    • Vercel: unset the shared NODE_EXTRA_CA_CERTS for apps/app and apps/portal; no env vars or tracing config are needed now.
    • No changes needed for Trigger.dev or apps/api; existing setups continue to work.

Written for commit d4f3371. Summary will update on new commits.

…TracingIncludes

Vercel deployments of apps/app and apps/portal use Turbopack (deployment
metadata `bundler: "turbopack"`). Next.js's `outputFileTracingIncludes` is
silently no-op'd under Turbopack — `next/dist/build/index.js` line ~1537
gates `collectBuildTraces` on `bundler !== Bundler.Turbopack`. So the
file-based approach from PR #2761 never landed the cert at
`/var/task/packages/db/certs/rds-global-bundle.pem` for App Router page
function bundles, producing this warning at every cold start:

  Warning: Ignoring extra certs from
  `/var/task/packages/db/certs/rds-global-bundle.pem`, load failed:
  error:80000002:system library

Connections still returned 200 because Node's default trust store happens
to verify the AWS RDS Proxy chain — but the bundle was never actually
loaded, defeating the verified-TLS work.

Fix: inline the PEM as a TypeScript string constant (RDS_CA_BUNDLE) and
pass it directly to the Postgres adapter via `ssl.ca`. Bundler-agnostic,
no env var needed, no tracing hacks. Generated by
`packages/db/scripts/generate-ca-bundle-ts.mjs` from the existing
`packages/db/certs/rds-global-bundle.pem` source.

Changes:
- packages/db/src/rds-ca-bundle.ts (generated, committed) — exports the
  PEM as a string constant.
- packages/db/src/ssl-config.ts — uses the inline bundle instead of
  reading NODE_EXTRA_CA_CERTS. Drops the throwing fallback (always have
  the cert now). Adds `ca` to the SslConfig type.
- apps/app/prisma/, apps/portal/prisma/, apps/framework-editor/prisma/ —
  inlined ca-bundle TS file + client uses `ssl.ca` directly. These
  duplicate the constant rather than importing from `@trycompai/db`
  because the Trigger.dev indexer pins to the npm-published version,
  which lags behind workspace source.
- apps/app/next.config.ts, apps/portal/next.config.ts — drop the now-
  redundant `outputFileTracingIncludes` for the cert.
- @trycompai/db: 2.1.1 → 2.2.0.
- Deploy checklist updated: `NODE_EXTRA_CA_CERTS` is no longer required
  on Vercel (and should be unset to silence the cold-start warning).

apps/api/prisma/client.ts is unchanged — the Docker runtime sets
`NODE_EXTRA_CA_CERTS` at the OS level and that path works fine.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
app Ready Ready Preview, Comment May 6, 2026 8:52pm
comp-framework-editor Ready Ready Preview, Comment May 6, 2026 8:52pm
portal Ready Ready Preview, Comment May 6, 2026 8:52pm

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 13 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

@Marfuen
Marfuen merged commit 2694ece into main May 6, 2026
11 checks passed
@Marfuen
Marfuen deleted the mariano/fix-rds-ca-turbopack branch May 6, 2026 20:53
@claudfuen

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 3.44.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Marfuen added a commit that referenced this pull request May 6, 2026
)

URGENT: production runtime fix. Staging is hitting:

  Error [PrismaClientKnownRequestError]
  Invalid `prisma.member.findFirst()` invocation
  Error opening a TLS connection: unable to get local issuer certificate
  code: 'P1011', driverAdapterError: TlsConnectionError

Cause: PR #2772 set `ssl.ca = RDS_CA_BUNDLE` in the prisma adapter, which
*replaces* Node's trust store rather than augmenting it. Our bundle only
contains the 108 RDS-specific regional self-signed CAs — it does NOT
contain Amazon Root CA 1, which is where AWS RDS Proxy chains terminate
(and which lives in Node's default Mozilla bundle). So the chain failed
to validate at runtime under the strict-TLS branch.

Why apps/app and apps/portal didn't trip this in earlier checks:
- The /auth route returned 200 because that codepath doesn't query the
  DB; it talks to apps/api over HTTP, and apps/api uses a different
  prisma client (Docker, NODE_EXTRA_CA_CERTS at OS level).
- DB-touching SSR routes (e.g., /[orgId]/overview) are exactly what the
  reported staging failure exercises.

Fix: drop the `ca:` field. Node's default trust store includes Amazon
Root CA 1, which is sufficient for chain validation against RDS Proxy.
Hostname check is still skipped (NLB topology — chain check still
rejects forged or wrong-CA certs). PRISMA_ALLOW_INSECURE_TLS=1 remains
the explicit insecure opt-out — the original Cubic finding fix is
preserved.

Files:
- packages/db/src/ssl-config.ts: drop RDS_CA_BUNDLE import + usage
- packages/db/src/client.test.ts: rewrite tests for new behavior (6 pass)
- apps/{app,portal,framework-editor}/prisma/client.ts: drop the ca: branch
- Delete: packages/db/{certs/rds-global-bundle.pem,src/rds-ca-bundle.ts,
  scripts/generate-ca-bundle-ts.mjs} and the inlined rds-ca-bundle.ts
  copies in apps/{app,portal,framework-editor}/prisma/ (~660KB removed)
- packages/db: 2.2.0 → 2.3.0 (also drops `certs` from `files` array)
- apps/api/prisma/client.ts: unchanged — Docker still uses
  NODE_EXTRA_CA_CERTS at OS level and that path is fine.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claudfuen pushed a commit that referenced this pull request May 6, 2026
## [3.44.2](v3.44.1...v3.44.2) (2026-05-06)

### Bug Fixes

* **db:** drop inlined RDS CA bundle, use Node default trust store ([#2775](#2775)) ([cd5046c](cd5046c)), closes [#2772](#2772)

This branch was successfully deployed

3 active deployments
Preview – app — d4f33712 Deployed May 6, 2026 by vercel[bot]
Preview – portal — d4f33712 Deployed May 6, 2026 by vercel[bot]
Preview – comp-framework-editor — d4f33712 Deployed May 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants