Skip to content

feat: carry the landing page and HubSpot visitor cookie from embedded forms - #204

Merged
josue-commits merged 16 commits into
developfrom
feat/embed-hubspot-visit-context
Sep 26, 2026
Merged

josue-commits merged 16 commits into
developfrom
feat/embed-hubspot-visit-context

Conversation

@josue-commits

Copy link
Copy Markdown
Contributor

Closes #199.

Stacked on #203 (spam protection). Do not merge before #203. This PR targets feat/spam-protection-captcha, so the diff shows only this work. Once #203 is in develop, retarget this PR to develop. It owns DB migration 0023.

An embedded form reached HubSpot without the visitor's tracking cookie (hubspotutk) and without the landing URL, so the contact lost its original source, page views and campaign. Embedded forms also lost the landing's UTMs, because the iframe only sees its own src. Only the host page can read those values, so embed.js now answers a small handshake from the form.

What it adds

  • Handshake. At mount, and on every partial and complete submit, the form asks its parent for the visit with dapta-forms:context-request and a nonce. embed.js answers with the values read at that moment: pageUri, pageName, pageId (HubSpot CMS pages) and hutk.
    • Who gets an answer: only frames marked data-dapta-forms, and only when the message origin matches the frame's src origin and the origin embed.js was loaded from. The reply goes to that exact targetOrigin. The frame accepts an answer only from its parent, carrying its own nonce.
    • Opting out: the HubSpot opt-outs (__hs_opt_out, __hs_do_not_track) drop the cookie. data-dapta-forms-context="off" on the iframe turns the whole handshake off. Nothing writes cookies.
    • Cost: the form waits at most 500 ms when the host has answered before, and 150 ms otherwise, for example with a bare iframe. The visit is resolved once, in parallel with the spam check, so the submit is still sent once. A late answer is cached for the next submit. Without an answer, pageUri falls back to the referrer.
  • Storage. A new nullable column, submission.visit (jsonb in Postgres, text in SQLite), with an additive migration 0023 in both dialects.
    • Upserts use COALESCE, so a write without a visit never erases one.
    • Parsing is tolerant: a malformed field is dropped, never refused.
      • hutk: 32 hex characters.
      • pageUri: absolute http(s), printable ASCII, at most 2048 characters.
      • pageId: digits only.
      • Characters Postgres refuses (\u0000, lone surrogates) never reach it: a page URL carrying one is dropped, a title or UTM value is cleaned, and a UTM key carrying one is skipped.
  • Landing UTMs merge into data.utm, all or nothing. UTMs already on the iframe src win.
  • HubSpot.
    • The mirror form submission ("Record a form submission in HubSpot" on) now sends context: { hutk, pageUri, pageName, pageId }, leaving out empty keys.
    • On a 400 it retries once with the old { pageName }. Nothing is retried after a successful post.
    • The contact note gets a "Page" line.
    • A cookie from another portal logs a warning.
  • Webhooks get a top-level visit: { pageUri, pageName, embedded, hutk? }. It is additive, and the ping example includes it.
  • Dashboard.
    • The response panel shows a "Page" row, with the title and the host. It is a link only for http(s) URLs.
    • A "HubSpot cookie received" chip shows when the cookie arrived.
    • CSV export adds a "Page URL" column at the end.
    • The admin API returns visit without the cookie, with a hubspotCookie boolean instead.
    • The cookie is never shown or logged. The webhook delivery history masks it, including inside what a receiver answers.
  • Minimization. The cookie is stored only when a destination can use it. It is snapshotted per destination and phase: the HubSpot mirror only posts on complete.
  • Direct links. When the form has its own HubSpot tracking ID, it reads its own hubspotutk. pageUri is the form URL without non-UTM params, because the rest are prefilled answers.
  • With spam protection on, partials still store the visit but are not delivered. The verified complete carries the merged visit.

Deploy notes

  • Migration 0023 is additive and runs on deploy.
  • Snippets already pasted pick this up on their next page load, because embed.js is served without cache. Bare iframes (no embed.js) send only the referrer origin.
  • Outbox rows queued before the deploy deliver exactly as before.
  • embed.js answers the handshake only for frames on the host it was loaded from, which is always the case with the editor snippet. A copy of the script served from another host keeps resizing and redirecting, but sends no visit.
  • For attribution in HubSpot, a customer needs three things:
    • the HubSpot connection with the forms and form-submissions-write scopes, and "Record a form submission in HubSpot" on;
    • the same portal in the connection and in the landing's tracking code;
    • the full snippet: the iframe plus embed.js.

Before production

  • Verify with a HubSpot test portal on a page that runs the tracking code (steps in feat(embed): pass the HubSpot visit (hutk, page URL, landing UTMs) from embedded forms #199).
    • Check the page views on the contact.
    • Check that the original source does not stay "Offline sources" when the contact is created milliseconds before the mirror submission. If it does, the mirror has to go first, which touches the retry rules.
  • Public docs (embed, tracking and privacy, HubSpot, webhook payload) are a follow-up in the docs repo.

Notes for review

  • COALESCE works on the whole object, as the issue specifies. A poorer visit, such as a referrer-only one after an iframe reload, can therefore replace a richer one. It can be a follow-up if it matters.
  • The issue's Legacy and Protocol sections disagree about a stale embed.js. The code follows Protocol, which falls back to the referrer.
  • Found on the base and not changed here: in Playwright WebKit, the builder live preview never leaves "waiting" (on develop too), so one v17 test fails in WebKit.

Testing

  • pnpm typecheck and pnpm lint pass. pnpm test: 2,680 passed.
  • @quill/db on Postgres: 340/340.
  • e2e v18-embed-hubspot-context: 16/16 in Chromium and WebKit. It covers:
    • a landing on another origin, with a fake hubspotutk and UTMs;
    • a bare iframe, off and opt-out;
    • spam protection on.
  • Unit specs cover:
    • host-visit, and embed.js in a vm;
    • the UTM merge;
    • tolerant parsing, including values Postgres refuses;
    • the HubSpot mirror context and its 400 retry;
    • the webhook envelope and signature, including a receiver that echoes the request;
    • COALESCE on both dialects;
    • the outbox snapshot, booking sync, CSV and the panel.
  • Publish gate passed, no dashes added, DCO and commitlint clean. No test reaches HubSpot or Cloudflare.

🤖 Generated with Claude Code

josue-commits and others added 16 commits September 25, 2026 13:11
A submission can now carry `visit`: the page it was answered on (the landing
that embeds the form, or the form's own URL), its title, HubSpot page and
portal ids, and the landing's HubSpot visitor cookie (`hutk`).

- `submissionVisitSchema` in @quill/types is tolerant field by field: a value
  that fails its rule is dropped and the submission goes on, so a malformed
  visit can never refuse one. It is string-built, like the rest of the
  package, and strips what Postgres cannot store in a JSON value.
- Migration 0023 adds the nullable `submission.visit` column in both dialects.
- `upsertSubmission` writes it with COALESCE in both UPDATEs and the INSERT: a
  write without a visit keeps the stored one, a write with one replaces it.
  The row it returns is the merged one.
- The dashboard read side maps it to a safe view (page, embedded, "HubSpot
  visitor linked") at the one place it turns rows into objects, so a route
  that spreads a row cannot hand the cookie out.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
The API stores the visit a submit reports and hands the merged row's visit
to every destination, snapshotted in the outbox so a retry sends the same.

- The HubSpot cookie is kept only when something on the form uses it (a
  webhook, or HubSpot recording the form submission), is snapshotted only for
  a destination that sends it on, and is never logged: a refused one is
  reported as "hutk dropped" without its value.
- HubSpot: the mirror submission sends `hutk`, `pageUri`, `pageName` and
  `pageId` as its context, empty keys left out. A page with no title is named
  after the form's public title. A 400 is retried once with the context it
  always sent, which is safe because a refused post creates no activity; any
  other refusal is not. The Note names the page, the detail says `+visit`, a
  cookie from another portal is reported, and HubSpot's own error text is
  scrubbed of the cookie before it is logged. Without a visit the post is
  byte for byte what it was.
- Webhook: a top-level `visit` (`pageUri`, `pageName`, `embedded`, `hutk`
  when there is one), under the signature. The delivery history records the
  body with the cookie hidden. The test ping shows an example visit.
- Booking sync reads the visit from the row and passes it to its mirror post.
- Dashboard: the list and the single response carry the safe view, and the
  CSV gains a "Page URL" column, last so no existing column moves.
- OpenAPI documents the field and the safe view.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
…anel

An embedded form cannot read the landing it sits on: its HubSpot cookie, URL,
title and campaign all live on another origin. The host page's embed.js can,
so the frame asks for them.

- embed.js answers `dapta-forms:context-request {id, v: 1}` with the page's
  URL, title, HubSpot page and portal ids, and `hubspotutk`. It answers only
  our frames, only while a frame still shows the origin of its own `src`, and
  targets that origin. It leaves the cookie out when the visitor opted out of
  HubSpot tracking or it is not HubSpot shaped, never writes a cookie, and
  answers only "off" for an iframe with `data-dapta-forms-context="off"`.
- lib/host-visit asks at mount (again at 1 s and 3 s while nothing answers)
  and before each partial and complete submit, accepting a reply only from its
  parent for its own nonce. It waits at most 500 ms, or 150 ms when nothing
  ever answered, and falls back to the referrer. On a direct link the page is
  the form's own URL without its prefill, with the form's own HubSpot cookie
  only when the form loads its own tracking ID.
- Both renderers resolve the visit once per submit, alongside the human check
  rather than after it, so a protected form still sends one submit; partials
  carry it too. The landing's utm_* fill `data.utm` all or nothing, and the
  iframe src's own campaign wins. The builder preview asks no one.
- The response panel (Responses and Summary) shows a "Page" row, a link only
  for a web address, and a "HubSpot visit linked" chip; never the cookie.
- Copy: the embed modal, the HubSpot form submission switch and the tracking
  ID hint say what the script now passes, in English and Spanish.
- qa/e2e/v18 covers the embedded, bare, off, opt-out, direct link and spam
  protection cases in Chromium and WebKit, with a preload that answers the
  challenge provider's token check locally.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
A frame with no src (srcdoc) resolves to the host page itself, so its
origin matched and embed.js would have answered it with the page context.
It is no form of ours: it now gets nothing.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
A pageUri carrying a lone surrogate passed the URL check, and the JSON it
became is refused by a Postgres jsonb column, so the whole submit failed with
a 500 and the answers were lost (SQLite stored it). A browser always
serializes its URL to printable ASCII, so anything else is dropped now, the
same as any other field that fails its rule.

Every other visit string was checked against the same probe: the page name
already replaces lone surrogates and controls, the ids and the cookie are
ASCII by their patterns, and the landing's UTMs decode to replacement
characters. The db spec now stores a visit parsed from hostile strings on
both dialects.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
… answers

The delivery history shows the receiver's answer verbatim, for a delivered
webhook and for a refused one. A receiver that answers with the request it
got (request bins and workflow tools do) put the visitor's hutk on screen.

Both answers now pass through one shared `scrubHutk`, which the HubSpot
adapter uses for its logs too (it replaces the adapter's own copy), and the
request transcript is built with it. The answer is scrubbed before it is cut
to length, so the cut can no longer keep a piece of the cookie.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
…iting

The frame accepted an answer only while its request was still waiting, so a
busy host page that answered after the 150 or 500 ms wait was dropped: the
next submit still saw a silent host, sent the referrer and waited the short
time again. An answer to any request the frame made now updates what it
knows of the page, and the host then counts as one that speaks. The submit
that timed out is not delayed; it already went with what it had.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
A HubSpot destination posts the form submission, the one call that takes the
cookie, on a complete only, yet its partial snapshot carried the cookie too.
The outbox snapshot now keeps it only for a webhook in a phase it fires for,
and for HubSpot recording the form submission on a complete. The row still
stores it whenever some phase will use it, since a complete without a visit
delivers the one its partial stored.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
`DestinationVisit` repeated every field of `SubmissionVisit` from
@quill/types, so the two could drift. It is now an alias of the contract type.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
…g cast

`SubmissionService.listSubmissions` had no caller, and since the visit it
would have handed out the raw HubSpot cookie. Booking sync now names an
untitled page from the form config it already parsed instead of casting the
raw one.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
The embed modal copy lives in the builder catalog, not in @quill/shared, and
the changeset now says the delivery history hides the cookie in a receiver's
answer too.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
The dashboard chip said "HubSpot visit linked" whenever the visitor's cookie
arrived, which is also true for a webhook-only form and after HubSpot refused
the visit. It now says what the flag knows, "HubSpot cookie received" /
"Cookie de HubSpot recibida", and the field is renamed `hubspotCookie`
before anything ships. The OpenAPI description says the cookie itself is never
returned and that the flag does not mean HubSpot accepted the visit.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
The landing's query now reaches the form, and a key like `utm_%00` decoded
to `utm_\u0000`, which Postgres refuses in a JSON key: the submit failed with
a 500 on every retry and the lead was lost. Both UTM sources, the landing's
and the form's own URL, now read through one `utmParams`: a key carrying a
control is skipped, controls leave a value, and an empty value is no
parameter. Checked on Postgres through a real browser and the API: the same
landing went from an error to the thank-you screen, stored without the pair.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
embed.js answered any iframe marked `data-dapta-forms` whose document still
showed the origin of its src, whatever that origin was, so a page editor who
marked a foreign iframe handed it hubspotutk and the full page URL. The
script now reads the host it was loaded from (`document.currentScript`) and
answers a context request only for a frame whose src is on that host. An
inline copy, which cannot know its host, keeps the src check alone. Resize,
redirect and scroll into view are unchanged.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
The Page row linked the page under the title the respondent's browser
reported, so a title like "Acme pricing" could dress up any link. The link
now reads "title · host", with the host taken from the URL it opens; an
untitled page still reads as its host alone.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
Strict mode refuses a complete under 2 s from the session's view, which is
recorded when the lazy frame hydrates. The spec timed its 2.1 s from the
landing's load instead, so a slow machine could submit too early and read
the check failing.

Refs #199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Josue Hernandez <josue@daptatech.com>
@josue-commits
josue-commits changed the base branch from feat/spam-protection-captcha to develop September 26, 2026 15:44
@josue-commits
josue-commits merged commit eb7ec04 into develop Sep 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(embed): pass the HubSpot visit (hutk, page URL, landing UTMs) from embedded forms

1 participant