Skip to content

Fix AtomRegistry missed updates through stale nodes and leaked superseded builds - #8543

Merged
tim-smart merged 7 commits into
Effect-TS:mainfrom
front-depiction:atom-registry-dependency-relation
Sep 28, 2026
Merged

tim-smart merged 7 commits into
Effect-TS:mainfrom
front-depiction:atom-registry-dependency-relation

Conversation

@front-depiction

@front-depiction front-depiction commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

AtomRegistry dropped the edges from a parent to its dependents as soon as the parent changed, and expected each dependent to add them back when it rebuilt. A dependent that stays stale never does. Its parents could be released under it (a keepAlive derived atom lost the state it read), and a change in one of those parents never reached it again, even after a listener was added. A build that wrote to one of its own inputs was rebuilt inside itself, and the outer build's lifetime was never disposed.

This revision replaces the earlier rewrite in this PR with a smaller change to the registry on main:

  • Edges are kept while a node is stale. A parent's change is pushed only to the children that depend on its current value, valid ones and running builds that already read it. A build drops the parents it did not read when it returns.
  • A stale node is brought up to date when it gains a listener, so an observed node is never left stale outside a batch.
  • A build superseded by a nested rebuild is disposed, and a node rebuilds until it is valid, since the nested result can go stale again before the outer read returns.
  • Observed dependents are counted instead of walking the children on every invalidation.
  • A batch drains its rebuilds and notifications when its function throws, and finalizers, listeners and idle sweeps run to completion when one of them throws.
  • A build that throws leaves its node stale and stales the dependents that hold or are reading its value. An observed node that stayed stale this way is rebuilt again when a parent commits, so it recovers once its inputs allow. A node that pulls a parent which produced its value, but failed to update another dependent, still takes the value and rethrows that failure once its own build is done. A source announces its own change even when a dependent's rebuild throws.

Atom.ts is unchanged. Four dependency/failure regressions and three batch-notification cases live in packages/effect/test/reactivity/Atom.test.ts. The original three fail on main; the fourth covers recovery after a derived build throws and verifies that the source listener is still notified when its dependent fails. The batch cases cover a late immediate subscriber after a first value, after a first value queued with an existing listener, and after initialValues is applied in a batch.

Behaviour changes worth knowing

  • subscribe without immediate brings a stale node up to date, so it can throw where it previously could not.
  • First and initial values follow the same queueing rule as changed values: a commit queues a notification only if the node has listeners at that moment. A queued notification reaches the listeners present when the batch flushes, so a subscriber added later in the batch can still receive it, including a second delivery after an immediate: true subscription. On main, a first value committed inside a batch was queued even without listeners, so a late subscriber heard it, or heard it twice, where a changed value would have stayed silent.

Validation

With the three batch-notification cases, the reactivity suite passes (134), as do pnpm check and pnpm lint. Before those cases were added, reactivity, hydration, React, Solid and Vue tests passed (296).

A randomized check drove random graphs of state and derived atoms (conditional reads, builds that throw for some inputs, keepAlive, non-lazy and idle-TTL variants) through sets, batches, reads, subscriptions with listeners that write back directly or in a nested batch, refreshes and scheduler flushes, comparing every value, edge and observed count against a pure model, requiring observed nodes to recover from failed builds, and checking that nothing leaks once released. It passes 100000 seeds here. main fails within the first ten, and the previous revision of this PR failed on nested rebuilds staled by a listener write.

Median ns per operation, each case in its own process:

workload main previous revision this
chain of 50, observed, set root 34483 11273 11057
chain of 50, lazy, set root then read 32908 11700 11503
diamond of 100, observed, set root 669047 216047 213844
batch of 5 sets, 200 observed dependents 100779 65505 67520
dynamic dependency toggled 898 826 834
create, read and release 20 nodes 16077 14826 15129
unchanged rebuild, dependent kept 796 414 489

Closes EFF-1604

@changeset-bot

changeset-bot Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: a1d42c8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 31 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/ai-typesafe Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 38.54 KB 38.54 KB 0.00 KB (0.00%)
basic.ts 6.88 KB 6.88 KB 0.00 KB (0.00%)
batching.ts 9.97 KB 9.97 KB 0.00 KB (0.00%)
brand.ts 6.57 KB 6.57 KB 0.00 KB (0.00%)
cache.ts 10.73 KB 10.73 KB 0.00 KB (0.00%)
config.ts 21.81 KB 21.81 KB 0.00 KB (0.00%)
differ.ts 20.95 KB 20.95 KB 0.00 KB (0.00%)
http-client.ts 22.11 KB 22.11 KB 0.00 KB (0.00%)
http-router.ts 33.33 KB 33.33 KB 0.00 KB (0.00%)
logger.ts 10.85 KB 10.85 KB 0.00 KB (0.00%)
metric.ts 8.83 KB 8.83 KB 0.00 KB (0.00%)
optic.ts 6.80 KB 6.80 KB 0.00 KB (0.00%)
pubsub.ts 15.00 KB 15.00 KB 0.00 KB (0.00%)
queue.ts 11.89 KB 11.89 KB 0.00 KB (0.00%)
schedule.ts 11.03 KB 11.03 KB 0.00 KB (0.00%)
schema-bigdecimal.ts 13.48 KB 13.48 KB 0.00 KB (0.00%)
schema-binary.ts 39.53 KB 39.53 KB 0.00 KB (0.00%)
schema-class.ts 20.65 KB 20.65 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 31.72 KB 31.72 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 26.92 KB 26.92 KB 0.00 KB (0.00%)
schema-string-transformation.ts 14.18 KB 14.18 KB 0.00 KB (0.00%)
schema-string.ts 11.73 KB 11.73 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.75 KB 15.75 KB 0.00 KB (0.00%)
schema-toArbitrary.ts 38.08 KB 38.08 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 25.13 KB 25.13 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.88 KB 19.88 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 20.04 KB 20.04 KB 0.00 KB (0.00%)
schema-toFormatter.ts 20.14 KB 20.14 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 24.64 KB 24.64 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 20.14 KB 20.14 KB 0.00 KB (0.00%)
schema.ts 19.86 KB 19.86 KB 0.00 KB (0.00%)
stm.ts 12.90 KB 12.90 KB 0.00 KB (0.00%)
stream.ts 9.82 KB 9.82 KB 0.00 KB (0.00%)

@front-depiction
front-depiction force-pushed the atom-registry-dependency-relation branch from 263aa59 to 1c01832 Compare September 25, 2026 20:36
@front-depiction front-depiction changed the title Make AtomRegistry faster and lighter by storing each dependency once Fix AtomRegistry missed updates and leaks, and cut its update cost Sep 25, 2026
@front-depiction
front-depiction force-pushed the atom-registry-dependency-relation branch from 1c01832 to 0176431 Compare September 25, 2026 20:37
@front-depiction
front-depiction marked this pull request as ready for review September 25, 2026 20:39
@front-depiction
front-depiction force-pushed the atom-registry-dependency-relation branch from 0176431 to 2a9c6fb Compare September 26, 2026 15:14
@tim-smart
tim-smart force-pushed the atom-registry-dependency-relation branch from 065ca04 to bd507cd Compare September 27, 2026 21:51
…eded builds

The registry dropped the edges from a parent to its dependents as soon as
the parent changed, and expected each dependent to add them back when it
rebuilt. A dependent that stays stale never does: its parents could be
released under it, and their next change never reached it. A build that
wrote to one of its own inputs was rebuilt inside itself, and the outer
build's lifetime was never disposed.

Edges are now kept while a node is stale. A parent's change is pushed only
to the children that depend on its current value, valid ones and running
builds that already read it, and a build drops the parents it did not read
when it returns. A stale node is brought up to date when it gains a
listener. A build superseded by a nested rebuild is disposed, and a node
rebuilds until it is valid. Observed dependents are counted instead of
walking the children on every invalidation. A batch drains even when its
function throws, and finalizers, listeners and idle sweeps run to
completion when one of them throws.
@tim-smart tim-smart changed the title Fix AtomRegistry missed updates and leaks, and cut its update cost Fix AtomRegistry missed updates through stale nodes and leaked superseded builds Sep 27, 2026
@tim-smart
tim-smart force-pushed the atom-registry-dependency-relation branch from 4eefb05 to bf4c921 Compare September 27, 2026 22:49
A build that threw left its node stale with no way back: a later change
in a parent skipped it as already stale, so a subscribed node never heard
another update, and a dependent kept a value the failed node no longer
stood behind. A source also skipped its own listeners when a dependent's
rebuild threw.

A build that throws now stales the dependents that hold or are reading
its value, and an observed node that stayed stale this way is rebuilt
again when a parent commits. The edge to a parent is kept even when the
parent could not produce a value, so its recovery reaches the reader. A
node that pulls a parent which produced its value but failed to update
another dependent still takes the value and rethrows once its own build
is done. A source announces its own change before rethrowing a
dependent's failure.
One predicate decides which dependents a parent's change reaches, and it
includes the retry of an observed dependent whose rebuild failed, so a
failed build reuses the same propagation as a commit. A tracked read
records itself in one place once the parent has answered. Committing a
value no longer special-cases the first value: invalidating dependents
returns early without children, and listeners are announced only when
there are some.
A value notifies the listeners present when it is committed. That was
already so for a changed value, and the previous commit made it so for a
first value; an initial value set before its first build now follows the
same rule instead of queueing a notification for listeners a batch adds
later.
@tim-smart
tim-smart merged commit 34cc788 into Effect-TS:main Sep 28, 2026
8 of 10 checks passed
kriegcloud added a commit to beep-effect/beep-effect that referenced this pull request Sep 28, 2026
…gistry

The snapshot's AtomRegistry.reset empties the node map before running node
finalizers (Effect-TS/effect#8543), so the graph-worker finalizer's
registry.get on the watchdog/boundary/failure request atoms created nodes on
a disposed registry and threw "registry is disposed" in three Session atom
tests. Cancellation now bumps a request counter only while its atom still
has a node; a request nobody can answer needs no cancellation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
kriegcloud added a commit to beep-effect/beep-effect that referenced this pull request Sep 28, 2026
… 1) (#1310)

* chore(deps): bump the Effect catalog to snapshot e5f7d12af9 (rc.118 + 1)

Move every Effect catalog entry from the pkg.pr.new snapshot of main
330b7475e2 to the snapshot of e5f7d12af9 (one commit after the
4.0.0-rc.118 tag). The Bun patches for effect and platform-node-shared
are re-keyed to the new URLs (upstream sources unchanged), the OSV
override is re-keyed, and the effect-vitest primitives graph, charter
fixtures, and inventory are re-pinned to the @effect/vitest@4.0.0-rc.118
tag (101 entries: the README gained a fixtures section).

HttpRouter.serve now owns the router it serves (Effect-TS/effect#8477),
so the professional-desktop sidecar builds its rpc server inside the
served app instead of as a sibling of it; the previous shape registered
the handlers on an unserved second protocol. The integration harness
follows the same shape. The knowledge command-surface parser drops the
pre-rc.118 effect/unstable/cli specifier.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ontology-client): skip request cancellation on a disposed atom registry

The snapshot's AtomRegistry.reset empties the node map before running node
finalizers (Effect-TS/effect#8543), so the graph-worker finalizer's
registry.get on the watchdog/boundary/failure request atoms created nodes on
a disposed registry and threw "registry is disposed" in three Session atom
tests. Cancellation now bumps a request counter only while its atom still
has a node; a request nobody can answer needs no cancellation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(epistemic-server): count SHACL violations without spying on the schema

The snapshot caches every schema's `make` as a lazily built, non-configurable
own property (Effect-TS/effect internal/schema/make.ts), so
`vi.spyOn(ShaclValidationViolation, "make")` throws "Attempting to change
value of a readonly property" once any test has built a violation. The
bounded-generation test now counts constructions through a module mock of
the layer's import, whose wrapper inherits the schema and shadows only
`make`. `vi` comes from `vitest` because only that import gets its `mock`
and `hoisted` calls hoisted; the effect-vitest inventory records the three
plain-vitest rows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

This branch is waiting to be deployed

1 waiting deployment
fork — a1d42c8d Waiting Sep 28, 2026 by tim-smart via approval-gate #16729
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants