Skip to content

Fix cluster request resets stranded by in-memory deduplication - #8317

Merged
tim-smart merged 12 commits into
mainfrom
agent/bob/e326ac81f5a7
Sep 20, 2026
Merged

tim-smart merged 12 commits into
mainfrom
agent/bob/e326ac81f5a7

Conversation

@tim-smart

@tim-smart tim-smart commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Release targeted storage claims when a reset request is suppressed by completed-request deduplication, including the exit-publication window before active-request cleanup. This lets the receiving runner recover resets initiated locally or through shared storage on another runner without waiting for the ten-minute claim timeout.

Active requests, including streams that have emitted chunks, retain their claims. A separate exit flag distinguishes completion from chunk publication; notifyLocal keeps its existing excludeReplies semantics. Targeted-release failures are contained so capped-address claims from the same batch still get released.

MessageStorage.resetRequests clears only selected request claims, preserving replies, processed state, and unrelated requests. SQL, memory, and noop implementations are included. Custom drivers must implement this new required member of the exported storage interfaces; the changeset documents that compatibility impact despite the rc patch classification.

Coverage and validation

  • Sharding, MessageStorage, and ClusterWorkflowEngine: 119 tests passed. Coverage includes local and shared-storage remote resets, exit publication before cleanup, active-stream claim retention, and capped-address release after targeted-release failure.
  • SQL storage integration: 65 tests passed across PostgreSQL, MySQL, and SQLite (EFFECT_INTEGRATION_TESTS=1). Storage contracts cover empty input, targeted release, and reply/processed-state preservation.
  • nix develop -c pnpm lint-fix, nix develop -c pnpm check, and git diff --check passed.
  • The implementation/documentation follow-up leaves all tests unchanged. Both previously failing review regressions now pass.

The deterministic Sharding tests use memory storage. Unlike SQL storage, it does not exclude requests with unacknowledged chunk replies, and it does not model PostgreSQL locking or concurrent transactions. The remote-reset case uses two adapters over shared persistence, not two processes. These limits are documented in the memory driver.

Independent re-review and the reporter's Bun/PostgreSQL workload (300 workflows, 20 concurrent) remain required before acceptance. That workload has not been run against this revision.

Closes EFF-1426
Closes #8312

@changeset-bot

changeset-bot Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: dfa11c2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 31 packages
Name Type
effect Patch
@effect/opentelemetry Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openai Patch
@effect/ai-openrouter Patch
@effect/ai-typesafe Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-janitor effect-janitor Bot added the 4.0 label Sep 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 37.71 KB 37.71 KB 0.00 KB (0.00%)
basic.ts 6.87 KB 6.87 KB 0.00 KB (0.00%)
batching.ts 9.95 KB 9.95 KB 0.00 KB (0.00%)
brand.ts 6.45 KB 6.45 KB 0.00 KB (0.00%)
cache.ts 10.77 KB 10.77 KB 0.00 KB (0.00%)
config.ts 21.83 KB 21.83 KB 0.00 KB (0.00%)
differ.ts 20.67 KB 20.67 KB 0.00 KB (0.00%)
http-client.ts 21.94 KB 21.94 KB 0.00 KB (0.00%)
http-router.ts 36.96 KB 36.96 KB 0.00 KB (0.00%)
logger.ts 10.88 KB 10.88 KB 0.00 KB (0.00%)
metric.ts 9.02 KB 9.02 KB 0.00 KB (0.00%)
optic.ts 6.70 KB 6.70 KB 0.00 KB (0.00%)
pubsub.ts 15.10 KB 15.10 KB 0.00 KB (0.00%)
queue.ts 11.85 KB 11.85 KB 0.00 KB (0.00%)
schedule.ts 10.96 KB 10.96 KB 0.00 KB (0.00%)
schema-bigdecimal.ts 13.40 KB 13.40 KB 0.00 KB (0.00%)
schema-binary.ts 39.82 KB 39.82 KB 0.00 KB (0.00%)
schema-class.ts 20.38 KB 20.38 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 31.18 KB 31.18 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 26.60 KB 26.60 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.96 KB 13.96 KB 0.00 KB (0.00%)
schema-string.ts 11.55 KB 11.55 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.70 KB 15.70 KB 0.00 KB (0.00%)
schema-toArbitrary.ts 37.24 KB 37.24 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.84 KB 24.84 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.61 KB 19.61 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 19.75 KB 19.75 KB 0.00 KB (0.00%)
schema-toFormatter.ts 19.85 KB 19.85 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 24.15 KB 24.15 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.89 KB 19.89 KB 0.00 KB (0.00%)
schema.ts 19.59 KB 19.59 KB 0.00 KB (0.00%)
stm.ts 12.80 KB 12.80 KB 0.00 KB (0.00%)
stream.ts 9.83 KB 9.83 KB 0.00 KB (0.00%)

@tim-smart
tim-smart merged commit 8d40572 into main Sep 20, 2026
13 checks passed
@tim-smart
tim-smart deleted the agent/bob/e326ac81f5a7 branch September 20, 2026 09:31
arrudaricardo pushed a commit to arrudaricardo/effect that referenced this pull request Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ClusterWorkflowEngine: reset run request is claimed and then skipped by the storage read loop, so the workflow stalls for 10 minutes

1 participant