Skip to content

feat(dreamer): gate the memory-pool tasks on session message activity - #465

Closed
TreedsSlop wants to merge 2 commits into
cortexkit:masterfrom
TreedsSlop:dreamer-activity-gates
Closed

TreedsSlop wants to merge 2 commits into
cortexkit:masterfrom
TreedsSlop:dreamer-activity-gates

Conversation

@TreedsSlop

@TreedsSlop TreedsSlop commented Sep 18, 2026

Copy link
Copy Markdown

verify, verify-broad, curate, compress-cues, and classify-memories gated
only on pool existence, so an untouched project still took the memory lease
and ran a whole-pool LLM pass on every cron slot with nothing new to process.

Require session activity since the last successful run in addition to the
pool half, reusing the MessageActivityProvider introduced for the
retrospective:

  verify / compress-cues / classify-memories: live pool AND activity
  verify-broad:  open cycle OR (live pool AND activity)
  curate:        raw status pool AND activity (curate owns expiry hygiene)

A missing provider or an unavailable message store is treated conservatively
as "run", so hosts without an indexed message store keep their previous
behavior.

Marking this as draft for now until I worked through the issues on the PR this is based on


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Gates the dreamer memory-pool tasks on session message activity, and switches the retrospective to measure real message timestamps instead of the registration-time updated_at column, so untouched projects stop paying for whole-pool LLM passes and stale sessions are no longer misclassified.

Behavior changes

  • verify, compress-cues, and classify-memories now require a live memory pool and session activity since the last successful run.
  • verify-broad runs on an open cycle, or on a closed cycle with a pool and activity.
  • curate requires a raw-status pool and activity, since it owns expiry hygiene.
  • A missing activity provider or an unavailable message store is treated conservatively as "run", preserving behavior for hosts without an indexed message store.

Retrospective fixes

  • Session eligibility is now driven by message-table timestamps, not session_projects.updated_at (which records binding/backfill time).
  • This fixes under-scan (new messages in previously eligible sessions were never re-read) and over-scan (backfilled sessions stayed eligible forever).
  • Providers without an indexed message store keep the updated_at filter as a fallback.

Written for commit 59a923f. Summary will update on new commits.

Review in cubic

Jona added 2 commits September 18, 2026 03:16
The retrospective gate tested `session_projects.updated_at > watermark`, but
updated_at is written only at first project binding (and by the backfill at
scan time) — it is not an activity timestamp:

- under-scan: once the watermark passed a session's registration time, new
  messages in that session could never re-trigger a run, and a session
  truncated at the per-session cap never had its tail read;
- over-scan: a backfilled old session is stamped with the backfill time,
  newer than every message it contains, so it stayed eligible forever and
  re-ran nightly to no effect.

Derive the activity signal from the message table at query time instead:

- a new MessageActivityProvider counts root sessions with a message newer
  than the watermark (indexed message table, sub-ms); a missing provider or
  an unavailable store is treated conservatively as "run" (the executor
  bails before any child session);
- the retrospective scanner makes message activity the eligibility driver,
  demoting the updated_at filter to the indexless-provider fallback.
verify, verify-broad, curate, compress-cues, and classify-memories gated
only on pool existence, so an untouched project still took the memory lease
and ran a whole-pool LLM pass on every cron slot with nothing new to process.

Require session activity since the last successful run in addition to the
pool half, reusing the MessageActivityProvider introduced for the
retrospective:

  verify / compress-cues / classify-memories: live pool AND activity
  verify-broad:  open cycle OR (live pool AND activity)
  curate:        raw status pool AND activity (curate owns expiry hygiene)

A missing provider or an unavailable message store is treated conservatively
as "run", so hosts without an indexed message store keep their previous
behavior.
@magic-alfonso

magic-alfonso Bot commented Sep 18, 2026

Copy link
Copy Markdown

Diagnosis confirmed at source, and it is a good catch: session_projects.updated_at is written by the upsert in session-project-storage.ts only when project_path changes, so it records binding (or backfill) time and is not an activity signal. The retrospective gate at task-gates.ts:392 treats it as one, which produces both the under-scan and the over-scan you describe, on every harness.

We are not taking this shape, for the reason you already named: it is OpenCode-only and reads opencode.db from the dreamer path, and the review bot's finding is right that the provider is applied to Pi and OMP registrations too, where it finds no sessions. Our rule for contributions is that a change has to cover every harness we support (OpenCode, Pi, OMP, and the Rust module lane where it applies), and we have been actively removing opencode.db reads from the plugin rather than adding them.

Your alternative (b) is the fix we want, and it is cross-harness by construction: a per-session activity high-water mark in our own store, keyed on the message time base the watermark already uses (max message time_created, not wall-clock), advanced from the message events every harness already delivers to the plugin (OpenCode message.updated, Pi and OMP message hooks), and backfilled once for existing rows from each harness's own store at migration so stale sessions do not keep a wrong value forever. That satisfies your (a), (b) and (c). The retrospective gate then reads it, and the memory-pool gating from #465 (a genuinely good idea: verify/curate/classify should not take the lease and run a whole-pool pass on an untouched project) becomes a one-line change on the same column.

We are doing it ourselves and will credit the diagnosis in the release notes; it has to hold on every harness, including the Rust transform mode where the dreamer's gates read module-owned state, so it lands with that work rather than as a standalone patch. Closing both as shaped.

@magic-alfonso magic-alfonso Bot closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant