Add run hook chain to replace session-lifecycle events - #151
Open
Sewer56 wants to merge 14 commits into
Open
Conversation
Introduce run hooks so agents can intercept, mutate, or block runs before and after each execution, using the same chain semantics as tool hooks. - Core: add RunHook, RunOriginal, and RunExecutor traits with RunConfig, RunOutput, and RunUsage types, all built on the shared hook chain - Provide on_run_start/on_run_end convenience wrappers: code before `original` is "start", code after is "end" - Replace session events with HookRunContext (adds model_name) and add a Failed EndReason for LLM errors, length limits, and content filters; keep a compact callback distinct from the run chain - SerdesAI: dispatch runs through HookSet::dispatch_run via new HookedAgent (run/run_stream) and SerdesRunExecutor; bridge tool hooks with HookedToolExecutor/CoreToolBridge and output_to_return plus return_to_output conversions - Split hook examples into focused standalone files that share a harness
- Removed 5 redundant tests from the hook_set tests module with no loss of coverage: two on_run_start/on_run_end wrapper-count tests duplicated the builder's run-hook registration checks, and the ordering tests were subsumed by the run-start-before-hooks test (including unwind order) and the merged end-reason test. - Merged the two end-reason tests into one that asserts on_run_end fires exactly once with the executor's EndReason for both Completed and Failed. - Added executor-output passthrough assertions so the notify-wrapper entry points still cover unchanged propagation of executor output content and end reason. hooks test count goes 19 -> 14; cargo test -p reloaded-code-core --lib hooks:: passes (29 passed, 0 failed).
The review workflow writes handoffs, ledgers, and verdict artifacts under artifact/ directories (repo root and src/). Ignore them so this scratch output never shows in git status or gets committed.
Define a run as one `agent.run()` call, start to finish, in the headless framework: no persistent conversation, branching, or multi-session switching. A run holds N steps, where one step is one LLM request plus its tool calls. Run hooks wrap that whole boundary; tool hooks fire inside a run under the same `run_id`. Also reword the doc comments to follow the docs style rules and fix rustdoc link syntax. Docs only; no code or behavior change.
- Moved the run-lifecycle types (`EndReason`, `HookRunContext`, `SessionCompactFn`) from the removed `hooks/session/` module into `hooks/run_hook/`, so the run hook chain owns its context types like `tool_hook` does. - Crate-root public API is unchanged; this is a pure relocation. - Folded the `hooks/mod.rs` doc list. - Fixed two test-only imports that referenced the removed module.
…ace fixture Add three mock-gated example binaries under examples/hooks/tool/ that exercise the tool hook surface end to end with realistic guardrails that static permission rules cannot express: a result-rewrite hook that scrubs secret values from a real read, a stateful hook that denies writes to files the run never read, and a two-hook chain that audits then hardens bash arguments before one real execution. The shared example fixture runs everything inside a hermetic tempfile workspace holding a secrets-bearing service.env and an unread write target, gains a two-tool-call scripted mock model helper for two-step scenarios, and its agent_config_with_tools helper builds permission rules that allow the named standard tools. Each example is registered in Cargo.toml behind the mock feature so default-feature builds are unchanged.
One test in the existing task.rs test module builds a runtime with a stateful ReadBeforeWriteHook and scripts the mock model with a test-local two-tools-then-text helper so a real agent run first reads a temp-workspace fixture, then attempts a write to a file the run never read. It asserts the real read executed and its result reached the model, the hook's explanatory denial replaced the write response without calling the original tool, the unread file was never created, and the run completed. The test runs under plain `cargo test -p reloaded-code-serdesai` and fails if tool hook dispatch or short-circuit wiring breaks.
…narios Every hook type in the hooks guide (tool hook observe/wrap, tool block, tool chain, run hook, run event, run chain) has a short inline snippet and a link to the runnable example binary it maps to. The tool-hook sections describe the realistic guardrail scenarios: a result rewrite that scrubs secret values from a real read, a stateful deny for writes to files the run never read, and an audit hook stacked with an argument-hardening hook around one real bash execution. The examples README gained a tool-hooks section matching the run-hooks section, and the shared-code note covers both hook kinds and the tool-permission config fixture.
Reference-style link definitions (`[`X`]: path`) sat mid-comment in 13 doc blocks, splitting prose from its sections. Moved all definitions to the bottom of each doc block and consolidated scattered ones (`Streamed`, `tools/custom`) so rustdoc renders sections contiguously. Verified: fmt, tests, clippy -D warnings, rustdoc -D warnings all pass. `cargo publish --dry-run` fails on pre-existing missing reloaded-code-provider-config/README.md, unrelated to this change.
Audit of task.rs's test module found two tests asserting nothing their neighbours did not already cover (public-wrapper variants of the no-callable-targets and max-depth scenarios) and four copies of the same fixtures spread across task.rs, handle.rs, build.rs, and the hook examples. This removes the redundancy without losing executed coverage: - Deleted `agent_build_context_omits_task_tool_when_no_targets_are_callable` (byte-identical scenario to `build_agent_skips_task_tool_when_no_targets_ are_callable`; the only delta, the 1-line `build()` delegate, stays covered by the hook end-to-end test) and `agent_build_context_omits_task_tool_when_max_depth_is_zero` (same production branch as `build_agent_omits_task_tool_at_max_depth`). - Merged the pattern-scoped and absent-permission Task-attach tests into one two-runtime test; both assertion sets remain. - Promoted `two_tools_then_text` into `mock` as a public generalisation of `tool_then_text` (shared `tool_call_response` helper, reuses `extract_tool_return_text`), replacing the test-local copy and the twin in `examples/hooks/shared.rs`; the tool-block example now imports it. - Moved agent/allow_tools/pattern_task/catalog/credentials/workspace_root fixtures into `agent_runtime::test_stubs` (pub(crate), cfg(test)); handle.rs and build.rs test modules consume them and build.rs's `agent_with_sampling` becomes a struct update on the shared fixture. - Replaced cfg-gated `TaskBuildContext` struct literals with the existing `new_for_test` constructor, decoupling tests from future field additions. Net -285 lines. Suite stays green (110 unit + 20 doc tests); the hook end-to-end test still solely covers `with_model_override`, `HookedAgent::run`, and the `HookedToolExecutor` wiring.
Automated by the rust-llm-tidy GitHub Action.
Contributor
rust-llm-tidy: ✅ fixes appliedI tidied the files below and pushed commit afb9b28. Changed files:
|
Contributor
rust-llm-tidy: ✅ all tidyAll files are tidy - no changes required. |
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- on_run_end now fires with EndReason::Failed when the executor errors, then propagates the error unchanged (plus tests). - HookedAgent::run generates a real run id for run-hook contexts instead of an empty string. - run_stream rejects non-text prompts on the hooked path and emits a RunComplete event with the real run id and message history. - SerdesRunExecutor keeps preamble order stable (system prompt, then preambles in configured order) and reports token usage from the response instead of defaults. - Tool-hook bridge restores untouched ToolReturn/ToolError values so images, tool_call_id, truncated markers, and structured validation errors reach the model unchanged; hook-modified results still convert. - ReadBeforeWrite examples key read authorization by (run_id, path). - tool_hook docs drop the false retry-by-cloning claim. - hooks.md fixes AuditHook nesting labels and documents on_run_end failure semantics. - Add missing reloaded-code-provider-config README so cargo publish --dry-run passes (pre-existing failure on main).
Automated by the rust-llm-tidy GitHub Action.
Contributor
rust-llm-tidy: ✅ fixes appliedI tidied the files below and pushed commit e91db14. Changed files:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hooks now intercept full agent runs in the SerdesAI pipeline. A
RunHookwraps the whole
agent.run()boundary: mutateRunConfigbeforeoriginal, skiporiginalto replace the run, observeRunOutputafterit.
Tool hooks now fire on real tool calls.
HookedToolExecutorbridges thecore and SerdesAI executor traits, so registered hooks intercept actual
tool execution end to end. The docs no longer mark hook wiring as work in
progress.
Why
Session start/end callbacks could only observe: no config changes, no
blocking, no shared ordering with tool hooks. One hook chain fixes all
three.
on_run_startandon_run_endsurvive as thinRunHookwrappers.Examples
Six runnable examples live in
src/reloaded-code-serdesai/examples/hooks/, all on the mock model.docs/src/hooks.mddocuments each scenario.Run:
cargo run --example serdesai-run-hook -p reloaded-code-serdesai --features mockinjects a preamble viaRunConfig.serdesai-run-chainshows twoRunHooks nesting in registration order.serdesai-run-eventcoverson_run_startandon_run_endclosures.Tool:
serdesai-tool-hookscrubsAPI_KEY=/TOKEN=fromreadresults.serdesai-tool-blockdenies awriteto a never-read file; the realtool never runs.
serdesai-tool-chainstacks an audit hook and a hardening hook viashared_tool_hook.Breaking changes
SessionContext,on_session_start,on_session_end.HookRunContext,on_run_start,on_run_endinstead.EndReasongains aFailedvariant.AgentRuntimeBuilder::buildreturnsHookedAgent, notAgent<(), String>.HookedAgent::runreturnsHookedAgentRunResult.run_streamemits a synthetic stream from the finaloutput.
AgentRunError::Other.on_session_compact, overHookRunContext.Verification
cargo test -p reloaded-code-core: 390 passed.cargo test -p reloaded-code-serdesai --features reloaded-code-serdesai/mock:110 passed, including
tool_hook_denies_write_to_never_read_file_during_agent_run.Not run: full
src/.cargo/verify.sh. Done when: it passes clean, coveringclippy, docs, blocking features, and publish dry-run.