Runtime: add unified operation deadlines

README.md

@@ -12,7 +12,7 @@ Build -> Flash -> Run -> Observe -> Diagnose -> Fix
 
 BenchPilot is **not** a CANoe clone. It does not aim to reproduce full vehicle-network simulation, CAPL, ADAS simulation or hundreds of analysis windows. CAN/CAN FD, DBC, ISO-TP, UDS and DoIP are added when they help complete the ECU development loop.
 
-> Current status: **real-bench software foundation ready for physical validation**. The resident Runtime, versioned local API, CLI and MCP adapter share one hardware state and safety boundary. Real `system-serial`, J-Link Commander and SCPI power drivers, non-destructive preflight/readiness checks, bounded operation/observation evidence and graceful Runtime shutdown are implemented. Windows/Linux CI is green. The next gate is validation against an actual ECU + J-Link + serial + bench supply, not adding more protocols.
+> Current status: **real-bench software foundation ready for physical validation**. The resident Runtime, versioned local API, CLI and MCP adapter share one hardware state and safety boundary. Real `system-serial`, J-Link Commander and SCPI power drivers, non-destructive preflight/readiness checks, bounded operation/observation evidence, Runtime-owned execution deadlines and graceful Runtime shutdown are implemented. Windows/Linux CI is green. The next gate is validation against an actual ECU + J-Link + serial + bench supply, not adding more protocols.
 
 ## Why BenchPilot?
 
@@ -89,7 +89,7 @@ The foundation transport is HTTP JSON bound to loopback only. `benchpilotd` refu
 The simulator behaves like one small physical bench:
 
 - virtual bench supply with inrush -> settle -> idle current;
-- virtual firmware boot log;
+- virtual firmware boot log with time-based line visibility;
 - flash/reset behavior;
 - shared state across power, serial and flash;
 - context-compressed serial wait observations;
@@ -153,6 +153,18 @@ dotnet run --project src/Benchpilot.Cli -- power check --lt-ma 100 --json
 dotnet run --project src/Benchpilot.Cli -- power off --json
 ```
 
+Long mutations and serial observations can also carry a Runtime execution budget:
+
+```bash
+dotnet run --project src/Benchpilot.Cli -- \
+  flash write build/app.elf --deadline-ms 30000 --json
+
+dotnet run --project src/Benchpilot.Cli -- \
+  serial wait Ready --timeout-ms 5000 --deadline-ms 7000 --json
+```
+
+`--deadline-ms` is deliberately different from a device/protocol timeout or `serial wait --timeout-ms`. The serial timeout is the semantic wait window: reaching it normally produces an unmatched assertion. The Runtime deadline is the outer execution budget shared by CLI/MCP/Agent workflows. When it expires, Runtime records `deadline_exceeded` in history and evidence and rejects even a late success returned by a driver that ignored cancellation.
+
 ### 3. Validate a physical bench before touching the ECU
 
 Start from the checked-in example profile and replace every `CHANGE_ME` value with your actual bench information:
@@ -213,7 +225,7 @@ With `benchpilotd` still running:
 dotnet run --project src/Benchpilot.Mcp
 ```
 
-The MCP process is only a stdio protocol adapter. It calls the same resident Runtime as CLI, so an Agent and a terminal observe the same ECU/bench state. The MCP `BenchValidate` tool exposes the same non-destructive readiness report as CLI.
+The MCP process is only a stdio protocol adapter. It calls the same resident Runtime as CLI, so an Agent and a terminal observe the same ECU/bench state. The MCP `BenchValidate` tool exposes the same non-destructive readiness report as CLI. Long power/flash/serial tools also expose an optional `deadlineMs`, enforced and audited by Runtime rather than by the MCP process.
 
 Example Agent task:
 
@@ -234,10 +246,13 @@ CLI exit codes are intentionally stable and machine-friendly:
 3 target/resource/operation/observation/evidence not found
 4 runtime/device unavailable or device/preflight error
 5 target/resource busy because another mutating operation is active
+6 Runtime execution deadline exceeded
 ```
 
 A `bench validate` exit code of `1` does **not** mean the readiness API failed. It means the report executed successfully but one or more blocking readiness checks failed; inspect the JSON checks and remediation fields.
 
+A deadline failure is returned as `code=deadline_exceeded` with `deadlineMs` and `deadlineAtUtc`; active and history records also expose deadline metadata. Emergency power-off intentionally has no Runtime deadline once accepted, because a safety shutdown must not be abandoned just because a shell budget expired.
+
 ## Resource / target profile
 
 BenchPilot does not assume that a real bench has one monolithic `hardware` driver. A target can combine independent vendor resources:
@@ -318,7 +333,7 @@ Future CAN/protocol/Flash/Studio projects plug into these boundaries rather than
 Near-term work remains a vertical slice rather than broad protocol coverage:
 
 1. validate `system-serial` + J-Link + SCPI power against one physical ECU and check in a repeatable known-good profile;
-2. correlate power/current context with flash/boot failures and strengthen real-bench evidence;
+2. add a richer vendor-neutral device/runtime error taxonomy and production-grade evidence/artifact references;
 3. CAN/CAN FD + DBC observations via SocketCAN and PCAN;
 4. ISO-TP + UDS;
 5. professional, hardware-aware UDS Flash Engine;
@@ -355,4 +370,4 @@ A future visual workflow editor and textual DSL will compile to the same typed e
 
 ## License
 
-GNU Affero General Public License v3.0. See [LICENSE](LICENSE).
+GNU Affero General Public License v3.0. See [LICENSE](LICENSE).

ROADMAP.md

@@ -46,22 +46,26 @@ These are now Runtime properties shared by CLI, MCP and future Studio clients, r
 - [x] active operation IDs with target, operation kind, resources and start time;
 - [x] cooperative cancellation through Runtime into supported drivers;
 - [x] busy responses identify the owning operation when available;
-- [x] bounded in-memory operation history with `completed` / `cancelled` / `faulted` execution states;
+- [x] bounded in-memory operation history with `completed` / `cancelled` / `faulted` / `deadline_exceeded` execution states;
 - [x] bounded mutation evidence keyed by operation ID;
 - [x] non-mutating observation IDs/history/cancellation/evidence for serial operations;
 - [x] bounded serial failure windows sourced from the driver's local line buffer;
 - [x] bounded cross-operation target context (power/current) correlated into flash/reset and boot-wait failures;
+- [x] Runtime-owned execution deadlines for long mutations/observations, distinct from caller cancellation and device/semantic timeouts;
+- [x] active/history/evidence/API/CLI/MCP deadline metadata with stable `deadline_exceeded` classification;
+- [x] late success from a driver that ignores cancellation is rejected after a Runtime deadline expires;
 - [x] explicit normal shutdown versus emergency shutdown semantics;
-- [x] emergency power-off bypasses mutation gates, is non-cancellable once accepted, and is audited;
+- [x] emergency power-off bypasses mutation gates, is non-cancellable once accepted, has no Runtime deadline, and is audited;
 - [x] destructive flash/reset confirmation policy;
 - [x] maximum voltage/current bench safety enforcement;
-- [x] stable validation / not-found / busy / cancelled / runtime-state API error classes;
+- [x] stable validation / not-found / busy / cancelled / deadline-exceeded / runtime-state API error classes;
 - [x] graceful host shutdown requests cancellation and drains Runtime-owned active work before releasing hardware resources;
 
+Runtime deadline semantics are intentionally separate from protocol/device timing. For example, `serial wait --timeout-ms` is the semantic observation window and may return a normal unmatched assertion, while `--deadline-ms` is the outer Runtime execution budget and produces `deadline_exceeded` when exhausted. Device drivers may still enforce narrower hardware/tool-specific timeouts internally.
+
 Still intentionally incomplete:
 
 - [ ] richer device/runtime error taxonomy for vendor-specific failures without leaking vendor SDK types into Core;
-- [ ] one Runtime-level deadline/timeout model across all long operations (drivers already enforce bounded device timeouts where required);
 - [ ] persistent evidence/artifact storage beyond the current bounded in-memory Runtime stores;
 - [ ] remote/team leases — local mutation locks are **not** a substitute for authenticated remote ownership.
 

scripts/smoke-runtime.sh

@@ -105,6 +105,46 @@ assert "capturedAtUtc" in ctx["metadata"]
 assert "ageMs" in ctx["metadata"]
 '
 
+# Runtime deadlines are not semantic serial timeouts. Make the execution budget
+# much shorter than the wait window and require the distinct CLI/API/history/
+# evidence classification all the way through the resident daemon.
+set +e
+deadline_json="$(cli serial wait __BENCHPILOT_DEADLINE__ --timeout-ms 5000 --deadline-ms 100 --json)"
+deadline_code=$?
+set -e
+printf '%s\n' "$deadline_json"
+if [[ $deadline_code -ne 6 ]]; then
+  echo "expected Runtime deadline to return exit code 6, got $deadline_code" >&2
+  exit 1
+fi
+printf '%s' "$deadline_json" | python3 -c '
+import json,sys
+r=json.load(sys.stdin)
+assert r["ok"] is False
+assert r["code"] == "deadline_exceeded"
+assert r["deadlineMs"] == 100
+assert r.get("deadlineAtUtc")
+'
+
+deadline_history="$(cli observe history --limit 1 --json)"
+printf '%s\n' "$deadline_history"
+deadline_observation_id="$(printf '%s' "$deadline_history" | python3 -c '
+import json,sys
+r=json.load(sys.stdin)["observations"][0]
+assert r["state"] == "deadline_exceeded"
+assert r.get("deadlineAtUtc")
+print(r["id"])
+')"
+deadline_evidence="$(cli observe evidence "$deadline_observation_id" --json)"
+printf '%s\n' "$deadline_evidence"
+printf '%s' "$deadline_evidence" | python3 -c '
+import json,sys
+r=json.load(sys.stdin)
+item=next(x for x in r["items"] if x["kind"] == "runtime.deadline")
+assert item["metadata"]["deadlineMs"] == "100"
+assert item["metadata"].get("deadlineAtUtc")
+'
+
 cli power check --lt-ma 100 --json
 cli power off --json
 # Completed mutations and observations remain available as bounded Runtime audit trails.

src/Benchpilot.Cli/Program.cs

@@ -39,6 +39,7 @@ public static async Task<int> Run(string[] args)
         {
             using var client = new BenchClient(BenchClient.ResolveEndpoint(parsed.Get("endpoint")));
             var target = parsed.Get("target");
+            var deadlineMs = parsed.GetNullableInt("deadline-ms");
             var command = parsed.Positionals[0].ToLowerInvariant();
             var subcommand = parsed.Positionals.Count > 1
                 ? parsed.Positionals[1].ToLowerInvariant()
@@ -134,14 +135,14 @@ public static async Task<int> Run(string[] args)
                 {
                     var voltage = parsed.GetDouble("voltage", 12);
                     var settleMs = parsed.GetInt("settle-ms", 2000);
-                    var result = await client.PowerOn(voltage, settleMs, target, cts.Token);
+                    var result = await client.PowerOn(voltage, settleMs, target, deadlineMs, cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
 
                 case ("power", "off"):
                 {
-                    var result = await client.PowerOff(target, cts.Token);
+                    var result = await client.PowerOff(target, deadlineMs, cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
@@ -175,15 +176,20 @@ public static async Task<int> Run(string[] args)
                 {
                     var firmware = RequirePositional(parsed, 2, "firmware path");
                     var confirmTarget = parsed.Get("confirm-target");
-                    var result = await client.Flash(firmware, target, confirmTarget, cts.Token);
+                    var result = await client.Flash(
+                        firmware,
+                        target,
+                        confirmTarget,
+                        deadlineMs,
+                        cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
 
                 case ("flash", "reset"):
                 {
                     var confirmTarget = parsed.Get("confirm-target");
-                    var result = await client.Reset(target, confirmTarget, cts.Token);
+                    var result = await client.Reset(target, confirmTarget, deadlineMs, cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
@@ -192,7 +198,7 @@ public static async Task<int> Run(string[] args)
                 {
                     var port = parsed.Get("port");
                     var baud = parsed.GetNullableInt("baud");
-                    var result = await client.SerialOpen(port, baud, target, cts.Token);
+                    var result = await client.SerialOpen(port, baud, target, deadlineMs, cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
@@ -201,7 +207,12 @@ public static async Task<int> Run(string[] args)
                 {
                     var pattern = RequirePositional(parsed, 2, "pattern");
                     var timeoutMs = parsed.GetInt("timeout-ms", 10000);
-                    var result = await client.SerialWaitFor(pattern, timeoutMs, target, cts.Token);
+                    var result = await client.SerialWaitFor(
+                        pattern,
+                        timeoutMs,
+                        target,
+                        deadlineMs,
+                        cts.Token);
                     Print(result, parsed.Json);
                     if (!result.Ok) return 4;
                     return result.Matched ? 0 : 1;
@@ -211,15 +222,20 @@ public static async Task<int> Run(string[] args)
                 {
                     var lines = parsed.GetInt("lines", 50);
                     var filter = parsed.Get("filter");
-                    var result = await client.SerialReadWindow(lines, filter, target, cts.Token);
+                    var result = await client.SerialReadWindow(
+                        lines,
+                        filter,
+                        target,
+                        deadlineMs,
+                        cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
 
                 case ("serial", "send"):
                 {
                     var data = RequirePositional(parsed, 2, "data");
-                    var result = await client.SerialSend(data, target, cts.Token);
+                    var result = await client.SerialSend(data, target, deadlineMs, cts.Token);
                     Print(result, parsed.Json);
                     return result.Ok ? 0 : 4;
                 }
@@ -237,10 +253,13 @@ public static async Task<int> Run(string[] args)
                 ex.Message,
                 ex.OperationId,
                 ex.BusyScope,
-                ex.BusyId), parsed.Json);
+                ex.BusyId,
+                ex.DeadlineMs,
+                ex.DeadlineAtUtc), parsed.Json);
             return ex.Code switch
             {
                 "busy" => 5,
+                "deadline_exceeded" => 6,
                 "cancelled" => 1,
                 "runtime_state" => 4,
                 _ => ex.StatusCode switch
@@ -301,27 +320,27 @@ benchpilot history                 [--limit N] [--json] [--endpoint URL]
   benchpilot evidence <operation-id> [--json] [--endpoint URL]
   benchpilot cancel <operation-id>   [--json] [--endpoint URL]
 
-  benchpilot observe list                    [--json] [--endpoint URL]
-  benchpilot observe history                 [--limit N] [--json] [--endpoint URL]
+  benchpilot observe list                      [--json] [--endpoint URL]
+  benchpilot observe history                   [--limit N] [--json] [--endpoint URL]
   benchpilot observe evidence <observation-id> [--json] [--endpoint URL]
-  benchpilot observe cancel <observation-id> [--json] [--endpoint URL]
+  benchpilot observe cancel <observation-id>   [--json] [--endpoint URL]
 
   benchpilot preflight              [--target ID] [--json]
   benchpilot bench validate         [--target ID] [--json]
 
-  benchpilot power on            [--target ID] [--voltage V] [--settle-ms N] [--json]
-  benchpilot power off           [--target ID] [--json]
+  benchpilot power on            [--target ID] [--voltage V] [--settle-ms N] [--deadline-ms N] [--json]
+  benchpilot power off           [--target ID] [--deadline-ms N] [--json]
   benchpilot power emergency-off [--target ID] [--json]
   benchpilot power current       [--target ID] [--window-ms N] [--json]
   benchpilot power check         [--target ID] [--lt-ma N] [--gt-ma N] [--json]
 
-  benchpilot flash write <firmware> [--target ID] [--confirm-target ID] [--json]
-  benchpilot flash reset            [--target ID] [--confirm-target ID] [--json]
+  benchpilot flash write <firmware> [--target ID] [--confirm-target ID] [--deadline-ms N] [--json]
+  benchpilot flash reset            [--target ID] [--confirm-target ID] [--deadline-ms N] [--json]
 
-  benchpilot serial open            [--target ID] [--port NAME] [--baud N] [--json]
-  benchpilot serial wait <pattern>  [--target ID] [--timeout-ms N] [--json]
-  benchpilot serial window          [--target ID] [--lines N] [--filter TEXT] [--json]
-  benchpilot serial send <data>     [--target ID] [--json]
+  benchpilot serial open            [--target ID] [--port NAME] [--baud N] [--deadline-ms N] [--json]
+  benchpilot serial wait <pattern>  [--target ID] [--timeout-ms N] [--deadline-ms N] [--json]
+  benchpilot serial window          [--target ID] [--lines N] [--filter TEXT] [--deadline-ms N] [--json]
+  benchpilot serial send <data>     [--target ID] [--deadline-ms N] [--json]
 
 Mutating operations and observations are intentionally separate. `operations`
 uses target/resource gates for state-changing work. `observe ...` reports
@@ -333,6 +352,13 @@ compact mutation evidence bundle. `observe history/evidence` provide the same
 A failed/unmatched serial wait captures only a small tail of the Runtime-owned
 serial line buffer, never the unbounded raw stream.
 
+`--deadline-ms` is a Runtime execution budget for supported mutation/observation
+operations. It is different from `serial wait --timeout-ms`: the latter is a
+semantic wait window and a normal unmatched assertion returns exit code 1;
+exceeding the Runtime deadline is an execution failure with code
+`deadline_exceeded` and exit code 6. Emergency power-off deliberately ignores
+Runtime deadlines so an accepted safety action cannot be abandoned by a shell.
+
 `preflight` is non-destructive. It checks configured resource readiness without
 power-cycling, resetting or flashing the target.
 
@@ -363,6 +389,7 @@ 2 validation error
   3 target/resource/operation/observation/evidence not found
   4 runtime/device unavailable or device/preflight error
   5 target/resource busy (another mutating operation is active)
+  6 Runtime deadline exceeded
 """);
     }
 }
@@ -454,4 +481,4 @@ public double GetDouble(string name, double defaultValue)
             ? parsed
             : throw new FormatException($"Option --{name} must be a number.");
     }
-}
+}