diff --git a/README.md b/README.md index 8447c99..d4e3144 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ Build -> Flash -> Run -> Observe -> Diagnose -> Fix BenchPilot is **not** a CANoe clone. It does not aim to reproduce full vehicle-network simulation, CAPL, ADAS simulation or hundreds of analysis windows. CAN/CAN FD, DBC, ISO-TP, UDS and DoIP are added when they help complete the ECU development loop. -> Current status: **real-bench software foundation ready for physical validation**. The resident Runtime, versioned local API, CLI and MCP adapter share one hardware state and safety boundary. Real `system-serial`, J-Link Commander and SCPI power drivers, non-destructive preflight/readiness checks, bounded operation/observation evidence and graceful Runtime shutdown are implemented. Windows/Linux CI is green. The next gate is validation against an actual ECU + J-Link + serial + bench supply, not adding more protocols. +> Current status: **real-bench software foundation ready for physical validation**. The resident Runtime, versioned local API, CLI and MCP adapter share one hardware state and safety boundary. Real `system-serial`, J-Link Commander and SCPI power drivers, non-destructive preflight/readiness checks, bounded operation/observation evidence, Runtime-owned execution deadlines and graceful Runtime shutdown are implemented. Windows/Linux CI is green. The next gate is validation against an actual ECU + J-Link + serial + bench supply, not adding more protocols. ## Why BenchPilot? @@ -89,7 +89,7 @@ The foundation transport is HTTP JSON bound to loopback only. `benchpilotd` refu The simulator behaves like one small physical bench: - virtual bench supply with inrush -> settle -> idle current; -- virtual firmware boot log; +- virtual firmware boot log with time-based line visibility; - flash/reset behavior; - shared state across power, serial and flash; - context-compressed serial wait observations; @@ -153,6 +153,18 @@ dotnet run --project src/Benchpilot.Cli -- power check --lt-ma 100 --json dotnet run --project src/Benchpilot.Cli -- power off --json ``` +Long mutations and serial observations can also carry a Runtime execution budget: + +```bash +dotnet run --project src/Benchpilot.Cli -- \ + flash write build/app.elf --deadline-ms 30000 --json + +dotnet run --project src/Benchpilot.Cli -- \ + serial wait Ready --timeout-ms 5000 --deadline-ms 7000 --json +``` + +`--deadline-ms` is deliberately different from a device/protocol timeout or `serial wait --timeout-ms`. The serial timeout is the semantic wait window: reaching it normally produces an unmatched assertion. The Runtime deadline is the outer execution budget shared by CLI/MCP/Agent workflows. When it expires, Runtime records `deadline_exceeded` in history and evidence and rejects even a late success returned by a driver that ignored cancellation. + ### 3. Validate a physical bench before touching the ECU Start from the checked-in example profile and replace every `CHANGE_ME` value with your actual bench information: @@ -213,7 +225,7 @@ With `benchpilotd` still running: dotnet run --project src/Benchpilot.Mcp ``` -The MCP process is only a stdio protocol adapter. It calls the same resident Runtime as CLI, so an Agent and a terminal observe the same ECU/bench state. The MCP `BenchValidate` tool exposes the same non-destructive readiness report as CLI. +The MCP process is only a stdio protocol adapter. It calls the same resident Runtime as CLI, so an Agent and a terminal observe the same ECU/bench state. The MCP `BenchValidate` tool exposes the same non-destructive readiness report as CLI. Long power/flash/serial tools also expose an optional `deadlineMs`, enforced and audited by Runtime rather than by the MCP process. Example Agent task: @@ -234,10 +246,13 @@ CLI exit codes are intentionally stable and machine-friendly: 3 target/resource/operation/observation/evidence not found 4 runtime/device unavailable or device/preflight error 5 target/resource busy because another mutating operation is active +6 Runtime execution deadline exceeded ``` A `bench validate` exit code of `1` does **not** mean the readiness API failed. It means the report executed successfully but one or more blocking readiness checks failed; inspect the JSON checks and remediation fields. +A deadline failure is returned as `code=deadline_exceeded` with `deadlineMs` and `deadlineAtUtc`; active and history records also expose deadline metadata. Emergency power-off intentionally has no Runtime deadline once accepted, because a safety shutdown must not be abandoned just because a shell budget expired. + ## Resource / target profile BenchPilot does not assume that a real bench has one monolithic `hardware` driver. A target can combine independent vendor resources: @@ -318,7 +333,7 @@ Future CAN/protocol/Flash/Studio projects plug into these boundaries rather than Near-term work remains a vertical slice rather than broad protocol coverage: 1. validate `system-serial` + J-Link + SCPI power against one physical ECU and check in a repeatable known-good profile; -2. correlate power/current context with flash/boot failures and strengthen real-bench evidence; +2. add a richer vendor-neutral device/runtime error taxonomy and production-grade evidence/artifact references; 3. CAN/CAN FD + DBC observations via SocketCAN and PCAN; 4. ISO-TP + UDS; 5. professional, hardware-aware UDS Flash Engine; @@ -355,4 +370,4 @@ A future visual workflow editor and textual DSL will compile to the same typed e ## License -GNU Affero General Public License v3.0. See [LICENSE](LICENSE). +GNU Affero General Public License v3.0. See [LICENSE](LICENSE). \ No newline at end of file diff --git a/ROADMAP.md b/ROADMAP.md index 138d0f5..92a8343 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -46,22 +46,26 @@ These are now Runtime properties shared by CLI, MCP and future Studio clients, r - [x] active operation IDs with target, operation kind, resources and start time; - [x] cooperative cancellation through Runtime into supported drivers; - [x] busy responses identify the owning operation when available; -- [x] bounded in-memory operation history with `completed` / `cancelled` / `faulted` execution states; +- [x] bounded in-memory operation history with `completed` / `cancelled` / `faulted` / `deadline_exceeded` execution states; - [x] bounded mutation evidence keyed by operation ID; - [x] non-mutating observation IDs/history/cancellation/evidence for serial operations; - [x] bounded serial failure windows sourced from the driver's local line buffer; - [x] bounded cross-operation target context (power/current) correlated into flash/reset and boot-wait failures; +- [x] Runtime-owned execution deadlines for long mutations/observations, distinct from caller cancellation and device/semantic timeouts; +- [x] active/history/evidence/API/CLI/MCP deadline metadata with stable `deadline_exceeded` classification; +- [x] late success from a driver that ignores cancellation is rejected after a Runtime deadline expires; - [x] explicit normal shutdown versus emergency shutdown semantics; -- [x] emergency power-off bypasses mutation gates, is non-cancellable once accepted, and is audited; +- [x] emergency power-off bypasses mutation gates, is non-cancellable once accepted, has no Runtime deadline, and is audited; - [x] destructive flash/reset confirmation policy; - [x] maximum voltage/current bench safety enforcement; -- [x] stable validation / not-found / busy / cancelled / runtime-state API error classes; +- [x] stable validation / not-found / busy / cancelled / deadline-exceeded / runtime-state API error classes; - [x] graceful host shutdown requests cancellation and drains Runtime-owned active work before releasing hardware resources; +Runtime deadline semantics are intentionally separate from protocol/device timing. For example, `serial wait --timeout-ms` is the semantic observation window and may return a normal unmatched assertion, while `--deadline-ms` is the outer Runtime execution budget and produces `deadline_exceeded` when exhausted. Device drivers may still enforce narrower hardware/tool-specific timeouts internally. + Still intentionally incomplete: - [ ] richer device/runtime error taxonomy for vendor-specific failures without leaking vendor SDK types into Core; -- [ ] one Runtime-level deadline/timeout model across all long operations (drivers already enforce bounded device timeouts where required); - [ ] persistent evidence/artifact storage beyond the current bounded in-memory Runtime stores; - [ ] remote/team leases — local mutation locks are **not** a substitute for authenticated remote ownership. diff --git a/scripts/smoke-runtime.sh b/scripts/smoke-runtime.sh index 9cac1be..c3e538f 100644 --- a/scripts/smoke-runtime.sh +++ b/scripts/smoke-runtime.sh @@ -105,6 +105,46 @@ assert "capturedAtUtc" in ctx["metadata"] assert "ageMs" in ctx["metadata"] ' +# Runtime deadlines are not semantic serial timeouts. Make the execution budget +# much shorter than the wait window and require the distinct CLI/API/history/ +# evidence classification all the way through the resident daemon. +set +e +deadline_json="$(cli serial wait __BENCHPILOT_DEADLINE__ --timeout-ms 5000 --deadline-ms 100 --json)" +deadline_code=$? +set -e +printf '%s\n' "$deadline_json" +if [[ $deadline_code -ne 6 ]]; then + echo "expected Runtime deadline to return exit code 6, got $deadline_code" >&2 + exit 1 +fi +printf '%s' "$deadline_json" | python3 -c ' +import json,sys +r=json.load(sys.stdin) +assert r["ok"] is False +assert r["code"] == "deadline_exceeded" +assert r["deadlineMs"] == 100 +assert r.get("deadlineAtUtc") +' + +deadline_history="$(cli observe history --limit 1 --json)" +printf '%s\n' "$deadline_history" +deadline_observation_id="$(printf '%s' "$deadline_history" | python3 -c ' +import json,sys +r=json.load(sys.stdin)["observations"][0] +assert r["state"] == "deadline_exceeded" +assert r.get("deadlineAtUtc") +print(r["id"]) +')" +deadline_evidence="$(cli observe evidence "$deadline_observation_id" --json)" +printf '%s\n' "$deadline_evidence" +printf '%s' "$deadline_evidence" | python3 -c ' +import json,sys +r=json.load(sys.stdin) +item=next(x for x in r["items"] if x["kind"] == "runtime.deadline") +assert item["metadata"]["deadlineMs"] == "100" +assert item["metadata"].get("deadlineAtUtc") +' + cli power check --lt-ma 100 --json cli power off --json # Completed mutations and observations remain available as bounded Runtime audit trails. diff --git a/src/Benchpilot.Cli/Program.cs b/src/Benchpilot.Cli/Program.cs index d8a742f..facaab6 100644 --- a/src/Benchpilot.Cli/Program.cs +++ b/src/Benchpilot.Cli/Program.cs @@ -39,6 +39,7 @@ public static async Task Run(string[] args) { using var client = new BenchClient(BenchClient.ResolveEndpoint(parsed.Get("endpoint"))); var target = parsed.Get("target"); + var deadlineMs = parsed.GetNullableInt("deadline-ms"); var command = parsed.Positionals[0].ToLowerInvariant(); var subcommand = parsed.Positionals.Count > 1 ? parsed.Positionals[1].ToLowerInvariant() @@ -134,14 +135,14 @@ public static async Task Run(string[] args) { var voltage = parsed.GetDouble("voltage", 12); var settleMs = parsed.GetInt("settle-ms", 2000); - var result = await client.PowerOn(voltage, settleMs, target, cts.Token); + var result = await client.PowerOn(voltage, settleMs, target, deadlineMs, cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } case ("power", "off"): { - var result = await client.PowerOff(target, cts.Token); + var result = await client.PowerOff(target, deadlineMs, cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -175,7 +176,12 @@ public static async Task Run(string[] args) { var firmware = RequirePositional(parsed, 2, "firmware path"); var confirmTarget = parsed.Get("confirm-target"); - var result = await client.Flash(firmware, target, confirmTarget, cts.Token); + var result = await client.Flash( + firmware, + target, + confirmTarget, + deadlineMs, + cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -183,7 +189,7 @@ public static async Task Run(string[] args) case ("flash", "reset"): { var confirmTarget = parsed.Get("confirm-target"); - var result = await client.Reset(target, confirmTarget, cts.Token); + var result = await client.Reset(target, confirmTarget, deadlineMs, cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -192,7 +198,7 @@ public static async Task Run(string[] args) { var port = parsed.Get("port"); var baud = parsed.GetNullableInt("baud"); - var result = await client.SerialOpen(port, baud, target, cts.Token); + var result = await client.SerialOpen(port, baud, target, deadlineMs, cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -201,7 +207,12 @@ public static async Task Run(string[] args) { var pattern = RequirePositional(parsed, 2, "pattern"); var timeoutMs = parsed.GetInt("timeout-ms", 10000); - var result = await client.SerialWaitFor(pattern, timeoutMs, target, cts.Token); + var result = await client.SerialWaitFor( + pattern, + timeoutMs, + target, + deadlineMs, + cts.Token); Print(result, parsed.Json); if (!result.Ok) return 4; return result.Matched ? 0 : 1; @@ -211,7 +222,12 @@ public static async Task Run(string[] args) { var lines = parsed.GetInt("lines", 50); var filter = parsed.Get("filter"); - var result = await client.SerialReadWindow(lines, filter, target, cts.Token); + var result = await client.SerialReadWindow( + lines, + filter, + target, + deadlineMs, + cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -219,7 +235,7 @@ public static async Task Run(string[] args) case ("serial", "send"): { var data = RequirePositional(parsed, 2, "data"); - var result = await client.SerialSend(data, target, cts.Token); + var result = await client.SerialSend(data, target, deadlineMs, cts.Token); Print(result, parsed.Json); return result.Ok ? 0 : 4; } @@ -237,10 +253,13 @@ public static async Task Run(string[] args) ex.Message, ex.OperationId, ex.BusyScope, - ex.BusyId), parsed.Json); + ex.BusyId, + ex.DeadlineMs, + ex.DeadlineAtUtc), parsed.Json); return ex.Code switch { "busy" => 5, + "deadline_exceeded" => 6, "cancelled" => 1, "runtime_state" => 4, _ => ex.StatusCode switch @@ -301,27 +320,27 @@ benchpilot history [--limit N] [--json] [--endpoint URL] benchpilot evidence [--json] [--endpoint URL] benchpilot cancel [--json] [--endpoint URL] - benchpilot observe list [--json] [--endpoint URL] - benchpilot observe history [--limit N] [--json] [--endpoint URL] + benchpilot observe list [--json] [--endpoint URL] + benchpilot observe history [--limit N] [--json] [--endpoint URL] benchpilot observe evidence [--json] [--endpoint URL] - benchpilot observe cancel [--json] [--endpoint URL] + benchpilot observe cancel [--json] [--endpoint URL] benchpilot preflight [--target ID] [--json] benchpilot bench validate [--target ID] [--json] - benchpilot power on [--target ID] [--voltage V] [--settle-ms N] [--json] - benchpilot power off [--target ID] [--json] + benchpilot power on [--target ID] [--voltage V] [--settle-ms N] [--deadline-ms N] [--json] + benchpilot power off [--target ID] [--deadline-ms N] [--json] benchpilot power emergency-off [--target ID] [--json] benchpilot power current [--target ID] [--window-ms N] [--json] benchpilot power check [--target ID] [--lt-ma N] [--gt-ma N] [--json] - benchpilot flash write [--target ID] [--confirm-target ID] [--json] - benchpilot flash reset [--target ID] [--confirm-target ID] [--json] + benchpilot flash write [--target ID] [--confirm-target ID] [--deadline-ms N] [--json] + benchpilot flash reset [--target ID] [--confirm-target ID] [--deadline-ms N] [--json] - benchpilot serial open [--target ID] [--port NAME] [--baud N] [--json] - benchpilot serial wait [--target ID] [--timeout-ms N] [--json] - benchpilot serial window [--target ID] [--lines N] [--filter TEXT] [--json] - benchpilot serial send [--target ID] [--json] + benchpilot serial open [--target ID] [--port NAME] [--baud N] [--deadline-ms N] [--json] + benchpilot serial wait [--target ID] [--timeout-ms N] [--deadline-ms N] [--json] + benchpilot serial window [--target ID] [--lines N] [--filter TEXT] [--deadline-ms N] [--json] + benchpilot serial send [--target ID] [--deadline-ms N] [--json] Mutating operations and observations are intentionally separate. `operations` uses target/resource gates for state-changing work. `observe ...` reports @@ -333,6 +352,13 @@ compact mutation evidence bundle. `observe history/evidence` provide the same A failed/unmatched serial wait captures only a small tail of the Runtime-owned serial line buffer, never the unbounded raw stream. +`--deadline-ms` is a Runtime execution budget for supported mutation/observation +operations. It is different from `serial wait --timeout-ms`: the latter is a +semantic wait window and a normal unmatched assertion returns exit code 1; +exceeding the Runtime deadline is an execution failure with code +`deadline_exceeded` and exit code 6. Emergency power-off deliberately ignores +Runtime deadlines so an accepted safety action cannot be abandoned by a shell. + `preflight` is non-destructive. It checks configured resource readiness without power-cycling, resetting or flashing the target. @@ -363,6 +389,7 @@ 2 validation error 3 target/resource/operation/observation/evidence not found 4 runtime/device unavailable or device/preflight error 5 target/resource busy (another mutating operation is active) + 6 Runtime deadline exceeded """); } } @@ -454,4 +481,4 @@ public double GetDouble(string name, double defaultValue) ? parsed : throw new FormatException($"Option --{name} must be a number."); } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Client/BenchClient.cs b/src/Benchpilot.Client/BenchClient.cs index c6b0f64..c1bb4a0 100644 --- a/src/Benchpilot.Client/BenchClient.cs +++ b/src/Benchpilot.Client/BenchClient.cs @@ -14,7 +14,9 @@ public BenchClientException( string message, string? operationId = null, string? busyScope = null, - string? busyId = null) + string? busyId = null, + int? deadlineMs = null, + DateTimeOffset? deadlineAtUtc = null) : base(message) { StatusCode = statusCode; @@ -22,6 +24,8 @@ public BenchClientException( OperationId = operationId; BusyScope = busyScope; BusyId = busyId; + DeadlineMs = deadlineMs; + DeadlineAtUtc = deadlineAtUtc; } public HttpStatusCode StatusCode { get; } @@ -29,6 +33,8 @@ public BenchClientException( public string? OperationId { get; } public string? BusyScope { get; } public string? BusyId { get; } + public int? DeadlineMs { get; } + public DateTimeOffset? DeadlineAtUtc { get; } } /// @@ -52,6 +58,8 @@ public BenchClient(Uri endpoint, HttpClient? httpClient = null) _ownsClient = httpClient is null; _http = httpClient ?? new HttpClient(); _http.BaseAddress = EnsureTrailingSlash(endpoint); + // Runtime deadlines, not HttpClient, own execution budgets. This keeps + // timeout classification/history/evidence deterministic across shells. _http.Timeout = Timeout.InfiniteTimeSpan; } @@ -146,34 +154,48 @@ public Task CancelObservation( public Task Preflight( string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/preflight", target), null, ct); + Send(HttpMethod.Post, WithExecutionOptions("api/v1/preflight", target, null), null, ct); public Task ValidateTargetReadiness( string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/validate", target), null, ct); + Send(HttpMethod.Post, WithExecutionOptions("api/v1/validate", target, null), null, ct); public Task PowerOn( double voltage, int settleMs, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/power/on", target), + PowerOn(voltage, settleMs, target, null, ct); + + public Task PowerOn( + double voltage, + int settleMs, + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/power/on", target, deadlineMs), new PowerOnRequest(voltage, settleMs), ct); public Task PowerOff(string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/power/off", target), null, ct); + PowerOff(target, null, ct); + + public Task PowerOff( + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/power/off", target, deadlineMs), null, ct); public Task EmergencyPowerOff( string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/power/emergency-off", target), null, ct); + Send(HttpMethod.Post, WithExecutionOptions("api/v1/power/emergency-off", target, null), null, ct); public Task ReadCurrent( int windowMs, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/power/current/read", target), + Send(HttpMethod.Post, WithExecutionOptions("api/v1/power/current/read", target, null), new CurrentReadRequest(windowMs), ct); public Task CheckCurrent( @@ -181,31 +203,46 @@ public Task CheckCurrent( double? gtMa, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/power/current/check", target), + Send(HttpMethod.Post, WithExecutionOptions("api/v1/power/current/check", target, null), new CurrentCheckRequest(ltMa, gtMa), ct); public Task Flash( string firmware, string? target = null, CancellationToken ct = default) => - Flash(firmware, target, null, ct); + Flash(firmware, target, null, null, ct); public Task Flash( string firmware, string? target, string? confirmTarget, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/flash/write", target), + Flash(firmware, target, confirmTarget, null, ct); + + public Task Flash( + string firmware, + string? target, + string? confirmTarget, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/flash/write", target, deadlineMs), new FlashRequest(firmware, confirmTarget), ct); public Task Reset(string? target = null, CancellationToken ct = default) => - Reset(target, null, ct); + Reset(target, null, null, ct); public Task Reset( string? target, string? confirmTarget, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/flash/reset", target), + Reset(target, confirmTarget, null, ct); + + public Task Reset( + string? target, + string? confirmTarget, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/flash/reset", target, deadlineMs), new ResetRequest(confirmTarget), ct); public Task SerialOpen( @@ -213,7 +250,15 @@ public Task SerialOpen( int? baud = null, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/serial/open", target), + SerialOpen(port, baud, target, null, ct); + + public Task SerialOpen( + string? port, + int? baud, + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/serial/open", target, deadlineMs), new SerialOpenRequest(port, baud), ct); public Task SerialWaitFor( @@ -221,7 +266,15 @@ public Task SerialWaitFor( int timeoutMs, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/serial/wait", target), + SerialWaitFor(pattern, timeoutMs, target, null, ct); + + public Task SerialWaitFor( + string pattern, + int timeoutMs, + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/serial/wait", target, deadlineMs), new SerialWaitRequest(pattern, timeoutMs), ct); public Task SerialReadWindow( @@ -229,14 +282,29 @@ public Task SerialReadWindow( string? filter, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/serial/window", target), + SerialReadWindow(lines, filter, target, null, ct); + + public Task SerialReadWindow( + int lines, + string? filter, + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/serial/window", target, deadlineMs), new SerialWindowRequest(lines, filter), ct); public Task SerialSend( string data, string? target = null, CancellationToken ct = default) => - Send(HttpMethod.Post, WithTarget("api/v1/serial/send", target), + SerialSend(data, target, null, ct); + + public Task SerialSend( + string data, + string? target, + int? deadlineMs, + CancellationToken ct = default) => + Send(HttpMethod.Post, WithExecutionOptions("api/v1/serial/send", target, deadlineMs), new SerialSendRequest(data), ct); private async Task Send( @@ -270,7 +338,9 @@ private async Task Send( apiError?.Error ?? $"BenchPilot runtime returned HTTP {(int)response.StatusCode}.", apiError?.OperationId, apiError?.BusyScope, - apiError?.BusyId); + apiError?.BusyId, + apiError?.DeadlineMs, + apiError?.DeadlineAtUtc); } var value = await response.Content.ReadFromJsonAsync(JsonOptions, ct); @@ -280,10 +350,21 @@ private async Task Send( "BenchPilot runtime returned an empty or invalid JSON response."); } - private static string WithTarget(string path, string? target) => - string.IsNullOrWhiteSpace(target) + private static string WithExecutionOptions( + string path, + string? target, + int? deadlineMs) + { + var query = new List(2); + if (!string.IsNullOrWhiteSpace(target)) + query.Add($"target={Uri.EscapeDataString(target)}"); + if (deadlineMs is { } value) + query.Add($"deadlineMs={value}"); + + return query.Count == 0 ? path - : $"{path}?target={Uri.EscapeDataString(target)}"; + : $"{path}?{string.Join("&", query)}"; + } private static Uri EnsureTrailingSlash(Uri endpoint) => endpoint.AbsoluteUri.EndsWith("/", StringComparison.Ordinal) @@ -295,4 +376,4 @@ public void Dispose() if (_ownsClient) _http.Dispose(); } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Mcp/Tools/FlashTools.cs b/src/Benchpilot.Mcp/Tools/FlashTools.cs index c7cdf3d..9085b5b 100644 --- a/src/Benchpilot.Mcp/Tools/FlashTools.cs +++ b/src/Benchpilot.Mcp/Tools/FlashTools.cs @@ -17,13 +17,15 @@ internal sealed class FlashTools public async Task Flash( [Description("Path to the firmware image, e.g. build/app.elf")] string firmware = "build/app.elf", [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, - [Description("Explicit destructive-operation confirmation. When required by policy, this must exactly match the resolved target id.")] string? confirmTarget = null) - => await _client.Flash(firmware, target, confirmTarget, CancellationToken.None); + [Description("Explicit destructive-operation confirmation. When required by policy, this must exactly match the resolved target id.")] string? confirmTarget = null, + [Description("Optional Runtime execution deadline in milliseconds. The Runtime records deadline_exceeded separately from caller cancellation and driver/device errors.")] int? deadlineMs = null) + => await _client.Flash(firmware, target, confirmTarget, deadlineMs, CancellationToken.None); [McpServerTool] [Description("Reset the target through its configured flash/debug capability. Profiles may require confirmTarget to exactly match the semantic target id.")] public async Task Reset( [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, - [Description("Explicit destructive-operation confirmation. When required by policy, this must exactly match the resolved target id.")] string? confirmTarget = null) - => await _client.Reset(target, confirmTarget, CancellationToken.None); -} + [Description("Explicit destructive-operation confirmation. When required by policy, this must exactly match the resolved target id.")] string? confirmTarget = null, + [Description("Optional Runtime execution deadline in milliseconds.")] int? deadlineMs = null) + => await _client.Reset(target, confirmTarget, deadlineMs, CancellationToken.None); +} \ No newline at end of file diff --git a/src/Benchpilot.Mcp/Tools/PowerTools.cs b/src/Benchpilot.Mcp/Tools/PowerTools.cs index d10ca39..4482533 100644 --- a/src/Benchpilot.Mcp/Tools/PowerTools.cs +++ b/src/Benchpilot.Mcp/Tools/PowerTools.cs @@ -17,17 +17,19 @@ internal sealed class PowerTools public async Task PowerOn( [Description("Supply voltage in volts, e.g. 12")] double voltage = 12, [Description("Settle window in milliseconds before reporting current")] int settleMs = 2000, - [Description("Semantic target id, e.g. 'radar'. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.PowerOn(voltage, settleMs, target, CancellationToken.None); + [Description("Semantic target id, e.g. 'radar'. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds. Distinct from device-specific timeouts.")] int? deadlineMs = null) + => await _client.PowerOn(voltage, settleMs, target, deadlineMs, CancellationToken.None); [McpServerTool] [Description("Switch off the target's bench supply. This normal shutdown respects the target mutation gate and will not interrupt an active flash/reset.")] public async Task PowerOff( - [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.PowerOff(target, CancellationToken.None); + [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds.")] int? deadlineMs = null) + => await _client.PowerOff(target, deadlineMs, CancellationToken.None); [McpServerTool] - [Description("Emergency safety shutdown. Switch off target power even when another mutating operation is active. Once Runtime accepts this safety action it is not cancellable by the caller, and the action is recorded in operation history. Use only when leaving the bench energized is more dangerous than interrupting the active operation.")] + [Description("Emergency safety shutdown. Switch off target power even when another mutating operation is active. Once Runtime accepts this safety action it is not cancellable by the caller and has no Runtime deadline. Use only when leaving the bench energized is more dangerous than interrupting the active operation.")] public async Task EmergencyPowerOff( [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) => await _client.EmergencyPowerOff(target, CancellationToken.None); @@ -46,4 +48,4 @@ public async Task CheckCurrent( [Description("Pass if current (mA) is above this")] double? gt = null, [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) => await _client.CheckCurrent(lt, gt, target, CancellationToken.None); -} +} \ No newline at end of file diff --git a/src/Benchpilot.Mcp/Tools/SerialTools.cs b/src/Benchpilot.Mcp/Tools/SerialTools.cs index b231952..aa747f3 100644 --- a/src/Benchpilot.Mcp/Tools/SerialTools.cs +++ b/src/Benchpilot.Mcp/Tools/SerialTools.cs @@ -17,29 +17,33 @@ internal sealed class SerialTools public async Task SerialOpen( [Description("Optional OS serial-port override. Omit to use the target resource profile.")] string? port = null, [Description("Optional baud-rate override. Omit to use the target resource profile.")] int? baud = null, - [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.SerialOpen(port, baud, target, CancellationToken.None); + [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds.")] int? deadlineMs = null) + => await _client.SerialOpen(port, baud, target, deadlineMs, CancellationToken.None); [McpServerTool] - [Description("Wait until a line containing the pattern appears on a target console, or until timeout. Returns only the matched event instead of an unbounded byte stream.")] + [Description("Wait until a line containing the pattern appears on a target console, or until timeout. Returns only the matched event instead of an unbounded byte stream. timeoutMs is the semantic wait window; deadlineMs is a separate Runtime execution budget.")] public async Task SerialWaitFor( [Description("Substring to wait for (case-insensitive), e.g. 'Ready'")] string pattern, - [Description("Timeout in milliseconds")] int timeoutMs = 10000, - [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.SerialWaitFor(pattern, timeoutMs, target, CancellationToken.None); + [Description("Semantic wait timeout in milliseconds. Expiry returns a normal unmatched result.")] int timeoutMs = 10000, + [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds. Expiry is reported as deadline_exceeded, not as an unmatched serial assertion.")] int? deadlineMs = null) + => await _client.SerialWaitFor(pattern, timeoutMs, target, deadlineMs, CancellationToken.None); [McpServerTool] [Description("Read a bounded trailing window from a target console, optionally filtered by a substring.")] public async Task SerialReadWindow( [Description("Number of trailing lines to return")] int lines = 50, [Description("Optional substring filter (case-insensitive)")] string? filter = null, - [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.SerialReadWindow(lines, filter, target, CancellationToken.None); + [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds.")] int? deadlineMs = null) + => await _client.SerialReadWindow(lines, filter, target, deadlineMs, CancellationToken.None); [McpServerTool] [Description("Send a line of text to a target serial console.")] public async Task SerialSend( [Description("Text to send")] string data, - [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null) - => await _client.SerialSend(data, target, CancellationToken.None); -} + [Description("Semantic target id. Omit to use defaultTarget when allowed.")] string? target = null, + [Description("Optional Runtime execution deadline in milliseconds.")] int? deadlineMs = null) + => await _client.SerialSend(data, target, deadlineMs, CancellationToken.None); +} \ No newline at end of file diff --git a/src/Benchpilot.Protocol/ApiModels.cs b/src/Benchpilot.Protocol/ApiModels.cs index 598c578..0f8bf0e 100644 --- a/src/Benchpilot.Protocol/ApiModels.cs +++ b/src/Benchpilot.Protocol/ApiModels.cs @@ -12,7 +12,9 @@ public record ApiError( string Error, string? OperationId = null, string? BusyScope = null, - string? BusyId = null); + string? BusyId = null, + int? DeadlineMs = null, + DateTimeOffset? DeadlineAtUtc = null); public record TargetSummary( string Id, @@ -41,7 +43,9 @@ public record OperationSummary( string Kind, IReadOnlyList ResourceIds, DateTimeOffset StartedAtUtc, - bool CancellationRequested); + DateTimeOffset? DeadlineAtUtc, + bool CancellationRequested, + bool DeadlineExceeded); public record OperationListResult( bool Ok, @@ -62,6 +66,7 @@ public record OperationHistorySummary( DateTimeOffset StartedAtUtc, DateTimeOffset CompletedAtUtc, int DurationMs, + DateTimeOffset? DeadlineAtUtc, string State, string? Error = null); @@ -76,7 +81,9 @@ public record ObservationSummary( string Kind, IReadOnlyList ResourceIds, DateTimeOffset StartedAtUtc, - bool CancellationRequested); + DateTimeOffset? DeadlineAtUtc, + bool CancellationRequested, + bool DeadlineExceeded); public record ObservationListResult( bool Ok, @@ -97,6 +104,7 @@ public record ObservationHistorySummary( DateTimeOffset StartedAtUtc, DateTimeOffset CompletedAtUtc, int DurationMs, + DateTimeOffset? DeadlineAtUtc, string State, string? Error = null); @@ -141,4 +149,4 @@ public record ResetRequest(string? ConfirmTarget = null); public record SerialOpenRequest(string? Port = null, int? Baud = null); public record SerialWaitRequest(string Pattern, int TimeoutMs = 10000); public record SerialWindowRequest(int Lines = 50, string? Filter = null); -public record SerialSendRequest(string Data); +public record SerialSendRequest(string Data); \ No newline at end of file diff --git a/src/Benchpilot.Runtime/BenchRuntime.cs b/src/Benchpilot.Runtime/BenchRuntime.cs index ed98cd8..4356bc7 100644 --- a/src/Benchpilot.Runtime/BenchRuntime.cs +++ b/src/Benchpilot.Runtime/BenchRuntime.cs @@ -158,13 +158,15 @@ internal async Task RunMutation( string operation, IReadOnlyCollection resourceIds, Func> action, - CancellationToken requestCancellation) + CancellationToken requestCancellation, + int? deadlineMs = null) { ObjectDisposedException.ThrowIf(_disposed, this); ArgumentException.ThrowIfNullOrWhiteSpace(targetId); ArgumentException.ThrowIfNullOrWhiteSpace(operation); ArgumentNullException.ThrowIfNull(resourceIds); ArgumentNullException.ThrowIfNull(action); + ValidateDeadline(deadlineMs); requestCancellation.ThrowIfCancellationRequested(); var normalizedResources = resourceIds @@ -219,7 +221,8 @@ internal async Task RunMutation( operation, normalizedResources, DateTimeOffset.UtcNow, - requestCancellation); + requestCancellation, + deadlineMs); if (!_activeOperations.TryAdd(operationId, active)) throw new InvalidOperationException($"Could not register active operation '{operationId}'."); @@ -227,10 +230,27 @@ internal async Task RunMutation( try { var result = await action(active.Token); + // Preserve the existing compatibility contract for a driver that + // ignores caller/drain cancellation and eventually returns. A + // Runtime deadline is different: once its wall-clock budget has + // expired, a late success is never accepted. + if (active.DeadlineExceeded) + throw new OperationCanceledException(active.Token); RecordEvidence(active, OperationEvidenceExtractor.FromResult(result)); RecordOperation(active, "completed", null); return result; } + catch (OperationCanceledException) when (active.DeadlineExceeded) + { + var deadline = active.CreateDeadlineException(); + RecordEvidence( + active, + OperationEvidenceExtractor.FromDeadline( + deadline.DeadlineMs, + deadline.DeadlineAtUtc)); + RecordOperation(active, "deadline_exceeded", deadline.Message); + throw deadline; + } catch (OperationCanceledException) { RecordEvidence(active, OperationEvidenceExtractor.FromCancellation()); @@ -260,21 +280,23 @@ internal async Task RunMutation( /// /// Runs a non-mutating observation without taking target/resource mutation /// gates. Observations can therefore run alongside flash/power operations, - /// while still receiving identity, cancellation, history and bounded - /// evidence owned by the resident Runtime. + /// while still receiving identity, cancellation, deadline, history and + /// bounded evidence owned by the resident Runtime. /// internal async Task RunObservation( string targetId, string observation, IReadOnlyCollection resourceIds, Func>> action, - CancellationToken requestCancellation) + CancellationToken requestCancellation, + int? deadlineMs = null) { ObjectDisposedException.ThrowIf(_disposed, this); ArgumentException.ThrowIfNullOrWhiteSpace(targetId); ArgumentException.ThrowIfNullOrWhiteSpace(observation); ArgumentNullException.ThrowIfNull(resourceIds); ArgumentNullException.ThrowIfNull(action); + ValidateDeadline(deadlineMs); requestCancellation.ThrowIfCancellationRequested(); var normalizedResources = resourceIds @@ -289,7 +311,8 @@ internal async Task RunObservation( observation, normalizedResources, DateTimeOffset.UtcNow, - requestCancellation); + requestCancellation, + deadlineMs); if (!_activeObservations.TryAdd(observationId, active)) { @@ -300,10 +323,23 @@ internal async Task RunObservation( try { var execution = await action(observationId, active.Token); + if (active.DeadlineExceeded) + throw new OperationCanceledException(active.Token); RecordObservationEvidence(active, execution.Evidence); RecordObservation(active, "completed", null); return execution.Result; } + catch (OperationCanceledException) when (active.DeadlineExceeded) + { + var deadline = active.CreateDeadlineException(); + RecordObservationEvidence( + active, + SerialObservationEvidenceExtractor.FromDeadline( + deadline.DeadlineMs, + deadline.DeadlineAtUtc)); + RecordObservation(active, "deadline_exceeded", deadline.Message); + throw deadline; + } catch (OperationCanceledException) { RecordObservationEvidence(active, SerialObservationEvidenceExtractor.FromCancellation()); @@ -327,6 +363,7 @@ internal async Task RunObservation( /// Runs a safety action without taking target/resource mutation gates. The /// action is intentionally not cancellable once accepted, but it is still /// assigned an operation id and written to bounded audit/evidence stores. + /// Emergency safety actions deliberately do not accept Runtime deadlines. /// internal async Task RunUngatedSafetyOperation( string targetId, @@ -363,6 +400,7 @@ internal async Task RunUngatedSafetyOperation( operation, normalizedResources, startedAt, + null, "completed", null); return result; @@ -381,6 +419,7 @@ internal async Task RunUngatedSafetyOperation( operation, normalizedResources, startedAt, + null, "faulted", BoundHistoryError(ex.Message)); throw; @@ -443,6 +482,7 @@ private void RecordOperation(ActiveMutation active, string state, string? error) active.Kind, active.ResourceIds, active.StartedAtUtc, + active.DeadlineAtUtc, state, error); @@ -452,6 +492,7 @@ private void RecordOperation( string kind, IReadOnlyList resourceIds, DateTimeOffset startedAt, + DateTimeOffset? deadlineAtUtc, string state, string? error) { @@ -465,6 +506,7 @@ private void RecordOperation( startedAt, completedAt, durationMs, + deadlineAtUtc, state, error); @@ -487,6 +529,7 @@ private void RecordObservation(ActiveObservation active, string state, string? e active.StartedAtUtc, completedAt, DurationMs(active.StartedAtUtc, completedAt), + active.DeadlineAtUtc, state, error); @@ -498,6 +541,12 @@ private void RecordObservation(ActiveObservation active, string state, string? e } } + private static void ValidateDeadline(int? deadlineMs) + { + if (deadlineMs is <= 0) + throw new BenchValidationException("Runtime deadlineMs must be greater than zero."); + } + private static int DurationMs(DateTimeOffset startedAt, DateTimeOffset completedAt) => (int)Math.Min( int.MaxValue, @@ -539,9 +588,7 @@ private sealed record MutationGateRequest(string Key, string Scope, string Id); private sealed class ActiveMutation : IDisposable { - private readonly object _sync = new(); - private CancellationTokenSource? _cancellation; - private bool _cancellationRequested; + private readonly RuntimeExecutionCancellation _cancellation; public ActiveMutation( string id, @@ -549,14 +596,18 @@ public ActiveMutation( string kind, IReadOnlyList resourceIds, DateTimeOffset startedAtUtc, - CancellationToken requestCancellation) + CancellationToken requestCancellation, + int? deadlineMs) { Id = id; TargetId = targetId; Kind = kind; ResourceIds = resourceIds; StartedAtUtc = startedAtUtc; - _cancellation = CancellationTokenSource.CreateLinkedTokenSource(requestCancellation); + _cancellation = new RuntimeExecutionCancellation( + requestCancellation, + deadlineMs, + startedAtUtc); } public string Id { get; } @@ -564,73 +615,37 @@ public ActiveMutation( public string Kind { get; } public IReadOnlyList ResourceIds { get; } public DateTimeOffset StartedAtUtc { get; } - - public CancellationToken Token - { - get - { - lock (_sync) - { - return _cancellation?.Token - ?? new CancellationToken(canceled: true); - } - } - } - - public BenchOperationInfo Snapshot() - { - lock (_sync) - { - return new BenchOperationInfo( - Id, - TargetId, - Kind, - ResourceIds, - StartedAtUtc, - _cancellationRequested || (_cancellation?.IsCancellationRequested ?? true)); - } - } - - public bool RequestCancel() - { - CancellationTokenSource? cancellation; - lock (_sync) - { - if (_cancellation is null) - return false; - - _cancellationRequested = true; - cancellation = _cancellation; - } - - try - { - cancellation.Cancel(); - return true; - } - catch (ObjectDisposedException) - { - return false; - } - } - - public void Dispose() - { - CancellationTokenSource? cancellation; - lock (_sync) - { - cancellation = _cancellation; - _cancellation = null; - } - cancellation?.Dispose(); - } + public int? DeadlineMs => _cancellation.DeadlineMs; + public DateTimeOffset? DeadlineAtUtc => _cancellation.DeadlineAtUtc; + public bool DeadlineExceeded => _cancellation.DeadlineExceeded; + public CancellationToken Token => _cancellation.Token; + + public BenchOperationInfo Snapshot() => + new( + Id, + TargetId, + Kind, + ResourceIds, + StartedAtUtc, + DeadlineAtUtc, + _cancellation.CancellationRequested, + DeadlineExceeded); + + public bool RequestCancel() => _cancellation.RequestCancel(); + + public BenchDeadlineExceededException CreateDeadlineException() => + new( + TargetId, + Kind, + DeadlineMs ?? throw new InvalidOperationException("Deadline metadata is unavailable."), + DeadlineAtUtc ?? throw new InvalidOperationException("Deadline metadata is unavailable.")); + + public void Dispose() => _cancellation.Dispose(); } private sealed class ActiveObservation : IDisposable { - private readonly object _sync = new(); - private CancellationTokenSource? _cancellation; - private bool _cancellationRequested; + private readonly RuntimeExecutionCancellation _cancellation; public ActiveObservation( string id, @@ -638,14 +653,18 @@ public ActiveObservation( string kind, IReadOnlyList resourceIds, DateTimeOffset startedAtUtc, - CancellationToken requestCancellation) + CancellationToken requestCancellation, + int? deadlineMs) { Id = id; TargetId = targetId; Kind = kind; ResourceIds = resourceIds; StartedAtUtc = startedAtUtc; - _cancellation = CancellationTokenSource.CreateLinkedTokenSource(requestCancellation); + _cancellation = new RuntimeExecutionCancellation( + requestCancellation, + deadlineMs, + startedAtUtc); } public string Id { get; } @@ -653,65 +672,31 @@ public ActiveObservation( public string Kind { get; } public IReadOnlyList ResourceIds { get; } public DateTimeOffset StartedAtUtc { get; } - - public CancellationToken Token - { - get - { - lock (_sync) - { - return _cancellation?.Token - ?? new CancellationToken(canceled: true); - } - } - } - - public BenchObservationInfo Snapshot() - { - lock (_sync) - { - return new BenchObservationInfo( - Id, - TargetId, - Kind, - ResourceIds, - StartedAtUtc, - _cancellationRequested || (_cancellation?.IsCancellationRequested ?? true)); - } - } - - public bool RequestCancel() - { - CancellationTokenSource? cancellation; - lock (_sync) - { - if (_cancellation is null) - return false; - - _cancellationRequested = true; - cancellation = _cancellation; - } - - try - { - cancellation.Cancel(); - return true; - } - catch (ObjectDisposedException) - { - return false; - } - } - - public void Dispose() - { - CancellationTokenSource? cancellation; - lock (_sync) - { - cancellation = _cancellation; - _cancellation = null; - } - cancellation?.Dispose(); - } + public int? DeadlineMs => _cancellation.DeadlineMs; + public DateTimeOffset? DeadlineAtUtc => _cancellation.DeadlineAtUtc; + public bool DeadlineExceeded => _cancellation.DeadlineExceeded; + public CancellationToken Token => _cancellation.Token; + + public BenchObservationInfo Snapshot() => + new( + Id, + TargetId, + Kind, + ResourceIds, + StartedAtUtc, + DeadlineAtUtc, + _cancellation.CancellationRequested, + DeadlineExceeded); + + public bool RequestCancel() => _cancellation.RequestCancel(); + + public BenchDeadlineExceededException CreateDeadlineException() => + new( + TargetId, + Kind, + DeadlineMs ?? throw new InvalidOperationException("Deadline metadata is unavailable."), + DeadlineAtUtc ?? throw new InvalidOperationException("Deadline metadata is unavailable.")); + + public void Dispose() => _cancellation.Dispose(); } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Runtime/BenchTarget.cs b/src/Benchpilot.Runtime/BenchTarget.cs index 1138b59..e558a35 100644 --- a/src/Benchpilot.Runtime/BenchTarget.cs +++ b/src/Benchpilot.Runtime/BenchTarget.cs @@ -30,9 +30,16 @@ public bool HasCapability(string capability) => public T Capability(string capability) where T : class => BoundCapability(capability).Capability; + public Task PowerOn( + double voltage, + int settleMs, + CancellationToken ct = default) => + PowerOn(voltage, settleMs, null, ct); + public async Task PowerOn( double voltage, int settleMs, + int? deadlineMs, CancellationToken ct = default) { if (voltage <= 0) @@ -65,7 +72,7 @@ _runtime.Profile.Safety.MaxCurrentMa is { } maxCurrentMa && } return value; - }, ct); + }, ct, deadlineMs); TargetContextEvidence.RecordPowerOn(_runtime, Id, result); return result; } @@ -75,7 +82,12 @@ _runtime.Profile.Safety.MaxCurrentMa is { } maxCurrentMa && /// mutation gates so it cannot interrupt an active flash/reset or another /// target currently using the same physical power supply. /// - public async Task PowerOff(CancellationToken ct = default) + public Task PowerOff(CancellationToken ct = default) => + PowerOff(null, ct); + + public async Task PowerOff( + int? deadlineMs, + CancellationToken ct = default) { var binding = BoundCapability("power"); var result = await _runtime.RunMutation( @@ -83,7 +95,8 @@ public async Task PowerOff(CancellationToken ct = default) "power.off", [binding.ResourceId], operationCt => binding.Capability.PowerOff(operationCt), - ct); + ct, + deadlineMs); TargetContextEvidence.RecordPowerOff(_runtime, Id, result); return result; } @@ -91,7 +104,8 @@ public async Task PowerOff(CancellationToken ct = default) /// /// Explicit safety escape hatch. It bypasses mutation gates, cannot be /// cancelled by a disconnected caller after Runtime accepts it, and is - /// always written to operation history for auditability. + /// always written to operation history for auditability. Emergency shutdown + /// deliberately has no Runtime deadline. /// public async Task EmergencyPowerOff(CancellationToken ct = default) { @@ -129,9 +143,16 @@ public async Task CheckCurrent( return result; } - public async Task Flash( + public Task Flash( string firmware, string? confirmTarget = null, + CancellationToken ct = default) => + Flash(firmware, confirmTarget, null, ct); + + public async Task Flash( + string firmware, + string? confirmTarget, + int? deadlineMs, CancellationToken ct = default) { if (string.IsNullOrWhiteSpace(firmware)) @@ -145,11 +166,18 @@ public async Task Flash( "flash.write", [binding.ResourceId], operationCt => binding.Capability.Flash(firmware, operationCt), - ct); + ct, + deadlineMs); } - public async Task Reset( + public Task Reset( string? confirmTarget = null, + CancellationToken ct = default) => + Reset(confirmTarget, null, ct); + + public async Task Reset( + string? confirmTarget, + int? deadlineMs, CancellationToken ct = default) { ValidateDestructiveConfirmation("reset", confirmTarget); @@ -161,12 +189,20 @@ public async Task Reset( "flash.reset", [binding.ResourceId], operationCt => binding.Capability.Reset(operationCt), - ct); + ct, + deadlineMs); } public Task SerialOpen( string? port = null, int? baud = null, + CancellationToken ct = default) => + SerialOpen(port, baud, null, ct); + + public Task SerialOpen( + string? port, + int? baud, + int? deadlineMs, CancellationToken ct = default) { if (port is not null && string.IsNullOrWhiteSpace(port)) @@ -187,12 +223,20 @@ public Task SerialOpen( identified, SerialObservationEvidenceExtractor.FromOpen(identified)); }, - ct); + ct, + deadlineMs); } public Task SerialWaitFor( string pattern, int timeoutMs, + CancellationToken ct = default) => + SerialWaitFor(pattern, timeoutMs, null, ct); + + public Task SerialWaitFor( + string pattern, + int timeoutMs, + int? deadlineMs, CancellationToken ct = default) { if (string.IsNullOrWhiteSpace(pattern)) @@ -237,12 +281,20 @@ public Task SerialWaitFor( return new ObservationExecution(identified, evidence); }, - ct); + ct, + deadlineMs); } public Task SerialReadWindow( int lines, string? filter = null, + CancellationToken ct = default) => + SerialReadWindow(lines, filter, null, ct); + + public Task SerialReadWindow( + int lines, + string? filter, + int? deadlineMs, CancellationToken ct = default) { if (lines <= 0) @@ -261,10 +313,17 @@ public Task SerialReadWindow( identified, SerialObservationEvidenceExtractor.FromWindow(identified)); }, - ct); + ct, + deadlineMs); } - public Task SerialSend(string data, CancellationToken ct = default) + public Task SerialSend(string data, CancellationToken ct = default) => + SerialSend(data, null, ct); + + public Task SerialSend( + string data, + int? deadlineMs, + CancellationToken ct = default) { if (data is null) throw new BenchValidationException("Serial data cannot be null."); @@ -282,7 +341,8 @@ public Task SerialSend(string data, CancellationToken ct = def identified, SerialObservationEvidenceExtractor.FromSend(identified, data.Length)); }, - ct); + ct, + deadlineMs); } private (string ResourceId, T Capability) BoundCapability(string capability) where T : class @@ -309,4 +369,4 @@ private void ValidateDestructiveConfirmation(string operation, string? confirmTa $"Destructive operation '{operation}' requires confirmTarget matching target id '{Id}'."); } } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Runtime/ObservationEvidence.cs b/src/Benchpilot.Runtime/ObservationEvidence.cs index 80cc5be..e30fd12 100644 --- a/src/Benchpilot.Runtime/ObservationEvidence.cs +++ b/src/Benchpilot.Runtime/ObservationEvidence.cs @@ -111,6 +111,20 @@ public static IReadOnlyList FromCancellation() => "Observation cancelled.") ]; + public static IReadOnlyList FromDeadline( + int deadlineMs, + DateTimeOffset deadlineAtUtc) => + [ + Item( + "runtime.deadline", + "Observation exceeded its Runtime deadline.", + metadata: new Dictionary + { + ["deadlineMs"] = deadlineMs.ToString(CultureInfo.InvariantCulture), + ["deadlineAtUtc"] = deadlineAtUtc.ToString("O", CultureInfo.InvariantCulture), + }) + ]; + public static IReadOnlyList FromException(Exception exception) { ArgumentNullException.ThrowIfNull(exception); @@ -228,4 +242,4 @@ private static BenchEvidenceItem BoundItem(BenchEvidenceItem item) if (string.IsNullOrEmpty(value)) return value; return value.Length <= maxLength ? value : value[..maxLength]; } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Runtime/ObservationTypes.cs b/src/Benchpilot.Runtime/ObservationTypes.cs index 358d108..9e036e7 100644 --- a/src/Benchpilot.Runtime/ObservationTypes.cs +++ b/src/Benchpilot.Runtime/ObservationTypes.cs @@ -6,7 +6,9 @@ public sealed record BenchObservationInfo( string Kind, IReadOnlyList ResourceIds, DateTimeOffset StartedAtUtc, - bool CancellationRequested); + DateTimeOffset? DeadlineAtUtc, + bool CancellationRequested, + bool DeadlineExceeded); public sealed record BenchObservationRecord( string Id, @@ -16,9 +18,10 @@ public sealed record BenchObservationRecord( DateTimeOffset StartedAtUtc, DateTimeOffset CompletedAtUtc, int DurationMs, + DateTimeOffset? DeadlineAtUtc, string State, string? Error = null); internal sealed record ObservationExecution( T Result, - IReadOnlyList Evidence); + IReadOnlyList Evidence); \ No newline at end of file diff --git a/src/Benchpilot.Runtime/OperationEvidence.cs b/src/Benchpilot.Runtime/OperationEvidence.cs index 1bd9f6a..26aec23 100644 --- a/src/Benchpilot.Runtime/OperationEvidence.cs +++ b/src/Benchpilot.Runtime/OperationEvidence.cs @@ -98,7 +98,7 @@ result is null // Explicit user/runtime cancellation is already self-explanatory and is // intentionally kept minimal. Cross-operation context is reserved for - // device-error or infrastructure-fault diagnosis. + // device-error, deadline or infrastructure-fault diagnosis. public static IReadOnlyList FromCancellation() => [ Item( @@ -107,6 +107,21 @@ public static IReadOnlyList FromCancellation() => "Operation cancelled.") ]; + public static IReadOnlyList FromDeadline( + int deadlineMs, + DateTimeOffset deadlineAtUtc) => + TargetContextEvidence.AppendFailureContext( + [ + Item( + "runtime.deadline", + "Operation exceeded its Runtime deadline.", + metadata: new Dictionary + { + ["deadlineMs"] = deadlineMs.ToString(CultureInfo.InvariantCulture), + ["deadlineAtUtc"] = deadlineAtUtc.ToString("O", CultureInfo.InvariantCulture), + }) + ]); + public static IReadOnlyList FromException(Exception exception) { ArgumentNullException.ThrowIfNull(exception); @@ -232,4 +247,4 @@ private static BenchEvidenceItem BoundItem(BenchEvidenceItem item) if (string.IsNullOrEmpty(value)) return value; return value.Length <= maxLength ? value : value[..maxLength]; } -} +} \ No newline at end of file diff --git a/src/Benchpilot.Runtime/RuntimeExecutionCancellation.cs b/src/Benchpilot.Runtime/RuntimeExecutionCancellation.cs new file mode 100644 index 0000000..b560df5 --- /dev/null +++ b/src/Benchpilot.Runtime/RuntimeExecutionCancellation.cs @@ -0,0 +1,112 @@ +namespace Benchpilot.Runtime; + +/// +/// Owns the three cancellation causes for one Runtime execution: caller/request +/// cancellation, explicit Runtime cancellation, and an optional operation +/// deadline. The first observed cause wins so history/API classification does +/// not depend on callback or thread scheduling order. +/// +internal sealed class RuntimeExecutionCancellation : IDisposable +{ + private const int CauseNone = 0; + private const int CauseCaller = 1; + private const int CauseExplicit = 2; + private const int CauseDeadline = 3; + + private readonly CancellationTokenSource _explicitCancellation = new(); + private readonly CancellationTokenSource? _deadlineCancellation; + private readonly CancellationTokenSource _linkedCancellation; + private readonly CancellationTokenRegistration _requestRegistration; + private readonly CancellationTokenRegistration _deadlineRegistration; + private int _cause; + private int _disposed; + + public RuntimeExecutionCancellation( + CancellationToken requestCancellation, + int? deadlineMs, + DateTimeOffset startedAtUtc) + { + DeadlineMs = deadlineMs; + DeadlineAtUtc = deadlineMs is { } value + ? startedAtUtc.AddMilliseconds(value) + : null; + + if (requestCancellation.IsCancellationRequested) + Interlocked.CompareExchange(ref _cause, CauseCaller, CauseNone); + + _requestRegistration = requestCancellation.Register( + static state => ((RuntimeExecutionCancellation)state!).MarkCause(CauseCaller), + this); + + if (deadlineMs is { } timeoutMs) + { + _deadlineCancellation = new CancellationTokenSource(timeoutMs); + _deadlineRegistration = _deadlineCancellation.Token.Register( + static state => ((RuntimeExecutionCancellation)state!).MarkCause(CauseDeadline), + this); + _linkedCancellation = CancellationTokenSource.CreateLinkedTokenSource( + requestCancellation, + _explicitCancellation.Token, + _deadlineCancellation.Token); + } + else + { + _linkedCancellation = CancellationTokenSource.CreateLinkedTokenSource( + requestCancellation, + _explicitCancellation.Token); + } + } + + public int? DeadlineMs { get; } + public DateTimeOffset? DeadlineAtUtc { get; } + public CancellationToken Token => _linkedCancellation.Token; + + public bool DeadlineExceeded + { + get + { + // Timer callbacks can be scheduled a little after their nominal due + // time. If a driver ignores cancellation and returns after the wall + // clock deadline, claim the deadline cause here before accepting the + // result. A caller/explicit cancellation that already won remains the + // cause because MarkCause is compare-exchange based. + if (DeadlineAtUtc is { } deadlineAtUtc && DateTimeOffset.UtcNow >= deadlineAtUtc) + MarkCause(CauseDeadline); + return Volatile.Read(ref _cause) == CauseDeadline; + } + } + + public bool CancellationRequested => Volatile.Read(ref _cause) != CauseNone || Token.IsCancellationRequested; + + public bool RequestCancel() + { + if (Volatile.Read(ref _disposed) != 0) + return false; + + MarkCause(CauseExplicit); + try + { + _explicitCancellation.Cancel(); + return true; + } + catch (ObjectDisposedException) + { + return false; + } + } + + private void MarkCause(int cause) => + Interlocked.CompareExchange(ref _cause, cause, CauseNone); + + public void Dispose() + { + if (Interlocked.Exchange(ref _disposed, 1) != 0) + return; + + _requestRegistration.Dispose(); + _deadlineRegistration.Dispose(); + _linkedCancellation.Dispose(); + _deadlineCancellation?.Dispose(); + _explicitCancellation.Dispose(); + } +} \ No newline at end of file diff --git a/src/Benchpilot.Runtime/RuntimeTypes.cs b/src/Benchpilot.Runtime/RuntimeTypes.cs index c4626a9..e731c55 100644 --- a/src/Benchpilot.Runtime/RuntimeTypes.cs +++ b/src/Benchpilot.Runtime/RuntimeTypes.cs @@ -15,6 +15,27 @@ public sealed class BenchTargetNotFoundException : BenchRuntimeException public BenchTargetNotFoundException(string message) : base(message) { } } +public sealed class BenchDeadlineExceededException : BenchRuntimeException +{ + public BenchDeadlineExceededException( + string targetId, + string operation, + int deadlineMs, + DateTimeOffset deadlineAtUtc) + : base($"Target '{targetId}' operation '{operation}' exceeded its Runtime deadline of {deadlineMs} ms.") + { + TargetId = targetId; + Operation = operation; + DeadlineMs = deadlineMs; + DeadlineAtUtc = deadlineAtUtc; + } + + public string TargetId { get; } + public string Operation { get; } + public int DeadlineMs { get; } + public DateTimeOffset DeadlineAtUtc { get; } +} + public sealed class BenchBusyException : BenchRuntimeException { public BenchBusyException( @@ -81,7 +102,9 @@ public sealed record BenchOperationInfo( string Kind, IReadOnlyList ResourceIds, DateTimeOffset StartedAtUtc, - bool CancellationRequested); + DateTimeOffset? DeadlineAtUtc, + bool CancellationRequested, + bool DeadlineExceeded); public sealed record BenchOperationRecord( string Id, @@ -91,5 +114,6 @@ public sealed record BenchOperationRecord( DateTimeOffset StartedAtUtc, DateTimeOffset CompletedAtUtc, int DurationMs, + DateTimeOffset? DeadlineAtUtc, string State, - string? Error = null); + string? Error = null); \ No newline at end of file diff --git a/src/Benchpilot.RuntimeHost/Program.cs b/src/Benchpilot.RuntimeHost/Program.cs index a845423..1cefd64 100644 --- a/src/Benchpilot.RuntimeHost/Program.cs +++ b/src/Benchpilot.RuntimeHost/Program.cs @@ -120,7 +120,9 @@ await context.Response.WriteAsJsonAsync(new ApiError( x.Kind, x.ResourceIds, x.StartedAtUtc, - x.CancellationRequested)) + x.DeadlineAtUtc, + x.CancellationRequested, + x.DeadlineExceeded)) .ToArray(); return Results.Json(new OperationListResult(true, operations)); }); @@ -140,6 +142,7 @@ await context.Response.WriteAsJsonAsync(new ApiError( x.StartedAtUtc, x.CompletedAtUtc, x.DurationMs, + x.DeadlineAtUtc, x.State, x.Error)) .ToArray(); @@ -207,7 +210,9 @@ await context.Response.WriteAsJsonAsync(new ApiError( x.Kind, x.ResourceIds, x.StartedAtUtc, - x.CancellationRequested)) + x.DeadlineAtUtc, + x.CancellationRequested, + x.DeadlineExceeded)) .ToArray(); return Results.Json(new ObservationListResult(true, observations)); }); @@ -227,6 +232,7 @@ await context.Response.WriteAsJsonAsync(new ApiError( x.StartedAtUtc, x.CompletedAtUtc, x.DurationMs, + x.DeadlineAtUtc, x.State, x.Error)) .ToArray(); @@ -299,16 +305,22 @@ await context.Response.WriteAsJsonAsync(new ApiError( app.MapPost($"{BenchpilotApi.Prefix}/power/on", async ( string? target, + int? deadlineMs, PowerOnRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).PowerOn(request.Voltage, request.SettleMs, ct))); + await Execute(() => runtime.Target(target).PowerOn( + request.Voltage, + request.SettleMs, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/power/off", async ( string? target, + int? deadlineMs, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).PowerOff(ct))); + await Execute(() => runtime.Target(target).PowerOff(deadlineMs, ct))); app.MapPost($"{BenchpilotApi.Prefix}/power/emergency-off", async ( string? target, @@ -332,45 +344,73 @@ await context.Response.WriteAsJsonAsync(new ApiError( app.MapPost($"{BenchpilotApi.Prefix}/flash/write", async ( string? target, + int? deadlineMs, FlashRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).Flash(request.Firmware, request.ConfirmTarget, ct))); + await Execute(() => runtime.Target(target).Flash( + request.Firmware, + request.ConfirmTarget, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/flash/reset", async ( string? target, + int? deadlineMs, ResetRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).Reset(request.ConfirmTarget, ct))); + await Execute(() => runtime.Target(target).Reset( + request.ConfirmTarget, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/serial/open", async ( string? target, + int? deadlineMs, SerialOpenRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).SerialOpen(request.Port, request.Baud, ct))); + await Execute(() => runtime.Target(target).SerialOpen( + request.Port, + request.Baud, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/serial/wait", async ( string? target, + int? deadlineMs, SerialWaitRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).SerialWaitFor(request.Pattern, request.TimeoutMs, ct))); + await Execute(() => runtime.Target(target).SerialWaitFor( + request.Pattern, + request.TimeoutMs, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/serial/window", async ( string? target, + int? deadlineMs, SerialWindowRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).SerialReadWindow(request.Lines, request.Filter, ct))); + await Execute(() => runtime.Target(target).SerialReadWindow( + request.Lines, + request.Filter, + deadlineMs, + ct))); app.MapPost($"{BenchpilotApi.Prefix}/serial/send", async ( string? target, + int? deadlineMs, SerialSendRequest request, BenchRuntime runtime, CancellationToken ct) => - await Execute(() => runtime.Target(target).SerialSend(request.Data, ct))); + await Execute(() => runtime.Target(target).SerialSend( + request.Data, + deadlineMs, + ct))); app.Logger.LogInformation( "BenchPilot Runtime '{BenchName}' listening on {Endpoint}. Profile: {Profile}. Drivers: {Drivers}", @@ -411,6 +451,17 @@ static async Task Execute(Func> operation) ex.BusyId), statusCode: StatusCodes.Status409Conflict); } + catch (BenchDeadlineExceededException ex) + { + return Results.Json( + new ApiError( + false, + "deadline_exceeded", + ex.Message, + DeadlineMs: ex.DeadlineMs, + DeadlineAtUtc: ex.DeadlineAtUtc), + statusCode: StatusCodes.Status408RequestTimeout); + } catch (OperationCanceledException) { return Results.Json( @@ -429,4 +480,4 @@ static async Task Execute(Func> operation) new ApiError(false, "internal", ex.Message), statusCode: StatusCodes.Status500InternalServerError); } -} +} \ No newline at end of file diff --git a/tests/Benchpilot.Core.Tests/BenchReadinessTests.cs b/tests/Benchpilot.Core.Tests/BenchReadinessTests.cs index d68c733..eb4db4d 100644 --- a/tests/Benchpilot.Core.Tests/BenchReadinessTests.cs +++ b/tests/Benchpilot.Core.Tests/BenchReadinessTests.cs @@ -27,7 +27,7 @@ public async Task Missing_required_capability_blocks_readiness_with_remediation( var result = await runtime.ValidateTargetReadiness("ecu"); Assert.False(result.ReadyForRealEcuLoop); - var check = Assert.Single(result.Checks.Where(x => x.Code == "capability.flash")); + var check = Assert.Single(result.Checks, x => x.Code == "capability.flash"); Assert.False(check.Passed); Assert.Contains("bindings.flash", check.Remediation, StringComparison.OrdinalIgnoreCase); } @@ -64,7 +64,7 @@ public async Task Simulator_backing_is_not_accepted_as_real_hardware_readiness() Assert.False(result.ReadyForRealEcuLoop); Assert.Equal("simulator", result.Mode); - var check = Assert.Single(result.Checks.Where(x => x.Code == "target.real-hardware")); + var check = Assert.Single(result.Checks, x => x.Code == "target.real-hardware"); Assert.False(check.Passed); Assert.NotNull(check.Remediation); } @@ -121,7 +121,7 @@ public async Task Change_me_placeholder_blocks_readiness_and_names_profile_path( } private static BenchReadinessCheck Find(TargetReadinessResult result, string code) => - Assert.Single(result.Checks.Where(x => x.Code == code)); + Assert.Single(result.Checks, x => x.Code == code); private static BenchRuntime CreateRuntime(BenchProfile profile, bool healthOk) { @@ -238,4 +238,4 @@ public Task Flash(string firmwarePath, CancellationToken ct = defau public Task Reset(CancellationToken ct = default) => Task.FromResult(new ResetResult(true)); } -} +} \ No newline at end of file diff --git a/tests/Benchpilot.Core.Tests/RuntimeDeadlineTests.cs b/tests/Benchpilot.Core.Tests/RuntimeDeadlineTests.cs new file mode 100644 index 0000000..8798db5 --- /dev/null +++ b/tests/Benchpilot.Core.Tests/RuntimeDeadlineTests.cs @@ -0,0 +1,229 @@ +using Benchpilot.Core; +using Benchpilot.Runtime; + +namespace Benchpilot.Core.Tests; + +public sealed class RuntimeDeadlineTests +{ + [Fact] + public async Task Mutation_deadline_has_distinct_history_exception_and_evidence() + { + using var runtime = NewRuntime(new BlockingDevice()); + + var error = await Assert.ThrowsAsync( + () => runtime.Target("ecu").Flash( + "app.elf", + confirmTarget: null, + deadlineMs: 60)); + + Assert.Equal("ecu", error.TargetId); + Assert.Equal("flash.write", error.Operation); + Assert.Equal(60, error.DeadlineMs); + + var history = Assert.Single(runtime.RecentOperations()); + Assert.Equal("deadline_exceeded", history.State); + Assert.NotNull(history.DeadlineAtUtc); + Assert.Contains("60 ms", history.Error); + + var evidence = Assert.IsType( + runtime.GetOperationEvidence(history.Id)); + var item = Assert.Single(evidence.Items); + Assert.Equal("runtime.deadline", item.Kind); + Assert.Equal("60", item.Metadata!["deadlineMs"]); + Assert.True(item.Metadata.ContainsKey("deadlineAtUtc")); + } + + [Fact] + public async Task Observation_deadline_is_not_semantic_serial_wait_timeout() + { + using var runtime = NewRuntime(new BlockingDevice()); + + var error = await Assert.ThrowsAsync( + () => runtime.Target("ecu").SerialWaitFor( + "READY", + timeoutMs: 5000, + deadlineMs: 60)); + + Assert.Equal("serial.wait", error.Operation); + var history = Assert.Single(runtime.RecentObservations()); + Assert.Equal("deadline_exceeded", history.State); + Assert.NotNull(history.DeadlineAtUtc); + + var evidence = Assert.IsType( + runtime.GetObservationEvidence(history.Id)); + var item = Assert.Single(evidence.Items); + Assert.Equal("runtime.deadline", item.Kind); + Assert.Equal("60", item.Metadata!["deadlineMs"]); + } + + [Fact] + public async Task Caller_cancellation_remains_cancelled_not_deadline() + { + using var runtime = NewRuntime(new BlockingDevice()); + using var cts = new CancellationTokenSource(); + + var flash = runtime.Target("ecu").Flash("app.elf", ct: cts.Token); + var active = Assert.Single(runtime.ActiveOperations); + cts.Cancel(); + + await Assert.ThrowsAnyAsync(async () => await flash); + + var history = Assert.Single(runtime.RecentOperations()); + Assert.Equal("cancelled", history.State); + Assert.Null(history.DeadlineAtUtc); + var evidence = Assert.IsType( + runtime.GetOperationEvidence(active.Id)); + Assert.Equal("runtime.cancelled", Assert.Single(evidence.Items).Kind); + } + + [Fact] + public async Task Late_success_from_uncooperative_driver_is_rejected_after_deadline() + { + using var runtime = NewRuntime(new LateSuccessDevice()); + + await Assert.ThrowsAsync( + () => runtime.Target("ecu").Flash( + "app.elf", + confirmTarget: null, + deadlineMs: 30)); + + var history = Assert.Single(runtime.RecentOperations()); + Assert.Equal("deadline_exceeded", history.State); + } + + [Fact] + public async Task Active_operation_exposes_deadline_without_marking_explicit_cancel() + { + using var runtime = NewRuntime(new BlockingDevice()); + var flash = runtime.Target("ecu").Flash( + "app.elf", + confirmTarget: null, + deadlineMs: 5000); + + var active = Assert.Single(runtime.ActiveOperations); + Assert.NotNull(active.DeadlineAtUtc); + Assert.False(active.CancellationRequested); + Assert.False(active.DeadlineExceeded); + + Assert.True(runtime.CancelOperation(active.Id)); + await Assert.ThrowsAnyAsync(async () => await flash); + } + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public async Task Non_positive_deadline_is_validation_error(int deadlineMs) + { + using var runtime = NewRuntime(new BlockingDevice()); + + var error = await Assert.ThrowsAsync( + () => runtime.Target("ecu").Flash( + "app.elf", + confirmTarget: null, + deadlineMs: deadlineMs)); + + Assert.Contains("deadlineMs", error.Message); + Assert.Empty(runtime.RecentOperations()); + } + + private static BenchRuntime NewRuntime(CompositeDevice device) + { + var profile = new BenchProfile + { + Name = "Deadline test bench", + DefaultTarget = "ecu", + Resources = new Dictionary(StringComparer.OrdinalIgnoreCase) + { + ["device.ecu"] = new() + { + Driver = "test", + Capabilities = ["flash", "serial"], + }, + }, + Targets = new Dictionary(StringComparer.OrdinalIgnoreCase) + { + ["ecu"] = new() + { + Bindings = new Dictionary(StringComparer.OrdinalIgnoreCase) + { + ["flash"] = "device.ecu", + ["serial"] = "device.ecu", + }, + }, + }, + }; + + var registry = new BenchResourceRegistry(profile); + registry.Register("device.ecu", device); + return new BenchRuntime(profile, registry); + } + + private abstract class CompositeDevice : IFlashTarget, ISerialChannel + { + public bool IsOpen => true; + + public abstract Task Flash( + string firmwarePath, + CancellationToken ct = default); + + public Task Reset(CancellationToken ct = default) => + Task.FromResult(new ResetResult(true)); + + public Task Open( + string? port = null, + int? baud = null, + CancellationToken ct = default) => + Task.FromResult(new SerialOpenResult(true, port ?? "TEST", baud ?? 115200)); + + public abstract Task WaitFor( + string pattern, + int timeoutMs, + CancellationToken ct = default); + + public Task ReadWindow( + int lines, + string? filter, + CancellationToken ct = default) => + Task.FromResult(new SerialWindowResult(true, Array.Empty())); + + public Task Send(string data, CancellationToken ct = default) => + Task.FromResult(new SerialSendResult(true)); + } + + private sealed class BlockingDevice : CompositeDevice + { + public override async Task Flash( + string firmwarePath, + CancellationToken ct = default) + { + await Task.Delay(Timeout.InfiniteTimeSpan, ct); + throw new InvalidOperationException("Unreachable after infinite cancellable delay."); + } + + public override async Task WaitFor( + string pattern, + int timeoutMs, + CancellationToken ct = default) + { + await Task.Delay(Timeout.InfiniteTimeSpan, ct); + throw new InvalidOperationException("Unreachable after infinite cancellable delay."); + } + } + + private sealed class LateSuccessDevice : CompositeDevice + { + public override async Task Flash( + string firmwarePath, + CancellationToken ct = default) + { + await Task.Delay(120, CancellationToken.None); + return new FlashResult(true, 1024, 120); + } + + public override Task WaitFor( + string pattern, + int timeoutMs, + CancellationToken ct = default) => + Task.FromResult(new SerialWaitResult(true, false, null, timeoutMs)); + } +} \ No newline at end of file diff --git a/tests/Benchpilot.Core.Tests/TargetContextEvidenceTests.cs b/tests/Benchpilot.Core.Tests/TargetContextEvidenceTests.cs index 53ee4a0..1fe7bd5 100644 --- a/tests/Benchpilot.Core.Tests/TargetContextEvidenceTests.cs +++ b/tests/Benchpilot.Core.Tests/TargetContextEvidenceTests.cs @@ -20,14 +20,14 @@ public async Task Failed_flash_includes_recent_power_and_current_context() var flash = await target.Flash("app.hex"); Assert.False(flash.Ok); - var operation = Assert.Single(runtime.RecentOperations(10).Where(x => x.Kind == "flash.write")); + var operation = Assert.Single(runtime.RecentOperations(10), x => x.Kind == "flash.write"); var evidence = Assert.IsType(runtime.GetOperationEvidence(operation.Id)); Assert.Contains(evidence.Items, x => x.Kind == "flash.result"); Assert.Contains(evidence.Items, x => x.Kind == "context.current-check"); Assert.Contains(evidence.Items, x => x.Kind == "context.current-reading"); Assert.Contains(evidence.Items, x => x.Kind == "context.power-on"); - var power = Assert.Single(evidence.Items.Where(x => x.Kind == "context.power-on")); + var power = Assert.Single(evidence.Items, x => x.Kind == "context.power-on"); Assert.Equal("12.4", power.Metadata!["voltageV"]); Assert.Equal("184", power.Metadata["currentMa"]); Assert.True(power.Metadata.ContainsKey("capturedAtUtc")); @@ -69,7 +69,7 @@ public async Task Successful_flash_does_not_bloat_evidence_with_target_context() var flash = await target.Flash("app.hex"); Assert.True(flash.Ok); - var operation = Assert.Single(runtime.RecentOperations(10).Where(x => x.Kind == "flash.write")); + var operation = Assert.Single(runtime.RecentOperations(10), x => x.Kind == "flash.write"); var evidence = Assert.IsType(runtime.GetOperationEvidence(operation.Id)); Assert.Single(evidence.Items); Assert.Equal("flash.result", evidence.Items[0].Kind); @@ -90,7 +90,7 @@ public async Task Failure_context_is_bounded_to_three_newest_entries() } await target.Flash("app.hex"); - var operation = Assert.Single(runtime.RecentOperations(10).Where(x => x.Kind == "flash.write")); + var operation = Assert.Single(runtime.RecentOperations(10), x => x.Kind == "flash.write"); var evidence = Assert.IsType(runtime.GetOperationEvidence(operation.Id)); var context = evidence.Items.Where(x => x.Kind.StartsWith("context.", StringComparison.Ordinal)).ToArray(); @@ -112,7 +112,7 @@ public async Task Recent_power_off_is_visible_in_later_failure_context() await target.PowerOff(); await target.Flash("app.hex"); - var operation = Assert.Single(runtime.RecentOperations(10).Where(x => x.Kind == "flash.write")); + var operation = Assert.Single(runtime.RecentOperations(10), x => x.Kind == "flash.write"); var evidence = Assert.IsType(runtime.GetOperationEvidence(operation.Id)); var context = evidence.Items.Where(x => x.Kind.StartsWith("context.", StringComparison.Ordinal)).ToArray(); @@ -224,4 +224,4 @@ public Task ReadWindow( public Task Send(string data, CancellationToken ct = default) => Task.FromResult(new SerialSendResult(true)); } -} +} \ No newline at end of file