diff --git a/.agents/upstream-review.md b/.agents/upstream-review.md index ac1894ec4b..0a3b36ea40 100644 --- a/.agents/upstream-review.md +++ b/.agents/upstream-review.md @@ -35,6 +35,9 @@ Historical groups are indexed in the linked archive. This file migration changes | Group / bounded head | Sources | Outcome and remaining scope | Pylon PR / verification | | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Explicit provider refresh cache bypass / `f193a6863c494fdfa36a1ef3132e42a7d7b42926` | `f193a6863c494fdfa36a1ef3132e42a7d7b42926` (#13109) | Implemented in PR #743, pending merge: explicit refresh bypasses the model manifest freshness/retry timer and provider-owned capability, model and package-version caches before status probes. Background probes retain their timers; Pylon's installed-provider `refreshModels` path remains. No behavior excluded. Cursor unchanged. | Cycle [#736](https://github.com/pylon-code/pylon/issues/736), [PR #743](https://github.com/pylon-code/pylon/pull/743); independent adversarial review, 246 focused tests, server typecheck and scoped lint/format. | +| Background GitHub PR quota / `eafb4a9340bd2c2271b7c8fac7eddd55beea3972` | `5975ec78b7eaff654e6183b99f8de7a05591e99d` (#13189), `18de6bb328059703b5d1b0661610076e3c0fa2d4` (#13198), `eafb4a9340bd2c2271b7c8fac7eddd55beea3972` (#13200) | Adapted all three: skip uncached recheck when a terminal link cannot settle, batch background PR summaries, and avoid owner-qualified head probes. Preserve Pylon credential scopes, GraphQL budget and per-environment routing; exclude upstream-only test fixture APIs. No cursor advance. | [GitHub quota #749](https://github.com/pylon-code/pylon/pull/749); focused server regressions, scoped lint and typecheck. | +| OTLP export kill switch / `e67abcf798f8c4d8458755e3b4dde02c2c1f628b` | `e67abcf798f8c4d8458755e3b4dde02c2c1f628b` (#13355) | Implemented in PR #746, pending merge: `OTEL_SDK_DISABLED` and the inherited `T3CODE_OTEL_SDK_DISABLED` override suppress server and desktop main-process OTLP traces, metrics, and logs from environment, bootstrap, or Settings endpoints; WSL inherits the flags. Local trace files and local logs remain active, and Pylon service identity is retained. No behavior excluded. Cursor unchanged. | Cycle [#736](https://github.com/pylon-code/pylon/issues/736), [PR #746](https://github.com/pylon-code/pylon/pull/746); independent adversarial review, 80 focused tests, server/desktop/shared typechecks and scoped lint/format. | | OTLP log export, desktop main process telemetry, and per-signal observability / `fa1e1715538f6f8ac076761f7821f68479f616ab` | `3bb06ad910389fcfc886bd5156f41dc73f307f80` (#12493), `82059df1523d070b00ab6f45159115f626a4977e` (#12520), `a8693eb4ed3a66d72e6b814117e7beeaf0fb5f8b` (#12540), `fa1e1715538f6f8ac076761f7821f68479f616ab` (#12657) | Adopted all four in Cycle #689 (PR #690). Server exports structured log records over OTLP with trace/span ID correlation, replaces `tracerLogger` to eliminate duplicate in-span span-event exports, and applies dedicated flusher/serialization layers. Desktop exports main process traces and logs over OTLP while withholding metric emissions until metrics are instrumented. OTLP settings and HTTP header sets are isolated per signal (traces, metrics, logs) across server and desktop configurations. Adversarial review findings resolved: preserved `pylon-server` runtime and service identity defaults, verified `DEFAULT_SIGNAL_EXPORT` fallback, tested backward compatibility of `ServerObservability` schema decoding, and validated log deduplication. Cursor unchanged. | Cycle [#689](https://github.com/pylon-code/pylon/issues/689), [PR #690](https://github.com/pylon-code/pylon/pull/690); independent adversarial review, new ServerLogger test suite, 9 desktop observability tests, all CI checks green. | | Web chat/diff polish, editor discovery, diff cache eviction, ACP elicitation, and preview recovery / `c14f6015bfe479d313355cb234af1a5c16dbb15f` | `c14f6015bfe479d313355cb234af1a5c16dbb15f` (#12453), `0ff87f251dafb32703d5f531e7b248ad0a581ee2` (#10831), `4a560b4e4ebb37efb7f57805ba79e37f5500bdca` (#12438), `9accc567670da5096e9a694c1adb0d35adb51d9d` (#12571), `dfbb11bdd7c3f1a5575cb55d3e3abb12be025727` (#12377), `408ff8ae9bd7eb2e7e90cbfd8b3fcfe63641bf23` (#12439), `cb3d95c17487f3d834a5537f7c3aa0106bc9b5ec` (#12523), `de6a230db0bb71d3fa91d8de0ae7067b68253f82` (#11294), `efb96939fbb135b9f4de3bff19447200b178b23a` (#7861), `5378f87f99175bded2b9241876319567875aca03` (#12535) | Adopted all ten across Cycle #685 Group A (PR #686) and Group B (PR #687). Retained whitespace when copying diff contents, kept linebreaks in citation content, opened chat popovers with relative placement, discovered installed external editors on desktop outside PATH, wrapped long titles in confirmation dialogs, showed plain text in collapsed thought previews, released oversized pull request diff cache entries, accepted ACP SDK elicitation requests (`session/elicitation` & `elicitation/create`), surfaced Codex app permission requests as approvable, and recovered preview host registration after request timeouts via connection queue eviction. Hardening and adversarial review findings resolved: normalized `acceptAlways` approval decision in Codex session runtime to grant permissions, enforced retirement epoch and quarantine boundaries on approval requests, scoped event emission with `CodexNotificationEpoch`, wrapped diff cache invalidation in `Effect.uninterruptible`, cleaned up mobile icon maps, and rebranded pre-existing PR tool descriptions to Pylon. Preserved Pylon Effect services, session lifecycles, and UI layout authority. Cursor unchanged. | Cycle [#685](https://github.com/pylon-code/pylon/issues/685), [PR #686](https://github.com/pylon-code/pylon/pull/686) and [PR #687](https://github.com/pylon-code/pylon/pull/687); independent adversarial reviews, 260+ focused unit tests across web, server, and effect-acp, full workspace typecheck, scoped format/lint. | | Web interaction, ACP session startup stderr, Claude continuation, and GTK4 snapshot / `52d08a14b9e475371658fabbb12c056c3d3dad67` | `8dd02470b1e7603b8d36a21ae27c510480351f18` (#12552), `7445aa733ada33e45289e5aa5055f79142556513` (#11263), `599c9776eb887d4fff42da7d2daf8aa07ee9a9e2` (#11268), `dcf8942304cfef382af55c0068cb2f04c86757fc` (#12636), `e36725682bd2935f1bcfdf81660cb569b5f3df29` (#12577), `63ff33756c47f127391962458d8388a647e8ed44` (#12624), `d6f291303ddc0c9a14f570266a4d9eff6d431593` (#12625), `52d08a14b9e475371658fabbb12c056c3d3dad67` (#12635) | Adopted all eight across Cycle #682 Group A (PR #683) and Group B (PR #684). Preserved draft question input when clicking options, desktop CSP screenshot annotations, restored providers settings heading, aligned PR detail menu glyphs. Empty Claude homePath correctly resolves and shares continuation with `~/.claude` or `CLAUDE_CONFIG_DIR`, surfaced ACP stderr excerpt in session startup failures instead of generic closed session errors, and extracted desktop accessible text for Flatpak and GTK4 apps in SnapShot. Adversarial review findings resolved: ACP stderr excerpt sanitized against tokens/credentials and pending buffer bounded; defensive empty `homePath` trimming and `CLAUDE_CONFIG_DIR` resolution in OAuth usage; desktop accessibility timeout preserved without premature concurrency overlap. Preserved Pylon provider rate limits and error reporting. Cursor unchanged. | Cycle [#682](https://github.com/pylon-code/pylon/issues/682), [PR #683](https://github.com/pylon-code/pylon/pull/683) and [PR #684](https://github.com/pylon-code/pylon/pull/684); independent adversarial reviews, 225+ focused tests across desktop and server, full workspace typecheck, scoped format/lint. | @@ -219,3 +222,5 @@ The review cursor stays unchanged; unrelated sources in this range remain unclas | PR cache reuse and quota efficiency / `6d1d549441be84f19696ab59ed7e2fbf305280d4` | `f4600d77dd7c2fa9f10e8f4500882e427fcc7e26` (#11888), `d612d12b8bb278af97b0029f8d48b2403a5f9ee6` (#12168) | Adopted applicable behavior: scoped cache revisions, canonical detail reuse, deduplicated refreshes and bounded quota probes. Preserve Pylon environment fences and observed GraphQL consumption. Exclude account-router-specific hooks and Forgejo tests: those dependencies are absent in Pylon; their independent source decisions remain open. No cross-environment credential routing introduced. | Cycle [#625](https://github.com/pylon-code/pylon/issues/625), [PR #626](https://github.com/pylon-code/pylon/pull/626); direct adversarial self-review, focused backend/client regressions and scoped types/lint. | | Authenticated GitHub PR media / `6d1d549441be84f19696ab59ed7e2fbf305280d4` | `32e8b2584556c0c55ce0d1d8f72b506cb771d60d` (#11706) | Adopted with Pylon path/context guards retained. Signed assets use the active server GitHub CLI account per request; bounded GitHub/CDN redirects strip credentials off credentialed hosts, stream ranges, constrain MIME and sandbox SVG. Older servers retain public fallback. GitHub Enterprise media unsupported. | Cycle [#625](https://github.com/pylon-code/pylon/issues/625), [PR #629](https://github.com/pylon-code/pylon/pull/629); direct adversarial self-review, credential/redirect/range regressions, scoped checks and browser evidence. | + +| Retained usage totals and newest shared scan / `7ed3f711caa36485001538bbe1cb0017b84aa7fe` | `d4d5d12e8ba086cfbf79ca3adeb4156b46ead665` (#10315), `675869d2b0f9e059d5950b80390c0cad1cb1590e` (#12304) | Adopted with Pylon adaptation: retain saved usage after transcript cleanup or moves and across restart; claim a shared fingerprint from the newest scan with a stable ID tie-break. Configured and standalone Antigravity homes retain their profile identity, partial status and per-directory SQLite limits. Stale source identities expire when no current or retained records use them. Existing provider-level wire buckets cannot split costs by individual home when two environments each own distinct and shared homes; that cross-environment overlap remains outside this port and needs source-attributed buckets. The 90-day retention bound remains. Cursor unchanged. | Cycle [#736](https://github.com/pylon-code/pylon/issues/736), [PR #762](https://github.com/pylon-code/pylon/pull/762); 46 focused tests and scoped server checks. | diff --git a/.github/scripts/thread-transfer-report.cjs b/.github/scripts/thread-transfer-report.cjs index 94a02b7806..264a92b1c8 100644 --- a/.github/scripts/thread-transfer-report.cjs +++ b/.github/scripts/thread-transfer-report.cjs @@ -61,7 +61,7 @@ function validateResult(value) { } assertExactKeys(value.scenario, SCENARIO_KEYS, "result.scenario"); - if (value.scenario.id !== "thread-transfer-v1") { + if (!["thread-transfer-v1", "thread-transfer-v2"].includes(value.scenario.id)) { throw new Error("result.scenario.id is not supported"); } for (const key of SCENARIO_KEYS.slice(1)) { @@ -157,7 +157,7 @@ function renderComment(input) { `| ${provider === "codex" ? "Codex" : "Claude"} | ${metric.label} | ${baselineObserved === undefined ? "—" : formatValue(baselineObserved, metric.kind)} | ${formatValue(observed, metric.kind)} | ${formatImpact(observed, baselineObserved, metric.kind)} | ${formatValue(ceiling, metric.kind)} | ${pass ? "✅" : "❌"} |`, ); - if (baseline && baseline.providers[provider].ceiling[metric.key] !== ceiling) { + if (comparable && baseline.providers[provider].ceiling[metric.key] !== ceiling) { ceilingChanges.push( `- ${provider === "codex" ? "Codex" : "Claude"} ${metric.label}: ${formatValue(baseline.providers[provider].ceiling[metric.key], metric.kind)} → ${formatValue(ceiling, metric.kind)}`, ); diff --git a/.github/scripts/thread-transfer-report.test.cjs b/.github/scripts/thread-transfer-report.test.cjs index 4935864e46..3c8ab2578a 100644 --- a/.github/scripts/thread-transfer-report.test.cjs +++ b/.github/scripts/thread-transfer-report.test.cjs @@ -290,3 +290,21 @@ test("preserves a successful result when a same-SHA rerun has no artifact", asyn assert.equal(published, true); assert.equal(updatedComment, false); }); + +test("accepts V2 without comparing it to the V1 scenario", () => { + const current = result(); + current.scenario.id = "thread-transfer-v2"; + current.providers.codex.ceiling = { ...current.providers.codex.ceiling, totalWireBytes: 3000000 }; + assert.equal(validateResult(current), current); + const comment = renderComment({ + current, + baseline: result(), + currentRun: { sha: "bbbbbbbb", conclusion: "success", url: "https://example.com/current" }, + baselineRun: { sha: "aaaaaaaa", matchesBase: true, url: "https://example.com/baseline" }, + }); + assert.match(comment, /fixture changed/); + assert.doesNotMatch(comment, /This PR changes transfer ceilings/); + assert.doesNotMatch(comment, /[+-]\d+\.\d+%/); + current.scenario.id = "unrecognized"; + assert.throws(() => validateResult(current), /not supported/); +}); diff --git a/apps/desktop/src/app/DesktopApp.ts b/apps/desktop/src/app/DesktopApp.ts index 719502636b..b5a7bb2007 100644 --- a/apps/desktop/src/app/DesktopApp.ts +++ b/apps/desktop/src/app/DesktopApp.ts @@ -1,4 +1,5 @@ import * as Cause from "effect/Cause"; +import * as Duration from "effect/Duration"; import * as Effect from "effect/Effect"; import * as Option from "effect/Option"; import * as Ref from "effect/Ref"; @@ -142,6 +143,20 @@ const handleFatalStartupError = Effect.fn("desktop.startup.handleFatalStartupErr const fatalStartupCause = (stage: string, cause: Cause.Cause) => handleFatalStartupError(stage, Cause.pretty(cause)).pipe(Effect.andThen(Effect.failCause(cause))); +export const stopAllPoolInstances = Effect.fn("desktop.app.stopAllPoolInstances")( + function* (): Effect.fn.Return { + // Stop every backend in the pool with a timeout to guarantee the quit + // path makes progress even if a backend hangs during teardown. + const pool = yield* DesktopBackendPool.DesktopBackendPool; + const instances = yield* pool.list; + yield* Effect.forEach( + instances, + (instance) => instance.stop({ timeout: Duration.seconds(5) }), + { concurrency: "unbounded" }, + ); + }, +); + const bootstrap = Effect.gen(function* () { const pool = yield* DesktopBackendPool.DesktopBackendPool; const primaryBackend = yield* pool.primary; @@ -313,18 +328,12 @@ const scopedProgram = Effect.scoped( const shutdown = yield* DesktopShutdown.DesktopShutdown; yield* Effect.addFinalizer(() => - Effect.gen(function* () { - const pool = yield* DesktopBackendPool.DesktopBackendPool; - // Stop every backend in the pool, not just the primary. The - // electronApp.quit() path can race ahead of the layer-scope - // cascade, so leaving the WSL instance for its parent scope - // finalizer means it gets hard-killed by the OS instead of - // receiving SIGTERM + grace. Stops run concurrently. - const instances = yield* pool.list; - yield* Effect.forEach(instances, (instance) => instance.stop(), { - concurrency: "unbounded", - }); - }).pipe(Effect.ensuring(shutdown.markComplete)), + // Stop every backend in the pool, not just the primary. The + // electronApp.quit() path can race ahead of the layer-scope + // cascade, so leaving the WSL instance for its parent scope + // finalizer means it gets hard-killed by the OS instead of + // receiving SIGTERM + grace. + stopAllPoolInstances().pipe(Effect.ensuring(shutdown.markComplete)), ); yield* startup; diff --git a/apps/desktop/src/app/DesktopObservability.test.ts b/apps/desktop/src/app/DesktopObservability.test.ts index d7ccfc43b1..215998631e 100644 --- a/apps/desktop/src/app/DesktopObservability.test.ts +++ b/apps/desktop/src/app/DesktopObservability.test.ts @@ -1,6 +1,7 @@ import * as NodeHttpClient from "@effect/platform-node/NodeHttpClient"; import * as NodeServices from "@effect/platform-node/NodeServices"; import { assert, describe, it } from "@effect/vitest"; +import * as ConfigProvider from "effect/ConfigProvider"; import * as Effect from "effect/Effect"; import * as FileSystem from "effect/FileSystem"; import * as Layer from "effect/Layer"; @@ -427,6 +428,36 @@ describe("DesktopObservability", () => { ); }); + it.effect("exports kill switch warnings through the configured logger", () => { + const requests: Array = []; + return Effect.gen(function* () { + const fileSystem = yield* FileSystem.FileSystem; + const baseDir = yield* fileSystem.makeTempDirectoryScoped({ + prefix: "t3-desktop-observability-test-", + }); + const environmentLayer = makeEnvironmentLayer(baseDir, true, { + T3CODE_OTLP_LOGS_URL: "https://collector.example.com/v1/logs", + }); + + yield* Effect.scoped( + Effect.void.pipe( + Effect.provide(DesktopObservability.layer.pipe(Layer.provideMerge(environmentLayer))), + ), + ); + + assert.include(requests[0]?.body ?? "", "OTEL_SDK_DISABLED=1 was read as false"); + }).pipe( + Effect.scoped, + Effect.provide( + Layer.mergeAll( + NodeServices.layer, + collectorLayer(requests), + ConfigProvider.layer(ConfigProvider.fromEnv({ env: { OTEL_SDK_DISABLED: "1" } })), + ), + ), + ); + }); + it.effect("reads every signal endpoint from Settings when the environment names none", () => { const requests: Array = []; return Effect.gen(function* () { @@ -491,4 +522,37 @@ describe("DesktopObservability", () => { Effect.provide(Layer.mergeAll(NodeServices.layer, collectorLayer(requests))), ); }); + + it.effect("stops every export when the OpenTelemetry SDK is disabled", () => { + const requests: Array = []; + return Effect.gen(function* () { + const fileSystem = yield* FileSystem.FileSystem; + const baseDir = yield* fileSystem.makeTempDirectoryScoped({ + prefix: "t3-desktop-observability-test-", + }); + const environmentLayer = makeEnvironmentLayer(baseDir); + yield* writeObservabilitySettings(environmentLayer, { + otlpTracesUrl: "https://settings.example.com/v1/traces", + otlpLogsUrl: "https://settings.example.com/v1/logs", + }); + + yield* Effect.scoped( + Effect.logInfo("desktop log stays local when disabled").pipe( + Effect.withSpan("desktop-disabled-test"), + Effect.provide(DesktopObservability.layer.pipe(Layer.provideMerge(environmentLayer))), + ), + ); + + assert.lengthOf(requests, 0); + }).pipe( + Effect.scoped, + Effect.provide( + Layer.mergeAll( + NodeServices.layer, + collectorLayer(requests), + ConfigProvider.layer(ConfigProvider.fromEnv({ env: { OTEL_SDK_DISABLED: "true" } })), + ), + ), + ); + }); }); diff --git a/apps/desktop/src/app/DesktopObservability.ts b/apps/desktop/src/app/DesktopObservability.ts index 9665721574..f8fd73a636 100644 --- a/apps/desktop/src/app/DesktopObservability.ts +++ b/apps/desktop/src/app/DesktopObservability.ts @@ -4,6 +4,7 @@ import { makeTraceSink, otlpSerializationLayer, } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import { parsePersistedServerObservabilitySettings, type PersistedServerObservabilitySettings, @@ -353,12 +354,18 @@ const readPersistedObservabilitySettings: Effect.Effect< * resolve traces against one revision of the file and logs against another. */ const resolveOtlpEndpoints = Effect.gen(function* () { + const otel = yield* OtelEnvironment.load; + if (otel.disabled) { + return { traces: undefined, metrics: undefined, logs: undefined, warnings: otel.warnings }; + } + const environment = yield* DesktopEnvironment.DesktopEnvironment; const persisted = yield* readPersistedObservabilitySettings; return { traces: Option.getOrUndefined(environment.otlpTracesUrl) ?? persisted.otlpTracesUrl, metrics: Option.getOrUndefined(environment.otlpMetricsUrl) ?? persisted.otlpMetricsUrl, logs: Option.getOrUndefined(environment.otlpLogsUrl) ?? persisted.otlpLogsUrl, + warnings: otel.warnings, }; }); @@ -671,7 +678,12 @@ const telemetryLayer = Layer.unwrap( // resource, // }).pipe(Layer.provide(serializationLayer)); - return Layer.mergeAll(loggerLayer, tracerLayer); + // Logged once the loggers above are installed, so the warnings use them. + const otelWarningsLayer = Layer.effectDiscard( + Effect.forEach(endpoints.warnings, (warning) => Effect.logWarning(warning)), + ); + + return otelWarningsLayer.pipe(Layer.provideMerge(Layer.mergeAll(loggerLayer, tracerLayer))); }), ); diff --git a/apps/desktop/src/backend/DesktopBackendConfiguration.test.ts b/apps/desktop/src/backend/DesktopBackendConfiguration.test.ts index 62a11381fe..de63e91b10 100644 --- a/apps/desktop/src/backend/DesktopBackendConfiguration.test.ts +++ b/apps/desktop/src/backend/DesktopBackendConfiguration.test.ts @@ -1055,6 +1055,49 @@ describe("DesktopBackendConfiguration", () => { }).pipe(Effect.scoped, Effect.provide(NodeServices.layer)), ); + it.effect("resolveWsl carries the kill switch into the distro", () => + Effect.gen(function* () { + const fileSystem = yield* FileSystem.FileSystem; + const baseDir = yield* fileSystem.makeTempDirectoryScoped({ + prefix: "t3-desktop-backend-config-test-", + }); + + const previousWslEnv = process.env.WSLENV; + const previousDisabled = process.env.OTEL_SDK_DISABLED; + try { + delete process.env.WSLENV; + process.env.OTEL_SDK_DISABLED = "true"; + + yield* Effect.gen(function* () { + const configuration = yield* DesktopBackendConfiguration.DesktopBackendConfiguration; + const config = yield* configuration.resolveWsl({ port: 5050, distro: null }); + + assert.equal(config.env.OTEL_SDK_DISABLED, "true"); + assert.include((config.env.WSLENV ?? "").split(":"), "OTEL_SDK_DISABLED"); + }).pipe( + Effect.provide( + DesktopBackendConfiguration.layer.pipe( + Layer.provideMerge(serverExposureLayer), + Layer.provideMerge(DesktopAppSettings.layerTest()), + Layer.provideMerge(DesktopWslServerTree.layerTest()), + Layer.provideMerge( + DesktopWslEnvironment.layerTest({ + isAvailable: true, + windowsToWslPath: () => Option.some("/mnt/c/repo/apps/server/src/index.ts"), + getDistroIp: () => Option.some("172.27.0.99"), + }), + ), + Layer.provideMerge(makeEnvironmentLayer(baseDir, { platform: "win32" })), + ), + ), + ); + } finally { + restoreEnv("WSLENV", previousWslEnv); + restoreEnv("OTEL_SDK_DISABLED", previousDisabled); + } + }).pipe(Effect.scoped, Effect.provide(NodeServices.layer)), + ); + it.effect("resolveWsl preserves existing WSLENV entries when forwarding backend secrets", () => Effect.gen(function* () { const fileSystem = yield* FileSystem.FileSystem; diff --git a/apps/desktop/src/backend/DesktopBackendConfiguration.ts b/apps/desktop/src/backend/DesktopBackendConfiguration.ts index 711405002f..8df5f799e1 100644 --- a/apps/desktop/src/backend/DesktopBackendConfiguration.ts +++ b/apps/desktop/src/backend/DesktopBackendConfiguration.ts @@ -96,6 +96,9 @@ const DESKTOP_BACKEND_ENV_NAMES = [ const WSL_FORWARDED_ENV_NAMES = [ "OPENAI_API_KEY", "ANTHROPIC_API_KEY", + // Otherwise the WSL server keeps exporting to endpoints from the bootstrap. + "T3CODE_OTEL_SDK_DISABLED", + "OTEL_SDK_DISABLED", "T3CODE_OTLP_HEADERS", "T3CODE_OTLP_PROTOCOL", ] as const; diff --git a/apps/desktop/src/backend/DesktopBackendManager.test.ts b/apps/desktop/src/backend/DesktopBackendManager.test.ts index 53ccf5a756..4bbecd7d74 100644 --- a/apps/desktop/src/backend/DesktopBackendManager.test.ts +++ b/apps/desktop/src/backend/DesktopBackendManager.test.ts @@ -23,6 +23,8 @@ import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/unstab import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process"; import * as DesktopBackendManager from "./DesktopBackendManager.ts"; +import * as DesktopApp from "../app/DesktopApp.ts"; +import * as DesktopBackendPool from "./DesktopBackendPool.ts"; import * as DesktopObservability from "../app/DesktopObservability.ts"; import * as DesktopTelemetryPublisher from "../telemetry/DesktopTelemetryPublisher.ts"; import * as DesktopWslEnvironment from "../wsl/DesktopWslEnvironment.ts"; @@ -1503,4 +1505,147 @@ describe("DesktopBackendManager", () => { }).pipe(Effect.provide(TestClock.layer())), ), ); + + it.effect("stopAllPoolInstances bounds the quit finalizer when backends hang", () => + Effect.scoped( + Effect.gen(function* () { + // Each backend's process-scope finalizer reports when it starts and + // when it finishes, keyed by instance name, so the test can prove + // both backends reached each milestone instead of inferring it from + // a shared flag or a clock advance. + const teardownStarted = yield* Queue.unbounded(); + const teardownFinished = yield* Queue.unbounded(); + const allowTeardown = yield* Deferred.make(); + + const makeInstance = (name: string) => + makeTestInstance({ + spawnerLayer: Layer.succeed( + ChildProcessSpawner.ChildProcessSpawner, + ChildProcessSpawner.make(() => + Effect.gen(function* () { + const scope = yield* Scope.Scope; + yield* Scope.addFinalizer( + scope, + Queue.offer(teardownStarted, name).pipe( + Effect.andThen(Deferred.await(allowTeardown)), + Effect.andThen(Queue.offer(teardownFinished, name)), + Effect.asVoid, + ), + ); + return makeProcess({ exitCode: Effect.never }); + }), + ), + ), + httpClientLayer: httpClientLayer(() => Effect.never), + }); + + const instance1 = yield* makeInstance("instance1"); + const instance2 = yield* makeInstance("instance2"); + + yield* instance1.start; + yield* instance2.start; + + const mockPool = Layer.succeed(DesktopBackendPool.DesktopBackendPool, { + list: Effect.succeed([instance1, instance2]), + get: () => Effect.succeed(Option.none()), + primary: Effect.die(new Error("primary not implemented")), + register: () => Effect.die(new Error("register not implemented")), + unregister: () => Effect.die(new Error("unregister not implemented")), + }); + + // Mirror the quit path: register stopAllPoolInstances as a scope + // finalizer and let the scope close run it, rather than calling it + // as an ordinary interruptible effect. + const quitFiber = yield* Effect.scoped( + Effect.addFinalizer(() => DesktopApp.stopAllPoolInstances()), + ).pipe(Effect.provide(mockPool), Effect.forkChild); + + const started = yield* Queue.takeN(teardownStarted, 2); + assert.deepEqual(started.toSorted(), ["instance1", "instance2"]); + + // Both backends are now hung in teardown. Advancing past the 5s + // budget must let the quit finalizer return without them. + yield* TestClock.adjust(Duration.seconds(5)); + yield* Fiber.join(quitFiber); + assert.equal(yield* Queue.size(teardownFinished), 0); + + // The timed-out closes keep running in the background and finish + // once the backends unblock. + yield* Deferred.succeed(allowTeardown, undefined); + const finished = yield* Queue.takeN(teardownFinished, 2); + assert.deepEqual(finished.toSorted(), ["instance1", "instance2"]); + }).pipe(Effect.provide(TestClock.layer())), + ), + ); + + it.effect("stopAllPoolInstances bounds an early shutdown callback", () => + Effect.scoped( + Effect.gen(function* () { + const ready = yield* Queue.unbounded(); + const hookStarted = yield* Queue.unbounded(); + const hookFinished = yield* Queue.unbounded(); + const releaseHook = yield* Deferred.make(); + + const makeInstance = (name: string) => + makeTestInstance({ + spawnerLayer: Layer.succeed( + ChildProcessSpawner.ChildProcessSpawner, + ChildProcessSpawner.make(() => + Effect.gen(function* () { + const scope = yield* Scope.Scope; + const closed = yield* Deferred.make(); + yield* Scope.addFinalizer( + scope, + Deferred.succeed(closed, undefined).pipe(Effect.asVoid), + ); + return makeProcess({ + exitCode: Deferred.await(closed).pipe( + Effect.as(ChildProcessSpawner.ExitCode(0)), + ), + kill: () => Deferred.succeed(closed, undefined).pipe(Effect.asVoid), + }); + }), + ), + ), + onReady: Queue.offer(ready, name).pipe(Effect.asVoid), + onShutdown: Queue.offer(hookStarted, name).pipe( + Effect.andThen(Deferred.await(releaseHook)), + Effect.andThen(Queue.offer(hookFinished, name)), + Effect.asVoid, + ), + }); + + const instance1 = yield* makeInstance("instance1"); + const instance2 = yield* makeInstance("instance2"); + yield* instance1.start; + yield* instance2.start; + assert.deepEqual((yield* Queue.takeN(ready, 2)).toSorted(), ["instance1", "instance2"]); + + const mockPool = Layer.succeed(DesktopBackendPool.DesktopBackendPool, { + list: Effect.succeed([instance1, instance2]), + get: () => Effect.succeed(Option.none()), + primary: Effect.die(new Error("primary not implemented")), + register: () => Effect.die(new Error("register not implemented")), + unregister: () => Effect.die(new Error("unregister not implemented")), + }); + const quitFiber = yield* Effect.scoped( + Effect.addFinalizer(() => DesktopApp.stopAllPoolInstances()), + ).pipe(Effect.provide(mockPool), Effect.forkChild); + + assert.deepEqual((yield* Queue.takeN(hookStarted, 2)).toSorted(), [ + "instance1", + "instance2", + ]); + yield* TestClock.adjust(Duration.seconds(5)); + yield* Fiber.join(quitFiber); + assert.equal(yield* Queue.size(hookFinished), 0); + + yield* Deferred.succeed(releaseHook, undefined); + assert.deepEqual((yield* Queue.takeN(hookFinished, 2)).toSorted(), [ + "instance1", + "instance2", + ]); + }).pipe(Effect.provide(TestClock.layer())), + ), + ); }); diff --git a/apps/desktop/src/backend/DesktopBackendManager.ts b/apps/desktop/src/backend/DesktopBackendManager.ts index 6f1ea139bd..997f6df459 100644 --- a/apps/desktop/src/backend/DesktopBackendManager.ts +++ b/apps/desktop/src/backend/DesktopBackendManager.ts @@ -1045,7 +1045,7 @@ export const makeBackendInstance = Effect.fn("makeBackendInstance")(function* ( }); }); - const stop = Effect.fn("desktop.backendInstance.stop")(function* (options?: { + const stopUnsafe = Effect.fn("desktop.backendInstance.stopUnsafe")(function* (options?: { readonly timeout?: Duration.Duration; }) { const { active, restartFiber, notifyShutdown } = yield* mutex.withPermits(1)( @@ -1137,6 +1137,15 @@ export const makeBackendInstance = Effect.fn("makeBackendInstance")(function* ( }); }); + const stop: DesktopBackendInstance["stop"] = (options) => { + const timeout = options?.timeout; + if (timeout === undefined) return stopUnsafe(); + return Effect.forkIn(stopUnsafe(), parentScope).pipe( + Effect.flatMap((fiber) => Fiber.join(fiber).pipe(Effect.timeoutOption(timeout))), + Effect.asVoid, + ); + }; + const waitForReady = (timeout: Duration.Duration): Effect.Effect => Effect.gen(function* () { const current = yield* Ref.get(state); @@ -1154,7 +1163,7 @@ export const makeBackendInstance = Effect.fn("makeBackendInstance")(function* ( Effect.map(Option.getOrElse(() => false)), ); - yield* Effect.addFinalizer(() => stop()); + yield* Effect.addFinalizer(() => stopUnsafe({ timeout: Duration.seconds(5) })); return { id: spec.id, diff --git a/apps/desktop/src/wsl/DesktopWslEnvironment.test.ts b/apps/desktop/src/wsl/DesktopWslEnvironment.test.ts index 8301c9ea33..fb360e2f63 100644 --- a/apps/desktop/src/wsl/DesktopWslEnvironment.test.ts +++ b/apps/desktop/src/wsl/DesktopWslEnvironment.test.ts @@ -168,7 +168,7 @@ describe("WSL runtime cache", () => { expect(nightly).not.toContain('"$HOME/.pylon-code/wsl-runtime"'); expect(buildWslRuntimeInvalidateScript("sha256-bbb", "pylon-code-nightly")).toContain( - ".pylon-code-nightly/wsl-runtime/sha256-bbb", + ".pylon-code-nightly/wsl-runtime", ); }); @@ -190,6 +190,10 @@ describe("WSL runtime cache", () => { expect(script).toContain("trap 'exit 1' HUP INT TERM"); expect(script).toContain('exec 9> "$runtime_lock"'); expect(script).toContain("flock -x 9"); + expect(script).toContain('probe_failure_marker="$runtime_parent/.1.2.3-x64.probe-failed"'); + expect(script.indexOf('if [ -f "$probe_failure_marker" ]')).toBeLessThan( + script.indexOf("if runtime_is_ready; then"), + ); expect(script).not.toContain("runtime_lock_pid"); expect(script).not.toContain("sleep 0.1"); expect(script).not.toContain('rm -rf "$runtime_lock"'); @@ -386,7 +390,7 @@ describe("WSL runtime cache", () => { // Without visible processes the retention rules cannot tell a live cache // from an abandoned one, so the sweep is skipped rather than guessed at. - expect(script).toContain("[ -d /proc/1 ] || exit 0"); + expect(script).toContain("[ -d /proc/1 ] && have_process_info=1"); // The guard has to gate the delete, not just exist. const inUseChecked = script.indexOf('! runtime_in_use "$candidate"'); @@ -400,9 +404,9 @@ describe("WSL runtime cache", () => { // Dot-prefixed, so `"$runtime_parent"/*` never matches them, and they carry // no ready marker either; without this pass a killed install leaks forever. - expect(script).toContain( - 'for scratch in "$runtime_parent"/.*.tmp.* "$runtime_parent"/.*.stale.*; do', - ); + expect(script).toContain('for scratch in "$runtime_parent"/.*.tmp.*; do'); + expect(script).toContain('for scratch in "$runtime_parent"/.*.stale.*; do'); + expect(script).toContain('! runtime_in_use "$runtime_parent/$original_name" || continue'); // Age guard: a scratch directory younger than this belongs to a live install. expect(script).toContain('find "$scratch" -maxdepth 0 -mmin +120'); }); @@ -412,9 +416,8 @@ describe("WSL runtime cache", () => { // Readiness is a presence check, so a tree whose pty.node is present but // unloadable stays ready forever unless the probe can revoke the marker. - expect(script).toContain( - 'rm -f "$HOME/.pylon-code/wsl-runtime/1.2.3_x64/.t3code-wsl-runtime-ready"', - ); + expect(script).toContain('rm -f "$runtime_parent/1.2.3_x64/.t3code-wsl-runtime-ready"'); + expect(script).toContain('touch "$runtime_parent/.1.2.3_x64.probe-failed"'); // Deleting the tree here would pull it out from under any backend still // running from it; the next install moves an unready root aside instead. expect(script).not.toContain("rm -rf"); @@ -503,6 +506,34 @@ describe.skipIf(posixShellRunner === null)("WSL runtime install script (executed expect(restored.stdout).toBe(SERVER_ENTRY_SOURCE); }); + it("backs off a proven native probe failure and retries after the cooldown", () => { + const fixture = createFixture(); + expect(fixture.install().status).toBe(0); + const invalidated = runShell( + [ + "set -eu", + `HOME=${sh(`${fixture.work}/home`)}`, + "export HOME", + buildWslRuntimeInvalidateScript(fixture.runtimeId, "pylon-code"), + ].join("\n"), + ); + expect(invalidated.status, invalidated.stderr).toBe(0); + + // No extraction or runtime path is offered during the cooling period. + const cooled = fixture.install(); + expect(cooled.status).toBe(2); + expect(parseWslRuntimeRoot(cooled.stdout)).toBeNull(); + expect(cooled.stderr).toContain("using the mounted tree"); + + const expired = runShell( + `set -eu\ntouch -d "11 minutes ago" ${sh(`${fixture.runtimeParent}/.${fixture.runtimeId}.probe-failed`)}`, + ); + expect(expired.status, expired.stderr).toBe(0); + const repaired = fixture.install(); + expect(repaired.status, repaired.stderr).toBe(0); + expect(parseWslRuntimeRoot(repaired.stdout)).toBe(fixture.runtimeRoot); + }); + it("falls back instead of launching a corrupted cache it cannot reinstall", () => { const fixture = createFixture(); expect(fixture.install().status).toBe(0); @@ -664,7 +695,7 @@ describe.skipIf(posixShellRunner === null)("WSL runtime install script (executed expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0); }); - it("removes an aged stale tree after replacing an active unready cache", () => { + it("keeps an aged stale tree until its original runtime process exits", () => { const fixture = createFixture(); expect(fixture.install().status).toBe(0); const result = runShell( @@ -683,15 +714,51 @@ describe.skipIf(posixShellRunner === null)("WSL runtime install script (executed `HOME=${sh(`${fixture.work}/home`)}`, "export HOME", buildWslRuntimePruneScript(fixture.runtimeId, "pylon-code"), - 'test ! -e "$stale"', + 'test -d "$stale"', "kill $active_pid", "wait $active_pid 2>/dev/null || true", + buildWslRuntimePruneScript(fixture.runtimeId, "pylon-code"), + 'test ! -e "$stale"', ].join("\n"), ); expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0); }); + it("does not sweep aged extraction scratch while its install lock is held", () => { + const fixture = createFixture(); + const result = runShell( + [ + "set -eu", + `runtime_parent=${sh(fixture.runtimeParent)}`, + 'mkdir -p "$runtime_parent"', + `held_scratch="$runtime_parent/.${fixture.runtimeId}.tmp.test"`, + 'mkdir -p "$held_scratch"', + 'touch -d "180 minutes ago" "$held_scratch"', + // The writer signals only after flock has acquired the lock. -F and + // exec keep $! as the exact lock-owning sleep process for cleanup. + 'lock_ready="$runtime_parent/lock-ready.pipe"', + 'mkfifo "$lock_ready"', + `flock -x -F "$runtime_parent/.${fixture.runtimeId}.install.lock" sh -c 'printf "ready\\n" > "$1"; exec sleep 30' sh "$lock_ready" >/dev/null 2>&1 &`, + "lock_pid=$!", + 'trap \'kill "$lock_pid" 2>/dev/null || true; wait "$lock_pid" 2>/dev/null || true\' EXIT', + 'locked=$(timeout 5 cat "$lock_ready")', + 'test "$locked" = ready', + 'rm "$lock_ready"', + `HOME=${sh(`${fixture.work}/home`)}`, + "export HOME", + buildWslRuntimePruneScript(fixture.runtimeId, "pylon-code"), + 'test -d "$held_scratch"', + "kill $lock_pid", + "wait $lock_pid 2>/dev/null || true", + "trap - EXIT", + buildWslRuntimePruneScript(fixture.runtimeId, "pylon-code"), + 'test ! -e "$held_scratch"', + ].join("\n"), + ); + expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0); + }); + it("prunes old and markerless caches without touching retained, active, locked, or unrelated roots", () => { const result = runShell( [ diff --git a/apps/desktop/src/wsl/DesktopWslEnvironment.ts b/apps/desktop/src/wsl/DesktopWslEnvironment.ts index 3adf7d2498..f9c18a01ec 100644 --- a/apps/desktop/src/wsl/DesktopWslEnvironment.ts +++ b/apps/desktop/src/wsl/DesktopWslEnvironment.ts @@ -22,7 +22,9 @@ const WSLPATH_TIMEOUT = Duration.seconds(10); const PROBE_TIMEOUT = Duration.seconds(10); const TOOLCHAIN_TIMEOUT = Duration.seconds(10); const BUILD_TIMEOUT = Duration.minutes(5); -const RUNTIME_INSTALL_TIMEOUT = Duration.minutes(2); +// A cold extraction reads the archive from /mnt/c while holding an install lock. +// Slow disks and antivirus scans can take longer than two minutes. +const RUNTIME_INSTALL_TIMEOUT = Duration.minutes(10); const RUNTIME_PRUNE_TIMEOUT = Duration.seconds(30); const RUNTIME_INVALIDATE_TIMEOUT = Duration.seconds(15); const USER_HOME_TIMEOUT = Duration.seconds(5); @@ -261,6 +263,7 @@ const shellQuote = (value: string): string => `'${value.replaceAll("'", "'\\''") // that install wrote. const WSL_RUNTIME_READY_MARKER = ".t3code-wsl-runtime-ready"; const WSL_RUNTIME_SELECTED_MARKER = ".t3code-wsl-runtime-selected"; +const WSL_RUNTIME_PROBE_FAILURE_COOLDOWN_MINUTES = 10; // Pylon's own runtime home inside the distro, per channel — not `~/.t3`, which // belongs to T3 Code. This is a correctness boundary rather than branding: // `pruneRuntimes` deletes every `sha256-*` sibling it does not recognise as @@ -330,9 +333,18 @@ export const buildWslRuntimeInstallScript = ( "}", 'mkdir -p "$runtime_parent"', `runtime_lock="$runtime_parent/.${safeRuntimeId}.install.lock"`, + `probe_failure_marker="$runtime_parent/.${safeRuntimeId}.probe-failed"`, "trap 'exit 1' HUP INT TERM", 'exec 9> "$runtime_lock"', "flock -x 9", + // A staged binary that failed a real node-pty load while the mounted tree + // worked is unlikely to be repaired by reinstalling identical archive + // bytes on every desktop launch. Retry after a bounded cooling period. + `if [ -f "$probe_failure_marker" ] && find "$probe_failure_marker" -maxdepth 0 -mmin -${WSL_RUNTIME_PROBE_FAILURE_COOLDOWN_MINUTES} -print -quit | grep -q .; then`, + " printf 'The staged WSL runtime failed its native probe recently; using the mounted tree until retry.\\n' >&2", + " exit 2", + "fi", + 'rm -f -- "$probe_failure_marker"', "if runtime_is_ready; then", ` touch "$runtime_root/${WSL_RUNTIME_SELECTED_MARKER}"`, ` printf 'runtimeRoot:%s\\n' "$runtime_root"`, @@ -436,13 +448,16 @@ export const buildWslRuntimePruneScript = ( 'prune_lock="$runtime_parent/.prune.lock"', 'exec 8> "$prune_lock"', "flock -x 8", - // Without a way to see the distro's processes we cannot tell which caches - // are load-bearing, and the retention rules below are not safe on their own. - "[ -d /proc/1 ] || exit 0", + // Old extraction scratch can be swept without process info only after + // checking its stable per-runtime install lock. A timed-out host call does + // not prove that WSL's tar process has exited. + "have_process_info=0", + "[ -d /proc/1 ] && have_process_info=1", "runtime_in_use() {", ' grep -qF -- "$1/" /proc/[0-9]*/cmdline 2>/dev/null', "}", 'previous_runtime=""', + 'if [ "$have_process_info" = 1 ]; then', 'for candidate in "$runtime_parent"/sha256-*; do', ' [ -d "$candidate" ] || continue', ' [ "$candidate" != "$current_runtime" ] || continue', @@ -470,14 +485,38 @@ export const buildWslRuntimePruneScript = ( " continue", " fi", ' rm -rf -- "$candidate"', + // Keep the per-runtime flock pathname. Removing it after unlocking can + // split concurrent installers across old and new inodes. These tiny files + // are stable coordination state, even after their cache tree is pruned. " flock -u 9", "done", + "fi", // Interrupted installs use dot-prefixed names under this dedicated parent. - 'for scratch in "$runtime_parent"/.*.tmp.* "$runtime_parent"/.*.stale.*; do', + 'for scratch in "$runtime_parent"/.*.tmp.*; do', ' [ -d "$scratch" ] || continue', ` find "$scratch" -maxdepth 0 -mmin +${String(ORPHANED_RUNTIME_SCRATCH_MAX_AGE_MINUTES)} -print -quit | grep -q . || continue`, + " scratch_name=${scratch##*/}", + " scratch_runtime=${scratch_name#.}", + " scratch_runtime=${scratch_runtime%%.tmp.*}", + ' exec 9> "$runtime_parent/.${scratch_runtime}.install.lock"', + " flock -n 9 || continue", ' rm -rf -- "$scratch"', + " flock -u 9", "done", + 'if [ "$have_process_info" = 1 ]; then', + ' for scratch in "$runtime_parent"/.*.stale.*; do', + ' [ -d "$scratch" ] || continue', + ` find "$scratch" -maxdepth 0 -mmin +${String(ORPHANED_RUNTIME_SCRATCH_MAX_AGE_MINUTES)} -print -quit | grep -q . || continue`, + " stale_name=${scratch##*/}", + " original_name=${stale_name#.}", + " original_name=${original_name%%.stale.*}", + ' ! runtime_in_use "$runtime_parent/$original_name" || continue', + ' exec 9> "$runtime_parent/.${original_name}.install.lock"', + " flock -n 9 || continue", + ' rm -rf -- "$scratch"', + " flock -u 9", + " done", + "fi", ].join("\n"); }; @@ -495,7 +534,13 @@ export const buildWslRuntimeInvalidateScript = ( const safeRuntimeId = sanitizeWslRuntimeId(runtimeId); return [ "set -eu", - `rm -f "$HOME/.${runtimeHomeDirName}/wsl-runtime/${safeRuntimeId}/${WSL_RUNTIME_READY_MARKER}"`, + `runtime_parent=${wslRuntimeParent(runtimeHomeDirName)}`, + `[ -d "$runtime_parent" ] || exit 0`, + `runtime_lock="$runtime_parent/.${safeRuntimeId}.install.lock"`, + 'exec 9> "$runtime_lock"', + "flock -x 9", + `rm -f "$runtime_parent/${safeRuntimeId}/${WSL_RUNTIME_READY_MARKER}"`, + `touch "$runtime_parent/.${safeRuntimeId}.probe-failed"`, ].join("\n"); }; diff --git a/apps/mobile/src/components/AppSymbol.tsx b/apps/mobile/src/components/AppSymbol.tsx index 2f65531fa6..aba1023f79 100644 --- a/apps/mobile/src/components/AppSymbol.tsx +++ b/apps/mobile/src/components/AppSymbol.tsx @@ -7,6 +7,7 @@ import IconAlertTriangle from "@tabler/icons-react-native/IconAlertTriangle"; import IconApps from "@tabler/icons-react-native/IconApps"; import IconArchive from "@tabler/icons-react-native/IconArchive"; import IconArrowBackUp from "@tabler/icons-react-native/IconArrowBackUp"; +import IconArrowDown from "@tabler/icons-react-native/IconArrowDown"; import IconArrowDownCircle from "@tabler/icons-react-native/IconArrowDownCircle"; import IconArrowRightCircle from "@tabler/icons-react-native/IconArrowRightCircle"; import IconArrowUp from "@tabler/icons-react-native/IconArrowUp"; @@ -96,6 +97,7 @@ import { withUniwind } from "uniwind"; const ANDROID_ICON_BY_SF_SYMBOL: Partial> = { "arrow.branch": IconGitBranch, "arrow.clockwise": IconRefresh, + "arrow.down": IconArrowDown, "arrow.down.circle": IconArrowDownCircle, "arrow.right.circle": IconArrowRightCircle, "arrow.triangle.branch": IconGitBranch, diff --git a/apps/mobile/src/features/usage/UsageRouteScreen.tsx b/apps/mobile/src/features/usage/UsageRouteScreen.tsx index a515f2ddd6..131ad22c3a 100644 --- a/apps/mobile/src/features/usage/UsageRouteScreen.tsx +++ b/apps/mobile/src/features/usage/UsageRouteScreen.tsx @@ -286,6 +286,19 @@ export function UsageRouteScreen() { {merged.duplicateSources.join(", ")} ) : null} + {merged.approximateEnvironments.length > 0 ? ( + + Totals may count shared usage more than once because these environments run older + servers or have directories whose filesystem identity could not be read:{" "} + {selectedEnvironments + .filter((environment) => + merged.approximateEnvironments.includes(environment.environmentId), + ) + .map((environment) => environment.label) + .join(", ")} + . + + ) : null} {isPending ? ( Scanning provider transcripts… diff --git a/apps/server/scripts/acp-mock-agent.ts b/apps/server/scripts/acp-mock-agent.ts index 1aea931038..651eaf302a 100644 --- a/apps/server/scripts/acp-mock-agent.ts +++ b/apps/server/scripts/acp-mock-agent.ts @@ -38,6 +38,8 @@ const emitXAiAskUserQuestionThenHang = const emitContentThenHang = process.env.T3_ACP_EMIT_CONTENT_THEN_HANG === "1"; const emitPlanThenHang = process.env.T3_ACP_EMIT_PLAN_THEN_HANG === "1"; const emitActiveToolThenHang = process.env.T3_ACP_EMIT_ACTIVE_TOOL_THEN_HANG === "1"; +const emitGrokMonitorPostTurnPoll = process.env.T3_ACP_EMIT_GROK_MONITOR_POST_TURN_POLL === "1"; +const emitGrokBackgroundTaskStarted = process.env.T3_ACP_EMIT_GROK_BACKGROUND_TASK_STARTED === "1"; const emitForeignSessionUpdates = process.env.T3_ACP_EMIT_FOREIGN_SESSION_UPDATES === "1"; const emitPrivateThoughtChunk = process.env.T3_ACP_EMIT_PRIVATE_THOUGHT_CHUNK === "1"; const assistantMessageIdMode = process.env.T3_ACP_ASSISTANT_MESSAGE_IDS; @@ -935,6 +937,108 @@ const program = Effect.gen(function* () { return yield* Effect.never; } + if (emitGrokMonitorPostTurnPoll) { + const monitorCallId = "call-monitor-1"; + const pollCallId = "call-monitor-poll-1"; + const taskId = "01a05f41-5107-7550-821e-79e8d1cd7687"; + const description = "Watch count-sheet Typst unit until done"; + writeJsonRpcNotification("session/update", { + sessionId: requestedSessionId, + update: { + sessionUpdate: "tool_call", + toolCallId: monitorCallId, + title: "monitor", + kind: "other", + status: "pending", + rawInput: { description }, + _meta: { + "x.ai/tool": { version: 1, name: "monitor", kind: "task", namespace: "grok_build" }, + }, + }, + }); + writeJsonRpcNotification("session/update", { + sessionId: requestedSessionId, + update: { + sessionUpdate: "tool_call_update", + toolCallId: monitorCallId, + status: "completed", + rawInput: { description }, + rawOutput: { + type: "Monitor", + taskId, + timeoutMs: 36_000_000, + }, + }, + }); + writeJsonRpcNotification("_x.ai/session/prompt_complete", { + sessionId: requestedSessionId, + promptId: promptIdFromRequestMeta(request) ?? "mock-xai-prompt-1", + stopReason: "end_turn", + agentResult: null, + }); + yield* Effect.sleep("120 millis"); + writeJsonRpcNotification("session/update", { + sessionId: requestedSessionId, + update: { + sessionUpdate: "tool_call", + toolCallId: pollCallId, + title: "get_command_or_subagent_output", + kind: "other", + status: "completed", + rawInput: { variant: "TaskOutput", task_ids: [taskId], timeout_ms: 0 }, + rawOutput: { + type: "TaskOutput", + Result: { + task_id: taskId, + command: `[monitor] ${description}`, + status: "completed", + exit_code: 0, + output: "Monitor finished.", + }, + }, + }, + }); + return yield* Effect.never; + } + + if (emitGrokBackgroundTaskStarted) { + const toolCallId = "call-fb9d0000-0000-0000-0000-000000000026"; + const command = "sleep 40; echo done-a"; + writeJsonRpcNotification("session/update", { + sessionId: requestedSessionId, + update: { + sessionUpdate: "tool_call", + toolCallId, + title: "run_terminal_command", + kind: "execute", + status: "in_progress", + rawInput: { command }, + }, + }); + writeJsonRpcNotification("session/update", { + sessionId: requestedSessionId, + update: { + sessionUpdate: "tool_call_update", + toolCallId, + status: "completed", + rawOutput: { + type: "BackgroundTaskStarted", + task_id: toolCallId, + task_type: "bash", + status: "running", + command, + }, + }, + }); + writeJsonRpcNotification("_x.ai/session/prompt_complete", { + sessionId: requestedSessionId, + promptId: promptIdFromRequestMeta(request) ?? "mock-xai-prompt-1", + stopReason: "end_turn", + agentResult: null, + }); + return yield* Effect.never; + } + if (emitInterleavedAssistantToolCalls) { const toolCallId = "tool-call-1"; diff --git a/apps/server/src/bin.test.ts b/apps/server/src/bin.test.ts index 42fbb7815c..d6163bb463 100644 --- a/apps/server/src/bin.test.ts +++ b/apps/server/src/bin.test.ts @@ -16,6 +16,7 @@ import { import * as NetService from "@t3tools/shared/Net"; import { HostProcessEnvironment } from "@t3tools/shared/hostProcess"; import { DEFAULT_SIGNAL_EXPORT } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import { assert, it } from "@effect/vitest"; import * as Effect from "effect/Effect"; import * as DateTime from "effect/DateTime"; @@ -106,6 +107,7 @@ const makeCliTestServerConfig = (baseDir: string) => otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "t3-server", + otelEnvironment: OtelEnvironment.none, mode: "web", port: 0, host: "127.0.0.1", diff --git a/apps/server/src/checkpointing/Utils.ts b/apps/server/src/checkpointing/Utils.ts index adc089f624..88ae5b9369 100644 --- a/apps/server/src/checkpointing/Utils.ts +++ b/apps/server/src/checkpointing/Utils.ts @@ -3,10 +3,12 @@ import { CheckpointRef, ProjectId, type ThreadId } from "@t3tools/contracts"; const CHECKPOINT_REFS_PREFIX = "refs/t3/checkpoints"; +export function checkpointRefPrefixForThread(threadId: ThreadId): string { + return `${CHECKPOINT_REFS_PREFIX}/${Encoding.encodeBase64Url(threadId)}/`; +} + export function checkpointRefForThreadTurn(threadId: ThreadId, turnCount: number): CheckpointRef { - return CheckpointRef.make( - `${CHECKPOINT_REFS_PREFIX}/${Encoding.encodeBase64Url(threadId)}/turn/${turnCount}`, - ); + return CheckpointRef.make(`${checkpointRefPrefixForThread(threadId)}turn/${turnCount}`); } export function resolveThreadWorkspaceCwd(input: { diff --git a/apps/server/src/cli/config.test.ts b/apps/server/src/cli/config.test.ts index b15787efeb..20fcf45891 100644 --- a/apps/server/src/cli/config.test.ts +++ b/apps/server/src/cli/config.test.ts @@ -18,6 +18,7 @@ import { } from "@t3tools/contracts"; import * as NetService from "@t3tools/shared/Net"; import { DEFAULT_SIGNAL_EXPORT } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import * as NodeServices from "@effect/platform-node/NodeServices"; import * as TestConsole from "effect/testing/TestConsole"; import { Command } from "effect/unstable/cli"; @@ -58,7 +59,8 @@ it.layer(NodeServices.layer)("cli config resolution", (it) => { otlpTracesExport: DEFAULT_SIGNAL_EXPORT, otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, - otlpServiceName: "t3-server", + otlpServiceName: "pylon-server", + otelEnvironment: OtelEnvironment.none, devAllowedOrigins: [], } as const; @@ -582,6 +584,109 @@ it.layer(NodeServices.layer)("cli config resolution", (it) => { }), ); + it.effect("zeroes an endpoint stored in Settings when the SDK is disabled", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const baseDir = yield* fs.makeTempDirectoryScoped({ prefix: "t3-cli-config-otel-off-" }); + const derivedPaths = yield* deriveExplicitServerPaths(baseDir, undefined); + yield* fs.makeDirectory(path.dirname(derivedPaths.settingsPath), { recursive: true }); + yield* fs.writeFileString( + derivedPaths.settingsPath, + // @effect-diagnostics-next-line preferSchemaOverJson:off + `${JSON.stringify({ + observability: { + otlpTracesUrl: "http://localhost:4318/v1/traces", + otlpMetricsUrl: "http://localhost:4318/v1/metrics", + otlpLogsUrl: "http://localhost:4318/v1/logs", + }, + })}\n`, + ); + + const resolved = yield* resolveServerConfig( + { + mode: Option.some("desktop"), + port: Option.some(4888), + host: Option.none(), + baseDir: Option.some(baseDir), + cwd: Option.none(), + devUrl: Option.none(), + noBrowser: Option.none(), + bootstrapFd: Option.none(), + autoBootstrapProjectFromCwd: Option.none(), + logWebSocketEvents: Option.none(), + tailscaleServeEnabled: Option.none(), + tailscaleServePort: Option.none(), + }, + Option.none(), + ).pipe( + Effect.provide( + Layer.mergeAll( + ConfigProvider.layer(ConfigProvider.fromEnv({ env: { OTEL_SDK_DISABLED: "true" } })), + NetService.layer, + ), + ), + ); + + // The switch beats every source, including an endpoint stored in Settings. + expect(resolved.otlpTracesUrl).toBeUndefined(); + expect(resolved.otlpMetricsUrl).toBeUndefined(); + expect(resolved.otlpLogsUrl).toBeUndefined(); + expect(resolved.otelEnvironment.disabled).toBe(true); + }), + ); + + it.effect("lets T3CODE_OTEL_SDK_DISABLED=false override an ambient OTEL_SDK_DISABLED=true", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const baseDir = yield* fs.makeTempDirectoryScoped({ prefix: "t3-cli-config-otel-on-" }); + const derivedPaths = yield* deriveExplicitServerPaths(baseDir, undefined); + yield* fs.makeDirectory(path.dirname(derivedPaths.settingsPath), { recursive: true }); + yield* fs.writeFileString( + derivedPaths.settingsPath, + // @effect-diagnostics-next-line preferSchemaOverJson:off + `${JSON.stringify({ + observability: { + otlpTracesUrl: "http://localhost:4318/v1/traces", + }, + })}\n`, + ); + + const resolved = yield* resolveServerConfig( + { + mode: Option.some("desktop"), + port: Option.some(4888), + host: Option.none(), + baseDir: Option.some(baseDir), + cwd: Option.none(), + devUrl: Option.none(), + noBrowser: Option.none(), + bootstrapFd: Option.none(), + autoBootstrapProjectFromCwd: Option.none(), + logWebSocketEvents: Option.none(), + tailscaleServeEnabled: Option.none(), + tailscaleServePort: Option.none(), + }, + Option.none(), + ).pipe( + Effect.provide( + Layer.mergeAll( + ConfigProvider.layer( + ConfigProvider.fromEnv({ + env: { T3CODE_OTEL_SDK_DISABLED: "false", OTEL_SDK_DISABLED: "true" }, + }), + ), + NetService.layer, + ), + ), + ); + + expect(resolved.otelEnvironment.disabled).toBe(false); + expect(resolved.otlpTracesUrl).toBe("http://localhost:4318/v1/traces"); + }), + ); + it.effect("forces noBrowser and disables auto-bootstrap for headless startup presentation", () => Effect.gen(function* () { const { join } = yield* Path.Path; diff --git a/apps/server/src/cli/config.ts b/apps/server/src/cli/config.ts index c876450505..3e7d685492 100644 --- a/apps/server/src/cli/config.ts +++ b/apps/server/src/cli/config.ts @@ -4,6 +4,7 @@ import { OtlpProtocol, type SignalExport, } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import { parsePersistedServerObservabilitySettings } from "@t3tools/shared/serverSettings"; import { DesktopBackendBootstrap, PortSchema } from "@t3tools/contracts"; import * as Config from "effect/Config"; @@ -106,7 +107,9 @@ const EnvServerConfig = Config.all({ otlpExportIntervalMs: Config.int("T3CODE_OTLP_EXPORT_INTERVAL_MS").pipe( Config.withDefault(10_000), ), - otlpServiceName: Config.string("T3CODE_OTLP_SERVICE_NAME").pipe(Config.withDefault("t3-server")), + otlpServiceName: Config.string("T3CODE_OTLP_SERVICE_NAME").pipe( + Config.withDefault("pylon-server"), + ), otlpHeaders: Config.schema(OtlpHeadersFromString, "T3CODE_OTLP_HEADERS").pipe( Config.option, Config.map(Option.getOrUndefined), @@ -372,7 +375,9 @@ export const resolveServerConfig = ( ); const logLevel = Option.getOrElse(cliLogLevel, () => env.logLevel); - // Pylon's own OTLP variables name no signal, so the one answer they give + const otel = yield* OtelEnvironment.load; + + // Pylon's inherited OTLP variables name no signal, so the one answer they give // is the answer for all three. const signalExport: SignalExport = { protocol: env.otlpProtocol, @@ -387,20 +392,24 @@ export const resolveServerConfig = ( traceBatchWindowMs: env.traceBatchWindowMs, traceMaxBytes: env.traceMaxBytes, traceMaxFiles: env.traceMaxFiles, - otlpTracesUrl: - env.otlpTracesUrl ?? - bootstrap?.otlpTracesUrl ?? - persistedObservabilitySettings.otlpTracesUrl, - otlpMetricsUrl: - env.otlpMetricsUrl ?? - bootstrap?.otlpMetricsUrl ?? - persistedObservabilitySettings.otlpMetricsUrl, - otlpLogsUrl: - env.otlpLogsUrl ?? bootstrap?.otlpLogsUrl ?? persistedObservabilitySettings.otlpLogsUrl, + otlpTracesUrl: otel.disabled + ? undefined + : (env.otlpTracesUrl ?? + bootstrap?.otlpTracesUrl ?? + persistedObservabilitySettings.otlpTracesUrl), + otlpMetricsUrl: otel.disabled + ? undefined + : (env.otlpMetricsUrl ?? + bootstrap?.otlpMetricsUrl ?? + persistedObservabilitySettings.otlpMetricsUrl), + otlpLogsUrl: otel.disabled + ? undefined + : (env.otlpLogsUrl ?? bootstrap?.otlpLogsUrl ?? persistedObservabilitySettings.otlpLogsUrl), otlpTracesExport: signalExport, otlpMetricsExport: signalExport, otlpLogsExport: signalExport, otlpServiceName: env.otlpServiceName, + otelEnvironment: otel, mode, port, cwd, diff --git a/apps/server/src/cli/pair.ts b/apps/server/src/cli/pair.ts index 801d877c9c..6fc0da6a5c 100644 --- a/apps/server/src/cli/pair.ts +++ b/apps/server/src/cli/pair.ts @@ -16,6 +16,7 @@ import { } from "@t3tools/contracts"; import { resolveWorktreeT3Home } from "@t3tools/shared/devHome"; import { DEFAULT_SIGNAL_EXPORT } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import { buildTailscaleHttpsBaseUrl, DEFAULT_TAILSCALE_SERVE_PORT, @@ -326,6 +327,7 @@ const makePairServerConfig = Effect.fn(function* (input: { otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "pylon-server", + otelEnvironment: OtelEnvironment.none, mode: "web", port: state.port, host: state.host, diff --git a/apps/server/src/config.ts b/apps/server/src/config.ts index 573d61432e..b6738325ac 100644 --- a/apps/server/src/config.ts +++ b/apps/server/src/config.ts @@ -17,6 +17,7 @@ import * as Schema from "effect/Schema"; import { sweepStalePendingAttachments } from "./attachmentStore.ts"; import { DEFAULT_SIGNAL_EXPORT, type SignalExport } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; export const DEFAULT_PORT = 3773; @@ -81,6 +82,7 @@ export class ServerConfig extends Context.Service< readonly otlpMetricsExport: SignalExport; readonly otlpLogsExport: SignalExport; readonly otlpServiceName: string; + readonly otelEnvironment: OtelEnvironment.OtelEnvironment; readonly mode: RuntimeMode; readonly port: number; readonly host: string | undefined; @@ -215,6 +217,7 @@ const makeTest = Effect.fn("ServerConfig.makeTest")(function* ( otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "pylon-server", + otelEnvironment: OtelEnvironment.none, cwd, baseDir, ...derivedPaths, diff --git a/apps/server/src/device/LocalDeviceHost.test.ts b/apps/server/src/device/LocalDeviceHost.test.ts index 037e05186b..e6b608aa45 100644 --- a/apps/server/src/device/LocalDeviceHost.test.ts +++ b/apps/server/src/device/LocalDeviceHost.test.ts @@ -20,17 +20,21 @@ import * as NetService from "@t3tools/shared/Net"; import * as ServerConfig from "../config.ts"; import * as ProcessRunner from "../processRunner.ts"; -const diagnose = (files: ReadonlyArray, environment: NodeJS.ProcessEnv) => +const diagnose = ( + files: ReadonlyArray, + environment: NodeJS.ProcessEnv, + platform: NodeJS.Platform = "darwin", +) => LocalDeviceHost.__testing.platformReason("android").pipe( Effect.provideService(HostProcessEnvironment, environment), - Effect.provideService(HostProcessPlatform, "darwin"), + Effect.provideService(HostProcessPlatform, platform), Effect.provideService( FileSystem.FileSystem, FileSystem.makeNoop({ exists: (file) => Effect.succeed(files.includes(file)), }), ), - Effect.provide(NodePath.layer), + Effect.provide(platform === "win32" ? NodePath.layerWin32 : NodePath.layerPosix), ); describe("Android SDK availability", () => { @@ -46,7 +50,50 @@ describe("Android SDK availability", () => { const reason = yield* diagnose(["/sdk/platform-tools/adb", "/sdk/emulator/emulator"], { ANDROID_HOME: "/sdk", }); - expect(reason).toContain("Command-line Tools (latest)"); + expect(reason).toContain("Command-line Tools (latest) are missing"); + }), + ); + + it.effect("explains how to upgrade legacy command-line tools", () => + Effect.gen(function* () { + const root = "/test/home/Library/Android/sdk"; + const reason = yield* diagnose( + [`${root}/platform-tools/adb`, `${root}/emulator/emulator`, `${root}/tools/bin/avdmanager`], + { HOME: "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/test/home" }, + ); + expect(reason).toContain("older, unsupported version"); + expect(reason).toContain(root); + expect(reason).toContain("Android Studio's SDK Manager under SDK Tools"); + }), + ); + + it.effect("recognizes legacy command-line tools on Windows", () => + Effect.gen(function* () { + const reason = yield* diagnose( + [ + "C:\\Android\\Sdk\\platform-tools\\adb.exe", + "C:\\Android\\Sdk\\emulator\\emulator.exe", + "C:\\Android\\Sdk\\tools\\bin\\avdmanager.bat", + ], + { ANDROID_HOME: "C:\\Android\\Sdk" }, + "win32", + ); + expect(reason).toContain("older, unsupported version"); + }), + ); + + it.effect("accepts latest tools even if legacy tools are installed", () => + Effect.gen(function* () { + const reason = yield* diagnose( + [ + "/sdk/platform-tools/adb", + "/sdk/emulator/emulator", + "/sdk/tools/bin/avdmanager", + "/sdk/cmdline-tools/latest/bin/avdmanager", + ], + { ANDROID_HOME: "/sdk" }, + ); + expect(reason).toBeNull(); }), ); diff --git a/apps/server/src/device/LocalDeviceHost.ts b/apps/server/src/device/LocalDeviceHost.ts index 28143f5b8e..139034a178 100644 --- a/apps/server/src/device/LocalDeviceHost.ts +++ b/apps/server/src/device/LocalDeviceHost.ts @@ -109,8 +109,11 @@ const platformReason = Effect.fn("LocalDeviceHost.platformReason")(function* ( return `Android SDK Platform-Tools are missing from ${sdk.root}. Install them in Android Studio's SDK Manager.`; if (!sdk.emulator) return `Android Emulator is missing from ${sdk.root}. Install it in Android Studio's SDK Manager.`; - if (!sdk.avdmanager) + if (!sdk.avdmanager) { + if (sdk.legacyAvdmanager) + return `The Android SDK command-line tools in ${sdk.root} appear to be an older, unsupported version. Install Android SDK Command-line Tools (latest) in Android Studio's SDK Manager under SDK Tools.`; return `Android SDK Command-line Tools (latest) are missing from ${sdk.root}. Install them in Android Studio's SDK Manager.`; + } return null; }); @@ -160,10 +163,15 @@ const androidSdk = Effect.gen(function* () { platform === "win32" ? "avdmanager.bat" : "avdmanager", ), ); - return { root, adb, emulator, avdmanager }; + const legacyAvdmanager = + !avdmanager && + (yield* exists( + path.join(root, "tools", "bin", platform === "win32" ? "avdmanager.bat" : "avdmanager"), + )); + return { root, adb, emulator, avdmanager, legacyAvdmanager }; } } - return { root: null, adb: false, emulator: false, avdmanager: false }; + return { root: null, adb: false, emulator: false, avdmanager: false, legacyAvdmanager: false }; }); const deviceHostEnvironment = ( diff --git a/apps/server/src/environment/ServerEnvironment.test.ts b/apps/server/src/environment/ServerEnvironment.test.ts index a1b9493b13..71f228b3dc 100644 --- a/apps/server/src/environment/ServerEnvironment.test.ts +++ b/apps/server/src/environment/ServerEnvironment.test.ts @@ -11,6 +11,7 @@ import * as PlatformError from "effect/PlatformError"; import * as Schema from "effect/Schema"; import { DEFAULT_SIGNAL_EXPORT } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import * as ServerSecretStore from "../auth/ServerSecretStore.ts"; import { @@ -60,6 +61,7 @@ const makeServerConfig = Effect.fn(function* (baseDir: string) { otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "t3-server", + otelEnvironment: OtelEnvironment.none, cwd: process.cwd(), baseDir, mode: "web", diff --git a/apps/server/src/environment/ServerEnvironment.ts b/apps/server/src/environment/ServerEnvironment.ts index 951003a271..d63e44bcd9 100644 --- a/apps/server/src/environment/ServerEnvironment.ts +++ b/apps/server/src/environment/ServerEnvironment.ts @@ -226,6 +226,7 @@ export const make = Effect.gen(function* () { threadAutoSettlement: true, threadRestartContinuation: true, projectSettingsOverrides: true, + worktreeSubmodules: true, defaultRuntimeMode: true, threadSnooze: true, environmentThemes: true, diff --git a/apps/server/src/git/GitManager.test.ts b/apps/server/src/git/GitManager.test.ts index d4d22be1d2..a1b1062666 100644 --- a/apps/server/src/git/GitManager.test.ts +++ b/apps/server/src/git/GitManager.test.ts @@ -15,6 +15,7 @@ import * as PlatformError from "effect/PlatformError"; import * as References from "effect/References"; import * as Schema from "effect/Schema"; import * as Scope from "effect/Scope"; +import { TestClock } from "effect/testing"; import { ChildProcessSpawner } from "effect/unstable/process"; import { expect } from "vite-plus/test"; import type { @@ -1302,7 +1303,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { prListByHeadSelector: { // Fake gh returns raw JSON stdout, matching the CLI boundary under test. // @effect-diagnostics-next-line preferSchemaOverJson:off - "contributor:feature/deleted-fork-branch": JSON.stringify([ + "feature/deleted-fork-branch": JSON.stringify([ { number: 218, title: "Deleted fork branch PR", @@ -1332,8 +1333,9 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { updatedAt: "2026-04-05T15:00:00.000Z", }); expect( - ghCalls.some((call) => call.includes("--head contributor:feature/deleted-fork-branch")), + ghCalls.some((call) => call.includes("--head feature/deleted-fork-branch --state all")), ).toBe(true); + expect(ghCalls.some((call) => call.includes("--head contributor:"))).toBe(false); }), ); @@ -1484,6 +1486,61 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { }), ); + it.effect("branch PR lookup rechecks open PRs every minute and settled answers less often", () => + Effect.gen(function* () { + const repoDir = yield* makeTempDir("t3code-git-manager-"); + yield* initRepo(repoDir); + const remoteDir = yield* createBareRemote(); + yield* runGit(repoDir, ["remote", "add", "origin", remoteDir]); + for (const branch of ["feature/open-pr", "feature/merged-pr", "feature/no-pr"]) { + yield* runGit(repoDir, ["checkout", "-b", branch, "main"]); + yield* runGit(repoDir, ["push", "-u", "origin", branch]); + } + const pullRequest = (number: number, headRefName: string, state: string) => ({ + number, + title: headRefName, + url: `https://github.com/pingdotgg/codething-mvp/pull/${number}`, + baseRefName: "main", + headRefName, + state, + updatedAt: "2026-04-07T15:00:00Z", + }); + const { manager, ghCalls } = yield* makeManager({ + ghScenario: { + prListByHeadSelector: { + "feature/open-pr": encodeCliJson([pullRequest(301, "feature/open-pr", "OPEN")]), + "feature/merged-pr": encodeCliJson([pullRequest(302, "feature/merged-pr", "MERGED")]), + }, + }, + }); + const lookupAll = Effect.forEach( + ["feature/open-pr", "feature/merged-pr", "feature/no-pr"], + (branch) => manager.branchPullRequest({ cwd: repoDir, branch }), + ); + const prListCalls = () => ghCalls.filter((call) => call.startsWith("pr list ")); + + yield* lookupAll; + expect(prListCalls()).toHaveLength(3); + + yield* TestClock.adjust("61 seconds"); + yield* lookupAll; + expect(prListCalls()).toHaveLength(4); + expect(prListCalls().at(-1)).toContain("--head feature/open-pr"); + + // Just inside the 5-minute window only the open PR is asked again. + yield* TestClock.adjust("238 seconds"); + yield* lookupAll; + expect(prListCalls()).toHaveLength(5); + expect(prListCalls().at(-1)).toContain("--head feature/open-pr"); + + // Just past it the settled answers expire too. + yield* TestClock.adjust("2 seconds"); + yield* lookupAll; + expect(prListCalls()).toHaveLength(7); + expect(prListCalls().slice(-2).join("\n")).not.toContain("--head feature/open-pr"); + }), + ); + it.effect("branch PR lookup propagates provider failures", () => Effect.gen(function* () { const repoDir = yield* makeTempDir("t3code-git-manager-"); @@ -1780,8 +1837,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const { manager } = yield* makeManager({ ghScenario: { prListByHeadSelector: { - "alice:feature": output, - "fork:feature": output, feature: output, }, }, @@ -1948,10 +2003,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const { manager, ghCalls } = yield* makeManager({ ghScenario: { prListSequence: [ - // @effect-diagnostics-next-line preferSchemaOverJson:off - JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - JSON.stringify([]), // @effect-diagnostics-next-line preferSchemaOverJson:off JSON.stringify([ { @@ -1987,7 +2038,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { updatedAt: "2026-03-10T07:00:00.000Z", }); expect(ghCalls).toContain( - "pr list --head jasonLaster:statemachine --state all --limit 20 --json number,title,url,baseRefName,headRefName,state,isDraft,mergedAt,closedAt,updatedAt,isCrossRepository,headRepository,headRepositoryOwner", + "pr list --head statemachine --state all --limit 100 --json number,title,url,baseRefName,headRefName,state,isDraft,mergedAt,closedAt,updatedAt,isCrossRepository,headRepository,headRepositoryOwner", ); }), 20_000, @@ -2019,7 +2070,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { ghScenario: { prListByHeadSelector: { // @effect-diagnostics-next-line preferSchemaOverJson:off - "contributor:main": JSON.stringify([ + main: JSON.stringify([ { number: 777, title: "Fork PR from main", @@ -2053,7 +2104,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { updatedAt: "2026-03-10T07:00:00.000Z", }); expect(ghCalls).toContain( - "pr list --head contributor:main --state all --limit 20 --json number,title,url,baseRefName,headRefName,state,isDraft,mergedAt,closedAt,updatedAt,isCrossRepository,headRepository,headRepositoryOwner", + "pr list --head main --state all --limit 100 --json number,title,url,baseRefName,headRefName,state,isDraft,mergedAt,closedAt,updatedAt,isCrossRepository,headRepository,headRepositoryOwner", ); }), 20_000, @@ -2118,34 +2169,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { updatedAt: "2026-04-01T10:00:00Z", }, ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "pingdotgg:effect-atom": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "my-org/upstream:effect-atom": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "pingdotgg:upstream/effect-atom": JSON.stringify([ - { - number: 1518, - title: "Wrong PR", - url: "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/pingdotgg/t3code/pull/1518", - baseRefName: "main", - headRefName: "upstream/effect-atom", - state: "OPEN", - updatedAt: "2026-04-01T10:00:00Z", - }, - ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "my-org/upstream:upstream/effect-atom": JSON.stringify([ - { - number: 1518, - title: "Wrong PR", - url: "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/pingdotgg/t3code/pull/1518", - baseRefName: "main", - headRefName: "upstream/effect-atom", - state: "OPEN", - updatedAt: "2026-04-01T10:00:00Z", - }, - ]), }, }, }); @@ -2356,7 +2379,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { prListByHeadSelector: { // Fake gh returns raw JSON stdout, matching the CLI boundary under test. // @effect-diagnostics-next-line preferSchemaOverJson:off - "contributor:feature/fork-plain": JSON.stringify([ + "feature/fork-plain": JSON.stringify([ { number: 89, title: "Fork PR pushed without -u", @@ -2376,9 +2399,8 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const status = yield* manager.status({ cwd: repoDir }); expect(status.pr?.number).toBe(89); - expect(ghCalls.some((call) => call.includes("--head contributor:feature/fork-plain"))).toBe( - true, - ); + expect(ghCalls.some((call) => call.includes("--head feature/fork-plain"))).toBe(true); + expect(ghCalls.some((call) => call.includes("--head contributor:"))).toBe(false); expect(ghCalls.some((call) => call.includes("--head main"))).toBe(false); }), ); @@ -2413,7 +2435,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { prListByHeadSelector: { // Fake gh returns raw JSON stdout, matching the CLI boundary under test. // @effect-diagnostics-next-line preferSchemaOverJson:off - "contributor:feature/fork-settle": JSON.stringify([ + "feature/fork-settle": JSON.stringify([ { number: 91, title: "Fork PR to settle", @@ -3697,7 +3719,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { ); it.effect( - "returns existing cross-repo PR metadata using the fork owner selector", + "returns existing cross-repo PR metadata found under the bare branch name", () => Effect.gen(function* () { const repoDir = yield* makeTempDir("t3code-git-manager-"); @@ -3716,8 +3738,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const { manager, ghCalls } = yield* makeManager({ ghScenario: { prListSequence: [ - // @effect-diagnostics-next-line preferSchemaOverJson:off - JSON.stringify([]), // @effect-diagnostics-next-line preferSchemaOverJson:off JSON.stringify([ { @@ -3749,7 +3769,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { expect(result.pr.number).toBe(142); expect( ghCalls.some((call) => - call.includes("pr list --head octocat:statemachine --state open --limit 1"), + call.includes("pr list --head statemachine --state open --limit 100"), ), ).toBe(true); expect(ghCalls.some((call) => call.startsWith("pr create "))).toBe(false); @@ -3815,30 +3835,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { headRefName: "upstream/effect-atom", }, ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "pingdotgg:effect-atom": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "my-org/upstream:effect-atom": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "pingdotgg:upstream/effect-atom": JSON.stringify([ - { - number: 1518, - title: "Wrong PR", - url: "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/pingdotgg/t3code/pull/1518", - baseRefName: "main", - headRefName: "upstream/effect-atom", - }, - ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "my-org/upstream:upstream/effect-atom": JSON.stringify([ - { - number: 1518, - title: "Wrong PR", - url: "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/pingdotgg/t3code/pull/1518", - baseRefName: "main", - headRefName: "upstream/effect-atom", - }, - ]), }, }, }); @@ -3858,7 +3854,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { ); it.effect( - "prefers owner-qualified selectors before bare branch names for cross-repo PRs", + "picks the fork PR among same-named branches from other repositories", () => Effect.gen(function* () { const repoDir = yield* makeTempDir("t3code-git-manager-"); @@ -3890,9 +3886,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { baseRefName: "main", headRefName: "statemachine", }, - ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "octocat:statemachine": JSON.stringify([ { number: 142, title: "Existing fork PR", @@ -3909,8 +3902,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { }, }, ]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - "fork-seed:statemachine": JSON.stringify([]), }, }, }); @@ -3922,11 +3913,9 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { expect(result.pr.status).toBe("opened_existing"); expect(result.pr.number).toBe(142); - - const ownerSelectorCallIndex = ghCalls.findIndex((call) => - call.includes("pr list --head octocat:statemachine --state open --limit 1"), - ); - expect(ownerSelectorCallIndex).toBeGreaterThanOrEqual(0); + expect( + ghCalls.some((call) => /--head [^ ]*:/u.test(call) && call.startsWith("pr list")), + ).toBe(false); expect(ghCalls.some((call) => call.startsWith("pr create "))).toBe(false); }), 12_000, @@ -3955,7 +3944,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { ghScenario: { prListByHeadSelector: { // @effect-diagnostics-next-line preferSchemaOverJson:off - "octocat:statemachine": JSON.stringify([ + statemachine: JSON.stringify([ { number: 142, title: "Existing fork PR", @@ -3973,11 +3962,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { }, ]), // @effect-diagnostics-next-line preferSchemaOverJson:off - "fork-seed:statemachine": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off "t3code/pr-142/statemachine": JSON.stringify([]), - // @effect-diagnostics-next-line preferSchemaOverJson:off - statemachine: JSON.stringify([]), }, }, }); @@ -3990,10 +3975,10 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { expect(result.pr.status).toBe("opened_existing"); expect(result.pr.number).toBe(142); - const openLookupCalls = ghCalls.filter((call) => call.includes("--state open --limit 1")); + const openLookupCalls = ghCalls.filter((call) => call.includes("--state open --limit 100")); expect(openLookupCalls).toHaveLength(1); expect(openLookupCalls[0]).toContain( - "pr list --head octocat:statemachine --state open --limit 1", + "pr list --head statemachine --state open --limit 100", ); }), 12_000, @@ -4018,12 +4003,10 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const { manager, ghCalls } = yield* makeManager({ ghScenario: { prListSequenceByHeadSelector: { - "octocat:statemachine": [ + statemachine: [ `[{"number":41,"title":"Ambiguous fork PR","url":"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/pingdotgg/codething-mvp/pull/41","baseRefName":"main","headRefName":"statemachine","state":"OPEN"}]`, `[{"number":142,"title":"Add stacked git actions","url":"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/pingdotgg/codething-mvp/pull/142","baseRefName":"main","headRefName":"statemachine","state":"OPEN","isCrossRepository":true,"headRepository":{"nameWithOwner":"octocat/codething-mvp"},"headRepositoryOwner":{"login":"octocat"}}]`, ], - "fork-seed:statemachine": ["[]"], - statemachine: ["[]"], }, }, }); @@ -4191,7 +4174,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { }), ); - it.effect("generates PR content against the remote base when the local base is stale", () => + it.effect("generates PR content from branch changes when the remote base advances", () => Effect.gen(function* () { const repoDir = yield* makeTempDir("t3code-git-manager-"); yield* initRepo(repoDir); @@ -4223,7 +4206,15 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { yield* runGit(repoDir, ["push", "-u", "origin", "feature/remote-base"]); yield* runGit(repoDir, ["config", "branch.feature/remote-base.gh-merge-base", "main"]); + NodeFS.writeFileSync(NodePath.join(peerDir, "later-main.txt"), "unrelated\n"); + yield* runGit(peerDir, ["add", "later-main.txt"]); + yield* runGit(peerDir, ["commit", "-m", "Later main commit"]); + yield* runGit(peerDir, ["push", "origin", "main"]); + yield* runGit(repoDir, ["fetch", "origin"]); + let generatedCommitSummary = ""; + let generatedDiffSummary = ""; + let generatedDiffPatch = ""; const { manager } = yield* makeManager({ ghScenario: { prListSequence: ["[]", "[]"], @@ -4231,6 +4222,8 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { textGeneration: { generatePrContent: (input) => { generatedCommitSummary = input.commitSummary; + generatedDiffSummary = input.diffSummary; + generatedDiffPatch = input.diffPatch; return Effect.succeed({ title: "Feature PR", body: "Feature body" }); }, }, @@ -4244,6 +4237,11 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { expect(result.pr.status).toBe("created"); expect(generatedCommitSummary).toContain("Feature commit"); expect(generatedCommitSummary).not.toContain("Remote base commit"); + expect(generatedCommitSummary).not.toContain("Later main commit"); + expect(generatedDiffSummary).toContain("feature.txt"); + expect(generatedDiffSummary).not.toContain("later-main.txt"); + expect(generatedDiffPatch).toContain("feature.txt"); + expect(generatedDiffPatch).not.toContain("later-main.txt"); }), ); @@ -4344,7 +4342,7 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { const { manager, ghCalls } = yield* makeManager({ ghScenario: { prListSequenceByHeadSelector: { - "octocat:statemachine": [ + statemachine: [ // @effect-diagnostics-next-line preferSchemaOverJson:off JSON.stringify([]), // @effect-diagnostics-next-line preferSchemaOverJson:off @@ -4366,10 +4364,6 @@ it.layer(GitManagerTestLayer)("GitManager", (it) => { }, ]), ], - // @effect-diagnostics-next-line preferSchemaOverJson:off - "fork-seed:statemachine": [JSON.stringify([])], - // @effect-diagnostics-next-line preferSchemaOverJson:off - statemachine: [JSON.stringify([])], }, }, }); diff --git a/apps/server/src/git/GitManager.ts b/apps/server/src/git/GitManager.ts index 8bec757d12..0c93732bfd 100644 --- a/apps/server/src/git/GitManager.ts +++ b/apps/server/src/git/GitManager.ts @@ -31,6 +31,7 @@ import { ModelSelection, type ProjectId, SourceControlProviderError, + type SourceControlProviderKind, type SourceControlWritingStyleSettings, type ThreadId, } from "@t3tools/contracts"; @@ -143,6 +144,12 @@ const STATUS_RESULT_CACHE_CAPACITY = 2_048; // exponentially via prLookupFailureTtl, so throttling pressure still drops // under 429s instead of amplifying it. const PR_LOOKUP_CACHE_TTL = Duration.seconds(60); +// Answers without an open PR ("no PR yet", merged, closed) only change when +// someone opens a PR, and the paths that do that in-app (turn end, push, +// create PR, user refresh) bypass this cache. Re-asking every minute for each +// idle branch was the bulk of background GitHub quota use, so these wait out +// a longer TTL and a PR opened outside the app shows up within minutes. +const PR_LOOKUP_NO_OPEN_PR_CACHE_TTL = Duration.minutes(5); const PR_LOOKUP_FAILURE_BASE_TTL = Duration.seconds(20); const PR_LOOKUP_FAILURE_MAX_TTL = Duration.minutes(15); const PR_LOOKUP_CACHE_CAPACITY = 2_048; @@ -578,6 +585,26 @@ function parseCustomCommitMessage(raw: string): { subject: string; body: string }; } +// Without the owner selector, a bare branch name also lists same-named +// branches on other forks (`main`, `patch-1`), so GitHub probes ask for a full +// page and let matchesBranchHeadContext pick the right head. gh fetches up to +// 100 in one request, and GitHub prices a first:100 connection like first:1. +const GITHUB_HEAD_BRANCH_PROBE_LIMIT = 100; + +// `gh pr list --head` filters on the head ref name alone and accepts anything, so an +// `owner:branch` or `remote:branch` selector silently lists zero pull requests +// while spending a GraphQL call. Git branch names cannot contain ":", and the +// bare head branch is always among the selectors, so GitHub probes skip them and +// leave the owner check to matchesBranchHeadContext. +function probeableHeadSelectors( + providerKind: SourceControlProviderKind, + headSelectors: ReadonlyArray, +): ReadonlyArray { + return providerKind === "github" + ? headSelectors.filter((selector) => !selector.includes(":")) + : headSelectors; +} + function appendUnique(values: string[], next: string | null | undefined): void { const trimmed = next?.trim() ?? ""; if (trimmed.length === 0 || values.includes(trimmed)) { @@ -1126,7 +1153,9 @@ export const make = Effect.gen(function* () { timeToLive: (exit, key) => { if (Exit.isSuccess(exit)) { prLookupFailureStreakByKey.delete(key); - return PR_LOOKUP_CACHE_TTL; + return exit.value.latest?.state === "open" + ? PR_LOOKUP_CACHE_TTL + : PR_LOOKUP_NO_OPEN_PR_CACHE_TTL; } return nextPrLookupFailureTtl(key); }, @@ -1610,12 +1639,14 @@ export const make = Effect.gen(function* () { | "isCrossRepository" >, ) { - for (const headSelector of headContext.headSelectors) { - const pullRequests = yield* (yield* sourceControlProvider(cwd)).listChangeRequests({ + const provider = yield* sourceControlProvider(cwd); + const headSelectors = probeableHeadSelectors(provider.kind, headContext.headSelectors); + for (const headSelector of headSelectors) { + const pullRequests = yield* provider.listChangeRequests({ cwd, headSelector, state: "open", - limit: 1, + limit: provider.kind === "github" ? GITHUB_HEAD_BRANCH_PROBE_LIMIT : 1, }); const normalizedPullRequests = pullRequests.map(toPullRequestInfo); @@ -1640,12 +1671,13 @@ export const make = Effect.gen(function* () { ) { const parsedByNumber = new Map(); - for (const headSelector of headContext.headSelectors) { - const pullRequests = yield* (yield* sourceControlProvider(cwd)).listChangeRequests({ + const provider = yield* sourceControlProvider(cwd); + for (const headSelector of probeableHeadSelectors(provider.kind, headContext.headSelectors)) { + const pullRequests = yield* provider.listChangeRequests({ cwd, headSelector, state: "all", - limit: 20, + limit: provider.kind === "github" ? GITHUB_HEAD_BRANCH_PROBE_LIMIT : 20, }); for (const pr of pullRequests.map(toPullRequestInfo)) { @@ -2557,11 +2589,15 @@ export const make = Effect.gen(function* () { }); } - const worktree = yield* gitCore.createWorktree({ - cwd: input.cwd, - refName: localPullRequestBranch, - path: null, - }); + const worktree = yield* gitCore.createWorktree( + { cwd: input.cwd, refName: localPullRequestBranch, path: null }, + { + submodules: yield* projectSettingsFor(input).pipe( + Effect.map((settings) => settings.worktreeSubmodules), + Effect.orElseSucceed(() => null), + ), + }, + ); yield* ensureExistingWorktreeUpstream(worktree.worktree.path); yield* maybeRunSetupScript(worktree.worktree.path); diff --git a/apps/server/src/git/GitWorkflowService.ts b/apps/server/src/git/GitWorkflowService.ts index 9f3176cee8..d11109c2fd 100644 --- a/apps/server/src/git/GitWorkflowService.ts +++ b/apps/server/src/git/GitWorkflowService.ts @@ -86,6 +86,7 @@ export class GitWorkflowService extends Context.Service< readonly fetchRemote: (input: { readonly cwd: string; readonly remoteName: string; + readonly refName?: string; }) => Effect.Effect; readonly remoteExists: (input: { readonly cwd: string; diff --git a/apps/server/src/observability/Layers/Observability.ts b/apps/server/src/observability/Layers/Observability.ts index 8f7b607745..a754933e5d 100644 --- a/apps/server/src/observability/Layers/Observability.ts +++ b/apps/server/src/observability/Layers/Observability.ts @@ -20,6 +20,7 @@ import * as BrowserTraceCollector from "../BrowserTraceCollector.ts"; export const ObservabilityLive = Layer.unwrap( Effect.gen(function* () { const config = yield* ServerConfig.ServerConfig; + const traces = config.otlpTracesExport; const metrics = config.otlpMetricsExport; // The trace serializer stays in the returned context because the browser @@ -86,6 +87,15 @@ export const ObservabilityLive = Layer.unwrap( resource, }).pipe(Layer.provide(otlpSerializationLayer(metrics.protocol))); - return Layer.mergeAll(ServerLoggerLive, traceReferencesLayer, tracerLayer, metricsLayer); + // Logged once the server's loggers are installed, so the warnings use them. + const otelWarningsLayer = Layer.effectDiscard( + Effect.forEach(config.otelEnvironment.warnings, (warning) => Effect.logWarning(warning)), + ); + + return otelWarningsLayer.pipe( + Layer.provideMerge( + Layer.mergeAll(ServerLoggerLive, traceReferencesLayer, tracerLayer, metricsLayer), + ), + ); }), ); diff --git a/apps/server/src/orchestration/Layers/CheckpointReactor.test.ts b/apps/server/src/orchestration/Layers/CheckpointReactor.test.ts index 9a4d3b81b6..bcc7f23d12 100644 --- a/apps/server/src/orchestration/Layers/CheckpointReactor.test.ts +++ b/apps/server/src/orchestration/Layers/CheckpointReactor.test.ts @@ -7,10 +7,12 @@ import * as NodeChildProcess from "node:child_process"; import { VcsProcessTimeoutError, VcsProcessSpawnError, + VcsProcessExitError, ProviderDriverKind, ProviderRuntimeEvent, ProviderSession, ProviderInstanceId, + type OrchestrationEvent, } from "@t3tools/contracts"; import { CommandId, @@ -28,9 +30,11 @@ import * as Deferred from "effect/Deferred"; import * as Effect from "effect/Effect"; import * as Exit from "effect/Exit"; import * as Layer from "effect/Layer"; +import * as Logger from "effect/Logger"; import * as ManagedRuntime from "effect/ManagedRuntime"; import * as PubSub from "effect/PubSub"; import * as Queue from "effect/Queue"; +import * as Schema from "effect/Schema"; import { it as effectIt } from "@effect/vitest"; import * as RuntimeReceiptBus from "../Services/RuntimeReceiptBus.ts"; import * as Scope from "effect/Scope"; @@ -50,6 +54,7 @@ import * as ThreadBackgroundLiveness from "../ThreadBackgroundLiveness.ts"; import * as ThreadPlanProgress from "../ThreadPlanProgress.ts"; import { RuntimeReceiptBusTest } from "./RuntimeReceiptBus.ts"; import { OrchestrationEventStoreLive } from "../../persistence/Layers/OrchestrationEventStore.ts"; +import { OrchestrationEventStore } from "../../persistence/Services/OrchestrationEventStore.ts"; import { OrchestrationCommandReceiptRepositoryLive } from "../../persistence/Layers/OrchestrationCommandReceipts.ts"; import { SqlitePersistenceMemory } from "../../persistence/Layers/Sqlite.ts"; import { @@ -71,6 +76,7 @@ import { PullRequestService } from "../../pullRequest/PullRequestService.ts"; const asProjectId = (value: string): ProjectId => ProjectId.make(value); const asTurnId = (value: string): TurnId => TurnId.make(value); +const encodeUnknownJson = Schema.encodeSync(Schema.fromJsonString(Schema.Unknown)); type LegacyProviderRuntimeEvent = { readonly type: string; @@ -314,6 +320,10 @@ describe("CheckpointReactor", () => { readonly checkpointLookupFailure?: ( cwd: string, ) => VcsProcessTimeoutError | VcsProcessSpawnError | undefined; + readonly checkpointCaptureFailure?: (cwd: string) => VcsProcessExitError | undefined; + readonly checkpointDiffFailure?: (cwd: string) => VcsProcessExitError | undefined; + readonly checkpointRepositoryFailure?: (cwd: string) => VcsProcessExitError | undefined; + readonly logMessages?: Array; readonly workspaceRefresh?: (cwd: string) => Effect.Effect; readonly hasSession?: boolean; readonly seedFilesystemCheckpoints?: boolean; @@ -343,6 +353,8 @@ describe("CheckpointReactor", () => { options?.providerName ?? ProviderDriverKind.make("codex"), options?.conversationRollback ?? "relative", ); + const historicalRollbackEvents = Effect.runSync(PubSub.unbounded()); + const historicalRollbackReady = Effect.runSync(Deferred.make()); const orchestrationLayer = OrchestrationEngineLive.pipe( Layer.provide(OrchestrationProjectionSnapshotQueryLive), Layer.provide(ThreadBackgroundLiveness.layer), @@ -353,6 +365,24 @@ describe("CheckpointReactor", () => { Layer.provide(RepositoryIdentityResolver.layer), Layer.provide(SqlitePersistenceMemory), ); + const historicalEngineLayer = Layer.effect( + OrchestrationEngineService, + Effect.gen(function* () { + const engine = yield* OrchestrationEngineService; + return OrchestrationEngineService.of({ + ...engine, + streamDomainEvents: Stream.merge( + engine.streamDomainEvents, + Stream.unwrap( + PubSub.subscribe(historicalRollbackEvents).pipe( + Effect.tap(() => Deferred.succeed(historicalRollbackReady, undefined)), + Effect.map(Stream.fromSubscription), + ), + ), + ), + }); + }), + ).pipe(Layer.provide(orchestrationLayer)); const projectionSnapshotLayer = OrchestrationProjectionSnapshotQueryLive.pipe( Layer.provide(ThreadBackgroundLiveness.layer), Layer.provide(ThreadPlanProgress.layer), @@ -404,7 +434,7 @@ describe("CheckpointReactor", () => { ); const layer = CheckpointReactorLive.pipe( - Layer.provideMerge(orchestrationLayer), + Layer.provideMerge(historicalEngineLayer), Layer.provideMerge(projectionSnapshotLayer), Layer.provideMerge(RuntimeReceiptBusTest), Layer.provideMerge(Layer.succeed(ProviderService, provider.service)), @@ -420,6 +450,18 @@ describe("CheckpointReactor", () => { const failure = options?.checkpointLookupFailure?.(input.cwd); return failure ? Effect.fail(failure) : store.hasCheckpointRef(input); }, + captureCheckpoint: (input) => { + const failure = options?.checkpointCaptureFailure?.(input.cwd); + return failure ? Effect.fail(failure) : store.captureCheckpoint(input); + }, + diffCheckpoints: (input) => { + const failure = options?.checkpointDiffFailure?.(input.cwd); + return failure ? Effect.fail(failure) : store.diffCheckpoints(input); + }, + isGitRepository: (cwd) => { + const failure = options?.checkpointRepositoryFailure?.(cwd); + return failure ? Effect.fail(failure) : store.isGitRepository(cwd); + }, })), ), ).pipe(Layer.provide(VcsDriverRegistry.layer)), @@ -430,9 +472,56 @@ describe("CheckpointReactor", () => { Layer.provideMerge(ServerConfigLayer), Layer.provideMerge(NodeServices.layer), ); - - runtime = ManagedRuntime.make(layer); + const layerWithLogger = + options?.logMessages === undefined + ? layer + : layer.pipe( + Layer.provideMerge( + Logger.layer( + [ + Logger.make((event) => { + options.logMessages?.push(Logger.formatStructured.log(event)); + }), + ], + { mergeWithExisting: false }, + ), + ), + ); + + runtime = ManagedRuntime.make(layerWithLogger); const engine = await runtime.runPromise(Effect.service(OrchestrationEngineService)); + const eventStore = await runtime.runPromise( + Effect.service(OrchestrationEventStore).pipe( + Effect.provide(OrchestrationEventStoreLive.pipe(Layer.provide(SqlitePersistenceMemory))), + ), + ); + const publishLegacyRevert = (command: { + readonly commandId: CommandId; + readonly threadId: ThreadId; + readonly turnCount: number; + readonly createdAt: string; + }) => + Effect.gen(function* () { + const event = yield* eventStore.append({ + eventId: EventId.make(`legacy-${command.commandId}`), + aggregateKind: "thread", + aggregateId: command.threadId, + occurredAt: command.createdAt, + commandId: command.commandId, + causationEventId: null, + correlationId: command.commandId, + metadata: {}, + type: "thread.checkpoint-revert-requested", + payload: { + threadId: command.threadId, + turnCount: command.turnCount, + createdAt: command.createdAt, + }, + }); + const persisted = yield* Stream.runCollect(engine.readEvents(event.sequence - 1)); + expect(Array.from(persisted).some((stored) => stored.eventId === event.eventId)).toBe(true); + yield* PubSub.publish(historicalRollbackEvents, event); + }); const snapshotQuery = await runtime.runPromise(Effect.service(ProjectionSnapshotQuery)); const reactor = await runtime.runPromise(Effect.service(CheckpointReactor)); const checkpointStore = await runtime.runPromise( @@ -450,6 +539,7 @@ describe("CheckpointReactor", () => { Queue.offer(receipts, receipt), ).pipe(Effect.forkIn(testScope, { startImmediately: true })); yield* reactor.start().pipe(Scope.provide(testScope)); + yield* Deferred.await(historicalRollbackReady); return receipts; }), ); @@ -539,6 +629,7 @@ describe("CheckpointReactor", () => { return { engine, + publishLegacyRevert, readModel: () => Effect.runPromise(snapshotQuery.getSnapshot()), provider, workspaceRefresh, @@ -653,6 +744,152 @@ describe("CheckpointReactor", () => { }), ); + for (const stage of ["capture", "diff"] as const) { + effectIt.effect(`keeps ${stage} VCS failure details out of public activity and logs`, () => + Effect.gen(function* () { + let fail = false; + const logMessages: Array = []; + const failure = (cwd: string) => + fail + ? new VcsProcessExitError({ + operation: "checkpoint-private-operation", + command: "git-private-command-canary", + cwd, + exitCode: 128, + detail: "private-stderr-and-credential-canary", + }) + : undefined; + const harness = yield* Effect.promise(() => + createHarness({ + seedFilesystemCheckpoints: false, + ...(stage === "capture" ? { checkpointCaptureFailure: failure } : {}), + ...(stage === "diff" ? { checkpointDiffFailure: failure } : {}), + logMessages, + }), + ); + const threadId = ThreadId.make("thread-1"); + const turnId = asTurnId(`turn-private-${stage}`); + harness.provider.emit({ + type: "turn.started", + eventId: EventId.make(`evt-private-${stage}-start`), + provider: ProviderDriverKind.make("codex"), + createdAt: "2026-01-01T00:00:00.000Z", + threadId, + turnId, + }); + expect(yield* harness.nextReceipt).toMatchObject({ type: "checkpoint.baseline.captured" }); + fail = true; + harness.provider.emit({ + type: "turn.completed", + eventId: EventId.make(`evt-private-${stage}-complete`), + provider: ProviderDriverKind.make("codex"), + createdAt: "2026-01-01T00:00:01.000Z", + threadId, + turnId, + payload: { state: "completed" }, + }); + yield* Effect.promise(harness.drain); + const thread = (yield* Effect.promise(harness.readModel)).threads[0]; + const publicActivity = encodeUnknownJson( + thread?.activities.filter((activity) => activity.kind === "checkpoint.capture.failed"), + ); + expect(publicActivity).toContain("checkpoint.capture.failed"); + const emittedLogs = encodeUnknownJson(logMessages); + if (stage === "diff") { + expect(emittedLogs).toContain("failed to derive checkpoint file summary"); + expect(emittedLogs).toContain("git-exit"); + } + for (const privateCanary of [ + harness.cwd, + "git-private-command-canary", + "private-stderr-and-credential-canary", + ]) { + expect(publicActivity).not.toContain(privateCanary); + expect(emittedLogs).not.toContain(privateCanary); + } + }), + ); + } + + effectIt.effect("keeps repository errors out of public revert failure activity", () => + Effect.gen(function* () { + let fail = false; + const harness = yield* Effect.promise(() => + createHarness({ + checkpointRepositoryFailure: (cwd) => + fail + ? new VcsProcessExitError({ + operation: "private-revert-operation", + command: "git-private-revert-command", + cwd, + exitCode: 128, + detail: "private-revert-stderr-canary", + }) + : undefined, + }), + ); + fail = true; + yield* harness.publishLegacyRevert({ + commandId: CommandId.make("cmd-private-revert-failure"), + threadId: ThreadId.make("thread-1"), + turnCount: 0, + createdAt: "2026-01-01T00:00:00.000Z", + }); + yield* Effect.promise(harness.drain); + const thread = (yield* Effect.promise(harness.readModel)).threads[0]; + const publicActivity = encodeUnknownJson( + thread?.activities.filter((activity) => activity.kind === "checkpoint.revert.failed"), + ); + expect(publicActivity).toContain("checkpoint.revert.failed"); + expect(publicActivity).toContain("git-exit"); + for (const privateCanary of [ + harness.cwd, + "git-private-revert-command", + "private-revert-stderr-canary", + ]) { + expect(publicActivity).not.toContain(privateCanary); + } + }), + ); + + effectIt.effect("logs checkpoint workspace refresh failure without its path or cause", () => + Effect.gen(function* () { + const logMessages: Array = []; + const harness = yield* Effect.promise(() => + createHarness({ + seedFilesystemCheckpoints: false, + workspaceRefresh: () => Effect.die(new Error("private-refresh-cause-canary")), + logMessages, + }), + ); + const threadId = ThreadId.make("thread-1"); + const turnId = asTurnId("turn-private-refresh"); + harness.provider.emit({ + type: "turn.started", + eventId: EventId.make("evt-private-refresh-start"), + provider: ProviderDriverKind.make("codex"), + createdAt: "2026-01-01T00:00:00.000Z", + threadId, + turnId, + }); + expect(yield* harness.nextReceipt).toMatchObject({ type: "checkpoint.baseline.captured" }); + harness.provider.emit({ + type: "turn.completed", + eventId: EventId.make("evt-private-refresh-complete"), + provider: ProviderDriverKind.make("codex"), + createdAt: "2026-01-01T00:00:01.000Z", + threadId, + turnId, + payload: { state: "completed" }, + }); + yield* Effect.promise(harness.drain); + const emittedLogs = encodeUnknownJson(logMessages); + expect(emittedLogs).toContain("failed to refresh checkpoint workspace entries"); + expect(emittedLogs).not.toContain(harness.cwd); + expect(emittedLogs).not.toContain("private-refresh-cause-canary"); + }), + ); + effectIt.effect( "appends isolation failure activity when restoreFiles is true in a shared workspace", () => @@ -719,24 +956,26 @@ describe("CheckpointReactor", () => { (failureKind) => Effect.gen(function* () { let failLookup = false; + const logMessages: Array = []; const harness = yield* Effect.promise(() => createHarness({ seedFilesystemCheckpoints: false, + logMessages, checkpointLookupFailure: (cwd) => !failLookup ? undefined : failureKind === "timeout" ? new VcsProcessTimeoutError({ operation: "test.refLookup", - command: "git", + command: "git-private-ref-canary", cwd, timeoutMs: 30000, }) : new VcsProcessSpawnError({ operation: "test.refLookup", - command: "git", + command: "git-private-ref-canary", cwd, - cause: new Error("transient lookup spawn failure"), + cause: new Error("private-ref-cause-canary"), }), }), ); @@ -763,6 +1002,16 @@ describe("CheckpointReactor", () => { payload: { state: "completed" }, }); yield* Effect.promise(harness.drain); + const emittedLogs = encodeUnknownJson(logMessages); + expect(emittedLogs).toContain(failureKind === "timeout" ? "git-timeout" : "git-spawn"); + for (const privateCanary of [ + harness.cwd, + "git-private-ref-canary", + "private-ref-cause-canary", + checkpointRefForThreadTurn(threadId, 0), + ]) { + expect(emittedLogs).not.toContain(privateCanary); + } const ref = checkpointRefForThreadTurn(threadId, 1); expect(gitShowFileAtRef(harness.cwd, ref, "README.md")).toBe("new snapshot\n"); expect(yield* harness.nextReceipt).toMatchObject({ diff --git a/apps/server/src/orchestration/Layers/CheckpointReactor.ts b/apps/server/src/orchestration/Layers/CheckpointReactor.ts index 109150811b..ef3d211311 100644 --- a/apps/server/src/orchestration/Layers/CheckpointReactor.ts +++ b/apps/server/src/orchestration/Layers/CheckpointReactor.ts @@ -46,6 +46,30 @@ import { RollbackWorkspace } from "../../rollback/RollbackWorkspace.ts"; import * as PullRequestService from "../../pullRequest/PullRequestService.ts"; const nowIso = Effect.map(DateTime.now, DateTime.formatIso); +// VCS errors can carry absolute paths and command diagnostics in `message`. +// Only these fixed categories may enter public activities or server logs. +const checkpointFailureKind = (error: unknown) => { + if (typeof error !== "object" || error === null || !("_tag" in error)) return "unknown"; + switch (error._tag) { + case "VcsProcessSpawnError": + return "git-spawn"; + case "VcsProcessExitError": + return "git-exit"; + case "VcsProcessTimeoutError": + return "git-timeout"; + case "VcsProcessStdinWriteError": + case "VcsProcessOutputReadError": + case "VcsProcessOutputLimitError": + case "VcsProcessMissingExitCodeError": + return "git-io"; + case "VcsRepositoryDetectionError": + return "repository-detection"; + case "VcsUnsupportedOperationError": + return "unsupported"; + default: + return "unknown"; + } +}; const rollbackUnavailable = { state: "unavailable" as const, reason: @@ -126,7 +150,7 @@ export const make = Effect.gen(function* () { Cause.hasInterruptsOnly(cause) ? Effect.failCause(cause) : Effect.logWarning("failed to refresh checkpoint workspace entries", { - cwd, + failureKind: "workspace-entry-refresh", }), ), ), @@ -365,8 +389,8 @@ export const make = Effect.gen(function* () { Effect.catch((error) => Effect.logWarning("checkpoint capture previous ref lookup failed", { threadId: input.threadId, - checkpointRef: fromCheckpointRef, - category: error._tag, + fromTurnCount, + category: checkpointFailureKind(error), }).pipe(Effect.as(false)), ), ); @@ -429,7 +453,7 @@ export const make = Effect.gen(function* () { appendCaptureFailureActivity({ threadId: input.threadId, turnId: input.turnId, - detail: `Checkpoint captured, but turn diff summary is unavailable: ${error.message}`, + detail: `Checkpoint captured, but turn diff summary is unavailable (${checkpointFailureKind(error)}).`, createdAt: input.createdAt, }), ), @@ -438,7 +462,7 @@ export const make = Effect.gen(function* () { threadId: input.threadId, turnId: input.turnId, turnCount: input.turnCount, - detail: error.message, + failureKind: checkpointFailureKind(error), }).pipe(Effect.as([])), ), ); @@ -642,8 +666,7 @@ export const make = Effect.gen(function* () { Effect.logWarning("failed to refresh local git status after turn completion", { threadId: event.threadId, turnId: event.turnId ?? null, - cwd: sessionRuntime.value.cwd, - detail: error.message, + failureKind: checkpointFailureKind(error), }).pipe(Effect.as(null)), ), ); @@ -682,8 +705,7 @@ export const make = Effect.gen(function* () { Effect.catch((error) => Effect.logWarning("failed to refresh pull request status after turn completion", { threadId: input.threadId, - cwd: input.cwd, - detail: error.message, + failureKind: checkpointFailureKind(error), }), ), ); @@ -752,7 +774,7 @@ export const make = Effect.gen(function* () { } return Effect.logWarning("failed to follow worktree branch drift", { threadId: input.threadId, - cause: Cause.pretty(cause), + failureKind: "unexpected", }); }), ); @@ -769,7 +791,7 @@ export const make = Effect.gen(function* () { ? Effect.failCause(cause) : Effect.logWarning("failed to refresh git status after turn completion", { threadId: event.threadId, - cause: Cause.pretty(cause), + failureKind: "unexpected", }), ), ), @@ -994,7 +1016,7 @@ export const make = Effect.gen(function* () { appendRevertFailureActivity({ threadId: event.payload.threadId, turnCount: event.payload.turnCount, - detail: error.message, + detail: `Checkpoint revert could not be verified (${checkpointFailureKind(error)}).`, createdAt, }), ), @@ -1063,7 +1085,7 @@ export const make = Effect.gen(function* () { appendCaptureFailureActivity({ threadId: event.threadId, turnId, - detail: error.message, + detail: `Checkpoint capture failed (${checkpointFailureKind(error)}).`, createdAt, }).pipe(Effect.catch(() => Effect.void)), ), @@ -1106,7 +1128,7 @@ export const make = Effect.gen(function* () { return Effect.logWarning("checkpoint reactor failed to process input", { source: input.source, eventType: input.source === "saga" ? "rollback.saga.reconcile" : input.event.type, - cause: input.source === "saga" ? "rollback saga step failed" : Cause.pretty(cause), + failureKind: input.source === "saga" ? "rollback-saga" : "unexpected", }); }), ); diff --git a/apps/server/src/orchestration/Layers/ProviderCommandReactor.test.ts b/apps/server/src/orchestration/Layers/ProviderCommandReactor.test.ts index a3be323be0..19b1a0fc1e 100644 --- a/apps/server/src/orchestration/Layers/ProviderCommandReactor.test.ts +++ b/apps/server/src/orchestration/Layers/ProviderCommandReactor.test.ts @@ -2005,6 +2005,100 @@ describe("ProviderCommandReactor", () => { }), ); + effectIt.effect( + "keeps exact admission pending while a provider reports a running prior turn", + () => + Effect.gen(function* () { + const sendEntered = yield* Deferred.make(); + const releaseSend = yield* Deferred.make(); + const requestId = CommandId.make("cmd-running-provider-exact-admission"); + const messageId = asMessageId("message-running-provider-exact-admission"); + const harness = yield* Effect.promise(() => + createHarness({ + startSessionEffect: (session) => Effect.succeed({ ...session, status: "running" }), + sendTurnEffect: () => + Deferred.succeed(sendEntered, undefined).pipe( + Effect.andThen(Deferred.await(releaseSend)), + Effect.as({ threadId: ThreadId.make("thread-1"), turnId: asTurnId("new-turn") }), + ), + }), + ); + + yield* harness.engine.dispatch({ + type: "thread.turn.start", + commandId: requestId, + threadId: ThreadId.make("thread-1"), + message: { messageId, role: "user", text: "follow up", attachments: [] }, + interactionMode: DEFAULT_PROVIDER_INTERACTION_MODE, + runtimeMode: "approval-required", + createdAt: isoAt(0), + }); + yield* Deferred.await(sendEntered); + let thread = (yield* Effect.promise(() => harness.readModel())).threads.find( + (entry) => entry.id === ThreadId.make("thread-1"), + ); + expect(thread?.session?.status).toBe("starting"); + expect(thread?.session?.pendingTurnRequestId).toBe(requestId); + expect(thread?.session?.pendingTurnMessageId).toBe(messageId); + + yield* Deferred.succeed(releaseSend, undefined); + yield* Effect.promise(() => + waitFor(async () => { + const thread = (await harness.readModel()).threads.find( + (entry) => entry.id === ThreadId.make("thread-1"), + ); + return thread?.session?.activeTurnId === asTurnId("new-turn"); + }), + ); + thread = (yield* Effect.promise(() => harness.readModel())).threads.find( + (entry) => entry.id === ThreadId.make("thread-1"), + ); + expect(thread?.session?.status).toBe("running"); + expect(thread?.session?.activeTurnRequestId).toBe(requestId); + expect(thread?.session?.pendingTurnRequestId).toBeUndefined(); + }), + ); + + for (const status of ["error", "closed"] as const) { + effectIt.effect(`rejects exact admission when the provider session is ${status}`, () => + Effect.gen(function* () { + const requestId = CommandId.make(`cmd-terminal-provider-${status}`); + const harness = yield* Effect.promise(() => + createHarness({ + startSessionEffect: (session) => Effect.succeed({ ...session, status }), + }), + ); + yield* harness.engine.dispatch({ + type: "thread.turn.start", + commandId: requestId, + threadId: ThreadId.make("thread-1"), + message: { + messageId: asMessageId(`message-terminal-provider-${status}`), + role: "user", + text: "follow up", + attachments: [], + }, + interactionMode: DEFAULT_PROVIDER_INTERACTION_MODE, + runtimeMode: "approval-required", + createdAt: isoAt(0), + }); + yield* Effect.promise(() => + waitFor(async () => { + const thread = (await harness.readModel()).threads.find( + (entry) => entry.id === ThreadId.make("thread-1"), + ); + return thread?.session?.status === "error"; + }), + ); + const thread = (yield* Effect.promise(() => harness.readModel())).threads.find( + (entry) => entry.id === ThreadId.make("thread-1"), + ); + expect(thread?.session?.failedTurnRequestId).toBe(requestId); + expect(harness.sendTurn).not.toHaveBeenCalled(); + }), + ); + } + effectIt.effect("does not disarm when raw turn start ingestion never accepts the CAS", () => Effect.gen(function* () { const testClock = yield* TestClock.make(); @@ -3784,11 +3878,14 @@ describe("ProviderCommandReactor", () => { await waitFor(() => harness.startSession.mock.calls.length === 1); expect(harness.pruneWorktrees).toHaveBeenCalledWith({ cwd: "/tmp/provider-project" }); - expect(harness.createWorktree).toHaveBeenCalledWith({ - cwd: "/tmp/provider-project", - refName: "feature/restore", - path: worktreePath, - }); + expect(harness.createWorktree).toHaveBeenCalledWith( + { + cwd: "/tmp/provider-project", + refName: "feature/restore", + path: worktreePath, + }, + { submodules: null }, + ); expect(harness.createWorktree.mock.invocationCallOrder[0]).toBeLessThan( harness.startSession.mock.invocationCallOrder[0]!, ); diff --git a/apps/server/src/orchestration/Layers/ProviderCommandReactor.ts b/apps/server/src/orchestration/Layers/ProviderCommandReactor.ts index e5fcf3d49c..40caffc583 100644 --- a/apps/server/src/orchestration/Layers/ProviderCommandReactor.ts +++ b/apps/server/src/orchestration/Layers/ProviderCommandReactor.ts @@ -640,8 +640,14 @@ const make = Effect.gen(function* () { }); // A directory deleted without `git worktree remove` leaves an admin entry // that makes `git worktree add` refuse the path; prune clears it. + const submodules = yield* projectSettingsForThread(thread.id).pipe( + Effect.map((settings) => settings.worktreeSubmodules), + Effect.orElseSucceed(() => null), + ); yield* gitWorkflow.pruneWorktrees({ cwd }).pipe( - Effect.andThen(gitWorkflow.createWorktree({ cwd, refName: branch, path: worktreePath })), + Effect.andThen( + gitWorkflow.createWorktree({ cwd, refName: branch, path: worktreePath }, { submodules }), + ), Effect.catchCause((cause) => Cause.hasInterruptsOnly(cause) ? Effect.failCause(cause) @@ -1048,10 +1054,23 @@ const make = Effect.gen(function* () { detail: `Provider session '${session.threadId}' started without a provider instance or session incarnation id.`, }); } + if ( + options?.pendingTurnStart === true && + (session.status === "error" || session.status === "closed") + ) { + return yield* new ProviderAdapterRequestError({ + provider: providerErrorLabel(session.provider), + method: "thread.turn.start", + detail: `Provider session '${session.threadId}' is ${session.status}; it cannot accept a new turn.`, + }); + } const sessionBinding: OrchestrationSession = { threadId, + // The provider can still report the previous turn as running while + // a new exact admission is reserved. The reservation owns the + // projected lifecycle until its matching turn.started is accepted. status: - options?.pendingTurnStart === true && session.status === "ready" + options?.pendingTurnStart === true ? "starting" : mapProviderSessionStatusToOrchestrationStatus(session.status), providerName: session.provider, diff --git a/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.test.ts b/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.test.ts index 9bb1d25e74..dd5dd4f3dc 100644 --- a/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.test.ts +++ b/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.test.ts @@ -68,6 +68,7 @@ import * as ThreadPlanProgress from "../ThreadPlanProgress.ts"; import { ProviderRuntimeIngestionLive, runtimeEventToActivities, + splitBufferedAssistantText, } from "./ProviderRuntimeIngestion.ts"; import { DEFAULT_THREAD_TITLE } from "../threadTitles.ts"; import { OrchestrationEngineService } from "../Services/OrchestrationEngine.ts"; @@ -488,7 +489,23 @@ describe("ProviderRuntimeIngestion", () => { }); }), ).pipe(Layer.provide(projectionSnapshotLayer)); + // Advance pacing deterministically while keeping real sleeps and deadlines. + let clockOffsetMs = 0; + const realClock = Effect.runSync(Effect.service(Clock.Clock)); + const shiftedClock: Clock.Clock = { + currentTimeMillisUnsafe: () => realClock.currentTimeMillisUnsafe() + clockOffsetMs, + currentTimeMillis: Effect.sync(() => realClock.currentTimeMillisUnsafe() + clockOffsetMs), + currentTimeNanosUnsafe: () => + realClock.currentTimeNanosUnsafe() + BigInt(clockOffsetMs) * 1_000_000n, + currentTimeNanos: Effect.sync( + () => realClock.currentTimeNanosUnsafe() + BigInt(clockOffsetMs) * 1_000_000n, + ), + monotonicTimeNanosUnsafe: () => realClock.monotonicTimeNanosUnsafe(), + monotonicTimeNanos: realClock.monotonicTimeNanos, + sleep: (duration) => realClock.sleep(duration), + }; const layer = ProviderRuntimeIngestionLive.pipe( + Layer.provide(Layer.succeed(Clock.Clock, shiftedClock)), Layer.provideMerge(orchestrationLayer), Layer.provideMerge(ingestionProjectionSnapshotLayer), // Single shared liveness instance across ingestion (writer), the @@ -587,6 +604,9 @@ describe("ProviderRuntimeIngestion", () => { dispatch, readModel: () => testRuntime.runPromise(snapshotQuery.getSnapshot()), emit: provider.emit, + advanceClock: (ms: number) => { + clockOffsetMs += ms; + }, emitAndDrain, sqlCount: sqlCounter.count, readTurn: (turnId: TurnId) => @@ -1663,6 +1683,27 @@ describe("ProviderRuntimeIngestion", () => { expect(thread?.session?.pendingTurnRequestId).toBe(requestId); expect(thread?.messages.some((message) => message.role === "assistant")).toBe(false); + // The prior turn can finish on the same provider session after the new + // admission is reserved. Its terminal event must not clear that pending + // admission before the matching turn.started arrives. + await harness.emitAndDrain([ + { + type: "turn.completed", + eventId: asEventId("evt-runtime-session-b-prior-turn-completed"), + provider: ProviderDriverKind.make("codex"), + providerInstanceId: ProviderInstanceId.make("codex"), + threadId, + turnId: asTurnId("turn-runtime-session-b-prior"), + admissionRequestId: CommandId.make("cmd-runtime-session-b-prior"), + sessionIncarnationId: sessionB, + createdAt: "2026-01-01T00:00:02.500Z", + payload: { state: "completed" }, + }, + ]); + thread = (await harness.readModel()).threads.find((entry) => entry.id === threadId); + expect(thread?.session?.status).toBe("starting"); + expect(thread?.session?.pendingTurnRequestId).toBe(requestId); + const currentTurnId = asTurnId("turn-runtime-session-b"); harness.emit({ type: "turn.started", @@ -4965,6 +5006,145 @@ describe("ProviderRuntimeIngestion", () => { expect(finalMessage?.streaming).toBe(false); }); + it("delivers finished paragraphs while the rest of the message stays buffered", async () => { + const harness = await createHarness(); + const now = "2026-01-01T00:00:00.000Z"; + const codex = ProviderDriverKind.make("codex"); + const threadId = asThreadId("thread-1"); + const turnId = asTurnId("turn-paragraph-flush"); + const itemId = asItemId("item-paragraph-flush"); + + harness.emit({ + type: "turn.started", + eventId: asEventId("evt-paragraph-started"), + provider: codex, + createdAt: now, + threadId, + turnId, + }); + await waitForThread( + harness.readModel, + (thread) => thread.session?.status === "running" && thread.session?.activeTurnId === turnId, + ); + + // Each delta lands well outside the pacing window of the one before. + const emitDelta = (eventId: string, delta: string) => { + harness.advanceClock(1_000); + harness.emit({ + type: "content.delta", + eventId: asEventId(eventId), + provider: codex, + createdAt: now, + threadId, + turnId, + itemId, + payload: { streamKind: "assistant_text", delta }, + }); + }; + + emitDelta("evt-paragraph-1", "First paragraph.\n\nSecond para"); + const afterFirst = await waitForThread(harness.readModel, (thread) => + thread.messages.some( + (message: ProviderRuntimeTestMessage) => message.id === `assistant:${itemId}`, + ), + ); + expect( + afterFirst.messages.find((m: ProviderRuntimeTestMessage) => m.id === `assistant:${itemId}`), + ).toMatchObject({ + text: "First paragraph.\n\n", + streaming: true, + }); + + // An open code block holds the whole block until its closing fence lands. + emitDelta("evt-paragraph-2", "graph.\n\n```ts\nconst a = 1;\n\nconst b = 2;\n"); + await harness.drain(); + expect( + (await harness.readModel()).threads + .find((t) => t.id === threadId) + ?.messages.find((m: ProviderRuntimeTestMessage) => m.id === `assistant:${itemId}`)?.text, + ).toBe("First paragraph.\n\nSecond paragraph.\n\n"); + + emitDelta("evt-paragraph-3", "```\n\nTail without newline"); + harness.emit({ + type: "item.completed", + eventId: asEventId("evt-paragraph-completed"), + provider: codex, + createdAt: now, + threadId, + turnId, + itemId, + payload: { itemType: "assistant_message", status: "completed" }, + }); + const finalThread = await waitForThread(harness.readModel, (thread) => + thread.messages.some( + (message: ProviderRuntimeTestMessage) => + message.id === `assistant:${itemId}` && !message.streaming, + ), + ); + expect( + finalThread.messages.find((m: ProviderRuntimeTestMessage) => m.id === `assistant:${itemId}`) + ?.text, + ).toBe( + "First paragraph.\n\nSecond paragraph.\n\n```ts\nconst a = 1;\n\nconst b = 2;\n```\n\nTail without newline", + ); + }); + + it("holds paragraphs that finish inside the pacing window and lands them together", async () => { + const harness = await createHarness(); + const codex = ProviderDriverKind.make("codex"); + const threadId = asThreadId("thread-1"); + const turnId = asTurnId("turn-paced"); + const itemId = asItemId("item-paced"); + // Every delta carries the same event time, like OpenCode does for one + // part. Pacing must follow the server clock, not the event stamp. + const now = "2026-01-01T00:00:00.000Z"; + + harness.emit({ + type: "turn.started", + eventId: asEventId("evt-paced-started"), + provider: codex, + createdAt: now, + threadId, + turnId, + }); + await waitForThread( + harness.readModel, + (thread) => thread.session?.status === "running" && thread.session?.activeTurnId === turnId, + ); + // Emit is fire-and-forget, so drain after each delta before moving the + // clock. Otherwise the worker reads a clock that has already advanced. + let clockMs = 0; + const emitDelta = async (eventId: string, delta: string, offsetMs: number) => { + harness.advanceClock(offsetMs - clockMs); + clockMs = offsetMs; + await harness.emitAndDrain([ + { + type: "content.delta", + eventId: asEventId(eventId), + provider: codex, + createdAt: now, + threadId, + turnId, + itemId, + payload: { streamKind: "assistant_text", delta }, + }, + ]); + }; + const messageText = async () => + (await harness.readModel()).threads + .find((t) => t.id === threadId) + ?.messages.find((m: ProviderRuntimeTestMessage) => m.id === `assistant:${itemId}`)?.text; + + await emitDelta("evt-paced-1", "One.\n\n", 0); + await emitDelta("evt-paced-2", "Two.\n\n", 100); + await emitDelta("evt-paced-3", "Three.\n\n", 200); + // The first paragraph lands right away. The next two are inside the window. + expect(await messageText()).toBe("One.\n\n"); + + await emitDelta("evt-paced-4", "Four.\n\n", 500); + expect(await messageText()).toBe("One.\n\nTwo.\n\nThree.\n\nFour.\n\n"); + }); + it("spills oversized buffered deltas and still finalizes full assistant text", async () => { const harness = await createHarness(); const now = "2026-01-01T00:00:00.000Z"; @@ -6797,3 +6977,94 @@ describe("ProviderRuntimeIngestion", () => { expect(thread.session?.lastError).toBe("runtime still processed"); }); }); + +describe("splitBufferedAssistantText", () => { + it("keeps a partial trailing line buffered", () => { + expect(splitBufferedAssistantText("one\n\ntwo")).toEqual({ ready: "one\n\n", rest: "two" }); + expect(splitBufferedAssistantText("one\ntwo")).toEqual({ ready: "", rest: "one\ntwo" }); + }); + + it("does not split inside an open fence and delivers the block at its closing fence", () => { + const open = "intro\n\n```\ncode\n\nmore\n"; + expect(splitBufferedAssistantText(open)).toEqual({ + ready: "intro\n\n", + rest: "```\ncode\n\nmore\n", + }); + expect(splitBufferedAssistantText(`${open}\`\`\`\nafter`)).toEqual({ + ready: `${open}\`\`\`\n`, + rest: "after", + }); + }); + + it("does not treat a fence with an info string as a closing fence", () => { + const text = "```\n```javascript\nstill code\n\nmore\n"; + expect(splitBufferedAssistantText(text)).toEqual({ ready: "", rest: text }); + }); + + it("treats a fence indented four or more spaces as code, not a closing fence", () => { + const text = "```\n ```\n\nstill code\n"; + expect(splitBufferedAssistantText(text)).toEqual({ ready: "", rest: text }); + expect(splitBufferedAssistantText("```\n ```\nafter")).toEqual({ + ready: "```\n ```\n", + rest: "after", + }); + }); + + it("keeps a fence nested under a list item open across its blank lines", () => { + const text = "- step\n\n ```ts\n a\n\n b\n ```\n\nafter\n"; + expect(splitBufferedAssistantText(text)).toEqual({ + ready: "- step\n\n ```ts\n a\n\n b\n ```\n\n", + rest: "after\n", + }); + }); + + it("does not treat a no-break-space line as blank", () => { + expect(splitBufferedAssistantText("para\n\u00a0\ncont\n\nnext")).toEqual({ + ready: "para\n\u00a0\ncont\n\n", + rest: "next", + }); + }); + + it("treats CRLF blank lines as boundaries", () => { + expect(splitBufferedAssistantText("one\r\n\r\ntwo")).toEqual({ + ready: "one\r\n\r\n", + rest: "two", + }); + }); + + it("only closes a fence with the same marker of equal or greater length", () => { + const text = "````\n```\nstill code\n\n````\n\nout\n"; + expect(splitBufferedAssistantText(text)).toEqual({ + ready: "````\n```\nstill code\n\n````\n\n", + rest: "out\n", + }); + expect(splitBufferedAssistantText("~~~\n```\n\nx\n")).toEqual({ + ready: "", + rest: "~~~\n```\n\nx\n", + }); + }); + + it("delivers tight list items one at a time", () => { + expect(splitBufferedAssistantText("## Steps\n\n- one\n- two\n- thr")).toEqual({ + ready: "## Steps\n\n- one\n- two\n", + rest: "- thr", + }); + expect(splitBufferedAssistantText("1. one\n2. two\n more\n3. t")).toEqual({ + ready: "1. one\n2. two\n more\n", + rest: "3. t", + }); + }); + + it("keeps a partial list marker and list-like code buffered", () => { + expect(splitBufferedAssistantText("intro\n-")).toEqual({ ready: "", rest: "intro\n-" }); + expect(splitBufferedAssistantText("intro\n1.")).toEqual({ ready: "", rest: "intro\n1." }); + // `intro\n- \n` would parse as a setext heading, so a bare marker with only + // trailing whitespace is not a boundary on the partial line either. + expect(splitBufferedAssistantText("intro\n- ")).toEqual({ ready: "", rest: "intro\n- " }); + expect(splitBufferedAssistantText("- one\n")).toEqual({ ready: "", rest: "- one\n" }); + expect(splitBufferedAssistantText("```\n- one\n- two\n")).toEqual({ + ready: "", + rest: "```\n- one\n- two\n", + }); + }); +}); diff --git a/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts b/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts index 7aea6f4d5d..4012a4089f 100644 --- a/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts +++ b/apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts @@ -27,6 +27,7 @@ import { } from "@t3tools/contracts"; import * as Cache from "effect/Cache"; import * as Cause from "effect/Cause"; +import * as Clock from "effect/Clock"; import * as Crypto from "effect/Crypto"; import * as Context from "effect/Context"; import * as Data from "effect/Data"; @@ -158,6 +159,7 @@ const BUFFERED_PROPOSED_PLAN_BY_ID_TTL = Duration.minutes(120); const TASK_DESCRIPTION_BY_TASK_CACHE_CAPACITY = 10_000; const TASK_DESCRIPTION_BY_TASK_TTL = Duration.minutes(120); const MAX_BUFFERED_ASSISTANT_CHARS = 24_000; +const MIN_ASSISTANT_DELIVERY_INTERVAL_MS = 400; const STRICT_PROVIDER_LIFECYCLE_GUARD = process.env.T3CODE_STRICT_PROVIDER_LIFECYCLE_GUARD !== "0"; type TurnStartRequestedDomainEvent = Extract< @@ -268,6 +270,51 @@ function hasRenderableAssistantText(text: string | undefined): boolean { return (text?.trim().length ?? 0) > 0; } +// A closing fence may be indented at most three spaces more than its opener. +const MARKDOWN_FENCE_PATTERN = /^( *)(`{3,}|~{3,})/; +const BLANK_LINE_PATTERN = /^[ \t]*$/; +// A complete list marker lets the previous item land before the next is done. +const LIST_ITEM_START_PATTERN = /^[ \t]*(?:[-*+]|\d{1,9}[.)])[ \t]/; + +/** Return text whose Markdown shape is complete, leaving the active block buffered. */ +export function splitBufferedAssistantText(text: string): { ready: string; rest: string } { + let openFence: { marker: string; indent: number } | null = null; + let boundary = -1; + let lineStart = 0; + for (;;) { + const newline = text.indexOf("\n", lineStart); + const line = text + .slice(lineStart, newline === -1 ? text.length : newline) + .replace(/[ \t\r]+$/, ""); + if (openFence === null && lineStart > 0 && LIST_ITEM_START_PATTERN.test(line)) { + boundary = lineStart; + } + if (newline === -1) break; + const fenceMatch = MARKDOWN_FENCE_PATTERN.exec(line); + if (fenceMatch) { + const indent = fenceMatch[1]!.length; + const marker = fenceMatch[2]!; + if (openFence === null) { + openFence = { marker, indent }; + } else if ( + marker[0] === openFence.marker[0] && + marker.length >= openFence.marker.length && + indent <= openFence.indent + 3 && + line.length === indent + marker.length + ) { + openFence = null; + boundary = newline + 1; + } + } else if (openFence === null && BLANK_LINE_PATTERN.test(line) && lineStart > 0) { + boundary = newline + 1; + } + lineStart = newline + 1; + } + return boundary === -1 + ? { ready: "", rest: text } + : { ready: text.slice(0, boundary), rest: text.slice(boundary) }; +} + function proposedPlanIdForTurn(threadId: ThreadId, turnId: TurnId): string { return `plan:${threadId}:turn:${turnId}`; } @@ -1618,6 +1665,11 @@ const make = Effect.gen(function* () { timeToLive: BUFFERED_MESSAGE_TEXT_BY_MESSAGE_ID_TTL, lookup: () => Effect.succeed(""), }); + const lastAssistantDeliveryAtByMessageId = yield* Cache.make({ + capacity: BUFFERED_MESSAGE_TEXT_BY_MESSAGE_ID_CACHE_CAPACITY, + timeToLive: BUFFERED_MESSAGE_TEXT_BY_MESSAGE_ID_TTL, + lookup: () => Effect.succeed(0), + }); const assistantSegmentStateByTurnKey = yield* Cache.make({ capacity: TURN_MESSAGE_IDS_BY_TURN_CACHE_CAPACITY, @@ -1809,7 +1861,7 @@ const make = Effect.gen(function* () { }); }); - const appendBufferedAssistantText = (messageId: MessageId, delta: string) => + const appendBufferedAssistantText = (messageId: MessageId, delta: string, atMillis: number) => Cache.getOption(bufferedAssistantTextByMessageId, messageId).pipe( Effect.flatMap((existingText) => Effect.gen(function* () { @@ -1817,6 +1869,30 @@ const make = Effect.gen(function* () { onNone: () => delta, onSome: (text) => `${text}${delta}`, }); + const { ready, rest } = splitBufferedAssistantText(nextText); + const lastDeliveredAt = Option.getOrUndefined( + yield* Cache.getOption(lastAssistantDeliveryAtByMessageId, messageId), + ); + if ( + (lastDeliveredAt === undefined || + atMillis - lastDeliveredAt >= MIN_ASSISTANT_DELIVERY_INTERVAL_MS) && + hasRenderableAssistantText(ready) && + rest.length <= MAX_BUFFERED_ASSISTANT_CHARS + ) { + if (rest.length > 0) { + yield* commitRuntimeMutation( + Cache.set(bufferedAssistantTextByMessageId, messageId, rest), + ); + } else { + yield* commitRuntimeMutation( + Cache.invalidate(bufferedAssistantTextByMessageId, messageId), + ); + } + yield* commitRuntimeMutation( + Cache.set(lastAssistantDeliveryAtByMessageId, messageId, atMillis), + ); + return ready; + } if (nextText.length <= MAX_BUFFERED_ASSISTANT_CHARS) { yield* commitRuntimeMutation( Cache.set(bufferedAssistantTextByMessageId, messageId, nextText), @@ -1843,7 +1919,11 @@ const make = Effect.gen(function* () { ); const clearBufferedAssistantText = (messageId: MessageId) => - commitRuntimeMutation(Cache.invalidate(bufferedAssistantTextByMessageId, messageId)); + commitRuntimeMutation(Cache.invalidate(bufferedAssistantTextByMessageId, messageId)).pipe( + Effect.andThen( + commitRuntimeMutation(Cache.invalidate(lastAssistantDeliveryAtByMessageId, messageId)), + ), + ); const appendBufferedProposedPlan = (planId: string, delta: string, createdAt: string) => Cache.getOption(bufferedProposedPlanById, planId).pipe( @@ -2752,7 +2832,13 @@ const make = Effect.gen(function* () { : "buffered", ); if (assistantDeliveryMode === "buffered") { - const spillChunk = yield* appendBufferedAssistantText(assistantMessageId, assistantDelta); + // Provider event timestamps may be fixed for an entire part; pace + // delivery by the ingestion clock instead. + const spillChunk = yield* appendBufferedAssistantText( + assistantMessageId, + assistantDelta, + yield* Clock.currentTimeMillis, + ); if (spillChunk.length > 0) { yield* orchestrationEngine.dispatch({ type: "thread.message.assistant.delta", diff --git a/apps/server/src/orchestration/PullRequestSyncReactor.test.ts b/apps/server/src/orchestration/PullRequestSyncReactor.test.ts index 414f76f324..1406eae239 100644 --- a/apps/server/src/orchestration/PullRequestSyncReactor.test.ts +++ b/apps/server/src/orchestration/PullRequestSyncReactor.test.ts @@ -1,9 +1,11 @@ import { + EventId, ProjectId, ProviderInstanceId, PullRequestOperationError, ThreadId, type OrchestrationCommand, + type OrchestrationEvent, type OrchestrationProjectShell, type OrchestrationShellSnapshot, type OrchestrationThreadShell, @@ -18,6 +20,7 @@ import * as Crypto from "effect/Crypto"; import * as Deferred from "effect/Deferred"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; +import * as PubSub from "effect/PubSub"; import * as Queue from "effect/Queue"; import * as Ref from "effect/Ref"; import * as Stream from "effect/Stream"; @@ -31,6 +34,7 @@ import { } from "./Services/OrchestrationEngine.ts"; import { ProjectionSnapshotQuery } from "./Services/ProjectionSnapshotQuery.ts"; import * as PullRequestSyncReactor from "./PullRequestSyncReactor.ts"; +import { resolveAutoSettlementAt } from "./ThreadSettlementPolicy.ts"; const NOW = "2026-08-28T12:00:00.000Z"; const PROJECT_ID = ProjectId.make("sync-project"); @@ -152,6 +156,7 @@ function makeSummary( } interface HarnessOptions { + readonly onDispatch?: (command: SyncCommand | LinkCommand) => Effect.Effect; readonly invalidate?: PullRequestService["Service"]["invalidate"]; readonly snapshot: OrchestrationShellSnapshot; readonly summary?: ( @@ -165,6 +170,7 @@ interface HarnessOptions { const makeHarness = Effect.fn("makePullRequestSyncHarness")(function* (options: HarnessOptions) { const activation = yield* Deferred.make(); const snapshots = yield* Ref.make(options.snapshot); + const events = yield* PubSub.unbounded(); const snapshotReads = yield* Queue.unbounded(); const syncCommands = yield* Ref.make>([]); const linkCommands = yield* Ref.make>([]); @@ -188,11 +194,13 @@ const makeHarness = Effect.fn("makePullRequestSyncHarness")(function* (options: const dispatch: OrchestrationEngineShape["dispatch"] = (command) => { if (command.type === "thread.pull-request-link.sync") { return Ref.update(syncCommands, (recorded) => [...recorded, command]).pipe( + Effect.andThen(options.onDispatch?.(command) ?? Effect.void), Effect.as({ sequence: 1 }), ); } if (command.type === "thread.pull-request.link") { return Ref.update(linkCommands, (recorded) => [...recorded, command]).pipe( + Effect.andThen(options.onDispatch?.(command) ?? Effect.void), Effect.as({ sequence: 1 }), ); } @@ -213,6 +221,9 @@ const makeHarness = Effect.fn("makePullRequestSyncHarness")(function* (options: readEvents: () => Stream.empty, dispatch, streamDomainEvents: Stream.empty, + subscribeDomainEvents: PubSub.subscribe(events).pipe( + Effect.map((subscription) => Stream.fromSubscription(subscription)), + ), latestSequence: Effect.succeed(0), }), Layer.succeed(ServerActivation, Deferred.await(activation)), @@ -220,6 +231,7 @@ const makeHarness = Effect.fn("makePullRequestSyncHarness")(function* (options: ); return { + events, activation, snapshots, snapshotReads, @@ -274,6 +286,42 @@ function applySync( } describe("PullRequestSyncReactor", () => { + it.effect("syncs a newly linked merged PR without waiting for the periodic sweep", () => + Effect.scoped( + Effect.gen(function* () { + yield* TestClock.setTime(Date.parse(NOW)); + const fixture = yield* makeHarness({ + snapshot: makeSnapshot([makeThread("one")]), + summary: (input) => + Effect.succeed(makeSummary(input, { state: "merged", mergedAt: NOW })), + }); + yield* Effect.gen(function* () { + const reactor = yield* startAndSweep(fixture); + const link = makeLink(42); + yield* Ref.set( + fixture.snapshots, + makeSnapshot([makeThread("one", { pullRequests: [link] })]), + ); + yield* PubSub.publish(fixture.events, { + type: "thread.pull-request-linked", + sequence: 2, + eventId: EventId.make("linked"), + aggregateKind: "thread", + aggregateId: ThreadId.make("one"), + occurredAt: NOW, + commandId: null, + causationEventId: null, + correlationId: null, + metadata: {}, + payload: { threadId: ThreadId.make("one"), link, updatedAt: NOW }, + }); + yield* Queue.take(fixture.snapshotReads); + yield* reactor.drain; + assert.strictEqual((yield* Ref.get(fixture.syncCommands))[0]?.snapshot.state, "merged"); + }).pipe(Effect.provide(fixture.layer)); + }), + ), + ); it.effect("retries a failed stack read after the summary becomes terminal", () => Effect.scoped( Effect.gen(function* () { @@ -303,6 +351,7 @@ describe("PullRequestSyncReactor", () => { yield* Effect.gen(function* () { const reactor = yield* startAndSweep(fixture); const commands = yield* Ref.get(fixture.syncCommands); + assert.deepStrictEqual(commands, []); yield* Ref.update(fixture.snapshots, (snapshot) => applySync(snapshot, commands)); yield* sweepAgain(fixture, reactor); assert.strictEqual(attempts, 2); @@ -315,6 +364,85 @@ describe("PullRequestSyncReactor", () => { ), ); + it.effect("retries a failed sibling link before publishing a terminal snapshot", () => + Effect.scoped( + Effect.gen(function* () { + yield* TestClock.setTime(Date.parse(NOW)); + let failSibling = true; + const fixture = yield* makeHarness({ + snapshot: makeSnapshot([ + makeThread("one", { pullRequests: [makeLink(7, { state: "closed" })] }), + ]), + summary: (input) => + Effect.succeed(makeSummary(input, { state: "merged", mergedAt: NOW })), + stack: () => + Effect.succeed({ + id: "stack", + number: 7, + url: "/owner/repository/stacks/7", + base: "main", + layers: [ + { number: 7, headBranch: "feature", state: "merged" }, + { number: 8, headBranch: "sibling", state: "open" }, + ], + }), + onDispatch: (command) => + command.type === "thread.pull-request.link" && failSibling + ? Effect.die("temporary link failure") + : Effect.void, + }); + yield* Effect.gen(function* () { + const reactor = yield* startAndSweep(fixture); + assert.deepStrictEqual(yield* Ref.get(fixture.syncCommands), []); + failSibling = false; + yield* sweepAgain(fixture, reactor); + assert.strictEqual((yield* Ref.get(fixture.syncCommands))[0]?.snapshot.state, "merged"); + assert.strictEqual((yield* Ref.get(fixture.linkCommands)).at(-1)?.number, 8); + }).pipe(Effect.provide(fixture.layer)); + }), + ), + ); + + it.effect("concurrent stack reads persist a shared sibling once before syncing both roots", () => + Effect.scoped( + Effect.gen(function* () { + const readsReady = yield* Deferred.make(); + let reads = 0; + let linked = false; + const fixture = yield* makeHarness({ + snapshot: makeSnapshot([makeThread("one", { pullRequests: [makeLink(7), makeLink(8)] })]), + stack: () => + Effect.gen(function* () { + if (++reads === 2) yield* Deferred.succeed(readsReady, undefined); + yield* Deferred.await(readsReady); + return { + id: "stack", + number: 7, + url: "/owner/repository/stacks/7", + base: "main", + layers: [{ number: 9, headBranch: "sibling", state: "open" as const }], + }; + }), + onDispatch: (command) => + Effect.gen(function* () { + if (command.type === "thread.pull-request.link") { + yield* Effect.yieldNow; + assert.strictEqual(linked, false); + linked = true; + } else { + assert.strictEqual(linked, true); + } + }), + }); + yield* Effect.gen(function* () { + yield* startAndSweep(fixture); + assert.strictEqual((yield* Ref.get(fixture.linkCommands)).length, 1); + assert.strictEqual((yield* Ref.get(fixture.syncCommands)).length, 2); + }).pipe(Effect.provide(fixture.layer)); + }), + ), + ); + it.effect("explicit refresh reads a changed stack even when its PR summary is unchanged", () => Effect.scoped( Effect.gen(function* () { @@ -631,20 +759,38 @@ describe("PullRequestSyncReactor", () => { base: "main", layers: [ { number: 41, headBranch: "layer-1", state: "merged" }, - { number: 42, headBranch: "layer-2", state: "open" }, + { number: 42, headBranch: "layer-2", state: "merged" }, { number: 43, headBranch: "layer-3", state: "open" }, ], }; + let thread = makeThread("one", { + pullRequests: [ + makeLink(42, { state: "open" }), + makeLink(41, { state: "merged" }, { source: "stack-dismissed" }), + ], + }); const fixture = yield* makeHarness({ - snapshot: makeSnapshot([ - makeThread("one", { - pullRequests: [ - makeLink(42), - makeLink(41, { state: "merged" }, { source: "stack-dismissed" }), - ], - }), - ]), + snapshot: makeSnapshot([thread]), + summary: (input) => + Effect.succeed(makeSummary(input, { state: "merged", mergedAt: NOW })), stack: () => Effect.succeed(stack), + onDispatch: (command) => + Effect.sync(() => { + thread = + command.type === "thread.pull-request.link" + ? { ...thread, pullRequests: [...thread.pullRequests, makeLink(command.number)] } + : applySync(makeSnapshot([thread]), [command]).threads[0]!; + // Every projected event may wake settlement, including the terminal root update. + assert.isNull( + resolveAutoSettlementAt({ + thread, + pullRequest: null, + now: NOW, + autoSettleAfterDays: null, + autoSettleOnMerge: true, + }), + ); + }), }); yield* Effect.gen(function* () { diff --git a/apps/server/src/orchestration/PullRequestSyncReactor.ts b/apps/server/src/orchestration/PullRequestSyncReactor.ts index 0a39fa5d91..2298530210 100644 --- a/apps/server/src/orchestration/PullRequestSyncReactor.ts +++ b/apps/server/src/orchestration/PullRequestSyncReactor.ts @@ -22,6 +22,8 @@ import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; import * as Schedule from "effect/Schedule"; import type * as Scope from "effect/Scope"; +import * as Semaphore from "effect/Semaphore"; +import * as Stream from "effect/Stream"; import * as PullRequestService from "../pullRequest/PullRequestService.ts"; import { forkParked } from "../serverActivation.ts"; @@ -149,7 +151,7 @@ export const make = Effect.gen(function* () { (cause: Cause.Cause): Effect.Effect => Cause.hasInterruptsOnly(cause) ? Effect.failCause(cause) : Effect.logWarning(message, fields); - const sweep = Effect.fn("PullRequestSyncReactor.sweep")(function* () { + const sweep = Effect.fn("PullRequestSyncReactor.sweep")(function* (requestedKey?: string) { const snapshot = yield* snapshots.getShellSnapshot(); const now = yield* DateTime.now; const nowMs = DateTime.toEpochMillis(now); @@ -173,6 +175,7 @@ export const make = Effect.gen(function* () { // Layers auto-linked this sweep, so two links of one thread that share a // stack do not both try to add the same sibling. const linkedThisSweep = new Set(); + const persistence = yield* Semaphore.make(1); const syncEntry = Effect.fn("PullRequestSyncReactor.syncEntry")(function* ( entry: LinkEntry, @@ -185,21 +188,8 @@ export const make = Effect.gen(function* () { link.snapshot === null || !snapshotFieldsEqual(link.snapshot, fields) || !stacksEqual(link.stack, nextStack); - if (changed) { - const uuid = yield* crypto.randomUUIDv4; - yield* engine.dispatch({ - type: "thread.pull-request-link.sync", - commandId: CommandId.make(`server:pr-sync:${thread.id}:${uuid}`), - threadId: thread.id, - host: link.host, - repository: link.repository, - number: link.number, - snapshot: { ...fields, syncedAt: nowIso }, - stack: nextStack, - }); - } - if (fetchedStack === null || fetchedStack.stack === null) return; - for (const layer of fetchedStack.stack.layers) { + // Persist discovered siblings before a terminal snapshot can trigger settlement. + for (const layer of fetchedStack?.stack?.layers ?? []) { const layerKey = { host: link.host, repository: link.repository, number: layer.number }; const dedupeKey = `${thread.id}:${threadPullRequestKeyOf(layerKey)}`; if (linkedThisSweep.has(dedupeKey)) continue; @@ -211,25 +201,29 @@ export const make = Effect.gen(function* () { } const url = siblingPullRequestUrl(link.url, layer.number); if (url === null) continue; + const uuid = yield* crypto.randomUUIDv4; + yield* engine.dispatch({ + type: "thread.pull-request.link", + commandId: CommandId.make(`server:pr-stack-link:${thread.id}:${uuid}`), + threadId: thread.id, + ...layerKey, + url, + source: "stack", + }); linkedThisSweep.add(dedupeKey); + } + if (changed) { const uuid = yield* crypto.randomUUIDv4; - yield* engine - .dispatch({ - type: "thread.pull-request.link", - commandId: CommandId.make(`server:pr-stack-link:${thread.id}:${uuid}`), - threadId: thread.id, - ...layerKey, - url, - source: "stack", - }) - .pipe( - Effect.catchCause( - logSkipped("pull request stack layer link skipped", { - threadId: thread.id, - number: layer.number, - }), - ), - ); + yield* engine.dispatch({ + type: "thread.pull-request-link.sync", + commandId: CommandId.make(`server:pr-sync:${thread.id}:${uuid}`), + threadId: thread.id, + host: link.host, + repository: link.repository, + number: link.number, + snapshot: { ...fields, syncedAt: nowIso }, + stack: nextStack, + }); } }); @@ -270,8 +264,11 @@ export const make = Effect.gen(function* () { ) : null; if (needsStack) { - if (fetchedStack === null) retryStacks.add(key); - else retryStacks.delete(key); + if (fetchedStack === null) { + retryStacks.add(key); + return; + } + retryStacks.delete(key); } // The host answered, so the cadence clock ticks even if a dispatch below is rejected. lastSyncedAt.set(key, nowMs); @@ -281,9 +278,13 @@ export const make = Effect.gen(function* () { entries, (entry) => syncEntry(entry, fields, fetchedStack).pipe( - Effect.catchCause( - logSkipped("pull request sync skipped", { threadId: entry.thread.id, key }), - ), + persistence.withPermits(1), + Effect.catchCause((cause) => { + if (!Cause.hasInterruptsOnly(cause)) retryStacks.add(key); + return logSkipped("pull request sync skipped", { threadId: entry.thread.id, key })( + cause, + ); + }), ), { discard: true }, ); @@ -292,22 +293,30 @@ export const make = Effect.gen(function* () { yield* Effect.forEach( groups, ([key, entries]) => - isDue(key, entries, nowMs) + (requestedKey === undefined || requestedKey === key) && isDue(key, entries, nowMs) ? syncGroup(key, entries).pipe( Effect.catchCause(logSkipped("pull request sync skipped", { key })), ) : Effect.void, - { concurrency: 8, discard: true }, + // As wide as one batched summary read, so the sweep's reads on a host arrive together and + // GitHub answers them in one request rather than one `gh pr view` apiece. + { concurrency: 25, discard: true }, ); }); - const worker = yield* makeDrainableWorker(() => - sweep().pipe(Effect.catchCause(logSkipped("pull request sync sweep failed", {}))), + const worker = yield* makeDrainableWorker((key: string | undefined) => + sweep(key).pipe(Effect.catchCause(logSkipped("pull request sync sweep failed", {}))), ); const start: PullRequestSyncReactor["Service"]["start"] = Effect.fn( "PullRequestSyncReactor.start", )(function* () { + const events = yield* engine.subscribeDomainEvents; + yield* forkParked( + Stream.runForEach(events, (event) => + event.type === "thread.pull-request-linked" ? requestSync(event.payload.link) : Effect.void, + ), + ); yield* forkParked( Effect.gen(function* () { yield* worker.enqueue(undefined); @@ -318,8 +327,9 @@ export const make = Effect.gen(function* () { const requestSync: PullRequestSyncReactor["Service"]["requestSync"] = (key) => Effect.suspend(() => { - requested.set(threadPullRequestKeyOf(key), ++requestGeneration); - return worker.enqueue(undefined); + const syncKey = threadPullRequestKeyOf(key); + requested.set(syncKey, ++requestGeneration); + return worker.enqueue(syncKey); }); return { start, drain: worker.drain, requestSync } satisfies PullRequestSyncReactor["Service"]; diff --git a/apps/server/src/orchestration/RelayWorkerBridge.test.ts b/apps/server/src/orchestration/RelayWorkerBridge.test.ts index 727e5260ce..ab4a135e61 100644 --- a/apps/server/src/orchestration/RelayWorkerBridge.test.ts +++ b/apps/server/src/orchestration/RelayWorkerBridge.test.ts @@ -213,6 +213,65 @@ describe("RelayWorkerBridge direct receipts", () => { ), ).toEqual([]); }); + + it("binds the installed Claude plugin's unversioned job result", () => { + const result = { + type: "tool_result", + content: [ + { + type: "text", + text: JSON.stringify({ + id: jobId, + status: "queued", + providerType: "codex", + createdAt: "2026-09-24T00:00:00.000Z", + }), + }, + ], + }; + expect( + relayBindingFromToolEvent( + toolEvent({ + toolName: "mcp__plugin_relay-orchestrator_relay__relay_delegate", + result, + }), + ), + ).toEqual([ + { kind: "job", id: jobId, threadId: "thread-a", turnId: "turn-a", toolCallId: "tool-a" }, + ]); + expect( + relayBindingFromToolEvent( + toolEvent({ + toolName: "mcp__relay__relay_delegate", + result, + }), + ), + ).toEqual([ + { kind: "job", id: jobId, threadId: "thread-a", turnId: "turn-a", toolCallId: "tool-a" }, + ]); + expect( + relayBindingFromToolEvent( + toolEvent({ + toolName: "mcp__plugin_relay-orchestrator_relay__relay_panel", + result: { + type: "tool_result", + content: [ + { + type: "text", + text: JSON.stringify({ + id: panelId, + jobs: [jobId], + unstarted: [{ index: 1 }], + }), + }, + ], + }, + }), + ), + ).toEqual([ + { kind: "panel", id: panelId, threadId: "thread-a", turnId: "turn-a", toolCallId: "tool-a" }, + ]); + }); }); describe("Relay observe v1", () => { @@ -267,7 +326,10 @@ const args = process.argv.slice(2); calls.push(args); fs.writeFileSync(callsPath, JSON.stringify(calls)); const id = args[0] === "observe" && args[1] === "--panel" ? args[2] : args[1]; -if (args[0] === "observe" && state[id]) console.log(JSON.stringify(state[id])); +if (args[0] === "observe" && state[id]?.schemaVersion === 1) console.log(JSON.stringify(state[id])); +else if (args[0] === "observe" && state[id]) console.log(JSON.stringify({ usage: "relay init | status [id]", request: "JSON from --request file" })); +else if (args[0] === "status" && state[id]) console.log(JSON.stringify(state[id])); +else if (args[0] === "panel-result" && state[id]) console.log(JSON.stringify(state[id])); else if (args[0] === "cancel" && state[id]) console.log(JSON.stringify({id, status: "cancelling"})); else process.exit(1); `, @@ -362,14 +424,11 @@ persistence("Relay persisted observer and controls", (it) => { toolCallId: "tool-a", status: "completed", data: { - toolName: "mcp__relay__relay_delegate", + toolName: "mcp__plugin_relay-orchestrator_relay__relay_delegate", result: { type: "tool_result", tool_use_id: "tool-a", - content: encodeJson({ - content: [{ type: "text", text: "Worker dispatched" }], - structuredContent: { schemaVersion: 1, kind: "job", jobId, attempt: 1 }, - }), + content: [{ type: "text", text: encodeJson({ id: jobId, status: "queued" }) }], }, }, }; @@ -395,6 +454,55 @@ persistence("Relay persisted observer and controls", (it) => { effort: "high", cancellable: true, }); + const legacyJobId = "job-77777777-7777-4777-8777-777777777777"; + cli.setState({ + [jobId]: job, + [legacyJobId]: { + id: legacyJobId, + status: "running", + providerType: "codex", + model: "gpt-6-sol", + effort: "low", + createdAt: "2026-09-24T00:00:00.000Z", + updatedAt: "2026-09-24T00:00:01.000Z", + }, + }); + yield* bridge.recordToolResult( + toolEvent({ + toolName: "mcp__plugin_relay-orchestrator_relay__relay_delegate", + itemId: "legacy-tool", + turnId: "legacy-turn", + result: { + type: "tool_result", + content: [ + { + type: "text", + text: encodeJson({ + id: legacyJobId, + status: "queued", + providerType: "codex", + }), + }, + ], + }, + }), + ); + yield* bridge.reconcile; + const legacyRows = yield* sql` + SELECT turn_id AS turnId, payload_json AS payload + FROM projection_thread_activities + WHERE kind = 'task.started' + AND json_extract(payload_json, '$.taskId') = ${`relay:${legacyJobId}`} + `; + expect(legacyRows).toHaveLength(1); + expect(legacyRows[0]?.turnId).toBe("legacy-turn"); + expect(decodeJson(legacyRows[0]?.payload)).toMatchObject({ + taskId: `relay:${legacyJobId}`, + source: "relay", + model: "gpt-6-sol", + effort: "low", + toolUseId: "legacy-tool", + }); const wrongThread = yield* Effect.result( bridge.cancel(ThreadId.make("thread-b"), `relay:${jobId}`), ); @@ -412,7 +520,7 @@ persistence("Relay persisted observer and controls", (it) => { yield* unavailable.reconcile; yield* unavailable.reconcile; const idleRows = - yield* sql`SELECT payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.progress' AND summary = 'Relay observer unavailable'`; + yield* sql`SELECT payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.progress' AND summary = 'Relay observer unavailable' AND json_extract(payload_json, '$.taskId') = ${`relay:${jobId}`}`; expect(idleRows).toHaveLength(1); expect(decodeJson(idleRows[0]?.payload)).toMatchObject({ taskId: `relay:${jobId}`, @@ -442,6 +550,7 @@ persistence("Relay persisted observer and controls", (it) => { const repeatedOutage = yield* sql` SELECT payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.progress' AND summary = 'Relay observer unavailable' + AND json_extract(payload_json, '$.taskId') = ${`relay:${jobId}`} ORDER BY sequence `; expect(repeatedOutage).toHaveLength(2); @@ -466,7 +575,7 @@ persistence("Relay persisted observer and controls", (it) => { }); yield* reconnected.reconcile; const completed = - yield* sql`SELECT payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.completed'`; + yield* sql`SELECT payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.completed' AND json_extract(payload_json, '$.taskId') = ${`relay:${jobId}`}`; expect(completed).toHaveLength(1); expect(decodeJson(completed[0]?.payload)).toMatchObject({ taskId: `relay:${jobId}`, @@ -515,6 +624,7 @@ persistence("Relay persisted observer and controls", (it) => { const pendingResume = yield* sql` SELECT turn_id AS turnId, payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.progress' AND activity_id LIKE 'relay-observer-unavailable:%' + AND json_extract(payload_json, '$.taskId') = ${`relay:${jobId}`} ORDER BY sequence DESC LIMIT 1 `; expect(pendingResume).toHaveLength(1); @@ -530,7 +640,7 @@ persistence("Relay persisted observer and controls", (it) => { cli.setState({ [jobId]: { ...job, attempt: 2, sequence: 1, status: "running" } }); yield* resumedBridge.reconcile; const resumed = - yield* sql`SELECT turn_id AS turnId, payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.started' ORDER BY created_at`; + yield* sql`SELECT turn_id AS turnId, payload_json AS payload FROM projection_thread_activities WHERE kind = 'task.started' AND json_extract(payload_json, '$.taskId') = ${`relay:${jobId}`} ORDER BY created_at`; expect(resumed).toHaveLength(2); expect(decodeJson(resumed[1]?.payload)).toMatchObject({ taskId: `relay:${jobId}`, @@ -944,12 +1054,143 @@ persistence("Relay persisted observer and controls", (it) => { Effect.provideService(ServerEnvironment, fakeEnvironment), Effect.provideService(ThreadBackgroundLivenessService, liveness), ); + const callsBeforeBoot = cli.calls().length; yield* bootAfterChurn.reconcile; + const bootCalls = cli.calls().slice(callsBeforeBoot); + const observed = (id: string) => + bootCalls.findIndex((args) => args[0] === "observe" && args[1] === id); + expect(observed(legacyJobId)).toBeGreaterThan(-1); + expect(observed(completedPanelJobId)).toBeGreaterThan(-1); + expect(observed(completedPanelJobId)).toBeLessThan(observed(legacyJobId)); expect( cli .calls() .filter((args) => args[0] === "observe" && args[1]?.startsWith("job-00000000-")), ).toHaveLength(0); + + const missingId = "job-cccccccc-cccc-4ccc-8ccc-cccccccccccc"; + yield* sql` + INSERT INTO projection_thread_activities ( + activity_id, thread_id, turn_id, tone, kind, summary, payload_json, sequence, created_at + ) VALUES ( + ${`relay-binding:${missingId}`}, 'thread-a', 'turn-missing', 'info', + 'relay.binding', 'Relay worker bound', + ${encodeJson({ kind: "job", id: missingId, toolCallId: "tool-missing", environmentId: "test-environment" })}, + 9999, '2026-09-24T00:00:00.000Z' + ) + `; + const missingBridge = yield* makeWithCliPath(cli.path).pipe( + Effect.provideService(OrchestrationEngineService, engine), + Effect.provideService(ServerEnvironment, fakeEnvironment), + Effect.provideService(ThreadBackgroundLivenessService, liveness), + ); + yield* missingBridge.reconcile; + yield* missingBridge.reconcile; + yield* missingBridge.reconcile; + const missingRows = yield* sql` + SELECT payload_json AS payload FROM projection_thread_activities + WHERE kind = 'task.progress' AND json_extract(payload_json, '$.taskId') = ${`relay:${missingId}`} + `; + expect(missingRows).toHaveLength(1); + expect(decodeJson(missingRows[0]?.payload)).toMatchObject({ + status: "idle", + summary: "Relay observer unavailable", + cancellable: false, + }); + const missingPolls = cli + .calls() + .filter((args) => args[0] === "observe" && args[1] === missingId).length; + yield* missingBridge.reconcile; + expect( + cli.calls().filter((args) => args[0] === "observe" && args[1] === missingId), + ).toHaveLength(missingPolls); + + const legacyPanelId = "panel-88888888-8888-4888-8888-888888888888"; + const priorId = "job-99999999-9999-4999-8999-999999999999"; + const currentId = "job-aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"; + const otherId = "job-bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"; + const legacyDate = "2026-09-24T00:00:01.000Z"; + const legacyJobs = [ + { + id: currentId, + status: "running", + panelId: legacyPanelId, + panelMemberIndex: 0, + providerType: "codex", + createdAt: legacyDate, + updatedAt: legacyDate, + }, + { + id: otherId, + status: "running", + panelId: legacyPanelId, + panelMemberIndex: 1, + providerType: "claude-alt", + createdAt: legacyDate, + updatedAt: legacyDate, + }, + ]; + cli.setState({ + [legacyPanelId]: { + id: legacyPanelId, + request: { members: [{}, {}] }, + jobs: legacyJobs, + previousJobs: [priorId], + unstarted: [], + complete: false, + createdAt: legacyDate, + }, + [priorId]: { + id: priorId, + status: "failed", + panelId: legacyPanelId, + panelMemberIndex: 0, + createdAt: legacyDate, + updatedAt: legacyDate, + }, + [currentId]: legacyJobs[0], + [otherId]: legacyJobs[1], + }); + yield* bootAfterChurn.recordToolResult( + toolEvent({ + toolName: "mcp__plugin_relay-orchestrator_relay__relay_panel", + itemId: "legacy-panel-tool", + turnId: "legacy-panel-turn", + result: { + type: "tool_result", + content: [ + { + type: "text", + text: encodeJson({ + id: legacyPanelId, + jobs: [currentId, otherId], + unstarted: [], + }), + }, + ], + }, + }), + ); + yield* bootAfterChurn.reconcile; + const legacyMemberId = `relay-panel:${legacyPanelId}:member:0`; + const legacyMember = yield* sql` + SELECT turn_id AS turnId, payload_json AS payload + FROM projection_thread_activities WHERE kind = 'task.started' + AND json_extract(payload_json, '$.taskId') = ${legacyMemberId} + `; + expect(legacyMember).toHaveLength(1); + expect(legacyMember[0]?.turnId).toBe("legacy-panel-turn"); + expect(decodeJson(legacyMember[0]?.payload)).toMatchObject({ + taskId: legacyMemberId, + attempt: 2_000_001, + parentAgentId: `relay-panel:${legacyPanelId}`, + }); + expect( + (yield* bootAfterChurn.cancel(ThreadId.make("thread-a"), legacyMemberId)).disposition, + ).toBe("cancel-requested"); + expect(cli.calls().some((args) => args[0] === "cancel" && args[1] === currentId)).toBe( + true, + ); }).pipe(Effect.ensuring(Effect.sync(() => cli.cleanup()))); }, ); diff --git a/apps/server/src/orchestration/RelayWorkerBridge.ts b/apps/server/src/orchestration/RelayWorkerBridge.ts index cd05b9e9b4..1671b3fea6 100644 --- a/apps/server/src/orchestration/RelayWorkerBridge.ts +++ b/apps/server/src/orchestration/RelayWorkerBridge.ts @@ -17,6 +17,7 @@ import { type OrchestrationThreadActivity, } from "@t3tools/contracts"; import * as Context from "effect/Context"; +import * as Clock from "effect/Clock"; import * as Effect from "effect/Effect"; import * as DateTime from "effect/DateTime"; import * as Layer from "effect/Layer"; @@ -154,7 +155,7 @@ function relayReceiptFromToolEvent(event: ProviderRuntimeEvent): RelayToolReceip return undefined; } const result = item?.result ?? data?.result; - const structured = relayStructuredResult(result); + const structured = relayStructuredResult(result) ?? relayLegacyResult(result); if (!structured || structured.schemaVersion !== 1) return undefined; const base = { threadId: event.threadId, @@ -193,6 +194,48 @@ function relayReceiptFromToolEvent(event: ProviderRuntimeEvent): RelayToolReceip return undefined; } +/** The installed Claude plugin still returns its job/panel record as text. */ +function relayLegacyResult(value: unknown): Record | undefined { + const object = record(value); + const content = object?.content; + if (!Array.isArray(content) || content.length > 4) return undefined; + const text = record(content[0])?.text; + if (typeof text !== "string" || text.length > MAX_MCP_RECEIPT_CODE_UNITS) return undefined; + try { + const parsed = record(JSON.parse(text)); + if (!parsed) return undefined; + if ( + typeof parsed.id === "string" && + JOB_ID.test(parsed.id) && + typeof parsed.status === "string" && + parsed.status !== "not_dispatched" + ) { + return { + schemaVersion: 1, + kind: "job", + jobId: parsed.id, + ...(nonNegativeInteger(parsed.attempt) !== undefined ? { attempt: parsed.attempt } : {}), + }; + } + if ( + typeof parsed.id === "string" && + PANEL_ID.test(parsed.id) && + Array.isArray(parsed.jobs) && + parsed.jobs.length <= 8 + ) { + return { + schemaVersion: 1, + kind: "panel", + panelId: parsed.id, + jobIds: parsed.jobs.filter((id): id is string => typeof id === "string" && JOB_ID.test(id)), + }; + } + } catch { + /* Invalid plugin text is not a dispatch receipt. */ + } + return undefined; +} + function relayStructuredResult(value: unknown, depth = 0): Record | undefined { if (depth > 4) return undefined; const object = record(value); @@ -280,6 +323,79 @@ export function parseObservation( return undefined; } +function legacyObservation( + value: unknown, + expectedId: string, + priorJobSlots: ReadonlyMap = new Map(), +): RelayJobObservation | RelayPanelObservation | undefined { + const object = record(value); + if (!object || object.id !== expectedId) return undefined; + if (JOB_ID.test(expectedId)) { + const attempt = nonNegativeInteger(object.attempt) ?? 1; + const sequence = Date.parse(String(object.updatedAt ?? object.createdAt)); + if (attempt < 1 || !Number.isSafeInteger(sequence) || typeof object.status !== "string") + return undefined; + return parseObservation( + { + schemaVersion: 1, + kind: "job", + id: expectedId, + attempt, + sequence, + status: object.status, + providerType: object.providerType, + model: object.actualModel ?? object.model, + effort: object.effort, + panelId: object.panelId, + panelMemberIndex: object.panelMemberIndex, + createdAt: object.createdAt, + updatedAt: object.updatedAt, + }, + expectedId, + ); + } + if ( + !PANEL_ID.test(expectedId) || + !Array.isArray(object.jobs) || + !Array.isArray(object.unstarted) || + object.jobs.length > 8 + ) + return undefined; + const request = record(object.request); + const requested = request?.members; + if (!Array.isArray(requested) || requested.length < 2 || requested.length > 8) return undefined; + const jobs = object.jobs.map(record); + const members = requested.map((_, index) => { + const prior = Array.isArray(object.previousJobs) ? object.previousJobs : []; + const slotAttempt = 1 + prior.filter((id) => priorJobSlots.get(id) === index).length; + const job = jobs.find((candidate) => candidate?.panelMemberIndex === index); + const jobId = typeof job?.id === "string" && JOB_ID.test(job.id) ? job.id : null; + return { + index, + slotAttempt, + state: jobId ? "started" : "unstarted", + jobId, + ...(jobId ? { job: legacyObservation(job, jobId) } : {}), + }; + }); + return parseObservation( + { + schemaVersion: 1, + kind: "panel", + id: expectedId, + createdAt: object.createdAt, + complete: object.complete === true, + members, + }, + expectedId, + ); +} + +function legacyObserveUnavailable(value: unknown): boolean { + const object = record(value); + return typeof object?.usage === "string" && typeof object.request === "string"; +} + function observationFingerprint(observation: RelayJobObservation | RelayPanelObservation): string { return NodeCrypto.createHash("sha256") .update(JSON.stringify(observation)) @@ -363,6 +479,7 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu const seen = new Map(); const currentPanelJobs = new Map>(); const failedObservations = new Map(); + const missingObservationRetryAt = new Map(); const unavailable = new Set(); const outageEpochs = new Map(); const pendingPanelDispatchPolls = new Map(); @@ -370,6 +487,7 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu string, { attempt: number; usage: RelayUsageRollup | undefined } >(); + const legacyPriorJobSlots = new Map>(); const append = Effect.fn("RelayWorkerBridge.append")(function* ( binding: RelayBinding, @@ -457,14 +575,14 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu }); const readBindingPage = Effect.fn("RelayWorkerBridge.readBindingPage")(function* ( - afterRowId: number, + beforeRowId: number, ) { const rows = yield* sql` SELECT rowid AS rowId, thread_id AS threadId, turn_id AS turnId, payload_json AS payload FROM projection_thread_activities - WHERE rowid > ${afterRowId} AND kind = 'relay.binding' + WHERE rowid < ${beforeRowId} AND kind = 'relay.binding' AND json_extract(payload_json, '$.environmentId') = ${environmentId} - ORDER BY rowid ASC LIMIT 256 + ORDER BY rowid DESC LIMIT 256 `; return rows.map((raw) => ({ rowId: nonNegativeInteger(record(raw)?.rowId) ?? 0, @@ -729,19 +847,57 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu }, ); - const runCli = Effect.fn("RelayWorkerBridge.runCli")(function* (args: ReadonlyArray) { + const runCli = Effect.fn("RelayWorkerBridge.runCli")(function* ( + args: ReadonlyArray, + timeout = 12_000, + ) { if (!enabled || !cliPath) return Option.none<{ stdout: string; stderr: string }>(); return yield* Effect.tryPromise({ try: () => execFile(process.execPath, [cliPath, ...args], { - timeout: 12_000, - maxBuffer: MAX_OBSERVE_BYTES, + timeout, + maxBuffer: args[0] === "panel-result" ? MAX_MCP_RECEIPT_CODE_UNITS : MAX_OBSERVE_BYTES, windowsHide: true, }), catch: () => undefined, }).pipe(Effect.option); }); + const parseLegacyCliObservation = Effect.fn("RelayWorkerBridge.parseLegacyCliObservation")( + function* (value: unknown, id: string) { + if (!PANEL_ID.test(id)) return legacyObservation(value, id); + const previousJobs = record(value)?.previousJobs; + if (!Array.isArray(previousJobs) || previousJobs.length > 64) return undefined; + const previousIds = previousJobs.filter( + (previousId): previousId is string => + typeof previousId === "string" && JOB_ID.test(previousId), + ); + if (previousIds.length !== previousJobs.length) return undefined; + const priorSlots = legacyPriorJobSlots.get(id) ?? new Map(); + const missing = previousIds.filter((previousId) => !priorSlots.has(previousId)); + // Old retry records are immutable. Cache resolved slots and bound an + // unavailable history lookup so a current panel cannot stall the sweep. + const resolved = yield* Effect.forEach( + missing, + (previousId) => + Effect.gen(function* () { + const result = yield* runCli(["status", previousId], 2_000); + const decoded = Option.isSome(result) ? decodeJson(result.value.stdout) : Option.none(); + const job = Option.isSome(decoded) ? record(decoded.value) : undefined; + const index = nonNegativeInteger(job?.panelMemberIndex); + return job?.id === previousId && job.panelId === id && index !== undefined && index <= 7 + ? ([previousId, index] as const) + : undefined; + }), + { concurrency: 8 }, + ); + if (resolved.some((entry) => entry === undefined)) return undefined; + for (const entry of resolved) if (entry) priorSlots.set(entry[0], entry[1]); + legacyPriorJobSlots.set(id, priorSlots); + return legacyObservation(value, id, priorSlots); + }, + ); + const readLatestTaskActivity = Effect.fn("RelayWorkerBridge.readLatestTaskActivity")(function* ( binding: RelayBinding, ) { @@ -964,7 +1120,35 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu const payload = latest.payload; const completedAttempt = nonNegativeInteger(payload?.attempt); const completedSequence = nonNegativeInteger(payload?.relaySequence); - if (completedAttempt === undefined || completedSequence === undefined) return; + if (completedAttempt === undefined || completedSequence === undefined) { + // A recovered receipt may outlive its Relay files. Retain the binding + // for a later return, but do not recheck every missing historical job + // on every sweep ahead of live workers. + if (!unavailable.has(binding.id)) { + yield* append( + binding, + "task.progress", + { + taskId, + title: binding.kind === "panel" ? "Relay panel" : "Relay worker", + attempt: 1, + relaySequence: 0, + outageEpoch: 1, + status: "idle", + summary: "Relay observer unavailable", + detail: "Relay observer unavailable", + ...bindingPayload(binding), + cancellable: false, + }, + "Relay observer unavailable", + `relay-observer-unavailable:${binding.id}:1`, + "1:0:1", + ); + unavailable.add(binding.id); + } + missingObservationRetryAt.set(binding.id, (yield* Clock.currentTimeMillis) + 60_000); + return; + } const pending = latest.kind === "task.completed" ? yield* pendingGenerationAfter(binding, latest.rowId, completedAttempt) @@ -972,6 +1156,7 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu if (latest.kind === "task.completed") { if (!pending) { active.delete(binding.id); + legacyPriorJobSlots.delete(binding.id); seen.delete(binding.id); failedObservations.delete(binding.id); unavailable.delete(binding.id); @@ -1010,16 +1195,27 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu }); const poll = Effect.fn("RelayWorkerBridge.poll")(function* (binding: RelayBinding) { + if ((missingObservationRetryAt.get(binding.id) ?? 0) > (yield* Clock.currentTimeMillis)) return; const result = yield* runCli( binding.kind === "job" ? ["observe", binding.id] : ["observe", "--panel", binding.id], ); - if (Option.isNone(result)) return yield* markObserverUnavailable(binding); - const parsed = decodeJson(result.value.stdout); - const observation = parseObservation( + const parsed = Option.isSome(result) ? decodeJson(result.value.stdout) : Option.none(); + let observation = parseObservation( Option.isSome(parsed) ? parsed.value : undefined, binding.id, ); + if (!observation && Option.isSome(parsed) && legacyObserveUnavailable(parsed.value)) { + const legacy = yield* runCli( + binding.kind === "job" ? ["status", binding.id] : ["panel-result", binding.id], + ); + const decoded = Option.isSome(legacy) ? decodeJson(legacy.value.stdout) : Option.none(); + observation = yield* parseLegacyCliObservation( + Option.isSome(decoded) ? decoded.value : undefined, + binding.id, + ); + } if (!observation) return yield* markObserverUnavailable(binding); + missingObservationRetryAt.delete(binding.id); const wasUnavailableInMemory = unavailable.delete(binding.id); const lastPersisted = seen.has(binding.id) ? undefined : yield* readLatestTaskActivity(binding); const wasUnavailable = @@ -1154,6 +1350,7 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu fingerprint, ); active.delete(binding.id); + legacyPriorJobSlots.delete(binding.id); seen.delete(binding.id); failedObservations.delete(binding.id); unavailable.delete(binding.id); @@ -1165,6 +1362,7 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu // idle panel. A new continuation or member resume receipt reactivates // observation; polling this unchanged snapshot forever is needless. active.delete(binding.id); + legacyPriorJobSlots.delete(binding.id); seen.delete(binding.id); failedObservations.delete(binding.id); unavailable.delete(binding.id); @@ -1304,12 +1502,14 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu // the crash window after command dispatch and before observer registration. if (!bootstrapped) { yield* recoverPersistedToolReceipts(); - let afterRowId = 0; + // Replay inserts historical bindings in receipt order. Visit newest + // bindings first so current workers project before old missing jobs. + let beforeRowId = Number.MAX_SAFE_INTEGER; for (;;) { - const page = yield* readBindingPage(afterRowId); + const page = yield* readBindingPage(beforeRowId); if (page.length === 0) break; for (const { rowId, binding } of page) { - afterRowId = Math.max(afterRowId, rowId); + beforeRowId = Math.min(beforeRowId, rowId); if (binding && !active.has(binding.id) && (yield* shouldObserveOnBoot(binding, rowId))) { active.set(binding.id, binding); } @@ -1404,13 +1604,25 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu ) return yield* new ProviderCancelSessionAgentError({ reason: "agent-not-active" }); const current = yield* runCli(["observe", "--panel", panelId]); - if (Option.isNone(current)) - return yield* new ProviderCancelSessionAgentError({ reason: "request-failed" }); - const decodedPanel = decodeJson(current.value.stdout); - const observedPanel = parseObservation( + const decodedPanel = Option.isSome(current) + ? decodeJson(current.value.stdout) + : Option.none(); + let observedPanel = parseObservation( Option.isSome(decodedPanel) ? decodedPanel.value : undefined, panelId, ); + if ( + !observedPanel && + Option.isSome(decodedPanel) && + legacyObserveUnavailable(decodedPanel.value) + ) { + const legacy = yield* runCli(["panel-result", panelId]); + const decoded = Option.isSome(legacy) ? decodeJson(legacy.value.stdout) : Option.none(); + observedPanel = yield* parseLegacyCliObservation( + Option.isSome(decoded) ? decoded.value : undefined, + panelId, + ); + } if (observedPanel?.kind !== "panel") return yield* new ProviderCancelSessionAgentError({ reason: "request-failed" }); const member = observedPanel.members.find((candidate) => candidate.index === index); @@ -1441,12 +1653,17 @@ export const makeWithCliPath = Effect.fn("RelayWorkerBridge.makeWithCliPath")(fu if (!binding) return yield* new ProviderCancelSessionAgentError({ reason: "agent-not-active" }); const id = binding.id; const observed = yield* runCli(["observe", id]); - if (Option.isNone(observed)) - return yield* new ProviderCancelSessionAgentError({ reason: "request-failed" }); - const decodedJob = decodeJson(observed.value.stdout); - if ( - parseObservation(Option.isSome(decodedJob) ? decodedJob.value : undefined, id)?.kind !== "job" - ) { + const decodedJob = Option.isSome(observed) ? decodeJson(observed.value.stdout) : Option.none(); + let current = parseObservation(Option.isSome(decodedJob) ? decodedJob.value : undefined, id); + if (!current && Option.isSome(decodedJob) && legacyObserveUnavailable(decodedJob.value)) { + const legacy = yield* runCli(["status", id]); + const decoded = Option.isSome(legacy) ? decodeJson(legacy.value.stdout) : Option.none(); + current = yield* parseLegacyCliObservation( + Option.isSome(decoded) ? decoded.value : undefined, + id, + ); + } + if (current?.kind !== "job") { return yield* new ProviderCancelSessionAgentError({ reason: "request-failed" }); } const result = yield* runCli(["cancel", id]); diff --git a/apps/server/src/orchestration/ThreadSettlementReactor.test.ts b/apps/server/src/orchestration/ThreadSettlementReactor.test.ts index 455cb37a61..99a02d8e5b 100644 --- a/apps/server/src/orchestration/ThreadSettlementReactor.test.ts +++ b/apps/server/src/orchestration/ThreadSettlementReactor.test.ts @@ -1,10 +1,12 @@ import { DEFAULT_SERVER_SETTINGS, + EventId, ProjectId, ProviderInstanceId, PullRequestOperationError, ThreadId, type OrchestrationCommand, + type OrchestrationEvent, type OrchestrationProjectShell, type OrchestrationShellSnapshot, type OrchestrationThreadShell, @@ -171,6 +173,7 @@ const makeHarness = Effect.fn("makeThreadSettlementHarness")(function* (options: const settingsReads = yield* Queue.unbounded(); const settingsChanges = yield* PubSub.unbounded(); const mergedPullRequests = yield* PubSub.unbounded(); + const domainEvents = yield* PubSub.unbounded(); const commands = yield* Ref.make>([]); const branchCalls = yield* Ref.make< ReadonlyArray<{ readonly cwd: string; readonly branch: string }> @@ -261,6 +264,9 @@ const makeHarness = Effect.fn("makeThreadSettlementHarness")(function* (options: readEvents: () => Stream.empty, dispatch, streamDomainEvents: Stream.empty, + subscribeDomainEvents: PubSub.subscribe(domainEvents).pipe( + Effect.map((subscription) => Stream.fromSubscription(subscription)), + ), latestSequence: Effect.succeed(0), }), Layer.succeed(ServerSettingsService, serverSettings), @@ -283,6 +289,7 @@ const makeHarness = Effect.fn("makeThreadSettlementHarness")(function* (options: summaryRecovery, invalidatedCwds, updateSettings, + publishEvent: (event: OrchestrationEvent) => PubSub.publish(domainEvents, event), publishMerge: PubSub.publish(mergedPullRequests, { projectId: PROJECT_ID, repository: "owner/repository", @@ -306,12 +313,12 @@ const startHarness = Effect.fn("startThreadSettlementHarness")(function* ( describe("ThreadSettlementReactor", () => { it.effect( - "settles all-terminal links from snapshots and keeps open or unsynced links active", + "settles synced terminal links immediately while open, unsynced, or running threads wait", () => Effect.scoped( Effect.gen(function* () { yield* TestClock.setTime(Date.parse(NOW)); - const link = (number: number, state: "open" | "merged" | null) => ({ + const link = (number: number, state: "open" | "closed" | "merged" | null) => ({ host: "example.test", repository: "owner/repository", number, @@ -331,14 +338,30 @@ describe("ThreadSettlementReactor", () => { updatedAt: NOW, syncedAt: NOW, mergedAt: state === "merged" ? NOW : null, + closedAt: state === "closed" ? NOW : null, }, }); + const runningSession = { + threadId: ThreadId.make("running"), + status: "running" as const, + providerName: "Codex", + runtimeMode: "full-access" as const, + activeTurnId: null, + lastError: null, + updatedAt: NOW, + }; + const threads = [ + makeThread("merged", { pullRequests: [link(1, "open"), link(2, "merged")] }), + makeThread("closed", { pullRequests: [link(1, "open")] }), + makeThread("open", { pullRequests: [link(1, "open"), link(2, "open")] }), + makeThread("unsynced", { pullRequests: [link(1, "open"), link(2, null)] }), + makeThread("running", { + pullRequests: [link(1, "open")], + session: runningSession, + }), + ]; const fixture = yield* makeHarness({ - snapshot: makeSnapshot([ - makeThread("merged", { pullRequests: [link(1, "merged"), link(2, "merged")] }), - makeThread("open", { pullRequests: [link(1, "merged"), link(2, "open")] }), - makeThread("unsynced", { pullRequests: [link(1, "merged"), link(2, null)] }), - ]), + snapshot: makeSnapshot(threads), settings: { ...DEFAULT_SERVER_SETTINGS, sidebarAutoSettleOnMerge: true }, branchPullRequest: () => Effect.die("linked threads must not query the branch"), pullRequestSummary: () => Effect.die("linked threads must use their snapshots"), @@ -346,9 +369,66 @@ describe("ThreadSettlementReactor", () => { yield* Effect.gen(function* () { const reactor = yield* ThreadSettlementReactor.ThreadSettlementReactor; yield* startHarness(reactor, fixture.activation, fixture.snapshotReads); + assert.deepStrictEqual(yield* Ref.get(fixture.commands), []); + const eventBase = { + sequence: 2, + eventId: EventId.make("pull-request-synced"), + aggregateKind: "thread" as const, + occurredAt: NOW, + commandId: null, + causationEventId: null, + correlationId: null, + metadata: {}, + }; + for (const thread of threads) { + const terminalLink = link(1, thread.id === "closed" ? "closed" : "merged"); + yield* Ref.update(fixture.snapshots, (snapshot) => ({ + ...snapshot, + threads: snapshot.threads.map((current) => + current.id === thread.id + ? { ...current, pullRequests: [terminalLink, ...current.pullRequests.slice(1)] } + : current, + ), + })); + yield* fixture.publishEvent({ + ...eventBase, + type: "thread.pull-request-synced", + aggregateId: thread.id, + payload: { + threadId: thread.id, + host: terminalLink.host, + repository: terminalLink.repository, + number: terminalLink.number, + snapshot: terminalLink.snapshot!, + stack: null, + updatedAt: NOW, + }, + }); + yield* Queue.take(fixture.snapshotReads); + yield* reactor.drain; + } + assert.deepStrictEqual( + (yield* Ref.get(fixture.commands)).map(({ threadId }) => threadId), + [ThreadId.make("merged"), ThreadId.make("closed")], + ); + const readySession = { ...runningSession, status: "ready" as const }; + yield* Ref.update(fixture.snapshots, (snapshot) => ({ + ...snapshot, + threads: snapshot.threads.map((thread) => + thread.id === readySession.threadId ? { ...thread, session: readySession } : thread, + ), + })); + yield* fixture.publishEvent({ + ...eventBase, + type: "thread.session-set", + aggregateId: readySession.threadId, + payload: { threadId: readySession.threadId, session: readySession }, + }); + yield* Queue.take(fixture.snapshotReads); + yield* reactor.drain; assert.deepStrictEqual( (yield* Ref.get(fixture.commands)).map(({ threadId }) => threadId), - [ThreadId.make("merged")], + [ThreadId.make("merged"), ThreadId.make("closed"), ThreadId.make("running")], ); assert.deepStrictEqual(yield* Ref.get(fixture.branchCalls), []); assert.deepStrictEqual(yield* Ref.get(fixture.summaryCalls), []); @@ -357,6 +437,109 @@ describe("ThreadSettlementReactor", () => { ), ); + it.effect("rechecks only the affected thread after link and unlink events", () => + Effect.scoped( + Effect.gen(function* () { + yield* TestClock.setTime(Date.parse(NOW)); + const link = (number: number, state: "open" | "merged") => ({ + host: "example.test", + repository: "owner/repository", + number, + url: `https://example.test/owner/repository/pull/${number}`, + source: "manual" as const, + linkedAt: NOW, + stack: null, + snapshot: { + state, + title: "Review", + headBranch: "feature", + baseBranch: "main", + isDraft: false, + updatedAt: NOW, + syncedAt: NOW, + mergedAt: state === "merged" ? NOW : null, + closedAt: null, + }, + }); + const merged = link(1, "merged"); + const open = link(2, "open"); + const fixture = yield* makeHarness({ + snapshot: makeSnapshot([ + makeThread("new-link"), + makeThread("removed-link", { pullRequests: [open, merged] }), + ]), + settings: { + ...DEFAULT_SERVER_SETTINGS, + sidebarAutoSettleOnMerge: true, + sidebarAutoSettleAfterDays: null, + }, + branchPullRequest: () => Effect.die("linked event must use the projected snapshot"), + pullRequestSummary: () => Effect.die("linked event must not query a provider"), + }); + yield* Effect.gen(function* () { + const reactor = yield* ThreadSettlementReactor.ThreadSettlementReactor; + yield* startHarness(reactor, fixture.activation, fixture.snapshotReads); + assert.deepStrictEqual(yield* Ref.get(fixture.commands), []); + const eventBase = { + sequence: 2, + eventId: EventId.make("link-change"), + aggregateKind: "thread" as const, + occurredAt: NOW, + commandId: null, + causationEventId: null, + correlationId: null, + metadata: {}, + }; + + yield* Ref.update(fixture.snapshots, (snapshot) => ({ + ...snapshot, + threads: snapshot.threads.map((thread) => + thread.id === ThreadId.make("new-link") + ? { ...thread, pullRequests: [merged] } + : thread, + ), + })); + yield* fixture.publishEvent({ + ...eventBase, + type: "thread.pull-request-linked", + aggregateId: ThreadId.make("new-link"), + payload: { threadId: ThreadId.make("new-link"), link: merged, updatedAt: NOW }, + }); + yield* Queue.take(fixture.snapshotReads); + yield* reactor.drain; + + yield* Ref.update(fixture.snapshots, (snapshot) => ({ + ...snapshot, + threads: snapshot.threads.map((thread) => + thread.id === ThreadId.make("removed-link") + ? { ...thread, pullRequests: [merged] } + : thread, + ), + })); + yield* fixture.publishEvent({ + ...eventBase, + type: "thread.pull-request-unlinked", + aggregateId: ThreadId.make("removed-link"), + payload: { + threadId: ThreadId.make("removed-link"), + host: open.host, + repository: open.repository, + number: open.number, + updatedAt: NOW, + }, + }); + yield* Queue.take(fixture.snapshotReads); + yield* reactor.drain; + assert.deepStrictEqual( + (yield* Ref.get(fixture.commands)).map(({ threadId }) => threadId), + [ThreadId.make("new-link"), ThreadId.make("removed-link")], + ); + assert.deepStrictEqual(yield* Ref.get(fixture.summaryCalls), []); + }).pipe(Effect.provide(fixture.layer)); + }), + ), + ); + it("distinguishes a project that inherits the threshold from one that disables it", () => { const inherits = ThreadSettlementReactor.autoSettlementSettingsKey({ ...DEFAULT_SERVER_SETTINGS, @@ -417,12 +600,12 @@ describe("ThreadSettlementReactor", () => { makeThread("reused-detected", { branch: "reused", branchPullRequest: previous }), makeThread("foreign-branch-pr", { branch: "foreign", linkedPullRequest: previous }), makeThread("resumed-manual", { - branch: "main", + branch: "resumed", linkedPullRequest: previous, latestUserMessageAt: "2026-08-28T00:00:00.000Z", }), makeThread("resumed-detected", { - branch: "main", + branch: "resumed", branchPullRequest: previous, latestUserMessageAt: "2026-08-28T00:00:00.000Z", }), @@ -458,6 +641,9 @@ describe("ThreadSettlementReactor", () => { new Set((yield* Ref.get(fixture.commands)).map((command) => command.threadId)), new Set([ThreadId.make("retained-terminal"), ThreadId.make("foreign-branch-pr")]), ); + assert.isFalse( + (yield* Ref.get(fixture.branchCalls)).some(({ branch }) => branch === "resumed"), + ); }).pipe(Effect.provide(fixture.layer)); }), ), diff --git a/apps/server/src/orchestration/ThreadSettlementReactor.ts b/apps/server/src/orchestration/ThreadSettlementReactor.ts index b9041d2976..ff6f995df4 100644 --- a/apps/server/src/orchestration/ThreadSettlementReactor.ts +++ b/apps/server/src/orchestration/ThreadSettlementReactor.ts @@ -1,4 +1,9 @@ -import { CommandId, type ServerSettings as ServerSettingsValue } from "@t3tools/contracts"; +import { + CommandId, + type OrchestrationEvent, + type ServerSettings as ServerSettingsValue, + type ThreadId, +} from "@t3tools/contracts"; import { resolveProjectSettings } from "@t3tools/shared/projectSettings"; import { makeDrainableWorker } from "@t3tools/shared/DrainableWorker"; import * as Cause from "effect/Cause"; @@ -84,6 +89,7 @@ export const make = Effect.gen(function* () { const sweep = Effect.fn("ThreadSettlementReactor.sweep")(function* ( mergedPullRequest: PullRequestService.PullRequestMergeEvent | null, + threadId?: ThreadId, ) { const settings = yield* settingsService.getSettings; if (!autoSettlementConfigured(settings)) { @@ -94,7 +100,11 @@ export const make = Effect.gen(function* () { const projects = new Map(snapshot.projects.map((project) => [project.id, project])); // A merge rechecks all candidates, including branches that discovery has // not linked yet. Those lookups can still have cached the PR as open. - const candidates = snapshot.threads.filter((thread) => isAutoSettlementCandidate(thread, now)); + const candidates = snapshot.threads.filter( + (thread) => + (threadId === undefined || thread.id === threadId) && + isAutoSettlementCandidate(thread, now), + ); // Return the thread when it still needs a pull request decision. A rejected // dispatch skips it for this snapshot instead of retrying through a lookup. @@ -199,9 +209,30 @@ export const make = Effect.gen(function* () { }; const groups = Map.groupBy(lookupCandidates, lookupKey); + const wouldSettle = Effect.fn("ThreadSettlementReactor.wouldSettle")(function* ( + group: ReadonlyArray<(typeof candidates)[number]>, + pullRequest: SettlementPullRequest, + ) { + const currentSettings = yield* settingsService.getSettings; + const decisionNow = DateTime.formatIso(yield* DateTime.now); + return group.some((thread) => { + const { settings } = resolveProjectSettings(currentSettings, thread.projectId); + return ( + resolveAutoSettlementAt({ + thread, + pullRequest, + now: decisionNow, + autoSettleAfterDays: settings.sidebarAutoSettleAfterDays, + autoSettleOnMerge: settings.sidebarAutoSettleOnMerge, + }) !== null + ); + }); + }); + const pullRequestFor = Effect.fn("ThreadSettlementReactor.pullRequestFor")(function* ( - thread: (typeof candidates)[number], + group: ReadonlyArray<(typeof candidates)[number]>, ) { + const thread = group[0]!; const reference = thread.linkedPullRequest ?? thread.branchPullRequest; if (reference != null) { const matchesMerge = @@ -226,10 +257,21 @@ export const make = Effect.gen(function* () { }, { recoverTransientFailure: false }, ); + const terminal = { + state: summary.state, + closedAt: summary.closedAt ?? null, + mergedAt: summary.mergedAt ?? null, + } satisfies SettlementPullRequest; const cwd = lookupCwdByThreadId.get(thread.id); if (summary.state !== "open" && thread.branch !== null && cwd !== undefined) { // A reused branch can already have a new open PR while discovery // is replacing its old link. Do not let settlement win that race. + // Only pay for the uncached lookup when this sweep would otherwise + // settle: a terminal link that settles nothing (resumed thread, + // settle-on-merge off) would re-query the host every minute. A + // group that becomes eligible after this check waits for the next + // sweep rather than settling on the unverified link. + if (!(yield* wouldSettle(group, terminal))) return undefined; const current = yield* git.branchPullRequest( { cwd, branch: thread.branch }, { refresh: true }, @@ -243,11 +285,7 @@ export const make = Effect.gen(function* () { return current; } } - return { - state: summary.state, - closedAt: summary.closedAt ?? null, - mergedAt: summary.mergedAt ?? null, - } satisfies SettlementPullRequest; + return terminal; } if (thread.branch === null) return null; const cwd = lookupCwdByThreadId.get(thread.id); @@ -261,7 +299,8 @@ export const make = Effect.gen(function* () { groups.values(), (group) => Effect.gen(function* () { - const pullRequest = yield* pullRequestFor(group[0]!); + const pullRequest = yield* pullRequestFor(group); + if (pullRequest === undefined) return; yield* Effect.forEach(group, (thread) => settleThread(thread, pullRequest), { discard: true, }); @@ -279,8 +318,11 @@ export const make = Effect.gen(function* () { ); }); - const runSweep = (mergedPullRequest: PullRequestService.PullRequestMergeEvent | null) => - sweep(mergedPullRequest).pipe( + const runSweep = ( + mergedPullRequest: PullRequestService.PullRequestMergeEvent | null, + threadId?: ThreadId, + ) => + sweep(mergedPullRequest, threadId).pipe( Effect.catchCause((cause) => Cause.hasInterruptsOnly(cause) ? Effect.failCause(cause) @@ -289,13 +331,36 @@ export const make = Effect.gen(function* () { }), ), ); - const worker = yield* makeDrainableWorker(() => runSweep(null)); + const worker = yield* makeDrainableWorker((threadId: ThreadId | undefined) => + runSweep(null, threadId), + ); + + const processEvent = (event: OrchestrationEvent) => { + switch (event.type) { + case "thread.pull-request-linked": + case "thread.pull-request-synced": + case "thread.pull-request-unlinked": + // Merge notifications can arrive before the linked snapshot is projected. + // Recheck the persisted state so terminal links settle without the timer. + return worker.enqueue(event.payload.threadId); + case "thread.session-set": + if ( + event.payload.session.status !== "running" && + event.payload.session.status !== "starting" + ) { + return worker.enqueue(event.payload.threadId); + } + break; + } + return Effect.void; + }; const start: ThreadSettlementReactor["Service"]["start"] = Effect.fn( "ThreadSettlementReactor.start", )(function* () { const settingsChanges = yield* settingsService.subscribeChanges; const mergedPullRequests = yield* pullRequests.subscribeMerges; + const events = yield* engine.subscribeDomainEvents; const initialSettings = yield* settingsService.getSettings.pipe(Effect.orDie); let lastSettlementSettings = autoSettlementSettingsKey(initialSettings); yield* forkParked( @@ -314,7 +379,8 @@ export const make = Effect.gen(function* () { return worker.enqueue(undefined); }), ); - yield* forkParked(Stream.runForEach(mergedPullRequests, runSweep)); + yield* forkParked(Stream.runForEach(mergedPullRequests, (event) => runSweep(event))); + yield* forkParked(Stream.runForEach(events, processEvent)); }); return { start, drain: worker.drain } satisfies ThreadSettlementReactor["Service"]; diff --git a/apps/server/src/orchestration/decider.sessionLifecycle.test.ts b/apps/server/src/orchestration/decider.sessionLifecycle.test.ts index f60a02abe8..79b2158d40 100644 --- a/apps/server/src/orchestration/decider.sessionLifecycle.test.ts +++ b/apps/server/src/orchestration/decider.sessionLifecycle.test.ts @@ -199,6 +199,12 @@ it.layer(NodeServices.layer)("session lifecycle CAS decider", (it) => { readModel: makeReadModel(current), }); expect(stale).toEqual([]); + + const runningWithPendingRequest = yield* decideOrchestrationCommand({ + command: { ...command, session: { ...acceptedSession, status: "running" } }, + readModel: makeReadModel(current), + }); + expect(runningWithPendingRequest).toEqual([]); }), ); diff --git a/apps/server/src/orchestration/decider.ts b/apps/server/src/orchestration/decider.ts index 8c9a180a60..5afe8eb9a2 100644 --- a/apps/server/src/orchestration/decider.ts +++ b/apps/server/src/orchestration/decider.ts @@ -2629,6 +2629,7 @@ export const decideOrchestrationCommand = Effect.fn("decideOrchestrationCommand" thread.session.activeTurnId !== null || command.session.pendingTurnRequestId !== command.requestId || command.session.pendingTurnMessageId !== command.messageId || + command.session.status !== "starting" || command.session.providerInstanceId !== command.modelSelection.instanceId || command.session.runtimeMode !== command.runtimeMode || command.session.sessionIncarnationId === undefined || diff --git a/apps/server/src/provider/Drivers/ClaudeDriver.ts b/apps/server/src/provider/Drivers/ClaudeDriver.ts index cbcf626858..a5461c1e18 100644 --- a/apps/server/src/provider/Drivers/ClaudeDriver.ts +++ b/apps/server/src/provider/Drivers/ClaudeDriver.ts @@ -12,11 +12,7 @@ * * @module provider/Drivers/ClaudeDriver */ -import { - ClaudeSettings, - ProviderDriverKind, - type ServerProviderUsageLimits, -} from "@t3tools/contracts"; +import { ClaudeSettings, ProviderDriverKind } from "@t3tools/contracts"; import * as Cache from "effect/Cache"; import * as Duration from "effect/Duration"; import * as Crypto from "effect/Crypto"; @@ -29,7 +25,12 @@ import * as Schema from "effect/Schema"; import { HttpClient } from "effect/unstable/http"; import { ChildProcessSpawner } from "effect/unstable/process"; -import { retainUsageLimits } from "../providerUsageRetention.ts"; +import { + authenticatedUsageIdentity, + matchingAccountUsage, + retainUsageLimitsForAccount, + type AccountUsageReading, +} from "../providerUsageRetention.ts"; import { makeClaudeTextGeneration } from "../../textGeneration/ClaudeTextGeneration.ts"; import * as BackgroundPolicy from "../../background/BackgroundPolicy.ts"; import { ServerConfig } from "../../config.ts"; @@ -194,7 +195,7 @@ export const ClaudeDriver: ProviderDriver = { cwd, processEnv, ); - const lastKnownUsage = yield* Ref.make(undefined); + const lastKnownUsage = yield* Ref.make(undefined); const usageProbeCache = yield* Cache.makeWith( () => probeClaudeUsageLimits(effectiveConfig, processEnv, cwd).pipe( @@ -236,15 +237,17 @@ export const ClaudeDriver: ProviderDriver = { usageProbeCache, `${capabilitiesCacheKey}:${accountIdentity ?? "unknown"}`, ).pipe( - Effect.map((result) => { - if (!result) return undefined; - return result.accountIdentity && - accountIdentity && - result.accountIdentity !== accountIdentity - ? undefined - : result.usageLimits; - }), - Effect.flatMap((usageLimits) => retainUsageLimits(lastKnownUsage, usageLimits)), + Effect.map((result) => matchingAccountUsage(accountIdentity, result)), + Effect.flatMap((usageLimits) => + retainUsageLimitsForAccount( + lastKnownUsage, + authenticatedUsageIdentity({ + status: "authenticated", + email: accountIdentity, + }), + usageLimits, + ), + ), ), resolveClaudeModelCatalog(manifest), ), @@ -333,6 +336,7 @@ export const ClaudeDriver: ProviderDriver = { accentColor, enabled, snapshot, + invalidateCaches: Cache.invalidateAll(capabilitiesProbeCache), snapshotForCwd, adapter, textGeneration, diff --git a/apps/server/src/provider/Drivers/CodexDriver.test.ts b/apps/server/src/provider/Drivers/CodexDriver.test.ts index 3020920d28..83bb4202e9 100644 --- a/apps/server/src/provider/Drivers/CodexDriver.test.ts +++ b/apps/server/src/provider/Drivers/CodexDriver.test.ts @@ -29,6 +29,7 @@ import { import { CodexDriver } from "./CodexDriver.ts"; import { writeFakeCli } from "../../testUtils/fakeCli.ts"; import { SHARED_USAGE_CACHE_DIR_ENV } from "../sharedUsageReadCache.ts"; +import { readSharedUsageEntry, sharedUsageReadKey } from "../sharedUsageReadCache.ts"; const encodeFixtureString = Schema.encodeEffect(Schema.fromJsonString(Schema.String)); @@ -61,32 +62,58 @@ const noSpawn = ChildProcessSpawner.make(() => ); it.layer(testLayer)("CodexDriver", (it) => { - for (const readFails of [false, true]) { - it.effect( - `redeems through the initialized CLI and ${readFails ? "preserves a confirmed result when refresh fails" : "reads fresh limits before returning"}`, - () => - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const directory = yield* fs.makeTempDirectoryScoped({ prefix: "pylon-reset-credit-" }); - const logPath = NodePath.join(directory, "requests.jsonl"); - const priorCache = process.env[SHARED_USAGE_CACHE_DIR_ENV]; - yield* Effect.acquireRelease( + for (const scenario of [ + { + name: "reads fresh limits before returning", + readFails: false, + switchBefore: false, + switchOnConsume: false, + }, + { + name: "preserves a confirmed result when refresh fails", + readFails: true, + switchBefore: false, + switchOnConsume: false, + }, + { + name: "rejects a changed live account before consuming", + readFails: false, + switchBefore: true, + switchOnConsume: false, + }, + { + name: "does not cache a result after the account changes mid-reset", + readFails: false, + switchBefore: false, + switchOnConsume: true, + }, + ]) { + it.effect(`redeems through the initialized CLI and ${scenario.name}`, () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "pylon-reset-credit-" }); + const logPath = NodePath.join(directory, "requests.jsonl"); + const accountPath = NodePath.join(directory, "account.txt"); + yield* fs.writeFileString(accountPath, "owner@example.test"); + const priorCache = process.env[SHARED_USAGE_CACHE_DIR_ENV]; + yield* Effect.acquireRelease( + Effect.sync(() => { + process.env[SHARED_USAGE_CACHE_DIR_ENV] = NodePath.join(directory, "usage-cache"); + }), + () => Effect.sync(() => { - process.env[SHARED_USAGE_CACHE_DIR_ENV] = NodePath.join(directory, "usage-cache"); + if (priorCache === undefined) delete process.env[SHARED_USAGE_CACHE_DIR_ENV]; + else process.env[SHARED_USAGE_CACHE_DIR_ENV] = priorCache; }), - () => - Effect.sync(() => { - if (priorCache === undefined) delete process.env[SHARED_USAGE_CACHE_DIR_ENV]; - else process.env[SHARED_USAGE_CACHE_DIR_ENV] = priorCache; - }), - ); - const quotedLogPath = yield* encodeFixtureString(logPath); - const quotedDirectory = yield* encodeFixtureString(directory); - const binaryPath = writeFakeCli({ - directory, - name: "codex", - source: ` - import { appendFileSync } from "node:fs"; + ); + const quotedLogPath = yield* encodeFixtureString(logPath); + const quotedDirectory = yield* encodeFixtureString(directory); + const quotedAccountPath = yield* encodeFixtureString(accountPath); + const binaryPath = writeFakeCli({ + directory, + name: "codex", + source: ` + import { appendFileSync, readFileSync, writeFileSync } from "node:fs"; import { createInterface } from "node:readline"; const lines = createInterface({ input: process.stdin }); lines.on("line", line => { @@ -95,9 +122,14 @@ it.layer(testLayer)("CodexDriver", (it) => { if (request.id === undefined) return; let result; if (request.method === "initialize") result = { userAgent: "codex/1.0", codexHome: ${quotedDirectory}, platformFamily: "unix", platformOs: "macos" }; - else if (request.method === "account/rateLimitResetCredit/consume") result = { outcome: "reset" }; + else if (request.method === "account/read") result = { account: { type: "chatgpt", email: readFileSync(${quotedAccountPath}, "utf8"), planType: "plus" }, requiresOpenaiAuth: false }; + else if (request.method === "skills/list" || request.method === "model/list") result = { data: [] }; + else if (request.method === "account/rateLimitResetCredit/consume") { + if (${scenario.switchOnConsume}) writeFileSync(${quotedAccountPath}, "other@example.test"); + result = { outcome: "reset" }; + } else if (request.method === "account/rateLimits/read") { - if (${readFails}) { + if (${scenario.readFails}) { process.stdout.write(JSON.stringify({ id: request.id, error: { code: -32000, message: "read failed" } }) + "\\n"); return; } @@ -106,28 +138,59 @@ it.layer(testLayer)("CodexDriver", (it) => { process.stdout.write(JSON.stringify({ id: request.id, result }) + "\\n"); }); `, - }); - const instance = yield* CodexDriver.create({ - instanceId: ProviderInstanceId.make("codex-reset-test"), - displayName: "Reset test", - enabled: false, - environment: [], - config: { ...CodexDriver.defaultConfig(), binaryPath, homePath: directory }, - }); - const result = yield* instance.consumeResetCredit!({ requestId: "stable-reset-attempt" }); - expect(result.outcome).toBe("reset"); - if (readFails) expect(result.warning).toContain("updated limits could not be read"); - else expect(result.warning).toBeUndefined(); - const requests = yield* fs.readFileString(logPath); - expect(requests).toContain('"method":"initialized"'); + }); + const instance = yield* CodexDriver.create({ + instanceId: ProviderInstanceId.make("codex-reset-test"), + displayName: "Reset test", + enabled: true, + environment: [], + config: { ...CodexDriver.defaultConfig(), binaryPath, homePath: directory }, + }); + yield* instance.snapshot.refresh; + const accountCacheKey = sharedUsageReadKey([ + "codex", + directory, + "email:owner@example.test", + ]); + const cacheBefore = yield* readSharedUsageEntry( + NodePath.join(directory, "usage-cache"), + accountCacheKey, + ); + if (scenario.switchBefore) yield* fs.writeFileString(accountPath, "other@example.test"); + const result = yield* instance.consumeResetCredit!({ + requestId: "stable-reset-attempt", + }).pipe(Effect.exit); + if (scenario.switchBefore) expect(result._tag).toBe("Failure"); + else { + expect(result._tag).toBe("Success"); + if (result._tag === "Success") { + expect(result.value.outcome).toBe("reset"); + if (scenario.readFails) + expect(result.value.warning).toContain("updated limits could not be read"); + else if (scenario.switchOnConsume) + expect(result.value.warning).toContain("account changed"); + else expect(result.value.warning).toBeUndefined(); + } + } + const requests = yield* fs.readFileString(logPath); + expect(requests).toContain('"method":"initialized"'); + if (scenario.switchBefore) { + expect(requests).not.toContain('"method":"account/rateLimitResetCredit/consume"'); + } else { expect(requests).toContain('"idempotencyKey":"stable-reset-attempt"'); expect(requests.indexOf('"method":"initialize"')).toBeLessThan( requests.indexOf('"method":"account/rateLimitResetCredit/consume"'), ); expect(requests.indexOf('"method":"account/rateLimitResetCredit/consume"')).toBeLessThan( - requests.indexOf('"method":"account/rateLimits/read"'), + requests.lastIndexOf('"method":"account/rateLimits/read"'), ); - }).pipe(Effect.scoped), + } + if (scenario.switchBefore || scenario.switchOnConsume) { + expect( + yield* readSharedUsageEntry(NodePath.join(directory, "usage-cache"), accountCacheKey), + ).toEqual(cacheBefore); + } + }).pipe(Effect.scoped), ); } diff --git a/apps/server/src/provider/Drivers/CodexDriver.ts b/apps/server/src/provider/Drivers/CodexDriver.ts index a258ac4c06..873f2d9047 100644 --- a/apps/server/src/provider/Drivers/CodexDriver.ts +++ b/apps/server/src/provider/Drivers/CodexDriver.ts @@ -21,11 +21,7 @@ * * @module provider/Drivers/CodexDriver */ -import { - CodexSettings, - ProviderDriverKind, - type ServerProviderUsageLimits, -} from "@t3tools/contracts"; +import { CodexSettings, ProviderDriverKind } from "@t3tools/contracts"; import * as DateTime from "effect/DateTime"; import * as Crypto from "effect/Crypto"; import * as Effect from "effect/Effect"; @@ -44,11 +40,13 @@ import { ServerSettingsService } from "../../serverSettings.ts"; import { ProviderDriverError } from "../Errors.ts"; import { makeCodexAdapter } from "../Layers/CodexAdapter.ts"; import { + accountProbeStatus, checkCodexProviderStatus, makePendingCodexProvider, probeCodexSkillsForCwd, withCodexAppServerClient, } from "../Layers/CodexProvider.ts"; +import { readCodexAccountId } from "../codexAccountIdentity.ts"; import { CodexResetCreditCoordinator, CODEX_RESET_CREDIT_TIMEOUT, @@ -78,7 +76,13 @@ import { makeProviderSnapshotSettingsSource, type ProviderSnapshotSettings, } from "../providerUpdateSettings.ts"; -import { retainSnapshotUsageLimits, preferFreshUsageReading } from "../providerUsageRetention.ts"; +import { + authenticatedUsageIdentity, + retainSnapshotUsageLimits, + preferFreshUsageReading, + usageReadingForAuth, + type AccountUsageReading, +} from "../providerUsageRetention.ts"; import { codexContinuationIdentity, materializeCodexShadowHome, @@ -211,8 +215,8 @@ export const CodexDriver: ProviderDriver = { // Subscription capacity rides the same probe as install/auth status but // is read over the network, so it fails independently. Keep the last // good reading through those failures rather than blanking the gauge. - const lastKnownUsage = yield* Ref.make(undefined); - const redeemedUsage = yield* Ref.make(undefined); + const lastKnownUsage = yield* Ref.make(undefined); + const redeemedUsage = yield* Ref.make(undefined); const checkProvider = modelManifest.refreshInBackground.pipe( Effect.andThen( Effect.zipWith( @@ -221,7 +225,11 @@ export const CodexDriver: ProviderDriver = { Effect.gen(function* () { const resetReading = yield* Ref.get(redeemedUsage); const nowMs = yield* Effect.clockWith((clock) => clock.currentTimeMillis); - const published = preferFreshUsageReading(snapshot, resetReading, nowMs); + const published = preferFreshUsageReading( + snapshot, + usageReadingForAuth(resetReading, snapshot.auth), + nowMs, + ); return yield* retainSnapshotUsageLimits(lastKnownUsage, published); }), ), @@ -307,77 +315,148 @@ export const CodexDriver: ProviderDriver = { const accountKey = homeLayout.effectiveHomePath ?? homeLayout.sharedHomePath; const consumeResetCredit: NonNullable = (input) => - resetCreditCoordinator - .redeem(accountKey, input.requestId, (idempotencyKey) => - Effect.gen(function* () { - const result = yield* Effect.gen(function* () { - const { client, outcome } = yield* Effect.gen(function* () { - const { client } = yield* withCodexAppServerClient({ - binaryPath: effectiveConfig.binaryPath, - homePath: effectiveConfig.homePath, - launchArgs: resolveCodexLaunchArgs(effectiveConfig.launchArgs, processEnv), - cwd: process.cwd(), - environment: processEnv, - }); - const { outcome } = yield* client.request( - "account/rateLimitResetCredit/consume", - { idempotencyKey }, - ); - return { client, outcome }; - }).pipe(Effect.timeout(CODEX_RESET_CREDIT_TIMEOUT)); - const reading = yield* client - .request("account/rateLimits/read", undefined) - .pipe(Effect.timeout("5 seconds"), Effect.result); - if (reading._tag === "Failure") - return { - outcome, - warning: - "Codex answered the reset request, but its updated limits could not be read. Refresh to check.", - }; - const checkedAt = DateTime.formatIso(yield* DateTime.now); - const limits = usageLimitsFromCodexRateLimits(reading.success, checkedAt); - if (!limits) - return { - outcome, - warning: - "Codex answered the reset request without reporting updated limits. Refresh to check.", - }; - yield* Ref.set(redeemedUsage, limits); - yield* writeSharedUsageEntry( - yield* resolveSharedUsageCacheDir, - sharedUsageReadKey(["codex", accountKey]), - { version: 1, readAt: checkedAt, usageLimits: limits }, + Effect.gen(function* () { + const selectedIdentity = authenticatedUsageIdentity((yield* snapshot.getSnapshot).auth); + if (!selectedIdentity) { + return yield* Effect.fail( + new ProviderDriverError({ + driver: DRIVER_KIND, + instanceId, + detail: "Codex account identity is not verified.", + }), + ); + } + // A pending or completed attempt belongs to one account, even if the + // same CLI home signs in as another account before a retry. + const attemptKey = sharedUsageReadKey(["codex-reset", accountKey, selectedIdentity]); + return yield* resetCreditCoordinator.redeem( + attemptKey, + input.requestId, + (idempotencyKey) => + Effect.gen(function* () { + const identityBefore = authenticatedUsageIdentity( + (yield* snapshot.getSnapshot).auth, ); - return { outcome }; - }).pipe(Effect.scoped); - // The direct account read bypasses the shared cache. Even if an older - // status probe finishes now, checkProvider overlays this fresh reading. - const refreshed = yield* snapshot.refresh.pipe(Effect.exit); - return refreshed._tag === "Failure" && !result.warning - ? { - ...result, - warning: - "Codex answered the reset request, but the provider status could not refresh. Refresh to check.", + if (identityBefore !== selectedIdentity) { + return yield* Effect.fail( + new ProviderDriverError({ + driver: DRIVER_KIND, + instanceId, + detail: "Codex account changed before the reset credit could be used.", + }), + ); + } + const result = yield* Effect.gen(function* () { + const { client, sharedHomePath, outcome } = yield* Effect.gen(function* () { + const { client, sharedHomePath } = yield* withCodexAppServerClient({ + binaryPath: effectiveConfig.binaryPath, + homePath: effectiveConfig.homePath, + launchArgs: resolveCodexLaunchArgs(effectiveConfig.launchArgs, processEnv), + cwd: process.cwd(), + environment: processEnv, + }); + const liveAccount = yield* client.request("account/read", {}); + const liveIdentity = authenticatedUsageIdentity( + accountProbeStatus(liveAccount, yield* readCodexAccountId(sharedHomePath)) + .auth, + ); + if (!identityBefore || liveIdentity !== identityBefore) { + return yield* Effect.fail( + new ProviderDriverError({ + driver: DRIVER_KIND, + instanceId, + detail: "Codex account changed before the reset credit could be used.", + }), + ); + } + const { outcome } = yield* client.request( + "account/rateLimitResetCredit/consume", + { idempotencyKey }, + ); + return { client, sharedHomePath, outcome }; + }).pipe(Effect.timeout(CODEX_RESET_CREDIT_TIMEOUT)); + const reading = yield* client + .request("account/rateLimits/read", undefined) + .pipe(Effect.timeout("5 seconds"), Effect.result); + if (reading._tag === "Failure") + return { + outcome, + warning: + "Codex answered the reset request, but its updated limits could not be read. Refresh to check.", + }; + const checkedAt = DateTime.formatIso(yield* DateTime.now); + const limits = usageLimitsFromCodexRateLimits(reading.success, checkedAt); + if (!limits) + return { + outcome, + warning: + "Codex answered the reset request without reporting updated limits. Refresh to check.", + }; + const accountAfter = yield* client + .request("account/read", {}) + .pipe(Effect.option); + const liveIdentityAfter = + accountAfter._tag === "Some" + ? authenticatedUsageIdentity( + accountProbeStatus( + accountAfter.value, + yield* readCodexAccountId(sharedHomePath), + ).auth, + ) + : undefined; + const snapshotIdentityAfter = authenticatedUsageIdentity( + (yield* snapshot.getSnapshot).auth, + ); + if ( + identityBefore && + liveIdentityAfter === identityBefore && + snapshotIdentityAfter === identityBefore + ) { + yield* Ref.set(redeemedUsage, { + identity: identityBefore, + usageLimits: limits, + }); + yield* writeSharedUsageEntry( + yield* resolveSharedUsageCacheDir, + sharedUsageReadKey(["codex", accountKey, identityBefore]), + { version: 1, readAt: checkedAt, usageLimits: limits }, + ); + } else { + return { + outcome, + warning: + "Codex answered the reset request, but the account changed before its limits could be verified. Refresh to check.", + }; } - : result; - }).pipe( - Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), - Effect.provideService(FileSystem.FileSystem, fileSystem), - Effect.provideService(Path.Path, path), - ), - ) - .pipe( - Effect.mapError( - (cause) => - new ProviderDriverError({ - driver: DRIVER_KIND, - instanceId, - detail: - "Codex could not redeem the reset credit. Retry to check the same attempt.", - cause, - }), - ), + return { outcome }; + }).pipe(Effect.scoped); + // The direct account read bypasses the shared cache. Even if an older + // status probe finishes now, checkProvider overlays this fresh reading. + const refreshed = yield* snapshot.refresh.pipe(Effect.exit); + return refreshed._tag === "Failure" && !result.warning + ? { + ...result, + warning: + "Codex answered the reset request, but the provider status could not refresh. Refresh to check.", + } + : result; + }).pipe( + Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), + Effect.provideService(FileSystem.FileSystem, fileSystem), + Effect.provideService(Path.Path, path), + ), ); + }).pipe( + Effect.mapError( + (cause) => + new ProviderDriverError({ + driver: DRIVER_KIND, + instanceId, + detail: "Codex could not redeem the reset credit. Retry to check the same attempt.", + cause, + }), + ), + ); return { instanceId, diff --git a/apps/server/src/provider/Drivers/CursorDriver.ts b/apps/server/src/provider/Drivers/CursorDriver.ts index 5466af802e..8715dd9835 100644 --- a/apps/server/src/provider/Drivers/CursorDriver.ts +++ b/apps/server/src/provider/Drivers/CursorDriver.ts @@ -137,11 +137,11 @@ export const CursorDriver: ProviderDriver = { }); const textGeneration = yield* makeCursorTextGeneration(effectiveConfig, processEnv); - const discoverModels = yield* makeCursorModelDiscovery(effectiveConfig, processEnv); + const modelDiscovery = yield* makeCursorModelDiscovery(effectiveConfig, processEnv); const checkProvider = checkCursorProviderStatus( effectiveConfig, processEnv, - discoverModels, + modelDiscovery.discover, ).pipe( Effect.map(stampIdentity), Effect.provideService(Crypto.Crypto, crypto), @@ -196,6 +196,7 @@ export const CursorDriver: ProviderDriver = { accentColor, enabled, snapshot, + invalidateCaches: modelDiscovery.invalidate, snapshotForCwd: (cwd) => !effectiveConfig.enabled ? snapshot.getSnapshot diff --git a/apps/server/src/provider/Layers/ClaudeAdapter.test.ts b/apps/server/src/provider/Layers/ClaudeAdapter.test.ts index 028322f50c..4565e8dccd 100644 --- a/apps/server/src/provider/Layers/ClaudeAdapter.test.ts +++ b/apps/server/src/provider/Layers/ClaudeAdapter.test.ts @@ -10179,4 +10179,70 @@ describe("ClaudeAdapterLive", () => { Effect.provide(harness.layer), ); }); + + it.effect("attributes live subagent tool heartbeats when Claude omits task_id", () => { + const harness = makeHarness(); + return Effect.gen(function* () { + const adapter = yield* ClaudeAdapter; + const progressFiber = yield* adapter.streamEvents.pipe( + Stream.filter((event) => event.type === "tool.progress"), + Stream.take(4), + Stream.runCollect, + Effect.forkChild, + ); + const session = yield* adapter.startSession({ + threadId: THREAD_ID, + provider: ProviderDriverKind.make("claudeAgent"), + runtimeMode: "full-access", + }); + yield* adapter.sendTurn({ threadId: session.threadId, input: "Delegate work" }); + + harness.query.emit({ + type: "system", + subtype: "task_started", + task_id: "task-heartbeat", + description: "Inspect code", + task_type: "local_agent", + tool_use_id: "tool-task-heartbeat", + uuid: "task-heartbeat-start", + session_id: "sdk-heartbeat", + } as unknown as SDKMessage); + const heartbeat = (uuid: string, parentToolUseId: string, taskId?: string) => + harness.query.emit({ + type: "tool_progress", + tool_use_id: `tool-${uuid}`, + tool_name: "Bash", + elapsed_time_seconds: 30, + parent_tool_use_id: parentToolUseId, + ...(taskId ? { task_id: taskId } : {}), + uuid, + session_id: "sdk-heartbeat", + } as unknown as SDKMessage); + heartbeat("live", "tool-task-heartbeat"); + heartbeat("explicit", "tool-task-heartbeat", "task-explicit"); + heartbeat("unknown", "tool-unknown"); + harness.query.emit({ + type: "system", + subtype: "task_notification", + task_id: "task-heartbeat", + status: "completed", + uuid: "task-heartbeat-end", + session_id: "sdk-heartbeat", + } as unknown as SDKMessage); + heartbeat("late", "tool-task-heartbeat"); + + const progress = Array.from(yield* Fiber.join(progressFiber)); + assert.deepEqual( + progress.map((event) => + event.type === "tool.progress" && event.payload.taskId + ? String(event.payload.taskId) + : undefined, + ), + ["task-heartbeat", "task-explicit", undefined, undefined], + ); + }).pipe( + Effect.provideService(Random.Random, makeDeterministicRandomService()), + Effect.provide(harness.layer), + ); + }); }); diff --git a/apps/server/src/provider/Layers/ClaudeAdapter.ts b/apps/server/src/provider/Layers/ClaudeAdapter.ts index 3f74b6b855..e45474b80a 100644 --- a/apps/server/src/provider/Layers/ClaudeAdapter.ts +++ b/apps/server/src/provider/Layers/ClaudeAdapter.ts @@ -4346,6 +4346,13 @@ export const makeClaudeAdapter = Effect.fn("makeClaudeAdapter")(function* ( }; if (message.type === "tool_progress") { + // Claude omits task_id on subagent tool heartbeats but includes the + // launching Task tool's id. Recover only a still-live agent: a late + // heartbeat for a completed task must not revive its activity row. + const owningTaskId = agentIdForParentToolUse(context.taskAgents, message.parent_tool_use_id); + const taskId = + message.task_id ?? + (owningTaskId && context.liveTaskIds.has(owningTaskId) ? owningTaskId : undefined); yield* offerRuntimeEvent(context.sessionIncarnationId, { ...base, type: "tool.progress", @@ -4353,7 +4360,7 @@ export const makeClaudeAdapter = Effect.fn("makeClaudeAdapter")(function* ( toolUseId: message.tool_use_id, toolName: message.tool_name, elapsedSeconds: message.elapsed_time_seconds, - ...(message.task_id ? { taskId: RuntimeTaskId.make(message.task_id) } : {}), + ...(taskId ? { taskId: RuntimeTaskId.make(taskId) } : {}), ...(message.parent_tool_use_id !== null ? { parentToolUseId: message.parent_tool_use_id } : {}), diff --git a/apps/server/src/provider/Layers/CodexProvider.test.ts b/apps/server/src/provider/Layers/CodexProvider.test.ts index 94cd8ba582..fe319f45a9 100644 --- a/apps/server/src/provider/Layers/CodexProvider.test.ts +++ b/apps/server/src/provider/Layers/CodexProvider.test.ts @@ -240,6 +240,8 @@ it("labels non-ChatGPT account types", () => { }); const CODEX_HOME = "/home/someone/.codex"; +const ACCOUNT_A = "email:a@example.com"; +const ACCOUNT_B = "email:b@example.com"; const usageAt = (checkedAt: string): ServerProviderUsageLimits => ({ source: "codexAppServer", @@ -272,7 +274,7 @@ const readAfterMissedRead = (input: { readonly age: Duration.Duration }) => else process.env[SHARED_USAGE_CACHE_DIR_ENV] = priorCache; }), ); - const cacheKey = sharedUsageReadKey(["codex", CODEX_HOME]); + const cacheKey = sharedUsageReadKey(["codex", CODEX_HOME, ACCOUNT_A]); const readAt = DateTime.formatIso(DateTime.makeUnsafe(0)); yield* writeSharedUsageEntry(cacheDir, cacheKey, { version: 1, @@ -283,6 +285,7 @@ const readAfterMissedRead = (input: { readonly age: Duration.Duration }) => yield* TestClock.adjust(input.age); const result = yield* readCodexRateLimitsShared({ sharedHomePath: CODEX_HOME, + accountIdentity: ACCOUNT_A, read: MISSED_READ, }); return { result, entry: yield* readSharedUsageEntry(cacheDir, cacheKey), readAt }; @@ -311,4 +314,34 @@ it.layer(NodeServices.layer)("codex capacity across a missed read", (it) => { assert.deepStrictEqual(result, {}); }), ); + + it.effect("does not read account A's shared quota after the same home signs into B", () => + Effect.gen(function* () { + const fileSystem = yield* FileSystem.FileSystem; + const cacheDir = yield* fileSystem.makeTempDirectoryScoped({ prefix: "pylon-codex-usage-" }); + const priorCache = process.env[SHARED_USAGE_CACHE_DIR_ENV]; + yield* Effect.acquireRelease( + Effect.sync(() => { + process.env[SHARED_USAGE_CACHE_DIR_ENV] = cacheDir; + }), + () => + Effect.sync(() => { + if (priorCache === undefined) delete process.env[SHARED_USAGE_CACHE_DIR_ENV]; + else process.env[SHARED_USAGE_CACHE_DIR_ENV] = priorCache; + }), + ); + const readAt = DateTime.formatIso(yield* DateTime.now); + yield* writeSharedUsageEntry(cacheDir, sharedUsageReadKey(["codex", CODEX_HOME, ACCOUNT_A]), { + version: 1, + readAt, + usageLimits: usageAt(readAt), + }); + const result = yield* readCodexRateLimitsShared({ + sharedHomePath: CODEX_HOME, + accountIdentity: ACCOUNT_B, + read: MISSED_READ, + }); + assert.deepStrictEqual(result, {}); + }).pipe(Effect.scoped), + ); }); diff --git a/apps/server/src/provider/Layers/CodexProvider.ts b/apps/server/src/provider/Layers/CodexProvider.ts index 871578cd02..b52b6fe3b2 100644 --- a/apps/server/src/provider/Layers/CodexProvider.ts +++ b/apps/server/src/provider/Layers/CodexProvider.ts @@ -55,7 +55,7 @@ import { sharedUsageReadKey, writeSharedUsageEntry, } from "../sharedUsageReadCache.ts"; -import { isRetainedUsageFresh } from "../providerUsageRetention.ts"; +import { authenticatedUsageIdentity, isRetainedUsageFresh } from "../providerUsageRetention.ts"; import packageJson from "../../../package.json" with { type: "json" }; const isCodexAppServerSpawnError = Schema.is(CodexErrors.CodexAppServerSpawnError); @@ -402,14 +402,22 @@ const failedRead = Effect.fn("readCodexRateLimitsShared.failed")(function* ( */ export const readCodexRateLimitsShared = Effect.fn("readCodexRateLimitsShared")(function* (input: { readonly sharedHomePath: string; + readonly accountIdentity: string | undefined; readonly read: Effect.Effect; }): Effect.fn.Return< Pick, never, FileSystem.FileSystem | Path.Path > { + // A home can sign out and into another account without changing path. An + // unidentified account may make a live read, but must never reuse the old + // home's shared quota file. + if (!input.accountIdentity) { + const rateLimits = yield* input.read; + return rateLimits ? { rateLimits } : {}; + } const cacheDir = yield* resolveSharedUsageCacheDir; - const cacheKey = sharedUsageReadKey(["codex", input.sharedHomePath]); + const cacheKey = sharedUsageReadKey(["codex", input.sharedHomePath, input.accountIdentity]); const nowMs = yield* Effect.clockWith((clock) => clock.currentTimeMillis); const shared = decideSharedUsageRead(yield* readSharedUsageEntry(cacheDir, cacheKey), nowMs); if (shared.kind === "fresh") return { sharedUsageLimits: shared.usageLimits }; @@ -526,11 +534,14 @@ const probeCodexAppServerProvider = Effect.fn("probeCodexAppServerProvider")(fun } satisfies CodexAppServerProviderSnapshot; } - // The account id is read alongside the app-server requests rather than - // before the spawn: the probe's timeout is measured from its start, and a - // filesystem read ahead of the spawn would let it expire with nothing to - // close. - const [skillsResponse, models, rateLimitsRead, accountId] = yield* Effect.all( + // Read the account id after the app-server starts, then bind the shared + // quota file to this account before looking it up. A home path alone is + // reused after sign-out and cannot identify a subscription. + const accountId = yield* readCodexAccountId(sharedHomePath); + const accountIdentity = authenticatedUsageIdentity( + accountProbeStatus(accountResponse, accountId).auth, + ); + const [skillsResponse, models, rateLimitsRead] = yield* Effect.all( [ client.request("skills/list", { cwds: [input.cwd], @@ -539,6 +550,7 @@ const probeCodexAppServerProvider = Effect.fn("probeCodexAppServerProvider")(fun accountResponse.account?.type === "chatgpt" ? readCodexRateLimitsShared({ sharedHomePath, + accountIdentity, read: client .request("account/rateLimits/read", undefined) .pipe( @@ -551,7 +563,6 @@ const probeCodexAppServerProvider = Effect.fn("probeCodexAppServerProvider")(fun : Effect.succeed>( {}, ), - readCodexAccountId(sharedHomePath), ], { concurrency: "unbounded" }, ); @@ -659,7 +670,7 @@ const makePendingCodexProvider = ( }); }); -function accountProbeStatus( +export function accountProbeStatus( account: CodexAppServerProviderSnapshot["account"], accountId: string | undefined, ): { diff --git a/apps/server/src/provider/Layers/CursorProvider.test.ts b/apps/server/src/provider/Layers/CursorProvider.test.ts index 7337a19010..dd314b0ec9 100644 --- a/apps/server/src/provider/Layers/CursorProvider.test.ts +++ b/apps/server/src/provider/Layers/CursorProvider.test.ts @@ -626,7 +626,7 @@ describe("discoverCursorModelsViaAcp", () => { apiEndpoint: "", customModels: [], }; - const discover = yield* makeCursorModelDiscovery(settings, { + const { discover, invalidate } = yield* makeCursorModelDiscovery(settings, { ...process.env, T3_ACP_REQUEST_LOG_PATH: requestLogPath, }); @@ -639,6 +639,10 @@ describe("discoverCursorModelsViaAcp", () => { yield* fileSystem.writeFileString(requestLogPath, ""); expect(yield* discover(about)).toEqual(first); expect(yield* fileSystem.readFileString(requestLogPath)).toBe(""); + yield* invalidate; + expect(yield* discover(about)).toEqual(first); + expect(yield* fileSystem.readFileString(requestLogPath)).toContain("initialize"); + yield* fileSystem.writeFileString(requestLogPath, ""); yield* discover({ ...about, version: "2026.08.12" }); expect(yield* fileSystem.readFileString(requestLogPath)).toContain("initialize"); yield* fileSystem.writeFileString(requestLogPath, ""); diff --git a/apps/server/src/provider/Layers/CursorProvider.ts b/apps/server/src/provider/Layers/CursorProvider.ts index 1d80fa20b8..e46989ab62 100644 --- a/apps/server/src/provider/Layers/CursorProvider.ts +++ b/apps/server/src/provider/Layers/CursorProvider.ts @@ -588,8 +588,11 @@ export const makeCursorModelDiscovery = Effect.fn("makeCursorModelDiscovery")(fu Exit.isSuccess(exit) && exit.value.length > 0 ? Duration.minutes(30) : Duration.zero, }, ); - return (about: Pick) => - Cache.get(cache, JSON.stringify([about.version, about.auth])); + return { + discover: (about: Pick) => + Cache.get(cache, JSON.stringify([about.version, about.auth])), + invalidate: Cache.invalidateAll(cache), + }; }); function getCursorFallbackModels( diff --git a/apps/server/src/provider/Layers/GrokAdapter.test.ts b/apps/server/src/provider/Layers/GrokAdapter.test.ts index b14edfcc0b..e38de31156 100644 --- a/apps/server/src/provider/Layers/GrokAdapter.test.ts +++ b/apps/server/src/provider/Layers/GrokAdapter.test.ts @@ -25,8 +25,10 @@ import { TurnId, type ProviderRuntimeEvent, } from "@t3tools/contracts"; +import { HostProcessPlatform } from "@t3tools/shared/hostProcess"; import { ServerConfig } from "../../config.ts"; +import { execScriptSource, writeFakeCli } from "../../testUtils/fakeCli.ts"; import { grokPromptSettlementBelongsToContext, isGrokEnterPlanModeToolCall, @@ -34,8 +36,7 @@ import { nextGrokPlanModeActive, selectGrokPermissionOptionId, } from "./GrokAdapter.ts"; -import { execScriptSource, writeFakeCli } from "../../testUtils/fakeCli.ts"; -import { HostProcessPlatform } from "@t3tools/shared/hostProcess"; + const decodeGrokSettings = Schema.decodeSync(GrokSettings); const __dirname = NodePath.dirname(NodeURL.fileURLToPath(import.meta.url)); @@ -251,6 +252,65 @@ it.layer(grokAdapterTestLayer)("GrokAdapterLive", (it) => { }), ); + for (const taskType of ["monitor", "shell"] as const) { + it.effect(`emits the ${taskType} background lifecycle`, () => + Effect.gen(function* () { + const threadId = ThreadId.make(`grok-background-${taskType}`); + const wrapperPath = yield* Effect.promise(() => + makeMockGrokWrapper({ + [taskType === "monitor" + ? "T3_ACP_EMIT_GROK_MONITOR_POST_TURN_POLL" + : "T3_ACP_EMIT_GROK_BACKGROUND_TASK_STARTED"]: "1", + }), + ); + const adapter = yield* makeTestAdapter(wrapperPath); + const events: ProviderRuntimeEvent[] = []; + const finished = yield* Deferred.make(); + const eventsFiber = yield* Stream.runForEach(adapter.streamEvents, (event) => + Effect.gen(function* () { + events.push(event); + if (event.type === (taskType === "monitor" ? "task.completed" : "turn.completed")) { + yield* Deferred.succeed(finished, undefined); + } + }), + ).pipe(Effect.forkChild); + yield* adapter.startSession({ threadId, cwd: process.cwd(), runtimeMode: "full-access" }); + yield* adapter.sendTurn({ threadId, input: "watch the unit" }); + yield* Deferred.await(finished).pipe(Effect.timeout("3 seconds")); + + const started = events.find((event) => event.type === "task.started"); + const taskId = + taskType === "monitor" + ? "01a05f41-5107-7550-821e-79e8d1cd7687" + : "call-fb9d0000-0000-0000-0000-000000000026"; + assert.equal(started?.payload.taskType, taskType); + assert.equal(started?.payload.taskId, taskId); + if (taskType === "monitor") { + const completed = events.find((event) => event.type === "task.completed"); + const turnEnd = events.findIndex((event) => event.type === "turn.completed"); + assert.equal(completed?.payload.status, "completed"); + assert.equal(completed?.payload.taskId, taskId); + assert.equal(completed?.turnId, undefined); + assert.isAtLeast(turnEnd, 0); + assert.isAbove( + events.findIndex((event) => event.type === "task.completed"), + turnEnd, + ); + assert.deepEqual( + events + .slice(turnEnd + 1) + .filter((event) => event.type === "item.updated" || event.type === "item.completed"), + [], + ); + } else { + assert.equal(started?.payload.description, "sleep 40; echo done-a"); + } + yield* Fiber.interrupt(eventsFiber); + yield* adapter.stopSession(threadId); + }).pipe(TestClock.withLive), + ); + } + it.effect("sends runtime context with the current model without changing saved prompts", () => Effect.gen(function* () { const threadId = ThreadId.make("grok-runtime-context"); diff --git a/apps/server/src/provider/Layers/GrokAdapter.ts b/apps/server/src/provider/Layers/GrokAdapter.ts index 1f0053da34..2a92dba3d3 100644 --- a/apps/server/src/provider/Layers/GrokAdapter.ts +++ b/apps/server/src/provider/Layers/GrokAdapter.ts @@ -68,6 +68,10 @@ import { normalizeGrokReasoningEffort, resolveGrokAcpBaseModelId, } from "../acp/GrokAcpSupport.ts"; +import { + buildGrokBackgroundTaskEvents, + type GrokBackgroundTaskRecord, +} from "../acp/XAiBackgroundTasks.ts"; import { extractGrokPlanMarkdownFromToolCallData, extractXAiAskUserQuestions, @@ -174,6 +178,10 @@ interface GrokSessionContext { currentModelId: string | undefined; currentReasoningEffort: string | undefined; stopped: boolean; + /** Live monitor and shell identities, with their originating turns. */ + readonly backgroundTasks: Map; + readonly completedBackgroundTaskIds: Map; + readonly ambiguousBackgroundTaskIds: Set; } function settlePendingApprovalsAsCancelled( @@ -1337,6 +1345,9 @@ export function makeGrokAdapter(grokSettings: GrokSettings, options?: GrokAdapte ? normalizeGrokReasoningEffort(requestedStartReasoningEffort) : currentStartReasoningEffort, stopped: false, + backgroundTasks: new Map(), + completedBackgroundTaskIds: new Map(), + ambiguousBackgroundTaskIds: new Set(), }; const nf = yield* Stream.runDrain( @@ -1359,6 +1370,30 @@ export function makeGrokAdapter(grokSettings: GrokSettings, options?: GrokAdapte } const notificationTurnId = resolveNotificationTurnId(ctx); + if ( + event._tag === "ToolCallUpdated" && + !ctx.stopped && + (notificationTurnId === undefined || + !ctx.interruptedTurnIds.has(notificationTurnId)) + ) { + for (const taskEvent of buildGrokBackgroundTaskEvents({ + tasks: ctx.backgroundTasks, + completedTaskIds: ctx.completedBackgroundTaskIds, + ambiguousTaskIds: ctx.ambiguousBackgroundTaskIds, + toolCallId: event.toolCall.toolCallId, + rawInput: event.toolCall.data.rawInput, + rawOutput: event.toolCall.data.rawOutput, + toolCallStatus: event.toolCall.status, + turnId: notificationTurnId, + })) { + yield* offerRuntimeEvent(ctx.sessionIncarnationId, { + ...taskEvent, + ...(yield* makeEventStamp()), + provider: PROVIDER, + threadId: ctx.threadId, + }); + } + } if ( notificationTurnId === undefined || ctx.interruptedTurnIds.has(notificationTurnId) diff --git a/apps/server/src/provider/Layers/ProviderService.test.ts b/apps/server/src/provider/Layers/ProviderService.test.ts index 1e228af47d..a961d5ab50 100644 --- a/apps/server/src/provider/Layers/ProviderService.test.ts +++ b/apps/server/src/provider/Layers/ProviderService.test.ts @@ -57,6 +57,7 @@ import * as Option from "effect/Option"; import * as PubSub from "effect/PubSub"; import * as Ref from "effect/Ref"; import * as Result from "effect/Result"; +import * as Scheduler from "effect/Scheduler"; import * as Schema from "effect/Schema"; import * as Scope from "effect/Scope"; import * as Stream from "effect/Stream"; @@ -6917,6 +6918,34 @@ class RuntimeReaper extends Context.Service()( describe("agent browser access", () => { const projectId = ProjectId.make("project-browser-access"); + it.effect("subscribes to adapter events before exposing the provider service", () => + Effect.gen(function* () { + const codex = makeFakeCodexAdapter(); + const events = yield* PubSub.unbounded(); + let subscribed = false; + const adapter: ProviderAdapterShape = { + ...codex.adapter, + streamEvents: Stream.unwrap( + Effect.map(PubSub.subscribe(events), (subscription) => { + // This runs only after PubSub has registered the subscription, not merely when + // the stream wrapper is constructed. + subscribed = true; + return Stream.fromSubscription(subscription); + }), + ), + }; + yield* Effect.gen(function* () { + yield* ProviderService.ProviderService; + assert.isTrue(subscribed); + }).pipe(Effect.provide(makeAgentBrowserProviderLayer(false, { ...codex, adapter }, {}))); + }).pipe( + // Keep this one fiber from yielding before the assertion. The old deferred fork then + // reproducibly leaves `subscribed` false; immediate startup passes without a timer. + Effect.provideService(Scheduler.PreventSchedulerYield, true), + Effect.provide(NodeServices.layer), + ), + ); + const makeBrowserAccessProjectionLayer = ( threadId: ThreadId, status: "found" | "missing" | "failed" = "found", diff --git a/apps/server/src/provider/Layers/ProviderService.ts b/apps/server/src/provider/Layers/ProviderService.ts index 2fc810c8a3..e999423390 100644 --- a/apps/server/src/provider/Layers/ProviderService.ts +++ b/apps/server/src/provider/Layers/ProviderService.ts @@ -1758,6 +1758,8 @@ const makeProviderService = Effect.fn("makeProviderService")(function* ( // Publish the new adapter set before any replacement stream can emit. yield* Ref.set(subscribedAdapters, next); for (const [id, adapter] of newSubscriptions) { + // A provider can emit a receipt synchronously during an API call. Subscribe before + // returning from reconciliation so that first receipt cannot outrun a deferred fork. yield* Stream.runForEach(adapter.streamEvents, (event) => processRuntimeEvent( { @@ -1767,7 +1769,7 @@ const makeProviderService = Effect.fn("makeProviderService")(function* ( }, event, ), - ).pipe(Effect.forkScoped); + ).pipe(Effect.forkScoped({ startImmediately: true })); } }); diff --git a/apps/server/src/provider/ModelManifest.test.ts b/apps/server/src/provider/ModelManifest.test.ts index f779280e6a..8f8e6b4d56 100644 --- a/apps/server/src/provider/ModelManifest.test.ts +++ b/apps/server/src/provider/ModelManifest.test.ts @@ -363,6 +363,84 @@ describe("ModelManifest service", () => { ); }); + it.live("explicit refresh bypasses fresh memory and disk caches", () => { + let fetchCount = 0; + const updated: ModelManifestData = { + ...REMOTE_MANIFEST, + currentModels: { codex: ["gpt-reloaded"] }, + }; + return Effect.gen(function* () { + const service = yield* make; + assert.deepStrictEqual(yield* service.refresh, REMOTE_MANIFEST); + assert.deepStrictEqual(yield* service.refresh, REMOTE_MANIFEST); + assert.strictEqual(fetchCount, 1); + + const rebooted = yield* make; + assert.deepStrictEqual(yield* rebooted.refresh, REMOTE_MANIFEST); + assert.strictEqual(fetchCount, 1); + assert.deepStrictEqual(yield* rebooted.forceRefresh, updated); + assert.strictEqual(fetchCount, 2); + assert.deepStrictEqual(yield* rebooted.current, updated); + assert.deepStrictEqual(yield* (yield* make).current, updated); + }).pipe( + Effect.scoped, + Effect.provide( + serviceLayers({ + prefix: "model-manifest-force-refresh-test", + response: () => Response.json(fetchCount++ === 0 ? REMOTE_MANIFEST : updated), + }), + ), + ); + }); + + it.live("explicit refresh retries immediately after failure and preserves last-good data", () => { + let fetchCount = 0; + return Effect.gen(function* () { + const service = yield* make; + assert.deepStrictEqual(yield* service.refresh, REMOTE_MANIFEST); + assert.deepStrictEqual(yield* service.forceRefresh, REMOTE_MANIFEST); + assert.deepStrictEqual(yield* service.current, REMOTE_MANIFEST); + assert.deepStrictEqual(yield* (yield* make).current, REMOTE_MANIFEST); + assert.strictEqual(fetchCount, 2); + assert.deepStrictEqual(yield* service.forceRefresh, REMOTE_MANIFEST); + assert.strictEqual(fetchCount, 3); + }).pipe( + Effect.scoped, + Effect.provide( + serviceLayers({ + prefix: "model-manifest-force-retry-test", + response: () => + fetchCount++ === 1 + ? new Response(null, { status: 503 }) + : Response.json(REMOTE_MANIFEST), + }), + ), + ); + }); + + it.live("explicit refresh bypasses the retry delay after an initial failure", () => { + let fetchCount = 0; + return Effect.gen(function* () { + const service = yield* make; + assert.deepStrictEqual(yield* service.refresh, BUNDLED_MODEL_MANIFEST); + assert.deepStrictEqual(yield* service.refresh, BUNDLED_MODEL_MANIFEST); + assert.strictEqual(fetchCount, 1); + assert.deepStrictEqual(yield* service.forceRefresh, REMOTE_MANIFEST); + assert.strictEqual(fetchCount, 2); + }).pipe( + Effect.scoped, + Effect.provide( + serviceLayers({ + prefix: "model-manifest-force-initial-retry-test", + response: () => + fetchCount++ === 0 + ? new Response(null, { status: 503 }) + : Response.json(REMOTE_MANIFEST), + }), + ), + ); + }); + it.live("prefers a fetched manifest over the bundle and caches it to disk", () => Effect.gen(function* () { const service = yield* make; diff --git a/apps/server/src/provider/ModelManifest.ts b/apps/server/src/provider/ModelManifest.ts index bcf9ccfa8b..0ba775e231 100644 --- a/apps/server/src/provider/ModelManifest.ts +++ b/apps/server/src/provider/ModelManifest.ts @@ -341,6 +341,8 @@ export class ModelManifest extends Context.Service< readonly current: Effect.Effect; /** Manifest after a TTL-gated remote refresh; never fails. */ readonly refresh: Effect.Effect; + /** Explicit refresh bypasses freshness and retry timers, retaining last-good data. */ + readonly forceRefresh: Effect.Effect; /** Forks `refresh` into the service's own scope. Drivers call this from * provider checks: the fetch is process-shared state, so it must survive * the teardown of whichever instance happened to trigger it. */ @@ -352,6 +354,7 @@ export class ModelManifest extends Context.Service< const BundledOnlyModelManifest: ModelManifest["Service"] = { current: Effect.succeed(BUNDLED_MODEL_MANIFEST), refresh: Effect.succeed(BUNDLED_MODEL_MANIFEST), + forceRefresh: Effect.succeed(BUNDLED_MODEL_MANIFEST), refreshInBackground: Effect.void, }; @@ -388,7 +391,7 @@ export const make = Effect.gen(function* () { }), ); - const refresh = Effect.fn("ModelManifest.refresh")(function* () { + const refresh = Effect.fn("ModelManifest.refresh")(function* (force = false) { yield* ensureDiskCacheLoaded; const now = yield* Clock.currentTimeMillis; // A timestamp in the future means the wall clock moved backwards (the @@ -396,8 +399,8 @@ export const make = Effect.gen(function* () { // it as expired: the refetch rewrites both timestamps and self-heals. const isWithin = (sinceMs: number | null, windowMs: number) => sinceMs !== null && now >= sinceMs && now - sinceMs < windowMs; - if (isWithin(fetchedAtMs, MANIFEST_TTL_MS)) return manifest; - if (isWithin(lastAttemptMs, MANIFEST_RETRY_MS)) return manifest; + if (!force && isWithin(fetchedAtMs, MANIFEST_TTL_MS)) return manifest; + if (!force && isWithin(lastAttemptMs, MANIFEST_RETRY_MS)) return manifest; // The same switch that gates provider CLI update checks. It stops network // fetches only: a manifest already cached on disk from an earlier fetch @@ -450,6 +453,7 @@ export const make = Effect.gen(function* () { return ModelManifest.of({ current: ensureDiskCacheLoaded.pipe(Effect.map(() => manifest)), refresh: guardedRefresh, + forceRefresh: refreshSemaphore.withPermits(1)(refresh(true)), refreshInBackground: Effect.forkIn(guardedRefresh, serviceScope).pipe(Effect.asVoid), }); }); diff --git a/apps/server/src/provider/ProviderDriver.ts b/apps/server/src/provider/ProviderDriver.ts index 46b0f1cf80..2ca4fedcf7 100644 --- a/apps/server/src/provider/ProviderDriver.ts +++ b/apps/server/src/provider/ProviderDriver.ts @@ -84,6 +84,8 @@ export interface ProviderInstance { readonly enabled: boolean; readonly snapshot: ServerProviderShape; readonly snapshotForCwd?: (cwd: string) => Effect.Effect; + /** Invalidate provider-owned discovery caches before an explicit refresh. */ + readonly invalidateCaches?: Effect.Effect; readonly adapter: ProviderAdapterShape; readonly textGeneration: TextGeneration.TextGeneration["Service"]; readonly refreshModels?: () => Effect.Effect; diff --git a/apps/server/src/provider/acp/CursorTransportFailure.test.ts b/apps/server/src/provider/acp/CursorTransportFailure.test.ts index dcd4f9996b..fdde19b934 100644 --- a/apps/server/src/provider/acp/CursorTransportFailure.test.ts +++ b/apps/server/src/provider/acp/CursorTransportFailure.test.ts @@ -38,6 +38,8 @@ describe("CursorTransportFailure", () => { "Error: ConnectError: [unauthenticated] sign in", "Error: ConnectError: [permission_denied] subscription required", "Error: HTTP 500 from the application being debugged", + "Error: RetriableError: [internal] Failed to run step, exceeded max retries", + "Error: RetriableError: [internal] Failed to run step, exceeded max retries\n at step (cli.js:1:2)", ])("preserves prose, code and non-transport errors: %s", (message) => { expect(failureFor([...message])).toBeUndefined(); }); diff --git a/apps/server/src/provider/acp/CursorTransportFailure.ts b/apps/server/src/provider/acp/CursorTransportFailure.ts index 99765c74eb..e6cadb23dc 100644 --- a/apps/server/src/provider/acp/CursorTransportFailure.ts +++ b/apps/server/src/provider/acp/CursorTransportFailure.ts @@ -1,6 +1,7 @@ const maxLineLength = 4096; +// Cursor also uses RetriableError for agent-loop failures; preserve those diagnostics. const transportError = - /^Error: (?:RetriableError: .+|ConnectError: \[(?:unavailable|aborted|deadline_exceeded)\].*)$/; + /^Error: (?:RetriableError: (?!\[internal\]).+|ConnectError: \[(?:unavailable|aborted|deadline_exceeded)\].*)$/; const serverError = "Something went wrong communicating with the server. Please try again."; interface ReplyState { diff --git a/apps/server/src/provider/acp/XAiBackgroundTasks.test.ts b/apps/server/src/provider/acp/XAiBackgroundTasks.test.ts new file mode 100644 index 0000000000..3f77ca451a --- /dev/null +++ b/apps/server/src/provider/acp/XAiBackgroundTasks.test.ts @@ -0,0 +1,274 @@ +import { describe, expect, it } from "vite-plus/test"; +import { TurnId } from "@t3tools/contracts"; + +import { + buildGrokBackgroundTaskEvents, + type GrokBackgroundTaskRecord, +} from "./XAiBackgroundTasks.ts"; + +const turnId = TurnId.make("turn-1"); +const monitor = { type: "Monitor", taskId: "monitor-1", timeoutMs: 60_000 }; +const shell = { type: "BackgroundTaskStarted", task_id: "shell-1", command: "sleep 40" }; + +function mapper() { + const tasks = new Map(); + const completedTaskIds = new Map(); + const ambiguousTaskIds = new Set(); + const update = ( + rawOutput: unknown, + overrides: Partial[0]> = {}, + ) => + buildGrokBackgroundTaskEvents({ + tasks, + completedTaskIds, + ambiguousTaskIds, + toolCallId: "call-1", + rawInput: { description: "Watch" }, + rawOutput, + toolCallStatus: "completed", + turnId, + ...overrides, + }); + return { tasks, completedTaskIds, ambiguousTaskIds, update }; +} + +describe("Grok background tasks", () => { + it.each([ + [monitor, "monitor-1", "monitor", "Watch"], + [shell, "shell-1", "shell", "sleep 40"], + ] as const)("starts and deduplicates %j", (output, id, taskType, description) => { + const { tasks, update } = mapper(); + expect(update(output)).toEqual([ + { + type: "task.started", + turnId, + payload: { taskId: id, taskType, description, title: description, toolUseId: "call-1" }, + }, + ]); + expect(update(output)).toEqual([]); + expect(tasks.size).toBe(1); + }); + + it("accepts automatic backgrounding before the tool becomes terminal", () => { + const { update } = mapper(); + expect(update(shell, { toolCallStatus: "inProgress" })[0]?.type).toBe("task.started"); + expect(update(monitor, { toolCallStatus: "inProgress" })).toEqual([]); + expect(update(monitor, { toolCallStatus: "failed" })).toEqual([]); + }); + + it.each([ + ["running", null, "task.progress", undefined], + ["pending", null, "task.progress", undefined], + ["completed", 0, "task.completed", "completed"], + ["success", 0, "task.completed", "completed"], + ["succeeded", 0, "task.completed", "completed"], + ["failed", 1, "task.completed", "failed"], + ["error", 1, "task.completed", "failed"], + ["stopped", 137, "task.completed", "stopped"], + ["killed", 137, "task.completed", "stopped"], + ["cancelled", 137, "task.completed", "stopped"], + [undefined, 0, "task.completed", "completed"], + [undefined, 1, "task.completed", "failed"], + ])("maps poll status %s / exit %s", (status, exit_code, type, expectedStatus) => { + const { tasks, update } = mapper(); + update(monitor); + const events = update({ + type: "TaskOutput", + Result: { + task_id: "monitor-1", + command: "[monitor:Watch]", + status, + exit_code, + output: "\n result\nmore", + }, + }); + expect(events).toEqual([ + { + type, + turnId, + payload: { + taskId: "monitor-1", + taskType: "monitor", + description: "Watch", + title: "Watch", + toolUseId: "call-1", + summary: "result", + ...(expectedStatus ? { status: expectedStatus } : {}), + }, + }, + ]); + expect(tasks.size).toBe(type === "task.progress" ? 1 : 0); + }); + + it.each([undefined, TurnId.make("turn-2")])( + "does not attribute old tasks to a later turn: %s", + (laterTurnId) => { + const { tasks, update } = mapper(); + update(shell); + const events = update( + { + type: "TaskOutput", + Result: { task_id: "shell-1", command: "sleep 40", status: "completed" }, + }, + { turnId: laterTurnId }, + ); + expect(events).toHaveLength(1); + expect(events[0]?.turnId).toBeUndefined(); + expect(tasks.size).toBe(0); + }, + ); + + it("starts unknown poll tasks before progress or completion and ignores subagents", () => { + const { tasks, update } = mapper(); + const events = update({ + type: "TaskOutput", + MultiResult: { + results: [ + { task_id: "shell-1", command: "sleep 40", status: "running" }, + { task_id: "monitor-1", command: "[monitor] Watch", status: "completed" }, + { task_id: "agent-1", command: "[subagent:executor] work", status: "running" }, + ], + }, + }); + expect(events.map(({ type }) => type)).toEqual([ + "task.started", + "task.progress", + "task.started", + "task.completed", + ]); + expect(events.map(({ payload }) => payload.taskType)).toEqual([ + "shell", + "shell", + "monitor", + "monitor", + ]); + expect([...tasks.keys()]).toEqual(["shell-1"]); + expect(events.map((event) => event.turnId)).toEqual([ + undefined, + undefined, + undefined, + undefined, + ]); + }); + + it("retires only successfully killed tasks, including mixed results", () => { + const { tasks, update } = mapper(); + update(shell); + update(monitor); + const result = { type: "KillTask", Result: { task_id: "shell-1", outcome: "killed" } }; + expect(update(result, { toolCallStatus: "failed" })).toEqual([]); + expect(tasks.size).toBe(2); + const events = update({ + type: "KillTask", + MultiResult: { + results: [ + result.Result, + { task_id: "monitor-1", outcome: "error" }, + { task_id: "unknown", outcome: "killed" }, + ], + }, + }); + expect(events).toEqual([ + { + type: "task.completed", + turnId, + payload: { + taskId: "shell-1", + taskType: "shell", + description: "sleep 40", + title: "sleep 40", + toolUseId: "call-1", + status: "stopped", + }, + }, + ]); + expect([...tasks.keys()]).toEqual(["monitor-1"]); + }); + + it("ignores repeated terminal polls and gives a reused provider id a new runtime identity", () => { + const { tasks, completedTaskIds, update } = mapper(); + update(shell); + const terminal = { + type: "TaskOutput", + Result: { task_id: "shell-1", command: "sleep 40", status: "completed" }, + }; + expect(update(terminal).map((event) => event.type)).toEqual(["task.completed"]); + expect(update(terminal)).toEqual([]); + expect(completedTaskIds.get("shell-1")).toBe("call-1"); + expect(update(shell, { toolCallId: "call-2", turnId: TurnId.make("turn-2") })).toMatchObject([ + { type: "task.started", payload: { taskId: "shell-1#call-2", toolUseId: "call-2" } }, + ]); + expect(tasks.get("shell-1")?.payload.taskId).toBe("shell-1#call-2"); + }); + + it("remembers terminal identities for the session even after many other completions", () => { + const { completedTaskIds, update } = mapper(); + const completed = (id: string) => ({ + type: "TaskOutput", + Result: { task_id: id, command: "echo done", status: "completed" }, + }); + update(completed("old-task")); + for (let index = 0; index < 513; index += 1) { + update(completed(`task-${index}`)); + } + expect(completedTaskIds.size).toBe(514); + expect(update(completed("old-task"))).toEqual([]); + }); + + it("fails closed when two live tool calls claim the same provider task id", () => { + const { tasks, ambiguousTaskIds, update } = mapper(); + update(shell); + expect(update(shell, { toolCallId: "call-2" })).toMatchObject([ + { type: "task.completed", payload: { taskId: "shell-1", status: "stopped" } }, + ]); + expect(ambiguousTaskIds.has("shell-1")).toBe(true); + expect(tasks.has("shell-1")).toBe(false); + expect( + update({ + type: "TaskOutput", + Result: { task_id: "shell-1", command: "sleep 40", status: "completed" }, + }), + ).toEqual([]); + }); + + it("bounds long descriptions and output lines before projecting them", () => { + const { update } = mapper(); + const [started] = update(monitor, { + rawInput: { description: "a".repeat(2_000) }, + }); + expect(started?.type).toBe("task.started"); + if (started?.type !== "task.started") throw new Error("expected task.started"); + expect(started?.payload.description).toHaveLength(200); + const [progress] = update({ + type: "TaskOutput", + Result: { + task_id: "monitor-1", + command: "[monitor] Watch", + status: "running", + output: "b".repeat(2_000), + }, + }); + expect(progress?.type).toBe("task.progress"); + if (progress?.type !== "task.progress") throw new Error("expected task.progress"); + expect(progress?.payload.summary).toHaveLength(512); + }); + + it.each([ + null, + [], + {}, + { type: "Text", text: "subagent_id: fake\ntype: executor\ndescription: fake" }, + { type: "Monitor", taskId: " " }, + { type: "BackgroundTaskStarted", task_id: "shell-1" }, + { + type: "TaskOutput", + MultiResult: { + results: [null, {}, { task_id: "task", command: "sleep 40", exit_code: Infinity }], + }, + }, + ])("ignores malformed or unrelated outputs: %j", (output) => { + const { tasks, update } = mapper(); + expect(update(output)).toEqual([]); + expect(tasks.size).toBe(0); + }); +}); diff --git a/apps/server/src/provider/acp/XAiBackgroundTasks.ts b/apps/server/src/provider/acp/XAiBackgroundTasks.ts new file mode 100644 index 0000000000..85dbd5b767 --- /dev/null +++ b/apps/server/src/provider/acp/XAiBackgroundTasks.ts @@ -0,0 +1,183 @@ +import { + RuntimeTaskId, + type ProviderRuntimeTaskStartedEvent, + type ProviderRuntimeTaskProgressEvent, + type ProviderRuntimeTaskCompletedEvent, + type TurnId, +} from "@t3tools/contracts"; + +type TaskEvent = + | Pick + | Pick + | Pick; + +export interface GrokBackgroundTaskRecord { + readonly rawTaskId: string; + readonly payload: { + readonly taskId: RuntimeTaskId; + readonly taskType: "monitor" | "shell"; + readonly description: string; + readonly title: string; + readonly toolUseId?: string; + }; + readonly turnId: TurnId | undefined; +} + +function record(value: unknown): Record { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : {}; +} + +function text(value: unknown): string | undefined { + return typeof value === "string" ? value.trim() || undefined : undefined; +} + +function lifecycle(status: unknown, exitCode: unknown) { + switch (text(status)?.toLowerCase()) { + case "pending": + case "running": + return "running"; + case "completed": + case "success": + case "succeeded": + return "completed"; + case "failed": + case "error": + return "failed"; + case "stopped": + case "killed": + case "cancelled": + return "stopped"; + default: + return typeof exitCode === "number" && Number.isFinite(exitCode) + ? exitCode === 0 + ? "completed" + : "failed" + : undefined; + } +} + +/** Map Grok's discriminated tool results, including notifications after the turn ends. */ +export function buildGrokBackgroundTaskEvents(input: { + readonly tasks: Map; + readonly completedTaskIds: Map; + readonly ambiguousTaskIds: Set; + readonly toolCallId: string; + readonly rawInput: unknown; + readonly rawOutput: unknown; + readonly toolCallStatus: string | undefined; + readonly turnId?: TurnId | undefined; +}): TaskEvent[] { + const { tasks, completedTaskIds, ambiguousTaskIds, toolCallId, toolCallStatus, turnId } = input; + const output = record(input.rawOutput); + const events: TaskEvent[] = []; + if ( + toolCallStatus !== "completed" && + toolCallStatus !== "failed" && + output.type !== "BackgroundTaskStarted" + ) { + return events; + } + const attribution = (task: GrokBackgroundTaskRecord) => + task.turnId !== undefined && task.turnId === turnId ? { turnId } : {}; + const start = ( + id: string, + taskType: "monitor" | "shell", + description: string, + toolUseId?: string, + ) => { + if (ambiguousTaskIds.has(id)) return undefined; + const known = tasks.get(id); + if (known) { + if (!toolUseId || known.payload.toolUseId === toolUseId) return known; + // Polls identify only the provider ID. Once two live tool calls reuse it, + // their later results cannot be attributed safely to either activation. + tasks.delete(id); + completedTaskIds.set(id, known.payload.toolUseId); + ambiguousTaskIds.add(id); + events.push({ + type: "task.completed", + payload: { ...known.payload, status: "stopped" }, + ...attribution(known), + }); + return undefined; + } + const previousToolUseId = completedTaskIds.get(id); + if (completedTaskIds.has(id) && (!toolUseId || previousToolUseId === toolUseId)) { + return undefined; + } + const task: GrokBackgroundTaskRecord = { + rawTaskId: id, + payload: { + taskId: RuntimeTaskId.make(completedTaskIds.has(id) ? `${id}#${toolUseId}` : id), + taskType, + description: description.slice(0, 200), + title: description.slice(0, 200), + ...(toolUseId ? { toolUseId } : {}), + }, + // Polls can rediscover older tasks without establishing their originating turn. + turnId: toolUseId ? turnId : undefined, + }; + completedTaskIds.delete(id); + tasks.set(id, task); + events.push({ type: "task.started", payload: task.payload, ...attribution(task) }); + return task; + }; + const complete = ( + task: GrokBackgroundTaskRecord, + status: "completed" | "failed" | "stopped", + summary?: string, + ) => { + tasks.delete(task.rawTaskId); + completedTaskIds.delete(task.rawTaskId); + completedTaskIds.set(task.rawTaskId, task.payload.toolUseId); + events.push({ + type: "task.completed", + payload: { ...task.payload, status, ...(summary ? { summary } : {}) }, + ...attribution(task), + }); + }; + + if (output.type === "Monitor" && toolCallStatus === "completed") { + const id = text(output.taskId); + if (id) start(id, "monitor", text(record(input.rawInput).description) ?? "Monitor", toolCallId); + } else if (output.type === "BackgroundTaskStarted") { + const id = text(output.task_id) ?? text(output.taskId); + const command = text(output.command); + if (id && command) start(id, "shell", command.split("\n")[0]!.slice(0, 200), toolCallId); + } else if (output.type === "TaskOutput" || output.type === "KillTask") { + const results = record(output.MultiResult).results; + for (const value of Array.isArray(results) ? results : [output.Result]) { + const result = record(value); + const id = text(result.task_id); + if (!id) continue; + if (output.type === "KillTask") { + const task = tasks.get(id); + if (task && toolCallStatus === "completed" && result.outcome === "killed") + complete(task, "stopped"); + continue; + } + const command = text(result.command); + const status = lifecycle(result.status, result.exit_code); + if (!command || !status || command.startsWith("[subagent:")) continue; + const task = start(id, /^\[monitor[:\]]/.test(command) ? "monitor" : "shell", command); + if (!task) continue; + const summary = text(result.output) + ?.split("\n") + .find((line) => line.trim()) + ?.trim() + .slice(0, 512); + if (status === "running") { + events.push({ + type: "task.progress", + payload: { ...task.payload, ...(summary ? { summary } : {}) }, + ...attribution(task), + }); + } else { + complete(task, status, summary); + } + } + } + return events; +} diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.test.ts b/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.test.ts index ca3c2ae7d6..15db0b597b 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.test.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.test.ts @@ -6661,6 +6661,39 @@ describe("PrimeAgentDaemonAdapter", () => { }, }); } + yield* offer(captures, { + _tag: "ChildUpdated", + child: { + id: "child-private-progress", + parentId: "parent", + activeSessionId: "native-private-100", + model: "child-model", + label: "private-safe-label", + status: "running", + tokenCount: 10, + toolUseCount: 1, + activity: { kind: "executing", toolName: "bash" }, + progressNote: "Checking the failing test", + lastActivityAt: 1_000, + }, + }); + yield* awaitObservedType(subscription.observed, "task.progress"); + yield* offer(captures, { + _tag: "ChildUpdated", + child: { + id: "child-private-progress", + parentId: "parent", + activeSessionId: "native-private-100", + model: "child-model", + label: "private-safe-label", + status: "running", + tokenCount: 10, + toolUseCount: 1, + activity: { kind: "executing", toolName: "bash" }, + progressNote: "Checking the failing test", + lastActivityAt: 2_000, + }, + }); captures.agentRoster = [ { id: "child-private-progress", @@ -6676,6 +6709,7 @@ describe("PrimeAgentDaemonAdapter", () => { activity: { kind: "executing", toolName: "bash" }, answerPreview: "private-answer-roster", recap: "private-recap-roster", + progressNote: "Checking the failing test", }, ]; expect( @@ -6718,17 +6752,21 @@ describe("PrimeAgentDaemonAdapter", () => { yield* awaitObservedType(subscription.observed, "session.goal.updated"); const durableEvents = subscription.events.slice(durableStart); - expect(durableEvents.filter((event) => event.type === "task.progress")).toEqual([]); + expect(durableEvents.filter((event) => event.type === "task.progress")).toEqual([ + expect.objectContaining({ + payload: expect.objectContaining({ summary: "Checking the failing test" }), + }), + ]); const completed = durableEvents.filter((event) => event.type === "task.completed"); expect(completed).toHaveLength(1); expect(completed[0]).toMatchObject({ payload: { taskId: "child-private-progress", status: "failed", + summary: "Failed", typedUsage: { totalTokens: 12, toolUses: 1, durationMs: 1_500 }, }, }); - expect(completed[0]?.payload).not.toHaveProperty("summary"); const persisted = encodeUnknownJson(durableEvents); expect(persisted).not.toContain("private-answer"); expect(persisted).not.toContain("private-recap"); diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.ts b/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.ts index 18a0fb462b..3ecc76033b 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonAdapter.ts @@ -531,6 +531,7 @@ type PrimeAgentDaemonChildDurableProjection = { readonly messageable: boolean; readonly waiting: boolean; readonly lastToolName: string | undefined; + readonly progressNote: string | undefined; readonly tokenCount: number | undefined; readonly toolUseCount: number | undefined; readonly terminalDurationMs: number | undefined; @@ -560,6 +561,7 @@ function durableChildProjection( child.status === "running" && child.activity?.kind !== "waiting" ? durableChildText(child.activity?.toolName) : undefined, + progressNote: active ? durableChildText(child.progressNote) : undefined, tokenCount: durableChildCount(child.tokenCount), toolUseCount: durableChildCount(child.toolUseCount), terminalDurationMs: active ? undefined : durableChildCount(child.durationMs), @@ -581,6 +583,7 @@ function durableChildChanged( left.messageable !== right.messageable || left.waiting !== right.waiting || left.lastToolName !== right.lastToolName || + left.progressNote !== right.progressNote || left.tokenCount !== right.tokenCount || left.toolUseCount !== right.toolUseCount || left.terminalDurationMs !== right.terminalDurationMs diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonEvents.test.ts b/apps/server/src/provider/prime/PrimeAgentDaemonEvents.test.ts index 1a7e865f87..7829ea5431 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonEvents.test.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonEvents.test.ts @@ -819,6 +819,26 @@ describe("PrimeAgentDaemonEvents", () => { _tag: "ChildUpdated", child: { id: "child-1", status: "running", activity: { toolName: "ipython" } }, }); + expect( + decodePrimeAgentDaemonEvent( + sessionEvent({ + type: "rlm_child_update", + child: { + ...child, + progressNote: "Checking tests", + lastActivityAt: 1_700_000_000_000, + activityStaleMs: 600_000, + }, + }), + ), + ).toMatchObject({ + _tag: "ChildUpdated", + child: { + progressNote: "Checking tests", + lastActivityAt: 1_700_000_000_000, + activityStaleMs: 600_000, + }, + }); const queueChanged = decodePrimeAgentDaemonEvent( sessionEvent({ type: "session_action_update", actions }), ); diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonEvents.ts b/apps/server/src/provider/prime/PrimeAgentDaemonEvents.ts index de46966fe7..335895f01d 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonEvents.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonEvents.ts @@ -387,6 +387,9 @@ const rlmChild = Schema.Struct({ toolUseCount: Schema.optional(Schema.Number), tokenCount: Schema.optional(Schema.Number), recap: Schema.optional(Schema.String), + progressNote: Schema.optional(Schema.String), + lastActivityAt: Schema.optional(Schema.Finite), + activityStaleMs: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))), sessionDir: Schema.String, activity: Schema.optional( Schema.Struct({ @@ -1309,6 +1312,9 @@ export type PrimeDaemonEvent = ( readonly toolUseCount?: number | undefined; readonly tokenCount?: number | undefined; readonly recap?: string | undefined; + readonly progressNote?: string | undefined; + readonly lastActivityAt?: number | undefined; + readonly activityStaleMs?: number | undefined; readonly activity?: | { readonly kind: "waiting" | "writing" | "executing"; @@ -1656,6 +1662,9 @@ function mapChild( toolUseCount: value.toolUseCount, tokenCount: value.tokenCount, recap: optionalBounded(value.recap, MAX_PREVIEW_LENGTH), + progressNote: optionalBounded(value.progressNote, 512), + lastActivityAt: value.lastActivityAt, + activityStaleMs: value.activityStaleMs, activity: value.activity, error: optionalBounded(value.error, MAX_PREVIEW_LENGTH), }; diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.test.ts b/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.test.ts index a4f078db40..4b42987701 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.test.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.test.ts @@ -660,6 +660,9 @@ describe("mapPrimeAgentDaemonRuntimeEventDrafts", () => { tokenCount: 90, answerPreview: "private live answer", recap: "private recap", + progressNote: "Checking the failing test", + lastActivityAt: 1_700_000_000_000, + activityStaleMs: 600_000, error: "private transient error", activity: { kind: "executing", toolName: "bash" }, }, @@ -676,6 +679,7 @@ describe("mapPrimeAgentDaemonRuntimeEventDrafts", () => { taskId: RuntimeTaskId.make("child-1"), description: "Review tests", status: "running", + summary: "Checking the failing test", typedUsage: { totalTokens: 90, toolUses: 4, durationMs: 250 }, lastToolName: "bash", taskType: "subagent", @@ -693,6 +697,8 @@ describe("mapPrimeAgentDaemonRuntimeEventDrafts", () => { expect(JSON.stringify(running)).not.toContain("private live answer"); expect(JSON.stringify(running)).not.toContain("private recap"); expect(JSON.stringify(running)).not.toContain("private transient error"); + expect(JSON.stringify(running)).not.toContain("activityStaleMs"); + expect(JSON.stringify(running)).not.toContain("lastActivityAt"); const terminal = mapPrimeAgentDaemonRuntimeEventDrafts({ ...context, @@ -716,12 +722,12 @@ describe("mapPrimeAgentDaemonRuntimeEventDrafts", () => { payload: { taskId: RuntimeTaskId.make("child-1"), status: "failed", + summary: "Failed", typedUsage: { totalTokens: 100 }, messageable: false, timelineBypass: true, }, }); - expect(terminal[0]?.payload).not.toHaveProperty("summary"); expect(JSON.stringify(terminal)).not.toContain("private terminal"); }); diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.ts b/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.ts index b2912965b9..566f6f90c3 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonRuntimeEvents.ts @@ -307,7 +307,8 @@ function childDraft( }; case "running": { const lastToolName = boundedNonEmpty(child.activity?.toolName, MAX_SCALAR_LENGTH); - if (child.activity?.kind === "waiting") { + const progressNote = boundedNonEmpty(child.progressNote?.trim(), 512); + if (child.activity?.kind === "waiting" && progressNote === undefined) { return { ...runtimeBase(input), type: "task.updated", @@ -320,7 +321,8 @@ function childDraft( payload: { taskId, description, - status: "running", + status: child.activity?.kind === "waiting" ? "waiting" : "running", + ...(progressNote === undefined ? {} : { summary: progressNote }), ...(typedUsage === undefined ? {} : { typedUsage }), ...(lastToolName === undefined ? {} : { lastToolName }), ...linkage, @@ -337,6 +339,8 @@ function childDraft( taskId, status: child.status === "done" ? "completed" : child.status === "error" ? "failed" : "stopped", + summary: + child.status === "done" ? "Completed" : child.status === "error" ? "Failed" : "Stopped", ...(typedUsage === undefined ? {} : { typedUsage }), ...linkage, }, diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.test.ts b/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.test.ts index c28abbde45..022be73c24 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.test.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.test.ts @@ -12668,6 +12668,111 @@ describe("PrimeAgentDaemonSessionRuntime", () => { }), ); + it.effect("rejects a private leaf response from before daemon reconnect", () => + Effect.scoped( + Effect.gen(function* () { + let releaseOldState!: (value: unknown) => void; + let observeOldRequest!: () => void; + let currentLeaf = "leaf-source-private"; + let firstRequest = true; + const oldRequestStarted = new Promise((resolve) => { + observeOldRequest = resolve; + }); + const test = fixture({ + rawSnapshot: { + ...snapshot(), + state: { ...snapshot().state, leafId: "leaf-source-private" }, + }, + getStateImpl: () => { + if (firstRequest) { + firstRequest = false; + observeOldRequest(); + return new Promise((resolve) => { + releaseOldState = resolve; + }); + } + return Promise.resolve({ + sessionId: "session-1", + activeSessionId: "active-secret-1", + leafId: currentLeaf, + }); + }, + }); + const runtime = yield* test.make(); + const inspection = yield* runtime.inspectConversationLeaf.pipe( + Effect.result, + Effect.forkChild({ startImmediately: true }), + ); + yield* Effect.promise(() => oldRequestStarted); + yield* Effect.promise(() => + test.emit({ type: "connection_status", status: "reconnecting" }), + ); + yield* Effect.promise(() => test.emit({ type: "session_resynced", snapshot: snapshot(9) })); + expect(runtime.resolveReconnectSnapshot(1, true)).toBe(true); + yield* Effect.promise(() => test.emit({ type: "connection_status", status: "connected" })); + releaseOldState({ + sessionId: "session-1", + activeSessionId: "active-secret-1", + leafId: "leaf-target-private", + }); + const result = yield* Fiber.join(inspection); + expect(result._tag).toBe("Failure"); + expect(yield* runtime.inspectConversationLeaf).toBe("leaf-source-private"); + currentLeaf = "leaf-third-private"; + expect(yield* runtime.inspectConversationLeaf).toBe("leaf-third-private"); + }), + ), + ); + + it.effect("rejects a private navigation response from before daemon reconnect", () => + Effect.scoped( + Effect.gen(function* () { + let releaseOldNavigation!: (value: unknown) => void; + let observeOldNavigation!: () => void; + let currentLeaf = "leaf-source-private"; + const oldNavigationStarted = new Promise((resolve) => { + observeOldNavigation = resolve; + }); + const test = fixture({ + rawSnapshot: { + ...snapshot(), + state: { ...snapshot().state, leafId: currentLeaf }, + }, + getStateImpl: () => + Promise.resolve({ + sessionId: "session-1", + activeSessionId: "active-secret-1", + leafId: currentLeaf, + }), + navigateTreeImpl: () => { + observeOldNavigation(); + return new Promise((resolve) => { + releaseOldNavigation = resolve; + }); + }, + }); + const runtime = yield* test.make(); + const navigation = yield* runtime + .navigateConversationLeaf({ + desiredLeafId: "leaf-target-private", + allowedSourceLeafId: "leaf-source-private", + }) + .pipe(Effect.result, Effect.forkChild({ startImmediately: true })); + yield* Effect.promise(() => oldNavigationStarted); + yield* Effect.promise(() => + test.emit({ type: "connection_status", status: "reconnecting" }), + ); + yield* Effect.promise(() => test.emit({ type: "session_resynced", snapshot: snapshot(9) })); + expect(runtime.resolveReconnectSnapshot(1, true)).toBe(true); + yield* Effect.promise(() => test.emit({ type: "connection_status", status: "connected" })); + currentLeaf = "leaf-third-private"; + releaseOldNavigation({ cancelled: false }); + expect((yield* Fiber.join(navigation))._tag).toBe("Failure"); + expect(yield* runtime.inspectConversationLeaf).toBe("leaf-third-private"); + }), + ), + ); + it.effect("rejects a third leaf and keeps response-loss target proof inspectable", () => Effect.gen(function* () { let leafId = "leaf-source-private"; diff --git a/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.ts b/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.ts index 4dc5f1cce0..fb01b7f2ec 100644 --- a/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.ts +++ b/apps/server/src/provider/prime/PrimeAgentDaemonSessionRuntime.ts @@ -5605,6 +5605,11 @@ export const makePrimeAgentDaemonSessionRuntime = Effect.fn("makePrimeAgentDaemo Predicate.isFunction(connection!.getState) && Predicate.isFunction(connection!.navigateTree); + const conversationRequestIsCurrent = (generation: number) => + generation === connectionGeneration && + (reconnectResolution === undefined || reconnectResolution.settled) && + client.hello?.supervisorGeneration?.trim() === conversationRuntimeGeneration; + const readConversationLeaf = Effect.fn( "PrimeAgentDaemonSessionRuntime.inspectConversationLeaf", )(function* () { @@ -5617,6 +5622,14 @@ export const makePrimeAgentDaemonSessionRuntime = Effect.fn("makePrimeAgentDaemo ); } const getState = yield* requireMethod("inspect-conversation-leaf", connection!.getState); + const requestGeneration = connectionGeneration; + if (!conversationRequestIsCurrent(requestGeneration)) { + return yield* runtimeError( + "inspect-conversation-leaf", + "request-failed", + "Prime Agent conversation connection changed before inspection.", + ); + } const output = yield* Effect.tryPromise({ try: () => getState.call(connection), catch: () => @@ -5636,6 +5649,13 @@ export const makePrimeAgentDaemonSessionRuntime = Effect.fn("makePrimeAgentDaemo ), }), ); + if (!conversationRequestIsCurrent(requestGeneration)) { + return yield* runtimeError( + "inspect-conversation-leaf", + "request-failed", + "Prime Agent conversation connection changed during inspection.", + ); + } const decoded = decodePrivateConversationLeafState(output); if ( Option.isNone(decoded) || @@ -5705,6 +5725,14 @@ export const makePrimeAgentDaemonSessionRuntime = Effect.fn("makePrimeAgentDaemo "navigate-conversation-leaf", connection!.navigateTree, ); + const requestGeneration = connectionGeneration; + if (!conversationRequestIsCurrent(requestGeneration)) { + return yield* runtimeError( + "navigate-conversation-leaf", + "request-failed", + "Prime Agent conversation connection changed before navigation.", + ); + } const output = yield* Effect.tryPromise({ try: () => navigate.call(connection, input.desiredLeafId, { summarize: false }), catch: () => @@ -5724,6 +5752,13 @@ export const makePrimeAgentDaemonSessionRuntime = Effect.fn("makePrimeAgentDaemo ), }), ); + if (!conversationRequestIsCurrent(requestGeneration)) { + return yield* runtimeError( + "navigate-conversation-leaf", + "request-failed", + "Prime Agent conversation connection changed during navigation.", + ); + } const result = decodePrivateConversationNavigationResult(output); if (Option.isNone(result) || result.value.cancelled || result.value.aborted === true) { return yield* runtimeError( diff --git a/apps/server/src/provider/providerUsageRetention.test.ts b/apps/server/src/provider/providerUsageRetention.test.ts index 1a43b35335..e82dfbdd7e 100644 --- a/apps/server/src/provider/providerUsageRetention.test.ts +++ b/apps/server/src/provider/providerUsageRetention.test.ts @@ -5,10 +5,14 @@ import * as Ref from "effect/Ref"; import * as TestClock from "effect/testing/TestClock"; import { + authenticatedUsageIdentity, isRetainedUsageFresh, + matchingAccountUsage, retainSnapshotUsageLimits, - retainUsageLimits, + retainUsageLimitsForAccount, preferFreshUsageReading, + usageReadingForAuth, + type AccountUsageReading, } from "./providerUsageRetention.ts"; const NOW = Date.parse("2026-08-06T12:00:00.000Z"); @@ -71,17 +75,67 @@ describe("isRetainedUsageFresh", () => { }); }); -describe("retainUsageLimits", () => { +const ACCOUNT_A = "email:a@example.com"; +const ACCOUNT_B = "email:b@example.com"; + +it("binds an authenticated Codex reading to both account id and live email", () => { + assert.strictEqual( + authenticatedUsageIdentity({ + status: "authenticated", + accountId: "account-1", + email: "A@Example.com", + }), + "account:account-1|email:a@example.com", + ); + assert.isUndefined(authenticatedUsageIdentity({ status: "unknown", email: "a@example.com" })); +}); + +it("does not overlay a direct reset reading onto another or unverified account", () => { + const direct = { identity: ACCOUNT_A, usageLimits: usage("2026-08-06T11:59:00.000Z") }; + assert.strictEqual( + usageReadingForAuth(direct, { status: "authenticated", email: "a@example.com" }), + direct.usageLimits, + ); + assert.isUndefined( + usageReadingForAuth(direct, { status: "authenticated", email: "b@example.com" }), + ); + assert.isUndefined(usageReadingForAuth(direct, { status: "unknown" })); +}); + +it("accepts a Claude OAuth read only when its account probe agrees with the current account", () => { + const limits = usage("2026-08-06T11:59:00.000Z"); + assert.strictEqual( + matchingAccountUsage("B@Example.com", { + accountIdentity: "b@example.com", + usageLimits: limits, + }), + limits, + ); + assert.isUndefined( + matchingAccountUsage("b@example.com", { + accountIdentity: "a@example.com", + usageLimits: limits, + }), + ); + assert.isUndefined( + matchingAccountUsage("b@example.com", { accountIdentity: undefined, usageLimits: limits }), + ); +}); + +describe("retainUsageLimitsForAccount", () => { it.effect("remembers a successful reading", () => Effect.gen(function* () { yield* TestClock.setTime(NOW); - const lastKnown = yield* Ref.make(undefined); + const lastKnown = yield* Ref.make(undefined); const fresh = usage("2026-08-06T11:59:00.000Z"); - const result = yield* retainUsageLimits(lastKnown, fresh); + const result = yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, fresh); assert.strictEqual(result, fresh); - assert.strictEqual(yield* Ref.get(lastKnown), fresh); + assert.deepStrictEqual(yield* Ref.get(lastKnown), { + identity: ACCOUNT_A, + usageLimits: fresh, + }); }), ); @@ -90,9 +144,12 @@ describe("retainUsageLimits", () => { Effect.gen(function* () { yield* TestClock.setTime(NOW); const previous = usage("2026-08-06T11:59:00.000Z"); - const lastKnown = yield* Ref.make(previous); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: previous, + }); - const result = yield* retainUsageLimits(lastKnown, undefined); + const result = yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, undefined); assert.strictEqual(result, previous); }), @@ -102,9 +159,12 @@ describe("retainUsageLimits", () => { Effect.gen(function* () { yield* TestClock.setTime(NOW); const ancient = usage("2020-01-01T00:00:00.000Z"); - const lastKnown = yield* Ref.make(ancient); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: ancient, + }); - const result = yield* retainUsageLimits(lastKnown, undefined); + const result = yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, undefined); assert.isUndefined(result); // Cleared as well, so it cannot resurface later. @@ -114,9 +174,53 @@ describe("retainUsageLimits", () => { it.effect("has nothing to show before any reading succeeds", () => Effect.gen(function* () { - const lastKnown = yield* Ref.make(undefined); + const lastKnown = yield* Ref.make(undefined); + + assert.isUndefined(yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, undefined)); + }), + ); + + it.effect("clears account A after an authenticated switch to B with a failed read", () => + Effect.gen(function* () { + yield* TestClock.setTime(NOW); + const previous = usage("2026-08-06T11:59:00.000Z"); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: previous, + }); + assert.isUndefined(yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_B, undefined)); + assert.isUndefined(yield* Ref.get(lastKnown)); + assert.isUndefined(yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, undefined)); + }), + ); + + it.effect("does not retain an unidentified account's reading", () => + Effect.gen(function* () { + const lastKnown = yield* Ref.make(undefined); + const fresh = usage("2026-08-06T11:59:00.000Z"); + assert.strictEqual(yield* retainUsageLimitsForAccount(lastKnown, undefined, fresh), fresh); + assert.isUndefined(yield* Ref.get(lastKnown)); + }), + ); - assert.isUndefined(yield* retainUsageLimits(lastKnown, undefined)); + it.effect("hides an A reading during unknown auth, then reuses it only for verified A", () => + Effect.gen(function* () { + yield* TestClock.setTime(NOW); + const previous = usage("2026-08-06T11:59:00.000Z"); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: previous, + }); + assert.isUndefined(yield* retainUsageLimitsForAccount(lastKnown, undefined, undefined)); + assert.deepStrictEqual(yield* Ref.get(lastKnown), { + identity: ACCOUNT_A, + usageLimits: previous, + }); + assert.strictEqual( + yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_A, undefined), + previous, + ); + assert.isUndefined(yield* retainUsageLimitsForAccount(lastKnown, ACCOUNT_B, undefined)); }), ); }); @@ -124,9 +228,10 @@ describe("retainUsageLimits", () => { describe("retainSnapshotUsageLimits", () => { const snapshot = (input: { readonly auth: "authenticated" | "unauthenticated" | "unknown"; + readonly email?: string; readonly usageLimits?: ServerProviderUsageLimits; }) => ({ - auth: { status: input.auth }, + auth: { status: input.auth, ...(input.email ? { email: input.email } : {}) }, ...(input.usageLimits ? { usageLimits: input.usageLimits } : {}), }); @@ -136,11 +241,14 @@ describe("retainSnapshotUsageLimits", () => { Effect.gen(function* () { yield* TestClock.setTime(NOW); const previous = usage("2026-08-06T11:59:00.000Z"); - const lastKnown = yield* Ref.make(previous); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: previous, + }); const result = yield* retainSnapshotUsageLimits( lastKnown, - snapshot({ auth: "authenticated" }), + snapshot({ auth: "authenticated", email: "a@example.com" }), ); assert.strictEqual(result.usageLimits, previous); @@ -150,16 +258,20 @@ describe("retainSnapshotUsageLimits", () => { it.effect("returns the probe untouched when it carries its own reading", () => Effect.gen(function* () { yield* TestClock.setTime(NOW); - const lastKnown = yield* Ref.make(undefined); + const lastKnown = yield* Ref.make(undefined); const probed = snapshot({ auth: "authenticated", + email: "a@example.com", usageLimits: usage("2026-08-06T11:59:00.000Z"), }); const result = yield* retainSnapshotUsageLimits(lastKnown, probed); assert.strictEqual(result, probed); - assert.strictEqual(yield* Ref.get(lastKnown), probed.usageLimits); + assert.deepStrictEqual(yield* Ref.get(lastKnown), { + identity: ACCOUNT_A, + usageLimits: probed.usageLimits, + }); }), ); @@ -168,9 +280,10 @@ describe("retainSnapshotUsageLimits", () => { it.effect("clears the reading when the account signs out", () => Effect.gen(function* () { yield* TestClock.setTime(NOW); - const lastKnown = yield* Ref.make( - usage("2026-08-06T11:59:00.000Z"), - ); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: usage("2026-08-06T11:59:00.000Z"), + }); const result = yield* retainSnapshotUsageLimits( lastKnown, @@ -181,4 +294,20 @@ describe("retainSnapshotUsageLimits", () => { assert.strictEqual(yield* Ref.get(lastKnown), undefined); }), ); + + it.effect("does not attach account A's reading to an authenticated B snapshot", () => + Effect.gen(function* () { + yield* TestClock.setTime(NOW); + const lastKnown = yield* Ref.make({ + identity: ACCOUNT_A, + usageLimits: usage("2026-08-06T11:59:00.000Z"), + }); + const result = yield* retainSnapshotUsageLimits( + lastKnown, + snapshot({ auth: "authenticated", email: "b@example.com" }), + ); + assert.isUndefined(result.usageLimits); + assert.isUndefined(yield* Ref.get(lastKnown)); + }), + ); }); diff --git a/apps/server/src/provider/providerUsageRetention.ts b/apps/server/src/provider/providerUsageRetention.ts index b9e1b5a3c3..5fb3384d2e 100644 --- a/apps/server/src/provider/providerUsageRetention.ts +++ b/apps/server/src/provider/providerUsageRetention.ts @@ -58,32 +58,89 @@ export function isRetainedUsageFresh(input: { * number beats no number when deciding where to send work, and `checkedAt` * travels with it so the client can say how old it is. */ -export function retainUsageLimits( - lastKnown: Ref.Ref, +export interface AccountUsageReading { + readonly identity: string; + readonly usageLimits: ServerProviderUsageLimits; +} + +/** Only stable, affirmative account identity can authorize a retained reading. */ +export function authenticatedUsageIdentity(auth: ServerProviderAuth): string | undefined { + if (auth.status !== "authenticated") return undefined; + // Account id and live CLI email are independent signals. Keeping both + // prevents a changing auth.json from assigning B's CLI read to A's id. + if (auth.accountId && auth.email) { + return `account:${auth.accountId}|email:${auth.email.trim().toLowerCase()}`; + } + if (auth.accountId) return `account:${auth.accountId}`; + if (auth.email) return `email:${auth.email.trim().toLowerCase()}`; + return undefined; +} + +/** A direct post-reset read may overlay only the same authenticated account. */ +export function usageReadingForAuth( + reading: AccountUsageReading | undefined, + auth: ServerProviderAuth, +): ServerProviderUsageLimits | undefined { + const identity = authenticatedUsageIdentity(auth); + return identity && reading?.identity === identity ? reading.usageLimits : undefined; +} + +/** A separate OAuth usage read must agree with the authenticated account. */ +export function matchingAccountUsage( + expectedEmail: string | undefined, + result: + | { + readonly accountIdentity: string | undefined; + readonly usageLimits: ServerProviderUsageLimits | undefined; + } + | undefined, +): ServerProviderUsageLimits | undefined { + if (!result) return undefined; + if ( + expectedEmail && + result.accountIdentity?.trim().toLowerCase() !== expectedEmail.trim().toLowerCase() + ) { + return undefined; + } + return result.usageLimits; +} + +export function retainUsageLimitsForAccount( + lastKnown: Ref.Ref, + identity: string | undefined, usageLimits: ServerProviderUsageLimits | undefined, ): Effect.Effect { return Effect.gen(function* () { if (usageLimits) { - yield* RefModule.set(lastKnown, usageLimits); + if (identity) yield* RefModule.set(lastKnown, { identity, usageLimits }); return usageLimits; } const retained = yield* RefModule.get(lastKnown); if (!retained) return undefined; + // A temporarily unknown auth result cannot authorize publication, but it + // also cannot prove the signed-in account changed. Hold the reading for a + // later matching identity without showing it during the unknown interval. + if (identity === undefined) return undefined; + if (retained.identity !== identity) { + yield* RefModule.set(lastKnown, undefined); + return undefined; + } const nowMs = yield* Effect.clockWith((clock) => clock.currentTimeMillis); - if (isRetainedUsageFresh({ checkedAt: retained.checkedAt, nowMs })) return retained; + if (isRetainedUsageFresh({ checkedAt: retained.usageLimits.checkedAt, nowMs })) { + return retained.usageLimits; + } yield* RefModule.set(lastKnown, undefined); return undefined; }); } /** - * Apply {@link retainUsageLimits} to a whole probed snapshot. + * Apply account-scoped retention to a whole probed snapshot. * - * A signed-out account has no capacity to retain, so its reading is cleared - * rather than carried; anything else keeps the last good reading through a - * failed or timed-out read. Codex reads its windows over the network inside - * the status probe, where one slow answer would otherwise blank the gauge - * until the next poll. + * A signed-out account has no capacity to retain. A failed or timed-out read + * may reuse the last reading only while the authenticated account identity + * remains the same. Codex reads its windows over the network inside the + * status probe, where one slow answer would otherwise blank the gauge. */ export function retainSnapshotUsageLimits< Snapshot extends { @@ -91,7 +148,7 @@ export function retainSnapshotUsageLimits< readonly usageLimits?: ServerProviderUsageLimits | undefined; }, >( - lastKnown: Ref.Ref, + lastKnown: Ref.Ref, snapshot: Snapshot, ): Effect.Effect { return Effect.gen(function* () { @@ -100,7 +157,11 @@ export function retainSnapshotUsageLimits< const { usageLimits: _usageLimits, ...withoutUsage } = snapshot; return withoutUsage as Snapshot; } - const usageLimits = yield* retainUsageLimits(lastKnown, snapshot.usageLimits); + const usageLimits = yield* retainUsageLimitsForAccount( + lastKnown, + authenticatedUsageIdentity(snapshot.auth), + snapshot.usageLimits, + ); if (usageLimits === snapshot.usageLimits) return snapshot; return usageLimits ? { ...snapshot, usageLimits } : snapshot; }); diff --git a/apps/server/src/pullRequest/GitHubPullRequestCli.test.ts b/apps/server/src/pullRequest/GitHubPullRequestCli.test.ts index 8416083061..ed8710ee53 100644 --- a/apps/server/src/pullRequest/GitHubPullRequestCli.test.ts +++ b/apps/server/src/pullRequest/GitHubPullRequestCli.test.ts @@ -1,5 +1,6 @@ import { afterEach, assert, expect, it, vi } from "@effect/vitest"; import * as Effect from "effect/Effect"; +import * as Fiber from "effect/Fiber"; import * as Layer from "effect/Layer"; import * as Schema from "effect/Schema"; import * as TestClock from "effect/testing/TestClock"; @@ -193,77 +194,137 @@ afterEach(() => { }); layer("GitHubPullRequestCli.layer", (it) => { - it.effect("reads linked pull request status with the overview fields in one request", () => + it.effect("reads linked pull requests on one host together, filed back by position", () => Effect.gen(function* () { + const node = (number: number) => ({ + number, + title: `Pull request ${number}`, + url: `https://github.com/acme/web/pull/${number}`, + author: { __typename: "User", login: "octocat", name: "Octo Cat", avatarUrl: null }, + baseRefName: "main", + headRefName: `feat/${number}`, + state: "OPEN", + isDraft: false, + mergeable: "MERGEABLE", + reviewDecision: null, + latestReviews: { nodes: [{ state: "APPROVED", author: { login: "reviewer" } }] }, + additions: 12, + deletions: 3, + changedFiles: 2, + updatedAt: "2026-08-24T12:34:56.000Z", + mergedAt: null, + closedAt: null, + commits: { nodes: [{ commit: { statusCheckRollup: { state: "SUCCESS" } } }] }, + }); mockedExecute.mockReturnValueOnce( Effect.succeed( output( // @effect-diagnostics-next-line preferSchemaOverJson:off JSON.stringify({ - number: 7, - title: "Reuse the summary", - url: "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/acme/web/pull/7", - author: { login: "octocat", name: "Octo Cat" }, - baseRefName: "main", - headRefName: "feat/summary", - state: "OPEN", - isDraft: false, - mergeable: "MERGEABLE", - reviewDecision: "APPROVED", - additions: 12, - deletions: 3, - changedFiles: 2, - createdAt: "2026-08-20T00:00:00.000Z", - updatedAt: "2026-08-24T12:34:56.000Z", - reviewRequests: [], - labels: [], - statusCheckRollup: [ - { __typename: "CheckRun", status: "COMPLETED", conclusion: "SUCCESS", name: "ci" }, - ], - body: "", + data: { s0: { pullRequest: node(7) }, s1: { pullRequest: node(8) } }, }), ), ), ); const cli = yield* GitHubPullRequestCli.GitHubPullRequestCli; - const summary = yield* cli.getPullRequestSummary({ - cwd: "/w", - repository: "acme/web", - host: "github.com", - number: 7, - }); + const reads = yield* Effect.forEach( + [7, 8], + (number) => + cli.getPullRequestSummary({ + cwd: "/w", + repository: "acme/web", + host: "github.com", + number, + }), + { concurrency: "unbounded" }, + ).pipe(Effect.forkChild); + yield* TestClock.adjust("10 millis"); + const [seven, eight] = yield* Fiber.join(reads); assert.deepStrictEqual( { - number: summary.number, - state: summary.state, - headBranch: summary.headBranch, - isDraft: summary.isDraft, - author: summary.author?.login, - additions: summary.additions, - deletions: summary.deletions, - changedFiles: summary.changedFiles, - reviewDecision: summary.reviewDecision, - checksState: summary.checksState, - mergeability: summary.mergeability, + number: seven?.number, + state: seven?.state, + headBranch: seven?.headBranch, + author: seven?.author?.login, + changedFiles: seven?.changedFiles, + reviewDecision: seven?.reviewDecision, + checksState: seven?.checksState, + mergeability: seven?.mergeability, }, { number: 7, state: "open", - headBranch: "feat/summary", - isDraft: false, + headBranch: "feat/7", author: "octocat", - additions: 12, - deletions: 3, changedFiles: 2, reviewDecision: "approved", checksState: "passing", mergeability: "mergeable", }, ); + assert.strictEqual(eight?.headBranch, "feat/8"); expect(mockedExecute).toHaveBeenCalledOnce(); - expect(mockedExecute.mock.calls[0]?.[0]?.args).toEqual([ + const document = callAt(0).args.at(-1) ?? ""; + expect(document).toContain( + 's0: repository(owner: "acme", name: "web") { pullRequest(number: 7)', + ); + expect(document).toContain("pullRequest(number: 8)"); + }), + ); + + it.effect("reads a pull request the batch said nothing about on its own", () => + Effect.gen(function* () { + mockedExecute + .mockReturnValueOnce(Effect.succeed(output('{"data":{"s0":{"pullRequest":null}}}'))) + .mockReturnValueOnce( + Effect.succeed( + output( + // @effect-diagnostics-next-line preferSchemaOverJson:off + JSON.stringify({ + number: 7, + title: "Reuse the summary", + url: "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/acme/web/pull/7", + author: { login: "octocat", name: "Octo Cat" }, + baseRefName: "main", + headRefName: "feat/summary", + state: "OPEN", + isDraft: false, + mergeable: "MERGEABLE", + reviewDecision: "APPROVED", + additions: 12, + deletions: 3, + changedFiles: 2, + createdAt: "2026-08-20T00:00:00.000Z", + updatedAt: "2026-08-24T12:34:56.000Z", + reviewRequests: [], + labels: [], + statusCheckRollup: [ + { + __typename: "CheckRun", + status: "COMPLETED", + conclusion: "SUCCESS", + name: "ci", + }, + ], + body: "", + }), + ), + ), + ); + const cli = yield* GitHubPullRequestCli.GitHubPullRequestCli; + + const read = yield* cli + .getPullRequestSummary({ cwd: "/w", repository: "acme/web", host: "github.com", number: 7 }) + .pipe(Effect.forkChild); + yield* TestClock.adjust("10 millis"); + const summary = yield* Fiber.join(read); + + assert.strictEqual(summary.headBranch, "feat/summary"); + assert.strictEqual(summary.checksState, "passing"); + assert.strictEqual(mockedExecute.mock.calls.length, 2); + expect(callAt(1).args).toEqual([ "pr", "view", "7", @@ -272,7 +333,46 @@ layer("GitHubPullRequestCli.layer", (it) => { "--json", expect.stringContaining("statusCheckRollup"), ]); - expect(mockedGetPullRequest).not.toHaveBeenCalled(); + }), + ); + + it.effect("does not fan out summary reads after a rejected GraphQL batch", () => + Effect.gen(function* () { + for (const response of [ + Effect.succeed(output('{"data":null,"errors":[{"message":"API rate limit exceeded"}]}')), + Effect.fail( + new GitHubCli.GitHubCliRateLimitError({ + command: "gh", + cwd: "/w", + cause: new Error("API rate limit exceeded"), + }), + ), + ]) { + mockedExecute.mockReset().mockReturnValueOnce(response); + const cli = yield* GitHubPullRequestCli.GitHubPullRequestCli; + const reads = yield* Effect.all([ + cli + .getPullRequestSummary({ + cwd: "/w", + repository: "acme/web", + host: "github.com", + number: 7, + }) + .pipe(Effect.forkChild), + cli + .getPullRequestSummary({ + cwd: "/w", + repository: "acme/web", + host: "github.com", + number: 8, + }) + .pipe(Effect.forkChild), + ]); + yield* TestClock.adjust("10 millis"); + const exits = yield* Effect.forEach(reads, (read) => Fiber.await(read)); + expect(exits.every((exit) => exit._tag === "Failure")).toBe(true); + expect(mockedExecute).toHaveBeenCalledOnce(); + } }), ); diff --git a/apps/server/src/pullRequest/GitHubPullRequestCli.ts b/apps/server/src/pullRequest/GitHubPullRequestCli.ts index d90d7378fa..b6c9dae750 100644 --- a/apps/server/src/pullRequest/GitHubPullRequestCli.ts +++ b/apps/server/src/pullRequest/GitHubPullRequestCli.ts @@ -1,7 +1,12 @@ import { runGitHubStackAction, type GitHubStackActionError } from "./githubStackActions.ts"; +import * as Cause from "effect/Cause"; import * as Context from "effect/Context"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; +import * as Exit from "effect/Exit"; +import * as Option from "effect/Option"; +import * as Request from "effect/Request"; +import * as RequestResolver from "effect/RequestResolver"; import * as Result from "effect/Result"; import * as Schema from "effect/Schema"; import { @@ -36,6 +41,7 @@ import { decodeActorAvatarsJson, decodePullRequestActivityJson, decodePullRequestDetailJson, + type GitHubPullRequestSummary, decodePullRequestFilesJson, decodePullRequestHeadsJson, decodePullRequestListJson, @@ -43,6 +49,8 @@ import { decodePullRequestSearchJson, decodePullRequestStacksJson, decodePullRequestStatsJson, + decodePullRequestSummariesJson, + GitHubSummaryBatchUnavailableError, decodeReactionSubjectScopeJson, decodeRepositoryAccessJson, decodeReviewerCandidatesJson, @@ -53,6 +61,7 @@ import { decodeReviewThreadCommentsJson, decodeReviewThreadsJson, buildPullRequestStatsGraphQlQuery, + buildPullRequestSummariesGraphQlQuery, buildPullRequestStackMembershipsGraphQlQuery, decodePullRequestStackMembershipsJson, encodeGraphQlRequestJson, @@ -393,6 +402,23 @@ export interface GitHubPullRequestStat { */ const STAT_ALIASES_PER_REQUEST = 25; const STAT_REQUEST_CONCURRENCY = 4; +/** + * How long a summary read waits for company. The background sync asks for every linked pull + * request at once, and each read reaches the resolver after its own cache check, so a batch + * needs a moment longer than one scheduler tick to gather them. + */ +const SUMMARY_BATCH_WINDOW = "10 millis"; + +class PullRequestSummaryRead extends Request.Class< + { + readonly cwd: string; + readonly repository: string; + readonly host: string; + readonly number: number; + }, + ProviderChangeRequestSummary, + GitHubPullRequestCliError +> {} export interface GitHubPullRequestSearchBatch { /** Rows across every repository asked for, newest update first, each naming its own. */ @@ -1467,6 +1493,139 @@ export const make = Effect.gen(function* () { ), ); + // One `gh pr view` either way; asking for the detail fields costs nothing extra and hands + // the thread overview its author, diff stat, review decision and checks in the same read. + const viewPullRequestSummary = (input: PullRequestSummaryRead) => + github + .execute({ + cwd: input.cwd, + args: [ + "pr", + "view", + String(input.number), + ...repositoryArgs(input), + "--json", + PULL_REQUEST_DETAIL_JSON_FIELDS, + ], + }) + .pipe( + Effect.flatMap((result) => { + const decoded = decodePullRequestDetailJson(result.stdout.trim()); + if (!Result.isSuccess(decoded)) { + return Effect.fail( + new GitHubPullRequestReadError({ + command: "gh", + cwd: input.cwd, + operation: "getPullRequestSummary", + cause: decoded.failure, + }), + ); + } + const detail = decoded.success; + return Effect.succeed({ + number: detail.number, + title: detail.title, + url: detail.url, + headBranch: detail.headBranch, + baseBranch: detail.baseBranch, + state: detail.state, + updatedAt: detail.updatedAt, + closedAt: detail.closedAt ?? null, + mergedAt: detail.mergedAt ?? null, + isDraft: detail.isDraft, + author: detail.author, + additions: detail.additions, + deletions: detail.deletions, + changedFiles: detail.changedFiles, + reviewDecision: detail.reviewDecision, + checksState: detail.checksState, + mergeability: detail.mergeability, + }); + }), + ); + + /** + * Summaries asked for together, on one host under one credential, share aliased GraphQL reads + * of twenty-five: the background sync reads every linked pull request each minute, and one + * `gh pr view` apiece is most of what it spends. Whatever the batch cannot answer — a selector + * GraphQL cannot address, a pull request GitHub returned nothing for — is read on its own. + */ + const summaryResolver = RequestResolver.makeGrouped({ + key: ({ request, context }) => + JSON.stringify([ + request.host.toLowerCase(), + Context.getOrElse(context, GitHubCli.PinnedGitHubCredential, () => null) + ?.credentialFingerprint ?? null, + Context.getOrElse(context, SourceControlRateLimit.CredentialScope, () => ""), + ]), + resolver: (entries) => { + const [first] = entries; + const batchable = entries.filter( + (entry) => buildPullRequestSummariesGraphQlQuery([entry.request]) !== null, + ); + const query = buildPullRequestSummariesGraphQlQuery(batchable.map((entry) => entry.request)); + const batched = + query === null + ? Effect.succeed(new Map()) + : graphqlRead({ + cwd: first.request.cwd, + host: first.request.host, + operation: "getPullRequestSummary", + query, + decode: decodePullRequestSummariesJson, + }); + return batched.pipe( + // A GraphQL error anywhere fails the whole document — one repository gone or out of + // reach — so a batch that could not be read leaves every entry to its own read. A paused + // budget is the exception: reading one at a time would only spend what is being saved. + Effect.catchCauseIf( + (cause) => + !Cause.hasInterruptsOnly(cause) && + !Cause.findErrorOption(cause).pipe( + Option.exists( + (error) => + error._tag === "SourceControlRateLimitPausedError" || + error._tag === "GitHubCliRateLimitError" || + (error._tag === "GitHubPullRequestReadError" && + error.cause instanceof GitHubSummaryBatchUnavailableError), + ), + ), + (cause) => + Effect.logDebug("batched pull request summary read failed", { cause }).pipe( + Effect.as(new Map()), + ), + ), + Effect.flatMap((summaries) => { + const unanswered = entries.filter((entry) => { + const summary = summaries.get(batchable.indexOf(entry)); + if (summary === undefined) return true; + entry.completeUnsafe(Exit.succeed(summary)); + return false; + }); + return Effect.forEach( + unanswered, + (entry) => + viewPullRequestSummary(entry.request).pipe( + Effect.exit, + Effect.map((exit) => entry.completeUnsafe(exit)), + ), + { concurrency: STAT_REQUEST_CONCURRENCY, discard: true }, + ); + }), + Effect.catchCause((cause) => + Effect.sync(() => { + for (const entry of entries) entry.completeUnsafe(Exit.failCause(cause)); + }), + ), + ); + }, + }).pipe( + RequestResolver.setDelay(SUMMARY_BATCH_WINDOW), + RequestResolver.batchN(STAT_ALIASES_PER_REQUEST), + ); + const getPullRequestSummary: GitHubPullRequestCli["Service"]["getPullRequestSummary"] = (input) => + Effect.request(new PullRequestSummaryRead(input), summaryResolver); + return GitHubPullRequestCli.of({ getViewerLogin: (input) => github.execute({ cwd: input.cwd, args: ["api", "user", "--jq", ".login"] }).pipe( @@ -1678,56 +1837,7 @@ export const make = Effect.gen(function* () { ).pipe(Effect.map((results) => results.flat())); }, - // One `gh pr view` either way; asking for the detail fields costs nothing extra and hands - // the thread overview its author, diff stat, review decision and checks in the same read. - getPullRequestSummary: (input) => - github - .execute({ - cwd: input.cwd, - args: [ - "pr", - "view", - String(input.number), - ...repositoryArgs(input), - "--json", - PULL_REQUEST_DETAIL_JSON_FIELDS, - ], - }) - .pipe( - Effect.flatMap((result) => { - const decoded = decodePullRequestDetailJson(result.stdout.trim()); - if (!Result.isSuccess(decoded)) { - return Effect.fail( - new GitHubPullRequestReadError({ - command: "gh", - cwd: input.cwd, - operation: "getPullRequestSummary", - cause: decoded.failure, - }), - ); - } - const detail = decoded.success; - return Effect.succeed({ - number: detail.number, - title: detail.title, - url: detail.url, - headBranch: detail.headBranch, - baseBranch: detail.baseBranch, - state: detail.state, - updatedAt: detail.updatedAt, - closedAt: detail.closedAt ?? null, - mergedAt: detail.mergedAt ?? null, - isDraft: detail.isDraft, - author: detail.author, - additions: detail.additions, - deletions: detail.deletions, - changedFiles: detail.changedFiles, - reviewDecision: detail.reviewDecision, - checksState: detail.checksState, - mergeability: detail.mergeability, - }); - }), - ), + getPullRequestSummary, getPullRequestDetail, listWorkflowRunsRequiringApproval, diff --git a/apps/server/src/pullRequest/PullRequestService.test.ts b/apps/server/src/pullRequest/PullRequestService.test.ts index a4f866b59a..e84703f241 100644 --- a/apps/server/src/pullRequest/PullRequestService.test.ts +++ b/apps/server/src/pullRequest/PullRequestService.test.ts @@ -1081,8 +1081,13 @@ it.effect("publishes a merge for immediate settlement only after host confirmati // Queueing succeeds while the host still reports an open PR. yield* service.runAction({ ...reference, action: "merge" }); + const queuedRefresh = Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)); confirmationFails = true; yield* service.runAction({ ...reference, action: "merge" }); + assert.isAbove( + Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)), + queuedRefresh, + ); confirmationFails = false; state = "merged"; yield* TestClock.setTime(Date.parse(mergedAt)); @@ -1101,6 +1106,53 @@ it.effect("publishes a merge for immediate settlement only after host confirmati ), ); +it.effect("refreshes every reader before a queued merge confirmation finishes", () => + Effect.scoped( + Effect.gen(function* () { + const confirmationStarted = yield* Deferred.make(); + const confirm = yield* Deferred.make(); + const reference = { projectId: "p1" as ProjectId, repository: "acme/web", number: 1 }; + const service = yield* makeService({ + projects: [ + project({ id: "p1", title: "web", workspaceRoot: "/a", repository: "acme/web" }), + ], + providers: [ + fakeProvider("github", { + getChangeRequestSummary: () => + Effect.gen(function* () { + yield* Deferred.succeed(confirmationStarted, undefined); + yield* Deferred.await(confirm); + return changeRequest(1, "2026-09-16T00:00:00.000Z"); + }), + }), + ], + }); + const merges = yield* service.subscribeMerges; + const observedMerge = yield* Stream.runHead(merges).pipe( + Effect.forkChild({ startImmediately: true }), + ); + const readers = yield* Effect.forEach([0, 1], () => + Stream.runHead(service.subscribeRefreshes).pipe( + Effect.forkChild({ startImmediately: true }), + ), + ); + const action = yield* service + .runAction({ ...reference, action: "merge" }) + .pipe(Effect.forkChild({ startImmediately: true })); + yield* Deferred.await(confirmationStarted); + const revisions = yield* Effect.forEach(readers, (reader) => + Fiber.join(reader).pipe(Effect.map(Option.getOrThrow)), + ); + assert.isAbove(revisions[0]!, 0); + assert.strictEqual(revisions[0], revisions[1]); + assert.isUndefined(action.pollUnsafe()); + yield* Deferred.succeed(confirm, undefined); + yield* Fiber.join(action); + assert.isUndefined(observedMerge.pollUnsafe()); + }), + ), +); + it.effect("refuses an action this viewer may not take, and says what access it takes", () => Effect.gen(function* () { let ran: string | null = null; @@ -2254,10 +2306,16 @@ it.effect("invalidates the cached activity after reacting, like the other mutati ], }); + yield* service.refreshAfterTurn(reference.projectId); + const previousRefresh = Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)); yield* service.activity(reference); assert.strictEqual(activityCalls, 1); yield* service.setReaction({ ...reference, content: "heart", reacted: true }); + assert.isAbove( + Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)), + previousRefresh, + ); yield* service.activity(reference); assert.strictEqual(activityCalls, 2); @@ -3192,31 +3250,106 @@ it.effect("explicit and turn invalidations make the next listing ask the host ag }), ); -it.effect("a mutation makes the next listing ask the host again, with no client asking", () => - Effect.gen(function* () { - let hostCalls = 0; - const service = yield* makeService({ - projects: [project({ id: "p1", title: "web", workspaceRoot: "/a", repository: "acme/web" })], - providers: [ - fakeProvider("github", { - listChangeRequests: () => { - hostCalls += 1; - return Effect.succeed({ items: [], truncated: false, continues: false }); - }, - }), - ], - }); +it.effect("close and reopen notify subscribed readers after invalidating their cached state", () => + Effect.scoped( + Effect.gen(function* () { + let hostCalls = 0; + let state: "open" | "closed" = "open"; + const reference = { projectId: "p1" as ProjectId, repository: "acme/web", number: 1 }; + const service = yield* makeService({ + projects: [ + project({ id: "p1", title: "web", workspaceRoot: "/a", repository: "acme/web" }), + ], + providers: [ + fakeProvider("github", { + getChangeRequestSummary: () => + Effect.succeed({ ...changeRequest(1, "2026-09-16T00:00:00.000Z"), state }), + runAction: (input) => + Effect.sync(() => { + state = input.action === "close" ? "closed" : "open"; + }), + listChangeRequests: () => { + hostCalls += 1; + return Effect.succeed({ items: [], truncated: false, continues: false }); + }, + }), + ], + }); - yield* service.list({ state: "open" }); - yield* service.runAction({ - projectId: "p1" as ProjectId, - repository: "acme/web", - number: 1, - action: "close", - }); - yield* service.list({ state: "open" }); - assert.strictEqual(hostCalls, 2); - }), + yield* service.refreshAfterTurn(reference.projectId); + yield* service.list({ state: "open" }); + assert.strictEqual((yield* service.summary(reference)).state, "open"); + for (const action of ["close", "reopen"] as const) { + const refreshed = yield* service.subscribeRefreshes.pipe( + Stream.drop(1), + Stream.take(1), + Stream.mapEffect(() => + Effect.gen(function* () { + yield* service.list({ state: "open" }); + return yield* service.summary(reference); + }), + ), + Stream.runHead, + Effect.forkChild({ startImmediately: true }), + ); + yield* service.runAction({ ...reference, action }); + assert.strictEqual( + Option.getOrThrow(yield* Fiber.join(refreshed)).state, + action === "close" ? "closed" : "open", + ); + } + assert.strictEqual(hostCalls, 3); + }), + ), +); + +it.effect("explicit invalidation refreshes origin readers after a routed host mutation", () => + Effect.scoped( + Effect.gen(function* () { + let state: "open" | "closed" = "open"; + const reference = { projectId: "p1" as ProjectId, repository: "acme/web", number: 1 }; + const service = yield* makeService({ + projects: [ + project({ id: "p1", title: "web", workspaceRoot: "/a", repository: "acme/web" }), + ], + providers: [ + fakeProvider("github", { + getChangeRequestSummary: () => + Effect.succeed({ ...changeRequest(1, "2026-09-16T00:00:00.000Z"), state }), + }), + ], + }); + yield* service.refreshAfterTurn(reference.projectId); + let revision = Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)); + // The sync reactor invalidates before reading; it must not notify itself again. + yield* service.invalidate({ reference }); + assert.strictEqual( + Option.getOrThrow(yield* Stream.runHead(service.subscribeRefreshes)), + revision, + ); + assert.strictEqual((yield* service.summary(reference)).state, "open"); + + for (const nextState of ["closed", "open"] as const) { + const refreshed = yield* service.subscribeRefreshes.pipe( + Stream.drop(1), + Stream.take(1), + Stream.mapEffect((nextRevision) => + service + .summary(reference) + .pipe(Effect.map((summary) => ({ revision: nextRevision, state: summary.state }))), + ), + Stream.runHead, + Effect.forkChild({ startImmediately: true }), + ); + state = nextState; + yield* service.invalidate({ reference }, { notifyReaders: true }); + const result = Option.getOrThrow(yield* Fiber.join(refreshed)); + assert.strictEqual(result.state, nextState); + assert.isAbove(result.revision, revision); + revision = result.revision; + } + }), + ), ); it.effect("does not cache a failed listing", () => diff --git a/apps/server/src/pullRequest/PullRequestService.ts b/apps/server/src/pullRequest/PullRequestService.ts index c61ce781a8..ff53821727 100644 --- a/apps/server/src/pullRequest/PullRequestService.ts +++ b/apps/server/src/pullRequest/PullRequestService.ts @@ -214,7 +214,10 @@ export class PullRequestService extends Context.Service< readonly setLabels: ( input: PullRequestLabelChangeInput, ) => Effect.Effect; - readonly invalidate: (input: PullRequestInvalidateInput) => Effect.Effect; + readonly invalidate: ( + input: PullRequestInvalidateInput, + options?: { readonly notifyReaders?: boolean }, + ) => Effect.Effect; } >()("t3/pullRequest/PullRequestService") {} @@ -2732,10 +2735,12 @@ export const make = Effect.gen(function* () { return { stats: [...held, ...result.stats] }; }); - const invalidate: PullRequestService["Service"]["invalidate"] = (input) => { + const invalidate: PullRequestService["Service"]["invalidate"] = Effect.fn( + "PullRequestService.invalidate", + )(function* (input, options) { const reference = input.reference; if (reference !== undefined) { - return canonicalRef(reference).pipe( + yield* canonicalRef(reference).pipe( Effect.flatMap((ref) => readCache .invalidate(refScope(ref)) @@ -2743,12 +2748,15 @@ export const make = Effect.gen(function* () { ), Effect.ignore, ); - } - return Effect.sync(() => { + } else { listingsEpoch = ++epochCounter; viewersByHost.clear(); - }).pipe(Effect.andThen(Cache.invalidateAll(viewerFlights))); - }; + yield* Cache.invalidateAll(viewerFlights); + } + if (options?.notifyReaders) { + yield* SubscriptionRef.set(pullRequestRefreshes, ++epochCounter); + } + }); const refreshAfterTurn: PullRequestService["Service"]["refreshAfterTurn"] = (projectId) => Effect.suspend(() => { @@ -2767,9 +2775,7 @@ export const make = Effect.gen(function* () { .pipe(Effect.andThen(SubscriptionRef.set(pullRequestRefreshes, listingsEpoch))); }); - // A mutation's own client re-reads right after it, and every other client's next read must - // see the action too — so a write forgets the change request it touched and the listings its - // state change reorders, for everyone, without any client asking. + // Invalidate before notifying every client so mounted readers immediately fetch the edit. const invalidatedByMutation = ( method: (input: I) => Effect.Effect, @@ -2787,6 +2793,7 @@ export const make = Effect.gen(function* () { }), ), ); + yield* SubscriptionRef.set(pullRequestRefreshes, listingsEpoch); }); const runActionAndInvalidate: PullRequestService["Service"]["runAction"] = Effect.fn( "PullRequestService.runActionAndInvalidate", @@ -2798,6 +2805,7 @@ export const make = Effect.gen(function* () { ); bumpRefEpoch({ ...ref, repository }); listingsEpoch = ++epochCounter; + yield* SubscriptionRef.set(pullRequestRefreshes, listingsEpoch); if (input.action === "merge") { // A successful merge action can merely enqueue the PR or enable auto-merge. const confirmed = yield* summaryUncached({ ...input, repository }).pipe( diff --git a/apps/server/src/pullRequest/gitHubPullRequestJson.test.ts b/apps/server/src/pullRequest/gitHubPullRequestJson.test.ts index 2c3ac6f7c0..801c439a7d 100644 --- a/apps/server/src/pullRequest/gitHubPullRequestJson.test.ts +++ b/apps/server/src/pullRequest/gitHubPullRequestJson.test.ts @@ -2,6 +2,8 @@ import * as Result from "effect/Result"; import { describe, expect, it } from "vite-plus/test"; import { + buildPullRequestSummariesGraphQlQuery, + decodePullRequestSummariesJson, buildReviewSubmissionJson, buildPullRequestStackMembershipsGraphQlQuery, decodePullRequestStackMembershipsJson, @@ -1708,3 +1710,54 @@ describe("pull request stack membership batches", () => { ); }); }); + +describe("batched pull request summaries", () => { + it("refuses a repository GraphQL cannot address, rather than writing it into the document", () => { + expect( + buildPullRequestSummariesGraphQlQuery([{ repository: 'acme/web") { x } #', number: 1 }]), + ).toBeNull(); + expect( + buildPullRequestSummariesGraphQlQuery([{ repository: "acme/web", number: 0 }]), + ).toBeNull(); + expect(buildPullRequestSummariesGraphQlQuery([])).toBeNull(); + }); + + it("files each answer by its alias and skips what GitHub or the decoder could not give", () => { + const decoded = decodePullRequestSummariesJson( + JSON.stringify({ + data: { + s0: { + pullRequest: { + number: 7, + title: "Merged", + url: "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/acme/web/pull/7", + author: { __typename: "Bot", login: "renovate", avatarUrl: "https://a/r.png" }, + headRefName: "feat/seven", + baseRefName: "main", + state: "MERGED", + mergedAt: "2026-08-24T00:00:00Z", + closedAt: "2026-08-24T00:00:00Z", + updatedAt: "2026-08-24T00:00:00Z", + commits: { nodes: [{ commit: { statusCheckRollup: { state: "FAILURE" } } }] }, + }, + }, + s1: { pullRequest: null }, + s2: { pullRequest: { number: 9 } }, + rateLimit: { cost: 1 }, + }, + }), + ); + expect(Result.isSuccess(decoded)).toBe(true); + if (!Result.isSuccess(decoded)) return; + expect([...decoded.success.keys()]).toEqual([0]); + expect(decoded.success.get(0)).toMatchObject({ + number: 7, + state: "merged", + mergedAt: "2026-08-24T00:00:00Z", + author: { login: "renovate", isBot: true }, + checksState: "failing", + mergeability: "unknown", + additions: 0, + }); + }); +}); diff --git a/apps/server/src/pullRequest/gitHubPullRequestJson.ts b/apps/server/src/pullRequest/gitHubPullRequestJson.ts index 1ebe34ae27..f45416bc91 100644 --- a/apps/server/src/pullRequest/gitHubPullRequestJson.ts +++ b/apps/server/src/pullRequest/gitHubPullRequestJson.ts @@ -1725,6 +1725,139 @@ export function decodePullRequestStatsJson( return Result.succeed(stats); } +/** + * The fields a linked thread keeps current, for many pull requests in one aliased read. Same + * shape as the search row where the two overlap: the checks arrive as GitHub's one-word rollup + * rather than the whole check list `gh pr view` hands back, which is what keeps a batch cheap. + */ +const PULL_REQUEST_SUMMARY_SELECTION = + "number title url state isDraft mergeable reviewDecision additions deletions changedFiles " + + "updatedAt mergedAt closedAt headRefName baseRefName " + + "author { __typename login avatarUrl ... on User { name } } " + + "latestReviews(first: 20) { nodes { state author { login } } } " + + "commits(last: 1) { nodes { commit { statusCheckRollup { state } } } }"; + +/** + * Summaries for pull requests anywhere on one host, one aliased lookup each. Checked and written + * into the document the way `buildPullRequestStatsGraphQlQuery` does, so null means "do not send". + */ +export function buildPullRequestSummariesGraphQlQuery( + changeRequests: ReadonlyArray<{ readonly repository: string; readonly number: number }>, +): string | null { + if (changeRequests.length === 0) return null; + const selections: string[] = []; + for (const [index, changeRequest] of changeRequests.entries()) { + const [owner, name, ...rest] = changeRequest.repository.trim().split("/"); + if (rest.length > 0 || owner === undefined || name === undefined) return null; + if (!REPOSITORY_PART.test(owner) || !REPOSITORY_PART.test(name)) return null; + if (!Number.isSafeInteger(changeRequest.number) || changeRequest.number <= 0) return null; + selections.push( + ` s${index}: repository(owner: "${owner}", name: "${name}") { pullRequest(number: ${changeRequest.number}) { ${PULL_REQUEST_SUMMARY_SELECTION} } }`, + ); + } + return `query PullRequestSummaries {\n${selections.join("\n")}\n}`; +} + +const RawSummarySchema = Schema.Struct({ + ...RawSearchItemSchema.fields, + changedFiles: Schema.optional(Schema.NullOr(Schema.Int)), + additions: Schema.optional(Schema.NullOr(Schema.Int)), + deletions: Schema.optional(Schema.NullOr(Schema.Int)), + closedAt: Schema.optional(Schema.NullOr(Schema.String)), + createdAt: Schema.optional(Schema.String), +}); +const decodeSummaries = decodeJsonResult( + Schema.Struct({ + errors: Schema.optional(Schema.Array(Schema.Unknown)), + data: Schema.optional( + Schema.NullOr( + Schema.Record( + Schema.String, + Schema.NullOr(Schema.Struct({ pullRequest: Schema.optional(Schema.Unknown) })), + ), + ), + ), + }), +); +const decodeSummaryEntry = Schema.decodeUnknownExit(RawSummarySchema); + +/** A failed GraphQL document is not a set of missing aliases to retry individually. */ +export class GitHubSummaryBatchUnavailableError extends Error {} + +export interface GitHubPullRequestSummary { + readonly number: number; + readonly title: string; + readonly url: string; + readonly headBranch: string; + readonly baseBranch: string; + readonly state: PullRequestState; + readonly isDraft: boolean; + readonly closedAt: string | null; + readonly mergedAt: string | null; + readonly updatedAt: string; + readonly author: PullRequestActor | null; + readonly additions: number; + readonly deletions: number; + readonly changedFiles: number; + readonly reviewDecision: PullRequestReviewDecision | null; + readonly checksState: PullRequestChecksState | null; + readonly mergeability: PullRequestMergeability; +} + +/** + * Summaries by the position they were asked in. A pull request GitHub answered nothing for, or + * one whose fields no longer decode, is absent rather than failing the rest of the batch. + */ +export function decodePullRequestSummariesJson( + raw: string, +): Result.Result< + ReadonlyMap, + DecodeFailure | GitHubSummaryBatchUnavailableError +> { + const decoded = decodeSummaries(raw); + if (!Result.isSuccess(decoded)) return Result.fail(decoded.failure); + if (decoded.success.data == null && (decoded.success.errors?.length ?? 0) > 0) { + return Result.fail(new GitHubSummaryBatchUnavailableError("GitHub rejected the summary batch")); + } + const summaries = new Map(); + for (const [alias, value] of Object.entries(decoded.success.data ?? {})) { + const index = /^s(\d+)$/.exec(alias)?.[1]; + if (index === undefined || value?.pullRequest == null) continue; + const entry = decodeSummaryEntry(value.pullRequest); + if (!Exit.isSuccess(entry)) continue; + const pr = entry.value; + summaries.set(Number(index), { + number: pr.number, + title: pr.title, + url: pr.url, + headBranch: pr.headRefName, + baseBranch: pr.baseRefName, + state: toState(pr), + isDraft: pr.isDraft ?? false, + closedAt: trimmed(pr.closedAt), + mergedAt: trimmed(pr.mergedAt), + updatedAt: pr.updatedAt, + author: toActor(pr.author), + additions: pr.additions ?? 0, + deletions: pr.deletions ?? 0, + changedFiles: pr.changedFiles ?? 0, + reviewDecision: toReviewDecisionWithReviews( + pr.reviewDecision, + (pr.latestReviews?.nodes ?? []).flatMap((review) => (review === null ? [] : [review])), + ), + // One enum for the head commit, dressed as a single check like the search row's. + checksState: rollupChecksState( + (pr.commits?.nodes ?? []).flatMap((commitNode) => { + const state = trimmed(commitNode?.commit?.statusCheckRollup?.state); + return state === null ? [] : [{ state }]; + }), + ), + mergeability: toMergeability(pr.mergeable), + }); + } + return Result.succeed(summaries); +} + export function decodePullRequestDetailJson( raw: string, ): Result.Result { diff --git a/apps/server/src/relay/AgentAwarenessRelay.test.ts b/apps/server/src/relay/AgentAwarenessRelay.test.ts index 02931d0534..2ea053a668 100644 --- a/apps/server/src/relay/AgentAwarenessRelay.test.ts +++ b/apps/server/src/relay/AgentAwarenessRelay.test.ts @@ -325,6 +325,9 @@ describe.sequential("signRelayAgentActivityPublishProof", () => { const projectId = "project-1" as ProjectId; const activeThreadId = "thread-active" as ThreadId; const idleThreadId = "thread-idle" as ThreadId; + const oldCompletedId = "thread-old-completed" as ThreadId; + const newCompletedId = "thread-new-completed" as ThreadId; + const freshMessageId = "thread-fresh-message" as ThreadId; const baseThread = { projectId, @@ -351,6 +354,7 @@ describe.sequential("signRelayAgentActivityPublishProof", () => { expect( AgentAwarenessRelay.resolveAgentAwarenessRelayActiveThreadIds({ environmentId, + startedAt: Date.parse(now), projects: [ { id: projectId, @@ -374,6 +378,44 @@ describe.sequential("signRelayAgentActivityPublishProof", () => { ...baseThread, id: idleThreadId, }, + { + ...baseThread, + id: oldCompletedId, + latestTurn: { + turnId: "turn-old" as TurnId, + state: "completed", + requestedAt: "2026-05-24T00:00:00.000Z", + startedAt: "2026-05-24T00:00:00.000Z", + completedAt: "2026-05-24T00:01:00.000Z", + assistantMessageId: null, + }, + }, + { + ...baseThread, + id: newCompletedId, + latestTurn: { + turnId: "turn-new" as TurnId, + state: "completed", + requestedAt: "2026-05-25T00:00:01.000Z", + startedAt: "2026-05-25T00:00:01.000Z", + completedAt: "2026-05-25T00:00:02.000Z", + assistantMessageId: null, + }, + }, + { + ...baseThread, + id: freshMessageId, + latestUserMessageAt: "2026-05-25T00:00:01.000Z", + session: { + threadId: freshMessageId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: "2026-05-25T00:00:02.000Z", + }, + }, { ...baseThread, id: "thread-missing-project" as ThreadId, @@ -389,7 +431,101 @@ describe.sequential("signRelayAgentActivityPublishProof", () => { }, ], }), - ).toEqual([activeThreadId]); + ).toEqual([activeThreadId, newCompletedId]); + expect( + AgentAwarenessRelay.resolveAgentAwarenessRelayActiveThreadIds({ + environmentId, + startedAt: Date.parse(now), + pendingAdmissionThreads: new Set([newCompletedId]), + projects: [{ id: projectId, title: "Pylon" }], + threads: [ + { + ...baseThread, + id: newCompletedId, + latestTurn: { + turnId: "turn-new" as TurnId, + state: "completed", + requestedAt: "2026-05-25T00:00:01.000Z", + startedAt: "2026-05-25T00:00:01.000Z", + completedAt: "2026-05-25T00:00:02.000Z", + assistantMessageId: null, + }, + }, + ], + }), + ).toEqual([]); + // A provider running event proves work even when a no-checkpoint turn + // settles to ready without materializing latestTurn. + expect( + AgentAwarenessRelay.resolveAgentAwarenessRelayActiveThreadIds({ + environmentId, + startedAt: Date.parse(now), + freshLifecycleThreads: new Map([ + [ + freshMessageId, + { + phase: "completed", + sequence: 2, + sessionStatus: "ready", + sessionUpdatedAt: "2026-05-25T00:00:02.000Z", + sessionIncarnationId: undefined, + }, + ], + ]), + projects: [{ id: projectId, title: "Pylon" }], + threads: [ + { + ...baseThread, + id: freshMessageId, + session: { + threadId: freshMessageId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: "2026-05-25T00:00:02.000Z", + }, + }, + ], + }), + ).toEqual([freshMessageId]); + // A running event may arrive before an old ready shell catches up, even + // when both states were stamped in the same millisecond. + expect( + AgentAwarenessRelay.resolveAgentAwarenessRelayActiveThreadIds({ + environmentId, + startedAt: Date.parse(now), + freshLifecycleThreads: new Map([ + [ + freshMessageId, + { + phase: "running", + sequence: 1, + sessionStatus: null, + sessionUpdatedAt: "2026-05-25T00:00:01.000Z", + sessionIncarnationId: undefined, + }, + ], + ]), + projects: [{ id: projectId, title: "Pylon" }], + threads: [ + { + ...baseThread, + id: freshMessageId, + session: { + threadId: freshMessageId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: "2026-05-25T00:00:01.000Z", + }, + }, + ], + }), + ).toEqual([]); }); it.effect("signs the activity publish JWT and rejects tampering", () => diff --git a/apps/server/src/relay/AgentAwarenessRelay.ts b/apps/server/src/relay/AgentAwarenessRelay.ts index 144d43a029..91f8bf45cf 100644 --- a/apps/server/src/relay/AgentAwarenessRelay.ts +++ b/apps/server/src/relay/AgentAwarenessRelay.ts @@ -267,8 +267,44 @@ export function resolveAgentAwarenessRelayPublishSnapshot(input: { }; } +type FreshLifecycleProof = { + readonly phase: "running" | "completed" | "failed"; + readonly sequence: number; + readonly sessionStatus: "ready" | "idle" | "error" | null; + readonly sessionUpdatedAt: string; + readonly sessionIncarnationId: string | undefined; +}; + +function terminalWorkSinceStart( + thread: OrchestrationThreadShell, + startedAt: number, + phase: "completed" | "failed", + freshLifecycle?: FreshLifecycleProof, +): boolean { + // An observed new lifecycle takes precedence over a checkpoint from the + // previous turn, which can remain in the shell during admission. + if (freshLifecycle === undefined) { + return Date.parse(thread.latestTurn?.completedAt ?? "") > startedAt; + } + if (freshLifecycle.phase === "running") return false; + // For an observed terminal transition, the projected shell must have caught + // up to that same session even when an older checkpoint is still present. + const session = thread.session; + return ( + session !== null && + ((phase === "completed" && freshLifecycle.phase === "completed") || + (phase === "failed" && freshLifecycle.phase === "failed")) && + session.status === freshLifecycle.sessionStatus && + session.sessionIncarnationId === freshLifecycle.sessionIncarnationId && + Date.parse(session.updatedAt) >= Date.parse(freshLifecycle.sessionUpdatedAt) + ); +} + export function resolveAgentAwarenessRelayActiveThreadIds(input: { readonly environmentId: EnvironmentId; + readonly startedAt: number; + readonly freshLifecycleThreads?: ReadonlyMap; + readonly pendingAdmissionThreads?: ReadonlySet; readonly projects: ReadonlyArray>; readonly threads: ReadonlyArray; }): ReadonlyArray { @@ -279,12 +315,22 @@ export function resolveAgentAwarenessRelayActiveThreadIds(input: { if (!project) { return false; } + const state = projectThreadAwareness({ + environmentId: input.environmentId, + project, + thread, + }); return ( - projectThreadAwareness({ - environmentId: input.environmentId, - project, - thread, - }) !== null + state !== null && + (state.phase !== "completed" && state.phase !== "failed" + ? true + : !input.pendingAdmissionThreads?.has(thread.id) && + terminalWorkSinceStart( + thread, + input.startedAt, + state.phase, + input.freshLifecycleThreads?.get(thread.id), + )) ); }) .map((thread) => thread.id); @@ -298,8 +344,12 @@ export const make = Effect.gen(function* () { const orchestrationEngine = yield* OrchestrationEngine.OrchestrationEngineService; const crypto = yield* Crypto.Crypto; const cloudLinkKeyPair = yield* getOrCreateEnvironmentKeyPairFromSecretStore(secrets); + const startedAt = (yield* DateTime.now).epochMilliseconds; const activeSnapshotPublishedRef = yield* Ref.make(false); const publishedStateByThreadRef = yield* Ref.make(new Map()); + const publishedNonNullThreads = new Set(); + const freshLifecycleThreads = new Map(); + const pendingAdmissionThreads = new Set(); const readSecretString = (name: string) => secrets @@ -417,6 +467,19 @@ export const make = Effect.gen(function* () { }); const publishIdentity = agentAwarenessPublishIdentity(snapshot.state); const publishedStateByThread = yield* Ref.get(publishedStateByThreadRef); + if (snapshot.state?.phase === "completed" || snapshot.state?.phase === "failed") { + // A new admission can leave the previous completed checkpoint projected + // until its running shell arrives. This fence applies even when the + // thread has an older published state. + if (pendingAdmissionThreads.has(threadId)) return; + const proof = freshLifecycleThreads.get(threadId); + if ( + (proof !== undefined || !publishedNonNullThreads.has(threadId)) && + (Option.isNone(thread) || + !terminalWorkSinceStart(thread.value, startedAt, snapshot.state.phase, proof)) + ) + return; + } if (publishedStateByThread.get(threadId) === publishIdentity) { // The projection is back at (or never left) the last published state, so // any pending deferred confirmation is moot. Leaving the deadline in @@ -444,7 +507,7 @@ export const make = Effect.gen(function* () { const requiresConfirmation = (snapshot.state === null && publishedStateByThread.get(threadId) !== agentAwarenessPublishIdentity(null)) || - (snapshot.state?.phase === "completed" && !publishedStateByThread.has(threadId)); + (snapshot.state?.phase === "completed" && !publishedNonNullThreads.has(threadId)); if (requiresConfirmation) { const nowMs = (yield* DateTime.now).epochMilliseconds; const deadline = publishConfirmDeadlines.get(threadId); @@ -498,6 +561,15 @@ export const make = Effect.gen(function* () { nextPublishedStates.set(threadId, publishIdentity); return nextPublishedStates; }); + if (snapshot.state === null) { + freshLifecycleThreads.delete(threadId); + pendingAdmissionThreads.delete(threadId); + } else { + publishedNonNullThreads.add(threadId); + if (snapshot.state.phase === "completed" || snapshot.state.phase === "failed") { + freshLifecycleThreads.delete(threadId); + } + } }); const publishThread: AgentAwarenessRelay["Service"]["publishThread"] = (threadId) => @@ -529,6 +601,9 @@ export const make = Effect.gen(function* () { const snapshot = yield* snapshotQuery.getShellSnapshot(); const activeThreadIds = resolveAgentAwarenessRelayActiveThreadIds({ environmentId, + startedAt, + freshLifecycleThreads, + pendingAdmissionThreads, projects: snapshot.projects, threads: snapshot.threads, }); @@ -610,12 +685,69 @@ export const make = Effect.gen(function* () { ); yield* forkParked( Stream.runForEach(orchestrationEngine.streamDomainEvents, (event) => { + // This PubSub stream starts with the subscription and does not replay + // past events. A timestamp fence would discard valid events emitted + // later in the same millisecond as startup. const threadId = eventThreadId(event); if (threadId === null) { return Effect.logDebug("agent activity publishing ignored event without thread id", { eventType: event.type, }); } + if ( + (event.type === "thread.turn-start-requested" || + event.type === "thread.message-sent") && + event.metadata.historyImport !== true + ) { + freshLifecycleThreads.delete(threadId); + pendingAdmissionThreads.add(threadId); + publishConfirmDeadlines.delete(threadId); + } + if (event.type === "thread.session-set" && event.metadata.historyImport !== true) { + const session = event.payload.session; + const previous = freshLifecycleThreads.get(threadId); + if (session.status === "running") { + pendingAdmissionThreads.delete(threadId); + publishConfirmDeadlines.delete(threadId); + freshLifecycleThreads.set(threadId, { + phase: "running", + sequence: event.sequence, + sessionStatus: null, + sessionUpdatedAt: session.updatedAt, + sessionIncarnationId: session.sessionIncarnationId, + }); + } else if (session.status === "error") { + pendingAdmissionThreads.delete(threadId); + publishConfirmDeadlines.delete(threadId); + freshLifecycleThreads.set(threadId, { + phase: "failed", + sequence: event.sequence, + sessionStatus: "error", + sessionUpdatedAt: session.updatedAt, + sessionIncarnationId: session.sessionIncarnationId, + }); + } else if (session.status === "ready" || session.status === "idle") { + if ( + previous?.phase === "running" && + event.sequence > previous.sequence && + session.sessionIncarnationId === previous.sessionIncarnationId + ) { + freshLifecycleThreads.set(threadId, { + phase: "completed", + sequence: event.sequence, + sessionStatus: session.status, + sessionUpdatedAt: session.updatedAt, + sessionIncarnationId: session.sessionIncarnationId, + }); + } + } else { + // A new admission or stop invalidates an older completion proof + // even while the projector still serves the prior ready shell. + freshLifecycleThreads.delete(threadId); + pendingAdmissionThreads.add(threadId); + publishConfirmDeadlines.delete(threadId); + } + } if (!shouldPublishAgentAwarenessEvent(event)) { return Effect.logDebug( "agent activity publishing ignored event without activity changes", diff --git a/apps/server/src/relay/AgentAwarenessRelayStartup.test.ts b/apps/server/src/relay/AgentAwarenessRelayStartup.test.ts new file mode 100644 index 0000000000..95887f6342 --- /dev/null +++ b/apps/server/src/relay/AgentAwarenessRelayStartup.test.ts @@ -0,0 +1,488 @@ +import * as NodeServices from "@effect/platform-node/NodeServices"; +import type { + EnvironmentId, + ExecutionEnvironmentDescriptor, + OrchestrationEvent, + OrchestrationProjectShell, + OrchestrationShellSnapshot, + OrchestrationThreadShell, + ProjectId, + ThreadId, + TurnId, +} from "@t3tools/contracts"; +import { CommandId, ProviderInstanceId } from "@t3tools/contracts"; +import { describe, expect, it } from "@effect/vitest"; +import * as DateTime from "effect/DateTime"; +import * as Effect from "effect/Effect"; +import * as Layer from "effect/Layer"; +import * as Option from "effect/Option"; +import * as Queue from "effect/Queue"; +import * as Stream from "effect/Stream"; +import * as TestClock from "effect/testing/TestClock"; + +import * as ServerSecretStore from "../auth/ServerSecretStore.ts"; +import * as ServerEnvironment from "../environment/ServerEnvironment.ts"; +import { + OrchestrationEngineService, + type OrchestrationEngineShape, +} from "../orchestration/Services/OrchestrationEngine.ts"; +import { + ProjectionSnapshotQuery, + type ProjectionSnapshotQueryShape, +} from "../orchestration/Services/ProjectionSnapshotQuery.ts"; +import { + RELAY_ENVIRONMENT_CREDENTIAL_SECRET, + RELAY_URL_SECRET, + PUBLISH_AGENT_ACTIVITY_SECRET, +} from "../cloud/config.ts"; +import * as AgentAwarenessRelay from "./AgentAwarenessRelay.ts"; + +const encodeSecret = (value: string): Uint8Array => new TextEncoder().encode(value); + +function makeMemorySecretStore() { + const values = new Map(); + const store = { + get: ((name) => + Effect.sync(() => { + const value = values.get(name); + return value === undefined ? Option.none() : Option.some(Uint8Array.from(value)); + })) satisfies ServerSecretStore.ServerSecretStore["Service"]["get"], + set: ((name, value) => + Effect.sync(() => { + values.set(name, Uint8Array.from(value)); + })) satisfies ServerSecretStore.ServerSecretStore["Service"]["set"], + create: ((name, value) => + Effect.sync(() => { + values.set(name, Uint8Array.from(value)); + })) satisfies ServerSecretStore.ServerSecretStore["Service"]["create"], + getOrCreateRandom: ((name, bytes) => + Effect.sync(() => { + const existing = values.get(name); + if (existing) { + return existing; + } + const generated = new Uint8Array(bytes); + values.set(name, generated); + return generated; + })) satisfies ServerSecretStore.ServerSecretStore["Service"]["getOrCreateRandom"], + remove: ((name) => + Effect.sync(() => { + values.delete(name); + })) satisfies ServerSecretStore.ServerSecretStore["Service"]["remove"], + } satisfies ServerSecretStore.ServerSecretStore["Service"]; + return { + store, + setString: (name: string, value: string) => store.set(name, encodeSecret(value)), + }; +} + +describe.sequential("AgentAwarenessRelay startup", () => { + it.effect("does not alert for historical completions after startup", () => + Effect.scoped( + Effect.gen(function* () { + const secrets = makeMemorySecretStore(); + const now = yield* DateTime.now; + const old = DateTime.formatIso(DateTime.add(now, { days: -7 })); + // Live PubSub events can share the startup millisecond and are still new. + const afterStartup = DateTime.formatIso(now); + const threadId = "thread-old" as ThreadId; + const freshDoneThreadId = "thread-new-done" as ThreadId; + const invalidatedThreadId = "thread-new-admission" as ThreadId; + const projectId = "project-1" as ProjectId; + const environmentId = "env-1" as EnvironmentId; + const project = { + id: projectId, + title: "T3 Code", + workspaceRoot: "/workspace", + repositoryIdentity: null, + defaultModelSelection: null, + scripts: [], + createdAt: old, + updatedAt: old, + } satisfies OrchestrationProjectShell; + const completedTurn = { + turnId: "turn-1" as TurnId, + state: "completed", + requestedAt: old, + startedAt: old, + completedAt: old, + assistantMessageId: null, + } as const; + const completedThread = { + id: threadId, + projectId, + title: "Old task", + modelSelection: { instanceId: ProviderInstanceId.make("codex"), model: "gpt-5.4" }, + runtimeMode: "full-access", + interactionMode: "default", + branch: null, + worktreePath: null, + pullRequests: [], + latestTurn: completedTurn, + createdAt: old, + updatedAt: old, + archivedAt: null, + settledOverride: null, + settledAt: null, + session: null, + latestUserMessageAt: old, + hasPendingApprovals: false, + hasPendingUserInput: false, + hasActionableProposedPlan: false, + } satisfies OrchestrationThreadShell; + let currentThread: OrchestrationThreadShell | null = completedThread; + let resolveFreshDoneQuery: (() => void) | undefined; + const freshDoneQuery = new Promise((resolve) => { + resolveFreshDoneQuery = resolve; + }); + let watchFreshDoneQuery = false; + let invalidatedQueries = 0; + let resolveInvalidatedQueries: (() => void) | undefined; + const invalidatedQuery = new Promise((resolve) => { + resolveInvalidatedQueries = resolve; + }); + let freshDoneQueries = 0; + let resolveFreshReadyQuery: (() => void) | undefined; + const freshReadyQuery = new Promise((resolve) => { + resolveFreshReadyQuery = resolve; + }); + let publishes = 0; + const publishPaths: string[] = []; + const countFor = (id: ThreadId) => + publishPaths.filter((path) => path.includes(`/threads/${id}/`)).length; + let expectedThreadId = threadId; + let resolveNewQuery: (() => void) | undefined; + const newQuery = new Promise((resolve) => { + resolveNewQuery = resolve; + }); + let resolvePublished: (() => void) | undefined; + const published = new Promise((resolve) => { + resolvePublished = resolve; + }); + const events = yield* Queue.unbounded(); + const originalFetch = globalThis.fetch; + globalThis.fetch = ((input: Parameters[0]) => { + publishes += 1; + const path = String(input); + publishPaths.push(path); + if (path.includes(`/threads/${expectedThreadId}/`)) resolvePublished?.(); + return Promise.resolve(Response.json({ ok: true, deliveries: [] })); + }) as unknown as typeof fetch; + yield* Effect.addFinalizer(() => + Effect.sync(() => { + globalThis.fetch = originalFetch; + }), + ); + yield* secrets.setString(RELAY_URL_SECRET, "https://relay.example.test"); + yield* secrets.setString(RELAY_ENVIRONMENT_CREDENTIAL_SECRET, "relay-credential"); + yield* secrets.setString(PUBLISH_AGENT_ACTIVITY_SECRET, "true"); + + const layer = Layer.mergeAll( + Layer.succeed(ServerSecretStore.ServerSecretStore, secrets.store), + Layer.succeed(ServerEnvironment.ServerEnvironment, { + getEnvironmentId: Effect.succeed(environmentId), + getDescriptor: Effect.succeed({ + environmentId, + label: "Test Desktop", + platform: { os: "darwin", arch: "arm64" }, + serverVersion: "0.0.0-test", + capabilities: { repositoryIdentity: true }, + } satisfies ExecutionEnvironmentDescriptor), + }), + Layer.succeed(OrchestrationEngineService, { + streamDomainEvents: Stream.fromQueue(events), + } as OrchestrationEngineShape), + Layer.succeed(ProjectionSnapshotQuery, { + getShellSnapshot: () => + Effect.succeed({ + snapshotSequence: 1, + projects: [project], + threads: currentThread ? [currentThread] : [], + updatedAt: old, + } satisfies OrchestrationShellSnapshot), + getThreadShellById: (requestedThreadId: ThreadId) => + Effect.sync(() => { + if (currentThread?.session?.lastError === "new failure") resolveNewQuery?.(); + if (requestedThreadId === invalidatedThreadId) { + invalidatedQueries += 1; + if (invalidatedQueries >= 3) resolveInvalidatedQueries?.(); + } + if (watchFreshDoneQuery && currentThread?.id === freshDoneThreadId) { + freshDoneQueries += 1; + resolveFreshDoneQuery?.(); + if (freshDoneQueries >= 2) resolveFreshReadyQuery?.(); + } + return Option.fromNullishOr( + currentThread?.id === requestedThreadId ? currentThread : null, + ); + }), + getProjectShellById: () => Effect.succeed(Option.some(project)), + } as unknown as ProjectionSnapshotQueryShape), + ); + + yield* Effect.gen(function* () { + const relay = yield* AgentAwarenessRelay.AgentAwarenessRelay; + yield* relay.start(); + yield* relay.publishThread(threadId); + expect(publishes).toBe(0); + + currentThread = { + ...completedThread, + latestTurn: null, + latestUserMessageAt: DateTime.formatIso(DateTime.add(now, { seconds: 1 })), + session: { + threadId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: DateTime.formatIso(DateTime.add(now, { seconds: 2 })), + }, + }; + expect( + AgentAwarenessRelay.resolveAgentAwarenessRelayPublishSnapshot({ + environmentId, + threadId, + thread: Option.some(currentThread), + project: Option.some(project), + }).state?.phase, + ).toBe("completed"); + yield* relay.publishThread(threadId); + expect(publishes).toBe(0); + + currentThread = { + ...completedThread, + session: { + threadId, + status: "error", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: "old failure", + updatedAt: old, + }, + }; + yield* relay.publishThread(threadId); + expect(publishes).toBe(0); + + currentThread = { + ...completedThread, + latestTurn: null, + session: { + threadId, + status: "error", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: "new failure", + updatedAt: afterStartup, + }, + }; + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 1, + eventId: "event-new-failure", + commandId: CommandId.make("command-new-failure"), + aggregateKind: "thread", + aggregateId: threadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { threadId, session: currentThread.session }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Effect.promise(() => newQuery); + yield* Effect.promise(() => published); + expect(publishes).toBe(1); + + let resolveCompleted: (() => void) | undefined; + const completed = new Promise((resolve) => { + resolveCompleted = resolve; + }); + resolvePublished = resolveCompleted; + currentThread = { + ...completedThread, + latestTurn: null, + session: { + threadId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: afterStartup, + }, + }; + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 2, + eventId: "event-fast-completion", + commandId: CommandId.make("command-fast-completion"), + aggregateKind: "thread", + aggregateId: threadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { threadId, session: { ...currentThread.session, status: "running" } }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 3, + eventId: "event-fast-completion-ready", + commandId: CommandId.make("command-fast-completion-ready"), + aggregateKind: "thread", + aggregateId: threadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { threadId, session: currentThread.session }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Effect.promise(() => completed); + expect(publishes).toBe(2); + + expectedThreadId = freshDoneThreadId; + currentThread = { + ...completedThread, + id: freshDoneThreadId, + latestTurn: null, + session: { + threadId: freshDoneThreadId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: afterStartup, + }, + }; + yield* relay.publishThread(freshDoneThreadId); + expect(countFor(freshDoneThreadId)).toBe(0); + let resolveFirstDone: (() => void) | undefined; + const firstDone = new Promise((resolve) => { + resolveFirstDone = resolve; + }); + resolvePublished = resolveFirstDone; + watchFreshDoneQuery = true; + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 4, + eventId: "event-first-done", + commandId: CommandId.make("command-first-done"), + aggregateKind: "thread", + aggregateId: freshDoneThreadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { + threadId: freshDoneThreadId, + session: { ...currentThread.session, status: "running" }, + }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Effect.promise(() => freshDoneQuery); + expect(countFor(freshDoneThreadId)).toBe(0); + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 5, + eventId: "event-first-done-ready", + commandId: CommandId.make("command-first-done-ready"), + aggregateKind: "thread", + aggregateId: freshDoneThreadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { threadId: freshDoneThreadId, session: currentThread.session }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Effect.promise(() => freshReadyQuery); + yield* Effect.yieldNow; + expect(countFor(freshDoneThreadId)).toBe(0); + yield* TestClock.adjust("5 seconds"); + yield* Effect.promise(() => firstDone); + expect(countFor(freshDoneThreadId)).toBe(1); + + let resolveTombstone: (() => void) | undefined; + const tombstone = new Promise((resolve) => { + resolveTombstone = resolve; + }); + resolvePublished = resolveTombstone; + currentThread = null; + yield* relay.publishThread(freshDoneThreadId); + expect(countFor(freshDoneThreadId), publishPaths.join(", ")).toBe(1); + yield* TestClock.adjust("5 seconds"); + yield* Effect.promise(() => tombstone); + expect(countFor(freshDoneThreadId)).toBe(2); + yield* relay.publishThread(freshDoneThreadId); + expect(countFor(freshDoneThreadId)).toBe(2); + + // A new admission invalidates the prior Done proof while the shell + // still projects the previous ready state. + currentThread = { + ...completedThread, + id: invalidatedThreadId, + latestTurn: { ...completedTurn, completedAt: afterStartup }, + session: { + threadId: invalidatedThreadId, + status: "ready", + providerName: "Codex", + runtimeMode: "full-access", + activeTurnId: null, + lastError: null, + updatedAt: afterStartup, + }, + }; + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 6, + eventId: "event-admission-running", + commandId: CommandId.make("command-admission-running"), + aggregateKind: "thread", + aggregateId: invalidatedThreadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { + threadId: invalidatedThreadId, + session: { ...currentThread.session, status: "running" }, + }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 7, + eventId: "event-admission-ready", + commandId: CommandId.make("command-admission-ready"), + aggregateKind: "thread", + aggregateId: invalidatedThreadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { threadId: invalidatedThreadId, session: currentThread.session }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Queue.offer(events, { + type: "thread.session-set", + sequence: 8, + eventId: "event-new-admission-starting", + commandId: CommandId.make("command-new-admission-starting"), + aggregateKind: "thread", + aggregateId: invalidatedThreadId, + actor: { kind: "provider" }, + metadata: {}, + payload: { + threadId: invalidatedThreadId, + session: { ...currentThread.session, status: "starting" }, + }, + occurredAt: afterStartup, + } as unknown as OrchestrationEvent); + yield* Effect.promise(() => invalidatedQuery); + yield* TestClock.adjust("5 seconds"); + expect(countFor(invalidatedThreadId)).toBe(0); + }).pipe( + Effect.provide( + AgentAwarenessRelay.layer.pipe( + Layer.provide(layer), + Layer.provideMerge(NodeServices.layer), + ), + ), + ); + }), + ), + ); +}); diff --git a/apps/server/src/rollback/RollbackSagaRunner.test.ts b/apps/server/src/rollback/RollbackSagaRunner.test.ts index 05711e5df2..2e6456593b 100644 --- a/apps/server/src/rollback/RollbackSagaRunner.test.ts +++ b/apps/server/src/rollback/RollbackSagaRunner.test.ts @@ -11,10 +11,13 @@ import * as NodeServices from "@effect/platform-node/NodeServices"; import { assert, it } from "@effect/vitest"; import * as Effect from "effect/Effect"; import * as Fiber from "effect/Fiber"; +import * as Layer from "effect/Layer"; +import * as Logger from "effect/Logger"; import * as Option from "effect/Option"; import * as Stream from "effect/Stream"; import { CheckpointStore } from "../checkpointing/CheckpointStore.ts"; +import { checkpointRefForThreadTurn } from "../checkpointing/Utils.ts"; import { OrchestrationEngineService } from "../orchestration/Services/OrchestrationEngine.ts"; import { RuntimeReceiptBus, @@ -39,6 +42,7 @@ const now = "2026-08-31T00:00:00.000Z"; const privateTargetCanary = "PRIVATE_TARGET_LEAF_CANARY"; const privateSourceCanary = "PRIVATE_SOURCE_LEAF_CANARY"; const privatePreimageCanary = "/private/preimage/canary"; +const privateFailureCanary = "PRIVATE_PROMPT_TOOL_CREDENTIAL_ERROR_CANARY"; type ProviderMode = "success" | "unknown-target" | "stayed-source" | "wrong-target"; @@ -53,9 +57,9 @@ const makeState = (operationId: string): RollbackSagaState => ({ targetRevision: 1, sourceTurnId: null, targetTurnId: null, - sourceCheckpointRef: CheckpointRef.make("refs/t3/checkpoints/thread-runner/turn/2"), + sourceCheckpointRef: checkpointRefForThreadTurn(threadId, 2), sourceCheckpointOid: "a".repeat(40), - targetCheckpointRef: CheckpointRef.make("refs/t3/checkpoints/thread-runner/turn/1"), + targetCheckpointRef: checkpointRefForThreadTurn(threadId, 1), targetCheckpointOid: "b".repeat(40), targetCheckpointDigest: "target-tree-digest", providerInstanceId, @@ -111,6 +115,7 @@ const makeEnvironment = ( const providerApplies: string[] = []; const commands: OrchestrationCommand[] = []; const runtimeReceipts: OrchestrationRuntimeReceipt[] = []; + const emittedLogs: Array = []; const preimage: RollbackWorkspacePreimage = { backupPath: privatePreimageCanary, @@ -235,9 +240,9 @@ const makeEnvironment = ( return Effect.void; case "unknown-target": providerDigest = "provider-target"; - return Effect.fail("provider-timeout"); + return Effect.fail(privateFailureCanary); case "stayed-source": - return Effect.fail("provider-failed"); + return Effect.fail(privateFailureCanary); case "wrong-target": providerDigest = "provider-wrong"; return Effect.void; @@ -296,7 +301,7 @@ const makeEnvironment = ( workspaceCalls.push("cleanupPreimage"); if (cleanupFailures > 0) { cleanupFailures -= 1; - return Effect.fail("cleanup-failed"); + return Effect.fail(privateFailureCanary); } preimageCleaned = true; return Effect.void; @@ -352,7 +357,19 @@ const makeEnvironment = ( Effect.provideService(CheckpointStore, checkpointStore as never), Effect.provideService(RuntimeReceiptBus, receipts), Effect.provideService(RollbackFaultInjector, fault), - Effect.provide(NodeServices.layer), + Effect.provide( + Layer.mergeAll( + Logger.layer( + [ + Logger.make( + (event) => void emittedLogs.push(Logger.formatStructured.log(event)), + ), + ], + { mergeWithExisting: false }, + ), + NodeServices.layer, + ), + ), ); }; @@ -389,7 +406,19 @@ const makeEnvironment = ( projectionCommits, commands, runtimeReceipts, + emittedLogs, }), + assertPrivateDataAbsentFromPublicOutput: () => { + const publicOutput = JSON.stringify({ commands, runtimeReceipts, emittedLogs }); + for (const canary of [ + privateTargetCanary, + privateSourceCanary, + privatePreimageCanary, + privateFailureCanary, + ]) { + assert.notInclude(publicOutput, canary); + } + }, }; }; @@ -409,6 +438,7 @@ it.effect("commits last, clears private state, and never publishes private canar const runner = yield* environment.makeRunner(); yield* runner.run("operation-success", false); const snapshot = environment.snapshot(); + environment.assertPrivateDataAbsentFromPublicOutput(); assert.equal(snapshot.record.state.phase, "complete"); assert.isTrue(snapshot.record.terminal); @@ -550,6 +580,21 @@ it.effect("compensates workspace and provider when the provider stays at source" }), ); +it.effect("does not replay a persisted preimage under a mismatched checkpoint owner", () => + Effect.gen(function* () { + const environment = makeEnvironment("operation-owner-mismatch", "stayed-source"); + environment.setPersistedState({ + sourceCheckpointRef: CheckpointRef.make("refs/t3/checkpoints/foreign/turn/2"), + }); + const runner = yield* environment.makeRunner(); + yield* runner.run("operation-owner-mismatch", false); + const snapshot = environment.snapshot(); + assert.equal(snapshot.record.state.phase, "manual-recovery"); + assert.isFalse(snapshot.workspaceCalls.includes("restorePreimage")); + assert.equal(snapshot.workspaceDigest, "workspace-target"); + }), +); + it.effect( "fails closed in manual recovery when inspection finds an unrelated provider anchor", () => @@ -694,6 +739,7 @@ it.effect("releases only the worker owner and retries post-commit cleanup idempo assert.isFalse(complete.lease); assert.equal(complete.projectionCommits, 1); assert.isTrue(complete.preimageCleaned); + environment.assertPrivateDataAbsentFromPublicOutput(); }), ); @@ -750,6 +796,7 @@ for (const faultLabel of compensationFaultLabels) { assert.equal(snapshot.providerDigest, "provider-source"); assert.equal(snapshot.projectionCommits, 0); assert.isFalse(snapshot.lease); + environment.assertPrivateDataAbsentFromPublicOutput(); }), ); } @@ -767,6 +814,7 @@ it.effect("restores durable manual recovery status after a crash before status p yield* restartedRunner.run(operationId, true); const snapshot = environment.snapshot(); assert.equal(snapshot.record.state.phase, "manual-recovery"); + environment.assertPrivateDataAbsentFromPublicOutput(); assert.isFalse(snapshot.record.terminal); assert.isTrue(snapshot.lease); const statuses = snapshot.commands.filter( @@ -810,6 +858,7 @@ for (const faultLabel of restartFaultLabels) { yield* restartedRunner.run(operationId, true); const snapshot = environment.snapshot(); assert.equal(snapshot.record.state.phase, "complete"); + environment.assertPrivateDataAbsentFromPublicOutput(); assert.isTrue(snapshot.record.terminal); assert.isFalse(snapshot.lease); assert.equal(snapshot.workspaceDigest, "workspace-target"); @@ -874,6 +923,7 @@ for (const failure of ["provider", "projection"] as const) { snapshot.record.state.lastErrorCode, failure === "provider" ? "provider-target-retry-exhausted" : "projection-commit-cas-failed", ); + environment.assertPrivateDataAbsentFromPublicOutput(); if (failure === "projection") assert.deepEqual(snapshot.providerApplies, ["target", "source"]); const status = snapshot.commands.findLast( diff --git a/apps/server/src/rollback/RollbackSagaRunner.ts b/apps/server/src/rollback/RollbackSagaRunner.ts index 9a0ca895e5..7567868a7b 100644 --- a/apps/server/src/rollback/RollbackSagaRunner.ts +++ b/apps/server/src/rollback/RollbackSagaRunner.ts @@ -189,18 +189,26 @@ export const make = Effect.gen(function* () { record = workspaceStarted.value; const preimage = privatePreimage(record.state); if (preimage !== null) { - workspaceProved = yield* workspace - .restorePreimage({ - cwd: record.state.workspaceCwd, - preimage, - }) - .pipe( - Effect.tap(() => after("side-effect:workspace-compensated", record.operationId)), - Effect.match({ - onFailure: () => false, - onSuccess: (receipt) => receipt.digest === preimage.digest, - }), - ); + const validOwner = + record.state.sourceCheckpointRef === + checkpointRefForThreadTurn(record.state.threadId, record.state.sourceRevision) && + record.state.targetCheckpointRef === + checkpointRefForThreadTurn(record.state.threadId, record.state.targetRevision); + workspaceProved = validOwner + ? yield* workspace + .restorePreimage({ + cwd: record.state.workspaceCwd, + threadId: record.state.threadId, + preimage, + }) + .pipe( + Effect.tap(() => after("side-effect:workspace-compensated", record.operationId)), + Effect.match({ + onFailure: () => false, + onSuccess: (receipt) => receipt.digest === preimage.digest, + }), + ) + : false; } const workspaceComplete = yield* update(record, { @@ -332,6 +340,7 @@ export const make = Effect.gen(function* () { .capturePreimage({ operationId: state.operationId, cwd: state.workspaceCwd, + threadId: state.threadId, targetCheckpointOid: state.targetCheckpointOid, }) .pipe(Effect.result); diff --git a/apps/server/src/rollback/RollbackWorkspace.test.ts b/apps/server/src/rollback/RollbackWorkspace.test.ts index 9aa0b9c2ef..0ecba08840 100644 --- a/apps/server/src/rollback/RollbackWorkspace.test.ts +++ b/apps/server/src/rollback/RollbackWorkspace.test.ts @@ -8,6 +8,8 @@ import * as NodeFSP from "node:fs/promises"; import * as NodePath from "node:path"; import * as NodeOS from "node:os"; import * as NodeUtil from "node:util"; +import { ThreadId } from "@t3tools/contracts"; +import { checkpointRefForThreadTurn } from "../checkpointing/Utils.ts"; import * as ServerConfig from "../config.ts"; import { RollbackWorkspace, layer as RollbackWorkspaceLive } from "./RollbackWorkspace.ts"; @@ -33,6 +35,9 @@ layer("RollbackWorkspace", (it) => { (cwd) => Effect.gen(function* () { const workspace = yield* RollbackWorkspace; + const threadId = ThreadId.make("thread-test"); + const targetRef = checkpointRefForThreadTurn(threadId, 1); + const sourceRef = checkpointRefForThreadTurn(threadId, 2); yield* Effect.promise(async () => { await run(cwd, ["init", "-b", "main"]); @@ -64,7 +69,7 @@ layer("RollbackWorkspace", (it) => { "-m", "target checkpoint", ]); - await run(cwd, ["update-ref", "refs/t3/checkpoints/thread-test/turn/1", targetOid]); + await run(cwd, ["update-ref", targetRef, targetOid]); await run(cwd, ["reset", "--hard", "HEAD"]); await NodeFSP.writeFile(NodePath.join(cwd, "tracked.txt"), "staged\n"); @@ -83,7 +88,7 @@ layer("RollbackWorkspace", (it) => { NodePath.join(cwd, "ignored-target.txt"), "source ignored path\n", ); - await run(cwd, ["update-ref", "refs/t3/checkpoints/thread-test/turn/2", "HEAD"]); + await run(cwd, ["update-ref", sourceRef, "HEAD"]); }); const sourceStatus = yield* Effect.promise(() => @@ -95,11 +100,12 @@ layer("RollbackWorkspace", (it) => { ); const target = yield* workspace.resolveCheckpoint({ cwd, - checkpointRef: "refs/t3/checkpoints/thread-test/turn/1", + checkpointRef: targetRef, }); const preimage = yield* workspace.capturePreimage({ operationId: "operation-workspace", cwd, + threadId, targetCheckpointOid: target.oid, }); assert.notEqual(preimage.digest.length, 0); @@ -149,17 +155,11 @@ layer("RollbackWorkspace", (it) => { "refs/heads/main", ); assert.equal( - yield* Effect.promise(() => - run(cwd, ["rev-parse", "refs/t3/checkpoints/thread-test/turn/2"]), - ), + yield* Effect.promise(() => run(cwd, ["rev-parse", sourceRef])), yield* Effect.promise(() => run(cwd, ["rev-parse", "HEAD"])), ); - yield* Effect.promise(async () => { - await run(cwd, ["update-ref", "-d", "refs/t3/checkpoints/thread-test/turn/2"]); - await run(cwd, ["update-ref", "refs/t3/checkpoints/rogue/turn/99", "HEAD"]); - }); - const restored = yield* workspace.restorePreimage({ cwd, preimage }); + const restored = yield* workspace.restorePreimage({ cwd, threadId, preimage }); assert.equal(restored.digest, preimage.digest); assert.equal( yield* Effect.promise(() => run(cwd, ["status", "--porcelain=v1", "-uall"])), @@ -179,6 +179,16 @@ layer("RollbackWorkspace", (it) => { ), sourceRefs, ); + const driftRef = checkpointRefForThreadTurn(threadId, 3); + yield* Effect.promise(() => run(cwd, ["update-ref", driftRef, "HEAD"])); + const driftRestore = yield* workspace + .restorePreimage({ cwd, threadId, preimage }) + .pipe(Effect.exit); + assert.equal(driftRestore._tag, "Failure"); + assert.equal( + yield* Effect.promise(() => run(cwd, ["rev-parse", driftRef])), + yield* Effect.promise(() => run(cwd, ["rev-parse", "HEAD"])), + ); assert.equal( yield* Effect.promise(() => NodeFSP.readFile(NodePath.join(cwd, "nested", "untracked.txt"), "utf8"), @@ -222,6 +232,11 @@ layer("RollbackWorkspace", (it) => { (root) => Effect.gen(function* () { const workspace = yield* RollbackWorkspace; + const threadId = ThreadId.make("linked"); + const siblingThreadId = ThreadId.make("sibling"); + const targetRef = checkpointRefForThreadTurn(threadId, 1); + const siblingFirstRef = checkpointRefForThreadTurn(siblingThreadId, 1); + const siblingSecondRef = checkpointRefForThreadTurn(siblingThreadId, 2); const main = NodePath.join(root, "main"); const linked = NodePath.join(root, "linked"); yield* Effect.promise(async () => { @@ -237,7 +252,7 @@ layer("RollbackWorkspace", (it) => { await run(linked, ["add", "tracked.txt"]); const tree = await run(linked, ["write-tree"]); const checkpoint = await run(linked, ["commit-tree", tree, "-m", "linked checkpoint"]); - await run(linked, ["update-ref", "refs/t3/checkpoints/linked/turn/1", checkpoint]); + await run(linked, ["update-ref", targetRef, checkpoint]); await run(linked, ["reset", "--hard", "HEAD"]); await NodeFSP.writeFile(NodePath.join(linked, "tracked.txt"), "linked pre-image\n"); await run(linked, ["add", "tracked.txt"]); @@ -249,15 +264,37 @@ layer("RollbackWorkspace", (it) => { assert.equal(mainIdentity.gitCommonDir, linkedIdentity.gitCommonDir); assert.notEqual(mainIdentity.workspaceKey, linkedIdentity.workspaceKey); + const siblingFirst = yield* Effect.promise(async () => { + const oid = await run(main, [ + "commit-tree", + "HEAD^{tree}", + "-m", + "sibling first checkpoint", + ]); + await run(main, ["update-ref", siblingFirstRef, oid]); + return oid; + }); + const checkpoint = yield* workspace.resolveCheckpoint({ cwd: linked, - checkpointRef: "refs/t3/checkpoints/linked/turn/1", + checkpointRef: targetRef, }); const preimage = yield* workspace.capturePreimage({ operationId: "operation-linked", cwd: linked, + threadId, targetCheckpointOid: checkpoint.oid, }); + assert.deepEqual(preimage.ownedRefs, [{ ref: targetRef, oid: checkpoint.oid }]); + const siblingAdvanced = yield* Effect.promise(() => + run(main, ["commit-tree", "HEAD^{tree}", "-m", "sibling advanced checkpoint"]), + ); + yield* Effect.promise(async () => { + // Both worktrees share the Git common dir, but hold different + // rollback leases. B may write its refs while A is compensating. + await run(main, ["update-ref", siblingFirstRef, siblingAdvanced]); + await run(main, ["update-ref", siblingSecondRef, siblingFirst]); + }); yield* workspace.applyCheckpoint({ cwd: linked, checkpointOid: checkpoint.oid }); assert.equal( yield* Effect.promise(() => @@ -265,8 +302,26 @@ layer("RollbackWorkspace", (it) => { ), "checkpoint\n", ); - const restored = yield* workspace.restorePreimage({ cwd: linked, preimage }); + // Simulate a pre-upgrade persisted preimage containing all repository + // checkpoint refs. B's saved OID must have no recovery authority. + const legacyPreimage = { + ...preimage, + ownedRefs: [...preimage.ownedRefs, { ref: siblingFirstRef, oid: siblingFirst }], + }; + const restored = yield* workspace.restorePreimage({ + cwd: linked, + threadId, + preimage: legacyPreimage, + }); assert.equal(restored.digest, preimage.digest); + assert.equal( + yield* Effect.promise(() => run(main, ["rev-parse", siblingFirstRef])), + siblingAdvanced, + ); + assert.equal( + yield* Effect.promise(() => run(main, ["rev-parse", siblingSecondRef])), + siblingFirst, + ); assert.equal( yield* Effect.promise(() => NodeFSP.readFile(NodePath.join(linked, "tracked.txt"), "utf8"), diff --git a/apps/server/src/rollback/RollbackWorkspace.ts b/apps/server/src/rollback/RollbackWorkspace.ts index 1cce852ca6..a7a0e36826 100644 --- a/apps/server/src/rollback/RollbackWorkspace.ts +++ b/apps/server/src/rollback/RollbackWorkspace.ts @@ -9,6 +9,8 @@ import * as NodeFS from "node:fs"; import * as NodeFSP from "node:fs/promises"; import * as NodePath from "node:path"; import * as NodeUtil from "node:util"; +import type { ThreadId } from "@t3tools/contracts"; +import { checkpointRefPrefixForThread } from "../checkpointing/Utils.ts"; import { ServerConfig } from "../config.ts"; const execFileAsync = NodeUtil.promisify(NodeChildProcess.execFile); @@ -63,10 +65,12 @@ export interface RollbackWorkspaceShape { readonly capturePreimage: (input: { readonly operationId: string; readonly cwd: string; + readonly threadId: ThreadId; readonly targetCheckpointOid: string; }) => Effect.Effect; readonly restorePreimage: (input: { readonly cwd: string; + readonly threadId: ThreadId; readonly preimage: RollbackWorkspacePreimage; }) => Effect.Effect; readonly applyCheckpoint: (input: { @@ -229,37 +233,43 @@ async function clearMutablePaths( } } -async function restoreCheckpointRefs( +async function assertCheckpointRefsUnchanged( cwd: string, - ownedRefs: ReadonlyArray<{ readonly ref: string; readonly oid: string }>, + threadId: ThreadId, + savedRefs: ReadonlyArray<{ readonly ref: string; readonly oid: string }>, ): Promise { - const currentRaw = await git(cwd, [ - "for-each-ref", - "--format=%(refname)%00%(objectname)", - "refs/t3/checkpoints", - ]); - const currentRefs = currentRaw + if (!Array.isArray(savedRefs)) throw new Error("invalid-checkpoint-preimage"); + const prefix = checkpointRefPrefixForThread(threadId); + const expected = new Map(); + for (const entry of savedRefs) { + if (typeof entry?.ref !== "string" || typeof entry.oid !== "string") + throw new Error("invalid-checkpoint-preimage"); + // Older durable preimages contain every repository ref. Their only authority + // comes from the saga's persisted thread identity, never from this list. + if (!entry.ref.startsWith(prefix)) continue; + if (!/^[0-9a-f]{40,64}$/u.test(entry.oid) || expected.has(entry.ref)) + throw new Error("invalid-checkpoint-preimage"); + expected.set(entry.ref, entry.oid); + } + const current = await listThreadCheckpointRefs(cwd, threadId); + if (current.length !== expected.size || current.some(({ ref, oid }) => expected.get(ref) !== oid)) + throw new Error("checkpoint-ref-drift"); +} + +async function listThreadCheckpointRefs( + cwd: string, + threadId: ThreadId, +): Promise> { + const prefix = checkpointRefPrefixForThread(threadId); + const raw = await git(cwd, ["for-each-ref", "--format=%(refname)%00%(objectname)", prefix]); + return raw .split("\n") .filter(Boolean) - .map((line) => line.split("\0")[0] ?? ""); - const desired = new Map(ownedRefs.map((entry) => [entry.ref, entry.oid])); - const commands = [ - ...currentRefs.filter((ref) => !desired.has(ref)).map((ref) => `delete ${ref}`), - ...ownedRefs.map((entry) => `update ${entry.ref} ${entry.oid}`), - ]; - if (commands.length === 0) return; - await new Promise((resolve, reject) => { - const child = NodeChildProcess.execFile( - "git", - ["update-ref", "--stdin"], - { cwd, timeout: 30_000, windowsHide: true }, - (error) => { - if (error) reject(error); - else resolve(); - }, - ); - child.stdin?.end(`${commands.join("\n")}\n`); - }); + .map((line) => { + const [ref = "", oid = ""] = line.split("\0"); + return { ref, oid }; + }) + .filter(({ ref }) => ref.startsWith(prefix)); } async function resolveHead(cwd: string): Promise<{ symbolic: string | null; oid: string | null }> { @@ -407,18 +417,7 @@ const make = Effect.gen(function* () { ); if (indexExisted) await NodeFSP.copyFile(indexPath, NodePath.join(backupPath, "index")); const head = await resolveHead(identity.cwd); - const refsRaw = await git(identity.cwd, [ - "for-each-ref", - "--format=%(refname)%00%(objectname)", - "refs/t3/checkpoints", - ]); - const ownedRefs = refsRaw - .split("\n") - .filter(Boolean) - .map((line) => { - const [ref = "", oid = ""] = line.split("\0"); - return { ref, oid }; - }); + const ownedRefs = await listThreadCheckpointRefs(identity.cwd, input.threadId); const receipt = await inspectWorkspace(identity.cwd, mutablePaths); return { backupPath, @@ -443,6 +442,7 @@ const make = Effect.gen(function* () { beforeHead.oid !== input.preimage.headOid ) throw new Error("head-drift"); + await assertCheckpointRefsUnchanged(identity.cwd, input.threadId, input.preimage.ownedRefs); await clearMutablePaths(identity.cwd, input.preimage.paths); await copyBackup(NodePath.join(input.preimage.backupPath, "workspace"), identity.cwd); if (input.preimage.indexExisted) { @@ -454,7 +454,7 @@ const make = Effect.gen(function* () { } else { await NodeFSP.rm(input.preimage.indexPath, { force: true }); } - await restoreCheckpointRefs(identity.cwd, input.preimage.ownedRefs); + await assertCheckpointRefsUnchanged(identity.cwd, input.threadId, input.preimage.ownedRefs); const receipt = await inspectWorkspace(identity.cwd, input.preimage.paths); if (receipt.digest !== input.preimage.digest) throw new Error("preimage-postcondition"); return receipt; diff --git a/apps/server/src/server.test.ts b/apps/server/src/server.test.ts index e3eb8af1cb..ad3522a770 100644 --- a/apps/server/src/server.test.ts +++ b/apps/server/src/server.test.ts @@ -142,6 +142,12 @@ import { OrchestrationEventStore } from "./persistence/Services/OrchestrationEve import { RollbackSagaRepository } from "./persistence/Services/RollbackSagas.ts"; import { PersistenceSqlError } from "./persistence/Errors.ts"; import * as ProviderRegistry from "./provider/Services/ProviderRegistry.ts"; +import * as ModelManifest from "./provider/ModelManifest.ts"; +import { + ProviderVersionCache, + type ProviderVersionCacheEntry, +} from "./provider/providerMaintenance.ts"; +import type { ProviderInstance } from "./provider/ProviderDriver.ts"; import { ProviderAdapterUnsupportedOperationError, ProviderValidationError, @@ -225,6 +231,7 @@ import { } from "../integration/TransferBudgetReport.integration.ts"; import { symlinksSupported } from "@t3tools/shared/testing/symlinks"; import { DEFAULT_SIGNAL_EXPORT, otlpSerializationLayer } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; const defaultProjectId = ProjectId.make("project-default"); const defaultThreadId = ThreadId.make("thread-default"); @@ -506,6 +513,8 @@ const buildAppUnderTest = (options?: { antigravityInstallation?: Partial; environmentTheme?: Partial; providerRegistry?: Partial; + modelManifest?: Partial; + providerVersionCache?: Map; providerService?: Partial; rollbackSagaRepository?: Partial; serverSettings?: Partial; @@ -568,6 +577,7 @@ const buildAppUnderTest = (options?: { otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "t3-server", + otelEnvironment: OtelEnvironment.none, mode: "desktop", port: 0, host: "127.0.0.1", @@ -767,6 +777,14 @@ const buildAppUnderTest = (options?: { getInstance: () => Effect.succeed(undefined), ...options?.layers?.providerInstances, }), + Layer.mock(ModelManifest.ModelManifest)({ + current: Effect.succeed(ModelManifest.BUNDLED_MODEL_MANIFEST), + refresh: Effect.succeed(ModelManifest.BUNDLED_MODEL_MANIFEST), + forceRefresh: Effect.succeed(ModelManifest.BUNDLED_MODEL_MANIFEST), + refreshInBackground: Effect.void, + ...options?.layers?.modelManifest, + }), + Layer.succeed(ProviderVersionCache, options?.layers?.providerVersionCache ?? new Map()), Layer.mock(EnvironmentTheme.EnvironmentThemeService)({ current: Effect.succeed([]), streamChanges: Stream.empty, @@ -6428,6 +6446,93 @@ it.layer(NodeServices.layer)("server router seam", (it) => { }).pipe(Effect.provide(NodeHttpServer.layerTest)), ); + for (const mode of ["all", "targeted", "background"] as const) { + it.effect(`provider refresh invalidates owned caches before probing (${mode})`, () => { + const driver = ProviderDriverKind.make("codex"); + const instanceIds = [ProviderInstanceId.make("codex"), ProviderInstanceId.make("codex_work")]; + const packageNames = ["@example/personal", "@example/work"]; + const versionCache = new Map( + packageNames.map((name) => [ + name, + { expiresAt: Number.MAX_SAFE_INTEGER, version: "1.0.0" }, + ]), + ); + const invalidated: string[] = []; + const freshMaintenance: string[] = []; + let manifestRefreshed = false; + let probed = false; + const instances = instanceIds.map( + (instanceId, index) => + ({ + instanceId, + driverKind: driver, + continuationIdentity: { driverKind: driver, continuationKey: instanceId }, + displayName: undefined, + enabled: true, + invalidateCaches: Effect.sync(() => { + invalidated.push(instanceId); + }), + snapshot: { + resolveMaintenance: (options) => + Effect.sync(() => { + assert.isTrue(options?.fresh); + freshMaintenance.push(instanceId); + return makeManualOnlyProviderMaintenanceCapabilities({ + provider: driver, + packageName: packageNames[index]!, + }); + }), + getSnapshot: Effect.never, + refresh: Effect.never, + streamChanges: Stream.empty, + }, + adapter: {} as ProviderInstance["adapter"], + textGeneration: {} as ProviderInstance["textGeneration"], + }) satisfies ProviderInstance, + ); + const expected = + mode === "background" ? [] : mode === "targeted" ? [instanceIds[1]!] : instanceIds; + const probe = Effect.sync(() => { + probed = true; + assert.equal(manifestRefreshed, mode !== "background"); + assert.deepEqual(invalidated.toSorted(), expected.toSorted()); + assert.deepEqual(freshMaintenance.toSorted(), expected.toSorted()); + for (let index = 0; index < instanceIds.length; index++) { + assert.equal( + versionCache.has(packageNames[index]!), + !expected.includes(instanceIds[index]!), + ); + } + return []; + }); + return Effect.gen(function* () { + yield* buildAppUnderTest({ + layers: { + modelManifest: { + forceRefresh: Effect.sync(() => { + manifestRefreshed = true; + return ModelManifest.BUNDLED_MODEL_MANIFEST; + }), + }, + providerVersionCache: versionCache, + providerInstances: { listInstances: Effect.succeed(instances) }, + providerRegistry: { refresh: () => probe, refreshInstance: () => probe }, + }, + }); + const wsUrl = yield* getWsServerUrl("/ws"); + yield* Effect.scoped( + withWsRpcClient(wsUrl, (client) => + client[WS_METHODS.serverRefreshProviders]({ + ...(mode === "targeted" ? { instanceId: instanceIds[1]! } : {}), + ...(mode !== "background" ? { refreshModels: true } : {}), + }), + ), + ); + assert.isTrue(probed); + }).pipe(Effect.provide(NodeHttpServer.layerTest)); + }); + } + it.effect("serves config on reconnect without starting provider probes", () => Effect.gen(function* () { const refresh = vi.fn(() => Effect.never); @@ -11119,6 +11224,7 @@ it.layer(NodeServices.layer)("server router seam", (it) => { assert.deepEqual(fetchRemote.mock.calls[0]?.[0], { cwd: "/tmp/project", remoteName: "origin", + refName: "main", }); assert.deepEqual(remoteBranchExists.mock.calls[0]?.[0], { cwd: "/tmp/project", diff --git a/apps/server/src/serverLogger.test.ts b/apps/server/src/serverLogger.test.ts index d6abdfd197..c259526a2a 100644 --- a/apps/server/src/serverLogger.test.ts +++ b/apps/server/src/serverLogger.test.ts @@ -9,6 +9,7 @@ import * as HttpClient from "effect/unstable/http/HttpClient"; import * as HttpClientResponse from "effect/unstable/http/HttpClientResponse"; import { DEFAULT_SIGNAL_EXPORT } from "@t3tools/shared/observability"; +import * as OtelEnvironment from "@t3tools/shared/otelEnvironment"; import * as ServerConfig from "./config.ts"; import { ServerLoggerLive } from "./serverLogger.ts"; @@ -57,6 +58,7 @@ const configLayer = (overrides: Partial) = otlpMetricsExport: DEFAULT_SIGNAL_EXPORT, otlpLogsExport: DEFAULT_SIGNAL_EXPORT, otlpServiceName: "pylon-server", + otelEnvironment: OtelEnvironment.none, cwd: baseDir, baseDir, ...derivedPaths, diff --git a/apps/server/src/usage/UsageService.test.ts b/apps/server/src/usage/UsageService.test.ts index ec1c88ccfd..549d0a96a0 100644 --- a/apps/server/src/usage/UsageService.test.ts +++ b/apps/server/src/usage/UsageService.test.ts @@ -8,7 +8,9 @@ import * as NodeSqlite from "node:sqlite"; import { assert, describe, it } from "@effect/vitest"; import * as NodeServices from "@effect/platform-node/NodeServices"; import { HostProcessEnvironment } from "@t3tools/shared/hostProcess"; +import { mergeUsage } from "@t3tools/shared/usageMerge"; import { + EnvironmentId, ProviderDriverKind, ProviderInstanceId, ServerProviderInstancesMutationId, @@ -139,7 +141,7 @@ describe("UsageService", () => { await NodeFSP.mkdir(NodePath.join(claudeHome, "projects"), { recursive: true }); await NodeFSP.writeFile( NodePath.join(claudeHome, "projects", "session.jsonl"), - claudeLine(2, 7), + claudeLine(1, 5) + claudeLine(2, 7), ); await NodeFSP.mkdir(NodePath.join(codexHome, "sessions"), { recursive: true }); await NodeFSP.symlink(codexHome, alias, "junction"); @@ -148,14 +150,15 @@ describe("UsageService", () => { [ { type: "session_meta", payload: { id: "codex-account-session" } }, { type: "turn_context", payload: { model: "gpt-5.6-sol" } }, - { + // A-B-A at one timestamp must preserve both equal A events. + ...[11, 12, 11].map((outputTokens) => ({ type: "event_msg", timestamp: "2026-08-01T10:00:00Z", payload: { type: "token_count", - info: { last_token_usage: { input_tokens: 10, output_tokens: 11 } }, + info: { last_token_usage: { input_tokens: 10, output_tokens: outputTokens } }, }, - }, + })), ] .map((line) => encodeUnknownJsonString(line)) .join("\n") + "\n", @@ -215,7 +218,21 @@ describe("UsageService", () => { ), ); const summary = yield* service.readSummary(WINDOW); - assert.strictEqual(totalOutputTokens(summary), 36); + assert.strictEqual(totalOutputTokens(summary), 59); + yield* Effect.promise(() => + NodeFSP.rename( + NodePath.join(codexHome, "sessions", "rollout.jsonl"), + NodePath.join(codexHome, "sessions", "moved.jsonl"), + ), + ); + const moved = yield* service.readSummary(WINDOW); + assert.deepStrictEqual(moved.buckets, summary.buckets); + yield* Effect.promise(() => + NodeFSP.rm(NodePath.join(codexHome, "sessions"), { recursive: true }), + ); + const removed = yield* service.readSummary(WINDOW); + assert.deepStrictEqual(removed.buckets, summary.buckets); + const sources = summary.sources.filter((source) => source.status === "ok"); assert.strictEqual(sources.length, 4); assert.strictEqual( @@ -226,6 +243,20 @@ describe("UsageService", () => { sources.filter((source) => source.fingerprint.provider === "codex").length, 1, ); + assert.strictEqual( + sources.reduce( + (sum, source) => + sum + + (source.buckets ?? []).reduce((total, bucket) => total + bucket.totals.outputTokens, 0), + 0, + ), + totalOutputTokens(summary), + ); + assert.isTrue( + sources.every((source) => + source.buckets?.every((bucket) => bucket.provider === source.fingerprint.provider), + ), + ); }).pipe(Effect.scoped), ); @@ -458,6 +489,128 @@ describe("UsageService", () => { }).pipe(Effect.scoped), ); + it.live("preserves saved tokens, costs and sessions after transcript cleanup and restart", () => + Effect.gen(function* () { + const { transcript, settings, home } = yield* setup; + const alias = NodePath.join(home, "claude-alias"); + yield* Effect.promise(() => + NodeFSP.symlink(NodePath.join(home, "claude"), alias, "junction"), + ); + const content = claudeLine(1, 5); + yield* Effect.promise(() => NodeFSP.writeFile(transcript, content)); + yield* Effect.gen(function* () { + const service = yield* UsageService.make; + const first = yield* service.readSummary(WINDOW); + assert.strictEqual(totalOutputTokens(first), 5); + assert.isAbove(first.buckets[0]?.costUsd ?? 0, 0); + + yield* Effect.promise(() => NodeFSP.rm(transcript)); + const deleted = yield* service.readSummary(WINDOW); + assert.deepStrictEqual(deleted.buckets, first.buckets); + assert.deepStrictEqual(deleted.sources, first.sources); + + const restarted = yield* UsageService.make; + const restored = yield* restarted.readSummary(WINDOW); + assert.deepStrictEqual(restored.buckets, first.buckets); + assert.deepStrictEqual(restored.sources, first.sources); + + // A moved transcript must not count the saved usage twice. + yield* Effect.promise(() => NodeFSP.writeFile(transcript + ".jsonl", content)); + const moved = yield* restarted.readSummary(WINDOW); + assert.deepStrictEqual(moved.buckets, first.buckets); + assert.strictEqual(moved.sources[0]?.distinctSessions, 1); + + const replacementProjects = NodePath.join(home, "replacement-projects"); + yield* Effect.promise(() => NodeFSP.mkdir(replacementProjects)); + yield* Effect.promise(() => + NodeFSP.rm(NodePath.join(home, "claude", "projects"), { recursive: true }), + ); + const afterRootCleanup = yield* UsageService.make; + const missingRoot = yield* afterRootCleanup.readSummary(WINDOW); + assert.deepStrictEqual(missingRoot.buckets, first.buckets); + assert.strictEqual(missingRoot.sources[0]?.distinctSessions, 1); + assert.strictEqual(missingRoot.sources[0]?.status, "ok"); + assert.deepStrictEqual(missingRoot.sources[0]?.fingerprint, first.sources[0]?.fingerprint); + yield* Effect.promise(async () => { + const projects = NodePath.join(home, "claude", "projects"); + await NodeFSP.rename(replacementProjects, projects); + await NodeFSP.writeFile(NodePath.join(projects, "new.jsonl"), claudeLine(2, 7)); + }); + const recreated = yield* afterRootCleanup.readSummary(WINDOW); + assert.strictEqual(totalOutputTokens(recreated), 12); + assert.deepStrictEqual(recreated.sources[0]?.fingerprint, first.sources[0]?.fingerprint); + + const merged = mergeUsage( + [ + { + environmentId: EnvironmentId.make("cleanup-test"), + label: "test", + summary: recreated, + }, + { + environmentId: EnvironmentId.make("other-environment"), + label: "before cleanup", + summary: first, + }, + ], + missingRoot.contractVersion, + ); + assert.strictEqual(merged.outputTokens, 12); + assert.strictEqual(merged.sessions, 1); + assert.strictEqual(merged.costUsd, recreated.buckets[0]?.costUsd); + + const outsideWindow = yield* restarted.readSummary({ + ...WINDOW, + sinceDay: UsageDay.make("2026-08-02"), + }); + assert.deepStrictEqual(outsideWindow.buckets, []); + assert.strictEqual(outsideWindow.sources[0]?.distinctSessions, 0); + }).pipe( + Effect.provide( + serviceLayers({ + prefix: "usage-service-cleanup-test", + home, + settings: { providers: { ...settings.providers, claudeAgent: { homePath: alias } } }, + ratesDocument: { + "claude-fable-5": { input_cost_per_token: 1e-5, output_cost_per_token: 5e-5 }, + }, + }), + ), + ); + }).pipe(Effect.scoped), + ); + + it.live("drops stale source identities after an empty account home is replaced", () => + Effect.gen(function* () { + const { home, settings } = yield* setup; + const replacement = NodePath.join(home, "replacement-claude"); + yield* Effect.promise(() => + NodeFSP.mkdir(NodePath.join(replacement, "projects"), { recursive: true }), + ); + yield* Effect.gen(function* () { + const config = yield* ServerConfig.ServerConfig; + const settingsService = yield* ServerSettings.ServerSettingsService; + const service = yield* UsageService.make; + yield* service.readSummary(WINDOW); + const cachePath = NodePath.join(config.stateDir, "usage-scan-cache.json"); + const oldDir = NodePath.join(home, "claude", "projects"); + assert.include(yield* Effect.promise(() => NodeFSP.readFile(cachePath, "utf8")), oldDir); + + yield* settingsService.updateSettings({ + providers: { claudeAgent: { homePath: replacement } }, + }); + yield* service.readSummary(WINDOW); + const persisted = yield* Effect.promise(() => NodeFSP.readFile(cachePath, "utf8")); + assert.notInclude(persisted, oldDir); + assert.include(persisted, NodePath.join(replacement, "projects")); + }).pipe( + Effect.provide( + serviceLayers({ prefix: "usage-service-source-prune-test", home, settings }), + ), + ); + }).pipe(Effect.scoped), + ); + it.live("does not share an in-flight scan after custom prices change", () => Effect.gen(function* () { const { transcript, settings, home } = yield* setup; @@ -769,6 +922,120 @@ describe("UsageService", () => { ), ); + it.live("retains Antigravity usage after conversation cleanup and service restart", () => + Effect.gen(function* () { + yield* setup; + const config = yield* ServerConfig.ServerConfig; + const instanceId = ProviderInstanceId.make("antigravity-retained"); + const profileDir = resolveAntigravityProfileDirectory(config.stateDir, instanceId); + const convDir = NodePath.join(profileDir, "antigravity-acp", "conversations"); + yield* Effect.promise(() => NodeFSP.mkdir(convDir, { recursive: true })); + const dbPath = NodePath.join(convDir, "retained.db"); + seedAntigravityDb(dbPath, [ + { + idx: 0, + blob: encodeSyntheticGenMetadataBlob({ + modelName: "gemini-2.5-pro", + timestampSeconds: 1785578400n, + inputTokens: 100, + outputTokens: 40, + }), + }, + ]); + + const service = yield* UsageService.make; + const first = yield* service.readSummary(WINDOW); + yield* Effect.promise(() => NodeFSP.rm(convDir, { recursive: true })); + const afterCleanup = yield* service.readSummary(WINDOW); + const restarted = yield* UsageService.make; + const afterRestart = yield* restarted.readSummary(WINDOW); + + for (const [phase, summary] of [ + ["cleanup", afterCleanup], + ["restart", afterRestart], + ] as const) { + assert.deepStrictEqual(summary.buckets, first.buckets, phase); + const source = summary.sources.find( + (item) => + item.fingerprint.provider === "antigravity" && + item.fingerprint.resolvedHomePath === convDir, + ); + assert.isDefined(source); + assert.strictEqual(source.status, "ok"); + assert.strictEqual(source.scannedFiles, 1); + } + }).pipe( + Effect.provide( + serviceLayers({ + prefix: "usage-service-antigravity-retained-test", + home: "", + settings: { + providers: {}, + providerInstances: { + [ProviderInstanceId.make("antigravity-retained")]: { + driver: "antigravity", + enabled: true, + }, + }, + }, + }), + ), + Effect.scoped, + ), + ); + + it.live( + "retains standalone Antigravity history after its directory is removed and service restarts", + () => + Effect.gen(function* () { + yield* setup; + const config = yield* ServerConfig.ServerConfig; + const convDir = NodePath.join(config.stateDir, "standalone-antigravity", "conversations"); + yield* Effect.promise(() => NodeFSP.mkdir(convDir, { recursive: true })); + seedAntigravityDb(NodePath.join(convDir, "session.db"), [ + { + idx: 0, + blob: encodeSyntheticGenMetadataBlob({ + modelName: "gemini-2.5-pro", + timestampSeconds: 1785578400n, + inputTokens: 100, + outputTokens: 40, + }), + }, + ]); + + yield* Effect.gen(function* () { + const firstService = yield* UsageService.make; + const first = yield* firstService.readSummary(WINDOW); + assert.isTrue(first.buckets.some((bucket) => bucket.provider === "antigravity")); + yield* Effect.promise(() => NodeFSP.rm(convDir, { recursive: true })); + + for (const summary of [ + yield* firstService.readSummary(WINDOW), + yield* (yield* UsageService.make).readSummary(WINDOW), + ]) { + assert.deepStrictEqual(summary.buckets, first.buckets); + const source = summary.sources.find( + (item) => + item.fingerprint.provider === "antigravity" && + item.fingerprint.resolvedHomePath === convDir, + ); + assert.isDefined(source); + assert.strictEqual(source.status, "ok"); + } + }).pipe(Effect.provideService(UsageService.StandaloneAntigravityConversations, convDir)); + }).pipe( + Effect.provide( + serviceLayers({ + prefix: "usage-service-standalone-antigravity-retained-test", + home: "", + settings: { providers: {}, providerInstances: {} }, + }), + ), + Effect.scoped, + ), + ); + it.live( "reports honest missing status when configured Antigravity instance has no conversations dir on disk", () => diff --git a/apps/server/src/usage/UsageService.ts b/apps/server/src/usage/UsageService.ts index 122b89436e..7ad01d09c4 100644 --- a/apps/server/src/usage/UsageService.ts +++ b/apps/server/src/usage/UsageService.ts @@ -14,6 +14,8 @@ * @module UsageService */ import * as NodeOS from "node:os"; +// @effect-diagnostics nodeBuiltinImport:off - Context.Reference defaults cannot request Path. +import * as NodePath from "node:path"; import { ClaudeSettings, @@ -90,6 +92,18 @@ const MAX_HOURLY_WINDOW_MS = 24 * 60 * 60 * 1000; /** Longest window the UI offers, plus slack. Older entries are pruned. */ const CACHE_RETENTION_DAYS = 90; +const MAX_ANTIGRAVITY_FILES_PER_DIR = 500; +const MAX_ANTIGRAVITY_BYTES_PER_DIR = 256 * 1024 * 1024; +const MAX_ANTIGRAVITY_RECORDS_PER_DIR = 50_000; + +/** The standalone Antigravity home belongs to this server environment. */ +export const StandaloneAntigravityConversations = Context.Reference( + "StandaloneAntigravityConversations", + { + defaultValue: () => + NodePath.join(NodeOS.homedir(), ".gemini", "antigravity-cli", "conversations"), + }, +); const decodeCodexSettings = Schema.decodeOption(CodexSettings); const decodeClaudeSettings = Schema.decodeOption(ClaudeSettings); @@ -110,6 +124,11 @@ const encodeRatesCache = Schema.encodeEffect( const ScanCacheJson = Schema.fromJsonString(Schema.Unknown as unknown as Schema.Codec); const decodeScanCacheFile = Schema.decodeUnknownEffect(ScanCacheJson); const encodeScanCacheFile = Schema.encodeEffect(ScanCacheJson); +const encodeUsageRecordKey = Schema.encodeSync(ScanCacheJson); +const CachedSource = Schema.Struct({ dir: Schema.String, volumeId: Schema.String }); +const decodeCachedSources = Schema.decodeUnknownOption( + Schema.Struct({ sources: Schema.Record(Schema.String, CachedSource) }), +); export class UsageService extends Context.Service< UsageService, @@ -156,7 +175,12 @@ export const make = Effect.gen(function* () { const hostEnvironment = yield* HostProcessEnvironment; const fileCache: ScanCache = new Map(); + const sourceCache = new Map(); let cacheDirty = false; + const isWithinDirectory = (filePath: string, dir: string) => { + const relative = path.relative(dir, filePath); + return relative !== ".." && !relative.startsWith(".." + path.sep) && !path.isAbsolute(relative); + }; const ratesCachePath = path.join(config.stateDir, "usage-model-rates.json"); const scanCachePath = path.join(config.stateDir, "usage-scan-cache.json"); @@ -250,9 +274,16 @@ export const make = Effect.gen(function* () { /** Resolves the transcript directory for each provider. */ const resolveTranscriptDirs = Effect.fn("UsageService.resolveTranscriptDirs")(function* ( settings: ServerSettingsValue, + retentionCutoffMs: number, ) { - const dirs: Array<{ provider: UsageProviderKind; dir: string; fileName?: string }> = []; + const dirs: Array<{ + provider: UsageProviderKind; + dir: string; + volumeId: string; + fileName?: string; + }> = []; const seen = new Set(); + const activeSourceKeys = new Set(); for (const driver of ["claudeAgent", "codex", "grok"] as const) { // Disabled accounts still have history. Explicit default slots replace // the legacy settings, just as they do in the provider registry. @@ -289,14 +320,42 @@ export const make = Effect.gen(function* () { ); } const directory = path.resolve(home, provider === "claude" ? "projects" : "sessions"); - // Account aliases and Codex auth overlays can share the same history. + const sourceKey = provider + "\0" + directory; + const previous = sourceCache.get(sourceKey); + // Keep canonical paths and source fingerprints stable after root cleanup, + // including aliases and clients merging pre-cleanup environment summaries. const dir = yield* fileSystem .realPath(directory) - .pipe(Effect.orElseSucceed(() => directory)); + .pipe(Effect.orElseSucceed(() => previous?.dir ?? directory)); + const currentVolumeId = yield* Effect.promise(() => readDirectoryVolumeId(dir)); + const hasRetainedHistory = fileCache + .entries() + .some( + ([filePath, entry]) => + entry.provider === provider && + entry.mtimeMs >= retentionCutoffMs && + entry.records.length + entry.tailRecords.length > 0 && + isWithinDirectory(filePath, dir), + ); + // A recreated directory still reports the retained history under its old identity. + const volumeId = + previous?.dir === dir && (hasRetainedHistory || !currentVolumeId) + ? previous.volumeId || currentVolumeId + : currentVolumeId; + if (previous?.dir !== dir || previous.volumeId !== volumeId) { + sourceCache.set(sourceKey, { dir, volumeId }); + cacheDirty = true; + } const key = `${provider}\0${dir}`; + activeSourceKeys.add(sourceKey); if (seen.has(key)) continue; seen.add(key); - dirs.push({ provider, dir, ...(provider === "grok" ? { fileName: "updates.jsonl" } : {}) }); + dirs.push({ + provider, + dir, + volumeId, + ...(provider === "grok" ? { fileName: "updates.jsonl" } : {}), + }); } } // Antigravity profile roots: resolve configured instances @@ -324,21 +383,69 @@ export const make = Effect.gen(function* () { } // Standalone ~/.gemini/antigravity-cli/conversations when present - const standaloneDir = path.join( - NodeOS.homedir(), - ".gemini", - "antigravity-cli", - "conversations", - ); + const standaloneDir = yield* StandaloneAntigravityConversations; const standaloneExists = yield* fileSystem .exists(standaloneDir) .pipe(Effect.catchCause(() => Effect.succeed(false))); - if (standaloneExists) { + const hasStandaloneHistory = fileCache + .entries() + .some( + ([filePath, entry]) => + entry.provider === "antigravity" && + entry.mtimeMs >= retentionCutoffMs && + entry.records.length + entry.tailRecords.length > 0 && + isWithinDirectory(filePath, standaloneDir), + ); + if (standaloneExists || hasStandaloneHistory) { antigravityRoots.add(standaloneDir); } for (const root of antigravityRoots) { - dirs.push({ provider: "antigravity" as const, dir: root }); + const directory = path.resolve(root); + const sourceKey = "antigravity\0" + directory; + const previous = sourceCache.get(sourceKey); + // Pylon's Antigravity profile fingerprint is the configured profile + // path, not its realpath. Keep it stable across profile cleanup. + const dir = directory; + const currentVolumeId = yield* Effect.promise(() => readDirectoryVolumeId(dir)); + const hasRetainedHistory = fileCache + .entries() + .some( + ([filePath, entry]) => + entry.provider === "antigravity" && + entry.mtimeMs >= retentionCutoffMs && + entry.records.length + entry.tailRecords.length > 0 && + isWithinDirectory(filePath, dir), + ); + const volumeId = + previous?.dir === dir && (hasRetainedHistory || !currentVolumeId) + ? previous.volumeId || currentVolumeId + : currentVolumeId; + if (previous?.dir !== dir || previous.volumeId !== volumeId) { + sourceCache.set(sourceKey, { dir, volumeId }); + cacheDirty = true; + } + activeSourceKeys.add(sourceKey); + dirs.push({ provider: "antigravity", dir, volumeId }); + } + + // Old configured homes can disappear from settings. Keep their identity + // only while an in-retention cached transcript could use it again. + for (const [sourceKey, source] of sourceCache) { + if (activeSourceKeys.has(sourceKey)) continue; + const provider = sourceKey.split("\0", 1)[0]; + const hasRetainedHistory = fileCache + .entries() + .some( + ([filePath, entry]) => + entry.provider === provider && + entry.mtimeMs >= retentionCutoffMs && + entry.records.length + entry.tailRecords.length > 0 && + isWithinDirectory(filePath, source.dir), + ); + if (hasRetainedHistory) continue; + sourceCache.delete(sourceKey); + cacheDirty = true; } return dirs; @@ -359,6 +466,11 @@ export const make = Effect.gen(function* () { ); if (document === null) return; for (const [path, entry] of decodeScanCache(document)) fileCache.set(path, entry); + const sources = decodeCachedSources(document); + if (Option.isSome(sources)) { + for (const [key, source] of Object.entries(sources.value.sources)) + sourceCache.set(key, source); + } }), ); @@ -366,7 +478,10 @@ export const make = Effect.gen(function* () { if (!cacheDirty) return; // Cleared only after the write lands, so a failed persist is retried on // the next scan instead of leaving disk permanently stale. - yield* encodeScanCacheFile(encodeScanCache(fileCache)).pipe( + yield* encodeScanCacheFile({ + ...encodeScanCache(fileCache), + sources: Object.fromEntries(sourceCache), + }).pipe( Effect.flatMap((serialized) => fileSystem.writeFileString(scanCachePath, serialized)), Effect.map(() => { cacheDirty = false; @@ -417,7 +532,8 @@ export const make = Effect.gen(function* () { ); // A read failure is not an empty transcript: caching it under this // (size, mtime) would silently drop the file's usage until it changes. - if (parsed === null) return []; + if (parsed === null) + return cached?.provider === provider ? [...cached.records, ...cached.tailRecords] : []; // Stored already de-duplicated within the file, which is 99% of all // duplicates. The aggregator still runs the cross-file dedupe pass. One @@ -452,24 +568,25 @@ export const make = Effect.gen(function* () { readonly status?: "ok" | "missing" | "partial" | "failed"; readonly malformedRecords?: number; readonly message?: string | null; + readonly nativeBudget?: { + readonly files: number; + readonly bytes: number; + readonly records: number; + }; } const collectDirs = Effect.fn("UsageService.collectDirs")(function* ( windowStartMs: number, settings: ServerSettingsValue, + retentionCutoffMs: number, ) { // The home resolvers ask for `Path` themselves; satisfy them from the // instance we already hold so the scan stays context-free. - const dirs = yield* resolveTranscriptDirs(settings).pipe( + const dirs = yield* resolveTranscriptDirs(settings, retentionCutoffMs).pipe( Effect.provideService(Path.Path, path), ); const scanned: ScannedDir[] = []; - // Bound native SQLite work across all instance roots, including warm-cache records. - let nativeFiles = 0; - let nativeBytes = 0; - let nativeRecords = 0; - for (const { provider, dir, fileName } of dirs) { - const volumeId = yield* Effect.promise(() => readDirectoryVolumeId(dir)); + for (const { provider, dir, volumeId, fileName } of dirs) { const exists = yield* fileSystem .exists(dir) .pipe(Effect.catchCause(() => Effect.succeed(false))); @@ -479,6 +596,10 @@ export const make = Effect.gen(function* () { } if (provider === "antigravity") { + // Native SQLite limits apply to each profile directory independently. + let nativeFiles = 0; + let nativeBytes = 0; + let nativeRecords = 0; const dirEntries = yield* fileSystem .readDirectory(dir) .pipe(Effect.catchCause(() => Effect.succeed(null))); @@ -497,10 +618,6 @@ export const make = Effect.gen(function* () { const dbFileNames = dirEntries.filter((name) => name.endsWith(".db")).sort(); - const MAX_ANTIGRAVITY_FILES_PER_DIR = 500; - const MAX_ANTIGRAVITY_BYTES_PER_DIR = 256 * 1024 * 1024; - const MAX_ANTIGRAVITY_RECORDS_PER_DIR = 50_000; - const parsedFiles: { path: string; records: readonly UsageRecord[] }[] = []; let dirMalformedRows = 0; let dirTruncated = false; @@ -683,6 +800,7 @@ export const make = Effect.gen(function* () { status, malformedRecords: dirMalformedRows, message, + nativeBudget: { files: nativeFiles, bytes: nativeBytes, records: nativeRecords }, }); continue; } @@ -749,11 +867,13 @@ export const make = Effect.gen(function* () { const windowStartMs = (hourlyWindow?.sinceTimeMs ?? DateTime.toEpochMillis(windowStart.value)) - MTIME_SLACK_MS; + const retentionCutoffMs = startedAtMs - CACHE_RETENTION_DAYS * 24 * 60 * 60 * 1000; + // Pricing only matters once records are aggregated, so the rate table // loads while transcripts stream instead of gating them: a cold rates // fetch on a slow network no longer delays the scan by its own timeout. const [, scannedDirs] = yield* Effect.all( - [ensureRates(false), collectDirs(windowStartMs, settings)], + [ensureRates(false), collectDirs(windowStartMs, settings, retentionCutoffMs)], { concurrency: 2 }, ); @@ -768,8 +888,6 @@ export const make = Effect.gen(function* () { }); const sources: UsageSource[] = []; - const livePaths = new Set(); - const walkedRoots: string[] = []; for (const { provider, @@ -779,60 +897,114 @@ export const make = Effect.gen(function* () { status, malformedRecords, message, + nativeBudget, } of scannedDirs) { - if (files === null) { - sources.push({ - fingerprint: { hostId, provider, resolvedHomePath: dir, volumeId }, - status: "missing", - scannedFiles: 0, - skippedFiles: 0, - malformedRecords: 0, - distinctSessions: 0, - message: "No transcript directory on this environment.", - }); - continue; + const retainedFiles = [...(files ?? [])]; + let retainedTruncated = false; + let retainedNativeFiles = nativeBudget?.files ?? 0; + let retainedNativeBytes = nativeBudget?.bytes ?? 0; + let retainedNativeRecords = nativeBudget?.records ?? 0; + const livePaths = new Set(retainedFiles.map((file) => file.path)); + // Cleanup may remove transcripts, but the usage we already saved still + // contributes to this source. Keep the normal aggregation and dedupe path. + for (const [filePath, entry] of fileCache) { + if ( + entry.provider !== provider || + entry.mtimeMs < retentionCutoffMs || + livePaths.has(filePath) || + !isWithinDirectory(filePath, dir) + ) + continue; + if (provider === "antigravity") { + const recordCount = entry.records.length + entry.tailRecords.length; + if ( + retainedNativeFiles + 1 > MAX_ANTIGRAVITY_FILES_PER_DIR || + retainedNativeBytes + entry.size > MAX_ANTIGRAVITY_BYTES_PER_DIR || + retainedNativeRecords + recordCount > MAX_ANTIGRAVITY_RECORDS_PER_DIR + ) { + retainedTruncated = true; + continue; + } + retainedNativeFiles++; + retainedNativeBytes += entry.size; + retainedNativeRecords += recordCount; + } + retainedFiles.push({ path: filePath, records: [...entry.records, ...entry.tailRecords] }); } - - walkedRoots.push(dir); let scannedFiles = 0; let skippedFiles = 0; // Distinct per directory. Buckets carry per-cell session counts, but a // session spans days and models, so clients total this figure instead. const sessionIds = new Set(); + // Keep attribution at the physical directory boundary. The primary + // aggregator decides scan-wide de-duplication first, so source buckets + // partition the same accepted records as the legacy flat buckets. + const sourceAggregator = new UsageAggregator({ + timeZone: input.timeZone, + sinceDay: input.sinceDay, + untilDay: input.untilDay, + resolution: input.resolution ?? "day", + ...hourlyWindow, + rates, + priceOverrides: createOverrideRateTable(settings.usagePriceOverrides), + }); - for (const file of files) { - livePaths.add(file.path); + for (const file of retainedFiles) { if (file.records.length === 0) { skippedFiles += 1; continue; } scannedFiles += 1; + const codexEventOccurrences = new Map(); for (const record of file.records) { - // Only sessions that contributed in-window count: the mtime slack - // admits boundary files whose records fall outside the range. - if (aggregator.add(record) && record.sessionId.length > 0) { - sessionIds.add(record.sessionId); + let usageRecord = record; + if (record.provider === "codex" && record.sessionId.length > 0) { + // Match moved rollout copies without collapsing repeated equal events + // within one rollout (timestamps can have only second precision). + const key = encodeUsageRecordKey([ + record.provider, + record.sessionId, + record.timestampMs, + record.model, + record.totals, + ]); + const occurrence = (codexEventOccurrences.get(key) ?? 0) + 1; + codexEventOccurrences.set(key, occurrence); + usageRecord = { ...record, dedupeKey: key + ":" + occurrence }; + } + // Only sessions contributing in-window count; the mtime slack can + // admit boundary files whose records fall outside the range. + if (aggregator.add(usageRecord)) { + sourceAggregator.add(usageRecord); + if (record.sessionId.length > 0) sessionIds.add(record.sessionId); } } } sources.push({ fingerprint: { hostId, provider, resolvedHomePath: dir, volumeId }, - status: status ?? "ok", + // Saved records remain available after cleanup. Preserve Pylon's native + // partial/failed scan status when an Antigravity read was incomplete. + status: retainedTruncated + ? "partial" + : files === null + ? scannedFiles === 0 + ? "missing" + : "ok" + : status === "failed" && scannedFiles > 0 + ? "partial" + : (status ?? "ok"), + buckets: sourceAggregator.finish().buckets, scannedFiles, skippedFiles, malformedRecords: malformedRecords ?? 0, distinctSessions: sessionIds.size, - message: message ?? null, + message: + files === null ? "No transcript directory on this environment." : (message ?? null), }); } - const pruned = pruneScanCache(fileCache, { - livePaths, - walkedRoots, - windowStartMs, - retentionCutoffMs: startedAtMs - CACHE_RETENTION_DAYS * 24 * 60 * 60 * 1000, - }); + const pruned = pruneScanCache(fileCache, retentionCutoffMs); if (pruned > 0) cacheDirty = true; yield* persistScanCache(); diff --git a/apps/server/src/usage/usageScanCache.test.ts b/apps/server/src/usage/usageScanCache.test.ts index fdb0aabafa..cc1bdbcc16 100644 --- a/apps/server/src/usage/usageScanCache.test.ts +++ b/apps/server/src/usage/usageScanCache.test.ts @@ -194,88 +194,17 @@ describe("pruneScanCache", () => { it("drops entries older than retention", () => { const cache = cacheWith([["/old.jsonl", 500, [record()]]]); - const removed = pruneScanCache(cache, { - livePaths: new Set(), - walkedRoots: ["/"], - windowStartMs: 400, - retentionCutoffMs, - }); + const removed = pruneScanCache(cache, retentionCutoffMs); expect(removed).toBe(1); expect(cache.size).toBe(0); }); - it("drops in-window entries whose file has disappeared", () => { + it("keeps entries whose file has disappeared", () => { const cache = cacheWith([["/gone.jsonl", 5000, [record()]]]); - pruneScanCache(cache, { - livePaths: new Set(), - walkedRoots: ["/"], - windowStartMs: 4000, - retentionCutoffMs, - }); - - expect(cache.size).toBe(0); - }); - - it("keeps entries outside the walked window that are still within retention", () => { - // Viewing 7 days must not evict the 30-day entries, which that walk never - // looked for and so cannot prove are gone. - const cache = cacheWith([["/older-but-valid.jsonl", 2000, [record()]]]); - - const removed = pruneScanCache(cache, { - livePaths: new Set(), - walkedRoots: ["/"], - windowStartMs: 4000, - retentionCutoffMs, - }); - - expect(removed).toBe(0); - expect(cache.size).toBe(1); - }); - - it("keeps entries the walk saw", () => { - const cache = cacheWith([["/live.jsonl", 5000, [record()]]]); - - pruneScanCache(cache, { - livePaths: new Set(["/live.jsonl"]), - walkedRoots: ["/"], - windowStartMs: 4000, - retentionCutoffMs, - }); - - expect(cache.size).toBe(1); - }); -}); - -describe("pruneScanCache with an unwalked root", () => { - it("keeps in-window entries for a provider whose directory was not walked", () => { - // A missing provider root or failed settings read leaves livePaths without - // that provider's files. Its warm entries must survive the pass. - const cache = cacheWith([["/codex/sessions/a.jsonl", 5000, [record()]]]); - - const removed = pruneScanCache(cache, { - livePaths: new Set(), - walkedRoots: ["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/claude/projects"], - windowStartMs: 4000, - retentionCutoffMs: 1000, - }); - - expect(removed).toBe(0); - expect(cache.size).toBe(1); - }); - - it("keeps entries under a sibling path that only shares the walked root prefix", () => { - const cache = cacheWith([["/claude/projects-copy/a.jsonl", 5000, [record()]]]); - - const removed = pruneScanCache(cache, { - livePaths: new Set(), - walkedRoots: ["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/claude/projects"], - windowStartMs: 4000, - retentionCutoffMs: 1000, - }); + pruneScanCache(cache, retentionCutoffMs); - expect(removed).toBe(0); expect(cache.size).toBe(1); }); }); diff --git a/apps/server/src/usage/usageScanCache.ts b/apps/server/src/usage/usageScanCache.ts index 04f9597f42..8439d5ca74 100644 --- a/apps/server/src/usage/usageScanCache.ts +++ b/apps/server/src/usage/usageScanCache.ts @@ -14,9 +14,6 @@ * * @module usageScanCache */ -// @effect-diagnostics nodeBuiltinImport:off -import * as NodePath from "node:path"; - import type { UsageProviderKind } from "@t3tools/contracts"; import { GUARD_LENGTH, type TranscriptParsePosition } from "./usageTranscriptReader.ts"; @@ -302,47 +299,11 @@ function decodeCodexState(value: unknown): CodexScanState | null | undefined { }; } -export interface PruneOptions { - /** Files the walk just saw. Only meaningful inside the walked window. */ - readonly livePaths: ReadonlySet; - /** - * Roots the walk actually completed. Absence from `livePaths` only proves a - * file is gone when its root was walked: a provider whose directory failed to - * resolve this pass must not have its warm entries purged. - */ - readonly walkedRoots: readonly string[]; - /** Start of the walked window; entries older than this were not looked for. */ - readonly windowStartMs: number; - /** Entries older than this are dropped regardless. */ - readonly retentionCutoffMs: number; -} - -/** - * Drops aged-out entries, and entries for files that have disappeared. - * - * The walk only covers the requested window, so absence from `livePaths` only - * proves deletion for entries *inside* that window. Pruning everything the walk - * missed would evict the 30-day entries every time someone looked at 7 days. - * - * Replaces an earlier record cap that cleared the whole cache once exceeded, - * which meant a large enough window never warmed up at all. - */ -export function pruneScanCache(cache: ScanCache, options: PruneOptions): number { +/** Keeps saved usage after transcript cleanup, until the reporting retention expires. */ +export function pruneScanCache(cache: ScanCache, retentionCutoffMs: number): number { let removed = 0; for (const [path, entry] of cache) { - const agedOut = entry.mtimeMs < options.retentionCutoffMs; - const underWalkedRoot = options.walkedRoots.some((root) => { - const relative = NodePath.relative(root, path); - return ( - relative === "" || - (relative !== ".." && - !relative.startsWith(`..${NodePath.sep}`) && - !NodePath.isAbsolute(relative)) - ); - }); - const deleted = - underWalkedRoot && entry.mtimeMs >= options.windowStartMs && !options.livePaths.has(path); - if (agedOut || deleted) { + if (entry.mtimeMs < retentionCutoffMs) { cache.delete(path); removed += 1; } diff --git a/apps/server/src/vcs/GitVcsDriver.ts b/apps/server/src/vcs/GitVcsDriver.ts index 7d89a827ca..5808d8ad4e 100644 --- a/apps/server/src/vcs/GitVcsDriver.ts +++ b/apps/server/src/vcs/GitVcsDriver.ts @@ -30,6 +30,7 @@ import { type VcsRemoveWorktreeInput, type VcsStatusInput, type VcsStatusResult, + type WorktreeSubmodules, } from "@t3tools/contracts"; import { makeGitVcsDriverCore, @@ -128,6 +129,9 @@ export interface CreateWorktreeProgress { total: number; }) => Effect.Effect; readonly onSubmodulesStarted?: () => Effect.Effect; + readonly onSubmodulesDisabled?: (input: { + source: "settings" | "t3.json"; + }) => Effect.Effect; readonly onSubmoduleLine?: (line: string) => Effect.Effect; readonly onSubmodulesFinished?: (input: { ok: boolean; @@ -137,6 +141,8 @@ export interface CreateWorktreeProgress { export interface CreateWorktreeOptions { readonly progress?: CreateWorktreeProgress; + /** Null defers to the newly created checkout's t3.json. */ + readonly submodules?: WorktreeSubmodules | null; } export interface GitCommitProgress { diff --git a/apps/server/src/vcs/GitVcsDriverCore.test.ts b/apps/server/src/vcs/GitVcsDriverCore.test.ts index f93a74f5d3..691909d25d 100644 --- a/apps/server/src/vcs/GitVcsDriverCore.test.ts +++ b/apps/server/src/vcs/GitVcsDriverCore.test.ts @@ -26,6 +26,7 @@ import { GitCommandError, ReviewDiffPreviewInput, type ReviewDiffFileContentsInput, + type WorktreeSubmodules, } from "@t3tools/contracts"; import { ServerConfig } from "../config.ts"; import { gitCommandDuration } from "../observability/Metrics.ts"; @@ -2134,9 +2135,113 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { assert.equal(result.branch, current); }), ); + + it.effect("rejects a missing branch without restoring a matching dirty file", () => + Effect.gen(function* () { + const cwd = yield* makeTmpDir(); + const { initialBranch } = yield* initRepoWithCommit(cwd); + const driver = yield* GitVcsDriver.GitVcsDriver; + yield* writeTextFile(cwd, "obsolete-branch", "original\n"); + yield* git(cwd, ["add", "obsolete-branch"]); + yield* git(cwd, ["commit", "-m", "tracked file"]); + yield* git(cwd, ["branch", "obsolete-branch"]); + yield* git(cwd, ["branch", "-D", "obsolete-branch"]); + yield* writeTextFile(cwd, "obsolete-branch", "uncommitted work\n"); + + const result = yield* driver + .switchRef({ cwd, refName: "obsolete-branch" }) + .pipe(Effect.result); + + assert.equal(result._tag, "Failure"); + const fileSystem = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + assert.equal( + yield* fileSystem.readFileString(path.join(cwd, "obsolete-branch")), + "uncommitted work\n", + ); + assert.equal(yield* git(cwd, ["branch", "--show-current"]), initialBranch); + }), + ); + + it.effect("still creates and reuses remote tracking branches and allows detached refs", () => + Effect.gen(function* () { + const cwd = yield* makeTmpDir(); + const remote = yield* makeTmpDir("git-remote-"); + const { initialBranch } = yield* initRepoWithCommit(cwd); + const driver = yield* GitVcsDriver.GitVcsDriver; + yield* git(remote, ["init", "--bare"]); + yield* git(cwd, ["remote", "add", "origin", remote]); + yield* git(cwd, ["push", "origin", "HEAD:refs/heads/remote-only"]); + + for (let attempt = 0; attempt < 2; attempt += 1) { + const result = yield* driver.switchRef({ cwd, refName: "origin/remote-only" }); + assert.equal(result.refName, "remote-only"); + assert.equal( + yield* git(cwd, ["rev-parse", "--abbrev-ref", "@{upstream}"]), + "origin/remote-only", + ); + yield* driver.switchRef({ cwd, refName: initialBranch }); + } + const commit = yield* git(cwd, ["rev-parse", "HEAD"]); + const detached = yield* driver.switchRef({ cwd, refName: commit }); + assert.equal(detached.refName, null); + assert.equal(yield* git(cwd, ["rev-parse", "HEAD"]), commit); + }), + ); }); describe("worktree operations", () => { + it.effect( + "uses parallel checkout while preserving filters, hooks, and configured workers", + () => + Effect.gen(function* () { + const cwd = yield* makeTmpDir(); + const { initialBranch } = yield* initRepoWithCommit(cwd); + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const driver = yield* GitVcsDriver.GitVcsDriver; + yield* git(cwd, ["config", "filter.test.smudge", "sed s/original/filtered/g"]); + yield* writeTextFile(cwd, ".gitattributes", "asset.txt filter=test\n"); + yield* writeTextFile(cwd, "asset.txt", "original\n"); + yield* git(cwd, ["add", "."]); + yield* git(cwd, ["commit", "-m", "filtered asset"]); + yield* writeTextFile( + cwd, + ".git/hooks/post-checkout", + "#!/bin/sh\ngit config checkout.workers > checkout-workers\nexit 0\n", + ); + yield* fs.chmod(path.join(cwd, ".git/hooks/post-checkout"), 0o755); + + for (const [configured, expected] of [ + [null, "0"], + ["1", "1"], + ] as const) { + if (configured !== null) yield* git(cwd, ["config", "checkout.workers", configured]); + const worktreePath = path.join( + yield* makeTmpDir("git-worktrees-"), + `workers-${expected}`, + ); + yield* driver.createWorktree({ + cwd, + path: worktreePath, + refName: initialBranch, + newRefName: `feature/workers-${expected}`, + }); + assert.equal( + yield* fs.readFileString(path.join(worktreePath, "checkout-workers")), + `${expected}\n`, + ); + assert.equal( + yield* fs.readFileString(path.join(worktreePath, "asset.txt")), + "filtered\n", + ); + assert.equal( + yield* git(worktreePath, ["rev-parse", "HEAD"]), + yield* git(cwd, ["rev-parse", "HEAD"]), + ); + } + }), + ); it("parses checkout progress lines from git's stderr", () => { assert.deepStrictEqual(parseGitCheckoutProgressLine("Updating files: 78% (2104/2700)"), { percent: 78, @@ -2266,11 +2371,11 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { }), ); - it.effect("reports checkout progress while creating a worktree", () => + it.effect("reports checkout progress during parallel worktree creation", () => Effect.gen(function* () { const cwd = yield* makeTmpDir(); const { initialBranch } = yield* initRepoWithCommit(cwd); - for (let index = 0; index < 5; index += 1) { + for (let index = 0; index < 200; index += 1) { yield* writeTextFile(cwd, `file-${index}.txt`, `${index}\n`); } yield* git(cwd, ["add", "."]); @@ -2304,7 +2409,7 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { const updates = yield* Ref.get(seen); assert.isAbove(updates.length, 1); assert.equal(updates.at(-1)?.percent, 100); - assert.equal(updates.at(-1)?.total, 6); + assert.equal(updates.at(-1)?.total, 201); const completed = updates.map((update) => update.completed); assert.deepEqual( completed, @@ -2341,6 +2446,100 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { }), ); + it.effect("resolves the submodule mode from the option, then t3.json", () => + Effect.gen(function* () { + const fileSystem = yield* FileSystem.FileSystem; + const pathService = yield* Path.Path; + + const previousAllowedProtocol = process.env.GIT_ALLOW_PROTOCOL; + process.env.GIT_ALLOW_PROTOCOL = "file"; + yield* Effect.addFinalizer(() => + Effect.sync(() => { + if (previousAllowedProtocol === undefined) { + delete process.env.GIT_ALLOW_PROTOCOL; + } else { + process.env.GIT_ALLOW_PROTOCOL = previousAllowedProtocol; + } + }), + ); + + // inner -> nested, so a recursive init populates nested/NESTED.md and + // a top-level init leaves it empty. + const nestedRepo = yield* makeTmpDir("git-nested-"); + yield* initRepoWithCommit(nestedRepo); + yield* writeTextFile(nestedRepo, "NESTED.md", "# nested\n"); + yield* git(nestedRepo, ["add", "."]); + yield* git(nestedRepo, ["commit", "-m", "nested"]); + const innerRepo = yield* makeTmpDir("git-inner-"); + yield* initRepoWithCommit(innerRepo); + yield* writeTextFile(innerRepo, "INNER.md", "# inner\n"); + yield* git(innerRepo, ["submodule", "add", nestedRepo, "nested"]); + yield* git(innerRepo, ["add", "."]); + yield* git(innerRepo, ["commit", "-m", "inner"]); + + const cwd = yield* makeTmpDir(); + const { initialBranch } = yield* initRepoWithCommit(cwd); + yield* git(cwd, ["submodule", "add", innerRepo, "inner"]); + yield* git(cwd, ["commit", "-m", "add submodule"]); + const driver = yield* GitVcsDriver.GitVcsDriver; + const worktreesDir = yield* makeTmpDir("git-worktrees-"); + + const createWithMode = Effect.fn(function* ( + fileMode: WorktreeSubmodules, + branch: string, + submodules: WorktreeSubmodules | null = null, + ) { + yield* writeTextFile(cwd, "t3.json", `{ "worktreeSubmodules": "${fileMode}" }`); + yield* git(cwd, ["add", "t3.json"]); + // Consecutive cases may reuse a file mode to test the option alone. + yield* git(cwd, ["commit", "--allow-empty", "-m", `submodules: ${fileMode}`]); + const worktreePath = pathService.join(worktreesDir, branch); + const disabled = yield* Ref.make<"settings" | "t3.json" | false>(false); + yield* driver.createWorktree( + { cwd, path: worktreePath, refName: initialBranch, newRefName: branch }, + { + submodules, + progress: { onSubmodulesDisabled: ({ source }) => Ref.set(disabled, source) }, + }, + ); + return { + disabled: yield* Ref.get(disabled), + inner: yield* fileSystem.exists(pathService.join(worktreePath, "inner", "INNER.md")), + nested: yield* fileSystem.exists( + pathService.join(worktreePath, "inner", "nested", "NESTED.md"), + ), + }; + }); + + assert.deepEqual(yield* createWithMode("recursive", "recursive"), { + disabled: false, + inner: true, + nested: true, + }); + assert.deepEqual(yield* createWithMode("top-level", "top-level"), { + disabled: false, + inner: true, + nested: false, + }); + // A resolved setting outranks the file in both directions. + assert.deepEqual(yield* createWithMode("recursive", "setting-none", "none"), { + disabled: "settings", + inner: false, + nested: false, + }); + assert.deepEqual(yield* createWithMode("none", "setting-wins", "top-level"), { + disabled: false, + inner: true, + nested: false, + }); + assert.deepEqual(yield* createWithMode("none", "none"), { + disabled: "t3.json", + inner: false, + nested: false, + }); + }), + ); + it.effect("allows worktree removal to run longer than the default command timeout", () => Effect.gen(function* () { const delegate = yield* ChildProcessSpawner.ChildProcessSpawner; @@ -2540,6 +2739,67 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { }); describe("remote operations", () => { + for (const failure of ["offline", "auth", "timeout"] as const) { + it.effect(`does not retry a scoped fetch after ${failure}`, () => + Effect.gen(function* () { + const cwd = yield* makeTmpDir(); + yield* initRepoWithCommit(cwd); + yield* git(cwd, ["remote", "add", "origin", "https://example.invalid/repo.git"]); + const delegate = yield* ChildProcessSpawner.ChildProcessSpawner; + const started = yield* Deferred.make(); + const attempts: Array> = []; + const spawner = ChildProcessSpawner.make((command) => + Effect.gen(function* () { + if (!ChildProcess.isStandardCommand(command)) + return yield* Effect.die("unexpected command"); + if (command.args[0] !== "fetch") return yield* delegate.spawn(command); + attempts.push(command.args); + yield* Deferred.succeed(started, undefined); + return ChildProcessSpawner.makeHandle({ + ...makeNonRepositoryHandle(), + exitCode: + failure === "timeout" + ? Effect.never + : Effect.succeed(ChildProcessSpawner.ExitCode(128)), + stderr: Stream.encodeText( + Stream.make( + failure === "auth" + ? "fatal: Authentication failed" + : "fatal: Could not resolve host", + ), + ), + }); + }), + ); + const driver = yield* makeGitVcsDriverCore().pipe( + Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), + Effect.provide(ServerConfigLayer), + ); + const fetching = yield* driver + .fetchRemote({ cwd, remoteName: "origin", refName: "main" }) + .pipe(Effect.result, Effect.forkChild({ startImmediately: true })); + yield* Deferred.await(started); + if (failure === "timeout") { + yield* TestClock.adjust("31 seconds"); + yield* TestClock.adjust("31 seconds"); + } + const result = yield* Fiber.join(fetching); + assert.isTrue(Result.isFailure(result)); + assert.equal(attempts.length, 1); + if (Result.isFailure(result)) { + assert.equal( + result.failure.detail, + failure === "timeout" + ? "Git command timed out." + : failure === "auth" + ? "Git could not authenticate with the remote. Check Git credentials or SSH access on the server, then retry." + : "Git could not reach the remote. Check the server's network connection and remote host, then retry.", + ); + } + }), + ); + } + it.effect("explains a real fetch failure for a missing local remote", () => Effect.gen(function* () { const cwd = yield* makeTmpDir(); @@ -2579,7 +2839,15 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { assert.notEqual(beforeFetch, remoteHead); const driver = yield* GitVcsDriver.GitVcsDriver; - yield* driver.fetchRemote({ cwd, remoteName: "origin" }); + yield* git(peer, ["push", "origin", "HEAD:refs/heads/unrelated"]); + yield* driver.fetchRemote({ + cwd, + remoteName: "origin", + refName: initialBranch, + }); + assert.isFalse( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "unrelated" }), + ); assert.equal( yield* driver.remoteBranchExists({ @@ -2641,6 +2909,46 @@ it.layer(TestLayer)("GitVcsDriver core integration", (it) => { const status = yield* driver.statusDetails(worktreePath); assert.equal(status.aheadCount, 0); assert.equal(status.aheadOfDefaultCount, 0); + + // Local-only bases still use the old full-fetch fallback. + yield* driver.fetchRemote({ cwd, remoteName: "origin", refName: "local-only" }); + assert.isTrue( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "unrelated" }), + ); + + // A branch literally named origin/topic must retain the old lookup at + // refs/remotes/origin/origin/topic, not get stripped to origin/topic. + yield* git(peer, ["push", "origin", "HEAD:refs/heads/origin/topic"]); + yield* driver.fetchRemote({ cwd, remoteName: "origin", refName: "origin/topic" }); + assert.isTrue( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "origin/topic" }), + ); + + // A custom no-force fetch refspec keeps its full-fetch semantics. + yield* git(peer, ["push", "origin", "HEAD:refs/heads/custom-policy"]); + yield* git(cwd, ["config", "remote.origin.fetch", "refs/heads/*:refs/remotes/origin/*"]); + yield* driver.fetchRemote({ cwd, remoteName: "origin", refName: initialBranch }); + assert.isTrue( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "custom-policy" }), + ); + + // Pruning needs the full remote refspec, not only the selected branch. + yield* git(cwd, ["config", "remote.origin.fetch", "+refs/heads/*:refs/remotes/origin/*"]); + yield* git(cwd, ["config", "fetch.prune", "true"]); + yield* git(peer, ["push", "origin", ":refs/heads/custom-policy"]); + yield* driver.fetchRemote({ cwd, remoteName: "origin", refName: initialBranch }); + assert.isFalse( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "custom-policy" }), + ); + + // A remote configured to fetch all tags also keeps its full branch fetch. + yield* git(cwd, ["config", "--unset", "fetch.prune"]); + yield* git(cwd, ["config", "remote.origin.tagOpt", "--tags"]); + yield* git(peer, ["push", "origin", "HEAD:refs/heads/tag-policy"]); + yield* driver.fetchRemote({ cwd, remoteName: "origin", refName: initialBranch }); + assert.isTrue( + yield* driver.remoteBranchExists({ cwd, remoteName: "origin", refName: "tag-policy" }), + ); }), ); diff --git a/apps/server/src/vcs/GitVcsDriverCore.ts b/apps/server/src/vcs/GitVcsDriverCore.ts index 58191a9129..d476a37ebc 100644 --- a/apps/server/src/vcs/GitVcsDriverCore.ts +++ b/apps/server/src/vcs/GitVcsDriverCore.ts @@ -20,6 +20,7 @@ import { ChildProcess, ChildProcessSpawner } from "effect/unstable/process"; import { GitCommandError, + T3_PROJECT_FILE_NAME, type ReviewDiffFileContentsInput, type ReviewDiffPreviewInput, type ReviewDiffFileStat, @@ -30,6 +31,7 @@ import { dedupeRemoteBranchesWithLocalMatches, normalizeGitRemoteUrl } from "@t3 import { HostProcessPlatform } from "@t3tools/shared/hostProcess"; import { compactTraceAttributes } from "@t3tools/shared/observability"; import { decodeJsonResult } from "@t3tools/shared/schemaJson"; +import { parseT3ProjectFile } from "@t3tools/shared/t3ProjectFile"; import { gitCommandDuration, gitCommandsTotal, withMetrics } from "../observability/Metrics.ts"; import * as GitVcsDriver from "./GitVcsDriver.ts"; import { @@ -2282,13 +2284,15 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* const readRangeContext: GitVcsDriver.GitVcsDriver["Service"]["readRangeContext"] = Effect.fn( "readRangeContext", )(function* (cwd, baseRef) { - const range = `${baseRef}..HEAD`; + const commitRange = `${baseRef}..HEAD`; + // PR diffs start at the common ancestor when the base branch has advanced. + const diffRange = `${baseRef}...HEAD`; const [commitSummary, diffSummary, diffPatch] = yield* Effect.all( [ runGitStdoutWithOptions( "GitVcsDriver.readRangeContext.log", cwd, - ["log", "--oneline", range], + ["log", "--oneline", commitRange], { maxOutputBytes: RANGE_COMMIT_SUMMARY_MAX_OUTPUT_BYTES, appendTruncationMarker: true, @@ -2297,7 +2301,7 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* runGitStdoutWithOptions( "GitVcsDriver.readRangeContext.diffStat", cwd, - ["diff", "--stat", range], + ["diff", "--stat", diffRange], { maxOutputBytes: RANGE_DIFF_SUMMARY_MAX_OUTPUT_BYTES, appendTruncationMarker: true, @@ -2306,7 +2310,7 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* runGitStdoutWithOptions( "GitVcsDriver.readRangeContext.diffPatch", cwd, - ["diff", "--no-ext-diff", "--patch", "--minimal", range], + ["diff", "--no-ext-diff", "--patch", "--minimal", diffRange], { maxOutputBytes: RANGE_DIFF_PATCH_MAX_OUTPUT_BYTES, appendTruncationMarker: true, @@ -3070,23 +3074,32 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* const progress = options?.progress; const onCheckoutProgress = progress?.onCheckoutProgress; - yield* executeGit("GitVcsDriver.createWorktree", input.cwd, args, { - fallbackErrorDetail: "git worktree add failed", - timeoutMs: WORKTREE_ADD_TIMEOUT_MS, - ...(onCheckoutProgress - ? { - // Git only prints checkout progress when stderr is a tty or the - // delay elapsed. GIT_PROGRESS_DELAY=0 forces it through the pipe. - env: { GIT_PROGRESS_DELAY: "0", LC_ALL: "C" }, - progress: { - onStderrLine: (line) => { - const parsed = parseGitCheckoutProgressLine(line); - return parsed ? onCheckoutProgress(parsed) : Effect.void; + // Git defaults to a single checkout worker unless the caller opts in. + // Respect an explicit checkout.workers setting while enabling Git's + // automatic worker count for the common unset case. + const checkoutWorkers = (yield* readConfigValue(input.cwd, "checkout.workers")) ?? "0"; + yield* executeGit( + "GitVcsDriver.createWorktree", + input.cwd, + ["-c", `checkout.workers=${checkoutWorkers}`, ...args], + { + fallbackErrorDetail: "git worktree add failed", + timeoutMs: WORKTREE_ADD_TIMEOUT_MS, + ...(onCheckoutProgress + ? { + // Git only prints checkout progress when stderr is a tty or the + // delay elapsed. GIT_PROGRESS_DELAY=0 forces it through the pipe. + env: { GIT_PROGRESS_DELAY: "0", LC_ALL: "C" }, + progress: { + onStderrLine: (line) => { + const parsed = parseGitCheckoutProgressLine(line); + return parsed ? onCheckoutProgress(parsed) : Effect.void; + }, }, - }, - } - : {}), - }); + } + : {}), + }, + ); if (progress?.onWorktreeClaimed) { yield* progress.onWorktreeClaimed(worktreePath); @@ -3100,7 +3113,30 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* const hasSubmodules = yield* fileSystem .exists(path.join(worktreePath, ".gitmodules")) .pipe(Effect.orElseSucceed(() => false)); - if (hasSubmodules) { + const submoduleSetting = options?.submodules; + const submoduleMode = !hasSubmodules + ? "none" + : submoduleSetting != null + ? submoduleSetting + : yield* fileSystem.readFileString(path.join(worktreePath, T3_PROJECT_FILE_NAME)).pipe( + Effect.flatMap((contents) => { + const file = parseT3ProjectFile(contents); + return file === null + ? Effect.logWarning("t3.json is invalid; initializing submodules recursively", { + worktreePath, + }).pipe(Effect.as("recursive" as const)) + : Effect.succeed(file.worktreeSubmodules ?? "recursive"); + }), + Effect.orElseSucceed(() => "recursive" as const), + ); + if (hasSubmodules && submoduleMode === "none") { + if (progress?.onSubmodulesDisabled) { + yield* progress.onSubmodulesDisabled({ + source: submoduleSetting == null ? "t3.json" : "settings", + }); + } + } + if (submoduleMode !== "none") { if (progress?.onSubmodulesStarted) { yield* progress.onSubmodulesStarted(); } @@ -3108,7 +3144,9 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* yield* runGit( "GitVcsDriver.createWorktree.updateSubmodules", worktreePath, - ["submodule", "update", "--init", "--recursive"], + submoduleMode === "recursive" + ? ["submodule", "update", "--init", "--recursive"] + : ["submodule", "update", "--init"], onSubmoduleLine ? { env: { LC_ALL: "C" }, @@ -3273,28 +3311,89 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* env: STATUS_UPSTREAM_REFRESH_ENV, fallbackErrorDetail: `git fetch ${input.remoteName} failed`, }; - yield* executeGitWithStableDiagnostics("GitVcsDriver.fetchRemote", input.cwd, args, { - ...options, - allowNonZeroExit: true, - }).pipe( - Effect.flatMap((result) => - result.exitCode === 0 - ? Effect.void - : Effect.fail( - new GitCommandError({ - ...gitCommandContext({ - operation: "GitVcsDriver.fetchRemote", - cwd: input.cwd, - args, - }), - detail: fetchFailureDetail(result.stderr) ?? options.fallbackErrorDetail, - ...(result.exitCode === null ? {} : { exitCode: result.exitCode }), - stdoutLength: result.stdout.length, - stderrLength: result.stderr.length, - }), - ), - ), - ); + const runFetch = (fetchArgs: ReadonlyArray) => + executeGitWithStableDiagnostics("GitVcsDriver.fetchRemote", input.cwd, [...fetchArgs], { + ...options, + allowNonZeroExit: true, + }); + let fetchArgs: ReadonlyArray = args; + let result; + // An explicit refspec must not silently override a repository's custom + // remote.fetch mapping or its force-update policy. Scope the common + // default mapping; keep the existing full fetch for all other setups. + const defaultFetchRefspec = `+refs/heads/*:refs/remotes/${input.remoteName}/*`; + const configuredFetchRefspecs = input.refName + ? (yield* runGitStdout( + "GitVcsDriver.fetchRemote.readRefspecs", + input.cwd, + ["config", "--get-all", `remote.${input.remoteName}.fetch`], + true, + )) + .split(/\r?\n/) + .filter((line) => line.length > 0) + : []; + const pruneSettings = input.refName + ? yield* Effect.forEach( + [ + "fetch.prune", + "fetch.pruneTags", + `remote.${input.remoteName}.prune`, + `remote.${input.remoteName}.pruneTags`, + ], + (key) => + runGitStdout( + "GitVcsDriver.fetchRemote.readPruneSetting", + input.cwd, + ["config", "--type=bool", "--get", key], + true, + ).pipe(Effect.map((value) => value.trim() === "true")), + ) + : []; + const tagOption = input.refName + ? yield* readConfigValue(input.cwd, `remote.${input.remoteName}.tagOpt`) + : null; + if ( + input.refName && + parseRemoteRefWithRemoteNames(input.refName, [input.remoteName]) === null && + configuredFetchRefspecs.length === 1 && + configuredFetchRefspecs[0] === defaultFetchRefspec && + !pruneSettings.some(Boolean) && + tagOption !== "--tags" + ) { + const branch = input.refName; + const scopedArgs = [ + ...args, + `+refs/heads/${branch}:refs/remotes/${input.remoteName}/${branch}`, + ]; + fetchArgs = scopedArgs; + result = yield* runFetch(scopedArgs); + // A local-only base has no matching remote branch. Preserve the old + // full-fetch behavior so callers can still discover other remote refs. + if ( + result.exitCode !== 0 && + result.stderr + .split(/\r?\n/) + .includes(`fatal: couldn't find remote ref refs/heads/${branch}`) + ) { + fetchArgs = args; + result = yield* runFetch(args); + } + } else { + result = yield* runFetch(args); + } + if (result.exitCode !== 0) { + return yield* new GitCommandError({ + ...gitCommandContext({ + operation: "GitVcsDriver.fetchRemote", + cwd: input.cwd, + args: fetchArgs, + }), + detail: fetchFailureDetail(result.stderr) ?? options.fallbackErrorDetail, + ...(result.exitCode === null ? {} : { exitCode: result.exitCode }), + stdoutLength: result.stdout.length, + stderrLength: result.stderr.length, + }); + } }, ); @@ -3504,7 +3603,8 @@ export const makeGitVcsDriverCore = Effect.fn("makeGitVcsDriverCore")(function* ? ["checkout", localTrackingBranch] : ["checkout", input.refName]; - yield* executeGit("GitVcsDriver.switchRef.checkout", input.cwd, checkoutArgs, { + // A stale ref must not turn into a path checkout that discards local edits. + yield* executeGit("GitVcsDriver.switchRef.checkout", input.cwd, [...checkoutArgs, "--"], { timeoutMs: 10_000, fallbackErrorDetail: "git checkout failed", }); diff --git a/apps/server/src/ws.ts b/apps/server/src/ws.ts index 9d4328a8cc..fca63f651b 100644 --- a/apps/server/src/ws.ts +++ b/apps/server/src/ws.ts @@ -91,6 +91,7 @@ import { type WorktreeSetupSnapshot, } from "@t3tools/contracts"; import { resolveServerBackgroundActivitySettings } from "@t3tools/shared/backgroundActivitySettings"; +import { resolveProjectSettings } from "@t3tools/shared/projectSettings"; import { isDriveOrPosixAbsolutePath, isWindowsAbsolutePath } from "@t3tools/shared/path"; import { HttpRouter, HttpServerRequest, HttpServerRespondable } from "effect/unstable/http"; import { RpcSerialization, RpcServer } from "effect/unstable/rpc"; @@ -121,6 +122,8 @@ import { observeRpcStreamEffect as instrumentRpcStreamEffect, } from "./observability/RpcInstrumentation.ts"; import * as ProviderRegistry from "./provider/Services/ProviderRegistry.ts"; +import * as ModelManifest from "./provider/ModelManifest.ts"; +import * as ProviderMaintenance from "./provider/providerMaintenance.ts"; import * as ProviderService from "./provider/Services/ProviderService.ts"; import * as ProviderSessionDirectory from "./provider/Services/ProviderSessionDirectory.ts"; import * as ProviderMaintenanceRunner from "./provider/providerMaintenanceRunner.ts"; @@ -560,6 +563,8 @@ const makeWsRpcLayer = ( const deviceService = yield* DeviceService.DeviceService; const portDiscovery = yield* PortScanner.PortDiscovery; const providerRegistry = yield* ProviderRegistry.ProviderRegistry; + const modelManifest = yield* ModelManifest.ModelManifest; + const providerVersionCache = yield* ProviderMaintenance.ProviderVersionCache; const providerService = yield* ProviderService.ProviderService; const relayWorkerBridge = Option.getOrUndefined( yield* Effect.serviceOption(RelayWorkerBridge), @@ -1068,6 +1073,41 @@ const makeWsRpcLayer = ( return output; }); + // The branch checked out into a new worktree may have its own t3.json, + // so a null settings value intentionally leaves that read to the driver. + const resolveBootstrapWorktreeSubmodules = Effect.fnUntraced(function* (input: { + readonly threadId: ThreadId; + readonly projectId: ProjectId | null; + }) { + const settings = yield* serverSettings.getSettings.pipe(Effect.orElseSucceed(() => null)); + if (settings === null) return null; + const projectId = + input.projectId ?? + (yield* projectionSnapshotQuery.getThreadShellById(input.threadId).pipe( + Effect.map((thread) => Option.getOrNull(thread)?.projectId ?? null), + Effect.orElseSucceed(() => null), + )); + const project = + projectId === null + ? null + : yield* projectionSnapshotQuery.getProjectShellById(projectId).pipe( + Effect.map(Option.getOrNull), + Effect.orElseSucceed(() => null), + ); + return resolveProjectSettings(settings, projectId, project).settings.worktreeSubmodules; + }); + + const resolveWorktreeSubmodulesForCwd = Effect.fnUntraced(function* (cwd: string) { + const settings = yield* serverSettings.getSettings.pipe(Effect.orElseSucceed(() => null)); + if (settings === null) return null; + const project = yield* projectionSnapshotQuery.getActiveProjectByWorkspaceRoot(cwd).pipe( + Effect.map(Option.getOrNull), + Effect.orElseSucceed(() => null), + ); + return resolveProjectSettings(settings, project?.id ?? null, project).settings + .worktreeSubmodules; + }); + const dispatchBootstrapTurnStart = ( command: Extract, ): Effect.Effect<{ readonly sequence: number }, OrchestrationDispatchCommandError> => @@ -1342,6 +1382,7 @@ const makeWsRpcLayer = ( yield* gitWorkflow.fetchRemote({ cwd: prepareWorktree.projectCwd, remoteName: "origin", + refName: prepareWorktree.baseBranch, }); const remoteBaseExists = yield* gitWorkflow.remoteBranchExists({ cwd: prepareWorktree.projectCwd, @@ -1491,6 +1532,10 @@ const makeWsRpcLayer = ( } yield* worktreeSetupTracker.stageStatus(threadId, "checkout", "running"); let checkoutTotal: number | null = null; + const submodules = yield* resolveBootstrapWorktreeSubmodules({ + threadId, + projectId: targetProjectId ?? null, + }); const worktree = yield* gitWorkflow.createWorktree( { cwd: prepareWorktree.projectCwd, @@ -1500,6 +1545,7 @@ const makeWsRpcLayer = ( path: null, }, { + submodules, progress: { // Git has registered the directory at this point, so a // cancel during the submodule step can still remove it. @@ -1529,6 +1575,13 @@ const makeWsRpcLayer = ( worktreeSetupTracker.stageStatus(threadId, "submodules", "running"), ), ), + onSubmodulesDisabled: ({ source }) => + worktreeSetupTracker.stageStatus( + threadId, + "submodules", + "skipped", + `disabled in ${source}`, + ), onSubmoduleLine: (line) => { const submodulePath = /Submodule path '([^']+)'/.exec(line)?.[1]; return submodulePath === undefined @@ -2345,6 +2398,29 @@ const makeWsRpcLayer = ( observeRpcEffect( WS_METHODS.serverRefreshProviders, Effect.gen(function* () { + if (input.refreshModels) { + // Explicit refresh bypasses Pylon-owned caches; background probes + // keep their normal freshness windows. + yield* modelManifest.forceRefresh; + const instances = yield* providerInstances.listInstances; + yield* Effect.forEach( + instances.filter( + (instance) => + input.instanceId === undefined || input.instanceId === instance.instanceId, + ), + (instance) => + Effect.gen(function* () { + yield* instance.invalidateCaches ?? Effect.void; + const maintenance = yield* instance.snapshot.resolveMaintenance({ + fresh: true, + }); + if (maintenance.packageName) { + providerVersionCache.delete(maintenance.packageName); + } + }), + { concurrency: "unbounded", discard: true }, + ); + } let providers = yield* input.cwd !== undefined && input.instanceId !== undefined ? providerRegistry.refreshWorkspaceSnapshot({ instanceId: input.instanceId, @@ -3097,7 +3173,7 @@ const makeWsRpcLayer = ( [WS_METHODS.pullRequestsInvalidate]: (input) => observeRpcEffect( WS_METHODS.pullRequestsInvalidate, - pullRequests.invalidate(input).pipe( + pullRequests.invalidate(input, { notifyReaders: true }).pipe( // A reader asking for fresh host state also wants the thread badges it feeds to // catch up, including a merged link the sweep would otherwise never revisit. Effect.andThen( @@ -3582,7 +3658,10 @@ const makeWsRpcLayer = ( [WS_METHODS.vcsCreateWorktree]: (input) => observeRpcEffect( WS_METHODS.vcsCreateWorktree, - gitWorkflow.createWorktree(input).pipe(Effect.tap(() => refreshGitStatus(input.cwd))), + resolveWorktreeSubmodulesForCwd(input.cwd).pipe( + Effect.flatMap((submodules) => gitWorkflow.createWorktree(input, { submodules })), + Effect.tap(() => refreshGitStatus(input.cwd)), + ), { "rpc.aggregate": "vcs" }, ), [WS_METHODS.vcsRemoveWorktree]: (input) => diff --git a/apps/web/src/components/BranchToolbar.logic.ts b/apps/web/src/components/BranchToolbar.logic.ts index bd892b5a4d..5a3367c8ce 100644 --- a/apps/web/src/components/BranchToolbar.logic.ts +++ b/apps/web/src/components/BranchToolbar.logic.ts @@ -1,4 +1,10 @@ -import type { EnvironmentId, EnvironmentMachineKind, VcsRef, ProjectId } from "@t3tools/contracts"; +import type { + EnvironmentId, + EnvironmentMachineKind, + VcsRef, + ProjectId, + WorktreeSubmodules, +} from "@t3tools/contracts"; import * as Schema from "effect/Schema"; import { toSortableTimestamp } from "../lib/threadSort"; export { @@ -17,6 +23,12 @@ export interface EnvironmentOption { export const EnvMode = Schema.Literals(["local", "worktree"]); export type EnvMode = typeof EnvMode.Type; +export const WORKTREE_SUBMODULES_LABELS: Record = { + recursive: "Recursive", + "top-level": "Top level only", + none: "Do not initialize", +}; + const GENERIC_LOCAL_ENVIRONMENT_LABELS = new Set(["local", "local environment"]); function normalizeDisplayLabel(value: string | null | undefined): string | null { diff --git a/apps/web/src/components/ChatMarkdown.tsx b/apps/web/src/components/ChatMarkdown.tsx index 9ef4f6138d..26e4410c2f 100644 --- a/apps/web/src/components/ChatMarkdown.tsx +++ b/apps/web/src/components/ChatMarkdown.tsx @@ -3307,7 +3307,15 @@ const CHAT_MARKDOWN_COMPONENTS = { resetKeys={[codeBlock.code, language, diffThemeName, isStreaming]} fallback={
{children}
} > - {children}}> + {/* Keep the block's dimensions while highlighting loads so a + newly delivered code block does not flash as plain text. */} + + {children} + + } + > diff --git a/apps/web/src/components/CommandPalette.logic.test.ts b/apps/web/src/components/CommandPalette.logic.test.ts index 5cce941f86..2c59587e59 100644 --- a/apps/web/src/components/CommandPalette.logic.test.ts +++ b/apps/web/src/components/CommandPalette.logic.test.ts @@ -10,7 +10,9 @@ import { enumerateCommandPaletteItems, filterPinnedBrowseEntries, filterCommandPaletteGroups, + getExistingThreadForProjectAdd, reduceCommandPaletteUiState, + type CommandPaletteActionItem, type CommandPaletteGroup, } from "./CommandPalette.logic"; @@ -343,6 +345,55 @@ function makeThread(overrides: Partial = {}): Thread { }; } +describe("adding an existing project", () => { + it("starts a new thread when the latest unarchived thread is settled", () => { + const olderActive = makeThread({ + id: ThreadId.make("older-active"), + updatedAt: "2026-03-01T00:00:00.000Z", + }); + const newestSettled = makeThread({ + id: ThreadId.make("newest-settled"), + updatedAt: "2026-03-02T00:00:00.000Z", + settledOverride: "settled", + settledAt: "2026-03-02T00:00:00.000Z", + }); + expect( + getExistingThreadForProjectAdd([olderActive, newestSettled], PROJECT_ID, "updated_at"), + ).toBeNull(); + + expect( + getExistingThreadForProjectAdd( + [olderActive, { ...newestSettled, settledOverride: "active", settledAt: null }], + PROJECT_ID, + "updated_at", + )?.id, + ).toBe(newestSettled.id); + }); + + it("ignores archived threads while leaving ordinary active threads available", () => { + const active = makeThread({ id: ThreadId.make("active") }); + const archived = makeThread({ + id: ThreadId.make("archived"), + updatedAt: "2026-03-03T00:00:00.000Z", + archivedAt: "2026-03-03T00:00:00.000Z", + }); + expect(getExistingThreadForProjectAdd([active, archived], PROJECT_ID, "updated_at")?.id).toBe( + active.id, + ); + }); + + it("keeps snoozed threads eligible because snoozing does not settle them", () => { + const snoozed = makeThread({ + id: ThreadId.make("snoozed"), + snoozedAt: "2026-03-01T00:00:00.000Z", + snoozedUntil: "2026-03-04T00:00:00.000Z", + }); + expect(getExistingThreadForProjectAdd([snoozed], PROJECT_ID, "updated_at")?.id).toBe( + snoozed.id, + ); + }); +}); + describe("buildProjectActionItems", () => { it("shows the grouped display name but keeps the real title for icons", () => { const project = makeProject({ title: "fleet", workspaceRoot: "/Users/theo/Code/p/fleet" }); @@ -439,6 +490,94 @@ describe("buildThreadActionItems", () => { ]); }); + it("uses activity recency for equally relevant titles across active and remote threads", () => { + const remoteEnvironmentId = EnvironmentId.make("environment-remote"); + const threads = [ + makeThread({ + id: ThreadId.make("older-active"), + title: "Fix search input", + createdAt: "2026-03-20T00:00:00.000Z", + updatedAt: "2026-03-21T00:00:00.000Z", + }), + makeThread({ + id: ThreadId.make("newer-remote"), + environmentId: remoteEnvironmentId, + title: "Fix search results", + createdAt: "2026-03-01T00:00:00.000Z", + updatedAt: "2026-03-23T00:00:00.000Z", + }), + makeThread({ + id: ThreadId.make("older-inactive"), + title: "Fix search ranking", + createdAt: "2026-03-22T00:00:00.000Z", + updatedAt: "2026-03-20T00:00:00.000Z", + }), + ]; + const items = buildThreadActionItems({ + threads, + activeThreadId: ThreadId.make("older-active"), + projectTitleById: new Map(), + sortOrder: "created_at", + icon: null, + runThread: async () => undefined, + }); + const groups = filterCommandPaletteGroups({ + activeGroups: [], + query: "fix search", + isInSubmenu: false, + projectSearchItems: [], + threadSearchItems: items, + }); + + expect(groups[0]?.items.map((item) => item.value)).toEqual([ + "thread:newer-remote", + "thread:older-active", + "thread:older-inactive", + ]); + }); + + it("keeps stronger title matches ahead of recency and preserves stable ties", () => { + const items = buildThreadActionItems({ + threads: [ + makeThread({ + id: ThreadId.make("prefix-a"), + title: "Search alpha", + createdAt: "2026-03-03T00:00:00.000Z", + updatedAt: "2026-03-20T00:00:00.000Z", + }), + makeThread({ + id: ThreadId.make("substring"), + title: "Improve search", + createdAt: "2026-03-02T00:00:00.000Z", + updatedAt: "2026-03-25T00:00:00.000Z", + }), + makeThread({ + id: ThreadId.make("prefix-b"), + title: "Search beta", + createdAt: "2026-03-01T00:00:00.000Z", + updatedAt: "2026-03-20T00:00:00.000Z", + }), + ], + projectTitleById: new Map(), + sortOrder: "created_at", + icon: null, + runThread: async () => undefined, + }); + const groups = filterCommandPaletteGroups({ + activeGroups: [], + query: "search", + isInSubmenu: false, + projectSearchItems: [], + threadSearchItems: items, + }); + + expect(groups[0]?.items.map((item) => item.value)).toEqual([ + "thread:prefix-a", + "thread:prefix-b", + "thread:substring", + ]); + }); + it("preserves thread project-name matches when there is no stronger title match", () => { const group: CommandPaletteGroup = { value: "threads-search", @@ -583,6 +722,38 @@ describe("buildThreadActionItems", () => { expect(item?.description).toBe("Pylon · #feat/search"); }); + it("finds a thread by pasted ID while ranking a title match first", () => { + const byId = makeThread({ + id: ThreadId.make("thread-alpha-1234"), + title: "Unrelated work", + updatedAt: "2026-03-05T00:00:00.000Z", + }); + const byTitle = makeThread({ + id: ThreadId.make("thread-other-9999"), + title: "Fix thread-alpha-1234 flakes", + updatedAt: "2026-03-04T00:00:00.000Z", + }); + const items = buildThreadActionItems({ + threads: [byId, byTitle], + projectTitleById: new Map([[PROJECT_ID, "Pylon"]]), + sortOrder: "updated_at", + icon: null, + runThread: async () => undefined, + }); + const groups = filterCommandPaletteGroups({ + activeGroups: [], + query: " THREAD-ALPHA-1234 ", + isInSubmenu: false, + projectSearchItems: [], + settingsSearchItems: [], + threadSearchItems: items, + }); + expect(groups.flatMap((group) => group.items.map((item) => item.value))).toEqual([ + `thread:${byTitle.id}`, + `thread:${byId.id}`, + ]); + }); + it("prefers renderDescription when provided", () => { const [item] = buildThreadActionItems({ threads: [makeThread({ branch: "feat/search", worktreePath: "/tmp/wt" })], @@ -738,3 +909,30 @@ it.each([ "Implementation", ]); }); + +it("ranks secondary keybinding settings after primary settings matches", () => { + const item = (value: string, title: string, secondary = false): CommandPaletteActionItem => ({ + kind: "action", + value, + title, + searchTerms: [title, "General"], + icon: null, + run: async () => undefined, + secondary, + }); + const groups = filterCommandPaletteGroups({ + activeGroups: [], + query: "model", + isInSubmenu: false, + projectSearchItems: [], + settingsSearchItems: [ + item("setting:keybinding-modelPicker.toggle", "Model Picker: Toggle", true), + item("setting:default-model", "Default model"), + ], + threadSearchItems: [], + }); + expect(groups.flatMap((group) => group.items.map((entry) => entry.value))).toEqual([ + "setting:default-model", + "setting:keybinding-modelPicker.toggle", + ]); +}); diff --git a/apps/web/src/components/CommandPalette.logic.ts b/apps/web/src/components/CommandPalette.logic.ts index 57c1711de1..172e2254e9 100644 --- a/apps/web/src/components/CommandPalette.logic.ts +++ b/apps/web/src/components/CommandPalette.logic.ts @@ -2,6 +2,7 @@ import { threadPullRequestSearchTerms } from "@t3tools/shared/threadPullRequests import type { CommandPaletteLinkedThreads } from "../commandPaletteBus"; import { type EnvironmentId, + type ProjectId, type FilesystemBrowseEntry, type KeybindingCommand, THREAD_JUMP_KEYBINDING_COMMANDS, @@ -11,7 +12,12 @@ import type { SidebarThreadSortOrder } from "@t3tools/contracts/settings"; import * as Arr from "effect/Array"; import * as Result from "effect/Result"; import { type ReactNode } from "react"; -import { sortThreads } from "../lib/threadSort"; +import { + getLatestThreadForProject, + getThreadSortTimestamp, + sortThreads, + type ThreadSortInput, +} from "../lib/threadSort"; import { normalizeSearchText } from "../lib/utils"; import { formatRelativeTimeLabel } from "../timestampFormat"; import { type Project, type SidebarThreadSummary, type Thread } from "../types"; @@ -20,6 +26,19 @@ export const RECENT_THREAD_LIMIT = 12; export const ITEM_ICON_CLASS = "size-4 text-icon-muted"; export const ADDON_ICON_CLASS = "size-4"; +/** Adding an existing project should open its latest thread only while it remains active. */ +export function getExistingThreadForProjectAdd< + T extends { + readonly id: string; + readonly projectId: ProjectId; + readonly archivedAt: string | null; + readonly settledOverride: string | null; + } & ThreadSortInput, +>(threads: readonly T[], projectId: ProjectId, sortOrder: SidebarThreadSortOrder): T | null { + const latestThread = getLatestThreadForProject(threads, projectId, sortOrder); + return latestThread?.settledOverride === "settled" ? null : latestThread; +} + /** A PR's relations include archived threads that normal palette search omits. */ export function buildLinkedThreadActionItems( input: CommandPaletteLinkedThreads & { @@ -132,6 +151,8 @@ export interface CommandPaletteItem { readonly description?: ReactNode; readonly threadContentMatch?: CommandPaletteThreadContentMatch; readonly timestamp?: string; + /** Activity time used only to break equally relevant search matches. */ + readonly searchRecency?: number; readonly icon: ReactNode; readonly disabled?: boolean; /** Optional content rendered inline before the title text. */ @@ -139,6 +160,8 @@ export interface CommandPaletteItem { /** Optional content rendered inline after the title text (before the timestamp). */ readonly titleTrailingContent?: ReactNode; readonly shortcutCommand?: KeybindingCommand; + /** Keep detailed settings rows after primary matches in their group. */ + readonly secondary?: boolean; } export interface CommandPaletteActionItem extends CommandPaletteItem { @@ -302,12 +325,15 @@ export function buildThreadActionItems right.rank - left.rank || left.index - right.index) + .toSorted((left, right) => { + const secondaryOrder = + Number(left.item.secondary ?? false) - Number(right.item.secondary ?? false); + if (secondaryOrder !== 0) return secondaryOrder; + if (left.rank !== right.rank) return right.rank - left.rank; + const leftRecency = left.item.searchRecency ?? Number.NEGATIVE_INFINITY; + const rightRecency = right.item.searchRecency ?? Number.NEGATIVE_INFINITY; + if (leftRecency !== rightRecency) return rightRecency > leftRecency ? 1 : -1; + return left.index - right.index; + }) .map((entry) => entry.item); if (items.length === 0) { diff --git a/apps/web/src/components/CommandPalette.tsx b/apps/web/src/components/CommandPalette.tsx index ba543a8872..06b8c5b026 100644 --- a/apps/web/src/components/CommandPalette.tsx +++ b/apps/web/src/components/CommandPalette.tsx @@ -45,6 +45,7 @@ import { useLocation, useNavigate, useParams } from "@tanstack/react-router"; import * as Option from "effect/Option"; import { ArrowLeftIcon, + ChartNoAxesColumnIcon, CornerLeftUpIcon, FileSearchIcon, FolderIcon, @@ -155,6 +156,7 @@ import { filterPinnedBrowseEntries, getCommandPaletteInputPlaceholder, getCommandPaletteMode, + getExistingThreadForProjectAdd, ITEM_ICON_CLASS, RECENT_THREAD_LIMIT, reduceCommandPaletteUiState, @@ -196,6 +198,7 @@ import { } from "../sidebarProjectGrouping"; import type { Project } from "../types"; import { PullRequestGlyph } from "~/components/pullRequest/pullRequestIcons"; +import { readPullRequestListPreferences } from "~/components/pullRequest/pullRequestListPreferences"; const EMPTY_BROWSE_ENTRIES: FilesystemBrowseResult["entries"] = []; @@ -1931,6 +1934,34 @@ function OpenCommandPaletteDialog(props: { }, }); + if ( + environments.some( + (environment) => environment.serverConfig?.environment.capabilities.pullRequests === true, + ) + ) { + actionItems.push({ + kind: "action", + value: "action:pull-requests", + searchTerms: ["pull requests", "prs", "pr", "github", "review", "merge", "branch"], + title: "Open pull requests", + icon: , + run: async () => { + await navigate({ to: "/pull-requests", search: readPullRequestListPreferences() }); + }, + }); + } + + actionItems.push({ + kind: "action", + value: "action:usage", + searchTerms: ["usage", "use", "tokens", "cost", "spend", "limits", "stats", "analytics"], + title: "Open usage", + icon: , + run: async () => { + await navigate({ to: "/usage" }); + }, + }); + actionItems.push({ kind: "action", value: "action:settings", @@ -1990,6 +2021,7 @@ function OpenCommandPaletteDialog(props: { searchTerms: [item.title, SETTINGS_SECTION_LABELS[item.to], ...(item.searchTerms ?? [])], title: item.title, description: `Settings · ${SETTINGS_SECTION_LABELS[item.to]}`, + ...(item.secondary ? { secondary: true } : {}), icon: , run: async () => { await navigate({ @@ -2086,7 +2118,7 @@ function OpenCommandPaletteDialog(props: { cwd, ); if (existing) { - const latestThread = getLatestThreadForProject( + const latestThread = getExistingThreadForProjectAdd( threads.filter((thread) => thread.environmentId === existing.environmentId), existing.id, clientSettings.sidebarThreadSortOrder, diff --git a/apps/web/src/components/DiffPanel.tsx b/apps/web/src/components/DiffPanel.tsx index d954a7ff69..547856a426 100644 --- a/apps/web/src/components/DiffPanel.tsx +++ b/apps/web/src/components/DiffPanel.tsx @@ -72,7 +72,8 @@ import { import { DropdownMenu, DropdownMenuContent, - DropdownMenuItem, + DropdownMenuRadioGroup, + DropdownMenuRadioItem, DropdownMenuSub, DropdownMenuSubContent, DropdownMenuSubTrigger, @@ -644,6 +645,26 @@ export default function DiffPanel({ if (!routeThreadRef) return; useDiffPanelStore.getState().selectBranchBaseRef(routeThreadRef, baseRef); }; + // The scope menu has two radio groups: the top-level one treats the latest + // turn as "latest", while the turn sub-menu keys every turn by id so the + // latest turn is also marked there. + const selectedTurnValue = selectedTurn ? `turn:${selectedTurn.turnId}` : ""; + const selectedScopeValue = + selectedTurnId === null + ? selectedGitScope + : selectedTurn?.turnId === latestTurn?.turnId + ? "latest" + : selectedTurnValue; + const selectScopeValue = (value: string) => { + if (value === "unstaged" || value === "branch") { + selectGitScope(value); + } else if (value === "latest") { + if (latestTurn) selectTurn(latestTurn.turnId); + } else { + const turn = orderedTurnDiffSummaries.find((summary) => `turn:${summary.turnId}` === value); + if (turn) selectTurn(turn.turnId); + } + }; const headerRow = ( <> @@ -657,61 +678,42 @@ export default function DiffPanel({ - selectGitScope("unstaged")} - > - Working tree - - selectGitScope("branch")} - > - Branch changes - - { - if (latestTurn) selectTurn(latestTurn.turnId); - }} - > - Latest turn - + + + Working tree + + + Branch changes + + + Latest turn + + Turn - {orderedTurnDiffSummaries.map((summary) => { - const turnCount = - summary.checkpointTurnCount ?? - inferredCheckpointTurnCountByTurnId[summary.turnId] ?? - "?"; - return ( - selectTurn(summary.turnId)} - > - Turn {turnCount} - - {formatShortTimestamp(summary.completedAt, settings.timestampFormat)} - - - ); - })} + + {orderedTurnDiffSummaries.map((summary) => { + const turnCount = + summary.checkpointTurnCount ?? + inferredCheckpointTurnCountByTurnId[summary.turnId] ?? + "?"; + return ( + + + Turn {turnCount} + + {formatShortTimestamp(summary.completedAt, settings.timestampFormat)} + + + + ); + })} + diff --git a/apps/web/src/components/Sidebar.tsx b/apps/web/src/components/Sidebar.tsx index 0cce06b1d6..d2974a48d5 100644 --- a/apps/web/src/components/Sidebar.tsx +++ b/apps/web/src/components/Sidebar.tsx @@ -5251,7 +5251,9 @@ export default function Sidebar() { key={threadKey} id={threadKey} disabled={ - !draggableThreadKeys.has(threadKey) || optimisticDrop !== null + renamingThreadKey === threadKey || + !draggableThreadKeys.has(threadKey) || + optimisticDrop !== null } > {(bag) => diff --git a/apps/web/src/components/chat/MessageCopyButton.tsx b/apps/web/src/components/chat/MessageCopyButton.tsx index 4a5c3f170d..aaaa171f10 100644 --- a/apps/web/src/components/chat/MessageCopyButton.tsx +++ b/apps/web/src/components/chat/MessageCopyButton.tsx @@ -37,7 +37,7 @@ export const MessageCopyButton = memo(function MessageCopyButton({ copyToClipboard(text)} ref={ref} @@ -51,7 +51,7 @@ export const MessageCopyButton = memo(function MessageCopyButton({ {isCopied ? : } -

Copy to clipboard

+

Copy message

 ); diff --git a/apps/web/src/components/chat/MessagesTimeline.test.tsx b/apps/web/src/components/chat/MessagesTimeline.test.tsx index 4afd75636f..86836e9229 100644 --- a/apps/web/src/components/chat/MessagesTimeline.test.tsx +++ b/apps/web/src/components/chat/MessagesTimeline.test.tsx @@ -1357,7 +1357,7 @@ describe("MessagesTimeline", () => { />, ); - expect(markup).toContain('aria-label="Copy link"'); + expect(markup).toContain('aria-label="Copy message"'); expect(markup).toContain('data-user-message-collapsed="true"'); expect(markup).toContain('data-user-message-footer="true"'); }); diff --git a/apps/web/src/components/files/FileBreadcrumbs.tsx b/apps/web/src/components/files/FileBreadcrumbs.tsx index 4896c9ce2d..6f69aba45e 100644 --- a/apps/web/src/components/files/FileBreadcrumbs.tsx +++ b/apps/web/src/components/files/FileBreadcrumbs.tsx @@ -10,6 +10,8 @@ import { MenuGroup, MenuItem, MenuPopup, + MenuRadioGroup, + MenuRadioItem, MenuSeparator, MenuTrigger, } from "~/components/ui/menu"; @@ -143,36 +145,51 @@ function BreadcrumbMenuContent(props: { : "This folder is empty."} ) : ( - children.map((entry) => { - const isCurrentFile = entry.kind === "file" && entry.path === props.currentFilePath; - return ( - { - if (entry.kind === "directory") { - props.onDirectoryChange(entry.path); - return; - } - props.onOpenChange(false); - props.onOpenFile(entry.path); - }} - > - - - }> - {entry.label} - - - {entry.path} - - - {entry.kind === "directory" ? : null} - - ); - }) + // Files form a radio group keyed by path so the open file is marked as checked; + // directories only navigate the menu, so they stay plain items. + { + props.onOpenChange(false); + props.onOpenFile(path); + }} + > + {children.map((entry) => { + const isCurrentFile = entry.kind === "file" && entry.path === props.currentFilePath; + const row = ( + <> + + + }> + {entry.label} + + + {entry.path} + + + + ); + return entry.kind === "directory" ? ( + props.onDirectoryChange(entry.path)} + > + {row} + + + ) : ( + + {row} + + ); + })} + )} {entriesQuery.error && entriesQuery.data !== null ? ( diff --git a/apps/web/src/components/pullRequest/PullRequestCodeTab.tsx b/apps/web/src/components/pullRequest/PullRequestCodeTab.tsx index af1d40cd00..fe8052afa8 100644 --- a/apps/web/src/components/pullRequest/PullRequestCodeTab.tsx +++ b/apps/web/src/components/pullRequest/PullRequestCodeTab.tsx @@ -69,6 +69,8 @@ import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, + DropdownMenuRadioGroup, + DropdownMenuRadioItem, DropdownMenuTrigger, } from "../ui/menu"; import { toastManager } from "../ui/toast"; @@ -1048,31 +1050,31 @@ function PullRequestCodeTab({ - onSelectedCommitChange(null)} + onSelectedCommitChange(value === "all" ? null : value)} > - All commits - - {orderedCommits.slice(0, visibleCommitCount).map((entry) => ( - onSelectedCommitChange(entry.oid)} - > - {/* Headlines run long, and the abbreviated oid after one is what a reader - matches against the commit list on the host. */} - - {entry.messageHeadline}} - /> - {entry.messageHeadline} - - - {entry.oid.slice(0, 7)} - - - ))} + + All commits + + {orderedCommits.slice(0, visibleCommitCount).map((entry) => ( + + {/* Headlines run long, and the abbreviated oid after one is what a reader + matches against the commit list on the host. */} + + + {entry.messageHeadline}} + /> + {entry.messageHeadline} + + + {entry.oid.slice(0, 7)} + + + + ))} + {orderedCommits.length > visibleCommitCount ? ( // Kept out of the radio group: it changes how much of the list is on screen // rather than what the diff is scoped to. diff --git a/apps/web/src/components/pullRequest/PullRequestDetailPanel.tsx b/apps/web/src/components/pullRequest/PullRequestDetailPanel.tsx index 0080f4348d..665f085d27 100644 --- a/apps/web/src/components/pullRequest/PullRequestDetailPanel.tsx +++ b/apps/web/src/components/pullRequest/PullRequestDetailPanel.tsx @@ -149,12 +149,11 @@ import { handoffPrompt, handoffReviewComments, latestPullRequestReviewOutcomes, - loadingPullRequestCheckoutCommand, isStackedPullRequestBase, pullRequestActionMenuHasGroup, pullRequestActionNeedsHostRefresh, pullRequestComposerTarget, - pullRequestCheckoutCommand, + panelPullRequestCheckoutCommand, pullRequestFindingKey, pullRequestHandoffLabels, PULL_REQUEST_MERGE_METHOD_LABELS, @@ -765,14 +764,12 @@ export function PullRequestDetailPanel({ repositoryUrl !== null ? new URL(`/${encodeURIComponent(detail.author.login)}`, repositoryUrl).toString() : null; - const checkoutCommand = detail - ? pullRequestCheckoutCommand( - detail.provider, - detail.number, - detail.headBranch, - detail.headRepositoryNameWithOwner, - ) - : null; + const checkoutCommand = panelPullRequestCheckoutCommand({ + reference, + identity: repositoryIdentity, + summary: handoffSummary, + headRepositoryNameWithOwner: detail?.headRepositoryNameWithOwner, + }); const onCheckoutCommandError = useCallback((error: Error) => { toastManager.add({ type: "error", diff --git a/apps/web/src/components/pullRequest/pullRequestDetail.logic.test.ts b/apps/web/src/components/pullRequest/pullRequestDetail.logic.test.ts index 57affb3076..9c0a93ea0d 100644 --- a/apps/web/src/components/pullRequest/pullRequestDetail.logic.test.ts +++ b/apps/web/src/components/pullRequest/pullRequestDetail.logic.test.ts @@ -29,6 +29,7 @@ import { isPullRequestVerdictStale, isStackedPullRequestBase, loadingPullRequestCheckoutCommand, + panelPullRequestCheckoutCommand, isThreadOwnPullRequest, latestPullRequestReviewOutcomes, newestPullRequestCommitAt, @@ -114,6 +115,51 @@ describe("pull request checkout commands", () => { ), ).toBeNull(); }); + + it("keeps a checkout command available from the list summary while detail loads", () => { + const pullRequest = reference("github.com"); + expect( + panelPullRequestCheckoutCommand({ + reference: pullRequest, + identity: undefined, + summary: { provider: "github", number: 42, headBranch: "topic" }, + }), + ).toBe("gh pr checkout 42"); + expect( + panelPullRequestCheckoutCommand({ + reference: pullRequest, + identity: undefined, + summary: null, + }), + ).toBe("gh pr checkout 42"); + }); + + it("does not invent a checkout command for an unknown host or incomplete Bitbucket detail", () => { + expect( + panelPullRequestCheckoutCommand({ + reference: reference("forge.example"), + identity: undefined, + summary: null, + }), + ).toBeNull(); + expect( + panelPullRequestCheckoutCommand({ + reference: reference("bitbucket.org"), + identity: undefined, + summary: { provider: "bitbucket", number: 42, headBranch: "topic" }, + }), + ).toBeNull(); + expect( + panelPullRequestCheckoutCommand({ + reference: reference("bitbucket.org"), + identity: undefined, + summary: { provider: "bitbucket", number: 42, headBranch: "topic" }, + headRepositoryNameWithOwner: "acme/web", + }), + ).toBe( + "git clone --single-branch --branch topic https://bitbucket.org/acme/web.git t3code-pr-42", + ); + }); }); const TIMELINE_SOURCE: Pick< diff --git a/apps/web/src/components/pullRequest/pullRequestDetail.logic.ts b/apps/web/src/components/pullRequest/pullRequestDetail.logic.ts index b34cab9b5c..70b161c9c7 100644 --- a/apps/web/src/components/pullRequest/pullRequestDetail.logic.ts +++ b/apps/web/src/components/pullRequest/pullRequestDetail.logic.ts @@ -145,6 +145,23 @@ export function loadingPullRequestCheckoutCommand( return pullRequestCheckoutCommand(provider, reference.number, ""); } +/** Keep the checkout affordance stable while richer PR detail is loading. */ +export function panelPullRequestCheckoutCommand(input: { + readonly reference: PullRequestRef; + readonly identity: RepositoryIdentity | null | undefined; + readonly summary: Pick | null; + readonly headRepositoryNameWithOwner?: string | null | undefined; +}): string | null { + return input.summary + ? pullRequestCheckoutCommand( + input.summary.provider, + input.summary.number, + input.summary.headBranch, + input.headRepositoryNameWithOwner, + ) + : loadingPullRequestCheckoutCommand(input.reference, input.identity); +} + /** Activity changes only when the same host resource reports a newer revision. */ export function shouldRefreshPullRequestActivity( previous: { readonly key: string; readonly updatedAt: string } | null, diff --git a/apps/web/src/components/settings/KeybindingsSettings.tsx b/apps/web/src/components/settings/KeybindingsSettings.tsx index 75d0b8363a..88191afabc 100644 --- a/apps/web/src/components/settings/KeybindingsSettings.tsx +++ b/apps/web/src/components/settings/KeybindingsSettings.tsx @@ -9,6 +9,7 @@ import { TriangleAlertIcon, XIcon, } from "lucide-react"; +import { useLocation } from "@tanstack/react-router"; import { type KeyboardEvent, type ReactNode, @@ -69,7 +70,7 @@ import { whenNodeRemoveLabel, } from "./KeybindingsSettings.logic"; import { SettingsPageContainer, SettingsRow, SettingsSection } from "./settingsLayout"; -import { searchableSetting } from "./settingsSearch"; +import { keybindingSearchAnchorId, searchableSetting } from "./settingsSearch"; import { Tooltip, TooltipPopup, TooltipTrigger } from "../ui/tooltip"; import { useAtomCommand } from "../../state/use-atom-command"; @@ -858,7 +859,11 @@ interface KeybindingRowActions { onRemove: (row: KeybindingRow) => void; } -type KeybindingRowProps = KeybindingRowActions & { row: KeybindingRow; isSaving: boolean }; +type KeybindingRowProps = KeybindingRowActions & { + row: KeybindingRow; + isSaving: boolean; + anchorId?: string | undefined; +}; /** Shortcut pill that turns into a capture input when clicked, plus Save once the draft changes. */ function KeybindingKeyControl({ @@ -1069,11 +1074,12 @@ function KeybindingHoverRowMenu(props: { /** One binding as a settings row: pills flush right, actions fading in beside them on hover. */ function KeybindingSettingsRow(props: KeybindingRowProps) { - const { row, isSaving, allRows, variables, onSave, onReset, onRemove } = props; + const { row, isSaving, anchorId, allRows, variables, onSave, onReset, onRemove } = props; const editor = useKeybindingRowEditor({ row, allRows, onSave }); return ( } description={} @@ -1339,6 +1345,16 @@ function KeybindingsList(props: KeybindingsListProps) { onSave: rowActions.onSave, onCancel: onCancelAdd, }; + const anchorIds = useMemo(() => { + const ids = new Map(); + const seen = new Set(); + for (const row of rows) { + if (seen.has(row.command)) continue; + seen.add(row.command); + ids.set(row.id, keybindingSearchAnchorId(row.command)); + } + return ids; + }, [rows]); return (
{isAddingBinding ? : null} @@ -1346,6 +1362,7 @@ function KeybindingsList(props: KeybindingsListProps) { @@ -1400,6 +1417,12 @@ export function KeybindingsSettingsPanel() { const [savingCommand, setSavingCommand] = useState(null); const [isAddingBinding, setIsAddingBinding] = useState(false); const rows = useMemo(() => buildKeybindingRows(keybindings, query), [keybindings, query]); + const searchTargetId = useLocation({ select: (location) => location.hash.replace(/^#/, "") }); + const [handledSearchTargetId, setHandledSearchTargetId] = useState(searchTargetId); + if (searchTargetId !== handledSearchTargetId) { + setHandledSearchTargetId(searchTargetId); + if (searchTargetId.startsWith("keybinding-")) setQuery(""); + } const commandOptions = useMemo(() => buildKeybindingCommandOptions(keybindings), [keybindings]); const whenVariables = useMemo(() => buildWhenVariableOptions(), []); diff --git a/apps/web/src/components/settings/ProjectDefaultsSettings.tsx b/apps/web/src/components/settings/ProjectDefaultsSettings.tsx index 57991dd4bf..8c34ca7fa1 100644 --- a/apps/web/src/components/settings/ProjectDefaultsSettings.tsx +++ b/apps/web/src/components/settings/ProjectDefaultsSettings.tsx @@ -3,6 +3,7 @@ import { EnvironmentId, type ModelSelection, type ProviderInstanceId, + type WorktreeSubmodules, } from "@t3tools/contracts"; import { createModelSelection } from "@t3tools/shared/model"; import { useNavigate } from "@tanstack/react-router"; @@ -17,7 +18,7 @@ import { } from "../../providerInstances"; import { useEnvironments } from "../../state/environments"; import { EMPTY_SERVER_PROVIDERS } from "../../state/server"; -import { resolveEnvModeLabel } from "../BranchToolbar.logic"; +import { resolveEnvModeLabel, WORKTREE_SUBMODULES_LABELS } from "../BranchToolbar.logic"; import { ProviderModelPicker } from "../chat/ProviderModelPicker"; import { runtimeModeConfig, runtimeModeOptions } from "../chat/runtimeModeConfig"; import { PULL_REQUEST_MERGE_METHOD_LABELS } from "../pullRequest/pullRequestDetail.logic"; @@ -46,6 +47,11 @@ import { * environment defaults at an environment scope and project overrides at a * project or checkout scope; the scoped hooks route the write. */ +const WORKTREE_SUBMODULES_OPTIONS = ["recursive", "top-level", "none"] as const; +function isWorktreeSubmodules(value: string | null): value is WorktreeSubmodules { + return value !== null && (WORKTREE_SUBMODULES_OPTIONS as readonly string[]).includes(value); +} + export function ProjectDefaultsSettings({ category }: { category: ProjectSettingsCategory }) { const { scope, target, targets, connectedEnvironments } = useSettingsScope(); const settings = useScopedSettings(); @@ -71,6 +77,13 @@ export function ProjectDefaultsSettings({ category }: { category: ProjectSetting const mixedPermissions = useScopedSettingsMixed(["defaultRuntimeMode"]); const PermissionIcon = runtimeModeConfig[settings.defaultRuntimeMode].icon; const mixedWorkspace = useScopedSettingsMixed(["defaultThreadEnvMode"]); + const mixedSubmodules = useScopedSettingsMixed(["worktreeSubmodules"]); + const submoduleSettingSupported = + connectedEnvironments.length > 0 && + connectedEnvironments.every( + (environment) => + environment.serverConfig?.environment.capabilities.worktreeSubmodules === true, + ); const mixedBrowser = useScopedSettingsMixed(["enableAgentBrowserAccess"]); const mixedDevice = useScopedSettingsMixed(["enableAgentDeviceAccess"]); const mixedAutoPull = useScopedSettingsMixed(["defaultAutoPull"]); @@ -89,6 +102,7 @@ export function ProjectDefaultsSettings({ category }: { category: ProjectSetting category === "general" && checkout ? checkout.workspaceRoot : null, ); const repositoryEnvMode = t3File.file?.defaultThreadEnvMode ?? null; + const repositorySubmodules = t3File.file?.worktreeSubmodules ?? null; const inheritedEnvModeLabel = workspaceSource === "project" ? null @@ -285,6 +299,67 @@ export function ProjectDefaultsSettings({ category }: { category: ProjectSetting } /> + updateSettings({ worktreeSubmodules: null })} + /> + ) : null + } + control={ + + } + /> + {props.separator ? · : null} {props.prefix ? {props.prefix} : null} ) : null; const statusLineClassName = - "flex min-w-0 flex-wrap items-center gap-x-1.5 text-[13px] leading-[1.45] text-muted-foreground/80"; + "flex min-w-0 flex-wrap items-baseline gap-x-1.5 text-[13px] leading-[1.45] text-muted-foreground/80"; // The editor's status line. It doubles as the account identity row, so the // redacted email lives here rather than in the list, where a masked address diff --git a/apps/web/src/components/settings/RedactedSensitiveText.tsx b/apps/web/src/components/settings/RedactedSensitiveText.tsx index 95b31b4036..cc29228ae4 100644 --- a/apps/web/src/components/settings/RedactedSensitiveText.tsx +++ b/apps/web/src/components/settings/RedactedSensitiveText.tsx @@ -44,7 +44,7 @@ export function RedactedSensitiveText(props: {
); } @@ -685,6 +703,7 @@ function UsageEnvironmentFilter({ isPartial, duplicateSources, staleEnvironments, + approximateEnvironments, }: { readonly environments: readonly EnvironmentUsageStatus[]; readonly selectedEnvironments: readonly EnvironmentUsageStatus[]; @@ -694,6 +713,7 @@ function UsageEnvironmentFilter({ readonly isPartial: boolean; readonly duplicateSources: readonly string[]; readonly staleEnvironments: readonly string[]; + readonly approximateEnvironments: readonly string[]; }) { const [modelPricesOpen, setModelPricesOpen] = useState(false); const allSelected = selectedEnvironmentIds === null; @@ -708,7 +728,8 @@ function UsageEnvironmentFilter({ ).length; const hasIssue = selectedEnvironments.some((environment) => environment.error !== null) || - staleEnvironments.length > 0; + staleEnvironments.length > 0 || + approximateEnvironments.length > 0; return ( <> @@ -807,6 +828,7 @@ function UsageEnvironmentFilter({ environments={selectedEnvironments} duplicateSources={duplicateSources} staleEnvironments={staleEnvironments} + approximateEnvironments={approximateEnvironments} /> ) : null} diff --git a/apps/web/src/index.css b/apps/web/src/index.css index 90c63f62ba..d7c49958ff 100644 --- a/apps/web/src/index.css +++ b/apps/web/src/index.css @@ -1911,6 +1911,19 @@ code { background: transparent !important; } +/* A completed block can arrive before the assistant finishes. Animate only + fresh live blocks, and honor Pylon's panel-motion and reduced-motion controls. */ +@media (prefers-reduced-motion: no-preference) { + [data-panel-animations="true"] .chat-markdown[data-streaming] > *, + [data-panel-animations="true"] .chat-markdown[data-streaming] .chat-markdown-shiki { + transition: opacity var(--panel-animation-duration, 600ms) ease-out; + + @starting-style { + opacity: 0; + } + } +} + /* Diagnostics-style tables: row separators only, uppercase headers, and a scroll-fade container for horizontal overflow. The root chat-markdown wrapping rules (overflow-wrap: anywhere) would let columns shrink to single diff --git a/apps/web/src/lib/selectionActions.test.ts b/apps/web/src/lib/selectionActions.test.ts index 56f51863b6..d677b28587 100644 --- a/apps/web/src/lib/selectionActions.test.ts +++ b/apps/web/src/lib/selectionActions.test.ts @@ -24,7 +24,12 @@ function createSelectionSurface({ interactiveActions = false } = {}) { }); const element = Object.assign(new EventTarget(), { ownerDocument: document, - contains: (target: unknown): boolean => target === element, + contains: (target: unknown): boolean => target === element || target === buttonIcon, + closest: () => null, + }); + const button = new EventTarget(); + const buttonIcon = Object.assign(new EventTarget(), { + closest: (selector: string) => (selector === "button, [role=button]" ? button : null), }); const onSelection = vi.fn(); const onDismiss = vi.fn(); @@ -58,7 +63,7 @@ function createSelectionSurface({ interactiveActions = false } = {}) { const press = event("pointerdown", { button, isPrimary }); Object.defineProperty(press, "target", { value: target }); document.dispatchEvent(press); - if (target === element && !consumed) element.dispatchEvent(press); + if (element.contains(target) && !consumed) element.dispatchEvent(press); }; const focus = (target: EventTarget) => { document.activeElement = target; @@ -92,6 +97,7 @@ function createSelectionSurface({ interactiveActions = false } = {}) { view, document, element, + buttonIcon, actionElement, field, onSelection, @@ -177,6 +183,31 @@ describe("selection action gestures", () => { expect(surface.onSelection).not.toHaveBeenCalled(); }); + it.each([1, 2])( + "dismisses selection actions when a button is pressed after %i clicks", + (detail) => { + const surface = createSelectionSurface(); + surface.down(); + surface.up({ detail }); + surface.flush(); + surface.onSelection.mockClear(); + surface.onDismiss.mockClear(); + + surface.down({ target: surface.buttonIcon }); + surface.up(); + surface.change(); + surface.flush(1000); + expect(surface.onDismiss).toHaveBeenCalledWith("interaction"); + expect(surface.onSelection).not.toHaveBeenCalled(); + + surface.down(); + surface.change(); + surface.up(); + surface.flush(); + expect(surface.onSelection).toHaveBeenCalledOnce(); + }, + ); + it("ignores nonprimary pointers and non-left mouseup without ending a left drag", () => { const surface = createSelectionSurface(); surface.down({ isPrimary: false }); diff --git a/apps/web/src/lib/selectionActions.ts b/apps/web/src/lib/selectionActions.ts index 91e6c2e6a1..5954a770aa 100644 --- a/apps/web/src/lib/selectionActions.ts +++ b/apps/web/src/lib/selectionActions.ts @@ -101,6 +101,10 @@ export function observeSelectionActions({ }; const onSelectionStart = (event: PointerEvent) => { if (!event.isPrimary) return; + const target = event.target as Element | null; + // Buttons can preserve an existing text selection while opening a menu. + // Their release must not reopen selection actions beside the button. + if (target?.closest("button, [role=button]")) return; // A surface may consume a press for a link or terminal mouse reporting. gestureActive = event.button === 0 && !event.defaultPrevented; dismissed = !gestureActive; diff --git a/docs/operations/observability.md b/docs/operations/observability.md index 807a2bd3bc..3659377718 100644 --- a/docs/operations/observability.md +++ b/docs/operations/observability.md @@ -555,6 +555,18 @@ OTLP export: If the OTLP URLs are unset, local tracing still works, metrics stay in-process only, and logs stay on stdout only. +### The Kill Switch + +`T3CODE_OTEL_SDK_DISABLED` and `OTEL_SDK_DISABLED` turn off every OTLP export in both the server and +the desktop main process, overriding any endpoint from the environment or Settings. Local trace +files and stdout logs are unaffected. + +`T3CODE_OTEL_SDK_DISABLED` wins when set, so `T3CODE_OTEL_SDK_DISABLED=false` re-enables export on a +machine that sets `OTEL_SDK_DISABLED` for everything else. It accepts the usual boolean spellings +(`true`/`false`, `yes`/`no`, `on`/`off`, `1`/`0`, `y`/`n`). `OTEL_SDK_DISABLED` follows the +OpenTelemetry specification and only `true` disables export, so `OTEL_SDK_DISABLED=1` does not. +Values are case-insensitive and trimmed. An unrecognized value is ignored with a startup warning. + ### What Is Instrumented Today Current high-value span and metric boundaries include: diff --git a/docs/user/project-settings.md b/docs/user/project-settings.md index 52851e5ff7..c1656d683b 100644 --- a/docs/user/project-settings.md +++ b/docs/user/project-settings.md @@ -39,6 +39,10 @@ on each selected environment, and reset returns to the environment's shared list `t3.json` actions can be imported there. For workspace mode, a project's `t3.json` preference applies when the project has no override. +For new worktrees, choose whether git initializes submodules recursively, only at the top level, +or not at all. A project override takes precedence over its environment setting; when neither +is set, `worktreeSubmodules` in the newly checked out branch's `t3.json` applies, otherwise +initialization is recursive. This does not change existing worktrees. Browser access changes apply when an agent session next starts. ## Project icons diff --git a/docs/user/providers-claude.md b/docs/user/providers-claude.md index 190f7591bd..5c3b442f39 100644 --- a/docs/user/providers-claude.md +++ b/docs/user/providers-claude.md @@ -132,9 +132,16 @@ Use `https://openrouter.ai/api`, not `/api/v1`. If that Claude config directory login, run `/logout` in a Claude Code session using that directory before starting the router setup. Cached login credentials can conflict with the router token. -Verify requests with `/status` in a Claude session or in OpenRouter's activity dashboard. For -model-role overrides and current compatibility requirements, use the -[OpenRouter Claude Code guide](https://openrouter.ai/docs/cookbook/coding-agents/claude-code-integration). +Choose the model in Pylon's chat model picker. If the OpenRouter model is not listed, open that +Claude instance in **Settings → Providers**, add its full model ID with **Add custom model**, then +select it in the picker. Claude Code's `ANTHROPIC_DEFAULT_*_MODEL` variables map role aliases such +as `sonnet`; they do not replace a model ID explicitly selected in Pylon. Custom models may expose +fewer effort, thinking, or context controls than built-in models. + +Verify the router connection with `/status` in a Claude session using that config directory, and +check OpenRouter's activity dashboard for the model used by Pylon. For model-role overrides and +current compatibility requirements, use the +[OpenRouter Claude Code guide](https://openrouter.ai/docs/guides/coding-agents/claude-code-integration). ## Other routers diff --git a/docs/user/usage.md b/docs/user/usage.md index 3d9283cc80..924431d66d 100644 --- a/docs/user/usage.md +++ b/docs/user/usage.md @@ -13,7 +13,9 @@ Usage includes each configured account's history, including disabled accounts. C the account's home setting or its `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, or `GROK_HOME` environment variable. Use absolute paths in the account's environment settings; relative environment paths depend on each project's working directory and cannot be reliably discovered -by Usage. Accounts sharing a history directory count once. +by Usage. Accounts sharing an identifiable history directory count once, while distinct directories +keep their own totals. If an older server cannot separate those totals, or a directory's filesystem +identity is unavailable, Usage warns that shared history may be counted more than once. Antigravity totals come from saved native conversations, including standalone CLI history when available. Models without known prices still contribute tokens; set a custom model price to include @@ -55,7 +57,9 @@ not report that window. Refresh to update the readings. During native Claude sessions, account-wide quota reports update immediately. Model-specific weekly limits reconcile with the signed-in account after about a minute; provider throttling can delay them. -Pylon keeps the last reading when it cannot verify the account. +Pylon keeps a recent reading through a failed quota check when it can still verify the same +account. It hides that reading while account status is unknown, after sign-out, or after a different +account signs in. Antigravity Google-account limits belong to the account signed in to that Pylon provider instance. Settings and Limits show the model groups reported by Google. The thread composer selects the group diff --git a/infra/relay/migrations/postgres/20260918175607_long_thread_ids/migration.sql b/infra/relay/migrations/postgres/20260918175607_long_thread_ids/migration.sql new file mode 100644 index 0000000000..1bb8be0559 --- /dev/null +++ b/infra/relay/migrations/postgres/20260918175607_long_thread_ids/migration.sql @@ -0,0 +1,2 @@ +ALTER TABLE "relay_agent_activity_rows" ALTER COLUMN "thread_id" SET DATA TYPE varchar(512) USING "thread_id"::varchar(512);--> statement-breakpoint +ALTER TABLE "relay_delivery_attempts" ALTER COLUMN "thread_id" SET DATA TYPE varchar(512) USING "thread_id"::varchar(512); \ No newline at end of file diff --git a/infra/relay/migrations/postgres/20260918175607_long_thread_ids/snapshot.json b/infra/relay/migrations/postgres/20260918175607_long_thread_ids/snapshot.json new file mode 100644 index 0000000000..4e56b797b9 --- /dev/null +++ b/infra/relay/migrations/postgres/20260918175607_long_thread_ids/snapshot.json @@ -0,0 +1,1516 @@ +{ + "version": "8", + "dialect": "postgres", + "id": "4b6d0d21-8d78-4499-9dde-b42bcf633d05", + "prevIds": ["1109ed23-036c-4c39-a459-52422a7ffd99"], + "ddl": [ + { + "isRlsEnabled": false, + "name": "relay_agent_activity_rows", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_delivery_attempts", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_dpop_proofs", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_environment_credentials", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_environment_links", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_live_activities", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_managed_endpoint_allocations", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_managed_tunnel_limits", + "entityType": "tables", + "schema": "public" + }, + { + "isRlsEnabled": false, + "name": "relay_mobile_devices", + "entityType": "tables", + "schema": "public" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_id", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_public_key", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "varchar(512)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "thread_id", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "jsonb", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "state_json", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "type": "varchar(36)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(512)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "thread_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "device_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "kind", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "source_job_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(16)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "token_suffix", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "integer", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "apns_status", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "text", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "apns_reason", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(128)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "apns_id", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "text", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "transport_error", + "entityType": "columns", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "type": "varchar(128)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "thumbprint", + "entityType": "columns", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "jti", + "entityType": "columns", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "type": "integer", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "iat", + "entityType": "columns", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "expires_at", + "entityType": "columns", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "credential_id", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_id", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_public_key", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "credential_hash", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "revoked_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_id", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": "'T3 Environment'", + "generated": null, + "identity": null, + "name": "environment_label", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_public_key", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "endpoint_http_base_url", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "endpoint_ws_base_url", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(32)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "endpoint_provider_kind", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "boolean", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": "true", + "generated": null, + "identity": null, + "name": "notifications_enabled", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "boolean", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": "true", + "generated": null, + "identity": null, + "name": "live_activities_enabled", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "boolean", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": "false", + "generated": null, + "identity": null, + "name": "managed_tunnels_enabled", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_by_device_id", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "revoked_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_environment_links" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "device_id", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "text", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "activity_push_token", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "remote_start_queued_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "remote_started_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "ended_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "jsonb", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "last_aggregate_json", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "last_live_activity_delivery_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_live_activities" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "environment_id", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "hostname", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "tunnel_id", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "tunnel_name", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "dns_record_id", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "ready_at", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "type": "varchar(191)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_tunnel_limits" + }, + { + "type": "integer", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "max_tunnels", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_tunnel_limits" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_tunnel_limits" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_managed_tunnel_limits" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "user_id", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "device_id", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "text", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": "'iOS device'", + "generated": null, + "identity": null, + "name": "label", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(16)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "platform", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "integer", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "ios_major_version", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "integer", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "android_api_level", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "app_version", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(255)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "bundle_id", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(16)", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "aps_environment", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "text", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "push_token", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "text", + "typeSchema": null, + "notNull": false, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "push_to_start_token", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "jsonb", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "preferences_json", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "created_at", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "type": "varchar(64)", + "typeSchema": null, + "notNull": true, + "dimensions": 0, + "default": null, + "generated": null, + "identity": null, + "name": "updated_at", + "entityType": "columns", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "updated_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_agent_activity_rows_updated", + "entityType": "indexes", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "environment_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "thread_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "created_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_delivery_attempts_environment", + "entityType": "indexes", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "source_job_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_delivery_attempts_source_job", + "entityType": "indexes", + "schema": "public", + "table": "relay_delivery_attempts" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "expires_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_dpop_proofs_expires_at", + "entityType": "indexes", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "credential_hash", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_environment_credentials_hash", + "entityType": "indexes", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "environment_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "revoked_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_environment_credentials_environment", + "entityType": "indexes", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "environment_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "environment_public_key", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "revoked_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_environment_credentials_environment_key", + "entityType": "indexes", + "schema": "public", + "table": "relay_environment_credentials" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "environment_id", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + }, + { + "value": "revoked_at", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": false, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_environment_links_environment", + "entityType": "indexes", + "schema": "public", + "table": "relay_environment_links" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "activity_push_token", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_live_activities_activity_push_token", + "entityType": "indexes", + "schema": "public", + "table": "relay_live_activities" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "hostname", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_managed_endpoint_allocations_hostname", + "entityType": "indexes", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "tunnel_name", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_managed_endpoint_allocations_tunnel_name", + "entityType": "indexes", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "push_token", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_mobile_devices_push_token", + "entityType": "indexes", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "nameExplicit": true, + "columns": [ + { + "value": "push_to_start_token", + "isExpression": false, + "asc": true, + "nullsFirst": false, + "opclass": null + } + ], + "isUnique": true, + "where": null, + "with": "", + "method": "btree", + "concurrently": false, + "name": "idx_relay_mobile_devices_push_to_start_token", + "entityType": "indexes", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "columns": ["environment_id", "environment_public_key", "thread_id"], + "nameExplicit": false, + "name": "relay_agent_activity_rows_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_agent_activity_rows" + }, + { + "columns": ["thumbprint", "jti"], + "nameExplicit": false, + "name": "relay_dpop_proofs_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_dpop_proofs" + }, + { + "columns": ["user_id", "environment_id"], + "nameExplicit": false, + "name": "relay_environment_links_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_environment_links" + }, + { + "columns": ["user_id", "device_id"], + "nameExplicit": false, + "name": "relay_live_activities_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_live_activities" + }, + { + "columns": ["user_id", "environment_id"], + "nameExplicit": false, + "name": "relay_managed_endpoint_allocations_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_managed_endpoint_allocations" + }, + { + "columns": ["user_id", "device_id"], + "nameExplicit": false, + "name": "relay_mobile_devices_pkey", + "entityType": "pks", + "schema": "public", + "table": "relay_mobile_devices" + }, + { + "columns": ["id"], + "nameExplicit": false, + "name": "relay_delivery_attempts_pkey", + "schema": "public", + "table": "relay_delivery_attempts", + "entityType": "pks" + }, + { + "columns": ["credential_id"], + "nameExplicit": false, + "name": "relay_environment_credentials_pkey", + "schema": "public", + "table": "relay_environment_credentials", + "entityType": "pks" + }, + { + "columns": ["user_id"], + "nameExplicit": false, + "name": "relay_managed_tunnel_limits_pkey", + "schema": "public", + "table": "relay_managed_tunnel_limits", + "entityType": "pks" + } + ], + "renames": [] +} diff --git a/infra/relay/src/http/Api.test.ts b/infra/relay/src/http/Api.test.ts index 66b87e643a..bf7bd1f5a4 100644 --- a/infra/relay/src/http/Api.test.ts +++ b/infra/relay/src/http/Api.test.ts @@ -1,4 +1,15 @@ +import * as HttpServer from "effect/unstable/http/HttpServer"; +import { + RelayClientAuth, + RelayClientPrincipal, + type RelayClientDeviceRecord, +} from "@t3tools/contracts/relay"; +import * as EnvironmentLinker from "../environments/EnvironmentLinker.ts"; +import * as RelayTokens from "../auth/RelayTokens.ts"; +import * as Devices from "../agentActivity/Devices.ts"; import { createClerkClient, verifyToken } from "@clerk/backend"; +import * as NodeHttpPlatform from "@effect/platform-node/NodeHttpPlatform"; +import * as NodeServices from "@effect/platform-node/NodeServices"; import { describe, expect, it } from "@effect/vitest"; import { vi } from "vite-plus/test"; import * as Context from "effect/Context"; @@ -12,22 +23,21 @@ import * as Predicate from "effect/Predicate"; import * as Redacted from "effect/Redacted"; import * as TestClock from "effect/testing/TestClock"; import * as Tracer from "effect/Tracer"; +import * as Etag from "effect/unstable/http/Etag"; import * as HttpRouter from "effect/unstable/http/HttpRouter"; import * as HttpServerRequest from "effect/unstable/http/HttpServerRequest"; import * as HttpServerResponse from "effect/unstable/http/HttpServerResponse"; -import * as HttpServer from "effect/unstable/http/HttpServer"; -import * as HttpApiBuilder from "effect/unstable/httpapi/HttpApiBuilder"; import * as HttpApi from "effect/unstable/httpapi/HttpApi"; +import * as HttpApiBuilder from "effect/unstable/httpapi/HttpApiBuilder"; import { EnvironmentId } from "@t3tools/contracts"; import { - RelayApi, - RelayClientAuth, - RelayClientPrincipal, RelayEnvironmentAuth, - type RelayClientDeviceRecord, + RelayEnvironmentPrincipal, + RelayApi, } from "@t3tools/contracts/relay"; import { + RELAY_HTTP_ROUTER_CONFIG, RELAY_REQUEST_DEADLINE_MS, clientApi, relayCors, @@ -36,6 +46,7 @@ import { relayNotFoundRoute, relayDpopFailureReason, revokeEnvironmentLinkRecord, + serverApi, traceRelayHttpRequestWith, unlinkEnvironmentRecord, verifyRelayClientBearerToken, @@ -46,9 +57,8 @@ import * as RelayDb from "../db.ts"; import * as EnvironmentCredentials from "../environments/EnvironmentCredentials.ts"; import * as EnvironmentLinks from "../environments/EnvironmentLinks.ts"; import * as ManagedEndpointProvider from "../environments/ManagedEndpointProvider.ts"; -import * as EnvironmentLinker from "../environments/EnvironmentLinker.ts"; -import * as RelayTokens from "../auth/RelayTokens.ts"; -import * as Devices from "../agentActivity/Devices.ts"; +import * as AgentActivityPublisher from "../agentActivity/AgentActivityPublisher.ts"; +import * as EnvironmentPublishSignatures from "../environments/EnvironmentPublishSignatures.ts"; vi.mock("@clerk/backend", () => ({ createClerkClient: vi.fn(), @@ -606,6 +616,105 @@ describe("relay request tracing", () => { }); describe("relay routing fallback", () => { + it.effect("publishes activity for escaped delegated thread IDs longer than 100 characters", () => + Effect.gen(function* () { + const environmentId = "environment-1"; + const environmentPublicKey = "environment-public-key"; + const published: Array< + Parameters[0] + > = []; + const verified: Array< + Parameters< + EnvironmentPublishSignatures.EnvironmentPublishSignatures["Service"]["verify"] + >[0] + > = []; + const publisher = Layer.succeed(AgentActivityPublisher.AgentActivityPublisher, { + publish: (input) => + Effect.sync(() => { + published.push(input); + return { ok: true as const, deliveries: [] }; + }), + replayForLiveActivityRegistration: () => Effect.succeed(null), + }); + const signatures = Layer.succeed(EnvironmentPublishSignatures.EnvironmentPublishSignatures, { + verify: (input) => + Effect.sync(() => { + verified.push(input); + }), + }); + const auth = Layer.succeed(RelayEnvironmentAuth, { + environmentBearer: (effect) => + effect.pipe( + Effect.provideService(RelayEnvironmentPrincipal, { + environmentId, + environmentPublicKey, + }), + ), + }); + const routes = HttpApiBuilder.layer( + HttpApi.make("RelayApi").add(RelayApi.groups.server), + ).pipe( + Layer.provide(serverApi.pipe(Layer.provide([publisher, signatures]))), + Layer.provide(auth), + Layer.provide([NodeServices.layer, NodeHttpPlatform.layer, Etag.layerWeak]), + ); + const httpEffect = yield* HttpRouter.toHttpEffect( + Layer.mergeAll(routes, relayNotFoundRoute, relayCors), + ).pipe(Effect.provideService(HttpRouter.RouterConfig, RELAY_HTTP_ROUTER_CONFIG)); + const threadIds = [ + "b7c8c522-d244-43dc-875f-7224fce79912", + "t".repeat(512), + `thread:${"t".repeat(505)}`, + "thread:delegated-task:command%3Amcp%3Ab7c8c522-d244-43dc-875f-7224fce79912%3Adelegate-task%3Agreet-subagent-20260813", + ]; + for (const threadId of threadIds) { + const request = HttpServerRequest.fromWeb( + new Request( + `https://relay.test/v1/environments/${environmentId}/threads/${encodeURIComponent(threadId)}/agent-activity`, + { + method: "POST", + headers: { + authorization: "Bearer environment-credential", + "content-type": "application/json", + }, + body: '{"state":null,"proof":"signed-proof"}', + }, + ), + ); + const response = yield* httpEffect.pipe( + Effect.provideService(HttpServerRequest.HttpServerRequest, request), + ); + expect(response.status).toBe(200); + } + const overLimit = HttpServerRequest.fromWeb( + new Request( + `https://relay.test/v1/environments/${environmentId}/threads/${"t".repeat(513)}/agent-activity`, + { + method: "POST", + headers: { + authorization: "Bearer environment-credential", + "content-type": "application/json", + }, + body: '{"state":null,"proof":"signed-proof"}', + }, + ), + ); + const overLimitResponse = yield* httpEffect.pipe( + Effect.provideService(HttpServerRequest.HttpServerRequest, overLimit), + ); + expect(overLimitResponse.status).toBe(404); + expect(published).toEqual( + threadIds.map((threadId) => ({ + environmentId, + environmentPublicKey, + threadId, + state: null, + })), + ); + expect(verified.map((input) => input.threadId)).toEqual(threadIds); + }).pipe(Effect.scoped), + ); + it.effect("redirects the relay root to the API docs", () => Effect.gen(function* () { const request = HttpServerRequest.fromWeb(new Request("https://relay.test/")); diff --git a/infra/relay/src/http/Api.ts b/infra/relay/src/http/Api.ts index fcaeca6404..5a4d0c9ca4 100644 --- a/infra/relay/src/http/Api.ts +++ b/infra/relay/src/http/Api.ts @@ -71,6 +71,12 @@ import * as MobileRegistrations from "../agentActivity/MobileRegistrations.ts"; import { withSpanAttributes } from "../observability.ts"; import * as RelayDb from "../db.ts"; +// Delegated thread IDs carry escaped command provenance and exceed the router's +// default 100-character path parameter limit. Match the environment server. +export const RELAY_HTTP_ROUTER_CONFIG = { + maxParamLength: 512, +} as const; + const relayCorsAllowedMethods = ["GET", "POST", "DELETE", "OPTIONS"] as const; const relayCorsAllowedHeaders = [ "authorization", diff --git a/infra/relay/src/persistence/schema.ts b/infra/relay/src/persistence/schema.ts index d9a7074bc0..fd766c7589 100644 --- a/infra/relay/src/persistence/schema.ts +++ b/infra/relay/src/persistence/schema.ts @@ -138,7 +138,7 @@ export const relayAgentActivityRows = pgTable( { environmentId: varchar("environment_id", { length: 191 }).notNull(), environmentPublicKey: text("environment_public_key").notNull(), - threadId: varchar("thread_id", { length: 191 }).notNull(), + threadId: varchar("thread_id", { length: 512 }).notNull(), stateJson: jsonb("state_json").notNull().$type(), updatedAt: varchar("updated_at", { length: 64 }).notNull(), createdAt: varchar("created_at", { length: 64 }).notNull(), @@ -156,7 +156,7 @@ export const relayDeliveryAttempts = pgTable( createdAt: varchar("created_at", { length: 64 }).notNull(), userId: varchar("user_id", { length: 255 }), environmentId: varchar("environment_id", { length: 191 }), - threadId: varchar("thread_id", { length: 191 }), + threadId: varchar("thread_id", { length: 512 }), deviceId: varchar("device_id", { length: 255 }), kind: varchar("kind", { length: 64 }).notNull(), sourceJobId: varchar("source_job_id", { length: 64 }), diff --git a/infra/relay/src/worker.ts b/infra/relay/src/worker.ts index da52da67c0..7cb05920e8 100644 --- a/infra/relay/src/worker.ts +++ b/infra/relay/src/worker.ts @@ -23,6 +23,7 @@ import { healthApi, metadataApi, mobileApi, + RELAY_HTTP_ROUTER_CONFIG, relayClientAuthLayer, relayDpopClientAuthLayer, relayCors, @@ -351,6 +352,7 @@ export const ApiLive = Api.make( relayNotFoundRoute, ).pipe( HttpRouter.toHttpEffect, + Effect.provideService(HttpRouter.RouterConfig, RELAY_HTTP_ROUTER_CONFIG), withoutCapturedParentSpan, Effect.flatMap((httpEffect) => traceRelayHttpRequestWith(httpEffect, relayTraceLayer)), ); diff --git a/packages/client-runtime/src/state/pullRequests.test.ts b/packages/client-runtime/src/state/pullRequests.test.ts index 6187cf2e7b..fe6ea90a59 100644 --- a/packages/client-runtime/src/state/pullRequests.test.ts +++ b/packages/client-runtime/src/state/pullRequests.test.ts @@ -21,6 +21,7 @@ import * as EnvironmentSupervisor from "../connection/supervisor.ts"; import type { WsRpcProtocolClient } from "../rpc/protocol.ts"; import type { RpcSession } from "../rpc/session.ts"; import { + createLinkedPullRequestSummaryAtomFamily, createPullRequestEnvironmentAtoms, createPullRequestStackAtomFamily, } from "./pullRequests.ts"; @@ -134,6 +135,108 @@ it.effect("keeps concurrent diff file reads on different hosts separate", () => ), ); +it.effect("shares close, reopen, and merge with an untouched client's mounted PR readers", () => + Effect.scoped( + Effect.gen(function* () { + const revision = yield* SubscriptionRef.make(0); + let state: "open" | "closed" | "merged" = "open"; + const client = { + [WS_METHODS.pullRequestsSubscribeRefreshes]: () => SubscriptionRef.changes(revision), + [WS_METHODS.pullRequestsSummary]: () => Effect.sync(() => ({ state })), + [WS_METHODS.pullRequestsDetail]: () => Effect.sync(() => ({ state })), + [WS_METHODS.pullRequestsList]: () => + Effect.sync(() => ({ entries: [{ number: 1, state }] })), + [WS_METHODS.pullRequestsRunAction]: (input: { + readonly action: "close" | "reopen" | "merge"; + }) => + Effect.gen(function* () { + state = + input.action === "close" ? "closed" : input.action === "merge" ? "merged" : "open"; + yield* SubscriptionRef.update(revision, (value) => value + 1); + }), + } as unknown as WsRpcProtocolClient; + const writer = yield* makeTestRuntime(client); + const reader = yield* makeTestRuntime(client); + const target = { + environmentId: TARGET.environmentId, + input: { + projectId: ProjectId.make("project-1"), + host: "github.example.com", + repository: "acme/web", + number: 1, + }, + }; + const detail = reader.atoms.detail(target); + const summary = createLinkedPullRequestSummaryAtomFamily( + reader.runtime, + reader.atoms.refreshes, + )(target); + const list = reader.atoms.list({ + environmentId: TARGET.environmentId, + input: { state: "all" }, + }); + const unmountDetail = reader.registry.mount(detail); + const unmountSummary = reader.registry.mount(summary); + const unmountList = reader.registry.mount(list); + yield* Effect.addFinalizer(() => + Effect.sync(() => { + unmountDetail(); + unmountSummary(); + unmountList(); + }), + ); + expect((yield* AtomRegistry.getResult(reader.registry, detail)).state).toBe("open"); + expect((yield* AtomRegistry.getResult(reader.registry, summary)).state).toBe("open"); + expect((yield* AtomRegistry.getResult(reader.registry, list)).entries[0]?.state).toBe("open"); + + for (const [action, expected] of [ + ["close", "closed"], + ["reopen", "open"], + ["merge", "merged"], + ] as const) { + const detailChanged = Latch.makeUnsafe(); + const summaryChanged = Latch.makeUnsafe(); + const listChanged = Latch.makeUnsafe(); + const stops = [ + reader.registry.subscribe(detail, (result) => { + if (AsyncResult.isSuccess(result) && result.value.state === expected) { + detailChanged.openUnsafe(); + } + }), + reader.registry.subscribe(summary, (result) => { + if (AsyncResult.isSuccess(result) && result.value.state === expected) { + summaryChanged.openUnsafe(); + } + }), + reader.registry.subscribe(list, (result) => { + if (AsyncResult.isSuccess(result) && result.value.entries[0]?.state === expected) { + listChanged.openUnsafe(); + } + }), + ]; + yield* Effect.addFinalizer(() => Effect.sync(() => stops.forEach((stop) => stop()))); + const result = yield* Effect.promise(() => + writer.atoms.runAction.run(writer.registry, { + ...target, + input: { ...target.input, action }, + }), + ); + expect(AsyncResult.isSuccess(result)).toBe(true); + // The second client receives only the server push: no local refresh or timer tick. + yield* detailChanged.await; + yield* summaryChanged.await; + yield* listChanged.await; + expect((yield* AtomRegistry.getResult(reader.registry, detail)).state).toBe(expected); + expect((yield* AtomRegistry.getResult(reader.registry, summary)).state).toBe(expected); + expect((yield* AtomRegistry.getResult(reader.registry, list)).entries[0]?.state).toBe( + expected, + ); + stops.forEach((stop) => stop()); + } + }), + ), +); + it.effect("refreshes pull request activity after a comment is updated", () => Effect.scoped( Effect.gen(function* () { diff --git a/packages/client-runtime/src/state/subagentRuntime.test.ts b/packages/client-runtime/src/state/subagentRuntime.test.ts index ff066e045f..e1aa979c2e 100644 --- a/packages/client-runtime/src/state/subagentRuntime.test.ts +++ b/packages/client-runtime/src/state/subagentRuntime.test.ts @@ -71,6 +71,29 @@ function fold(rows: ReadonlyArray) { } describe("foldSubagentActivities", () => { + it("shows a Prime child note while active and its outcome after completion", () => { + const [agent] = fold([ + activity("task.progress", { + taskId: "prime-child-1", + taskType: "subagent", + title: "Review tests", + status: "running", + summary: "Checking the failing test", + }), + activity("task.completed", { + taskId: "prime-child-1", + taskType: "subagent", + status: "completed", + summary: "Completed", + }), + ]); + expect(agent).toMatchObject({ + status: "completed", + progress: "Checking the failing test", + result: "Completed", + }); + }); + it("shows the batch status limit after its parent turn ends without claiming a result", () => { const running = activity("task.progress", { taskId: "batch-1", diff --git a/packages/contracts/src/environment.test.ts b/packages/contracts/src/environment.test.ts index 1802ea1d0e..61ff6127e1 100644 --- a/packages/contracts/src/environment.test.ts +++ b/packages/contracts/src/environment.test.ts @@ -30,6 +30,16 @@ describe("ExecutionEnvironmentDescriptor", () => { expect(decodeDescriptor(descriptor).capabilities.pullRequests).toBeUndefined(); }); + it("requires an explicit worktree submodules capability", () => { + expect(decodeDescriptor(descriptor).capabilities.worktreeSubmodules).toBeUndefined(); + expect( + decodeDescriptor({ + ...descriptor, + capabilities: { ...descriptor.capabilities, worktreeSubmodules: true }, + }).capabilities.worktreeSubmodules, + ).toBe(true); + }); + it("preserves an advertised pull-request capability", () => { expect( decodeDescriptor({ diff --git a/packages/contracts/src/environment.ts b/packages/contracts/src/environment.ts index 5760c38758..49372cdbb4 100644 --- a/packages/contracts/src/environment.ts +++ b/packages/contracts/src/environment.ts @@ -58,6 +58,10 @@ export const ExecutionEnvironmentPlatform = Schema.Struct({ */ export const ThreadEnvMode = Schema.Literals(["local", "worktree"]); export type ThreadEnvMode = typeof ThreadEnvMode.Type; + +/** How new worktrees initialize git submodules. */ +export const WorktreeSubmodules = Schema.Literals(["recursive", "top-level", "none"]); +export type WorktreeSubmodules = typeof WorktreeSubmodules.Type; export type ExecutionEnvironmentPlatform = typeof ExecutionEnvironmentPlatform.Type; /** How a server can replace itself with another version when asked over RPC. @@ -109,6 +113,8 @@ export const ExecutionEnvironmentCapabilities = Schema.Struct({ threadRestartContinuation: Schema.optionalKey(Schema.Boolean), /** Server resolves `projectSettingsOverrides`; older servers ignore the key. */ projectSettingsOverrides: Schema.optionalKey(Schema.Boolean), + /** Server applies the worktree submodule initialization setting. */ + worktreeSubmodules: Schema.optionalKey(Schema.Boolean), /** Server accepts and applies the default permission mode for new threads. */ defaultRuntimeMode: Schema.optionalKey(Schema.Boolean), /** Server understands thread.snooze / thread.unsnooze commands. Same diff --git a/packages/contracts/src/rpc.ts b/packages/contracts/src/rpc.ts index 1d481cc512..5580d8a67a 100644 --- a/packages/contracts/src/rpc.ts +++ b/packages/contracts/src/rpc.ts @@ -567,7 +567,8 @@ const WsServerRefreshProvidersRpc = Rpc.make(WS_METHODS.serverRefreshProviders, */ instanceId: Schema.optional(ProviderInstanceId), cwd: Schema.optional(TrimmedNonEmptyString), - /** Explicit user request. Background status refreshes must not open agent sessions. */ + /** Explicit user request: bypass provider discovery caches and rediscover models. + * Background status refreshes must not open agent sessions. */ refreshModels: Schema.optional(Schema.Boolean), }), success: ServerProviderUpdatedPayload, diff --git a/packages/contracts/src/settings.test.ts b/packages/contracts/src/settings.test.ts index bc124651cb..471fde67d7 100644 --- a/packages/contracts/src/settings.test.ts +++ b/packages/contracts/src/settings.test.ts @@ -812,6 +812,13 @@ describe("provider enabled defaults", () => { }); describe("ServerSettings worktree defaults", () => { + it("defaults submodules to repository settings and accepts explicit modes", () => { + expect(decodeServerSettings({}).worktreeSubmodules).toBeNull(); + for (const mode of ["recursive", "top-level", "none"] as const) { + expect(decodeServerSettingsPatch({ worktreeSubmodules: mode }).worktreeSubmodules).toBe(mode); + } + expect(decodeServerSettingsPatch({ worktreeSubmodules: null }).worktreeSubmodules).toBeNull(); + }); it("defaults start-from-origin on for legacy configs", () => { expect(decodeServerSettings({}).newWorktreesStartFromOrigin).toBe(true); }); diff --git a/packages/contracts/src/settings.ts b/packages/contracts/src/settings.ts index a9569dedbd..0b59ae9c50 100644 --- a/packages/contracts/src/settings.ts +++ b/packages/contracts/src/settings.ts @@ -7,13 +7,14 @@ import * as Schema from "effect/Schema"; import * as SchemaTransformation from "effect/SchemaTransformation"; import { ForwardCompatibleNullable, + ForwardCompatibleOptional, NonNegativeInt, ProjectId, TrimmedNonEmptyString, TrimmedString, } from "./baseSchemas.ts"; import { UsageLimitSourceId } from "./usageLimitSourceId.ts"; -import { EnvironmentMachineKind, ThreadEnvMode } from "./environment.ts"; +import { EnvironmentMachineKind, ThreadEnvMode, WorktreeSubmodules } from "./environment.ts"; import { KeybindingShortcut } from "./keybindings.ts"; import { CustomModelSetting, @@ -1110,6 +1111,7 @@ export const PROJECT_SCOPED_SERVER_SETTING_KEYS = [ "defaultRuntimeMode", "defaultThreadEnvMode", "newWorktreesStartFromOrigin", + "worktreeSubmodules", "defaultAutoPull", "defaultProjectScripts", "enableAgentBrowserAccess", @@ -1135,6 +1137,7 @@ export const ProjectSettingsOverrides = Schema.Struct({ defaultRuntimeMode: Schema.optionalKey(RuntimeMode), defaultThreadEnvMode: Schema.optionalKey(ThreadEnvMode), newWorktreesStartFromOrigin: Schema.optionalKey(Schema.Boolean), + worktreeSubmodules: ForwardCompatibleOptional(WorktreeSubmodules), defaultAutoPull: Schema.optionalKey(Schema.Boolean), defaultProjectScripts: Schema.optionalKey(Schema.Array(ProjectScript)), enableAgentBrowserAccess: Schema.optionalKey(Schema.Boolean), @@ -1284,6 +1287,10 @@ export const ServerSettings = Schema.Struct({ newWorktreesStartFromOrigin: Schema.Boolean.pipe( Schema.withDecodingDefault(Effect.succeed(true)), ), + /** Null defers to the newly created checkout's t3.json, then recursive. */ + worktreeSubmodules: ForwardCompatibleNullable(WorktreeSubmodules).pipe( + Schema.withDecodingDefault(Effect.succeed(null)), + ), addProjectBaseDirectory: TrimmedString.pipe(Schema.withDecodingDefault(Effect.succeed(""))), textGenerationModelSelection: ModelSelection.pipe( Schema.withDecodingDefault( @@ -1572,6 +1579,7 @@ export const ServerSettingsPatch = Schema.Struct({ environmentIcon: Schema.optionalKey(Schema.NullOr(EnvironmentMachineKind)), defaultThreadEnvMode: Schema.optionalKey(ThreadEnvMode), newWorktreesStartFromOrigin: Schema.optionalKey(Schema.Boolean), + worktreeSubmodules: Schema.optionalKey(Schema.NullOr(WorktreeSubmodules)), addProjectBaseDirectory: Schema.optionalKey(TrimmedString), textGenerationModelSelection: Schema.optionalKey(ModelSelectionPatch), sourceControlWritingStyle: Schema.optionalKey( diff --git a/packages/contracts/src/t3ProjectFile.test.ts b/packages/contracts/src/t3ProjectFile.test.ts index 4352449c7a..b475b96347 100644 --- a/packages/contracts/src/t3ProjectFile.test.ts +++ b/packages/contracts/src/t3ProjectFile.test.ts @@ -60,4 +60,11 @@ describe("T3ProjectFile", () => { expect(decode({ defaultThreadEnvMode: "local" }).defaultThreadEnvMode).toBe("local"); expect(() => decode({ defaultThreadEnvMode: "remote" })).toThrow(); }); + + it("decodes worktree submodule modes and rejects unknown modes", () => { + for (const mode of ["recursive", "top-level", "none"] as const) { + expect(decode({ worktreeSubmodules: mode }).worktreeSubmodules).toBe(mode); + } + expect(() => decode({ worktreeSubmodules: "shallow" })).toThrow(); + }); }); diff --git a/packages/contracts/src/t3ProjectFile.ts b/packages/contracts/src/t3ProjectFile.ts index 08f7b17e27..bbcee1a540 100644 --- a/packages/contracts/src/t3ProjectFile.ts +++ b/packages/contracts/src/t3ProjectFile.ts @@ -1,7 +1,7 @@ import * as Schema from "effect/Schema"; import * as SchemaTransformation from "effect/SchemaTransformation"; -import { ThreadEnvMode } from "./environment.ts"; +import { ThreadEnvMode, WorktreeSubmodules } from "./environment.ts"; import { ProjectScriptIcon } from "./orchestration.ts"; /** File name of the checked-in T3 project file, resolved at the workspace root. */ @@ -86,6 +86,12 @@ export const T3ProjectFile = Schema.Struct({ 'Where new threads start for this repository: "worktree" for a fresh git worktree, "local" for the current checkout. A per-project setting in Pylon overrides this; when neither is set, the global default applies.', }), ), + worktreeSubmodules: Schema.optionalKey( + WorktreeSubmodules.annotate({ + description: + 'How new worktrees populate git submodules: "recursive" (default), "top-level", or "none".', + }), + ), scripts: Schema.optionalKey( Schema.Array(T3ProjectFileScript) .annotate({ diff --git a/packages/contracts/src/usage.ts b/packages/contracts/src/usage.ts index 3506d733d6..33ff1a8bbe 100644 --- a/packages/contracts/src/usage.ts +++ b/packages/contracts/src/usage.ts @@ -21,7 +21,7 @@ import { NonNegativeInt, TrimmedNonEmptyString } from "./baseSchemas.ts"; * client renders partial coverage when an environment reports an older version * rather than failing the whole page. */ -export const USAGE_CONTRACT_VERSION = 6 as const; +export const USAGE_CONTRACT_VERSION = 7 as const; /** * Oldest {@link UsageSummary} version a current client will still merge. @@ -150,6 +150,12 @@ export const UsageSource = Schema.Struct({ * those overcounts; this is the figure clients should total. */ distinctSessions: NonNegativeInt, + /** + * Buckets attributed to this directory after the server's scan-wide record + * de-duplication. Added in v7; older peers only report provider-wide buckets. + * Raw transcript records never cross the wire. + */ + buckets: Schema.optional(Schema.Array(UsageBucket)), message: Schema.NullOr(TrimmedNonEmptyString), }); export type UsageSource = typeof UsageSource.Type; diff --git a/packages/shared/package.json b/packages/shared/package.json index 73bbd85c31..ee4fc811b2 100644 --- a/packages/shared/package.json +++ b/packages/shared/package.json @@ -55,6 +55,10 @@ "types": "./src/observability.ts", "import": "./src/observability.ts" }, + "./otelEnvironment": { + "types": "./src/otelEnvironment.ts", + "import": "./src/otelEnvironment.ts" + }, "./httpObservability": { "types": "./src/httpObservability.ts", "import": "./src/httpObservability.ts" diff --git a/packages/shared/src/delimitedPreview.test.ts b/packages/shared/src/delimitedPreview.test.ts index 303e904285..0188f9da50 100644 --- a/packages/shared/src/delimitedPreview.test.ts +++ b/packages/shared/src/delimitedPreview.test.ts @@ -21,6 +21,35 @@ describe("delimited file previews", () => { ["", "three"], ]); }); + describe.each([",", "\t"] as const)("with delimiter %j", (delimiter) => { + it("preserves a final quoted empty record without requiring a line ending", () => { + for (const ending of ["", "\n", "\r\n"]) { + expect(parseDelimitedPreview(`name\r\nAlice\r\n""${ending}`, delimiter)).toEqual({ + rows: [["name"], ["Alice"], [""]], + truncated: false, + }); + } + expect(parseDelimitedPreview('""', delimiter)).toEqual({ + rows: [[""]], + truncated: false, + }); + }); + it("does not invent records for empty input or a trailing line ending", () => { + for (const prefix of ["", "\ufeff"]) { + expect(parseDelimitedPreview(prefix, delimiter).rows).toEqual([]); + expect(parseDelimitedPreview(`${prefix}name\r\n`, delimiter).rows).toEqual([["name"]]); + expect(parseDelimitedPreview(`${prefix}""`, delimiter).rows).toEqual([[""]]); + } + }); + it("keeps a quoted empty record at the row limit", () => { + const text = `${"name\n".repeat(99)}""`; + const preview = parseDelimitedPreview(text, delimiter); + expect(preview.rows).toHaveLength(100); + expect(preview.rows.at(-1)).toEqual([""]); + expect(preview.truncated).toBe(false); + expect(parseDelimitedPreview(`${text}\nextra`, delimiter).truncated).toBe(true); + }); + }); it("bounds rows, columns and cell length and reports partial content", () => { const table = parseDelimitedPreview( Array.from({ length: 101 }, () => diff --git a/packages/shared/src/delimitedPreview.ts b/packages/shared/src/delimitedPreview.ts index d4e24409a0..1cfbd41a78 100644 --- a/packages/shared/src/delimitedPreview.ts +++ b/packages/shared/src/delimitedPreview.ts @@ -16,12 +16,13 @@ export function parseDelimitedPreview(text: string, delimiter: "," | "\t") { let cell = ""; let quoted = false; let truncated = false; + let rowStart = text.charCodeAt(0) === 0xfeff ? 1 : 0; const endCell = () => { if (row.length < 30) row.push(cell); else truncated = true; cell = ""; }; - for (let index = text.charCodeAt(0) === 0xfeff ? 1 : 0; index < text.length; index++) { + for (let index = rowStart; index < text.length; index++) { const char = text[index]; if (char === '"') { if (quoted && text[index + 1] === '"') { @@ -41,12 +42,13 @@ export function parseDelimitedPreview(text: string, delimiter: "," | "\t") { rows.push(row); row = []; if (char === "\r" && text[index + 1] === "\n") index++; + rowStart = index + 1; if (rows.length === 100) return { rows, truncated: truncated || index < text.length - 1 }; } } else if (cell.length < 2000) cell += char; else truncated = true; } - if (cell.length || row.length) { + if (rowStart < text.length) { endCell(); rows.push(row); } diff --git a/packages/shared/src/otelEnvironment.test.ts b/packages/shared/src/otelEnvironment.test.ts new file mode 100644 index 0000000000..3b6dac4b50 --- /dev/null +++ b/packages/shared/src/otelEnvironment.test.ts @@ -0,0 +1,82 @@ +import { assert, describe, it } from "@effect/vitest"; +import * as ConfigProvider from "effect/ConfigProvider"; +import * as Effect from "effect/Effect"; + +import * as OtelEnvironment from "./otelEnvironment.ts"; + +const load = (env: Record) => + OtelEnvironment.load.pipe(Effect.provide(ConfigProvider.layer(ConfigProvider.fromEnv({ env })))); + +const SPEC_OFF = + "OTEL_SDK_DISABLED is set, so no telemetry is exported, whatever configured it; set T3CODE_OTEL_SDK_DISABLED=false to export anyway"; +const T3_OFF = + "T3CODE_OTEL_SDK_DISABLED is set, so no telemetry is exported, whatever configured it"; +const specIgnored = (value: string) => + `OTEL_SDK_DISABLED=${value} was read as false; the OpenTelemetry specification recognizes only the string true, so use OTEL_SDK_DISABLED=true or T3CODE_OTEL_SDK_DISABLED to say it any other way`; + +describe("OtelEnvironment", () => { + it.effect.each([ + { name: "nothing set", env: {}, disabled: false, warnings: [] }, + // OTEL_SDK_DISABLED follows the specification: only `true`, case-insensitively. + { name: "spec true", env: { OTEL_SDK_DISABLED: "true" }, disabled: true, warnings: [SPEC_OFF] }, + { name: "spec True", env: { OTEL_SDK_DISABLED: "True" }, disabled: true, warnings: [SPEC_OFF] }, + { + name: "spec padded", + env: { OTEL_SDK_DISABLED: " true " }, + disabled: true, + warnings: [SPEC_OFF], + }, + { name: "spec false", env: { OTEL_SDK_DISABLED: "false" }, disabled: false, warnings: [] }, + { + name: "spec 1", + env: { OTEL_SDK_DISABLED: "1" }, + disabled: false, + warnings: [specIgnored("1")], + }, + { + name: "spec padded yes", + env: { OTEL_SDK_DISABLED: " yes " }, + disabled: false, + warnings: [specIgnored("yes")], + }, + // T3CODE_OTEL_SDK_DISABLED takes Config.Boolean's values, case-insensitively. + { name: "t3 1", env: { T3CODE_OTEL_SDK_DISABLED: "1" }, disabled: true, warnings: [T3_OFF] }, + { + name: "t3 TRUE", + env: { T3CODE_OTEL_SDK_DISABLED: "TRUE" }, + disabled: true, + warnings: [T3_OFF], + }, + { name: "t3 n", env: { T3CODE_OTEL_SDK_DISABLED: "n" }, disabled: false, warnings: [] }, + { + name: "t3 false overrides spec true", + env: { T3CODE_OTEL_SDK_DISABLED: "false", OTEL_SDK_DISABLED: "true" }, + disabled: false, + warnings: [], + }, + { + name: "blank t3 falls through", + env: { T3CODE_OTEL_SDK_DISABLED: " ", OTEL_SDK_DISABLED: "true" }, + disabled: true, + warnings: [SPEC_OFF], + }, + { + name: "unreadable t3 warns and falls through", + env: { T3CODE_OTEL_SDK_DISABLED: "maybe", OTEL_SDK_DISABLED: "true" }, + disabled: true, + warnings: ["T3CODE_OTEL_SDK_DISABLED=maybe is not a yes or a no and was ignored", SPEC_OFF], + }, + { + name: "bad spec value still warns when t3 answered", + env: { T3CODE_OTEL_SDK_DISABLED: "false", OTEL_SDK_DISABLED: "yes" }, + disabled: false, + warnings: [specIgnored("yes")], + }, + ])("$name", ({ env, disabled, warnings }) => + Effect.gen(function* () { + const resolved = yield* load(env); + assert.strictEqual(resolved.disabled, disabled); + assert.deepStrictEqual(resolved.warnings, warnings); + }), + ); +}); diff --git a/packages/shared/src/otelEnvironment.ts b/packages/shared/src/otelEnvironment.ts new file mode 100644 index 0000000000..8bbf50198c --- /dev/null +++ b/packages/shared/src/otelEnvironment.ts @@ -0,0 +1,105 @@ +/** + * otelEnvironment: the OpenTelemetry kill switch, shared by the server and the + * desktop main process so both agree on what turns export off. + * + * `T3CODE_OTEL_SDK_DISABLED` is read first, so a machine that sets + * `OTEL_SDK_DISABLED` for everything else can still opt Pylon back in. + * + * @module otelEnvironment + */ +import * as Config from "effect/Config"; +import * as Effect from "effect/Effect"; +import * as Schema from "effect/Schema"; +import * as SchemaTransformation from "effect/SchemaTransformation"; + +export interface OtelEnvironment { + /** Whether OTLP export is off, whatever endpoint is configured. */ + readonly disabled: boolean; + /** Messages for the caller to log once at startup. */ + readonly warnings: ReadonlyArray; +} + +interface Flag { + /** `undefined` when the variable is unset, blank, or unreadable. */ + readonly value: boolean | undefined; + readonly warning?: string; +} + +const TrimmedLowercase = Schema.String.pipe( + Schema.decodeTo( + Schema.String, + SchemaTransformation.trim().compose(SchemaTransformation.toLowerCase()), + ), +); + +/** + * Reads a boolean that accepts `truthy` and `falsy`, ignoring case and padding. + * Any other value is ignored with a warning rather than failing startup. + */ +const flag = ( + name: string, + truthy: ReadonlyArray, + falsy: ReadonlyArray, + invalid: (value: string) => string, +) => + Config.schema( + TrimmedLowercase.pipe(Schema.decodeTo(Schema.Literals([...truthy, ...falsy]))), + name, + ).pipe( + Config.map((value): Flag => ({ value: truthy.includes(value) })), + Config.orElse(() => + Config.string(name).pipe( + Config.map((raw): Flag => { + const value = raw.trim(); + return value === "" + ? { value: undefined } + : { value: undefined, warning: invalid(value) }; + }), + ), + ), + Config.withDefault({ value: undefined }), + ); + +// `Config.Boolean`'s literals, which effect does not export on their own. +const T3CODE_TRUE = ["true", "yes", "on", "1", "y"]; +const T3CODE_FALSE = ["false", "no", "off", "0", "n"]; + +export const load: Effect.Effect = Config.all({ + t3: flag( + "T3CODE_OTEL_SDK_DISABLED", + T3CODE_TRUE, + T3CODE_FALSE, + (value) => `T3CODE_OTEL_SDK_DISABLED=${value} is not a yes or a no and was ignored`, + ), + // The specification: a boolean it defines is true "only by the + // case-insensitive string `true`", implementations "MUST NOT" accept other + // values as true, and should warn about unrecognized ones. + spec: flag( + "OTEL_SDK_DISABLED", + ["true"], + ["false"], + (value) => + `OTEL_SDK_DISABLED=${value} was read as false; the OpenTelemetry specification recognizes only the string true, so use OTEL_SDK_DISABLED=true or T3CODE_OTEL_SDK_DISABLED to say it any other way`, + ), +}).pipe( + Effect.map(({ t3, spec }) => { + const disabled = t3.value ?? spec.value ?? false; + const warnings = [t3.warning, spec.warning].filter((warning) => warning !== undefined); + if (disabled) { + warnings.push( + t3.value + ? "T3CODE_OTEL_SDK_DISABLED is set, so no telemetry is exported, whatever configured it" + : "OTEL_SDK_DISABLED is set, so no telemetry is exported, whatever configured it; set T3CODE_OTEL_SDK_DISABLED=false to export anyway", + ); + } + return { disabled, warnings }; + }), + // Every read above falls back instead of failing, so this cannot happen. + Effect.orDie, +); + +/** An environment that asked for nothing, for tests and for the pairing CLI. */ +export const none: OtelEnvironment = { + disabled: false, + warnings: [], +}; diff --git a/packages/shared/src/projectSettings.test.ts b/packages/shared/src/projectSettings.test.ts index 26efbda8dd..cd7ed360dd 100644 --- a/packages/shared/src/projectSettings.test.ts +++ b/packages/shared/src/projectSettings.test.ts @@ -19,6 +19,18 @@ const projectId = ProjectId.make("project-a"); const otherProjectId = ProjectId.make("project-b"); describe("resolveProjectSettings", () => { + it("lets a project override the worktree submodule mode and restore inheritance", () => { + const settings = applyServerSettingsPatch(DEFAULT_SERVER_SETTINGS, { + worktreeSubmodules: "top-level", + projectSettingsOverrides: { [projectId]: { worktreeSubmodules: "none" } }, + }); + expect(resolveProjectSettings(settings, projectId).settings.worktreeSubmodules).toBe("none"); + expect(resolveProjectSettings(settings, otherProjectId).settings.worktreeSubmodules).toBe( + "top-level", + ); + const inherited = clearProjectSettingsOverrides(settings, projectId, ["worktreeSubmodules"]); + expect(inherited).toBeNull(); + }); it("inherits every scopable key when the project has no overrides", () => { const resolved = resolveProjectSettings(DEFAULT_SERVER_SETTINGS, projectId); expect(resolved.settings).toBe(DEFAULT_SERVER_SETTINGS); diff --git a/packages/shared/src/t3ProjectFile.test.ts b/packages/shared/src/t3ProjectFile.test.ts index 924a01c441..42045cc428 100644 --- a/packages/shared/src/t3ProjectFile.test.ts +++ b/packages/shared/src/t3ProjectFile.test.ts @@ -36,6 +36,7 @@ describe("buildT3ProjectFileJsonSchema", () => { "defaultThreadEnvMode", "iconPath", "scripts", + "worktreeSubmodules", ]); expect(schema.required).toBeUndefined(); expect(schema.properties.iconPath?.description).toContain("Workspace-relative path"); diff --git a/packages/shared/src/usageMerge.test.ts b/packages/shared/src/usageMerge.test.ts index 3da3865022..60412b9f1d 100644 --- a/packages/shared/src/usageMerge.test.ts +++ b/packages/shared/src/usageMerge.test.ts @@ -41,6 +41,7 @@ function summary( homePath: string; volumeId?: string; distinctSessions?: number; + buckets?: readonly UsageBucket[]; }[], contractVersion: number = USAGE_CONTRACT_VERSION, ): UsageSummary { @@ -63,6 +64,7 @@ function summary( skippedFiles: 0, malformedRecords: 0, distinctSessions: source.distinctSessions ?? 1, + ...(source.buckets === undefined ? {} : { buckets: source.buckets }), message: null, })), pricing: { status: "fresh", source: "litellm", fetchedAt: null, knownModels: 10 }, @@ -75,6 +77,183 @@ function environment(id: string, usageSummary: UsageSummary): EnvironmentUsage { } describe("mergeUsage", () => { + it("keeps unique older homes while taking a newer shared-home scan", () => { + const shared = { provider: "claude" as const, hostId: "mac", homePath: "/shared" }; + const old = summary( + [bucket({ costUsd: 10 })], + [ + { ...shared, buckets: [bucket({ costUsd: 4 })] }, + { ...shared, homePath: "/unique-a", buckets: [bucket({ costUsd: 6 })] }, + ], + ); + const latest = summary( + [bucket({ costUsd: 16 })], + [ + { ...shared, buckets: [bucket({ costUsd: 5 })] }, + { ...shared, homePath: "/unique-b", buckets: [bucket({ costUsd: 11 })] }, + ], + ); + const merged = mergeUsage( + [ + environment("env-a", { ...old, readAt: "2026-08-07T00:00:00.000Z" }), + environment("env-b", { ...latest, readAt: "2026-08-08T00:00:00.000Z" }), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(22); + expect(merged.approximateEnvironments).toEqual([]); + }); + + it("counts overlapping Claude homes once while retaining both unique homes", () => { + const shared = { provider: "claude" as const, hostId: "mac", homePath: "/shared" }; + const uniqueA = { provider: "claude" as const, hostId: "mac", homePath: "/unique-a" }; + const uniqueB = { provider: "claude" as const, hostId: "mac", homePath: "/unique-b" }; + const four = bucket({ costUsd: 4, records: 1 }); + const six = bucket({ costUsd: 6, records: 1 }); + const eleven = bucket({ costUsd: 11, records: 1 }); + const merged = mergeUsage( + [ + environment( + "env-a", + summary( + [bucket({ costUsd: 10, records: 2 })], + [ + { ...shared, buckets: [four] }, + { ...uniqueA, buckets: [six] }, + ], + ), + ), + environment( + "env-b", + summary( + [bucket({ costUsd: 15, records: 2 })], + [ + { ...shared, buckets: [four] }, + { ...uniqueB, buckets: [eleven] }, + ], + ), + ), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(21); + expect(merged.records).toBe(3); + expect(merged.sessions).toBe(3); + expect(merged.duplicateSources).toHaveLength(1); + expect(merged.approximateEnvironments).toEqual([]); + }); + + it("does not collapse identical paths on different hosts or filesystems", () => { + const source = { provider: "claude" as const, hostId: "mac", homePath: "/same" }; + const merged = mergeUsage( + [ + environment( + "env-a", + summary([bucket({ costUsd: 4 })], [{ ...source, buckets: [bucket({ costUsd: 4 })] }]), + ), + environment( + "env-b", + summary( + [bucket({ costUsd: 6 })], + [{ ...source, hostId: "other", buckets: [bucket({ costUsd: 6 })] }], + ), + ), + environment( + "env-c", + summary( + [bucket({ costUsd: 11 })], + [{ ...source, volumeId: "moved-volume", buckets: [bucket({ costUsd: 11 })] }], + ), + ), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(21); + expect(merged.duplicateSources).toEqual([]); + }); + + it("keeps unknown filesystem identities separate and reports uncertain overlap", () => { + const source = { provider: "claude" as const, hostId: "mac", homePath: "/same" }; + const merged = mergeUsage( + [ + environment( + "env-a", + summary( + [bucket({ costUsd: 4 })], + [{ ...source, volumeId: "", buckets: [bucket({ costUsd: 4 })] }], + ), + ), + environment( + "env-b", + summary( + [bucket({ costUsd: 6 })], + [{ ...source, volumeId: "vol-mac", buckets: [bucket({ costUsd: 6 })] }], + ), + ), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(10); + expect(merged.duplicateSources).toEqual([]); + expect(merged.approximateEnvironments).toEqual(["env-a", "env-b"]); + }); + + it("marks legacy mixed-home totals approximate without dropping their unique usage", () => { + const shared = { provider: "claude" as const, hostId: "mac", homePath: "/shared" }; + const merged = mergeUsage( + [ + environment( + "env-a", + summary([bucket({ costUsd: 4 })], [{ ...shared, buckets: [bucket({ costUsd: 4 })] }]), + ), + environment( + "env-b", + summary( + [bucket({ costUsd: 15 })], + [shared, { ...shared, homePath: "/unique-b" }], + USAGE_CONTRACT_VERSION - 1, + ), + ), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(19); + expect(merged.approximateEnvironments).toEqual(["env-b"]); + expect(merged.staleEnvironments).toEqual([]); + }); + + it("falls back to flat provider totals when a source claims buckets for another provider", () => { + const shared = { provider: "claude" as const, hostId: "mac", homePath: "/shared" }; + const merged = mergeUsage( + [ + environment( + "env-a", + summary([bucket({ costUsd: 4 })], [{ ...shared, buckets: [bucket({ costUsd: 4 })] }]), + ), + environment( + "env-b", + summary( + [bucket({ costUsd: 15 })], + [ + { ...shared, buckets: [bucket({ provider: "codex", costUsd: 4 })] }, + { ...shared, homePath: "/unique-b", buckets: [bucket({ costUsd: 11 })] }, + ], + ), + ), + ], + USAGE_CONTRACT_VERSION, + ); + + expect(merged.costUsd).toBe(19); + expect(merged.providers.map((provider) => provider.provider)).toEqual(["claude"]); + expect(merged.approximateEnvironments).toEqual(["env-b"]); + }); + it("sums environments that read different transcript directories", () => { const merged = mergeUsage( [ @@ -147,6 +326,52 @@ describe("mergeUsage", () => { ).toEqual({ claude: 1, codex: 1 }); }); + it("uses the newest scan when environments share the same transcript directory", () => { + const source = { provider: "claude" as const, hostId: "mac", homePath: "/home/theo/.claude" }; + const environments = [ + environment("env-a", summary([bucket({ costUsd: 4, records: 2 })], [source])), + environment("env-b", { + ...summary([bucket()], [source]), + readAt: "2026-08-07T01:00:00.000Z", + }), + ]; + + for (const ordered of [environments, environments.toReversed()]) { + const merged = mergeUsage(ordered, USAGE_CONTRACT_VERSION); + expect(merged.costUsd).toBe(10); + expect(merged.records).toBe(5); + expect(merged.sessions).toBe(1); + expect(merged.contributingEnvironments).toEqual(["env-b"]); + expect(merged.duplicateSources).toEqual(["env-a: /home/theo/.claude"]); + } + }); + + it("uses stable environment ids for equal or invalid scan timestamps", () => { + const source = { provider: "claude" as const, hostId: "mac", homePath: "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/shared/.claude" }; + const invalid = environment("env-a", { + ...summary([bucket({ costUsd: 4 })], [source]), + readAt: "invalid", + }); + const equallyNew = [ + environment("env-b", summary([bucket({ costUsd: 10 })], [source])), + environment("env-c", summary([bucket({ costUsd: 20 })], [source])), + ]; + for (const ordered of [[invalid, ...equallyNew], [...equallyNew, invalid].toReversed()]) { + const merged = mergeUsage(ordered, USAGE_CONTRACT_VERSION); + expect(merged.costUsd).toBe(10); + expect(merged.contributingEnvironments).toEqual(["env-b"]); + } + const bothInvalid = mergeUsage( + [ + environment("env-z", { ...summary([bucket({ costUsd: 9 })], [source]), readAt: "bad" }), + invalid, + ], + USAGE_CONTRACT_VERSION, + ); + expect(bothInvalid.costUsd).toBe(4); + expect(bothInvalid.contributingEnvironments).toEqual(["env-a"]); + }); + it("excludes an environment reporting an older contract version", () => { const merged = mergeUsage( [ diff --git a/packages/shared/src/usageMerge.ts b/packages/shared/src/usageMerge.ts index e0cb0510eb..730b610a65 100644 --- a/packages/shared/src/usageMerge.ts +++ b/packages/shared/src/usageMerge.ts @@ -94,6 +94,8 @@ export interface MergedUsage { readonly duplicateSources: readonly string[]; readonly contributingEnvironments: readonly EnvironmentId[]; readonly staleEnvironments: readonly EnvironmentId[]; + /** Legacy mixed homes or unknown filesystem identities prevent exact attribution. */ + readonly approximateEnvironments: readonly EnvironmentId[]; } /** @@ -104,13 +106,16 @@ export interface MergedUsage { * home path, which is every Mac in a fleet, from collapsing into one source and * having one of them silently dropped. */ -function fingerprintKey(fingerprint: UsageSourceFingerprint): string { - return [ +function fingerprintKey(fingerprint: UsageSourceFingerprint, environmentId: EnvironmentId): string { + return JSON.stringify([ fingerprint.hostId, fingerprint.provider, fingerprint.resolvedHomePath, fingerprint.volumeId, - ].join(" "); + // An unreadable filesystem identity cannot prove two remote environments + // see the same directory, even if their hostnames and paths match. + fingerprint.volumeId === "" ? environmentId : "", + ]); } /** @@ -118,23 +123,39 @@ function fingerprintKey(fingerprint: UsageSourceFingerprint): string { * * Several environments on one machine (worktree servers, for instance) resolve * the same provider home and would otherwise double count every token. The - * first environment in a stable order claims a fingerprint; the rest have that - * provider's buckets dropped. Environments are sorted by id so the winner does - * not change between renders. + * most recently read summary claims a fingerprint; environment ids break ties + * so ownership stays stable when two summaries have the same read time. */ function claimSources(environments: readonly EnvironmentUsage[]): { readonly ownerByFingerprint: ReadonlyMap; readonly duplicates: readonly string[]; + readonly uncertainEnvironments: readonly EnvironmentId[]; } { const ownerByFingerprint = new Map(); const duplicates: string[] = []; + const weakMatches = new Map(); - const ordered = [...environments].sort((a, b) => a.environmentId.localeCompare(b.environmentId)); + const ordered = [...environments].sort( + (a, b) => + (Date.parse(b.summary.readAt) || 0) - (Date.parse(a.summary.readAt) || 0) || + a.environmentId.localeCompare(b.environmentId), + ); for (const environment of ordered) { for (const source of environment.summary.sources) { if (source.status === "missing") continue; - const key = fingerprintKey(source.fingerprint); + const weakKey = JSON.stringify([ + source.fingerprint.hostId, + source.fingerprint.provider, + source.fingerprint.resolvedHomePath, + ]); + const matches = weakMatches.get(weakKey) ?? []; + matches.push({ + environmentId: environment.environmentId, + volumeId: source.fingerprint.volumeId, + }); + weakMatches.set(weakKey, matches); + const key = fingerprintKey(source.fingerprint, environment.environmentId); if (ownerByFingerprint.has(key)) { duplicates.push(`${environment.label}: ${source.fingerprint.resolvedHomePath}`); continue; @@ -143,7 +164,17 @@ function claimSources(environments: readonly EnvironmentUsage[]): { } } - return { ownerByFingerprint, duplicates }; + const uncertainEnvironments = new Set(); + for (const matches of weakMatches.values()) { + if ( + matches.some((match) => match.volumeId === "") && + new Set(matches.map((match) => match.environmentId)).size > 1 + ) { + for (const match of matches) uncertainEnvironments.add(match.environmentId); + } + } + + return { ownerByFingerprint, duplicates, uncertainEnvironments: [...uncertainEnvironments] }; } /** Sources this environment owns after fingerprint claims, plus their buckets. */ @@ -153,12 +184,16 @@ function ownedContribution( ): { readonly buckets: readonly UsageBucket[]; readonly sessionsByProvider: ReadonlyMap; + readonly approximate: boolean; } { const ownedProviders = new Set(); const sessionsByProvider = new Map(); + const sourcesByProvider = new Map(); for (const source of environment.summary.sources) { if (source.status === "missing") continue; - const key = fingerprintKey(source.fingerprint); + const providerSources = sourcesByProvider.get(source.fingerprint.provider) ?? []; + sourcesByProvider.set(source.fingerprint.provider, [...providerSources, source]); + const key = fingerprintKey(source.fingerprint, environment.environmentId); if (ownerByFingerprint.get(key) === environment.environmentId) { const provider = source.fingerprint.provider; ownedProviders.add(provider); @@ -170,9 +205,45 @@ function ownedContribution( ); } } + const buckets: UsageBucket[] = []; + let approximate = false; + for (const [provider, sources] of sourcesByProvider) { + if (!ownedProviders.has(provider)) continue; + // Mixed-version peers still have provider-wide buckets. Keep those totals + // visible, but explicitly report their attribution as approximate when a + // provider spans both owned and duplicated directories. + const granular = sources.every( + (source) => + source.buckets !== undefined && + source.buckets.every((bucket) => bucket.provider === source.fingerprint.provider), + ); + if (granular) { + for (const source of sources) { + if ( + ownerByFingerprint.get(fingerprintKey(source.fingerprint, environment.environmentId)) === + environment.environmentId + ) { + buckets.push(...(source.buckets ?? [])); + } + } + } else { + buckets.push(...environment.summary.buckets.filter((bucket) => bucket.provider === provider)); + if ( + sources.some( + (source) => + ownerByFingerprint.get( + fingerprintKey(source.fingerprint, environment.environmentId), + ) !== environment.environmentId, + ) + ) { + approximate = true; + } + } + } return { - buckets: environment.summary.buckets.filter((bucket) => ownedProviders.has(bucket.provider)), + buckets, sessionsByProvider, + approximate, }; } @@ -213,6 +284,7 @@ const EMPTY_MERGED: MergedUsage = { duplicateSources: [], contributingEnvironments: [], staleEnvironments: [], + approximateEnvironments: [], }; /** @@ -242,7 +314,7 @@ export function mergeUsage( } } - const { ownerByFingerprint, duplicates } = claimSources(current); + const { ownerByFingerprint, duplicates, uncertainEnvironments } = claimSources(current); let costUsd = 0; let uncachedInputTokens = 0; @@ -289,9 +361,14 @@ export function mergeUsage( } >(); const contributingEnvironments: EnvironmentId[] = []; + const approximateEnvironments = new Set(uncertainEnvironments); for (const environment of current) { - const { buckets, sessionsByProvider } = ownedContribution(environment, ownerByFingerprint); + const { buckets, sessionsByProvider, approximate } = ownedContribution( + environment, + ownerByFingerprint, + ); + if (approximate) approximateEnvironments.add(environment.environmentId); if (buckets.length > 0) contributingEnvironments.push(environment.environmentId); for (const [providerKind, providerSessions] of sessionsByProvider) { @@ -444,5 +521,6 @@ export function mergeUsage( duplicateSources: duplicates, contributingEnvironments, staleEnvironments, + approximateEnvironments: [...approximateEnvironments], }; } diff --git a/packages/ssh/src/tunnel.test.ts b/packages/ssh/src/tunnel.test.ts index 822f260a63..4cfa1c09e4 100644 --- a/packages/ssh/src/tunnel.test.ts +++ b/packages/ssh/src/tunnel.test.ts @@ -118,6 +118,7 @@ describe("ssh tunnel scripts", () => { assert.include(script, "remote_node_satisfies_engine()"); assert.include(script, "function satisfiesSemverRange"); assert.include(script, "satisfiesSemverRange(rawVersion, range)"); + assert.include(script, 'prepend_path_if_dir "/home/linuxbrew/.linuxbrew/bin"'); assert.include(script, 'prepend_path_if_dir "$VOLTA_HOME/bin"'); assert.include(script, 'prepend_path_if_dir "$HOME/.asdf/shims"'); assert.include(script, 'prepend_path_if_dir "$HOME/.local/share/mise/shims"'); diff --git a/packages/ssh/src/tunnel.ts b/packages/ssh/src/tunnel.ts index b88872b087..32e5f4ffaf 100644 --- a/packages/ssh/src/tunnel.ts +++ b/packages/ssh/src/tunnel.ts @@ -336,6 +336,7 @@ ensure_remote_node_path() { prepend_path_if_dir "$HOME/.local/bin" prepend_path_if_dir "$HOME/bin" prepend_path_if_dir "/opt/homebrew/bin" + prepend_path_if_dir "/home/linuxbrew/.linuxbrew/bin" prepend_path_if_dir "/usr/local/bin" prepend_path_if_dir "/usr/bin" prepend_path_if_dir "/bin"