Skip to content

feat(pi): add validated MLflow OSS streaming - #449

Draft
dgokeeffe wants to merge 4 commits into
databricks:mainfrom
dgokeeffe:pr/pi-oss-sse
Draft

feat(pi): add validated MLflow OSS streaming#449
dgokeeffe wants to merge 4 commits into
databricks:mainfrom
dgokeeffe:pr/pi-oss-sse

Conversation

@dgokeeffe

@dgokeeffe dgokeeffe commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add Pi's validated MLflow/OSS provider and capability metadata
  • repair incomplete SSE termination from compatible OSS backends
  • restore the direct gateway URL and close the loopback proxy on normal exit, errors, SIGTERM and SIGHUP
  • enforce managed Pi model allowlists without falling back to unlisted discovery results
  • preserve model/session affinity and supplemental capability state

Why

Pi could not reliably use coding-oriented OSS models through AI Gateway. Some backends omit the terminal stream fields Pi expects, and an interrupted session could leave the persisted model configuration pointing at a dead loopback port. This adds the compatibility layer with bounded lifecycle and policy handling.

Stack

PR D of five. Depends on PR C (pr/capability-discovery). It is opened as a draft until A, B and C land; the isolated review delta is 763fa3e..5ac7937.

D and E are siblings and can be reviewed or landed independently once C is available.

  • A: #446 — Pi routing foundation
  • B: #447 — Claude discovery hardening
  • C: #448 — capability-driven discovery
  • D: #449 — this PR
  • E: #450 — sibling OpenCode PR

Validation

  • 99 final focused Pi/proxy tests passed after dependency replay
  • SSE termination, truncation, signal handling, cleanup and managed-allowlist cases are covered
  • Ruff lint and changed-file formatting passed
  • ty check src and git diff --check passed
  • fresh installed-build Pi inference passed on system.ai.deepseek-v4-pro-0813
  • the isolated proxy URL was restored and its listener was closed after inference
  • the complete integration stack passed 2,358 tests with 37 skips; its three failures reproduce unchanged on main

Residual risk

An uncatchable SIGKILL or power loss can still interrupt cleanup. A subsequent Pi launch repairs a stale loopback URL before starting a new proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant