feat: SLS-497 Add explicit prestart hooks to the runpod-python SDK

[jasonwang-runpod]

Problem

Workers often load models or start inference engines before calling runpod.serverless.start(). The SDK cannot observe that prestart work, so failures leave requests in IN_QUEUE status while useful info remains in worker logs.

Solution

Add explicit prestart hooks that the SDK supervises before handler execution.

• Workers can register sync or async hooks with @runpod.serverless.register_prestart_hook and provide an optional timeout for the prestart phase.
• Queue-based workers take jobs concurrently with prestart, but don't run the handler function until prestart succeeds.
• A failure or timeout is attached to a request as prestart_failed. Then the worker drains and exits.
• Local input and single-worker hosted API mode run prestart before invoking or serving the handler. Realtime and multi-process hosted API modes are unsupported.
• Failure payloads can carry the last 16 KB of the worker's stdout and stderr. RUNPOD_LOG_CAPTURE gates this: auto (default) captures only when hooks are registered, all always captures, off never does. Note: log capture doesn't see child processes.
• A worker that doesn't register any hooks is unaffected.

Testing

• uv run pytest -q: 710 passed, 94.7% coverage.
• An A40 sglang endpoint (runpod/worker-sglang:2.0.2) across three runs:
  • Unmodified template, no hooks: COMPLETED, and the payload carried output only, with no log field.
  • Same template with a prestart hook: COMPLETED, with the handler held 73 sec while engine started up.
  • Hook pointed at a missing model (intentional failure): FAILED with a prestart_failed payload naming the failing hook, rather than the request sitting in IN_QUEUE.

Supersedes #567.

[Copilot]

Pull request overview

Adds supervised prestart hooks across supported Serverless worker modes, including startup-failure reporting and bounded stdout/stderr capture.

Changes:

• Adds ordered sync/async prestart hooks with optional phase timeout.
• Gates handlers during initialization and drains failed queue workers.
• Captures bounded logs for startup and handler failures.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
runpod/serverless/__init__.py	Exposes hooks and validates runtime modes.
runpod/serverless/worker.py	Installs output capture for workers.
runpod/serverless/modules/rp_prestart.py	Implements hook registration and execution.
runpod/serverless/modules/rp_capture.py	Implements bounded contextual output capture.
runpod/serverless/modules/rp_scale.py	Integrates queue gating, failure delivery, and shutdown.
runpod/serverless/modules/rp_local.py	Runs hooks before local handlers.
runpod/serverless/modules/rp_fastapi.py	Runs hooks through FastAPI lifespan.
runpod/serverless/modules/rp_job.py	Attaches captured logs to handler failures.
docs/serverless/worker.md	Documents prestart configuration and modes.
tests/test_serverless/test_prestart.py	Tests the public hook contract and mode guards.
tests/test_serverless/test_prestart_lifecycle.py	Tests queue lifecycle and process termination.
tests/test_serverless/test_capture.py	Tests capture and output bounds.
tests/test_serverless/test_modules/test_local.py	Tests local prestart behavior.
tests/test_serverless/test_init.py	Verifies the new public export.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Suppressed comments (3)

docs/serverless/worker.md:77

• This overstates what the implementation captures. The active buffer is a ContextVar, and a normal threading.Thread starts without that context, so direct stdout/stderr writes from engine/background threads created by a hook or handler are omitted even while that hook or handler is running. Document the child-thread limitation (or propagate the context) so users do not rely on missing diagnostics.

Capture replaces `sys.stdout`/`sys.stderr` at worker startup, so it sees `print`
and direct stream writes made while a hook or handler runs. It does not see
child processes and log handlers created before startup.

runpod/serverless/modules/rp_capture.py:26

• MAX_CAPTURED_CHARS limits Unicode code points, not the encoded size sent to /job-done. A 16,384-character non-ASCII tail can occupy up to 64 KB in UTF-8 (and the inner json.dumps can expand characters further), so the advertised 16 KB bound and the payload-size protection are not enforced. Bound the encoded/serialized byte length while preserving valid character boundaries; the ring buffer, clip, and prestart payload slicing need to use the same byte-based contract.

MAX_CAPTURED_CHARS = 16 * 1024

tests/test_serverless/test_prestart.py:91

• This assertion has only 10 ms of scheduling slack. On a loaded CI runner the first 30 ms sleep can resume after the 40 ms phase deadline, making the reported hook first and intermittently failing the test. Make the first hook only yield once and have the second block indefinitely so the timeout deterministically occurs in second.

    def test_timeout_bounds_the_whole_phase_and_names_current_hook(self):
        async def first():
            await asyncio.sleep(0.03)

        async def second():
            await asyncio.sleep(0.03)

        with self.assertRaises(PrestartTimeout) as ctx:
            _run(run_prestart_hooks_async((first, second), timeout=0.04))