feat(dev): run agentex locally without Docker - #353
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Stand up the full backend as host processes with embedded datastores — no Docker daemon required — as a lighter alternative to the container stack. `./dev.sh local` (also `make dev-local` / `python -m scripts.dev_local`) provisions: - Postgres via bundled pgserver (unix socket) and Redis via bundled redislite - a Temporal dev server + UI and the agentex worker (--no-temporal to skip) - a local mongod, required for the full stack (--no-mongo / --lean to skip) - an optional OpenTelemetry collector (--no-otel to skip) then runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on SIGINT/SIGTERM. --lean is a minimal Postgres+Redis+API stack; --ephemeral uses a throwaway data dir. The runner is a small scripts/dev_local package (config / services / supervise / runner) so the pure config/env layer stays testable. App-side changes to make the no-Docker path robust (all no-ops when Mongo is configured, as it always is in Docker/prod): - The Temporal worker no longer crashes when MongoDB is unavailable — the Mongo CRUD adapter tolerates an unset database and errors only on real use, so the worker degrades like the API instead of taking down the stack. - Skip the Mongo connection entirely when MONGODB_URI is unset, removing a ~20s startup hang against the implicit localhost:27017 default. - In local mode the backend rewrites agents' host.docker.internal ACP host to loopback (AGENTEX_ACP_HOST_OVERRIDE), so default-scaffolded agents work without manifest edits. Also fix a frontend dev-server process leak in dev.sh (kill the whole make→npm→next tree and sweep orphans; report status by listening port), correct the MongoDB and OpenTelemetry install commands, and document local mode in README and CLAUDE.md.
356b3eb to
aa8bb61
Compare
| raw = acp_url_override | ||
|
|
||
| # Prefer the production deployment's URL when there's no explicit override. | ||
| if raw is None and agent.production_deployment_id: |
There was a problem hiding this comment.
@smoreinis can you take a look here just make sure it doesn't conflict with the preview workflow?
| """ | ||
| # In docker-free local mode, rewrite host.docker.internal -> the host-reachable | ||
| # override so the healthcheck matches how the request path dials the agent. | ||
| acp_url = resolve_acp_url(acp_url) |
There was a problem hiding this comment.
just making sure, no other places we need to do conversion right?
There was a problem hiding this comment.
yes, only running this mode
| if acp_url_override: | ||
| return acp_url_override | ||
| """Resolve the ACP URL for an agent, optionally overriding with a specific URL. | ||
|
|
There was a problem hiding this comment.
did we lose the override?
There was a problem hiding this comment.
hi, so ./dev.sh no-docker will pass _ACP_HOST_OVERRIDE_ENV variable, so the function resolve_acp_url will resolve this. Besides this, the logic of this function is kept the same
| Returns the URL unchanged when the override env var is unset or the URL does | ||
| not use the Docker sentinel host, so it is safe to call on every ACP dial. | ||
| """ | ||
| override = os.environ.get(_ACP_HOST_OVERRIDE_ENV) |
There was a problem hiding this comment.
is this where the override moved?
There was a problem hiding this comment.
yes, here we override the environment variable _ACP_HOST_OVERRIDE_ENV. This environment is set and passed from config.py file
|
|
||
| > **MongoDB is required for the full local stack** and is always started — the Temporal | ||
| > worker builds Mongo-backed repositories at startup, so a missing/unreachable Mongo | ||
| > makes the runner fail fast (with an install message) rather than crash the worker. |
danielmillerp
left a comment
danielmillerp
left a comment
There was a problem hiding this comment.
lots of small questions! also as far as testing on PC, not super sure. A lot of our clients do use PCs and I know there are Scaliens who have PCs. What problems do you anticipate?
There was a problem hiding this comment.
Overall this is rad.
One UX suggestion: in addition dev.sh, I think this would be really nice to have in the agentex CLI. I.E. you can run a single agent inline. therefore to start running a single agent you can start quick rather than having both the agentex backend and cli running. then this setup would be suitable for someone testing a multiagent flow.
Not sure if that is possible given the abstraction and integration, but wanted to throw it out there as a north star.
+1 to that! |
@danielmillerp So Im testing this on Windows (via AWS Workspaces) but the embedded Redis uses (the redislite package has no native Windows build (Redis ships no supported Windows server) so the dependency required to run locally without Docker won't even install on native Windows. |
Some findings from testing local dev on Windows:
@NiteshDhanpal had a clear view on how to scope this. His take: don't treat Windows-native local mode as a hard requirement for now (given the redislite/Redis limitation), and use a support matrix instead:
His main concern is a reliable fallback so customers aren't blocked when local setup is painful or impossible, and he thinks a cloud/dev-environment fallback is the most reliable option for that. |
I'm aligned @levilentz does that make sense for customers? Also CC: @lucyakoroleva for context per today's standup discussion |
@danielmillerp @aringuyen3 I think broadly this is a good first step. I think assuming that the customer will not have WSL or docker is the way to test this as that it normally the problem. can we just disable redis in this mode? |
I like that! not necessary to go from 0 to 1 to stream |
Default the no-docker embedded Redis to 6379 (was 6390) and the Temporal UI to 8080 (was 8233) so both modes expose the same URLs and quick start needs no port flags. Users can still pass --redis-port / --ui-port if a Docker service already holds a port. Also drop the now-unnecessary per-mode Temporal UI branching in dev.sh and add a README note pointing users to wait for the readiness message before starting an agent.
@danielmillerp @levilentz Redis is required. Agentex uses Redis for streaming and messaging. Referred to this PR #343 |
Add a "Choosing a Setup" section to WINDOWS.md: run docker-free local mode (./dev.sh no-docker) inside WSL2 when it's available, and fall back to the Docker-based PowerShell flow when WSL2 isn't (locked-down or managed environments). Includes a WSL2 quick-start subsection. Expand the README's Windows pointer to surface the same decision from the main entry point.
A previous dev stack whose supervisor died without reaping its children leaves processes bound to the API/datastore ports. The next launch then fails to bind a managed process and tears the whole stack back down, with the real EADDRINUSE buried in a background log. Add a preflight port-conflict check to both start paths that lists any process already listening on a needed port (with PID + command) and offers to kill it before launching: - TTY: prompts [y/N] to kill and continue, else aborts with a clear hint. - DEV_KILL_PORTS=1: kills without prompting (restart/CI). - Non-interactive without the opt-in: warns and continues (unchanged). Kills gracefully via kill_tree (TERM, then force-KILL survivors) so a supervisor and its datastore children go down together. The no-docker check honors the runner's port-override flags; Postgres is skipped there since it uses a Unix socket.
|
Want your agent to iterate on Greptile's feedback? Start a greploop in Cursor and it will work through the open comments and keep going until this PR reviews clean. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
3 participants
What
Adds a docker-free mode that runs the full agentex backend as host processes with embedded datastores — a lighter alternative to the Docker Compose stack, closer to a one-command
langgraph dev-style workflow.The bare
./dev.sh(Docker) is unchanged and now also accepts an explicit./dev.sh dockeralias. The docker-free mode is also available asmake dev-no-dockerandpython -m scripts.dev_nodocker.Why
Standing up a local environment previously required the full Docker stack. This lets a developer run the backend with a single command and no Docker daemon:
pgserver(unix socket) and Redis via bundledredislitemongod— always started; the stack requires it (the Temporal worker builds Mongo-backed repositories at boot)It runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on Ctrl-C / SIGTERM.
--ephemeraluses a throwaway data dir;--mongo-uripoints at an existing MongoDB instead of launching a localmongod.App-side change
Safe no-op wherever the backend runs in Docker / staging / prod (the env var it keys on is unset there):
ACP host rewrite for docker-free mode. Agents register their ACP URL at
host.docker.internal(the SDK default, so a Docker backend can reach an agent on the host), which a host-process backend can't resolve. The runner setsAGENTEX_ACP_HOST_OVERRIDE=127.0.0.1and the backend rewrites only that sentinel host to the override when dialing agents — in the ACP request path, the agent-API-key proxy path, and the Temporal healthcheck. When the env var is unset, the stored URL is used verbatim. Default-scaffolded agents work without manifest edits.Also
dev.sh(kill the wholemake → npm → nexttree and sweep orphans; report status by listening port).brew trustfirst) and OpenTelemetry (release binary; not in Homebrew) install commands.dev-no-dockeruv dependency group (pgserver,redislite,greenlet) pulled only for docker-free mode.README.mdandCLAUDE.md.Contract note
The runner is a package (
scripts/dev_nodocker/), so the direct invocation ispython -m scripts.dev_nodocker(notpython scripts/dev_nodocker.py)../dev.sh no-dockerandmake dev-no-dockerare unchanged for callers.Testing
Manually exercised on macOS (Docker stopped):
--lean— the API passes/healthzwithin ~1s and/readyzreports Postgres, Redis, and MongoDB all healthy; the Temporal worker stays up; teardown frees all ports with no orphaned processes.mongodabsent aborts with an actionable install message (or point at an external instance with--mongo-uri).python -m scripts.dev_nodocker; agents scaffolded byagentex initconnect without manifest edits (ACP host rewrite).Platform support: macOS and Linux only. Not supported on native Windows — the embedded Redis (
redislite) ships no Windows server build; run it under WSL2, whereredislite/pgserver/mongodbehave as on Linux (WSL2 path not yet verified).Greptile Summary
This PR introduces a docker-free local development mode (
./dev.sh no-docker) that provisions embedded datastores (bundled Postgres viapgserver, Redis viaredislite, auto-downloaded Temporal dev server, and a localmongod) and supervises the API server and worker as host processes — no Docker daemon required. It also adds an ACP host-rewrite utility (src/utils/acp_url.py) so that agents registered athost.docker.internalare dialed correctly when the backend itself runs on the host.scripts/dev_nodockerpackage: pure config module, service provisioners, subprocess supervisor, and orchestration runner. Previously flagged issues (crash returning exit code 0, empty-stringdeployment.acp_urlfalling through incorrectly) are both addressed in this revision.resolve_acp_url): applied consistently across all three ACP dial sites (ACP use case, API-key proxy, Temporal healthcheck activity). Safe no-op in Docker/staging/prod whenAGENTEX_ACP_HOST_OVERRIDEis unset.dev.shenhancements:no-dockersubcommand, mode-file tracking for stop/status/restart,kill_tree+sweep_stray_frontendsto fix the frontend process-leak bug, and a preflight port-conflict checker.Confidence Score: 5/5
agentex/src/utils/acp_url.py.Important Files Changed
host.docker.internalto a local override in no-docker mode. Logic is clean and safe; minor issue with credentials being dropped when rewriting the netloc.finallyblock.writer.wait_closed()usage.build_envproperly removes env vars for optional services that weren't started.resolve_configcovers ephemeral, custom data-dir, and default paths.deployment.acp_urlregression is correctly addressed — new code uses truthiness (if not raw) throughout, so an empty string from a deployment falls through toagent.acp_urlinstead of being dialed as"".acp_url_overridealso correctly passes throughresolve_acp_url.resolve_acp_urlto the agent-API-key proxy path so the host rewrite covers all three ACP dial sites.resolve_acp_urlto the Temporal healthcheck activity, completing the trio of ACP dial sites that need the host rewrite.no-dockersubcommand with mode-file tracking,preflight_port_check,kill_tree,sweep_stray_frontends, and proper stop/status awareness. OTel binary installer proceeds without verification if the checksums file download fails (previously flagged P2, not yet addressed).dev-no-dockertarget. Includes a TODO without a ticket number.Sequence Diagram
sequenceDiagram participant devsh as dev.sh no-docker participant runner as runner.run() participant pg as pgserver (embedded) participant redis as redislite (embedded) participant mongo as mongod (local) participant temporal as Temporal dev server participant api as uvicorn API participant worker as Temporal worker devsh->>runner: asyncio.run(run(cfg)) runner->>pg: provision_postgres(cfg) pg-->>runner: (server, database_url) runner->>redis: provision_redis(cfg) redis-->>runner: (server, redis_url) runner->>mongo: provision_mongo(cfg) mongo-->>runner: (proc, mongo_uri) runner->>temporal: provision_temporal(cfg) temporal-->>runner: (env, temporal_address) runner->>runner: build_env(...) runner->>runner: run_migrations(cfg, env) runner->>api: spawn("api", uvicorn ...) runner->>worker: spawn("worker", run_worker.py) runner->>runner: wait_for_health(/healthz) runner->>runner: asyncio.wait([stop, proc exits]) Note over runner: Ctrl-C / SIGTERM sets stop runner->>worker: terminate(SIGTERM → SIGKILL) runner->>api: terminate(SIGTERM → SIGKILL) runner->>temporal: env.shutdown() runner->>mongo: terminate(SIGTERM → SIGKILL) runner->>redis: teardown_redis() runner->>pg: pg_server.cleanup()Reviews (11): Last reviewed commit: "Merge branch 'main' into aringuyen/run-l..." | Re-trigger Greptile