feat(envd): tell reverse proxies not to buffer streaming responses - #3669
Open
eastagiletracker wants to merge 1 commit into
Open
eastagiletracker wants to merge 1 commit into
eastagiletracker wants to merge 1 commit into
Conversation
Server-streaming RPCs (Process.Start, Process.Connect, Filesystem.WatchDir) now respond with X-Accel-Buffering: no, so a proxy with response buffering on (the nginx default) forwards process output and watch events as they are written instead of holding them until its buffer fills. Unary and client-streaming RPCs are unchanged.
eastagiletracker
requested review from
ValentaTomas,
dobrac and
jakubno
as code owners
September 29, 2026 16:33
|
We require contributors to sign our Contributor License Agreement, and we don't have @eastagiletracker on file. You can sign our CLA at https://e2b.dev/docs/cla . Once you've signed, post a comment here that says '@cla-bot check' |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR proposes marking envd's server-streaming responses with
X-Accel-Buffering: noso reverse proxies stop holding process output and watch events (Fixes #2707). We include this PR work along with a full history of your repo at https://eastagiletracker.com/projects/751. You can sign in with your GitHub ID to claim ownership of the project.Process.Start,Process.ConnectandFilesystem.WatchDirare server-streaming, but nothing in their responses tells a reverse proxy that. Behind a stock nginx (proxy_buffering on), stream chunks sit in the proxy buffer until it fills, so an interactive PTY looks like it accepts input and prints nothing. On currentmain(85d0f38), mounting the realfilesystem.Handleon a chi mux and opening aWatchDirstream on a temp dir gets a livestartevent with onlyConnect-Accept-Encoding,Connect-Content-Encoding,Content-TypeandDatein the response headers, andX-Accel-Buffering="".The change adds a small connect interceptor in
packages/envd/internal/services/streamingthat setsX-Accel-Buffering: noon responses whoseSpec().StreamTypeincludes server streaming. It is wired into the existingconnect.WithInterceptors(...)chain infilesystem.Handleandprocess.Handle. Unary and client-streaming RPCs (ListDir,List,StreamInput, ...) pass through untouched, so they keep whatever buffering or caching the proxy applies today. The header is set before the handler runs, so it is also present when a stream ends in an error. Nothing is removed or renamed, and clients that ignore the header see no difference.Tests:
streaming/interceptor_test.gocovers a server stream (with a message and ending in an error),Process.Start, a unary call and a client stream.filesystem/handle_test.goandprocess/handle_test.gogo through the realHandlewiring. With the twoservice.goedits reverted,go test -run TestHandleDisablesProxyBuffering ./internal/services/filesystem/ ./internal/services/process/fails withexpected: []string{"no"} actual: []string(nil), and it passes with them.go test ./...inpackages/envdshows the same set of failures before and after the change. All of those failures are environment-dependent (cgroup, full-disk and network-mount tests on this machine), and none are in the packages touched here.golangci-lintv2.13.2 (from.tool-versions) reports 0 issues on the changed packages.How this was managed
This work was tracked as https://eastagiletracker.com/projects/751/stories/809774 on https://eastagiletracker.com/projects/751, a board imported from this repo's issues and pull requests (3,652 stories) and used to manage this change.
If you'd rather not receive contributions like this, reply
no-more-prson this pull request and we won't open any further ones on your repositories.Lawrence W. Sinclair
CEO / East Agile
linkedin.com/in/lwsinclair/
eastagile.com