You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Go SDK only exposes Filesystem.Read / Filesystem.ReadText, both of which materialize the entire file in memory. There is no streaming entry point at all.
// runtime/filesystem.go:78-110func (f*Filesystem) Read(ctx context.Context, pathstring, user...string) ([]byte, error) {
...returnio.ReadAll(resp.Body) // whole file into heap
}
// runtime/filesystem.go:114-120func (f*Filesystem) ReadText(ctx context.Context, pathstring, user...string) (string, error) {
data, err:=f.Read(ctx, path, user...) // and then copied again into a string...
}
Peak memory is therefore file size for Read, and roughly twice the file size for ReadText.
This is the Go counterpart of #41 (Java), which was addressed by #45.
Additional problem specific to Go: a hard cap on total read time
Unlike the Java SDK — which has a dedicated streaming client (RuntimeConfig.getOrCreateStreamingHttpClient(), readTimeout=0) — the Go SDK has a single shared HTTP client:
In Go, http.Client.Timeout is a total deadline that includes reading the response body. Consequences:
Even today, Read cannot fetch a file whose transfer takes longer than 60s, regardless of available memory.
A naive ReadStream that reuses this client would have its stream torn down at 60s — the same class of defect that was caught in the first review round of Implement InputStream-based file read API for large files #45, but harder to notice in Go because there is no ready-made streaming client to pick from.
Desired behavior
Add a streaming read that returns io.ReadCloser, and back it with a client that does not impose a total timeout, letting the caller's ctx govern cancellation (all Filesystem methods already take a ctx).
Scope: only one place to change
Unlike the Java SDK, which keeps two hand-maintained copies of Filesystem.java, the Go side has a single implementation. e2b.Sandbox embeds *runtime.Client:
So adding ReadStream to runtime/filesystem.go automatically makes it available to both the standalone runtime client and the e2b client (sb.Files.ReadStream(...)). No synchronization work is needed.
Suggested API
// ReadStream opens a stream to the file content. The caller must Close the// returned ReadCloser. Cancellation and deadlines are controlled via ctx.func (f*Filesystem) ReadStream(ctx context.Context, pathstring, user...string) (io.ReadCloser, error)
Return resp.Body directly. Go needs no equivalent of the Java FilterInputStream wrapper — closing the body already releases the connection.
Add a streaming client in Config with Timeout: 0, reusing the same Transport/connection pool. When a custom client was supplied via WithHTTPClient, shallow-copy it and zero out Timeout (mirroring Java's newBuilder().readTimeout(0)).
A ReadTextStream counterpart is not proposed: bufio.NewScanner(rc) already covers it, and adding it would be un-idiomatic in Go.
Read may share an internal helper with ReadStream to avoid duplicating URL/request construction, but it should keep using the existing finite-timeout client so its behavior does not change.
Documentation to update: the filesystem API tables in e2b/README.md:357, runtime/README.md:204, and their README_zh-CH.md counterparts.
Use case
Same as #41: sandbox workloads produce logs, dumps and other large artifacts. Fetching them through Read/ReadText is impractical both because of heap usage and because of the 60s total-request deadline.
Separate follow-up, out of scope here: the same 60s client is also shared with the two Connect RPC clients (runtime/client.go:39-40), and long-lived server streams (runtime/command_handle.go:97, runtime/commands.go:173) run over it. Command output streams lasting more than RequestTimeout are likely to be truncated for the same reason. This needs to be confirmed and tracked in its own issue.
Current behavior
The Go SDK only exposes
Filesystem.Read/Filesystem.ReadText, both of which materialize the entire file in memory. There is no streaming entry point at all.Peak memory is therefore file size for
Read, and roughly twice the file size forReadText.This is the Go counterpart of #41 (Java), which was addressed by #45.
Additional problem specific to Go: a hard cap on total read time
Unlike the Java SDK — which has a dedicated streaming client (
RuntimeConfig.getOrCreateStreamingHttpClient(),readTimeout=0) — the Go SDK has a single shared HTTP client:In Go,
http.Client.Timeoutis a total deadline that includes reading the response body. Consequences:Readcannot fetch a file whose transfer takes longer than 60s, regardless of available memory.ReadStreamthat reuses this client would have its stream torn down at 60s — the same class of defect that was caught in the first review round of Implement InputStream-based file read API for large files #45, but harder to notice in Go because there is no ready-made streaming client to pick from.Desired behavior
Add a streaming read that returns
io.ReadCloser, and back it with a client that does not impose a total timeout, letting the caller'sctxgovern cancellation (allFilesystemmethods already take actx).Scope: only one place to change
Unlike the Java SDK, which keeps two hand-maintained copies of
Filesystem.java, the Go side has a single implementation.e2b.Sandboxembeds*runtime.Client:So adding
ReadStreamtoruntime/filesystem.goautomatically makes it available to both the standaloneruntimeclient and thee2bclient (sb.Files.ReadStream(...)). No synchronization work is needed.Suggested API
Usage:
Implementation notes:
resp.Bodydirectly. Go needs no equivalent of the JavaFilterInputStreamwrapper — closing the body already releases the connection.ConfigwithTimeout: 0, reusing the sameTransport/connection pool. When a custom client was supplied viaWithHTTPClient, shallow-copy it and zero outTimeout(mirroring Java'snewBuilder().readTimeout(0)).ReadTextStreamcounterpart is not proposed:bufio.NewScanner(rc)already covers it, and adding it would be un-idiomatic in Go.Readmay share an internal helper withReadStreamto avoid duplicating URL/request construction, but it should keep using the existing finite-timeout client so its behavior does not change.e2b/README.md:357,runtime/README.md:204, and theirREADME_zh-CH.mdcounterparts.Use case
Same as #41: sandbox workloads produce logs, dumps and other large artifacts. Fetching them through
Read/ReadTextis impractical both because of heap usage and because of the 60s total-request deadline.Separate follow-up, out of scope here: the same 60s client is also shared with the two Connect RPC clients (
runtime/client.go:39-40), and long-lived server streams (runtime/command_handle.go:97,runtime/commands.go:173) run over it. Command output streams lasting more thanRequestTimeoutare likely to be truncated for the same reason. This needs to be confirmed and tracked in its own issue.