Add configurable backend timeouts

[AnthonyRonning]

Closes #33.

Summary:

• Add MAPLE_REQUEST_TIMEOUT_SECS and MAPLE_STREAM_IDLE_TIMEOUT_SECS config values, with 300s defaults.
• Wrap attestation setup, model listing, non-streaming completions, embeddings, and streaming response setup in request timeouts.
• Add a streaming idle timeout around backend SSE chunk reads so stalled streams emit an error event and terminate.
• Document the timeout variables across README, .env.example, Docker, Compose, and just helpers.

Note:

• The current opensecret SDK does not expose a reqwest ClientBuilder/custom client hook, so this applies the timeout at the proxy boundary rather than inside SDK construction.

Tests:

• cargo test
• cargo clippy --all-targets -- -D warnings

---

Summary by CodeRabbit

• New Features

  • Configurable request and streaming idle timeouts with sensible defaults; requests now time out with 504 and streaming emits an error event on idle timeout.

• Documentation

  • README and docs updated to document the new timeout environment variables and examples.

• Chores

  • Container, compose, and run targets updated to expose and pass the new timeout settings.

• Tests

  • Added tests for defaults, overrides, CLI validation, and stream idle behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: fcf7bbd3-68eb-4b4e-88fe-923a1883b58a

📥 Commits

Reviewing files that changed from the base of the PR and between 499129a and 82cfecb.

📒 Files selected for processing (1)

• CLAUDE.md

✅ Files skipped from review due to trivial changes (1)

• CLAUDE.md

---

📝 Walkthrough

Walkthrough

Adds configurable request and streaming idle timeouts (MAPLE_REQUEST_TIMEOUT_SECS, MAPLE_STREAM_IDLE_TIMEOUT_SECS), wires them into Config and client creation, applies tokio timeouts to backend calls and SSE idle waits, and updates Docker, compose, justfile, README, docs, and tests.

Changes

Configurable Request and Streaming Timeouts

Layer / File(s)	Summary
Timeout Configuration Contract src/config.rs, Cargo.toml	Module constants DEFAULT_REQUEST_TIMEOUT_SECS and DEFAULT_STREAM_IDLE_TIMEOUT_SECS are defined; struct fields request_timeout_secs and stream_idle_timeout_secs added with clap env var binding and positive-value validation; with_request_timeout_secs() and with_stream_idle_timeout_secs() builder methods; request_timeout() and stream_idle_timeout() return Duration. tokio dependency gains time feature. Tests added.
Client Initialization with Timeouts src/proxy.rs	Adds ProxyError alias; create_client_with_auth accepts request_timeout and wraps the attestation handshake in tokio::time::timeout; introduces timeout_response to convert handshake/request timeouts into 504 GATEWAY_TIMEOUT JSON OpenAIError.
Request Endpoint Timeout Wrapping src/proxy.rs	list_models, create_chat_completion (streaming and non-streaming), and create_embeddings wrap backend client calls in tokio::time::timeout and return timeout_response on timeout. Test config updated to include timeout fields.
Streaming Idle Timeout src/proxy.rs	SSE stream creation accepts stream_idle_timeout and wraps stream.next() in tokio::time::timeout; on idle timeout logs and emits an SSE error event and terminates without emitting [DONE]. New Tokio test validates the idle-timeout error emission.
Deployment and Documentation .env.example, Dockerfile, docker-compose.yml, justfile, README.md, CLAUDE.md, tests/health_test.rs	Environment variables documented and added to examples; Dockerfile and docker-compose set runtime ENV defaults; justfile prints and forwards variables; health tests now construct Config via Config::new(...).

Sequence Diagram(s)

sequenceDiagram
  participant ClientRequest as Client Request
  participant Proxy as Maple Proxy
  participant ClientCache as Client Cache
  participant Creator as create_client_with_auth
  participant Backend as OpenSecret backend

  ClientRequest->>Proxy: incoming request (api_key)
  Proxy->>ClientCache: lookup_or_create_client(api_key, request_timeout)
  ClientCache->>Creator: create_client_with_auth(request_timeout)
  Creator->>Backend: attestation handshake (timed)
  Backend-->>Creator: attestation success or error
  Creator-->>ClientCache: client or timeout_response
  ClientCache-->>Proxy: client or timeout_response
  Proxy->>Backend: backend call (timed)
  Backend-->>Proxy: response or timeout

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• OpenSecretCloud/maple-proxy#34: Modifies client creation/caching per API key; relates to client initialization changes in this PR.

Poem

🐰 I nibbled clocks into the code with care,
Timeouts now guard each roaming snare.
Streams that stall will send a cry,
No more hangs — we let them fly.
Hop, test, and deploy with a happy air.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding configurable backend timeouts, which aligns with the primary objective of the PR.
Linked Issues check	✅ Passed	The PR successfully implements all requirements from issue #33: configurable request timeout (MAPLE_REQUEST_TIMEOUT_SECS), streaming idle timeout (MAPLE_STREAM_IDLE_TIMEOUT_SECS), environment variable configuration, documentation, and timeout wrapping around relevant backend calls.
Out of Scope Changes check	✅ Passed	All changes are directly related to implementing configurable timeouts: configuration files, timeout logic in proxy handlers, builder methods, documentation, and test updates. No unrelated changes detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/configurable-timeouts

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

[devin-ai-integration]

Devin Review found 1 new potential issue.

⚠️ 1 issue in files not directly in the diff

⚠️ CLAUDE.md Configuration section not updated with new environment variables (CLAUDE.md:76-82)

The PR adds two new environment variables (MAPLE_REQUEST_TIMEOUT_SECS and MAPLE_STREAM_IDLE_TIMEOUT_SECS) in src/config.rs, .env.example, README.md, Dockerfile, docker-compose.yml, and justfile, but does not update the Configuration section of CLAUDE.md (lines 76-82) which serves as the canonical guidance file for this repository. The CLAUDE.md explicitly lists all available environment variables, and this list is now incomplete.

View 4 additional findings in Devin Review.