v0.3.0: Cloud native integration and S3-compatible persistence

[yifanxuaaa]

Goal

Deliver v0.3.0 cloud native integration so multiple machines can build Layers, Branches, and private Workspaces against shared LayerFS history, preserving CAS/CDC/COW identities and the v0.2 automatic Branch reconciliation contract.

Roadmap

Canonical planning document: docs/roadmap/0.3/README.md (initialized locally alongside this issue; pending commit/push).

Scope and acceptance checklist

• Select the smallest viable topology: evaluate an authoritative SQLite service with Litestream replication against immutable object storage with transactional metadata. Record the choice and evidence; both implementations are not required.
• Define remote SDK/CLI operations, workspace ownership, authentication, tenant access, cancellation, reconnect, and resource limits.
• Implement bounded local write buffers, background batched admission, safe duplicate uploads, verified local caching, and bounded missing-object fetches.
• Define local acceptance, local persistence, shared publication, remote replication, and fsync/Commit acknowledgements explicitly.
• Publish complete roots atomically with conditional Branch/LayerStack advancement; preserve v0.2 automatic clean reconciliation, structured conflicts, and exact-candidate validation.
• Prove idempotent publication after lost replies, protection against garbage-collection races, supported partition behavior, and fencing if failover is supported.
• Evaluate Litestream compatibility: current Store uses MEMORY journaling, synchronous OFF, EXCLUSIVE locking, and rejects WAL. Specify migration, locking/checkpoint changes, and durable synchronization; benchmark the changed contract.
• Pin dependencies and provide reproducible local S3-compatible tests without AWS, using an isolated endpoint such as MinIO.
• Exercise two isolated workers, identical/distinct chunk uploads, independent and shared Branches, competing Add, endpoint outage/catch-up, and interrupted uploads.
• Restore into an empty environment without original DB/WAL/cache access; verify roots, IDs, history, bytes, and integrity after writer loss before/after replicated acknowledgement.
• Measure cold/warm reads, concurrent startup, Commit backlog, p50/p95/p99, request count, transferred/replicated bytes, lag, CPU/RSS, and storage/request costs.
• Qualify the selected path on AWS with recorded topology, IAM, versions, acknowledgement boundaries, and measured behavior; local emulation does not establish AWS performance/durability.
• Freeze public contracts, migrations, recovery instructions, applicable regressions, source identity, and release evidence together.

Completion proof

Two machines build and publish shared history through public operations. An acknowledged remotely durable result remains readable after its originating worker disappears. Concurrent operations never silently overwrite accepted history or expose incomplete roots. Recovery, resource bounds, and performance claims are backed by retained evidence.

Boundaries

Litestream is an evaluation candidate, not a preselected backend. It replicates SQLite pages and does not merge independently writable Store copies; its optional VFS write mode assumes one writer. Do not replace v0.2 concurrent-Workspace reconciliation with one Workspace per Branch or manual stale-head rebasing. Active-active multi-region metadata, general CRDT semantics, and speculative sharding/packing are outside the initial scope.

References

• https://litestream.io/how-it-works/
• https://litestream.io/guides/vfs-write-mode/
• https://litestream.io/reference/sync/
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-guidelines.html