diff --git a/rfcs/0012-gateway-independent-core.md b/rfcs/0012-gateway-independent-core.md new file mode 100644 index 00000000..6b25aae4 --- /dev/null +++ b/rfcs/0012-gateway-independent-core.md @@ -0,0 +1,361 @@ +--- +title: Gateway-Independent OpenClaw Core +authors: + - Jason (Json) +created: 2026-06-15 +last_updated: 2026-06-15 +status: draft +issue: +rfc_pr: https://github.com/openclaw/rfcs/pull/17 +--- + +# Proposal: Gateway-Independent OpenClaw Core + +## Summary + +OpenClaw should make its core agent runtime independent of the Gateway lifecycle while preserving the existing `openclaw gateway` command, `gateway.*` configuration, port, authentication, Control UI, TUI, protocol, and plugin behavior. A Host-owned core would run conversations, sessions, cron, heartbeat, tasks, routing policy, and durable delivery intent without requiring any channel connection. The existing Gateway would become a compatibility-supervised composition of a Control Server protocol facade and transport-only Channel Gateways. The migration would be phased, additive at public boundaries, and invisible to existing users and external plugin developers. + +## Motivation + +The current Gateway is more than a channel gateway. It is the composition root and lifecycle owner for channel connections, the control protocol, agents, cron, heartbeat, tasks, plugin services, hooks, and local clients. This makes the process convenient to start, but it also makes unrelated systems share the same failure and restart boundary. + +A channel connection failure, a channel-specific configuration change, a Control Server restart, or a Gateway deployment can interrupt work that does not depend on that surface. Conversely, local agent conversations, cron jobs, and heartbeat work cannot operate as a complete product mode unless the Gateway composition remains available. + +This couples the product's core identity to its transport layer. OpenClaw should be an agent runtime with channel integrations, not a channel relay that happens to run agents. + +### Why detachment matters + +Detaching the Gateway from core functionality provides concrete product benefits: + +- **Core work survives channel outages.** Active agent turns, cron jobs, heartbeat work, and local conversations continue when a connector disconnects or restarts. +- **Local-first becomes a real runtime mode.** The TUI, Control UI, CLI, and other local clients can use the Host even when no messaging channels are configured. +- **Failure blast radius becomes smaller.** A Telegram, Discord, or other connector fault does not require restarting conversations, schedules, tasks, or unrelated connectors. +- **Configuration changes become owner-scoped.** A channel token change can restart only that connector; a Control Server TLS change can restart only the Control Server. +- **Security boundaries become clearer.** Channel credentials and untrusted transport payloads can remain at a narrower transport boundary. +- **Plugin contracts become clearer.** Channel plugins can target a serializable connector contract instead of depending on broad in-process Gateway state. +- **Deployment options improve.** OpenClaw can later separate processes or hosts without first encoding the current monolith as a distributed monolith. + +### Configuration restart behavior exposes the current coupling + +OpenClaw already validates configuration, computes changed paths, and hot-applies many changes. The reload planner distinguishes no-op, hot-reload, and restart-required changes. Channel plugins can declare config prefixes, and plugins can declare hot, no-op, and restart prefixes. + +The important limitation is not that OpenClaw lacks reload logic. It is that the fallback restart targets the process that owns everything. When a changed path requires `restartGateway: true`, the Gateway shutdown path stops plugin services, channels, agent harnesses, cron, heartbeat, tasks, and control connections together. + +The proposed architecture does not promise that every change becomes hot-reloadable. It changes the restart boundary: + +- today, a restart-required change restarts the shared Gateway composition; +- after this RFC, a restart-required change restarts the owner of the affected configuration; +- an explicit full service restart remains available and keeps its existing meaning. + +This is one of the clearest user-visible reasons to separate ownership even before OpenClaw uses separate operating-system processes. + +### Current-state evidence + +The source survey for this proposal used `openclaw/openclaw` `main` at commit `127e174c9e4de36f2eccb96300bb3efa69ae9a32`. + +| Current surface | Relevant source | Observed ownership | +| --- | --- | --- | +| Gateway composition and shutdown | `src/gateway/server.impl.ts`, `src/gateway/server-close.ts` | One lifecycle owns core systems, channels, plugins, and control connections. | +| Config reload planning and application | `src/gateway/config-reload.ts`, `src/gateway/config-reload-plan.ts`, `src/gateway/server-reload-handlers.ts` | Most changes are diffed and classified, but the broad fallback is a Gateway restart. | +| Cron | `src/cron/service-contract.ts`, `src/cron/service.ts`, `src/gateway/server-cron.ts` | The scheduler has a service boundary but is started and stopped by the Gateway. | +| Heartbeat | `src/infra/heartbeat-runner.ts` | Heartbeat has lifecycle structure but still reaches channel delivery concerns. | +| Local TUI | `src/tui/tui-backend.ts`, `src/tui/embedded-backend.ts` | Local execution exists, but core paths and Gateway-facing paths are not one canonical Host service surface. | +| Agent tools | `src/agents/openclaw-tools.ts`, `src/agents/tools/embedded-gateway-stub.ts` | Internal core behavior can route back through Gateway-shaped RPC and stubs. | +| Control UI | `ui/src/ui/gateway.ts`, `ui/src/ui/app-gateway.ts`, `ui/src/ui/controllers/cron.ts` | The UI treats the Gateway protocol as the owner of core and transport concerns. | +| Channel and plugin contracts | `src/gateway/server-channels.ts`, `src/channels/plugins/types.adapters.ts`, `src/plugins/types.ts` | Channel plugins have broad in-process access and lifecycle coupling. | + +These are useful existing seams. The proposal builds on them rather than replacing the product in one step. + +## Goals + +- Run conversations, sessions, cron, heartbeat, tasks, routing policy, and durable delivery intent when zero channels are configured or connected. +- Keep those core systems running through Channel Gateway and Control Server restarts. +- Preserve the public `openclaw gateway` command, `gateway.*` configuration, default port, authentication, protocol methods, subscriptions, Control UI, TUI, CLI, nodes, and hooks. +- Keep existing `openclaw.json` files valid without required new keys, environment variables, or normal-upgrade doctor migrations. +- Keep shipped external plugins working without changes through an automatic co-located compatibility mode. +- Add an additive connector contract for transport-only channel implementations. +- Establish one canonical core execution path and one canonical writer for durable core state. +- Make configuration validation, ownership, desired revision, active revision, and restart scope explicit. +- Make the logical ownership split before making a physical process split. +- Roll out each phase behind proof gates with complete backward compatibility. + +## Non-Goals + +- This RFC does not rename the public Gateway command, configuration namespace, protocol, or product surfaces. +- This RFC does not require users to manage additional processes, ports, sockets, or IPC settings. +- This RFC does not force external plugin developers to migrate immediately. +- This RFC does not deprecate the existing channel plugin contract in the initial implementation. +- This RFC does not make remote or multi-host Channel Gateways a first-phase feature. +- This RFC does not migrate all channels at once. +- This RFC does not add dual core execution paths, dual durable-state writers, or steady-state runtime fallbacks. +- This RFC does not promise that configuration changes never require a restart. +- This RFC does not promise instantaneous atomic configuration activation across independently restarting owners. + +## Proposal + +### 1. Establish four explicit runtime owners + +The public `openclaw gateway` service remains the user-facing entry point. Internally, it supervises four logical owners: + +```text + existing clients and plugins + TUI / Control UI / CLI / nodes / hooks + | + v + +-------------------------+ + | Control Server | + | existing Gateway facade | + +------------+------------+ + | + v ++----------------------+ +---------+---------+ +----------------------+ +| Compatibility |--->| Host |<-->| Channel Gateway(s) | +| Supervisor | | canonical core | | transport-only | +| public service owner | | and durable state | | connectors | ++----------------------+ +-------------------+ +----------------------+ +``` + +#### Compatibility Supervisor + +The Compatibility Supervisor owns the public service lifecycle: + +- starts the combined service for existing users; +- supervises logical owners and, later, child processes; +- exposes combined health and diagnostics; +- places legacy plugins in the correct co-located compatibility runtime; +- performs explicit full service restarts; +- does not own durable business state. + +#### Host + +The Host is the canonical owner of: + +- agent conversations and turns; +- session and transcript coordination; +- cron and scheduled work; +- heartbeat scheduling and work generation; +- tasks and background work; +- routing and product policy; +- durable delivery intent; +- core plugin services; +- canonical core state; +- configuration validation and revision coordination. + +The Host must start and operate when no channels or Control Server are available. + +#### Control Server + +The Control Server is the compatibility facade for the existing Gateway protocol. It owns: + +- current protocol authentication and authorization; +- current methods, events, and subscriptions; +- Control UI, TUI, CLI, node, and hook connections; +- Gateway HTTP, WebSocket, TLS, bind, and port configuration; +- projection of Host and connector health into existing protocol views. + +The Control Server does not own core business state. It calls Host service APIs and projects their results. + +#### Channel Gateways + +A Channel Gateway owns transport-specific concerns: + +- connect and authenticate to the channel; +- receive and acknowledge native events; +- normalize inbound events into portable ingress; +- render and send portable delivery requests; +- report receipts, typing, delivery state, and transport health; +- enforce channel-specific limits. + +A Channel Gateway does not own conversations, sessions, schedules, heartbeat policy, product commands, provider policy, or durable delivery intent. + +### 2. Enforce ownership invariants before splitting processes + +The implementation must enforce these invariants while all owners can still run in one process: + +1. The Host starts and passes its core proof suite with zero Channel Gateways. +2. Core services call Host-owned interfaces directly. They do not call the Control Server or an embedded Gateway stub. +3. The Control Server is a facade over Host APIs, not a second implementation of core behavior. +4. Channel Gateways receive portable ingress and delivery contracts. They do not reach into Host state. +5. Only the Host writes canonical core state. +6. Connector restarts cannot cancel unrelated active turns, cron work, heartbeat work, or tasks. +7. Control Server restarts cannot cancel Host-owned work. +8. Every configuration path has one declared owner before detached mode becomes the default. + +This logical split is the architecture change. Moving owners into separate processes is a later deployment step and proof mechanism. + +### 3. Add Host service APIs + +The Host exposes narrow internal service APIs for: + +- conversation start, continue, cancel, and status; +- session lookup, history, and lifecycle operations; +- schedules and cron execution; +- heartbeat scheduling and execution; +- task creation, status, cancellation, and completion; +- durable delivery intent and result reconciliation; +- health, capability, and configuration revision state. + +The APIs should use serializable values and closed result types even while calls are in-process. That prevents functions, callbacks, process globals, and transport objects from becoming accidental contracts. + +Local TUI and embedded use should call these Host APIs directly. The Control Server should call the same APIs. There must not be a separate local implementation and remote implementation of core behavior. + +### 4. Make delivery an explicit Host-to-connector contract + +The Host owns durable delivery intent. Channel Gateways own the attempt to render and deliver that intent. + +The additive connector contract should include serializable messages for: + +- inbound portable events; +- delivery requests; +- delivery accepted/rejected results; +- delivery receipts and final state; +- typing and presence requests; +- connector capability snapshots; +- connector and account health; +- configuration revision activation. + +Delivery identifiers and receipt reconciliation must prevent duplicate visible delivery when a connector restarts or retries. The connector contract must distinguish acceptance, upstream acknowledgement, and final delivery state where the transport supports those concepts. + +The Host may continue core work while a connector is unavailable. Delivery intent can remain pending, fail according to product policy, or be retried according to an explicit delivery policy. A connector outage must not implicitly become a Host outage. + +### 5. Make configuration lifecycle owner-scoped + +The Host owns a Config Coordinator that validates the complete configuration and coordinates secret-reference resolution once for the revision. It produces owner-scoped activation payloads so an owner receives only the configuration and secret material it needs; the Host does not need to retain connector plaintext after coordination. + +For each accepted configuration revision, the coordinator: + +1. validates the complete candidate configuration; +2. computes changed source paths; +3. maps every changed path to exactly one owner or an explicitly combined action; +4. asks affected owners to prepare the revision; +5. persists the desired revision; +6. applies hot changes or restarts only affected owners; +7. records the active revision for each owner; +8. exposes convergence, failure, retry, and rollback state. + +The system provides revisioned convergence rather than pretending that independently restarting owners activate a revision at one instant. + +Representative ownership: + +| Changed configuration | Owner and action | +| --- | --- | +| message or routing policy | Apply a new Host snapshot. | +| cron configuration | Reconfigure or restart only the Host scheduler subsystem. | +| `channels.telegram.*` | Reconfigure or restart only the affected Telegram connector or account. | +| another connector's account settings | Reconfigure or restart only that connector or account. | +| `gateway.port`, bind, TLS, or HTTP settings | Restart or reconfigure only the Control Server. | +| Host-owned plugin configuration | Reconfigure or restart the Host-owned plugin service. | +| explicit full restart request | Compatibility Supervisor restarts the complete service. | + +Unknown or ambiguous path ownership fails closed. During migration, it may require a combined service restart. Detached mode must not become the default until the path has owner metadata and proof. + +Existing reload semantics remain compatible: + +- `gateway.reload.mode: hybrid` gains owner-scoped actions where ownership is known; +- `gateway.reload.mode: hot` and `gateway.reload.mode: off` keep their shipped meanings; +- an explicit restart remains a full compatibility-supervised service restart; +- existing legacy plugin restart prefixes keep a safe co-located restart meaning; +- modern Host and connector contracts can declare narrower reload capabilities additively. + +### 6. Preserve all public user contracts + +The migration is intentionally invisible to existing users: + +- `openclaw gateway` remains the normal command. +- Existing service management continues to manage one OpenClaw service. +- Existing `gateway.*` configuration remains valid. +- Existing default port, authentication, Control UI URL, protocol, methods, events, and subscriptions remain valid. +- Existing TUI and CLI workflows remain valid. +- Existing configuration needs no manual edits. +- A normal upgrade does not require `openclaw doctor --fix` solely because of this architecture change. +- Existing monitoring can continue to observe the combined service, with additive owner-level health available for diagnosis. + +New internal owner configuration should be derived from existing configuration and defaults. User-facing configuration is added only if a later product requirement cannot be expressed through existing behavior. + +### 7. Preserve external plugin behavior through compatibility placement + +External plugin compatibility is a release gate, not a best-effort follow-up. + +Existing channel plugins use a broad in-process contract that can include functions, callbacks, runtime objects, and Gateway lifecycle assumptions. Those plugins cannot be moved across a process boundary without changing their contract. The Compatibility Supervisor therefore classifies them as legacy connector v1 plugins and runs them unchanged in a co-located compatibility runtime. + +Legacy compatibility rules: + +- existing plugin packages, manifests, config, and SDK imports continue to work; +- existing lifecycle and restart-prefix behavior remains available; +- no new permission or configuration is required; +- the plugin remains co-located with the compatibility service until it opts into a serializable connector contract; +- OpenClaw-owned and bundled callers migrate to the modern contract as it becomes available; +- no initial deprecation date is assigned to the legacy contract. + +Connector v2 is additive. It defines the serializable ingress, delivery, receipt, health, capability, lifecycle, and configuration-revision contract required for detached placement. Plugin authors can opt in when the capability is useful to them. + +Non-channel plugins remain Host-owned unless their contract explicitly belongs to another owner. + +### 8. Roll out in proof-gated phases + +The implementation proceeds in phases: + +1. **Contract and guardrails:** document owners, invariants, compatibility rules, dependency boundaries, and configuration ownership. +2. **Host kernel:** introduce the Host lifecycle and Config Coordinator while the service remains combined; move cron and heartbeat ownership. +3. **Canonical core service APIs:** move conversations, sessions, schedules, tasks, delivery intent, and local TUI execution behind Host APIs; remove internal Gateway RPC and embedded Gateway stubs. +4. **Control Server facade:** make the current protocol a facade over Host APIs and prove Control Server/config restarts do not interrupt Host work. +5. **Connector v2 and compatibility placement:** add the serializable connector contract, keep legacy v1 plugins co-located, and migrate bundled connectors incrementally. +6. **Supervised physical split:** add private IPC, leases, restart budgets, revision convergence, and separate processes only after logical boundaries pass fault-injection proof. + +The detailed dependency-oriented work packages, compatibility gates, and proof matrix are in [0012/implementation-plan.md](0012/implementation-plan.md). + +### 9. Gate default changes on real upgrade and fault proof + +No phase becomes the default solely because its unit tests pass. Each default change requires: + +- upgrade proof from the last stable published OpenClaw version with unchanged configuration; +- unchanged legacy external plugin fixture proof; +- zero-channel Host proof; +- active-turn survival through connector and Control Server restarts; +- owner-scoped configuration restart proof; +- duplicate-delivery and lease/fencing proof; +- current protocol compatibility proof; +- rollback proof to the prior combined placement. + +The combined compatibility placement remains the fallback deployment mode during rollout, but core behavior must still have one canonical Host implementation. Rollback changes placement, not business logic. + +## Rationale + +### Why this is the right product direction + +Users care that their conversations, schedules, tasks, and local workflows continue to work. They should not need to understand which internal process owns them. Separating the core runtime from channel connectivity makes OpenClaw more dependable without adding setup burden. + +The proposal also aligns product boundaries with operator expectations. A channel token rotation should affect that channel. A Control Server certificate change should affect the Control Server. A disconnected channel should not stop a local conversation or cron job. The new ownership model makes those expectations enforceable. + +### Why not a one-shot rewrite + +A one-shot rewrite would combine lifecycle changes, core service APIs, protocol projection, plugin SDK compatibility, channel migration, state ownership, configuration reload, and deployment changes in one release. That creates too many simultaneous failure modes and makes rollback ambiguous. + +The phased plan keeps the public product stable while moving one owner boundary at a time. Every slice must leave one canonical core path, preserve compatibility, and provide a rollback placement. + +### Why not keep the monolith and improve hot reload + +Improved hot reload is useful, but it does not isolate channel faults, control-plane restarts, shutdown ordering, resource leaks, or deployment changes. It also cannot make zero-channel Host operation a first-class mode while the Gateway remains the core owner. + +### Why not split processes first + +Splitting the current composition first would turn existing in-process coupling into IPC calls and create a distributed monolith. Logical ownership, serializable contracts, and one-writer state rules must be proven before process boundaries make failures slower and harder to diagnose. + +### Why not make remote Channel Gateways the first target + +Remote connectors introduce public networking, trust, authentication, versioning, support, and deployment contracts that are not required to obtain the main resilience benefits. Private supervised placement should prove the architecture first. Remote placement can be a separate product proposal if demand justifies it. + +### Why compatibility belongs at the public edge + +Complete backward compatibility does not require keeping the current internal architecture. It requires preserving shipped user and plugin contracts. The proposal keeps compatibility in the public service, protocol facade, configuration interpretation, and legacy plugin placement while allowing core internals to converge on one Host-owned design. + +## Unresolved questions + +- Which bundled channel should be the first connector v2 migration and proof case? +- Which private IPC transport best fits supervised local processes? +- What exact SDK capability names should declare connector v2 support and owner-scoped reload behavior? +- Which Host plugin services, if any, should later receive their own owner boundary? +- What evidence would justify offering remote Channel Gateways as a supported product mode? +- Should legacy co-located channel plugins ever receive a deprecation timeline? This RFC intentionally does not set one. +- Which owner-level health details should be projected through the existing Gateway protocol before any additive protocol methods are introduced? +- The RFC lifecycle requires a matching discussion thread in `maintainer-discussion`. diff --git a/rfcs/0012/implementation-plan.md b/rfcs/0012/implementation-plan.md new file mode 100644 index 00000000..9a47b1f3 --- /dev/null +++ b/rfcs/0012/implementation-plan.md @@ -0,0 +1,558 @@ +# Implementation Plan + +This sidecar is the dependency-oriented implementation and rollout plan for the Gateway-Independent OpenClaw Core RFC. It is intentionally more detailed than the RFC body so implementation PRs can remain focused while reviewers retain a shared target architecture, compatibility contract, and proof standard. + +## Strategy + +The implementation should move logical ownership before physical placement: + +1. Define the owner and contract. +2. Route all OpenClaw-owned callers through one canonical path. +3. Prove the owner can start, stop, reload, and fail independently while still in one process. +4. Add serializable boundaries and compatibility placement. +5. Split the owner into a supervised process only after fault-injection proof. + +Each implementation PR must preserve the public service and leave the tree in a releasable state. A phase is complete only when the old internal path is removed or explicitly retained as a shipped compatibility contract. + +### Hard implementation rules + +- The Host is the only canonical implementation of core behavior. +- The Host is the only writer of canonical core state. +- The Control Server and Channel Gateways are clients of Host services. +- Existing user configuration remains the source of truth. +- Public compatibility is automatic; users do not select a migration mode. +- Legacy external plugins remain unchanged in co-located compatibility placement. +- Connector v2 is additive and serializable. +- Unknown configuration ownership fails closed. +- Rollback changes owner placement, not core behavior. +- Process splitting is the last architectural step, not the first. +- No phase adds dual writes, read-through fallbacks, or a second core execution path. + +## Target Owner Matrix + +This matrix is the intended ownership result. Early phases can remain co-located, but they must converge toward these boundaries. + +| Capability | Canonical owner | Other owners' role | +| --- | --- | --- | +| Agent conversations and turns | Host | Control Server invokes and projects; connectors deliver ingress/results. | +| Sessions and transcript coordination | Host | Control Server exposes existing protocol views. | +| Cron and schedules | Host | Control Server exposes existing methods/events. | +| Heartbeat scheduling and work | Host | Connectors receive delivery requests only. | +| Tasks and background work | Host | Control Server exposes status and cancellation. | +| Routing and product policy | Host | Connectors report capabilities; they do not decide policy. | +| Durable delivery intent | Host | Connectors attempt delivery and report receipts. | +| Core plugin services | Host | Compatibility Supervisor places legacy services when required. | +| Complete config validation | Host Config Coordinator | Owners prepare and activate their slices. | +| Desired configuration revision | Host Config Coordinator | All owners report active revision. | +| Gateway protocol and auth | Control Server | Host supplies service results and events. | +| HTTP, WebSocket, TLS, bind, port | Control Server | Compatibility Supervisor observes health. | +| Channel authentication and connection | Channel Gateway | Host never handles native channel connection state. | +| Native event parsing and acknowledgement | Channel Gateway | Host receives portable ingress. | +| Native rendering and transport limits | Channel Gateway | Host sends portable delivery intent. | +| Combined service lifecycle | Compatibility Supervisor | Owners expose health, readiness, and shutdown. | +| Legacy plugin placement | Compatibility Supervisor | Legacy plugin contract remains co-located. | +| Canonical durable core state | Host | No other owner writes it. | + +## Configuration Ownership and Revision Contract + +Configuration reload is a first-class architecture proof because it exercises validation, ownership, lifecycle, compatibility, and observability together. + +### Coordinator algorithm + +For every candidate revision: + +1. Parse and validate the complete configuration using the current public schema. +2. Coordinate secret-reference resolution according to current behavior and produce owner-scoped activation payloads without requiring the Host to retain connector plaintext. +3. Compute changed source paths against the active desired revision. +4. Resolve every path through an owner registry. +5. Build one closed apply plan containing owner actions: + - `none` + - `hot-apply` + - `restart-subsystem` + - `restart-owner` + - `restart-service` +6. Ask every affected owner to prepare the candidate revision. +7. Reject the revision if preparation fails or path ownership is ambiguous. +8. Persist the desired revision and plan. +9. Execute owner actions in dependency order. +10. Record each owner's active revision and result. +11. Expose pending, converged, failed, retrying, and rolled-back states. + +The first implementation can run this algorithm in one process. The contract must not depend on that placement. + +### Ownership metadata + +Existing reload prefixes are migration input, not the final owner model. Add explicit owner metadata that maps current config-schema paths to a logical owner and supported apply actions. + +Owner metadata must: + +- cover every current configuration path before detached placement becomes default; +- compose with plugin-declared configuration prefixes; +- reject overlapping or ambiguous ownership; +- preserve existing `gateway.reload.mode` behavior; +- preserve existing legacy plugin restart prefixes in co-located mode; +- identify whether an action targets an account, connector, subsystem, owner, or whole service; +- be visible in diagnostics and testable without starting every integration. + +### Revision and rollback semantics + +- The desired revision is the complete accepted configuration, not a set of partial owner files. +- Each owner reports the desired revision it has prepared and the revision it has activated. +- A failed owner activation is visible; it is not silently treated as success. +- Retry and rollback policy is explicit per action. +- Rollback can restore the prior complete desired revision or return the service to combined placement. +- A crash during activation must not create a hidden split-brain state. +- The compatibility service reports healthy only according to a documented convergence policy. + +### Required config proof cases + +| Change | Expected proof | +| --- | --- | +| `messages.*` or routing policy | Host applies the new snapshot; connectors and Control Server continue. | +| cron settings | Host scheduler reconfigures or restarts; active conversations and connectors continue. | +| one Telegram account token | Only that account or Telegram connector restarts; Host work and other connectors continue. | +| one connector's transport settings | Only the affected connector restarts. | +| `gateway.port`, bind, TLS, or HTTP | Only the Control Server restarts; Host work and connectors continue. | +| Host-owned plugin settings | Only the plugin service or Host subsystem restarts. | +| mixed Host and connector change | Desired and active owner revisions are visible until convergence. | +| unknown path during migration | Combined restart or rejected revision according to the migration policy. | +| explicit full restart | Compatibility Supervisor restarts the complete service. | + +## Compatibility Strategy + +### Existing users + +The normal upgrade path must require no action: + +- same command; +- same config file and keys; +- same service manager entry; +- same default port and authentication; +- same UI, TUI, CLI, node, and hook workflows; +- same protocol methods and event shapes; +- no mandatory doctor migration; +- no operator-managed child process or IPC configuration. + +Owner-level diagnostics and health can be additive, but current combined health remains available. + +### External plugin developers + +External plugin developers should experience no break during rollout. + +| Plugin type | Initial behavior | Modern opt-in path | +| --- | --- | --- | +| Existing channel plugin | Runs unchanged in co-located legacy connector v1 placement. | Adopt connector v2 when detached placement or narrower lifecycle is useful. | +| Existing provider, tool, memory, service, or other core plugin | Remains Host-owned with existing public SDK behavior. | Adopt new Host capabilities only when needed. | +| New channel plugin | May use the existing contract. | Prefer connector v2 after the SDK is documented and stable. | +| Bundled OpenClaw channel | Migrated one at a time after connector v2 proof. | Must use the modern contract before detached default for that channel. | + +Legacy v1 placement is an explicit compatibility contract: + +- detect by declared capability and current plugin shape, not by plugin ID; +- keep broad in-process runtime access only inside the co-located boundary; +- preserve current lifecycle and reload behavior; +- emit diagnostics that explain placement without warning users that their valid setup is broken; +- do not assign a deprecation deadline in this RFC; +- prevent bundled/internal plugins from using the legacy path after they have a modern replacement. + +Connector v2 must be additive and versioned through capability negotiation. It should expose the smallest serializable contract required for: + +- ingress; +- delivery; +- receipts; +- typing and presence; +- health; +- capabilities; +- account lifecycle; +- configuration prepare/activate; +- shutdown and drain. + +### Protocol compatibility + +The current Gateway protocol remains the public protocol. The Control Server must preserve existing method, event, auth, and subscription behavior while changing the implementation behind it. + +Any owner-level diagnostics should first fit existing compatible response fields or additive methods/events. An incompatible protocol change requires a separate versioned proposal and is not part of this migration. + +## Phases + +### Phase 0: Contracts, baselines, and dependency guardrails + +**Objective:** Make the intended boundaries reviewable and prevent new coupling while behavior remains unchanged. + +Work: + +- Add an architecture decision record in `openclaw/openclaw`. +- Define the owner matrix and lifecycle state machine. +- Inventory Gateway-owned start, close, reload, and shutdown behavior. +- Inventory all configuration paths and existing reload-prefix declarations. +- Inventory external plugin-facing Gateway, channel, and lifecycle contracts. +- Add dependency checks that prevent new Host-to-Control Server and Host-to-channel imports. +- Add baseline tests for current user, protocol, config, and plugin behavior. +- Define combined placement and rollback semantics. + +Exit gates: + +- Every known lifecycle surface has a proposed owner. +- Every configuration path is inventoried. +- Legacy external plugin fixtures are captured. +- Boundary checks fail when new reverse dependencies are introduced. +- No user-visible behavior changes. + +### Phase 1: Host lifecycle and Config Coordinator + +**Objective:** Introduce the Host as a real lifecycle owner while everything remains in one process. + +Work: + +- Add `OpenClawHost` start, ready, drain, close, and health lifecycle. +- Make the Compatibility Supervisor start the Host. +- Move complete config validation and revision coordination into the Host. +- Generate owner-scoped apply plans while still executing combined behavior where needed. +- Move cron ownership into the Host. +- Move heartbeat scheduling and work generation into the Host. +- Introduce a Host delivery port so heartbeat and cron do not call channel adapters directly. + +Exit gates: + +- Host starts and runs with all channels disabled. +- Cron and heartbeat survive a simulated connector restart. +- Config Coordinator can explain ownership and planned action for every changed path. +- Current public command, config, and protocol remain unchanged. +- Shutdown ordering is deterministic and covered. + +### Phase 2: Canonical Host service APIs + +**Objective:** Make local and protocol clients use one core implementation. + +Work: + +- Add Host APIs for conversations, sessions, schedules, tasks, cancellation, and status. +- Add durable delivery-intent APIs and receipt reconciliation. +- Route embedded/local TUI execution through Host APIs. +- Route current Gateway-facing core methods through Host APIs. +- Remove internal Gateway RPC calls from core systems. +- Remove embedded Gateway stubs after all OpenClaw-owned callers migrate. +- Move core plugin services under Host lifecycle. + +Exit gates: + +- Local TUI and Control Server paths execute the same Host methods. +- No core subsystem requires a Control Server connection. +- No OpenClaw-owned core caller uses the legacy Gateway stub. +- Host is the only canonical writer of core state. +- Active work survives a simulated Control Server restart. + +### Phase 3: Control Server facade and owner-scoped reload + +**Objective:** Make the existing Gateway protocol a replaceable facade over the Host. + +Work: + +- Extract the current protocol/auth/server surface into the Control Server owner. +- Route existing methods, subscriptions, UI, TUI, CLI, node, and hook behavior to Host APIs. +- Project Host and connector health through existing compatible views. +- Assign Gateway HTTP, WebSocket, TLS, bind, and port configuration to the Control Server. +- Execute Control Server-only config restart plans. +- Add desired-versus-active revision diagnostics. + +Exit gates: + +- Existing protocol compatibility suites pass unchanged. +- Control Server can restart without cancelling active Host work. +- Control Server-owned config changes restart only the Control Server. +- Existing clients reconnect according to current behavior. +- Combined service health accurately represents Host and Control Server state. + +### Phase 4: Connector v2 and legacy compatibility placement + +**Objective:** Create a transport-only serializable channel contract without breaking existing plugins. + +Work: + +- Define connector v2 ingress, delivery, receipt, health, capability, lifecycle, and config-revision messages. +- Define account and connector identity, fencing, and delivery idempotency rules. +- Add the legacy v1 classifier and co-located placement. +- Add plugin diagnostics that show placement and capability. +- Migrate one bundled connector as the reference implementation. +- Prove owner-scoped connector and account restarts. +- Migrate remaining bundled connectors incrementally. + +Exit gates: + +- An unchanged external legacy channel plugin fixture passes. +- A connector v2 fixture passes both in-process and detached-boundary contract tests. +- The reference bundled connector can restart during an active Host turn without cancelling it. +- Connector-owned config changes restart only the affected connector or account. +- Duplicate-delivery proof passes across restart and retry cases. +- No bundled connector migrated to v2 uses broad Host internals. + +### Phase 5: Supervised process split + +**Objective:** Use process isolation as a deployment and fault-containment mechanism after logical boundaries are proven. + +Work: + +- Add private local IPC for Host, Control Server, and connector v2 messages. +- Add leases, fencing tokens, restart budgets, backoff, and drain deadlines. +- Add process-level desired and active revision convergence. +- Add combined and owner-level health reporting. +- Split the Control Server into a supervised process. +- Split migrated connector v2 implementations into supervised processes. +- Keep legacy v1 plugins in safe co-located placement. +- Preserve one service-manager entry and existing public command. + +Exit gates: + +- Process kill and restart fault-injection tests pass. +- Stale connector instances cannot deliver after fencing. +- Host work survives Control Server and connector process failure. +- Upgrade from the last stable package requires no config edits or doctor run. +- Rollback to combined placement is proven. +- Operator diagnostics identify failed owner, active revision, and recovery action. + +## Implementation Work Packages + +The phases above describe product milestones. These work packages are intended to become focused PRs or small PR stacks. + +### P0: Architecture decision and public invariants + +- Document owner boundaries, public invariants, lifecycle state machine, and rollback placement. +- Record the source survey and current behavior baselines. +- Name the compatibility contracts that must not change. + +### P1: Dependency and ownership guardrails + +- Add import/dependency checks for Host, Control Server, connector, and compatibility boundaries. +- Add configuration-path ownership inventory validation. +- Add a ratchet against new core calls through Gateway RPC or channel adapters. + +### P2: Host lifecycle shell + +- Introduce Host lifecycle, health, drain, and close. +- Start it from the existing service composition. +- Do not move behavior until lifecycle proof is stable. + +### P2A: Config Coordinator and apply-plan metadata + +- Move complete config validation and diff ownership to the Host. +- Add owner-scoped plan types and diagnostics. +- Initially execute combined restart for unknown ownership. + +### P3: Cron to Host + +- Make cron start, stop, reload, and execution Host-owned. +- Preserve current protocol and user behavior through projection. + +### P4: Heartbeat and delivery port + +- Make heartbeat scheduling and work Host-owned. +- Replace direct channel delivery access with the Host delivery port. +- Establish portable delivery intent before connector v2. + +### P5: Conversations, sessions, schedules, tasks, and delivery intent + +- Introduce narrow Host APIs. +- Route existing protocol handlers to them. +- Make Host state ownership explicit. + +### P6: Embedded TUI to Host + +- Route local TUI execution through the same Host APIs used by protocol clients. +- Prove zero-channel local conversation operation. + +### P7: Remove internal Gateway RPC and embedded stubs + +- Migrate all OpenClaw-owned callers. +- Delete the old internal path once no shipped external contract requires it. + +### P8: Control Server extraction + +- Make protocol/auth/HTTP/WebSocket lifecycle independently restartable. +- Assign Control Server configuration ownership. + +### P9: Existing client projection + +- Preserve UI, TUI, CLI, node, hook, method, event, and subscription behavior. +- Add reconnect and active-work-survival proof. + +### P10: Connector v2 contract + +- Define serializable messages and capability negotiation. +- Define delivery IDs, receipts, fencing, account lifecycle, and revision activation. +- Publish plugin SDK docs and fixtures. + +### P11: Legacy v1 classifier and diagnostics + +- Classify existing plugins into automatic co-located placement. +- Preserve current config, lifecycle, and restart-prefix behavior. +- Add compatibility fixtures and understandable diagnostics. + +### P12: First bundled connector migration + +- Select one representative connector. +- Prove in-process v2, owner-scoped reload, detached placement, restart, and delivery reconciliation. + +### P13: Remaining bundled connector migrations + +- Migrate in small channel-owned PRs. +- Keep unmigrated and external v1 connectors co-located. + +### P14: Supervisor, IPC, leases, and revision convergence + +- Add private IPC and supervised child placement. +- Add fencing, restart budgets, health aggregation, drain, and convergence state. + +### P15: Zero-touch upgrade and cohort rollout + +- Prove stable-package upgrade with unchanged config and plugins. +- Roll out detached placement by capability cohort. +- Preserve combined placement rollback until detached proof is mature. + +## Test and Proof Matrix + +### Core independence + +- Start Host with zero configured channels. +- Start Host with all configured channels disabled. +- Run a local TUI conversation without Control Server or channels. +- Run cron, heartbeat, and tasks without Control Server or channels. +- Restart and reconnect Control Server while a Host turn remains active. +- Crash and restart one connector while a Host turn remains active. + +### Compatibility + +- Upgrade from the last stable published package using an unchanged real-world config fixture. +- Start with existing `gateway.*` settings and service manager behavior. +- Exercise current protocol clients without changes. +- Load unchanged external legacy channel plugin fixtures. +- Load unchanged external non-channel plugin fixtures. +- Preserve `gateway.reload.mode` semantics. +- Preserve explicit full restart semantics. + +### Configuration lifecycle + +- Hot-apply a Host-owned change. +- Restart one Host subsystem. +- Restart one connector account. +- Restart one connector. +- Restart only the Control Server. +- Apply a mixed-owner revision and observe convergence. +- Reject or safely combine-restart an unknown-owner revision. +- Fail preparation and prove no silent partial activation. +- Crash during activation and prove revision state remains diagnosable. + +### Delivery and connector safety + +- Receive ingress once across connector reconnect. +- Deliver visible output once across retry and connector restart. +- Reject stale connector receipts after fencing. +- Drain accepted delivery before planned restart. +- Report pending, accepted, acknowledged, failed, and final delivery states. +- Exercise multi-account connector concurrency. + +### Lifecycle and fault injection + +- Kill Control Server process. +- Kill one connector process. +- Kill and restart the Compatibility Supervisor while preserving documented Host behavior. +- Exercise restart budget and backoff. +- Exercise drain deadline and forced shutdown. +- Verify shutdown ordering and resource cleanup. +- Verify owner health and desired/active revision diagnostics. + +### Security + +- Prove channel credentials remain connector-scoped where supported. +- Prove native untrusted payloads cross into Host only through portable ingress. +- Prove Control Server auth and authorization behavior remains unchanged. +- Prove private IPC rejects unauthorized or stale peers. +- Prove logs and diagnostics do not expose secrets. + +## Rollout Plan + +### Default placement progression + +1. Combined service with Host lifecycle present. +2. Combined service with canonical Host APIs and Control Server facade. +3. Combined service with connector v2 reference implementation. +4. Opt-in or capability-gated detached placement for internal proof. +5. Detached Control Server and migrated bundled connectors by default. +6. Legacy external v1 plugins remain automatically co-located. + +The user still runs one public service throughout this progression. + +### Cohort gates + +A connector or owner can move to detached default only when: + +- it declares the modern capability; +- its config paths have complete ownership metadata; +- its lifecycle and fault-injection suite passes; +- its delivery reconciliation suite passes where relevant; +- upgrade and rollback proof passes; +- diagnostics identify failures without requiring internal knowledge. + +### Rollback + +Rollback returns an owner to combined placement while preserving the same Host APIs and state ownership. It must not reactivate a removed core implementation or create dual writes. + +### Observability + +Additive diagnostics should expose: + +- combined service health; +- owner health and readiness; +- owner placement; +- desired configuration revision; +- active revision per owner; +- pending or failed apply action; +- restart count, budget, and next retry; +- connector account health; +- delivery backlog and failure classification without message content or secrets. + +## Risk Register + +| Risk | Severity | Mitigation and gate | +| --- | --- | --- | +| Dual core execution during migration | Critical | One canonical Host API per migrated capability; delete old internal path before phase exit. | +| Duplicate or stale channel delivery | Critical | Delivery IDs, receipts, idempotency, leases, fencing, and restart fault proof. | +| External plugin incompatibility | Critical | Automatic legacy v1 co-location, unchanged fixtures, additive v2, no initial deprecation. | +| Multiple writers to canonical state | Critical | Host-only state ownership and boundary checks. | +| Configuration split-brain | Critical | Complete validation, owner prepare, desired/active revisions, fail-closed ownership, recovery proof. | +| Hidden lifecycle globals | High | Inventory, dependency guardrails, deterministic shutdown tests, move one owner at a time. | +| Protocol behavior drift | High | Keep Control Server as compatibility facade; unchanged protocol suites and clients. | +| Expanded local trust boundary | High | Private authenticated IPC, least privilege, connector-scoped credentials, stale-peer fencing. | +| Operator complexity | High | One public service, existing config, combined health, additive owner diagnostics. | +| Rollback reintroduces two implementations | High | Rollback changes placement only; Host remains canonical. | +| Excess resource or startup cost | Medium | Measure per-owner startup, memory, connection, and restart costs before detached default. | +| Migration stalls with permanent mixed architecture | Medium | Phase exit gates, ownership inventory, and bundled-caller migration ratchets. | + +## Review and Landing Requirements + +Every implementation PR should state: + +- owner boundary changed; +- public and plugin compatibility impact; +- configuration paths affected; +- old internal path removed or retained contract; +- focused proof run; +- fault or upgrade proof when relevant; +- rollback behavior; +- remaining phase gate. + +Larger behavior, plugin SDK, protocol, security, and default-placement changes require owner review. A phase should not be considered complete while its compatibility or real-behavior proof is deferred. + +## Completion Criteria + +This RFC is implemented when: + +- Host core systems run and remain useful with zero channels and no Control Server; +- current public commands, config, protocol, UI, TUI, CLI, nodes, and hooks continue to work; +- existing external plugins work without changes; +- migrated connectors can restart independently; +- Control Server can restart independently; +- owner-scoped config changes affect only their owners; +- explicit full restart retains its current behavior; +- Host is the only canonical core implementation and state writer; +- stable-package upgrade and rollback proof pass; +- detached placement is an internal detail users do not need to configure or understand.