Skip to content

[WSLC Compose 2/5] Introduce the service operation engine - #42158

Merged
Blue (OneBlue) merged 13 commits into
feature/composefrom
user/dkbennett/compose-operation-engine
Nov 12, 2026
Merged

Blue (OneBlue) merged 13 commits into
feature/composefrom
user/dkbennett/compose-operation-engine

Conversation

@dkbennett

@dkbennett David Bennett (dkbennett) commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Summary of the Pull Request

Introduce command-scoped, service-owned Compose execution through a synchronous native COM API, with captured inputs, progress, cooperative cancellation, and a final result.

The initial Compose implementation was a quick proof of concept that established an end-to-end path through file loading, resource creation, and lifecycle commands. This PR builds on that foundation with explicit actions, invocation-scoped inputs, fresh resource discovery, and a common execution contract for clients.

This is PR 2/5. It adds the service operation engine without migrating the existing CLI or removing the cached Compose session. Full configuration convergence, Docker Compose compatibility gaps, and changes to existing Up behavior remain outside this PR.

PR Checklist

  • Closes: No linked issue.
  • Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
  • Tests: Added/updated if needed and all pass
  • Localization: All end user facing strings can be localized
  • Dev docs: Added/updated if needed
  • Documentation updated: If checked, please file a pull request on our docs repo and link it here: Not applicable; internal developer documentation only.

Detailed Description of the Pull Request / Additional comments

Execution contract

Each invocation supplies an explicit action and either captured documents or a normalized project key:

  1. IWSLCSession::ExecuteComposeOperation accepts the request, an optional progress callback, an optional manual-reset cancellation event, and result storage. Request buffers remain valid until the method returns.
  2. A call-scoped executor interprets the captured inputs and runs the action on the incoming execution thread. The reconciler discovers WSLC-labeled resources and serializes discovery, mutation, and final observation per project.
  3. The method returns after execution, callback delivery, and cleanup. On S_OK, the result contains the execution HRESULT, terminal status, optional localized failure message, project key, and final successful container observation. The caller frees the returned message and container array with CoTaskMemFree.

Invalid request envelopes and transport failures fail the method. Interpretation, execution, and observed cancellation are reported through the terminal result. Localized execution errors are retained even without a progress callback or when terminal notification fails.

There is no remote operation object, dedicated service execution worker, separately waitable completion event, or result-retrieval RPC. Native RPC reports service-process failure directly. Session loss leaves the execution outcome unknown; it does not establish cancellation or rollback and must not trigger an automatic retry.

Why the native boundary is synchronous

This matches the existing WSLC service API model and ties invocation lifetime to the native call. The service still owns interpretation, execution policy, project serialization, cancellation checkpoints, cleanup, and the final outcome.

Clients that need responsive input or I/O can run the call on a COM-initialized background thread and coordinate cancellation locally. A synchronous native boundary does not require sequential resource execution or prevent concurrent stream consumption. Those concerns are independent of the outer API shape; parallel resource scheduling and full service-owned attached-command lifetime are not implemented here.

A future public SDK can expose standard WinRT asynchronous operations with explicit cancellation bridging. This PR adds no WinRT facade or public SDK API.

Lifetime and correctness

  • Project-lock acquisition waits on the mutex, caller cancellation, session termination, and caller-process exit. Abandoned mutexes fail execution, and recursive execution cannot interleave project mutations.
  • Session shutdown drains registered invocations independently of the VM runtime lock and keeps cancelling outgoing callbacks until registrations drain. This covers gaps before callback RPC entry and execution without a VM lease.
  • Discovery and public listing share typed Docker enumeration. Recovery subscribes to container events and revalidates existence and state before publishing an implementation, preventing stale snapshots from restoring deleted containers.
  • Stop honors the configured default stop signal. Final observation uses cached wrapper properties, including the original image reference, so successfully stopped auto-remove containers can be reported as Deleted.

Scope and limits

Supported actions are Validate, Create, Up, Start, Attach, and Stop. Create reuses a nonempty discovered container set; Up retains its existing destructive replacement behavior. This is not a configuration-convergence or partial-Up recovery change.

Cancellation is cooperative while waiting for the project lock and between resource operations. It does not interrupt an in-flight Docker or container call. Completed mutations can remain visible, and late cancellation can still leave success. Failure recovery requires resource rediscovery; there is no durable operation journal, reconnection protocol, or transactional rollback.

Callbacks remain synchronous and must return promptly. A stalled resource callback retains the project lock and delays cancellation checkpoints; a stalled terminal callback delays return. User cancellation alone does not interrupt a stalled callback. Callbacks must not wait for the current operation or another operation requiring the same project lock. Isolating execution from unresponsive observers remains separate work.

The typed request/result contract is internal and shipped in lockstep. SchemaVersion identifies the payload revision, not independently negotiated COM ABI compatibility. Pre-existing normalizer data types and validation remain intact.

This layer emits status and resource-progress events. Attach returns container identities, not terminal streams. CLI migration and cached-session removal belong to layer 3, management commands to layer 4, and diagnostic/stream delivery with its producers, consumers, and tests to layer 5.

The developer guide documents input ownership, result semantics, callback threading, project locking, cancellation, and these limitations.

Validation Steps Performed

Local validation of the PR 2 revision:

  • .\bin\x64\debug\test.bat '/name:*Compose*': passed, 36/36 tests

The focused suite covers request validation and capture, unsupported inputs, discovery and resource ordering, cancellation checkpoints, late cancellation, partial state, localized errors, session-process exit, project serialization, blocked-callback shutdown, deletion during Up, configured stop signals, auto-removal, and the existing CLI lifecycle. These counts are for PR 2, not the combined follow-up stack.

Add captured requests, asynchronous operations, managed-resource discovery, per-project serialization, progress, and cooperative cancellation. Retain the synchronous session API for existing CLI callers and cover the operation engine plus the existing CLI lifecycle.

Validation: full Windows build; Compose tests 14/14; CLI tests 518/518; formatting and changed-source copyright checks passed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 675962d7-8ad8-45ed-8c6c-a0cce4f3d8c4
Copilot-Session: 4bce7a09-be93-4627-b729-aaada0f63c66
Exercise empty service rejection through completed operation results. Assert failed status and zero affected containers while preserving the original default network ID.

Validation: full Windows build; Compose tests 16/16; formatting and test copyright checks passed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 675962d7-8ad8-45ed-8c6c-a0cce4f3d8c4
Copilot-Session: 4bce7a09-be93-4627-b729-aaada0f63c66
Copilot AI balanced review requested due to automatic review settings November 3, 2026 20:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cancellation checks are missing between resource operations, container ordering differs between creation and discovery, and missing readiness handshakes make cancellation tests prone to timing failures.

3 open issues
What changed in this PR

Introduces a service-owned Compose operation engine for WSLC, giving each invocation captured inputs and an explicit asynchronous lifecycle. The existing CLI remains on the cached Compose-session implementation.

Changes:

  • Adds internal COM contracts for progress, cancellation, completion, and final results.
  • Adds label-based resource discovery and per-project serialized execution.
  • Expands lifecycle tests and documents operation ownership, concurrency, and limitations.
File Description
test/​windows/​wslc/​WSLCE2EComposeLifecycleTests.cpp Adds CLI lifecycle regression coverage.
test/​windows/​wslc/​WSLCComposeTests.cpp Tests operation results, cancellation, discovery, and concurrency.
test/​windows/​CMakeLists.txt Adds the OneCore library dependency.
src/​windows/​wslcsession/​WSLCSession.h Declares operation entry points and reconciler ownership.
src/​windows/​wslcsession/​WSLCSession.cpp Integrates operations, discovery, progress, and cancellation checkpoints.
src/​windows/​wslcsession/​WSLCComposeOperation.h Defines operation state and worker ownership.
src/​windows/​wslcsession/​WSLCComposeOperation.cpp Implements request capture, asynchronous execution, and result publication.
src/​windows/​wslcsession/​ComposeReconciler.h Declares project execution and synchronization helpers.
src/​windows/​wslcsession/​ComposeReconciler.cpp Implements serialized discovery, lifecycle actions, and final observation.
src/​windows/​wslcsession/​ComposeNormalizer.h Declares project-key validation helpers.
src/​windows/​wslcsession/​ComposeNormalizer.cpp Validates normalized project keys.
src/​windows/​wslcsession/​CMakeLists.txt Includes the new operation-engine sources.
src/​windows/​service/​inc/​wslc.idl Defines internal Compose operation contracts.
src/​windows/​common/​ExecutionContext.cpp Allows empty context scopes around internal COM calls.
msipackage/​package.wix.in Registers the new COM interfaces for marshaling.
doc/​docs/​technical-documentation/​wslc-compose-operations.md Documents lifecycle semantics, usage, and limitations.

Review effort: Balanced

Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Comment thread test/windows/wslc/WSLCComposeTests.cpp Resolved
Copilot AI balanced review requested due to automatic review settings November 3, 2026 20:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical discovery/create race can disconnect the creating caller’s wrapper, while missing readiness handshakes can cause cancellation tests to time out before reaching their assertions.

4 open issues

Review effort: Balanced

Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Sort normalized services to match resource discovery and assert returned order. Check cancellation before network creation and image pulls independently of progress reporting. Wait for shell trap readiness before the cancellation tests issue Stop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4c14c899-3f8c-4862-aeac-8a6f0482b7b7
Copilot AI balanced review requested due to automatic review settings November 3, 2026 21:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Container discovery can race with pending creation, inserting a duplicate implementation that leaves the creating caller’s implementation untracked and causes its returned COM wrapper to disconnect.

2 open issues
2 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 3, 2026 22:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the tests retain request buffers through completion, leaving the captured-input lifetime guarantee unverified when caller data or the source file changes.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@dkbennett
David Bennett (dkbennett) marked this pull request as ready for review November 4, 2026 18:50
Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Comment thread src/windows/service/inc/wslc.idl Resolved
Comment thread src/windows/service/inc/wslc.idl Resolved
Copilot AI balanced review requested due to automatic review settings November 9, 2026 20:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The PR description promises a service-owned operation object and completion-event contract that do not match the synchronous, call-scoped implementation.

1 open issue

Review effort: Balanced

Comment thread src/windows/service/inc/wslc.idl Resolved

@ggarzia-MSFT ggarzia-MSFT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. one nit that is probably addressed in a follow up. not approving, needs Blue (@OneBlue) 's review as well

Comment thread src/windows/service/inc/wslc.idl Resolved
Comment thread src/windows/service/inc/wslc.idl Resolved
Comment thread src/windows/service/inc/wslc.idl Resolved
Copilot AI balanced review requested due to automatic review settings November 10, 2026 22:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because a critical shutdown race can misreport cancellation as failure and trigger ordinary-failure cleanup during creation.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 10, 2026 22:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because shutdown cancellation may escape error handling and trigger cleanup of completed resources.

No open issues 🎉

Review effort: Balanced

Comment thread src/windows/common/ExecutionContext.cpp Resolved
Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Comment thread src/windows/wslcsession/WSLCSession.cpp Resolved
Comment thread test/windows/wslc/WSLCComposeTests.cpp Resolved
Comment thread test/windows/wslc/WSLCComposeTests.cpp Resolved
Comment thread test/windows/wslc/WSLCComposeTests.cpp Resolved
Copilot AI balanced review requested due to automatic review settings November 12, 2026 21:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A redundant callback registration can throw E_ABORT outside the catch block during session shutdown, causing an accepted invocation to report Failed instead of Cancelled.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 12, 2026 22:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed for an unpublished finding that the installer registers IComposeOperation despite the interface being absent from the service contract and unsupported by the generated proxy.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@OneBlue Blue (OneBlue) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@OneBlue

Copy link
Copy Markdown
Collaborator

Overriding the CI since the WSLC test failure WslcSdkTests::InstallWithDependencies_WslPackage_GhFallback404 is caused by a broken test assumption, and the target is a feature branch so risk is low

@OneBlue
Blue (OneBlue) merged commit a97120d into feature/compose Nov 12, 2026
14 of 17 checks passed
@OneBlue
Blue (OneBlue) deleted the user/dkbennett/compose-operation-engine branch November 12, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants