Skip to content

Cosmos: Cut over CosmosClient and DatabaseClient to driver#4147

Open
tvaron3 wants to merge 17 commits intoAzure:release/azure_data_cosmos-previewsfrom
tvaron3:feature/cutover-cosmos-database-to-driver
Open

Cosmos: Cut over CosmosClient and DatabaseClient to driver#4147
tvaron3 wants to merge 17 commits intoAzure:release/azure_data_cosmos-previewsfrom
tvaron3:feature/cutover-cosmos-database-to-driver

Conversation

@tvaron3
Copy link
Copy Markdown
Member

@tvaron3 tvaron3 commented Apr 10, 2026

Route create_database, read, create_container, and delete through CosmosDriver instead of the legacy gateway pipeline, following the pattern established by create_item/read_item.

  • Add DatabaseReference field to DatabaseClient
  • Remove stale #[allow(unused_variables)] annotations
  • Remove dead CosmosRequest/OperationType imports
  • Add CHANGELOG entry

Fixes #4127

Copilot AI review requested due to automatic review settings April 10, 2026 22:30
@tvaron3 tvaron3 requested a review from a team as a code owner April 10, 2026 22:30
@github-actions github-actions bot added the Cosmos The azure_cosmos crate label Apr 10, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR continues the Cosmos SDK “driver cutover” by routing additional control-plane operations (create_database, DatabaseClient::read, DatabaseClient::create_container, DatabaseClient::delete) through azure_data_cosmos_driver, aligning them with the existing driver-routed item operations.

Changes:

  • Added OperationOptions plumbing to database/container CRUD options and routed those operations through CosmosDriver::execute_operation.
  • Introduced DatabaseReference storage on DatabaseClient to build driver operations without relying on the legacy gateway request types.
  • Added apply_throughput_headers helper to merge throughput headers into driver OperationOptions while preserving user-provided custom headers.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
sdk/cosmos/azure_data_cosmos/src/options/mod.rs Adds operation: OperationOptions to create/read/delete database/container options (including changing read/delete DB options from unit structs).
sdk/cosmos/azure_data_cosmos/src/driver_bridge.rs Adds apply_throughput_headers to merge throughput-derived headers into driver operation options.
sdk/cosmos/azure_data_cosmos/src/clients/database_client.rs Cuts over read, create_container, and delete to driver operations using DatabaseReference.
sdk/cosmos/azure_data_cosmos/src/clients/cosmos_client.rs Cuts over create_database to driver operations and applies throughput headers via the bridge helper.
sdk/cosmos/azure_data_cosmos/CHANGELOG.md Adds an Unreleased entry describing the cutover and options support.

Comment thread sdk/cosmos/azure_data_cosmos/src/options/mod.rs Outdated
Comment thread sdk/cosmos/azure_data_cosmos/src/options/mod.rs Outdated
Comment thread sdk/cosmos/azure_data_cosmos/CHANGELOG.md
Comment thread sdk/cosmos/azure_data_cosmos/src/driver_bridge.rs Outdated
Route create_database, read, create_container, and delete
through CosmosDriver instead of the legacy gateway pipeline,
following the pattern established by create_item/read_item.

- Add DatabaseReference field to DatabaseClient
- Add OperationOptions to CRUD options types
- Add apply_throughput_headers helper in driver_bridge that
  merges throughput headers with existing custom headers
- Remove stale #[allow(unused_variables)] annotations
- Remove dead CosmosRequest/OperationType imports
- Add CHANGELOG entry

Fixes Azure#4127

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tvaron3 tvaron3 force-pushed the feature/cutover-cosmos-database-to-driver branch from 4e6ba28 to 509bf39 Compare April 10, 2026 22:38
@tvaron3 tvaron3 closed this Apr 10, 2026
@github-project-automation github-project-automation bot moved this from Todo to Done in CosmosDB Go/Rust Crew Apr 10, 2026
@tvaron3 tvaron3 reopened this Apr 11, 2026
@tvaron3 tvaron3 marked this pull request as draft April 11, 2026 01:24
tvaron3 and others added 5 commits April 11, 2026 20:44
Covers the autopilot path of apply_throughput_headers, verifying
that autoscale ThroughputProperties produces the x-ms-cosmos-offer-
autopilot-settings header and does not produce the manual throughput
header.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This artifact belongs to the coding harness workflow and
should not be committed to the repository.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… CRUD types

Per the ConfigurationOptions spec (§5.6), database/container CRUD
options types should not include OperationOptions. Throughput is
now a first-class typed field on the driver's CosmosRequestHeaders
instead of being passed as custom headers on OperationOptions.

- Add offer_throughput and offer_autopilot_settings fields to
  CosmosRequestHeaders with write_to_headers support
- Create throughput_headers module for SDK-to-driver conversion
- Revert CreateDatabaseOptions, ReadDatabaseOptions,
  DeleteDatabaseOptions, CreateContainerOptions to simple structs
- Remove apply_throughput_headers helper from driver_bridge

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace standalone throughput_headers module with a TryFrom impl
on ThroughputProperties, keeping conversion logic co-located with
the type it converts. This is the idiomatic Rust approach for
fallible type conversions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tvaron3 tvaron3 marked this pull request as ready for review April 13, 2026 05:44
@tvaron3
Copy link
Copy Markdown
Member Author

tvaron3 commented Apr 13, 2026

/azp run rust - cosmos - weekly

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

tvaron3 and others added 3 commits April 13, 2026 11:11
The driver adds Prefer: return=minimal by default for all write
operations, but database and container creates need the full
response body so callers can inspect the created resource. Set
ContentResponseOnWrite::Enabled for create_database and
create_container to match the previous gateway behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The driver already supports proxy_allowed on ConnectionPoolOptions
but the SDK never forwarded the allow_proxy setting. This caused
driver-routed operations to ignore HTTPS_PROXY even when proxy
was explicitly enabled via with_proxy_allowed(true).

Also consolidates the connection pool builder so emulator cert
config and proxy config share the same builder instance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The driver probes the endpoint during build(), which routes through
the proxy. The fake proxy doesn't implement CONNECT, so the probe
fails with ConnectionReset. Move build() into the spawned task so
the proxy connection signal fires regardless of whether the probe
succeeds. The test only needs to verify the proxy was contacted,
not that the operation succeeded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tvaron3 tvaron3 force-pushed the feature/cutover-cosmos-database-to-driver branch from d445347 to b20982d Compare April 14, 2026 00:31
@tvaron3
Copy link
Copy Markdown
Member Author

tvaron3 commented Apr 14, 2026

/azp run rust - cosmos - weekly

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

Comment thread sdk/cosmos/azure_data_cosmos/src/models/throughput_properties.rs Outdated
pub offer_throughput: Option<usize>,

/// JSON-serialized autoscale settings (`x-ms-cosmos-offer-autopilot-settings`).
pub offer_autopilot_settings: Option<String>,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is OK, since SDKs generally control serialization, but it feels a little weird. It feels like the driver should expose a model type here and do the serialization internally.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed for driver to expose model type

Comment on lines +113 to +124
if let Some(throughput) = self.offer_throughput {
headers.insert(
request_header_names::OFFER_THROUGHPUT.clone(),
HeaderValue::from(throughput.to_string()),
);
}
if let Some(autopilot) = self.offer_autopilot_settings.as_ref() {
headers.insert(
request_header_names::OFFER_AUTOPILOT_SETTINGS.clone(),
HeaderValue::from(autopilot.clone()),
);
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might conflict with the optimizations I'm doing in #4159

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll wait for your pr and address the merge conflicts

Per analogrelay's feedback:
- Replace TryFrom<&ThroughputProperties> with apply_headers method
  that mutates headers in place (simpler, no new struct creation)
- Add OfferAutoscaleSettings model to driver so serialization is
  handled internally instead of passing pre-serialized JSON
- Driver's write_to_headers() now serializes the autoscale struct

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

impl CreateDatabaseOptions {
/// Sets the throughput properties for the new database.
pub fn with_throughput(mut self, throughput: ThroughputProperties) -> Self {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates shared throughput databases, right @FabianMeiswinkel. Should we remove it, or else at least add docs saying to avoid it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am all in favor of a scream test - Means we are not able to test it ourselves anymore thought - so, we should keep an open mind

tvaron3 and others added 4 commits April 14, 2026 14:38
…iews' into feature/cutover-cosmos-database-to-driver

# Conflicts:
#	sdk/cosmos/azure_data_cosmos/src/clients/cosmos_client.rs
#	sdk/cosmos/azure_data_cosmos/src/clients/database_client.rs
#	sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_headers.rs
Database-level shared throughput is discouraged. Remove
CreateDatabaseOptions::with_throughput() and associated tests
as a scream test per discussion with Pilchie and Fabian.

Container-level throughput (CreateContainerOptions::with_throughput)
is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The OfferAutoscaleSettings model only carried max_throughput,
dropping the autoUpgradePolicy (increment_percent). This caused
the container_throughput_crud_autoscale emulator test to fail
because the increment percent was lost during header serialization.

Add AutoscaleAutoUpgradePolicy and AutoscaleThroughputPolicy
models to the driver and pass the full autoscale settings
from the SDK's apply_headers method.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tvaron3
Copy link
Copy Markdown
Member Author

tvaron3 commented Apr 16, 2026

/azp run rust - cosmos - weekly

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

- azure_core_test: remove redundant .into_iter() call
- azure_storage_blob: use sort_by_key instead of sort_by
- azure_data_cosmos_driver: use sort_by_key instead of sort_by

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tvaron3
Copy link
Copy Markdown
Member Author

tvaron3 commented Apr 16, 2026

/azp run rust - cosmos - weekly

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Copy Markdown
Member

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only signing off on changes outside sdk/cosmos; that said, to avoid needing my sign off on such changes for future iterations of this PR, you should rebase on main. Those changes were already fixed by @analogrelay and would be removed from this PR.

@analogrelay
Copy link
Copy Markdown
Member

I've also pulled those fixes up to our release branch, so you can just rebase on the release branch and get them

…iews' into feature/cutover-cosmos-database-to-driver

# Conflicts:
#	sdk/cosmos/azure_data_cosmos_driver/src/driver/routing/session_container.rs
#	sdk/storage/azure_storage_blob/src/clients/block_blob_client.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cosmos The azure_cosmos crate

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Cosmos: Cut over CosmosClient and DatabaseClient methods to driver

6 participants