Cosmos: Cut over CosmosClient and DatabaseClient to driver#4147
Cosmos: Cut over CosmosClient and DatabaseClient to driver#4147tvaron3 wants to merge 17 commits intoAzure:release/azure_data_cosmos-previewsfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR continues the Cosmos SDK “driver cutover” by routing additional control-plane operations (create_database, DatabaseClient::read, DatabaseClient::create_container, DatabaseClient::delete) through azure_data_cosmos_driver, aligning them with the existing driver-routed item operations.
Changes:
- Added
OperationOptionsplumbing to database/container CRUD options and routed those operations throughCosmosDriver::execute_operation. - Introduced
DatabaseReferencestorage onDatabaseClientto build driver operations without relying on the legacy gateway request types. - Added
apply_throughput_headershelper to merge throughput headers into driverOperationOptionswhile preserving user-provided custom headers.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/cosmos/azure_data_cosmos/src/options/mod.rs | Adds operation: OperationOptions to create/read/delete database/container options (including changing read/delete DB options from unit structs). |
| sdk/cosmos/azure_data_cosmos/src/driver_bridge.rs | Adds apply_throughput_headers to merge throughput-derived headers into driver operation options. |
| sdk/cosmos/azure_data_cosmos/src/clients/database_client.rs | Cuts over read, create_container, and delete to driver operations using DatabaseReference. |
| sdk/cosmos/azure_data_cosmos/src/clients/cosmos_client.rs | Cuts over create_database to driver operations and applies throughput headers via the bridge helper. |
| sdk/cosmos/azure_data_cosmos/CHANGELOG.md | Adds an Unreleased entry describing the cutover and options support. |
Route create_database, read, create_container, and delete through CosmosDriver instead of the legacy gateway pipeline, following the pattern established by create_item/read_item. - Add DatabaseReference field to DatabaseClient - Add OperationOptions to CRUD options types - Add apply_throughput_headers helper in driver_bridge that merges throughput headers with existing custom headers - Remove stale #[allow(unused_variables)] annotations - Remove dead CosmosRequest/OperationType imports - Add CHANGELOG entry Fixes Azure#4127 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4e6ba28 to
509bf39
Compare
Covers the autopilot path of apply_throughput_headers, verifying that autoscale ThroughputProperties produces the x-ms-cosmos-offer- autopilot-settings header and does not produce the manual throughput header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This artifact belongs to the coding harness workflow and should not be committed to the repository. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… CRUD types Per the ConfigurationOptions spec (§5.6), database/container CRUD options types should not include OperationOptions. Throughput is now a first-class typed field on the driver's CosmosRequestHeaders instead of being passed as custom headers on OperationOptions. - Add offer_throughput and offer_autopilot_settings fields to CosmosRequestHeaders with write_to_headers support - Create throughput_headers module for SDK-to-driver conversion - Revert CreateDatabaseOptions, ReadDatabaseOptions, DeleteDatabaseOptions, CreateContainerOptions to simple structs - Remove apply_throughput_headers helper from driver_bridge Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace standalone throughput_headers module with a TryFrom impl on ThroughputProperties, keeping conversion logic co-located with the type it converts. This is the idiomatic Rust approach for fallible type conversions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…r-cosmos-database-to-driver
|
/azp run rust - cosmos - weekly |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The driver adds Prefer: return=minimal by default for all write operations, but database and container creates need the full response body so callers can inspect the created resource. Set ContentResponseOnWrite::Enabled for create_database and create_container to match the previous gateway behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The driver already supports proxy_allowed on ConnectionPoolOptions but the SDK never forwarded the allow_proxy setting. This caused driver-routed operations to ignore HTTPS_PROXY even when proxy was explicitly enabled via with_proxy_allowed(true). Also consolidates the connection pool builder so emulator cert config and proxy config share the same builder instance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The driver probes the endpoint during build(), which routes through the proxy. The fake proxy doesn't implement CONNECT, so the probe fails with ConnectionReset. Move build() into the spawned task so the proxy connection signal fires regardless of whether the probe succeeds. The test only needs to verify the proxy was contacted, not that the operation succeeded. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
d445347 to
b20982d
Compare
|
/azp run rust - cosmos - weekly |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| pub offer_throughput: Option<usize>, | ||
|
|
||
| /// JSON-serialized autoscale settings (`x-ms-cosmos-offer-autopilot-settings`). | ||
| pub offer_autopilot_settings: Option<String>, |
There was a problem hiding this comment.
I suppose this is OK, since SDKs generally control serialization, but it feels a little weird. It feels like the driver should expose a model type here and do the serialization internally.
There was a problem hiding this comment.
changed for driver to expose model type
| if let Some(throughput) = self.offer_throughput { | ||
| headers.insert( | ||
| request_header_names::OFFER_THROUGHPUT.clone(), | ||
| HeaderValue::from(throughput.to_string()), | ||
| ); | ||
| } | ||
| if let Some(autopilot) = self.offer_autopilot_settings.as_ref() { | ||
| headers.insert( | ||
| request_header_names::OFFER_AUTOPILOT_SETTINGS.clone(), | ||
| HeaderValue::from(autopilot.clone()), | ||
| ); | ||
| } |
There was a problem hiding this comment.
This might conflict with the optimizations I'm doing in #4159
There was a problem hiding this comment.
I'll wait for your pr and address the merge conflicts
Per analogrelay's feedback: - Replace TryFrom<&ThroughputProperties> with apply_headers method that mutates headers in place (simpler, no new struct creation) - Add OfferAutoscaleSettings model to driver so serialization is handled internally instead of passing pre-serialized JSON - Driver's write_to_headers() now serializes the autoscale struct Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
||
| impl CreateDatabaseOptions { | ||
| /// Sets the throughput properties for the new database. | ||
| pub fn with_throughput(mut self, throughput: ThroughputProperties) -> Self { |
There was a problem hiding this comment.
This creates shared throughput databases, right @FabianMeiswinkel. Should we remove it, or else at least add docs saying to avoid it?
There was a problem hiding this comment.
I am all in favor of a scream test - Means we are not able to test it ourselves anymore thought - so, we should keep an open mind
…iews' into feature/cutover-cosmos-database-to-driver # Conflicts: # sdk/cosmos/azure_data_cosmos/src/clients/cosmos_client.rs # sdk/cosmos/azure_data_cosmos/src/clients/database_client.rs # sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_headers.rs
Database-level shared throughput is discouraged. Remove CreateDatabaseOptions::with_throughput() and associated tests as a scream test per discussion with Pilchie and Fabian. Container-level throughput (CreateContainerOptions::with_throughput) is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The OfferAutoscaleSettings model only carried max_throughput, dropping the autoUpgradePolicy (increment_percent). This caused the container_throughput_crud_autoscale emulator test to fail because the increment percent was lost during header serialization. Add AutoscaleAutoUpgradePolicy and AutoscaleThroughputPolicy models to the driver and pass the full autoscale settings from the SDK's apply_headers method. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run rust - cosmos - weekly |
|
Azure Pipelines successfully started running 1 pipeline(s). |
- azure_core_test: remove redundant .into_iter() call - azure_storage_blob: use sort_by_key instead of sort_by - azure_data_cosmos_driver: use sort_by_key instead of sort_by Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run rust - cosmos - weekly |
|
Azure Pipelines successfully started running 1 pipeline(s). |
heaths
left a comment
There was a problem hiding this comment.
Only signing off on changes outside sdk/cosmos; that said, to avoid needing my sign off on such changes for future iterations of this PR, you should rebase on main. Those changes were already fixed by @analogrelay and would be removed from this PR.
|
I've also pulled those fixes up to our release branch, so you can just rebase on the release branch and get them |
…iews' into feature/cutover-cosmos-database-to-driver # Conflicts: # sdk/cosmos/azure_data_cosmos_driver/src/driver/routing/session_container.rs # sdk/storage/azure_storage_blob/src/clients/block_blob_client.rs
Route
create_database,read,create_container, anddeletethrough CosmosDriver instead of the legacy gateway pipeline, following the pattern established bycreate_item/read_item.DatabaseReferencefield toDatabaseClient#[allow(unused_variables)]annotationsCosmosRequest/OperationTypeimportsFixes #4127