Skip to content

[Storage] remove download_into#4166

Merged
jaschrep-msft merged 3 commits intoAzure:mainfrom
jaschrep-msft:remove-download-into
Apr 17, 2026
Merged

[Storage] remove download_into#4166
jaschrep-msft merged 3 commits intoAzure:mainfrom
jaschrep-msft:remove-download-into

Conversation

@jaschrep-msft
Copy link
Copy Markdown
Member

download_into currently cannot account for its unsafe use of pointers when its future is dropped. It is being removed for now to return in a future release.

Streaming download does not encounter this issue, as it owns all its memory.

The Problem

To achieve the desired performance, managed downloads spawn workers in the async runtime which do the work of individual downloads into memory. download_into must write bytes directly into &mut [u8], which cannot be safely sent over a thread boundary.

However, these workers are not contained in the Futures which reference them. Those are only join handles. Dropping them does not stop the thread, it only loses any reference to its results.

This is already accounted for in the event of a failure. The async function does not return without forcing every thread to join, ensuring no stray references to memory that disobeys borrow rules. Dropping the future for this async function bypasses that.

Future Solutions

Assuming we will continue to use borrowed memory and continue to spawn workers, the solution is likely to write a Drop implementation that will go off when the async future is dropped. This drop() will be responsible for guaranteeing no further work is done with the borrowed memory.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Removes the BlobClient::download_into API (and its internal implementation/tests) due to unsoundness when the async future is dropped, avoiding unsafe cross-thread borrowed buffer usage.

Changes:

  • Removed BlobClient::download_into public API and the partitioned-transfer implementation that wrote into &mut [u8].
  • Deleted SendSlice (unsafe Send wrapper over raw slice pointers) and its module wiring.
  • Removed unit/integration tests that exercised download_into.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
sdk/storage/azure_storage_blob/tests/blob_client.rs Removes recorded integration tests that depended on download_into.
sdk/storage/azure_storage_blob/src/partitioned_transfer/download.rs Deletes unsafe download_into implementation and related worker helpers/tests.
sdk/storage/azure_storage_blob/src/models/slice.rs Removes SendSlice definition that enabled sending borrowed buffers across threads.
sdk/storage/azure_storage_blob/src/models/mod.rs Drops the slice module export after deletion.
sdk/storage/azure_storage_blob/src/clients/blob_client.rs Removes the public download_into method from BlobClient.

Comment thread sdk/storage/azure_storage_blob/src/clients/blob_client.rs
@github-actions github-actions bot added the Storage Storage Service (Queues, Blobs, Files) label Apr 14, 2026
@jaschrep-msft
Copy link
Copy Markdown
Member Author

@heaths @LarryOsterman it seems semver checks are now in CI. We haven't technically GA'd yet and this method has serious memory safety concerns. How can we get this through?

@jaschrep-msft jaschrep-msft merged commit 4758d30 into Azure:main Apr 17, 2026
13 checks passed
@jaschrep-msft jaschrep-msft deleted the remove-download-into branch April 17, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants