|
| 1 | +--- |
| 2 | +draft: false |
| 3 | +date: 2025-06-25 |
| 4 | +categories: |
| 5 | + - Release |
| 6 | +authors: |
| 7 | + - kylebarron |
| 8 | +links: |
| 9 | + - CHANGELOG.md |
| 10 | +--- |
| 11 | + |
| 12 | +# Releasing obstore 0.7! |
| 13 | + |
| 14 | +Obstore is the simplest, highest-throughput Python interface to Amazon S3, Google Cloud Storage, and Azure Storage, powered by Rust. |
| 15 | + |
| 16 | +This post gives an overview of what's new in obstore version 0.7. |
| 17 | + |
| 18 | +<!-- more --> |
| 19 | + |
| 20 | +Refer to the [changelog](../../CHANGELOG.md) for all updates. |
| 21 | + |
| 22 | +## Anonymous connections to Google Cloud Storage |
| 23 | + |
| 24 | +Obstore now supports anonymous connections to GCS. Pass [`skip_signature=True`][obstore.store.GCSConfig.skip_signature] to configure an anonymous connection. |
| 25 | + |
| 26 | +```py |
| 27 | +from obstore.store import GCSStore |
| 28 | + |
| 29 | +store = GCSStore( |
| 30 | + "weatherbench2", |
| 31 | + prefix="datasets/era5/1959-2023_01_10-full_37-1h-0p25deg-chunk-1.zarr", |
| 32 | + # Anonymous connection |
| 33 | + skip_signature=True, |
| 34 | +) |
| 35 | +store.list_with_delimiter()["objects"] |
| 36 | +``` |
| 37 | + |
| 38 | +Now prints: |
| 39 | + |
| 40 | +```py |
| 41 | +[{'path': '.zattrs', |
| 42 | + 'last_modified': datetime.datetime(2023, 11, 22, 9, 4, 54, 481000, tzinfo=datetime.timezone.utc), |
| 43 | + 'size': 2, |
| 44 | + 'e_tag': '"99914b932bd37a50b983c5e7c90ae93b"', |
| 45 | + 'version': None}, |
| 46 | + {'path': '.zgroup', |
| 47 | + 'last_modified': datetime.datetime(2023, 11, 22, 9, 4, 53, 465000, tzinfo=datetime.timezone.utc), |
| 48 | + 'size': 24, |
| 49 | + 'e_tag': '"e20297935e73dd0154104d4ea53040ab"', |
| 50 | + 'version': None}, |
| 51 | + {'path': '.zmetadata', |
| 52 | + 'last_modified': datetime.datetime(2023, 11, 22, 9, 4, 54, 947000, tzinfo=datetime.timezone.utc), |
| 53 | + 'size': 46842, |
| 54 | + 'e_tag': '"9d287796ca614bfec4f1bb20a4ac1ba3"', |
| 55 | + 'version': None}] |
| 56 | +``` |
| 57 | + |
| 58 | +## Obspec v0.1 compatibility |
| 59 | + |
| 60 | +Obstore provides an implementation for accessing Amazon S3, Google Cloud Storage, and Azure Storage, but some libraries may want to also support other backends, such as HTTP clients or more obscure things like SFTP or HDFS filesystems. |
| 61 | + |
| 62 | +Additionally, there's a bunch of useful behavior that could exist on top of Obstore: caching, metrics, globbing, bulk operations. While all of those operations are useful, we want to keep the core Obstore library as small as possible, tightly coupled with the underlying Rust `object_store` library. |
| 63 | + |
| 64 | +[Obspec](https://developmentseed.org/obspec/) exists to provide the abstractions for generic programming against object store backends. Obspec is essentially a formalization and generalization of the Obstore API, so if you're already using Obstore, very few changes are needed to use Obspec instead. |
| 65 | + |
| 66 | +Downstream libraries can program against the Obspec API to be fully generic around what underlying backend is used at runtime. |
| 67 | + |
| 68 | +For further information, refer to the [Obspec documentation](https://developmentseed.org/obspec/latest/) and the [Obspec announcement blog post](https://developmentseed.org/obspec/latest/blog/2025/06/25/introducing-obspec-a-python-protocol-for-interfacing-with-object-storage/). |
| 69 | + |
| 70 | +## Customize headers sent in requests |
| 71 | + |
| 72 | +`ClientConfig` now accepts a [`default_headers` key][obstore.store.ClientConfig.default_headers]. This allows you to add additional headers that will be sent by the HTTP client on every request. |
| 73 | + |
| 74 | +## Improvements to NASA Earthdata credential provider |
| 75 | + |
| 76 | +The NASA Earthdata credential provider now allows user to customize the host that handles credentialization. |
| 77 | + |
| 78 | +It also allows for more possibilities of passing credentials. Authentication information can be a NASA Earthdata token, NASA Earthdata username/password (tuple), or `None`, in which case, environment variables or a `~/.netrc` file are used, if set. |
| 79 | + |
| 80 | +See updated documentation on the [NASA Earthdata page](../../../../../api/auth/earthdata/). |
| 81 | + |
| 82 | +## Fixed creation of `AzureStore` from HTTPS URL |
| 83 | + |
| 84 | +Previously, this would create an incorrect AzureStore configuration: |
| 85 | + |
| 86 | +```py |
| 87 | +url = "https://overturemapswestus2.blob.core.windows.net/release" |
| 88 | +store = AzureStore.from_url(url, skip_signature=True) |
| 89 | +``` |
| 90 | + |
| 91 | +because it would interpret `release` as part of the within-bucket _prefix_, when it should really be interpreted as the _container name_. |
| 92 | + |
| 93 | +This is now fixed and this test passes: |
| 94 | + |
| 95 | +```py |
| 96 | +url = "https://overturemapswestus2.blob.core.windows.net/release" |
| 97 | +store = AzureStore.from_url(url, skip_signature=True) |
| 98 | + |
| 99 | +assert store.config.get("container_name") == "release" |
| 100 | +assert store.config.get("account_name") == "overturemapswestus2" |
| 101 | +assert store.prefix is None |
| 102 | +``` |
| 103 | + |
| 104 | +## Improved documentation |
| 105 | + |
| 106 | +- [New `Zarr` example](../../examples/zarr.md) |
| 107 | +- [New `stream-zip` example](../../examples/stream-zip.md) |
| 108 | + |
| 109 | +## All updates |
| 110 | + |
| 111 | +Refer to the [changelog](../../CHANGELOG.md) for all updates. |
0 commit comments