Repository navigation
Open AnnData Zarr from GET-only hosts and pre-0.8 stores - #170
Merged
Merged
Conversation
The Azimuth human fetus reference (azimuth-vitessce S3) showed "Request blocked": the bucket's CORS rule allows GET only, so HEAD responses carry no CORS headers and every HEAD probe failed like a network error. The store, written by anndata 0.7, also has no root .zattrs, and detection needed encoding-type anndata there. fetchHead sends HEAD and, when it fails to fetch on an origin where no HEAD has succeeded, a one-byte Range GET in its place, returned as a bodiless response whose Content-Length is the size from Content-Range. An origin that answers it gets the GET from then on. Every HEAD call site uses it. A .zarr URL without root attributes is AnnData when obs/.zattrs is a dataframe.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Azimuth human fetus reference (
https://azimuth-vitessce.s3.us-west-2.amazonaws.com/human-fetus/vitessce_ref.zarr, 100,000 cells × 63,561 genes) showed "Request blocked". Two things stood in the way:.zattrs. Detection named a store AnnData only fromencoding-type: anndatain its root attributes.Fix
fetchHeadinsrc/data/hostResponses.ts. When HEAD fails to fetch on an origin where no HEAD has succeeded, it sends a one-byte Range GET instead, cancels the body, and returns a bodiless response whoseContent-Lengthis the file size fromContent-Range. An origin that answers the GET gets it from then on, without HEAD. Origins where HEAD has worked never fall back, so hosts that hide their 404s (EBI FTP) cost no extra requests.probeZarrRootContent: a.zarrURL with no root attributes (missing or{}) is AnnData whenobs/.zattrsis a dataframe. Only URLs ending in.zarrget this check, so other folders make no extra request.Testing
datasetFactory.test.ts: a pre-0.8 store on a host whose HEAD fails, from detection throughgenePanel(), and a folder not named.zarrthat does not requestobs/.zattrs.hostResponses.test.ts: size fromContent-Range, a 404 passed on, a host that ignores Range, both requests failing, no fallback on an origin that answered HEAD (including a HEAD that succeeds while another is in flight).annotation.l1, 172annotation.l2classes), and picking SOX2 colours the cells.