Skip to content

Open AnnData Zarr from GET-only hosts and pre-0.8 stores - #170

Merged
prete merged 3 commits into
mainfrom
claude/get-only-anndata-zarr
Nov 18, 2026
Merged

prete merged 3 commits into
mainfrom
claude/get-only-anndata-zarr

Conversation

@prete

@prete prete commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Problem

The Azimuth human fetus reference (https://azimuth-vitessce.s3.us-west-2.amazonaws.com/human-fetus/vitessce_ref.zarr, 100,000 cells × 63,561 genes) showed "Request blocked". Two things stood in the way:

  • The bucket's CORS rule allows GET only. S3 answers HEAD without CORS headers, so the browser reports every HEAD probe as a network failure, and detection sends HEAD first.
  • The store was written by anndata 0.7, which wrote no root .zattrs. Detection named a store AnnData only from encoding-type: anndata in its root attributes.

Fix

  • New fetchHead in src/data/hostResponses.ts. When HEAD fails to fetch on an origin where no HEAD has succeeded, it sends a one-byte Range GET instead, cancels the body, and returns a bodiless response whose Content-Length is the file size from Content-Range. An origin that answers the GET gets it from then on, without HEAD. Origins where HEAD has worked never fall back, so hosts that hide their 404s (EBI FTP) cost no extra requests.
  • Every HEAD call site uses it: detection probes, H5AD size checks, zip directories, Visium HD clusters, segmentations and readers, Feature Slice and Xenium matrix budgets, and Diagnostics.
  • probeZarrRootContent: a .zarr URL with no root attributes (missing or {}) is AnnData when obs/.zattrs is a dataframe. Only URLs ending in .zarr get this check, so other folders make no extra request.

Testing

  • New tests in datasetFactory.test.ts: a pre-0.8 store on a host whose HEAD fails, from detection through genePanel(), and a folder not named .zarr that does not request obs/.zattrs.
  • New tests in hostResponses.test.ts: size from Content-Range, a 404 passed on, a host that ignores Range, both requests failing, no fallback on an origin that answered HEAD (including a HEAD that succeeds while another is in flight).
  • Full suite passed three runs in a row (2,745 tests), plus lint and typecheck. Two earlier back-to-back runs had 1–6 failures in the timing-sensitive App UI tests, which don't use the changed code.
  • In the app (Playwright, real bucket): the store opens in about 8 s with 74 requests, shows the UMAP and 6 obs columns (77 annotation.l1, 172 annotation.l2 classes), and picking SOX2 colours the cells.

The Azimuth human fetus reference (azimuth-vitessce S3) showed "Request
blocked": the bucket's CORS rule allows GET only, so HEAD responses carry
no CORS headers and every HEAD probe failed like a network error. The
store, written by anndata 0.7, also has no root .zattrs, and detection
needed encoding-type anndata there.

fetchHead sends HEAD and, when it fails to fetch on an origin where no
HEAD has succeeded, a one-byte Range GET in its place, returned as a
bodiless response whose Content-Length is the size from Content-Range.
An origin that answers it gets the GET from then on. Every HEAD call
site uses it. A .zarr URL without root attributes is AnnData when
obs/.zattrs is a dataframe.
@prete
prete merged commit c35b87e into main Nov 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant