Port VISION (visionpy) into scviva-tools as scviva.tools.vision - #19
Open
oieretxezarreta wants to merge 31 commits into
Open
Port VISION (visionpy) into scviva-tools as scviva.tools.vision#19oieretxezarreta wants to merge 31 commits into
oieretxezarreta wants to merge 31 commits into
Conversation
…ctor harreman/vision into a dedicated Harreman-VISION integration module, and add the missing core dependencies.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…weights ownership tracking - _infer_obs_columns() now excludes non-categorical columns whose values are all unique (e.g. barcode/identifier columns), which previously crashed differential expression on real metadata. - compute_signature_scores() restores adata.uns["signature_varm_key"] after the permutation-null step instead of leaving it pointed at the transient "random_signatures" set, which broke integrate_vision_hotspot_results(). - VisionAnalysis now tracks whether it built adata.obsp["weights"] itself via a uns marker, so it correctly reuses an externally-built graph (e.g. from Harreman) even when X_pca is also present, instead of silently rebuilding and overwriting it, while still rebuilding on a second setup() call. - Add Harreman+VISION single-cell tutorial notebook and regression tests for the above.
for more information, see https://pre-commit.ci
- Rename numer -> numerator in _gearys_c.py (codespell flagged it as a typo of "number"). - Wrap an over-length inline comment in test_harreman_vision_integration.py onto its own line.
…e wrapper, custom varm_key, small-signature cluster labels - scviva.tools.vision no longer eagerly imports diffexp (and therefore scanpy) at module load time; rank_genes_groups is now exposed lazily via module __getattr__, so plain `import scviva` no longer pulls in the full VISION/scanpy stack. - HarremanAnalysis.vs.compute_vision_signatures() now has explicit defaults for norm_data_key/signature_varm_key/signature_names_uns_key instead of forwarding **kwargs straight through, fixing the no-arg legacy call path (previously TypeError'd on missing required positional args). - load_signatures() now accepts a varm_key parameter and writes to adata.raw.varm[varm_key] instead of the hardcoded adata.varm["signatures"] when use_raw=True, fixing both custom-varm_key support and the raw/non-raw gene-count mismatch crash. VisionAnalysis.load_signatures() now forwards its varm_key argument through. - generate_permutations_null()'s <=5-signature branch re-indexes cluster centers by integer position instead of signature name, so random background cluster labels line up with real-signature cluster labels (previously always mismatched for small signature sets). - Re-executed docs/tutorials/Harreman_Vision_singlecell_tutorial.ipynb end-to-end against these fixes. Addresses the remaining findings from Ori's Codex review and the regression tests added in bbb08c5.
- signatures_from_file() now delegates to load_signatures() instead of duplicating its body. - compute_vision_signatures() now delegates to analyze_vision(scores_only=True) instead of duplicating its compute_signatures_anndata call.
…isium colon dataset Per Ori: replace the tutorial's locally-stored hemato_counts.csv.gz/ hemato_covariates.txt.gz data and local Hallmark GMT signature file with the same publicly hosted dataset and signature source used in Visium_colon_Harreman_pipeline.ipynb, so the whole notebook runs from hosted downloads only: - Dataset: load_visium_mouse_colon_dataset() (Parigi et al. mouse colon 10x Visium, hash-verified pooch download) instead of local CSVs. - Signatures: KEGG metabolic pathway gene sets fetched from exampledata.scverse.org instead of a local .gmt file. - Harreman setup now builds the spatial neighbor graph on "spatial_unrolled" (per-sample via sample_key="cond") instead of a PCA embedding, matching this being a spatial Visium dataset. - Drop purely technical/bookkeeping obs columns (array/pixel coordinates, per-spot barcodes, a QC flag constant after in-tissue filtering) before running VISION differential expression, so it only compares signatures across the biologically meaningful metadata. - Added a spatial visualization of the top autocorrelated KEGG pathway in place of the old cell-type bar chart (this dataset has no cell-type labels). Also fixes a real bug this surfaced: compute_obs_df_scores() / _gearysc_for_dataframe() crashed with a ZeroDivisionError on any constant (zero-variance) numeric obs column (e.g. an "in_tissue" QC flag that's 1 everywhere after filtering), because its rank-transformed Geary's C denominator is 0. Constant columns are now skipped and reported as "no detectable autocorrelation" (c_prime=0, pval=1) instead of crashing, with a regression test. Also .gitignore the /data/ and /docs/tutorials/data/ pooch download cache directories these dataset loaders create.
…pynb The old name no longer fit: the notebook now runs on the spatial Visium colon dataset (not a non-spatial single-cell one). Update the docs/tutorials toctree entry to match.
… (no Harreman) Per Oier: this tutorial should demonstrate VISION on its own, not just via the ha.vs accessor wrapper, since scviva.tools.vision.VisionAnalysis needs nothing from Harreman. - Drop the HarremanAnalysis import/setup and the Hotspot gene-module section entirely; VisionAnalysis now builds and owns its own KNN graph directly on "spatial_unrolled" via va.setup(). - Repurpose the former Hotspot-section cells into signature loading/scoring (va.load_signatures() / va.compute_signatures()). - Replace the old Vision-Hotspot integration section (nothing to integrate against anymore) with va.compute_differential_expression() showing which KEGG pathways differ most between Day 0 and Day 14 regeneration timepoints directly from per-cell signature scores. - Takeaways now point to Visium_colon_Harreman_pipeline.ipynb for how these same VISION signatures intersect with Harreman's Hotspot-equivalent gene modules via ha.vs.integrate_vision_hotspot_results(), rather than duplicating that integration here. Re-executed end-to-end; all 123 tests still pass.
Reset execution_count/trailing-newline formatting left by an editor touch; no content or output changes.
…layout Splits the flat vision/ folder into preprocessing/ (filters, normalization), tools/ (knn, projections, microclusters, signature, diffexp), and phylo/ (split into parsimony.py and tree_clustering.py), matching how harreman/ is organized by domain concern. Adds the missing va.tl.cluster_cells_tree, va.tl.pool_matrix, and va.tl.pool_metadata accessor methods, and moves the log2p1 helper into _utils.py to break a circular import introduced by the preprocessing/tools split. Adds docs/user_guide/models/vision.md following harreman.md's structure, per Ori's code-review feedback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes BH-FDR index scrambling, tie-correction/NaN bugs in diffexp,KNN self-loop bias, and several silent use_raw/signature-splitting regressions surfaced by review. Moves vision plotting functions out of plotting/harreman into plotting/vision to mirror Harreman's tools/plotting split, and adds test overage for diffexp.py and the moved plotting module.
…n tutorial API coverage - DestVI_tutorial.ipynb: score KEGG signatures with VisionAnalysis on deconvolved spots, correlate against DestVI cell-type proportions (Spearman) with a heatmap, and test per-cell-type pathway enrichment (Mann-Whitney) using the existing proportion thresholds. Also includes the CPU-compatibility fixes needed to run end-to-end on this machine (GPU driver too old for the installed torch/CUDA build): hide the GPU, swap rapids_singlecell's neighbors call for scanpy's CPU equivalent. - Vision_tutorial.ipynb: demonstrate attach_signatures, compute_one_vs_one_de, get_genes_by_signature/get_gene_expression, and the standalone rank_genes_groups function; drop the RESOLVI section to keep the tutorial VISION-exclusive. - Add DEAD_CODE.md cataloguing dead/unused VISION and Harreman code candidates found during this work, for review before any removal. - Ignore docs/tutorials/test/, gseapy's default EnrichR output dir. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
# Conflicts: # docs/tutorials/DestVI_tutorial.ipynb
for more information, see https://pre-commit.ci
…tabase storage mismatch) and unseeded RNGs across Harreman's hotspot, cell-communication, and KNN modules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/scviva/tools/vision/, following the structural conventions ofscviva.tools.harreman(aVisionAnalysisfacade class mirroringHarremanAnalysis, step methods with prerequisite gating,_constants.py/_results.py).harreman/vision/: removes the duplicated partial VISION port that lived there, and moves the actual Harreman↔VISION integration logic (integrate_vision_hotspot_results+ helpers) out ofhotspot/modules.pyintoharreman/vision/_integration.py, which now calls the canonicalscviva.tools.vision.signaturefunctions instead of a local copy.igraph,joblib,statsmodelspromoted to base deps; newvisionextra forbiopython, used only by optional PhyloVision tree features).filters→_normalization→projections→filters) introduced by consolidating a shared helper, by extracting it into_utils.py.Test plan
ruff check/ruff formatclean across the fullsrc/treepy_compileclean across the fullsrc/treescviva→scviva.tools.vision.VisionAnalysis→scviva.tools.harreman.HarremanAnalysis→scviva.tools.harreman.vision.integrate_vision_hotspot_resultssetup()→load_signatures()→compute_signatures()→compute_differential_expression()→results