GraphVIPER (Visibility and Image Parallel Execution Reduction) is a Dask based MapReduce package. It allows for mapping a dictionary of xarray.Datasets to Dask graph nodes followed by a reduce step.
GraphVIPER is in development and breaking API changes will happen.
The best place to start with GraphVIPER is doing the graph building tutorial.
git clone git@github.com:casangi/graphviper.git
cd graphviper
pip install -e '.[all]'
pre-commit installThe pre-commit install step sets up git hooks that automatically run code
formatting, linting, and import sorting (ruff; rules in [tool.ruff.lint] of pyproject.toml) and strip Jupyter notebook outputs
(nbstripout) on every commit. This keeps diffs clean and prevents large binary
outputs from bloating the repository. The same hooks run in CI (the
pre-commit workflow), so run them locally before pushing.
If pre-commit detects and makes any changes, those files will need to be
re-staged before committing. This is to allow the developer to inspect the
modified file before committing.