Migrate xcube development environments to Pixi

[forman]

Summary

Replace the current environment.yml-based development environments with Pixi across the xcube ecosystem.

The goal is to simplify environment setup, provide reproducible dependency resolution via lock files, and modernize the developer workflow while keeping the existing repository structure unchanged.

We should keep rtd-environment.yml since RTD specifically relies on it today. But the long-term plan is be to replace RTD anyway.
Should this take longer, we may define a pixi feature "docs".

Motivation

Today, each xcube repository provides an environment.yml that developers create using Conda or Mamba.

Pixi provides several advantages:

• reproducible environments through pixi.lock
• significantly simpler onboarding (pixi install)
• integrated task runner
• unified Conda + PyPI dependency management
• cross-platform lock files
• no need for developers to manually manage Conda environments

Proposed approach

Each repository should become an independent Pixi project. However, as a first step, only xcube should be migrated, plugins and companion libraries should be migrated separately. See #1239.

Instead of

environment.yml

each repository would contain

pyproject.toml
pixi.lock

with the Pixi configuration stored in pyproject.toml under tool.pixi.

The existing pyproject.toml already contains the package metadata, so no additional pixi.toml should be required:

pixi init --format pyproject
pixi import environment.yml
pixi install
pixi run pytest
git add pixi.lock
git remove environment.yml

Example repository layout:

xcube/
    pyproject.toml
    pixi.lock

xcube-stac/
    pyproject.toml
    pixi.lock

xcube-cmems/
    pyproject.toml
    pixi.lock

Developer workflow

Current:

mamba env create -f environment.yml
conda activate ...
pip install -e .
pytest

Proposed:

pixi install
pixi run tests

or

pixi shell

followed by the usual development commands.

Development model

This proposal does not suggest moving to a monorepo or a shared Pixi workspace.

Each repository remains independently developable and maintains its own lock file.

This means plugin development changes very little compared to today.

Developing xcube together with a plugin

Plugin repositories should normally depend on a released xcube version.

When simultaneously developing xcube and a plugin, developers should be able to temporarily use a local editable checkout of xcube (e.g. via a local path dependency) without committing those local paths to the repository.

[tool.pixi.pypi-dependencies]
xcube = { path = "../xcube", editable = true }
xcube-cmems = { path = ".", editable = true }

The exact workflow should be documented, but this proposal intentionally keeps local cross-repository development separate from the default development setup.

Migration plan

• migrate each environment.yml to Pixi
• store Pixi configuration in pyproject.toml
• commit pixi.lock
• remove environment.yml
• update contributor documentation
• optionally define common Pixi tasks (test, lint, format, docs, etc.)

Benefits

• simpler onboarding
• reproducible environments
• faster environment creation
• integrated task runner
• one project configuration file (pyproject.toml)
• consistent developer experience across all xcube repositories
• no change to the existing multi-repository development model

Recommended pixi task set

The following definitions allow for conveniently running the common, must-apply tasks for xcube and xcube plugins

• pixi run format run all code formatters
• pixi run checks run all code checkers
• pixi run tests run all tests

and also

• pixi run jl run jupyter-lab
• pixi run doc-serve run mkdocs serve
• pixi run doc-build run mkdocs build

Copy the following code block to the end of the pyproject.toml files of xcube and xcube plugins:

# pixi run ...

[tool.pixi.tasks]
# Top-level tasks
doc-serve = "mkdocs serve"
doc-build = "mkdocs build"
jl = "jupyter-lab"
tests = "pytest tests"
# The following are helpers for composite tasks that use `depends-on`
format-with-isort = "isort src tests"
format-with-ruff = "ruff format src tests"
check-with-ruff = "ruff check src"
check-with-mypy = "mypy src"
cov-base = "pytest --cov src/sen4cap_client --cov-report= --cov-append tests"
cov-report-html = "coverage html -d .cov-report && coverage report"
cov-report-xml = "coverage xml -o coverage.xml && coverage report"

# pixi run format

[tool.pixi.tasks.format]
depends-on = ["format-with-isort", "format-with-ruff"]

# pixi run checks

[tool.pixi.tasks.checks]
depends-on = ["check-with-ruff", "check-with-mypy"]

# pixi run coverage

[tool.pixi.tasks.coverage]
depends-on = ["cov-base", "cov-report-html"]

# pixi run coverage-ci

[tool.pixi.tasks.coverage-ci]
depends-on = ["cov-base", "cov-report-xml"]