Add multiplatform testing builds by Zeitsperre · Pull Request #438 · roocs/clisops

Zeitsperre · 2025-06-25T19:18:51Z

Pull Request Checklist:

This PR addresses an already opened issue (for bug fixes / features)
- This PR fixes issue Add GitHub CI tests for macOS, Windows and Anaconda #126
Tests for the changes have been added (for bug fixes / features)
Documentation has been added / updated (for bug fixes / features)
HISTORY.rst has been updated (with summary of main changes)

What kind of change does this PR introduce?:

Adds macOS builds to CI testing suite

Does this PR introduce a breaking change?:

No.

Other information:

The testing data is fetched currently from two different repositories. unix-style file locking is used when gathering these files, which isn't available on Windows. Additionally, xesmf is not available on Windows either (conda).

Will try seeing if tox builds could potentially work on Windows, otherwise I'll adjust the metadata to suggest that clisops doesn't support Windows.

coveralls · 2025-06-25T20:20:03Z

Pull Request Test Coverage Report for Build 15886172993

Details

1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.164%

Totals
Change from base Build 15885127333:	0.0%
Covered Lines:	3110
Relevant Lines:	3568

💛 - Coveralls

Zeitsperre · 2025-06-26T15:07:34Z

@cehbrecht Consider this PR a WIP; I need to get access to a Windows computer to see what's up, but essentially, there should be a simple way to load UNIX-style file paths on Windows and still have everything work as intended.

for more information, see https://pre-commit.ci

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

Copilot

Pull request overview

This PR extends the project’s CI pipeline toward multi-platform coverage (macOS + Windows alongside Linux) and updates the test harness/data-fetching utilities to better support running the test suite across different OS environments.

Changes:

Expand GitHub Actions test matrices to include macOS and Windows (PyPI/tox) and macOS (conda/micromamba), with per-OS caching.
Rework test-data gathering in tests/conftest.py and clisops/utils/testing.py (data registry handling, cache dir plumbing, worker coordination).
Update many tests and dataset-opening helpers to use xarray.open_mfdataset(..., data_vars="all", ...) to stabilize multi-file dataset behavior.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
`.github/workflows/main.yml`	Adds OS matrices and per-OS caching for tox/conda jobs.
`clisops/__version__.py`	Updates copyright year.
`clisops/project_utils.py`	Adjusts path handling for deducing projects, base dirs, ds-id mapping, and URL-to-path mapping.
`clisops/utils/dataset_utils.py`	Improves typing and opener kwargs; changes empty-input exception type.
`clisops/utils/testing.py`	Refactors test-data cache defaults/env handling, registry fetching, and adds custom Pooch downloader headers.
`tests/conftest.py`	Reworks session-level test-data gathering/copying and removes older base fixtures.
`tests/test_core_regrid.py`	Removes an unused fixture parameter.
`tests/test_core_subset.py`	Tweaks `open_mfdataset` options for multi-file datasets.
`tests/test_dataset_utils.py`	Adds `data_vars="all"` to multiple `open_mfdataset` calls.
`tests/test_file_namers.py`	Adds `data_vars="all"` to `open_mfdataset` calls used in naming tests.
`tests/test_ops_average.py`	Adds `data_vars="all"` and re-enables aux-variable test using portable paths.
`tests/test_ops_subset.py`	Makes dataset loader more explicit about decoding and uses portable file paths.
`tests/test_ops_xarray_mean.py`	Adds `data_vars="all"` when opening multi-file datasets.
`tests/test_output_utils.py`	Adds `data_vars="all"` for multi-file open in test helper.
`tests/test_project_utils.py`	Adjusts some project/path tests for portability and xarray opening behavior.

Comments suppressed due to low confidence (1)

.github/workflows/main.yml:201

This step is labeled macOS but is conditioned on ubuntu-latest, and it caches Linux paths (~/.cache/...) instead of the macOS ~/Library/Caches/... paths used elsewhere in this workflow. This makes macOS caching ineffective.

        if: matrix.os == 'ubuntu-latest'
        with:
          path: |
            ~/.cache/mini-esgf-data
            ~/.cache/xclim-testdata

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Zeitsperre · 2026-06-19T19:42:42Z

            if self._base_dir:
-                self._ds_id = ".".join(self._data_path.replace(self._base_dir, self._project).strip("/").split("/"))
+                relative_path = os.path.relpath(self._data_path, self._base_dir)
+                self._ds_id = ".".join(relative_path.split(os.sep))


@copilot Can you propose a fix here?

+        cmip5_base_dir = Path(project_utils.get_project_base_dir("cmip5"))
+        assert Path("/mnt/lustre/work/kd0956/CMIP5/data/cmip5").match(str(cmip5_base_dir))

-        c3s_cordex_base_dir = project_utils.get_project_base_dir("c3s-cordex")
-        assert c3s_cordex_base_dir == "/mnt/lustre/work/ik1017/C3SCORDEX/data/c3s-cordex"
+        c3s_cordex_base_dir = Path(project_utils.get_project_base_dir("c3s-cordex"))
+        assert Path("/mnt/lustre/work/ik1017/C3SCORDEX/data/c3s-cordex").match(str(c3s_cordex_base_dir))


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

… add-platforms

Zeitsperre · 2026-06-19T20:06:58Z

I've gone as far as I can without a Windows VM. Will try getting one soon to finish this.

Zeitsperre added 3 commits June 25, 2025 15:15

add other platforms

0b24d87

add a preload_test_data step for Windows builds

7725a16

do not build on Windows

f69ec05

Zeitsperre added 2 commits June 25, 2025 16:27

add Windows PyPI/tox build

de6a3c2

try using str instead of as_posix

e2a00d2

Zeitsperre self-assigned this Jun 26, 2025

Zeitsperre marked this pull request as draft June 30, 2025 19:59

Zeitsperre and others added 18 commits July 8, 2025 17:21

Merge branch 'master' into add-platforms

e14a48f

[pre-commit.ci] auto fixes from pre-commit.com hooks

e89ac9a

for more information, see https://pre-commit.ci

Merge branch 'master' into add-platforms

0202fb4

Merge branch 'master' into add-platforms

8c84a87

Merge branch 'master' into add-platforms

cae0055

testing data adjustments

20d86f0

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

add missing Path

accbfa4

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

adjust teardown

15b7e4d

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

adjustments

0605351

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

update registry mechanisms

ee007de

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

address warnings from xarray

6ff281c

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

add caching for Windows builds

d75a7e2

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

filepath fixes for multiplatform

22c23a1

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

fix typo

c5d6310

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

set env later

b87cfcb

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

use bash

ad38b85

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

small fixes

e47919a

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

fix cwd-dependent test

6fd1ed6

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

Zeitsperre requested a review from Copilot June 19, 2026 15:11

Copilot started reviewing on behalf of Zeitsperre June 19, 2026 15:11 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Zeitsperre and others added 3 commits June 19, 2026 19:11

Apply suggestions from code review

52aac5e

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

address review comments

15f8b85

Signed-off-by: Trevor James Smith <10819524+Zeitsperre@users.noreply.github.com>

Merge remote-tracking branch 'refs/remotes/origin/add-platforms' into…

dcb18d8

… add-platforms

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add multiplatform testing builds#438

Add multiplatform testing builds#438
Zeitsperre wants to merge 26 commits into
masterfrom
add-platforms

Zeitsperre commented Jun 25, 2025 •

edited

Loading

Uh oh!

coveralls commented Jun 25, 2025

Uh oh!

Zeitsperre commented Jun 26, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Zeitsperre Jun 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Zeitsperre commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Zeitsperre commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist:

What kind of change does this PR introduce?:

Does this PR introduce a breaking change?:

Other information:

Uh oh!

coveralls commented Jun 25, 2025

Pull Request Test Coverage Report for Build 15886172993

Details

💛 - Coveralls

Uh oh!

Zeitsperre commented Jun 26, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Zeitsperre Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Zeitsperre commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Zeitsperre commented Jun 25, 2025 •

edited

Loading