Skip to content

filesystem warehouse usability improvements - #2155

Draft
atravitz wants to merge 4 commits into
epic/execution_improvementsfrom
dev/warehouse_usability
Draft

filesystem warehouse usability improvements#2155
atravitz wants to merge 4 commits into
epic/execution_improvementsfrom
dev/warehouse_usability

Conversation

@atravitz

@atravitz atravitz commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

I think that creating a new filesystem warehouse and loading an existing one should have different guard rails.

otherwise, you:

  • can think you’re loading an existing warehouse with data in it, but instead a new, empty warehouse is created in a slightly different location
  • think you're loading a new warehouse, but you've just added tasks to an existing warehouse when you didn't mean to

LLM / AI generated code disclosure

LLMs or other AI-powered tools (beyond simple IDE use cases) were used in this contribution: yes / no
If yes, please provide details here:

Checklist

  • All new code is appropriately documented (user-facing code must have complete docstrings).
  • Added a news entry, or the changes are not user-facing.
  • Ran pre-commit: you can run pre-commit locally or comment on this PR with pre-commit.ci autofix.
  • Filled in the AI generated code disclosure.

Manual Tests: these are slow so don't need to be run every commit, only before merging and when relevant changes are made (generally at reviewer-discretion).

Developers certificate of origin

@ethanholz ethanholz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is great, a few minor notes but non-blocking

self.root_dir = pathlib.Path(root_dir)
if self.root_dir.is_dir() and not exist_okay:
raise ValueError(
"`root_dir` already exists. To load an existing Warehouse, use FileSystemWarehouse.load(`root_dir`)"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the error, but I am a little concerned about how it gets passed down to a user. If a user reruns a command using this under the hood will it make sense? Do we even care about that here?

@@ -1,5 +1,7 @@
# This code is part of OpenFE and is licensed under the MIT license.
# For details, see https://github.com/OpenFreeEnergy/gufe
from __future__ import annotations

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure you are aware but seems there are changes to this in Python 3.14 https://docs.python.org/3.14/whatsnew/3.14.html#from-future-import-annotations

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the link! I think this is fine in this instance, since it's pulling in Python 3.14 functionality.

@atravitz
atravitz force-pushed the epic/execution_improvements branch from a6fd672 to cc8edd5 Compare August 14, 2026 16:01
@atravitz

Copy link
Copy Markdown
Contributor Author

@ethanholz note that I have the same complaint about exorcist TaskStatusDB.from_filename() - it silently creates a new, empty, db if the path doesn't exist.

@atravitz
atravitz force-pushed the dev/warehouse_usability branch from fce955a to 9a4b3e0 Compare August 14, 2026 19:04
Comment thread src/openfe/storage/warehouse.py
@atravitz

Copy link
Copy Markdown
Contributor Author

If upon initial creation we create all the store top directories, then when pointing to a an existing warehouse, we could do a quick validation to make sure all the dirs exist.
Otherwise, if you point it to a generic non-warehouse directory, trying to access any of the stores would just create them in real time.

I think the broader question here is "what makes a directory a FileSystemWarehouse?" and how do we validate that.

@ethanholz

Copy link
Copy Markdown
Collaborator

If upon initial creation we create all the store top directories, then when pointing to a an existing warehouse, we could do a quick validation to make sure all the dirs exist. Otherwise, if you point it to a generic non-warehouse directory, trying to access any of the stores would just create them in real time.

I think the broader question here is "what makes a directory a FileSystemWarehouse?" and how do we validate that.

I think this is the right path forward!

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.57%. Comparing base (cc8edd5) to head (86ebbf7).

Files with missing lines Patch % Lines
src/openfe/orchestration/exorcist_utils.py 83.33% 3 Missing ⚠️
src/openfe/storage/warehouse.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@                       Coverage Diff                       @@
##           epic/execution_improvements    #2155      +/-   ##
===============================================================
- Coverage                        90.57%   90.57%   -0.01%     
===============================================================
  Files                              217      217              
  Lines                            21522    21549      +27     
===============================================================
+ Hits                             19494    19518      +24     
- Misses                            2028     2031       +3     
Flag Coverage Δ
fast-tests 90.57% <93.33%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants