Skip to content

Epic: execution improvements - #2113

Draft
atravitz wants to merge 3 commits into
mainfrom
epic/execution_improvements
Draft

Epic: execution improvements#2113
atravitz wants to merge 3 commits into
mainfrom
epic/execution_improvements

Conversation

@atravitz

Copy link
Copy Markdown
Contributor

This PR covers the first section (non-breaking changes) of the Exorcist & Warehouse Integration Plan

LLM / AI generated code disclosure

LLMs or other AI-powered tools (beyond simple IDE use cases) were used in this contribution: no
If yes, please provide details here:

Checklist

  • All new code is appropriately documented (user-facing code must have complete docstrings).
  • Added a news entry, or the changes are not user-facing.
  • Ran pre-commit: you can run pre-commit locally or comment on this PR with pre-commit.ci autofix.
  • Filled in the AI generated code disclosure.

Manual Tests: these are slow so don't need to be run every commit, only before merging and when relevant changes are made (generally at reviewer-discretion).

Developers certificate of origin

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.66532% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.57%. Comparing base (ca92919) to head (cc8edd5).

Files with missing lines Patch % Lines
src/openfe/orchestration/__init__.py 88.18% 15 Missing ⚠️
src/openfe/storage/warehouse.py 86.91% 14 Missing ⚠️
src/openfecli/commands/worker.py 93.75% 4 Missing ⚠️
src/openfe/tests/orchestration/test_worker.py 98.42% 3 Missing ⚠️
src/openfecli/plan_alchemical_networks_utils.py 87.50% 3 Missing ⚠️
.../openfe/tests/orchestration/test_exorcist_utils.py 98.61% 2 Missing ⚠️
src/openfe/tests/orchestration/conftest.py 97.22% 1 Missing ⚠️
src/openfecli/commands/plan_rbfe_network.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2113      +/-   ##
==========================================
- Coverage   95.20%   90.57%   -4.63%     
==========================================
  Files         213      217       +4     
  Lines       21134    21522     +388     
==========================================
- Hits        20121    19494     -627     
- Misses       1013     2028    +1015     
Flag Coverage Δ
fast-tests 90.57% <95.66%> (?)
slow-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

atravitz and others added 3 commits August 14, 2026 09:01
* refactor: remove result_server references in favor of gufe ExternalStorage (#1632)

* feat: remove result_server references in favor of gufe external_storage

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>

* fix: remove extra external_store

* test: change from result_server to external_storage

* test: remove external_store reference

* test: change extension to item per class changes

* refactor: remove result server import

* chore: remove metadata store from resultclient

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* refactor: remove result server

* refactor: remove metadata store

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* refactor: rename to WarehouseBaseClass (#1660)

* refactor(warehouse): rename to warehouse

* refactor: rename _ResultContainerTest to _ContainerTest

* refactor: rename _Container instances to _DataContainer

* feat/refactor: add setup store (#1671)

* refactor: remove extra implementations

We may end up adding these back later but for the time being, we are
going to remove them to keep life easy.

* refactor: remove load_* and store_* functions

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>

* refactor: keys are being stored really strangely

* refactor(warehouse): huge refactor to simplify working with Warehouse

* refactor(warehouse): remove dead code

* fix: delete failed due to incorrect object

* test: add a test for the FileSystemWarehouse

* refactor: remove dead code

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* docs: update warehouse docstrings

* feat(warehouse): leverage keyed_chain for object dedup

* Update openfe/storage/warehouse.py

* import Literal

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* docs(warehouse): change from return, to raises

* refactor(warehouse): move _key_exists to exists

* chore(warehouse): remove extra todo

* chore(warehouse): remove _load_stream

* fix(warehouse): deduplicate objects on the filesystem

* chore(warehouse): add typehint to exists

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>
Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

* Temporarily build pooch from main w/ hotfix (#1806)

* build with pooch@main to see if hotfix works

* add link

* Add support for result tokenizables to warehouse (#1763)

* feat(warehouse): add result tokenizable store

* docs(warehouse): add docs on the result store

* chore: add property for result store

* fix: use the correct function signature for handling setup

---------

Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

* test: cleanup warehouse test to be more modular (#1809)

* fix dict type

* add news item

* run mypy CI on PRs into all branches (not just main) (#2112)

* ignore a couple type check violations for now

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Ethan Holz <ethan.holz@omsf.io>
* refactor: remove result_server references in favor of gufe ExternalStorage (#1632)

* feat: remove result_server references in favor of gufe external_storage

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>

* fix: remove extra external_store

* test: change from result_server to external_storage

* test: remove external_store reference

* test: change extension to item per class changes

* refactor: remove result server import

* chore: remove metadata store from resultclient

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* refactor: remove result server

* refactor: remove metadata store

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* refactor: rename to WarehouseBaseClass (#1660)

* refactor(warehouse): rename to warehouse

* refactor: rename _ResultContainerTest to _ContainerTest

* refactor: rename _Container instances to _DataContainer

* feat/refactor: add setup store (#1671)

* refactor: remove extra implementations

We may end up adding these back later but for the time being, we are
going to remove them to keep life easy.

* refactor: remove load_* and store_* functions

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>

* refactor: keys are being stored really strangely

* refactor(warehouse): huge refactor to simplify working with Warehouse

* refactor(warehouse): remove dead code

* fix: delete failed due to incorrect object

* test: add a test for the FileSystemWarehouse

* refactor: remove dead code

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* docs: update warehouse docstrings

* feat(warehouse): leverage keyed_chain for object dedup

* Update openfe/storage/warehouse.py

* import Literal

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* Update openfe/storage/warehouse.py

Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>

* docs(warehouse): change from return, to raises

* refactor(warehouse): move _key_exists to exists

* chore(warehouse): remove extra todo

* chore(warehouse): remove _load_stream

* fix(warehouse): deduplicate objects on the filesystem

* chore(warehouse): add typehint to exists

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>
Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

* Temporarily build pooch from main w/ hotfix (#1806)

* build with pooch@main to see if hotfix works

* add link

* Add support for result tokenizables to warehouse (#1763)

* feat(warehouse): add result tokenizable store

* docs(warehouse): add docs on the result store

* chore: add property for result store

* fix: use the correct function signature for handling setup

---------

Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>

* test: cleanup warehouse test to be more modular (#1809)

* feat: add warehouse primitives for handling protocol units

* feat: inital worker for exorcist

* test: add tests for warehouse

* fix: can now return protocol unit

* refactor: make things more consistent

* test: initial test setup for orchestration subpackage

* test: initial exorcist utility testing

* refactor: provide a root path to the exorcist DB

* test: inital worker testing

* feat: add shared_store

* feat: add better handling for CLI application

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>

* test: add new worker tests

* feat: add exorcist worker to CLI

* test: add for worker CLI command

* docs: add numpy docstrings

* feat: add support for planning an RBFE to a Warehouse

* fix: correct edge direction for task graph

* refactor: remove extra debugging from warehouse

* refactor: cleanup handling of tasks for worker

* refactor: fix issues in the CLI for running the worker

* Update src/openfe/orchestration/__init__.py

* fix dict type

* add news item

* ignore a couple type check violations for now

* fix type checking

* add exorcist to docs build

---------

Signed-off-by: Ethan Holz <ethan.holz@omsf.io>
Co-authored-by: Alyssa Travitz <31974495+atravitz@users.noreply.github.com>
Co-authored-by: Alyssa Travitz <alyssa.travitz@omsf.io>
* only use protocolunit as key

* add specific protocol dag store

* add get_protocol_dags

* fix protocol dag store deduplication order

* type checking and require a name

* default task db to use warehouse name

* update docstrings

* update type hint for python 3.12
@atravitz
atravitz force-pushed the epic/execution_improvements branch from a6fd672 to cc8edd5 Compare August 14, 2026 16:01
@github-actions

Copy link
Copy Markdown

🚨 API breaking changes detected! 🚨

View workflow run

Griffe output
$ griffe check "openfe" -s src --no-inspection --no-color --verbose -a origin/main
src/openfe/tests/storage/test_resultclient.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_metadatastore.py:0: <module>:
Public object was removed

src/openfe/tests/storage/test_resultserver.py:0: <module>:
Public object was removed

src/openfe/storage/resultclient.py:0: <module>:
Public object was removed

src/openfe/storage/resultserver.py:0: <module>:
Public object was removed

src/openfe/storage/metadatastore.py:0: <module>:
Public object was removed


$ griffe check "openfecli" -s src --no-inspection --no-color --verbose -a origin/main
src/openfecli/commands/plan_rbfe_network.py:121: plan_rbfe_network(warehouse):
Parameter was added as required


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants