acc: run bundle/deploy/mlops-stacks locally - #5829
Merged
Merged
Conversation
Collaborator
Integration test reportCommit: fd5863d
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 3 slowest tests (at least 2 minutes):
|
radakam
force-pushed
the
acc-local-mlops-stacks
branch
from
July 6, 2026 08:14
cc3d0b3 to
456aec7
Compare
radakam
marked this pull request as ready for review
July 6, 2026 09:46
denik
reviewed
Jul 6, 2026
denik
left a comment
denik
left a comment
Contributor
There was a problem hiding this comment.
can we add Taskfile entry that upgrades mlops repo to the latest from github?
Flip bundle/deploy/mlops-stacks to Local = true so it runs against the local testserver in addition to cloud. Local runs cannot git-clone mlops-stacks from GitHub, so vendor a pinned copy of the template under acceptance/bundle/deploy/mlops-stacks/template/ and use it when CLOUD_ENV is unset. A test.toml Repl normalizes the init command in output so local and cloud golden files stay unified.
The offline copy vendored for the local run only needs the files required to render this test's config and satisfy its assertions: the template machinery (schema, library, run_validations, update_layout), the asserted root README, databricks.yml, the three included resource YAMLs, and the four referenced notebooks (BatchInference, Train, ModelValidation, ModelDeployment). Drop the remaining 112 files (docs, images, CICD workflows, tests, feature-store/MLflow-recipe/monitoring variants, binary parquet fixtures). They are either FS/recipe-gated (never rendered for this config) or synced but unreferenced, so removing them leaves output.txt byte-identical and the test passing on both engines. The cloud run is unaffected; it still clones mlops-stacks from GitHub.
…-stacks template
The vendored MLOps Stacks template contained a directory named using a
backtick-quoted named-template invocation, which is an illegal Go module
file path (breaking TestFilePath and module packaging repo-wide). Rename
it to {{.input_project_name}}; for this test the project name is always
lowercase-alphanumeric, so the alphanumeric_underscore transform is a
no-op and rendered output is unchanged.
Also exempt the vendored template from the whitespace linter via
.wsignore, matching the existing "Developed elsewhere" convention, so
upstream files are kept as-is.
On Windows $TESTDIR expands with backslash separators, so the local template path was rendered as ...\mlops-stacks/template and the repl that maps it to the cloud template name "mlops-stacks" did not match, failing the acceptance test on Windows. Match either separator before "mlops-stacks".
Add `task update-mlops-stacks-template`, backed by tools/update_mlops_stacks_template.py, to upgrade the vendored template to the latest upstream commit. It refreshes only the currently-vendored files (preserving the earlier pruning), undoes the illegal-Go-module-path directory rename when mapping vendored paths back to upstream, and records the new commit in template.REVISION.
Condense the update tool's docstring and the Windows-separator repl comment to state the reason without the extra prose.
The name already says what it does; keep only the non-obvious "why" comments.
This reverts commit ea16f30; keep the docstring.
Rename the task and its script for consistency, and resolve every upstream source before copying any. The previous loop copied as it went and only reported missing files afterwards, so a restructured upstream left the vendored template half-refreshed and silently broke the test. The closing hint also pointed at `./task test-update-templates`, which only matches TestAccept/bundle/templates and never regenerates this test.
Restore the CLOUD_ENV split so local runs init from the vendored copy while cloud runs keep exercising `bundle init mlops-stacks` against GitHub. Both legs share one golden file, so they must render the same bundle. Bump the vendored copy from 1c87ae24 to upstream HEAD 9b387898, the revision main's cloud-only fix (#5929) already targets. Upstream restructured in databricks/mlops-stacks#226: the four notebooks moved out of their notebooks/ subdirs and the resources now reference ../requirements.txt, which joins the vendored set. The local run then reproduces main's golden output byte for byte, so output.txt is unchanged.
radakam
force-pushed
the
acc-local-mlops-stacks
branch
from
July 30, 2026 12:56
33d60c0 to
a73d191
Compare
The task follows upstream HEAD and rewrites a test input, so it is neither a test-update-* task (those regenerate golden output from the current inputs) nor a generate-* task (those must reproduce byte-for-byte in generate-check). Name it for what it does and move it out from between the test-update-* entries so neither grouping is implied. Rename the script to match.
The task rewrote a test input and left updating the golden output to the person running it. Run the update from the task instead, scoped to this one test rather than the full test-update.
The vendored set was hand-picked and the bump script read it back off disk, so it could not tell why a file was vendored and errored out asking for manual reconciliation whenever upstream moved one. The selection rule lived only in a commit message. Derive it instead: render upstream once with this test's config and keep the template machinery plus every project file that produced output. The bump wipes and re-copies, so renames and deletions upstream need no intervention. This vendors 48 files rather than 16; the extra ones are files the template renders that the cloud leg already gets from its clone. Output is unchanged. .wsignore covers the vendored tree with a `**` pattern, which glob does not apply to dot-prefixed names, so the whitespace fixer rewrote upstream content under .github/ that the bump would restore on every run. Match hidden paths.
janniklasrose
approved these changes
Aug 3, 2026
Co-authored-by: Jan N Rose <janniklas.rose@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Run
bundle/deploy/mlops-stackslocally in addition to cloud.bundle init mlops-stacksclones from GitHub, which local acceptance tests cannot do, so this checks in a pinned copy of the template underacceptance/bundle/deploy/mlops-stacks/template/. Local runs init from that copy; cloud runs still clone, keeping the real git-clone path covered. Atest.tomlrepl rewrites the local path back tobundle init mlops-stacksso both legs share one golden file.Only the files the template renders for this config are vendored (48 of 105, pinned to
9b387898intemplate.REVISION). That set is derived by rendering upstream, so./task bump-mlops-stacksmoves to upstream HEAD and regenerates the golden output without manual reconciliation.Two notes: upstream's project directory is named with a backtick template invocation, an illegal Go module path that breaks
TestFilePathrepo-wide, so it is vendored as{{.input_project_name}}(a no-op transform here)..wsignoreexempts the vendored tree, which neededinclude_hiddeninvalidate_whitespace.pyto reach its.github/files.