Validate generated reference docs in CI #13451
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `doc` target now generates reference documentation files via `reference-help` and `check-doc` validates all generated files are committed. This replaces scattered unit tests that only validated 3 out of 4 files and had reliability issues. Previously, `mimir-flags-defaults.json` wasn't validated, leading to incorrect values being committed (e.g., 9223372036854776000 instead of 9223372036854775807 for max int64).
dimitarvdimitrov
requested review from
a team,
bubu11e,
vaxvms and
wilfriedroset
as code owners
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
aknuds1
approved these changes
Nov 10, 2025
The doc target depends on reference-help, so reference-help must also run in the container when BUILD_IN_CONTAINER=true.
reference-help should not run in the container target list because it would try to resolve its binary dependencies on the host. Instead, when doc runs in the container, it will call reference-help which will then build Linux binaries inside the container.
Split doc target into container and non-container versions. In container mode, doc enters the container with BUILD_IN_CONTAINER=false to avoid resolving reference-help dependencies on the host, which would build macOS binaries that can't run in the Linux container.
dimitarvdimitrov
added
the
changelog-not-needed
Split reference help validation from check-doc into its own target that always runs outside the container. This allows CI to validate reference documentation without needing the build container.
Contributor
Author
|
changed the approach and now we have |
Signed-off-by: Dimitar Dimitrov <[email protected]>
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the validation of reference help documentation by replacing Go-based tests with a Makefile-based check that runs in CI. The key motivation is to eliminate flaky tests while maintaining validation coverage for generated documentation files.
Key changes:
- Removed
TestConfigDescriptorIsUpToDateandTestHelpGo tests and replaced them with a newcheck-reference-helpMakefile target - Added the new check to the CI workflow to ensure reference documentation stays up to date
- Corrected max int64 values in gRPC keepalive configuration from incorrect values to the accurate
9223372036854775807
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/config-inspector/main_test.go | Removed the entire TestConfigDescriptorIsUpToDate test function, leaving only package declaration |
| cmd/mimir/main_test.go | Removed the TestHelp function that validated help output templates |
| Makefile | Added new check-reference-help target to validate reference documentation is up to date |
| .github/workflows/test-build-deploy.yml | Added step to run the new check-reference-help validation in CI |
| operations/mimir/mimir-flags-defaults.json | Corrected gRPC keepalive max connection values to accurate max int64 value |
| .github/workflows/flaky-tests.yml | Removed TestConfigDescriptorIsUpToDate from ignored tests list since the test no longer exists |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Dimitar Dimitrov <[email protected]>
alexweav
approved these changes
Nov 10, 2025
Contributor
alexweav
left a comment
alexweav
left a comment
There was a problem hiding this comment.
Seems reasonable, LGTM. I like the newer, split target approach more, make reference-help and make check-reference-help feels consistent
dimitarvdimitrov
deleted the
dimitar/build/validate-generated-reference-docs
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The check-doc CI step now validates generated markdown files, while a new check-reference-help step validates reference documentation files (help.txt.tmpl, help-all.txt.tmpl, config-descriptor.json, mimir-flags-defaults.json). Both checks always run outside the container.
Why
Previously, operations/mimir/mimir-flags-defaults.json wasn't validated in CI, leading to incorrect values being committed (e.g., 9223372036854776000 instead of 9223372036854775807 for max int64).
The reference-help target needs to run outside the container since it executes the compiled mimir binary and config-inspector tool directly. Splitting it into a separate check makes the separation clear and allows check-doc to continue running in the container for markdown validation.
Note
Adds a CI step and Makefile target to validate generated reference docs, corrects gRPC keepalive max int64 defaults, updates flaky-tests ignored list, and removes redundant help/config descriptor tests.
Check Reference Help Documentationstep in/.github/workflows/test-build-deploy.ymlrunningmake BUILD_IN_CONTAINER=false check-reference-help./.github/workflows/flaky-tests.ymlto only ignoreTestOurUpstreamTestCasesAreInSyncWithUpstream.check-reference-helptarget to verifycmd/mimir/help.txt.tmpl,cmd/mimir/help-all.txt.tmpl,cmd/mimir/config-descriptor.json, andoperations/mimir/mimir-flags-defaults.jsonare up to date.TestHelpincmd/mimir/main_test.go.tools/config-inspector/main_test.go(config descriptor up-to-date test).operations/mimir/mimir-flags-defaults.jsonforserver.grpc.keepalive.*defaults.Written by Cursor Bugbot for commit 32d5f93. This will update automatically on new commits. Configure here.