bootstrap: add reusable opt-in consumer setup

[tonyredondo]

Summary

• add reusable Bzlmod and WORKSPACE bootstrap primitives for Go and Python, with --config=test-optimization as the single user-facing opt-in
• make disabled Go instrumentation a structural no-op under the public test label, with no hidden payload, Bazel-metadata, or real Orchestrion targets
• keep disabled metadata and Orchestrion repositories deterministic and network-free
• build enabled Orchestrion with the Bazel-managed Go SDK instead of requiring host Go, honoring the consumer's configured Go module proxy
• preserve upgrade compatibility for existing static sync users while generated config-gated onboarding opts into enabled_by_env = True
• add the separate test_optimization_manifest_sync WORKSPACE rule and Bzlmod extension for invocation-scoped Go/Python monorepo onboarding
• consume a strict, fully expanded target/context manifest and generate one aggregate repository with target-to-context exports, narrow per-context/module labels, bundled doctor/uploader context, and an exact expected-target file
• preserve the existing static single-service and static multi-service APIs unchanged
• allow Go and Python macros to consume explicit generated labels so metadata changes invalidate only the affected service/module test actions
• keep target discovery and service derivation consumer-owned: this repository adds no committed target/service mapping, examples.bzl, Gazelle extension, inventory, or ownership gate
• support registry-backed rules_go v0.62.0 fork/profile materialization while retaining v0.60.0 as the default and v0.61.1 compatibility
• update first-party documentation and the Go/Python onboarding skills; other languages remain outside the automatic manifest flow

Behavioral contract

• Without --config=test-optimization, manifest sync ignores the internal manifest handoff, performs no metadata requests, and emits stable disabled exports.
• With the config, the consumer supplies one invocation-scoped manifest generated from the exact selected Go/Python labels.
• Reordering equivalent manifest input produces byte-identical generated output.
• Changing metadata for one service/module changes only that context's narrow Bazel inputs.
• Doctor validation uses the exact selected target set from the generated file.
• DD_TEST_OPTIMIZATION_SERVICES_MANIFEST is an internal command-to-Bazel handoff, not a public rollout switch.

Validation

The manifest/bootstrap implementation passed:

• core Starlark suite: 165 tests
• Go companion suite: 75 tests
• Python companion suite: 58 tests in both normal and hermetic modes
• Python tooling and full manifest-sync integration
• Unix mock-server integration
• full WORKSPACE and Bzlmod Go integrations
• same-output-root disabled-to-enabled transitions across maintained rules_go profiles v0.60.0, v0.61.1, and v0.62.0
• strict manifest rejection, deterministic output, Go/Python multi-context payloads, exact doctor targets, uploader dry-run enrichment, zero real uploads, and service/module cache isolation
• examples, Buildifier, Python syntax, workflow YAML, first-party Markdown links, and git diff --check
• the full Windows PowerShell mock-server/uploader integration harness
• the complete 50-check GitHub Actions matrix for current head 80ee9674de5a2d09d532ec40bdf0757d9ada2c25

The current head was also consumed by rules_test_optimization_tests head b1426b0c3289e110776d8fb188a8eba79662f24c. Local WORKSPACE and Bzlmod v0.62.0 E2E paths passed with metadata fetch, instrumented test execution, payload generation, doctor validation, and uploader dry-run enrichment.

On macOS, Bazel tests use --noexperimental_split_xml_generation because Bazel 8.5.1's experimental split-XML generator can segfault in generate-xml.sh.

Downstream

• Consumer fixtures: DataDog/rules_test_optimization_tests#95, pinned to this head.
• dd-source integration: ddoghq/dd-source#9412 implements invocation-scoped automatic target/service derivation without service BUILD-file edits; its Rule repin follows the consumer-fixture gate.

[datadog-prod-us1-4]

The orch_transition no longer writes @rules_go//go/private/orchestrion:enabled=true; the bool_flag defaults to False. Existing consumers who upgrade the module without re-running the bootstrap tool will silently build orch_go_test targets without Orchestrion instrumentation — tests still pass, so nothing surfaces the regression. All 22 adversarial scenarios (9 Go, 13 Python) passed; no unintended behavioral regressions found elsewhere.

_{📊 Validated against 22 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 14d4964 · What is Autotest? · Any feedback? Reach out in #autotest}