Skip to content

Commit 91d25c2

Browse files
mnriemCopilot
andauthored
feat(bundles): reconcile exact component pins and version conflicts (#4844)
* feat(bundles): reconcile exact component pins and version conflicts Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix selected bundle release installation and catalog outage validation Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix catalog validation precedence and selected workflow paths Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix catalog redirect and encoding validation Reject unsafe extension and preset catalog redirects during exact-ID lookups and classify malformed extension encodings. Cover stacked and legacy fetch paths with before-and-after regressions. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Validate deeply nested workflow and step catalogs Classify JSON recursion during parsing or cache writes as malformed catalog data, recover from poisoned caches, and keep non-fetch step catalog errors blocking exact-ID lookup. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Align bundled preset validation with installation Reject unsourced bundled preset pins that the installer cannot satisfy, classify extension HTTP protocol failures as unreachable catalogs, and make deep-cache regressions deterministic across Python versions. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reject missing catalog collections and bundled extension pin mismatches Require workflow and step collections in fetched or cached catalogs before exact-ID lookup may fall through. Align bundle validation with bundled-first extension installs while retaining explicit catalog sources and matching installed versions. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reject duplicate IDs in list-form workflow catalogs Treat duplicate workflow IDs within one list-form catalog as invalid during exact-ID lookup and search, while retaining priority-based resolution across distinct sources. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Report partial catalog outages during targeted lookup Preserve uncertainty when an exact bundle reference is absent from readable catalogs but another configured catalog could not be fetched. Retain winning-source and untargeted search behavior; cover all four catalog families and CLI validation. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Persist all installed bundle requirements for conflict checks Keep independently installed component pins separate from bundle-owned contributions. Reject incompatible later installs, retain requirements on remove, and preserve legacy records' known pins. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Protect other bundles' pins during unpinned installs Preserve ownership attribution while consulting all recorded pinned requirements in preflight. Reject unpinned installs or refreshes that may replace a required version and allow no-op sharing of a matching installation. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Describe bundle overlaps as shared requirements Avoid attributing ownership when another bundle only requires an independently installed component. Cover conflict reports and bundle info output for required-only records. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reject lower-priority catalog matches after source outages Preserve precedence for exact component lookups when an earlier catalog could not be fetched. Keep broad search available and warn when bundle validation cannot verify the winning source. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Protect shared bundle install requirements Reject sharing or refreshing a component when another bundle requires different source or preset priority/strategy. Check required-only records before any primitive mutation and retain compatible sharing. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Transfer ownership when bundle contributions remain needed Keep bundle-installed components attributed after removal or update drops the sole contributor while another bundle still requires them. Preserve independent ownership and cover sequential removal and update cleanup. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Classify malformed extension and step catalogs consistently Treat deeply nested extension JSON as catalog validation failures across network and cache paths, and classify duplicate step IDs as malformed catalog data. Cover targeted and discovery lookups with regressions. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reject non-string IDs in workflow catalogs Classify malformed list-form workflow IDs as catalog validation failures before duplicate detection or exact-version resolution. Cover targeted and search lookups for unhashable, numeric, boolean, and null IDs. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reject ambiguous bundle sources and malformed step IDs Fail closed when an explicit bundle component source names multiple active catalogs, including validation and direct primitive installation. Reject non-string list-form step IDs and update the former coercion regression. Simulate JSON parser recursion deterministically across Python versions so CI tests the exception contract rather than interpreter-specific depth limits. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Count required components in bundle list output Report the complete required component set even when every component was already installed independently. Preserve legacy record fallback and JSON provenance, document unversioned catalog pin behavior, and make unknown-step reference testing independent of network availability. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(bundle): confine exact-pin resolution to bundler Keep strict catalog precedence and selected historical releases within bundle validation and installation. Restore general catalog behavior to upstream main while preserving workflow and step preselected installer hooks. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(bundle): validate pinned current release metadata Reject malformed workflow and step install URLs and checksums during online bundle validation, including the advertised current release. Preserve legacy workflow digest compatibility and cover both valid and invalid pins. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(bundle): reject unsafe pinned extension URLs Validate selected current and historical extension release URLs during online bundle checks so invalid references fail before installation. Add regression coverage and keep catalog fixtures installable. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(bundle): validate pinned current component metadata Reject invalid current preset URLs and malformed declared extension/preset digests during online exact-pin validation. Keep omitted digests and unversioned legacy entries compatible; cover failing and valid cases. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * fix(bundle): reject conflicting catalog entry IDs Reject declared IDs that disagree with mapping keys before bundle release selection. Cover all four component catalogs, preserve matching or omitted IDs, and order test imports consistently. Assisted-by: GitHub Copilot (model: GPT-6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 381ce3b commit 91d25c2

27 files changed

Lines changed: 3313 additions & 265 deletions

‎docs/reference/bundles.md‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Bundles compose existing Spec Kit components — extensions, presets, workflows, and steps — into a single, versioned, installable unit. Where extensions and presets are primitives, a bundle is a curated stack that declares everything a team or role needs and installs it in one step through each component's own machinery. Bundles add no new runtime behavior of their own: they are a distribution and composition layer over the primitives you already use.
44

5-
A bundle is described by a `bundle.yml` manifest and is discovered through the same catalog stack as other components. Installing a bundle resolves its declared components against pinned versions, checks for the single cross-bundle conflict point (the active integration), and applies each component idempotently with full provenance tracking so it can be cleanly removed or refreshed later.
5+
A bundle is described by a `bundle.yml` manifest and is discovered through the same catalog stack as other components. Installing a bundle resolves its declared components against pinned versions, checks active-integration and shared-component-version conflicts, and applies each component idempotently with full provenance tracking so it can be cleanly removed or refreshed later.
66

77
For a concrete starting point, see the
88
[example bundle manifests](https://github.com/github/spec-kit/tree/main/examples/bundles)
@@ -55,7 +55,7 @@ specify bundle info <bundle_id> --versions
5555
| `--json` | Emit machine-readable JSON |
5656
| `--versions` | List the current and historical catalog releases |
5757

58-
Shows full metadata for a bundle along with the **fully expanded component set** it installs — every extension, preset, step, and workflow with its pinned version, plus preset priority and strategy. The output also includes a trust indicator (`verified` vs `community`) so you can judge trust before installing. This preview is the same plan `install` applies, so you can see exactly what will be added before committing. Foreseeable overlaps with components already provided by installed bundles are surfaced here as well.
58+
Shows full metadata for a bundle along with the **fully expanded component set** it installs — every extension, preset, step, and workflow with its pinned version, plus preset priority and strategy. The output also includes a trust indicator (`verified` vs `community`) so you can judge trust before installing. This preview is the same plan `install` applies, so you can see exactly what will be added before committing. Foreseeable overlaps and version conflicts with components already required by installed bundles are surfaced here as well; a bundle can require a component without having installed it.
5959

6060
Use `specify bundle info <bundle_id> --versions` to inspect the catalog history without downloading a bundle manifest. With `--json`, it emits the bundle ID, available versions, current version, source, and install policy.
6161

@@ -74,7 +74,9 @@ specify bundle install <bundle_id | path>
7474

7575
Installs a bundle's full component set through each primitive's machinery. The argument may be a catalog bundle id, or a local path to a built `.zip` artifact, a bundle directory, or a `bundle.yml` file; local sources install directly without consulting the catalog stack.
7676

77-
If the current directory is not yet a Spec Kit project, `install` initializes one first so a fresh checkout reaches a working state in a single command. `--integration` selects the integration when initializing a new project, and confirms the target when a bundle pins a specific integration but the project's active integration can't be determined (missing or unreadable `.specify/integration.json`). It does **not** override an already-initialized project's active integration: if a bundle targets a different integration than the project's, install aborts with no changes. Integration-agnostic bundles inherit the project's active integration. Without `--refresh`, installation is idempotent — components already present are skipped. Components installed outside any bundle are skipped and never adopted, so their installed version must match the manifest pin; if it doesn't, or can't be read, install and refresh stop before changing anything and name the component, so you can remove it or install the pinned version yourself. On failure, no provenance record is written (a failed install records nothing), and the components installed during that run are removed on a best-effort basis — removal errors are swallowed, so partial on-disk state may remain.
77+
If the current directory is not yet a Spec Kit project, `install` initializes one first so a fresh checkout reaches a working state in a single command. `--integration` selects the integration when initializing a new project, and confirms the target when a bundle pins a specific integration but the project's active integration can't be determined (missing or unreadable `.specify/integration.json`). It does **not** override an already-initialized project's active integration: if a bundle targets a different integration than the project's, install aborts with no changes. Integration-agnostic bundles inherit the project's active integration. Without `--refresh`, installation is idempotent — already-installed components matching their pins are skipped. A missing or different installed version cannot satisfy a pin. Only `--refresh` can repair a drifted version owned exclusively by this bundle; independently installed components are skipped and never adopted or replaced. On failure, no provenance record is written (a failed install records nothing), and the components installed during that run are removed on a best-effort basis — removal errors are swallowed, so partial on-disk state may remain.
78+
79+
Installed-bundle records store required component pins separately from contributed components. A compatible independently installed component satisfies a bundle's requirement but remains independently owned: its pin still blocks a later bundle from installing an incompatible version, while removing the bundle does not uninstall it. An unpinned install or refresh cannot replace a component another bundle requires at a pinned version; sharing an already installed matching version without refreshing remains allowed. Shared bundles must also agree on component source and, for presets, priority and strategy before installing or refreshing the shared component. Older records without the requirement list retain their known contributed pins; reinstall or refresh the bundle to record its full requirements.
7880

7981
A normal install rejects a change to an already-recorded bundle's version or owned component metadata (version, source, preset priority, or strategy), including removal of an owned component. This applies even if a local manifest keeps the same bundle version. Reordering unchanged components or adding new components does not require refresh. To apply changes to a local bundle without adding it to a catalog, pass the revised source with `--refresh`:
8082

@@ -129,17 +131,23 @@ specify bundle update [<bundle_id>]
129131

130132
Re-resolves a bundle and **refreshes** its components through each primitive's update path, bringing already-installed components up to the bundle's newly pinned versions while preserving primitive-level overrides (such as preset priority). Provide a bundle id, or use `--all` to update everything installed.
131133

132-
**Pinned catalog releases.** An extension or preset pinned to a version other than the one its catalog currently advertises installs that exact release when the winning catalog entry lists it under `releases`, using that release's own download URL and SHA-256 digest. The downloaded archive must declare the pinned ID and version. If the winning catalog entry has no release for the pinned version, install stops with an error rather than substituting the advertised release or falling through to a lower-priority catalog. Workflows and components bundled with Spec Kit still require the pin to match the version they resolve to.
134+
**Pinned catalog releases.** Extensions, presets, workflows, and steps with version pins select that exact release from the highest-priority active catalog entry. Historical releases must be advertised under `releases` with their own artifact URL and SHA-256 digest; the bundler never guesses an old URL from the current one or falls through to another catalog. Legacy extension and preset entries that advertise no version are an exception: the bundler uses the winning entry as-is and cannot enforce its pin or verify the installed archive's version. Publish a versioned entry to make that pin enforceable. The primitive installer uses the selected workflow or step record without re-reading the catalog, and verifies downloaded archive or workflow/step metadata against that release. A pinned workflow that ships with Spec Kit uses its bundled copy when the version matches, and the catalog release when it differs and network access is allowed. A step without a pin installs the current catalog release.
135+
136+
Without an explicit `source`, bundled extensions and presets take precedence over catalog releases. Validation rejects a pin that differs from the bundled version even when a matching catalog release exists; specify the winning catalog as `source` to opt into its release.
137+
138+
> **One installed version per component ID.** Bundles sharing a component must agree on its pinned version. A different or unknown pin from another bundle is rejected before installation, including during `bundle update`; refreshing one bundle cannot replace a version required by another.
139+
140+
A bundle may list a component ID only once per kind; duplicate references in the same manifest are invalid, even if their pins agree.
133141

134-
> **Pin enforcement is install-time only.** Idempotency checks are id-based, not version-aware: a component owned by a bundle that is already present is skipped during `install` without comparing its on-disk version to the manifest pin. Version pins are therefore guaranteed to be applied only when the bundler actually installs a component for the first time or refreshes it. Run `specify bundle update <bundle_id>` for catalog bundles or `specify bundle install <path> --refresh` for local sources to re-apply owned components at their pinned versions.
142+
An optional `source` in a `provides.<kind>` reference names the expected winning component catalog (as displayed by `specify <kind> catalog list`, or `specify workflow step catalog list` for steps). It is **not** an artifact URL and does not override catalog priority or install policy. When specified, the component is verified against that catalog even if already installed, rather than resolved from a Spec Kit-bundled copy; a different winning catalog or a discovery-only source prevents installation. If multiple active catalogs share the named source, bundle validation and installation reject it as ambiguous; give each catalog a unique name. Verifying an explicit source requires network access. Direct primitive `--from` URLs are a separate, explicitly requested route.
135143

136144
## Remove a Bundle
137145

138146
```bash
139147
specify bundle remove <bundle_id>
140148
```
141149

142-
Uninstalls only the components this bundle contributed, leaving any component that another installed bundle still needs in place (no collateral removals).
150+
Uninstalls only the components this bundle contributed, leaving any component that another installed bundle still needs in place (no collateral removals). If no other bundle owns a retained component, its contribution is transferred to a remaining requiring bundle so removing that bundle later can clean it up. Independently installed components are never adopted.
143151

144152
## List Installed Bundles
145153

@@ -151,7 +159,7 @@ specify bundle list
151159
| -------- | ---------------------------- |
152160
| `--json` | Emit machine-readable JSON |
153161

154-
Lists the bundles installed in the project with their versions, component counts, and install timestamps.
162+
Lists the bundles installed in the project with their versions, required component counts (including components supplied independently), and install timestamps.
155163

156164
## Initialize a Project with a Bundle
157165

@@ -177,7 +185,7 @@ specify bundle validate
177185
| `--path` | Bundle directory or `bundle.yml` (default: current directory) |
178186
| `--offline` | Verify references against bundled/installed components only |
179187

180-
Reports whether a `bundle.yml` is well-formed and whether every declared component reference resolves. References are checked against bundled components, the project's installed components, and — when online — the active catalogs. Validation fails only when a reference is definitively absent everywhere it could be checked: that is, when an active catalog is reachable and confirms the component is missing. References that cannot be verified — because validation is offline, or because a catalog is unreachable — are downgraded to warnings so authoring can continue, rather than failing the run.
188+
Reports whether a `bundle.yml` is well-formed and whether every declared component reference resolves at its pinned version. References are checked against matching bundled or installed components and — when online — the exact release in the winning install-allowed catalog. An explicit `source` is verified against the winning catalog instead of resolving locally. Missing releases, mismatched sources, discovery-only sources, and malformed catalog metadata fail validation; references that cannot be checked offline or because a catalog is unreachable produce warnings. If a higher-priority catalog is unreachable before a match is found, the lookup remains unverified; a lower-priority match cannot settle it. As with installation, legacy extension and preset entries advertising no version are accepted without verifying their pins.
181189

182190
## Build a Bundle Artifact
183191

‎src/specify_cli/bundles/_commands.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ def _bundle_overlaps(project_root: Path, manifest, *, offline: bool) -> list[str
136136
active_integration(project_root),
137137
load_records(project_root),
138138
)
139-
return list(report.overlaps)
139+
return [*report.overlaps, *report.version_clashes]
140140
except BundlerError:
141141
return []
142142

‎src/specify_cli/bundles/adapters.py‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@
2222
from .._assets import _locate_core_pack, _repo_root
2323
from .._download_security import MAX_JSON_CATALOG_BYTES, read_response_limited
2424
from . import BundlerError
25-
from .yamlio import loads_json
2625
from .catalogs import CatalogSource
2726
from .manifest import ComponentRef
27+
from .yamlio import loads_json
2828

2929
COMMUNITY_CATALOG_URL = (
3030
"https://raw.githubusercontent.com/github/spec-kit/main/"
@@ -330,6 +330,30 @@ def installed_version(
330330
manager = self._manager_for(component, project_root)
331331
return manager.installed_version(component)
332332

333+
def validate_source(self, project_root: Path, component: ComponentRef) -> None:
334+
if not self._allow_network:
335+
raise BundlerError(
336+
f"Cannot verify catalog source '{component.source}' for "
337+
f"{component.kind[:-1]} '{component.id}' offline; "
338+
"re-run without --offline."
339+
)
340+
from .references import _resolved_in_catalog
341+
342+
result = _resolved_in_catalog(project_root, component)
343+
if result is True:
344+
return
345+
if result is None:
346+
detail = "catalog unreachable"
347+
elif result is False:
348+
detail = "the winning install-allowed catalog has no matching release"
349+
else:
350+
detail = result
351+
raise BundlerError(
352+
f"Cannot verify {component.kind[:-1]} '{component.id}' at "
353+
f"{component.version or 'current'} from catalog "
354+
f"'{component.source}': {detail}."
355+
)
356+
333357
def install(self, project_root: Path, component: ComponentRef) -> None:
334358
manager = self._manager_for(component, project_root)
335359
manager.install(component)

‎src/specify_cli/bundles/command_list.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,6 @@ def bundle_list(
4040
console.print(
4141
f" [bold]{_escape_markup(str(record.bundle_id))}[/bold] "
4242
f"v{_escape_markup(str(record.version))} "
43-
f"[dim]({len(record.contributed_components)} components, "
43+
f"[dim]({len(record.required_components)} components, "
4444
f"installed {_escape_markup(str(record.installed_at))})[/dim]"
4545
)

0 commit comments

Comments
 (0)