[BUG] Reject schema-invalid manifest identity and policy keys at their parser boundaries

[danielmeppiel]

fix(validation): reject schema-invalid manifest and policy inputs

TL;DR

Reject invalid apm.yml identity values and wrong-typed apm-policy.yml values at their native parser boundaries. Preserve unknown-policy-key warnings through discovery, malformed results, cache round trips, and apm policy status JSON/table output.

Note

This intentionally tightens accepted input. The CHANGELOG includes migration guidance.

Problem (WHY)

Fixes #2137.

• apm audit --ci exited 0 when name was empty/list-valued or version was an integer.
• apm policy status --check silently defaulted cache: [] and dependencies.allow: "".
• The parser produced unknown-key warnings, but discovery discarded them before status output.
• Cached policies also lost warning metadata after the first fetch.

Approach (WHAT)

Boundary	Change
Manifest	Require non-empty string name and version in APMPackage.from_apm_yml.
Policy	Validate mapping/list native types and string dependency patterns in validate_policy.
Discovery	Carry warnings on successful and malformed results and persist them in cache metadata.
Status	Include warnings in JSON and table reports.
Migration	Document quoted versions, non-empty identities, and policy mapping/list requirements.

Implementation (HOW)

• src/apm_cli/models/apm_package.py validates required identity values before parsing the rest of the manifest.
• src/apm_cli/policy/parser.py diagnoses unknown keys and rejects wrong native YAML container/item types.
• src/apm_cli/policy/discovery.py preserves warnings across local, URL, GitHub, ADO, inheritance, malformed, stale, and cached paths; cache schema v4 invalidates warning-less legacy cache entries.
• src/apm_cli/commands/policy.py exposes warnings in both renderers.
• Parser, discovery, and command tests lock down the regressions; docs and the packaged APM usage guide describe the stricter contract.

Diagram

Legend: invalid native inputs now stop at their owning parser, while valid policy warnings remain visible through discovery and rendering.

flowchart LR
    M[apm.yml] --> MV[Manifest identity validation]
    P[policy.yml] --> PV[Policy schema validation]
    MV -->|valid| A[Audit]
    MV -->|invalid| E[Nonzero diagnostic]
    PV -->|errors| E
    PV -->|valid with warnings| D[Policy discovery and cache]
    D --> S[Policy status JSON or table]
    classDef new stroke-dasharray: 5 5;
    class MV,PV,D,S new;

Loading

Trade-offs

• Validation stays in each native parser instead of introducing a universal schema layer.
• Unknown top-level policy keys remain warnings for forward compatibility; known wrong-typed values are errors.
• Cache schema v4 invalidates existing policy cache metadata once so warnings are reliable on later status runs.

Benefits

1. Invalid manifest identities now make apm audit --ci exit 1.
2. Wrong-typed policy values produce actionable schema diagnostics instead of defaults.
3. Unknown keys remain visible in fresh, malformed, and cached policy reports.
4. Valid string identities and valid policy files retain their existing behavior.

Validation

Regression tests were written first and failed on the base revision for the missing validations and warning propagation.

249 passed, 1 warning in 0.83s

Targeted suites:

• tests/test_apm_package_models.py::TestAPMPackage
• tests/unit/test_apm_package.py
• tests/unit/policy/test_parser.py
• tests/unit/policy/test_discovery.py
• tests/unit/commands/test_policy_status.py

Manual CLI evidence:

policy invalid types + typo: exit 1, error populated, warnings includes enforcment
manifest empty name + integer version: exit 1
manifest list-valued name: exit 1
manifest valid-pkg + 1.0.0: exit 0

Lint evidence:

ruff check: passed
ruff format --check: 1422 files already formatted
pylint R0801: 10.00/10
source guards: passed
auth-signal lint: clean

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	apm audit --ci rejects empty, list-valued, or non-string manifest identity	Portability by manifest, DevX	tests/test_apm_package_models.py::TestAPMPackage::test_from_apm_yml_rejects_invalid_identity (regression-trap for #2137)	unit
2	Policy status diagnoses typoed keys and wrong native value types	Governed by policy, DevX	tests/unit/policy/test_parser.py::TestValidatePolicy::test_typo_and_wrong_typed_native_values_are_diagnosed (regression-trap for #2137)	unit
3	Policy status JSON preserves warnings with schema errors and cache hits	Governed by policy, DevX	tests/unit/commands/test_policy_status.py::TestStatusPolicySourceOverride::test_json_preserves_warning_when_policy_has_schema_errors; tests/unit/policy/test_discovery.py::TestCacheReadWrite::test_policy_warnings_survive_cache_round_trip	unit

How to test

• Run uv run apm audit --ci --no-policy -f json with name: "" and version: 123; expect exit 1 and an identity diagnostic.
• Repeat with name: [a, b]; expect exit 1.
• Use name: valid-pkg and version: 1.0.0; expect exit 0.
• Run uv run apm policy status --check --policy-source policy.yml --json with the issue repro policy; expect exit 1, both type errors, and an unknown-key warning.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

Tightens validation at the native parser boundaries for apm.yml manifest identity and apm-policy.yml policy schema types, and threads unknown-key warnings through policy discovery, caching, and apm policy status output so they remain visible in both JSON and table renderers.

Changes:

• Enforce non-empty string name/version in APMPackage.from_apm_yml.
• Extend validate_policy to reject wrong-typed YAML containers/items while keeping unknown top-level keys as warnings.
• Preserve and persist policy warnings through discovery + cache and surface them in apm policy status output; add regression tests and docs/changelog guidance.

Show a summary per file

File	Description
src/apm_cli/models/apm_package.py	Rejects empty / non-string manifest identity fields early in from_apm_yml.
src/apm_cli/policy/parser.py	Adds schema-type validation (mappings/lists/items) and preserves unknown-key warnings even when raising validation errors.
src/apm_cli/policy/discovery.py	Carries warnings through fetch/discovery/cache paths and bumps cache schema version for metadata changes.
src/apm_cli/commands/policy.py	Includes warnings in the status report and renders them in table output.
tests/test_apm_package_models.py	Adds regression coverage for invalid manifest identity field types/empties.
tests/unit/policy/test_parser.py	Adds regression coverage for typo warnings + wrong-typed native YAML values being diagnosed.
tests/unit/policy/test_discovery.py	Adds regression coverage that warnings survive a cache round-trip.
tests/unit/commands/test_policy_status.py	Adds regression coverage that status JSON surfaces/preserves warnings on both valid and malformed policies.
docs/src/content/docs/reference/policy-schema.md	Documents warning surfacing in --json and the new “wrong native type => error” behavior.
docs/src/content/docs/concepts/package-anatomy.md	Documents that name/version must be non-empty strings and numeric versions should be quoted.
packages/apm-guide/.apm/skills/apm-usage/package-authoring.md	Updates packaged usage guide to reflect stricter manifest identity requirements.
packages/apm-guide/.apm/skills/apm-usage/governance.md	Updates packaged governance guidance on unknown-key warnings vs wrong-typed known fields.
CHANGELOG.md	Adds an Unreleased entry with migration guidance for the tightened validation.

Review details

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Low