feat(codegen): discover and register decentralized extended subtypes into open unions

[gsmith85]

Working...

Summary

Implements Task 8 (PR 3c) of the two-layer ucp-schema code-generation pipeline: cross-file decentralized subtype discovery and registration (register_extended_subtypes in src/codegen/reify.rs), stacked on #87.

Changes

• Decentralized Extended Subtype Discovery & Registration (src/codegen/reify.rs):
  • Scans $defs after lower_conditional_unions for schemas whose allOf references a lowered union <ParentUnion> (anyOf/oneOf) and defines a const (or single-value enum) tag on <ParentUnion>'s discriminator property (either at top-level properties.<disc> or in an inline allOf branch).
  • Folds inline object allOf branches into the subtype, merges <ParentUnion>Base (or <ParentUnion>) properties and required into the subtype, removes {"$ref": "#/$defs/<ParentUnion>"} from allOf, and strips "default" from the subtype's discriminator property (invariant OU-4).
  • Appends the discovered subtype's discriminator tag to <ParentUnion>Base.properties.<disc>.not.enum (invariant OU-2) and inserts {"$ref": "#/$defs/<Subtype>"} into <ParentUnion>.anyOf immediately before the trailing {"$ref": "#/$defs/<ParentUnion>Base"} fallback branch (invariant OU-1).
  • Leaves non-union base object hierarchies (PaymentInstrument -> CardPaymentInstrument) untouched so flatten_single_object_allof flattens them directly.

Stacked PR Series Roadmap

1. PR 1a (feat(codegen): add identifier naming, UCP keyword stripping, and $defs ref rewriting #81): Identifier qualification, UCP keyword stripping, $ref rewriting (src/codegen/normalizer.rs)
2. PR 1b (feat(codegen): add directional schema slicing and $ref alignment #82): Directional schema slicing & $ref alignment (src/codegen/normalizer.rs)
3. PR 1c (feat(codegen): add capability reachability, composition, and generate_types() API #83): Capability/extension reachability, composition & generate_types() library API (src/codegen/{mod,reachability,hoist,compose}.rs)
4. PR 1d (feat(cli): add generate-types subcommand #84): CLI generate-types subcommand (src/bin/ucp-schema.rs, tests/generate_types_test.rs)
5. PR 2 (feat(codegen): add self-contained .well-known/ucp profile scoping for generate-types #85): Self-contained .well-known/ucp --profile scoping (src/codegen/profile.rs)
6. PR 3a (feat(codegen): add AST shape normalizers and inline conditional variant hoisting #86): Codegen AST shape normalizers & inline conditional variant hoisting (src/codegen/normalizer.rs, src/codegen/reify.rs)
7. PR 3b (feat(codegen): lower open string discriminator unions into ordered anyOf with <Union>Base #87): Open string discriminator union lowering (<Union>Base) (src/codegen/reify.rs)
8. PR 3c (this PR): Cross-file extended subtype discovery & registration (src/codegen/reify.rs)
9. PR 4a (python-sdk#101): Python SDK direct JSON Schema codegen (generate_models.sh, postprocess_models.py, src/ucp_sdk/models/)
10. PR 4b (js-sdk#87): JS/TS SDK direct JSON Schema codegen (generate_models.sh, scripts/generate-zod-from-types.mjs, src/spec_generated.ts)
11. PR 5 (feat(openapi): add profile-driven OpenAPI 3.1 service and type binding (generate-openapi) #89): OpenAPI 3.1 REST service & type binding (ucp-schema generate-openapi, src/openapi/mod.rs)

Test Plan

• cargo fmt --check
• cargo test --all-targets
• cargo clippy --all-targets -- -D warnings

[damaz91]

Clean implementation of Stage 7 decentralized subtype discovery (register_extended_subtypes) and Stage 3 extension overlay property relaxation (relax_extension_property_required)! Left a few inline notes on edge cases.

[damaz91]

Schema transition objects in ucp_request / ucp_response:

relax_extension_property_required handles string shorthand ("ucp_request": "required") and per-operation strings ("ucp_request": {"complete": "required"}), but if an extension property uses a schema transition object ({"transition": {"from": "required", "to": "omit", ...}} or per-operation {"complete": {"transition": {"from": "required", ...}}}), val.as_str() == Some("required") won't match transition.from (which resolver::parse_transition_value uses as the active visibility). Consider also relaxing "from": "required" inside nested transition objects.

[damaz91]

Inlined $ref discriminator constraints (pattern) leaking from <ParentUnion>Base into extended subtypes:

When a parent union's discriminator property is a $ref to an open string type with constraints (such as Policy.properties.type referencing ReverseDomainName), build_union_base_schema removes "$ref" and inlines ReverseDomainName's "pattern" onto PolicyBase.properties.type.

When register_extended_subtypes merges base_template (PolicyBase) into an extended subtype via inherit_parent_into_union_variant, PolicyBase.properties.type no longer has "$ref", so merge_schema_property retains "pattern" from PolicyBase. Stripping "pattern" (and "minLength", "maxLength", "format") from disc_map when pinning "const" ensures extended subtypes get the same clean {"type": "string", "const": "<tag>"} discriminator as variants lowered in lower_conditional_unions.

[damaz91]

Closed unions lose properties and required before register_extended_subtypes runs:

Here .or_else(|| defs.get(&parent_union)) (along with the fallback loop in collect_lowered_union_discriminators) is designed to support closed unions where <ParentUnion>Base was not synthesized.

However, lower_conditional_unions (lines 334–348) already removes "properties" and "required" from defs[&parent_union], leaving only {"title": "<ParentUnion>", "anyOf": [...]}. If a decentralized subtype extends a closed union, base_template (defs.get(&parent_union)) has no properties or required, so inherit_parent_into_union_variant strips {"$ref": "#/$defs/<ParentUnion>"} from allOf without inheriting any parent fields.

[damaz91]

Closed-union discriminator detection order:

For unions without <name>Base, this fallback inspects the first referenced branch and picks the first property (in alphabetical BTreeMap order) that has a const or single-value enum. If a variant defines another const property that sorts before the discriminator key (e.g., "api_version": {"const": "1.0"} before "type": {"const": "..."}), or if a non-discriminator anyOf union references an object with a const field, it will record the wrong discriminator property.

Checking that all $ref branches share a const/single-enum property key (or passing the lowered union discriminator map from lower_conditional_unions) avoids false matches.

[damaz91]

Preserving non-$ref / non-if composition branches in fold_inline_allof_branches_into_subtype:

In Stage 6, flatten_single_object_allof preserves allOf branches containing ["$ref", "if", "then", "else", "anyOf", "oneOf", "allOf", "contains"].

Here, fold_inline_allof_branches_into_subtype only guards branch_obj.contains_key("$ref") || branch_obj.contains_key("if"). If a subtype has an allOf branch with "anyOf", "oneOf", or "not" (e.g., {"anyOf": [{"required": ["a"]}, {"required": ["b"]}]}), Stage 6 keeps it in allOf, and then fold_inline_allof_branches_into_subtype silently drops it because it has neither "$ref"/"if" nor "properties". Guarding the same composition keywords (and merging dependentRequired if present) prevents dropping constraint branches.