Repository navigation
✨ Summary link types, and copy / collect on a link type: a chain's far ends as links - #2418
Merged
Merged
Conversation
…every NaN one value `sphinx_needs_testkit.conformance.same`, which holds a build's value to a pin in the derive and field-declaration runners, compared floats with `==`, so the `-0.0` pin of `fingerprint-defaults-compared-as-text` could not tell a build that kept the sign from one that dropped it, and a NaN pin could never pass. A float is now compared by its IEEE 754 bits, with every NaN mapped to one pattern, as ubCode's runner compares `f64::to_bits`; ints keep `==` and the type check, bools stay `is`. `tests/test_testkit_conformance.py` pins it (red first: 7 failed against the old `same`), and with the fix, editing that case's pin to `0.0` makes the case fail (`REQ_1.zero: -0.0, expected 0.0`). Both corpora pass unchanged. Closes #2216.
…a derive rule, a call or variant, or both The `needs.derive_scope` report at a `needextend` whose filter names a field computed later said "which a dynamic function or variant computes" also for a field a declared `derive` / `derive_by_type` rule computes, where the project wrote no dynamic function or variant. It now takes ubCode's three-way clause (`derive_scope_extend_filter`): "a derive rule", "a dynamic function or variant", or "a derive rule, a dynamic function or a variant" when the filter names fields of both. On one need a rule wins over a call on its (closed) field, as in ubCode. Only the "which ... computes" clause moved; the rest of the sentence, the code and the severity are unchanged. The two tests pinning the old text move with it, and a new test pins all three wordings in one project; against the old wording the three are red. Changelog: a new `Unreleased` section. Closes #2207.
ubCode has run the case since useblocks/ubcode#4038, through its runner's own `run_case`, by a dedicated test that also asserts the flag is still set, so the re-vendor of this file fails loudly there and says what to do. Re-measured now with ubCode's debug CLI on the case's project: every pinned value is the same in both tools and neither reports a contract finding (ubCode's `derive_sphinx_minimum` notice aside). The case runs in sphinx-needs either way; nothing else changes here. Two cases stay `python_only` (`union-over-a-member-computed-last`, `union-computed-last-read-in-its-step`). Closes #2214.
Six additions to `derive_conformance.toml`, four new cases and two edits (84 -> 88 cases, `EXPECTED_CASE_COUNT` with it): - `links-include-parts-of-imported-and-external-needs` (new): the parts of a `needimport`ed need (those its parsed content gives) and of a `needs_external_needs` need (those its record's `parts` give; its content is not parsed) are `include_parts` candidates of a `links` rule. - `filter-inline-spellings` (edit): `links_from_filter(..., include_parts=True)` links a part after its need, and `copy(filter=...)` takes needs only, never a part. - `fingerprint-nested-need` (new): a nested need's content and fingerprint, and its parent's; `parent_needs` does not enter. - `fingerprint-computed-tags-default` (new): a `[[...]]` default of `tags`, computed per need, compared with `[]` by the fingerprint. - `fingerprint-computed-link-default-from-global-options` (new): a link type's `[[...]]` default through `[needs.global_options]`. - `by-type-invalid-shapes` (edit): `entry_not_table` and `entry_invalid` declare `default = 7`, and an `impl` need is added, so the placeholder (null, on the invalid entry's type) and the default (7.0, on a type with no rule) are different values the case tells apart. Every pin was measured now in BOTH tools on the case's project: this tree with the `needs` builder, and ubCode's debug `ubc_cli` (0.38.0) through `build needs` / `build index`, read by its runner's rules (an uncarried link type reads as `[]`); all pins equal, and the contract findings the same multiset. One difference was found and is deliberately not pinned: an imported record's own `parts` key where its content gives no part (sphinx-needs takes the content's parts, none; ubCode the record's). A comment in the new case says so. Closes #2215.
…n upwards A new subsection under the `copy` kind, "Starting a list roll-up one rung up" (`needs_derive_rollup_rung_up`): `req_ids` starts on the specification as a `copy` of `id` over `satisfies` with `select = "list"`, and is collected on each rung below (implementation, test). One sentence says why the list does not start on the requirement itself: a plain `copy` of `id` reads one string, which an array field does not hold, and `select` takes a list only with `over` or `join`; both spellings are refused (`needs.derive_invalid`). The same chain read upwards gives each need the tests that reach it (`reached_by`: `collect id over tests_back` on the implementation, then `collect reached_by` over each rung's back links). The `collect` section points to it. Both examples, in the per-type spelling the page uses, and the two refused spellings, were built in this tree and with ubCode's debug CLI: every value and both refusals are the same in the two tools. The page was checked with a dummy Sphinx build of it alone (no markup errors); the full docs build was not run. Changelog: a Documentation entry under `Unreleased`. Closes #2379.
`all` over no candidate is `true`, which on a requirement nothing tests reads as "every test passed". A rule of kind `any` or `all` now takes `empty = true|false`, the value it gives there instead. No candidate is the candidate list once `where` has dropped the needs it does not hold on (a requirement that only a specification names, under `where = "type == 'test'"`, has none). Over candidates `empty` is not read, and a candidate leaving a boolean `field` unset still reads as false. Unset, `empty` is the kind's own value over no candidate, `all` `true` and `any` `false`, so no existing rule changes value. It is a boolean role (`_BOOLEAN_ROLES`): on any other kind the rule is refused as "kind '<kind>' does not take the role 'empty'", and a non-boolean value as "kind '<kind>': the role 'empty' must be a boolean" (`needs.derive_invalid`). Corpus: a new case `any-all-empty` (all / any x unset / true / false, with a test and with a boolean field; a `where` that empties the candidates, no link at all, and candidates on which `empty` is ignored), and two refusal rows in `derive-invalid-shapes` (`empty` on `sum`, a non-boolean `empty`). 88 -> 89 cases. The new case was red against the unfixed tree (30 value mismatches), and reading `empty` before `where` (over the raw `over` list) turns it red again. Docs: the any/all section (an example, `versionadded:: 9.2.0`), the kinds table, the roles table (`empty` row, and the `test` row points to it), and the "No candidate" bullet of "How rules are computed". Changelog: a ✨ entry under `Unreleased`. Closes #2377.
A `copy` with `over` or `join` now takes a `where`, which narrows its targets as it narrows the candidates of the other kinds and of `links` with `join`. It is tested on each candidate alone, BEFORE `select`: `"first"` takes the lowest-id target that passes `where` and sets the field, `"unique"` counts (and reports) only those, and `"list"` keeps their values (in written order over `over`, a need named twice read twice; in need-id order over `join`). A candidate the filter cannot be evaluated on (a field it lacks, a type error) is dropped without a warning, as `_passes` drops it for every other kind. A `copy` of the need's own field, or of the need `from` names, reads one need and still takes no `where`, now refused as "kind 'copy' takes the role 'where' only with 'over' or 'join'" (`needs.derive_invalid`), in the wording of the existing `select` refusal. `copy` over a link type already filtered its candidates through `where` (the role was simply refused); the join branch of `execute_rule` now filters the matches before splitting into `links` and `copy`. The order of the pass needed no change: `_RuleReads.read` already adds the names of a `where` as columns in the join branch for any kind, and as reads on each target in the `over` branch. Corpus: `copy-join-where` and `copy-over-where` (first / unique / list with a `where` that drops the lowest-id setter; a `where` that cannot be evaluated on that setter; a `where` dropping every candidate), and two refusal rows in `derive-invalid-shapes` (own-field and `from` copies with a `where`). 89 -> 91 cases. Both new cases were red against the unfixed tree. Their `where` reads a field computed in the same step, declared last, on candidates whose ids sort after the reader's: dropping the `where` reads from `functions/order.py` (in either branch) turns the matching case red, so the cases fence the order too. Docs: the `copy` section (an example, `versionadded:: 9.2.0`), the join section's "links and copy" and "The order" bullets, the kinds and roles tables (`where`, `select`), the "Candidates" bullet and the list of refusals. Changelog: a ✨ entry under `Unreleased`. Closes #2246.
…ng-up changelog line is exact
A link type whose `needs_links` entry says `summary = true` materialises a path rather than an edge of the trace (#2370). It is read where it is named -- a filter, a needtable column, a rule's `over` or `field`, a needflow's `:link_types:`, a schema rule -- and a need holds its links and back links as any link type's; it is left out of every view that reads all link types, which in sphinx-needs is a needflow without `:link_types:`: neither drawn nor walked from `:root_id:`. - a `union` cannot name a summary link type as a member (`needs.derive_invalid`, the union holds no links): a union is read as one set of edges - needs.json's schema marks the link type and its back links `"summary": true`; ub-needs-exchange passes it through (contract section 7.1 says so, a reader test pins it) - the stance is one per name: a need type's `links` entry giving `summary` is a `needs.config` finding and the key is ignored; in a rule it is no role (`needs.derive_invalid`); a value that is not a boolean is a `needs.config` finding - the derive corpus gains an optional `schema.<name>` pin (the testkit's `check_schema`, as the field-declaration corpus has) and two cases, `summary-key-shapes` and `union-refuses-summary-member` (93 cases); ubCode owes a re-vendor
…s as links `copy` of `id` over a link type, and `collect` of a link type over a link type, declared on a link type (#2370): `{kind = "copy", field = "id", over = "satisfies"}` on a spec and `{kind = "collect", field = "chain", over = "implements"}` on the rungs below give every need of the chain the requirements it traces to as links, and each requirement the needs that reach it as back links -- what the field form gives as id strings. - each link once by its target and part, the condition dropped; a dead link is carried and reported where it is written (`LinkSchema.path_on`, asked beside `union_on`) - such a link type is a summary; `summary = false` beside the rule is refused (`needs.derive_invalid`) and ignored, and a union cannot name it - computed with the link types (stratum 1): an `over` naming back links or a derived link type, a `field` naming a value field or back links, a copy of a field other than `id`, `select` other than "list", `after` are refused at load, not per need - per need, a path reads its `field` on each need `over` names, so collect-over-collect is ordered like an own-field roll-up and a ring of needs is `needs.derive_cycle`; where a call computes the need's own `over` in stratum 1 the path waits for it and reads the field through its column (as check_linked_values does) - the density guard counts the links; fingerprints leave them out, as every derived link type - corpus: `link-copy-id-over`, `link-collect-chain`, `link-collect-cycle`, `link-kinds-refusals`, `link-collect-dense` (98 cases), header rule added; ubCode owes a re-vendor - two tests that used `copy` as the example of a value kind on a link type now use `sum`
…e written, no copy - `over` may name a derived link type: a `union` is read through its members (`_RuleReads._static_targets`, recursively), so a `collect` of its own link type over `trace_up` is computed rung by rung and a ring through the union is exactly its members' `derive_cycle`s. Over a link type another kind of rule computes (or a union with such a member) the path reads its field through its column: a `copy`, or a `collect` of another link type, is computed; a `collect` of its own link type would read itself on every need and is refused at load with that reason - a dead link a copy or collect carries sets `has_dead_links` only on the need writing it; its `forbidden` class follows the link type it is written in, rung by rung (`_forbidden` walks the path's `over` to the writer); `allow_dead_links` beside a copy or collect is refused and ignored, as beside a union - `copy = true` beside `summary = true` is a `needs.config` finding, and nothing is copied into `links` - the `needs_flow_link_types` fallback leaves summary link types out too - tests: the own-over wait is pinned by link types sorting before `implements`; the forbidden class is pinned in html with both `allow_dead_links` values - corpus: `link-kinds-refusals`' over_derived is the still-refused shape (a collect of its own link type over a `links`-derived link type); four cases appended (`link-collect-over-union`, `link-collect-ring-through-union`, `link-collect-dead-links`, `link-path-where-reads-later`), 102; header: the schema pin's path and a missing pinned key, derived overs, has_dead_links, derive_scope of a later `where`. ubCode owes the re-vendor of this revision
master squash-merged the derive branch this one stacked on (#2384), and has since gained per-type declarations (#2395), extension types (#2400), c.is_a (#2401), the incremental rewrite rules (#2414) and more. Conflicts kept both sides: the changelog's Unreleased bullets; LinkSchema's severity beside summary in the basic-doc snapshot; a path's select default in derive.py; the derive corpus gains master's where-follows-inherits case ahead of this branch's eleven, 103 cases.
…elevant page A collect on a link type links a chain's far ends, needs the collecting need does not name, so its targets are not known before the needs are resolved: a changed over left the old far end's page showing the need in its back links. It is now classified as a links rule is (every page holding a need is re-written on a change), and so is a copy on a link type over a computed link type; a copy over a link type written in the needs links only the needs its own links name, and stays a read of its links. Scratch-vs-incremental cases for both kinds.
…here, rules on need types alone A need type's back_links count names a summary link type as any other and counts its links, those a copy or collect computes included; a copy or collect may be declared on need types alone (the link type a summary all the same) and read c.is_a in its where; a need type's summary stays a needs.config finding.
…mmary is read A copy or collect on a link type, refused by 9.1.0, now computes, and a link type's summary now changes a needflow's default, with no change to the configuration: an upgraded build re-reads every document once, rather than keeping the empty link lists and old diagrams on the pages it does not re-read.
… on a link type reads c.is_a The where of a path reads c.is_a as master's where-follows-inherits pins it for a value rule: with spec inheriting req, a copy keeps the spec, and a collect whose where refuses a req leaves the spec's chain out. 104 cases.
master's unknown-key report (#2388) names the keys of NeedLinksConfig and LinkOptionsType, and pins them: this branch adds summary to both.
…d-type warning says why the key is ignored
… ENV_DATA_VERSION 18
github-actions
Bot
added
pkg: sphinx-needs
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2418 +/- ##
==========================================
+ Coverage 94.84% 94.85% +0.01%
==========================================
Files 146 146
Lines 25790 25943 +153
==========================================
+ Hits 24460 24609 +149
- Misses 1330 1334 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
chrisjsewell
marked this pull request as ready for review
October 30, 2026 16:21
Documentation build overview
|
chrisjsewell
added a commit
that referenced
this pull request
Oct 31, 2026
A link type declared on need types alone can now be a summary. Mark it on the entry of the first need type that declares it: ```toml [needs.types.result.links.covers] summary = true ``` It is then a summary on every need type, exactly as `summary = true` on a `[needs.links.covers]` entry makes it: a needflow without `:link_types:` leaves it out, and the needs.json schema marks it, and its back links, `"summary": true`. Until now that key was reported (`needs.config`) and ignored, so such a link type could be a summary only when a `copy` or `collect` computed it. `summary` follows the one-name-one-shape rule that the link type's other keys (`incoming`, `copy`, `allow_dead_links`, the styles) already follow: - **Agreement.** The first need type declaring the link type sets `summary`, and its silence means `false`. Every other need type may repeat the value or leave it out. A need type giving another value is reported (`needs.type_conflict`) and its entry ignored. - **Computed link types.** A link type that a `copy` or `collect` computes is a summary in any case. That means a derived one, with the rule in a declaration that is used; a refused declaration computes nothing. A declaration may say so, and `summary = false` beside such a rule is reported (`needs.derive_invalid`) at that need type's entry and ignored, whichever need type writes it. The rule still applies on that type. This is what a `[needs.links]` entry already does. - **Link types declared in `needs_links`.** The `[needs.links]` entry keeps the stance. A need type's entry giving `summary` is the usual shape-key warning (`needs.config`), even when it repeats the link type's own value. - **`copy = true` beside `summary = true`.** On a link type declared on need types alone, this is now reported at the need type's entry (`needs_types['spec'].links['mitigates']`). Nothing is copied, so it is no longer also reported as deprecated. This amends the unreleased "Summary link types" entry (#2418); it is not a new change on its own. `ENV_DATA_VERSION` goes to 20, because the key now changes a needflow's default under an unchanged configuration. The shared corpora carry the rule: - `field_declaration_conformance.toml` gets five cases (agreed, disagreeing, a copy in a refused declaration, implied by a copy, beside a `[needs.links]` entry), and its count goes from 123 to 128. - `derive_conformance.toml` gets one case, `summary = false` beside a copy on the first and on a later declaration, and its count goes from 104 to 105. A first declaration's ignored `false` sets nothing, so a later one may still say `true`. - Both headers now say `summary` is a shape key. ubCode vendors both files, so this owes it a re-vendor and the matching change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read by the data-model side before going out of draft: needs.json's
"summary": truemarker and the exchange-contract sentence fit, and nothing collides with its in-flight work. The ubCode side is useblocks/ubcode#4452; this PR merges first.What
packages/sphinx-needs(+ one sentence inpackages/ub-needs-exchange's exchange contract, and a reader test).The stance.
summary = trueon a link type'sneeds_linksentry (any link type, authored or derived) says its links materialise a path,not an edge of the trace. A summary link type is read where it is named (a filter, a needtable column, a rule's
over/field, a needflow's:link_types:, a schema rule, a need type'sback_linkscount, which counts its computed links too), and a need holds its links and back linksas for any link type. It is left out of every view that reads all link types:
in sphinx-needs, the needflow default (without
:link_types:a summary link type is neither drawn nor walked from:root_id:, and theneeds_flow_link_typesfallback leaves it out too). A union cannot name one as a member (needs.derive_invalid).summarybeside the deprecatedcopy = trueis reported and the copy ignored, so a summary link never leaks intolinks. needs.json's schema marks the link type and its back links"summary": true, and ub-needs-exchange passes it through. The stance is one per name: a need type's entry giving it is aneeds.configfinding.The kinds. On a link type,
copytakesfield = "id"andover, and links each needovernames.collecttakesfield(a link type, a derivedone included) and
over, and gives the linksfieldholds on each needovernames. Both takewhere(c.is_a(...)included), and both may bedeclared on need types alone (
[needs.types.<t>.links.<l>]). Each link appears once by target and part,its condition dropped. A dead link is carried and reported where it is written, rendered as that link type allows, and does not mark the needs below it
as having dead links. Such a link type is a summary implicitly.
overmay name a union, read through its members, so a chain can climb atrace_up-styleunion rung by rung. What the link step cannot wait for is refused at load:
overnaming back links;collectof its own link type over a link type another rule computes (it would read itself on every need);fieldnaming a value field or back links;copyof a field other thanid;selectother than"list";after;allow_dead_linksbeside such a rule.Per need, a path is ordered after the needs it reads; needs reading one another's chain around a ring (through a union too) are
needs.derive_cycle,and only those. The density guard counts these links; fingerprints leave them out.
Incremental builds. A
collecton a link type links needs its need does not name (a chain's far ends), so it is classified as alinksruleis: in a project with one, every change of a need re-writes every need-relevant page. A
copyover a link type written in the needs links only theneeds its own links name, and stays a read of its links (a
copyover a computed link type is classified as unnamed too).ENV_DATA_VERSIONis 19 (17 is 🐛 A need with :hide: is located as a visible need in the same place is #2415's, 18 ✨ NeedItemSource: one class for where a need comes from, and add_need is deprecated in favour of create_need #2419's): acopy/collecton a link type, refused by 9.1.0, now computes, andsummarynow changes aneedflow's default, with the configuration unchanged.
Why
Closes #2370.
The far ends of a chain (the requirements a test result is evidence for, three hops up) were only expressible as a field: strings, with no back links
("the test results that reach me"), no
:need:rendering, no direct relationship in a graph consumer. The link form adds them. The stance it needs, that such a linkis a summary and not a trace edge, applies to authored and other derived link types too.
Bringing it up to master. The branch was built on the derive branch, since squash-merged as #2384; master was merged in (merge commits, no rebase).
The conflicts kept both sides: the changelog's Unreleased bullets,
LinkSchema'sseverity(#2395) besidesummaryin the basic-doc snapshot,a path's
selectdefault, and the corpus (master'swhere-follows-inheritswhere master has it, this branch's cases after it). Checked against what mastergained since: the incremental rules of #2414 (a
collecton a link type was read as an ordinaryoverand left the old far end's page stale: fixed,red first, with scratch-vs-incremental cases for both kinds); a
[needs.types.<t>.back_links.<l>]count (#2395) naming a summary or computed link type(allowed, counts every link, pinned);
c.is_a(#2401) in a path'swhere(pinned in a test and the corpus); a path declared on need types alone and a needtype's
summary(#2400, pinned); the needs.json writer's nullable rule and""-> null (#2395, #2400: fields only, besidesummaryon link types), andthe needflow change on master (a view category only). Later merges brought in #2415 (hidden-need location) and #2419 (
NeedItemSource), which tookversions 17 and 18, so this branch's bump is 19; every bump stays in
data.py's version notes and in the changelog's bump bullet.Corpus.
derive_conformance.tomlgains twelve cases on top of master's 92 (104, sha256c4721bcc6cb1532b4a39a79c3ba5325c75b006762ca5d25a5e1c78809923ba01;python_only 2, ubcode_only 1) and an optional
schema.<name>expectation, documented in the header; master's cases are untouched.ubCode re-vendors it and mirrors the stance and the kinds in useblocks/ubcode#4452, including its own consumers that read all link types:
the needflow default, the agent walks,
ubc diff's default, the Need Viewer tree and the Cypher schema flag.Follow-up, not in this PR.
summaryon a need type's own link entry is aneeds.configfinding today, so an authored link type declared onneed types alone cannot be a summary. It can be lifted under the one-name-one-shape rule (#2284): accept it there when every need type declaring
the name agrees, and report a disagreement as any other shape mismatch.
Tests
derived
overlifted for unions; a call-computedovermaking only that need a cycle, kept) and added mutations of their own, each now killed (the needflowdefault and fallback, the union refusal, load-time refusals, stratum order, implied summary, dead links, the
forbiddenrendering with bothallow_dead_linksvalues). After the first merge: mutations of the incremental classification (both directions), of the back links of a computed summarylink type, and of
c.is_ain awhere, each red.summaryis one now, and its test says so.uv run poe lint0,uv run poe typecheck0, fulluv run poe test-needs4244 passed / 15 skipped (after the ✨ NeedItemSource: one class for where a need comes from, and add_need is deprecated in favour of create_need #2419 merge),uv run poe test-ub-needs-exchange0 (246 passed),uv run poe smoke-needs0.docs-needswarns only on what intersphinx resolves (the inventories are unreachable from the sandbox); none of this PR's references.Checklist
uv run poe test-needs,uv run poe test-ub-needs-exchange) and they pass.docs/changelog.rsthas an entry under Unreleased.uv run poe lintanduv run poe typecheckpass.