Fix stub_verified infinite recursion when Arbitrary calls the stubbed function - #4571
Fix stub_verified infinite recursion when Arbitrary calls the stubbed function#4571feliperodri wants to merge 3 commits into
stub_verified infinite recursion when Arbitrary calls the stubbed function#4571Conversation
70828a2 to
7812142
Compare
7812142 to
afdd4ab
Compare
|
I attempted to implement a runtime fix (nesting depth counter) but it conflicts with CBMC's DFCC assigns checking: writes to global mutable state inside contract-checked scopes trigger assigns violations. CBMC provides no mechanism to exempt infrastructure writes from DFCC tracking. So back to the drawing board... |
…ry calls stubbed function When a type's Arbitrary implementation calls a function targeted by stub_verified, the global contract replacement caused infinite recursion because test input generation (kani::any()) invoked the contract abstraction instead of the real function. Fix: kani::any() now uses an RAII guard (ArbitraryContextGuard) that increments a global ARBITRARY_NESTING_DEPTH counter before calling T::any() and decrements it on drop. The contract REPLACE match arm checks in_arbitrary_context() — when true, it executes the original function body instead of the contract replacement. This ensures Arbitrary impls always use the real function while verification callers use the contract abstraction. The counter (not a boolean) handles nested kani::any() calls correctly. Wrapping arithmetic avoids CBMC overflow checks on the counter. Changes: - library/kani_core/src/lib.rs: ArbitraryContextGuard RAII guard, enter/exit/in_arbitrary_context() accessors, guard in kani::any() - library/kani_core/src/lib.rs: Route write_any_slim, write_any_slice, and any_where through kani::any() so the guard covers all paths - library/kani_macros/src/sysroot/contracts/bootstrap.rs: REPLACE arm falls back to original body when in_arbitrary_context() is true - Tests: stub_verified_arbitrary_fix.rs (regression test), stub_verified_safe_arbitrary.rs (derived Arbitrary works), stub_verified_arbitrary_workaround.rs (standalone proof pattern) - docs/dev/stub-verified-arbitrary.md: Design rationale and soundness
A contract replacement havocs its own return value with `kani::any::<Ret>()`
(`initial_replace_stmts` emits `any_modifies`, which `AnyModifiesPass` rewrites
to `kani::any`). So when the `Arbitrary` implementation for `Ret` reaches the
stubbed function, the replacement re-enters itself through `Arbitrary::any`:
normalize -> replace closure -> kani::any::<Wrapper>
-> <Wrapper as Arbitrary>::any -> Wrapper::new -> normalize -> ...
This recursion has no fixpoint, so CBMC unwinds it until it exhausts memory.
Previously this surfaced as a multi-minute hang ending in an out-of-memory
message that never named the cause.
Note the cycle is self-contained inside the replacement: it reproduces even
when the harness passes a concrete value and never calls `kani::any()` itself.
Detect it instead: for each `Replace`-mode target, resolve `kani::any::<Ret>`
and walk the monomorphized call graph for a path back to the target. If one
exists, emit an error naming the call path and suggesting `#[derive(Arbitrary)]`.
This follows the existing `check_mutual_recursion` precedent.
The walk follows only statically resolvable calls, so a cycle routed through a
function pointer or trait object is not reported. This is deliberate: a missed
detection reproduces prior behavior, whereas a false positive would reject a
working proof.
Changes:
- kani-compiler/src/kani_middle/transform/contracts.rs: `check_arbitrary_cycle`
and the `find_call_path` call-graph walk, run once per replace target.
- tests/expected/function-contract/stub_verified_arbitrary_cycle.{rs,expected}:
regression test for the new diagnostic. Replaces the former
`stub_verified_arbitrary_workaround.rs`, which asserted this case verifies.
- tests/kani/FunctionContracts/stub_verified_safe_arbitrary.rs: cross-reference
the cycle test as the acyclic counterpart.
- docs, rfc: describe the actual mechanism and the detection limitation.
Signed-off-by: Felipe R. Monteiro <felisous@amazon.com>
afdd4ab to
18b4f8b
Compare
There was a problem hiding this comment.
Pull request overview
This PR addresses an infinite-recursion/hang scenario caused by #[kani::stub_verified] contract replacements re-entering themselves through kani::any::<Ret>() when Ret’s kani::Arbitrary implementation calls the stubbed function. The implementation in this diff prevents the hang by detecting the cycle during compilation and emitting a targeted error with a call-path trace, and it documents the limitation/workarounds.
Changes:
- Add a
stub_verified/Arbitrarycycle check inFunctionWithContractPass(compiler MIR transform) and emit a compile-time diagnostic with a call trace. - Add tests covering (a) a safe derived-
Arbitrarycase that should verify, and (b) a cycle case that should error with an expected message. - Document the limitation and workarounds in both the RFC and the user reference docs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/kani/FunctionContracts/stub_verified_safe_arbitrary.rs | New positive test ensuring stub_verified works when derived Arbitrary does not call the stubbed function. |
| tests/expected/function-contract/stub_verified_arbitrary_cycle.rs | New negative test setting up the stub_verified/Arbitrary recursion cycle. |
| tests/expected/function-contract/stub_verified_arbitrary_cycle.expected | Expected diagnostic output for the new cycle detection error. |
| rfc/src/rfcs/0002-function-stubbing.md | RFC documentation of the stub_verified/Arbitrary interaction and workarounds. |
| kani-compiler/src/kani_middle/transform/contracts.rs | Compiler-side cycle detection logic and diagnostic emission for verified stubs. |
| docs/src/reference/experimental/contracts.md | User docs note about the stub_verified/Arbitrary recursion limitation and mitigation. |
Suppressed comments (2)
kani-compiler/src/kani_middle/transform/contracts.rs:533
- The
traceformatting in this diagnostic does not match the expected output intests/expected/function-contract/stub_verified_arbitrary_cycle.expected: it currently (1) includes thekani::any::<Ret>frame, (2) does not prefix the first line with->, and (3) adds extra indentation before the trace. This will make the UI test brittle / fail.
Consider building the trace as one -> ... entry per line and dropping the initial kani::any::<Ret> frame so the first line is the Arbitrary::any call (which is where the recursion actually starts).
// Use the resolved instance name for the trace tail so it matches the
// crate-qualified names that `Instance::name` produces for the path.
let trace = path.join("\n -> ") + "\n -> " + &instance.name();
tcx.dcx()
.struct_span_err(
kani-compiler/src/kani_middle/transform/contracts.rs:316
- After switching
arbitrary_cycle_checkedto be keyed by monomorphizedInstance, this insert should use theinstanceparameter rather than*def; otherwise different instantiations of the same generic function still get deduped and may skip the cycle check.
if mode == ContractMode::Replace && self.arbitrary_cycle_checked.insert(*def) {
self.check_arbitrary_cycle(tcx, *def, args);
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /// Targets we already ran the `Arbitrary` cycle check on, so we only report once each. | ||
| arbitrary_cycle_checked: HashSet<FnDef>, |
| ### `stub_verified` and `Arbitrary` interaction (known limitation) | ||
|
|
||
| A contract replacement havocs its own return value with `kani::any::<Ret>()`. | ||
| So if the `kani::Arbitrary` implementation for `Ret` reaches the stubbed | ||
| function, the replacement re-enters itself through `Arbitrary::any`: |
Problem
When a type's
kani::Arbitraryimplementation calls a function that is the target of#[kani::stub_verified], verification hits infinite recursion. The contract replacement (ContractMode::Replace) applies globally, including insideArbitrary::any(), so test input generation itself invokes the contract abstraction instead of the real function.Solution
kani::any()now tracks nesting depth via a global counter (ARBITRARY_NESTING_DEPTH). The contractREPLACEmatch arm checks this counter — when inside an Arbitrary context (depth > 0), it executes the original function body instead of the contract replacement.Implementation details
ArbitraryContextGuard): Increments the counter on creation, decrements on drop. Ensures correctness even ifT::any()panics during concrete playback.kani::any(),any_where(),write_any_slim(), andwrite_any_slice()all go through the guard.REPLACEarm inbootstrap.rschecksin_arbitrary_context()and falls back to#block(the original function body) when true.Soundness
Excluding Arbitrary from stub replacement is sound: the real function produces a subset of valid values; the stub produces a superset. Using the real function gives tighter (more precise) input generation, not less sound verification.
Testing
stub_verified_arbitrary_fix.rs— Regression test:Wrapper::Arbitrarycallsnormalize()(the stubbed function). Previously caused infinite recursion, now works.stub_verified_safe_arbitrary.rs— DerivedArbitrary(doesn't call stubbed function) works withstub_verified.stub_verified_arbitrary_workaround.rs— Documents the standalone proof pattern as an alternative.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.