EIP-8141: revalidate dependency-affected frame transactions on a new head - #12778
EIP-8141: revalidate dependency-affected frame transactions on a new head#12778Marchhill wants to merge 32 commits into
Conversation
|
Claude finished @Marchhill's task in 5m 29s —— View job Review: EIP-8141 dependency revalidation
The design is right and the layering is clean — That said, I found one issue I'd call blocking and several that should be resolved before merge. The headline one is that the new eviction path leaks payer reservations, and its effects compound with two of the others. Findings
Critical: the reservation leak (finding 1)Worth expanding here because it's the merge-blocker.
The fix is small — move the TestingThe three new tests are well-chosen for the paths they cover (definite rejection evicts, indeterminate retains, unrelated change list doesn't consult the simulator — the
Smaller notes
VerdictNot mergeable as-is: finding 1 is a remotely-triggerable permanent DoS on any paymaster, and findings 2/3 mean the feature both misses the reorg case it claims to cover and reintroduces the head-processing stall it sets out to avoid. Findings 1, 2 and 5 are each a few lines. Findings 3, 4, 6 and 7 are design calls that need either a fix or an explicit rationale in the code. I have not built or run the test suite here (review only, no changes pushed). |
6152ef9 to
6214489
Compare
…lidation # Conflicts: # src/Nethermind/Nethermind.TxPool/IFrameTxPrefixSimulator.cs
…lidation # Conflicts: # src/Nethermind/Nethermind.Consensus/Processing/FrameTxPrefixSimulator.cs
|
The eight table findings all had inline threads and are resolved there. Responding here to the "Smaller notes", which did not:
Separately, a summary-only note from #12624 applies here and is now fixed: an EIP-7702-delegated sender runs the delegate's code, so the delegate is a dependency of the prefix; the sender's own code hash only pins the designation. |
|
@claude review |
|
Claude finished @Marchhill's task in 7m 25s —— View job Review: EIP-8141 dependency revalidation (round 2)
Round-1 findings — all resolved
The "smaller notes" replies are all reasonable, including keeping New findings
A — blob pool. B — the new delegate dependency. This is finding 5's twin, opened up by the head commit. C — hash cache. TestingThe five new tests cover their paths well, and the reservation-lifecycle and reorg cases requested last round are both there. Remaining gaps map onto the findings: no blob-pool case (A), no delegation case at all (B), no "resubmittable after the condition clears" case (C). I could not build or run the suite in this environment ( VerdictMuch improved — the round-1 blockers are genuinely fixed, and #4 and #7 were fixed by removing the risky construct rather than patching it. Nothing here is a merge blocker on its own: A and B are silent coverage holes in the new feature rather than regressions, and C is a behavioural sharp edge inherited from the expiry path. All three are small changes, so I'd resolve them before merge rather than as follow-ups. |
|
@claude review |
|
Claude finished @Marchhill's task in 7m 15s —— View job Review: EIP-8141 dependency revalidation (round 3)
Round-2 findings — all resolved
I also re-walked round 1's reservation logic against the current code, since the new re-index wrapper sits on that path: all four New findings — all in the three commits added since round 2
1 — the only one I'd hold the merge for. TestingThe reservation-lifecycle, resubmittability, reorg and delegation cases requested in earlier rounds are all present, and the delegation test is well-constructed — it fails for the right reason (the second head touches only the delegate), rather than asserting index internals. The one real gap tracks finding 1: I could not build or run the suite in this environment ( VerdictThe revalidation feature itself now looks right to me — round 1 and round 2 findings are genuinely fixed, several by removing the risky construct rather than patching it, and the |
…lidation # Conflicts: # src/Nethermind/Nethermind.TxPool.Test/TxPoolTests.cs # src/Nethermind/Nethermind.TxPool/Filters/NotSupportedTxFilter.cs
The pass removed every candidate, which made the deadline order it computed unobservable and over-evicted a steady-state-full pool; it now stops at the freed slot and runs for the blob pool too. Rename the revalidation flag to what the caller asks of it: releasing a reservation that was never taken under-counts the payer just as badly as never releasing one.
|
@claude review |
|
Claude finished @Marchhill's task in 10m 20s —— View job Review: EIP-8141 dependency revalidation (round 4)
Round-3 findings — all resolved
I also re-walked the round-1/2 fixes against the current text, since the re-index wrapper and the blob-pool fallthrough both sit on the reservation path: all four I checked the load-bearing assumption behind the whole index and it holds: a storage-only write to a dependency account does surface in New findings
1 — the only one I'd resolve before merge. 2 is not a correctness problem, just a mismatch between what the code does (free exactly one slot per pool per head — I also chased two things that turned out not to be findings, recorded so they don't get re-raised: mutating TestingSeven new tests, and the cases requested across rounds 1–3 are all present and well-shaped — reservation lifecycle, resubmittability under the same sender, reorg, the two-step delegation case, and the
VerdictThis is in good shape. Across four rounds the fixes have consistently been the structural kind rather than patches — the lock-free index replaced outright, · branch |
| RemoveProcessedTransactions(args.Block); | ||
| RemoveExpiredFrameTransactions(args.Block); | ||
| RevalidateFrameTransactions(args.Block); | ||
| ShedNearlyExpiredFrameTransactions(args.Block); |
There was a problem hiding this comment.
Medium — the shed pass reads the pressure signal before the head's own bucket cleanup has run, so it can shed a valid transaction to free a slot that was about to free itself.
ShedNearlyExpiredFrameTransactions decides on pool.IsFull(), but UpdateBuckets() at line 419 has not run yet — and it removes transactions: UpdateBucket marks them (updateTx(transactions, tx, null, lastElement)) and TxDistinctSortedPool.UpdatePool then calls TryRemove(_transactionsToRemove[i].Hash!). Those are the transactions the new head invalidated: nonce already consumed by a sibling, sender balance now below tx.Value, and the blob nonce-gap cascade.
So on a head where the pool is at capacity and UpdateBuckets is about to drop, say, five stale transactions, this pass has already shed a live frame transaction with up to 24 s of deadline left for a slot nothing needed. That is the same over-eviction the previous round's break was added to prevent, just one layer up: the break bounds how many are shed, this bounds whether any should be.
Frequency is modest — RemoveProcessedTransactions (line 409) usually drops the whole included set first, so the pool is often already under capacity by the time this runs — but when it does fire it evicts a transaction the node would otherwise have kept, and the fix is to move the call:
| ShedNearlyExpiredFrameTransactions(args.Block); | |
| RevalidateFrameTransactions(args.Block); |
and place ShedNearlyExpiredFrameTransactions(args.Block); immediately after UpdateBuckets(); on line 419. Revalidation must stay where it is (it consumes _frameTxsToRevalidate, and eviction there is not capacity-driven), but shedding is purely a response to pressure and should read the pressure that actually remains.
| shedding.AsSpan().Sort(static (a, b) => | ||
| { | ||
| int byDeadline = a.Deadline.CompareTo(b.Deadline); | ||
| return byDeadline != 0 ? byDeadline : a.Fee.CompareTo(b.Fee); | ||
| }); | ||
|
|
||
| foreach ((_, _, Transaction tx) in shedding.AsSpan()) | ||
| { | ||
| // Stops as soon as a slot is free, so the order above decides who yields first and a | ||
| // transaction with life to spare keeps its place until it actually expires. | ||
| if (!pool.IsFull()) break; |
There was a problem hiding this comment.
Low — with the break in place the loop can only ever remove one transaction, so the full sort is a O(n log n) way to select a single minimum, and the plural framing overstates the pass.
The break is the right fix for last round's over-eviction, but it makes the shed exactly one transaction per pool per head: IsFull() is _cacheMap.Count >= _capacity and SortedPool.Remove calls UpdateIsFull(), so the first successful RemoveTransaction flips it and the second iteration breaks. Two consequences:
-
Only the minimum is ever read. Sorting a pool-sized
ArrayPoolList<(ulong, UInt256, Transaction)>(48 bytes/entry, up toSizeentries on a full mainnet pool) to consume element 0 is work the result never justifies — a single linear pass tracking the best(Deadline, Fee)gives the same answer with no list, no sort, and noComparison<T>indirection. That also removes the second full-pool walk per head this pass adds on top ofRemoveExpiredFrameTransactions' snapshot. -
The wording implies a set. "Sheds the pending frame transactions closest to expiry" (
<summary>), "sheds only as far as the pressure goes" (<remarks>), and the PR summary's "the pending frame transactions … are shed, nearest deadline first" all read as if several yield together. What the code does is free exactly one slot per head. That is a defensible policy — it is the smallest step that gives the near-expiry tier any effect at all — but it means the tier's throughput is ~1 tx/slot regardless of how much fee-ordered eviction is happening behind it, which a reader sizing the feature would want stated.
Either is fine; they just shouldn't disagree. If shedding one slot per head is intended, say so and drop the sort in favour of a min-scan. If it is not, the bound wants to be a small headroom target rather than the IsFull() threshold, and then the sort earns its place.
Related, minor: the fee tiebreak is the one part of the order no test pins — Shedding_takes_the_nearest_deadline_first_and_stops_at_the_freed_slot varies only the deadline, so a.Fee.CompareTo(b.Fee) is unexercised. A second [TestCase] with equal deadlines and different MaxPriorityFeePerGas would cover it.
| /// <remarks> | ||
| /// EIP-8141 "Direct Evaluation of Protocol-Defined Frames" names the sender, the payer and the | ||
| /// expiry verifier as that set. Helper contracts an opaque prefix reaches through <c>CALL*</c> are | ||
| /// not indexed yet, so a code change at one does not trigger revalidation (EIP8141-GAP). | ||
| /// </remarks> |
There was a problem hiding this comment.
Low — the GAP note names one of the two axes the account index cannot see.
Helper contracts reached through CALL* are the right thing to flag, and the account-granularity superset argument holds up: I checked both producers of Block.AccountChanges and a storage-only write does surface the account. StateProvider.ChangedAddresses() returns every key of _blockChanges, which PersistentStorageProvider.Commit populates via AccountExists for each address whose storage root moved; BlockAccessListBasedWorldState filters on HasStateChanges, which includes StorageChanges.Length > 0. So the sender/payer/verifier slot dependencies are genuinely covered by their accounts.
What is not covered, and is not mentioned, is block context. An opaque prefix may branch on TIMESTAMP, NUMBER, or BLOCKHASH — a paymaster that enforces its own deadline inline rather than through an expiry frame is the obvious case. Nothing about such a prefix appears in any change list, so it is never collected, and unlike a helper-contract code change (which at least has a triggering account) the invalidating input moves on every head. RemoveExpiredFrameTransactions covers this for the protocol expiry frame only.
Worth naming in the same sentence so the gap list is complete — one clause, no code change:
| /// <remarks> | |
| /// EIP-8141 "Direct Evaluation of Protocol-Defined Frames" names the sender, the payer and the | |
| /// expiry verifier as that set. Helper contracts an opaque prefix reaches through <c>CALL*</c> are | |
| /// not indexed yet, so a code change at one does not trigger revalidation (EIP8141-GAP). | |
| /// </remarks> | |
| /// <remarks> | |
| /// EIP-8141 "Direct Evaluation of Protocol-Defined Frames" names the sender, the payer and the | |
| /// expiry verifier as that set. Two kinds of dependency are outside it (EIP8141-GAP): helper | |
| /// contracts an opaque prefix reaches through <c>CALL*</c>, so a code change at one does not trigger | |
| /// revalidation; and block context an opaque prefix reads (<c>TIMESTAMP</c>, <c>NUMBER</c>), which no | |
| /// change list can describe — only the protocol expiry frame is swept, by | |
| /// <see cref="RemoveExpiredFrameTransactions"/>. | |
| /// </remarks> |
For the record I also chased the EIP-8272 axis and it is not a gap worth listing: a recent-root reference stays valid for RecentRootUsableWindow = 8191 slots (~27 h), far beyond any mempool residency, so ageing out is unreachable in practice — and had it been indexed against RecentRootAddress, whose storage is written every block, every such transaction would be revalidated on every head.
Changes
Closes the biggest correctness hole in the frame-tx mempool stack: a frame transaction was validated once, at submit, and never rechecked.
FrameTxDependencySetwas constructed by the payer resolver and never consumed — its doc comment promised "a later layer". This is that layer.FrameTxDependencyIndex— maps the chain-head accounts a pending prefix depends on (sender, resolved payer, and the expiry verifier when an expiry frame is present) back to the transaction. Maintained from the pool'sInserted/Removedevents, so it can never outlive pool membership.Head-change revalidation — the new block's changed-account list is intersected with the index before it is disposed, and only that subset is re-resolved after included and expired transactions have left. A head whose change list does not describe everything that moved — a reorg, or a non-sequential block — falls back to every indexed prefix; that is the same completeness test the account cache uses, computed once so the two cannot drift. Revalidating the whole pool per head would be its own denial-of-service vector, which is the point of the index.
Invalid-against-head eviction — a prefix that no longer resolves a payer, or whose payer can no longer cover the pool's exposure to it, is evicted immediately. That is the first tier of the spec's eviction order: such transactions never compete for pool space at all.
Reservation follows the payer — a revalidation that resolves a different payer releases the old reservation and takes a new one against the new payer's balance, evicting if it does not fit.
FrameTxSimulationResult.Indeterminate— a simulation rejected by a resource bound (busy, per-head budget spent, timed out) says nothing about validity. Admission still declines, but revalidation leaves the transaction pending; otherwise an exhausted budget would turn into a mass eviction.Near-expiry shedding — when the pool is full at a head boundary, the pending frame transactions whose deadline is within roughly two slots are shed, nearest deadline first and lowest effective priority fee first among equals. That is the spec's second and third eviction tiers applied where they carry real information: a transaction about to expire is worth little, so it yields its slot rather than displacing a live one through the pool's fee-ordered capacity eviction.
The
EIP8141-GAPlist inNotSupportedTxFilterloses the items now implemented and keeps canonical-paymaster reservation, the failed-APPROVEreplay bound, and a deadline-ordered pool index.Scope
CALL*/EXTCODE*; those are not yet dependencies, so a code change at one does not trigger revalidation. MarkedEIP8141-GAP.IComparer<Transaction>over all transaction types. A frame-only expiry tiebreak that returns 0 for mixed pairs is not transitive and would corrupt the sorted pool; a total order would rank every expiring frame transaction below every other transaction, which is a policy change well beyond the spec. Doing it properly needs a deadline-ordered index insideTxDistinctSortedPooland an eviction-preference hook onSortedPool, so this PR ships the bounded form instead: expired transactions are dropped proactively on each head, and the near-expiry shed pass covers the ordering that matters under pressure. What remains unordered is the deadline ranking among frame transactions with real life left.Types of changes
What types of changes does your code introduce?
Testing
Requires testing
If yes, did you write tests?
Notes on testing
TxPoolTestscovers a sponsored frame transaction evicted once its prefix stops validating against the new head, one retained when the simulation failed on a resource bound, one left untouched when the block's changed accounts miss its dependency set (asserting the simulator is not consulted), a reorg revalidating a transaction its change list never mentions, and a sponsor able to fund again after an eviction released its reservation.Nethermind.TxPool.Test726 passed / 1 skipped;Nethermind.Evm.Testframe suites 147 passed.dotnet format whitespaceclean.Documentation
Requires documentation update
Requires explanation in Release Notes