Repository navigation
1.2.0: what real sessions say about retirement, and memory for subagents - #29
Merged
Merged
Conversation
- A correction no longer replaces an unrelated one through a shared turn index; corrections replace only a same-session restatement (10 % shared content words). - Episodes take no part in replacement by text; AskUserQuestion answers retire nothing by wording. - A commit retires only records older than itself, and only a newer record naming an added word inherits; typed records of the turn go only if they name the value. - maintain restores, once, records those paths retired wrongly. - queued_command attachments are read; Agent SDK sessions are kept as episodes, untyped. - The catalogue lists a long sentence once; show no longer calls a budget-cut record retired. - SubagentStart hook, on by default (subagent-delivery off turns it off). - Startup text: ask muninn why in the records' language. - rusqlite 0.41.1, serde_json 1.0.154, sha2 0.11.1, tree-sitter 0.27.1. Measured in usage-v1, v2 and v3 (muninn-experiments).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1.2.0. The full list is in the CHANGELOG entry; this is the short version.
Reading 773 sessions and 1,200 subagent transcripts from ten stores turned up two problems.
Records the user really said were being retired, and subagents never got memory. Only counts
and record ids are published; the projects stay private.
Retirement (usage-v1). Of the 95 typed records retired with a replacement, 25 were real
losses. Replaying every session, 1.1.0 retires 20 of them and this branch 9. Both valid
retirements still happen. On the held-out loop7 and loop8 sets every figure equals 1.1.0's, and
the noise arm retires nothing.
maintainrestores, once, what the fixed paths had retired: 56records on copies of the ten stores, 13 of them real losses and neither valid one.
Subagents (usage-v3). A new
SubagentStarthook, on by default. A recorded rule was followedin 5 of 6 tasks with it and 1 of 6 without it, on one model.
Also: queued messages are read (28 of the 31 that were missing), the catalogue lists a long
sentence once,
muninn showno longer calls a budget-cut record retired, and the startup textsays to ask
muninn whyin the records' language (27 of 30 against 5 of 30, usage-v2). It alsotakes the four open dependency bumps (#22, #23, #24, #27).
Checks run locally: fmt, clippy,
cargo test --workspace --features exact-tokens, the faultsuite at 200 reps (14 of 14) and
init --check-budget(489 of 500 tokens) all pass.perf --strictfails on the full UserPromptSubmit on this Windows machine, as on 1.1.0. Over seveninterleaved runs of each build the p95 medians are 10.6 ms on 1.1.0 and 9.7 ms here.
SubagentStartreads 6.7 to 9.7 ms p95.Not fixed: a user statement read as a change of an earlier one on the same subject still retires
it, and that is most of what is still retired wrongly.
Results, pre-registration and amendments are in ilien-dev/muninn-experiments, branch
experiment/usage-replay-1.1.1.