feat(memory): continuous compaction with contextual hand summaries#1236
Closed
pbranchu wants to merge 11 commits into
Closed
feat(memory): continuous compaction with contextual hand summaries#1236pbranchu wants to merge 11 commits into
pbranchu wants to merge 11 commits into
Conversation
Lays the groundwork for per-user memory by giving every install a stable default-user UUID and tagging every session with an owning user. Sessions are now consistently user-scoped: - `Session::user_id: UserId` (required, not Option) — defaults to the kernel's persistent default user - `Session::parent_session_id: Option<SessionId>` — foundation for future tree-scoped cascade deletion of forked sessions (no producer yet) - `MessageSource` enum + optional `Message::source` — additive type that later PRs (structured extraction filtering) will read; no consumer here - `UserConfig::is_default: bool` — `[[users]]` blocks can attach display name and channel bindings to the persistent default identity Kernel boots the default user once and caches it process-wide: - `bootstrap_default_user` — load-or-generate the UUID from `kv_store[shared, "default_user_uuid"]`, install via `set_default_user_id`, then run a one-shot rewrite of legacy nil-UUID sessions, gated by the `default_user_bootstrap_done` sentinel - `resolve_user_id` (strict, HTTP boundary) — folds the deprecated "test" alias and the nil UUID to the default user with `warn!` logs so reserved-bucket abuse is auditable - `resolve_user_id_internal` (raw mapper) — preserves the pre-fix behaviour for in-process test callers - `AuthManager::new_with_default` — binds the `is_default = true` user (or the first user) to the persistent UUID Storage and migration: - Schema v9 adds `user_id` (NOT NULL, default nil UUID) and `parent_session_id` (nullable) to `sessions`, plus `(agent_id, user_id)` and `parent_session_id` indexes - `MemorySubstrate::rewrite_nil_user_sessions` — atomic transaction wrapping the legacy-bucket UPDATE; the kernel only sets the bootstrap-done sentinel after a clean rewrite, so a failure leaves the retry path intact Single-user installs see no behaviour change: everyone is the default user, one session per agent, same as today. Tests: - `MessageSource` deserialises cleanly from pre-field payloads (JSON + msgpack) and survives full round-trips - `UserConfig::is_default` defaults to `false` for existing configs - `AuthManager::new_with_default` honours `is_default`, falls back to first user, and is a no-op for empty configs - Migration v9 adds the columns and indexes, and a v8-built DB upgrades cleanly to v9 with pre-existing rows preserved - `default_user_id`/`test_user_id` are distinct - `create_session(agent, user)` round-trips through SQLite — both the default and an explicit user - `rewrite_nil_user_sessions` is idempotent, targeted, and atomic - Kernel: default-user UUID persists across kernel restarts; the strict filter folds `"test"` and nil to default while passing other UUIDs through; the internal mapper preserves the raw `"test"` alias Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cargo update bumps lettre from 0.11.21 to 0.11.22 to clear RUSTSEC-2026-0141. Pulls in transitive dependency updates as a side effect (mostly windows-sys/socket2 version consolidation) — no API surface change in our own code. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…trol API
Adds the storage layer, opt-in gate, and management/control HTTP API for
structured memory. No producer (extraction + dreamer) and no UI changes —
those land in follow-up PRs. Default agents see zero behavior change:
storage tables are created on migration but only populated by agents that
opt in via `[memory] system = "structured"` in their manifest.
Per-agent opt-in:
- `MemorySystem` enum (`Summarization` default / `Structured`) on
`AgentManifest`, with `MemoryConfig` wrapper carrying
`skip_serializing_if = is_default` so manifests that don't opt in
round-trip clean TOML.
- Same `[memory]` field on `HandAgentConfig`; `activate_hand` copies
through to the spawned manifest.
- `MemoryConfig::is_structured()` is the single gate consulted by every
call site that touches structured memory.
Schema migration v10 (PR 1 added v9) consolidates the three structured-
memory storage tables with denormalized columns from the start:
- `session_extractions(user_id, agent_id, ...)` — audit attribution
survives session deletes; idx on `(user_id, created_at)` for the
audit endpoint.
- `user_memory_topics` with `expires_at` + `embedding` columns.
- `user_agent_memory_topics` keyed by `(user_id, agent_id, topic)`.
Storage modules:
- `user_memory.rs` / `user_agent_memory.rs` — CRUD, embedding store, prune.
- `SessionExtraction` + `SessionExtractionStore` in `session.rs`.
- `MemorySubstrate::wipe_user(user_id) -> WipeUserCounts` wraps the three
bucket DELETEs in a single SQLite transaction so a partial failure rolls
back rather than leaving a user half-wiped.
- `MemorySubstrate::list_user_extraction_audit` joins with `sessions`
purely to surface the `session_deleted` flag — attribution comes from
the denormalized `user_id` column.
Kernel prompt-build gate:
- `build_user_memory_context(memory, user_id, agent_id, memory_cfg)`
returns `None` immediately when `memory_cfg.is_structured() == false`
(no SQLite roundtrip for default agents). Called from both prompt-build
sites in `kernel.rs`. `PromptContext::user_memory_context` carries the
value through to a new "What I Remember About You" section that is
skipped entirely for subagents and for empty indexes.
Control API (`/api/users/*`):
- `GET /api/users` — list users + default
- `GET /api/users/{user_id}/memory` — list topics
- `GET /api/users/{user_id}/memory/{topic}` — topic content
- `DELETE /api/users/{user_id}/memory/{topic}` — delete one (404 if absent)
- `DELETE /api/users/{user_id}/memory` — atomic wipe; returns per-bucket counts
- `GET /api/users/{user_id}/agents/{agent_id}/memory` — per-agent topics
- `DELETE /api/users/{user_id}/agents/{agent_id}/memory` — delete per-agent
- `GET /api/users/{user_id}/memory/audit` — extraction events with `session_deleted`
- `GET /api/users/{user_id}/memory/export` — JSON dump
- `parse_user_id()` accepts `"default"` or any non-nil UUID; rejects the
nil UUID (legacy anonymous-bucket sentinel) and the deprecated `"test"`
alias with 400.
- Module doc + per-handler `AUTHORIZATION:` comments call out the
single-tenant RBAC limitation (API-key holder == full memory admin).
PATCH /api/agents/:id/config gains `memory_system: Option<String>`,
validated against `MemorySystem`'s serde and persisted to disk via
`Registry::update_memory_config` + `persist_manifest_to_disk`. GET
/api/agents/:id surfaces both a flat `memory_system` string and the
nested `manifest.memory.system` shape so dashboards can read either form.
Tests (98 new):
- v10 migration creates all three tables + indexes and upgrades cleanly
from a v9 baseline without touching pre-existing rows.
- `wipe_user`: per-bucket counts, scope (user A's wipe leaves user B
untouched), idempotent zero-count run.
- `MemorySystem` defaults to `Summarization`; `[memory] system = "..."`
parses both variants; TOML round-trip skips `[memory]` for the default
case and re-emits it when opted in.
- `build_user_memory_context` returns `None` for default agents (even
with seeded topics), returns the formatted block when opted in with
topics, returns `None` when opted in with an empty index.
- `parse_user_id`: accepts `default` and any UUID, rejects nil UUID,
`"test"`, garbage, and empty string.
- Audit endpoint preserves attribution after session delete and flips
the `session_deleted` flag to true.
…amer
Adds the per-user memory producer code that populates the storage layer
introduced in PR 2. Entirely gated behind `manifest.memory.is_structured()`
— default (Summarization) agents see zero behaviour change, while opted-in
agents pay a structured-extraction LLM call when the context overflows and
a dream-consolidation pass when their session goes idle.
Components:
* `compactor::extract_structured()` — LLM-driven extraction producing
`SessionExtraction { facts, preferences, decisions, tasks, open_items }`.
Filters out `MessageSource::ContextInjection` so calendar/email summaries
do not bleed into long-term memory. Falls back to the existing extraction
(or an empty one) on LLM error or persistent parse failure — never
propagates an error to the agent turn.
* `compactor::needs_extraction()` + `count_tool_calls()` — trigger gates
on tokens-since-last or tool-calls-since-last thresholds.
* `compactor::SessionExtraction` re-export — runtime callers reach the
struct without pulling `openfang-memory` directly.
* `context_overflow::overflow_drain_count()` — peek at how many leading
messages overflow recovery would drain, without applying the trim, so
mini-dream can extract from them first.
* `mini_dream` module — in-loop consumer that, immediately before the
overflow recovery trims messages, runs extract + dream and persists the
resulting topics into the user memory store. Non-fatal: errors are
logged, never returned.
* `dreamer` module — session-end consolidation pass: merges all
`SessionExtraction` records accumulated during compaction plus the
recent message tail into 3–7 topic-organised user memory entries, with
conflict resolution (`supersedes` list) and expiry tagging.
* `agent_loop.rs` integration — two `if manifest.memory.is_structured()`
call sites (streaming + non-streaming) wire mini_dream into the iteration
prologue. Default agents skip entirely.
* `OpenFangKernel::trigger_session_dream()` + `run_session_lifecycle_loop()`
— background loop polls `agent_last_active` every 30 s; for each agent
idle longer than `[sessions] gap_secs`, fires the dream pass.
Per-agent gate: structured-memory only. Per-session gate: skip if no
real user activity (pure `[AUTONOMOUS TICK]` / `[SCHEDULED TICK]` /
`ContextInjection` sessions are no-ops — the production thundering-herd
fix from commit `aa4ec5c` on `branchu`).
* `SessionsConfig` (in `KernelConfig`) — `[sessions] gap_secs` knob,
default 300 s. Set to 0 to disable the dreamer loop entirely.
* `Message::context_injection()` helper — small constructor for the new
`ContextInjection`-tagged messages used by extract/dream tests and the
activity-gating predicate.
Tests (12 new, all green):
* `compactor::test_extract_structured_filters_context_injections`
* `compactor::test_extract_structured_fallback_on_empty_response`
* `compactor::test_needs_extraction_token_threshold`
* `compactor::test_needs_extraction_tool_calls_threshold`
* `compactor::test_count_tool_calls`
* `dreamer::test_dream_filters_context_injections`
* `dreamer::test_dream_result_has_topics`
* `dreamer::test_dream_conflict_resolution`
* `dreamer::test_dream_expiry_tagging`
* `dreamer::test_dream_fallback_on_parse_error`
* `mini_dream::test_structured_memory_gate_skips_default_agents`
* `mini_dream::test_structured_memory_gate_fires_for_opted_in_agents`
Built on top of `pr/memory-storage` (PR 2). No UI, route, schema, or
`MemorySystem`/`MemoryConfig` changes — that surface area is PR 2's.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… gate The dream activity gate (skip pure-tick / pure-context-injection sessions) was inlined in trigger_session_dream with zero direct test coverage. Pulled it out into a pub(crate) free function so the predicate can be unit-tested without spinning up a kernel, then added 8 tests covering the scenarios that matter for the thundering-herd fix: real user activity, autonomous ticks, scheduled ticks, mixed tick+real, pure context injections, empty sessions, assistant-only sessions, and ticks + context injections. This is the most critical test gap flagged by review — the predicate is the live protection against every heartbeat firing a dream pass.
… in dream extract_structured was declared Result<_, String> but every code path returned Ok(fallback()) — the Err arm was unreachable. Callers in mini_dream.rs and the kernel dream entry point had dead match/unwrap_or arms that could never trigger. Made the signature infallible and stripped the dead error handling from both call sites. While touching the kernel call site, also fixed the StubDriver fallback that wrapped resolve_driver().ok().unwrap_or_else(...). StubDriver.complete() always errors, so feeding it to extract_structured burned the full retry budget (3 attempts) producing nothing but warn logs before returning the fallback. Resolve the driver up front and return early when missing — matching the pattern that already existed for the later dream() call in the same function. No behavior change for the happy path; eliminates wasted retries and noisy warns when no driver is configured, and removes dead error code.
The lifecycle loop fires a dream task per expired agent every 30s. If a dream takes longer than gap_secs while the same agent keeps receiving activity, the loop could re-enter and spawn a second dream task for the same agent — racing on extractions, embeddings, and user-topic writes. Added agent_dream_locks: DashMap<AgentId, Arc<tokio::sync::Mutex<()>>> on OpenFangKernel and gate every spawned dream task on `try_lock` of the per-agent mutex. `try_lock` (not `lock`) so iterations never queue — if the previous dream is still running, this tick logs a debug and skips, and the next 30s tick will check again. The mutex is separate from agent_msg_locks because that one serializes user turns vs user turns for the same agent; dreams should not block user turns and vice versa — only dream-vs-dream needs serialization. Added test_agent_dream_locks_serialize_per_agent covering: (a) second dream for same agent fails try_lock while first is in flight, (b) a different agent is not blocked, (c) once the first releases the next dispatch can lock again.
… race doc Five smaller cleanups bundled together: - context_overflow: added test_overflow_drain_count_matches_recover_from_overflow (stage 1 + stage 2 + below-threshold paired tests) so the two implementations cannot silently drift apart. overflow_drain_count mirrors stages 1+2 of recover_from_overflow by hand; if either threshold moves, the paired test catches it. - compactor: marked count_tool_calls + needs_extraction with #[allow(dead_code)] and a TODO(PR4-or-later) explaining the intent. Today the extraction path is triggered by the context-overflow signal (overflow_drain_count → mini-dream); these helpers are the alternative cadence-based gate that will land in a follow-up PR once agent_loop tracks per-session counters. Keeping them next to CompactionConfig so the policy stays co-located. - compactor: tightened build_conversation_text visibility from pub to pub(crate). It is only called inside this crate (compactor and dreamer). - kernel: added a NOTE in run_session_lifecycle_loop explaining the benign race between the agent_last_active snapshot and the per-agent remove. The race only delays a dream by one gap_secs window; the dream itself reads the live session, so content correctness is preserved.
Dashboard surfaces for the structured memory feature. Per-agent Memory
System dropdown in the agent Config tab (opt-in). New Users page with
Memory and Extraction Audit tabs for viewing/deleting/exporting per-user
accumulated memory. Pure UI work — all backend routes and the PATCH
field already shipped in the memory storage PR.
- Memory System selector in the agent detail Config tab:
- <select id="memory-system"> in index_body.html with two options
("Summarization (default)" / "Structured (LLM extraction + dreamer)")
plus help text describing both modes.
- Wired to configForm.memory_system via Alpine x-model; saveConfig()
PATCHes the field along with the rest of the form.
- buildConfigForm reads memory_system from the flat field or the
nested manifest.memory.system shape exposed by GET /api/agents/:id.
- New Users page at #users route:
- Top-level sidebar entry under the System group; #users added to
validPages in app.js.
- Left rail: configured users, default user badged as "default".
- Right panel with Memory + Extraction Audit tabs.
- Memory tab: topic list with View / Delete actions, "Delete All
Memory" button, "Export JSON" header button that downloads
memory_<uid>_<yyyymmdd>.json from the export endpoint.
- Audit tab: per-event log (timestamp, agent name, session, per-field
counts) backed by /memory/audit.
- Topic viewer modal fetches full topic content on demand.
- New crates/openfang-api/static/js/pages/users.js; bundled into the
dashboard via webchat.rs alongside the other page scripts.
Styling matches the existing dashboard conventions (form-group,
form-select, text-xs.text-dim, card, tabs).
No backend changes: all consumed routes (GET /api/users,
GET/DELETE /api/users/{id}/memory[/{topic}], /memory/audit, /export)
and the memory_system PATCH field already exist on pr/memory-producer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Revival of upstream PR RightNow-AI#948 (closed 2026-04-04 without review), adapted to coexist with the memory-stack PRs (RightNow-AI#1224, RightNow-AI#1225, RightNow-AI#1226). What this adds ============== Continuous compaction is a proactive flavour on top of the standard reactive compaction. It fires on two triggers: - **Cadence trigger** — every `continuous_interval` user exchanges (post-loop, both streaming and non-streaming paths). - **Gap trigger** — when a new inbound channel message arrives more than `gap_secs` after the previous one for the same `(agent, user)` pair, compaction + context refresh runs *before* the new message is dispatched, so the LLM sees the refreshed context on the very next turn. When triggered, the kernel: 1. Runs the standard `compact_agent_session` pass. 2. Queries every configured `[[compaction.context_sources]]` hand in parallel with a `(from_ts, to_ts)` bounded window. - `from_ts` defaults to `now - gap_max_lookback_secs` on the first pass, then the previous compaction's timestamp on subsequent passes, so each refresh asks the hand only about the new window. 3. Truncates the combined payload to `context_token_cap` tokens (with a `…[truncated]` marker so the cap is visible in logs and prompts). 4. Injects the result via `Message::context_injection` (PR RightNow-AI#1224), which carries `MessageSource::ContextInjection` so the structured-memory dreamer (PR RightNow-AI#1226) skips it. Cross-PR integration ==================== - Uses `MessageSource::ContextInjection` so calendar/mail summaries do not bleed into long-term structured memory via the dreamer. - Tracks state per `(agent_id, user_id)` (PR RightNow-AI#1224 made sessions user-scoped). - Reuses `has_real_user_activity` (PR RightNow-AI#1226) so pure-tick sessions do not burn LLM/tool budget querying hands. - Mirrors the `agent_dream_locks` `try_lock`-and-skip pattern (PR RightNow-AI#1226) for the per-(agent, user) compaction lock — concurrent callers skip rather than queue. Naming ====== `[compaction] gap_secs` is intentionally separate from `[sessions] gap_secs` (PR RightNow-AI#1226). They measure the same wall-clock dimension (inactivity) but drive different subsystems with different defaults: 900s for compaction-trigger, 300s for dream-lifecycle. Renamed from the original PR: - `session_gap_secs` → `gap_secs` - `max_lookback_secs` → `gap_max_lookback_secs` Opt-in default ============== `continuous_interval = 0` by default — with no override the feature is fully off and existing deployments see no behaviour change. It kicks in only when `continuous_interval > 0` AND at least one `[[compaction.context_sources]]` is configured. Tests ===== - 3 unit tests for `needs_continuous_compaction` (off / fires on multiple / respects keep_recent boundary). - 4 unit tests for `truncate_to_token_cap` (under / over / cap=0 / multi-byte codepoint safety). - `test_agent_compaction_locks_serialize_per_pair` — verifies the per-(agent, user) lock has the same shape as the dream lock. - `test_inject_context_uses_context_injection_tag` — verifies the injection writes a `ContextInjection`-tagged message. - `test_extract_structured_skips_context_injection` — cross-PR integration test: confirms `extract_structured` (PR RightNow-AI#1225) filters out ContextInjection messages so the secret payload cannot leak into long-term memory. Documentation ============= `docs/CONTINUOUS_COMPACTION.md` covers triggers, configuration, the `gap_secs` naming collision, LLM/tool budget, debugging. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…dispatch latency
Two review fixes on the continuous-compaction PR:
1. Make `[compaction]` truly opt-in by default.
`[compaction] gap_secs` defaulted to 900, which meant deployments with
no `[compaction]` block still paid for `check_session_gap` on every
inbound channel message (lock acquisition + session read inside
`detect_and_run_session_gap`, even though it returned `false`
immediately).
- Change the default to `0` ("0 disables gap-triggered refresh").
- Add `channel_compaction_enabled()` getter on `ChannelBridgeHandle`,
implemented on `KernelBridgeAdapter` as
`continuous_interval > 0 || gap_secs > 0 || !context_sources.is_empty()`.
- Gate the `handle.check_session_gap(...)` call in the bridge on
that getter so the disabled path pays nothing — not even the lock
acquisition.
With no `[compaction]` block in config, behavior now matches
`pr/memory-ui` exactly (the stated invariant).
2. Document the blocking pre-dispatch await.
The gap trigger runs synchronously before `send_message` (by design
— the injected context must be in the session when the LLM reads it
for the first response after the gap). Worst-case it adds up to
~30s of latency from per-source timeouts. The cadence trigger uses
`tokio::spawn` and never blocks.
Add a "Latency considerations" section to docs/CONTINUOUS_COMPACTION.md
covering the worst-case math and operator guidance (keep context
sources small + fast, or disable `gap_secs` and rely on the
asynchronous cadence trigger for SLA-strict channels).
Also update the doc's opt-in section and the `[compaction] gap_secs`
row of the comparison table to reflect the new default.
Contributor
Author
|
Marking this draft. On re-reading issue #896 more carefully, an independent review surfaced that this resubmit does not address @jaberjaber23's 2026-05-12 feedback:
Will rework on the underlying branch and reopen (or open a fresh PR) once #1–#3 are addressed. Apologies for the noise. |
6 tasks
Contributor
Author
|
Closing — superseded by the proper 2-PR split @jaberjaber23 recommended on #896:
The split together addresses all four substantive concerns from the issue thread that this single-PR version missed (no new primitive, no untrusted-content wrap, no ephemeral isolation, no PR split). See the closing-PR comment above for details. |
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Revival of closed PR #948 (April 2026, lapsed without review). Adds continuous compaction triggered every N exchanges and gap-triggered session refresh when a user returns after a configurable idle window — both query explicitly-allowlisted hands for time-bounded context summaries and inject the result into the live session, tagged
MessageSource::ContextInjectionso it's excluded from structured memory extraction.Closes #896.
Depends on #1224, #1225, #1226, #1227 (the 4-PR structured-memory series). This builds on top of
MessageSource::ContextInjection,Session::user_id, thehas_real_user_activitypredicate, and the per-agent-lock pattern they introduce.Differences from the original closed #948
The original PR lapsed without review. This resubmit proactively addresses concerns a careful reviewer would raise — and integrates cleanly with the new memory architecture:
session_gap_secs(collides with #1226)[compaction] gap_secs— separate from #1226's[sessions] gap_secs(agent_id, user_id)lock keyed by sessions from #1224has_real_user_activityfrom #1226 — skips quiet-tick sessionscontinuous_interval = 0,gap_secs = 0, emptycontext_sources→ bridge skips the probe entirely (zero cost on stock configs)MessageSource::ContextInjection(from #1224) so the dreamer from #1226 doesn't extract facts from ittokio::spawnwith 30s timeout; per-source errors logged and skippedcontext_token_cap(default 2000 tokens) with UTF-8-safe truncationtry_lockon per-(agent, user) DashMap — concurrent compactions skip, don't queueHow it works
Cadence trigger (async, never blocks)
Every N exchanges (
continuous_interval, default 0 = off), after the agent loop finishes a turn:try_lock— skip if busyhas_real_user_activitypredicate — skip pure-tick / pure-context-injection sessionstokio::spawn(30s per-source timeout)[Context refresh — ts]user message taggedMessageSource::ContextInjectionFires via
tokio::spawnfrom the post-loop hook — does not block the agent loop response.Gap trigger (synchronous, blocks the user's first message)
When
[compaction] gap_secs > 0and the user returns after that interval, fires beforesend_messageso the injected context is in the session when the LLM reads it for the first response.Latency tradeoff documented in
docs/CONTINUOUS_COMPACTION.mdwith operator guidance (keep sources short, prefer fast hands, consider disabling for SLA-strict channels).Configuration
Opt-in invariant: with no
[compaction]block at all, the bridge's newchannel_compaction_enabled()getter returnsfalseand the probe is skipped entirely — zero cost on stock configs.Cross-PR integration
The injected
[Context refresh — ts]message is taggedMessageSource::ContextInjectionso:compactor::extract_structured(PR feat(memory): structured memory producer — extract + mini_dream + dreamer #1226) filters it out — the structured memory dreamer doesn't extract "facts" from a calendar hand's summaryhas_real_user_activitypredicate (PR feat(memory): structured memory producer — extract + mini_dream + dreamer #1226) treats it as non-user activity — quiet sessions with only context refreshes don't accidentally trigger dream consolidationIntegration test
test_extract_structured_skips_context_injectionexercises this end-to-end with a realRecordingDriverthat asserts the context-refresh payload never reaches the summarizer LLM.Test plan
cargo fmt --checkcleancargo clippy --workspace --tests --all-targets -- -D warningscleancargo build --workspacecleancargo test -p openfang-runtime --lib957/957 pass (incl. 3 newneeds_continuous_compactiontests)cargo test -p openfang-kernel --lib315/317 (only the 2 pre-existingtest_referenced_providers_*failures unrelated to this PR; 7 new tests for this feature all pass)ContextInjectiontag (verified via msgpack decode of persisted session)Reviewed independently
Two rounds of independent agent review. Round 1 found 2 yellow issues (default
gap_secs = 900broke the "stock config = no behavior change" invariant; pre-dispatch await blocking was undocumented). Both addressed with traceable commits before submission. Round 2 verified each fix is real and confirmed cross-PR integration with the dreamer.Documentation
docs/CONTINUOUS_COMPACTION.md(178 lines) covers triggers, moving parts, opt-in semantics, thegap_secsnaming distinction from PR #1226, activity gating, try_lock pattern, failure tolerance, token cap, latency considerations with operator guidance, cross-PR integration, and debug log lines.