Skip to content

Follow-up: Offline feedback-learning iteration and legible per-user rules #9360

Description

@Git-on-my-level

Follow-up: Offline feedback-learning iteration and legible per-user rules

Dependency

Blocked by #9352 Phase 0/4. Requires the epic's attribution telemetry to be live: every feedback event must already be attributable intervention → candidate/task/workstream → later outcome by stable IDs. This issue consumes that signal; it must not add new logging of raw private content.

Context

#9352 deliberately ships no online learning mechanism. Feedback (Do now / Later / Dismiss + Already handled / Not mine / Not useful), edits, completions, and auto-accept deletions are captured as attributable signal only. This issue is the offline iteration loop that turns that signal into better behavior — deliberately decoupled so the epic cannot block on learning infrastructure.

Scope

  1. Offline analysis pipeline over feedback + outcome telemetry: auto-accept precision by source/confidence band, "Not mine" rate by modality, dismissal patterns, 24h deletion of auto-accepted tasks, workstream association precision.
  2. Threshold and prompt iteration: recalibrate auto-accept thresholds, extraction policy fixtures, and evaluation golden-ranking fixtures from observed data. Changes land as fixture/prompt/threshold PRs against the shared policy spec in Epic: Rebuild Omi Tasks around quiet capture, focused goals, persistent workstreams, and contextual action #9352 — never as surface-specific prompt drift.
  3. Legible per-user rules (v2): derive inspectable, deletable per-user suppression/boost rules (e.g. "stop suggesting tasks from #general mentions") that the evaluation judgment consumes as context. Rules must be visible to the user before they take effect silently. This is the only sanctioned mechanism for personalized learning — no opaque per-user model weights.

Non-goals

Acceptance

  • Threshold/fixture changes are traceable to analyzed cohort data in the PR description.
  • A "Not mine" derived rule reduces equivalent ownership false-positives in the fixture suite without suppressing direct mentions.
  • Per-user rules are enumerable, human-readable, and deletable by the user.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions