diff --git a/AGENTS.md b/AGENTS.md index 82b5e08..05e5735 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,7 +11,7 @@ Reason in English. Communicate with humans in Chinese. Call the user “Sir”. - Expensive core-py internal design: `docs/30-unit-tdd/` - Runtime, packaging, migration, observability, and recovery: `docs/40-deployment/` - Vulnerability reporting: `SECURITY.md`; local security boundaries: `docs/30-unit-tdd/security-model.md` -- Volatile task control: `tasks/`; never treat it as durable truth +- Volatile task control: `tasks/`; never treat it as durable truth, but retain an active packet until its parent task closes - Mechanically enforceable facts: code, configuration, schemas, tests, assertions, lint, and CI - Repeated subtree hazards only: the nearest local `AGENTS.md` @@ -24,9 +24,12 @@ Resolve the semantic owner before adding durable material. A Unit is a logical r and [contribution workflow](https://github.com/InKCre/.github/blob/main/CONTRIBUTING.md) for branches, pull requests, release authority, and delivery boundaries; repository-local documents own exact commands. - Before a reference-sensitive, logic-altering, or non-obviously-local durable mutation, state the exact object, `From -> To`, side effects, blast radius, invariants, verification, and uncertainty. +- Before promoting behavior, evaluate delivery owner, durable owner, interface layer, and external capability owner independently. Importance, first-party distribution, current pressure, or successful acceptance on one axis does not prove another. +- Before owning external protocol mechanics, inspect existing dependencies and primary documentation and name the unsupported gap. Keep only the application-specific remainder. - Read the nearest local `AGENTS.md` before changing its subtree. Read shared Product or Product TDD only when that owner is implicated, then the relevant local Unit TDD or Deployment document. - Before a security-sensitive claim, read the security model and name actor, capability, asset, boundary, harm, and attack path. Missing defense in depth is hardening unless evidence shows a boundary violation. - Exclude `tasks/`, generated output, dependencies, environments, caches, and temporary directories from ordinary source and durable-doc search unless they are the evidence target. +- Clean task artifacts by parent-task lifecycle, not directory class, age, size, or completed child units. Splitting content must not create a second control authority. - Use sub-agents only when bounded isolation or parallel capacity repays assignment, validation, integration, conflict, and residual cost. Primary owns the Human relationship, global integration, and material residual. Pause for Human input when the requested change conflicts with Product/Technical truth, ownership across durable surfaces remains unclear, evidence cannot support a bug or architecture decision, or the shortcut would damage maintainability. diff --git a/app/business/source/main.py b/app/business/source/main.py index 2059441..f3bd111 100644 --- a/app/business/source/main.py +++ b/app/business/source/main.py @@ -7,7 +7,6 @@ from typing import Optional as Opt from app.engine import SessionLocal -from app.database_contract.profile import BUILTIN_SOURCE_TYPES_BY_ID from app.business.info_base.block import BlockManager from app.schemas.info_base.block import BlockForm, BlockModel from app.schemas.job import JobModel @@ -168,30 +167,15 @@ def sync_source_types( registered = cls._SOURCE_CLASSES if source_classes is None else source_classes with SessionLocal() as db: for source_type, source_cls in registered.items(): - builtin = BUILTIN_SOURCE_TYPES_BY_ID.get(source_type) stmt = sqlalchemy.dialects.postgresql.insert(SourceTypesModel).values( id=source_type, - description=( - builtin.description - if builtin is not None - else source_cls.__doc__ or "No description." - ), - config_schema=( - builtin.config_schema if builtin is not None else source_cls.__configschema__ - ), - collect_config_schema=( - builtin.collect_config_schema - if builtin is not None - else source_cls.__collectconfigcls__.model_json_schema() - ), + description=source_cls.__doc__ or "No description.", + config_schema=source_cls.__configschema__, + collect_config_schema=source_cls.__collectconfigcls__.model_json_schema(), backfill_config_schema=( - builtin.backfill_config_schema - if builtin is not None - else ( - None - if source_cls.__backfillconfigcls__ is None - else source_cls.__backfillconfigcls__.model_json_schema() - ) + None + if source_cls.__backfillconfigcls__ is None + else source_cls.__backfillconfigcls__.model_json_schema() ), ) stmt = stmt.on_conflict_do_update( diff --git a/app/database_contract/catalog.py b/app/database_contract/catalog.py index cbce9b3..91836f9 100644 --- a/app/database_contract/catalog.py +++ b/app/database_contract/catalog.py @@ -19,7 +19,6 @@ from .profile import ( BUILTIN_AI_DIALECTS, BUILTIN_JOB_TYPES, - BUILTIN_SOURCE_TYPES, BUILTIN_STORAGES, BUILTIN_STORAGE_TYPES, ) @@ -119,35 +118,6 @@ def reconcile_builtins(database_url: str | None = None) -> None: ), ) - for source_type in BUILTIN_SOURCE_TYPES: - cursor.execute( - sql.SQL( - """ - INSERT INTO {}.sources_types ( - id, description, config_schema, collect_config_schema, - backfill_config_schema - ) - VALUES (%s, %s, %s, %s, %s) - ON CONFLICT (id) DO UPDATE - SET description = EXCLUDED.description, - config_schema = EXCLUDED.config_schema, - collect_config_schema = EXCLUDED.collect_config_schema, - backfill_config_schema = EXCLUDED.backfill_config_schema - """ - ).format(sql.Identifier(PROTOCOL_SCHEMA)), - ( - source_type.id, - source_type.description, - Jsonb(source_type.config_schema), - Jsonb(source_type.collect_config_schema), - ( - None - if source_type.backfill_config_schema is None - else Jsonb(source_type.backfill_config_schema) - ), - ), - ) - for storage in BUILTIN_STORAGES: cursor.execute( sql.SQL( diff --git a/app/database_contract/profile.py b/app/database_contract/profile.py index 506b661..5fe1a49 100644 --- a/app/database_contract/profile.py +++ b/app/database_contract/profile.py @@ -26,15 +26,6 @@ class StorageTypeProfile: writable: bool -@dataclass(frozen=True) -class SourceTypeProfile: - id: str - description: str - config_schema: JsonObject - collect_config_schema: JsonObject - backfill_config_schema: JsonObject | None = None - - @dataclass(frozen=True) class JobTypeProfile: id: str @@ -64,14 +55,6 @@ def _object_schema( } -def _string(default: str = "") -> JsonObject: - return {"default": default, "type": "string"} - - -def _integer(default: int) -> JsonObject: - return {"default": default, "type": "integer"} - - def _positive_integer(default: int) -> JsonObject: return {"default": default, "exclusiveMinimum": 0, "type": "integer"} @@ -308,219 +291,6 @@ def _boolean(default: bool) -> JsonObject: ) -EMPTY_SOURCE_COMMAND_SCHEMA = { - "additionalProperties": False, - "properties": {}, - "title": "EmptySourceCommandConfig", - "type": "object", -} - -GITHUB_COLLECT_SCHEMA = { - "additionalProperties": False, - "properties": {"full": _boolean(False)}, - "title": "CollectConfig", - "type": "object", -} - -TWITTER_COLLECT_SCHEMA = { - "additionalProperties": False, - "properties": { - "full": _boolean(False), - "result_limit": {"default": 40, "maximum": 100, "minimum": 5, "type": "integer"}, - }, - "title": "CollectConfig", - "type": "object", -} - -FEED_COLLECT_SCHEMA = { - "additionalProperties": False, - "description": "Per-run overrides admitted by the RSS collection command.", - "properties": { - "download_enclosures": { - "anyOf": [{"type": "boolean"}, {"type": "null"}], - "default": None, - "title": "Download Enclosures", - }, - "fetch_full_text": { - "anyOf": [{"type": "boolean"}, {"type": "null"}], - "default": None, - "title": "Fetch Full Text", - }, - "target_storage_id": { - "anyOf": [{"type": "integer"}, {"type": "null"}], - "default": None, - "title": "Target Storage Id", - }, - }, - "title": "FeedCollectJobConfig", - "type": "object", -} - -_FEED_PROPERTIES = { - "feed_url": {"type": "string"}, - "request_timeout_seconds": _positive_integer(30), - "max_feed_bytes": _positive_integer(8 * 1024 * 1024), - "fetch_full_text": _boolean(True), - "max_article_bytes": _positive_integer(8 * 1024 * 1024), - "download_enclosures": _boolean(False), - "max_enclosure_bytes": _positive_integer(64 * 1024 * 1024), - "target_storage_id": _integer(-4), - "unidentified_item_behavior": { - "default": "create", - "enum": ["create", "discard"], - "type": "string", - }, - "user_agent": _string("InKCre RSS/0.1"), -} - - -def _feed_source_schema(title: str, description: str) -> JsonObject: - schema = _object_schema(title, description, _FEED_PROPERTIES) - schema["additionalProperties"] = False - schema["required"] = ["feed_url"] - return schema - - -MAIL_SOURCE_SCHEMA = { - "additionalProperties": False, - "description": "Configuration for one Mail access context.", - "properties": { - "protocol": {"const": "imap", "default": "imap", "type": "string"}, - "parameters": { - "additionalProperties": False, - "properties": { - "host": {"minLength": 1, "type": "string"}, - "port": {"default": 993, "maximum": 65535, "minimum": 1, "type": "integer"}, - "security": { - "default": "tls", - "enum": ["tls", "starttls", "plain"], - "type": "string", - }, - "username": {"minLength": 1, "type": "string"}, - "password": {"minLength": 1, "type": "string"}, - }, - "required": ["host", "username", "password"], - "title": "IMAPParameters", - "type": "object", - }, - "excluded_mailboxes": { - "anyOf": [ - { - "additionalProperties": False, - "properties": { - "names": { - "items": {"type": "string"}, - "title": "Names", - "type": "array", - }, - "special_uses": { - "items": {"type": "string"}, - "title": "Special Uses", - "type": "array", - }, - }, - "title": "MailboxExclusionPolicy", - "type": "object", - }, - {"type": "null"}, - ], - "default": None, - }, - "ordinary_mark_as_seen": _boolean(True), - "backfill_mark_as_seen": _boolean(False), - "synchronize_deletions": _boolean(False), - }, - "required": ["parameters"], - "title": "MailSourceConfig", - "type": "object", -} - -MAIL_BACKFILL_SCHEMA = { - "additionalProperties": False, - "description": "One exact historical Mail collection range.", - "properties": { - "since": {"format": "date", "title": "Since", "type": "string"}, - "before": { - "anyOf": [{"format": "date", "type": "string"}, {"type": "null"}], - "default": None, - "title": "Before", - }, - }, - "required": ["since"], - "title": "MailBackfillConfig", - "type": "object", -} - - -BUILTIN_SOURCE_TYPES = ( - SourceTypeProfile( - "extensions.github.stars.Source", - "GitHub Stars Source - collects starred repositories from GitHub.", - _object_schema( - "SourceConfig", - "Configuration of GitHub Stars Source.", - { - "github_token": _string(), - "include_private": _boolean(False), - "username": _string(), - }, - ), - GITHUB_COLLECT_SCHEMA, - ), - SourceTypeProfile( - "extensions.mail.source.Source", - "Mail Source - incrementally collects communication records through IMAP.", - MAIL_SOURCE_SCHEMA, - EMPTY_SOURCE_COMMAND_SCHEMA, - MAIL_BACKFILL_SCHEMA, - ), - SourceTypeProfile( - "extensions.rss.atom.Source", - "Atom Feed Source.", - _feed_source_schema( - "AtomSourceConfig", - "Configuration for Atom feed source.", - ), - FEED_COLLECT_SCHEMA, - ), - SourceTypeProfile( - "extensions.rss.rss.Source", - "RSS 2.0 Feed Source.", - _feed_source_schema( - "RssSourceConfig", - "Configuration for RSS 2.0 source.", - ), - FEED_COLLECT_SCHEMA, - ), - SourceTypeProfile( - "extensions.telegram.source.Source", - "Telegram Source - collects messages sent to the configured Telegram bot.", - _object_schema( - "SourceConfig", - "Configuration of Telegram Source.", - { - "bot_token": _string(), - "collect_method": { - "default": "default", - "enum": ["default", "webhook"], - "type": "string", - }, - }, - ), - EMPTY_SOURCE_COMMAND_SCHEMA, - ), - SourceTypeProfile( - "extensions.twitter.bookmark.Source", - "Twitter Bookmark as Source", - _object_schema( - "SourceConfig", - "Configuration for Twitter Bookmark Source.", - {}, - ), - TWITTER_COLLECT_SCHEMA, - ), -) - BUILTIN_STORAGES = ( StorageProfile(-1, "http", "HTTP", {}), StorageProfile(-4, "postgresql_binary", "PostgreSQL Binary", {}), @@ -528,5 +298,4 @@ def _feed_source_schema(title: str, description: str) -> JsonObject: BUILTIN_AI_DIALECTS_BY_ID = {item.id: item for item in BUILTIN_AI_DIALECTS} BUILTIN_STORAGE_TYPES_BY_ID = {item.id: item for item in BUILTIN_STORAGE_TYPES} -BUILTIN_SOURCE_TYPES_BY_ID = {item.id: item for item in BUILTIN_SOURCE_TYPES} BUILTIN_JOB_TYPES_BY_ID = {item.id: item for item in BUILTIN_JOB_TYPES} diff --git a/app/database_contract/readiness.py b/app/database_contract/readiness.py index 6614ff9..6a5606d 100644 --- a/app/database_contract/readiness.py +++ b/app/database_contract/readiness.py @@ -23,7 +23,6 @@ from .profile import ( BUILTIN_AI_DIALECTS, BUILTIN_JOB_TYPES, - BUILTIN_SOURCE_TYPES, BUILTIN_STORAGES, BUILTIN_STORAGE_TYPES, ) @@ -484,22 +483,6 @@ def _catalog_component(cursor) -> dict[str, Any]: ): problems.append(f"job_types:{profile.id}") - for profile in BUILTIN_SOURCE_TYPES: - cursor.execute( - sql.SQL( - "SELECT description, config_schema, collect_config_schema, " - "backfill_config_schema FROM {}.sources_types WHERE id = %s" - ).format(sql.Identifier(PROTOCOL_SCHEMA)), - (profile.id,), - ) - if cursor.fetchone() != ( - profile.description, - profile.config_schema, - profile.collect_config_schema, - profile.backfill_config_schema, - ): - problems.append(f"sources_types:{profile.id}") - for storage in BUILTIN_STORAGES: cursor.execute( sql.SQL("SELECT type, nickname, config FROM {}.storages WHERE id = %s").format( diff --git a/docs/30-unit-tdd/mail-extension.md b/docs/30-unit-tdd/mail-extension.md index e3ee937..b20bad0 100644 --- a/docs/30-unit-tdd/mail-extension.md +++ b/docs/30-unit-tdd/mail-extension.md @@ -20,8 +20,9 @@ Mail MIME Resolver -> semantic content Block + content Relation ``` -共享的 Source、Job/Cron、graph、Storage、Resolver、Peer 与 InfoBase route contracts 属于 Hub Product TDD；本文只拥有 -Python package topology、exact IDs、Mail schemas、IMAP checkpoint policy、graph grammar 与本 repo acceptance boundary。 +Hub Product TDD 只拥有 generic Source、Job/Cron、graph、Storage、Resolver、Peer 与 InfoBase route contracts；本文拥有 +完整的 Mail-specific product/technical contract，包括 Python package topology、exact IDs、Mail schemas、IMAP +checkpoint policy、graph grammar 与 acceptance boundary。 ## Package Topology diff --git a/docs/30-unit-tdd/memos-extension.md b/docs/30-unit-tdd/memos-extension.md index 00381d0..9fb202d 100644 --- a/docs/30-unit-tdd/memos-extension.md +++ b/docs/30-unit-tdd/memos-extension.md @@ -3,9 +3,9 @@ ## Purpose 本文件记录 core-py `memos` extension 的稳定本地架构。它实现 memo-family authority，并以 -Memos-compatible backend 作为首个 access mode。共享的 collection / organization / application、 -graph、resolver、storage 与 extension protocol contract 属于 Hub Product TDD；这里拥有 Python -package、resolver ID、relation grammar、API version、transaction 和测试边界。 +Memos-compatible backend 作为首个 access mode。Hub Product TDD 只拥有 generic collection / organization / +application、graph、resolver、storage 与 Extension contracts；本文拥有完整的 Memos-specific product/technical +contract，包括 Python package、resolver ID、relation grammar、API version、transaction 和 acceptance 边界。 ## Current Delivery Boundary diff --git a/docs/30-unit-tdd/rss-extension.md b/docs/30-unit-tdd/rss-extension.md index 43ae810..a28efec 100644 --- a/docs/30-unit-tdd/rss-extension.md +++ b/docs/30-unit-tdd/rss-extension.md @@ -16,8 +16,9 @@ source schedule/manual command -> optional full-text/enclosure materialization ``` -共享的 collection、graph、storage、resolver、effect vocabulary 与 authority 合同属于 Hub Product TDD； -这里拥有 Python package、exact IDs、config/state shape、relation grammar、事务与测试边界。 +Hub Product TDD 只拥有 generic collection、graph、storage、resolver、effect vocabulary 与 authority 合同； +本文拥有完整的 RSS/Atom-specific product/technical contract，包括 Python package、exact IDs、config/state shape、 +relation grammar、事务与 acceptance 边界。 ## Delivery Boundary diff --git a/docs/40-deployment/database-contract.md b/docs/40-deployment/database-contract.md index 25e627c..655f91c 100644 --- a/docs/40-deployment/database-contract.md +++ b/docs/40-deployment/database-contract.md @@ -94,7 +94,8 @@ abrupt loss 等待 lease 自然过期。 - role attributes、memberships、table/sequence/schema/function ACLs； - exact public relation/function set and internal-surface exclusion； - admitted RPC argument names/types、return database type、set shape、volatility and transport media type； -- checked-in built-in catalog； +- checked-in core-owned catalog；Extension-contributed Source types are published by their runtime and are not readiness + prerequisites before activation； - fixed development seed when selected。 只检查 function names/EXECUTE 权限不足以证明 wire contract。Readiness errors remain component-level and never expose diff --git a/docs/40-deployment/runtime-orchestration.md b/docs/40-deployment/runtime-orchestration.md index 35b2771..4f9ab20 100644 --- a/docs/40-deployment/runtime-orchestration.md +++ b/docs/40-deployment/runtime-orchestration.md @@ -25,6 +25,9 @@ Current bootstrap flow in `run.py`: 11. start the scheduler and register Peer refresh、Cron materialization and pending-Job checks 12. report readiness as true +Source classes are the sole schema authority for Extension-contributed Source types。Database initialization does not seed +disabled Extension Source schemas；Extension startup publishes them before step 7 persists the complete runtime registry。 + Database waiting is retryable and does not block `/livez`. A failure after the database preflight moves runtime state to `failed`; it is observable through `/readyz` and is not silently retried because extension startup can have partial effects. diff --git a/docs/_shared b/docs/_shared index 5595ede..3296867 160000 --- a/docs/_shared +++ b/docs/_shared @@ -1 +1 @@ -Subproject commit 5595edec65317c728cb27ed70b219d5e068e208b +Subproject commit 3296867c3d47f285686703a919c3adef0b3b3d43 diff --git a/tasks/extension-ownership-correction/packet.md b/tasks/extension-ownership-correction/packet.md new file mode 100644 index 0000000..cc5b9f8 --- /dev/null +++ b/tasks/extension-ownership-correction/packet.md @@ -0,0 +1,99 @@ +# Extension ownership correction + +- **Objective**: correct cross-unit ownership errors exposed by GitHub extension review before resuming that unit。 +- **Phase**: implementation complete；local verification passed，while owner-separated Hub/core commits、Hub push and Spoke + shared-ref bump remain delivery work requiring explicit authorization。 +- **Relation to the program**: this is an independent corrective task，not a new knowledge-lifecycle capability or an + Extension implementation unit。`knowledge-lifecycle-capabilities` remains active but its GitHub unit is paused until this + correction closes。 + +## Trigger and evidence + +The GitHub extension implementation passed its real-account journey but review exposed repeated promotion errors： + +- concrete Extension capabilities were promoted into Hub normative PRD/Product TDD merely because they were important and + first-party； +- Extension-contributed Source types were represented as core built-ins； +- adjacent persistence primitives were introduced without preserving the existing graph-command topology； +- an installed mature protocol client was replaced by handwritten transport without first proving a capability gap。 + +PR #79 also deleted the still-active `knowledge-lifecycle-capabilities` task packet by treating the whole `tasks/` subtree as +disposable historical material。That packet is being restored separately as task-control repair；this task owns the resulting +cross-unit product/technical corrections，not the historical packet reconstruction itself。 + +## Accepted scope + +1. **Hub/Spoke durable ownership** + - Hub normative docs do not claim Memos、RSS、Mail、GitHub or another concrete Extension capability merely because the + Extension is first-party。 + - Concrete names may remain only as explicitly non-normative implementation examples。 + - Normative protocol、graph、runtime and acceptance contracts live with the owning Extension in its Spoke-local Unit TDD + or README；verify that local owner exists before removing Hub text。 +2. **Core built-in versus Extension publication** + - Remove all Extension Source types from `BUILTIN_SOURCE_TYPES`，including GitHub、Mail、RSS、Telegram and Twitter。 + - Extension activation/runtime publication is the Source schema owner。 + - If caller evidence shows no true built-in Source remains，delete the empty profile/fallback abstraction instead of + preserving speculative structure。 +3. **Promotion and dependency guidelines** + - Evaluate delivery owner（core/Extension）、durable owner（Hub/Spoke）、interface layer（domain command/persistence + primitive）and external capability owner（existing dependency/InKCre）as independent axes。 + - Importance、first-party distribution、current unit pressure and successful acceptance are not promotion evidence。 + - Before implementing external protocol transport，inspect existing dependencies and primary documentation，then record + the exact unsupported gap；own only the remainder。 +4. **Task cleanup lifecycle** + - `tasks/` is not durable truth，but an active task packet is current collaboration authority。 + - Cleanup is gated by task lifecycle，not directory class、age、line count or completed child units。 + - Content may be split to control a monolith without creating a second control authority。 + +## Explicitly deferred to GitHub extension + +- PyGithub-backed snapshot adapter correction； +- GitHub graph reconciliation and real-account re-acceptance； +- symmetric batch persistence beneath `InfoBaseManager.submit_graph()`； +- GitHub release metadata and PR #80 implementation closure。 + +These remain recorded in +[`knowledge-lifecycle-capabilities/units/github-extension/packet.md`](../knowledge-lifecycle-capabilities/units/github-extension/packet.md) +and resume only after this task closes。 + +## Planned sequence + +1. Inspect Hub concrete Extension claims and confirm each Extension-local normative owner。 +2. Freeze the exact Hub deletions/normalizations and local Unit TDD additions。 +3. Trace Source type publication、database init/readiness and cold Extension restore callers。 +4. Decide whether the built-in Source profile abstraction is removed entirely or retained for a demonstrated core Source。 +5. Prepare an Impact Handshake covering Hub、shared ref、database contract/runtime and documentation guidelines。 +6. Wait for Sir's explicit start；then mutate Hub first，push its owner commit，apply core correction，and bump the shared ref + separately。 +7. Verify Hub links/ownership、database reset/readiness、Extension activation catalog publication and repository gates；close + this task before resuming GitHub extension。 + +## Guardrails + +- Do not broaden this into general Extension hardening or release work。 +- Do not modify GitHub-specific protocol/reconciliation behavior in this task。 +- Do not add speculative registries、fallbacks or tests merely to preserve deleted structure。 +- Keep Hub edits、Spoke-local implementation and shared-ref bump in owner-separated commits。 + +## Implementation and verification evidence + +- Restored the still-active `knowledge-lifecycle-capabilities` packet deleted by PR #79：102 deleted paths plus the original + program/capability-map structure，with GitHub integrated as a paused unit and stale graph-navigation control corrected。 +- Hub removes concrete Memos、RSS/Atom、Mail and GitHub capability、claim、workflow、realization and reference-integration + contracts。Generic memo-like capture、collection、source/enrichment、graph and Extension contracts remain。 +- The canonical shared-doc workflow now checks delivery owner、durable owner、interface layer and external capability owner + independently before promotion。 +- Core removes `SourceTypeProfile`、all Extension schema copies、`BUILTIN_SOURCE_TYPES` and its lookup，catalog seed/readiness + loops，and the `SourceManager.sync_source_types()` fallback。No true core built-in Source was found，so no empty abstraction remains。 +- Local Memos、RSS and Mail Unit TDDs explicitly own their complete Extension-specific product/technical contracts；core + contribution guidance records promotion、dependency reuse and active-task cleanup boundaries。 +- A clean worktree development database reset produced an empty `sources_types` catalog and passed database/Core readiness。 + Publishing GitHub、Mail、RSS、Atom、Telegram and Twitter through the real `SourceManager.sync_source_types()` path then + produced exactly six rows；description、config、collect and backfill schemas matched each registered Source class exactly。 +- Existing deployment rows are not forcibly deleted：they may represent installed Sources and are protected by lifecycle/FK + concerns。The correction removes false core authority; catalog garbage collection remains outside scope。 +- `pdm run check` passes：foundation、lock、migration integrity、format、Ruff、Pyrefly and the admitted test baseline（7 + passed，40 skipped）。Hub `git diff --check` passes and normative PRD/Product TDD contains no Memos、RSS/Atom、Mail or + GitHub implementation reference after correction。 +- Hub's existing SVC adoption remains on corpus/config 10.0.1 while the installed CLI is 14.0.0；`svc status` therefore + reports its pre-existing project-upgrade work。This correction does not mix an SVC adoption upgrade into the owner diff。 diff --git a/tasks/knowledge-lifecycle-capabilities/capability-map.md b/tasks/knowledge-lifecycle-capabilities/capability-map.md index 2322692..e4eac86 100644 --- a/tasks/knowledge-lifecycle-capabilities/capability-map.md +++ b/tasks/knowledge-lifecycle-capabilities/capability-map.md @@ -1,41 +1,150 @@ -# Knowledge lifecycle capability map +# Knowledge Lifecycle Capability Map -## Decomposition +本文只解释 program 如何拆分、为何按这个顺序讨论，以及 queued work 在哪里。它不维护 active +phase、当前问题或具体单元的设计；这些由 [program packet](packet.md) 与 active unit packet +负责。 -| Trunk | Goal | Remaining candidate units | -| --- | --- | --- | -| Collection | persist source-specific information as reusable graph facts | Twitter/GitHub/Telegram hardening；CalDAV；Nextcloud Files；Apple Notes；future Memos collectors/products | -| Organization | improve later use of information already in the info-base | breakdown/interpretation、merge、linking and later evidence-backed approaches | -| Application | recover or navigate useful information | perceptual feature retrieval、hybrid composition and later use surfaces | +## 1. Decomposition Basis + +### Capability trunks -This table is a queue，not a roadmap。The next unit is selected after each closure from current user value、newly exposed -dependencies and uncertainty。 +收集、整理、应用是本任务要增强的三组**动作能力**，不是信息状态，也不是一个强制的信息 +生命周期。 + +| Trunk | Goal | Known units | +| --- | --- | --- | +| Collection | 把 source-specific information 可靠地持久化到 info-base | 现有 sources、memo-like、CalDAV、Nextcloud Files、Apple Notes | +| Organization | 打理已经存在的 info-base，以改善 use 效果 | breakdown、merge、linking；允许由真实目标发现其他能力 | +| Application | 从 info-base 取得有用结果 | 特征检索、语义检索、图导航检索；indexing 只是支撑 | -## Implementable-unit boundary +### Vertical implementable units -A unit is vertical enough to explain and verify： +讨论与实现以一个具体 source、organization operation 或 retrieval mode 为纵切。每个 unit +必须能够独立说明： ```text user value / observable failure - -> native input or use request - -> authority and cross-boundary contracts - -> graph / projection behavior - -> public-boundary acceptance - -> bounded implementation increments + → native input or use request + → owner and cross-boundary contracts + → graph / projection behavior + → executable acceptance + → bounded implementation increments ``` -Shared mechanisms are not a fourth trunk。A vertical unit may expose pressure on Block、Relation、Resolver、Storage、Source、 -Extension、AI、Job/Cron or Peer contracts；promote a common mechanism only when the current unit cannot correctly proceed -without it or repeated units demonstrate a stable shared seam。 +这样拆分的依据是可观察价值、单一 owner 和可验收的端到端闭环，而不是文件夹、抽象层或 +先造公共框架的便利。 + +### Cross-cutting pressures + +block、relation、resolver、storage、source、extension/registry 与跨仓合同不是第四条主线。 +它们只在具体 vertical unit 打破现有假设时进入设计： + +`上游需求 → 被打破的假设 → 候选 owner → blast radius → evidence → decision` + +压力先记录到 [pressure-ledger.md](pressure-ledger.md)。一个通用机制通常需要两个真实 unit +重复证明；单个 unit 若没有它就无法正确交付，也可以推动最小局部改造，但不能借机设计 +万能框架。 + +## 2. Joint Information Semantics + +所有 unit 共享的已确认约束是： + +```text +block.content ───────────────┐ + ├─ resolver ─→ solved / use-facing representation +storage ─→ hydrated content ┤ +local relations ────────────┘ +``` + +- block 是 info-base 的基本持久信息单元；source-native Tweet、GithubRepo、FeedItem 等不会 + 另外形成并行 durable object store。 +- relation 使多个 blocks 形成 graph，并可能参与 root block 的完整意义。 +- storage 只负责在需要时按 pointer 取得 actual content；它不是 semantic owner。 +- block hydration 隐藏 inline/pointer 分支，resolver 联合 hydrated content 与 local relations 产生明确的 + 解释/projection。 +- collection 可以为了正确持久化 source information 而拆成多个 blocks/relations； + organization 则处理已经存在的 info-base，目的不同。 + +这些语义约束讨论方向，但不预先规定每个 source 的 graph shape。 + +## 3. Program Queue + +| Order | Unit family | State | Why here | +| --- | --- | --- | --- | +| 1 | [Memos extension](units/memos-extension/packet.md) | **Complete；backend MVP implemented** | Sir 的直接产品需求；released client E2E 已证明 memo canonical/graph/read contract；durable owner projections committed | +| 2 | [RSS extension hardening](units/rss-extension-hardening/packet.md) | **Complete；human-accepted 2026-08-03** | 已用 RSS/Atom vertical 建立 source instance → collect job → graph → resolver → state 的可信 collection baseline | +| 3 | [Mail extension](units/mail-extension/packet.md) | **Complete；implementation/J1–J4/promotion complete** | 高价值真实邮箱 corpus 已证明 protocol → graph → materialization → generic InfoBase browser 纵切，且 durable owner delivery 已关闭 | +| 4 | [GitHub extension](units/github-extension/packet.md) | **Paused — owner correction first** | 首轮真实账号验收完成；Hub/Spoke 与 core/Extension owner correction 关闭后再修正实现 | +| 5 | Remaining collection units | Queued | CalDAV、Nextcloud Files、Apple Notes 各暴露不同 access/identity/storage/runtime 压力，不提前压成一个 source framework | +| 6 | [Semantic retrieval](units/semantic-retrieval/packet.md) | **Complete** | real-provider、local/delegated Peer、rumination 与 shared-truth projection 均已验收关闭 | +| 7 | [Feature retrieval](units/feature-retrieval/packet.md) | **Complete** | Lexical increment 的实现、J1–J7、core/client promotion、真实 fork/cold-start 与 exact-main Pages delivery 均已验收；graph facts 与 hybrid composition 仍由相邻能力承担 | +| 8 | [Graph navigation retrieval](units/graph-navigation-retrieval/packet.md) | **Complete** | bounded neighborhood/path、peer-local topology、Graph View、preview/production acceptance 与 durable closure 已完成 | +| 9 | Other organization/application units | Queued | breakdown、merge、linking 等仍各自从真实 use/failure evidence 建立合同；hybrid retrieval 等基础 primitive 完成后再组合 | + +这不是永久开发顺序。active unit 结束时，应根据用户价值、已暴露依赖和不确定性重新选择下一个 +unit；不得仅因为表格编号自动启动。 + +### Memo-like queue boundary + +- active ownership unit 是 `memos-extension`；当前 MVP delivery scope 才是 Memos + 0.29.1-compatible backend，MoeMemos Android 2.0.4 是 acceptance client。 +- Memos collector 是同一 extension 的 future delivery scope；它会重新打开 external identity、 + reconciliation、cursor 与 delete observation 等问题，但不另建 canonical ownership unit。 +- flomo 已校正为正确产品名称，但没有已证明的 official-client replaceable-backend path； + backend 暂不启动，collector 也不驱动当前 CanonicalMemo v1。 +- 未来其他 memo products 通过 product/generation adapter 检验 memo-family canonical + boundary，不直接复制 Memos shape。 + +## 4. Why Collection Units Differ + +| Unit | Main pressure exposed | +| --- | --- | +| Memos extension / backend MVP | native-compatible API、CanonicalMemo、graph round-trip、terminal-user boundary、transaction | +| Memos/other collectors (future) | external identity、scan/event reconciliation、cursor、source deletion | +| CalDAV | discovery、sync-token、recurrence、timezone、participants、ETag conflict | +| Nextcloud Files | hierarchy、path vs file identity、rename/move、binary、remote storage、permissions | +| Apple Notes | local macOS runtime、TCC、Notes.app/iCloud eventual sync、offline bridge | + +这些压力共同演化 collection 能力，但 source-native semantics 不会为了获得一个统一 schema 而 +被压平。 + +## 5. Discussion and Delivery Order Inside One Unit + +每个 unit 使用同一组 gate；approval 顺序稳定，但 supporting artifacts 可以提前作为探针： + +1. **Product**：用户旅程、纳入/排除、成功与可观察失败。 +2. **Technical**：authority、topology、API/data contract、compatibility、failure/partial-effect semantics。 +3. **Acceptance**：native input → persisted graph → resolver/use output，以及错误和重复执行；可在 + Technical 阶段先形成草案以暴露缺口。 +4. **Implementation Plan + Preflight**：先形成 design-probing draft，再核实版本、代码地址、依赖、 + 环境和失败分支；任何新 owner/behavior 都退回 Technical/Acceptance 讨论，不留到 Execute。 + Technical/Acceptance 获批且 preflight questions 关闭后才冻结为 execution baseline。 +6. **Impact Handshake + explicit start**：Sir 审查 state diff 后才修改 durable docs 或代码。 +7. **Execute / Verify / Promote**：实现闭环，再把稳定 truth 投影到唯一 durable owner。 + +一次尽量只讨论一个会改变设计的问题。supporting evidence/acceptance/plan 可以提前探索下游 +gate，但不能把“已经写成草案”误当作“已经获批”或“可以 Execute”。 + +## 6. Evidence Boundary + +- Hub PRD/Product TDD 含有宽泛的 collection、organization、retrieval 与 extension claims， + 但其中边界可能源自旧设计困境，必须交叉验证。 +- core-py 已有 source lifecycle、extension loading、resolver/storage registries、graph + persistence 与 embedding primitives；具体正确性与承载能力按 unit preflight 验证。 +- client-web 有 source/extension 管理与 browser-extension loading，但不能据现状推导完整 + registry 或 retrieval product contract。 +- 详细三仓术语/实现证据在 [terminology-audit.md](terminology-audit.md)；讨论结果只在 + [decision register](decisions/index.md) 登记一次。 + +## 7. Withdrawn Frames -## Completed baseline +以下内容没有项目术语或已确认需求作为依据，不再用于组织讨论： -- **Memos extension**: Memos-compatible backend MVP and MoeMemos journey；collector/flomo scope remains future work。 -- **RSS extension**: RSS/Atom collection、identity、content acquisition、media materialization and Resolver/Storage baseline。 -- **Mail extension**: IMAP collection、Jobs/Crons、communication graph、materialization and generic InfoBase rendering。 -- **Semantic retrieval**: AI provider/model/profile、embedding maintenance、local/Peer retrieval and focal rumination。 -- **Feature retrieval**: Block-local lexical projection plus system-driven multimodal interpretation。 -- **Graph navigation retrieval**: bounded neighborhoods、shortest-by-hop paths and progressive InfoBase Graph View。 +- `observation`、audit、replay 或独立“图准入”阶段； +- 把 collection / organization / use 当作信息状态迁移； +- 把 block / relation / resolver / storage 的联合语义升级成一条先于用户能力的工作主线； +- 固定“先写 Hub docs、再造 registry、再批量实现所有 source”的 program DAG； +- 把 `SubGraphForm` 当作完整信息模型或 collection 产品产物。 -Canonical behavior and technical contracts are owned by `docs/_shared/10-prd`、`docs/_shared/20-product-tdd` and the relevant -Spoke-local durable docs—not by this queue。 +Durable docs 是获批设计的最终投影，不是讨论顺序的起点。候选更新统一进入 +[documentation-promotion.md](documentation-promotion.md)。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D001-D010.md b/tasks/knowledge-lifecycle-capabilities/decisions/D001-D010.md new file mode 100644 index 0000000..628287a --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D001-D010.md @@ -0,0 +1,76 @@ +# Decisions D-001–D-010 + +> [Decision register index](index.md) + +### D-001 — Program trunks + +- **Decision**: 本任务以收集、整理、应用三组能力及其纵向切片为主线。 +- **Implication**: 术语、info-base 联合模型、extension / registry 与跨仓边界都是横切合同， + 不能取代能力主线。 +- **Confidence**: Sir confirmed。 + +### D-002 — Actions, not states + +- **Decision**: collection、organization、use 是对信息执行的动作，不是信息状态。 +- **Implication**: 不建立未经需求证明的信息生命周期。 +- **Confidence**: Sir confirmed。 + +### D-003 — Collection persistence model + +- **Decision**: Tweet、GithubRepo、FeedItem 等是 source-specific 输入形状；collection 通过 + 持久化 blocks / relations 完成收集，不引入通用采集对象或并列 source object store。 +- **Implication**: graph 映射、identity 与更新语义按 source slice 讨论。 +- **Confidence**: Sir confirmed and code-aligned。 + +### D-004 — Joint information semantics + +- **Decision**: block 是基本持久信息单元，但其可用含义不由 block row 单独决定；resolver + 联合解释 raw content 与 local relations，storage 只在需要时取得 raw content。 +- **Implication**: collection、organization、application 的设计都必须明确消费或产生哪种表示。 +- **Confidence**: Sir confirmed and code/history-aligned。 + +### D-005 — Organization goal and open capability set + +- **Decision**: organization 为 use 优化 info-base；breakdown、merge、linking 是已知能力， + 不是完备枚举。 +- **Implication**: 允许从真实 use 场景发现其他 organization 能力，但不得无目标扩张。 +- **Confidence**: Sir confirmed。 + +### D-006 — Indexing boundary + +- **Decision**: indexing 不属于 organization；它只作为 application / retrieval 的支撑。 +- **Confidence**: Sir confirmed。 +- **Reconsideration status**: D-081 已关闭 semantic-retrieval 中的 owner 复审；organization 仍止于 + blocks/relations graph，embedding/retrieval 仍属于 info-base use/interface。 + +### D-007 — Existing truth is fallible + +- **Decision**: 现有 Hub、代码和 Sir 的判断都需要交叉验证；Hub 是获批稳定真相的最终归属， + 不是讨论中无需核验的前提。 +- **Confidence**: Sir confirmed。 + +### D-008 — memo-like product role + +- **Decision**: memo-like 指低摩擦收集用户想法、周围事物与零碎信息的多端应用类别；它是 + InKCre 的重要 collection surface，但不承担 info-base 的查阅或使用。 +- **Implication**: 本切片的产品价值首先是随时随地可靠收集，不能把 InKCre 的 use 能力 + 偷渡进 memo 客户端范围。 +- **Confidence**: Sir confirmed；flomo 官方产品描述与该类别相符。 + +### D-009 — Productized memo integrations + +- **Decision**: InKCre 不依赖不存在的通用 memo 协议，而是通过 core extensions 支持多款 + 有代表性的 memo 产品；Memos 是已确认代表。 +- **Implication**: 各产品可有独立 access 与 graph mapping；只有重复压力才提升为公共 + extension / registry 合同。 +- **Confidence**: Sir confirmed；现有 extension 边界与方案方向相容，承载能力待 Technical + 阶段验证。 + +### D-010 — Dual memo integration relationship + +- **Decision**: memo extension 同时支持 InKCre 作为 memo backend，以及 InKCre 从既有 + memo 服务进行 collection；两种关系都汇入 extension 的 collection boundary，再持久化 + 到 info-base。 +- **Implication**: 两条路径共享后半段 collection 语义，但各自保留独立的接入、身份、 + 变更与失败合同，不能用一个含混的“同步”合同覆盖。 +- **Confidence**: Sir confirmed。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D011-D020.md b/tasks/knowledge-lifecycle-capabilities/decisions/D011-D020.md new file mode 100644 index 0000000..204cdf8 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D011-D020.md @@ -0,0 +1,94 @@ +# Decisions D-011–D-020 + +> [Decision register index](index.md) + +### D-011 — Collection success from the user perspective + +- **Decision**: backend 路径中，memo 客户端显示保存成功即代表该 memo 已被 InKCre + 持久化到 info-base；collector 路径的可达能力和成功确认依赖目标产品提供的 export / + automatic-backup API。 +- **Implication**: backend endpoint 不能在 graph 持久化完成前返回成功；collector 的详细 + 成功、部分成功和可见性语义仍需按产品讨论。 +- **Confidence**: Sir confirmed；事务边界与各平台接口能力待 Technical / Acceptance + 阶段验证。 + +### D-012 — Memo graph boundary + +- **Decision**: 每个 memo 默认有一个承载 memo identity 与主要文本的根 block；图片和其他 + 附件在需要独立 storage / resolution / use 时成为 component blocks，并通过 relation + 保留角色。只有来源本身提供可独立寻址的正文片段时，才继续拆分 text blocks。 +- **Implication**: 是否拆 block 由独立身份和使用价值决定，不由 MIME type 机械决定； + 附件顺序没有被证明是 memo 的通用 graph 语义。 +- **Confidence**: Sir confirmed；relation payload contract remains open。 + +### D-013 — Attachment association is unordered by default + +- **Decision**: memo attachment relations 默认为无序关联。若正文显式引用附件，正文是位置语义 + 的唯一 authority；只有来源明确把附件顺序定义为稳定语义时，extension 才另行保留。 +- **Implication**: memo 不使用 `attachment:` 作为通用合同，也不因附件排序单独推动 + relation schema 变化。 +- **Confidence**: Sir confirmed the default rule。Memos 0.29.1 preflight later proved a product-specific + exception：its request order is deliberately persisted and returned，see D-040。 + +### D-014 — Durable documentation changes are accumulated + +- **Decision**: 讨论中产生的稳定结论、新架构理解和待纠正文档先记录在 task packet 的 + `documentation-promotion.md`，不逐次修改 durable docs；形成内聚批次后再统一审查和应用。 +- **Implication**: “统一应用”指一个协调的 promotion batch，不改变 Hub source、Spoke + shared-ref 与 Spoke-local docs 必须按 owner 分离操作的规则。 +- **Confidence**: Sir confirmed。 + +### D-015 — Memos is the first reference product + +- **Decision**: 首个 memo → graph mapping 以 Memos 为 reference product。 +- **Implication**: 当前先由 Memos extension backend MVP 的真实 native shape 检验 + canonical/graph/read + 边界；未来 collector 作为独立 unit 再检验 reconciliation。只有重复压力才提升为通用 + memo extension 合同，不以不存在的通用 memo schema 起步。 +- **Confidence**: Sir confirmed and official API evidence available。 + +### D-016 — Memos compatibility starts at the current API era + +- **Decision**: 不支持旧历史版本；实现只面向最新版本，或最近一次 breaking change 之后的 + 当前 API generation。方案必须预留未来 breaking generations 的多版本适配能力，不能把 + 当前 shape 假设为永久合同。 +- **Implication**: 首版精确 target 已由 D-031 固定为 0.29.1；0.27–0.28 与 0.30 只作为 + compatibility/research evidence。未来 adapter retirement policy 仍需在出现第二代时确定。 +- **Confidence**: Sir confirmed at policy level。 + +### D-017 — Comments are independently collected memos + +- **Decision**: Memos comment 从首版开始完整收集为独立 memo block，并以 relation 连接 + parent；它复用 memo 的 graph mapping，不扁平化进 parent content。 +- **Implication**: 当前 backend 必须以独立 comment fixture 覆盖 create/read/change/delete； + future collector 若采集 comments，也复用同一 graph semantics。parent lifecycle 不能导致 + comment graph 静默丢失或残留。 +- **Confidence**: Sir confirmed and Memos native model aligned。 + +### D-018 — Current-generation adapter policy + +- **Decision**: 每个 memo product adapter 支持当前 API generation；breaking release 产生新 + adapter，新旧 generation 可在迁移期短期并存，旧 adapter 不永久保留。`latest` + main-branch schema 不等于 released wire contract。 +- **Implication**: backend 只暴露其配置声明的 released generation，当前为 0.29.1；future + collector 必须识别 product generation 并拒绝 unsupported generation。 +- **Confidence**: Sir confirmed。 + +### D-019 — CanonicalMemo is the memo-family semantic boundary + +- **Decision**: memo extension 由 product-specific、generation-specific adapters 将 Memos / + flomo 等 native shapes 转换为 `CanonicalMemo`，再由 canonical mapping 写入 info-base + graph；任何单一产品的 shape 都不能直接成为 memo extension 的核心模型。 +- **Implication**: `CanonicalMemo` 只覆盖 memo-like 产品族，不是系统级 collected object， + 也不形成与 block / relation graph 并列的 durable store。 +- **Confidence**: Sir confirmed at topology level；persistence/version boundary 已由 D-020/D-026 + 关闭，exact v1 wire later closed by D-042。 + +### D-020 — CanonicalMemo is persisted in root block content + +- **Decision**: `CanonicalMemo` 的序列化表示直接存入 memo root block 的 `content`；它不是 + 只在 adapter / mapper 间存在的 transient model。resolver 读取 CanonicalMemo content， + 并联合 local relations 解释 attachments、parent、references 等 graph context。 +- **Implication**: CanonicalMemo 是 durable、versioned content contract，但仍属于 + block / relation graph，不构成与 info-base 并列的 object store。 +- **Confidence**: Sir confirmed。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D021-D030.md b/tasks/knowledge-lifecycle-capabilities/decisions/D021-D030.md new file mode 100644 index 0000000..c8b51c7 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D021-D030.md @@ -0,0 +1,102 @@ +# Decisions D-021–D-030 + +> [Decision register index](index.md) + +### D-021 — Backend implements a minimal compatible API + +- **Decision**: InKCre-as-backend 除 collection/write endpoints 外，还实现目标客户端正常 + 工作所需的 read endpoints；只支持经过验证的最小兼容子集，不复刻完整产品 backend。 +- **Implication**: endpoint scope 必须从目标 client journey 与实际调用证据得出；adapter + 需要 native request → CanonicalMemo，也需要 graph / CanonicalMemo → native response。 +- **Confidence**: Sir confirmed at product-policy level。 + +### D-022 — Canonical content excludes graph-owned components + +- **Decision**: CanonicalMemo content 排除 attachments、parent 与 memo references；这些具有 + 独立 identity 或 graph value 的信息只由 blocks / relations 表达。 +- **Implication**: 这是 canonical-content / graph 的通用 authority pattern；resolver + 联合 root content 与 relations 解释完整 memo，禁止重复持久同一事实。 +- **Confidence**: Sir confirmed。 + +### D-023 — Backend reads are resolver-mediated + +- **Decision**: product backend read adapter 不直接读取和解释 block / relation rows；它通过 + memo resolver 获取 graph 的 solved result，再转换为 product-native response。 +- **Implication**: resolver 是 graph → memo semantics 的 owner，adapter 是 memo semantics + → product protocol 的 owner。 +- **Confidence**: Sir confirmed。 + +### D-024 — Concrete capability pressure may evolve core contracts + +- **Decision**: 讨论从 memo-like、Apple Notes 等具体上游需求出发；当它们暴露 info-base、 + collection、organization、use 或 extension 的缺口时，显式记录传导链并允许系统合同演进。 +- **Implication**: 不把现有 core 当作不可变前提，也不把尚未由具体压力证明的横切抽象提前 + 升级为主线。 +- **Confidence**: Sir confirmed。 + +### D-025 — Local memo identity is the block ID + +- **Decision**: 在 InKCre info-base 内，memo 的 local identity 使用 `block.id`，不在 + CanonicalMemo content 中复制一个 canonical `id`。 +- **Implication**: source-native identity 可以帮助 resolver 匹配已有 block，但它不取代 + `block.id` 作为 info-base local identity。 +- **Confidence**: Sir confirmed for local identity。 + +### D-026 — Canonical content generation is selected by resolver identity + +- **Decision**: CanonicalMemo payload 不携带 schema version；memo root block 通过 versioned + resolver identity 绑定 exact canonical decoder。新的 product API generation 不自动产生 + 新 canonical generation，新的 canonical shape 才产生新的 resolver identity。 +- **Implication**: registry 必须保留仍被已持久 blocks 引用的 decoder generations；未知 + resolver identity 必须明确失败，不能猜测、fallback 或把新 decoder 用于旧 content。 + 当前不增加 `BlockModel.content_schema_version`。 +- **Confidence**: Sir confirmed the recommendation；current core-py / client-web registries have + implementation gaps to address later。 + +### D-027 — Resolver-owned, best-effort cross-system consistency + +- **Decision**: 不建立通用 source binding table。source-native provenance / identity 在对 use + 或精确 reconciliation 有价值时持久到 resolver-owned `block.content`；future memo + collector 中即 CanonicalMemo。resolver 基于这些事实尽可能匹配已有 block。 +- **Consistency boundary**: 不追求低 ROI 的跨系统全局一致性。无法可靠匹配而产生的 + duplicate、revision fork 或 weak link 进入已有 info-base 后，由 organization 通过 + merge / linking / 其他能力优化 use 效果。 +- **Safety invariant**: resolver 不确定时宁可产生可整理的重复，不得模糊命中并覆盖 + 错误 block。organization 只处理已有 info-base 的后果，不得把 collection 失败 + 伪装成成功。 +- **Confidence**: Sir confirmed the ROI / responsibility direction；exact CanonicalMemo fields and + resolver matching contract remain open。 + +### D-028 — Source ID is an acceptable reconciliation-scope fallback + +- **Decision**: reconciliation 优先采用 source-native memo ID 及其稳定 external namespace； + 产品不提供稳定 instance identity 时，允许使用 InKCre `source_id` 作为本地 best-effort + namespace。future Memos collector 因此可以使用 source ID + instance-wide memo UID 精确 + 匹配；当前 backend 自己是 memo authority，不需要这层 mapping。 +- **Semantic limit**: source ID 不是 external provenance truth，也不承诺 source 被重建、合并 + 或重复配置后仍能识别同一外部 memo。可用的 instance locator 仍可作为 provenance / + diagnosis fact 持久化，但不能被描述成 immutable instance ID。 +- **Failure rule**: scope 失效时允许 duplicate / fork 并交给 organization 改善 use；禁止以 + content、时间或 mutable username 做 fuzzy overwrite。 +- **Confidence**: Sir confirmed source-ID fallback and best-effort exact identity。 + +### D-029 — memo-like delivery is backend-first + +- **Decision**: `memos-extension` 的首个 MVP delivery scope 是 Memos-compatible backend； + collector 延期为同一 extension 的 future scope。backend MVP 只实现选定 released + generation 的最小 API subset，MoeMemos 是首个 compatibility acceptance client，而不是 + API authority。 +- **Implication**: “minimum API”由 upstream Memos contract 与 MoeMemos 真实调用的交集界定； + 不复刻完整 Memos server，也不实现 client-private protocol。flomo 与 collectors 均不属于 + backend MVP，但不因此被建模为另一个 extension ownership unit。 +- **Confidence**: Sir confirmed product priority, MoeMemos target, and deferral of flomo/collectors。 + +### D-030 — flomo backend is deferred + +- **Decision**: `FlowMo` 已校正为 `flomo / 浮墨笔记`。公开产品合同没有显示官方 flomo 客户端 + 支持 custom backend；在不存在可使用客户端/protocol 的情况下，本任务不实现 flomo-like + backend。 +- **Implication**: flomo 不再驱动当前 CanonicalMemo/API scope；未来只有发现受支持的 + replaceable-backend path、独立兼容客户端，或启动 collector task 时才重开。 +- **Confidence**: Sir confirmed deferral if backend path is unavailable；official docs support the + feasibility finding。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D031-D040.md b/tasks/knowledge-lifecycle-capabilities/decisions/D031-D040.md new file mode 100644 index 0000000..55d1fc8 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D031-D040.md @@ -0,0 +1,153 @@ +# Decisions D-031–D-040 + +> [Decision register index](index.md) + +### D-031 — First Memos backend generation is 0.29.1 + +- **Decision**: 首版 Memos-compatible backend 以 released Memos 0.29.1 API generation 为 + protocol target，MoeMemos 2.0.4 为首个 compatibility acceptance client；不支持更旧 + historical generations。 +- **Implication**: v0.30 native shape research 仍可检验 canonical semantics，但不能直接成为 + 首版 wire contract。MoeMemos 后续支持 v0.30 时，通过 generation adapter 演进，不在首版 + 同时实现两代。 +- **Confidence**: Sir confirmed。 + +### D-032 — Memo-authored time is not block persistence time + +- **Decision**: block `created_at / updated_at` 只描述 info-base row persistence；CanonicalMemo + semantic minimum 使用 nullable、timezone-aware `created_at / updated_at` 表达 memo-side + authored/source times。 +- **Implication**: backend 按 memo service contract 建立这些时间；future collector 在来源没有 + 可信时间时保持 null，不能用 collection time、block time 或运行机器 timezone 猜测填补。 +- **Confidence**: Sir confirmed the authority distinction；exact v1 serialization/default behavior + later closed by D-042。 + +### D-033 — Deployment is the product user/owner scope + +- **Decision**: 当前 InKCre 产品不建立 deployment 内的 terminal-user、tenant 或 per-user + ownership/ACL domain model；一个 deployment 表达单一 user/owner context。多个 + `ClientModel` 是围绕同一 info-base 的 runtime peers，不是多个人类用户。 +- **External identity boundary**: source account、external author 或兼容协议中的 `user` 可以是 + configuration、provenance 或 wire projection，但不会自动升级为 InKCre core User。 +- **Memos realization**: backend 投影一个 deployment-scoped Memos-compatible profile/settings； + memo rows 不增加 user/tenant owner，也不复用 `ClientModel` 表示人。Bearer credential 的 + exact contract later closed by D-039/D-047。 +- **Evolution rule**: future multi-user support 若出现，必须作为显式的新产品/跨单元变更处理， + 不能由某个 adapter 静默引入。 +- **Confidence**: Sir confirmed at program level；current code/shared topology aligns with the + absence of a terminal-user model, while durable shared docs do not yet state this product policy。 + +### D-034 — Missing updateMask uses adapter-local key-presence inference + +- **Decision**: Memos 0.29.1 adapter 接受 MoeMemos 2.0.4 缺少 `updateMask` 的 PATCH，并从原始 + JSON 中实际出现的可更新 keys 推导 mask。这是明确命名、测试和隔离的 compatibility shim， + 不是严格 upstream parity。 +- **Explicit-mask rule**: 请求提供合法 `updateMask` 时采用 Memos 0.29.1 语义，不额外推导或 + 扩张 mask。 +- **Presence rule**: 推导依据是 key presence，不是 truthiness 或反序列化后的 non-null values； + `false`、`""`、`[]` 都是有效更新。未知/不可更新 key、空 inferred mask 明确失败；`null` + 除非上游字段合同明确允许，否则也失败。 +- **Boundary**: shim 只属于 Memos 0.29.1 generation adapter，不下沉为 CanonicalMemo、memo + application service 或 core 的通用 PATCH 规则。 +- **Confidence**: Sir selected option 3 after reviewing the upstream/client conflict and inference + semantics。 + +### D-035 — Memos extension is the implementable ownership unit + +- **Decision**: 当前可实现单元的稳定 identity 是 `memos-extension`；`memos-backend` 只表示该 + extension 的首个 MVP delivery scope，不是独立 unit。 +- **Ownership consequence**: CanonicalMemo、memo graph mapping、memo resolver 与 + product/generation adapters 都属于 Memos extension。backend、collector 或未来其他接入关系 + 是该 owner 下可分别过 gate 的 delivery scopes。 +- **Approval boundary**: backend MVP 已获批的 Product/Technical/Acceptance 结论不自动批准 + collectors、flomo 或未来 generations；它们仍需自己的 scope/gates，但不重复发明 canonical + owner。 +- **Confidence**: Sir corrected the prior unit/MVP conflation。 + +### D-036 — Authentication is composed by route tree + +- **Decision**: public、peer JWT and extension-owned authentication are route policies，not three + mutually exclusive `ExtensionBase` modes。Core protected routers and ordinary extension routers use + `require_peer_jwt` by default。An extension that needs public or custom-auth routes explicitly uses an + auth-neutral root and composes child routers with no dependency、the peer dependency or its own + verifier。Memos uses a public detection child router plus a Memos-credential-protected child router。 +- **FastAPI consequence**: remove the catch-all peer `JWTMiddleware` and reuse its validator through + `Security`/`Depends` at router boundaries。Parent-router dependencies are additive and cannot be removed + by a child，which is why mixed-policy extensions require an explicitly auth-neutral root rather than + public exceptions under a protected parent。 +- **ExtensionBase seam**: retain one overridable dependency hook whose default is + `Security(require_peer_jwt)`；do not add a public/peer/self enum、path override、auth registry、extension + sub-app or middleware solely for these requirements。 +- **Management boundary**: backend MVP does not add `/memos/admin/*`。Credential setup/change/revoke + should use the existing peer-authenticated extension configuration surface rather than inventing a + parallel Memos administration API。 +- **Documentation requirement**: implementation plan、local Unit TDD and any promoted cross-unit + contract must preserve a minimal code-shaped example showing the core peer router、the default + `ExtensionBase` hook and the Memos public/protected child-router composition；prose alone is + insufficient。 +- **Follow-on decision**: D-039 owns the confirmed PAT config/update semantics and exact public detection + endpoint set；the route-auth composition remains D-036。 +- **Confidence**: Sir explicitly accepted this route-composition refinement and requested that the + minimal example survive into real implementation/documentation work。 + +### D-037 — Memos credential is long-lived by default + +- **Decision**: the deployment-scoped Memos Bearer credential has no time-based expiry by default。It + remains valid until explicitly replaced or revoked through the existing extension configuration + surface。 +- **Implication**: backend MVP does not add refresh tokens、login sessions、automatic rotation or + periodic re-authentication。Missing、invalid、replaced or revoked credentials fail authentication； + mobile sync must not stop merely because time elapsed。 +- **Boundary**: D-039 later fixed token input、ordinary raw config persistence and no replacement overlap + from InKCre's actual config trust boundary。 +- **Confidence**: Sir explicitly confirmed the default lifetime。 + +### D-038 — Extension enable/disable is same-process hot + +- **Decision**: extension `enable/disable` should change its HTTP route availability in the running + process；a normal toggle does not require an application restart。For this project's current risk and + production profile，directly adding/removing an extension-owned route set is an acceptable mechanism。 +- **Runtime shape**: one retained router/route-set handle per extension is the mutation boundary。 + `ExtensionManager` is the only writer；enable publishes one exact route set，disable unpublishes it + before `on_close()`，and re-enable must not accumulate duplicate routes。 +- **Complexity boundary**: MVP does not add a permanent per-route running dependency、readiness gate、 + request-drain generation、isolated ASGI dispatcher or restart-bound activation model。Framework- + specific route-cache/OpenAPI invalidation is localized behind the runtime host and protected by + lifecycle tests。 +- **Deployment boundary**: this contract assumes the current single-process/single-replica web runtime。 + A future multi-worker or multi-replica topology must reopen runtime-state propagation；it does not + preemptively complicate this unit。 +- **Confidence**: Sir explicitly preferred best-effort hot activation and accepted direct route mutation + after calibrating both failure likelihood and consequence for this non-critical production context。 + +### D-039 — Memos PAT is ordinary validated extension configuration + +- **Decision**: backend accepts one deployment-scoped、upstream-shaped + `memos_pat_[0-9A-Za-z]{32}` Bearer credential。Its raw value is ordinary Memos extension config：it is + persisted、loaded into runtime and returned through the existing peer-authenticated config surface。 + Omitted field preserves，`null` revokes，a valid string establishes or immediately replaces；the token + remains valid until replace/revoke，without overlap、expiry、refresh or Memos-specific admin API。 +- **Route contract**: only `GET /memos/api/v1/instance/profile` is public； + `/memos/api/v1/status` remains unregistered (`404`) so MoeMemos falls through to the v1 adapter；all + other implemented Memos and `/memos/file/*` routes require this PAT。 +- **Generic config consequence**: extension config update is current config + shallow patch → validate + through the existing extension `config_cls` → persist normalized JSON → update the running config。 + Invalid input or persistence failure leaves durable/runtime state unchanged。No Memos-only secret + transform or read projection is introduced。 +- **Boundary**: a future generic encrypted/redacted config facility may migrate Memos alongside other + credential-bearing configs；this MVP does not pretend that only this token is secret while current + Twitter、Telegram、IMAP and GitHub credentials remain recoverable。 +- **Confidence**: Sir explicitly accepted ordinary raw config persistence and the generic update + direction after reviewing the security/complexity trade-off；D-036/D-037 and released-client evidence + close the remaining route and lifetime parts of O-018。 + +### D-040 — Memos attachment order is a product-specific graph fact + +- **Decision**: D-013's unordered default remains the memo-family rule，but the Memos 0.29.1 adapter must + preserve attachment request order。The tagged server deliberately rewrites attachment timestamps in + reversed request order and lists them by that ordering；the repeated attachment field is therefore not + incidental transport order。 +- **Implication**: order belongs to attachment relations，not CanonicalMemo or attachment block content。 + Reordering changes the root → attachment relation set/targets without changing attachment identity or + raw bytes。D-044 later fixed the exact predicate/content grammar。 +- **Confidence**: deductive consequence of Sir-confirmed D-013 plus pinned Memos 0.29.1 source evidence。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D041-D050.md b/tasks/knowledge-lifecycle-capabilities/decisions/D041-D050.md new file mode 100644 index 0000000..ff0bd5c --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D041-D050.md @@ -0,0 +1,126 @@ +# Decisions D-041–D-050 + +> [Decision register index](index.md) + +### D-041 — Graph completeness is not an observable guarantee + +- **Decision**: backend mutation failure is not required to leave a perfectly pre-command graph，and the + MVP does not promise “no partial graph”。Do not add cross-component compensation、replay or cleanup + machinery solely to enforce that guarantee。 +- **Success boundary**: D-011 remains：a successful memo save means its primary memo root/change was + persisted。The protocol response reflects the committed state；the exact HTTP result for subordinate + attachment/comment failure remains fixture-owned。 +- **Engineering latitude**: use one PostgreSQL transaction where it is already the simplest local + implementation，but treat that as an implementation property，not a product/Acceptance invariant。 + Failure may leave orphan components、raw blobs or stale relations for later cleanup/organization。 +- **Confidence**: Sir explicitly rejected the “无部分 graph” guarantee after preflight review。 + +### D-042 — CanonicalMemo v1 root wire + +- **Decision**: CanonicalMemo v1 root content owns `body`、nullable timezone-aware `created_at` / + `updated_at`、`archived`、`visibility` and `pinned`。Backend-created memos always establish both semantic + times；future collectors may leave unavailable source times null。 +- **Wire**: deterministic JSON；timestamps are UTC RFC3339；visibility uses canonical lowercase + `private | protected | public`；unknown keys are rejected。`archived` is canonical semantics rather than + copying Memos' generation-specific `state` enum。 +- **Exclusions**: local identity remains `block.id`；canonical generation remains the resolver identity； + attachments、parent and references remain graph-only。 +- **Confidence**: Sir reviewed the proposed exact shape and reported no remaining issue。 + +### D-043 — Attachment raw bytes use PostgreSQL binary storage + +- **Decision**: add a generic PostgreSQL-backed binary storage。The raw table owns only a generated pointer + and `BYTEA` bytes；the attachment block owns attachment identity、filename、media type、decoded size and + its storage pointer。 +- **Reason**: this preserves graph/storage ownership and avoids inline base64 in canonical content or a + DB + filesystem compensation protocol。A shared PostgreSQL transaction may be used where convenient， + but D-041 means atomic graph completeness is not an Acceptance guarantee。 +- **Boundary**: exact table/symbol names are implementation addresses；no parallel Memos object store is + introduced。 +- **Confidence**: Sir explicitly approved PostgreSQL binary storage。 + +### D-044 — Memos v1 relation payload grammar + +- **Decision**: root → attachment relation content is `attachment:`；comment → parent is + `parent`；memo → referenced memo is `reference`。Resolver validates the grammar and rejects duplicate + attachment positions for one root。 +- **Implication**: reorder rewrites relation positions/targets without changing attachment identity or raw + bytes。This is a Memos extension contract on the existing open string payload，not a generic relation + schema change。 +- **Confidence**: follows D-040 and Sir's review that the remaining plan had no issue。 + +### D-045 — Repair client-web extension config routing + +- **Decision**: the MVP delivery includes a separate client-web fix changing generic extension config save + from `/{extension_id}/config` to core's `/extensions/{extension_id}/config` contract，with a focused + request-shape test。 +- **Boundary**: direct database editing remains possible in the trusted deployment；the path repair enables + the existing core-API flow and must remain a sibling-repo change batch/commit。 +- **Confidence**: Sir explicitly requested the repair after the preflight finding。 + +### D-046 — Memo delete uses primary success plus best-effort owned cleanup + +- **Decision**: successful memo delete guarantees that the target root is no longer returned by memo + list/read。The service then best-effort removes comment roots、parent/attachment relations、exclusively- + owned attachment blocks and raw bytes。 +- **Safety boundary**: cleanup residue is allowed under D-041；reference targets and any block without + proven exclusive ownership must not be deleted。Traversal is cycle-bounded；repeated delete of an unknown + root returns `404` in the 0.29.1 adapter。 +- **Confidence**: Sir explicitly accepted the minimum deletion contract。 + +### D-047 — Exact API fixtures are a bounded executable compatibility contract + +- **Decision**: fixtures cover only the approved Memos 0.29.1 backend subset，not the complete upstream + server API。Each endpoint fixture pins request/query/header、response/status/error shape and the associated + CanonicalMemo/graph/resolver effect where relevant。 +- **Evidence layers**: upstream fixtures express tagged Memos 0.29.1 wire；MoeMemos fixtures express the + 2.0.4 real call sequence and deliberate compatibility deviations such as missing `updateMask`；InKCre + fixtures express block/relation/storage and resolver output。 +- **Test realization**: pure adapter/serialization fixtures、ASGI route/auth fixtures and PostgreSQL + graph/storage fixtures remain separate；the pinned APK is the final client-level evidence。 +- **Confidence**: Sir confirmed this interpretation and approved it as the remaining exact API contract。 + +### D-048 — Memos extension separates family core、product generations and access modes + +- **Decision**: the stable Memos extension core owns CanonicalMemo、graph mapping/predicates、application + commands and resolvers。Product/generation adapters（Memos 0.29.1、future flomo generations）own only + native wire ↔ canonical/solved translation。Access modes（backend、future collector）own transport、 + scheduling/cursors/reconciliation and invoke the same extension core。 +- **Test consequence**: one canonical/graph contract suite is reusable by every adapter/access mode；each + product generation keeps its own request/response/error fixtures；backend and collector orchestration + tests remain separate。 +- **Complexity boundary**: design the package seams and dependency direction now，but do not invent a + generic adapter registry、collector framework or flomo DTO before a second concrete implementation + supplies pressure。 +- **Confidence**: Sir explicitly required the implementation/tests to leave room for flomo and collectors； + the boundary follows confirmed D-019/D-035 without expanding the current MVP。 + +### D-049 — Runtime acceptance is black-box-first，with static proof carrying structural verification + +- **Decision**: new units should put most type、signature、dependency-direction、exhaustiveness and other + structural verification into static mechanisms。Runtime tests should preferentially drive the public or + deployed boundary and observe durable/user-visible effects；direct schema/default/helper/parser tests are + not the default acceptance shape。 +- **Double/live boundary**: hermetic CI may use a protocol double behind the real transport boundary and + real persistence/runtime path。Where external availability is cheap，an opt-in live smoke should consume + a real service/feed and assert stable invariants rather than volatile exact data。 +- **Targeted-test exception**: retain a narrow white-box/pure test only when the behavior is valuable，cannot + be proven statically，and cannot be observed reliably through the black-box path；the implementation plan + must state that reason。Legacy low-value tests are removed when their replacement proof exists。 +- **Memos history**: this preference does not retroactively invalidate the closed Memos layered fixtures； + its released MoeMemos APK journey remains the final black-box evidence，while the extra layers were an + accepted complexity trade-off for that unit。 +- **Confidence**: Sir explicitly stated a general preference for black-box tests and assigning most + verification to static checking，and required RSS acceptance to use real RSS/Atom or at least a double。 + +### D-050 — Feed-authored content remains authority；fetched full text is independent enrichment + +- **Decision**: RSS/Atom-authored title、summary and content remain source-authored canonical facts。A + successful item-link fetch does not overwrite them；the extracted full article is an independent graph + component/enrichment with its own retrieval、failure and change behavior。 +- **Use consequence**: resolver/use-facing text may prefer the full-text enrichment when available，while + preserving access to the original feed representation。Enrichment failure alone does not turn an otherwise + valid feed-item collection into failure。 +- **Boundary**: exact component resolver、relation grammar、fetch policy and whether enrichment ships in the + first MVP slice remain Technical/Product-scope work；the authority split is fixed。 +- **Confidence**: Sir explicitly accepted this model。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D051-D060.md b/tasks/knowledge-lifecycle-capabilities/decisions/D051-D060.md new file mode 100644 index 0000000..681acef --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D051-D060.md @@ -0,0 +1,180 @@ +# Decisions D-051–D-060 + +> [Decision register index](index.md) + +### D-051 — RSS hardening is a behavior rewrite behind the existing extension identity + +- **Decision**: retain the `rss` extension product/artifact identity，but design a new MVP implementation。 + Existing collectors、`seen_ids`、unversioned resolver、graph shape and white-box tests are failure/requirement + evidence rather than an incremental modification base。 +- **Library boundary**: mature third-party libraries should own RSS/Atom parsing and，if admitted，article + extraction。InKCre continues to own HTTP policy、canonical mapping、native identity/reconciliation、graph、 + source job/state and observable failure behavior；a third-party feed reader must not introduce a parallel + feed/entry store or application model。 +- **Complexity boundary**: do not rewrite a parser，do not create a second extension，and do not use the rewrite + to redesign every source or build a universal source framework。 +- **Confidence**: Sir explicitly approved the behavior-rewrite framing and mature-library preference。 + +### D-052 — Full-text enrichment ships in the RSS MVP and is enabled by default + +- **Decision**: the first RSS rewrite delivery includes item-link full-text enrichment as a second vertical + slice after canonical feed collection。It is enabled by default and source configuration may explicitly + disable it。 +- **Trigger/failure**: do not infer truncation from arbitrary body length or similar heuristics。For an + eligible item link，new or meaningfully updated items receive one best-effort enrichment attempt under the + approved fetch policy；failure does not fail the primary feed-item collection。Unchanged items are not + unconditionally re-fetched on every collection run。 +- **Authority**: D-050 remains unchanged：feed-authored content is authority，full text is an independent + component，and use-facing text may prefer the component when present。 +- **Confidence**: Sir required enrichment to be enabled by default and accepted the other proposed MVP + boundaries。 + +### D-053 — Feed-item reconciliation uses a best-effort exact identity ladder + +- **Decision**: Atom uses `atom:id` first；RSS uses `guid` within the strongest stable feed scope；when the + protocol ID is absent，use the canonical alternate link。When neither a native ID nor a link exists，do not + manufacture an exact identity from a normalized-payload fingerprint；apply the source instance's explicit + `create` / `discard` unidentified-item policy，defaulting to `create`。 +- **Change behavior**: the same exact identity updates the existing local block。Native-ID changes and + content/time similarity never trigger fuzzy overwrite；possible duplicates remain available for later + organization。`create` treats every encounter with an unidentifiable item as a new block and records that + exact native identity was unavailable；it does not claim idempotency or reconciliation。`discard` skips the + item with an observable diagnostic。 +- **Scope/provenance**: prefer a protocol-proven stable feed identity；fall back to the local source instance + when RSS provides no stable external scope。Identity/provenance facts live in versioned feed canonical + content，not a generic binding table。The info-base identity remains `block.id`。 +- **Confidence**: Sir accepted the native-ID/link ladder，then explicitly withdrew the fingerprint fallback as + unnecessary complexity。After checking the low expected incidence，Sir selected new-block collection and a + source-configurable create/discard policy；default `create` follows that stated product preference。 + +### D-054 — Feed/channel information is an independent graph block + +- **Decision**: persist feed/channel-native information as an independent block；persist each item as its own + block and connect it to the feed block through a relation。Do not use source-instance configuration as a + substitute for feed-native information，and do not duplicate mutable feed metadata into every item root。 +- **Boundary**: a source instance owns collection configuration and runtime state；a feed block owns collected + feed/channel facts and may be independently resolved，queried and graph-navigated。Relation direction/content， + canonical feed shape and feed reconciliation remain Technical/Product follow-ups。 +- **Confidence**: Sir explicitly accepted this separation after reviewing its graph and authority consequences。 + +### D-055 — Feed/channel reconciliation uses protocol identity before local URL scope + +- **Decision**: identify an Atom feed by `atom:feed/atom:id` first；otherwise use the feed-declared + `atom:link[rel=self]` when available；otherwise use the configured feed URL within the local source-instance + scope。The same exact feed identity updates the existing feed block。 +- **Change behavior**: a configured-URL change forms a new feed block when neither protocol identity nor a + declared self link proves continuity。HTTP redirect handling and URL normalization are transport/Technical + details and must not silently invent cross-feed equivalence。 +- **Boundary**: the local source instance remains collection config/runtime state；its ID only scopes the last + fallback and is not persisted as the feed's external identity。 +- **Confidence**: Sir explicitly accepted the proposed feed identity ladder。 + +### D-056 — A source-time admission watermark may reduce unidentified-item duplicates + +- **Decision**: under RSS source policy `create`，an item lacking exact native ID/link may be filtered by a + low-cost source-time admission watermark。For the first successful contentful snapshot，or when the item has + no parseable source-native time，create a new block。Thereafter create only when the item's effective native + time is later than the previous successfully committed snapshot observation time；otherwise filter it。 + Policy `discard` continues to skip every unidentified item。 +- **Watermark fact**: capture `snapshot_observed_at` when the complete feed response is received，but advance the + RSS source state only after that collection succeeds。Do not substitute collect-job completion time；do not + advance it for `304 Not Modified`。Prefer a valid native `updated` time，then `published`/`pubDate`；exact field + projection remains Technical design。 +- **Semantics**: this is an admission heuristic，not item identity，reconciliation，deduplication proof or an + ordering contract。Scan all items rather than short-circuiting on document position。Late publication， + backdating or publisher clock skew may cause a new unidentified item to be filtered；that bounded loss is + accepted for this rare fallback instead of adding content fingerprints or snapshot-diff machinery。 +- **Reusable pattern**: when exact identity is unavailable，a cheap independently observed monotonic watermark + may bound repeated side effects，provided its weaker guarantee and failure modes stay explicit and the system + does not promote the heuristic into authority。 +- **Confidence**: Sir explicitly accepted the precise RSS watermark behavior and requested that the general + pattern/mind be queued for durable documentation promotion。 + +### D-057 — Enclosures are graph components with manual and source-policy materialization + +- **Graph authority**: persist each RSS/Atom enclosure as an independent metadata block and connect it to its item + through a relation。The enclosure metadata block owns the native URL，media type，declared + length/title and other supported feed metadata；the item root does not duplicate that association。 +- **Download surfaces**: the RSS extension exposes a peer-authenticated manual materialization endpoint whose + inputs identify enclosure metadata blocks。The command must pass each block through its exact installed resolver + rather than parsing `block.content` directly，then return the newly materialized semantic content block。Source config also + provides an automatic enclosure-download policy and selects the target + writable storage。 +- **Responsibility boundary**: the resolver interprets and validates enclosure semantics；an extension-owned + application service performs network/storage/graph side effects；storage only persists and retrieves actual-content + bytes。The downloaded block remains distinct from the enclosure metadata block so collection + authority is not overwritten by local materialization。 +- **Materialized semantics**: do not model the downloaded result as `StoredBinary` merely because storage holds + bytes。Create the semantic media block selected by the materialized content kind—normally audio，video or + image—and point that block at the selected storage。PDF，EPUB and ZIP also receive exact content resolver IDs + in this delivery；unknown or not-yet-supported downloaded kinds fall back to a concrete file + block with best-effort MIME type。Do not silently revive the rejected generic external `resource`/binding + model merely because the endpoint response was described as a “resource block”。 +- **Storage pressure**: existing `WritableStorage` plus built-in `postgresql_binary` can prove the strategy，but + its whole-bytes synchronous write contract is not evidence of large enclosure/S3 suitability。Technical + design must decide whether this unit adds a streaming capability and S3-compatible storage or verifies only + the existing target while retaining storage-ID configurability。 +- **Confidence**: Sir accepted the independent enclosure graph，required the manual extension endpoint，and + required source-configured automatic download into storage in this unit，even if that exposes a need for a + new storage type。Sir then corrected the materialized-block proposal：raw binary is a storage representation， + while the block should use content semantics；Sir explicitly included PDF/EPUB/ZIP and selected file with + MIME type as the unknown/unsupported fallback。 + +### D-058 — RSS may propagate media/storage corrections horizontally without a new unit + +- **Decision**: keep `rss-extension-hardening` as the sole active implementable unit。Media resolver，storage + and Memos attachment corrections discovered by the RSS vertical are horizontal state diffs inside this same + discussion/design/acceptance/implementation loop，not a separately sequenced foundation unit。 +- **Current Memos fact**: Memos attachments currently use `extensions.memos.attachment.v1`；their canonical + content combines protocol attachment metadata with a PostgreSQL `blob_id` and the repository hard-codes + storage `-4`。They do not use image/video/audio/file semantic blocks。 +- **Required correction**: this unit must move Memos attachment actual content onto the corrected + media/file + storage path while retaining the already accepted Memos 0.29.1 backend behavior。Whether the + protocol attachment remains a metadata block or its identity is projected directly from a media/file block + remains the next Product decision。 +- **Blast-radius rule**: cross-owner changes still need explicit addresses，state diffs and regression proof； + keeping one unit does not make RSS the durable owner of common media/storage contracts。 +- **Confidence**: Sir explicitly rejected a mechanically split foundation unit and requested that existing + Memos attachments be corrected in the same work if they do not already use semantic media blocks。 + +### D-059 — Independently useful protocol metadata has its own block + +- **Decision**: retain a Memos attachment metadata block and connect it to one + image/audio/video/PDF/EPUB/ZIP/file semantic content block。Memo ordered ownership targets the attachment metadata block； + its resolver joins the content relation to project the Memos-native attachment。Unattached uploads remain + discoverable through the exact Memos attachment resolver ID。 +- **Authority split**: the metadata block owns protocol identity，role，lifecycle and authored/declaration + provenance；the exact resolver ID expresses content kind，resolver solved content owns byte-derived facts，and + storage owns actual content by pointer。Do not duplicate one authority fact merely to make both blocks self-contained。 +- **Cleanup**: deleting a metadata block may clean its semantic content block/stored object only under explicit exclusive- + ownership proof。A shared semantic content block survives metadata-block deletion。 +- **Reusable pattern**: use `metadata block → semantic content block → storage` only when protocol/source-authored + metadata has independently useful identity，role or lifecycle。Do not mechanically add a metadata block when + the input's only meaning is already carried by the semantic content block。RSS enclosure and Memos attachment are the two current reference + pressures。 +- **Confidence**: Sir explicitly selected the two-layer Memos graph and identified it as a common pattern。 + +### D-060 — Block owns storage hydration without a second persistent pointer field + +- **Persistent shape**: retain the existing conditional meaning of `BlockModel.content`。When `storage` is + null，`content` is inline actual content；when `storage` is non-null，`content` is that storage instance's + opaque pointer。Do not add `storage_pointer` or a duplicate `BlockRecord` representation。Keep `content` + non-null and change storage deletion from `SET NULL` to `RESTRICT` while blocks reference it。 +- **Hydration API**: `await block.get_hydrated_content()` is the single general actual-content read path。It + returns inline `content` directly or resolves the pointer through the selected storage；resolver and other + consumers that need actual content must not interpret `storage` themselves。 +- **Lazy instance cache**: cache the returned actual content in ORM-non-mapped + `block._hydrated_content`。Use a distinct unloaded sentinel supplied by `PrivateAttr(default_factory=...)` so + an actual value cannot be confused with “not loaded” and Pydantic cannot deepcopy the sentinel identity。 + Newly loaded instances begin unloaded；the controlled block update path must invalidate the cache whenever + persisted `content` or `storage` changes。This is model-instance memoization，not a promise that an external + storage object cannot change；never assign hydrated bytes back to the mapped `content` column。 +- **Terminology**: retire the ambiguous “real content / raw content” pair。`content` is the persisted inline + value or storage pointer；`hydrated content` is the actual data returned by the block；resolver-specific + interpretation remains a separate semantic result。 +- **Metadata boundary**: MIME，size，checksum，filename/provenance and other semantic metadata must receive an + explicit authority in the media design；they must not be encoded into the opaque pointer or duplicated in + `content` merely because a storage-backed block otherwise has no inline metadata field。 +- **Confidence**: after comparing a separate persistent/runtime representation with the smaller active-record + change，Sir selected conditional persisted `content` plus block-owned lazy hydration/cache and accepted + storage-deletion `RESTRICT`。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D061-D070.md b/tasks/knowledge-lifecycle-capabilities/decisions/D061-D070.md new file mode 100644 index 0000000..b895fc2 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D061-D070.md @@ -0,0 +1,182 @@ +# Decisions D-061–D-070 + +> [Decision register index](index.md) + +### D-061 — Storage hydration is a peer-local capability + +- **Shared versus local**: storage type semantics，storage instance configuration and block pointer are shared + protocol state；the executable handler is a capability registered in each peer runtime。Schema/migration + authority does not make core-py a central content service。 +- **Hydration**: each peer's block hydration path selects its local handler。A missing handler is an explicit + unsupported-capability failure，not an implicit request to core-py or another peer。 +- **Delegation boundary**: future cross-peer execution would require generic capability discovery and an explicit + peer command contract。It must not be hidden inside ordinary block hydration or privilege one named runtime。 +- **Current unit coverage**: client-web must support the full content lifecycle of the built-in + `postgresql_binary` storage during this RSS/Memos horizontal slice。D-063–D-066 define complete CRUD，storage + independence，cache refresh and the exact PostgREST transport。 +- **Confidence**: Sir explicitly accepted the peer-local handler contract and required client-web PostgreSQL + binary support，while reserving approval of the low-level transport。 + +### D-062 — Storage transports opaque content bytes; resolver owns interpretation + +- **Storage contract**: a storage implementation locates and reads/writes a block's actual content as opaque + bytes（or an equivalent streaming byte source）。It does not parse JSON/HTML，decode media，or decide whether + the information is image，video，audio，PDF or another semantic kind。 +- **Resolver contract**: resolver owns content-kind interpretation and parsing，using the block's exact resolver ID + plus graph/metadata context。MIME may inform that interpretation，but it is not created as semantic + truth merely because a storage transport returned a header or holds bytes。 +- **Type naming pressure**: storage types should name access/persistence mechanics，not information kinds。The + existing `http_image` / `http_video` / `http_html` / `http_json` / `http_text` split and the now-redundant + “binary” emphasis in `postgresql_binary` require explicit redesign；this decision does not yet freeze their + replacement names or migration shape。 +- **Backing records**: an implementation may use a private/protocol backing relation to hold its bytes。That + relation is not a storage type，storage instance，block or semantic media object。`storage_blobs` is an accepted + name for the PostgreSQL implementation's backing relation；the earlier concern was conceptual classification， + not its table name。 +- **Confidence**: Sir identified this as a general pattern and explicitly assigned bytes transport/storage to + storage and image/video/PDF interpretation to resolver。 + +### D-063 — Client-web PostgreSQL storage covers complete CRUD + +- **Scope**: client-web `packages/core` must implement local create，read，update and delete support for the + built-in PostgreSQL bytes storage in this unit；read-only hydration is insufficient。 +- **Transport boundary**: the browser peer talks to the admitted database surface through PostgREST，not through + a privileged core-py content proxy。The precise raw-binary read/write function or media-handler shape remains + subject to Sir's review and a black-box proof against the pinned PostgREST v14.15 runtime。 +- **Contract projection**: the admitted backing relation/functions must be added to the generated client-web + database contract through its normal generation path，not by hand-editing generated TypeScript。 +- **Storage independence**: CRUD operates on opaque pointer plus bytes and never discovers，loads or mutates a + referencing block。The generic writable capability therefore needs a real update operation；for PostgreSQL the + natural candidate is updating `storage_blobs.data` under the same `blob_id`，not mandatory copy-on-write。 +- **Confidence**: Sir explicitly expanded this unit from PostgreSQL binary hydration to complete client-web + write/read/delete support，including the update part of CRUD。 + +### D-064 — Block record time does not claim hydrated-content freshness + +- **Timestamp meaning**: `block.updated_at` reports mutation of the persisted block record。When `storage` is + non-null，the referenced object may live outside the protocol database and change independently；the timestamp + therefore cannot generally be interpreted as the last modification time of hydrated content。 +- **Dependency direction**: block selects a storage and carries its opaque pointer；storage does not depend back on + block。Storage update/delete must not search for or mutate referencing blocks merely to manufacture block-level + cache or timestamp coherence。 +- **Cache boundary**: `_hydrated_content` / client-web's corresponding private cache is at most an instance-local + snapshot。A mutation coordinated by the same block/application path can invalidate its known instance，but no + generic cross-instance or cross-peer freshness guarantee is inferred。The explicit refresh/bypass API remains to + be decided。 +- **Derived data**: embeddings，indexes and other derived interpretations can become stale when externally stored + content changes without an InKCre command。Detection/reconciliation belongs to collection or organization policy， + not to the base storage CRUD contract。 +- **Confidence**: Sir corrected the earlier copy-on-write rationale by identifying external storage mutability and + rejected a reverse dependency from storage to block。 + +### D-065 — Hydrated-content cache is an explicitly refreshable instance snapshot + +- **Default read**: `await block.get_hydrated_content()` in core-py and the equivalent client-web method lazily + hydrate once and reuse the value held by that block instance。 +- **Explicit refresh**: callers that require a new storage read pass an explicit refresh option。Refresh bypasses + the cached snapshot，reads through the selected peer-local storage handler and replaces the instance cache with + the result。 +- **API projection**: Python uses `await block.get_hydrated_content(refresh=True)`；client-web exposes the same + semantic through `await block.getHydratedContent({ refresh: true })` while retaining idiomatic language shape。 +- **Non-guarantees**: this contract adds no TTL，background polling，storage-version inference，cross-instance cache + invalidation or cross-peer coherence。A storage handler may later use provider validation features internally， + but the base block API does not require them。 +- **Confidence**: Sir explicitly accepted the instance-local cache plus explicit refresh contract。 + +### D-066 — PostgreSQL browser CRUD uses raw C/R and relation U/D + +- **Create**: client-web sends bytes to an admitted `create_storage_blob(bytea)` RPC as + `application/octet-stream`。The function has PostgREST's supported single unnamed `bytea` argument and returns the + generated blob UUID used in the opaque pointer。 +- **Read**: client-web calls an admitted read RPC with `blob_id` and `Accept: application/octet-stream`。The function + returns an `application/octet-stream` domain over `bytea` so the raw transport consumes an `ArrayBuffer` rather + than the normal JSON response decoder。 +- **Update/Delete**: update PATCHes the exact `storage_blobs` row with PostgreSQL's `\\x...` bytea representation； + delete uses the exact UUID-filtered relation DELETE。This preserves in-place object identity and uses the existing + authenticated peer table authority。 +- **Why hybrid**: raw Create/Read avoid encoding overhead on the common，large-data paths。PostgREST raw function + input supports only one unnamed binary argument，so relation Update avoids inventing a custom identifier header or + binary envelope；its roughly 2× hex upload representation is accepted for the less frequent path。 +- **Client transport**: `packages/core` owns one small authenticated raw PostgREST fetch path for Create/Read；typed + generated PostgREST relation operations remain the path for Update/Delete。The admitted functions and + `storage_blobs` relation are projected into the generated database contract，not hand-maintained client types。 +- **Verification**: black-box the four operations through the pinned PostgREST v14.15 runtime，including byte-exact + round-trip，same-pointer update，missing UUID behavior，JWT/ACL denial and cache refresh。 +- **Confidence**: Sir explicitly accepted the raw Create/Read plus relation Update/Delete wire contract。 + +### D-067 — Media metadata follows its authority; no generic block metadata is added + +- **Protocol/source authority**: filename，protocol-declared MIME/length，source URL and source-authored timestamps + remain in the canonical content of a metadata block（for example RSS Enclosure or MemosAttachment）。A metadata + block is an ordinary block that owns protocol/source-authored facts about related content；it is not a separate + wrapper type or source-module abstraction。Those facts are not copied onto the related media/file semantic content block。 +- **Storage authority**: `blob_id`，object key，provider version and other retrieval mechanics belong only to the + selected storage's opaque pointer/config。They are not semantic media metadata。 +- **Resolver authority**: image/audio/video/PDF/EPUB/ZIP/file kind is expressed by exact resolver ID。Detected + MIME，actual byte size，checksum，dimensions，duration and similar byte-derived facts belong to solved content； + organization may materialize those that have durable use value as graph enrichment。 +- **Graph shape**: a metadata block relates to a semantic content block。The metadata block keeps protocol identity/role/ + metadata/lifecycle；the semantic content block keeps resolver identity plus inline content or an opaque storage pointer。 + The same semantic content block may therefore be referenced without collapsing distinct source facts。 +- **Schema restraint**: this unit does not add `blocks.metadata`。The current pressure is resolved by assigning + existing facts to metadata-block content，pointer/config，solved content and optional organization enrichment；a + generic JSONB field would add an unbounded competing authority without a remaining requirement。 +- **Memos correction**: `CanonicalAttachment.blob_id` moves out of the attachment metadata block content and becomes + the related semantic content block's storage pointer；the metadata block retains Memos attachment fields needed for native + list/read/delete behavior。 +- **Confidence**: Sir explicitly accepted this authority split，identified it as the correct design and promoted it + as a common pattern。 + +### D-068 — S3 is valuable but sequenced with Nextcloud Files, not RSS + +- **Decision basis**: the general utility of S3-compatible object storage is sufficient future product pressure； + it does not need to be justified by the rare RSS enclosure that cannot fit in memory。 +- **Current unit**: RSS enclosure materialization targets the already approved writable PostgreSQL storage。This + unit does not implement S3-compatible storage and does not make very-large，non-materializable enclosure streaming + an acceptance condition。 +- **Abstraction restraint**: do not add a speculative streaming writable-storage contract solely for future S3。 + Keep the current unit's byte-oriented contract and explicit download bounds；introduce/reshape streaming when an + actual storage implementation exercises it。 +- **Sequencing**: design and implement S3-compatible storage with the future Nextcloud Files extension，where object + storage，file scale，hierarchy/listing and incremental file synchronization provide stronger reference pressure。 + This is deferral to an identified unit，not rejection of S3。 +- **Confidence**: Sir explicitly rejected large RSS files as the S3 decision basis and selected the Nextcloud Files + extension as the more appropriate implementation point。 + +### D-069 — Each extension owns media-classification policy; core offers mechanisms + +- **Ownership**: no global MIME/kind evidence ladder is imposed across Memos，RSS，Atom or future extensions。The + extension adapter that understands its protocol owns which declared/observed evidence selects an exact resolver ID。 +- **Common capability**: `ResolverManager` may deepen its existing resolver registry with reusable MIME + normalization/detection helpers and MIME-to-registered-resolver matching，including an opt-in default helper。 + Extensions decide which evidence to provide，its order，when to call the helper and whether to override its result。 + Do not add a standalone media module or hide a mandatory precedence policy inside `ResolverBase`。 +- **Memos evidence**: Memos 0.29.1 Attachment has a protocol `type` MIME field alongside filename and bytes。Upstream + normalizes and accepts a provided type，falling back to filename extension and Go content detection only when it is + empty。The current backend MVP also requires a valid non-empty type。Memos can therefore treat it as its stable + declared-media-type input without mandatory byte sniffing。 +- **Feed evidence**: RSS 2.0 enclosure requires `url`，`length` and MIME `type` attributes。Atom link `type` is + optional and advisory；RFC 4287 says a dereferenced server response media type is authoritative over that hint。 + RSS and Atom adapters therefore need distinct explicit policies even inside one extension family。 +- **Terminology**: call protocol/HTTP values “declared/observed media type”，byte magic a “content byte signature”， + and reserve “signature” without qualification for contexts that actually define it；they are not interchangeable。 +- **Current correction**: the existing RSS canonical stores enclosure URLs only and drops type/length。The rewrite + must preserve the protocol attributes before any classification/materialization policy can be correct。 +- **Confidence**: Sir rejected the proposed universal ladder，assigned policy to each extension and allowed only a + reusable ResolverManager mechanism where it does not erase extension-specific semantics。 + +### D-070 — Memos Attachment uses its declared MIME without mandatory sniffing + +- **Input authority**: the Memos product adapter requires and normalizes `Attachment.type` as its protocol-declared + MIME。The MemosAttachment metadata block preserves that exact declaration for native list/read/download projection。 +- **Resolver selection**: the extension maps the normalized MIME through `ResolverManager` to the semantic content block's + exact resolver ID。Known media types select image/audio/video/PDF/EPUB/ZIP as applicable；an unknown or unsupported + MIME selects file while retaining the declared MIME on the metadata block。 +- **No universal verification**: byte-signature sniffing is not mandatory for a successful MoeMemos upload and a + mismatch does not override the declared Memos field or reject the upload。A future diagnostic/enrichment may inspect + bytes，but it is outside this compatibility write contract。 +- **Graph consequence**: MemosAttachment remains the protocol metadata block；its content relation points to the + media/file semantic content block whose storage-backed `content` is only the opaque pointer。Backend reads reconstruct + Memos MIME from the metadata block，not from storage or globally detected solved content。 +- **Confidence**: Sir accepted this Memos classification policy and corrected “protocol-declared media type” versus + “content byte signature”。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D071-D080.md b/tasks/knowledge-lifecycle-capabilities/decisions/D071-D080.md new file mode 100644 index 0000000..64023a6 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D071-D080.md @@ -0,0 +1,191 @@ +# Decisions D-071–D-080 + +> [Decision register index](index.md) + +### D-071 — RSS 2.0 enclosure type is the primary resolver-selection evidence + +- **Protocol basis**: a conforming RSS 2.0 enclosure has required `url`，byte `length` and MIME `type` attributes。 + The canonical RSS Enclosure metadata block preserves all three rather than reducing the component to its URL。 +- **Primary mapping**: a valid，specific `enclosure.type` is passed through `ResolverManager` to select the semantic + content block's exact resolver ID。An unknown but valid declared MIME falls back to file while the metadata block retains it。 +- **Fallback boundary**: HTTP response Content-Type，filename/URL evidence and optional byte detection participate + only when the RSS declaration is absent，invalid，generic（for example `application/octet-stream`）or cannot provide + a usable mapping。They do not silently rewrite the metadata-block field。 +- **Malformed feed boundary**: this classification decision does not yet decide whether a non-conforming enclosure + with missing required attributes is retained with diagnostics or discarded；that belongs to RSS item failure + policy。 +- **Confidence**: Sir explicitly accepted valid，specific RSS `enclosure.type` as the primary resolver-selection + evidence。 + +### D-072 — Atom enclosure uses dereferenced HTTP type before its advisory hint + +- **Protocol basis**: Atom `link.type` is optional advisory MIME；RFC 4287 says it does not override the actual media + type returned when `href` is dereferenced。The canonical Atom Enclosure metadata block preserves `href` and optional + `type`，`length` and `title` attributes exactly。 +- **Primary mapping**: materialization first uses a valid，specific HTTP Content-Type to select the exact resolver + ID。If the response type is absent，invalid，generic or unusable，the adapter tries the valid Atom + `link.type` hint。 +- **Fallback boundary**: only after both protocol-specific signals are unusable may the Atom adapter call optional + `ResolverManager` filename/URL or byte-detection mechanisms；an unidentified result becomes file。 +- **No source rewrite**: observed HTTP type and fallback results never overwrite the Atom metadata block's declared + `type`。A specific HTTP/link conflict selects the resolver ID from HTTP while retaining the original hint for + provenance。 +- **Confidence**: Sir explicitly accepted this Atom enclosure classification policy。 + +### D-073 — Resolver makes graph application-usable; lazy graph materialization is allowed + +- **Role**: resolver is the application-facing interpretation boundary for a block and its relevant graph。Its job is + not limited to pure decoding；it makes graph information usable by application consumers。 +- **Optional outcomes**: text and embedding-string representations may be unsupported or absent。The abstract base + methods remain mandatory so every concrete resolver explicitly implements the capability surface；an unsupported + resolver raises `UnsupportedResolverCapability`，and a supported resolver may return `None` for a block with no + meaningful value。Neither case is represented by a fake empty string。 +- **Lazy materialization**: resolution may call AI or another organization capability and persist missing derived + blocks/relations。A read-triggered write is not inherently incorrect；it can be a lazy-loading/materialized-view + behavior。 +- **Actual problem**: callers currently cannot express whether remote work，cost，latency and graph mutation are + allowed，and current image resolution mixes those effects with import-time credentials，fixed storage assumptions + and ad hoc persistence。The rewrite must expose a materialization policy and make its writes idempotent/ + concurrency-safe enough for the unit's acceptance，rather than ban side effects。 +- **Peer projection**: exact method names remain peer-local，but core-py and client-web must distinguish cached solved + projection refresh from permission to materialize missing graph information。 +- **Interface status**: D-074 closes the ordinary default and shared option vocabulary；D-075 closes the abstract + method/unsupported-result boundary and exact common resolver IDs。 +- **Confidence**: Sir identified the mandatory text/embedding contract as historical debt，accepted its refactor and + corrected the proposed pure-read restriction with resolver's application-facing/lazy-loading role。 + +### D-074 — Resolver materializes missing graph by default；refresh is cache replacement only + +- **Ordinary default**: resolver application use may materialize missing derived graph by default。A caller that + requires a side-effect-free attempt explicitly sets `materialize_missing=False`（TypeScript + `materializeMissing: false`）。This option is permission to create an absent derivation，not a command to replace + one that already exists。 +- **Refresh contract**: on a cache-bearing read，`refresh=True`（TypeScript `refresh: true`）bypasses the reusable + instance/local snapshot，re-reads the currently available authority and replaces that cache。It does not itself + authorize materialization，request AI，or regenerate an existing derivation；those effects remain governed by + `materialize_missing` or an explicit organization command。The two controls are orthogonal。 +- **Adjacent vocabulary**: `invalidate` discards a cached value without replacing it；`recompute` explicitly + regenerates an existing derived representation and belongs to organization；`reload` is not an alias for refreshing + information and should be reserved for a runtime/config/module lifecycle when such an API exists。New InKCre-owned + APIs must not use an unqualified `force` boolean for any of these meanings。A third-party protocol-owned `force` + parameter，such as the Memos API query shape，is preserved as protocol fidelity rather than renamed。 +- **Relation direction**: `include_in` / `include_out`（TypeScript `includeIn` / `includeOut`）are the stable direct- + relation selectors relative to the subject block：incoming means the block is `to_`，outgoing means it is `from_`。 + They do not imply recursive graph traversal。 +- **Scope restraint**: this vocabulary does not require every method to expose every option。It fixes the name when + that semantic control exists，while peer-local method names and Python/TypeScript casing remain idiomatic。 +- **Current migration pressure**: client-web resolver methods currently call cache replacement `force`；the common + resolver rewrite should migrate those InKCre-owned options to `refresh`。Legacy source-job `full` is not promoted： + it currently combines scan breadth，incremental-boundary bypass，ordering and pagination effects across sources，so + each retained behavior must be named from its source-specific contract instead of preserving one generic boolean。 +- **Confidence**: Sir selected ordinary missing-materialization with an explicit read-only override，required + `refresh` to become stable durable vocabulary and requested promotion of other genuinely common parameters。The + relation selectors are already aligned across core-py/client-web；the `full` exclusion follows direct code evidence。 + +### D-075 — Exact resolver IDs version semantic-content contracts + +- **Exact IDs**: new common semantic content blocks use `core.text.v1`，`core.html.v1`，`core.image.v1`， + `core.audio.v1`，`core.video.v1`，`core.pdf.v1`，`core.epub.v1`，`core.zip.v1` and `core.file.v1`。 + `core` means shared InKCre-owned semantics rather than core-py execution authority。 +- **Version meaning**: the `v1` suffix is the resolver contract version。It advances only for an incompatible + persisted/solved/graph contract change，not for a parser release or file-format minor version。Use conventional + `version` language rather than introducing `generation` for this axis。 +- **Block roles**: protocol/source-authored identity，declaration，role and lifecycle remain in a metadata block；the + related semantic content block owns its exact resolver ID plus inline actual content or a storage pointer。Both + are ordinary blocks and remain connected by the accepted `content` relation。 +- **Solved content**: `core.text.v1` solves to Unicode text and `core.html.v1` to decoded HTML source。The seven + byte-oriented contracts share `byte_size` and nullable `detected_media_type`；image/audio/video/PDF/EPUB/ZIP add + their approved bounded typed facts，and file adds no pretend format-specific facts。 +- **Capability methods**: capability is invoked directly on a resolver instance。`ResolverBase.get_text()` and + `get_str_for_embedding()` remain abstract so every concrete resolver declares behavior。An unsupported + implementation raises `UnsupportedResolverCapability`；a supported implementation may return `None` when the + particular block has no meaningful value。`ResolverManager` selects/constructs resolvers and owns shared registry/ + matching mechanisms，not instance capability dispatch。 +- **Hard cut-off**: remove the bare `text`，`html`，`image` and `video` implementations and update every in-repo + producer，consumer and test in the same coherent pass。Do not retain compatibility decoders or migrate old rows； + reads of retired IDs fail explicitly，and client-web must not silently fall back to text。 +- **Scope restraint**: the minimum does not require OCR，speech transcription or PDF/EPUB/ZIP child-graph expansion。 + Exact parser dependencies，bounded inspection，charset authority，peer-local solved types and Memos attachment + resolver-version consequences remain Technical preflight。 +- **Confidence**: Sir accepted the nine exact IDs and hard cut-off，kept abstract capability methods，moved calls back + onto resolver instances，preferred `semantic content block`，and selected conventional `version` terminology。 + +### D-076 — Memos attachment v1 receives a one-time atomic migration to v2 + +- **Decision**: `extensions.memos.attachment.v1` rows are migrated atomically to the D-059/D-067 two-block graph， + rather than discarded or supported through a permanent v1 decoder。The existing metadata block ID remains the + Memos protocol identity and is rewritten as inline `extensions.memos.attachment.v2` canonical metadata without + `blob_id`。A new `core..v1` semantic content block receives minimal opaque PostgreSQL pointer JSON containing + the existing blob UUID，and one metadata → semantic `content` relation connects them。Blob bytes are not copied； + existing memo → attachment `attachment:` relations remain unchanged。 +- **Failure boundary**: each row conversion is transactional；a failed conversion leaves the v1 block、blob and owner + relations unchanged。After a successful migration the runtime registers only v2，so this is data preservation at a + breaking boundary，not indefinite version compatibility。 +- **Production evidence**: a 2026-08-02 read-only query against canonical Neon `production` found Alembic head + `d9f4e2a1b7c3`，no `storage_blobs` table and no `extensions.memos.attachment.v1` rows。The migration is therefore an + empty forward step for the current public demo，while remaining necessary for another database that already ran the + Memos/PostgreSQL-binary implementation。The same production snapshot contains retired bare resolver rows + (`html=1`、`image=28`、`text=8`、`video=3`)；D-075 intentionally leaves those rows unsupported without migration。 +- **Confidence**: Sir explicitly accepted the one-time atomic migration and clarified that canonical production is a + public demo，so ordinary diagnostic access should be practical rather than governed as a high-criticality service。 + +### D-077 — Durable projection follows verified implementation；publication remains owner-specific + +- **Decision**: during discussion，new architecture understanding stays in the task packet so unstable ideas do not + churn durable docs。After product/technical design is stable and implementation supplies evidence，the unit loop + projects accepted truth into the correct Hub、Unit TDD、deployment or peer-local owner；this does not require a + separate “documentation unit”。 +- **Operation boundary**: editing durable owner worktrees is distinct from commit、push、Hub publication and Spoke + shared-ref bump。The latter operations still require explicit authorization and separate owner commits；Hub source + must be published before any Spoke ref moves。 +- **Correction**: earlier packet language incorrectly treated durable projection itself as a post-unit publication + gate。Sir clarified that only discussion-time mutation was deferred，not implementation-time reconciliation。 +- **Confidence**: direct clarification from Sir after RSS implementation completion；consistent with the repository's + one-authority rule and Hub/Spoke workflow。 + +### D-078 — RSS runtime integration is sufficient close authority；generic test harness waits for a second pressure + +- **Acceptance decision**: the real-transport HTTP double → source/job → migrated PostgreSQL graph → storage/ + resolver → source-state journey，combined with real-format semantic bytes、migration/PostgREST probes and full + core-py/client-web regression，is sufficient acceptance authority for the RSS MVP。 +- **Observation boundary**: this is business-runtime vertical integration rather than a full deployment/process-level + public-API black box。Opt-in live RSS/Atom smoke was not selected in the final run and proves only fetch/parse when + enabled。Additional transient HTTP、whole-feed malformed、enrichment/storage/resolver failure、process-interruption + and scheduler exact-one-job probes remain non-blocking future hardening，not retroactive close gates。 +- **Infrastructure decision**: retain the already shared hermetic environment and on-demand real-format asset + generator。Keep RSS protocol routes、identity revisions、job/state assertions and graph cleanup local until a second + external-source unit proves the same test shape；then extract the smallest common harness without flattening + source-specific semantics。 +- **State consequence**: `rss-extension-hardening` is Complete after Sir's 2026-08-03 review；no acceptance follow-up + remains active。 +- **Confidence**: Sir explicitly accepted the final acceptance scheme and authorized task-packet completion/cleanup； + the infrastructure boundary also follows the program's two-real-pressure abstraction rule。 + +### D-079 — Semantic retrieval returns ranked information matches；generation is downstream use + +- **Product boundary**: semantic retrieval accepts a semantic query and returns ranked information matches。It does + not own answer generation、prompt assembly、chat state or an agent loop。 +- **Consumer boundary**: the same retrieval capability may be called directly by a search experience or indirectly by + an agent/RAG workflow。Those consumers do not change the retrieval result or quality contract into a generated-answer + contract。 +- **Acceptance consequence**: relevance、ranking、coverage、freshness and observable retrieval failures can be tested + independently of a generative model's answer quality。A generated answer is not evidence that retrieval found the + right information。 +- **Current-code consequence**: existing `/sink/rag` composition is not the semantic-retrieval product authority； + client-webext's own deprecation of core-side generation is supporting evidence，not the sole basis for the boundary。 +- **Confidence**: Sir explicitly accepted this boundary as the first semantic-retrieval Product decision。 + +### D-080 — Semantic retrieval quality drives organization breakdown；no block-segment layer + +- **Quality premise**: useful vector retrieval needs appropriate semantic granularity；whole heterogeneous/long + information blocks cannot be assumed to form good retrieval units。 +- **Information-model boundary**: this unit does not introduce `block segment` or another authoritative information + layer。When reusable retrieval granularity is required，organization breakdown expresses it as ordinary blocks/relations + in the info-base。 +- **Unit consequence**: semantic-retrieval is still the active vertical deliverable，but its design/implementation may + include the minimum organization behavior required to produce a high-quality searchable graph。This is an intended + cross-trunk pressure，not a reason to split a detached framework unit。 +- **Open ownership**: this decision does not yet assign vector index artifacts or embedding generation to organization + versus application/retrieval；that explicit conflict with D-006 remains the next review question。 +- **Confidence**: Sir directly required chunk quality to be handled through organization breakdown and rejected an + additional segment concept。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D081-D090.md b/tasks/knowledge-lifecycle-capabilities/decisions/D081-D090.md new file mode 100644 index 0000000..55125e9 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D081-D090.md @@ -0,0 +1,165 @@ +# Decisions D-081–D-090 + +> [Decision register index](index.md) + +### D-081 — Organization owns graph preparation；embedding and retrieval remain use-side projections + +- **Supersession**: this closes D-080's ownership question and supersedes D-006 only where that early frame assigned + embedding/indexing to organization。Organization owns graph preparation；embedding records、vector comparison and + physical ANN acceleration remain use-side derived support。 +- **Organization boundary**: organization changes the info-base itself by creating/updating/deleting ordinary + blocks/relations，including D-080 breakdown at useful semantic granularity。 +- **Use boundary**: embedding generation、durable embedding records、query-vector generation and vector comparison are + use/interface-side derived support；they do not become graph authority merely because organization may trigger their + refresh after a graph change。 +- **Coherence reason**: candidate and query vectors must be generated in the same embedding profile/vector space；the + use-side embedding/retrieval owner keeps that compatibility contract together while consuming organization-produced + graph entities。 +- **Vocabulary correction**: `embedding` is one vector representation；an entity-to-vector embedding record is a + derived projection；a physical ANN/database index is an optional acceleration structure。Do not use bare `index` to + collapse these three meanings。 +- **Confidence**: Sir accepted that organization is bounded by blocks/relations and retrieval belongs to info-base + use/interface；the vocabulary distinction answers the follow-up ambiguity without changing that boundary。 + +### D-082 — Every graph entity is a candidate；embedding profile/record are stable technical terms + +- **Candidate universe**: every persisted block/relation is considered by default；there is no product allowlist that + removes graph entities from semantic retrieval in advance。 +- **Per-profile availability**: an entity enters vector comparison only when it can provide meaningful input for the + selected embedding profile。Unavailable text for image/audio、empty information or purely structural relation labels + are profile-specific capability outcomes，not deletion from the info-base or exclusion from future multimodal/other + retrieval modes。 +- **Stable technical terms**: `embedding` is one derived vector；`embedding profile` names the compatible vector-space + contract；`embedding record` maps one block/relation plus its input snapshot/profile to a derived vector。These names + are promoted for Technical design and later durable projection。 +- **Physical-index distinction**: a PostgreSQL/pgvector HNSW、IVFFlat or other ANN index is an optional physical + acceleration structure，not an alias for either an embedding or the set of embedding records。 +- **Open shape**: profile identity/fields/lifecycle and embedding-record entity reference、snapshot、vector layout and + persistence remain mandatory review items；this decision intentionally does not freeze them。 +- **Confidence**: Sir explicitly accepted the candidate/availability rule and required embedding profile/record to + become stable technical vocabulary。 + +### D-083 — Persist immutable profiles；retain entity-specific embedding-record tables + +- **Profile persistence**: `EmbeddingProfile` has its own shared protocol relation。Vector-space-affecting changes + create a new immutable profile identity；active/default selection and credentials remain outside that identity。 +- **Record persistence**: continue from the existing `block_embeddings` and `relation_embeddings` relations rather + than introducing one polymorphic graph-entity table。Each becomes a true record table keyed by profile plus its real + block/relation FK，so entity deletion keeps referential/cascade behavior and multiple profiles can coexist。 +- **Cross-entity retrieval**: separate record relations do not change the Product result universe；retrieval can combine + ranked block/relation candidates without weakening their database identities。 +- **Deletion**: profile deletion is restricted while either record relation still references it；profile retirement or + active selection is not implemented by mutating its vector-space contract。 +- **Still open**: exact profile fields/ID、record input snapshot/status shape、migration and physical ANN strategy remain + review items。 +- **Confidence**: Sir explicitly accepted immutable persisted profiles and preferred evolving the two existing + entity-specific embedding relations。 + +### D-084 — Keep MVP profile symmetric；retrieval options are query-scoped；ANN is deferred + +- **Profile scope**: MVP does not persist or expose `candidate_input_options` / `query_input_options`。The selected + embedding contract is symmetric；a future model with proven asymmetric task/instruction/prefix requirements must add + an explicit versioned profile-contract field rather than hiding transformations in code。 +- **Metric boundary**: distance/similarity metric does not belong to immutable EmbeddingProfile identity。It is selected + when vectors are compared。 +- **Naming**: replace the proposed `RetrievalPolicy` domain object with query-scoped `VectorRetrievalOptions`。Options may + receive runtime defaults，but MVP does not add a persisted policy table or named-policy lifecycle。 +- **Physical strategy**: MVP uses exact pgvector comparison and does not create HNSW/IVFFlat indexes。ANN enters only + after representative scale/latency evidence justifies its DDL、recall and maintenance cost。 +- **Confidence**: Sir explicitly requested omitting unused asymmetric input options，confirmed query-time retrieval + parameters，and agreed HNSW is currently unnecessary。 + +### D-085 — Share AI provider/model identity；execute through peer-local dialects and typed services + +- **Shared registry**: persist deployment-scoped `AIProvider` instances and their `AIModel` children as one shared AI + Registry protocol。A provider references an exact dialect and owns that dialect adapter's config；a model owns its + provider-native identity、capabilities and modality declarations。 +- **Peer execution**: each peer owns an installed versioned dialect-adapter map and a + typed `AIService` that resolves model → provider → dialect before invoking embedding、language or future operations。 + “Global” means shared identity/config protocol，not a core-py network proxy or singleton execution authority。 +- **EmbeddingProfile consequence**: profile references `ai_model_id` instead of duplicating provider/dialect/model + strings。It does not repeat model-owned modalities；it owns selected invocation parameters that affect vector + compatibility，including dimensions。 +- **Naming rule**: omit redundant contextual prefixes，but prefer a precise domain noun over generic `type`。 + `capabilities` names supported operations；`modalities` names accepted information forms，and the two axes must not be + conflated。 +- **Unit boundary**: the semantic-retrieval vertical includes this minimum shared AI substrate because both embedding + and existing language calls already depend on it；its architecture supports multiple providers/capabilities without + requiring every future AI capability in this unit。 +- **Confidence**: Sir explicitly accepted the topology/unit boundary and identified modalities as AIModel properties + rather than EmbeddingProfile parameters。 + +### D-086 — Keep capability/modalities as typed AIModel JSON + +- **Persistence**: persist capability declarations in `ai_models.capabilities` as typed JSON rather than adding an + `ai_model_capabilities` relation。The declaration is model-owned，low-reuse configuration without an independent + identity or lifecycle，so normalization would add navigation and CRUD cost without enough relationship value。 +- **Shape**: each capability item uses locally sufficient `type` plus `input_modalities` and `output_modalities`；this + preserves capability-specific modality meaning without ambiguous flat `capabilities[] × modalities[]` arrays。 +- **Profile consequence**: EmbeddingProfile derives modality support from the referenced AIModel and does not duplicate + it。Dimensions remains a profile selection because it is an optional embedding invocation parameter that changes the + vector contract。 +- **Still open**: exact JSON discriminated-union schema、capability vocabulary and the remaining AIProvider/AIModel + columns are Technical design work；this decision does not approve mutable-profile freshness semantics。 +- **Confidence**: Sir preferred JSON over a child table，accepted the capability item fields and distinguished model + properties from profile parameters。 + +### D-087 — Choose typed relation versus owned JSON by identity、lifecycle and relationship pressure + +- **Owned JSON default**: keep a value inside its owner's typed JSON when it has no independent durable identity or + lifecycle，is rarely reused or queried independently，and is created/changed/deleted with the owner。AIModel + capability declarations are the current positive example。 +- **Dedicated-relation pressure**: prefer a table when values are independently referenced、have their own lifecycle or + ownership、participate as FK/integrity endpoints、need independent query/index/constraint behavior or are reused by + many durable rows。These are evidence dimensions rather than a mechanical score；relationship integrity can outweigh + low row count。 +- **Generic-config boundary**: a generic config table is suitable for leaf selections/defaults and owner-scoped payloads， + not automatically for every low-cardinality domain value。If other durable rows need a typed reference to a value， + hiding it in config JSON spends integrity and clarity to save a table。 +- **Promotion pressure**: this is a cross-unit persistence-design heuristic and should be projected to the appropriate + Product TDD/Unit TDD owner after implementation verifies it；discussion-time authority remains this packet。 +- **Confidence**: Sir explicitly identified the AI capability decision as a reusable table-creation criterion；the + relationship extension follows the same first-principles distinction and remains testable against Profile placement。 + +### D-088 — Embedding profile selection belongs to each use owner + +- **Selection**: semantic retrieval、organization linking and future embedding consumers explicitly own or accept the + EmbeddingProfile reference they use。A deployment/global config surface may persist those defaults。 +- **No generic router**: MVP does not add a global condition/routing-rule engine。One retrieval execution uses one explicit + profile；switching vector spaces is an explicit selection and multi-profile rank fusion is a separate capability。 +- **Provider distinction**: AIService routing from a concrete AIModel to its provider/dialect does not authorize + transparent failover to a different embedding model/vector space。 +- **Confidence**: Sir explicitly accepted consumer-owned routing；profile-definition persistence was subsequently settled + by D-089。 + +### D-089 — EmbeddingProfile remains a dedicated mutable relation + +- **Persistence**: retain an `embedding_profiles` typed relation。A profile is the stable relationship endpoint shared by + block/relation embedding records and query execution；it therefore meets D-087's identity、FK/integrity and reuse + pressures even when profile count is small。 +- **Config boundary**: a deployment/global config surface may own each use's selected/default profile reference，but it + does not embed Profile definitions as generic JSON。 +- **Mutation**: profile vector-contract fields may be updated in place。Older records are selected for rebuild when their + database-maintained `updated_at` precedes the profile watermark；the known late-write race is accepted rather than + adding a profile/record revision column。 +- **Supersession**: this supersedes D-083's strict profile-row immutability while preserving separate profile persistence、 + multiple profiles、entity-specific record tables and restrict-on-referenced-profile deletion。 +- **Confidence**: Sir explicitly accepted retaining the Profile table，rejected the extra version field on marginal- + benefit grounds and proposed timestamp-driven in-place rebuilds。 + +### D-090 — PostgreSQL owns selected shared-row `updated_at` semantics + +- **Authority**: for shared protocol values whose timestamp means “this database row last changed”，PostgreSQL owns + `updated_at` through a reusable `BEFORE UPDATE` trigger。SQLAlchemy `onupdate` cannot be authority because equal peers + also write through PostgREST/direct database protocol。 +- **Change boundary**: touch the timestamp only for an actual row change，avoiding no-op update churn。The trigger remains + shallow and does not modify/delete embedding records；maintenance compares timestamps separately。 +- **Graph consequence**: block content pointer、storage selection and resolver identity changes all invalidate its current + embedding input，and relation endpoint/content changes do likewise。A generic selected-table row trigger replaces the + current block-content-only behavior。 +- **Exclusions**: source-authored time、job lifecycle events and bytes changing behind an unchanged storage pointer are + not database-row `updated_at` semantics and receive no inferred timestamp update from this rule。 +- **Risk posture**: Profile/record timestamp comparison remains best-effort under concurrent late writes as accepted by + D-089；the trigger supplies cross-peer consistency，not serializable embedding maintenance。 +- **Confidence**: Sir identified the existing content-only block trigger as an implementation error and preferred a + PostgreSQL `BEFORE UPDATE` mechanism；the selected-table/exclusion boundary preserves earlier storage and peer rules。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D091-D100.md b/tasks/knowledge-lifecycle-capabilities/decisions/D091-D100.md new file mode 100644 index 0000000..41f199f --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D091-D100.md @@ -0,0 +1,169 @@ +# Decisions D-091–D-100 + +> [Decision register index](index.md) + +### D-091 — AIDialect is a registered type；AIProvider is its configurable instance + +- **Type/instance pattern**: model AIDialect after source type and storage type。The shared dialect catalog owns an exact + versioned ID、description and config schema；AIProvider owns one deployment-scoped configuration instance referencing + that dialect。 +- **Adapter binding**: every peer-local dialect component binds the exact shared dialect ID to its adapter + implementation。Catalog presence does not imply local implementation availability；AIService reports unsupported when + its peer lacks the adapter。 +- **Provider shape**: AIProvider owns durable identity、administrative name、dialect FK、typed config、`enabled` and + PostgreSQL-owned `created_at` / `updated_at`。Raw credentials may remain ordinary validated provider config under the + accepted security boundary。 +- **Lifecycle**: disabling a provider stops new calls while retaining config and all model/profile/embedding state。 + Provider deletion is restricted while AIModel children exist；display name is not routing identity。 +- **Confidence**: Sir accepted `enabled` and explicitly confirmed the dialect adapter/type correspondence with existing + source/storage registry patterns。 + +### D-092 — Use Manager naming and database-generated bigint AI identities + +- **Manager naming**: call the peer-local dialect registration/catalog/adapter-resolution component + `AIDialectManager`，matching SourceManager、StorageManager and ResolverManager roles。client-web may internally use AI + SDK's ProviderRegistry，but that library mechanism does not own InKCre domain vocabulary。This supersedes the + `AIDialectRegistry` name in D-085/D-091 without changing their topology。 +- **Identity strategy**: AIProvider and AIModel use PostgreSQL-generated bigint identities rather than UUIDs。All peers + create records through one shared PostgreSQL authority；a sequence/identity column already coordinates concurrent + creation，while offline preallocation、multi-primary merge and cross-database global identity are not requirements。 +- **AIModel shape**: provider FK + `native_model_id` is unique and immutable。`name` is nullable descriptive metadata；UI + falls back to native model ID。Capabilities JSON、enabled and database-owned timestamps retain their accepted owners。 +- **Model changes**: switching provider-native model identity creates a new AIModel，then updates the consuming Profile so + timestamp invalidation is explicit。Model deletion is restricted while Profiles refer to it。 +- **Confidence**: Sir questioned the inconsistent Registry name、made model name optional、identified the absence of a + distributed-ID requirement and accepted the remaining AIModel contract。 + +### D-093 — AIManager owns the whole local AI routing module + +- **Boundary correction**: use one domain-level `AIManager`，not AIDialectManager plus AIService。AIManager owns its + internal dialect-adapter map、dialect catalog sync、Provider/Model management、capability checks and typed execution + routing。This supersedes D-092's manager name and D-085's separate AIService surface without changing shared database + topology。 +- **Library boundary**: client-web may use AI SDK ProviderRegistry internally，but an implementation collection does not + become an InKCre module merely because a library names it Registry。 +- **Extraction heuristic**: registry/cache/map is commonly an internal mechanism。Promote it to a separate module only + when independent consumers、lifecycle、policy or reuse pressure justify the extra interface；otherwise keep the total + manager deep。This common pattern should be projected with other cross-unit design heuristics after implementation。 +- **Confidence**: Sir identified the issue as module decomposition rather than vocabulary and related it to the existing + total SourceManager/StorageManager pattern；local code evidence confirms those managers contain their registries。 + +### D-094 — Name typed references by role；require deterministic Profile dimensions + +- **Profile-shape closure**: this closes D-083's open Profile ID/field shape for MVP；D-089 separately supersedes that + decision's immutability，while D-101 later closes the EmbeddingRecord shape。 +- **Reference naming**: in typed persistence/domain models，name a reference by its semantic role—`AIModel.provider`、 + `EmbeddingProfile.ai_model`、`Block.storage`、`Block.resolver`—without mechanically appending `_id` or `_type` when the + schema/type already establishes representation。 +- **DTO qualification**: `_id` remains justified when a boundary must distinguish a scalar reference from an expanded + object，when multiple representations coexist or when an external protocol fixes the name。Directional role words such + as `from`/`to` are meaning，not redundant type prefixes；existing DTOs require later evidence-led audit rather than a + blind rename。 +- **Identity heuristic**: do not default internal rows to UUID。With one database creation authority，use generated integer + identity；require offline allocation、multi-primary merge、cross-database identity or another concrete pressure before + spending UUID complexity。AIProvider、AIModel and EmbeddingProfile use generated bigint。 +- **Profile shape**: EmbeddingProfile owns bigint `id`、nullable descriptive `name`、`ai_model` FK、required positive + `dimensions` and database timestamps。MVP has no enabled、normalization or generic config field。 +- **Boundary validation**: every embedding response must match Profile dimensions before persistence。The check is cheap + and prevents a provider/adapter defect from entering the shared record protocol。 +- **Confidence**: Sir explicitly promoted the reference-naming and UUID choices as common patterns，rejected nullable + dimensions as low-value nondeterminism and accepted easy result-shape validation。 + +### D-095 — AIManager is graph-blind；embedding projection stays use-side + +- **AI boundary**: AIManager accepts AI registry references、typed capability inputs/options and returns typed AI outputs。 + It does not know Block、Relation、Resolver、EmbeddingProfile graph semantics or traversal。 +- **Projection owner**: the embedding/retrieval component resolves graph entities through generic resolver capabilities， + constructs the model-compatible input and owns EmbeddingRecord lifecycle before/after calling AIManager。 +- **Resolver boundary**: remove `get_str_for_embedding()` and do not replace it with another embedding-named resolver + method。Resolvers make graph content generally applicable；they do not receive model/profile configuration。 +- **Multimodal restraint**: AIModel modality declarations do not require one universal graph-to-AI object。Add generic + resolver modality capabilities only from concrete model/content pressure；unsupported entity/profile pairs remain + explicit unavailable outcomes。 +- **Confidence**: Sir said this separation is mandatory；current resolver evidence confirms most embedding methods merely + duplicate text while the few differences are projection-semantics questions，not reasons to couple AI execution back + into resolvers。 + +### D-096 — Resolver `get_text()` is the sole general Block text projection + +- **Contract**: remove `get_str_for_embedding()`。`get_text()` owns one complete、generally useful textual projection of + a Block；it is neither a UI label nor input tailored to one embedding model。Text-modality embedding consumes it without + source-specific concatenation outside the resolver。 +- **Supersession**: this supersedes D-073's embedding-string capability wording and D-075's requirement that + `get_str_for_embedding()` remain an abstract Resolver method。D-073's application-facing/lazy-materialization role and + D-075's exact Resolver IDs、abstract `get_text()` and unsupported-capability behavior remain in force。 +- **Exact behavior**: FeedItem combines title、summary and full text/authored content；Memo uses body；HTML uses rendered + text；attachment metadata uses filename without embedding-specific MIME decoration。Resolvers without text capability + raise `UnsupportedResolverCapability`；supported-but-empty Blocks return `None`。 +- **Granularity**: organization breakdown，not resolver text projection，owns persistent semantic chunking。 +- **Evolution**: add no scenario parameter in MVP。If real retrieval evidence later proves one general projection + insufficient，a use-side projection context may evolve the resolver call without introducing Resolver → AI dependency； + its shape must come from the concrete failure。 +- **Confidence**: Sir accepted the exact contract and preferred later evidence-led optimization over speculative context + parameters。 + +### D-097 — Relation is a dynamic property whose generic projection belongs to RelationManager + +- **Information model**: interpret Relation direction as semantics：`from` is subject、`content` is a dynamic property and + `to` is its object/value—“to is from's relation-content”。Relation is not useful as isolated content；it is a directed + property assertion over two Blocks。 +- **Type boundary**: dynamic properties avoid a core ontology that attempts to enumerate every world-object class。They do + not abolish type as an implementation/contract concept；source/storage types、resolver IDs and AI capability types remain + valid adapter/decoder discriminators inside InKCre。 +- **Owner correction**: RelationManager owns the one generic Relation projection because Relation has no resolver-family + variants。The use-side retrieval owner consumes that projection；AIManager remains graph-blind。D-099 subsequently names + that owner SemanticRetrievalManager。This refines D-095's generic “embedding owner assembles” wording。 +- **Endpoint shape**: full Resolver `get_text()` is too large and can bury the property。Relation projection requires a + concise title/name-like endpoint representation；exact capability naming and fallback bounds remain T-010 review。 +- **Freshness consequence**: a Relation embedding input depends on the Relation plus both endpoint projections。Record + freshness must represent those dependencies without trigger cascades from Block updates into Relation rows。 +- **Confidence**: Sir introduced the dynamic-property model，confirmed the continuing role of implementation types，placed + projection ownership in RelationManager and rejected full endpoint text。 + +### D-098 — Semantic retrieval embeds resolver-qualified directed Relation assertions + +- **Selected projection**: retain Relation embeddings。RelationManager serializes subject/from label、exact + relation.content property and value/to label while preserving direction；raw relation content alone is not an admitted + instance projection。 +- **Endpoint label**: every Resolver provides a concise `get_label()` containing a readable resolver self-name plus + title/name/identifier，例如 `feed ` or `github user <username>`。Do not leak exact resolver implementation IDs into + semantic text。Labels are general graph-reference capability，not model prompts or full content summaries；D-102 owns the + final no-metadata boundary。 +- **Retrieval depth**: Block embeddings represent object information through `get_text()`；Relation embeddings represent + directed dynamic properties between identifiable object kinds。Semantic retrieval may return either existing Blocks or + Relations，and a Relation match remains graph-navigable to both endpoints。 +- **Freshness**: Relation record input depends on relation direction/content plus both endpoint labels。The record snapshot + must expose staleness when any dependency changes；do not trigger-cascade Block updates into Relation rows。 +- **Direction audit**: subject/property/value semantics become a producer invariant。Existing/in-scope graph builders must + be audited；the current mail `EmailAddress --from--> Email` is evidence of a likely reversed edge。 +- **Alternatives rejected for MVP**: content-only per-edge embeddings duplicate identical property vectors and lack + endpoint relevance；Block-only semantic retrieval loses direct relational matching。Distinct property discovery remains + a possible later tool，not this record shape。 +- **Confidence**: Sir explicitly preferred endpoint labels + relation content so semantic retrieval provides a deeper + object-and-property surface，then corrected the proposed qualifier concept/ownership through D-100/D-102。 + +### D-099 — SemanticRetrievalManager owns the use capability + +- **Public manager**: `SemanticRetrievalManager` owns Profile selection、Block/Relation projection consumption、embedding- + record maintenance、query embedding、vector comparison and ranked result contract。 +- **AI boundary**: AIManager owns raw provider/model embedding execution only and remains graph-blind。 +- **Internal boundary**: do not expose a generic EmbeddingManager in MVP。Keep embedding-record mechanics internal；if + organization linking or another real consumer proves the same graph/profile/record lifecycle，extract + `InfoBaseEmbeddingManager` then。 +- **Confidence**: Sir preferred the use-capability name over a mechanism-level manager；the boundary follows D-093's + evidence-led module extraction rule。 + +### D-100 — Resolver friendly-name registration proposal（superseded by D-102） + +- **Terminology correction**: do not add a Resolver-output `semantic kind`。Each exact Resolver registration owns a + stable canonical `friendly_name`，and `get_label()` uses it with an optional instance title/name/identifier to return a + concise reference such as `feed <title>` or `github user <username>`。This refines D-098's endpoint-label wording。 +- **Type boundary**: `friendly_name` is readable resolver contract metadata，not a classification field on Block and not + an ontology of world-object types。The exact resolver ID continues to own dispatch/version compatibility but does not + enter semantic text。 +- **Stability**: the value is canonical and non-localized across conforming peer implementations。A change affecting + Relation semantic input advances the exact resolver contract rather than mutating an invisible embedding dependency。 +- **Persistence proposal**: semantic retrieval currently proves only peer-local ResolverManager registration metadata； + it does not yet prove a shared `resolver_types` table。Persistence remains T-012's explicit review point。 +- **Confidence**: Sir rejected the newly invented `semantic kind` and proposed each Resolver's own `friendly_name`， + potentially as resolver-type registration metadata。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D101-D110.md b/tasks/knowledge-lifecycle-capabilities/decisions/D101-D110.md new file mode 100644 index 0000000..39f7815 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D101-D110.md @@ -0,0 +1,169 @@ +# Decisions D-101–D-110 + +> [Decision register index](index.md) + +### D-101 — Embedding records use profile/entity identity and timestamp-derived freshness + +- **Record-shape closure**: this closes D-082/D-083's remaining EmbeddingRecord identity、snapshot、vector-layout and + freshness questions for MVP。D-084 already deferred physical ANN strategy；migration mechanics remain implementation + planning rather than an open domain shape。 +- **Shape**: `block_embeddings` has `(profile, block)` primary key；`relation_embeddings` has `(profile, relation)` + primary key。Each stores a variable-dimension vector plus database-owned `created_at` / `updated_at`。 +- **Freshness**: a Block record compares its timestamp against Profile and Block；a Relation record compares against + Profile，Relation and both endpoint Blocks。Do not trigger-cascade endpoint changes into Relation rows。 +- **No digest**: MVP does not persist `input_digest`。Deterministic exact resolver contracts plus shared row timestamps + supply the accepted best-effort invalidation model；the known concurrent late-write race remains accepted。 +- **Dimension safety**: candidate selection excludes records whose actual vector length differs from the selected Profile + dimensions before applying a pgvector distance operator。 +- **Confidence**: Sir accepted the proposed record shape/freshness/no-digest boundary while separately correcting the + endpoint qualifier terminology。 + +### D-102 — The concrete Resolver owns its complete endpoint label + +- **Code fact**: current core-py Resolver has no domain `name`；it has exact `__rsotype__` and Python's implementation + `__name__`。client-web has JavaScript constructor `.name`，but peer implementation spellings already differ and are not + stable semantic contracts。 +- **Surface**: `get_label()` directly returns the complete resolver-qualified label，including readable self-name plus an + optional instance identifier。Do not split a single current use into `semantic_kind`、`friendly_name` registration + metadata or a newly introduced Resolver `name` capability。This supersedes D-100's registration-metadata proposal。 +- **Persistence**: do not create `resolver_types` or any other shared catalog for this label。The concrete Resolver is + already installed on the executing peer，and no independent discovery、FK or metadata lifecycle has been proven。 +- **Correction boundary**: repository-owned malformed Relation producers are corrected at their collection/graph writer + and，when justified by exact endpoint evidence，one-time migration。Do not add organization repair jobs，read-time graph + normalization or SemanticRetrievalManager mutation。 +- **Confidence**: Sir asked to reuse Resolver name if it existed，then clarified that readable naming is used only by the + Resolver and must not become registration metadata or a persisted resolver-type table。Local code/peer inspection + disproved the existing domain-name premise while preserving that ownership intent。 + +### D-103 — Correct malformed Relations at producers；hard-cut the legacy Twitter note hook + +- **Producer audit**: Memos、RSS、core image、GitHub、Twitter attachment/URL and Mail recipient/cc directions already + satisfy subject/property/value。Mail sender is reversed and must become email → `from` → sender address。 +- **Mail history**: implementation includes a targeted one-time migration selected by relation content plus exact + endpoint resolver identities。It must not reinterpret arbitrary caller-authored `from` relations。 +- **Twitter boundary**: hard-cut the graph-writing body of `Source._organize()` to an explicit no-op and remove the direct + test that treats it as a feature。Do not move reply fetching into organization，semantic retrieval or another runtime + repair loop。Future bookmark-note collection owns a new explicit source/command design and relation grammar。 +- **Historical restraint**: existing `bookmarked for` rows are not renamed or deleted without an approved future grammar + and exact endpoint evidence。Persisted direction/content remains authority in the meantime。 +- **Confidence**: Sir authorized producer audit/correction，forbade organization/runtime repair and explicitly approved + hard-cutting the legacy Twitter behavior。 + +### D-104 — Persist only successful EmbeddingRecords；derive freshness and keep attempt outcomes local + +- **Derived state**: missing means no profile/entity record；fresh means D-101 timestamp dependencies and vector + dimensions match；stale means an existing record fails those predicates。Do not persist a duplicate `dirty` flag or + trigger-cascade endpoint changes。 +- **Record boundary**: embedding tables contain only successful vectors。They receive no nullable embedding，status， + error or availability columns，and MVP adds no separate shared entity-availability table。 +- **Attempt outcomes**: maintenance reports `embedded`、`unavailable` or `failed`。Unsupported/empty projection and a + peer-local unknown Resolver are unavailable；content/storage/provider/adapter/shape failures are failed。Neither + outcome deletes an old record，while retrieval excludes it whenever stale。 +- **Peer reason**: local Resolver installation and temporal provider/network failure are not global properties of a graph + entity。Persisting them as shared state could suppress a capable peer or outlive the failure。A future durable job may + own execution diagnostics without moving them onto embedding records。 +- **Confidence**: Sir explicitly accepted successful-record-only persistence and rejected status/error/dirty columns or + a separate global availability/error relation for MVP。 + +### D-105 — Embedding maintenance is resumable without a durable job relation + +- **Progress**: successful EmbeddingRecords are the idempotent progress markers for ordinary missing/stale maintenance。 + After interruption，a new invocation scans only work that remains；MVP adds no maintenance-job identity、claim、cursor、 + retry state or item ledger。 +- **Operation**: scheduled and explicit paths call the same bounded `SemanticRetrievalManager.maintain()`。Scanning + continues past unavailable entities until an executable batch is filled or the candidate set ends，preventing a stable + unsupported prefix from starving later IDs。 +- **Transactions**: projection/storage/provider work occurs outside database transactions。A complete response batch is + checked for cardinality/order/dimensions before a short atomic upsert transaction。 +- **Rebuild**: administrative full regeneration is a distinct `rebuild` operation，not `refresh`。It selects records + older than the operation-start cutoff so its own writes are not selected again in one pass。Restarting may duplicate + some successful calls；that low-harm cost is accepted over durable job complexity。 +- **Concurrency**: peers may duplicate an AI call and the last valid compatible upsert wins。MVP adds no lease，advisory + lock or claim relation solely to prevent occasional duplicate cost。 +- **Confidence**: Sir explicitly accepted repeatable batch maintenance/rebuild without a durable embedding-maintenance + job。 + +### D-106 — Automatic maintenance follows only SemanticRetrievalManager's default Profile + +- **Selection**: SemanticRetrievalManager owns one nullable deployment-default EmbeddingProfile reference。Retrieve、 + maintain and rebuild may explicitly select another Profile，but do not infer multi-profile execution or ranking fusion。 +- **Automatic scope**: scheduled maintenance processes only the current default；a defined but unselected Profile does not + authorize recurring provider cost。Without a default，the scheduler no-ops and defaulted retrieval reports configuration + unavailable。 +- **Switching**: changing the default starts future automatic maintenance in the new vector space and leaves old Profiles/ + records intact for explicit evaluation、rollback or retention。 +- **Peer operations**: worker participation、interval and maximum per-run work are peer-local runtime settings。Logical + input batches belong to SemanticRetrievalManager；AIManager/dialect adapters own provider translation/chunking and must + preserve one output per ordered input。Neither resource limit belongs to EmbeddingProfile。 +- **Confidence**: Sir explicitly accepted automatic maintenance of only the deployment-default Profile and explicit + maintenance for all others。 + +### D-107 — Use shared `configs` with exact schema contracts；reject a semantic-retrieval singleton table + +- **Correction**: reject `semantic_retrieval_configs`。A deployment has only one such value and its lifecycle is wholly + the owner's configuration lifecycle；the earlier claim of independent record lifecycle was unsupported and did not + justify a dedicated table。 +- **Shared shape**: introduce deployment-scoped `configs(key, schema, value, created_at, updated_at)`，distinct from the + per-peer config currently stored on legacy `clients` rows（future `peers.config`）。The concise name `configs` is + sufficient in project context。 +- **Schema identity**: `schema` is an exact，versioned，cross-peer contract ID such as + `core.semantic_retrieval.config.v1`，not a Python class/import path。core-py DeploymentConfigManager maps it to a Pydantic model； + other peers register equivalent local validators。Unknown/colliding schemas fail explicitly。 +- **Value typing**: `value` is owner-shaped JSONB validated as a complete model。Semantic retrieval stores nullable + `default_profile` there。This preserves protocol/application typing；JSON does not provide an ordinary PostgreSQL FK，so + database-level delete restriction is a distinct trade-off rather than proof that all typed-reference value is lost。 +- **Module boundary**: `ConfigContract` in the generic configuration module owns model-driven validation、normalization、shallow patch + preparation and schema projection without deployment or persistence semantics。The deployment-scope configs module uses + it and owns the `configs` relation、exact schema registry and DeploymentConfigManager CRUD；the config owner owns its + key's use semantics and behavioral consequences。 +- **Confidence**: Sir rejected the low-value singleton，proposed a simple deployment-scope key/value config relation plus + schema-model registry，selected the table name `configs` and accepted exact schema-contract IDs。 + +### D-108 — DeploymentConfigManager exposes honest replace and shallow-patch operations + +- **Replace**: `PUT /configs/{key}` supplies schema plus one complete value，validates it before mutation and creates or + replaces the row。Only complete replace may change schema，making schema-generation migration and its new value atomic。 +- **Patch**: `PATCH /configs/{key}` requires an existing row，keeping its schema，shallow-merges current object plus patch， + validates the complete result and persists normalized JSON。 +- **Failures**: unknown/colliding schema and persisted invalid value fail explicitly；DeploymentConfigManager never exposes an + unvalidated raw-dict fallback。 +- **Owner boundary**: DeploymentConfigManager owns exact schema resolution and shared `configs` row operations；it delegates + model-driven validation、normalization and shallow-patch preparation to the persistence-neutral generic configuration + module。It does not own owner-specific reference semantics、scheduler behavior or a generic live-apply callback registry。 +- **HTTP correction**: new APIs use PUT for complete replacement and PATCH for partial update。The historical extension + PUT-as-patch behavior is not copied as a new generic contract。 +- **Confidence**: Sir explicitly accepted complete replace plus shallow patch and schema changes only through replace。 + +### D-109 — Use Peer for technical runtime-node semantics；retain Client as user-facing product language + +- **Technical terminology**: when architecture、shared protocol or implementation describes an InKCre runtime node + participating in one info-base，use `Peer`。Domain symbols、database relations/fields、runtime settings、wire claims and + technical docs move from `client` to `peer` rather than preserving a misleading “client means peer” disclaimer。 +- **User-facing terminology**: marketing、landing pages and non-technical/user-facing documents continue to call the + installed applications `clients`。The client/server implication is useful at that product surface because it describes + the user's application role，not the equal technical authority of participating runtimes。 +- **Naming consequence**: first-party product/repository identities such as `client-web`、`client-ios` and + `client-webext` remain unchanged。Their internal technical domain symbols/protocol contracts may still migrate to Peer。 +- **Semantic，not lexical**: HTTP/test clients、OAuth/native protocol `client_id`、third-party SDK clients and external + apps that genuinely consume a backend retain their native meaning。 +- **Data/compatibility direction**: the future implementation should rename existing protocol state in place so runtime + identity/data survive，while avoiding indefinite old-name aliases。Historical migration files remain historical；new + migration and synchronized peer implementations own the breaking contract change。 +- **Confidence**: Sir selected `peer` for technical semantics and explicitly retained `client` on marketing、landing and + non-technical user-facing surfaces，closing the repository/product slug question。 + +### D-110 — Deployment config validates structure；the use owner resolves references only when used + +- **Write/read path**: `SemanticRetrievalConfig` is a Pydantic structural contract consumed by the deployment-scope + configs module。Config PUT/PATCH/get does not pass through SemanticRetrievalManager and does not query + EmbeddingProfile existence。 +- **Reference semantics**: `default_profile: bigint | null` is a typed reference representation，not a database FK or a + promise of current referential integrity。DeploymentConfigManager may persist and return a structurally valid dangling + value。 +- **Use-time defense**: SemanticRetrievalManager resolves the selected Profile only when retrieval、maintenance or rebuild + uses it。A missing referenced row fails explicitly as invalid/unusable configuration；it is distinct from a missing config + row or `default_profile = null`，which means not configured。 +- **Deletion**: Profile deletion receives no reverse restriction or schema-specific database trigger from JSON config。 + Existing ordinary FKs from embedding records continue to enforce their own integrity independently。 +- **Confidence**: Sir corrected the previously leaked owner-specific write validation and selected use-time defense plus + explicitly repairable dangling references as sufficient。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D111-D120.md b/tasks/knowledge-lifecycle-capabilities/decisions/D111-D120.md new file mode 100644 index 0000000..4cf7ea5 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D111-D120.md @@ -0,0 +1,160 @@ +# Decisions D-111–D-120 + +> [Decision register index](index.md) + +### D-111 — Name generic config mechanics ConfigContract and persisted configs DeploymentConfig + +- **Generic module**: `app.configuration` owns `ConfigContract[Model]`，a persistence-neutral wrapper over one local + Pydantic model's complete validation、normalized JSON、shallow patch preparation and JSON Schema projection。It has no + registry、deployment key、database session or live owner lifecycle。 +- **Deployment domain**: `DeploymentConfigModel` maps the shared `configs` relation；`DeploymentConfigManager` owns exact + schema-ID registration/resolution and row CRUD。The explicit prefix prevents it from being mistaken for the generic + configuration mechanism；the database relation and HTTP resource remain concise `configs` / `/configs/{key}`。 +- **Owner models**: `SemanticRetrievalConfig` and extension/source/storage-specific config models remain ordinary + owner-defined Pydantic contracts。They may be wrapped by ConfigContract without becoming DeploymentConfig rows。 +- **Local topology**: persisted schema/business modules use the explicit `deployment_config` domain name；peer runtime + settings remain `app.settings` and are not folded into either config module。 +- **Confidence**: Sir explicitly approved `ConfigContract` and `DeploymentConfigManager` after reviewing their exact + ownership and call sequence。 + +### D-112 — AIManager MVP owns embedding and text generation；hard-cut legacy AI authority + +- **Capabilities**: this unit implements typed `embedding` and text-generation/language capability execution through the + peer-local AIManager。The architecture remains extensible to other capability types，but MVP does not implement every + declared modality/capability。 +- **Single authority**: retained core-py language consumers migrate with embedding consumers to AIModel → AIProvider → + AIDialect routing。The process-global OpenAI client and environment-owned provider/model authority in `libs/ai.py` are + hard-cut，not preserved through compatibility wrappers。 +- **Scope restraint**: an unused historical VLM helper does not justify multimodal-generation implementation and is + removed with the legacy module。RAG answer generation remains outside semantic-retrieval acceptance even if a retained + downstream RAG path uses AIManager。 +- **Organization pressure**: future/minimum breakdown may consume the same typed text-generation capability without + creating another provider registry or making AIManager aware of graph/organization semantics。 +- **Confidence**: Sir explicitly selected the hard cut over an embedding-only migration after reviewing the split-authority + consequence。 + +### D-113 — Retrieval matches wrap complete existing graph entities + +- **Shape**: semantic retrieval returns a discriminated union of Block and Relation matches。Each match owns + `type`、the complete ordinary `entity` row and query-derived ranking metadata；it does not become a graph entity or + replace the entity's identity。 +- **Graph use**: a Relation match exposes its stored direction/content and immediate `from_` / `to_` navigation；a Block + match exposes its resolver/storage/content references。Consumers do not need a second fetch merely to identify the + matched graph object。 +- **Projection boundary**: the match does not duplicate Resolver `get_text()` or RelationManager's semantic projection。 + A consumer needing solved text resolves the returned entity through the ordinary graph capability；no transient chunk or + segment is introduced。 +- **Confidence**: Sir explicitly accepted this as the natural result shape over reference-only or projection-duplicating + alternatives。 + +### D-114 — Public retrieval score is cosine similarity，not distance or confidence + +- **Score**: MVP exposes one `score = 1 - cosine_distance` per match；higher means more similar。It does not duplicate the + exactly derivable distance field。 +- **Interpretation**: score lies in `[-1, 1]` for finite non-zero cosine inputs。It is ranking evidence，not probability、 + confidence or a cross-model quality measure；values are comparable only within one query/Profile/metric execution。 +- **Context**: SemanticRetrievalResult exposes the selected Profile reference and exact `metric = "cosine"` once at the + result root rather than repeating them per match。Score is execution-derived and is never persisted on graph entities or + embedding records。 +- **Validation**: query and candidate vectors must be finite and non-zero before cosine comparison；invalid provider output + fails explicitly rather than entering NaN/undefined ordering。 +- **Confidence**: Sir explicitly approved higher-is-better cosine similarity as the sole public ranking value。 + +### D-115 — MVP semantic candidate filtering is graph-entity type only + +- **Filter**: `VectorRetrievalOptions.entity_types` is a required non-empty set drawn from `block | relation`，defaulting + to both。It filters the candidate tables before global mixed ranking；it does not classify world objects。 +- **Deferred predicates**: MVP does not expose resolver、source、time or Relation-property filters。Resolver is a Block + decoder contract rather than an object class；source/time do not have one uniform graph authority field；property + predicates belong to graph/feature retrieval。 +- **Composition**: consumers needing those constraints compose semantic matches with feature retrieval or graph + navigation rather than turning the first semantic API into a universal predicate language。 +- **Confidence**: Sir explicitly accepted entity type as the sole MVP candidate-filter dimension。 + +### D-116 — Retrieval is bounded top-k with optional score threshold and no pagination + +- **Cutoffs**: `VectorRetrievalOptions.limit` defaults to 20 and is constrained to `1..100`；`min_score` is nullable in + `[-1, 1]` and defaults to null。Candidates below the optional threshold are removed before global mixed ranking and + limit。 +- **No universal threshold**: score distributions vary by Profile/model/corpus，so core does not invent a default + relevance threshold。No qualifying candidate is a successful empty result，not an error。 +- **No pagination**: semantic retrieval is a bounded top-k answer，not a stable ordered collection。More than the useful + top results is pressure to refine the query or improve breakdown/projection/Profile/embedding quality，not to page + through low-relevance matches。Concurrent graph/embedding maintenance also makes offset/cursor traversal unstable。 +- **Physical-index distinction**: improving HNSW/another ANN structure primarily changes performance/approximation，not + semantic relevance；“optimize the index” improves result quality only when it means graph granularity、projection、 + EmbeddingProfile or embedding-record preparation。 +- **Confidence**: Sir explicitly accepted the cutoff contract and strengthened the no-pagination rationale。 + +### D-117 — Peer authority is equal；execution capabilities are heterogeneous + +- **Product correction**: Peer means equal participation in shared info-base/protocol authority，not identical runtime + capabilities。A browser Peer cannot be assumed to run long-lived schedulers、source collectors or every AI/resolver + implementation merely because it can read/write the same database。 +- **Database boundary**: reject the proposed shared semantic-comparison RPC。PostgreSQL binary-storage functions are + bounded atomic byte-transport helpers；semantic retrieval owns Profile/freshness/ranking/result business behavior and + remains in an application capability implementation for maintainability。 +- **Interaction shapes**: source collection is asynchronous delegated work whose durable domain job is the coordination + medium；semantic retrieval is synchronous request-response and must not invent a retrieval-job relation。A caller/provider + C/S edge is acceptable，but it is local to one capability interaction rather than a global primary-server topology。 +- **New pressure**: avoiding ad-hoc C/S edges now proves the need for exact Peer capability advertisement、discovery、 + routing and typed invocation contracts。D-061's previously future-only explicit cross-peer capability contract is + activated by a second concrete execution shape；it must not collapse domain jobs and synchronous requests into one + universal job/command object。 +- **Confidence**: Sir rejected database-owned retrieval business logic，identified real Peer capability asymmetry and + framed manageable request-response delegation as the actual design problem。 + +### D-118 — Peer capability discovery and domain invocation are separate contracts + +- **Discovery boundary**: treat heterogeneous Peer execution as a deployment-local service-discovery problem。PeerManager + owns local capability registration、self-advertisement、provider discovery and authenticated connectivity；it does not + interpret a capability's method/path、request/response schema or business behavior。 +- **Capability identity**: every advertised Peer Capability uses an exact versioned contract ID。Discovery matches the + complete ID rather than a friendly label、unversioned feature name or fuzzy compatibility claim。 +- **Invocation boundary**: each owning domain retains its typed endpoint/remote adapter and interaction semantics。 + Discovery returns a provider Peer；the domain contract decides how that Peer is called。 +- **Guardrails**: never add a generic `/capabilities/{id}/invoke` endpoint or generic delegation-job relation。Asynchronous + domains retain their own job state/claim/outcome models；synchronous domains use ordinary request-response contracts。 +- **Open semantic**: this decision does not yet define whether advertisement means implementation support、current + reachability or dependency readiness，nor does it select persistence、freshness or multi-provider routing behavior。 +- **Confidence**: Sir explicitly approved the positioning/model，exact versioned capability IDs，discovery/invocation + separation and both guardrails as mandatory boundaries。 + +### D-119 — Minimal discovery owns support and liveness；readiness remains service-internal + +- **Semantic closure**: this closes D-118's advertisement-meaning question：advertisement claims exact implementation + support，the Peer-scoped lease supplies liveness，and readiness remains internal。D-120–D-136 subsequently close the + shared shape、renewal、routing and invocation mechanism。 +- **Discovery facts**: the minimal shared discovery surface answers only which exact versioned capabilities a Peer + advertises and whether that Peer is online。It does not grow into dependency health、request feasibility、load or a + complete service-registry product。 +- **Routing**: consumers do not manually discover then inspect online state。PeerManager owns an internal routing layer + that selects an online provider for an exact capability；the domain remote adapter consumes that route and retains its + typed invocation contract。 +- **Provider equivalence**: Peers advertising the same exact capability ID implement the same capability contract and are + eligible provider instances。This does not erase provider identity or authorize a different capability version。 +- **Readiness boundary**: readiness/configuration/dependency feasibility belongs inside the capability service and its + domain failure semantics；it is neither discovery metadata nor a generic invocation concern。 +- **Retry restraint**: online routing does not by itself authorize automatic post-dispatch retry。Whether an operation is + idempotent/replay-safe remains domain-owned，so generic routing may select before dispatch but cannot blindly replay an + arbitrary failed request。 +- **Open mechanism**: exact Peer liveness representation、renewal/expiry and route selection remain Technical review；no + `online` boolean、heartbeat or lease shape is approved by this decision。 +- **Confidence**: Sir explicitly selected support+liveness as the minimal discovery surface，required routing to hide + online-provider selection and kept readiness internal to the service。 + +### D-120 — Peer owns a capability snapshot and expiring liveness lease + +- **Capability persistence direction**: persist a Peer-owned `capabilities: text[]` full snapshot of exact IDs rather + than independent service/catalog rows or append-only registration events。Runtime capability-owner lifecycle changes + recompute/publish the snapshot，so restart and hard cut-off naturally remove stale IDs。 +- **Liveness direction**: represent online state with one Peer-scoped expiring lease such as `lease_expires_at`，not a + durable `online` boolean or per-capability health rows。Abrupt process loss becomes offline through expiry；readiness + never renews or withdraws the lease。 +- **Ordering constraint**: data shape is accepted as the minimal model，but exact columns/update protocol cannot drive the + design before PeerManager、runtime composition、Extension lifecycle and domain-adapter module topology is approved。 +- **Reopened projection**: D-122's inbound-owned wire contract and the subsequent delegate topology create real pressure + for each advertised capability entry to carry an inbound interface descriptor，not only an ID。Peer ownership/full- + snapshot/lease remain accepted；the `text[]` projection is no longer frozen。 +- **Confidence**: Sir confirmed the proposed minimal persistence shape，then correctly elevated module topology as the + immediate design priority。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D121-D130.md b/tasks/knowledge-lifecycle-capabilities/decisions/D121-D130.md new file mode 100644 index 0000000..dd4dc31 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D121-D130.md @@ -0,0 +1,196 @@ +# Decisions D-121–D-130 + +> [Decision register index](index.md) + +### D-121 — One Peer control module；capability dependencies point inward；connection abstraction reopened + +- **Accepted topology**: retain PeerManager as the only public common Manager for local registration、advertisement、lease + and exact-ID online-provider routing。Runtime composition and Extension lifecycle register/withdraw owner snapshots； + each callable domain retains provider route/local Manager and a concrete remote adapter。 +- **Dependency direction retained**: capability-side code depends on PeerManager，while PeerManager stores/matches opaque + exact IDs and imports no concrete capability module。D-122 supersedes this decision's earlier concrete-adapter and + method/path ownership hypothesis without changing that dependency direction。 +- **Transport layering**: FastAPI/ASGI is the server-side HTTP transport。A FastAPI semantic-retrieval route is an inbound + domain adapter above that transport，mapping authenticated/parsed HTTP values to SemanticRetrievalManager；it is not the + transport server itself。 +- **Reopened abstraction resolved by D-122**: `PeerConnection` risked conflating selected-provider data、connection + lifecycle、authenticated HTTP transport and arbitrary domain invocation；D-122 removes it and PeerTarget entirely。 +- **Confidence**: Sir accepted the remaining module topology，identified the PeerConnection risk and requested explicit + review of concrete capability coupling and inbound-route/transport terminology。 + +### D-122 — Model explicit Peer inbound/outbound counterparts；transport details are inbound-owned + +- **Naming**: replace the vague role-name `DomainPeerAdapter` with concrete domain `PeerOutbound` names such as + `SemanticRetrievalPeerOutbound`。The provider counterpart is its `PeerInbound`。 +- **Wire authority correction**: SemanticRetrievalPeerOutbound owns the typed domain-facing `retrieve()` operation but + does not independently declare HTTP method/path。Those are transport-facing contract details owned by the matching + inbound，with a registry maintaining exact capability → inbound binding and enabling selection of the corresponding + outbound。 +- **Inbound aggregation**: from the Peer capability boundary，HTTP/ASGI/FastAPI route/controller may be treated together + as the inbound。The controller remains a thin adapter and does not acquire business logic；SemanticRetrievalManager + remains the provider implementation。 +- **Transport module**: `PeerHTTPTransport` has enough cohesive responsibility（peer endpoint、peer JWT、HTTP execution、 + transport failures）to be a real class/module。Rejecting a needless AI registry does not impose a general ban on + explicit registries or transport classes。 +- **Reopened terminology**: subsequent delegate review distinguishes the shared inbound/outbound wire protocol from the + caller-side outbound implementation。The then-proposed HTTP ID includes HTTP plus peer-JWT Authorization behavior and + is not merely a transport label；D-123 later fixes its exact name as `core.peer.protocol.http.v1` and the local adapter + as PeerHTTPOutbound。 +- **Active Record correction**: Active Record is not itself a failure pattern and may remain appropriate for Peer/Source + data behavior。The actual guardrail is one authority per concern；arbitrary capability invocation must not migrate onto + Peer merely because Peer is an Active Record。 +- **Removed abstractions**: cancel both PeerConnection and PeerTarget。Neither has distinct proven state or behavior after + inbound/outbound/transport responsibilities are separated。 +- **Open registry question**: whether capability → inbound/outbound pairing is runtime-local registration metadata or a + shared/persisted protocol surface remains unresolved and must precede the final module dependency graph。 +- **Confidence**: Sir approved DomainPeerAdapter's role after concrete naming，corrected wire-contract ownership，accepted + PeerHTTPTransport as a class，clarified the Active Record stance and explicitly approved removing PeerConnection/Target。 + +### Active pressure after D-122 — Delegate by advertised inbound interface + +This is a working proposal，not yet a confirmed decision: + +- a caller that knows the SemanticRetrieval contract but lacks a local implementation asks + `PeerManager.delegate(exact capability ID, ...)`；it does not select a concrete semantic-retrieval outbound adapter； +- each provider's active SemanticRetrieval inbound contributes an advertised interface descriptor containing an exact + Peer Protocol ID and protocol-owned properties such as HTTP method/path； +- PeerManager owns candidate routing/failover and resolves the advertised protocol through a local outbound registry； + `core.peer.protocol.http.v1` therefore selects PeerHTTPOutbound rather than a SemanticRetrieval-specific HTTP adapter； +- SemanticRetrieval retains request/result typing and validation，while PeerManager/outbound remain domain-blind； +- this is an internal delegation mechanism，not a generic `/capabilities/{id}/invoke` network endpoint and not a generic + delegation job。 +- **Terminology direction**: do not retain `transport` as a separate domain concept unless a later implementation proves + an independent contract。Peer Protocol is shared wire behavior；PeerOutbound is its caller-local executable adapter。 + +### D-123 — Advertise exact Peer Protocol separately from its parameters + +- **Protocol identity**: use the self-describing exact ID `core.peer.protocol.http.v1` for the shared HTTP Peer Protocol。 + The protocol includes HTTP plus peer-JWT Authorization、wire encoding and protocol-level failure behavior；`transport` + is not retained as a separate domain concept。 +- **Advertisement shape**: each capability snapshot entry owns an inbound object with separate `protocol` and + `parameters` fields。The protocol exact ID discriminates the parameter schema；HTTP parameters currently include method + and path。Do not flatten protocol identity and protocol-specific parameters into one object namespace。 +- **Local execution**: PeerOutboundRegistry maps exact Peer Protocol ID to the caller-local executable adapter，for + example `core.peer.protocol.http.v1` → `PeerHTTPOutbound`。Outbound is implementation identity，not the shared + advertisement value。 +- **Updated snapshot direction**: D-120's Peer-owned full-snapshot and lease ownership remain；its former `text[]` + projection is superseded by structured capability entries carrying inbound protocol descriptors。 +- **Confidence**: Sir accepted Peer Protocol、outbound mapping and the remaining delegate topology，corrected the + advertisement nesting and selected the explicit HTTP protocol ID。 + +### D-124 — Inbound publishes outbound parameters；delegation is one-shot + +- **Parameter semantics**: `inbound.parameters` are protocol-defined parameters published by the provider inbound and + consumed to construct/configure the caller-local PeerOutbound for that inbound。For + `core.peer.protocol.http.v1`，method/path are therefore PeerHTTPOutbound instance parameters，not generic capability + fields or separately authored caller mapping。 +- **Authority/consumer distinction**: inbound remains authority for how it is entered；outbound is the consumer/executor + of the advertised parameter object。The concise field name `parameters` remains valid because its schema is already + discriminated by sibling `protocol`。 +- **Delegate lifecycle**: `PeerManager.delegate()` is one-shot。Each call observes current online candidates and inbound + advertisements，constructs the protocol-selected outbound for an attempted Peer，executes the call and releases it。 + It returns no bound delegation/connection/target object for reuse。 +- **Deferred interaction mode**: long-lived WebSocket/session/multiplexed behavior is not forced into this one-shot + contract；it receives a separate design only when a real capability requires it。 +- **Confidence**: Sir corrected the direction of protocol parameters and explicitly approved one-shot delegation while + deferring WebSocket-style work。 + +### D-125 — Delegation crosses a JSON value boundary；Peer remains capability-business-blind + +- **Payload/result**: MVP `PeerManager.delegate()` accepts one normalized JSON value and returns one JSON value。The + capability owner serializes/normalizes its typed request before delegation and validates/reconstructs the typed result + afterward。 +- **Ownership**: SemanticRetrieval owns request/result models、domain validation and domain failure interpretation。 + PeerManager understands only exact capability identity、provider/inbound discovery、protocol-selected outbound + execution and opaque JSON transfer；PeerOutbound understands its protocol but not capability business。 +- **Protocol consequence**: `core.peer.protocol.http.v1` owns JSON wire encoding/decoding in addition to peer-JWT HTTP + behavior。A future non-JSON or streaming interaction must prove and introduce an appropriate protocol/boundary rather + than weakening this MVP contract pre-emptively。 +- **Guardrail**: a JSON delegation method inside PeerManager does not create a generic remote invoke endpoint；providers + retain their ordinary inbound paths and typed controllers。 +- **Confidence**: Sir explicitly approved the JSON boundary because Peer should understand capability delegation，not the + delegated capability's business。 + +### D-126 — SemanticRetrievalManager owns local-or-delegate selection；HTTP parameters must not imply body-only + +- **Unified domain entry**: SemanticRetrievalManager is the typed public domain facade on every Peer。It uses the local + implementation when registered；otherwise it normalizes the request，calls PeerManager.delegate() and validates the + returned JSON value。Do not add a second SemanticRetrieval facade without independent pressure。 +- **Provider path**: a Peer advertises SemanticRetrieval only when a local implementation exists。Its inbound invokes an + explicit non-delegating local execution path，preventing accidental cross-Peer delegation loops。 +- **JSON/binding distinction**: D-125's JSON value is the logical delegate payload/result boundary，not a requirement that + every HTTP inbound bind the request to an `application/json` body。D-127 supersedes this decision's initial proposal to + encode query versus body as an exclusive parameter。 +- **Media-type pressure**: future HTTP capabilities may need request/response media type or Content-Type/Accept behavior。 + Preserve a nested protocol-parameter space for that evolution；do not flatten or freeze the current method/path-only + example as the entire `core.peer.protocol.http.v1` contract。 +- **Still open**: the exact HTTP parameter schema and JSON-to-query constraints remain Technical review；non-JSON binary/ + streaming payloads still require an explicit future boundary rather than being smuggled through JSON。 +- **Confidence**: Sir approved the unified Manager/local path topology and identified HTTP query/media-type support as a + necessary correction to the prematurely narrow inbound example。 + +### D-127 — Separate static inbound parameters from per-call protocol payload；HTTP query/body may coexist + +- **Open-edge correction**: this begins closure of D-126's body/query schema question；D-130 fixes destination parameters + and D-135 freezes the complete MVP normalized HTTP envelope。 +- **Correction**: withdraw `request.location: query | body` and the invented `media_type` field。A standard HTTP request + may contain both query and body，so exclusive placement is not a valid protocol model。 +- **Static/dynamic split**: `inbound.parameters` configure construction of the protocol-specific outbound with static + interface facts such as method/path。The one-shot JSON passed through delegate is a protocol payload；for HTTP it may + independently contain both `query` and `body` members。 +- **Domain coupling is valid**: the SemanticRetrieval inbound belongs to SemanticRetrievalManager and may own the codec + between its typed domain request/result and the HTTP protocol payload。PeerManager still treats that JSON as opaque； + PeerHTTPOutbound interprets only the HTTP protocol envelope。 +- **HTTP vocabulary**: when a real requirement needs body representation metadata，use the standard `content_type` term， + not a newly invented `media_type` synonym。Whether content type is fixed inbound configuration or per-call payload is + decided by the concrete requirement；do not freeze it now。 +- **Current SemanticRetrieval**: its HTTP payload uses a JSON body and no query。Supporting query in the protocol shape + does not add unused SemanticRetrieval query semantics。 +- **Confidence**: Sir rejected the exclusive location model，required standard HTTP query+body compatibility，preferred + existing Content-Type terminology and confirmed that domain-owned inbound may couple to protocol payload。 + +### D-128 — Capability-owned inbound codec produces the delegated protocol payload + +- **Delegate input**: the JSON value supplied to `PeerManager.delegate()` is already a protocol-specific payload produced + by the capability owner's inbound codec，not an unmapped typed domain request。 +- **SemanticRetrieval example**: `SemanticRetrievalManager` validates and normalizes its typed request，then the + SemanticRetrieval-owned HTTP inbound codec maps it to the HTTP protocol payload；the current contract produces a JSON + `body` and no `query`。 +- **Provider path**: the corresponding provider inbound reconstructs the typed request and calls the explicit + non-delegating local execution path。 +- **Peer boundary**: `PeerManager` treats the protocol payload and result JSON as opaque；the selected `PeerOutbound` + interprets only the advertised Peer Protocol envelope。Neither learns SemanticRetrieval business semantics。 +- **Evolution**: if a capability later supports another Peer Protocol，that capability owns the corresponding codec。Do + not add a generic mapping DSL to `PeerManager` or the outbound registry without concrete pressure。 +- **Confidence**: Sir explicitly accepted that delegation receives codec-produced protocol payload rather than the raw + domain request。 + +### D-129 — Generic failover ends at provable non-execution + +- **Safe automatic failover**: `PeerManager` may try another eligible Peer only when it can prove that the selected + capability was not executed。This includes candidate rejection before dispatch and a Peer Protocol response that + explicitly guarantees the capability execution path was not entered。 +- **No generic replay after ambiguity**: a response containing a domain result/error is returned as-is。A timeout、reset + or other failure after the request may have reached the provider is outcome-unknown and must not be replayed + automatically by generic routing。 +- **Readiness remains internal**: an explicit protocol-level non-execution result can enable safe routing without adding + readiness to discovery。PeerManager understands only the execution boundary，not the provider's dependency condition。 +- **Future domain policy**: a capability owner may later add an explicit replay-safe policy when concrete use requires + it；PeerManager must not infer idempotency from HTTP method、capability name or current implementation behavior。 +- **Failure surface**: no eligible online provider and outcome-unknown dispatch are distinct explicit failures；neither + is silently converted into an empty retrieval result。 +- **Confidence**: Sir explicitly accepted failover only when capability non-execution can be proven。 + +### D-130 — HTTP destination belongs to the advertised inbound parameters + +- **Address authority**: `core.peer.protocol.http.v1` inbound parameters publish the absolute target `url` together with + `method`。`PeerHTTPOutbound` therefore receives all static construction parameters from the selected inbound descriptor； + it does not recover an origin from a separate Peer field。 +- **Peer shape**: the future `peers` relation has no global `rest_api_url` or renamed generic `base_url`。Peer identity and + liveness remain protocol-neutral，while each advertised inbound owns the endpoint at which that contract is callable。 +- **Accepted duplication**: repeating an origin across several capability snapshot entries is low-cost derived state。It + avoids a second protocol-endpoint map and permits different capability endpoints without changing Peer identity。 +- **Migration**: legacy `clients.rest_api_url` may inform the hard-cut migration/bootstrap，but it is not retained as a + second endpoint authority。An online runtime republishes its complete capability snapshot with absolute URLs。 +- **Confidence**: Sir explicitly accepted absolute `url` in HTTP inbound parameters and removal of global + `rest_api_url` from the Peer persistence model。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D131-D140.md b/tasks/knowledge-lifecycle-capabilities/decisions/D131-D140.md new file mode 100644 index 0000000..a30f357 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D131-D140.md @@ -0,0 +1,182 @@ +# Decisions D-131–D-140 + +> [Decision register index](index.md) + +### D-131 — Peer is one identity row with labels、config、snapshot and lease + +- **Relation**: hard-cut the technical `clients` relation to `peers` with `id: uuid`、required descriptive `name`、 + `labels: text[]`、`config: jsonb`、`config_schema: jsonb`、`capabilities: jsonb`、nullable + `lease_expires_at: timestamptz` and database-owned `created_at` / `updated_at`。 +- **No child lifecycle**: capability entries form one Peer-owned full snapshot and the lease is one Peer-scoped fact； + neither receives a child table、independent identity or generic service entity。 +- **Labels retained**: `labels` remain human/admin-authored Peer grouping metadata。They do not advertise capability、 + influence routing or substitute for liveness/readiness。 +- **Config boundary**: `config/config_schema` remain per-Peer application configuration and local `ConfigContract` + projection。They are not deployment-scoped `configs` rows and do not enter `DeploymentConfigManager`'s schema registry。 +- **Removed fields**: no global HTTP endpoint、persisted `online`、`last_seen` or readiness field is added。 +- **Confidence**: Sir accepted the proposed single-row shape except explicitly retained `labels`。 + +### D-132 — Lease owner supplies TTL；database owns expiry calculation + +- **Renewal contract**: the shared database helper is `renew_peer_lease(peer uuid, ttl_seconds integer) -> timestamptz`。 + It atomically writes and returns `statement_timestamp() + ttl_seconds`，using database time while allowing each Peer + deployment model to choose its own duration。 +- **Why TTL varies**: an always-on process、browser/runtime and scale-to-zero service can have materially different + renewal actors and safe expiry margins。The Peer Protocol must not impose one process-residency assumption。 +- **Liveness meaning**: the lease claims that the advertised inbound remains routable，not necessarily that one + application process is continuously resident。A deployment control plane may renew for a scale-to-zero Peer when it + owns the wakeable endpoint contract。 +- **Validation**: TTL must be positive and representable；MVP adds no arbitrary global maximum or persisted duplicate TTL + field。The expiry remains the sole shared liveness authority。 +- **Isolation**: ordinary Peer row updates never renew the lease。Unknown Peer renewal fails，and the helper is + `SECURITY INVOKER` rather than a privilege-escalation surface。 +- **Confidence**: Sir rejected a protocol-fixed TTL because Peer runtime/deployment models differ，while accepting the + remaining explicit database-time renewal contract。 + +### D-133 — Delegation uses caller-local randomized eligible-candidate ordering + +- **Eligibility**: a remote candidate must advertise the exact capability ID、hold a lease unexpired by database time、 + publish a valid inbound descriptor、use a Peer Protocol supported by the caller's outbound registry and not be the + caller Peer itself。 +- **Ordering**: each one-shot delegation randomly orders its eligible candidates locally。Do not derive priority from + UUID、lease duration/expiry、labels or snapshot order。 +- **Failover**: traverse that sequence only under D-129's provable-non-execution rule。A malformed advertisement or locally + unsupported Peer Protocol makes that candidate ineligible and produces diagnostics rather than invalidating other + candidates。 +- **No scheduler state**: MVP adds no weight、priority、load、stickiness、shared round-robin cursor、circuit breaker or + persisted routing state。A future cost/latency/locality requirement may introduce caller-owned routing policy。 +- **Failure**: no eligible candidate produces explicit `CapabilityDelegationUnavailable`，never an empty semantic result。 +- **Confidence**: Sir explicitly accepted caller-local randomized ordering among all eligible Peers。 + +### D-134 — HTTP non-execution uses `InkCre-Peer-Execution: not-executed` + +- **Accepted mechanism**: `core.peer.protocol.http.v1` uses one exact response header to guarantee that the request did + not enter capability execution。`PeerHTTPOutbound` converts it to a protocol-neutral internal `not_executed` outcome； + `PeerManager` never parses HTTP headers。 +- **Conservative default**: connection failure proven before dispatch is also `not_executed`。Every HTTP response lacking + the exact header is potentially executed regardless of status and is returned to the domain codec without automatic + failover。 +- **No readiness leak**: the header communicates only the execution boundary，not health、readiness or failure reason。 +- **Exact field**: freeze `InkCre-Peer-Execution: not-executed`。IETF BCP 178 / RFC 6648 deprecates the `X-` convention + for newly defined textual protocol parameters，while RFC 9205 recommends an application identifier prefix such as + `InkCre-` for application-specific HTTP fields。 +- **Infrastructure behavior**: FastAPI/Starlette can emit the field without special handling；ordinary reverse proxies do + not use `X-` as a generic pass-through marker。A browser caller requires CORS `Access-Control-Expose-Headers` for this + exact field。Acceptance must exercise the deployed proxy path rather than infer preservation from framework tests。 +- **Safe degradation**: if an intermediary removes the field，the caller conservatively treats the response as potentially + executed and stops failover。Field loss reduces availability but cannot authorize an unsafe replay。 +- **Confidence**: Sir accepted the mechanism，then accepted the non-`X-` exact name after reviewing standards and runtime + infrastructure behavior。 + +### D-135 — Peer HTTP v1 exchanges normalized request/response envelopes + +- **Protocol-shape closure**: together with D-127 and D-130，this closes D-126's remaining MVP HTTP parameter/payload + question。Binary/streaming remains an explicit future protocol version，not an unresolved part of HTTP v1。 +- **Static/dynamic split**: advertised HTTP inbound parameters own `method` and absolute `url`。Each one-shot request + payload may contain normalized `query`、`headers` and JSON `body` simultaneously；the response payload contains + `status`、normalized `headers` and JSON `body`。 +- **Normalization**: header/query names are canonicalized and repeated values are represented as `string[]`。Presence of + a body remains distinct from omission。Binary/streaming content requires a later Peer Protocol version rather than + smuggling bytes through JSON。 +- **Reserved HTTP authority**: `PeerHTTPOutbound` owns peer-JWT `Authorization`、authority/framing and hop-by-hop fields； + capability codecs cannot overwrite them。It consumes D-134's execution header before returning an ordinary response + envelope。 +- **Domain parity**: every other status/header/body value is preserved for the capability-owned codec，which reconstructs + the same typed result or failure exposed by local execution。PeerManager keeps both envelopes opaque。 +- **Current use**: SemanticRetrieval currently emits a JSON body and no query or additional domain headers。Generic HTTP + expressiveness does not add retrieval options without product evidence。 +- **Confidence**: Sir explicitly accepted the normalized HTTP request/response envelope and its authority boundaries。 + +### D-136 — Business and Peer Protocol inbound/outbound are orthogonal architectural roles + +- **Topology closure**: this closes D-122's registry-ownership question for MVP：capability-owned codecs and inbound + registration remain runtime-local collaborators；Peer advertisement persists only the protocol descriptor needed by + a caller's local outbound registry。No shared generic inbound/outbound catalog is introduced。 +- **Two views**: distinguish concrete Business inbound/outbound paths from Peer Protocol inbound/outbound mechanics。 + These names describe composed call roles，not a requirement for one class、module or persisted entity per box。 +- **SemanticRetrieval outbound**: caller-side `SemanticRetrievalManager` validates the domain call and uses its protocol + codec，then `PeerManager.delegate()` selects a provider/protocol and a `PeerHTTPOutbound` executes the normalized wire + request。The role spans all of those collaborators。 +- **SemanticRetrieval inbound**: provider-side HTTP/FastAPI protocol boundary、SemanticRetrieval route/codec and the + non-delegating local `SemanticRetrievalManager` execution path together form the Business inbound。 +- **Peer HTTP roles**: `PeerHTTPOutbound` is a cohesive caller-local implementation。Peer HTTP inbound is the reusable + protocol behavior composed into domain routes（peer JWT、normalized envelope、execution marker and applicable CORS）， + whether implemented by helpers/dependencies or a class。 +- **Manager coupling**: SemanticRetrievalManager's ownership of protocol codecs is therefore an intentional Business-edge + responsibility，not proof that retrieval business logic belongs in HTTP or that a second public remote Manager is + needed。Keep a private collaborator when it improves depth/readability。 +- **Terminology restraint**: HTTP remains a concrete Peer Protocol (`core.peer.protocol.http.v1`)。Do not revive a generic + persisted `transport` field、TransportManager or a claim that HTTP alone defines the protocol。 +- **Confidence**: Sir derived and explicitly accepted the two-view composition，including why a service-shaped Manager may + legitimately participate in protocol mapping without a one-role-one-class topology。 + +### D-137 — Breakdown is semantic graph factorization，not a mandatory part tree + +- **Product correction**: breakdown may replace one heterogeneous Block with an N-Block/Relation graph and delete the + original container。Preserving the input Block plus technical child chunks is not the universal product behavior。 +- **Relation value**: do not emit mandatory `part:<order>` relations merely to record that splitting occurred。Breakdown + should persist useful discovered dynamic properties/roles such as `highlight` or `need_adjustment` when supported by the + information。Composition/order relations remain possible only where document structure itself has use value。 +- **Expansion**: breakdown may also materialize information embedded in the input，for example resolving an authored URL + into an independent Block and linking it with the most specific supported semantic relation。This is graph enrichment， + not an excuse for LLM-authored provenance claims。 +- **Evidence preservation**: Sir retained the extractive/source-faithful premise。LLM may identify boundaries and + relationships，while fetched URL content has its own external-content authority；generated summaries/paraphrases must + remain a distinguishable future output rather than masquerading as authored fragments。 +- **Open identity boundary**: deleting a Block can invalidate its ID、incoming/outgoing relations and a source/protocol + lifecycle that addresses it directly。The next review must distinguish replaceable aggregation Blocks from identity- + bearing roots such as a Memos backend memo before approving destructive execution semantics。 +- **Confidence**: Sir explicitly corrected root preservation、rejected mandatory part relations and added semantic + relations plus embedded-URL materialization；identity-safe deletion remains under review。 + +### D-138 — Breakdown is an organization approach；MVP is additive and non-destructive + +- **Terminology correction**: breakdown is a solution/approach by which organization can improve the info-base，not an + `OrganizationBreakdownCommand`、mandatory runtime phase or preselected public method。Concrete operations may compose + breakdown、linking、resolver materialization and other approaches according to their owner。 +- **MVP mutation boundary**: retain the input Block unchanged and add useful ordinary Blocks/Relations。Do not delete、 + replace or generically rewire the original Block or its existing relations in this unit。 +- **Future replacement**: destructive substitution is not forbidden forever，but it requires an explicit owner-approved + replacement contract because its present marginal benefit is low while identity/lifecycle/relation risk is high。It is + excluded rather than partially designed now。 +- **D-137 correction**: D-137's semantic-factorization、non-mandatory-part and embedded-information directions remain；its + suggestion that current breakdown may delete the input is superseded。 +- **Confidence**: Sir explicitly distinguished approach from command and rejected destructive replacement for the current + scope unless a future design expressly authorizes it。 + +### D-139 — Valuable derived subgraphs may use an interpretation anchor + +- **Correction**: withdraw the proposed rule that absence of a domain-semantic anchor forces no-op。A derived subgraph can + itself improve use even when the source does not assert a specific property such as `highlight` or `reference`。 +- **Two anchor kinds**: prefer a concrete information relation when supported by evidence；otherwise connect the input + Block to one representative entry Block of the derived graph through an interpretation/breakdown relation。Only one + anchor path is required，not a direct relation to every derived Block。 +- **Markdown example**: organization may factor a Markdown document into title、quote and other ordinary Blocks/Relations， + then anchor the source document to the title/entry Block as its interpretation。The subgraph's internal structure，not + a mandatory `part:<order>` list，carries its use value。 +- **Authority meaning**: the anchor says “this graph is an interpretation of that authored Block”；it does not claim the + interpretation replaces、exhausts or mutates source authority。 +- **Exact vocabulary pending**: `interpretation` is the active recommendation because it is a noun fitting the dynamic- + property reading（target is source's interpretation）and does not turn the breakdown approach itself into persisted + output identity。Sir's `breakdown/interpret` wording establishes the fallback role；exact content awaits review。 +- **Confidence**: Sir explicitly rejected semantic-anchor-or-no-op and supplied the Markdown/title interpretation graph + example。 + +### D-140 — Breakdown is an organization behavior/representation category；interpretation is an approach + +- **Taxonomy correction**: supersede D-138's wording that called breakdown itself an approach。`breakdown` is a naming + umbrella/category for organization behavior and resulting representation：information previously aggregated in one + Block becomes a more useful multi-Block/Relation graph。 +- **Concrete approach**: `interpretation` is the first approved organization approach under that category。It reads an + authored/solved input，materializes a source-faithful structured graph and anchors that graph without replacing source + authority。A concrete implementation may combine parser、LLM、resolver materialization and linking mechanics。 +- **Persisted vocabulary**: freeze exact relation content `interpretation` for the fallback anchor from source Block to a + representative entry Block。This names the derived graph's relation to the source，not an execution command or class。 +- **Pending-name closure**: this closes D-139's exact-vocabulary question and supersedes its tentative + `interpretation/breakdown` wording with exact persisted relation content `interpretation`。 +- **No one-to-one implementation claim**: neither `Breakdown` nor `Interpretation` automatically requires a Manager、 + public method、job or single plugin class。Those code/runtime boundaries follow concrete execution pressure。 +- **Scope restraint**: this decision does not automatically reclassify `merge` or `linking`；their category/approach + semantics remain available for later evidence-based review。 +- **Confidence**: Sir explicitly reframed breakdown as umbrella/category and identified interpretation-like concepts as + actual organization approaches，with breakdown closer to representation/behavior。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D141-D150.md b/tasks/knowledge-lifecycle-capabilities/decisions/D141-D150.md new file mode 100644 index 0000000..e75f156 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D141-D150.md @@ -0,0 +1,180 @@ +# Decisions D-141–D-150 + +> [Decision register index](index.md) + +### D-141 — Breakdown focuses on one Block-in-graph；resolver projections are understanding capabilities + +- **True input**: the focal Block is the information authority around which breakdown behavior unfolds。Do not redefine + resolver text or a synthetic three-field DTO as the thing being organized。 +- **Graph context**: understanding the focal Block requires its position in the graph，so direct incoming/outgoing + Relations belong in the analysis horizon。They are context around the input rather than co-equal replacement inputs； + deeper traversal requires separate evidence。 +- **Resolver role**: resolver projections are ways to understand/apply the Block。Text is the current interpretation + capability needed by the MVP，not the essence or identity of breakdown。Future image/audio/structured approaches can + use other resolver capabilities without changing the focal-Block model。 +- **Content caveat**: a particular inline Block can sometimes be understood by reading `content` directly，but generic + organization cannot rely on that：storage-backed `content` is an opaque pointer and canonical inline JSON still needs + its exact resolver contract。Resolver use preserves the existing hydration/interpretation boundary。 +- **Rumination insight**: Sir identified breakdown's essence as “反刍”：reconsidering one Block in graph context to produce + a more useful representation。Whether `rumination` becomes the stable approach name and `interpretation` becomes only + the persisted output relation remains the next terminology review；no class/job is implied either way。 +- **Confidence**: Sir explicitly supplied the focal Block、direct-graph-context and understanding rationale；the storage + caveat is verified current architecture fact。 + +### D-142 — Rumination owns its understanding representation；breakdown leaves the technical vocabulary + +- **Content correction**: Sir's product-level “Block content” means the actual content carried by the Block after any + storage pointer has been resolved，not the persisted `Block.content` column in isolation。The established persisted- + versus-hydrated distinction remains valid；D-141 incorrectly turned that field-level distinction into a general product + prohibition。 +- **Approach-owned adaptation**: the concrete rumination implementation decides which representation its reasoning core + needs。A text LLM can use suitable hydrated text；another implementation may ask a resolver for text、image、audio or + structured understanding。Resolver `get_text()` is therefore one valuable path，not a mandatory universal boundary。 +- **Stable taxonomy**: `rumination` is the concrete organization approach：reconsider one focal Block in its direct graph + context and materialize a more useful ordinary graph。`interpretation` remains the exact persisted fallback relation + from the focal Block to a representative entry Block of the resulting subgraph。 +- **Retired technical name**: `breakdown` may remain as historical/product shorthand for an aggregated expression becoming + a richer graph，but it does not justify `Breakdown` classes、methods、registries、DTOs、jobs or other code symbols。 +- **D-140/D-141 correction**: retain the useful behavior observation、focal-Block authority and direct-relation context，but + withdraw breakdown as a stable technical category and withdraw mandatory resolver-text adaptation。 +- **Confidence**: Sir explicitly corrected the content meaning，accepted rumination and judged breakdown unimportant enough + that it may not appear in code。 + +### D-143 — Hydration exposes content；resolver owns understanding + +- **Boundary correction**: hydration resolves a storage pointer into the actual content carried by a Block；it does not + make that content a generic application input。Resolver capabilities own how an application understands that Block。 +- **No string shortcut**: the text-LLM rumination implementation requests resolver `get_text()` even when hydrated content + happens to be a string。Inspecting the runtime value and bypassing the resolver would duplicate resolver knowledge and + make semantically different Blocks look equivalent merely because they share a storage representation。 +- **Common pattern**: a reasoning core declares the understanding capability it needs；the Block's resolver adapts actual + hydrated content to that capability。Hydration remains below the resolver boundary，while rumination remains above it。 +- **Outcome correction**: absence of the required understanding capability does not mean rumination is inapplicable to the + Block。It means this concrete implementation cannot understand the Block。That expected limitation must remain distinct + from resolver/provider/storage execution failure and produces no graph mutation。 +- **D-142 correction**: withdraw D-142's suggestion that a text LLM may consume hydrated text directly。Its approach-owned + choice is the required capability（text），not permission to bypass the resolver that supplies it。 +- **Confidence**: Sir explicitly distinguished the essence of hydrated content from the resolver-owned understanding + boundary and corrected `not_applicable` to “I cannot understand it”。 + +### D-144 — Rumination exposes shallow best-effort completion semantics + +- **Public promise**: rumination uses the capabilities currently available to complete one best-effort consideration of + the focal Block in graph context；it does not promise that the graph will change。 +- **Silent normal completion**: inability to understand the Block with the selected implementation and a completed + understanding that yields no useful graph output both satisfy the public promise。Neither requires a public + `not_applicable`、`cannot_understand` or `no_change` result。 +- **Failure boundary**: if the one attempt itself cannot complete，the module may expose one abstraction-appropriate + high-level failure。Storage、resolver、AI-provider and persistence exception taxonomies do not become public rumination + contracts。 +- **No resilience subsystem now**: MVP does not add internal retry、compensating rollback、degradation or fallback policy。 + Existing dependency/transaction behavior may operate normally，but rumination does not acquire orchestration machinery + without concrete product pressure。 +- **Depth pattern**: a deep module may use rich internal outcomes when they are needed for its own decisions，while keeping + its public completion semantics shallow。This is permission，not a requirement to prebuild an unused outcome algebra； + distinctions with no caller or internal decision value should not enter the interface。 +- **UNIX analogy boundary**: hiding an implementation event is correct when the module still satisfies its abstraction- + level postcondition；returning success after failing that postcondition would be a lie，not depth。 +- **Promotion pressure**: record the general depth/error-surface rule for later Product TDD reconciliation；keep the exact + rumination execution contract in this unit until implementation evidence stabilizes it。 +- **Confidence**: Sir explicitly approved best-effort/no-mutation completion，warned against premature recovery machinery + and identified the shallow-public/rich-internal rule as a common pattern。 + +### D-145 — MVP graph context is a bounded direct-relation snapshot + +- **Focal understanding**: the text-LLM implementation obtains the focal Block's text through its resolver `get_text()`。 +- **Relation projection**: each direct Relation enters context with its direction and exact content，preserving the dynamic- + property reading `to is from's <relation-content>` rather than flattening endpoints into an unordered neighbor list。 +- **Neighbor projection**: the other endpoint is represented by an opaque Block reference plus resolver name and resolver + `get_label()` output。Its complete semantic content is not loaded into the default MVP context。 +- **Boundary value**: this snapshot exposes the focal Block's immediate graph position without silently turning every + neighbor into another substantive input or creating recursive expansion、token-budget and traversal policy。 +- **Future Agent path**: a later Agent-shaped rumination implementation may receive graph-navigation tools and actively + choose which neighbors or deeper paths to understand。MVP does not prebuild tool calls、agent loops or exploration + policy，and that future implementation does not change focal-Block authority。 +- **Confidence**: Sir judged this bounded context sufficient for MVP and identified active tool-driven graph exploration + as a possible future direction。 + +### D-146 — AgentManager is a reusable graph-blind orchestration module + +- **Approved topology**: organization approaches own domain prompts、tool selection and mutation policy；AgentManager owns + reusable model/tool orchestration；AIManager remains the model → provider → dialect executor。Neither reusable manager + understands info-base graph semantics。 +- **Why a separate manager**: system-prompt/tool composition and typed tool dispatch have reuse across organization + approaches and therefore form an independent deep-module boundary。This differs from an AIDialectManager，which would + only expose AIManager's internal adapter mechanism。 +- **MVP tool boundary**: rumination supplies only one mutation tool，`submit_graph`。It does not expose Block、Relation or + graph-navigation reading tools；focal resolver text plus the approved direct-relation snapshot remains the prepared + model input。 +- **Agent-loop scope**: introducing Agent infrastructure now enables native function calling and reusable tool contracts； + it does not authorize recursive exploration、autonomous tool discovery or a broad application-tool registry。 +- **Legacy removal**: the handwritten `BlockManager.query_by_reasoning()` `FOUND/CONTINUE` string protocol、CSV context and + recursive pseudo-agent are failure evidence and are deleted by the implementation hard cut rather than adapted。 +- **Confidence**: Sir explicitly approved the module topology，authorized MVP Agent use with submit-only tooling and + requested removal of the early reasoning-query implementation。 + +### D-147 — GraphForm is the producer-facing connected-graph command + +- **Naming hard cut**: replace `SubGraphForm` with `GraphForm`。The command is not required to be rooted or recursively + nested，so the shorter established graph noun is clearer。 +- **Form/model split**: add producer-facing `BlockForm` and `RelationForm` that exclude database identity、timestamps and + other table-owned state。Do not expose `BlockModel` / `RelationModel` as command or tool-call fields。 +- **Flat graph shape**: `GraphForm` may use non-persisted local references to declare new/existing Blocks and Relations as + one connected graph，rather than forcing a recursive tree。Exact reference and validation grammar remains the next + technical review。 +- **Persistence ownership**: `InfoBaseManager` maps the validated `GraphForm` to ordinary persisted Blocks/Relations and + continues to own graph insertion/fetchsert coordination。No LLM-only graph proposal domain is added。 +- **LLM authority**: the function-calling schema can accept `GraphForm` JSON directly while organization-owned tool + constraints/handler supply technical facts that the model does not own，including fixed resolver/storage policy。 +- **Confidence**: Sir approved the producer-command hard cut，identified direct persistence-model reuse as the actual + defect，selected `GraphForm` and accepted a non-nested representation。 + +### D-148 — Persist Agent definitions；keep execution history backend-extensible + +- **Persisted identity**: Agent persistence means reusable Agent definitions，not Agent runs。A definition is the + composition of one system prompt and an unordered set of exact Agent Tool references；tool order has no semantics and + duplicates are invalid。 +- **Runtime tool binding**: an Agent Tool consists of exact ID、typed schema and executable handler。The persisted Agent + references exact IDs；AgentManager's peer-local registry binds each ID to its schema/handler because executable callables + are not database values。 +- **MVP history**: every execution uses a short-lived in-memory message history。No messages or execution state are + persisted in this unit。 +- **Extension seam**: Agent domain keeps message-history storage behind an internal backend contract so a future persistent + implementation need not rewrite orchestration。Do not implement a persistent backend before product pressure exists。 +- **Everything is message**: future persistence stores one ordered message history；assistant tool calls and tool results + remain message variants rather than acquiring separate tables/lifecycles。 +- **No checkpoint domain**: checkpoint/resume state is intentionally excluded，including from the long-range architecture， + because it would create disproportionate execution lifecycle complexity。 +- **Exactly-once ownership**: AgentManager does not promise tool-call exactly-once。A tool that needs that guarantee owns its + own identity/idempotency/persistence contract behind its handler。 +- **Confidence**: Sir explicitly fixed definition-only persistence、unordered tools、in-memory MVP history、message-only + future persistence、no checkpoints and tool-owned exactly-once semantics。 + +### D-149 — Registry-owning Managers expose decorator registration + +- **Common pattern**: when Python runtime implementations are discovered by exact ID，the owning Manager should offer a + decorator registration surface for the implementation function or class。Agent Tools、Source types and Resolvers are + current pressure examples。 +- **Manager ownership**: do not create one global Registry module merely to share decorator syntax。AgentManager、 + SourceManager、ResolverManager and other registry-owning Managers retain their own exact-ID、metadata、duplicate and + lifecycle rules。 +- **Decorator behavior**: registration validates and binds the implementation at import/load time and returns the decorated + object unchanged。The syntax should expose identity visibly rather than rely on hidden `__init_subclass__` side effects。 +- **Promotion pressure**: record this as a shared implementation pattern for later Product TDD/local architecture + reconciliation once the Agent implementation proves the exact API shape。 +- **Confidence**: Sir explicitly requested decorator-based registration across Agent Tool functions and earlier class-based + registries。 + +### D-150 — Flat GraphForm coexists with the renamed nested graph form + +- **Correction to D-147**: `GraphForm` takes over the concise canonical name for the new non-nested producer command，but + the existing recursively nested form is retained under a more exact new name rather than deleted。 +- **Existing consumers**: extensions and other producers may continue using the nested representation after the explicit + rename and migration to `BlockForm` / `RelationForm`；they are not forced to build flat local-reference tables。 +- **One persistence authority**: InfoBaseManager accepts/normalizes the supported producer forms before performing the same + Block/Relation persistence coordination。The second representation does not create a second graph domain or persistence + path。 +- **Exact name pending**: the retained form's name must describe its nested serialization without claiming that the + resulting info-base graph is necessarily a tree。`NestedGraphForm` is the active recommendation。 +- **Confidence**: Sir explicitly retained the existing representation for extensions while assigning `GraphForm` to the + new flat tool-friendly command。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D151-D160.md b/tasks/knowledge-lifecycle-capabilities/decisions/D151-D160.md new file mode 100644 index 0000000..41354b0 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D151-D160.md @@ -0,0 +1,175 @@ +# Decisions D-151–D-160 + +> [Decision register index](index.md) + +### D-151 — Rename the retained recursive form to StarGraphForm + +- **Exact name**: the existing `SubGraphForm` becomes `StarGraphForm`。Each form unit owns one center Block plus its direct + incoming/outgoing arcs，and an arc endpoint may recursively contain another `StarGraphForm`。 +- **Topology meaning**: the payload composes stars recursively；the name does not claim that the complete persisted + info-base graph is one mathematical star。 +- **Exposed limitation**: root-relative nesting cannot naturally express arbitrary cross-links、cycles or one newly + declared Block shared by several branches without duplication/reference workarounds。This is a representation limit， + not a defect in graph persistence itself。 +- **GraphForm contrast**: the new flat `GraphForm` declares Blocks once and connects them through reference placeholders， + allowing an arbitrary connected Block/Relation graph while remaining a producer command。 +- **Consumer continuity**: extensions may keep the star-shaped authoring style after the explicit rename and migration to + `BlockForm` / `RelationForm`；they are not forced onto reference placeholders。 +- **D-150 closure**: supersede the active `NestedGraphForm` recommendation with exact `StarGraphForm`。 +- **Confidence**: Sir proposed StarGraph as the clearer structural name and connected it directly to the current form's + limitation versus flat reference-based GraphForm；the remaining Agent decisions were approved。 + +### D-152 — Exact retained-form name is StarsGraphForm + +- **Name correction**: supersede D-151's singular `StarGraphForm` with exact `StarsGraphForm`。The plural is intentional： + one payload recursively composes multiple root-relative star units。 +- **Meaning unchanged**: each unit still centers one Block and its direct in/out arcs；the resulting persisted graph is not + constrained to one star topology。 +- **Confidence**: Sir explicitly selected the plural name after reviewing the recursive composition meaning。 + +### D-153 — GraphForm uses ID-free placeholders with caller-owned bindings + +- **Flat command**: `GraphForm` declares new `BlockForm` values once under unique opaque placeholder names and declares + `RelationForm` endpoints through those placeholders。 +- **Existing Blocks**: a caller supplies a separate placeholder → existing Block ID binding map。Database IDs do not enter + the GraphForm JSON or become LLM-authored values。 +- **Rumination authority**: its `submit_graph` handler owns bindings for `focal` and the direct-context neighbors。The model + may reference only the opaque names it was given。 +- **Validation**: every relation endpoint must resolve exactly once from either the GraphForm's new-block declarations or + caller bindings；the two namespaces cannot collide，new placeholders are unique and dangling references fail before their + relation is persisted。 +- **Persistence**: InfoBaseManager resolves caller bindings，fetchserts/inserts new Blocks，maps their placeholders to actual + IDs and then persists Relations through the same coordination path。 +- **Confidence**: Sir accepted the completely ID-free GraphForm and caller-owned reference-binding boundary。 + +### D-154 — Agent definition has a minimal model-independent persisted shape + +- **Relation**: persist `agents` with database-generated bigint `id`、required descriptive `name`、required + `system_prompt`、`tools: text[]` and database-owned `created_at` / `updated_at`。 +- **Tool-set semantics**: exact Agent Tool IDs are an unordered set。Canonicalize storage order and reject duplicates so a + reorder does not create false updates。 +- **Model routing**: Agent does not reference AIModel。The invoking organization/use owner selects a model at execution + time，preserving the accepted definition `Agent = system prompt + Agent Tools`。 +- **Scope restraint**: do not add enabled、description、generic config、run/session state、history or checkpoint fields。 +- **Execution snapshot**: an in-memory execution uses the Agent definition it loaded；a concurrent definition update does + not mutate that run's prompt/tool set。 +- **Missing runtime tool**: inability to bind one persisted exact tool ID prevents that Agent execution and is surfaced as + one high-level Agent failure；do not add readiness、fallback or hidden tool substitution。 +- **Confidence**: Sir explicitly approved the proposed minimal Agent definition persistence shape。 + +### D-155 — GraphForm uses signed IDs；Agent runtime owns typed tool I/O validation + +- **Self-contained command**: remove caller reference bindings and the domain-specific handler context from + `submit_graph`。Rumination includes real focal/direct-neighbor Block IDs in the initial user message；future graph-reading + tools may accept the same real identities directly。 +- **Signed namespace**: GraphForm uses non-zero signed Block IDs。Positive values refer to existing persisted Blocks； + negative values identify new Blocks only within that GraphForm；zero is invalid。After insertion，InfoBaseManager maps + negative IDs to database-generated positive IDs。 +- **Exact field name**: use `id`，not `ref`。A negative value is still a valid form-scoped identifier，and Form vocabulary + already means database-managed fields are absent unless explicitly stated。Relation endpoints use the same signed-ID + namespace。 +- **Form validation**: Pydantic field/model contracts own structural facts available without I/O：negative IDs on new + Block declarations、non-zero endpoints、unique new IDs and resolution of every negative endpoint within the form。 +- **Runtime validation**: AgentManager derives tool input and output contracts from handler annotations，validates the LLM + JSON into the declared Pydantic input before invocation，validates the handler result against its declared output，then + emits the tool-result message。This is analogous to FastAPI's route boundary。 +- **Domain execution**: existence of positive Block IDs and graph persistence facts require info-base I/O and remain inside + InfoBaseManager。The thin `submit_graph` handler delegates；it does not duplicate schema or domain validation。 +- **D-153 correction**: retain flat arbitrary connectivity and local placeholder benefits，but supersede string opaque + placeholders、caller bindings and the claim that GraphForm itself is database-ID-free。 +- **Confidence**: Sir approved signed local IDs，selected exact `id` and moved tool I/O validation to the Agent runtime。 + +### D-156 — Agent Tools validate inputs only；PostgreSQL owns existing-ID integrity + +- **Input boundary**: AgentManager derives the Tool's LLM-visible schema from its Pydantic input annotation and validates + call arguments before handler invocation。 +- **Validation message**: on invalid arguments，serialize Pydantic's own `ValidationError` into the tool-result message。 + Add only a thin transport wrapper if required；do not create a parallel error model、translation table or validation + taxonomy。 +- **Output boundary**: Tool return annotations are enforced by ordinary static type checking。AgentManager does not perform + Pydantic/runtime output validation or publish a second output schema。Serialization failure remains an implementation + failure，not another validation layer。 +- **Database integrity**: InfoBaseManager does not pre-query whether positive GraphForm Block IDs exist。It performs the + graph command and PostgreSQL foreign keys remain the sole existence/integrity authority for persisted Relation endpoints。 +- **Structural validation unchanged**: GraphForm still validates no-I/O invariants such as negative new IDs、non-zero + endpoints、uniqueness and internal resolution of negative IDs。 +- **D-155 correction**: supersede runtime Tool-output validation and the proposed positive-ID existence check，while keeping + signed IDs、thin handler delegation and Agent-owned Pydantic input validation。 +- **Confidence**: Sir explicitly selected FK-owned existence、static-only output checking and direct Pydantic validation + errors with at most a thin wrapper。 + +### D-157 — AI owns canonical Messages；capabilities and dialects are orthogonal axes + +- **Message ownership**: provider-neutral Message contracts belong to the AI module boundary because AIManager consumes and + returns them；AgentManager depends on AIManager and stores the same canonical messages through its history backend rather + than defining a duplicate Agent message DTO。 +- **Canonical union**: use explicit System、User、Assistant and Tool message variants。Assistant owns optional text plus + ToolCalls；ToolMessage owns call identity plus JSON content。Tool calls/results remain messages，not independent domain + entities。 +- **Capability partition**: organize AI implementation by operation capability，beginning with embedding and the message- + based chat/language-model capability used by AgentManager。The exact second capability/module name remains the next naming + review。 +- **Dialect partition**: a dialect such as `core.openai-compatible.v1` owns provider/model/config client construction and + wire translation across every capability it supports。Do not create `chat-openai-compatible` or + `embedding-openai-compatible` dialect types/adapters。 +- **Dependency direction**: AgentManager composes prompts、history and tools，then calls the AI message capability；AIManager + and its dialect adapter remain unaware of Agent definitions、history backends and graph semantics。 +- **Future history**: an optional persistent Agent history backend stores the canonical ordered Message union。Backend row + identity/timestamps remain backend mechanics and do not enter the current message contract。 +- **Confidence**: Sir approved the Message ownership/union and explicitly separated capability modules from a shared + OpenAI-compatible dialect adapter。 + +### D-158 — Exact message capability and module name is chat + +- **Exact name**: use `chat` for the AI capability type、module and AIManager operation that accepts canonical Messages plus + optional Tools and returns an AssistantMessage。 +- **Not a product**: `chat` does not mean the Chat InKCre UI/product journey and does not move conversation/session/history + ownership into AIManager。 +- **Not a wire endpoint**: an AIDialect may implement `chat` through OpenAI Chat Completions、Responses or another + provider-native API。The capability name does not create endpoint-specific dialect identities。 +- **Rejected name**: do not use `llm` as the capability/module name；LLM describes a model category rather than the typed + operation performed。 +- **Stable surface**: the initial capability partition and methods are `embedding` / `chat` and `AIManager.embed()` / + `AIManager.chat()`；AgentManager consumes `chat`。 +- **Confidence**: Sir explicitly approved exact `chat` after reviewing the capability/model/product/wire distinction。 + +### D-159 — AI capability features declare model-offering support and intersect dialect/runtime support + +- **Capability-shape closure**: together with D-157/D-158's exact `embedding` / `chat` capability partition，this closes + D-086's remaining MVP capability vocabulary/discriminated-item question。The common typed JSON fields are `type`、 + `input_modalities`、`output_modalities` and `features`；capability modules own their exact item variants。 +- **Shape**: every typed item in `AIModel.capabilities` includes `features: string[]` alongside `type`、 + `input_modalities` and `output_modalities`。The JSON representation is an array，but its semantic value is an unordered + duplicate-free set；canonicalize ordering before persistence。 +- **Exact feature**: the Agent-required chat feature is `tool_calling`。Feature names are interpreted inside their owning + capability type；do not add a global feature registry or table。 +- **Name choice**: use `features`，not `extra_supports`。These are first-class capability facts，not secondary extras，and + `supports` is not a clear field noun。 +- **Joint support**: an OpenAI-compatible wire dialect does not prove that one configured model offering can call tools。 + Effective support requires the requested feature to be declared by the provider-bound `AIModel`，implemented by the + selected peer-local dialect adapter and enabled/satisfied by that provider's configuration。 +- **Ownership**: `AIModel.capabilities` declares the effective behavior of that provider + native-model offering；it is not + a timeless intrinsic fact about a model family。The dialect adapter owns canonical Tool/Message wire translation and + validates dialect-specific provider configuration。 +- **Agent gate**: AgentManager requires `chat.features` to contain `tool_calling`；AIManager rejects execution before a + provider request when the model declaration、local adapter support or required provider configuration is absent。 +- **Evidence**: vLLM's OpenAI-compatible server requires model-specific tool-call parsers and sometimes chat templates， + while Hugging Face documents that tool schemas must be rendered in the format expected by the model's training/template。 + Therefore dialect-level transport support and model-level semantic support are independent necessary conditions。 +- **Confidence**: Sir explicitly requested `features: string[]` now and asked that model-versus-dialect ownership be + established from implementation facts。 + +### D-160 — Provider-neutral feature name is tool_calling + +- **Exact name**: keep `tool_calling` as the `chat.features` value。Do not rename the canonical feature to + `function_calling`。 +- **Stable meaning**: the model can receive caller-supplied Tools with structured input schemas，return structured + ToolCalls，and consume corresponding ToolMessages after caller-side execution。 +- **Vocabulary boundary**: `function` may remain a concrete tool kind or wire term in OpenAI-compatible、Gemini or another + dialect。The dialect adapter translates that provider vocabulary to the canonical Tool/ToolCall/ToolMessage contract。 +- **No overclaim**: `tool_calling` does not imply support for every provider-managed built-in tool、MCP integration or + future tool kind。Those require their own concrete contracts/features when introduced。 +- **Why**: OpenAI and Gemini use Function Calling for custom functions inside a broader Tools surface，Anthropic uses Tool + Use，vLLM uses Tool Calling and MCP standardizes Tools。The provider-neutral umbrella therefore follows the cross- + provider Tool vocabulary and the existing Agent domain rather than one dialect's current function representation。 +- **Confidence**: Sir explicitly approved the researched terminology distinction and exact canonical name。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D161-D170.md b/tasks/knowledge-lifecycle-capabilities/decisions/D161-D170.md new file mode 100644 index 0000000..e7213cc --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D161-D170.md @@ -0,0 +1,195 @@ +# Decisions D-161–D-170 + +> [Decision register](index.md) + +### D-161 — Agent loop bound counts model calls per invocation + +- **Exact parameter**: `AgentManager.run(..., max_model_calls: int)` owns the sole MVP loop bound。 +- **Counted event**: increment the budget once for every attempted `AIManager.chat()` call。Do not count ToolCalls、tool + executions、Messages or loosely named iterations。 +- **Natural completion**: an AssistantMessage with no ToolCalls ends the run naturally，independent of whether its text is + empty。 +- **Multiple calls**: one AssistantMessage may contain multiple ToolCalls without consuming additional model-call budget； + the subsequent `AIManager.chat()` consumes the next unit after tool results are appended。 +- **Ownership**: `max_model_calls` is invocation policy selected by the caller，not persisted Agent-definition identity or + configuration。 +- **Why**: model calls directly bound provider cost、latency and non-termination，while tool/message counts vary with + parallel calls and result representation。 +- **Open edge**: the public completion semantics when the next model call would exceed the bound remains the next review。 +- **Confidence**: Sir explicitly approved `max_model_calls` as the only loop-bound unit。 + +### D-162 — Model-call limit is a typed non-exception termination + +- **Result**: `AgentManager.run()` returns an ephemeral `AgentRunResult` containing the complete in-memory Messages and + `termination: Literal["completed", "max_model_calls"]`。 +- **Completed**: use `completed` only when an AssistantMessage contains no ToolCalls and therefore ends naturally。 +- **Limit**: when another model call would exceed the invocation budget，return `max_model_calls` without making a cleanup + call or raising a limit exception。 +- **Side effects**: preserve every Tool effect already completed。Do not retry、roll back or compensate，and do not imply + that the caller should replay the run。 +- **Why typed**: silent indistinguishability is insufficient for a future consumer that needs a final answer，while an + exception would misclassify a caller-selected bound and could encourage unsafe replay after Tool side effects。Rumination + may intentionally ignore the termination value。 +- **Persistence**: AgentRunResult and termination are execution-local values，not new persisted run/session entities；the + approved future history backend still stores Messages only。 +- **Confidence**: Sir explicitly approved this non-exception typed outcome。 + +### D-163 — Same-turn ToolCalls are order-independent executions and failures are isolated + +- **Multiplicity**: accept every ToolCall emitted by one model turn；do not constrain one Assistant output to one call even + when the MVP Agent exposes only `submit_graph`。 +- **Order semantics**: provider-returned list order is not an execution dependency or result-order contract。Calls in one + turn have not observed one another's results and therefore form an order-independent execution batch correlated by + exact call IDs。 +- **Execution**: a first implementation may execute the batch sequentially，but must not promise that order。Concurrent + execution is the preferred later optimization when technically appropriate。 +- **Barrier**: collect one Tool result for every call before the next `AIManager.chat()`；completion order may differ from + call presentation order。 +- **Failure isolation**: one ToolCall failure becomes that call's error result and does not cancel or skip remaining calls。 + Preserve successful side effects；do not roll back the batch or retry failed calls implicitly。 +- **Feature scope**: this remains ordinary `tool_calling` semantics and does not introduce a separate + `parallel_tool_calling` feature merely because the model emits multiple same-turn calls。 +- **Confidence**: Sir approved multiple calls，removed strict ordering，preferred future concurrency and explicitly required + remaining calls to continue after one failure。 + +### D-164 — One ToolResultMessage owns the complete result batch + +- **Correction**: supersede D-157's single-result `ToolMessage` detail。The canonical Message union is SystemMessage、 + UserMessage、AssistantMessage and `ToolResultMessage`；do not introduce one independent ToolCallMessage per call。 +- **Assistant output**: AssistantMessage owns optional text plus `tool_calls: ToolCall[]` from one model call。This preserves + the batch boundary even when a provider exposes calls as independent wire items。 +- **Result shape**: `ToolResultMessage.results` is a non-empty list of `{tool_call_id, content, is_error}` values。ToolResult + is nested content，not an independently persisted Message/entity。 +- **Batch invariant**: result call IDs are unique and their set exactly equals the preceding AssistantMessage's ToolCall ID + set。Result array order is non-authoritative；the complete ToolResultMessage immediately follows that AssistantMessage + before any new ordinary Message/model call。 +- **Adapter mapping**: an OpenAI Chat adapter splits one canonical result batch into multiple `role=tool` messages；an + OpenAI Responses adapter splits it into `function_call_output` items；Anthropic/Gemini adapters group results into their + user content blocks/parts as required。 +- **Why**: AgentManager already waits at a result barrier，so multiple single-result Messages would falsely model individual + executions as conversation turns and allow one dialect's wire shape to control the provider-neutral domain。 +- **Confidence**: Sir proposed and explicitly approved the batch-shaped ToolResultMessage after wire-contract review。 + +### D-165 — Failed ToolResult content has three exposure layers without an error DTO + +- **Stable result**: every nested ToolResult keeps `is_error: bool` plus unconstrained JSON `content`；do not add a common + Agent error schema or wrapper object。 +- **Schema failure**: invalid model arguments produce `is_error=true` with Pydantic ValidationError's native JSON value as + content。Do not translate it into a second validation vocabulary。 +- **Declared Tool failure**: a handler may raise a thin `ToolExecutionError(content: JsonValue)` to expose actionable、 + Tool-owned failure information to the model。This exception carries content only and does not define a DTO hierarchy。 +- **Unexpected failure**: catch an ordinary unexpected handler Exception，record a stable generic JSON value in that call's + error result and log the complete exception/traceback internally。Do not expose database/framework exception strings as + provider-facing behavior。 +- **Success**: a successful ToolResult contains the handler return value's JSON serialization and `is_error=false`。 +- **Batch behavior**: every failure layer remains isolated to its call；remaining calls execute and the complete result batch + is returned。No retry or rollback is introduced。 +- **Why**: the model receives actionable contract-owned detail where one exists，while Agent behavior does not couple to + unstable internal implementation text and operators retain full diagnostics。 +- **Confidence**: Sir explicitly approved the three-layer content-exposure boundary。 + +### D-166 — Tool choice is protocol control；rumination meaning belongs to the system prompt + +- **Protocol evidence**: OpenAI and Anthropic expose exact `tool_choice` controls with `auto` semantics；Gemini exposes the + equivalent `functionCallingConfig.mode=AUTO`。This is existing LLM chat/tool protocol vocabulary，not an InKCre-created + organization concept。 +- **MVP execution**: AgentManager uses canonical `tool_choice="auto"` for every run。The dialect adapter maps it to the + provider equivalent；do not persist it in the Agent definition。 +- **No Agent option yet**: `AgentManager.run()` does not expose required、none or forced-tool selection in MVP。Concrete + demand must justify broadening that invocation interface。 +- **Semantic owner**: the rumination Agent system prompt explicitly instructs the model to call `submit_graph` only for a + meaningful rumination and otherwise end honestly without a call。Protocol `auto` merely permits this choice and does not + own the decision criterion。 +- **No-op**: an AssistantMessage without ToolCalls is natural `completed` termination and a valid rumination no-op，not a + Tool or Agent failure。 +- **Mixed output**: if one AssistantMessage contains text and ToolCalls，preserve the text in history and execute the complete + call batch。 +- **Confidence**: Sir accepted protocol-level auto selection and explicitly placed honest meaningful-rumination guidance in + the system prompt。 + +### D-167 — ToolCall state drives the loop；tool_choice belongs to Agent definition + +- **Message condition**: state the execution rule only as：when AssistantMessage has ToolCalls，execute that complete batch。 + Text presence/absence does not participate in this condition and must not appear in the contract wording。 +- **Adjacency**: after ToolCalls，finish the batch and append its complete ToolResultMessage first。Only after that message is + ready may a UserMessage be appended when the caller has an actual new-user-input need；the ordinary Agent loop appends no + synthetic UserMessage between ToolResultMessage and the next model call。 +- **Agent ownership**: supersede D-166's non-persistence detail。`tool_choice` is stable reusable Agent behavior and belongs + to the persisted Agent definition，not invocation policy。 +- **Storage choice**: add an explicit required `tool_choice` column rather than a generic extra-parameters/config bag。It is + an established、named cross-provider behavior with direct runtime effect，so hiding it in an open bag would reduce schema + clarity。MVP persists exact `auto` and can widen the typed contract when another mode has a concrete consumer。 +- **Execution**: AgentManager reads the persisted value and passes its canonical meaning to AIManager/dialect mapping。Do + not add a per-run override in MVP。 +- **Separate budget**: `max_model_calls` remains caller-owned invocation policy；making `tool_choice` Agent-owned does not + move runtime budgets into Agent definition。 +- **Prompt boundary**: `tool_choice=auto` permits no call；the rumination system prompt still owns the semantic instruction + to call `submit_graph` only for meaningful rumination。 +- **Confidence**: Sir explicitly corrected the message-state wording and promoted `tool_choice` into Agent definition，while + allowing implementation judgment between explicit and generic persistence shapes。 + +### D-168 — Agent tool_choice is nullable and null means unspecified + +- **Correction**: supersede D-167's required-column detail。Persist `agents.tool_choice` as an explicit nullable column；do + not assume every AI dialect/provider/model offering supports a tool-selection control。 +- **Null semantics**: `null` means the Agent declares no protocol-level tool-selection policy。The dialect adapter omits the + provider option and uses the provider/model's ordinary behavior；`null` must not be normalized to or reported as `auto`。 +- **Non-null semantics**: a non-null value is an exact Agent requirement。AIManager/dialect support must represent it；if + not，reject before the provider request rather than silently dropping or weakening it。 +- **Rumination MVP**: persist `null` for the rumination Agent。Its system prompt still asks the model to call `submit_graph` + only for meaningful rumination and otherwise end honestly，maximizing compatibility without claiming an unavailable wire + control。 +- **Shape**: keep the named column rather than introducing generic extras；nullability corrects capability variance without + obscuring a stable semantic field。 +- **Confidence**: Sir identified platform capability variance and explicitly proposed nullable persistence。 + +### D-169 — AgentManager.run materializes the definition prompt as the first SystemMessage + +- **Assembly owner**: AgentManager.run loads one Agent-definition snapshot and constructs the initial history。The caller + and AIManager do not separately assemble or override the Agent system prompt。 +- **Initial history**: materialize `SystemMessage(content=agent.system_prompt)` as the first Message，followed by the caller's + initial UserMessage。 +- **Authority**: the Agent definition remains the reusable system-prompt authority。The materialized SystemMessage is this + run's execution snapshot，not a second configuration authority。 +- **History semantics**: AgentRunResult's complete Messages include that SystemMessage。A future message-history backend may + preserve the exact prompt used by an old run，so later Agent-definition edits do not reinterpret old history。 +- **AI boundary**: AIManager receives canonical Messages and has no separate system-prompt parameter；dialect adapters own + provider role/input translation。 +- **Rumination boundary**: focal Block、direct relations、understood content and real graph IDs belong to the initial + UserMessage assembled by the rumination owner，not the reusable Agent definition or SystemMessage。 +- **Confidence**: Sir explicitly confirmed that run assembles the SystemMessage from Agent definition。 + +### D-170 — Agent creates an active in-memory-persisted Thread；each turn is one cancellable Task + +- **Agent definition**: supersede D-154/D-161's caller-selected model/budget details。Persist required `model` and required + positive `max_model_calls_per_turn` in Agent definition alongside name、system_prompt、tools、nullable tool_choice and + database timestamps。The per-turn model-call budget is reusable Agent behavior，not run input。 +- **Deep run entry**: `AgentManager.run(agent_id, input: UserMessage) -> Thread` is the caller surface。It loads one + `AgentDefinitionModel` rather than accepting model、tools、tool choice and budget as exploded parameters。 +- **System prompt boundary**: run materializes `SystemMessage(agent.system_prompt)` before entering Thread。Thread does not + snapshot system_prompt as another field；the resulting Message is already part of its initial history。 +- **Internal dependency**: Thread is an internal Agent-domain module and receives the complete AgentDefinitionModel。It may + depend on that model and internally snapshot model、tools、tool_choice and max_model_calls_per_turn without a flattened + constructor owned by external callers。 +- **Thread state**: the persistence unit consists of canonical Messages plus the snapshotted model、tools、nullable + tool_choice and max_model_calls_per_turn。Later Agent-definition edits do not change an existing Thread's behavior。 +- **Persistence seam**: use the exact term **thread persistence backend**。It is replaceable，but MVP implements only an + in-memory backend；do not add database Thread/Message persistence now。 +- **Initial turn**: run creates the Thread with the materialized SystemMessage，then starts the first turn with `input`； + `start_turn()` alone appends that UserMessage，so the initial input is not duplicated。Run returns the active Thread + immediately rather than awaiting first-turn completion。 +- **Turn execution**: the entire turn executes in one private coroutine。`start_turn(input)` schedules it as one + `asyncio.Task[TurnTermination]`，stores the runtime-only `current_turn` handle and returns that Task。A caller awaits the + Task or aborts via Task cancellation；the handle is not persisted and no Turn table/domain entity is introduced。 +- **Concurrency**: one Thread permits at most one active turn。Starting another before `current_turn.done()` fails + explicitly。 +- **Outcome correction**: supersede D-162's AgentRunResult。Thread is the run result/history handle；the Task owns per-turn + completion/limit outcome，and a cancelled Task represents abort。 +- **Completion closure**: this also closes D-161's remaining public-completion open edge。The Task returns typed + `completed` / `max_model_calls` termination，while cancellation represents abort；D-171 later closes Tool-batch commit + behavior under that same Turn owner。 +- **Rumination provisioning**: the withdrawn negative-ID/seed proposal is not part of the plan。How the rumination Agent + definition is provisioned remains a later implementation/configuration decision。 +- **Confidence**: Sir approved Agent-owned model/budget、Thread snapshots、replaceable in-memory-only thread persistence、the + coroutine/Task distinction and immediate-return run，and explicitly corrected terminology、definition dependency and + system-prompt snapshot semantics。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D171-D180.md b/tasks/knowledge-lifecycle-capabilities/decisions/D171-D180.md new file mode 100644 index 0000000..a55af44 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D171-D180.md @@ -0,0 +1,241 @@ +# Decisions D-171–D-180 + +> [Decision register](index.md) + +### D-171 — Turn owns a structured-concurrent Tool batch and commits only closed message pairs + +- **Turn ownership**: `start_turn()` already schedules the complete turn as one `asyncio.Task`。Do not wrap the whole Tool + batch in another child Task merely to await it；that layer would add no lifecycle、cancellation or observation value。 +- **Per-call concurrency**: when one AssistantMessage contains multiple ToolCalls，the Turn directly enters one structured + batch and schedules one child Task per call。The calls run concurrently and remain order-independent；there is no + sequential-execution compatibility promise。 +- **Failure isolation**: each child converts its ordinary validation/handler failure into its own ToolResult，so one + failure does not cancel siblings。Turn cancellation is different：it propagates to the batch and all unfinished child + Tasks；do not `shield()` Tool execution from abort。 +- **History authority**: Tool execution returns one complete `ToolResultMessage` and does not write Thread history。The + Turn runtime is the sole history writer。 +- **Closed commit**: after the whole result barrier，the thread persistence backend atomically appends the + AssistantMessage and its immediately following ToolResultMessage as one message batch。Abort before that commit leaves + neither half in history；already completed Tool side effects remain and are not rolled back。 +- **Defensive tail recovery**: `start_turn()` may repair exactly one recoverable persisted tail：an AssistantMessage with + ToolCalls and no following ToolResultMessage。Repair removes that entire trailing AssistantMessage before appending the + new UserMessage。Never erase only `tool_calls`，which would falsify the model-authored Message；never rewrite an older + ambiguous history segment。 +- **Future exception**: an independently observable/background Tool batch would need concrete product pressure and its own + lifecycle contract。It is not implied by ordinary async Tool execution。 +- **Confidence**: Sir approved single-writer closed-batch history、abort propagation and whole-tail recovery，then + identified that batch-level `create_task()` was redundant because concurrency belongs to the individual ToolCalls。 + +### D-172 — Resolver draft schemas are discovered on demand and id_start identifies the star Block + +- **Rooted Resolver semantics**: a Resolver draft describes one resolver-owned subject plus any directly or recursively + materialized supporting graph。`draft_graph` accepts a caller-selected negative `id_start=-1` by default，and the new + Block at exact `id_start` is always that subject/star Block。Other draft-local IDs continue below that start，so the + caller can compose non-colliding drafts without hidden allocator state。 +- **No duplicate entry fact**: do not return `entry_id`。It is exactly derivable from the call's `id_start` and duplicating + it would create a second authority that could disagree with the graph。 +- **Progressive schema discovery**: the initial rumination context exposes only available exact Resolver IDs and compact + descriptions。`get_draft_graph_schema(resolvers=[...])` accepts a set/list constrained to those exact IDs and returns + the selected descriptions plus resolver-owned draft-input JSON Schemas。The detailed union is not injected into every + model request。 +- **Domain-owned construction**: every draft-capable Resolver owns its description、Pydantic draft-input model and graph + construction。The generic `draft_graph` Tool dispatches by exact Resolver ID and validates `input` through that + Resolver's model；it does not expose persisted `block.content` schemas or a closed global relation-content enum。 +- **Effect boundary**: schema discovery and draft construction do not mutate info-base or storage。They need not promise + mathematical purity or prohibit resolver-owned external reads/computation。`submit_graph` remains the only rumination + Tool that writes the info-base。 +- **Submit result**: successful submission returns only new Block identity mappings，for example + `{"blocks":[{"local_id":-1,"id":731}]}`。It does not return full persisted rows or repeat relation results that the + current Agent journey does not use。 +- **Tool set correction**: supersede the earlier exactly-one-Tool claim。MVP rumination binds the specialized + `get_draft_graph_schema` discovery Tool、the non-persisting `draft_graph` construction Tool and the sole mutating + `submit_graph` Tool；it still exposes no graph-reading/navigation Tool。 +- **Open representation seam**: whether Resolver returns a retained rooted `StarsGraphForm` that the Agent Tool converts， + whether `submit_graph` accepts one/many rooted forms，or whether Resolver directly returns flat GraphForm is not yet + confirmed。Do not let the `id_start` invariant silently decide this type boundary。 +- **Confidence**: Sir selected on-demand schemas、caller-supplied `id_start` and removal of redundant `entry_id`，confirmed + that `id_start` is the star Block ID，accepted the submit result mapping and selected five reusable Agent Tool patterns + for later durable promotion。Sir explicitly reopened the rooted-versus-flat representation seam before it was closed。 + +### D-173 — Resolver drafting is rooted behavior over the sole flat GraphForm command + +- **Exact boundary**: `Resolver.draft_graph(input, id_start=-1) -> GraphForm`。The returned GraphForm's Block at exact + `id_start` is the Resolver's subject/star Block；the rooted guarantee is behavior，not a second serialized graph type。 +- **One command authority**: `submit_graph(graph: GraphForm)` accepts only GraphForm。Do not retain `StarsGraphForm`，accept + `star_graph[]` at the write boundary or make the Agent translate between two graph command models。 +- **Extension continuity**: preserve the existing extension pattern in which each Resolver owns native/canonical input → + one subject Block plus supporting Blocks/Relations。The hard cut changes representation and call sites，not that + ownership or rooted authoring model。All InKCre-owned extension/source producers migrate coherently from recursive + `SubGraphForm` construction；no compatibility wrapper is required。 +- **Authoring depth**: if flat ID allocation/merging makes several migrated producers mechanically noisy，a code-internal + builder may hide those mechanics and return GraphForm。It is not another Form、wire shape or InfoBaseManager input，and + implementation preflight must justify its exact API from repeated producer code。 +- **Composition**: independent Resolver drafts may be submitted independently。A caller that needs cross-draft Relations + observes the draft results，uses non-colliding signed IDs and composes them into one GraphForm；that is semantic graph + composition rather than format conversion。 +- **Supersession**: this closes D-172's representation seam and supersedes D-151/D-152's retained `StarsGraphForm` + decision。D-151/D-152 remain historical evidence for the still-preserved rooted Resolver behavior。 +- **Confidence**: Sir approved the “semantically rooted，structurally one GraphForm” contract and immediately required + confirmation that existing extension Resolver star-graph production remains a supported pattern。 + +### D-174 — StarsGraphForm remains the Resolver/extension authoring form；Agent draft output is GraphForm + +- **Correction**: D-173 incorrectly expanded the approved Agent-facing GraphForm boundary into deletion of + `StarsGraphForm` across all producers。Supersede D-173's “do not retain StarsGraphForm” and mandatory extension/source + flat-form migration claims；D-151/D-152's retained recursive authoring representation remains in force。 +- **Resolver/extension boundary**: an extension/source Resolver may continue to construct one subject Block plus recursive + in/out arcs as `StarsGraphForm`。Its migration removes persisted `BlockModel` / `RelationModel` fields in favor of + producer `BlockForm` / `RelationForm`，but preserves the star-centered authoring pattern and recursive representation。 +- **Agent boundary**: the Agent Tool named `draft_graph` still returns flat GraphForm。Its handler invokes the selected + Resolver's rooted construction and normalizes StarsGraphForm before exposing the ToolResult；the LLM never converts the + two representations itself。 +- **ID invariant**: normalization assigns exact `id_start` to the root/star Block and remaining negative IDs below it。 + `entry_id` remains redundant because the Tool call and resulting GraphForm share this invariant。 +- **Write boundary**: the Agent's `submit_graph` Tool continues to accept only GraphForm。Retaining StarsGraphForm for + producer ergonomics does not require `submit_graph(star_graph[])` or a second Agent-visible command。 +- **Remaining seam**: implementation design must place the reusable StarsGraphForm → GraphForm normalizer so extension + collection and Agent drafting share one conversion authority rather than duplicate traversal/ID allocation。Whether the + convenience entry lives on the form、InfoBaseManager or a smaller graph-command collaborator remains closure/preflight + work。 +- **Confidence**: Sir noticed that earlier decisions retained StarGraph and challenged the accidental global deletion。 + The correction follows the already approved distinction between existing extension Resolver authoring and the + Agent-facing draft result。 + +### D-175 — InfoBaseManager owns graph normalization；Agent draft_graph is a thin wrapper over internal creation + +> **Superseded by D-176**：the normalizer ownership remains，but the invented `InfoBaseManager.create_graph()` facade and +> InfoBaseManager → ResolverManager dependency do not。 + +- **Normalizer owner**: the single reusable `StarsGraphForm -> GraphForm` normalizer belongs to InfoBaseManager。Do not + place traversal/signed-ID allocation in AgentManager、individual Agent Tool handlers、ResolverManager or each extension。 +- **Internal facade**: internal managers use `InfoBaseManager.create_graph(...) -> GraphForm` as the non-persisting graph- + creation facade。It coordinates exact Resolver selection/input validation through ResolverManager，invokes the selected + Resolver's rooted `create_graph(...) -> StarsGraphForm` implementation and normalizes the result with caller-supplied + `id_start`。 +- **Agent adapter**: Agent-visible `draft_graph(resolver, input, id_start=-1)` is a separately registered Tool contract and + a thin wrapper around `InfoBaseManager.create_graph(...)`。It owns only Agent Tool input/output adaptation；it does not + copy Resolver dispatch、schema validation、star construction or normalization logic。 +- **Effect naming**: both internal `create_graph` and Agent `draft_graph` construct and return an unpersisted GraphForm。 + Only `submit_graph(GraphForm)` mutates the info-base。The different names distinguish an ordinary internal capability + from a model-facing proposal Tool，not different graph semantics。 +- **Dependency topology**: InfoBaseManager may depend on ResolverManager for exact dispatch/validation；ResolverManager and + concrete Resolvers do not depend on AgentManager。AgentManager invokes the registered wrapper without learning graph + construction internals。 +- **Confidence**: Sir assigned the shared normalizer to InfoBaseManager and explicitly distinguished the Agent Tool from + the internal graph-creation method while defining the former as a wrapper over the latter。 + +### D-176 — Resolver.create_graph owns semantic construction；Agent draft_graph wraps it and InfoBase only normalizes + +> **Partially corrected by D-177**：Resolver/InfoBase ownership remains，but Agent runtime—not ResolverManager or the Tool +> handler—owns Tool input validation；`Resolver.create_graph()` receives ordinary `input`。 + +- **Correction**: supersede D-175's invented `InfoBaseManager.create_graph()` facade and the resulting + InfoBaseManager → ResolverManager/Resolver construction dependency。InfoBaseManager does not decide how resolver-native + input becomes information；that is Resolver authority。 +- **Internal semantic method**: concrete `Resolver.create_graph(validated_input) -> StarsGraphForm` remains the ordinary + internal graph-construction method used by extension/source and other Managers。It owns resolver-specific Block content、 + supporting Blocks and Relation grammar。 +- **Agent wrapper**: Agent-visible `draft_graph(resolver, input, id_start=-1) -> GraphForm` is a thin wrapper around the + selected Resolver's `create_graph()`。Its adapter work is limited to ResolverManager exact-ID dispatch/input validation， + calling the same semantic method and projecting its StarsGraphForm result through InfoBaseManager normalization。 +- **InfoBase boundary**: InfoBaseManager owns reusable `StarsGraphForm -> GraphForm` normalization（including deterministic + signed-ID allocation from `id_start`）and `submit_graph(GraphForm)` persistence。It does not import Resolver semantics、 + choose a Resolver or expose a graph factory that hides Resolver ownership。 +- **Internal callers**: non-Agent Managers may call the concrete Resolver `create_graph()` directly and use the same + InfoBaseManager normalization/write path as needed。AgentManager remains only an execution/Tool boundary and owns none of + the three domain operations。 +- **Meaning of thin**: the Tool wrapper adds boundary validation、dispatch and result-shape adaptation，but no alternative + graph-construction policy。Calling the shared normalizer does not make it a wrapper over InfoBaseManager；its semantic + operation remains Resolver.create_graph。 +- **Confidence**: Sir identified the inverted ownership immediately and restated that Agent `draft_graph` wraps the + Resolver creation method，while the previously assigned shared normalizer alone belongs to InfoBaseManager。 + +### D-177 — Agent runtime validates draft_graph arguments；Resolver.create_graph receives ordinary input + +- **Correction**: supersede D-176's `Resolver.create_graph(validated_input)` wording and its assignment of draft-input + validation to ResolverManager。The Resolver method's domain contract does not record where or how its input was + validated。 +- **Agent boundary**: Agent runtime validates the complete `draft_graph` ToolCall before invoking its handler，using the + bound Tool contract and the selected Resolver's code-owned Pydantic input model。Pydantic ValidationError remains the + already approved per-ToolCall error result；the handler does not repeat validation。 +- **Handler input**: the registered `draft_graph` function receives validated Tool arguments。It uses the exact Resolver ID + to obtain the Resolver through ResolverManager，passes the nested ordinary `input` value to + `Resolver.create_graph(input)`，then asks InfoBaseManager to normalize the returned StarsGraphForm。 +- **Resolver boundary**: `Resolver.create_graph(input) -> StarsGraphForm` owns semantic graph construction only。It does not + receive a `validated_input` wrapper、inspect Agent runtime state or know whether its caller is an Agent Tool or another + internal Manager。 +- **ResolverManager boundary**: ResolverManager supplies exact-ID lookup and the Resolver-owned draft-input contract used + when AgentManager binds/validates the Tool。It does not become a second runtime validation layer inside the handler。 +- **Internal callers**: another Manager passes the resolver-native typed input expected by the concrete Resolver and relies + on ordinary Python/type/domain contracts；it is not forced through Agent Tool validation merely to call create_graph。 +- **Confidence**: Sir corrected both the input name and validation owner，reasserting that Agent runtime validation ends at + the Tool handler boundary and must not leak into Resolver.create_graph。 + +### D-178 — A runtime boundary turns raw input into ordinary typed input once + +- **Boundary rule**: the framework/runtime boundary that receives raw external input owns deserialization and validation + into the command's typed input。For an Agent ToolCall，that boundary is Agent runtime；for another transport，its own + framework adapter owns the equivalent step。 +- **Ordinary internal input**: functions called after that conversion receive ordinary typed/domain input。Do not add + `validated_input` names、`Validated[T]` wrappers or boolean state merely to restate that the boundary already validated + the payload。 +- **No overreach**: this is not a claim that only one invariant exists in the whole call path。A Pydantic model still owns + its intrinsic structural invariants；a later module may enforce a genuinely different invariant that it owns；PostgreSQL + still owns referential integrity。The rule removes duplicate validation/state，not distinct authorities。 +- **Current application**: Agent runtime validates the complete `draft_graph` / `submit_graph` Tool arguments；their + handlers receive typed arguments；`Resolver.create_graph(input)` receives ordinary resolver-native input；GraphForm and + PostgreSQL retain their already approved structural/FK responsibilities。 +- **Common-pattern pressure**: record this as U-031 for later durable promotion，with `submit_graph` and + `draft_graph -> Resolver.create_graph` as current evidence。 +- **Confidence**: after requiring the abstract wording to be replaced with concrete callers and responsibilities，Sir + approved this boundary → ordinary typed-input formulation。 + +### D-179 — Producer-form grammar was already closed；do not reopen it as a design question + +- **Closure correction**: D-147、D-152、D-155 and D-156 already close the producer-command grammar。The unit packet + incorrectly kept “exact producer Form fields” in its discussion queue merely because the approved rules had not been + rendered as one field/schema table。 +- **Base creation Forms**: `BlockForm` / `RelationForm` contain producer-owned values needed to create their respective + graph rows and omit database-generated identity、timestamps and other database-managed state。A surrounding graph + representation supplies connection/reference mechanics rather than leaking persisted Models into either base Form。 +- **Batch-reference exception**: when one command creates Blocks that Relations in the same command must reference，flat + GraphForm adds one command-local non-zero signed Block-ID namespace。Negative IDs declare Blocks to create；positive IDs + reference already persisted Blocks；Relations connect through those same IDs；zero is invalid。 +- **Representation continuity**: `StarsGraphForm` keeps recursive Resolver/extension authoring over BlockForm/RelationForm； + flat GraphForm keeps arbitrary connected composition and is the sole Agent submit command。InfoBaseManager owns their + approved normalization/write mechanics。 +- **Implementation detail**: exact Python collection/container projection should now be written in the implementation plan + from these contracts and repository conventions。It is not another product/architecture choice unless preflight exposes + a materially different behavior or authority boundary。 +- **Common-pattern pressure**: promote the distinction between ordinary id-free creation Forms and a batch command's + command-local negative identities as U-032。 +- **Confidence**: Sir identified that both the Form/database-managed-field rule and negative-ID mutual-reference rule had + already been confirmed and rejected reopening them as the next discussion topic。 + +### D-180 — Organization config selects rumination Agent；transport is only a projection + +- **Coordination correction**: do not frame Agent provisioning around an HTTP API。AgentDefinition and deployment config + are shared-database facts among equal Peers and may be edited through that authority directly。An HTTP CRUD surface is an + optional projection，not the coordination topology；Peer capability delegation remains the explicit request-response + exception in this unit。 +- **No approach Manager**: rumination currently remains one function on the enclosing domain owner， + `OrganizationManager.ruminate(...)`。Its present complexity does not justify a `RuminationManager` class or independent + approach lifecycle。 +- **Selection persistence**: the deployment-scoped `configs` row at key `core.organization.rumination` uses exact schema + `core.organization.rumination.config.v1` and owner-defined value `{"agent": int}`。The withdrawn Agent seed/default + proposal remains withdrawn；the invocation accepts no per-call Agent override。 +- **Schema ownership**: the organization-owned Pydantic model owns the value shape。ConfigContract supplies generic + model-driven mechanics；DeploymentConfigManager resolves the exact registered schema contract and persists the shared + row。Do not attribute the business schema to DeploymentConfigManager。 +- **Reference boundary**: `agent` is `int`，not an unjustified positive-integer refinement。The config path does not prove + existence；`OrganizationManager.ruminate()` resolves the Agent reference when used and distinguishes missing config from + a dangling reference。Agent deletion has no reverse restriction。 +- **Exact invocation**: restore and retain the approved signature + `AgentManager.run(agent_id, initial_message)`。OrganizationManager constructs the focal/direct-relation initial message + and passes the selected ID without renaming or exploding this contract。 +- **Discussion-guideline pressure**: the unit control surface now records authority-before-transport、earned class/module + boundaries、approved-signature continuity、owner-schema/generic-mechanics separation and proportional constraints as + steerable taste rather than universal rules。 +- **Confidence**: Sir corrected the HTTP-first frame、rejected a low-value RuminationManager、restored the exact run + signature、identified schema ownership leakage and rejected an unsupported positive-ID constraint，while accepting the + deployment-config selection topology。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D181-D190.md b/tasks/knowledge-lifecycle-capabilities/decisions/D181-D190.md new file mode 100644 index 0000000..438eb7b --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D181-D190.md @@ -0,0 +1,200 @@ +# Decisions D-181–D-190 + +> [Decision register index](index.md) + +### D-181 — Organization awaits rumination Turn and exposes shallow completion + +- **Public shape**: `OrganizationManager.ruminate(block_id)` is async and returns `None` after the active Agent Turn + completes；it does not expose Thread as the organization result。 +- **Agent contract continuity**: OrganizationManager calls the already approved + `AgentManager.run(agent_id, initial_message)`，receives its active Thread and awaits that Thread's current Turn Task。 + AgentManager itself still returns Thread；the organization layer deliberately hides that lower-level execution handle。 +- **Normal completion**: a naturally completed Turn returns `None` whether it submitted a graph or chose no useful write。 + Inability to understand the focal Block also returns `None` without starting an Agent run，preserving the approved shallow + best-effort semantics。 +- **Incomplete execution**: `max_model_calls` maps to one organization-level failure because the consideration did not + naturally finish。Completed Tool effects remain；do not add retry、rollback or compensation。 +- **Cancellation**: cancelling the caller's rumination coroutine propagates to its awaited Turn Task。Completed effects + remain，and no organization job、run entity、abort endpoint or Thread projection is introduced。 +- **Classification correction**: “do not make each approach a Manager” is not promoted as a common discussion pattern。 + Avoiding `RuminationManager` is a unit-local anti-pattern based on rumination's present small surface and the enclosing + `OrganizationManager` topology。 +- **Confidence**: Sir accepted the proposed completion contract and corrected the attempted generalization of a local + module-boundary judgment。 + +### D-182 — Repeated rumination is additive；direct context is not freshness authority + +- **Independent attempt**: each `OrganizationManager.ruminate()` invocation uses the latest bounded context and performs a + new best-effort consideration。Do not add a durable run、last-rumination timestamp、content fingerprint、idempotency key or + per-Block execution lock in the MVP。 +- **Graph effect**: negative GraphForm IDs request new Blocks and only explicit positive IDs reuse existing Blocks。Repeated、 + concurrent or uncertain-retry execution may therefore create duplicate Blocks/Relations；no exactly-once or semantic + deduplication guarantee is made。 +- **Reconciliation boundary**: accept duplicate graph facts as an MVP side effect。If they later harm use，organization + merge/linking/reconciliation is the proper product pressure；rumination does not silently acquire semantic-equivalence + detection now。 +- **Snapshot limitation**: direct relations may indicate that an earlier rumination probably produced an interpretation， + but they cannot prove whether that output needs updating。The snapshot has no complete dependency/freshness account for + focal content、Resolver behavior、Agent definition、neighbor facts or the deeper derived graph。 +- **No skip inference**: OrganizationManager does not skip execution or report prior output current merely because a direct + derived relation exists。The Agent may choose no submit from the context it sees，but that is a best-effort semantic no-op， + not freshness/update correctness。 +- **Future correction seam**: explicit reevaluation/freshness semantics may be added after real use exposes the missing + update behavior；the MVP does not pretend its limited context has already solved it。 +- **Confidence**: Sir accepted independent additive execution and its duplicate risk，while correcting the claim that direct + context could decide whether prior rumination requires an update。 + +### D-183 — Acceptance corpus starts from deterministic sources and real graph producers + +- **Authority**: use repository-owned deterministic source data，then exercise real product producers/runtime into a + disposable PostgreSQL info-base。Do not make hand-inserted target Blocks/Relations the primary corpus authority。 +- **Source journeys**: Memos enters through real API request/response behavior；RSS/Atom uses real protocol documents served + by a controllable local protocol double；a compound document passes through its real Resolver and rumination path。 +- **Runtime depth**: acceptance materializes real graph rows、EmbeddingRecords and retrieval results in PostgreSQL rather + than stopping at schema/helper fixtures。 +- **Expected evidence**: source artifacts may be committed，but expected results name entity identity、Relations and rank/ + relevance judgments rather than freezing complete internal graph or embedding payloads。 +- **Production role**: the public-demo production database may supplement exploratory evaluation but cannot own automated + pass/fail because its contents change independently。 +- **Audit finding**: this Acceptance discussion exposed that organization triggering was never closed。Execution entry、 + Agent selection、completion and repetition do not answer whether explicit、event-driven or periodic invocation exists； + the packet therefore reopens that upstream edge before continuing Acceptance。 +- **Confidence**: Sir accepted the proposed corpus authority and explicitly identified the missing periodic-trigger + discussion。 + +### D-184 — MVP rumination is explicit single-Block execution only + +- **Trigger**: rumination runs only after an explicit request naming one focal Block and enters through + `OrganizationManager.ruminate(block_id)`。 +- **Automatic exclusions**: do not add collection-completion/new-Block hooks、periodic scans、batch candidate selection or an + organization job in the MVP。Collection remains successful independently of organization AI availability、cost or output。 +- **Why now**: the unit has no correct candidate-eligibility、reevaluation/freshness or budget policy。Direct-relation + context cannot decide whether prior rumination needs updating，and periodic execution would amplify accepted duplicate + effects without adding understanding。 +- **Future seam**: automatic organization requires separate candidate、freshness、cost/concurrency and Peer-safe claiming/ + execution design。It may call the same explicit rumination function after those policies select a Block。 +- **Confidence**: Sir explicitly accepted on-demand single-Block invocation and exclusion of collection-triggered and + periodic automatic execution。 + +### D-185 — Rumination is the second Peer capability；client-web triggers it from Block details + +- **Capability/HTTP**: add exact `core.organization.rumination.v1` with fixed + `POST /organization/ruminate`、JSON `{"block": int}` and `204 No Content` success。The fixed URL fits the approved absolute- + inbound descriptor without inventing path templates；the action does not create a Rumination/run resource。 +- **Delegation**: client-web's organization-domain facade delegates through PeerManager；the provider inbound calls a non- + delegating OrganizationManager local path。Do not add a generic capability invoke endpoint or expose provider selection + to UI/domain callers。 +- **Mutating failover**: only pre-dispatch or exact Peer Protocol `not-executed` permits another provider。Normal `204`、 + execution failures and outcome-unknown transport failures do not replay；the latter may otherwise duplicate graph facts。 +- **UI**: place one explicit Ruminate action in the existing selected-Block `BlockDetailsPanel`。Show pending/completed/error， + reload the graph after success and tell the user to refresh/inspect after outcome unknown。Do not add confirmation、 + progress、automatic retry、cancel API or new-entity highlighting in the MVP。 +- **Legacy hard cut**: delete `Client(rest_api_url).request()` and its convenience methods when Peer delegation lands，and + delete the already rejected global `rest_api_url` projection。Do not retain an endpoint-selecting escape path for future + business capabilities；direct database Active Records remain valid for shared facts。 +- **Open address source**: D-130 approved absolute inbound URLs but did not close how a runtime learns its public address。 + Current `CLIENT_BASE_URL` comes from `CORE_PUBLIC_URL` and feeds legacy `rest_api_url`；the old path must be renamed/ + remodeled rather than silently deleted with the still-needed deployment fact。 +- **Confidence**: Sir approved the exact HTTP/UI/second-capability proposal，required removal of legacy Client request + routing and identified the missing inbound-address acquisition decision。 + +### D-186 — Peer config owns HTTP public base；capability URLs are derived advertisement + +- **Authority correction**: “deployment supplies peer-local runtime config” does not require a new environment-only + setting。Persist the provider's HTTP public base in its existing owner-specific `peers.config`，initially field + `http_public_base_url` for the core-py config model。 +- **Operational path**: deployment may edit the Peer row directly；the client-web Client administration view hard-cuts to + a Peer view capable of editing the same config under `config_schema`。This does not use deployment-scoped `configs` or + DeploymentConfigManager。 +- **Projection**: provider runtime reads its config and combines the base with registered fixed inbound paths when + publishing/refreshing its complete capability snapshot。Config is authority；each advertised absolute URL is derived + routable state，not another authored base-URL authority。 +- **Legacy removal**: delete `settings.client_base_url / CLIENT_BASE_URL`、the Compose `CORE_PUBLIC_URL` projection、 + `clients.rest_api_url` and legacy Client request routing。Do not add a parallel public-address environment setting or + table。 +- **Address semantics**: the configured base is absolute HTTP(S)，may include a deployment path prefix and has no query、 + fragment or credentials。Do not infer it from bind host/port or request headers。Absent config permits local execution + but omits the HTTP inbound advertisement。 +- **Planning seam**: exact snapshot refresh/change-detection cadence remains implementation-plan work；it must consume the + one Peer config authority rather than introduce another address source。 +- **Confidence**: Sir rejected the environment/config elaboration，pointed to direct database editing and the existing + client administration config surface，and accepted the remaining composition/unset semantics。 + +### D-187 — Acceptance uses entity judgments and test-owned symbolic references + +- **Judgment unit**: each natural-language query judges real Block/Relation identities as `primary`、`relevant` or an + explicitly selected `distractor`。A Relation may be the primary answer；unjudged entities are not automatically declared + irrelevant or distractors。 +- **Ranking evidence**: expected results constrain entity rank/coverage，not one provider's exact floating score。The next + Acceptance step will set the minimum rank/quality threshold separately rather than smuggle it into the judgment labels。 +- **Rumination evidence**: before rumination，a coarse source may remain relevant；after rumination，a newly materialized + specific Block/Relation may become the primary entity。The expected result still names graph entities，not transient + chunks or generated answers。 +- **Corpus quality**: deterministic source artifacts should be authentic、substantive and approachable material in the + software/AI/knowledge-systems problem space，valuable enough that the intended user might genuinely save them。Toy prose + is not the primary semantic-quality authority。 +- **Reference isolation**: human-readable symbolic names such as `block:memo.marginal_benefit` exist only in the Acceptance + corpus manifest/harness。After real producers ingest the sources，the harness resolves those names to actual database IDs + using observable source/graph facts。Never add test aliases、fixture IDs or Acceptance-only selectors to production + tables、Pydantic/domain models、producer payloads、runtime managers or public capability contracts。 +- **No test-shaped product path**: the harness must exercise public/real producer and retrieval boundaries as they exist for + product use。If an entity cannot be identified without changing production semantics solely for the test，the corpus or + harness is wrong；the implementation is not adapted to the fixture。 +- **Confidence**: Sir accepted the entity-judgment model，required professionally relevant real material and explicitly + prohibited Acceptance references from shaping or polluting the implementation。 + +### D-188 — Retrieval quality is gated per scenario，not by an aggregate score + +- **Primary rank**: every accepted query must place at least one entity judged `primary` within the single global top three + Block/Relation results。 +- **Hard distractors**: the highest-ranked `primary` must outrank every explicitly selected `distractor` for that query。 + Unjudged entities remain outside this assertion。 +- **Relevant entities**: `relevant` means useful supporting evidence，not mandatory recall；the MVP does not require every + relevant entity to appear in the bounded result。 +- **No aggregate escape**: MRR、NDCG、a corpus average or exact provider floating scores do not own pass/fail in the MVP。 + Aggregate metrics may be reported diagnostically，but they cannot hide a failed accepted journey。 +- **Rumination benefit**: for at least one coarse-source journey，the newly materialized specific semantic entity must enter + the top three and outrank the original coarse document after rumination。Its absence before graph materialization is a + structural fact，not a fabricated zero-score baseline。 +- **Confidence**: Sir accepted the proposed per-query rank、distractor and rumination-improvement thresholds。 + +### D-189 — Retrieval never hides candidate maintenance inside query execution + +- **Operation split**: `maintain` / `rebuild` generate or replace candidate EmbeddingRecords；`retrieve` embeds only its + query input and compares already-fresh candidate records。Retrieval does not scan、repair or persist missing/stale + candidate embeddings as an implicit side effect。 +- **Why the boundary differs from lazy resolution**: a Resolver may lazily resolve one explicitly requested Block，whereas + candidate maintenance ranges over an otherwise unbounded graph set with provider cost and partial availability。It must + remain an explicit/bounded maintenance operation rather than make query latency and failure meaning indeterminate。 +- **Freshness journey**: Acceptance ingests through real producers，maintains，retrieves，then updates Block + `content`/`storage`/`resolver` and Relation endpoint/content dependencies through real managers。Before another maintain， + the old records may remain stored but must be excluded as stale；after maintain，replacement records and query semantics + must reflect the update。 +- **Availability journey**: an entity whose projection is unavailable remains missing/stale without blocking later + available candidates in the same maintenance scan。Provider/adapter/model failure is reported as `failed` and cannot + write an invalid record；retrieval continues to use only other fresh records。 +- **Scheduling evidence**: automatic maintenance wiring/configuration calls the same manager operation，but Acceptance does + not wait for a wall-clock scheduler to prove the maintenance algorithm。 +- **Confidence**: Sir accepted the explicit maintenance/query split and the proposed real-update freshness Acceptance。 + +### D-190 — Peer Acceptance proves domain paths and conservative failover without a real-proxy smoke test + +- **Local path**: when the caller Peer has a local capability implementation，its domain facade executes locally without + reading an advertisement or using Peer HTTP outbound。 +- **Delegated paths**: a Peer without local support delegates exact `core.semantic_retrieval.v1` or + `core.organization.rumination.v1` through PeerManager、the real HTTP protocol/JWT/codec boundary and a provider inbound + that enters only the non-delegating local implementation。Acceptance must detect any delegation loop。 +- **Success semantics**: retrieval returns the ordinary ranked typed result；rumination returns `204 No Content` and the + client-web journey reloads its graph。The two paths need domain-equivalent outcomes，not identical internal call traces or + provider floating scores。 +- **Conservative failover**: expired/malformed/ineligible candidates and provable pre-dispatch failures may be skipped；the + exact `InkCre-Peer-Execution: not-executed` response may select another provider。An ordinary domain/HTTP response or an + outcome-unknown post-dispatch timeout/reset stops without replay for both capabilities。MVP does not add a retrieval- + specific replay policy merely because read execution would usually be lower harm。 +- **Evidence shape**: use process-level black-box evidence for local bypass，a two-Peer real HTTP integration for delegation + and a two-provider integration for lease/pre-dispatch/`not-executed`/outcome-unknown branches。 +- **Proxy correction**: do not require a real reverse-proxy smoke test。Existing HTTP/RFC/MDN evidence is sufficient for + the ordinary header and browser exposure mechanics；automated coverage may assert the protocol header and CORS exposure + at the application boundary without constructing deployment infrastructure solely to repeat that proof。 +- **Confidence**: Sir accepted the Peer runtime matrix and explicitly removed the proposed real reverse-proxy smoke test + because the existing standards evidence is already strong enough。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D191-D200.md b/tasks/knowledge-lifecycle-capabilities/decisions/D191-D200.md new file mode 100644 index 0000000..2abbf4d --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D191-D200.md @@ -0,0 +1,174 @@ +# Decisions D-191–D-200 + +> [Decision register index](index.md) + +### D-191 — Peer delegation supports an exact target；Extension management is its first consumer + +- **Two routing modes**: PeerManager exposes capability-routed delegation to any eligible provider and exact-target + delegation to one named Peer。The target constraint is a generic routing input，not Extension knowledge inside the Peer + domain。 +- **Target semantics**: exact-target delegation verifies that the selected Peer is live、advertises the exact capability and + has a caller-supported inbound protocol。It never substitutes another Peer，because another runtime cannot satisfy a + command whose business object is the selected Peer itself。 +- **Extension pressure**: deleting legacy `Client.request()` otherwise breaks client-web's existing remote Extension config、 + enable and disable operations。Migrate those consumers to exact `core.extension.management.v1` and a target-Peer domain + facade rather than preserving a generic endpoint-selected escape path。 +- **Provider execution**: the fixed Extension-management inbound enters the target Peer's non-delegating local + ExtensionManager，preserving config validation and hot lifecycle effects。PeerManager treats its payload as opaque and + does not learn Extension actions or schemas。 +- **Hard-cut consequence**: remove `Client.request()`、`rest_api_url` and their convenience paths after all known consumers + move。Do not silently regress remote Extension management，invent generic `/capabilities/{id}/invoke`，or add desired-state + polling/reconciliation merely to avoid modeling the actual synchronous command。 +- **Scope judgment**: this third exact capability is implementation pressure caused by the already approved Client→Peer + hard cut，not a new product unit or permission to convert every old HTTP route into a Peer capability。 +- **Confidence**: Sir accepted the exact Extension-management capability direction and explicitly judged specified-Peer + delegation to be a valuable general Peer capability。 + +### D-192 — One delegate entry owns optional exact-Peer routing + +- **Interface correction**: do not expose separate `delegate()` and `delegate_to()` methods。Use one + `PeerManager.delegate(capability, payload, *, route_to_peer: PeerRef | None = None)` entry。 +- **Null semantics**: `route_to_peer=None` uses the approved randomized eligible-provider sequence and bounded failover。 + A non-null value restricts the candidate set to exactly that Peer，so no alternate Peer can be substituted。 +- **Boundary**: `route_to_peer` is caller-local routing policy。It is not serialized into the capability payload、HTTP + request or advertisement，and PeerManager still treats capability semantics as opaque。 +- **Identity correction**: the type is `PeerRef` over the already approved UUID `peers.id`，not legacy `ClientRef` and not + an integer。User-facing product language may still say client，but technical/domain/database names use Peer。 +- **Extension application**: `core.extension.management.v1` uses the same delegate entry with `route_to_peer` set to the + selected runtime。Its discriminated enable/disable/config command remains one domain capability rather than three + transport-specific calls。 +- **Confidence**: Sir proposed the simpler optional routing parameter and confirmed that `ClientRef(int)` was stale + terminology rather than a request to reopen Peer identity。 + +### D-193 — An unreleased resolver contract may be corrected in place before v1 becomes durable compatibility truth + +- **Release boundary**: an exact resolver version protects persisted data and compatibility commitments that must continue + to be interpreted；it does not preserve every intermediate implementation written during local development。 +- **RSS consequence**: `extensions.rss.feed_item.v1` has not reached staging、preview or another retained deployment。 + Correct its `get_text()` in place to the approved title + summary + full-text/authored-content projection；do not create + v2、a row migration or a compatibility decoder solely for disposable local state。 +- **Local state**: reset the local development database during execution rather than adding production-shaped migration + complexity for data that has no retention requirement。This authorization does not imply resetting staging、preview、 + production or any other shared database。 +- **Future boundary**: once an exact resolver contract has retained deployment data or an external compatibility + commitment，an incompatible projection change advances its resolver contract version unless that deployment is also + explicitly declared disposable。 +- **Confidence**: Sir stated that RSS v1 has not been pushed even to staging/preview and explicitly allowed the local + database to be reset；the previous v2 proposal had mistaken local development state for released durable truth。 + +### D-194 — Remaining in-scope extension Resolver IDs hard-cut to exact v1 identities + +- **Exact IDs**: replace the remaining legacy registrations with `extensions.mail.email.v1`、 + `extensions.mail.newsletter.v1`、`extensions.mail.email_address.v1`、`extensions.github.repo.v1`、 + `extensions.github.user.v1`、`extensions.telegram.message.v1` and `extensions.learn_english.lexical.v1`。 +- **Contract boundary**: these names become the first retained exact contracts；do not mint v2 merely because an + unversioned local implementation existed first。Twitter's already selected `extensions.twitter.tweet.v1` remains。 +- **Compatibility**: do not retain aliases or legacy decoders。Disposable local/shared data is reset under D-195 rather + than making unsupported legacy Resolver strings part of the new compatibility surface。 +- **Confidence**: Sir accepted the exact hard cut after the full concrete ID list was presented。 + +### D-195 — Shared databases may be rebuilt cleanly；retain the reviewed Alembic chain + +- **Data authority**: current local、production and preview data has no retention requirement for this unit。Sir explicitly + authorized resetting production and staging/preview as well as local state，with an optional small dump first。 +- **Selected baseline strategy**: rebuild application schemas from empty state through the reviewed append-only Alembic + chain plus this unit's new revisions；do not squash the nine existing revisions into one new monolithic baseline。Those + revisions own already verified PostgREST functions、ACLs、storage protocol and trigger behavior，and their size does not + repay a second migration-integrity hard cut。 +- **Safety sequence**: before production reset，create a logical dump outside the repository on WorkSSD、record its digest + and create/verify a Neon recovery branch。Reset only the exact canonical production application schemas/lineage，then + run the normal migration/database-contract initialization path。Do not mutate archived staging lineage。 +- **Preview topology**: there is no active staging deployment。Bring the already data-free preview baseline to the same + repository head and sanitize it through the guarded workflow；future preview branches inherit that clean state。 +- **Consequence**: schema migrations still express target structure and remain reviewable，but no Resolver-row、Mail-edge、 + Client→Peer data or legacy embedding compatibility is promised。After rebuild，reconcile checked-in production discovery + with the actual contract revision and migration head。 +- **Confidence**: Sir granted the destructive authority and stated the shared data is unimportant；the retained-chain + choice follows direct evidence that only nine revisions exist and that a second hard cut would add mechanism and + monolithic migration risk without meaningful product value。 + +### D-196 — Exact Resolver execution matrix closes text、label and producer obligations + +- **Label contract**: every concrete Resolver returns one stable、non-localized、Block-local label shaped as an owned kind + plus optional identifier。It may hydrate/inspect its focal Block but cannot traverse Relations、invoke AI or materialize + another Block。Missing identifier falls back to the owned kind rather than unsupported capability。 +- **Normalization**: optional identifiers collapse whitespace；free-text identifiers use the first non-empty logical line + and a 96-Unicode-code-point bound。Exact resolver IDs are dispatch/version identity and never leak into semantic text。 +- **Text consolidation**: remove `get_str_for_embedding()` completely。Mail subject/body、GitHub repository language/topics + and Telegram text/media facts enter their one generally useful `get_text()` projection；Memos Attachment stays filename- + only。Byte document/media Resolvers without extracted text remain explicitly unsupported even when PDF/EPUB metadata can + supply a useful concise label。 +- **Freshness boundary**: graph-aware Resolver solved values do not make labels graph-aware。Relation embedding depends + only on the Relation row and two endpoint Block-local labels，so its accepted timestamp freshness formula remains + complete rather than hiding neighbor dependencies。 +- **Execution authority**: the complete per-ID matrix lives in the semantic-retrieval Resolver execution checklist and is + verified through registry、hydration、unsupported/null and producer-parity black boxes。 +- **Confidence**: Sir explicitly accepted the presented text/label matrix after its normalization、Block-locality and + per-family behavior were summarized。 + +### D-197 — Test valuable InKCre-owned behavior，not each incident or dependency implementation + +- **Cause before case**: a fixed defect does not automatically earn a regression test。First identify whether the failure + was a valuable InKCre-owned behavioral contract that a legitimate future implementation change could break，a structural + rule already owned by types/static checks，a misuse of a mature dependency，or evidence of missing shared infrastructure。 +- **Proof placement**: prefer a black-box test at an InKCre-owned observable boundary。Use a focused lower-level test only + under the D-049 exception。When the corrected architecture makes the invalid path structurally unavailable，prove that + architecture with its owning type/static/integration mechanism rather than copying the dependency's own tests。 +- **Infrastructure incidents**: when one adapter exposes a cross-cutting gap such as absent observability redaction，do not + assert one field's `repr` and mistake it for system safety。Record the infrastructure pressure；when that boundary exists， + test it once at the shared owner and remove incident-shaped duplicates。 +- **Semantic retrieval consequence**: keep `OpenAICompatibleConfig.api_key` secret-typed and unwrap only at SDK client + construction，but remove the adapter test that merely repeats Pydantic `SecretStr` representation behavior。The current + unit does not claim a complete logging/tracing/exception redaction pipeline。 +- **Confidence**: Sir rejected the incident-specific redaction regression and identified wrong tool/library usage plus a + missing architecture/observability baseline as the actual error class。 + +### D-198 — Low-cost information ownership is the product foundation；MVP maturity follows job and acceptable cost + +- **Product foundation**: InKCre first lowers the cost of collecting、organizing and basically using an info-base so a + sufficiently large body of information can later create emergent value。A source unit need not pre-answer the specific + knowledge artifact or creative output that every collected item will produce。 +- **Terminology boundary**: the system holds information；knowledge exists in the user's mind。Information becomes + knowledge only through human understanding and valuable application，so PRD language must not treat the two as + interchangeable database objects。 +- **Maturity judgment**: MVP / MLP is determined by whether the intended user job is done and whether its costs、losses or + side effects are acceptable。Protocol completeness、field coverage、identity machinery、attachment support and feature + count are evidence only，not automatic maturity gates。 +- **Source effects**: mutating an external source can be an intentional production workflow rather than an accidental side + effect。Mail `mark_as_seen` is such a candidate and is already configurable；its user semantics still require product + acceptance。 +- **Confidence**: Sir corrected the feature-completeness framing，restated the information/knowledge boundary and linked + low-cost info-base ownership to the program's root objective。 + +### D-199 — Mail extension is the next active implementable unit + +- **Unit identity**: `mail-extension` is the active ownership unit。The current delivery scope preserves that identity and + evolves the existing PoC toward a trustworthy collection baseline and useful/low-cost product experience；`hardening` or + `MLP` are scope descriptions，not replacement unit identities。 +- **Evidence boundary**: the old implementation is requirement and failure evidence，not an incremental-design authority。 + Its missing `Message-ID` reconciliation、reply/reference links or MIME attachment materialization are candidate + pressures，not pre-approved MVP blockers。 +- **Vertical**: real mail sources → trustworthy collection baseline → persisted graph and resolver/use representation → + mail-demanded organization and info-base query improvements → necessary client-web journey。This vertical may evolve + cross-cutting capabilities without claiming to complete their entire ownership trunks。 +- **Acceptance pressure**: replace schema/helper-led confidence with a real IMAP protocol → collect job → committed graph → + resolver/use black-box authority。Exact corpus、server boundary and failure horizon remain Product/Acceptance questions。 +- **Confidence**: Sir explicitly selected Mail as the next active implementable unit and accepted the proposed vertical and + old-implementation evidence boundary。 + +### D-200 — Mail ultimately owns a complete email client/agent direction，grounded in complete communication records + +- **Terminal product direction**: Mail is not complete when it merely imports selected incoming messages into the + info-base。InKCre should eventually become a complete email client/agent，closing the loop from communication capture and + understanding through organization、query and intentional email actions。 +- **Information foundation**: the product should retain as complete a communication record as practical，rather than model + Mail only as an inbox intake stream。Incoming、sent、archived and other mailbox/folder scopes are product inputs to be + refined，not a prior decision to ingest every server object indiscriminately。 +- **Bidirectional consequence**: `mark_as_seen` is an early intentional Mail action，not an exceptional violation of a + read-only source boundary。Future compose/reply/send、move/archive、label、delete or agent actions remain separate scope + decisions and are not silently approved by the terminal goal。 +- **Iteration boundary**: the terminal email client/agent direction does not require the current delivery scope to implement + the whole product。Each iteration must select an observable vertical while preserving the long-term topology instead of + locking Mail into a collector-only abstraction。 +- **Confidence**: Sir explicitly selected complete communication history and named a complete InKCre email agent/client as + Mail extension's ultimate goal。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D201-D210.md b/tasks/knowledge-lifecycle-capabilities/decisions/D201-D210.md new file mode 100644 index 0000000..c0c145d --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D201-D210.md @@ -0,0 +1,148 @@ +# Decisions D-201–D-210 + +> [Decision register index](index.md) + +### D-201 — The current Mail delivery scope establishes the communication-record foundation，not outbound composition + +- **Current outcome**: collect and continuously update as complete a multi-account communication record as practical，then + make it available through Resolver/use、the mail-demanded minimum basic query and the necessary client-web journey。 +- **Intentional write-back**: retain and verify configurable `mark_as_seen` as the current scope's remote Mail action。It + proves that the topology must not assume read-only collection without opening every future mutation now。 +- **Deferred actions**: compose、reply and send are not part of this delivery scope。Draft lifecycle、sending protocols、 + identity selection、idempotency、approval、failure recovery and agent execution remain future Mail scopes that must pass + their own gates。 +- **Architecture consequence**: current design must preserve a path toward the complete email client/agent direction，but + “future-compatible” does not authorize speculative outbound abstractions or generic action frameworks。 +- **Confidence**: Sir explicitly accepted this current-scope boundary after confirming D-200's terminal product direction。 + +### D-202 — One Mail Source represents one account；Extension config owns default mailbox exclusions + +- **Product boundary**: one Mail Source represents one email account。Its collection coverage is selected from that + account's mailboxes/folders，rather than creating a separate Source per folder or placing multiple accounts behind one + indistinguishable Source identity。 +- **Default coverage**: pursue a complete communication record by including inbox、sent、archive and user-created folders + by default，while drafts、spam and trash are default exclusion categories。Provider-specific labels/folder names are + adapter evidence，not shared product vocabulary。 +- **Configuration ownership**: Mail extension config owns the deployment-wide default excluded mailbox/folder policy。A + Source consumes that policy when expressing one account's actual coverage；the exact source override and merge semantics + remain a Technical question rather than an implicit decision here。 +- **User control**: default exclusions do not make any mailbox permanently unsupported。The eventual configuration contract + must allow deliberate coverage choices，subject to the later Technical design。 +- **Confidence**: Sir accepted the proposed account/mailbox product boundary and explicitly assigned default excluded + folders/mailboxes to extension config。 + +### D-203 — Source config can override Mail extension default mailbox exclusions + +- **Default flow**: Mail extension config owns the deployment-wide default excluded mailbox/folder set，and a newly + configured Source derives its initial exclusion value from that extension-level default。 +- **Source authority**: each Mail Source can explicitly configure its own excluded mailboxes/folders。The concrete account's + value is not permanently coupled to later extension-default changes merely because it originated from that default。 +- **Open Technical edge**: exact create-time defaulting、nullable/omitted representation、later reset-to-default behavior and + update merge semantics remain Technical questions。Do not infer dynamic inheritance or silently overwrite a Source's + explicit value。 +- **Confidence**: Sir explicitly clarified both Source-level configurability and the extension-default origin。 + +### D-204 — Ordinary Mail collection starts at Source setup；history uses explicit bounded backfill，not `full` + +- **Default collection**: setting up a Mail Source does not automatically ingest its prior account history。Ordinary + collection observes information produced after the Source's setup boundary and then continues forward。 +- **Historical action**: collecting earlier messages is an explicit、manually requested `backfill` operation with a caller- + supplied boundary。A complete communication record is a supported user outcome，not an unbounded side effect of Source + creation。 +- **Vocabulary hard cut**: do not reuse legacy `full` for history。`full` conflates historical range、reconciliation、 + replacement and other effects；D-074 already rejected it as common stable vocabulary。Mail uses `backfill` for this exact + intent。 +- **Cross-source direction**: ordinary collection versus explicit bounded history is a candidate common Source contract，but + its ordinary scope and boundary shape remain source-native。A current-state Source such as files/calendar may not equate + “history” with objects that merely existed before setup，so Mail does not silently rewrite every Source behavior。 +- **Independence**: embedding、organization and other derived/use-support operations have their own selection policies and + do not define or justify Mail backfill range。 +- **Confidence**: Sir replaced automatic first-history collection with explicit bounded history，rejected `full` semantics + and requested a common Source history vocabulary without mixing collection and organization concerns。 + +### D-205 — `collect` is the umbrella ingress action；`backfill` is a specialized collect intent + +- **Stable meaning**: collect means bringing information from outside InKCre into the info-base。This is the Source + domain's umbrella action and does not imply one universal time、cursor or snapshot strategy。 +- **Containment，not siblings**: backfill is a special kind/intent of collect that explicitly reaches source-native history + outside the Source's ordinary collection horizon。Do not model “collect” and “backfill” as parallel top-level + capabilities or lifecycles。 +- **Common mental model**: every Source may define its ordinary collection scope；a history-capable Source may additionally + accept an explicit bounded backfill collect。Sources without meaningful history need not invent the operation。 +- **Technical restraint**: this product vocabulary does not yet choose separate methods、a mode discriminator、job schema or + boundary type。Those are Technical decisions after existing Source/job implementations are audited。 +- **Confidence**: Sir accepted the common Source vocabulary but corrected the proposed parallelism and supplied collect's + system-external → system-internal meaning。 + +### D-206 — `enrichment` is reserved for additive Organization work over an existing graph + +- **Organization meaning**: enrichment starts from information already in the info-base and **adds** useful information or + structure to improve use。Addition is the defining emphasis；recomputing or replacing a projection is not enrichment merely + because it improves use。It is an Organization approach/category，not a synonym for every operation that obtains more bytes + or creates another Block。 +- **Mail example**: after an email is collected，discovering a URL in its body and expanding the referenced content into a + connected graph is enrichment。Resolver、AI、external fetch or other mechanisms may participate without taking ownership + away from Organization。 +- **Collection naming correction**: source-owned acquisition remains valid，but should use precise collection language。 + RSS collection-time linked-page work is `full-text acquisition`，not `full-text enrichment`；mail MIME parts are attachment + collection/acquisition。The ownership does not change merely because the label is corrected。 +- **Taxonomy restraint**: enrichment may overlap with linking、interpretation or other Organization approaches in a concrete + solution。This decision reserves its domain meaning without declaring a complete or mutually exclusive Organization + taxonomy。 +- **Confidence**: Sir accepted source-owned full-text acquisition，reserved enrichment for Organization and identified + post-collection Mail URL expansion as the canonical example。 + +### D-207 — Source already owns collection；avoid the redundant `source-owned collection` phrase + +- **Ownership correction**: Source is the collection-behavior owner by definition。Do not say `source-owned collection` as + though another collection owner must be disambiguated。 +- **Precise alternatives**: use `collection` for the domain action，`source acquisition` when contrasting external acquisition + with Organization enrichment，or an exact operation name such as `full-text acquisition` or `attachment acquisition`。 +- **No topology change**: this naming correction does not move RSS full-text acquisition、Mail attachment acquisition or any + other accepted behavior between owners。 +- **Confidence**: Sir identified the phrase as structurally redundant after the collection/enrichment boundary was fixed。 + +### D-208 — Remote Mail deletion does not delete info-base information by default + +- **Authority boundary**: the info-base is not a mirror of the mailbox。Once collected，an email remains information owned by + the info-base even if the remote mailbox later deletes or expunges it。 +- **Default behavior**: retain the email graph and record that it is deleted/absent on the mailbox side。The exact persisted + representation remains Technical；a mailbox-scoped membership fact must not be prematurely flattened into one global + email-deleted flag。 +- **Optional destructive sync**: a Source may eventually opt into synchronizing remote deletion to info-base deletion，but + the feature defaults off and is only eligible when the protocol/server exposes trustworthy incremental deletion evidence。 + Do not implement it by periodically traversing and diffing the entire mailbox merely to claim support。 +- **Move/membership caution**: disappearance from one mailbox may mean move、label removal or per-mailbox deletion rather + than global message deletion。Technical design must respect mailbox scope before any destructive effect。 +- **Confidence**: Sir explicitly chose retention，allowed conditional opt-in synchronized deletion and prohibited traversal- + based emulation；the scoped-fact caution follows the accepted account/mailbox boundary。 + +### D-209 — Mail Source owns remote-deletion behavior；current membership uses relation presence，not a deletion tombstone + +- **Owner correction**: InfoBase does not understand Mail or choose retention policy。Mail Source owns collection behavior； + when it observes remote removal under the default policy，it submits the mailbox-membership mutation and does not submit + deletion of the collected email graph。 +- **Current-state graph**: an Email's current presence in a Mailbox is represented by their membership relation。Removal or + move deletes the old membership relation；move also creates the new one。Exact direction and relation content remain + Technical vocabulary questions。 +- **No historical tombstone**: do not create `has been deleted from` or equivalent merely to remember a past membership。 + That would introduce event/history semantics without a current use requirement。Source sync state owns processed-change + progress；the graph expresses current known membership。 +- **Optional destructive policy**: D-208's opt-in synchronized deletion remains a Source policy。If later enabled and + protocol evidence proves its condition，Mail Source may issue graph deletion；InfoBase still only executes a domain command + and does not infer Mail semantics。 +- **Confidence**: Sir corrected the behavioral owner and proposed relation removal rather than a deleted marker；the rejection + of a deletion tombstone follows the current-state product requirement and prior restraint against invented audit/history。 + +### D-210 — Mail collection synchronizes membership and remote state；flags are graph tags，not Email content fields + +- **Continuous facts**: ordinary Mail collection keeps current mailbox membership and relevant remote message state in sync。 + Failure to refresh those facts does not invalidate already collected Email content。 +- **Flag modeling**: canonical Mail flags/keywords are independent Mail Flag Blocks connected to Email Blocks through + Relations。They are mutable、shared/queryable tag-like information and do not belong inside canonical Email root content。 +- **State distinction**: Seen and Answered have distinct mail-state semantics and are not collapsed into the tag-like Flag + model merely because a provider/protocol may transport them together。Their exact graph representation remains Technical。 +- **Membership modeling**: Mailbox is an independent Block；Email↔Mailbox Relations express current inclusion。The exact + predicate/direction must be chosen for clear graph reading，not copied blindly from IMAP field names。 +- **Confidence**: Sir explicitly required membership/remote-state synchronization，distinguished flags from Seen/Answered and + selected independent Mail Flag Blocks over root-content attributes。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D211-D220.md b/tasks/knowledge-lifecycle-capabilities/decisions/D211-D220.md new file mode 100644 index 0000000..904b1ca --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D211-D220.md @@ -0,0 +1,175 @@ +# Decisions D-211–D-220 + +> [Decision register index](index.md) + +### D-211 — Manual and scheduled Mail collection both create collect jobs；execution is future multi-Peer + +- **Product freshness**: the current Mail delivery scope supports manual and scheduled collection with bounded、configurable + delay。Near-real-time IMAP IDLE/long-lived push is not an Acceptance requirement。 +- **One execution semantic**: manual invocation and scheduling are job-creation paths，not two kinds of collection behavior。 + Both produce ordinary collect jobs that follow the same claim、execution、result and diagnostic contract。 +- **Peer topology**: core-py is the current capable executor，not the permanent owner of Mail runtime。Extension + implementations are expected to become multi-Peer；a future background-capable native Apple-universal InKCre Peer may + synchronize at a higher cadence because it can act as a mail client，while a browser Peer may remain incapable。 +- **Scope restraint**: this direction does not yet introduce distributed scheduling、cross-Peer cadence negotiation、job + routing、duplicate-schedule suppression or background execution in client-web。Technical design must avoid locking the + extension contract to core-py without prebuilding unsupported Peer machinery。 +- **Confidence**: Sir explicitly selected manual + scheduled collect-job execution，rejected the need for IDLE and supplied + the future native-Peer mail-client example。 + +### D-212 — A collect job is a CronJob-like execution envelope，not Source-internal completeness + +- **Common Source pattern**: a collect job records and executes one manual/scheduled invocation of Source collection or + synchronization。It is analogous to a CronJob run，not a declaration that one mailbox、page、item set or whole remote + source must be completed atomically。 +- **Deep Source boundary**: mailbox traversal、item processing、cursor/checkpoint progression、partial effects、failure + isolation and continuation belong inside the Source implementation。Do not extend generic job semantics into those + source-native units。 +- **Shallow job completion**: a job completes when the Source invocation returns under its own public completion semantics。 + It fails when execution raises beyond that boundary、is aborted or the runtime fails。Completion does not claim a message + count or complete synchronization horizon。 +- **Mail consequence**: one mailbox failure may be handled internally while remaining mailboxes continue and useful Email + graphs persist。The generic job does not acquire a mailbox-scoped transaction、cursor or “completed with errors” algebra + merely to expose that detail。 +- **Correction**: withdraw the proposal that any included-mailbox failure must automatically make the collect job fail with + mailbox-scoped job diagnostics。A Mail Source may still log or retain source-specific diagnostics/state，and may choose to + propagate an unrecoverable error，without changing the generic job contract。 +- **Confidence**: Sir identified Mailbox as a Mail management unit rather than the collected information unit，compared the + job to CronJob and explicitly required this as a common Source pattern。 + +### D-213 — Collect jobs are one-shot and have no retry lifecycle + +- **Terminal execution**: one collect job represents one invocation and terminates as completed、failed or aborted。The job + domain does not retry、resume or reopen that record。 +- **New run，new job**: the next scheduled tick or manual request creates a new job。A later Source invocation may continue + from Source-owned state and naturally cover previously uncollected information，but it is not an attempt of the old job。 +- **Excluded machinery**: do not add job attempt rows、retry count、backoff policy、parent/retry lineage or “resume failed + job” semantics to ordinary Source collection。 +- **Local-operation boundary**: a Source implementation could eventually perform a bounded transient I/O retry inside one + invocation if separately justified，but generic job semantics neither require nor expose it。 +- **Confidence**: Sir restated the CronJob-like boundary as the exact rule that a job has no retry。 + +### D-214 — Mail collection records attachment metadata，not attachment bytes；durable materialization is enrichment + +- **Collection boundary**: ordinary and backfill Mail collection do not fetch every attachment's actual bytes or write them + to Storage。They collect enough attachment metadata/remote reference information to preserve the Email graph and support + later retrieval；the exact canonical shape remains Technical。 +- **Industry evidence**: IMAP supports body-structure and per-part fetch；mainstream clients use mixed lazy/configurable + policies rather than one universal default to durably download all attachments。The evidence is retained in the Mail unit + evidence file。 +- **Organization path**: when an existing Email/Attachment metadata graph causes InKCre to fetch bytes and persist a new + semantic content Block for better use，that additive operation is Organization enrichment under D-206。 +- **Use distinction**: transiently fetching/streaming an attachment because the user opens or downloads it is use，not + automatically enrichment。It only becomes enrichment when the info-base is durably augmented。 +- **Mechanism boundary**: Mail Source/extension may provide authenticated remote-part access as a mechanism，but that does not + make the post-collection materialization decision part of collection。Exact tool/resolver/source interfaces remain + Technical questions。 +- **Confidence**: Sir chose metadata-only collection unless mainstream clients established universal default downloads； + official protocol/client evidence does not establish that premise，and Sir identified durable attachment expansion as a + useful Organization enrichment case。 + +### D-215 — Enrichment classifies additive behavior；Mail Resolver owns attachment materialization + +- **Correction to D-206/D-214**: enrichment is an Organization behavior/approach classification，not automatic implementation + ownership by an Organization module。The previous wording incorrectly fused “what kind of graph improvement is this?” + with “which deep module performs it?”。 +- **Resolver owner**: the Mail attachment Resolver owns the deep capability that turns an Attachment metadata Block into + applicable semantic content and may durably materialize the resulting semantic content graph。Exact method/form contracts + remain Technical，but callers do not reproduce MIME、remote access、Storage or graph details。 +- **Delegation topology**: the Resolver may delegate authenticated remote-part acquisition to the Mail Source/extension，then + use Storage and InfoBase persistence capabilities。Those mechanisms retain their own narrow responsibilities；Organization + does not absorb them merely because the resulting additive change is enrichment。 +- **Trigger independence**: Organization may select/trigger attachment enrichment；use may also trigger the same Resolver + lazy-materialization path。The classification does not require every call to pass through OrganizationManager or another + invented manager。 +- **Transient boundary retained**: streaming remote bytes without durable graph augmentation remains use；durable addition of + semantic content can be classified as enrichment while still being Resolver-executed。 +- **Confidence**: Sir explicitly rejected Organization ownership of byte download/semantic-content persistence，assigned the + operation to Mail Resolver with possible Source delegation，identified the classification/ownership distinction and later + explicitly accepted external Resolver ownership with internal delegation deferred。 + +### D-216 — Collect textual Mail body；treat non-text inline MIME parts as lazy content + +- **Body collection**: collect authored textual body representations needed to understand the Email，including `text/plain` + and `text/html` when present。Exact canonical storage/alternative-selection shape remains Technical。 +- **Inline binary boundary**: CID-referenced images and other non-text inline MIME parts follow the attachment metadata-only + policy。Collection records their metadata/reference but does not download/persist bytes by default。 +- **Use/materialization**: viewing may fetch/stream an inline part on demand；durable semantic-content addition uses the + corresponding Mail Resolver materialization capability and can be classified as enrichment under D-215。 +- **Remote HTML resources**: external images/resources referenced by HTML are not automatically fetched during collection。 + Later use or Organization behavior must make any retrieval/materialization policy explicit。 +- **Fidelity consequence**: the collected Email remains semantically readable through its textual body，while fully faithful + visual rendering can require online/on-demand parts。This is an accepted product trade-off，not a hidden completeness + claim。 +- **Confidence**: Sir explicitly accepted the proposed inline MIME parts boundary。 + +### D-217 — Native reply/reference facts form Email Relations；no generic Thread Block or collection-time inference + +- **Collected structure**: collect source-native reply/reference facts and express resolvable structure as directed Relations + between Email Blocks。Protocol headers such as Message-ID、In-Reply-To and References are mechanisms/evidence，not a + feature-completeness maturity checklist。 +- **No generic Thread entity**: do not create a universal Thread Block merely to group messages。A conversation/thread view is + derived from the Email relation graph unless a future source-native object proves independent information value and + identity。 +- **No collection inference**: collection does not guess missing reply links from subject、participants or time。Such + best-effort graph addition belongs to Organization linking when it improves use。 +- **Unresolved edge**: references whose target Email is not yet collected still need a faithful non-invented representation， + but placeholder/reference shape、later reconciliation and relation creation remain Technical questions。 +- **Direction vocabulary**: the intended meaning is reply Email → replied-to parent Email。Exact relation content is chosen in + Technical design for readable dynamic-property grammar rather than copied blindly from header names。 +- **Confidence**: Sir explicitly accepted the proposed reply/thread structure design after reiterating that protocol fields + are not automatic MVP criteria。 + +### D-218 — Mail renders Email Blocks through generic info-base surfaces；no Mail browsing page + +- **Product correction**: do not build an inbox/folder/message-list page or Mail-specific browsing/query product for this + delivery scope。That would reproduce an ordinary email client interface without proving InKCre-specific use value。 +- **Client-web pattern**: the Mail client-web extension supplies an Email Resolver and resolver-owned content component，like + the existing Twitter extension's `TweetResolver.contentComp`。Generic `BlockContent`/Block-detail/graph/query-result + surfaces render the Email Block through its registered Resolver component。 +- **Rendered information**: the Email component can present authored body、participants、mailbox membership、flags、native + reply/reference relations and attachment metadata by consuming Resolver solved content/relations。Exact compact/detail + presentation remains client-web design。 +- **Capability，not imitation**: the terminal email client/agent direction describes complete mail capabilities and actions， + not a mandate to copy a traditional mailbox information architecture。Mail-specific actions may later appear where useful + without requiring a dedicated Mail application page。 +- **Query boundary**: Mail can pressure generic info-base query/navigation，but this decision does not invent a Mail-only + filter API or UI。Any new query behavior must be justified as a generic info-base capability or a separately accepted + product surface。 +- **Withdrawn proposal**: withdraw the proposed Mail-specific account/mailbox browse list and structured filter journey as + the current client-web minimum。 +- **Evidence**: client-web's Twitter extension registers `extensions.twitter.tweet.v1` with `contentComp`；generic + `BlockContent` resolves and mounts the component，and the generic Block details panel already consumes `BlockContent`。 +- **Confidence**: Sir rejected a conventional Mail browsing page and explicitly selected the Twitter extension Block-renderer + pattern for Email Blocks。 + +### D-219 — Do not pre-authorize a generic query increment for Mail + +- **Current boundary**: this unit does not invent a generic query capability merely because Mail might benefit from one。 + Existing generic Block/graph surfaces and the Email renderer are the accepted client-web baseline。 +- **Evidence gate**: only if preflight proves collected Email Blocks cannot be reasonably reached through existing generic + surfaces may the unit propose the smallest generic query improvement that removes that actual blocker。 +- **Ownership**: any accepted increment belongs to generic info-base query/use，not a Mail-only filter endpoint or UI。It does + not automatically close the future feature-retrieval or graph-navigation ownership units。 +- **Confidence**: Sir explicitly accepted no predefined query increment and the preflight/blocker gate。 + +### D-220 — Reply/reference rendering is Block navigation；`contentComp` hides a focal-graph presentation contract + +- **Interaction**: Email presentation does not render reply/reference facts as ordinary inline links or a relation list。 + It exposes actions such as “查看回复” that navigate the generic info-base UI to the corresponding Email Block。Exact labels + for parent versus replies and multiple targets remain client-web design。 +- **Navigation owner**: the renderer identifies a target Block；the generic client-web Block-navigation surface performs the + transition。Do not embed a Mail-specific page route into the Email component。 +- **Interface problem**: existing `contentComp` naming implies rendering `Block.content`，while Tweet and Email require a + focal Block plus resolver-owned local graph projection。This is obscurity in the shared client-web Resolver contract，not a + reason to forbid graph-aware rendering。 +- **Deep boundary**: Resolver—not the Vue component—interprets hydrated content and local Relations into solved content for + the focal Block。The renderer consumes that projection and invokes explicit Resolver/navigation capabilities instead of + independently rebuilding graph semantics。 +- **Current Tweet evidence**: `TweetResolver` loads attachment Relations/Blocks and mutates a parsed Tweet-shaped solved + object before `contentTweet` renders it。This proves star-neighborhood demand but also exposes canonical-content versus + presentation-projection ambiguity that Technical design should correct rather than copy blindly。 +- **Open naming/API**: renaming `contentComp`/`BlockContent` and choosing solved-content/presentation props are Technical + questions。Product fixes only graph-aware focal-Block rendering and generic target-Block navigation。 +- **Confidence**: Sir corrected reply/reference interaction，identified the contentComp/star-graph mismatch and accepted the + no-predefined-query boundary。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D221-D230.md b/tasks/knowledge-lifecycle-capabilities/decisions/D221-D230.md new file mode 100644 index 0000000..8c1907e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D221-D230.md @@ -0,0 +1,200 @@ +# Decisions D-221–D-230 + +> [Decision register index](index.md) + +### D-221 — `block.resolver` selects Block behavior；client-web presents it through `BlockRenderer` + +- **Behavior contract**: `block.resolver` is not merely a display discriminator or content decoder。Its exact、versioned + resolver identity selects how the focal Block's persisted/hydrated content and required local graph context become usable + capabilities and solved content。 +- **Presentation consequence**: client-web replaces the misleading `contentComp` frame with `BlockRenderer`。A registered + renderer presents resolver-produced solved content for a focal Block；it does not independently reconstruct Mail、Tweet or + other graph semantics。 +- **Navigation clarification**: an Email renderer may expose actions such as “查看回复” and identify the target Block，but + does not own a Mail-specific URL/page。The generic hosting Block/details/graph surface decides how that target becomes the + current/open Block。This is the already-accepted generic discovery/open/render path，not a second Mail navigation product。 +- **Content layers**: persisted `block.content` representation、hydrated content and solved content are distinct。Hydration + only hides inline-versus-Storage access and returns the focal Block payload；solved content is a resolver-owned、derived、 + use-facing projection that can include required local Relations/related Blocks。Solved content is not durable graph + authority and must not masquerade as the canonical root content merely to simplify a renderer。 +- **Naming edge**: domain-specific solved projections need one non-redundant field for their canonical focal content plus + graph-derived fields。`SolvedEmail.email` / `SolvedTweet.tweet` are rejected as awkward candidates；the exact shared field + name remains under immediate Technical review。 +- **Promotion pressure**: PRD glossary should retain the user/product meaning of Block/Resolver；Product TDD should own the + exact behavior-selection and hydrated-versus-solved topology；client-web local architecture should own the + `BlockRenderer` interface and generic navigation mechanism。Promotion follows implementation evidence and owner-specific + workflow。 +- **Confidence**: Sir accepted `BlockRenderer` because `block.resolver` makes Block behavior coherent，identified this as a + deeper Block/Resolver relationship requiring durable clarification，required hydrated and solved content to remain + distinct，and accepted that solved content not impersonate canonical root content。 + +### D-222 — Graph surface exclusively owns cross-Block navigation；graph-aware solved roots use `.root` + +- **Navigation correction to D-221**: the graph surface—not a generic set that includes Block details—owns selection、focus、 + graph-position and route consequences when moving to another Block。`BlockDetails` only presents/acts on its current focal + Block and must not know or open another Block。 +- **Renderer implication**: a resolver-owned renderer can expose a semantic action and target Block reference，but the exact + delivery path must bypass Block-details ownership and terminate at the graph surface。Event forwarding、injected + navigation capability or another shallow UI contract remain Technical candidates；none is approved yet。 +- **Open hierarchy**: accepting the `BlockRenderer` name does not close the renderer/details composition。Technical design + must distinguish the resolver-selected semantic representation from the generic details/inspection shell before fixing + component names、props or event topology。 +- **Solved-root name**: graph-aware solved projections use `.root` for the focal Block's canonical parsed content；for example + `SolvedEmail.root: CanonicalEmail` and `SolvedTweet.root: CanonicalTweet`。Relation-derived fields remain siblings and do + not mutate or masquerade as `.root`。 +- **Cross-unit consistency**: RSS already supplies evidence for `.root`；Memos currently uses `.canonical` and must be audited + during preflight rather than silently left as competing stable vocabulary or mechanically migrated without blast-radius + evidence。 +- **Confidence**: Sir explicitly assigned routing to graph surface，forbade Block details from opening/knowing another Block， + kept the BlockRenderer design open for further discussion and accepted `.root`。 + +### D-223 — Resolver registers a `SolvedContentRenderer`；`BlockInspector` exposes “view content” + +- **Naming correction to D-221/D-222**: withdraw `BlockRenderer` as the exact client-web contract name。It ambiguously claims + the whole Block UI and collides with the generic details layer。The resolver-selected component is + `SolvedContentRenderer` because its direct subject is the resolver's solved content。 +- **View semantic**: the user-facing “查看内容” action means view the focal Block's solved content：obtain the exact Resolver， + resolve its hydrated content/local graph context，then render the resulting projection。The UI label need not expose the + technical word “solved”。It does not mean displaying the literal `block.content` column or storage pointer。 +- **Inspector role**: rename/reframe `BlockDetailsPanel` as `BlockInspector`。It owns generic persistence facts and + current-Block commands such as rumination and view content；having commands does not turn it into a graph navigation or + domain rendering owner。 +- **Navigation retained**: a `SolvedContentRenderer` can present relation-derived actions from its solved projection，but + GraphSurface still exclusively executes cross-Block navigation。The renderer-to-GraphSurface bridge remains an open + Technical edge。 +- **Dead boundary removal**: remove `BlockDetailsPanelProps.relations` and the graph-surface binding that supplies it。The prop + is unused and would make the inspector unnecessarily know other Block references；Resolver/solved-content construction + owns the required local Relations。 +- **Still open**: exact controller/component split、renderer props、view placement and navigation bridge are not closed by + this naming decision。 +- **Confidence**: Sir proposed `SolvedContentRenderer` and the “view solved content” interpretation，accepted + `BlockInspector` in light of its rumination entry point，and explicitly approved deleting the unused relations prop。 + +### D-224 — Do not replace Resolver behavior with render context；navigation is an InfoBase routing problem + +- **Domain ownership**: `BlockInspector` and GraphSurface both belong to the InfoBase domain。Their coordination must be + modeled inside that domain rather than treated as generic app-component callback plumbing。 +- **Evidence correction**: current renderers' limited use of the supplied Resolver instance is not evidence that + `SolvedContentRenderer` should be denied the full Resolver。Simple present renderers are an incomplete demand sample；the + Resolver is the selected deep behavior contract and future renderer-owned actions may legitimately need it。 +- **Historical separation**: the proven Module Federation failure was duplicate `@inkcre/core` runtime instances and split + static registries，fixed by sharing core as a singleton。The explicit dynamic import in Resolver relation loading addresses + a Block/Relation module cycle。Core renderer assignment in the host prevents a core-package → app-component dependency， + while extension Resolver/renderer co-location follows the extension-owned direction。None of these facts requires removing + Resolver from renderer props。 +- **Withdrawn candidate**: withdraw the generic `SolvedContentRenderContext` callback bag as the primary solution。It only + moves navigation calls into an ambient object without giving solved-content viewing、cross-Block transition、history or + GraphSurface realization a first-class model。 +- **Promoted design question**: consider a first-class InfoBase router analogous in role to Vue Router，with + `SolvedContentView` as a first-class InfoBase destination and GraphSurface as its route realization/outlet。Exact location + model、history、URL adapter、singleton/injection shape and renderer access remain Technical questions，not accepted schema。 +- **Renderer props remain open**: a full Resolver may remain in `SolvedContentRendererProps` alongside solved content；do not + remove it until the first-class view/router topology proves a smaller interface without losing Resolver behavior。 +- **Confidence**: Sir assigned Inspector/GraphSurface to InfoBase，rejected present non-use as an exclusion argument，recalled + dependency-cycle pressure，rejected context as solving the real problem and proposed first-class solved-content view or an + InfoBase-global router for further design。 + +### D-225 — InfoBaseRouter owns minimal navigation state；GraphSurface realizes routes + +- **Renderer props**: accept `SolvedContentRendererProps<SolvedContentT, ResolverT>` with both the typed solved content and + the complete exact Resolver instance。The solved value is the renderer's presentation subject；the Resolver remains its + deep behavior surface rather than being replaced by a callback/context projection。 +- **Router ownership**: InfoBaseRouter is a first-class InfoBase-domain navigation module。It owns the current InfoBase route + and navigation/history operations；it does not own Block loading、graph layout、selection visuals or Resolver execution。 +- **Route realization**: GraphSurface is the route outlet/realizer。It observes the InfoBase route and owns loading、selecting、 + focusing and presenting the corresponding graph/Block state。This preserves the earlier rule that BlockInspector cannot + open another Block without forcing extension renderers to know GraphSurface or app URLs。 +- **First-class destination**: `SolvedContentView` is an MVP route destination，not an implementation detail nested inside a + render context。It owns Resolver acquisition/lifecycle、solved-content loading/error/refresh and mounting the Resolver's + `SolvedContentRenderer`。 +- **MVP route scope**: admit only graph-with-no-focal-Block、inspect-one-Block and view-one-Block's-solved-content states。 + Do not build arbitrary extension route registration、generic screen routing、Relation inspection or speculative graph-scope + navigation now。 +- **Transport boundary**: the InfoBase route model contains domain locations，not Vue route names、paths or URLs。A web adapter + may synchronize it with Vue Router/browser history later；Module Federation renderers depend only on the shared InfoBase + contract。 +- **Confidence**: Sir explicitly accepted the full Resolver/solved-content props，strongly accepted GraphSurface route + realization、InfoBaseRouter and the proposed MVP positioning。 + +### D-226 — InfoBase routes express domain locations，not their GraphSurface realization + +- **Correction to D-225**: GraphSurface is the current route realizer，not the default/index InfoBase destination。Naming an + InfoBase route `graph` inverts `surface realizes route` into `route selects graph` and prematurely makes one presentation + topology part of the domain location。 +- **Surface independence**: the same InfoBase routes may later be realized by GraphSurface、ListSurface or another admitted + InfoBase surface。A realizer chooses how to load、arrange、focus and present the route；the route does not claim that graph + is the only or default interface merely because the info-base authority is a graph。 +- **Global location name**: use `overview` for the no-focal-entity InfoBase location。It expresses a global field of view + without specifying graph、list、grid or another surface。 +- **Selection edge**: how the application chooses the active surface realizer is separate from the InfoBase location model + and remains open。Do not add a speculative `surface` discriminator to the route merely to preserve today's GraphSurface。 +- **Exact shape remains open**: `overview` is accepted as the global route vocabulary；the focal-Block destinations still + require one coherent resource/action naming review before the exact discriminated union is frozen。 +- **Confidence**: Sir rejected the proposed `graph` route because it breaks the realizer relationship，identified future + ListSurface as counter-evidence and supplied `overview` as the non-misleading global-view name。 + +### D-227 — `overview | block | solved-content` is the minimal surface-independent InfoBase route vocabulary + +- **Exact domain shape**: + ```ts + type InfoBaseRoute = + | { name: 'overview' } + | { name: 'block'; block: BlockRef } + | { name: 'solved-content'; block: BlockRef } + ``` +- **Location semantics**: `block` means focus one Block through the active InfoBase surface；the current GraphSurface may + select its node and present `BlockInspector`，while a future ListSurface may select a row and realize the same location。 + `BlockInspector` is therefore a realization，not route vocabulary。 +- **Content semantics**: `solved-content` names the first-class use-facing view of one Block's resolver-derived projection； + it does not expose literal `block.content`、hydration mechanics or a presentation surface。 +- **MVP exclusions**: do not add a `surface` discriminator、Relation routes、extension-defined routes or speculative route + parameters。Active surface selection remains a separate application concern。 +- **Confidence**: Sir explicitly accepted the proposed three-way discriminated union after correcting the earlier + GraphSurface-specific route model。 + +### D-228 — InfoBaseRouter adapts the existing history authority；`back` must retain literal history semantics + +- **No second history**: do not create an independent InfoBase history stack or promote `InfoBaseHistory` into another + domain module。The browser/Vue Router history remains the single runtime history authority。 +- **Adapter level**: InfoBaseRouter depends on a replaceable history-adapter contract implemented by the web Vue Router + integration and by an in-memory test adapter。This contract is an implementation boundary owned inside the router module， + not a separately meaningful InfoBase concept。 +- **Dependency shape**: callers depend only on InfoBaseRouter；InfoBaseRouter translates its domain route operations through + the injected adapter；the adapter owns Vue-route/URL mapping and reflects browser history changes back as the current + `InfoBaseRoute`。 +- **Literal back semantics**: `back` means traverse the existing history backwards。It must not be simulated by pushing a + guessed parent/current-Block route，because that adds a new entry and can produce loops or a history order different from + the user's actual navigation。 +- **Confidence**: Sir accepted one history authority and replaceable adaptation，rejected the unnecessary independent + `InfoBaseHistory` abstraction and corrected the proposed `push(block)` substitute for `back`。 + +### D-229 — InfoBaseRouter MVP exposes only `current + push + back` + +- **Public interface**: InfoBase callers observe one read-only current `InfoBaseRoute`、push an explicit domain route and + traverse one literal history step backwards。 + ```ts + interface InfoBaseRouter { + readonly current: Readonly<Ref<InfoBaseRoute>> + push(route: InfoBaseRoute): Promise<void> + back(): void + } + ``` +- **Semantic distinction**: `push(block)` is an explicit navigation that creates a new history entry；`back()` returns to + the actual preceding entry。A UI command that means “open this Block” must not be labelled or implemented as history back。 +- **MVP exclusion**: do not expose `replace()` until a real InfoBase-domain caller requires replacement semantics。A web + adapter may still use Vue Router replacement internally for transport canonicalization without promoting it to the public + domain interface。 +- **Confidence**: Sir explicitly accepted the three-operation MVP interface after the literal-back correction。 + +### D-230 — InfoBaseRouter `current` is null outside an active InfoBase application location + +- **Exact type correction**: the accepted public property is + `Readonly<Ref<InfoBaseRoute | null>>`，not an always-present `InfoBaseRoute`。 +- **Null semantics**: `null` means the current application location is not realized by an InfoBase surface—for example， + Sources or Settings。It is not a fourth InfoBase route and does not enter the `InfoBaseRoute` discriminated union。 +- **Authority reason**: inventing `overview` or retaining the last InfoBase location while the Vue Router is elsewhere would + make `current` cease to describe the actual application location and would introduce hidden secondary state。 +- **Active-surface invariant**: while GraphSurface or a future admitted InfoBase surface realizes the current application + location，`current` must decode to exactly one of `overview | block | solved-content`。 +- **Confidence**: Sir explicitly accepted nullable current after the existing non-InfoBase application routes exposed the + omitted edge。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D231-D240.md b/tasks/knowledge-lifecycle-capabilities/decisions/D231-D240.md new file mode 100644 index 0000000..5490ad0 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D231-D240.md @@ -0,0 +1,208 @@ +# Decisions D-231–D-240 + +### D-231 — Web application routes select GraphSurface and project one of the three InfoBase routes + +- **Two-layer location model**: a web application route selects the active InfoBase surface and carries one + surface-independent `InfoBaseRoute`。The surface selection is transport/application state；it does not enter the domain + discriminated union。 +- **Accepted GraphSurface mapping**: + + | Vue Router URL | Active surface | `InfoBaseRouter.current` | + | --- | --- | --- | + | `/info-base/graph` | GraphSurface | `{ name: 'overview' }` | + | `/info-base/graph/blocks/:block` | GraphSurface | `{ name: 'block', block }` | + | `/info-base/graph/blocks/:block/content` | GraphSurface | `{ name: 'solved-content', block }` | + +- **Projection boundary**: the web adapter derives `current` directly from the Vue Router route，encodes `push()` into a + named Vue route and delegates `back()` to Vue Router/browser history。It must not mirror the location into independent + mutable state。 +- **Surface independence evidence**: a future ListSurface may use a different application path while decoding to the same + `InfoBaseRoute`；this possibility explains why `graph` belongs in the URL mapping but not the domain route。 +- **Outside mapping**: application routes outside the admitted InfoBase surface records project to `current = null`。 +- **Confidence**: Sir explicitly accepted the proposed GraphSurface URL mapping and its application-route/domain-route + separation。 + +### D-232 — InfoBaseRouter is a singleton client capability port，not a shared router implementation + +- **Original pressure**: InfoBaseRouter replaces renderer context by giving host、GraphSurface/ListSurface and federated + resolver renderers one stable InfoBase-navigation address through the shared `@inkcre/core` singleton。 +- **Shared ownership**: `@inkcre/core` owns the fixed `InfoBaseRoute` union、the `InfoBaseRouter` contract and one runtime + implementation binding/proxy。It does not own navigation state、a history stack or a static/dynamic route registry。 +- **Client ownership**: each client implements the complete `InfoBaseRouter` contract against its own navigation authority。 + client-web derives nullable `current` from Vue Router、maps the accepted application routes in both directions and + implements `push/back` directly；a future native client may realize the same contract differently。 +- **Consumer topology**: GraphSurface、a future ListSurface、BlockInspector、SolvedContentView and resolver-owned renderers + consume the singleton Router。Surfaces realize its current domain route into UI state；renderers issue navigation intent。 +- **No shared transport decomposition**: withdraw the proposed shared `InfoBaseRouterHistoryAdapter`、generic location and + `InfoBaseRouteCodec`。A client may privately factor its implementation if evidence warrants，but those are not shared + contracts。 +- **State distinction**: the singleton binding may retain one configured implementation reference；that is runtime + composition，not duplicated route/history state。The Router implementation remains the projection of the client's one + navigation authority。 +- **MVP route boundary**: the accepted union currently has only overview and Block-based destinations；Relation navigation + is not silently added by this topology。 +- **Confidence**: Sir reconstructed the original context-removal pressure，identified surfaces as Router consumers and + explicitly confirmed that each client should supply the concrete full implementation。 + +### D-233 — InfoBaseRouter reuses the existing singleton-binding pattern without extracting a generic binding module + +- **Binding API**: `@inkcre/core` keeps one module-scoped `InfoBaseRouter | null` implementation reference，configured by + host bootstrap through `setInfoBaseRouter()` and required by consumers through `getInfoBaseRouter()`。 +- **Failure semantics**: `getInfoBaseRouter()` fails fast before configuration；it must not return `null` because nullable + `router.current` already has the separate stable meaning “current app location is outside InfoBase”。 +- **Existing precedent**: `MFImplementation` already uses the same module-scoped implementation + set/get + fail-fast + singleton-binding pattern。`InkRouter` validates shared contract + host implementation but uses a different，component- + scoped Vue binding。Stateful Config adapter initialization is not this pattern。 +- **Pattern，not abstraction**: do not introduce `createRuntimeBinding<T>()`、a binding registry or another generic module in + this unit。The reusable result is the small implementation pattern and its applicability criteria；two concrete closures + remain clearer than another layer。 +- **Lifecycle exclusions**: no optional `isConfigured()`、clear、runtime hot-swap or multi-provider semantics are required。 +- **Confidence**: Sir accepted `setInfoBaseRouter/getInfoBaseRouter`，asked for the existing singleton-binding precedent and + clarified that “pattern” did not authorize widening the scope into a generic runtime-binding abstraction。 + +### D-234 — Route syntax and Block existence have separate owners + +- **Malformed/unmapped application location**: if client-web cannot decode an admitted route record and its `block` + parameter into a valid integer `BlockRef`，the location does not project an `InfoBaseRoute`；`current` is `null` and the + application-level unmatched/not-found behavior owns presentation。 +- **No existence lookup in Router**: a syntactically valid `block` or `solved-content` route is projected even when the + referenced row may not exist。InfoBaseRouter performs no database query and does not turn entity absence into routing + syntax failure。 +- **Realizer/view ownership**: GraphSurface loads the focal Block needed for `block`；SolvedContentView loads the focal Block + and Resolver projection needed for `solved-content`。The owning surface/view presents a missing-Block outcome locally。 +- **Boundary value**: URL/application-route syntax、domain navigation and entity lifecycle remain decoupled；direct links do + not require the Router to understand persistence。 +- **Confidence**: Sir explicitly accepted the malformed-route versus missing-Block separation。 + +### D-235 — GraphSurface keeps the graph and realizes both focal routes as self-closing popups + +- **Correction**: a first-class route destination is addressable navigation state，not necessarily an application page or + replacement surface body。Withdraw the proposal that `solved-content` replaces GraphSurface's main graph content。 +- **Exact realization**: + - `overview` → graph canvas without a focal popup； + - `block` → graph canvas plus BlockInspector popup/side panel； + - `solved-content` → graph canvas plus SolvedContentView popup。 +- **Popup ownership**: BlockInspector owns its close interaction and interprets close as literal `InfoBaseRouter.back()`。 + GraphSurface does not translate an Inspector close event into a guessed destination。SolvedContentView is the same popup + category and follows the same close/back principle。 +- **History consequence**: transitioning `block A → solved-content A` creates the admitted push entry；closing solved content + traverses back to the actual prior route and therefore restores BlockInspector when that was the prior state。 +- **Surface ownership remains**: GraphSurface observes `router.current`、keeps/focuses the graph and mounts the route-selected + popup。The popup may issue Router commands but does not own graph selection or another Block's UI realization。 +- **Direct-link consequence**: direct `solved-content` realization includes the GraphSurface canvas behind its popup；avoid + adding a page-replacement optimization that changes the accepted interaction model。 +- **Confidence**: Sir explicitly corrected Inspector close to owner-local back semantics and clarified that + SolvedContentView is a GraphSurface popup analogous to the Inspector side panel，not a page。 + +### D-236 — InfoBaseView is a navigation host；route destinations may own behavior-bearing containers + +- **Stable vocabulary**: GraphSurface and a future ListSurface are `InfoBaseView` implementations。An InfoBaseView is an + embedded navigation host that observes InfoBaseRouter and realizes its current route through a persistent base surface + plus a `route destination outlet`。These are architecture terms，not approval for a new base class/component。 +- **General UI rule**: presentation-neutral content does not choose whether it appears in a popup、drawer、card、list item or + another container；the parent page/surface/view/group normally owns that composition。 +- **Exact exception criterion**: when the container's open/dismiss lifecycle is part of an addressable route destination's + behavior contract—for example，dismiss has the exact semantic `InfoBaseRouter.back()`—the shell belongs to the destination + component rather than arbitrary parent layout。This is not a general license for content components to self-wrap。 +- **Exact component names**: use `BlockInspectorPopup` and `SolvedContentPopup` for the shell-owning route destinations。 + Withdraw `SolvedContentView` because it misleadingly suggests a page/general view rather than the admitted popup + realization。 +- **Separation retained**: `SolvedContentRenderer` remains presentation-neutral and does not know about Popup or Router； + `SolvedContentPopup` owns Popup、Block/Resolver/solved-content lifecycle and mounts the renderer。Do not extract a separate + presentation-neutral BlockInspector until an actual second container use requires it。 +- **Durable ownership**: promote the container/content rule and exception criterion to shared Product TDD after + implementation evidence；client-web local architecture owns the InfoBaseView/destination-outlet composition。A UI agent + skill should eventually derive the rule from durable truth，but skill build/delivery infrastructure is not expanded by the + Mail unit。 +- **Confidence**: Sir explicitly accepted the exception criterion and navigation-host framing，requested stable + `presentation-neutral content`、`InfoBaseView` and `route destination outlet` vocabulary，and supplied the two exact Popup + names while rejecting the misleading View implication。 + +### D-237 — InfoBaseRoute is GraphSurface's sole focal-Block authority + +- **Remove duplicate state**: delete GraphSurface's local `selectedBlock`/selected-Block-id authority。For `block` and + `solved-content` routes，the focal `BlockRef` is derived directly from `InfoBaseRouter.current`；`overview` has no focal + Block。 +- **Realizer knowledge**: GraphSurface is an InfoBaseView implementation and route realizer，so it is correct for it to + understand the stable `InfoBaseRoute.name` variants and map them to graph-node focus plus route-destination-outlet + composition。This is admitted domain coupling，not transport leakage。 +- **Direction**: node interaction issues `router.push({ name: 'block', block })`；route observation updates focus without + issuing another push。Browser back/forward and direct URLs therefore have one authority and cannot form a navigation loop。 +- **Loading edge**: before graph data loads，the route itself retains focal identity。After load，GraphSurface resolves and + focuses the matching node。A syntactically valid missing/out-of-projection Block leaves the route intact and produces no + invented focal node；the owning popup presents the missing outcome。 +- **Both focal destinations**: `block` and `solved-content` focus the same referenced graph node while mounting their + respective Popup destinations。 +- **Confidence**: Sir explicitly accepted route as the only focal authority and confirmed that GraphSurface understanding + `route.name` semantics is correct。 + +### D-238 — Route-owned Popups accept BlockRef and own their complete resource lifecycle + +- **Exact inputs**: both `BlockInspectorPopup` and `SolvedContentPopup` receive only the focal `BlockRef` from the + InfoBaseView route destination outlet；they do not require a parent-loaded `Block` object。 +- **Inspector ownership**: BlockInspectorPopup owns `Block.get()`、loading、missing/error presentation、Inspector commands + and Popup dismiss/back behavior。 +- **Solved-content ownership**: SolvedContentPopup owns `Block.get()`、Resolver acquisition、solved-content loading/error/ + refresh/disposal、Popup dismiss/back behavior and mounting the presentation-neutral `SolvedContentRenderer`。 +- **Surface ownership**: GraphSurface independently uses the same route BlockRef to find/focus a node in its graph + projection；it does not pass its incidental cached Block object into the Popup or own destination missing state。 +- **Correction to D-234**: preserve “Router never queries persistence” and “destination side owns missing entity”，but move + block-route loading from GraphSurface into BlockInspectorPopup now that the route destination boundary is explicit。 +- **Accepted cost**: the current GraphSurface may already hold the Block from `getAll()`，so this can add one inexpensive + read。Avoid parent/destination lifecycle coupling；if duplicate reads become material，solve them through a proper Block + cache/manager rather than an incidental prop shortcut。 +- **Confidence**: Sir explicitly accepted BlockRef-only popup inputs and destination-owned Block lifecycle。 + +### D-239 — Mail occurrence identity is mailbox-scoped；MVP consumes OBJECTID MAILBOXID when available + +- **Logical locator**: the protocol-correct remote occurrence locator is `remote mailbox identity + UIDVALIDITY + UID`。 + `Block.id` remains the local Email identity；`Message-ID` remains a best-effort Email reconciliation key。 +- **No universal account identity**: a configured mail address、username or credential set does not prove a stable remote + account identity。An authenticated IMAP connection may expose personal、other-user and shared namespaces；therefore + `account + mailbox name` is not a universally correct cross-Source mailbox identity。 +- **Base fallback**: absent stronger server/provider evidence，use a Source-scoped mailbox binding + UIDVALIDITY + UID。This + is exact within that access context but intentionally does not claim arbitrary cross-Source equality。 +- **MVP OBJECTID support**: after authentication，discover `OBJECTID` via CAPABILITY。When advertised，consume the + `MAILBOXID` response required on successful SELECT/EXAMINE and retain it as stronger mailbox-binding evidence。Do not issue + a separate STATUS per selected mailbox merely to retrieve MAILBOXID。 +- **Network cost**: one post-authentication CAPABILITY round trip per connection is admitted；SELECT/EXAMINE is already part + of collection and carries MAILBOXID without another mailbox-specific query。 +- **Comparison scope**: MAILBOXID is not globally unique by specification；cross-Source binding is allowed only when the + adapter can establish a comparable server/login scope or stronger provider-native scope。It must not compare bare + MAILBOXID values from arbitrary Sources。 +- **Scope exclusion**: consuming OBJECTID's `EMAILID` or `THREADID` is not implied by this MAILBOXID decision；those values + alter Email/thread reconciliation and require their own explicit edge。 +- **Next edge**: decide the exact graph/persistence placement of Source-scoped and MAILBOXID-backed mailbox bindings plus + occurrence-local UIDVALIDITY/UID。 +- **Confidence**: Sir accepted the mailbox-identity correction and explicitly approved MVP MAILBOXID capability support + provided the protocol cost is no more than one additional query；preflight confirms that bound。 + +### D-240 — Every Source has one graph projection；Source state never owns a collected-item ledger + +- **One-to-one projection**: every Source instance owns exactly one Source Block in the info-base。`SourceModel.block` is + the unique stable reference from the operational Source row to that projection；the field follows entity-reference + naming and is not called `block_id`。 +- **Authority split**: SourceModel remains the authority for config、state、schedule and collection execution。The Source + Block is the graph-visible projection used for provenance、navigation and resolver-mediated interpretation；it does not + become a second config/state authority or execute collection。 +- **Graph provenance**: source-native graph roots connect to the Source Block rather than embedding a `SourceRef` as their + only provenance fact。Natural intermediate anchors remain useful：for example，Mail connects Email occurrences through + Mailbox Blocks，and Mailbox Blocks connect to Source Blocks；RSS items may remain under a Feed Block that connects to its + Source Block。 +- **Mail placement**: Mailbox Block content owns remote mailbox identity evidence；the Email–Mailbox membership fact owns + the occurrence-local `UIDVALIDITY + UID` snapshot。The Source access path is expressed by a Mailbox–Source Block + relation，not copied into every Email–Mailbox relation。 +- **State boundary**: Source state may retain bounded cursors and validators such as last-observed UID、Twitter + `latest_tweet_id` or HTTP validators。It must not retain a list/set of every collected native item merely to prevent + duplicates；exact replay/reconciliation evidence is graph-owned and reachable from the Source Block。 +- **Cursor is not a ledger**: a cursor may reduce remote traversal but cannot prove idempotency by itself。For example，the + current Twitter `latest_tweet_id` can remain a cutoff，while exact native Tweet identity is reconciled against Tweets + already connected to that Source Block。 +- **No mandatory synthetic intermediate anchor**: domain-natural Feed/Mailbox/etc. Blocks still organize collected + information。The universal addition is the Source Block at the provenance end，not a requirement to replace those + domain objects with a generic collection-anchor Block。 +- **Open relation vocabulary**: exact Mailbox–Source and directly-collected-item–Source direction/predicate names remain the + next edge。Direction is semantic and should eliminate redundant inverse `... by` predicates where the active direction + is clearer。 +- **Confidence**: Sir corrected the earlier natural-anchor-only interpretation，explicitly approved exactly one Source + Block per Source instance，and excluded collected-item ledgers while retaining bounded cursors/validators。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D241-D250.md b/tasks/knowledge-lifecycle-capabilities/decisions/D241-D250.md new file mode 100644 index 0000000..9bae751 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D241-D250.md @@ -0,0 +1,256 @@ +# Decisions D-241–D-250 + +### D-241 — Relation producers prefer normalized active predicates；Mail provenance direction is frozen + +- **Directed-predicate model**: Relation direction participates in meaning。A persisted relation is a directed + `(from Block, predicate/content, to Block)` fact；the earlier dynamic-property reading is an important shape，not a + universal English grammar for every relation。 +- **Normalization pattern**: when an inverse passive spelling adds no independent meaning，prefer reversing the endpoints + and using the active predicate：`managed by` → `manages`、`collected by` → `collects`。The active predicate uses the + grammatically useful third-person singular form rather than retaining the passive participle `collected`。 +- **Purpose**: this is representational normalization。It reduces synonymous relation vocabulary and lets rumination、 + semantic projection or other graph-to-description paths form useful text without first recognizing avoidable passive + cases。Modern LLMs would usually understand the inverse spelling，so the marginal benefit does not justify enforcement。 +- **Guideline，not restriction**: do not add Relation validation、a global predicate registry or a ban on `... by` content。 + Producers own their relation grammar and should apply this common pattern when it makes the representation simpler；an + inverse phrase remains valid when a producer has a real reason to use it。 +- **Canonical Mail chain**: + + ```text + Source Block --manages--> Mailbox Block + Mailbox Block --contains { UIDVALIDITY, UID }--> Email Block + ``` + + The structured membership content uses `type = "contains"` plus the occurrence-local locator fields。Deleting remote + membership deletes the `contains` relation；ending one Source's management of a Mailbox deletes the corresponding + `manages` relation。 +- **Directly collected objects**: when a source-native item has no natural intermediate anchor，use the same normalized + direction，for example `Source Block --collects--> Tweet Block`，rather than `Tweet --collected_by--> Source`。 +- **Durable promotion pressure**: shared PRD should state that direction is product-significant and enables type-independent + dynamic relationships；Product TDD should own the canonical-direction/producer-grammar guidance and the rule against + duplicating inverse facts merely for traversal convenience。 +- **Confidence**: Sir supplied the normalization rationale，corrected `collected` to active `collects`，explicitly kept the + pattern non-mandatory and accepted the canonical Mail provenance chain。 + +### D-242 — Deleting an operational Source preserves its Source Block and collection provenance + +- **Independent survival**: deleting SourceModel terminates the operational collector but does not delete its Source Block、 + existing `collects` / `manages` relations or any collected graph。 +- **Information boundary**: removing credentials、schedule、cursor and future collection ability must not silently rewrite + information already admitted to the info-base or erase where it came from。Source deletion is not an implicit + “uncollect” command。 +- **Availability boundary**: a Source Block relation proves provenance/binding history，not that a live executor or valid + credentials currently exist。Remote Mail operations must resolve a still-live SourceModel associated with that Block； + absence makes the capability unavailable without invalidating the graph。 +- **Projection consequence**: Source Block content must remain minimally interpretable after the SourceModel row is gone； + it cannot be only an empty shell whose entire label/meaning is dynamically read from live config。The exact canonical + content and whether SourceModel should retain a separate identity remain the next edge。 +- **Recreation**: creating a later operational Source does not silently reclaim an old Source Block merely because its + config looks similar。Continuity would require explicit exact evidence and is outside this decision。 +- **Confidence**: Sir explicitly accepted preservation of the Source Block、provenance relations and collected graph after + operational Source deletion。 + +### D-243 — SourceModel is a sequence-free sidecar keyed by its Source Block identity + +- **Shared identity**: `sources.block` is both the SourceModel primary key and a foreign key to `blocks.id`。There is no + separate `sources.id`；the SourceRef and its Source BlockRef carry the same integer identity while retaining distinct + domain/static types at their call sites。 +- **No competing allocator**: `sources.block` has no default、identity or sequence。Source creation first inserts the + Source Block，then inserts the Source sidecar with that exact Block ID，within one transaction。This prevents + collisions between independent sequences and prevents a committed operational Source without its mandatory Block。 +- **Lifecycle direction**: deleting the Source sidecar leaves the Block and graph intact。Deleting the Source Block while a + live sidecar exists may cascade to the sidecar；the foreign-key direction must not make Source deletion cascade upward + into Block deletion。 +- **Reference consequence**: collect-job `source` and other operational Source references target `sources.block`。No + runtime code inspects an integer to decide whether it is a SourceRef or BlockRef；the owning interface already supplies + that semantic type。 +- **Correct graph wording**: Relations remain Block-to-Block and know nothing about the `sources` table。A Source producer + reads its own `sources.block` identity and uses that BlockRef as the `from`/`to` endpoint when writing provenance；it is + inaccurate to say that the graph itself “uses sources.block”。 +- **Actual benefit**: shared identity structurally enforces the one-Block sidecar topology and removes a reverse mapping + when resolving a Source Block to a live SourceModel。Withdraw the overstated claim that separate IDs would force every + cross-boundary operation to dynamically distinguish two kinds of references。 +- **Confidence**: Sir accepted the shared identity subject to Block-first allocation/no Source sequence，and corrected the + graph-producer boundary and the earlier exaggerated two-ID cost。 + +### D-244 — Source and Source Block retain distinct IDs；the Block stores the historical Source descriptor + +- **Supersedes D-243 identity topology**: the shared-PK/FK design has insufficient ROI after correcting its overstated + mapping cost。SourceModel retains its own `sources.id` / SourceRef；Source Block retains its ordinary `blocks.id` / + BlockRef。 +- **Projection binding**: add `sources.block BIGINT NOT NULL UNIQUE REFERENCES blocks(id) ON DELETE RESTRICT`。The column + has no default、identity or sequence。`NOT NULL + UNIQUE + FK` proves exactly one valid Source Block per operational + Source and at most one live Source projection binding per Block。 +- **Lifecycle fit**: the two IDs name objects with different lifetimes：the operational Source may be deleted while the + historical graph entity persists。Deleting a live Source Block is restricted；deleting SourceModel does not affect the + Block。 +- **No runtime ambiguity**: Source producers read `SourceModel.block` and use that BlockRef as a Relation endpoint；reverse + resolution uses the unique `sources.block` index。No caller dynamically guesses whether an integer is a SourceRef or + BlockRef。 +- **Exact Source Block contract**: resolver ID is `core.source.v1` and canonical content is: + + ```json + { + "id": 17, + "type": "extensions.mail.imap.Source", + "nickname": "Work Mail" + } + ``` + + `id` is the historical SourceRef；`type` is the exact Source type；`nickname` is optional user-authored naming。Block ID + and Block timestamps remain Block fields and do not enter this content。 +- **Nickname authority**: remove `sources.nickname`；Source Block content is its sole authority so the name survives + operational Source deletion without projection synchronization。Source config、state and `collect_at` remain solely in + SourceModel。 +- **Derived liveness**: active/deleted state is not copied into Block content。A unique SourceModel whose `block` points to + this Block means operationally live；absence means historical projection。 +- **Creation pressure**: content now requires Source ID while SourceModel requires a non-null BlockRef。The implementation + must allocate and persist both rows atomically without giving `sources.block` a sequence。Current client-web performs a + direct PostgREST `INSERT sources`，so the exact shared creation command/RPC boundary is the next edge。 +- **Confidence**: after explicit ROI reassessment，Sir approved separate IDs and required the Source ID in canonical Block + content while retaining the previously proposed resolver、type/nickname and authority boundaries。 + +### D-245 — Source Block is a lazy one-to-one graph anchor，not a mandatory Source-creation projection + +- **Supersedes D-240/D-244 mandatory timing**: a Source may exist without a Source Block until a graph producer actually + needs a provenance endpoint。`sources.block` is nullable；Source creation remains an ordinary Source-row insertion and + does not require a cross-table RPC。 +- **Cardinality**: `sources.block BIGINT NULL UNIQUE REFERENCES blocks(id) ON DELETE RESTRICT`。A Source has zero or one + current Source Block binding；once materialized，the binding is stable and no Source may bind a second Block or share the + same live anchor。PostgreSQL UNIQUE correctly permits multiple null Source rows。 +- **Anchor positioning**: the Block is best named/understood as the Source's info-base anchor。Its existence means this + operational Source has needed a graph identity/provenance endpoint，not merely that a Source row was configured。 +- **Lazy command**: before first writing `collects` / `manages` provenance，the producer asks SourceManager to ensure the + anchor。The manager locks the Source row，returns the existing BlockRef when present，or creates `core.source.v1` content + `{id, type, nickname}` and sets `sources.block`。Concurrent producers must converge on one anchor。 +- **Collection invariant**: no newly collected graph may omit Source provenance merely because `sources.block` started + null。The producer ensures the anchor as part of entering the graph；nullable means “not yet needed”，not “optional after + collection”。Collected-item ledgers remain excluded from Source state。 +- **Lifecycle retained**: deleting SourceModel leaves an existing anchor and provenance graph intact。A Source that never + entered the graph leaves no useless Block on deletion。 +- **Cross-peer consequence**: current client-web PostgREST `INSERT sources` can remain a single-table operation。Withdraw + D-244's pressure for a mandatory shared `create_source` RPC and the source-ID/Block-ID allocation cycle at initial Source + creation。 +- **Reopened nickname edge**: because a pre-anchor Source still needs a user-facing nickname，D-244's proposal to remove + `sources.nickname` cannot stand unchanged。The exact live authority versus historical anchor snapshot semantics are the + next decision。 +- **Confidence**: Sir proposed relaxing mandatory projection，positioned Source Block as an on-demand anchor and retained + one-to-one Source/Block correspondence when an anchor exists。 + +### D-246 — SourceModel remains authority；Source anchor content is only a resolver-friendly projection + +- **Corrects D-244 nickname ownership**: do not remove `sources.nickname`。SourceModel remains the authority for Source ID、 + type、nickname、config、state and schedule while the operational row exists。Calling the Block a projection precludes + making that projection the upstream authority。 +- **Projected content**: `core.source.v1` content remains `{id, type, nickname}`，copied from SourceModel when the lazy + anchor is materialized。These fields exist so Source Resolver can implement stable `get_label()` / `get_text()` directly + from Block content and the historical anchor remains intelligible after Source deletion。 +- **No authority succession**: deleting SourceModel does not cause the projection to “become the authority”。The upstream + operational fact no longer exists；the surviving Block is a last-known historical projection/provenance record。 +- **Derived duplication is explicit**: duplicate `id/type/nickname` is admitted because it serves an independently durable + graph projection。It must not be described as two peer authorities or used to write operational Source state backward + from Block content。 +- **Use boundary**: Source Resolver label/text should not require a live SourceModel lookup merely to interpret the anchor； + live capability/config access remains a separate SourceManager concern。 +- **Remaining synchronization edge**: normal Source mutation/anchor-use paths should keep an existing projection useful， + but exact refresh timing must be chosen proportionally；do not add a database trigger/RPC merely because a manually + bypassed write could leave low-impact descriptive content stale。 +- **Common-pattern pressure**: an independently durable projection may copy authority-owned facts for bounded downstream + use，but projection status never transfers ownership。This belongs in shared technical authority guidance after + implementation evidence。 +- **Confidence**: Sir corrected the authority inversion and clarified that projected nickname/type exist only to simplify + Source Resolver `get_label/get_text`。 + +### D-247 — SourceManager.ensure_block owns lazy anchor creation and projection refresh + +- **Shallow command**: `SourceManager.ensure_block(source, session) -> BlockRef` guarantees that the supplied Source has + one bound Source anchor and that its projected `{id, type, nickname}` equals the SourceModel facts observed by this call。 + Callers do not branch on created/existing or implement projection synchronization。 +- **Concurrency**: the command locks the Source row，reuses `source.block` when present and creates/binds exactly one + `core.source.v1` Block when null。Concurrent producers converge on the same anchor through the row lock plus unique FK。 +- **Refresh is part of the postcondition**: for an existing anchor，compare canonical content and update only when the + authority-owned projection fields changed。Do not add a separate `refresh` flag here；“current projection at ensure time” + is the command's stable meaning，not an optional reload effect。 +- **Transaction ownership**: the command uses the caller-supplied database session。A producer may create/refresh the + anchor and write the first `collects` / `manages` relation in the same transaction；SourceManager does not commit behind + the caller。 +- **Use separation**: `core.source.v1.get_label/get_text` read Block content only。They do not query live SourceModel or + write the graph。Operational Source access continues through SourceManager。 +- **Proportional consistency**: normal producer use repairs stale descriptive projection and an actual Block content update + participates in existing updated-at/embedding invalidation mechanics。Do not add trigger、RPC、background synchronizer or + retry merely to cover manual writes that bypass admitted Source mutation/use paths。 +- **Confidence**: Sir explicitly accepted the combined existence/current-projection postcondition and caller-session + transaction boundary。 + +### D-248 — Mailbox Blocks remain Source-scoped；cross-Source reconciliation targets Email，not Mailbox + +- **Stable boundary，not MVP deferral**: one Mailbox Block represents one mailbox as observed/managed through one Source + access context。Two Sources produce distinct Mailbox Blocks even when evidence suggests that they expose the same remote + mailbox。 +- **Canonicalization rejected**: do not merge Mailbox Blocks across Sources。Proving mailbox equality is expensive and + provider/protocol scoped；non-identity fields such as name、attributes or access behavior may disagree；the resulting + canonical Mailbox offers little downstream value relative to the reconciliation complexity。 +- **Useful reconciliation retained**: comparable `MAILBOXID` scope plus `UIDVALIDITY + UID` may still prove that occurrences + observed through two source-scoped Mailboxes belong to the same Email Block。`Message-ID` remains the best-effort logical + Email reconciliation rung。Each Mailbox then keeps its own `contains` relation to that shared Email Block。 +- **Graph truth**: each Mailbox Block has one owning `Source Block --manages--> Mailbox Block` provenance path。The graph + preserves multiple real access paths instead of collapsing them into one mailbox object with conflicting bindings。 +- **Rename value**: within one Source scope，stable MAILBOXID can prove continuity across mailbox rename and update the same + Mailbox Block。Absent exact continuity evidence，a renamed/replaced mailbox may form a new Block rather than fuzzy merge。 +- **Organization exclusion**: collection does not ask Organization to repair this deliberate source-scoped model by merging + Mailboxes。A future independently useful cross-mailbox relation may be added only for an actual use case；it does not + replace the observed Blocks。 +- **Common-pattern pressure**: when canonical identity is costly、non-ID facts conflict and merge gives little use value， + retain source-scoped observations and reconcile only the downstream entity where the benefit is concrete。 +- **Confidence**: Sir accepted the source-scoped model as a long-term boundary and emphasized the poor ROI and conflict + handling cost of Mailbox merge。 + +### D-249 — Canonical Mailbox is a Source-scoped protocol projection with scoped MAILBOXID + +- **Exact resolver/content**: resolver ID is `extensions.mail.mailbox.v1`。Canonical content is + `{name, delimiter, attributes, mailbox_id}`；`delimiter` and the whole `mailbox_id` object are nullable。 +- **Mailbox vocabulary**: use protocol `name` rather than inventing a filesystem `path`。`attributes` contains deduplicated、 + stably sorted semantic LIST attributes；discard transient synchronization hints `\\Marked` / `\\Unmarked` rather than + churning the Block for no use value。 +- **Scoped Object ID**: when present，`mailbox_id` is + `{value, access_scope: {host, port, username}}`。Never persist/compare a bare RFC 8474 value。The non-secret access fields + are best-effort comparison evidence，not a claim that configured username is a protocol-proven MailAccount identity； + uncertain scope prevents exact cross-Source occurrence reconciliation。 +- **Authority placement**: SourceRef remains in the `manages` graph path；UIDVALIDITY/UID remain in the `contains` + occurrence relation。Do not copy either into Mailbox content。 +- **Explicit omissions**: no message counts、namespace classification、derived path or duplicated selectability/ + subscription booleans。They currently lack independent use value or duplicate `attributes` and would create volatile or + competing facts。 +- **Resolver use projection**: `get_label()` returns mailbox name；`get_text()` projects name plus semantic attributes and + omits access-scope descriptors from semantic retrieval。 +- **Confidence**: Sir explicitly accepted the exact proposed shape，including the scoped `mailbox_id` object and exclusions。 + +### D-250 — Mail collection performs source-native semantic decomposition into a graph + +- **Amends the accepted Email root**: `extensions.mail.email.v1` root content is reduced to nullable + `{message_id, subject, authored_at}`。Plain-text and HTML authored bodies are independent semantic content Blocks rather + than two nested representations inside Email content。 +- **Independent body blocks**: ordinary decoded `text/plain` / `text/html` bodies become independently usable Blocks；their + Email-body role is graph-owned rather than nested in root content。Whether they directly use existing semantic content + resolvers or require any Mail-owned metadata layer remains part of the exact next-edge restraint review。 +- **MIME part boundary**: MIME defines multipart bodies as body parts，while attachment/inline is presentation disposition。 + An attachment or non-text inline part with independently useful filename、declared media type、disposition、content ID、 + source locator or lifecycle is a Mail-owned MIME-part metadata Block even when its bytes are not collected。 +- **Existing authority pattern reused**: initial collection may leave that metadata Block without a materialized child。 + Later materialization adds `metadata Block --content--> semantic content Block -> Storage`；the metadata Block owns + source/protocol facts，the semantic content Block owns resolver identity and inline/pointer content，and Storage owns bytes。 + The retired “real/raw content” vocabulary does not return。 +- **Native decomposition pattern**: when a Source already has trustworthy native semantics for useful components，collection + should admit an ordinary graph rather than flattening everything into one root Block or waiting for Organization to infer + the same structure。This improves independent retrieval、organization and graph use without moving collection into the + Organization lifecycle。 +- **Restraint**: decomposition stops where a component lacks independent use、relation、identity or lifecycle。Do not dump a + MIME parser AST into the info-base or persist pure multipart/container mechanics unless they carry presentation/structure + semantics that a real use path needs。 +- **Open exact grammar**: direct semantic body versus metadata wrapper、body/part relation content、MIME ordering/alternative + grouping and the body-part metadata schema are the next technical edge；this decision freezes decomposition/ownership，not + those exact forms。 +- **Durable promotion pressure**: shared collection Product TDD should own “collect graph，not just Block” plus the + source-native decomposition/restraint test；it composes with the existing metadata → semantic content → storage pattern。 +- **Confidence**: Sir explicitly moved text/HTML bodies into Blocks，recognized attachment as a MIME body-part candidate，and + promoted source-informed decomposition as a common collection/product-design pattern。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D251-D260.md b/tasks/knowledge-lifecycle-capabilities/decisions/D251-D260.md new file mode 100644 index 0000000..020ae36 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D251-D260.md @@ -0,0 +1,206 @@ +# Decisions D-251–D-260 + +### D-251 — Email body representations directly reuse core semantic content Blocks + +- **Exact body resolvers**: decoded ordinary `text/plain` and `text/html` authored bodies persist as `core.text.v1` and + `core.html.v1` Blocks。Do not add `EmailTextBody` / `EmailHtmlBody` resolver types or metadata wrapper Blocks。 +- **Role placement**: the owning Email Relation makes a generic semantic content Block an Email body representation。The + core Block remains independently retrievable/reusable without absorbing Mail provenance or role into its content。 +- **Why split**: each body representation has independent reading、retrieval、embedding、organization and graph value。The + decomposition is source-native and useful even if no other producer ever reuses the core resolver。 +- **Reuse as evidence，not cause**: clean reuse of an existing deep module is a strong signal that a proposed component + boundary follows already-recognized semantics；it is not sufficient justification for decomposition。Do not manufacture + entities whose only benefit is code reuse，nor distort meaningful source structure to fit a reusable abstraction。 +- **Metadata restraint**: charset/transfer encoding used only to decode the collected body do not earn a durable wrapper or + separate lifecycle。If future source-native metadata gains independent use value，it must pass the metadata-Block test on + its own rather than retroactively redefining the body content authority。 +- **Confidence**: Sir explicitly accepted direct `core.text.v1` / `core.html.v1` reuse and identified reuse as an important + decomposition signal while warning against decomposition for reuse's sake。 + +### D-252 — Mail body and MIME-part relations preserve bounded source order + +- **Exact extension-owned grammar**: Email body/component owner Relations use `body:<order>`、`attachment:<order>` and + `inline:<order>`。Each role has an independent contiguous zero-based sequence；the target resolver/content owns the actual + representation or metadata kind。 +- **Body meaning**: `body:<order>` enumerates the selected main authored alternatives in increasing MIME preference；the + last supported representation is preferred。A single ordinary body uses `body:0`。 +- **Component meaning**: attachment and inline order preserve source encounter/presentation order within their own role。 + HTML-to-inline binding uses MIME Content-ID metadata rather than ordinal coincidence。 +- **Bounded fidelity**: do not persist a complete MIME tree、alternative-group IDs or multipart containers in the MVP。 + Source-native part locators remain metadata facts for remote fetch（the IMAP adapter maps its section path）。A specific + multipart container may later earn a Block only when actual rendering/retrieval/materialization pressure shows its + grouping semantics cannot be represented otherwise。 +- **Why order earns persistence**: unlike an arbitrary component ordering，MIME mixed/alternative ordering has protocol + presentation/preference meaning and is available at negligible collection cost。The relation grammar reuses the proven + extension-owned `role:<order>` pattern without modifying generic Relation schema。 +- **Confidence**: Sir explicitly accepted the proposed three relation grammars and their bounded no-parser-AST scope。 + +### D-253 — Canonical Mail MIME-part separates fetch metadata from materialized content facts + +- **Exact resolver/content**: resolver ID is `extensions.mail.mime_part.v1`。Canonical content is + `{part_id, media_type, charset, disposition, filename, content_id, description, transfer_encoding, encoded_size, + content_location}`；only `part_id` and normalized effective `media_type` are required。 +- **Adapter-neutral locator**: `part_id` is the source-native identifier/locator within the Email。The IMAP adapter maps its + body-section path；future Mail adapters may map their own native part identifier。The owning Source interprets it during + delegated remote fetch，so the canonical schema does not expose an IMAP-only `section` field。 +- **Protocol/source facts**: disposition preserves nullable source value independently from the canonical owner-relation + role；filename is the adapter-selected canonical filename；Content-ID drops surrounding brackets；description and content + location preserve optional rendering/reference facts。 +- **Decode facts**: `charset` is the one promoted MIME parameter with a proven need for text/HTML transcoding；do not retain + the arbitrary parameter bag。`transfer_encoding` and non-negative `encoded_size` are nullable canonically；IMAP records + BODYSTRUCTURE's effective encoding and transfer-encoded octet count。 +- **No actual-byte claim**: encoded size is not decoded semantic-content byte size。Detected MIME、decoded size、checksum、 + dimensions and duration remain byte-derived Resolver facts after materialization。BODYSTRUCTURE MD5、language、line count、 + arbitrary parameters and disposition timestamps remain excluded until a use path earns them。 +- **Materialization path**: the MIME-part metadata Resolver selects an exact core semantic resolver through ResolverManager、 + writes decoded bytes through configured WritableStorage and adds one `content` relation；Storage still owns bytes only。 +- **Confidence**: Sir explicitly accepted the complete proposed shape and authority exclusions。 + +### D-254 — EmailAddress is a shared address entity while participant meaning belongs to Relations + +- **Exact resolver/content**: retain `extensions.mail.email_address.v1` and hard-cut its content to + `{"address":"Local.Part@example.com"}`。The Block does not own a display name、participant role or message occurrence。 +- **Useful reconciliation**: reconcile EmailAddress Blocks across messages and Sources by exact canonical addr-spec。Unlike + Mailbox equality，this has a low-cost stable key and concrete graph-navigation/retrieval value after contextual names are + removed from the shared entity。 +- **Canonicalization boundary**: parse one valid addr-spec；preserve local-part Unicode and case；normalize equivalent quoted + forms to minimally quoted serialization；lowercase an IDNA A-label DNS domain；retain a canonical bracketed address + literal。Do not Unicode-normalize local parts or apply provider-specific plus-tag、dot-folding or alias policy。 +- **Relation-owned participation**: From、Sender、Reply-To、To、Cc and Bcc are all represented as directional Email → + EmailAddress Relations。The relation owns the role、source occurrence order and optional message-authored display name； + none is copied into Email root content or used to mutate a global preferred name。 +- **Identity restraint**: distinct addresses remain distinct even when they likely identify one person。Future + Organization/linking may represent that interpretation without weakening collection-time exact identity。 +- **Projection**: `get_label()` and `get_text()` use the canonical address only；Solved Email rendering obtains contextual + participant names from the owning Relations。 +- **Confidence**: Sir explicitly accepted the EmailAddress shape/identity placement and emphasized that recipient、sender + and carbon-copy facts must all be EmailAddress + Relation graph facts。 + +### D-255 — Participant Relations preserve role、order and contextual display name + +- **Exact direction/content**: every participant occurrence is an Email → EmailAddress Relation whose content is canonical + compact JSON `{role, order, display_name}`。`role` is exactly one of `from`、`sender`、`reply_to`、`to`、`cc`、`bcc`； + `order` is required and non-negative；`display_name` is nullable。 +- **Occurrence authority**: each role owns an independent zero-based source-order sequence。One address may have multiple + Relations when it appears in multiple roles；the occurrence Relation，not the shared address Block，owns its decoded + display name。 +- **Protocol meaning retained**: From and Sender are not collapsed。To、Cc and Bcc remain distinct destination facts，but + collection records only Bcc values actually observable in the message and never infers stripped/undelivered recipients。 +- **Structured string rationale**: Relation.content remains the generic storage surface，while the Mail extension owns a + canonical JSON schema/serializer because display names cannot be safely embedded in a delimiter grammar。 +- **Bounded group fidelity**: actual addr-spec members of an RFC address group become ordinary participant Relations，but + the MVP does not retain group labels or empty groups。They are not communication endpoints and currently lack enough + renderer、navigation or retrieval return to justify a Block/hyperedge model。 +- **Confidence**: Sir explicitly accepted the complete participant grammar including omission of address-group labels。 + +### D-256 — Native references create incomplete Email anchors and later complete the same graph node + +- **Exact reply grammar**: In-Reply-To produces reply Email → referenced Email `parent:<order>` Relations；References + produces referencing Email → referenced Email `reference:<order>` Relations。Each header owns a contiguous zero-based + sequence；the same target may carry both source-distinct facts。 +- **Reference-known target**: when a valid semantic Message-ID names an Email not yet collected，create/reconcile the same + `extensions.mail.email.v1` Block with `{message_id, subject:null, authored_at:null}`。Do not introduce a placeholder + resolver、unresolved-reference table or Source-state ledger。 +- **Later completion**: actual collection uses the accepted Message-ID ladder，updates the existing root facts and adds the + body/participant/mailbox/provenance graph。Inbound Relations remain stable because the domain node never changes identity。 +- **Fidelity boundary**: only parsed semantic msg-id values earn anchors。Malformed header residue is not promoted merely + for wire fidelity；Collection neither infers missing thread links nor rewrites pathological source-native cycles。 +- **Source reference-anchor pattern**: trustworthy stable external reference may justify an identity-bearing incomplete + domain Block before content acquisition，then normal reconciliation completes that node。This lets Source collection + represent useful graph structure without inventing a parallel placeholder lifecycle。 +- **Remote-content analogy，not type collapse**: the pattern resembles content remaining on a remote storage authority，as + Email bytes remain on IMAP until requested；that analogy does not make an IMAP Source a generic Block Storage or change + the accepted Source/Resolver ownership boundary in this unit。 +- **Confidence**: Sir explicitly accepted the reply/reference grammar and incomplete-Email strategy，identified its + storage-like character and promoted it as a reusable Source collect pattern while accepting the current bounded model。 + +### D-257 — Mailbox occurrence persists only the locator required for on-demand access + +- **Exact direction/content**: one current IMAP occurrence is a Mailbox → Email Relation with canonical compact JSON + `{type:"contains", uid_validity, uid}`。Both protocol integers are required and positive；together with the Source-scoped + Mailbox they locate the remote occurrence。 +- **Multiplicity**: a Mailbox/Email pair may have multiple contains Relations when Message-ID reconciliation maps multiple + remote UIDs to one Email Block。The relation represents a concrete occurrence，not a collapsed boolean membership。 +- **Removal/move**: reliable remote removal deletes only the exact occurrence Relation。Move is old occurrence removal plus + new occurrence addition；neither deletes the Email graph nor creates a tombstone。 +- **UID epoch reset**: when UIDVALIDITY changes，delete stale old-epoch occurrence Relations、reset permitted Source + cursor/validator state and incrementally rebuild through later ordinary bounded collect jobs。No one job must finish a + mailbox and no job retry semantic is introduced。 +- **INTERNALDATE correction**: do not persist INTERNALDATE。It is neither identity nor a fetch locator，and no current + renderer/query/Organization path consumes it。Source may use it transiently for IMAP queries or backfill bounds without + copying it into graph authority。 +- **Collection value test**: source-native availability is not persistence value。A fact enters the collected graph only + when it serves identity、reconciliation、later on-demand access、an accepted use path or independently valuable structure。 +- **Confidence**: Sir challenged INTERNALDATE under the demand-driven collection principle，then explicitly accepted the + reduced shape and UIDVALIDITY reset behavior and requested the same over-collection audit across Mail content/relations。 + +### D-258 — Canonical Mailbox retains user role and identity，not transient LIST structure/access scope + +- **Exact reduced content**: supersede D-249's wider shape with `{name, special_uses, mailbox_id}`。`name` is required； + `special_uses` is a normalized list；`mailbox_id` is a nullable bare OBJECTID value。 +- **Value-bearing fields**: name supports display、remote operations and Source-scoped fallback identity；recognized + standards-backed special-use roles such as Sent/Drafts/Junk/Trash/Archive support collection/use without localized-name + guessing；MAILBOXID supports rename continuity inside the owning Source。 +- **Removed delimiter**: the full mailbox name is sufficient for current IMAP operations，and no accepted hierarchy + renderer/query consumes the LIST delimiter。Source may use it transiently during discovery。 +- **Narrowed attributes**: do not persist generic LIST attributes such as subscription、selectability、child hints or + transient marked state。Only adapter-understood special-use roles with product meaning enter `special_uses`；unknown + extension attributes do not persist merely because the server returned them。 +- **Removed access scope**: permanent Source-scoped Mailbox identity and `Source --manages--> Mailbox` already own scope。 + Copying host/port/username into the Block is redundant、can become stale and no longer supports cross-Source merging under + D-248。 +- **Projection**: `get_label()` uses name；`get_text()` may include recognized special uses，but never credentials-like + access descriptors or transient structural flags。 +- **Confidence**: Sir explicitly accepted the reduction after requesting a cross-model over-collection audit。 + +### D-259 — MIME tree position belongs to the owner Relation and replaces duplicate component order + +- **Unified component grammar**: supersede D-252's `body:<order>`、`attachment:<order>` and `inline:<order>` with canonical + compact JSON `{role, part_id}`，where role is exactly `body`、`attachment` or `inline` and part_id is the numeric MIME-tree + path such as `1.2`。 +- **One structural authority**: MIME numbering already identifies the Email-relative component position、preserves source + order and maps to the IMAP section fetch locator。Compare parsed numeric segments，not strings；do not persist another role + order。`(Email Block, part_id)` is the bounded identity，not a global MIME-part reconciliation key。 +- **Correct placement**: part_id moves out of the MIME-part Block because it is meaningful only relative to the owning Email。 + The metadata Resolver obtains it from its incoming owner Relation for later on-demand fetch。 +- **Reduced MIME metadata**: supersede D-253 by removing `part_id` and `disposition` from Block content。Exact content becomes + `{media_type, charset, filename, content_id, description, transfer_encoding, encoded_size, content_location}`；media_type + is required and the other facts remain nullable according to their protocol semantics。 +- **Description retained**: Content-Description is MIME-authored human semantic metadata，not generated file summary；it has + label/text/retrieval value before bytes are materialized。 +- **Body-part labels versus references**: Content-ID and Content-Location remain intrinsic target labels。A collected HTML + body that resolves one also creates HTML body → MIME-part metadata `{type:"embeds", reference}`，whose reference is the + authored body URI。The label and contextual edge are distinct authorities。 +- **Media terminology**: IMAP returns body type/subtype separately；MIME calls their normalized `type/subtype` Content-Type + value a media type。`media_type` is therefore the canonical field name，not a claim that IMAP has one wire field token。 +- **Other fields**: charset/transfer_encoding support later exact decode/transcode；filename labels/downloads；encoded_size + bounds pre-download display/policy；content labels support body rendering。BODYSTRUCTURE availability alone still does not + justify excluded disposition、MD5、language、line count、arbitrary parameters or timestamps。 +- **Confidence**: Sir clarified that part_id was being tested as an order authority，accepted Content-ID/Location as both + attributes and Relations、confirmed media_type and explicitly accepted the unified `{role, part_id}` replacement。 + +### D-260 — Exact IMAP occurrence is the sole collection identity for Email Blocks + +- **Identity hard cut**: supersede D-256's Message-ID completion/reconciliation behavior and D-257's many-occurrences-per- + Email multiplicity。One exact remote occurrence identified by `(Source-scoped Mailbox Block, UIDVALIDITY, UID)` creates or + reuses exactly one Email Block；distinct occurrence locators never reconcile to the same Email Block during Collection。 +- **Accepted duplication**: identical content、the same RFC Message-ID or the same server EMAILID across different UIDs or + Mailboxes still produces distinct Email Blocks。UIDVALIDITY reset invalidates the old locator epoch；re-discovered messages + create new Blocks。Mailbox move/copy likewise creates the destination occurrence's own Block while previously collected + information remains。This deliberately prefers a bounded stable collection key over low-confidence content deduplication。 +- **Email root evidence**: exact canonical content becomes + `{message_id, email_id, subject, authored_at}`；all fields are nullable。`email_id` preserves the bare server-native + OBJECTID EMAILID when available，but neither it nor Message-ID participates in collection identity or uniqueness。 +- **OBJECTID boundary**: keep OBJECTID capability consumption and MAILBOXID rename continuity。EMAILID is fetched/persisted + as optional source evidence only；THREADID remains out of scope。 +- **Flag topology consequence**: because one Email Block represents at most one live collected occurrence，a Mailbox-scoped + MailFlag may directly tag that Email without repeating UIDVALIDITY/UID or introducing MailOccurrence。The exact flag/state + vocabulary remains the next edge。 +- **Reference consequence**: Message-ID remains useful authored metadata and reply/reference evidence，but it can no longer + create an incomplete Email that later absorbs an occurrence。The reply/reference target model must be redesigned before + implementation；D-256's relation intent remains evidence while its anchor/completion mechanism is withdrawn。 +- **Risk review**: this decision is confirmed but explicitly open to evidence-driven reconsideration after comparing the + systemic duplicate-Block cost against the much rarer same-Mailbox canonical-occurrence/flag ambiguity in the superseded + design。Reconsideration must be a new decision rather than silently rewriting D-260。 +- **Confidence**: Sir explicitly chose mailbox-local stable occurrence identity over content deduplication，accepted + duplicate Email Blocks and requested optional EMAILID persistence while rejecting EMAILID/Message-ID reconciliation。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D261-D270.md b/tasks/knowledge-lifecycle-capabilities/decisions/D261-D270.md new file mode 100644 index 0000000..0354041 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D261-D270.md @@ -0,0 +1,284 @@ +# Decisions D-261–D-270 + +### D-261 — Canonical Email separates remote occurrences；all durable IMAP flags use locator-qualified Relations + +- **Canonical/occurrence split**：supersede D-260's one-Email-per-locator hard cut。An Email Block again represents the + best-effort canonical authored email，while each exact remote occurrence remains a distinct Mailbox → Email Relation + located by `(Source-scoped Mailbox, UIDVALIDITY, UID)`。Several locators may therefore point to the same Email Block。 +- **Collection authority retained**：the exact locator remains the first idempotency and remote-access authority；canonical + reconciliation only decides the Email endpoint for a previously unseen locator。The exact `EMAILID` / Message-ID + reconciliation order remains a separate pending edge；no content fingerprint is introduced。 +- **Accepted topology**： + + ```text + Mailbox --contains {type:"contains", uid_validity, uid}--> canonical Email + Mailbox --has--> MailFlag + MailFlag --tags {type:"tags", uid_validity, uid}--> canonical Email + ``` + + `contains` owns only current membership and its exact locator。It does not also store Seen、Answered、Deleted or another + flag；sharing the locator does not make those facts one lifecycle。 +- **One flag representation**：IMAP `\Seen`、`\Answered`、`\Flagged`、`\Deleted`、`\Draft` and observed keywords all + use MailFlag Blocks plus locator-qualified `tags` Relations。Their product behaviors differ，but IMAP exposes and mutates + them through the same FLAGS/STORE category；that difference does not justify two persistence shapes。 +- **Occurrence precision**：a MailFlag belongs to one Mailbox vocabulary and `tags` qualifies one application with the + exact UID epoch/UID。This preserves correct state when one canonical Email has multiple occurrences，including multiple + occurrences in the same Mailbox。A tag Relation without the matching `contains` occurrence is invalid Source output。 +- **Deleted lifecycle**：`\Deleted` remains an ordinary flag fact while the occurrence still exists。After reliable EXPUNGE + evidence，the Source removes that occurrence's `contains` and locator-qualified flag Relations；it does not reinterpret + `\Deleted` itself as absence or as a tombstone。 +- **Recent exclusion**：do not persist deprecated `\Recent`。It has session-derived rather than durable mailbox-state value； + collection does not turn it into a long-lived graph fact。 +- **Retained D-260 evidence**：Canonical Email content still includes nullable `email_id` alongside Message-ID、subject and + authored time。The empirical duplicate-risk study remains decision evidence and now justifies the correction；D-260 stays + in history but no longer governs identity or flags。 +- **Reference consequence**：canonical Email makes Message-ID reference anchors viable again，but D-256 is not silently + restored。Exact canonical reconciliation and incomplete-anchor completion must be reviewed together as the next edge。 +- **Common pattern**：facts in one source protocol category that share address scope and mutation mechanism should default + to one graph representation。Different UI meaning or business behavior belongs to resolver/use/action policy unless it + proves a genuinely different authority or lifecycle。 +- **Confidence**：Sir explicitly selected `canonical Email + locator-qualified MailFlag Relation` and then required Seen、 + Answered and Deleted to remain flags rather than being mixed into `contains`。 + +### D-262 — Plain MailFlag Relations preserve the common path；same-Mailbox duplicate occurrences do not reconcile + +- **Targeted correction**：supersede only D-261's locator-qualified `tags` content。Retain canonical Email、exact + `contains {uid_validity,uid}` occurrence locators、unified MailFlag representation and no MailOccurrence Block。 +- **Exact topology**： + + ```text + Mailbox --contains {type:"contains", uid_validity, uid}--> best-effort canonical Email + Mailbox --has--> MailFlag + MailFlag --tags--> best-effort canonical Email + ``` + + The MailFlag's incoming `has` Relation already supplies Mailbox scope；therefore plain `tags` loses no precision across + different Mailboxes。 +- **Source-owned invariant**：one `(Mailbox Block, Email Block)` pair has at most one live `contains` Relation。When a second + UID in the same Mailbox matches the same EMAILID/Message-ID evidence，Collection creates a separate Email Block rather + than reconciling it to that endpoint。This is a Mail producer invariant，not a generic InfoBase relation rule。 +- **Best-effort canonical boundary**：cross-Mailbox reconciliation remains available and owns the high-value deduplication + path；only duplicate occurrences inside one Mailbox give up canonical collapse。Canonical Email is therefore deliberately + best-effort，not a global uniqueness claim。 +- **Exact remote operation**：from `MailFlag --tags--> Email`，resolve the owning Mailbox through `Mailbox --has--> MailFlag` + and then the unique `Mailbox --contains--> Email` Relation。Its UIDVALIDITY + UID is the remote STORE locator。Zero or + multiple matches is invalid Mail Source output and must never be guessed into a remote mutation。 +- **One flag lifecycle**：`\Seen`、`\Answered`、`\Flagged`、`\Deleted`、`\Draft` and observed keywords all use plain + `tags`；deprecated `\Recent` remains excluded。Removing a flag removes its tag Relation。Reliable EXPUNGE removes the + exact `contains` Relation and the ordinary tags whose Mailbox/Email scope depended on it。 +- **Why this dominates the prior candidates**：it retains cross-Mailbox canonicalization、exact remote actions、uniform + MailFlag semantics and ordinary graph Relations，while removing locator duplication、cross-Relation JSON references and + UID noise from generic Relation text/semantic retrieval。The narrow lost case degrades into duplicate Email Blocks rather + than ambiguous mutable state。 +- **Evidence calibration**：the directional Enron sample observed no strict same-owner/same-folder duplicate-content group， + while cross-folder duplication was material。This is not a protocol guarantee；it supports spending complexity on the + common cross-Mailbox path and handling the rare same-Mailbox case through bounded non-reconciliation。 +- **Common pattern**：when an exact representation burdens every common-path fact to protect a rare case，prefer a simple + representation plus an explicit producer invariant if the rare case can degrade safely and locally。Do not trade broad + semantic/maintenance cost for low-marginal-return canonicalization。 +- **Confidence**：Sir explicitly accepted the plain-tag topology and its same-Mailbox non-reconciliation cost after comparing + the four explored occurrence/flag models，and judged the result to combine their useful properties best。 + +### D-263 — Email reconciliation is one linear strongest-to-weakest evidence ladder + +- **Recovered authority**：retain D-239/D-248's accepted exact-occurrence and Message-ID rungs；D-260 only temporarily + superseded their use，not their historical decision authority。D-261/D-262 restore best-effort canonical Email。 +- **Exact linear order**： + + 1. same Source-scoped Mailbox + UIDVALIDITY + UID → reuse the known local occurrence endpoint； + 2. adapter-proven comparable OBJECTID scope + same MAILBOXID + UIDVALIDITY + UID → reuse the same exact remote + occurrence observed through another Source-scoped Mailbox； + 3. adapter-proven comparable OBJECTID scope + same EMAILID → reconcile identical immutable message content； + 4. same valid semantic Message-ID → best-effort logical Email reconciliation； + 5. no usable match → create another Email Block。 + +- **D-262 guard**：before accepting any cross-occurrence candidate，exclude an Email already connected to the current + Mailbox by another live UID。This keeps ordinary MailFlag Relations exact；the same authored content may safely become a + second best-effort canonical Email Block inside one Mailbox。 +- **EMAILID placement**：EMAILID follows exact occurrence matching because it proves immutable-content equality，not the + same remote occurrence；it precedes authored Message-ID because it is a stronger server assertion within its valid + namespace。Bare EMAILID values are never compared across arbitrary Sources。 +- **Linear execution**：evaluate rungs in order and stop at the first usable match。Do not combine evidence into scores、 + majority votes or fuzzy confidence；do not add a content fingerprint fallback。A stronger exact rung is not demoted merely + because a weaker identifier is absent。 +- **Flag compatibility**：RFC 8474 permits occurrences sharing EMAILID to have different keywords。That does not weaken the + rung：Mailbox-scoped MailFlags and D-262's same-Mailbox guard keep mutable state separate from immutable-content + reconciliation。 +- **Still-open failure edge**：multiple eligible candidates at one rung、contradictory stronger/weaker evidence and partial + anchor completion need one conservative collision policy；they do not reopen the ladder order。 +- **Reference consequence**：D-256 Message-ID-only incomplete Email anchors use this same ladder when later collected，rather + than a separate reference-specific reconciliation algorithm。 +- **Confidence**：Sir confirmed the recovered ladder and accepted scoped EMAILID between exact occurrence and Message-ID， + emphasizing that the resulting ladder remains sufficiently clear and linear。 + +### D-264 — A reconciliation rung may reuse an existing Email only after exact-one resolution + +- **Cardinality contract**：after scope checks and the D-262 same-Mailbox eligibility guard，one D-263 rung can resolve to + zero、one or multiple candidate Email Blocks。Only exactly one candidate authorizes reuse。 +- **Mail execution rule**：zero candidates means this rung did not locate an existing endpoint，so continue to the next + weaker rung；one candidate returns it；multiple candidates stop the ladder and create another Email Block。Do not choose + by oldest/minimum Block ID and do not ask weaker evidence to arbitrate stronger ambiguity。 +- **Identifier precision**：zero and multiple both fail to locate one existing local entity，but they do not mean the + source-native token ceases to be an identifier in every domain。Zero may mean the externally identified object has not + been created locally；multiple means uniqueness is violated in the current comparison scope。In both cases the token may + remain durable evidence but cannot authorize existing-entity reuse for this operation。 +- **No hidden evidence composition**：continuing after a multiple-match rung would turn the linear ladder into implicit + cross-rung scoring/intersection。That is a different reconciliation model and is not introduced merely to reduce harmless + duplicate Blocks。 +- **Safe degradation**：Collection still needs an endpoint for the newly observed occurrence，so its shallow outcome is + create。Other domains may preserve an unresolved reference、skip or report a typed internal outcome；the common rule is + only “do not act on an existing entity without exact-one resolution”。 +- **Reference caveat**：D-256 reference-anchor behavior must apply the same exact-one rule，but its no-match/ambiguous shallow + outcome is reviewed separately because a reference observation does not yet carry collected Email content requiring a + new complete endpoint。 +- **Common pattern**：identifier usability is scoped and cardinality-based。A reference/reconciliation/mutation boundary + may act on an existing entity only when the identifier resolves to exactly one eligible referent；never hide zero/many + behind an arbitrary deterministic pick。 +- **Confidence**：Sir accepted immediate create on an ambiguous Mail rung and identified the exact-one resolution rule as a + reusable identifier design pattern。 + +### D-265 — Null identity facts may complete；contradictory non-null identity facts reject cross-occurrence reconciliation + +- **Compatibility runs after location**：D-264 exact-one resolution identifies one candidate endpoint；it does not by itself + prove that every newly observed identity fact is compatible with that candidate。Compatibility is a post-location guard， + not another ladder rung and not evidence scoring。 +- **Identity completion rule**：for `message_id` and a scope-comparable `email_id`，existing null + incoming non-null fills + the missing fact；existing non-null + incoming null preserves the existing fact；equal non-null facts are compatible。 + Collection never erases useful identity evidence merely because one observation omitted it。 +- **Contradiction rule**：when both sides provide different non-null values for the same comparable identity fact，reject + that cross-occurrence candidate and create another Email endpoint for the newly observed occurrence。Do not overwrite an + identity value and do not continue to weaker rungs to make the contradiction disappear。 +- **Concrete cross-checks**：an EMAILID-rung candidate with a different non-null Message-ID is incompatible。A Message-ID-rung + candidate with a different non-null EMAILID is incompatible only when the adapter has proved the EMAILID comparison + scope；bare EMAILIDs from incomparable scopes cannot contradict one another。 +- **Exact occurrence exception**：known/comparable exact occurrence locators are remote-occurrence authority rather than + best-effort logical reconciliation。They keep their existing endpoint even if a later observation reports contradictory + identity evidence；the producer must not fork one exact occurrence or overwrite the stored identity merely to conceal a + data-integrity/adapter inconsistency。 +- **Content is not identity**：`subject` and `authored_at` remain canonical Email content but are not ladder identifiers。 + Their null/non-null update policy does not become a content fingerprint or a reconciliation veto through this decision。 +- **Reusable mechanism pressure**：the ladder now has enough high-value common mechanics to justify a future Source-domain + utility。It should own ordered async rung execution、zero/one/many cardinality、short-circuiting and rung-labelled typed + outcomes such as matched/ambiguous/contradictory/exhausted。Each Source/adapter still owns comparison scope、candidate + queries、eligibility、identity compatibility and the command's shallow outcome；the utility must not become a cross-Source + identity-policy god object。Its exact API/name waits for implementation preflight rather than being invented in docs。 +- **Common pattern**：reconciliation may complete absent identity evidence but must not overwrite comparable contradictory + identity evidence。Extract repeated ladder orchestration only after its common mechanics are clear，while leaving domain + evidence semantics and effects with the domain owner。 +- **Confidence**：Sir explicitly accepted null completion/non-null identity contradiction rejection，confirmed the subtle + distinction has no adverse effect on the current Mail use，and identified the reconciliation-ladder utility as a + high-ROI future abstraction。 + +### D-266 — Reply/reference anchors use the same locate-then-reuse-or-create model + +- **Restored D-256 mechanism**：a parsed semantic Message-ID in In-Reply-To/References targets an ordinary + `extensions.mail.email.v1` Block。There is no placeholder resolver、unresolved-reference table or Source-state ledger。 +- **Exact-one reuse**：resolve the Message-ID within the eligible Email candidate scope。Exactly one candidate means the + relation targets that Email。The reference observation does not use subject/time inference、content fingerprints or an + arbitrary persistence-order tie-breaker。 +- **Zero and ambiguous create**：zero candidates and multiple candidates both mean Collection cannot locate one existing + referent，so create a new incomplete Email Block with + `{message_id:<value>,email_id:null,subject:null,authored_at:null}` and target it。Do not drop the source-native reference、 + point at all candidates or pretend one ambiguous candidate is authoritative。 +- **Bounded duplicate trade-off**：an ambiguous Message-ID may therefore produce another incomplete anchor，and repeated + ambiguous observations may produce more than one best-effort node。This is a safe local degradation：it preserves the + authored graph fact without manufacturing false equivalence；later Organization may merge when justified。 +- **Later collection**：the ordinary D-263 ladder locates an anchor just like any other Email。Exact-one plus D-265-compatible + facts reuses and completes the same Block by filling root facts and adding body、participant、mailbox and provenance + graph。Multiple matching anchors remain ambiguous，so Collection creates another complete Email rather than rewriting + existing reference targets under uncertainty。 +- **No anchor state machine**：“incomplete” is descriptive，derived from the ordinary Email's currently available facts and + graph。It is not a persisted status、separate lifecycle or special reconciliation rung。 +- **Common pattern**：identifier-driven ingestion follows `locate → exact-one reuse / otherwise create` unless the domain + command has a concrete reason to skip or preserve a non-entity unresolved value。A reference observation changes the + shape of the created entity，not the safety rule for reusing an existing one。 +- **Confidence**：Sir accepted zero/ambiguous anchor creation because reference handling is the same locate-and-reuse model + and benefits from behavior consistent with ordinary Collection reconciliation。 + +### D-267 — MailFlag content retains semantic description in addition to its name + +- **Canonical shape direction**：MailFlag is not name-only。Its content includes the flag/keyword `name` and a nullable + `description` so the Block remains semantically useful outside Mail-specific UI。 +- **Value**：description contributes to resolver text、semantic retrieval、LLM graph interpretation and generic inspection， + just as MIME Content-Description makes an attachment metadata Block useful before bytes are materialized。A currently + missing dedicated consumer does not erase that basic semantic value。 +- **Authority distinction remains explicit**：MIME-part description is source-authored message content；Base IMAP FLAGS does + not transmit a description for each flag/keyword。MailFlag description must therefore come from a standards-backed or + provider/adapter-owned semantic mapping when available，or remain null。It must not be documented as a raw IMAP flag fact。 +- **Still under review**：whether adapter-supplied canonical description is the sole authority，and how later provider- + supplied/user-authored descriptions would coexist，must be frozen before the complete MailFlag contract closes。 +- **Confidence**：Sir explicitly required persisted MailFlag description and connected its use value to the already + accepted attachment-description decision。 + +### D-268 — MailFlag canonical content and description authority are frozen；observed FLAGS is authoritative state + +- **Exact resolver/content**：use exact resolver ID `extensions.mail.flag.v1` and canonical content + `{name:string,description:string|null}`。Do not add kind、system/permanent/mutable booleans、Mailbox/Source references or + description provenance fields。 +- **Mailbox-scoped identity**：one Mailbox owns at most one MailFlag Block per ASCII case-insensitive flag name through + `Mailbox --has--> MailFlag`。Different Mailboxes retain different Blocks even for `\Seen`，because their tag relations + must derive one exact remote occurrence scope。 +- **Name projection**：known system flags and registered keywords use their standards-backed spelling。Unknown keywords + preserve their first observed spelling for readable generic rendering while subsequent identity comparison remains ASCII + case-insensitive；a casing-only later observation does not create or rename a Block。 +- **Description ladder**：a source/provider-native semantic description wins when an adapter can actually obtain one； + otherwise known standard flags/keywords use stable、non-localized standards-backed descriptions；unknown values remain + null。Do not infer prose from an arbitrary keyword token。 +- **Authority**：the owning Mail adapter selects this canonical description。Base IMAP does not provide per-flag + descriptions，so it is adapter-produced canonical semantic metadata，not a falsely claimed FLAGS wire field。Mailbox → + Source provenance already identifies the adapter；duplicating `description_source` would add derivable authority。 +- **No user-authority collision**：Source refresh may reproduce its canonical projection，but user-authored interpretations + do not edit this description。They belong to additional info-base graph facts unless a future remote provider action + proves a different authority model。 +- **Authoritative occurrence snapshot**：when Collection obtains the complete FLAGS list for one exact occurrence，the + filtered set（excluding deprecated `\Recent`）is authoritative for that observation：ensure MailFlag/`tags` for present + names and remove existing tags absent from the snapshot。This is replacement/reconciliation，not append-only ingestion。 + How scheduled jobs discover changed old occurrences remains a separate sync-capability edge。 +- **Confidence**：Sir accepted provider-native → standards-backed → null description authority after explicitly requiring + description to persist for the same semantic-use reason as attachment description。 + +### D-269 — Scheduled Mail synchronization follows QRESYNC → CONDSTORE → new-occurrence-only capability degradation + +- **QRESYNC path**：when the server/mailbox supports QRESYNC with persistent mod-sequences，use CHANGEDSINCE flag deltas and + VANISHED removal evidence to reconcile already-collected occurrence state without a full mailbox scan。 +- **CONDSTORE-only path**：use CHANGEDSINCE to discover existing-occurrence metadata/flag changes。Do not claim reliable + remote removal discovery；CONDSTORE alone still requires UID FETCH/SEARCH comparison to find expunges。 +- **Base path**：without either capability，ordinary scheduled collection discovers new occurrences and their current flag + snapshots only。It does not repeatedly scan every UID to simulate old flag/removal synchronization。 +- **Remote action coherence**：when InKCre itself successfully changes a remote flag，the action updates the corresponding + graph fact directly；it does not wait for a later scheduled job to rediscover its own effect。 +- **Known-UID authority**：QRESYNC known UIDs derive from live Mailbox `contains` Relations。Do not reintroduce a collected- + item ledger in Source state merely to build the protocol argument。 +- **Checkpoint discipline**：advance a mailbox's HIGHESTMODSEQ only after applying the returned delta。If one one-shot job + aborts/fails first，a later ordinary job uses the old checkpoint and idempotently observes the delta again；this is not a + job retry/reopen lifecycle。 +- **Deletion boundary**：this ladder implements the earlier product rule that remote removal is synchronized only through + reliable protocol-native evidence。No fallback full UID inventory traversal is smuggled in as “incremental sync”。 +- **Placement clarification pending**：a QRESYNC/CONDSTORE checkpoint needs prior UIDVALIDITY even for an empty Mailbox，but + occurrence Relations already carry UIDVALIDITY as locator evidence。The exact distinction/placement of those two uses is + the next narrow review；it does not reopen the accepted capability ladder。 +- **Confidence**：Sir accepted the full capability ladder and then correctly challenged the imprecise statement that + “UIDVALIDITY goes in Source state”。 + +### D-270 — UIDVALIDITY persists separately as occurrence-locator evidence and sync-checkpoint validation + +- **Occurrence placement retained**：every live `Mailbox --contains {uid_validity,uid}--> Email` Relation keeps the UID epoch + required to interpret that exact remote occurrence locator。It remains useful after Source state/credentials disappear and + preserves the historical epoch instead of being reinterpreted through a later Mailbox state。 +- **Checkpoint placement**：Source state owns a per-Source-scoped-Mailbox synchronization checkpoint containing the last + accepted UIDVALIDITY alongside HIGHESTMODSEQ and the forward new-UID collection cursor。That UIDVALIDITY validates whether + those cursors may be reused on the next SELECT/QRESYNC cycle。 +- **Why derivation is insufficient**：an empty Mailbox、a Mailbox whose final occurrence was expunged or a graph from which + all membership Relations were removed still needs its prior synchronization epoch。No `contains` Relation exists from + which the checkpoint validator can then be recovered。 +- **Same value，different fact/lifecycle**：Relation UIDVALIDITY is occurrence locator evidence；checkpoint UIDVALIDITY is a + precondition for Source-owned cursor reuse。A current epoch commonly gives them the same numeric value，but neither copy + can replace the other's authority or lifecycle。 +- **Mailbox content remains clean**：do not place UIDVALIDITY、HIGHESTMODSEQ or collection cursors in the Mailbox Block。 + Frequent sync updates would turn operational state into info-base content and cause unrelated Block timestamp/embedding + invalidation。 +- **No table yet**：a dedicated mailbox-sync-state table does not currently earn its lifecycle/complexity over existing + Source state，which is explicitly allowed to retain cursors and validators。Implementation preflight may reopen this only + with concrete concurrency/size/query evidence。 +- **Confidence**：Sir accepted the two placements after confirming their distinct scopes、empty-Mailbox requirement and + lifecycle behavior。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D271-D280.md b/tasks/knowledge-lifecycle-capabilities/decisions/D271-D280.md new file mode 100644 index 0000000..cd988bf --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D271-D280.md @@ -0,0 +1,192 @@ +# Decisions D-271–D-280 + +### D-271 — Message-ID-only reconciliation lazy-duplicates Emails that carry remote MIME components + +- **Correction to runtime guessing**：withdraw the proposal that MIME materialization choose an occurrence by comparing + “obviously matching” metadata and trying another UID when it appears different。Metadata similarity does not prove byte + identity，and the wrong bytes could be durably promoted as authoritative semantic content without detection。 +- **Eligibility guard**：known/comparable exact occurrence and scoped EMAILID rungs may reuse a canonical Email because they + establish the same occurrence or identical immutable message content。At the weaker Message-ID rung，if either the + existing Email graph or the incoming occurrence contains attachment/inline MIME metadata，the candidate is ineligible and + Collection creates another Email Block。 +- **Sparse-anchor exception**：a D-266 Message-ID-only reference anchor has not asserted source-native content or a remote + MIME tree，so the first actual occurrence may still reuse and complete it。The guard applies after an Email has acquired + collected content/occurrence semantics，not merely because its Message-ID was referenced。 +- **No blanket UID rule**：D-262 remains narrower for flags：a second UID in the same Mailbox cannot reuse one Email，while + different Mailboxes can share it because MailFlag scope stays exact。D-271 adds a separate weak-identity guard only where + remote MIME byte access would otherwise lose exactness；it does not make every different UID a new Email。 +- **Materialization consequence**：if one Email has several live occurrence locators and remote MIME components，the accepted + producer invariant means those locators were reconciled through exact occurrence/EMAILID evidence。Resolver may therefore + select an operational locator without guessing semantic equality；no per-part fetch-binding layer or metadata heuristic is + required for this MVP。 +- **Failure-risk criterion**：heuristic acceptability depends on probability、harm、detectability、recoverability and user + correction cost together。A rare wrong choice is still unacceptable when it silently writes incorrect durable content and + offers no natural recovery path。A visible duplicate that Organization can later merge is safer than an invisible false + semantic-content assertion。 +- **Product-interaction boundary**：do not compensate for missing provenance by asking users to “try another UID/MIME part”。 + That would expose protocol ambiguity as an incoherent product interaction and make users debug internal Source topology。 +- **Confidence**：Sir accepted the Message-ID MIME eligibility guard，rejected metadata-based guessing as unstable and + explained that its rare but severe、undetectable and effectively unrecoverable failure makes it unacceptable。 + +### D-272 — Mail Source and Resolver are sibling clients of a shared protocol adapter + +- **Correction**：withdraw the proposed MIME Resolver → operational Mail Source invocation edge。Collection and + materialization are separate domain behaviors；neither behavior becomes the other's protocol facade。 +- **Topology**：Mail Source and Mail MIME Resolver both depend on an extension-owned protocol adapter。The Source uses it + while orchestrating collection/synchronization；the Resolver uses it while materializing an exact remote MIME part。 +- **Config boundary**：each caller resolves and validates the operational access config it needs and passes that typed config + to the adapter。The Resolver may follow Source/Mailbox provenance to the operational Source row as data authority，but it + does not instantiate or invoke the Source implementation。The adapter does not accept a SourceRef and become the owner of + database lookup。 +- **Adapter ownership**：the adapter owns connection、authentication and protocol mechanics。It does not own Source state or + checkpoints、collection policy、identity reconciliation、graph persistence、Storage selection or semantic materialization + policy。 +- **Caller ownership**：the Source retains mailbox scope、collection horizon、checkpoint、reconciliation and collection-graph + production。The Resolver retains graph-context/locator selection and the Storage + semantic-graph materialization effects。 +- **Protocol extensibility**：IMAP is the current concrete adapter，not the permanent Mail domain boundary。A future POP3 or + other Mail protocol can provide another adapter without making its Resolver call that protocol's Source implementation。 + This does not require the current unit to pre-design POP3 locator or collection semantics。 +- **Storage vocabulary**：`WritableStorage` is already a real core capability class：`Storage` owns reading and + `WritableStorage(Storage)` adds transactional create/update/delete operations。It is not merely prose shorthand，and no + symmetric `ReadableStorage` abstraction is introduced without separate value。 +- **Confidence**：Sir corrected the dependency direction to Source / Resolver → common adapter，confirmed caller-owned typed + config and identified future POP3 support as the architectural reason to preserve this seam。 + +### D-273 — Mail is the Source/Resolver identity while IMAP and POP3 are adapter choices + +- **Correction to protocol-shaped domains**：do not create parallel IMAP Source / IMAP Resolver and POP3 Source / POP3 + Resolver families。There is one Mail Source type and one protocol-neutral family of Mail Resolvers；each configured Mail + Source instance selects one concrete protocol adapter。Resolvers that need remote access recover that same selection from + Source provenance。 +- **Stable topology**：`Mail Source -> selected Mail protocol adapter` and `Mail Resolvers -> selected Mail protocol adapter` + are the stable dependency directions。IMAP is the only current delivery adapter；POP3 is a future adapter option，not a + second Mail domain。 +- **What remains common**：canonical Email、EmailAddress、body/MIME metadata graph、reply/reference graph、Source Block + provenance、Resolver projections、Storage materialization and client rendering remain Mail-domain contracts。They are not + duplicated per protocol。 +- **IMAP-specific reassignment**：UIDVALIDITY/UID occurrence evidence、MAILBOXID/EMAILID、QRESYNC/CONDSTORE，IMAP FLAGS， + SELECT/EXAMINE and `mark_as_seen` wire behavior belong to the IMAP adapter path inside the current Mail Source behavior。 + Their accepted current-delivery semantics remain valid；they are no longer presented as requirements every future Mail + adapter must implement。 +- **Deferred compatibility**：the current IMAP locator relation shape remains the only current delivery shape。A future + POP3 adapter may earn a protocol-discriminated locator variant/version；do not pre-design that grammar or widen current + forms merely for hypothetical compatibility。 +- **Existing-code consequence**：the PoC's IMAP-specific `Source` class/config is failure and migration evidence，not the + target ownership shape。The behavior rewrite may hard-cut it into a Mail Source plus IMAP adapter without compatibility + preservation。 +- **Confidence**：Sir explicitly supplied the single Mail Source / Mail Resolver family → IMAP or POP3 adapter topology and + allowed the protocol-specific topology only if preserving the former would invalidate most prior work。Review shows that + most product/canonical-graph decisions remain intact；the change primarily reassigns protocol-specific technical facts。 + +### D-274 — Protocol adapters interpret checkpoints while Mail Source owns their durable lifecycle + +- **Adapter authority**：each Mail protocol adapter declares and understands its protocol-specific checkpoint schema， + interprets the current validated checkpoint and returns a proposed next checkpoint with its collection result。IMAP may + therefore own UIDVALIDITY/HIGHESTMODSEQ/cursor transition mechanics；a future POP3 adapter may own UIDL mechanics。 +- **Source authority**：Mail Source owns the durable Source-state slot、collection intent、accepted-effect boundary and the + decision to persist the proposed checkpoint。A proposal is not durable progress merely because adapter I/O completed。 +- **No persistence inversion**：the adapter does not query or mutate Source rows and does not commit state。Conversely，the + generic Mail Source does not branch on or reinterpret protocol-native checkpoint fields；doing so would reduce adapters + to shallow network clients and scatter protocol logic into the domain owner。 +- **Validation placement**：the selected adapter exposes the typed checkpoint model needed by the caller/runtime validation + path。The adapter receives validated config/state snapshots and returns a typed proposal；it does not duplicate application + validation or database lifecycle。 +- **Confidence**：Sir explicitly accepted the split：Adapter interprets and proposes checkpoint transitions，while Source + decides and persists them。 + +### D-275 — Mail Source persists a public protocol choice，not an internal adapter identity + +- **Identity correction**：each Mail Source instance corresponds to one public Mail protocol such as IMAP or POP3。The + protocol is an external standard fact，not an InKCre capability identity and not something registered as + `extensions.mail.protocol.<name>.v1`。 +- **Adapter distinction**：an IMAP/POP3 adapter is code-owned machinery implementing/translating the corresponding public + protocol for the Mail domain。Its Python class or implementation version is not persisted merely to select protocol + behavior。Protocol-standard version/capability negotiation remains wire behavior inside that adapter。 +- **Selection consequence**：Mail Source config durably states one protocol and its access parameters。Mail Source and a + remote-I/O Mail Resolver derive the code adapter from that protocol through one shared explicit construction seam；they do + not each branch independently。The exact config/factory shape remains the next design edge。 +- **No invented registry**：with the currently closed protocol set and one implementation per protocol，do not add a + `MailManager`、runtime adapter registry、persisted adapter catalog or versioned adapter ID。A future need for multiple + replaceable implementations of one protocol must provide its own ROI before changing this boundary。 +- **Clarification of D-273**：D-273's “selected protocol adapter” means the code implementation derived from the Source's + selected standard protocol，not a second durable selection field or protocol-shaped Source/Resolver identity。 +- **Confidence**：Sir rejected the proposed versioned adapter registry，emphasized one protocol per Source and distinguished + public IMAP/POP3 standards from InKCre's adapter implementations。 + +### D-276 — Mail Source config separates public protocol from typed parameters + +- **Protocol field**：`MailProtocol` names the public standard choice `imap | pop3`。It is a typed value in Mail Source + config，not an InKCre table/entity，registered capability or namespaced/versioned exact ID。 +- **Config shape**：Mail Source config has sibling `protocol` and `parameters` fields。`protocol` discriminates the typed + schema of `parameters`；common Mail-domain policy such as mailbox exclusion remains outside the protocol parameter object。 +- **Separation rationale**：do not mix the selector/identity and the selected protocol's construction parameters in one + object namespace。The same structural rule already proved useful for Peer inbound interfaces，where protocol identity and + outbound-construction parameters have different authority and consumers。 +- **Identity-format distinction**：the common pattern does not force every protocol onto one identity scheme。Peer's + `core.peer.protocol.http.v1` is an InKCre-owned exact wire contract；Mail's `imap`/`pop3` are public standards。Both still + keep protocol and parameters structurally separate。 +- **Open implementation edge**：the exact Pydantic union，current-runtime support validation and shared code factory remain + Technical work；D-276 does not reintroduce an adapter registry or promise POP3 implementation in this unit。 +- **Confidence**：Sir explicitly confirmed `MailProtocol == imap | pop3`，accepted the proposed config shape and recognized + it as a reuse of the earlier inbound-interface common pattern。 + +### D-277 — MailProtocol enumerates currently supported config values，not known future standards + +- **Correction to D-276**：the current exact type is `MailProtocol = Literal["imap"]`，not `Literal["imap", "pop3"]`。 + Knowing that POP3 is a public Mail protocol does not make it a currently supported Source configuration value。 +- **No speculative validity**：the typed config/schema must not accept a value whose parameters，adapter behavior and + acceptance contract do not exist。A source that validates but cannot collect would turn type-level capability truth into + a future roadmap claim。 +- **Expansion rule**：POP3 implementation later expands `MailProtocol`、the discriminated `parameters` union，adapter + construction and protocol acceptance together。JSON persistence requires no advance enum reservation or database + migration for that future change。 +- **Confidence**：Sir explicitly corrected the current type to `Literal["imap"]`。 + +### D-278 — A small create_mail_adapter seam centralizes protocol implementation construction + +- **Factory contract**：retain one extension-owned `create_mail_adapter(protocol, parameters)` function used by Mail Source + and every Mail Resolver that needs remote protocol I/O。It maps the validated public protocol choice and its typed + parameters to the corresponding code adapter。 +- **Current behavior**：with `MailProtocol = Literal["imap"]`，the function constructs `IMAPAdapter`。This remains useful + as the sole construction seam rather than making each caller depend on construction details。 +- **Future extension**：when POP3 becomes a real supported protocol，expanding the typed config plus one factory branch is + sufficient for adapter construction；callers retain the same dependency。 +- **Bounded abstraction**：the function is not a Manager，registry，catalog，persisted identity or runtime plugin system。It + owns only protocol/parameter pairing and adapter construction；connection lifetime，commands and output contracts remain + separate design edges。 +- **Confidence**：Sir explicitly retained `create_mail_adapter` as a low-cost，high-return abstraction whose future POP3 + extension is localized to the factory。 + +### D-279 — One domain command owns one async-context Mail adapter instance + +- **Construction boundary**：`create_mail_adapter(protocol, parameters)` is side-effect free and returns a fresh adapter。 + Network connection，authentication and protocol capability negotiation begin only on async-context entry。 +- **Lifetime**：one Source `collect` invocation uses one adapter/context across its mailbox traversal；one Resolver + materialization command uses its own short-lived adapter/context。Instances and connections are not shared across jobs， + commands or config snapshots。 +- **Cleanup invariant**：async-context exit owns deterministic logout/connection release on normal completion，exception or + cancellation。Callers do not duplicate `try/finally` protocol cleanup or understand partial connection state。 +- **No premature pool**：do not add a cross-command connection cache，singleton or pool now。If measured connection cost later + earns pooling，the adapter may hide it behind the same context contract without creating another public lifetime owner。 +- **Common-pattern pressure**：bind external-resource lifetime to the smallest meaningful domain-command scope using the + language's native resource mechanism。Keep factories free of I/O and let the deep adapter own acquisition/cleanup。 +- **Confidence**：Sir explicitly accepted one async-context adapter per command and identified it as an excellent use of + Python's resource-cleanup mechanism。 + +### D-280 — Mail adapters expose canonical Mail operations，not protocol primitives + +- **Deep boundary**：the adapter contains IMAP/POP3 wire mechanics and exposes Mail-domain acquisition/materialization + operations。Mail Source and Resolvers do not orchestrate `SELECT`、`SEARCH`、`UID FETCH`、QRESYNC，BODY sections or + equivalent protocol commands。 +- **Collection direction**：conceptually，the adapter accepts a typed Mail collection request plus its typed current + checkpoint and produces protocol-neutral Mail facts plus proposed checkpoint progress。Exact request/batch model names and + streaming granularity remain the next design edge。 +- **Remote-content direction**：conceptually，the adapter accepts a typed exact remote locator plus MIME `part_id` and + returns decoded bytes。Exact failure/output metadata remain open，but protocol transfer representation does not leak into + Storage or semantic resolver contracts。 +- **No persistence transfer**：the adapter returns no Block/Relation/GraphForm and performs no InfoBase or Source-state + writes。Mail Source retains reconciliation、canonical graph production/effects and checkpoint acceptance；Mail Resolver + retains semantic resolver selection、WritableStorage and materialized graph effects。 +- **Extensibility consequence**：future POP3 changes the adapter behind the canonical operations，not the Source/Resolver + domain family。A factory that merely hides construction would not achieve this if callers still consumed IMAP primitives。 +- **Confidence**：Sir explicitly accepted canonical Mail operations rather than protocol primitives as the adapter boundary。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D281-D290.md b/tasks/knowledge-lifecycle-capabilities/decisions/D281-D290.md new file mode 100644 index 0000000..c995f9a --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D281-D290.md @@ -0,0 +1,158 @@ +# Decisions D-281–D-290 + +### D-281 — Mail Source alone owns collect；Adapter exposes canonical remote access operations + +- **Correction to the D-280 sketch**：do not put `collect()`、`MailCollectRequest` or `MailCollectBatch` on MailAdapter。 + Collection is the Source-domain command that brings external information into InfoBase，not a protocol adapter behavior。 +- **Adapter surface**：the Adapter exposes remote-reading/change-stream and exact-part-fetch operations using canonical Mail + inputs/results while hiding IMAP primitives。Likely implementation vocabulary includes mailbox discovery，change reading + and part fetch；exact names/models are implementation-owned and no longer require design-by-design review。 +- **Checkpoint fit**：the Adapter may still interpret its typed checkpoint and pair canonical remote facts with proposed + progress。Mail Source calls those operations from `Source.collect()`，performs reconciliation/graph effects and alone + accepts/persists progress under D-274。 +- **Mail-client analogy boundary**：ordinary clients may call their overall behavior sync/fetch，but that vocabulary does not + transfer InKCre collection ownership。The adapter's remote access is subordinate machinery；it does not independently + synchronize a local InfoBase or complete a collection command。 +- **Delegated detail**：exact Adapter request/result/batching APIs may be finalized during implementation planning/preflight + without further review as long as D-272–D-280 remain intact：Source/Resolver sibling callers，public protocol + parameters， + code factory，per-command async resource scope，canonical Mail values，no graph/state persistence and no protocol primitive + leakage。Any pressure that changes those boundaries returns to discussion。 +- **Confidence**：Sir identified `collect` as Source semantics and delegated the remaining concrete MailAdapter interface + shape provided the previously frozen boundaries continue to hold。 + +### D-282 — Writable materialization Storage follows a Source-wide fallback policy + +- **Promotion from Mail pressure**：selecting the writable target for remote-content materialization is a general Source + concern，not an IMAP parameter or a Mail-only protocol fact。Mail MIME materialization is the first consumer that exposes + the common requirement。 +- **Resolution order**：use the Source instance's explicit writable Storage when present；otherwise use the + deployment-scoped default writable Storage；when neither is configured，use the always-available built-in PostgreSQL + binary Storage (`-4`) as the hard-coded final default。 +- **Missing versus invalid**：only absence advances to the next fallback。An explicitly selected Storage that is missing or + does not implement `WritableStorage` is a configuration/capability error；silently replacing it with a later default would + hide the operator's intended policy。 +- **Ownership boundary**：the target is local materialization policy and remains outside `protocol.parameters`。Mail Resolver + resolves the policy only after provenance identifies the exact Source/occurrence；the protocol Adapter never chooses or + writes Storage。 +- **Still active**：because this pressure is now Source-wide，the durable location of the per-Source explicit selection must + be settled before retaining it inside `MailSourceConfig`。 +- **Confidence**：Sir approved Source-configured target Storage and promoted a deployment default plus PostgreSQL binary + fallback as a general Source requirement。 + +### D-283 — Per-Source writable target is the nullable `sources.storage` reference + +- **Persistence**：promote the explicit target out of `MailSourceConfig` into nullable `sources.storage`。It is a common + Source-instance reference to `storages.id`，not a source-type-specific protocol/config fact。 +- **Meaning**：a non-null value is the Source instance's explicit materialization target；null means inherit D-282's + deployment default and then built-in PostgreSQL binary fallback。It does not mean materialization is disabled。 +- **Integrity**：the ordinary FK should use deletion `RESTRICT`，matching a Block's Storage reference；removing a selected + Storage first requires changing the Source policy rather than silently changing its destination。 +- **Deep capability seam**：Source-domain policy resolves the selected ID；StorageManager should provide one common + `get_writable_storage(...)`-like seam that returns the `WritableStorage` capability or raises a clear capability error，so + Memos、RSS、Mail and future Sources do not repeat `isinstance` checks。 +- **Confidence**：Sir approved promoting one `sources.storage` field。 + +### D-284 — Storage Registry owns code/catalog capability consistency；no defensive writable getter + +- **Correction to D-283's tentative seam**：do not add `StorageManager.get_writable_storage()` merely to rediscover or + defend against a catalog/class mismatch at each use。It has no independent domain command or useful call timing。 +- **System boundary**：Storage registration/bootstrap is the boundary that knows both the registered implementation class and + the persisted Storage capability projection。It must derive/validate writable support there，so ordinary use can rely on + one established invariant rather than repeat `isinstance` defense。 +- **Operational distinction**：`writable` means the implementation contract admits a write operation；it does not claim that + a concrete write will succeed against current credentials、quota、network or remote state。Those remain ordinary runtime + failures。 +- **Active modeling point**：because implementation capability is registered per Storage type，decide whether the durable + projection belongs on `storage_types` rather than duplicating it on every `storages` instance；then bind + `sources.storage` to that invariant。 +- **Confidence**：Sir rejected the defensive getter and fixed registry/system-boundary validation as the owner of + code/catalog consistency。 + +### D-285 — `storage_types.writable` is the persisted write-capability projection + +- **Location and spelling**：persist `writable` on `storage_types`，not `storages`。Use the established English spelling and + align it with the existing `WritableStorage` capability class。 +- **Authority**：the exact Storage type implementation contract owns whether writes are supported。Storage registry derives + the projection from the registered class (`issubclass(..., WritableStorage)`) and syncs/validates the catalog at the + registration/bootstrap boundary；individual instances inherit the capability from `storages.type`。 +- **Meaning**：`writable = true` says the Storage protocol/implementation admits write commands。It does not guarantee a + particular write will succeed under current credentials、network、quota or remote state。 +- **Source invariant**：a non-null `sources.storage` must reference a Storage instance whose referenced type is writable。 + Database integrity rejects creation/update that would establish a read-only target and rejects later Storage type or + capability changes that would invalidate an existing Source reference。 +- **Use path**：ordinary construction remains `StorageManager.get_storage()`；there is no parallel writable getter or repeated + catalog/class rediscovery at use time。 +- **Confidence**：Sir confirmed the column belongs to `storage_types` after recalling the existing type catalog。 + +### D-286 — MIME-part `content` Relation is the durable materialization authority + +- **Completion fact**：one outgoing `content` Relation from the MIME-part metadata Block to a core semantic content Block is + the durable fact that the part has been materialized。Neither a Source checkpoint、local Resolver cache nor Storage bytes + alone substitutes for this graph fact。 +- **Idempotency**：an existing exact Relation selects/reuses its semantic child；absence permits materialization；multiplicity + is graph-integrity failure rather than permission to choose an arbitrary child。 +- **Layering**：the metadata Block remains the source-authored remote-part projection；the child remains the actual semantic + content using D-282–D-285's writable target policy。The Relation is the additive bridge between them。 +- **Still active**：the concrete concurrent create sequence、shallow public completion/failure outcome and Solved Resolver + projection remain design-closure details；this decision freezes their durable authority，not every implementation step。 +- **Confidence**：Sir explicitly approved using the `content` Relation as materialization authority after the long design + review。 + +### D-287 — Existing MIME content short-circuits provenance routing；duplicate children are benign graph facts + +- **Correction to D-286 multiplicity**：one or more existing outgoing `content` Relations means content is already + materialized。The Resolver does not fail merely because more than one semantic child exists and never downloads another + child for that reason。 +- **Short circuit**：the Resolver checks existing children before traversing Email/Mailbox/Source provenance or resolving a + target writable Storage。Existing child Resolvers hydrate through each child's own persisted `block.storage`；that ordinary + content read is not Source-policy resolution。 +- **Concurrency**：the producer still uses a lock/recheck to reduce duplicate creation，but correctness does not depend on + exactly-one enforcement。A rare concurrent duplicate has low harm；all graph facts remain usable and later Organization + may merge/remove redundancy when doing so improves use。 +- **Ownership nuance**：Organization's ability to clean redundancy does not excuse an avoidable duplicate producer path；it + only makes residual benign concurrency a repairable info-base quality issue rather than a materialization failure。 +- **Confidence**：Sir corrected both the unnecessary Source/Storage traversal and the earlier graph-integrity treatment of + multiple children。 + +### D-288 — Resolver solving returns semantic completion，not command-mechanics status + +- **Stable informal contract**：`Resolver.get_solved_content(...)` returns the solved use-facing content after any permitted + lazy work。It does not expose whether internal mechanics created、reused、raced or fetched that result unless such a fact is + itself part of the domain's solved semantics。 +- **Depth**：this shallow completion contract prevents every Resolver caller from understanding internal materialization + algorithms and status algebra。Implementation diagnostics may retain those facts without expanding the public result。 +- **Documentation owner**：promote the contract into the Resolver base method/class docstring（and equivalent peer contract） + rather than repeating it as a Mail-only decision or relying on convention remembered by individual implementers。 +- **Confidence**：Sir identified this as a complexity-containment property of the existing Resolver contract，not merely a + choice to omit one Mail result field。 + +### D-289 — SolvedMimePart keeps singular content semantics while graph redundancy remains visible + +- **Correction to the plural proposal**：`SolvedMimePart` represents one MIME part's content，so its solved field is singular + `content: SolvedContentChild | None`。Returning a collection merely because graph concurrency can leave redundant + `content` Relations would leak producer residue and change the use-facing business cardinality。 +- **Representative selection**：when several valid children exist，materialization does not fail or create another child；the + Resolver chooses one stable representative（lowest persisted Relation identity is sufficient）and solves it。This is a + read/use projection over semantically interchangeable facts，not identifier reconciliation authorizing mutation。 +- **Graph retained**：the Resolver does not delete or hide the extra Relations from graph navigation。Organization may later + merge/remove redundant children to improve InfoBase quality；the shallow solved projection need not wait for cleanup。 +- **Direct content**：`SolvedContentChild` retains the child Block plus its Resolver's actual solved content，so callers receive + content rather than only a BlockRef while preserving resolver identity and navigation capability。 +- **Confidence**：Sir identified that plural solved output changed “one attachment's content” semantics and approved choosing + any valid child instead；stable deterministic choice is the low-cost implementation consequence。 + +### D-290 — Database integrity prevents `sources.storage` from selecting a read-only Storage type + +- **Invariant**：a non-null `sources.storage` must reference an existing Storage instance whose referenced + `storage_types.writable` is true。This is enforced at the shared PostgreSQL boundary，not left to a particular peer's form or + materialization call。 +- **PostgreSQL mechanism**：retain the ordinary FK to `storages.id` and use a constraint trigger for the cross-table capability + predicate。A PostgreSQL `CHECK` constraint cannot query `storages → storage_types` and therefore cannot express this + invariant without denormalizing writable state。 +- **Mutation closure**：the constraint also rejects changing a referenced Storage's type or registered type capability in a + way that would leave an existing Source targeting a read-only type。 +- **Registry remains authority**：Storage registry/bootstrap still derives and synchronizes `storage_types.writable` from the + registered implementation contract；the database constraint consumes that projection and does not rediscover Python + capability at use time。 +- **Confidence**：Sir required a database check preventing read-only Source targets；the constraint-trigger mechanism is the + PostgreSQL-valid realization of that check under the already approved type-level projection。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D291-D300.md b/tasks/knowledge-lifecycle-capabilities/decisions/D291-D300.md new file mode 100644 index 0000000..49a6e13 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D291-D300.md @@ -0,0 +1,172 @@ +# Decisions D-291–D-300 + +### D-291 — Singular graph reads do not promise stable selection or promote duplicate policy + +- **Correction to D-289 representative selection**：`SolvedMimePart.content` remains singular，but when several valid + outgoing `content` Relations exist，the Resolver may use any one of their target Blocks。It does not order by Relation ID + or promise that repeated reads select the same child。 +- **ROI boundary**：redundant children are a low-probability best-effort compromise，not an encouraged graph shape。Adding + stable-choice rules or duplicate-specific helpers would spread residue handling through ordinary use code and falsely + promote the compromise into a supported behavior family。 +- **Deep read seam**：InfoBaseManager provides one generic singular related-Block query，conceptually + `get_related_block(block, direction, relation_content) -> BlockModel | None`。Its database implementation may join/filter + the graph and apply `LIMIT 1` without `ORDER BY`；the contract promises neither uniqueness、ordering nor stable selection。 + Callers that need graph multiplicity continue to use the ordinary all-Relations query。 +- **Scope distinction**：this arbitrary singular projection is valid only where any matching fact satisfies the use-facing + meaning，as with semantically interchangeable MIME content children。It does not weaken D-264/U-034：identity + reconciliation or mutation still requires exact-one resolution and must not hide ambiguity behind this query。 +- **Materialization effect**：an existing child still short-circuits Source/provenance/target-Storage routing and is solved + directly。No duplicate detection、repair or representative-selection policy is added to the MIME Resolver；Organization may + independently reduce redundancy when that improves use。 +- **Confidence**：Sir rejected stable representative selection because its marginal value is too small and it would turn a + tolerated residual race into fragmented or apparently encouraged duplicate-handling behavior。 + +### D-292 — MIME-closure pattern review retains only five cross-unit candidates + +- **Retained promotion candidates**：retain U-040（fallback advances only on absence）、U-041（deep solving returns semantic + completion）、U-042（tolerated residue does not shape common APIs）、U-044（concurrency machinery follows expected harm） + and U-047（implementation-owned capabilities are projected once and durable references are data-boundary enforced）。 +- **Withdrawn promotion candidates**：withdraw U-043（completion-first production-path short circuit）、U-045（shared + matching mechanics versus domain evidence precedence）and U-046（semantic-model completion versus lower-level side + effects）from common-pattern promotion。Their concrete Mail design consequences remain valid under D-286/D-287 and the + MIME materialization contract；the correction concerns cross-unit elevation，not those local decisions。 +- **Selectivity rule**：a locally correct explanation is not automatically valuable durable guidance。Promotion requires + enough recurring decision leverage to justify another stable project-wide concept；otherwise implementation/design truth + stays with its narrower owner。 +- **Confidence**：Sir explicitly selected items 1、2、3、5 and 8 from the reviewed pattern list as worth retaining。 + +### D-293 — Collect owns its Job；the missing common capability is Cron + +- **Boundary correction**：`SourceCollectJob` remains a Collect-domain execution object with Collect-owned input、state、 + status and effects。Do not turn it into the generic schedule/firing ledger merely because scheduled collection is its + first recurring producer。 +- **Missing capability**：the current process-local APScheduler wiring is not a durable Cron mechanism：database schedule + edits are not hot-applied，different Peers can produce duplicate firings，and schedule/timezone/misfire authority is not + modeled。Introduce an MVP Cron capability rather than extending Collect Job with generic scheduling concerns。 +- **Withdrawn detail**：withdraw the tentative `sources_collect_jobs.scheduled_for` plus partial unique + `(source, scheduled_for)` design as the owner of firing convergence。Cron must own recurring definition and firing + coordination；a successful firing creates an ordinary Collect-domain Job through the Collect-owned command-creation seam。 +- **Still open**：freeze whether the MVP adds only generic Cron over domain-owned Jobs or also extracts a generic Job + lifecycle。Do not infer the latter merely from shared scheduling pressure。 +- **Confidence**：Sir identified that the existing Job is Collect-domain and the actual missing mechanism is Cron，while + inviting an MVP-level treatment instead of another Source-local patch。 + +### D-294 — Scheduled collection requires distributed occurrence materialization，not merely a Cron abstraction + +- **Problem correction**：the required behavior is periodic email collection。Collect Job exists because collection must run + in the background；the unresolved problem is how several capable Peers may observe one due schedule while creating no more + than one durable Collect Job for that occurrence。 +- **Legacy diagnosis**：`sources.collect_at` appeared sufficient only while core-py was accidentally the sole consumer。If + client-web or several core-py deployments consume the same schedule，each process-local scheduler can create a distinct + Job；claiming individual Job rows cannot repair that earlier duplication。 +- **Not the solution**：a generic Job lifecycle is not the core requirement，and renaming the process-local scheduler to a + Cron domain does not provide convergence by itself。 +- **Database boundary**：PostgreSQL must not autonomously evaluate schedules or implement domain Job creation。It may be the + shared serialization boundary used by application Peers，but the application remains responsible for determining a due + occurrence and materializing its domain command。This respects scale-to-zero operation and keeps business behavior out of + database procedures/triggers。 +- **Still open**：freeze the smallest application-plus-database protocol that atomically elects one materializer per canonical + schedule occurrence，creates the Collect-owned Job，and leaves execution to ordinary capable-Peer claiming。 +- **Confidence**：Sir traced the requirement from periodic Mail collection through background Collect Jobs and rejected both + generic-Job distraction and database-owned scheduling/business execution。 + +### D-295 — Active-Job uniqueness is the primary Peer-arbitration primitive；the Cron cursor records progress + +- **Primary convergence**：for a Cron-backed Source collect，the durable rule that one Cron may have at most one non-terminal + `SourceCollectJob` is the main database arbitration primitive when several Peers concurrently try to materialize work。It + also prevents a slow collection from accumulating overlapping Jobs。 +- **Cursor name and meaning**：Cron stores nullable `last_scheduled_for`，the canonical occurrence represented by the last + successfully materialized Job。Do not call it `last_run_at`：creation does not prove that the Job has started or completed， + and the scheduled occurrence must not drift to the materializer's wall-clock timestamp。 +- **Atomic boundary**：the winning Job insertion and `last_scheduled_for` advance commit in one transaction；a losing attempt + does not advance the cursor。Database constraints arbitrate application writes，while application code still evaluates the + schedule and constructs the Collect command。 +- **Separate guarantees**：non-terminal uniqueness arbitrates active overlap；the design must still close the stale-reader + window in which the first Job reaches a terminal status before another Peer acts on the old Cron cursor。The smallest exact + occurrence fence remains under review rather than being hidden by probabilistic timing assumptions。 +- **Confidence**：Sir accepted `last_scheduled_for` and identified one-non-terminal-Job-per-Cron as the core Peer-concurrency + mechanism，with overlap prevention as an additional benefit。 + +### D-296 — Global Cron persists a typed Job template；Source remains unaware of user scheduling composition + +- **No Collect binding table**：do not create `SourceCollectCronBinding`。A global Cron persists `job_type` plus typed + `job_parameters`，and `core.source.collect.v1` parameters already carry the called Source reference。The user composes Cron + and collect Job；Source merely executes collect when called and neither owns nor knows whether that call was scheduled。 +- **Correct creator direction**：Cron may hold nullable `last_job -> jobs.id` once Job is global。Job remains unaware of Cron、 + manual creation or any other producer。 +- **Distinct concurrency facts**：inside the locked Cron transaction，`last_scheduled_for` proves that the canonical occurrence + was already dispatched；`last_job` exposes whether the previously produced Job remains non-terminal and therefore prevents + later occurrences from accumulating overlapping Jobs。`last_job` alone cannot identify a just-dispatched occurrence。 +- **Global execution registry**：Job owns an exact-type Handler Registry as an internal `JobManager` capability，not a separate + manager。A registered definition supplies at least parameter validation、description and async handling；the durable + `job_types` catalog is its schema/description projection rather than executable authority。 +- **Cron scope**：Cron copies its typed Job template into a durable Job；it has no domain-target registry or generic invocation + protocol and does not execute Jobs。 +- **Confidence**：Sir rejected the redundant binding because Source does not own user scheduling，asked for the implied Job + Handler Registry，and accepted `last_job` as necessary for one-non-terminal-Job-per-Cron once its role was made explicit。 + +### D-297 — client-web becomes a real capable Job worker，not merely a Job CRUD surface + +- **Delivery requirement**：within `mail-extension`，`@inkcre/core` and client-web gain the ability to discover、atomically + claim、execute and conditionally close Jobs for locally registered handlers。The existing UI-only ability to insert and read + `sources_collect_jobs` is insufficient。 +- **Collect dispatch**：client-web requires a Source runtime registry so enabled extensions can register executable Source + implementations。The common `core.source.collect.v1` Job handler resolves the referenced Source and dispatches only when its + Source type is locally supported。 +- **Eligibility is parameter-sensitive**：having the generic collect Job handler does not make a Peer capable of every Source + type。Eligibility is checked before the atomic pending-to-running update；an ordinary browser must not claim Mail/IMAP work + without a registered Mail Source implementation。 +- **Browser boundary**：an open client-web may participate as a Peer worker，but browser lifetime cannot guarantee periodic + background availability。It is not the sole wake-up authority for durable Cron；this limitation does not reduce its real + claim/execute capability while active。 +- **Database boundary**：single-row conditional PostgREST updates can implement Job claim/close without a business RPC。This + decision does not yet require client-web to perform Cron's multi-row materialization transaction。 +- **Confidence**：Sir explicitly requires client-web Collect Job processing in this unit；repository evidence shows it + currently provides creation/view UI only and has no Source execution registry。 + +### D-298 — Cron uses one five-field UNIX expression under one deployment timezone + +- **Schedule representation**：`crons.schedule` is one standard five-field UNIX cron expression。The MVP does not invent an + interval union、seconds field or timezone-bearing Cron dialect。 +- **Timezone authority**：deployment config key `core.cron` owns one schema containing `timezone` as an IANA timezone name； + absence resolves to the hard-coded stable default `UTC`。Every Peer evaluates durable Cron rows against this same authority， + never its process、browser or operating-system timezone。 +- **Why not `CRON_TZ`**：timezone remains semantically distinct from the five-field expression；implementation-specific + crontab environment directives do not become InKCre's persisted protocol。Per-Cron timezone override is deferred until a + real single-deployment multi-timezone need appears。 +- **Change behavior**：changing `core.cron.timezone` changes the interpretation of every Cron on subsequent locked checks；no + cached process-local timezone authority or duplicated per-row projection is maintained。 +- **Confidence**：Sir accepted the five-field expression plus deployment-scoped timezone and UTC fallback。 + +### D-299 — Global Job persists registry-typed parameters/state JSON rather than domain detail tables + +- **Tables**：`job_types` projects exact type ID、description and `parameters_schema`；`jobs` owns an `int8` identity、type、 + JSONB parameters/state、status and lifecycle timestamps。Remove `sources_collect_jobs` rather than retaining it as a joined + Source-specific detail table。 +- **Deep-module boundary**：typed JSON is admitted because the runtime Handler Registry supplies the parameter model and + executable meaning for each exact type，while `JobManager` owns only durable background-command lifecycle、claim、dispatch + and conditional close。No arbitrary generic invocation API、retry、Cron provenance or domain checkpoint enters Job。 +- **Collect projection**：`core.source.collect.v1` parameters carry `source` and typed collect config；its handler owns Source + lookup and state semantics。Database FK coverage is intentionally traded for one cross-Peer command representation that Cron + can copy and both Python and TypeScript workers can validate/execute。 +- **Open execution edge**：the proposed boolean `can_handle(parameters)` is not frozen。Eligibility-to-claim races must be + resolved before the Handler definition is accepted；a Peer must not consume a Job merely because a capability disappeared + after a stale boolean check。 +- **Confidence**：Sir accepted registry-typed JSONB over domain detail tables and explicitly reopened `can_handle` because it + affects Job handling races。 + +### D-300 — Job eligibility is a best-effort pre-claim filter；atomic claim remains the concurrency authority + +- **Required order**：a worker reads a pending Job、validates its parameters、calls the local Handler's + `can_handle(parameters)` and only then attempts the atomic `PENDING -> RUNNING` claim。False means skip without mutating the + Job；a zero-row conditional update means another Peer won and the worker abandons its attempt。 +- **No post-claim eligibility gate**：do not claim first and then discover ordinary local incapability；that would consume work + another capable Peer could execute or require a reverse `RUNNING -> PENDING` transition。 +- **Accepted TOCTOU**：the local capability may disappear between a true check and claim/handling。Do not add execution leases、 + registry draining、requeue or retry machinery for this edge。A Cron-created failure is naturally superseded by a later + occurrence；a one-shot/manual Job visibly fails so the user can act。The expected harm does not justify stronger coordination。 +- **Scope of `can_handle`**：it answers local implementation eligibility，including parameter-dependent Source type support；it + does not probe IMAP/network/credentials or promise execution success。External unavailability remains ordinary Handler + failure。 +- **Confidence**：Sir clarified that the intended race was eligibility-before-versus-after-claim，selected the pre-claim + ordering and explicitly accepted the narrow capability-disappearance consequence on marginal-benefit grounds。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D301-D310.md b/tasks/knowledge-lifecycle-capabilities/decisions/D301-D310.md new file mode 100644 index 0000000..4e35c19 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D301-D310.md @@ -0,0 +1,174 @@ +# Decisions D-301–D-310 + +### D-301 — Cron coalesces missed active occurrences；disabled time creates no debt + +- **Active downtime**：when no Cron-capable Peer is awake across several occurrences，the first later check materializes at + most one Job for the latest due occurrence。It does not recreate every missed tick。 +- **Non-overlap continuation**：when `last_job` remains non-terminal，later occurrences are likewise coalesced。After that Job + closes，a later check may create one Job for the latest then-due occurrence。 +- **Disabled interval**：occurrences while a Cron is explicitly disabled are not debt。Re-enabling starts a new active window + and waits for an occurrence after that boundary rather than immediately compensating for disabled time。 +- **No retry reinterpretation**：a failed Cron-created Job remains failed；a later Job comes from a later schedule occurrence， + not retry state or replay of the failed Job。 +- **Persistence pressure**：the Cron model must preserve `last_scheduled_for` as the last actually materialized canonical + occurrence while also representing the start of the current active window。Do not silently overload one timestamp with both + meanings；the exact activation field/update rule remains to be frozen。 +- **Confidence**：Sir accepted coalesced active-period catch-up and no catch-up for manual disable/re-enable。 + +### D-302 — Correction：Cron has no misfire recovery；only the current matching occurrence may dispatch + +- **Supersedes D-301 catch-up**：withdraw active-period coalesced catch-up。If no capable Cron checker observes an occurrence + while its schedule minute is current，that occurrence is lost。Cron neither records misfire debt nor dispatches immediately + when a Peer later returns。 +- **Reference behavior**：ordinary cron checks whether jobs match the current minute and does not catch up downtime；Anacron or + persistent timers add that as a separate mechanism。Proxmox VE likewise makes `repeat-missed` opt-in with default false and + describes skip-missed as the expected default。InKCre adopts the simpler ordinary-cron behavior without an MVP option。 +- **Current-occurrence protocol**：a Peer derives the canonical current schedule minute in the deployment timezone and only + considers a Cron whose expression matches it。Under the Cron row lock，equal `last_scheduled_for` means that occurrence + already created a Job；a non-terminal `last_job` suppresses creation。Otherwise the Peer creates one Job and atomically sets + both fields。 +- **No `active_from`**：creation、enablement、definition edits and timezone edits require no catch-up boundary。They simply + affect subsequent checks；if the expression matches the minute currently being observed，that minute may run。 +- **Manual now**：run-now is an ordinary direct `JobManager.create(job_type, job_parameters)` using the Cron template。It does + not mutate Cron schedule progress or pretend that a missed occurrence ran。 +- **Confidence**：Sir explicitly replaced catch-up with “missed means missed；wait for the next occurrence”，asked for Unix + cron/PVE comparison and separated manual immediate dispatch from Cron semantics；external evidence supports that model。 + +### D-303 — Job owns its execution budget；Mail backfill is bounded resumable progress + +- **Reject Peer-coupled abandonment**：do not add `jobs.peer` or infer Job death from Peer discovery lease expiry。Peer + discovery、Cron materialization、Job execution and Mail collection are independent lifecycles；coupling Job terminal state to + Peer liveness would make ordinary long-running work depend on delegation infrastructure without solving the actual timeout + requirement。 +- **Job-owned limit**：replace the current universal five-minute running timeout with an explicit configurable execution + timeout owned by the Job envelope。The exact persistence/default/terminal-status shape remains to be frozen；it must not be + hidden inside Mail parameters or inferred from Peer state。 +- **Mail execution meaning**：a Mail collection/backfill Job advances as far as its execution budget permits。The Source owns + traversal and durable checkpoints，so a later independently created Job continues from the remaining frontier rather than + replaying already checkpointed work。 +- **No completeness or retry claim**：timeout、load limits or a finite invocation do not make an entire historical horizon a + must-success/must-fail unit。A later manual or scheduled Job is a new invocation，not a retry、attempt lineage or Cron + catch-up；partial graph effects remain valid。 +- **Confidence**：Sir rejected the proposed Peer-lease shortcut as over-coupling，located the problem in Job runtime policy， + and clarified that Mail backfill is deliberately best-effort、incremental and resumable across Jobs。 + +### D-304 — Timeout is an execution outcome；Source checkpointing is recommended resilience，not a contract + +- **Terminal outcome**：add terminal Job status `timed_out`。It means only that this invocation exhausted its configured + execution budget；it does not declare the Source horizon、Mail backfill or already-created graph effects failed or complete。 + A Handler that voluntarily finishes its bounded work before expiration remains `finished`；an escaping runtime/domain error + remains `failed`。 +- **Late-close protection**：timeout recovery conditionally transitions only an overdue `running` Job to `timed_out`。A worker + that returns later may close only a still-`running` row and therefore cannot overwrite the terminal timeout outcome。 +- **Checkpoint guidance**：Source implementations should persist useful checkpoints incrementally because collect may stop at + any time，including Job timeout and process/system failure。This is a resilience guideline，not a required Source capability + or a precondition for running a collect Job。 +- **Allowed weaker Source**：a Source without checkpoints remains valid。Its later invocation may rescan external data and + depend on Source-owned identity/reconciliation；generic Job does not inspect checkpoint support、enforce progress cadence、 + resume domain traversal、roll back partial effects or create retries。 +- **Confidence**：Sir accepted `timed_out` and explicitly classified checkpointing as recommended interruption tolerance rather + than a mandatory Source contract。 + +### D-305 — Job snapshots an effective timeout resolved from type default and creator override + +- **Persisted layers**：`job_types.default_timeout_seconds` is the non-null default execution budget for an exact Job type； + `jobs.timeout_seconds` is the non-null effective value snapshotted when an individual Job is created； + `crons.job_timeout_seconds` is a nullable override carried by the Cron Job template。A direct/manual create command may supply + the same nullable override。 +- **Resolution**：explicit creator override wins；otherwise Job creation reads the persisted Job-type default。The resolved + value is copied onto Job and later changes to Cron or Job type do not reinterpret an already-created execution envelope。 +- **Queue boundary**：the budget starts only after successful `pending -> running` claim。Pending queue time does not consume + runtime；expiration is derived from `started_at + timeout_seconds`。 +- **Representation**：use positive integer seconds rather than PostgreSQL `interval` or an implicit unit，keeping the contract + direct and portable across PostgreSQL/PostgREST、Python and TypeScript workers。 +- **Topology**：Cron merely supplies an optional creator override and materializes the Job；it does not monitor timeout or + understand execution。A Cron-capable Peer need not implement the Job Handler in order to resolve the persisted type default。 +- **Confidence**：Sir accepted the three-layer type-default、creator-override、Job-snapshot model。 + +### D-306 — Running timeout converges through local cancellation plus database-authoritative closure + +- **Local enforcement**：the worker executing a claimed Job establishes a local deadline from its snapshotted + `timeout_seconds` and attempts to cancel the Handler when that deadline expires。Runtime-specific cancellation propagation is + best-effort；Job does not promise process isolation or interruption of every blocking external library call。 +- **Distributed recovery**：any Job worker's maintenance pass may conditionally transition an overdue `running` row to + terminal `timed_out` when `started_at + timeout_seconds <= database now`。This also converges Jobs whose original worker or + process disappeared，without recording an execution Peer。 +- **Single authority**：the conditional database transition is authoritative。The original worker may close only a still- + `running` Job，so a late return cannot replace `timed_out` with `finished` or `failed`。Rare post-timeout effects from a + cancellation-insensitive operation remain valid partial effects。 +- **Excluded machinery**：do not add `jobs.peer`、Peer heartbeat coupling、execution lease、requeue、retry or per-Job process + isolation for this MVP。Their complexity is not justified by the residual cancellation edge。 +- **Confidence**：Sir accepted the local-cancellation plus database-convergence mechanism and its best-effort cancellation + boundary。 + +### D-307 — Mail backfill is exact-range best effort without a durable continuation checkpoint + +- **Corrected separation**：ordinary collect and backfill are separate Source-owned collect intents。Backfill does not use、 + complement、advance or derive its bounds from the ordinary setup/synchronization checkpoint；the two may overlap and share + only graph production、identity and reconciliation machinery。 +- **Execution semantics**：one backfill Job attempts as much as possible inside its caller-specified exact range and Job + timeout。It does not promise range completeness。After timeout，the user may submit a narrower range or a larger timeout； + there is no implicit continuation command or campaign lifecycle。 +- **No durable backfill cursor**：do not persist per-range backfill checkpoints in Source state。A later backfill may rescan the + range，but already-collected occurrences reconcile to existing graph facts，making the residual remote-scan cost cheaper than + checkpoint identity、merge、invalidation and cleanup machinery。 +- **Supersedes D-303 implication**：D-303's general resumable-progress rationale does not require Mail backfill continuation。 + Ordinary Mail synchronization still benefits from its existing protocol checkpoint；checkpointing remains only a general + Source resilience recommendation under D-304。 +- **Scheduling meaning**：a fixed historical range has no useful recurring meaning。Backfill is a manual collect intent；the + exact enforcement boundary between ordinary UI/command paths and generic Cron remains to be closed without coupling Cron to + Mail semantics。 +- **Confidence**：Sir rejected durable backfill checkpoint ROI，identified range/timeout adjustment as the user control，and + clarified that fixed-range backfill is naturally non-recurring。 + +### D-308 — Ordinary Source collect and Source backfill are distinct exact Job types + +- **Correction**：withdraw the proposed `intent: "collect" | "backfill"` union inside one `core.source.collect.v1` Job。 + Product vocabulary places backfill under the collect umbrella，but durable command identity follows distinct invocation + parameters、eligibility and scheduling behavior rather than that category hierarchy。 +- **Exact types**：`core.source.collect.v1` represents ordinary Source collection and may be created manually or by Cron； + `core.source.backfill.v1` represents explicit bounded historical collection and is created only by the manual product path。 + Neither parameters shape contains an `intent` discriminator。 +- **Source capability**：a Source type may implement ordinary collection without supporting backfill。Source registration + projects/validates their command config schemas independently；the backfill projection is absent when unsupported。 +- **Handler routing**：the Job Handler Registry owns separate exact handlers。Parameter-sensitive `can_handle` for ordinary + collect requires the referenced Source type's ordinary implementation；backfill requires that Source type's explicit + backfill implementation。 +- **Cron boundary**：ordinary scheduled collection templates use only `core.source.collect.v1`。No Cron/Mail semantic coupling + or recurring-backfill parameter union is needed to keep the supported path exact。 +- **Supersedes prior drafts**：D-299's `core.source.collect.v1` projection now applies only to ordinary collection；the R2b draft + union and its `intent` field are withdrawn。 +- **Confidence**：Sir corrected the false premise directly：ordinary collect and backfill collect were never intended to share + one Job type。 + +### D-309 — Mail seen policy is two linear Source config facts；Cron does not judge Job recurrence value + +- **Seen-policy correction**：replace one ordinary Source `mark_as_seen` plus a backfill Job-local override with two explicit + Source config facts：`ordinary_mark_as_seen` and `backfill_mark_as_seen`（exact spelling may be finalized mechanically）。 + Defaults remain ordinary `true` and backfill `false`。Each command reads exactly its own persisted policy；there is no + fallback、inheritance or nonlinear override mapping between Source config and Job parameters。 +- **Backfill parameters**：the bounded backfill Job carries its range，not a temporary seen-policy override。Changing remote + mutation policy is a Source configuration operation and remains visible across invocations。 +- **Cron correction**：withdraw D-307/D-308's `manual-only` enforcement implication。A fixed-range backfill is naturally poor + recurring work，but global Cron deliberately materializes any valid Job template without judging its business usefulness。 + Do not add `manual_only`、`scheduleable`、`can_schedule` or a Cron-specific parameter schema。 +- **Product guidance versus validity**：ordinary UI may present backfill as a manual action and need not encourage a recurring + template，but a user-authored Cron targeting `core.source.backfill.v1` remains valid and executes normally。 +- **Confidence**：Sir required direct independent config facts instead of an override graph and rejected turning a marginal- + value product judgment into generic scheduling restriction machinery。 + +### D-310 — Remote seen mutation is best-effort and does not gate accepted collection progress + +- **Primary accepted effect**：once an occurrence's graph、membership and observed remote flags are durably committed，Mail + collection has accepted that information。The ordinary mailbox checkpoint may advance from this primary fact boundary。 +- **Orthogonal configured action**：the matching Source seen policy triggers `STORE \\Seen` only after graph commit。Success + updates the local Seen tag fact；failure records bounded Job diagnostics but does not hold or regress the ordinary checkpoint。 + Backfill applies its independent policy under the same best-effort rule and has no checkpoint。 +- **ROI correction**：retrying the whole observed delta to recover one seen mutation can pin a Mailbox on deterministic + external failure、repeat fetch/parse/reconciliation and couple future collection to a workflow convenience。One message + remaining unseen is the smaller、visible harm；do not add a pending-action ledger、implicit retry or checkpoint barrier。 +- **Discussion correction**：this result is a natural consequence of existing marginal-utility、lifecycle-separation、one-shot + Job and shallow-completion patterns。The rejected checkpoint barrier was a dominated option and should not have been + presented as requiring human product judgment。 +- **Confidence**：Sir immediately rejected the re-observation ROI and identified the process failure：accepted common design + patterns must filter proposals before questions are escalated。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D311-D320.md b/tasks/knowledge-lifecycle-capabilities/decisions/D311-D320.md new file mode 100644 index 0000000..7bc54ea --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D311-D320.md @@ -0,0 +1,175 @@ +# Decisions D-311–D-320 + +### D-311 — A Mail Source cannot silently rebind to another remote access context + +- **Source meaning**: one configured Mail Source may rotate credentials，but once successfully used it remains bound to the + same locally described remote access context。An unchanged `sources.id` does not prove continuity after protocol、endpoint + or login changes。 +- **Why rebind is unsafe**: ordinary checkpoints、Source-scoped Mailbox graph and lazy remote MIME references all depend on + the prior access context。Clearing only checkpoint state would still mix provenance and can make previously collected remote + content unreachable；the protocol exposes no portable account identity that could make an automatic continuity guess safe。 +- **Validator state**: the selected Adapter projects an exact non-secret access binding。For IMAP it contains public protocol、 + normalized host、port、security mode and login name；rotatable credential secrets such as password are excluded。The Mail + Source persists this binding in its Source-owned validator state after the first successful Adapter entry and before + producing Mail graph effects。 +- **Use-time defense**: later Source commands compare the live projection with the persisted binding before collection。 + Remote-I/O Mail Resolvers require the same match but do not initialize、replace or otherwise own Source state。A mismatch is + a repairable configuration error：restore the prior binding fields or create a new Source for the new context。 +- **No implicit recovery**: do not clear state、delete graph、rewrite Mailbox scope、guess account equivalence or silently + rebind。A future explicit migration/rebind operation would need its own product value、effect boundary and recovery design。 +- **Pattern reuse**: this is a Mail application of U-023/U-024：incremental state and durable remote references remain valid + only inside the access scope that produced them；configuration change alone is not continuity proof。 +- **Confidence**: Sir explicitly agreed that a Mail Source cannot be silently rebound after reviewing the checkpoint、graph + provenance and lazy-content consequences。 + +### D-312 — Email HTML is preserved at collection and made safe at the browser-rendering boundary + +- **Authority**: collection and Storage preserve the original HTML body；they do not sanitize、rewrite or discard sender + content。Safety transformation belongs where passive external bytes would otherwise acquire browser execution or network + capability：the solved-content parsing/rendering boundary。 +- **Body choice**: `SolvedEmail` prefers an available HTML body for faithful reading and falls back to the plain-text body。 + This is presentation policy，not a change to the canonical Mail graph or either body Block。 +- **Baseline rendering contract**: sanitize HTML with a mature maintained library such as DOMPurify，then render it inside a + sandboxed iframe without script、form or same-origin capability。Do not implement an application-owned HTML parser or XSS + filter，and do not treat CSP alone as the sanitizer。 +- **Remote privacy**: automatic external resource loading is disabled by default。A user click may open a normalized + `http`/`https` link；opening an Email does not automatically download an attachment or fetch a remote image/tracker。 +- **Inline MIME**: a CID reference may be rewritten to a client-local object URL only when the corresponding semantic content + child is already materialized。An unmaterialized inline part remains a placeholder/metadata presentation with an explicit + materialization action。 +- **Scope**: this is basic enforcement at a concrete untrusted-HTML/browser boundary，not a generic security-hardening program + or permission to accumulate speculative defenses elsewhere。 +- **Confidence**: Sir explicitly accepted HTML-first、isolated rendering and remote resources disabled by default，and + confirmed that mature libraries should own sanitizer attack coverage。 + +### D-313 — Mail Acceptance uses deterministic real IMAP plus an optional external-provider smoke + +- **Hard authority**: the repeatable acceptance gate runs against an ephemeral Dovecot server through real IMAP sockets。 + Production MailAdapter behavior is exercised black-box；tests do not replace it with a mock or call parser/schema helpers + as a substitute for collection evidence。 +- **Corpus setup**: acceptance-owned `.eml` artifacts contain useful、readable real technical material and are installed into + the disposable mailbox through IMAP `APPEND`。This is valid at the current Source boundary：the unit consumes an existing + mailbox and does not own SMTP delivery。Corpus aliases/assertions remain test-owned and must not shape production identity、 + resolver or graph behavior。 +- **Provider reality**: a separate opt-in smoke may use environment-supplied credentials for a dedicated external IMAP + account to detect TLS、capability、mailbox、flag and MIME deviations in providers such as Gmail or Outlook。Network、rate + limits and provider availability make it diagnostic evidence，not the ordinary blocking authority。 +- **Browser proof**: client-web Playwright consumes the graph produced by the real-IMAP run and exercises GraphSurface → + BlockInspectorPopup → SolvedContentPopup，cross-Block navigation、explicit MIME materialization and the D-312 HTML + execution/remote-resource boundary。 +- **Test allocation**: schemas、typing and pure mechanical contracts stay with static analysis or narrowly justified tests。 + The existing Mail schema/parser-helper tests are not carried forward merely for line coverage。 +- **Confidence**: Sir explicitly accepted this two-level evidence authority with Dovecot as the hard gate and a real provider + as an optional smoke。 + +### D-314 — Four vertical journeys are sufficient Mail-unit Acceptance；do not add a focused negative-path suite now + +- **Blocking scope**: the unit is accepted through four Dovecot/client-web journeys：ordinary incremental Mail graph、explicit + backfill and collection policies、lazy MIME materialization、and generic InfoBase solved-content use。 +- **Coverage judgment**: these journeys traverse the production IMAP Adapter、Source Job commands、InfoBase graph、Resolvers、 + Storage and client-web where relevant。They prove the unit's primary value and lifecycle boundaries without demanding one + automated scenario for every frozen internal contract。 +- **No negative suite**: do not add dedicated failure injection for partial Mailbox graph failure、checkpoint CAS loss、 + post-commit Seen failure、access-binding mismatch、Cron/Job races、timeout recovery or other low-frequency negative paths in + this unit。Do not add the proposed scripted OBJECTID peer solely for coverage。 +- **Contract retained**: omitting those tests does not withdraw their accepted design semantics。Implementation should remain + readable and use database/type/library guarantees where applicable，but Acceptance does not manufacture faults or test-only + production abstractions to prove them now。 +- **Client-web worker consequence**: the required generic Job-worker/handler-registration implementation remains in scope， + but this unit does not add an acceptance-owned Source handler merely to exercise it。Its types、build and ordinary code-path + verification are implementation checks；the browser must not claim unsupported IMAP work。 +- **Confidence**: Sir judged the four vertical journeys sufficient for this unit and explicitly deferred negative-path tests。 + +### D-315 — MIME-part materialization is the Mail extension's first exact Peer delegation + +- **Capability**: register the exact capability ID `extensions.mail.mime_part.materialize.v1`。It is owned by the Mail + extension and delegates one request-response command；it is not generic Resolver delegation、generic capability invocation + or a Job。 +- **Caller**: a Peer that can solve the MIME-part metadata graph but cannot reach the configured Mail protocol—currently the + browser client—is allowed to delegate only when no semantic content child exists and explicit materialization is requested。 +- **Provider**: the selected capable Peer invokes the provider-local、non-delegating + `MailMimePartResolver.get_solved_content(materialize_missing=true)` path。The Resolver remains the materialization owner and + may use the Mail Adapter、Storage and InfoBase through their accepted boundaries。 +- **Payload boundary**: the request identifies the MIME-part metadata Block；success returns the resulting semantic child + Block，not raw bytes、Storage internals or `created/existing` mechanics。The caller resolves/hydrates that child through its + ordinary shared-database/Storage path。 +- **Lifecycle**: Mail extension start/close registers/unregisters the matching inbound route and republishes the Peer + capability snapshot。This is the first extension-owned Peer delegation and establishes no universal extension invocation + surface。 +- **Confidence**: Sir explicitly accepted the derived exact capability and its role as the first extension Peer delegation。 + +### D-316 — The next active direction is provisionally `feature-retrieval`，but its name and capability boundary remain open + +- **Selection**: after Mail closure，the next active application unit investigates what retrieval ability is missing beyond + semantic similarity and graph navigation。`feature-retrieval` is a provisional task address，not yet durable vocabulary or + an assumption that the answer is a field-filter feature list。 +- **Product pressure**: Memos、RSS and Mail can now produce useful graph state，but a person who does not already know a + BlockRef has no scalable、explainable entry path；the current GraphSurface loads the whole graph and known-ID focus is only + navigation after location。 +- **Multimodal scope**: text is not the only admissible evidence。Image、audio、video、file and source/graph facts may expose + useful authored or derived evidence，but no modality-specific extraction、index or UI is pre-approved merely because it can + be enumerated。 +- **Discovery rule**: begin from real information-finding jobs，separate query evidence、matching/ranking mechanism and result + explanation，then determine whether the missing capability is predicate、lexical、example/similarity、hybrid or another + model。Do not define the unit metaphysically by piling up operators。 +- **Confidence**: Sir fully accepted the unit direction，explicitly requested modalities beyond traditional text，and warned + that both his current idea and the provisional vocabulary must be tested against the actual gap left by semantic and graph + retrieval。 + +### D-317 — D-316 identifies a product capability pressure，not yet an implementable unit + +- **Correction**: the broad goal “locate unknown information from remembered、explainable clues” is valid，but it spans + structured facts、lexical evidence、multiple modalities、possible similarity mechanisms、query composition and use-facing + presentation。Treating that whole space as one active implementable unit would turn a useful product intent into an abstract + horizontal program。 +- **Effect on D-316**: retain its capability-gap discovery and multimodal guardrail，but withdraw the implication that + `feature-retrieval` is already the next bounded implementation unit。The name remains only a candidate exploration address。 +- **Selection gate**: return to no active implementable unit until one real “recall by clues” journey can define a narrow + observable outcome、bounded evidence types and an end-to-end acceptance corpus without claiming the whole capability。 +- **Confidence**: Sir accepted the capability positioning but explicitly rejected its current breadth and abstraction as an + implementable-unit boundary；the assistant agrees this is a decomposition-layer error。 + +### D-318 — `feature-retrieval` remains the active Unit；deliverability comes from internal increments + +- **Correction to D-317**: D-317 over-corrected a real scope problem。A Unit may own several independently deliverable + increments；its breadth does not require demoting a coherent product and technical ownership boundary into a non-Unit + exploration。 +- **Unit boundary**: feature retrieval locates information through matchable features of the target or its projections，rather + than through graph position/relationships or semantic similarity alone。Lexical features and perceptual features both belong + to this Unit；modality does not itself define a separate retrieval family。 +- **Adjacent ownership**: fact/relationship recall through graph topology belongs to graph-navigation-retrieval。Hybrid recall + composes already-established primitive retrieval capabilities and is deliberately later work，not a primitive of this Unit。 +- **Execution boundary**: lexical retrieval is the first required increment。Perceptual retrieval remains in the Unit boundary + but is not required in the first increment；each increment receives its own product design、technical design、plan and + acceptance journey before implementation。 +- **Confidence**: Sir explicitly established this decomposition and rejected the assistant's withdrawal of the Unit identity。 + +### D-319 — Retrieval ownership follows the current info-base representation，not an intrinsic/extrinsic ontology + +- **Decision**: a fact such as a PDF filename、page count or MIME does not permanently belong to one retrieval family because + it is an “intrinsic property”。While it remains embedded in Block content or a Resolver lexical projection，feature retrieval + is the available discovery path；after Organization externalizes it into Blocks/Relations，graph-navigation-retrieval owns + its graph-visible location and exploration。 +- **Organization meaning**: this representation transition is a concrete way Organization improves use：it promotes useful + evidence from content that must be interpreted into graph facts that can be navigated and composed。 +- **Feature boundary**: the pre-Organization fallback is likely lexical retrieval through Resolver output；it does not require + feature retrieval to become a universal structured-content predicate engine。Exact numeric or typed predicates are not + implied merely because the source content contains such values。 +- **Confidence**: Sir corrected the intrinsic-property formulation and the assistant agrees that current representation is the + stable ownership basis。 + +### D-320 — Lexical retrieval stays primitive but permits Resolver-owned missing materialization + +- **Primitive boundary**: lexical retrieval is a required atomic increment inside feature retrieval。Its V1 query remains + explicit lexical matching；Chinese contiguous fragments do not depend on linguistic segmentation，and hybrid composition is + left to downstream capabilities。 +- **Record model**: retain one rebuildable `block_lexical_records` projection，symmetric with semantic retrieval's derived + record pattern，without promoting the index to information authority。 +- **Effect decision**: lexical projection maintenance calls the Resolver with `materialize_missing=true`。This is permission， + not an obligation：a Resolver may satisfy the lexical context from existing metadata，or may create a missing derivation such + as OCR when its exact contract supports that effect。The exact Resolver owns the graph mutation；at the product-lifecycle level + this is an Organization effect。Feature retrieval owns only the request for projection and its derived record。 +- **Read boundary**: ranked retrieval still does not repair or maintain records inside the query。The explicit/scheduled lexical + maintenance path is the retrieval-driven trigger for permitted organization。 +- **Confidence**: Sir accepted primitive/atomic query semantics and the record model，and explicitly required missing + materialization to remain enabled for retrieval-driven organization。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D321-D330.md b/tasks/knowledge-lifecycle-capabilities/decisions/D321-D330.md new file mode 100644 index 0000000..c351167 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D321-D330.md @@ -0,0 +1,132 @@ +# Decisions D-321–D-330 + +### D-321 — Lexical projection must preserve available document body recall + +- **Completeness decision**: examples of PDF metadata do not narrow lexical retrieval to metadata。When an exact Resolver can + obtain document body text，that text must remain lexically searchable；metadata is additive，not a replacement for body + evidence。 +- **Graph boundary**: if body text already exists or is materialized as semantic-content Blocks，those Blocks are independently + indexed and the root file record need not duplicate the full body merely to preserve recall。If no child representation is + created，the Resolver may project available body text directly into the root lexical record。 +- **Materialization boundary**: explicit/scheduled lexical maintenance may pass `materialize_missing=true` so an exact Resolver + can produce missing body text，including OCR when supported。Encrypted、invalid or otherwise unsupported content remains an + explicit capability limit rather than silently pretending metadata is the complete document text。 +- **Observed gap**: the current `core.pdf.v1` Resolver inspects PDF metadata but reports text projection unsupported，so PDF body + recall is new implementation/acceptance scope rather than an existing guarantee。 +- **Confidence**: Sir confirmed that PDF body recall must not be lost and accepted explicit/scheduled lexical maintenance as + the materialization trigger。 + +### D-322 — Lexical projection is Block-local and non-recursive + +- **Projection boundary**: `get_text(context="lexical")` describes lexical evidence carried by the focal Block。It does not + recursively copy text owned by adjacent Blocks，including a semantic-content child reached through a `content` Relation。 +- **Record consequence**: every searchable Block may have its own `block_lexical_records` row，but the same semantic-content + body is not mechanically indexed on both the parent file Block and its text child。For a materialized PDF，the root record + carries its label/metadata and the child record carries extracted/OCR body text。 +- **Retrieval boundary**: Lexical Retrieval does not perform graph-aware result collapsing or parent/child deduplication。Two + genuinely distinct Blocks may naturally share lexical evidence；only recursive projection duplication is excluded upstream。 +- **Confidence**: Sir identified and accepted Block-local、non-recursive projection as the governing principle。 + +### D-323 — Multimodal-to-text recall remains inside the lexical increment + +- **Scope correction**: deferring perceptual retrieval does not defer lexical recall over faithful textual representations of + image、audio or video。OCR、speech transcription、source-native subtitle/caption extraction and comparable modality-to-text + materialization remain in the lexical increment。 +- **Composition rule**: an exact media Resolver may materialize derived text Blocks；those ordinary Blocks receive their own + lexical records under D-322。The lexical query engine still matches text and does not become a pixel/audio/video matcher。 +- **Remaining design pressure**: exact required textualization types and the boundary between faithful extraction and generated + interpretation remain product-design review，not part of this confirmed scope decision。 +- **Confidence**: Sir corrected the premature claim that only maintenance topology remained and explicitly required image、 + video and audio → text lexical retrieval in this increment。 + +### D-324 — Retrieval maintenance runs through durable Cron and typed Jobs + +- **Lexical execution**: explicit and scheduled lexical maintain/rebuild commands use the shared typed Job lifecycle；Cron is + the optional recurring Job-template owner，one capable Peer claims execution，and thin Job Handlers invoke the domain manager。 +- **Progress semantics**: successful derived records are natural resumable progress。No additional checkpoint、retry or + peer-local scheduling lifecycle is introduced，and ranked retrieval remains read-only。 +- **Semantic correction**: migrate semantic maintain/rebuild to the same Cron/Job topology and remove its legacy direct + peer-local maintenance timer，so retrieval indexes do not retain two scheduling authorities。 +- **Confidence**: Sir accepted Cron/Job execution for lexical maintain/rebuild and recommended the semantic migration。 + +### D-325 — Model-authored media interpretation is Organization，not materialization + +- **Required scope**: this increment must actively support generating image/video/audio descriptions、summaries or comparable + model-authored interpretations；it is insufficient to index them only if another future owner happens to create them。 +- **Ownership**: faithful OCR/transcription/source-native extraction may be Resolver-owned missing materialization。A generated + description or summary adds an interpretation and is therefore owned by Organization，not by `materialize_missing` and not by + lexical-record maintenance。 +- **Composition**: Organization persists interpretation Blocks/Relations through its ordinary graph command path；the resulting + text Blocks are independently indexed by lexical maintenance under the same Block-local contract as any other text。 +- **Trigger continuity**: D-184 remains authoritative：active generation means an explicit request for one focal Block through + `OrganizationManager.ruminate(block_id)`，not a collection hook、periodic scan or lexical-maintenance side effect。 +- **Remaining design pressure**: the typed multimodal Agent-input seam remains technical review。The design should extend the + existing focal-Block rumination approach rather than inventing a manager solely for media。 +- **Confidence**: Sir explicitly required active generation in this increment and corrected the ownership category from + materialization to Organization。 + +### D-326 — Media interpretation is system-driven and consumes solved content + +- **Trigger correction**: “active generation” means InKCre initiates media interpretation without a per-Block user request。 + This supersedes D-184's automatic-execution exclusion for this new media-interpretation approach；ordinary focal rumination + remains explicitly invoked unless separately redesigned。 +- **Resolver seam correction**: Organization may consume `Resolver.get_solved_content()`。Solved content is the existing typed、 + use-facing Block interpretation and may contain bytes or structured values；it is not restricted to JSON or text。 +- **Rejected addition**: do not add Resolver-owned `TextPart`/`MediaPart` or another parallel projection merely to feed + Organization。Any structured multimodal payload needed at the AI call belongs to the AI message/dialect boundary and does not + become a new InfoBase concept。 +- **Context restraint**: the existence of projection context leaves room for evidence-led solved variants，but this decision + does not add a new `get_solved_content()` context before a concrete Resolver requires one。 +- **Remaining design pressure**: automatic candidate selection、trigger/Job shape、attempt/freshness semantics and the minimal + solved-content → AI invocation adaptation must now be designed explicitly。 +- **Confidence**: Sir defined active generation as non-user-driven，identified solved content as the sufficient Resolver-owned + seam and rejected the unsupported parallel content-part abstraction。 + +### D-327 — Automatic media interpretation selects missing-only candidates + +- **Candidate rule**: the automatic approach considers `core.image.v1`、`core.audio.v1` and `core.video.v1` Blocks that do not + already have the exact outgoing `interpretation` Relation used by this approach。 +- **Freshness restraint**: Relation existence means only “an interpretation is present”。It does not prove that the result is + current、best or based on the latest bytes/model/context；automatic recompute is not introduced。 +- **Execution**: a typed bounded Organization Job scans deterministic candidates and continues after one candidate is + unavailable、fails or produces no graph。Successful interpretation Relations are natural progress for later Jobs。 +- **No ledger**: do not add attempt、freshness、retry or approach-run records。A still-missing Block may be reconsidered by a + later independently scheduled Job；explicit future `recompute` remains separate。 +- **Confidence**: Sir accepted the proposed missing-only automatic candidate policy。 + +### D-328 — Media interpretation uses an independent persisted Agent + +- **Agent identity**: system-driven media interpretation selects its own persisted reusable AgentDefinition；it does not reuse + the deployment selection or semantic identity of the explicit rumination Agent。 +- **Reuse boundary**: the two AgentDefinitions may reference the same AI Model or Tool IDs，but independently own system prompt、 + selected tools、nullable tool choice and per-Turn model-call budget。 +- **Organization topology**: the automatic approach runs the selected Agent through AgentManager and lets the existing validated + Graph Tool path persist additive interpretation output，rather than parsing one direct model string in Organization code。 +- **Configuration pressure**: the deployment config receives a separate media-interpretation Agent reference；exact key/schema + and Job parameters remain technical-design freeze items。 +- **Confidence**: Sir explicitly confirmed that media interpretation is another independent Agent。 + +### D-329 — Media interpretation routes to one Agent per modality + +- **Routing**: the Organization approach explicitly selects separate persisted Agent references for image、audio and video；it + does not require one Agent/AI Model to support every modality。 +- **Config shape**: one deployment config owns required integer references `image_agent`、`audio_agent` and `video_agent`。The + same Agent ID may intentionally fill more than one field，and reference existence remains a use-time defense。 +- **Ownership**: Organization chooses the Agent from the focal Resolver/media modality。AIManager routes the selected Agent's + model through provider/dialect capabilities but does not select or fail over among Agents。 +- **Partial availability**: one missing、dangling or incapable modality Agent becomes a bounded candidate diagnostic and does not + prevent the same Job from processing candidates whose selected Agents are usable。 +- **Confidence**: Sir explicitly required per-modality Agent routing in the current increment。 + +### D-330 — Faithful media text signals remain separate graph children + +- **Graph shape**: faithful textualization does not collapse every signal from one media Block into one aggregate text value。 + Each semantically distinct signal becomes its own ordinary `core.text.v1` child Block。 +- **Relation vocabulary**: relation content names the information role rather than the extraction implementation：visual written + language uses `text`，spoken language uses `transcript`，and source-native subtitles use `subtitle`。OCR、ASR and provider/model + names do not become graph predicates。 +- **Interpretation boundary**: model-authored description、summary or comparable understanding remains an `interpretation` + graph and is not merged with faithful text children。 +- **Retrieval consequence**: every child receives its own Block-local lexical record。No implicit aggregation、cross-signal + deduplication or copying of complete child text into the parent lexical record is introduced。 +- **Confidence**: Sir accepted separate child Blocks for distinct textual signals and the proposed semantic relation roles。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D331-D340.md b/tasks/knowledge-lifecycle-capabilities/decisions/D331-D340.md new file mode 100644 index 0000000..1007eb8 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D331-D340.md @@ -0,0 +1,118 @@ +# Decisions D-331–D-340 + +### D-331 — Media interpretation is one parameterless convergence Job + +- **Exact contract**: `core.organization.media_interpretation.v1` has an empty parameter object。Cron repeats the same static + Job template；it does not choose a Block or media modality。 +- **Candidate ownership**: Organization selects all currently missing image、audio and video candidates at execution time。 + Successful `interpretation` Relations remove their Blocks from later missing-only scans；timeout preserves completed graph + effects，and a later Job re-derives the remaining candidate set from current graph state。 +- **Routing placement**: modality is a property of each candidate and selects `image_agent`、`audio_agent` or `video_agent` + inside Organization。It is not a Job parameter and does not require three modality-specific Crons or Job types。 +- **State boundary**: Job state may report bounded per-modality counts and diagnostics but does not own a cursor、checkpoint、 + attempt ledger or retry protocol。 +- **Confidence**: Sir identified the static-Cron-template mismatch，accepted the corrected parameterless convergence Job and + confirmed per-modality routing remains candidate-local。 + +### D-332 — Lexical records have one exclusive lifecycle owner + +- **Ownership**: `block_lexical_records` is derived state owned exclusively by Lexical Retrieval。Only + `LexicalRetrievalManager.maintain/rebuild` creates or updates its rows。 +- **Upstream independence**: Resolver materialization and Organization interpretation may add or change authoritative + info-base graph facts，but neither writes lexical records、calls lexical maintenance nor receives a retrieval callback。 +- **Convergence**: independently invoked lexical maintenance scans current Blocks and projects resulting graph changes into + records。The temporal composition does not merge the lifecycles or give Organization ownership of an index。 +- **Confidence**: Sir explicitly reviewed this boundary，identified the abbreviated topology's ambiguity and accepted the + baseline after this ownership was confirmed。 + +### D-333 — Canonical Chat gains provider-neutral multimodal User content parts + +- **Contract**: `UserMessage.content` becomes a non-empty ordered tuple of discriminated text、image、audio or video parts。 + Media parts carry actual bytes plus standard MIME；they do not carry Block/Storage references、provider URLs or Resolver + solved-content types。 +- **AI boundary**: AIManager derives required input modalities from canonical Message history and validates Model + dialect + capability。The exact dialect alone maps parts to protocol fields/base64 and may internally aggregate streaming text and + ToolCall deltas into the existing complete `AssistantMessage` result。 +- **Lifecycle**: Agent Thread still persists canonical Messages。Streaming remains dialect-internal and does not create a + second Thread/Turn state model。 +- **Confidence**: Sir accepted the complete pre-execution baseline after provider/corpus and branch preflight。 + +### D-334 — Faithful media materialization uses exact Resolver-owned Model references + +- **Ownership**: image/audio/video Resolvers own faithful extraction prompts、information roles and graph writes，so they use + direct AI Model references rather than AgentDefinitions。Organization media interpretation continues to use independent + per-modality Agents。 +- **Configuration**: exact deployment configs `core.resolver.image`、`core.resolver.audio` and `core.resolver.video` bind only + the Model roles each Resolver needs。Source-native subtitle extraction needs no Model reference。 +- **Failure boundary**: references are resolved defensively at use time。Missing、disabled or modality-incapable Models make + only that exact derivation unavailable；AIManager performs no automatic Model fallback。 +- **Confidence**: Sir accepted the complete pre-execution baseline after provider/corpus and branch preflight。 + +### D-335 — Lexical and semantic maintenance use exact typed Jobs + +- **Execution**: lexical maintain/rebuild and semantic maintain/rebuild are four exact Job types whose thin Handlers invoke + their domain managers。The legacy semantic peer-local timer is removed so Cron → Job is the sole recurring authority。 +- **Eligibility**: Job `can_handle` uses best-effort local AIModel/Agent executability checks before atomic claim；it does not + probe providers、promise quota/key health、rollback a claim after TOCTOU failure or choose a fallback Model。 +- **Media partiality**: the parameterless interpretation Job is locally eligible when at least one configured modality Agent + is executable。It processes capable candidates，records bounded diagnostics and leaves the rest missing for later independent + Jobs without adding cursor/checkpoint state。 +- **Confidence**: Sir accepted the complete pre-execution baseline after the implementation plan and failure branches were + replayed。 + +### D-336 — Alibaba multimedia extensions receive an exact cross-capability dialect + +- **Protocol fact**: OpenAI Chat Completions itself defines `image_url` and base64 `input_audio` but not `video_url`。Alibaba + Model Studio extends that surface with video and URL audio；the OpenAI SDK's ability to send those fields is not protocol + identity。 +- **Exact identity**: retain `core.openai-compatible.v1` for the standard supported subset and add + `core.alibaba-model-studio.v1` for Alibaba-specific extensions。Do not create `core.openai-chat.v1` because D-157/D-158 make + `chat` one capability inside a dialect that may span capabilities and native endpoints。 +- **Reuse boundary**: both adapters may use the same OpenAI SDK and internal message/Tool translation helpers；shared mechanics + do not merge their exact wire contracts。 +- **Confidence**: Sir explicitly accepted `core.alibaba-model-studio.v1` after reviewing the OpenAI/Alibaba protocol delta。 + +### D-337 — Storage URLs are optional AI transfer hints behind hydrated content + +- **Content authority**: Resolver/Organization continues to supply actual hydrated bytes + MIME through AI Chat content parts。 + HTTP Storage therefore remains lazy byte storage and does not force a copy into PostgreSQL binary Storage merely for AI use。 +- **Storage capability**: `Storage.get_transfer_url(pointer) -> str | None` may expose an origin/scoped URL that another backend + could attempt to fetch。It is a transfer hint，not content authority、a public-access promise or a provider upload lifecycle。 +- **MVP ladder**: the exact dialect prefers inline bytes inside its stable bound；when inline transfer is unavailable/oversized， + it may use an accepted transfer URL；otherwise the exact operation is unavailable。No charged-call retry、silent truncation、 + transcoding or hidden staging is implied。 +- **Confidence**: Sir explicitly accepted URL-as-transfer-hint、the Storage method and the MVP ladder。 + +### D-338 — Multimodal ContentParts belong only to the AI chat contract + +- **Vocabulary**: `TextContentPart`、`ImageContentPart`、`AudioContentPart` and `VideoContentPart` are canonical AI `chat` + Message concepts。They do not become Resolver、Storage or InfoBase entities。 +- **Transfer field**: the three media variants carry actual bytes + MIME and may carry `transfer_url: str | None` under D-337。 + A `MediaContentPart` name，if implementation needs it at all，is only a static union alias and has no independent lifecycle or + contract authority。 +- **Confidence**: Sir confirmed the ownership boundary and explicitly accepted optional `transfer_url`。 + +### D-339 — Render Free is the sleeping self-host profile, not the reference Peer runtime + +- **Immediate host**: the GitHub-only fork journey targets two Render Free Docker web services for core-py and standalone + PostgREST，backed by Neon Free。Cloudflare Python Worker remains a separate event-driven/runtime-port problem；Neon Data API + remains a later role/auth migration despite passing the required PostgREST binary wire spike。 +- **Controller authority**: one manual GitHub workflow owns database convergence、Render service convergence and exact-commit + deploy。Render auto-deploy is disabled；Heroku remains the exact always-on/reference delivery。 +- **Availability boundary**: the profile explicitly permits 15-minute idle sleep、roughly one-minute cold wake、shared monthly + free-instance hours and missed process-owned Cron/lease work。It is an interactive demo profile，not continuous scheduling + equivalence。 +- **Confidence**: Sir accepted Render after the Cloudflare、Neon Data API and conventional-container comparison。 + +### D-340 — Self-host signing authority remains owner-private；public JWT publication is deferred + +- **Authority**: the current HS256 `JWT_SECRET` is full admitted-Peer signing authority，not a role-scoped reader secret。 + Fork owners store it only as a private GitHub secret and supply it privately to their own client session。README、workflow + summary、deployment profile and built clients contain no signing key。 +- **Delivery effect**: missing Render account state blocks only a live service-creation/probe，not implementation of the + controller、workflow、documentation or simulated provider acceptance。The checked-in path may be completed before account + registration，then closed by one real fork deployment later。 +- **Public demo**: publishing the canonical demo key is withdrawn for this increment。A real guest/read-only journey needs an + explicit admission surface and safe projections；it cannot be achieved by documenting another role beside the same HS256 key。 +- **Confidence**: Sir withdrew immediate JWT publication after reviewing provider/source credential reachability and explicitly + authorized implementation without waiting for Render account registration。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D341-D350.md b/tasks/knowledge-lifecycle-capabilities/decisions/D341-D350.md new file mode 100644 index 0000000..56d6548 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D341-D350.md @@ -0,0 +1,147 @@ +# Decisions D-341–D-350 + +### D-341 — Fork-based self-hosting and canonical production demo are independent dimensions + +- **Distribution**: forking `core-py` is a browser-only onboarding mechanism for an owner to create their own independent + InKCre deployment。It does not make that deployment an instance、projection or lifecycle child of InKCre's canonical + production environment。 +- **Runtime**: the Render + Neon profile may currently select Free instances with demo-grade availability。That is a hosting-plan + property；business/domain code must not branch on whether a deployment is self-hosted or canonical。 +- **Canonical production**: calling the maintained production environment a public demo only lowers the value/retention boundary + of its own data。It does not publish JWT signing authority、define self-host credentials or own the fork workflow。 +- **Naming**: delivery surfaces name the exact Render + Neon self-host profile。`fork` remains a user journey，and `demo` remains + an environment/availability description where applicable；neither becomes controller identity。 +- **Confidence**: Sir identified the conflation after configuring the real provider inputs and accepted this correction before + remote acceptance。 + +### D-342 — Preview-backed end-to-end acceptance remains manually invoked evidence + +- **Authority**: core/client preview deployments provide the real runtime for this Unit's end-to-end black-box journeys，while + the Render + Neon self-host workflow proves fork-based onboarding independently。 +- **Execution**: acceptance may be performed manually or through an explicitly invoked script。The current end-to-end journey is + not promoted into an automatic CI test、scheduled workflow or permanent required check。 +- **Regression split**: existing static、unit、migration、artifact and preview checks remain automated regression contracts；the + manual black-box journey observes the deployed system without reshaping production code or introducing fixture-driven hooks。 +- **Confidence**: Sir explicitly selected preview-backed E2E acceptance and prohibited promoting the current journey into an + automated test。 + +### D-343 — Graph navigation retrieval is the next application unit and is distinct from Resolver interpretation + +- **Unit selection**: `graph-navigation-retrieval` is the next active implementable unit after feature retrieval closure。 +- **Application role**: the unit starts from an addressable graph entity or a prior recall result and obtains an explicitly + navigable subset of existing Block/Relation authority；its Product and Technical shape remains under discussion。 +- **Resolver boundary**: Resolver remains the focal-Block interpretation boundary。Its ability to inspect direct Relations、 + construct solved content or lazily materialize missing derivation is not itself graph-navigation retrieval and must not be + renamed or reused as though the two capabilities had the same contract。 +- **Confidence**: Sir selected the unit and explicitly required the design to preserve this distinction。 + +### D-344 — One addressed graph entity expands by one hop as the atomic navigation primitive + +- **Primitive**: one graph-navigation request addresses one Block or Relation and returns the existing graph facts needed + to cross one hop。A Block expansion returns selected incident Relations and their opposite endpoint Blocks；a Relation + expansion returns that Relation and both endpoint Blocks。 +- **Composition**: caller-held scene/context may merge results and select another returned entity for the next expansion； + multi-hop exploration is composed from repeated one-hop requests rather than making an unconstrained N-hop neighborhood + the atomic contract。 +- **Boundedness pressure**: hub-like Source/Mailbox Blocks can have very high degree，so direction、selection、limit and + continuation remain required Technical/Product discussion；the primitive does not imply “load every neighbor”。 +- **MVP scope remains open**: Sir accepted one-hop as a foundation but judged it insufficient as the complete MVP；path、 + pattern and graph-view requirements remain under investigation。 +- **Confidence**: Sir explicitly accepted the proposed one-entity one-hop ability and the fan-out concern。 + +### D-345 — Bounded Block connection and an aggressively improved Graph view complete the MVP Product surface + +- **Second primitive**: add a bounded path operation between two known Blocks。It returns inspectable Block/Relation path + evidence under an explicit hop bound；it does not claim that shortest-by-hop is semantic relevance or introduce a general + Cypher/SPARQL-like pattern language。 +- **Graph view consumer**: client-web `InfoBase Graph view` is a first-class unit surface，not optional polish。It consumes + recall seeds、one-hop deltas and connection paths instead of preserving whole-info-base loading as its product model。 +- **UX scope**: the unit may aggressively improve canvas node/edge rendering、hover/active/focus/defocus states、camera + behavior、incremental layout and the positioning/size/content hierarchy of Block Inspector and solved-content popup。 +- **Reference use**: `yoheinakajima/graphcon-deck` supplies interaction evidence for on-canvas communities、overview ↔ + focused-scene camera travel、context dimming/push-out and hover detail。It is inspiration，not an authored-layout/data-model + contract for InKCre's dynamic graph。 +- **Still open**: exact Graph scene state、community stability、incremental layout、camera/user-control arbitration、popup + composition and initial overview seeding remain Product/Technical discussion。 +- **Confidence**: Sir accepted the proposed path design/judgments/trade-offs and explicitly expanded the unit to an + aggressive Graph-view UI/UX improvement using the linked deck as a reference。 + +### D-346 — One focus-set model unifies Graph scene attention and preserves spatial continuity + +- **Scene state**: client-web Graph scene owns the bounded Blocks/Relations already obtained、stable home positions、 + current selection and a transient focus set；it does not treat the whole info-base as its default scene or persist layout + as retrieval authority。 +- **One interaction primitive**: focal one-hop delta、bounded connection path、multi-selection and computed community can + each produce a focus set。The same focus operation drives focal/context contrast、internal/boundary Relation emphasis、 + reversible peripheral displacement and camera fit rather than creating separate state machines per feature。 +- **Incremental layout**: merging a graph delta preserves existing home positions、seeds new entities near the focal + entity and locally settles only the affected area。Expansion must not restart a whole-scene force simulation and destroy + the user's spatial mental map。 +- **Camera ownership**: automatic camera movement follows explicit selection、community enter、expansion、path and explicit + refocus。Manual pan/zoom suspends automatic ownership；background updates do not steal the camera。Reduced-motion mode + substitutes immediate position/contrast updates for spatial travel。 +- **Community boundary**: algorithmic community detection is an optional producer of focus sets，not a second scene model + or a required graph-navigation retrieval primitive。 +- **Confidence**: Sir accepted this focus-set model after GraphCon's authored scene behavior was separated from InKCre's + dynamic runtime graph。 + +### D-347 — Graph view consumes InkCre UI；real generic gaps may be fixed in the design-system owner + +- **Visual-system guardrail**: the Graph view must follow InkCre UI's existing visual language and reuse public + `@inkcre/ui-web` components/tokens by product intent。It must not repeat earlier client-web work that formed a parallel + local component/token language。 +- **Cross-repo scope**: `../design` is an allowed implementation surface in this unit when Graph view exposes a genuine + reusable primitive、token or accessibility gap。This permission does not make every Graph-specific node、edge、focus or + layout behavior a design-system component。 +- **Evidence before abstraction**: exact design changes remain subject to inspection and review；the unit may improve + InkCre UI, but must not add `InkGraph*` abstractions merely to make application composition look reusable。 +- **Confidence**: Sir explicitly required stronger visual-language/component reuse and invited this unit to improve the + InkCre UI owner where necessary。 + +### D-348 — InfoBase route outlets stay modeless over the desktop Graph navigation host + +- **Desktop behavior**: `BlockInspectorPopup` and `SolvedContentPopup` render without a scrim so the visible Graph remains + an available navigation host rather than being demoted to an inert backdrop。 +- **Responsive behavior**: on narrow screens Solved Content may occupy the available viewport because simultaneous canvas + interaction is no longer spatially useful；this is responsive presentation，not a different route or content contract。 +- **Navigation semantics**: both components remain InfoBase route destination outlets。Their explicit close action invokes + `InfoBaseRouter.back()`；absence of a scrim does not invent outside-click dismissal or a second history authority。 +- **Design-system pressure**: provide the no-scrim behavior through the generic low-level InkCre popup primitive with a + backward-compatible default，rather than duplicating a client-web-only overlay implementation。 +- **Confidence**: Sir explicitly accepted modeless desktop outlets、persistent Graph-host availability and the narrow-screen + Solved Content adaptation。 + +### D-349 — A Graph node is a variable-size preview surface，not an inline full-content window + +- **No inline mode switch**: reject inline expand and any preview/full toggle inside a draggable Graph node。Selection、 + dragging and embedded controls would compete for the same pointer/keyboard surface and turn navigation into window + management。 +- **Node content**: the node itself presents a non-complete preview and may have content-driven dimensions；it never claims + to contain the full Block experience。Freer node size is not permission to put complete solved content or business actions + on canvas。 +- **Outlet roles**: Block Inspector owns metadata、Relations and Block actions。Solved Content owns focused reading and may + contain real domain interactions such as a future Mail reply workflow；it is not merely a larger copy of a node preview。 +- **Resize status**: manual node resizing remains only an investigated option and is not selected for MVP。The fact that + Vue Flow can resize nodes does not establish product value，especially when resize handles can conflict with navigation。 +- **Confidence**: Sir rejected inline expansion，confirmed preview-only canvas content and the two outlet roles，and stated + that resizing is possible but not preferred。 + +### D-350 — Preview and full content are two renderings of one solved-content authority + +- **One interpreted value**: Resolver `solvedContent` remains the single application-facing interpretation of a Block。 + Preview must not introduce a second `previewContent` projection merely because it appears in another UI context。 +- **Two presentation contracts**: client-side Resolver registration distinguishes `previewRenderer` from + `solvedContentRenderer`。Both may consume the same Resolver instance and solved content；preview selects a concise、 + interaction-free presentation，while solved-content presentation may be complete and domain-interactive。 +- **Intrinsic size**: a Graph node's dimensions follow the actual rendered preview within scene-owned bounds。Resolver does + not return canvas width/height hints，and Graph does not assign every Block a fixed card size。 +- **Loading boundary**: Graph may lazily solve only previews that enter its bounded scene/viewport with + `materializeMissing=false`。This does not change solved-content semantics or allow every preview to materialize graph + state。 +- **Cross-surface value**: preview rendering belongs to InfoBase presentation rather than Graph alone；a future List view + can consume the same contract without moving Block/Resolver semantics into the design system。 +- **Correction pressure**: existing Tweet/Text renderers currently truncate full solved presentation for Graph needs，while + Mail solved presentation already carries actions。Implementation should split these presentation roles rather than + preserve that accidental contract。 +- **Confidence**: Sir accepted intrinsic sizing、the diagnosis that preview/full presentations are conflated and the rule + that both are renderings of the same solved content。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D351-D360.md b/tasks/knowledge-lifecycle-capabilities/decisions/D351-D360.md new file mode 100644 index 0000000..cab26d3 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D351-D360.md @@ -0,0 +1,163 @@ +# Decisions D-351–D-360 + +> [Decision register index](index.md) + +### D-351 — Graph-navigation retrieval is presentation-free；InfoBase View owns presentation + +- **Retrieval result**: graph navigation returns persisted Block/Relation authority plus only operation-specific structural + evidence required to navigate it，such as ordered path identity or bounded continuation。 +- **Excluded projections**: label、preview、solved content、node dimensions、layout、focus and “new to this scene” state do + not enter provider results。They are produced after retrieval by the consuming InfoBase View and its local Resolver/UI + capabilities。 +- **Owner boundary**: Resolver owns solved-content interpretation；InfoBase View owns its preview/full rendering and scene + composition；GraphNavigationRetrievalManager owns selection of existing graph facts。Peer delegation does not collapse + those owners merely because a remote runtime executes retrieval。 +- **Consequence**: a scene computes graph delta by entity identity when it merges a response，then lazily resolves visible + previews。The provider neither knows caller scene state nor renders presentation fallbacks。 +- **Confidence**: Sir stated this is a necessary boundary because presentation belongs to InfoBase View，not info-base + retrieval。 + +### D-352 — One-hop expansion is endpoint-closed、directional and cursor-bounded + +- **Block expansion**: return the addressed focal Block、one bounded deterministic page of incident Relations and every + opposite endpoint Block。An existing isolated Block succeeds with itself and no Relations；a missing Block is not-found。 +- **Relation expansion**: return the addressed Relation and both endpoint Blocks；a missing Relation is not-found。Every + result is endpoint-closed，so no returned Relation refers to an omitted Block。 +- **Selection**: Block expansion accepts `in`、`out` or `both` direction with `both` as default，a default limit of 20 and + hard maximum 100。Relations are ordered by stable descending Relation ID，which is an ordering identity rather than a + semantic recency claim。 +- **Continuation**: use an exclusive nullable Relation-ID `next_cursor` and fetch `limit + 1` to distinguish a complete + result from a truncated page。Insertions after the first page wait for explicit refresh rather than destabilizing the + current continuation sequence。 +- **Exact filtering only**: optional exact Relation-content values enter MVP。Do not add prefix、regex、JSONPath or an + implicit parser for arbitrary string/JSON Relation content；those would be a query language with different complexity。 +- **No caller-state leakage**: do not return frontier or new/existing flags。Every returned entity is a possible next + address，and caller scene merge determines graph delta。 +- **Confidence**: Sir accepted the complete proposed one-hop contract and exact-content-only filter boundary。 + +### D-353 — Bounded path returns one honest shortest-by-hop result under two independent bounds + +- **Contract names**: the path is addressed by `from` and `to` Blocks，not `source` and `target`。This aligns with persisted + Relation direction and avoids collision with the upstream InfoBase `Source` domain。Python may use `from_` with wire alias + `from` or explicit `from_block` / `to_block` parameters because `from` is a language keyword；that syntax does not rename + the public concept。 +- **Path selection**: return at most one shortest-by-hop path。Default direction is `both`，with optional `in` / `out` and + the same exact Relation-content filter as one-hop。Persisted Relation rows retain true direction even when traversal walks + against it。 +- **Bounds**: use both a hop bound and an explored-Block budget because one hub can explode within one hop。Working defaults + are 4 / hard maximum 8 hops and 1,000 / hard maximum 10,000 explored Blocks；preflight may tune exact values without + removing either public bound。 +- **Outcomes**: distinguish `found`、`not_found` within the requested bounds and `limit_reached` before exhaustive bounded + search。Do not report budget exhaustion as absence。A Block connected to itself yields one Block and zero Relations。 +- **Evidence**: a found result contains only the final endpoint-closed graph and an ordered Block-ID / Relation-ID path with + `len(blocks) = len(relations) + 1`；do not expose search working sets。 +- **No hidden ranking**: MVP does not penalize hubs、special-case Source/Mailbox or return multiple/semantically ranked paths。 + Such policies are future composition，not shortest-path retrieval。 +- **Technical ownership**: GraphNavigationRetrievalManager owns bounded bidirectional BFS and issues batched frontier fact + queries；do not place traversal business logic in a PostgreSQL RPC。Migration preflight must provide appropriate Relation + endpoint indexes。 +- **Confidence**: Sir corrected the endpoint names to `from` / `to` for consistency and precision，and accepted the remaining + path contract。 + +### D-354 — GraphModel is the endpoint-closed read shape for persisted graph authority + +- **Shape**: `GraphModel` contains persisted `BlockModel[]` and `RelationModel[]` with unique entity IDs and an invariant + that both endpoints of every Relation occur in its Blocks。 +- **Read/write distinction**: `GraphModel` is the read-side graph value；producer `GraphForm` remains an ID-free/signed-ref + write command。Neither is a database table，and GraphModel does not claim to represent the whole info-base。 +- **Reuse boundary**: one-hop and path results wrap GraphModel with only their navigation evidence。InfoBase View and Agent + consumers may reuse the shape without importing Graph Form submission semantics or presentation fields。 +- **Confidence**: Sir explicitly accepted adding GraphModel。 + +### D-355 — Graph navigation executes locally in database-capable Peers；MVP has no delegation capability + +- **Peer-parallel implementation**: client-web `@inkcre/core GraphNavigationRetrievalManager` executes through existing + PostgREST Block/Relation authority；core-py `GraphNavigationRetrievalManager` executes through SQLModel。Both implement the + same domain contract locally rather than requiring identical code。 +- **Why local**: unlike semantic retrieval、Mail collection or another asymmetric provider capability，graph navigation + needs no Peer-local model、secret connection or background worker。All current consumers can read the same deployment + graph authority，so delegation would add network latency and core-py availability dependency without new capability。 +- **MVP exclusion**: do not add `core.graph_navigation_retrieval.v1` Peer advertisement、outbound or fixed HTTP inbound。 + This is not opposition to remote access；it is absence of a current consumer that needs it。 +- **Future extension**: if a future consumer cannot access database authority，add a remote adapter behind the same domain + manager methods。Do not redesign GraphModel、one-hop or path contracts merely to introduce transport。 +- **Parity proof**: use one shared behavioral corpus/contract to verify TypeScript/PostgREST and Python/SQLModel outcomes， + including ordering、continuation、filters、path bounds and endpoint closure。 +- **Confidence**: Sir accepted peer-parallel local execution and cancellation of Graph Navigation Peer delegation/inbound。 + +### D-356 — InfoBase View may use recall for an unresolved scene；it does not own recall + +- **Owner correction**: InfoBase View owns presentation、navigation-host composition and realization of an existing graph + selection。Rendering or invoking a lexical-recall control in a fallback state does not move recall into the View domain。 +- **Initial modes**: Graph View may initialize from one random focal Block、a bounded random Block set or an unresolved/404- + like state that composes a Home-like lexical recall surface。These are alternative scene-seeding modes，not new graph- + navigation retrieval primitives。 +- **Composition boundary**: a recall component owns query、results and LexicalRetrievalManager use，then hands selected Block + authority to Graph View as seeds。GraphSurface does not implement lexical ranking/search，and the design system does not + absorb the product-aware recall component。 +- **Random boundary**: random Block selection is a basic seed policy over Block authority；it does not claim to be global + graph overview、feature retrieval or semantic relevance。 +- **Confidence**: Sir rejected lexical recall as part of InfoBase View's definition and supplied random-focal、random-set and + recall-backed unresolved modes as the correct composition alternatives。 + +### D-357 — Random focal is the default Graph scene initializer；other seed modes remain explicit + +- **Default**: Graph overview selects one random Block as focal and performs one bounded one-hop expansion，producing an + immediately navigable local scene without claiming relevance or a global overview。 +- **Alternative modes**: bounded random Blocks provide a serendipitous、possibly disconnected seed set；an unresolved/404- + like state composes external recall to obtain an intentional seed。Neither alternative changes Graph Navigation retrieval。 +- **Empty/missing behavior**: an empty info-base or missing addressed Block can enter the unresolved recall-backed state；do + not substitute recent Blocks or all graph authority merely to fill canvas。 +- **Confidence**: Sir accepted random focal as default and accepted the remaining initial-mode boundaries。 + +### D-358 — Application-level Recall opens globally and delegates result presentation to the active InfoBase View + +- **Global launcher**: mount one application-owned Recall/Search launcher at app level and open it with `Ctrl+K`，with + `Meta+K` as platform-equivalent support。Keyboard registration、lexical query semantics and destination selection do not + belong to GraphSurface or the design system。 +- **Presentation routing**: Recall results default to the List View。When the current page already is or hosts an InfoBase + View，that active View presents/consumes the result according to its own surface semantics；Graph uses selected Blocks as + scene seeds，while List renders ranked recall results。 +- **Ownership remains separate**: composing or hosting recall does not make lexical retrieval part of InfoBase View。Recall + owns query/result capability；the selected View owns presentation of the result it receives。 +- **Reusable application component**: Home/List and the Graph unresolved state may reuse the application-aware Recall/Search + composition rather than duplicating LexicalRetrievalManager and routing behavior。 +- **Design-system promotion**: add a generic `InkSearchBar` only for domain-neutral query input/submission/clear/loading and + accessible presentation。It must not know keyboard-global registration、Lexical Retrieval、InfoBase View or destination + routing。`SearchBar` is chosen over the overly broad `Composer`；application Recall composes it。 +- **Confidence**: Sir proposed the application-level `Ctrl+K` launcher and active-View/default-List result routing，accepted + application Recall reuse，and delegated the generic component judgment；the promotion decision follows the approved + abstraction threshold and current non-conforming Home search evidence。 + +### D-359 — URL query is the Recall-to-View handoff authority；do not add a global result store + +- **Handoff**: app-level Recall submission writes `?q=` onto the current route when it is or hosts an InfoBase View；otherwise + it navigates to List overview with that query。The active View consumes the query through the shared application recall + composition and presents the result according to its surface semantics。 +- **View behavior**: List replaces its ranked result presentation for the query；Graph merges matched Blocks into its scene + and focuses that seed set。This is use of Recall output，not Recall ownership by either View。 +- **One state authority**: URL query owns navigation、refresh、deep-link and browser-history semantics。Do not synchronize a + second global recall-result store with URL and View-local state；runtime results may remain ordinary query caches/local + state。 +- **Extensibility**: current routes can declare InfoBase-View hosting through application route metadata。A future hosting + page can opt in without the global launcher importing concrete List/Graph components or expanding InfoBaseRouter into a + recall router。 +- **Confidence**: Sir accepted the URL-query handoff and explicitly judged it elegant。 + +### D-360 — Relation is an addressable InfoBase route with its own modeless Inspector + +- **Router contract**: extend InfoBaseRoute with `{ name: "relation", relation: RelationRef }` alongside overview、Block and + Block solved-content routes。List/Graph client adapters add the corresponding Relation destination path。 +- **Graph interaction**: selecting an edge/label focuses that Relation and both endpoint Blocks。A direct Relation route + executes relation expansion to seed an endpoint-closed GraphModel before realizing the outlet。Opening its modeless + `RelationInspectorPopup` is a separate inspection action as corrected by D-363。 +- **Inspector content**: present persisted direction as from-Block label/preview → Relation content → to-Block label/preview， + plus Relation identity、updated time and navigation to either endpoint。Presentation may resolve endpoint labels locally。 +- **No solved-content fiction**: Relation has no Resolver and does not gain a Solved Content route。Its directed dynamic- + property value remains first-class without pretending it is a Block。 +- **Component boundary**: retain explicit Block and Relation Inspector components。Do not preemptively create a generic + EntityInspector；only a genuinely repeated application popup shell may be extracted later，and it does not enter the + design system merely because both Inspectors use it。 +- **Cross-surface value**: List can use the same Relation route for future semantic-retrieval Relation matches，so this is an + InfoBase navigation contract rather than Graph-only edge decoration。 +- **Confidence**: Sir accepted the Relation route and independent modeless Relation Inspector design。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D361-D370.md b/tasks/knowledge-lifecycle-capabilities/decisions/D361-D370.md new file mode 100644 index 0000000..c054b03 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D361-D370.md @@ -0,0 +1,164 @@ +# Decisions D-361–D-370 + +> [Decision register index](index.md) + +### D-361 — Graph canvas owns progressive one-hop exploration；Inspectors remain entity-local + +- **Exploration owner**: changing the focal Graph entity may invoke bounded one-hop retrieval directly from the canvas。 + One-hop is not hidden behind a Block Inspector action；the canvas is the progressive graph-navigation surface。 +- **Inspector boundary**: `BlockInspectorPopup` and `RelationInspectorPopup` inspect only the addressed entity itself。 + They may expose entity-local facts and ordinary actions，but do not own graph expansion、path discovery or scene scope。 +- **Interaction constraint**: Graph nodes remain draggable preview surfaces，so progressive expansion must be expressed by + focal selection and canvas-level state instead of embedded node buttons or inline expansion controls。 +- **Open design**: discrete focal-scene scales with bounded entity budgets are a promising model，but their exact semantics、 + relation to camera zoom and scene-retention behavior remain under review rather than being fixed by this decision。 +- **Confidence**: Sir explicitly required one-hop exploration in the Graph canvas and narrowed both Inspectors to inspecting + their addressed Block/Relation；the multi-scale focal proposal was intentionally presented as a hypothesis。 + +### D-362 — Progressive focal exploration uses a bounded active scene over a reusable session cache + +- **Active scene**: the visible Graph scene contains the current focal entity and its bounded direct context at the selected + exploration scale。Changing focal does not retain every entity visited during the session merely to create continuity。 +- **Session cache**: Graph View may retain already retrieved Blocks、Relations、continuation state and layout measurements in + its runtime cache。Returning to a known focal can therefore avoid unnecessary retrieval and preserve spatial familiarity。 +- **Continuity rule**: entities shared by the old and new bounded scenes retain their positions and transition continuously； + entities outside the new scene leave visibility。The previous focal remains visible only when admitted by the new scene， + for example when it is a direct neighbor of the new focal。 +- **Two scales**: exploration scale controls the bounded graph neighborhood；camera zoom controls only visual magnification。 + Pan、pinch and wheel zoom do not silently issue retrieval requests or alter the exploration budget。 +- **Relation focal**: a focal Relation admits that Relation and its two endpoints。Selecting an endpoint can then make the + Block focal and continue ordinary one-hop exploration。 +- **Confidence**: Sir accepted the bounded-active-scene plus session-cache model after comparing it with full replacement and + unbounded accumulation。 + +### D-363 — Primary Graph activation navigates；inspection is an explicit secondary action + +- **Primary activation**: a single click/tap on a Block or Relation changes the focal entity and updates the bounded active + scene。It does not automatically open an Inspector and interrupt progressive exploration。 +- **Canonical inspection action**: a canvas-level contextual `Inspect` action opens the Inspector for the current focal + entity。The action belongs to the Graph navigation host，not to draggable node/edge content。 +- **Shortcuts**: double activation and keyboard `Enter` may invoke the same inspection action。They are accelerators rather + than the only discoverable route。 +- **Close semantics**: closing an Inspector remains `InfoBaseRouter.back()`；it does not undo or mutate the Graph scene that + was already formed by focal navigation。 +- **Responsibility summary**: primary interaction navigates the graph；secondary interaction inspects one entity；Inspectors + remain entity-local；the canvas owns exploration。 +- **Confidence**: Sir accepted this exact interaction split and the correction to D-360。 + +### D-364 — Focal Graph canvas does not own path discovery；application Search may compose it + +- **Canvas boundary**: the focal Graph canvas owns progressive bounded one-hop navigation。No demonstrated focal-mode need + justifies placing `Find path` in its toolbar or inventing a two-Block canvas-selection workflow。 +- **Ownership correction**: application-owned Search may expose a path-discovery mode and invoke the bounded `find_path` + primitive itself。It must not model this as Graph handing control to lexical Recall merely to acquire a second endpoint。 +- **Capability remains atomic**: bounded path retrieval remains in `GraphNavigationRetrievalManager` and its peer-local + implementations。Removing a Graph-canvas command does not remove the primitive or collapse it into lexical retrieval。 +- **Presentation independence**: Search may later choose an InfoBase View to realize a path result；Graph's suitability for + displaying a path does not make GraphSurface the owner or initiator of path discovery。 +- **Confidence**: Sir found no focal-canvas path need，rejected the Graph-to-Recall target-picker flow and accepted path as an + application-owned Search operation instead。 + +### D-365 — Exploration scale is one Graph View density preference with discrete Relation budgets + +- **View-level state**: the active Graph View holds one exploration scale and carries it across focal changes。It expresses + the user's current desired scene density rather than becoming per-entity graph metadata or per-focal preference state。 +- **Budget unit**: each scale caps admitted incident Relations；endpoint closure determines the corresponding Block count。 + A Relation focal still admits exactly its two endpoints，so the scale has no material effect there。 +- **Discrete model**: use three UI levels with preflight starting points around compact `8`、standard `20` and broad `50` + Relations；standard is the default。Exact numbers are visual calibration values，not durable retrieval-contract constants。 +- **Continuation**: increasing scale continues the current ordered one-hop request from its cursor；decreasing scale hides the + excess but retains session cache。When a next cursor exists，the UI may say that more connections exist without issuing a + separate total-count query。 +- **Camera/layout trigger**: focal changes and explicit exploration-scale changes may update layout and perform bounded camera + fitting。Ordinary camera pan/zoom remains presentation-only as required by D-362。 +- **Confidence**: Sir accepted one unified Graph View density scale instead of remembering separate scales for each focal。 + +### D-366 — Focal layout is deterministic and scene-scoped；manual positions are session projections + +- **Focal topology**: hard-cut the old all-database/community/layout-selector machinery from focal mode。A bounded Block + one-hop scene is a star/multistar，so it uses a deterministic focal radial layout rather than asking users to choose among + force、dagre、grid、circular and radial algorithms。 +- **Intrinsic measurement**: layout consumes measured preview node bounds and may use multiple rings as density grows。It does + not assume equal circular nodes or use one fixed force-collision radius for heterogeneous resolver previews。 +- **Transition**: shared entities preserve measured positions long enough to transition into the next deterministic scene； + semantic focal/scale changes perform one bounded camera fit after layout settles。Continuous force simulation and repeated + timer-based fitting are not the interaction model。 +- **Parallel Relations**: multiple Relations between the same endpoint pair receive deterministic sibling lanes so edges、 + labels and hit targets remain distinguishable。 +- **Drag authority**: manual node positions are runtime projections cached by focal-scene identity and exploration scale。 + Returning to that session scene may restore them；another focal scene owns another layout。Positions are not Block/Relation + authority and are not persisted to the info-base。 +- **Accessibility**: transitions honor reduced-motion preferences。 +- **Community boundary**: this hard cut does not deny future community scenes；community remains an optional scene producer， + not a meaningful layout selector over a one-hop star。 +- **Confidence**: Sir accepted scene-scoped session retention for dragged positions after implementation preflight exposed the + old all-database layout topology and its mismatch with intrinsic one-hop scenes。 + +### D-367 — Graph focal uses minimal query state；InfoBaseRouter owns only the optional outlet + +- **Layering**: GraphSurface owns concrete focal-scene navigation；the client implementation of `InfoBaseRouter` owns an + optional Block、Relation or Solved Content outlet over that scene。Do not add Graph-specific focal semantics to the shared + `InfoBaseRoute` contract。 +- **URL shape correction**: keep the Graph page path stable and encode the reconstructive focal reference as role-named + optional parameters `focal_block=<id>` or `focal_relation=<id>`。Exactly one may be active。Do not introduce a discriminator + plus generic reference when the small closed alternatives already have distinct semantic reference roles。 +- **Minimal route authority**: only focal identity enters route state。Exploration scale、retrieval continuation、measured and + dragged positions、camera and session cache remain runtime presentation state。 +- **Outlet paths**: Block/Relation/Solved Content routes remain concrete nested destinations while preserving the focal query。 + `InfoBaseRouter.push()` adds an outlet；Inspector `back()` removes it and reveals the same focal scene。 +- **History**: focal activation pushes a query-state transition。Random default initialization replaces the empty Graph URL + with its chosen focal，avoiding an artificial history entry while making refresh reproducible。 +- **Search handoff**: `q` owns the recall-seed scene only while no focal is selected。Choosing a focal pushes a canonical focal + query without `q`；browser Back returns to the prior query-seed scene instead of keeping two competing scene authorities。 +- **Confidence**: Sir accepted scene/outlet separation and explicitly selected query parameters when scene state must be + represented in routing，then corrected the query shape to role-named references。 + +### D-368 — Resolver renderers are shared contracts；View loading orchestration remains specialized + +- **Shared contract**: Resolver classes expose presentation-neutral preview and solved-content renderers over the same solved + content contract。This is the useful maximum common denominator across InfoBase Views。 +- **No shared View workflow**: Graph and List are not forced through one loading/layout component merely because they consume + the same renderer contract。Graph owns focal-first bounded concurrency and measured radial reflow；List may own independent + row/virtualization/loading behavior appropriate to its surface。 +- **Graph progression**: Graph renders structural shells immediately，resolves focal first，then proactively resolves admitted + neighbors through bounded concurrency。Failure remains local to one preview；queued work for a departed scene may be + cancelled；no hidden retry is added。 +- **Context completeness**: scale-bounded Graph Relations are structural evidence and cannot masquerade as a Resolver's full + relation context。Resolver retains its own complete-context loading contract。 +- **Layout response**: intrinsic preview measurements are batched before deterministic reflow；individual completions do not + each take camera ownership。 +- **Retrieval wording**: Graph Navigation Retrieval returns bounded、direction-preserving、endpoint-closed graph evidence for + any application consumer。Fast navigability is a Graph View objective，not the retrieval domain's defining purpose。 +- **Confidence**: Sir accepted the progressive Graph preview model while explicitly rejecting both a lowest-common-denominator + View workflow and UI-oriented wording for the retrieval capability。 + +### D-369 — Graph direction is soft visual emphasis，not scene filtering + +- **Scene membership**: focal Graph View retrieves/adopts its scale-bounded `both` neighborhood regardless of the current + direction emphasis。Direction does not remove Relations or endpoint Blocks from the active scene。 +- **Presentation**: `all`、`incoming` and `outgoing` select an emphasis mode。Relations outside the active direction and Blocks + connected only through those Relations receive a disabled-like lower-opacity presentation but remain interactive。 +- **No structural churn**: changing direction performs no retrieval、layout、camera fit or scene-cache fork。Accordingly， + direction is not part of the `focal + scale` scene cache identity。 +- **Interaction**: hover/activation can temporarily restore strong contrast for a dimmed Relation；clicking it remains a valid + focal-navigation action。Opacity alone is not used as the only accessible indication of the selected emphasis control。 +- **Atomic API remains**: the one-hop retrieval contract still supports hard `in/out/both` direction selection for callers + that genuinely need a directional result。Graph View's softer consumption does not weaken that domain primitive。 +- **Confidence**: Sir explicitly replaced the proposed Graph hard filter with disabled-like visual treatment for Relations + outside the currently effective direction。 + +### D-370 — Focal emphasis preserves intrinsic size and uses restrained contrast hierarchy + +- **No size semantics**: focal status never enlarges a Block Node。Intrinsic dimensions remain owned by the resolver preview + presentation，so focus does not masquerade as information importance or force avoidable radial reflow。 +- **Contrast first**: focal emphasis may use a clearer hairline border and an extremely subtle or absent halo；the stronger + mechanism is lowering non-focal/context contrast while keeping text、direction and interaction accessible。 +- **Relations**: incident Relations gain crisp contrast relative to contextual Relations，supporting focal comprehension + without decorating the Block with multiple redundant signals。 +- **Restraint**: elevation stays minimal and is not a default focal signal；z-index is used only to solve an actual overlap/hit + problem，not as ritual state styling in a non-overlapping automatic layout。 +- **Exploratory visual direction**: pursue a restrained、cool、professional and sharp language through geometry、spacing、 + typography、hairlines、neutral color and precise motion。Do not prematurely freeze token values or mistake gradients、large + radii、heavy shadows、glass effects and saturated accents for polish。 +- **Confidence**: Sir completely accepted size-preserving focus，preferred context weakening over accumulated focal effects， + cautioned against elevation/z-index excess and supplied the desired visual character as an exploratory direction。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/D371-D380.md b/tasks/knowledge-lifecycle-capabilities/decisions/D371-D380.md new file mode 100644 index 0000000..ef83dc1 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/D371-D380.md @@ -0,0 +1,94 @@ +# Decisions D-371–D-380 + +> [Decision register index](index.md) + +### D-371 — Neighborhood retrieval uses role-specific methods；deep interfaces optimize clarity，not superficial width + +- **Methods**: `GraphNavigationRetrievalManager` exposes `get_block_neighborhood(...)` and + `get_relation_neighborhood(relation)` rather than a generic graph-entity method。`find_path(...)` remains independent。 +- **Vocabulary**: `neighborhood` names the graph-theoretic evidence returned by retrieval；`expand` remains a possible + consumer interaction that changes a Graph scene and does not leak into the domain method name。 +- **Different contracts stay visible**: Block neighborhood owns direction、exact contents、limit and cursor；Relation + neighborhood is exactly the Relation plus its endpoints and needs none of those options。A generic method would merely move + discriminator and valid-option knowledge into every caller。 +- **Results**: Block retrieval returns `BlockNeighborhood | None`，where the result contains only `graph` and nullable + `next_cursor`；Relation retrieval returns `GraphModel | None`。The request already owns the focal reference，so the response + does not repeat it as a second authority。 +- **Deep-module lesson**: a good deep interface is not one with the fewest method names at any cost。It is one that exposes + the clearest stable semantic operations while hiding routing、query construction、pagination and endpoint-closure + complexity from callers。 +- **Confidence**: Sir accepted the exact role-specific neighborhood methods and graph-theory vocabulary，and explicitly + strengthened the interpretation of deep-module interface quality。 + +### D-372 — Path outcomes form a discriminated union with evidence only on success + +- **Union**: `find_path(...)` returns `PathFound | PathNotFound | PathLimitReached`，discriminated by exact `status` values + `found`、`not_found` and `limit_reached`。Do not use one flat model with nullable graph/path fields and invalid combinations。 +- **Found evidence**: only `PathFound` carries the endpoint-closed `GraphModel` plus ordered `block_path` and + `relation_path` references，with `len(block_path) = len(relation_path) + 1`。 +- **Zero-hop case**: when `from == to` and the Block exists，the found path contains that one Block、no Relations and a + one-Block GraphModel。 +- **Shallow negative outcomes**: `PathNotFound` proves absence within the requested hop bound；`PathLimitReached` says the + explored-Block budget prevented that proof。Neither leaks the traversal working set or fabricates a partial path graph。 +- **Confidence**: Sir accepted the outcome-specific discriminated union over nullable result fields。 + +### D-373 — Random focal is a Graph initialization policy over a Block-owned random-row primitive + +- **Ownership**: Graph View decides to initialize from a random focal；Block access owns obtaining one random persisted Block。 + Do not add `get_random_focal` to Graph Navigation Retrieval or load every Block into GraphSurface for browser sampling。 +- **Interfaces**: core-py adds `BlockManager.get_random() -> BlockModel | None`；client-web's Active Record peer adds + `Block.getRandom() -> Promise<Block | null>`。The differing style follows each peer's existing Block boundary while the + behavior remains equivalent。 +- **Mechanism**: count current Blocks，choose a non-cryptographic random offset in `[0, count)`，then read one row under stable + ID ordering。This avoids a PostgreSQL business RPC and avoids transferring the whole ID set。 +- **Race semantics**: an empty table or a count/fetch deletion race may return missing and enter the already-approved + unresolved Search state。Do not add a hidden retry for this low-harm initialization race。 +- **MVP stop line**: implement one random Block only。The accepted bounded-random-set alternative remains a possible later + explicit initializer；it does not currently justify efficient uniform sampling-without-replacement machinery。 +- **Confidence**: Sir accepted the owner、two peer-native interfaces、count/offset mechanism and singular MVP boundary。 + +### D-374 — Equal shortest paths have no public tie-break；Acceptance verifies semantic properties + +- **Contract correction**: `find_path` guarantees one valid shortest-by-hop path，not a preferred path among equal-hop + alternatives。Relation database identity does not become semantic ranking or a public lexicographic tie-break。 +- **Implementation freedom**: a peer may iterate Relations in stable ID order for reproducibility and debugging，but callers + cannot depend on which equal shortest path that implementation happens to return across peers or versions。 +- **Acceptance**: unique-shortest fixtures may assert an exact path。Ambiguous fixtures assert validity、direction/filter + compliance and shortest length against the legal result set；peer parity compares contract properties rather than exact + incidental selection。 +- **Consumer stability**: a UI that wants one fetched path to remain visually stable retains that `PathFound` result for its + session。It does not force a use-neutral provider to expose a UI-motivated tie-break。 +- **Confidence**: Sir accepted withdrawal of the previously proposed public Relation-ID tie-break and identified + Acceptance-driven incidental design as an anti-pattern worth retaining。 + +### D-375 — Application Search owns path intent；Graph realizes the routed result + +- **Search mode**: application-owned Search exposes `Find path` beside its default Recall behavior。The mode owns selecting + `from` and `to` Block references；Graph canvas does not gain a path command or endpoint-selection state machine。 +- **Endpoint picking**: Search may accept exact Block references and may use lexical retrieval to suggest endpoint candidates。 + That is internal Search composition，not Graph delegating an incomplete path operation back to Recall。 +- **Handoff**: Search writes reconstructive `path_from` and `path_to` query references on the Graph route。This path-address + form is mutually exclusive with focal and `q` seed forms；no global result store is introduced。 +- **Realization**: GraphSurface consumes the routed intent，calls `find_path` and presents found/not-found/limit-reached scene + states without automatic bound expansion or retry。It realizes the result but does not own the Search operation。 +- **Exit**: activating any entity in a found path removes path query state and enters the corresponding focal scene。Browser + Back restores the previous Search/scene authority。 +- **Confidence**: Sir accepted the application Search mode and query handoff after previously rejecting a Graph-owned target + picker。 + +### D-376 — Successful graph reads are endpoint-closed without claiming cross-statement snapshots + +- **Snapshot boundary**: neither peer promises one database snapshot across the multiple statements/HTTP requests needed to + read Relations and endpoint Blocks。Do not introduce a PostgreSQL business RPC、Serializable transaction or snapshot token + for this low-probability concurrent-mutation case。 +- **Neighborhood repair**: a missing focal returns `None`。Relations whose endpoint Blocks cannot be obtained are omitted + before GraphModel construction，so every successful page remains endpoint-closed；the original page still determines its + cursor and no hidden retry occurs。 +- **Relation neighborhood**: if the addressed Relation or either endpoint can no longer be obtained，return `None` rather + than a dangling graph。 +- **Path assembly**: traversal retains the Relation rows it already observed and assembles the candidate from that + operation-local read rather than rereading them to manufacture a concurrency error。If candidate endpoint Blocks have + disappeared before closure can be assembled，return ordinary `not_found`；do not make callers understand an internal + cross-statement race or perform a hidden retry。 +- **Confidence**: the implementation self-review corrected the earlier error-escalation choice under Sir's operational + guideline：the owning module resolves low-value edge/state inconsistency and keeps the public completion semantic shallow。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/index.md b/tasks/knowledge-lifecycle-capabilities/decisions/index.md new file mode 100644 index 0000000..3a82969 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/index.md @@ -0,0 +1,66 @@ +# Decision Register + +> Task-state decision memory. Hub and code remain the final owners after promotion. + +This directory is the single decision authority for the task packet。Decisions are sharded mechanically in groups of ten +so one stable ID has one predictable address；the shard boundary does not imply product or technical ownership。 + +## Navigation + +| Range | Current subject span | +| --- | --- | +| [D-001–D-010](D001-D010.md) | Program boundaries → memo-like integration relationships | +| [D-011–D-020](D011-D020.md) | Collection success / memo graph → CanonicalMemo persistence | +| [D-021–D-030](D021-D030.md) | Memos backend/resolver/identity → flomo deferral | +| [D-031–D-040](D031-D040.md) | Memos 0.29.1、deployment、auth/config → attachment order | +| [D-041–D-050](D041-D050.md) | Memos graph/API/storage close → feed-content authority | +| [D-051–D-060](D051-D060.md) | RSS rewrite、identity、enclosures → Block hydration | +| [D-061–D-070](D061-D070.md) | Peer storage/resolver/media contracts → Memos MIME policy | +| [D-071–D-080](D071-D080.md) | RSS/Atom media、resolver IDs、unit close → semantic-quality pressure | +| [D-081–D-090](D081-D090.md) | Organization/use boundary、embeddings、AI topology → row timestamps | +| [D-091–D-100](D091-D100.md) | AI Provider/Model/Manager → Relation semantic projection | +| [D-101–D-110](D101-D110.md) | Embedding freshness/config → deployment-config reference defense | +| [D-111–D-120](D111-D120.md) | Config naming、retrieval results → Peer discovery/lease | +| [D-121–D-130](D121-D130.md) | Peer module/delegation/protocol → HTTP inbound destination | +| [D-131–D-140](D131-D140.md) | Peer persistence/wire completion → early organization taxonomy | +| [D-141–D-150](D141-D150.md) | Rumination understanding/depth → Agent/Graph forms | +| [D-151–D-160](D151-D160.md) | StarsGraphForm、GraphForm references → AI tool-calling terminology | +| [D-161–D-170](D161-D170.md) | Agent loop bound → cancellable Thread/turn topology | +| [D-171–D-180](D171-D180.md) | Structured-concurrent Tool batch → Agent validation / graph commands / rumination Agent selection | +| [D-181–D-190](D181-D190.md) | Organization-facing rumination completion → Peer Acceptance closure | +| [D-191–D-200](D191-D200.md) | Exact-target Peer delegation → clean shared-database rebuild | +| [D-201–D-210](D201-D210.md) | Mail communication-record foundation → current scope | +| [D-211–D-220](D211-D220.md) | Mail collect-job freshness → multi-Peer direction | +| [D-221–D-230](D221-D230.md) | Resolver behavior/content layers → SolvedContentRenderer and BlockInspector | +| [D-231–D-240](D231-D240.md) | InfoBase web route projection → Source Block provenance | +| [D-241–D-250](D241-D250.md) | Relation predicate normalization → Mail source-native graph decomposition | +| [D-251–D-260](D251-D260.md) | Core Email body reuse → exact-occurrence Email identity | +| [D-261–D-270](D261-D270.md) | Canonical Email restoration → Mail sync checkpoint placement | +| [D-271–D-280](D271-D280.md) | Remote MIME reconciliation safety → current edge | +| [D-281–D-290](D281-D290.md) | Mail adapter command ownership → writable Source targets | +| [D-291–D-300](D291-D300.md) | Singular graph reads without duplicate-policy promotion → current edge | +| [D-301–D-310](D301-D310.md) | Cron coalescing/activation semantics → current edge | +| [D-311–D-320](D311-D320.md) | Mail access-context continuity → retrieval capability/unit distinction | +| [D-321–D-330](D321-D330.md) | Lexical document-body completeness → current edge | +| [D-331–D-340](D331-D340.md) | Media-interpretation convergence → multimodal AI、maintenance Jobs and Render self-host profile | +| [D-341–D-350](D341-D350.md) | Self-host/canonical-production separation → Graph focus-set / UI ownership boundary | +| [D-351–D-360](D351-D360.md) | InfoBase View initialization and recall ownership → active-scene navigation | +| [D-361–D-370](D361-D370.md) | Graph scale/layout/route state → endpoint-closed retrieval outcomes | +| [D-371–D-380](D371-D380.md) | Random focal/path ownership → concurrent-read and visual Acceptance closure | +| [Withdrawn frames](withdrawn.md) | Explicitly rejected organizing frames and proposals | + +## Register Rules + +- Decision IDs remain monotonic across shards；append the next ID to its numeric shard。 +- A later correction gets a new ID and names the superseded decision；do not rewrite history merely to remove disagreement。 +- Unit packets and design files cite decision IDs and link to this index or the exact shard；they do not duplicate decision + authority。 +- When a shard reaches ten decisions，create the next fixed-width range file and add one index row。 +- Task-state truth is promoted to the owning durable document only after implementation evidence and owner reconciliation。 + +## Current Edge + +- Latest confirmed decision: [D-376](D371-D380.md)。 +- Active unit: [graph-navigation-retrieval](../units/graph-navigation-retrieval/packet.md)。 +- Active surface: Product、Technical、Acceptance、implementation plan and preflight are frozen；Impact Handshake is the + final review surface，then execution waits for Sir's new explicit “开始”。 diff --git a/tasks/knowledge-lifecycle-capabilities/decisions/withdrawn.md b/tasks/knowledge-lifecycle-capabilities/decisions/withdrawn.md new file mode 100644 index 0000000..9795991 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/decisions/withdrawn.md @@ -0,0 +1,29 @@ +# Withdrawn Frames + +> [Decision register index](index.md) + + +- 以 `observation`、audit、replay 或“图准入”组织任务。 +- O-017 top-level protocol mount：MoeMemos 2.0.4 的 Retrofit endpoints 是 relative paths，登录 + host 保留 path，attachment URL 也在 host path 后追加 `file/...`；配置 + `https://<deployment>/memos/` 即可复用现有 `/{extension_id}` routes。原先“必须占用根级 + `/api/v1`/`/file`”的前提不成立。 +- 把 collection / organization / use 建模成信息生命周期。 +- 把 block / relation / resolver / storage 联合模型升级为独立前置主线。 +- 为了讨论依赖而将 application 移到 organization 之前。 +- 把 `SubGraphForm` 当作完整产品模型或 collection 产品产物。 +- 把 CanonicalMemo 仅建模为 transient normalized / solved model，而不持久化到 block + content。 +- 为 source-native identity 建立通用 `resource`、opaque `source_key` 或 + `source_block_bindings` 持久模型。 +- 把公开 IMAP/POP3 protocol 建模为 versioned InKCre adapter ID，并为一个当前 closed、one-implementation- + per-protocol 的集合引入 `MailManager`、runtime adapter registry 或 persisted adapter catalog。Protocol 是 + Source 配置的外部标准事实；adapter 是代码实现机制。 +- 在 `MailAdapter` 上暴露 `collect()` / `MailCollectRequest` / `MailCollectBatch`。`collect` 是 Source 将外部信息 + 带入 InfoBase 的领域 command；Adapter 只暴露 canonical Mail 级别的远端读取/变更流与 part fetch。 +- 为防御 Storage catalog 与实现类不一致而增加 `StorageManager.get_writable_storage()`。该一致性属于 Storage + registry/bootstrap 系统边界；每次使用时重新发现同一能力既没有独立领域语义，也会泄漏 registry 复杂度。 +- 将“先读 durable completion fact 再重建生产路径”（原候选 U-043）、“共享 matching mechanics 而 evidence + precedence 归领域 owner”（原候选 U-045）以及“semantic completion 不由底层副作用拥有”（原候选 U-046）提升为 + project-wide common patterns。它们在 Mail MIME materialization 内仍是有效设计解释，但 D-292 复审认为其跨单元 + 决策杠杆不足，不值得增加 durable vocabulary。 diff --git a/tasks/knowledge-lifecycle-capabilities/design-taste.md b/tasks/knowledge-lifecycle-capabilities/design-taste.md new file mode 100644 index 0000000..0230b2a --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/design-taste.md @@ -0,0 +1,76 @@ +# Design Taste and Discussion Filter + +> **Status: experimental，task-wide，not law.** This is active working-memory control，not durable product/technical truth。 +> Sir will judge and revise it through actual discussion experience。Canonical common-pattern descriptions remain in +> [documentation promotion](documentation-promotion.md)；this file keeps the small set needed before proposing architecture or +> asking Sir for a decision。 + +## Experimental Discussion Model + +The unit of progress is a more coherent、evidence-backed current system model，not another answered question or a longer +decision register。Sir's preference to ask one question at a time is an upper bound on simultaneous human review，not a +requirement to manufacture one question after every answer。 + +Before turning an unresolved point into a human question： + +1. reconcile the latest accepted decision with the owning Product/Technical model and derive its natural consequences； +2. classify each proposed value by authority、scope/cardinality and lifecycle。When several owners/lifecycles interact，draw a + small topology before designing their interfaces； +3. when behavior is recurring、asynchronous、partial or state-dependent，replay at least two executions in a sequence/state + model and identify the persisted fact that makes the second execution different； +4. eliminate choices already dominated by confirmed constraints and marginal utility； +5. if one coherent answer remains，record/present the derived result without asking。Only surviving credible forks enter human + review，one at a time。 + +This workflow is deliberately experimental。Topology and sequence models are tools selected when they expose the relevant +dependency or time behavior，not compulsory diagram artifacts for every small naming or mechanical decision。 + +## Before Escalating a Design Question + +Run every candidate through these filters first： + +1. **Authority and lifecycle**：does the proposal make one owner/progress cursor depend on an orthogonal lifecycle merely + because the mechanisms are adjacent？If so，separate them unless correctness evidence requires coupling。 +2. **Marginal utility**：compare unresolved harm and recovery topology against dependency、obscurity、maintenance and new + failure modes。Stop when the remaining harm is cheaper than the next mechanism（U-011、U-033、U-037、U-044）。 +3. **Deep-module completion**：keep public completion semantics shallow；do not force callers or generic infrastructure to + understand internal residue、retry、created/existing or domain completeness that they cannot use（U-041、U-042）。 +4. **Primary versus orthogonal effects**：do not hold accepted primary progress behind a lower-value best-effort side effect + when that failure neither invalidates the primary fact nor prevents safe future operation（U-048 candidate）。 +5. **Natural consequence**：derive low-risk names、mechanical validation、ordinary error mapping and dominated choices without + asking Sir to select them。Record the result and expose it at the batch boundary。 + +A human decision question is justified only when at least two **credible、non-dominated** answers remain after those filters， +and choosing among them materially changes observable product behavior、authority、public contract、irreversible effects or a +high-cost failure/recovery path。Missing evidence should trigger exploration，not a speculative choice。 + +## Operational and safety reasoning discipline + +1. Do not turn ordinary implementation review into a broad safety or security audit。Safety reasoning starts only from a + specific actor、capability、asset、boundary、harm and attack path confirmed to exist in the current scope。 +2. Prefer conventional platform/library controls and their normal verification surfaces。A novel security design or bespoke + security verification needs a concrete uncovered attack path and demonstrated return。 +3. Operational safeguards must preserve observability：record actionable internal context even when the public completion + semantic is intentionally shallow。 +4. Do not escalate an ordinary edge/state race as `fail-fast` or `fail-closed` work for the Human or caller。Reconcile it at + the owning boundary、return the domain's ordinary completion outcome，or expose a repair action only when the caller can + meaningfully perform one。 + +## Current Failure Reference + +Mail ordinary collection had already persisted valid graph facts。Making a failed `mark_as_seen` attempt block the mailbox +checkpoint would repeatedly re-fetch the delta、possibly pin progress on a permanent external error and couple collection +authority to a workflow convenience。The isolated harm—one message remains unseen and the Job reports a diagnostic—is cheaper +and recoverable。This was a dominated proposal and should never have been escalated as a product decision。 + +Media-interpretation routing exposed the deeper discussion failure：after accepting per-modality Agents，the next response +treated “produce another decision question” as progress and mechanically projected modality into Cron/Job parameters。The +existing facts already implied one parameterless convergence Job：Cron owns a static template，Organization derives modality per +candidate，and graph state changes the next candidate set。A topology plus two-occurrence sequence would have made that +implication explicit，but the root correction is to make model reconciliation—not question production—the work unit。 + +## Scope Discipline + +- Unit-specific anti-patterns stay in the owning unit packet；do not promote them merely because they occurred once。 +- Common-pattern candidates remain non-durable until implementation evidence passes the promotion test。 +- This filter guides discussion；it does not turn taste into validation rules or prohibit evidence-backed exceptions。 diff --git a/tasks/knowledge-lifecycle-capabilities/documentation-promotion.md b/tasks/knowledge-lifecycle-capabilities/documentation-promotion.md new file mode 100644 index 0000000..9fd9f6d --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/documentation-promotion.md @@ -0,0 +1,496 @@ +# Durable Documentation Promotion Plan + +## Control + +- **Mode**: Memos/RSS/semantic-retrieval/Mail shared promotion published。Mail PRD/Product-TDD truth is Hub `067c60a`； + core-py `8e07da8` and client-web `056c265` consume that exact published head through pure shared-ref commits。The Mail + implementation/local-doc owners remain separately committed as core-py `d3cded7` and client-web `1e69938`。 +- **Apply gate**: unresolved discussion pressure remains here；stable design + verified implementation triggers + durable projection during unit completion。Commit、push、Hub publication、shared-ref bump 与 production mutation + remain separately authorized operations。 +- **Owner rule**: Hub source、Spoke shared-ref、core-py Unit TDD 与 client-web docs 分属不同 + owner/operation，不混入一个 commit，也不在 `docs/_shared/**` 直接编辑。 + +## Promotion Test + +候选内容必须同时满足： + +1. 已从讨论假设升级为获批、稳定且可复用的产品或技术合同； +2. 有一手产品事实、当前实现证据或 acceptance fixture 支撑； +3. 唯一 owner 已确定，不复制同一事实； +4. 不把某个 Spoke 的偶然类名或临时 workaround 升级为共享合同； +5. 与已有 durable claim 冲突时明确写出 `From → To`，不静默叠加。 + +## Known Corrections to Existing Hub Truth + +以下纠错已在 Hub source worktree 中按 owner 应用；本表保留 From → To 的 promotion provenance： + +| Existing pressure | Intended correction | +| --- | --- | +| collection 被描述为产生 blocks/relations，同时 organization 又像是把 collected information 转成 blocks/relations | collection 为了持久化 source information 可以拆成 graph；organization 打理**已经存在**的 info-base，以改善 use | +| collection / organization / use 容易被读成信息状态或固定 lifecycle | 三者是对信息执行的能力动作，不新增未经需求证明的 lifecycle | +| breakdown/merge/linking 容易被当成 organization 的完备枚举 | 它们只是已知能力，目标始终是为 use 优化 info-base | +| indexing 容易被归入 organization | indexing 只作为 application/retrieval 支撑 | +| source-native objects 可能被误解成 graph 之外的持久模型 | Tweet/GithubRepo/FeedItem 等通过 blocks/relations 持久化；不建立通用 collected god object | +| resolver 的 `v1` / `v2` 轴被叫作 `generation` | 按开发者惯例叫 `resolver contract version`；这是一致性/自解释修正，不是声称 `generation` 在其他上下文均错误 | +| graph 中的 source/protocol object 被叫作 `wrapper` | 它是普通 block；在拥有与关联 semantic content 有关的 protocol/source-authored facts 时按职责叫 `metadata block`，不新增 wrapper type | + +## Candidate Hub PRD Batch + +### Program-level product truth projected to Hub source + +- collection、organization、use 的动作边界，以及 indexing 的归属。 +- InKCre 首先降低用户收集、整理和基本使用 info-base 的成本，使 information 的规模积累有机会产生质变； + 它不要求每个 source unit 预先证明某个具体知识产出。info-base 持有 information，knowledge 只存在于 + 用户脑内；被用户理解并用于价值落地的 information 才成为 knowledge，PRD 不应混用二者。 +- MVP / MLP 由用户 job、所得价值与可接受代价界定，不由协议完整性、数据字段或 feature 数量机械判定。 + 对 source system 的状态改变也不天然是错误副作用；它可以是生产工作流的一部分，但应服从清晰、可配置的 + 用户选择。该原则当前由 Mail product discussion 提出，等待 unit evidence 后决定最终 PRD 表述。 +- Mail 的长期产品方向不是只读邮件 collector：InKCre 应以尽可能完整的 communication record 为信息基础， + 最终成为完整的 email client/agent，覆盖收集、理解、组织、查询与有意邮件行动。该终局不自动把所有 + compose/reply/send、mailbox mutation 或 autonomous-agent behavior 纳入当前 delivery scope。 +- 当前 Mail delivery scope 只承诺 communication-record foundation、持续更新、resolver/use、邮件所需 minimum + basic query、必要 client-web journey 与可配置 `mark_as_seen`；compose/reply/send 作为 future scope 重新过 gate。 +- Mail Source setup 默认只开始 forward collection；历史邮件由用户显式运行可指定边界的 backfill collect。 + `collect` 是把系统外部信息带入 info-base 的总动作，`backfill` 是特殊 collect intent，不是平行能力。Legacy + `full` 不再承载历史收集语义；current-state 与 history-capable Sources 仍拥有不同 source-native boundaries。 +- `enrichment` 保留为 Organization 从既有 graph 出发、为改善 use 而增加信息或结构的 approach vocabulary。 + RSS collection-time `full-text enrichment` 应在对应 owner 中纠正为 `full-text acquisition`，不改变其既有 + source ownership 或行为；Mail 入库后展开正文 URL 是 enrichment 的 reference example。 +- Source 本身就是 collection behavior 的 owner，不使用冗余的 `source-owned collection`；按上下文称 + collection、source acquisition 或具体 acquisition operation。 +- Mail Source—not InfoBase—owns remote-deletion behavior。默认只移除 Email–Mailbox membership relation，不提交 + email graph deletion，也不新增 `has been deleted from` tombstone；opt-in synchronized deletion 默认关闭，且只在 + 协议提供可信增量删除证据时考虑，不通过全量遍历模拟。 +- Mailbox 与 tag-like Mail Flag 是独立 Blocks；membership/flag assignment 由 Relations 表达。Seen/Answered 不与 + tag-like flags 强行合并，exact canonical graph vocabulary 等待 Technical design。 +- Mail manual/scheduled collection 使用同一种 collect-job execution semantic；scheduler 只创建 jobs。本轮不要求 + IMAP IDLE。core-py 是当前 capable executor，不是 Extension 的永久 runtime owner；future native Peers 可以拥有 + 不同同步频率，但当前不提前引入 distributed scheduler 或跨 Peer cadence coordination。 +- Collect job 是 CronJob-like Source invocation envelope，不是 source-internal completeness/transaction unit。Source + 拥有 mailbox/page/item traversal、checkpoint、partial effects 与 failure isolation；job completed 只表示 Source + invocation 按其 shallow public completion semantic 返回，不承诺某个同步 horizon。 +- Collect job 是 one-shot、无 retry lifecycle；terminal 后不 reopen。后续 manual/scheduled invocation 创建新 job， + Source 仅通过自己的 state 继续收集，不引入 attempts、retry lineage 或 job backoff。 +- Mail collection 默认只持有 attachment metadata/remote reference，不下载实际 bytes 或写入 Storage。对既有 + Email/Attachment graph 进行 durable materialization、增加 semantic content Block 可以分类为 Organization + enrichment，但 Mail Resolver 拥有 materialization 深接口，并可委托 Source/extension 取得 bytes；classification + 不等于 implementation ownership。用户打开时的 transient fetch/stream 属于 use。 +- Mail collection 保存 authored `text/plain`/`text/html` body；CID image 等 non-text inline parts 只保存 + metadata/reference，查看时 lazy fetch，持久化仍走 Mail Resolver materialization。HTML remote resources 不自动 + 抓取，完整视觉 fidelity 可以依赖按需网络读取。 +- Mail collection 将 source-native reply/reference facts 表达为 reply Email → parent Email Relations；不创建 generic + Thread Block，也不按 subject/time/participants 推断。Thread view 从 graph 派生，missing-link inference 属于 + Organization linking。 +- client-web 不建立 Mail 专用 inbox/folder/message-list 页面或 Mail-only query UI。Mail extension 参考 Twitter + extension，通过 Email Resolver/content component 在 generic BlockContent、details、graph/query-result surfaces 中 + 渲染 Email Block；完整 email client/agent 是能力方向，不是传统邮箱 UI 复刻要求。 +- reply/reference 在 Email renderer 中表现为请求跳转到目标 Block 的 action（如“查看回复”），不是普通 links；只有 + GraphSurface 拥有 target Block selection/focus/route，Block details 只知道当前 focal Block。`contentComp`/ + `BlockContent` 当前名称与接口掩盖 renderer 实际消费 focal Block + Resolver-owned local graph projection，需在 + client-web Technical owner 中纠正。 +- `block.resolver` 应被明确为 exact Block behavior contract 的选择，而不只是 decoder/display discriminator； + client-web Resolver 注册 `SolvedContentRenderer` 呈现 solved projection，`BlockInspector` 通过“查看内容”动作进入 + view-solved-content lifecycle 并保留 rumination 等 current-Block commands。PRD 保留 Block/Resolver 的产品含义， + Product TDD 明确 persisted representation → hydrated content → solved content 的 authority/derivation 边界， + client-web local architecture 记录 controller/renderer/Inspector/GraphSurface composition 与 target-Block navigation + bridge。Graph-aware solved content 不得伪装成 canonical root content；`.root` 是 focal Block canonical parsed content， + relation-derived fields 只作为 siblings。 +- BlockInspector、GraphSurface、solved-content viewing 与 cross-Block navigation 属于 InfoBase domain。不要把 + navigation 缩成 renderer callback/context；实现证据应验证一等 `SolvedContentPopup` 与 InfoBaseRouter-like domain + locations 是否成为正确 owner，并让 GraphSurface 负责 route realization。Module Federation singleton/Resolver + registration evidence 不能被误读为禁止 `SolvedContentRenderer` 使用完整 Resolver。 +- `SolvedContentRendererProps` 同时包含 exact Resolver 与 typed solved content。InfoBaseRouter/GraphSurface/ + SolvedContentPopup 已成为 accepted client-web InfoBase topology：router owns location/history operations，GraphSurface + 是当前 route realizer，SolvedContentPopup owns Resolver/popup lifecycle；route 不以 graph/list 命名 presentation + surface。surface-independent domain routes 固定为 `overview | block | solved-content`，两个 focal routes 携带 + `BlockRef`；未来 ListSurface 可以实现同一 locations。InfoBaseRouter 不建立第二套 history 或独立的 + `InfoBaseHistory` domain module，而通过 router-internal replaceable adapter 使用 Vue Router/browser authority； + `back` 保持 literal history traversal。MVP public interface 仅为 read-only `current`、`push(route)` 与 `back()`； + `current` 在 app location 不属于任何 InfoBase surface 时为 `null`，不是第四种 domain route；不公开无 caller 的 + `replace()`，也不建立 arbitrary extension route registry。GraphSurface application URLs 映射为 + `/info-base/graph`、`/info-base/graph/blocks/:block` 与 `/info-base/graph/blocks/:block/content`；URL 中的 `graph` + 选择 current surface，不进入 domain route。adapter 从 Vue Router 直接派生 current，不维护 location mirror。 + InfoBaseRouter 的 shared topology 不是 history/codec composition：`@inkcre/core` 只拥有 fixed contract 与 singleton + implementation binding，各 client 完整实现自己的 navigation projection；GraphSurface/ListSurface/renderers 是 + consumers。共享层无 route/history state 或 route registry，已撤回 shared `InfoBaseRouterHistoryAdapter` 与 codec。 + Singleton binding 使用已有 MFImplementation 同类的 module-scoped set/get/fail-fast pattern；不要因为两个实例就 + 抽出 generic runtime-binding helper/registry。缺失 binding 是 bootstrap composition error，不是 optional state。 + InfoBaseRouter 只验证/投影 client route syntax，不查询 Block existence；malformed/unmapped location 归 app-level + not-found，合法 BlockRef 指向 missing row 时由 GraphSurface/SolvedContentPopup 的加载生命周期呈现。 + GraphSurface 对三种 route 的 realization 是 graph-only、graph + `BlockInspectorPopup`、graph + + `SolvedContentPopup`；一等 destination 不等于页面。popup 自己拥有 close 并调用 literal Router.back，surface 不猜 + parent route。GraphSurface/ListSurface 稳定归类为 `InfoBaseView` navigation hosts，并以 route destination outlet + 实现 route。`SolvedContentView` 撤回。 +- Product TDD 应沉淀 UI container/content rule：presentation-neutral content 不选择 popup/drawer/card/list item 等 + container，通常由 parent composition owner 组装；只有 container lifecycle 本身成为 addressable route + destination behavior（例如 dismiss → Router.back）时，destination component 才自带 shell。该规则未来应派生进 + UI agent skill，但 Mail unit 不负责补建尚不成熟的 UI skill build/delivery infrastructure。 +- InfoBaseRoute 是 InfoBaseView 的唯一 focal-Block authority；GraphSurface 作为 realizer 正确理解 stable + `route.name` 并把它映射为 graph focus/outlet composition，不保存另一份 selected-Block identity，也不在 realization + 时反向 push。Vue route name/path 仍只属于 client implementation。 +- `BlockInspectorPopup`/`SolvedContentPopup` 的 route input 都是 BlockRef；destination 自己拥有 Block loading、 + missing/error 与 close/back lifecycle，SolvedContentPopup 还拥有 Resolver/solving/refresh/dispose。InfoBaseView 的 + graph/list projection 偶然持有的 Block 不是 popup resource provider；未来只在真实成本下引入 Block cache owner。 +- Mail unit 不预设 generic query increment；只有 preflight 证明现有 generic surfaces 无法合理触达 Email Blocks 时， + 才提出移除 blocker 的最小 generic query change。 +- memo-like 是独立、多端、低摩擦的 collection surface，用于记录想法、周围事物与零碎 + 信息，不承担 info-base 的查阅/use。 +- 长期产品方向允许 memo extension 分别支持 backend 与 collector，但两者是独立产品关系、 + 独立 deliverable，不共享一个含混的“同步”合同。 +- Memos 是首个 memo → graph reference product。 +- 当前 InKCre 不建立 terminal-user、tenant 或 per-user ownership/ACL；deployment 是单一 + user/owner context，多 runtime `client` 是 peers 而不是人类用户。external account/user 只在 + source/protocol 边界拥有其原生语义。 +- Extension APIs default to core peer authentication, but an extension may explicitly own its protocol + route-auth composition；Memos uses an auth-neutral root with separately classified public and + protocol-protected child routers。This does not create a core User and does not require a Memos + administration API。Promoted technical documentation must retain a minimal code-shaped example of + this topology instead of reducing it to prose-only “self-auth mode”。 +- Memos credential is deployment-scoped and long-lived by default until explicit replacement/revocation； + it is ordinary raw extension config with validated merge/persist/live-apply ordering；no refresh-token、 + session or Memos-specific secret lifecycle is introduced。 +- Extension enable/disable normally changes route availability in the running single-process deployment； + restart is not the ordinary activation boundary。 + +### Memos extension and backend MVP truth projected to Hub source + +- implementable ownership unit 是 Memos extension；CanonicalMemo、graph mapping、resolver 与 + product/generation adapters 由该 extension 拥有。Memos-compatible backend 只是首个 MVP + delivery scope。 +- 当前 MVP 是 Memos 0.29.1-compatible backend，MoeMemos Android 2.0.4 是 compatibility + acceptance client 而非 API authority。 +- backend 只实现目标 journey 所需的最小 write/read surface，不复刻完整 Memos server。 +- 客户端显示 write 成功代表 primary memo mutation 已持久化；D-041 不保证 graph completeness， + failure 可留下 orphan/stale components。先行成功上传的 unattached attachment 是独立资源。 +- comment 是独立 memo，并以 parent relation 连接；comments 需要独立 fixture，即使当前 APK + core sync 不调用它。 +- flomo backend、collectors、Memos 0.30/older generations、social/share/explore 默认不属于该 + unit。 +- Memos current-user/settings 是 deployment-scoped compatibility projection，不新增 User/tenant + tables，也不把 `ClientModel` 当人类用户。 + +### Product truth still blocked + +- 当前 backend MVP 没有剩余 Product blocker；其余未决项属于 Technical/Acceptance gates。 +- 其他 collection units 的用户可见 partial success、delay、delete 与 compatibility semantics。 +- feature retrieval 与 graph-navigation retrieval 的完整可观察行为与质量门槛。Semantic retrieval MVP 已形成 + 稳定合同、real-provider 实现证据和 Hub source projection，不再与另外两类 retrieval 捆绑。 + +### Semantic retrieval batch projected to Hub source + +- semantic retrieval 返回按相似度排序的现有 Blocks/Relations 与 score metadata，不生成 answer、transient chunk + 或 Chat/RAG product behavior； +- organization 保持为改善 use 的 graph mutation；本 MVP 的 explicit rumination 围绕一个 focal Block 反刍， + 可以 additive no-op/增图，但不替代或删除原 Block； +- embedding records 是 profile-scoped、可重建的 use support，information authority 仍是 graph；candidate maintenance + 与 retrieval 分离，stale records 在显式 maintenance 前不可用； +- exact capability discovery 与 invocation 分离；Peer advertisement 只陈述 capability/inbound/lease，业务 facade + 通过 PeerManager 做 opaque delegation，provider inbound 必须进入 non-delegating local path； +- generic capability invoke endpoint、delegation job、readiness advertisement、persistent Agent Thread/checkpoint、ANN/ + HNSW、pagination 和 transient segment layer 均不属于本 MVP； +- Acceptance authority 是经过真实 Memos/RSS/Atom/HTML/storage/rumination 边界形成的 pinned corpus graph；可读 alias + 只属于测试 harness，不进入 production model/API。Deterministic vectors 证明 control flow，不替代 credentialed + provider semantic-quality authority。 + +## Candidate Hub Product TDD Batch + +### Cross-unit contracts projected to Hub source + +- block 是基本持久信息单元；block hydration 隐藏 inline/pointer 分支，resolver 联合 hydrated content 与 + local relations 得到 solved/use-facing interpretation，storage 只负责按 pointer 取得 actual content。 +- source-specific input 通过 extension mapping 持久化为 block/relation graph；`SubGraphForm` + 是 write form，不是完整信息模型。 +- memo root content 直接保存 memo-family `CanonicalMemo`；attachments、parent、references 只由 + graph components/relations 表达，backend read 只消费 resolver solved result。 +- attachment relation 默认无序；正文内显式 attachment reference 是位置 authority。Memos 0.29.1 + 是已证明的 source-defined order exception，应在现有 relation payload 中保留，不推动通用 + relation schema 变化。 +- comment 复用 memo root mapping，并通过 parent relation 连接。 +- product API version 与 CanonicalMemo resolver contract version 是正交版本轴；CanonicalMemo decoder + 由 versioned resolver identity 选择，不在 payload/BlockModel 重复 schema version。 +- info-base local memo identity 是 `block.id`；不建立 generic `resource`、`source_key` 或 source + binding table。 +- future collector 采用 best-effort exact reconciliation；匹配不足宁可产生可整理 duplicate， + 不得 content/time fuzzy overwrite。 +- 复杂度投入应由边际效益而不是理论完备驱动：先识别仍未解决的问题造成的实际损失，再选择能以最低 + dependency/obscurity 成本消除主要损失的机制，并在新增机制的边际收益不再覆盖其维护与错误成本时停止。 + D-056 是 reference pressure：content fingerprint 试图制造更强 identity，却带来 normalization/stability + 成本；独立 source-time watermark 以更弱、显式的保证取得足够的 duplicate reduction。具体 heuristic + 仍不得被描述为 identity、reconciliation 或 correctness proof。 +- Cache/effect controls use stable，orthogonal vocabulary across peers：`refresh` bypasses and replaces an existing + local snapshot from current authority；`materialize_missing` permits creation only when a required derivation is + absent；`recompute` is an explicit organization command that regenerates an existing derivation；`invalidate` + discards a cache without reading a replacement。`refresh` itself neither grants AI/graph mutation nor requests + recomputation。Python uses snake_case and TypeScript uses camelCase。 +- New InKCre-owned APIs do not use `force` or `reload` as aliases for `refresh`。Protocol-owned names remain exact， + including a third-party `force` query parameter。Legacy source-job `full` is not a stable cross-source contract and + must not be promoted before its mixed scan/reconciliation/order/pagination effects are separated。 +- Direct relation selection uses `include_in` / `include_out`（TypeScript `includeIn` / `includeOut`）relative to the + subject block：incoming has the subject as `to_`，outgoing has it as `from_`；neither option implies recursive graph + traversal。 +- D-039 已关闭 deployment-scoped Memos PAT：ordinary raw extension-config lifecycle、generic validated + update ordering、exact public profile/v0-status-404 与 immediate replace/revoke。 +- D-041/D-042/D-043/D-044 已关闭 partial-graph boundary、CanonicalMemo v1、PostgreSQL binary storage + 与 Memos relation grammar；D-045 要求单独修复 client-web config path；D-046/D-047/D-048 关闭 + owned cleanup、exact fixtures 与 family/product/access-mode extensibility seams。 + +### Implementation evidence used for promotion + +- installed extension decoder retention、unknown resolver explicit failure、core-py route/config runtime + 与 client-web config path 已由 implementation/tests 证明。 +- D-047 exact fixtures、D-046 deletion behavior、PostgreSQL binary storage、partial cleanup residue 与 + official MoeMemos APK journey 均有 executable evidence。 +- existing `/{extension_id}` route 已由 MoeMemos pathful base URL 复用；route-auth composition、hot + lifecycle、caller-session graph commands 与 writable storage 已实现。证据仍不支持 top-level mount、 + generic resource binding 或通用 extension/resolver registry redesign。 + +### Deferred technical scope + +- collector scan/webhook/export、cursor、external identity 与 reconciliation。 +- flomo/其他 memo product adapters 及其 fidelity contract。 +- feature/semantic/graph navigation、index/projection invalidation 与 retrieval UI。 + +## Spoke Unit TDD Promotion Applied + +core-py local Unit TDD promotion 已应用，只记录本仓内部 implementation architecture，例如： + +- memo extension package、route/service/resolver/storage/transaction boundaries； +- graph mutation 与 solved result 的 internal contracts； +- auth/config 的 local wiring； +- Memos extension 的 0.29.1 backend adapter 对 missing `updateMask` 的 raw-JSON key-presence + inference 与 negative + cases； +- tests、migrations 与 failure/residue handling 的实现真相。 + +具体 ownership 已投影到 `docs/30-unit-tdd/memos-extension.md` 与更新后的 +`business-pipeline-and-authority.md`；临时 implementation observation 未被提升为共享合同。 + +## Architecture Understanding Log + +这些发现保留为 provenance；已通过 Promotion Test 的内容现在由对应 durable owner 陈述，不以本 log 作为 +并行 authority。U-009 仍只属于 task/workflow evidence： + +- **U-001 — Joint graph semantics**: block/relation/resolver/storage 共同决定信息如何保存与解释。 +- **U-002 — Attachment position**: association 默认无序；显式 inline reference 才拥有位置。 +- **U-003 — Ordered slots are not linked lists**: `root --slot--> component` 是 slot mapping； + 当前 relation fetchsert identity 也不直接支持只换 `to_` 的 slot semantics。 +- **U-004 — Text storage can carry a grammar**: JSON/text 并非天然无结构，但也不会自动获得 + canonical identity、typed query 或统一解释；resolver 不是 relation content 的唯一消费者。 +- **U-005 — No historical support is not no version architecture**: 当前 target 已固定 0.29.1； + future breaking generation 仍需要显式 adapter/generation boundary。 +- **U-006 — Family canonical is not a god object**: CanonicalMemo 属于 memo extension，且就在 + graph root content 内；它不与 info-base 竞争 authority。 +- **U-007 — Upstream needs evolve core contracts**: 具体 source/use pressure 可以传导到 + info-base、collection、organization、application 与 extension，但必须保留证据链。 +- **U-008 — Persistence time and authored time differ**: block row time、memo-authored/source time、 + collection observation time 不得互相替代。 +- **U-009 — A plan can be a design probe before it is executable**: 代码地址、依赖和可验收纵切可以 + 在 Technical/Acceptance 阶段暴露遗漏合同；只有上游获批且分叉关闭后，才冻结为 execution + baseline。 +- **U-010 — Client base URL participates in route compatibility**: protocol annotations 的 relative + path 必须与 client 对 configurable host path 的保留/拼接一起判断；不能只看到 `api/v1` 就推断 + server-root mount。MoeMemos 可用 `/memos/` base URL 复用当前 extension namespace。 +- **U-011 — Complexity follows marginal utility**: 不以理论上还能更完整为继续设计的充分理由；比较 + unresolved harm、机制覆盖率、dependency/obscurity 与长期维护成本。选择足够有效的低成本机制后停止， + 同时把剩余风险与弱保证写清楚。D-056 的 time watermark 是实例，不是这条方法本身。 +- **U-012 — Storage representation is not information kind**: storage 可以把 audio、video、image 或其他 + information 的 actual content 都保存为 bytes；这不使它们成为 `binary block`。block/resolver 按信息语义 + 命名和解释，storage 只按 pointer 保存/取得 actual content。RSS enclosure 与 Memos attachment 已形成两个 + 当前 reference pressures；PDF/EPUB/ZIP 使用 concrete generations，unknown/unsupported 使用带 MIME 的 + file fallback。 +- **U-013 — Metadata block can describe related content**: 当 protocol/source object 拥有可独立使用的 + identity、metadata、role 或 lifecycle 时，使用 `metadata block → semantic content block → storage-backed content` + 分离 provenance、信息语义与物理保存；resolver 联合 graph 投影 native/use-facing value。这两者都是普通 + block 的职责命名，不新增 wrapper 类型；没有独立意义的 input 不机械增加 metadata block。RSS + enclosure 与 Memos attachment 是当前 reference pressures。 +- **U-014 — One block read contract hides conditional persistence**: `block.content` 在 inline block 上是 + actual content，在 storage-backed block 上是 opaque pointer；通用 consumer 通过 + `get_hydrated_content()` 取得 actual content，不自行解释 storage。hydration 可缓存在 ORM 非映射的 + private state，但绝不能覆盖 mapped pointer。该 read contract 取代含混的 real/raw content 双重命名， + 也不为追求字段纯粹性提前增加第二套 block representation。 +- **U-015 — Storage mechanics do not define content semantics**: storage type 只描述如何按 pointer + 定位、读取或写入 opaque content bytes；stream 是 bytes 的 execution representation。resolver 才根据 + exact resolver ID、graph 与 metadata 把内容解释为 image/video/audio/PDF 等信息。实现 backing table 不是 + storage type 或 semantic block；不要按 media kind 复制 HTTP/S3/PostgreSQL storage families。 +- **U-016 — Metadata follows authority, not a generic container**: protocol/source-declared filename/MIME/length/URL/time + 留在 metadata block canonical content；storage retrieval mechanics 留在 opaque pointer/config； + content kind 由 exact resolver ID 表达，byte-derived facts 由 solved content 拥有；只有确有长期 use value 的 + derived facts 才由 organization 物化为 graph enrichment。不要仅因 storage-backed block 的 `content` + 被 pointer 占用，就增加无边界的通用 block metadata JSON。 +- **U-017 — Effect words name orthogonal controls**: `refresh`、`materialize_missing`、`recompute` 与 `invalidate` + 分别拥有 cache replacement、missing derivation permission、existing derivation regeneration 与 cache eviction + 语义；不能用 `force`/`reload` 把这些 effect 压回一个模糊 boolean。该合同只约束确实提供相应能力的 API， + 不要求所有方法机械增加同一 options bag。 +- **U-018 — Relation direction is subject-relative**: `include_in` / `include_out` 是相对 subject block 的 direct + relation selectors，并在 Python/TypeScript 仅做 casing 投影；它们不是 graph traversal depth/mode。 +- **U-019 — Repeated spelling is not yet a common contract**: 多个 source 的 `full` 共享拼写，却混合扩大扫描、 + 绕过增量 cutoff、改变顺序与延续分页等效果。Promotion 以稳定语义而非出现次数为准；`full` 当前是待拆解 + vocabulary debt，不是应被固化的 common parameter。 +- **U-020 — Conventional version language beats a new synonym**: 当一个轴表达 API、persisted shape 或 resolver + contract 的 breaking evolution 时，优先使用通行的 `version`，并用限定词说明是哪一种 version；不再用 + `generation` 创造项目内同义词。现有 task packet 中把 product/API/canonical `generation` 当作 `version` + 使用的历史段落属于待批量纠正的 terminology debt，不构成新的领域概念。 +- **U-021 — Readiness proves the executable wire contract**: database protocol readiness 不能只检查 schema、 + relation/function names 与 ACL；对 admitted RPC 还要验证 argument names/types、return database type、set/ + volatility shape 和 media-type transport。PostgREST 14 的 raw `bytea` response 需要显式 + `application/octet-stream` domain，而 raw request 只要求 single unnamed `bytea` parameter；这两者不是同一 + capability。内部 trigger/helper function 必须留在 internal schema，不能因为 authenticated peer 需要 + EXECUTE 就进入 public protocol schema。 +- **U-022 — Writable storage owns pointer serialization**: application/extension command 只应提交 actual bytes 并 + 得到可直接持久化到 `block.content` 的 opaque pointer string；storage handler 自己拥有 internal key → pointer + grammar。调用者硬编码 PostgreSQL `blob_id` JSON 会让 future S3/Nextcloud storage 反向泄漏进 source domain。 + Python 的低层 caller-session write 可以保留 storage-native key，但 common create seam 应与 client-web 一样 + 返回 pointer string。 +- **U-023 — Incremental state must name its authority scope**: ETag/Last-Modified 只对产生它们的 configured + request URL 有效；source-time watermark 只对产生它的 exact feed graph root 有效。cursor/timestamp 本身不是 + 可跨 identity 重用的事实，因此 state 同时保存 scope reference（本例为 configured URL 与 feed block ID）； + config 或 native identity 改变时 reset unrelated heuristic，而不是让旧 cursor 误删/误跳新 source facts。 +- **U-024 — Source config change is not proven feed continuity**: RSS feed continuity 依次由 source-scoped native + feed ID、declared self URL、source-scoped configured URL 证明。无法 exact match 时创建新的 feed root，旧 feed/ + items 保留；同一 declared identity 下 configured URL 可以更新。`source_instance_id` 是 scope，不足以单独 + 证明两次外部 information 来自同一 feed。 +- **U-025 — Schedules create commands, not hidden effects**: manual collection 与 scheduler trigger 都先创建普通 + PENDING collect job，再由同一个 atomic-claim runner 执行。schedule 是 command creation policy，不应成为绕过 + job diagnostics、status、retry 和 source-state semantics 的第二条 effect path。 +- **U-026 — Disclose dynamic schemas progressively**: Agent 初始上下文只提供完成语义选择所需的 compact exact + identities 与 descriptions；大型、稀疏使用或 runtime-dependent 的具体 input schemas 通过领域专用查询 Tool + 按需取得。不要把所有可选能力的联合 schema 固定注入每次模型调用，也不要为此建立万能反射服务。 +- **U-027 — Domain owners produce the canonical downstream command**: 领域实现拥有其输入 schema、description 与 + 语义转换，并直接产生下游 authority 接受的 canonical command。Agent/runtime 只提供注册、路由和 typed + validation；不要复制领域 schema，也不要增加只为跨 Tool 转换而存在的中间 DTO。Resolver-owned + StarsGraphForm authoring 通过一个领域拥有的 normalizer 产生 canonical GraphForm，而非交给 LLM 转换，是当前 + reference pressure。 +- **U-028 — Separate discovery、proposal and commit by effect**: schema/capability discovery、non-persisting proposal + construction 与 durable mutation 使用不同的窄 Tool 边界；写入集中到唯一明确 command，但不扩大成通用 + capability invocation、通用事务或 delegation job。 +- **U-029 — Do not persist or transmit derivable authority twice**: 当一个值可以通过稳定、低成本且无歧义的不变量 + 从同一 command/result 推导时，不再添加第二个字段表达它。Resolver draft 的 `id_start` 已固定为 star Block ID， + 因而额外 `entry_id` 只会制造可分歧的重复 authority。 +- **U-030 — Models choose semantics；code enforces mechanics**: LLM/Agent 负责需要语义判断的能力选择、关系表达和 + 是否提交；领域模块负责 exact routing、schema validation、identifier allocation、结构不变量与持久化。不要 + 用模型处理可确定的机械转换，也不要让通用 runtime 接管领域判断。 +- **U-031 — Runtime boundary turns raw input into ordinary typed input**: 接收外部/模型 raw payload 的 + framework/runtime boundary 负责把它反序列化、验证为 typed input；随后被调用的函数接收普通 typed/domain + input，不再用 `validated_input` 命名、`Validated[T]` wrapper 或额外状态重复表达“边界已经验证过”。这不取消 + 各层独有的不变量：Pydantic model 继续拥有自身结构约束，后续模块仍可检查自己拥有的不同约束，数据库继续 + 拥有 referential integrity。`submit_graph` 与 `draft_graph → Resolver.create_graph` 是当前 reference pressure。 +- **U-032 — Batch-local identity solves mutually referencing creation**: 普通 creation Form 不携带数据库生成的 + identity、timestamps 或其他 database-managed state。当一个批量 command 必须同时声明待创建实体并让同批关系 + 引用它们时，command envelope 可以引入仅在该 command 内有效的 identity namespace；对于 bigint row identity， + InKCre 使用非零 signed ID：负数声明待创建实体，正数引用已有实体，零无效。该 exception 属于批量引用机制， + 不把数据库生成字段重新泄漏进所有 base Forms。GraphForm 是当前 reference pressure。 +- **U-033 — Spend common-path complexity only for material marginal return**: 当一个更精确的表示要求每个高频事实 + 永久携带额外 identity、状态或一致性关系，却只保护罕见边缘情况时，先判断边缘情况能否通过明确的 producer + invariant 安全、局部地退化。只有退化不造成错误 mutation、信息丢失或 authority 混淆，并且 common path 的 + 语义、维护与 use 收益显著时，才选择较简单表示；这不是“忽略 edge case”，而是把复杂度放在实际产生回报的 + 分支。Mail 的 plain `tags` + same-Mailbox duplicate non-reconciliation 是当前 reference pressure：罕见情况增加 + best-effort canonical Email Block，而不是让每条 flag Relation 重复 locator 或产生可变状态歧义。 +- **U-034 — Identifier usability requires exact-one resolution in an explicit scope**: identifier 不是脱离 namespace、 + comparison scope 与 operation 的绝对标签。一个 reference/reconciliation/mutation boundary 只有在候选经过 scope + 与 eligibility 过滤后恰好解析为一个既有实体时，才能据此复用或修改该实体。零候选表示当前本地无 referent， + 多候选表示当前 scope 内不唯一；两者都不能被 arbitrary first/min-ID selection 隐藏，但 token 仍可作为 + source-native evidence。具体 shallow outcome 由领域 command 决定：继续较弱 ladder、创建、保留 unresolved 或 + skip；common contract 只禁止在非 exact-one resolution 上作用于某个既有实体。Mail D-264 是当前 reference + pressure。 +- **U-035 — Reconciliation completes absence but preserves contradiction；shared ladders own mechanics, not evidence**: + 一个 exact-one candidate 的 identity fact 为空时，可以用后续同 scope 的 non-null evidence 补全；双方同一 identity + fact 均为 non-null 且冲突时，不得靠覆盖值、继续较弱 rung 或内容猜测掩盖矛盾。若多个 Source 已经重复出现 + strongest-to-weakest ladder，通用 Source-domain mechanism 可以内聚 ordered async execution、candidate cardinality、 + short-circuit 与 rung-labelled typed outcome；comparison scope、evidence meaning、eligibility、compatibility 和最终 + command effect 仍由各领域 owner 决定。Mail D-265 是当前 reference pressure；exact utility API/name 等待 + implementation preflight。 +- **U-036 — Locate controls reuse；input kind controls the newly created representation**: identifier/reference ingestion + 先在明确 scope 内 locate existing domain entity；只有 exact-one 才授权复用，zero/many 则不把 ambiguity 隐藏为 + arbitrary choice。若 command 仍必须表达输入事实，它可以创建一个新的领域实体；reference-only observation 创建 + sparse/incomplete entity，full collection 创建完整或可继续物化的 entity，但二者不需要不同的 placeholder type 或 + reconciliation lifecycle。Mail D-266 是当前 reference pressure。 +- **U-037 — Judge heuristics by expected harm and recovery topology，not error probability alone**: “猜错概率低”不足以 + 证明 heuristic 值得采用；同时评估错误后果、系统能否检测、是否能自动/自然恢复，以及是否把内部歧义转嫁为用户 + 纠错操作。不可检测地把错误外部 bytes 持久化为 semantic content，且只能让用户尝试另一个 locator 才可能纠正， + 即使低概率也不应采用。优先选择可见、局部、不会伪造 authority 且可由 Organization 修复的退化，例如产生 + best-effort duplicate。Mail D-271 是当前 reference pressure。 +- **U-038 — Keep protocol identity separate from protocol parameters**: protocol 回答“使用哪套通信合同”， + parameters 回答“如何构造/进入该合同的一个具体 endpoint”；两者 authority、schema 与 consumer + 不同，不应展平或混合在同一 object namespace。Protocol 字段应当判别 parameters schema，但 protocol + identity 必须忠于其真实 authority：可以是 InKCre-owned exact/versioned Peer wire contract，也可以是公开 + IMAP/POP3 standard，不因结构复用而强行内部化。Peer D-123/D-124 与 Mail D-275/D-276 是当前 + reference pressure。该分离不要求 typed protocol vocabulary 预先列举已知但未支持的标准；Mail D-277 + 将当前有效值收窄为 `Literal["imap"]`。 +- **U-039 — Bind external-resource lifetime to a domain command with native language scope**: factory 只构造对象、 + 不产生 I/O；领域 command 通过语言原生 resource scope 取得、使用并释放外部连接。这使 exception、 + cancellation 和 normal completion 共享一个清理 authority，调用者无需理解 partial connection state；也不应 + 在没有实测回报前扩张成 cross-command cache/pool/lifecycle manager。Mail D-279 是当前 reference + pressure，Python async context manager 是其当前 concrete mechanism。 +- **U-040 — Resolve common Source materialization policy through explicit → deployment default → built-in fallback**: + 外部 bytes 的目标 Storage 是 Source-domain local policy，不是具体协议参数。显式 Source 选择优先，其次是 + deployment-scoped 默认 writable Storage；二者均缺失时使用一定可用的内置 PostgreSQL binary Storage。只有 + “未配置”才能进入下一层；已配置但不存在或不可写必须暴露配置/能力错误，不能被 fallback 静默掩盖。Mail + D-282 是第一个 reference pressure；D-283 将 per-Source explicit reference 提升为 nullable + `sources.storage`。D-284 进一步确认 code/catalog 能力一致性属于 Storage registry/bootstrap 系统边界，而不是 + 通过每次使用时的 defensive getter 重复发现。D-285 将 derived capability projection 固定为 + `storage_types.writable`；Source reference 只能选择其 type 可写的 Storage instance。 +- **U-041 — Resolver solving exposes semantic completion，not internal command status**（candidate Product TDD / Unit TDD）： + `get_solved_content` 返回调用者需要的 use-facing solved content；内部 lazy materialization 是 create、reuse、race + 还是 fetch，不应自动扩大成 public outcome。只有该事实本身属于领域 solved semantics 时才暴露。这个浅完成合同 + 应由 Resolver base docstring 与 peer-equivalent contract 拥有，让调用者无需理解深模块内部状态代数。Mail D-288 + 是当前 reference pressure。 +- **U-042 — Do not let tolerated residue shape the common API**（candidate Product TDD / Unit TDD）：低概率、低损害、 + best-effort 容忍的冗余或竞态残留不是受鼓励的领域行为；不要为了让它“更可预测”而在高频路径散布稳定选择、 + duplicate-aware 分支或专用 utility，否则会把妥协提升成事实上的公共合同。公共深接口应表达正常 use operation： + 例如 InfoBaseManager 的 singular related-Block read 只返回任一满足关系谓词的 Block，不承诺 uniqueness、order 或 + repeat-read stability；use-facing output 的 cardinality 继续服从领域语义，而不是被持久化冗余改写；需要观察全部 + graph facts 的调用者仍使用普通多值查询。该原则不适用于 identity reconciliation/mutation：后者继续要求 + U-034 的 exact-one resolution。Mail D-289/D-291 是当前 reference pressure。 +- **U-044 — Match concurrency machinery to expected harm，while keeping the normal result valid**（candidate Product TDD / + Unit TDD）：即使 deployment 是 single-user，自动化和异步入口仍会产生并发，因此不能假设 race 不存在；但也不因 + 理论 race 自动引入 exactly-once、专用唯一约束、回滚协议或复杂 duplicate lifecycle。先保证竞态残留不使正常 + command 失败或改变其领域结果，再用 lock/recheck 等局部机制减少残留；只有损害足以支撑成本时才升级更强保证。 + Organization 可修复低损害冗余，但不能成为 producer 放弃低成本预防的理由。Mail D-287 是当前 reference pressure。 +- **U-047 — Derive implementation-owned capabilities once，then enforce durable references at the data boundary** + （candidate Product TDD / Unit TDD）：当能力由注册的实现类拥有、而持久 reference 的合法性依赖该能力时， + registry/bootstrap 将实现 contract 投影到 catalog，数据库约束保证引用不会进入不可能状态，普通 use path + 直接依赖这个已建立的不变量。不要让每个 caller 反复 rediscover `isinstance`，也不要让可编辑 catalog 反过来成为 + 代码能力的 authority。`WritableStorage → storage_types.writable → sources.storage` constraint 是当前 reference + pressure（D-284/D-285/D-290）。 +- **U-048 — Do not gate primary progress on an orthogonal best-effort effect**（candidate Product TDD / Unit TDD）：先明确 + command 的 primary accepted effect；若另一项配置行为失败既不否定已接受事实、也不妨碍安全推进，而且为了重试它 + 会阻塞高价值进度、重复大量工作或引入新的 ledger/retry lifecycle，则该行为应在 primary commit 后 best-effort + 执行并留下有界 diagnostics，而不劫持 progress cursor。只有 side effect 本身属于 correctness boundary 时才允许 + gate。Mail D-310 的 graph/checkpoint 与 `mark_as_seen` 是当前 reference pressure；Memos primary delete + best-effort + cleanup 提供了较早的同类 evidence。 +- **U-049 — Visual emphasis should spend the fewest sufficient signals**: 当 focus 可以通过 context contrast、清晰 + hairline、关系线权重与 camera framing 建立时，不再机械叠加尺寸、阴影、halo、z-index 和高饱和色。内容的 + intrinsic presentation 不应因 selection/focus 被误写成 importance。Graph focal scene 是当前 evidence；是否 + 提升为跨 UI design guidance，等待实际 shell 的实现验收证明，具体 token 数值不在 task packet 中冻结。 +- **U-050 — Deep interfaces optimize caller understanding，not method-count minimalism**: 把不同语义、参数集合和 + outcome 的操作塞进一个 generic method，只会把 discriminator、合法组合与分支知识推给每个 caller。优先暴露 + 少量但角色清晰、语义内聚的入口，由深模块隐藏查询、路由、分页和结构不变量。Graph 的 Block/Relation + neighborhood methods 是当前 evidence；这不是鼓励为每个细节拆方法，而是以 caller 需要理解多少为判断标准。 +- **U-051 — Acceptance must not promote incidental implementation choices**: 当多个输出都满足同一有价值的领域 + invariant 时，Acceptance 应验证合法结果集合与 observable properties，而不是为了 exact fixture、跨实现逐字 + 相同或某个 consumer 的便利，反向固定数据库 identity、偶然顺序或当前算法。只有唯一选择本身具有产品价值 + 时才进入公共合同；equal-shortest path 是当前 reference pressure。 + +## Apply Checklist + +1. **Memos/RSS implementation done** — confirmed decisions、exclusions 与 acceptance evidence 已冻结。 +2. **Hub source projected and published** — PRD claims/workflows、knowledge capability contract、authority/topology + 与 claim matrix 已吸收 Memos、RSS 及 common patterns；`48b069f` 已作为 published `95c4023` 的 ancestor 到达 + Hub main。 +3. **Core-py local projected and committed** — Memos/RSS Unit TDD、business pipeline、database runtime v2 与最近 + local guides 已和 implementation reconcile；commit `835f89a` 未编辑 `docs/_shared`。 +4. **Client-web local projected and committed** — peer hydration、exact semantic resolvers、PostgreSQL CRUD 与 + safe browser handles 已进入 local architecture；commit `765b22f` 未编辑其 `docs/_shared`。 +5. **Verification complete** — Hub `git diff --check` + SVC noop；45 relative links resolved；core-py owner docs + Ruff-format/repository-lint green；client-web complete `pnpm check` green。Core-py full formatter only retains four + unrelated pre-existing guide drifts。 +6. **Owner-separated publication complete** — Hub 先发布 `95c4023`；core-py `cc8f90a` 与 client-web `8324293` + 随后各自只提交 `docs/_shared` gitlink。client-web remote 后续被观察为已同步；core-py push 与 production + migration 仍是独立 operation。 +7. **Tactical guides repaired** — retired semantic HTTP IDs、raw-content domain terminology、scheduler dual-path、 + Memos attachment v1 与 client-web pointer-rendering docs 已修正。 +8. **Semantic retrieval projected and verified** — core-py `semantic-retrieval.md`、business-pipeline and runtime/ + development docs，client-web Peer/runtime architecture，and the Hub source PRD/Product TDD match I0–I8 implementation。 + Credentialed DashScope Acceptance is 6/6；Hub links/diff/SVC noop are green。Hub publication and both exact Spoke + shared-ref commits are complete without mixing owners。 diff --git a/tasks/knowledge-lifecycle-capabilities/packet.md b/tasks/knowledge-lifecycle-capabilities/packet.md index a9425d0..5452d60 100644 --- a/tasks/knowledge-lifecycle-capabilities/packet.md +++ b/tasks/knowledge-lifecycle-capabilities/packet.md @@ -1,41 +1,170 @@ -# Knowledge lifecycle capabilities - -- **Objective**: improve InKCre's collection、organization and application capabilities through independently valuable、 - vertically acceptable units。 -- **Guardrails**: - - collection、organization and application are actions，not information states；Blocks/Relations remain info-base - authority； - - source、resolver、storage、extension and Peer mechanisms evolve only under concrete vertical pressure； - - one implementable unit is active at a time；Product、Technical、Acceptance and preflight precede an Impact Handshake， - then source implementation waits for Sir's explicit start； - - use current Hub/local durable docs as truth；completed task history remains in Git and is not a second authority； - - ask Sir only about credible non-dominated forks that materially change product behavior、authority、public contract or - expensive recovery。Derive ordinary names、mechanical consequences and low-risk implementation choices locally。 -- **Verification**: each unit closes through its real public boundary and projects stable truth to exactly one durable owner； - use the least complex evidence that adequately proves the contract，without turning manual、scripted and automated evidence - into a mandatory maturation sequence。 -- **Current Truth**: - - Memos backend、RSS hardening、Mail、semantic retrieval、lexical feature retrieval and graph-navigation retrieval are - implemented、accepted and reflected in current Hub/Spoke durable docs； - - graph-navigation retrieval closed through core-py PR #78、client-web PR #85 and ui `@inkcre/ui-web@1.4.0`； - - no capability unit is currently active；baseline cleanup has established the next-unit repository baseline； - - historical decisions、plans and acceptance evidence were removed from the checkout after promotion review；Git history - remains the recovery path。 -- **Next Step**: after baseline cleanup closes，select one unit from the remaining queue in - [capability-map.md](capability-map.md) using current user value、dependency pressure and uncertainty—not table order。 - -## Discussion and delivery loop +# Knowledge Lifecycle Capabilities + +- **Objective**: 增强 InKCre 的收集、整理与应用能力，并让每个可实现单元从产品设计、 + 技术设计、验收、实现计划与 preflight 可审计地进入实现。 +- **Guardrails**: 收集、整理、应用是能力动作而非信息状态；block / relation graph 是 + info-base 的持久 authority；横切机制只由具体单元的真实压力推动；durable docs 与业务代码 + 各自只有在完成对应 Impact Handshake 且 Sir 明确“开始”后才修改，并按 owner 分离操作。 +- **Verification**: 每个 active unit 必须拥有自己的可执行验收合同、阶段 gate、Impact + Handshake 与验证结果；D-049 要求结构性验证优先交给 static mechanisms，runtime acceptance + black-box-first。Program 完成还要求所有获批 durable truth 回到唯一 owner。 +- **Current Truth**: program 拆分和术语基线已经形成；当前产品不建立 terminal-user、tenant + 或 per-user ownership/ACL，deployment 是单一 owner context，其中的 InKCre runtime nodes 统一称为 + peers（D-033/D-109）。[Memos extension](units/memos-extension/packet.md) backend MVP 的 Product、Technical、 + Acceptance、Implementation、official APK E2E 均已完成，core-py commit 是 `304a5c8`，独立 + client-web config-path fix 是 `f2ab107`；该 implementation unit 已完成。 + [RSS extension hardening](units/rss-extension-hardening/packet.md) 的 B0–B8 implementation/verification 与 durable + reconciliation 也已完成，并于 2026-08-03 通过 Sir 的最终验收复审； + Hub PRD/Product TDD、core-py Unit/deployment docs 与 client-web info-base architecture 已跟随已验证实现投影并 + 分别提交为 Hub `48b069f`、core-py `835f89a`、client-web `765b22f`。Hub commit 已随 + semantic-retrieval shared batch 发布，两个 Spoke 也已消费该 published Hub head；final verification 显示 client-web + remote main 已由外部/自动 push 到 `8324293`，而 core-py main 仍仅本地。core-py push 与 production migration + 仍是独立授权操作。 + RSS hardening 方向与 + black-box-first acceptance strategy 已获 Sir 接受；D-050 固定 feed-authored content / independent + full-text enrichment authority，D-051 固定保留 extension identity 的 behavior rewrite 与成熟第三方库 + boundary，D-052 固定 full-text enrichment 进入 MVP、默认开启且可关闭，D-053 固定 best-effort exact + native identity/reconciliation ladder、排除 payload fingerprint，并允许 source config 在罕见的 + unidentifiable item 上选择 create（默认）或 discard，D-054 固定 feed/channel 是独立 information block。 + D-055 固定 feed/channel exact identity ladder，D-056 固定 unidentified-item source-time admission + watermark 及其非 identity 边界，D-057 固定 enclosure graph、manual extension command 与 + source-configured automatic materialization，并明确下载结果应按语义成为 audio/video/image 等 media + block，PDF/EPUB/ZIP 也进入 scope，unknown/unsupported fallback 为带 MIME 的 file block。D-058 固定这些 + 横向 media/storage/Memos 修正留在同一 RSS unit，D-059 固定 MemosAttachment metadata block → semantic + content block，并抽象出有适用条件的 common pattern。D-060 固定保留 `content` 的 inline-value/storage-pointer + 条件语义，由 `BlockModel.get_hydrated_content()` 统一延迟读取实际内容并缓存到非映射 private attribute， + 不引入 `storage_pointer` 或 `BlockRecord`。client-web `packages/core` 已证实拥有平行的 Block/resolver/ + storage hydration 实现，因此属于同一横向 contract 的 downstream implementation surface；hydration + 由各 peer 的本地 storage handler 承担，缺失 handler 明确失败，不默认委托 core-py。D-061 同时要求 + client-web 在本轮支持 PostgreSQL binary。D-062 进一步固定 storage 只运输/保存 opaque content bytes， + resolver 才拥有 image/video/PDF 等信息解释；现有按 content kind 拆分的 HTTP storage types 因而进入重构 + 压力面。D-063 将 client-web 范围扩为 create/read/update/delete 的完整 CRUD；D-064 澄清 + `block.updated_at` 只表示 block record 时间，storage 不反向依赖 block，hydrated cache 也不承诺跨实例或 + 跨 peer freshness。D-065 固定 instance-local snapshot + explicit refresh 合同；D-066 固定 raw + Create/Read + relation Update/Delete 的 PostgREST wire shape。D-067 固定 media metadata 按 source、 + storage、resolver、organization authority 分层，不新增 `blocks.metadata`，并将其提升为 common pattern + 候选。D-068 将 S3-compatible storage 排入 future Nextcloud Files unit；RSS 不以超大 enclosure/streaming + 作为验收条件，也不提前增加 streaming abstraction。D-069 撤回跨 extension 的 global classification + ladder：extension adapter 拥有 evidence policy，`ResolverManager` 只提供 opt-in common mechanisms，不新增 + media module。D-070 固定 Memos `Attachment.type` → normalized MIME → exact resolver ID，unknown → file； + 不做 mandatory byte sniff/mismatch rejection。D-071 固定合法、具体的 RSS `enclosure.type` 为 primary + resolver-selection evidence，fallback evidence 不覆盖 metadata-block declaration。D-072 固定 Atom materialization 以 + specific HTTP Content-Type 优先于 advisory `link.type`，再调用 adapter-owned fallback。D-073 固定 resolver + 使 graph 可被 application 使用、text/embedding projection optional，并允许受控 lazy graph + materialization。D-074 进一步固定 ordinary resolution 默认允许 materialize missing graph，显式 + `materialize_missing=False` 得到 read-only attempt；`refresh` 只拥有 cache bypass/replacement 语义，并与 + `recompute`、`invalidate`、relation `include_in/include_out` 形成跨 peer 稳定 vocabulary。Legacy source-job + `full` 因混合多种 effect 不被提升为通用合同。D-075 固定九个 + `core.<kind>.v1` exact semantic content resolver IDs，abstract text/embedding capability methods，resolver- + instance invocation，metadata block → semantic content block 命名，以及裸 `text/html/image/video` hard cut-off。 + [Semantic retrieval](units/semantic-retrieval/packet.md) 的 I0–I8 均已实现，core-py closure commit 为 `b80e5fd`， + client-web closure commit 为 `ca4899c`；pinned corpus、真实 producer/storage/runtime vertical、deterministic + rumination/ranking、local/delegated Peer journeys、本地 durable projection 以及 DashScope real-provider 6/6 + Acceptance 均已通过。Hub `95c4023` 已发布；core-py `cc8f90a` 与 client-web `8324293` 分别以纯 ref commit + 消费该 exact shared truth。该 implementable unit 已关闭。 +- **Next Step**: 暂停 [GitHub extension](units/github-extension/packet.md) 的 review correction；先由独立的 + [`extension-ownership-correction`](../extension-ownership-correction/packet.md) task 修正 Hub/Spoke durable owner、 + core built-in/Extension catalog ownership 与相应 promotion guideline。该 correction 关闭后，再回到 GitHub + extension 修正其 batch graph persistence 与 PyGithub integration。 + +## Program Boundary + +- **Collection**: 现有 sources、memo-like、CalDAV、Nextcloud Files、Apple Notes。 +- **Organization**: 以改善 use 为目标；breakdown、merge、linking 是已知能力，不是完备枚举。 +- **Application**: 特征检索、语义检索、图导航检索；indexing 是应用支撑，不属于 + organization。 +- `block.get_hydrated_content()` 统一提供 actual content；resolver 联合 hydrated content 与 local relations + 得到 use-facing interpretation。这是联合信息语义，不是第四条能力主线。 +- Hub 现有内容和 Sir 的判断都是需要核验的证据；二者都不是自证前提。 +- deployment-scoped single-user 是当前产品边界；外部 source account 或协议中的 `user` 不 + 自动成为 InKCre core domain user。 + +## Active Implementable Unit + +当前 program 内没有 active implementable Unit；跨 unit 的 +[`extension-ownership-correction`](../extension-ownership-correction/packet.md) 是当前唯一 active task。它不是新的 +knowledge-lifecycle capability unit，而是修复 PR #79 暴露的 task-control 损坏和 GitHub review 暴露的 owner +错误。修正关闭后恢复 GitHub extension unit。 + +[GitHub extension](units/github-extension/packet.md) 已完成首轮实现和真实账号 acceptance，但 review 发现 durable +owner、core/Extension catalog、batch graph interface 与 external protocol client ownership 错误；当前 **Paused**， +不得合并 PR #18/#80 的现状。 + +[Graph navigation retrieval](units/graph-navigation-retrieval/packet.md) 已完成 core-py PR #78、client-web PR #85、 +`@inkcre/ui-web@1.4.0`、preview/production acceptance 与 durable closure。 + +[Feature retrieval](units/feature-retrieval/packet.md) 已完成实现、J1–J7、真实 NASA/DashScope、core/client promotion、 +独立 Render + Neon fork/cold-start 与 exact-main Pages delivery 验收；perceptual/hybrid future pressure 不重新打开其 +已关闭 lexical increment。 + +[Mail extension](units/mail-extension/packet.md) 的 Product、Technical、Acceptance、Implementation、Verify、Promote +与 owner-separated delivery 均已关闭。 +本轮保留 extension identity，把旧实现当作需求与失败证据；先建立可信 collection baseline，再由真实邮件 +场景推动 organization、info-base basic use/query 与 client-web 的必要演进。MVP / MLP 由用户 job、价值与 +可接受代价决定，不以协议完整性或 feature checklist 代替产品判断（D-198/D-199）。 + +[Semantic retrieval](units/semantic-retrieval/packet.md) 的 Product、Technical、Acceptance、Implementation Plan、 +Preflight、Impact Handshake、Execute、Verify 与 shared-truth promotion 均已关闭。 + +[RSS extension hardening](units/rss-extension-hardening/packet.md) 已完成，不因 semantic retrieval 消费其 +resolver/hydration contract 而重新打开。 + +[Memos extension](units/memos-extension/packet.md) 已关闭；future collector/product generations 不继承其 +backend MVP approval。 + +同一时刻最多只有一个 unit 标记 Active。supporting documents 不维护独立 phase 或 `Current question`； +它们由 unit packet 路由。 + +## Delivery Loop ```text -current model + evidence - -> Product contract - -> Technical contract <-> Acceptance <-> implementation-plan probe - -> preflight / failure-branch simulation - -> frozen execution baseline + Impact Handshake - -> explicit start - -> implementation -> verification -> durable projection -> closure +Product contract + → Technical contract ↔ Acceptance draft ↔ Implementation-plan probe + → evidence preflight / branch simulation + → approved Acceptance contract + frozen Execution baseline + → Impact Handshake + → explicit “开始” + → Execute + → Verify / Promote ``` -Topology is used when ownership crosses modules；sequence/state models are used when behavior repeats、waits or partially -persists。They are reasoning tools，not mandatory artifacts。When one coherent solution follows from established constraints， -present the result for review instead of manufacturing another decision。 +- **Experimental task-wide discussion loop**：current model reconciliation → authority/scope/lifecycle classification → + topology and/or multi-execution sequence when behavior crosses owners or time → dominated-option removal → at most one + credible human fork。A question is an output of unresolved model pressure，not the unit of discussion progress。This protocol + is under Sir's experiential review and is not a law；see [design taste](design-taste.md)。 +- Product 明确用户旅程、范围、非目标、成功和可观察失败。 +- Technical 明确 owner、topology、data/API contract、compatibility 与 failure/partial-effect semantics。 +- Acceptance 在实现前固定 public/runtime input、持久 graph、resolver/native output、错误与重复执行 + behavior。优先由静态机制证明可机械检查的事实；需要动态证据时，先以真实 transport + persistence 的手工或 + 脚本化 black-box journey 验证，反复成熟且证明回归收益后才考虑提升为自动化测试。新增自动化测试需要 Sir + 显式批准；white-box fixture 只有在 D-049 exception 成立时保留。 +- Implementation-plan probe 可以在 Technical/Acceptance 审查中提前展开增量、代码地址、依赖与 + 验证顺序，用它暴露遗漏的设计；此时它不授权实现。 +- Preflight 可以在 design probe 后执行，核实版本、地址、运行环境并遍历实现分支；它发现新的 + owner/behavior 时必须退回相应 Technical/Acceptance gate，而不是把问题留到 Execute。 +- 只有 Technical/Acceptance 获批、preflight 暴露的 questions 关闭后，plan 才冻结为 Execution + baseline。若计划后来又暴露新的 owner/behavior 分叉，继续退回对应 gate。 +- Execute 必须同时具备完成的 Impact Handshake 和 Sir 对该 state diff 的明确“开始”。 + +## Program Navigation + +- Active design/discussion filter: [design taste](design-taste.md) +- Capability topology and queued work: [capability-map.md](capability-map.md) +- Single decision authority: [decision register](decisions/index.md) +- Cross-cutting pressures: [pressure-ledger.md](pressure-ledger.md) +- Terminology and repository evidence: [terminology-audit.md](terminology-audit.md) +- Peer terminology migration evidence: [peer-terminology-migration.md](peer-terminology-migration.md) +- Durable-doc promotion queue: [documentation-promotion.md](documentation-promotion.md) +- Track maps: [collection](tracks/collection.md), [organization](tracks/organization.md), + [application](tracks/application.md) + +## Retention and Promotion + +- Task files are working memory, not durable truth owners。 +- An active task packet is nevertheless the current collaboration authority。Cleanup follows the parent task lifecycle；a + completed child unit、large file count、age or the volatility of `tasks/` does not authorize deleting an active packet。 + Split content when needed，but retain one program control authority。 +- 获批决定只在 `decisions/` register 陈述一次；unit/design/evidence 通过 decision ID 或链接引用。 +- 讨论中尚未稳定的 durable-doc pressure 只进入 `documentation-promotion.md`；design 冻结且 implementation + 提供证据后，按 PRD、Product TDD、Unit TDD 等 owner 形成内聚批次并随 unit closure 应用。Commit/push、 + Hub publication 与 shared-ref bump 仍按 owner 独立授权。 diff --git a/tasks/knowledge-lifecycle-capabilities/peer-terminology-migration.md b/tasks/knowledge-lifecycle-capabilities/peer-terminology-migration.md new file mode 100644 index 0000000..ae54d70 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/peer-terminology-migration.md @@ -0,0 +1,68 @@ +# Peer Terminology Migration Evidence + +> D-109 的 cross-unit working ledger。它记录迁移地址与边界，不是 durable protocol owner，也不授权实现。 + +## Selected Semantic Diff + +```text +technical runtime-node domain: Client -> Peer +user-facing product/app language: Client remains Client +``` + +这是 technical domain hard rename，不是给旧词增加“其实是 peer”的注释，也不是全仓字符串替换。 + +## Proven Migration Surfaces + +### Shared database/runtime protocol + +- `clients` relation and its row/schema names；the structural migration renames it to `peers`，but D-195's clean shared- + database rebuild removes any retained-row compatibility requirement。 +- development baseline client constants and catalog/readiness SQL。 +- deployment profile `client` section and `core.client_id`。 +- peer JWT's current exact issuer value `inkcre-client`。 +- extension enablement references whose values are current client IDs。 +- generated database/runtime contracts and migrations consumed by equal peers。 + +### core-py + +- `ClientID`、`ClientModel`、`ClientManager` and `app.schemas.client` / `app.business.client`。 +- `client_id`、`client_name`、`client_base_url` settings and corresponding `CLIENT_*` environment names when they denote + the local InKCre runtime node。 +- bootstrap registration、tests and local durable docs that use the domain concept。 + +### client-web and other peer implementations + +- `packages/core/src/client/client.ts` currently mixes a database Client record with peer-to-peer behavior；the domain + record/API should become Peer/PeerRef and a peer module。 +- active-record mappings、extension enablement APIs、generated DB types、runtime profile fields、JWT contract、tests and + developer/admin UI labels that explicitly expose technical runtime topology。Ordinary user-facing UI remains Client。 +- Hub Product TDD and technical glossary currently preserve the Client domain and must be corrected through the Hub + workflow after implementation evidence。PRD/user-facing product language can retain Client；only claims that explicitly + explain peer topology need the technical term。 + `docs/_shared/**` is not edited from the Spoke working context。 + +## Excluded from Mechanical Rename + +- FastAPI/HTTP/TestClient variables and third-party HTTP library client classes。 +- OAuth/Twitter/GitHub native `client_id`、`client_secret` or SDK client vocabulary。 +- Memos/MoeMemos and other external apps when they genuinely act as clients of an extension backend。 +- marketing、landing、non-technical/user-facing documents and first-party product/repository identities such as + `client-web`、`client-ios` and `client-webext`。 +- historical migration filenames/revision history；a new migration expresses the protocol rename。 + +## Closed Product/Repository Boundary + +First-party repository/product slugs remain user-facing Client identities and are not renamed。Within those repositories， +technical domain symbols、database contracts and runtime topology still use Peer。`rokid-studio-client` likewise receives +no rename merely from this decision；if its internal architecture participates as an InKCre Peer，only that technical +surface enters the semantic migration。 + +## Approved Implementation Shape（semantic-retrieval unit） + +1. inventory technical Peer surfaces across participating repositories/deployments without renaming product/repository + identities； +2. update Hub technical contracts and each peer implementation as coordinated owner-specific changes； +3. rename persisted protocol state in place and hard-cut technical symbols/env/wire names without indefinite + compatibility aliases； +4. regenerate projections and verify cross-peer bootstrap、authentication、extension enablement and shared DB access； +5. keep user-facing and genuine external-client vocabulary unchanged and prove this with targeted search/static checks。 diff --git a/tasks/knowledge-lifecycle-capabilities/pressure-ledger.md b/tasks/knowledge-lifecycle-capabilities/pressure-ledger.md new file mode 100644 index 0000000..2abc173 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/pressure-ledger.md @@ -0,0 +1,437 @@ +# Capability Pressure Ledger + +这里追踪具体 unit 如何传导出横切架构压力，不把候选 core change 自动升级为决定。每项遵循： + +`上游需求 → 被打破的假设 → 候选 owner → 影响 → evidence → status` + +active unit 是 [Graph navigation retrieval](units/graph-navigation-retrieval/packet.md)，当前 Execution baseline 已冻结并 +等待新的明确实施授权。下列 Mail/Feature pressures 保留为 completed-unit provenance，不自动成为 graph navigation +retrieval 的设计前提；新的横切压力必须来自本 unit 的 user journey、Acceptance 或 implementation evidence。 + +## Completed Mail-unit pressures + +### P-018 — Mail cannot be permanently modeled as a read-only collection adapter + +- **Upstream**: Mail should retain a practically complete communication record and ultimately make InKCre a complete email + client/agent。 +- **Broken assumption**: a Mail extension can finish at IMAP ingestion，or any write to the remote mailbox is an accidental + source-side effect。 +- **Candidate owner**: Mail extension owns the product vertical；exact collection、remote-action、Agent、Peer and client-web + boundaries must be earned by successive delivery scopes rather than forced into SourceBase。 +- **Impact**: mailbox coverage、state synchronization、outbound commands、authorization/approval、graph/query semantics and + client-web interaction can all become relevant。The terminal direction does not approve them all for this iteration。 +- **Evidence**: current `mark_as_seen` is already an intentional configurable write-back；Sir selected complete + communication records and a complete email client/agent as the terminal product direction。 +- **Status**: D-200 confirms the terminal pressure；D-201 fixes the current communication-record foundation、configurable + `mark_as_seen` and compose/reply/send deferral；D-202 fixes one-account-per-Source coverage and extension-owned default + exclusions；D-203 adds Source override；D-204 fixes ordinary post-setup collection and explicit bounded history。 + Continuous-update semantics remain an open Product question。 + +### P-019 — Source history needs specialized `backfill` collect semantics rather than overloaded `full` + +- **Upstream**: a Mail account may contain years of history，but Source creation should begin cheap forward collection and + must not silently trigger an unbounded import。 +- **Broken assumption**: one boolean `full` can express history range、reconciliation、replacement and source-specific + traversal effects。 +- **Candidate owner**: the Source domain may own collect as the umbrella ingress action and backfill as a specialized collect + intent，while each Source/extension owns whether history exists and the shape of its ordinary/history boundaries。 +- **Impact**: legacy Mail、RSS、GitHub and Twitter job configs use or expose `full` with non-identical behavior。A hard cut + requires per-source audit and cannot be inferred from Mail alone。 +- **Evidence**: D-074 already refused to promote `full`；D-204 establishes exact Mail product semantics。Files/calendar-like + current-state Sources demonstrate why “created before setup” is not universally historical data。 +- **Status**: D-205 confirms the common mental model；cross-source interface remains a pressure for Technical audit，not a + program-wide code change authorized by this Product decision。 + +### P-020 — Reserve `enrichment` for Organization rather than source acquisition + +- **Upstream**: Mail can expand a URL from an already collected email into a useful connected graph；RSS currently obtains + linked full text while collecting feed information。 +- **Broken assumption**: every additive fetch is “enrichment”，or the mechanical act of network fetching determines the + domain owner。 +- **Candidate owner**: enrichment classifies additive Organization behavior but does not own its implementation。The domain + Resolver may own a deep materialization capability；Source/extension owns collection-time or delegated remote acquisition， + while Storage/InfoBase retain bytes/graph mechanisms。 +- **Impact**: RSS task/durable docs and code vocabulary currently use `full-text enrichment` for a source-owned behavior。 + Rename pressure must preserve behavior and authority rather than silently redesigning RSS。 +- **Evidence**: D-206 fixes post-collection Mail URL expansion as the canonical enrichment example and retains RSS linked + full-text work as source-owned acquisition。 +- **Status**: D-206 reserves additive Organization meaning；D-207 removes redundant `source-owned collection` wording；D-215 + corrects classification versus implementation ownership。Apply owner-specific durable/code naming corrections with an + implementation unit after blast-radius audit，not by rewriting completed-unit history during discussion。 + +### P-021 — Mailbox deletion is a scoped external fact，not default info-base deletion + +- **Upstream**: a collected email can later disappear from one remote mailbox while remaining valuable in the info-base or + present through another folder/label。 +- **Broken assumption**: Mail must mirror remote deletion，or one missing listing proves global message deletion。 +- **Candidate owner**: Mail Source collection records mailbox-scoped presence/deletion facts and decides whether to issue + graph deletion；InfoBase persists the submitted graph commands without understanding Mail policy。 +- **Impact**: incremental sync capabilities、folder/label identity、move detection、optional graph deletion and Source config。 +- **Evidence**: D-208 fixes default retention and permits opt-in synchronized deletion only with trustworthy protocol-native + incremental evidence，not full traversal/diff emulation。 +- **Status**: D-209 fixes relation removal/no tombstone and corrects the owner；D-210 fixes Mailbox/Flag Block pressure。 + Protocol feasibility and exact predicates/state representation remain preflight/Technical questions。 + +### P-022 — Scheduled collection is job creation；Mail execution must not remain core-py-only + +- **Upstream**: current core-py can run Mail collection，while a future background-capable native InKCre Peer may act as a + higher-frequency mail client and browser Peers may remain incapable。 +- **Broken assumption**: scheduled collection is a second execution path，or Source/Extension runtime permanently belongs to + the Python server because it is implemented there first。 +- **Candidate owner**: Source job contract owns collection commands/results；scheduling creates jobs。Extension/Peer runtime + capability determines who can execute，without requiring every Peer to be symmetric。 +- **Impact**: scheduler placement、job claiming、future Peer capability advertisement/routing、cadence configuration and + duplicate creation may eventually interact。 +- **Evidence**: D-211 confirms one collect-job semantic、current core-py execution and future multi-Peer direction。 +- **Status**: Product/topology direction confirmed；no distributed scheduler or native-Peer implementation is authorized in + the current scope without Technical evidence。 + +### P-023 — Collect jobs must not absorb Source-internal work-unit semantics + +- **Upstream**: one Mail collection invocation can visit multiple mailboxes and persist useful messages despite a local + mailbox/item failure。 +- **Broken assumption**: generic job success means every source-native sub-scope was synchronized，or jobs must own each + mailbox cursor、transaction and partial outcome。 +- **Candidate owner**: collect job owns invocation/runtime status；Source owns its deep collection algorithm、state、partial + progress and public completion semantics。 +- **Impact**: Mail、RSS、GitHub、Twitter and future Sources must not shape generic job statuses around their private traversal + units。Observability may carry detail without turning it into job-domain completeness。 +- **Evidence**: D-212 fixes the CronJob analogy and withdraws the proposed mailbox-failure → job-failure rule。 +- **Status**: D-212 confirms the shallow invocation boundary；D-213 fixes one-shot/no-retry lifecycle。Technical audit must + check current job/source boundaries without assuming a broad job schema expansion。 + +### P-024 — Mail attachment bytes become Organization enrichment after metadata-only collection + +- **Upstream**: Mail collection can preserve attachment identity/metadata without eagerly downloading every MIME part；later + use may justify durable local content。 +- **Broken assumption**: a complete communication record or credible email client must persist all attachment bytes during + collection，or every later attachment fetch still belongs to Source collection。 +- **Candidate owner**: Mail collection owns attachment metadata/remote reference；Mail Resolver owns the materialization + capability and may delegate remote acquisition to Source/extension；the resulting additive graph change can be classified + as Organization enrichment。Use may perform transient fetch/stream without persistence。 +- **Impact**: Mail canonical graph、authenticated part access、Storage、semantic content Resolver、Organization tools and + client-web attachment behavior。 +- **Evidence**: IMAP4rev2 selective fetch plus official Apple/Gmail configurable/lazy behavior；D-214 owns the product + inference and the unit evidence file retains links。 +- **Status**: D-215 corrects the owner topology；D-216 fixes textual body versus lazy non-text inline parts。Exact technical + interfaces remain open。 + +### P-025 — Native Mail references and inferred linking have different owners + +- **Upstream**: reply/reference headers can identify directed Email relationships，while missing targets or incomplete + headers may tempt subject/time heuristics。 +- **Broken assumption**: collection should create a generic Thread entity or infer every plausible conversation edge to make + the graph useful。 +- **Candidate owner**: Mail collection owns source-native reply/reference facts；Organization linking owns additive inferred + relations over the existing graph。Use derives thread views from graph structure。 +- **Impact**: Email canonical content、relation vocabulary、unresolved external references、later reconciliation、query and + client-web thread navigation。 +- **Evidence**: D-217 fixes native directed relations、no generic Thread Block and no collection-time inference。 +- **Status**: Product boundary confirmed；unresolved-reference representation and exact relation grammar remain Technical。 + +### P-026 — Extension-specific Block rendering must not become an extension-specific browsing product + +- **Upstream**: client-web must render rich Email content，but InKCre's Mail value is not yet expressed as a traditional + inbox/folder/list workflow。 +- **Broken assumption**: a rich source type requires a dedicated page and source-specific query UI，or a complete email + client/agent direction implies copying ordinary email client information architecture。 +- **Candidate owner**: Mail client-web extension owns Email Resolver/renderer；GraphSurface owns Block placement、selection + and cross-Block navigation。Generic query owns any future cross-type query behavior。 +- **Impact**: extension Module Federation artifact、Resolver registration、BlockContent/detail/graph surfaces and query UI。 +- **Evidence**: D-218 plus current Twitter `TweetResolver.contentComp` → generic `BlockContent` implementation path。 +- **Status**: Product boundary confirmed；D-219 closes the predefined-query edge，while exact Email renderer remains a + Technical design question。 + +### P-027 — `contentComp` obscures solved-content presentation + +- **Upstream**: an Email requires root content plus participants、mailbox/flag/reply Relations and attachment metadata to + render；Tweet already loads attachment Relations and Blocks before rendering。 +- **Broken assumption**: a Resolver component only renders the literal `Block.content` column，or UI components should query + and reconstruct graph semantics themselves。 +- **Candidate owner**: client-web Resolver owns focal-Block solved projection and registers its `SolvedContentRenderer`；a + generic controller owns Resolver/view lifecycle。GraphSurface alone owns target transitions；`BlockInspector` owns only + persistence facts and current-Block commands such as view content/rumination。 +- **Impact**: `ResolverClass.contentComp`、`ContentCompProps`、`BlockContent`、`BlockDetailsPanel`/unused `relations` prop、 + Tweet solved type、future Email renderer、loading/error/refresh and navigation context。 +- **Evidence**: D-220 and current Twitter resolver/component/BlockContent code path。 +- **Status**: `SolvedContentRenderer`、`BlockInspectorPopup`、view-solved-content meaning and GraphSurface route realization + confirmed；generic render context withdrawn。First-class `SolvedContentPopup`/InfoBaseRouter and complete + Resolver + typed solved-content props are confirmed。D-226 corrects GraphSurface to current realizer rather than route； + `overview` is accepted，while focal route shape/operations and Vue Router adapter remain Technical。 + +### P-028 — Mail does not justify generic query work without a reachability blocker + +- **Upstream**: Mail needs generic rendering/use but its distinct query workflow is not yet understood。 +- **Broken assumption**: every rich source unit must add source-specific browsing/filters or opportunistically expand generic + query APIs。 +- **Candidate owner**: preflight proves reachability through current generic surfaces；only a real blocker can pressure generic + info-base query ownership。 +- **Impact**: client-web graph/start surfaces、feature retrieval、query APIs and unit scope。 +- **Evidence**: D-219 explicitly accepts no preplanned increment and a minimal blocker-driven exception。 +- **Status**: confirmed scope guardrail；no query implementation is currently approved。 + +### P-029 — Resolver identity selects Block behavior；hydration and solving are different layers + +- **Upstream**: Mail/Tweet rendering requires a focal Block's own payload plus local graph context，while the current + `contentComp` interface suggests literal column rendering and current solved models inconsistently call the canonical + focal value `canonical`、`root` or a repeated domain noun。 +- **Broken assumption**: `block.resolver` is only a decoder/display type，hydrated content and solved content are synonyms， + or a graph-derived projection may silently mutate/impersonate canonical root content。 +- **Candidate owner**: shared PRD glossary owns product meaning；Product TDD owns exact Block → Resolver behavior selection + and content-layer topology；client-web local architecture owns `BlockRenderer` and its host navigation contract。 +- **Impact**: Python/TypeScript Resolver contracts、solved models、Tweet/Mail renderers、generic BlockContent/details/graph + surfaces、refresh caching and durable documentation vocabulary。 +- **Evidence**: D-221 plus current `TweetResolver` graph lookup、`MemoResolver`/RSS graph-aware solved projections and the + existing shared hydration/Resolver contract。 +- **Status**: behavior/content boundary、`SolvedContentRenderer`、`BlockInspectorPopup`、`.root` and surface-independent + InfoBase view/router topology confirmed；exact focal route/adapter mechanics remain Technical。 + +### P-030 — Container ownership changes when its lifecycle becomes route-destination behavior + +- **Upstream**: GraphSurface/ListSurface must host addressable Block-inspection and solved-content destinations without + duplicating close/back logic or teaching presentation-neutral resolver content about client navigation。 +- **Broken assumption**: every container must be assembled by the parent regardless of behavior，or any component may wrap + itself merely because it currently appears in one Popup。 +- **Candidate owner**: shared Product TDD owns the general container/content rule and exact exception criterion；client-web + local architecture owns `InfoBaseView` navigation host、route destination outlet、`BlockInspectorPopup` and + `SolvedContentPopup` composition。 +- **Impact**: GraphSurface/ListSurface responsibilities、Popup ownership、route/back semantics、component naming、Resolver + renderer portability and later UI implementation guidance。 +- **Evidence**: D-235 establishes popup/back behavior；D-236 identifies presentation-neutral content、navigation host and + route destination outlet as the stable explanatory model。 +- **Status**: Technical vocabulary and ownership confirmed for implementation。Promote to Product TDD after evidence；a UI + agent skill should later derive from durable truth，but its build/delivery infrastructure is explicitly outside this unit。 + +## Carried cross-unit pressures + +### P-017 — Secret-safe observability is a shared boundary，not an adapter-by-adapter assertion + +- **Upstream**: a real AI provider failure rendered dialect configuration in an exception path and exposed a plaintext + credential。 +- **Broken assumption**: marking one runtime field secret or testing its `repr` proves that logs、tracebacks、structured + events and diagnostics cannot disclose secrets。 +- **Candidate owner**: future shared observability/error-reporting infrastructure should define sanitization at its + ingestion/rendering boundary；typed secret config remains the local producer-side baseline。 +- **Impact**: AI、Source、Storage、Extension and deployment diagnostics can all carry database-owned credentials。 +- **Evidence**: Pydantic `SecretStr` repairs ordinary model representation for the AI dialect，but an adapter-level test + would only repeat Pydantic behavior and would not exercise any repository-wide observability path。 +- **Status**: D-197 removes the false local proof and records the cross-cutting pressure。No observability subsystem is + introduced by the semantic-retrieval unit。 + +### P-001 — Canonical content requires a decoder-generation contract + +- **Upstream**: CanonicalMemo 直接持久化为 `block.content`，而 shape 未来可能演化。 +- **Broken assumption**: resolver identifier 可以解释所有历史 content，但 registry 没有明确的 + generation retention / unknown-generation contract。 +- **Candidate owner**: block resolver identity 选择 exact decoder；extension/resolver registry + 保留 live generations。 +- **Impact**: backend write/read、migration、client-web parity、extension unload。 +- **Evidence**: GitHubRepo、FeedItem、Tweet 等 JSON block contents 已有同类无版本压力；当前 + registry 只按 exact resolver string 发现实现。 +- **Status**: D-026 已确认 versioned resolver identity，不在 payload 或 block column 重复 + version；retention 与 explicit failure 等待 Technical gate/preflight。 + +### P-003 — Persistence time is not memo-authored time + +- **Upstream**: backend 必须 round-trip memo create/update time；future collector 还会导入历史 + memo。 +- **Broken assumption**: block `created_at / updated_at` 可以同时代表 row persistence 与用户 + authored/source time。 +- **Candidate owner**: CanonicalMemo root facts；block timestamps 继续只属于 persistence row。 +- **Impact**: chronological retrieval、display、update response、future reconciliation。 +- **Evidence**: BlockModel timestamps 由 insert/update 产生；GitHubRepo、FeedItem、Email 也另存 + source times。 +- **Status**: D-032/D-042 confirm authority and exact CanonicalMemo v1 serialization。 + +### P-004 — Compatible backend reads need a semantic boundary + +- **Upstream**: memo 客户端 write 后会 list/sync/read native resources。 +- **Broken assumption**: backend 只需 collection endpoint，或 API adapter 可以直接解释 graph + rows。 +- **Candidate owner**: memo resolver owns graph → solved memo；generation adapter owns solved memo + → native response。 +- **Impact**: resolver output、API adapter、client compatibility、acceptance fixtures。 +- **Evidence**: D-021/D-023；当前 Resolver 已承担 raw content + local relations 的联合解释。 +- **Status**: ownership、D-042 root facts and D-047 exact native fixtures confirmed。 + +### P-005 — Graph-owned components must not be copied into root content + +- **Upstream**: attachments、comments、parent 与 references 必须可独立保存、解析和更新。 +- **Broken assumption**: 为 adapter 方便，可以同时把同一 component reference 放进 serialized + root content 和 relations。 +- **Candidate owner**: root CanonicalMemo 只持 root facts；component blocks/relations 持结构。 +- **Impact**: mutation、resolver、native response、organization。 +- **Evidence**: attachment 需要独立 storage；comment 是完整 memo；references 有独立 graph + identity/use value。 +- **Status**: D-012/D-013/D-017/D-022/D-044 confirmed。 + +### P-008 — Protocol user identity is not a core User + +- **Upstream**: MoeMemos startup 需要 current user、GENERAL settings、Bearer token 和 + creator-scoped sync。 +- **Broken assumption**: InKCre JWT / `ClientModel` 可以直接表示终端 memo user。 +- **Candidate owner**: deployment-scoped memo backend configuration 投影一个 profile/settings； + Memos extension ordinary config owns the credential（D-039）。 +- **Impact**: auth、ownership、visibility/ACL、migrations、Hub、client-web administration。 +- **Evidence**: current JWT authenticates peer claims；ClientModel 是 peer deployment；block / + relation/source 没有 terminal-user owner。 +- **Status**: D-033/D-039/D-047 confirmed。Profile/settings/credential/wire fixtures are closed。 + +### P-009 — Native mutation needs explicit coordination without completeness guarantees + +- **Upstream**: create/update/delete memo 可能同时改变 root、attachments、comments、relations + 和 raw storage；客户端成功必须代表全部已持久化。 +- **Broken assumption**: 多个各自 commit 的 convenience write paths 可以在没有 coordinator 的情况 + 下自然表达 primary success、component order 与 cleanup。 +- **Candidate owner**: memo application service coordinates graph/storage commands；D-041 explicitly + rejects graph completeness as an observable guarantee。 +- **Impact**: idempotency、owned deletion、storage residue、HTTP success and diagnosis。 +- **Evidence**: current convenience create paths independently commit；relation FK cascade 不删除 + component blocks/raw storage。 +- **Status**: D-041 closes failure completeness；D-046 closes minimum owned cleanup。 + +### P-010 — Namespaced extension routing is sufficient (resolved) + +- **Upstream**: MoeMemos 2.0.4 accepts a user-provided host and declares `api/v1/...` as relative + Retrofit paths；attachment URLs append `file/...` to that same host path。 +- **Assumption tested**: API annotations containing `api/v1` do **not** imply that the backend must own + server-root `/api/v1`。With base URL `https://<deployment>/memos/`, they resolve under the existing + Memos extension namespace。 +- **Owner**: current `ExtensionBase` `/{extension_id}` router remains the route owner；Memos adapter owns + its relative protocol paths。 +- **Impact**: no top-level mount/alias or external-protocol route registry。Existing duplicate-start、 + ineffective-close and post-disable reachability remain lifecycle defects, not a reason to change path + ownership。 +- **Evidence**: released client login normalization preserves the supplied path；V1 annotations have no + leading slash；resource URI construction appends to the stored host。 +- **Status**: resolved；O-017 withdrawn。D-036/D-039 assign peer and Memos authentication to composed + route dependencies；the global middleware limitation is now an implementation pressure。 + +### P-011 — Memos PAT follows the current extension-config trust boundary + +- **Upstream**: single-user MoeMemos needs a long-lived/revocable Bearer credential distinct from + InKCre peer JWT。 +- **Broken assumption**: upstream Memos' hashed multi-user PAT storage must be copied even though InKCre + currently persists and returns other recoverable credentials through ordinary config。 +- **Candidate owner**: Memos extension config owns the deployment-scoped credential； + core/default-extension routes own peer-auth dependencies。Existing extension config surface owns + peer-authorized setup/change/clear commands。 +- **Impact**: token setup/read/rotation/revocation、config validation/update ordering、middleware error + behavior、operator setup。 +- **Evidence**: JWT middleware rejects non-peer tokens before handlers；extension config is JSONB，is + returned by extension APIs and already includes Twitter/Telegram/IMAP/GitHub credentials。 +- **Status**: D-039 confirms ordinary raw-PAT config persistence、exact public detection、immediate + replacement/revocation and validated generic update ordering；D-036/D-037 own route composition and + lifetime。No O-018 design remains open。 + D-033 continues to prohibit introducing core User/tenant ownership。 + +### P-012 — Memos attachments require writable raw storage + +- **Upstream**: selected client journey requires attachment create/list/delete and authenticated raw + download。 +- **Broken assumption**: `Storage.get_raw_content()` plus remote-URL implementations are sufficient for + all collected raw content。 +- **Owner**: D-043：graph component owns attachment identity/role/metadata；a PostgreSQL BYTEA-backed + generic storage owns only the pointer + raw bytes。 +- **Impact**: storage interface/backend、schema/migration if needed、file serving、validation、graph + + external side-effect failure/compensation。 +- **Evidence**: no storage put/delete API, upload/file response route, multipart dependency or local raw + backend exists。 +- **Status**: D-043 confirmed one small DB raw table；filesystem compensation and inline graph base64 are + excluded。D-041 means shared DB transactionality is available but not a product guarantee。 + +### P-013 — Native sync requires an addressable query contract + +- **Upstream**: MoeMemos requests NORMAL and ARCHIVED lists with creator filter, stable ordering, + `pageSize=200` and page tokens。 +- **Broken assumption**: recent blocks by resolver is equivalent to a protocol list/query projection。 +- **Candidate owner**: memo extension query repository consumes canonical/graph authority and exposes + adapter-ready pages；a derived projection/index is optional application support, never a second memo + authority。 +- **Impact**: D-042 fact ownership、cursor stability、query performance、conditional migrations and + client-web database projection。 +- **Evidence**: current `BlockManager.get_recent` only filters resolver and limits rows；state/visibility/ + pinned owner remains open。 +- **Status**: D-042 resolved the fact owner；do not add a memo table/index preemptively。 + +### P-014 — Route topology is not automatically a hot extension lifecycle + +- **Upstream**: core-py and client-web currently expose same-process extension enable/disable intent； + Memos backend would inherit that behavior if accepted。 +- **Broken assumption**: `include_router()` route composition can be paired with a symmetric close/ + disable operation, or leaving routes installed behind per-request running guards is complexity-free。 +- **Owner**: D-038 assigns same-process route availability to `ExtensionManager`。One retained + extension-owned router/route-set handle bounds direct add/remove and prevents duplicate registration。 +- **Impact**: core-py ExtensionManager、client-web enable/disable UX、OpenAPI、auth ownership、resource + shutdown and deployment restart semantics。 +- **Evidence**: core-py dynamically includes routes but never removes them；client-web remote enable/ + disable calls expect immediate activation/deactivation。Pinned FastAPI 0.139.2 retains included child + routers and versions effective-route/OpenAPI caches，so a localized child route-set mutation is viable； + current deployment is one web process/replica。 +- **Status**: D-038 accepted direct route mutation for this project's risk profile。Do not add scattered + running dependencies、request-drain generations or an isolated dispatcher in the MVP；reopen only if + deployment concurrency or extension resource lifetime makes the low-cost contract false。 + +### P-015 — Config update is a shared operation，not an extension-specific workaround + +- **Upstream**: Memos needs validated、atomic-as-observed config replacement while running；extension、 + source、storage and future configurable owners share the broad operation。 +- **Broken assumption**: persisting an unvalidated dict and validating only when applying runtime state is + an acceptable update order，or every owner should implement its own update pipeline。 +- **Candidate owner**: a generic config-update operation owns merge → `config_cls` validation → durable + write → live apply ordering；each target manager still owns address resolution、persistence and + reconfiguration consequences。 +- **Impact**: extension MVP implementation now；source/storage APIs and lifecycle only after their current + paths are explored。This is not evidence for a generic hook registry or a universal config table。 +- **Evidence**: ExtensionBase、SourceBase and StorageBase already declare typed `config_cls` seams，while + current extension update commits raw input before runtime validation。 +- **Status**: direction accepted by Sir；exact reusable interface and non-extension adoption remain a + later technical design item。D-039 fixes the extension-first ordering；Memos must not preemptively invent + unverified source/storage lifecycle。 + +### P-016 — Memos proves ordered and pre-memo attachment states + +- **Upstream**: Memos 0.29.1 deliberately returns attachments in request order；MoeMemos may upload them + with `memo=null` before creating the memo。 +- **Broken assumption**: attachment order has no current source pressure，or every successful upload is + already part of a memo graph transaction。 +- **Owner**: D-040/D-044 make order and grammar extension-owned。D-043 owns raw storage；D-046 owns + minimum cleanup semantics。 +- **Impact**: relation payload、PATCH set semantics、orphan list/delete、failure residue and acceptance + wording。 +- **Evidence**: tagged server reverses the requested list and rewrites `updated_ts` before ordered reads； + tagged MoeMemos uploads resources before memo create and sends nullable `memo` in streaming JSON。 +- **Status**: D-040/D-043/D-044/D-046 confirmed。 + +## Deferred future-unit pressures + +### P-002 — Collector reconciliation needs external identity + +- **Future upstream**: scan/webhook/update/delete 要命中同一 external memo。 +- **Pressure**: `block.id` 只表达 local identity；默认 `(resolver, content)` equality 不是稳定 + external identity。 +- **Direction**: resolver-owned CanonicalMemo 可保存可靠 provenance/identity facts，按 stable + external scope → local `source_id` fallback 的梯度做 best-effort exact match；不建 generic + source binding table，不做 content/time fuzzy overwrite。 +- **Status**: D-027/D-028 direction confirmed；collector deferred，不参与当前 v1 backend gate。 + +### P-006 — A family body representation cannot copy every product protocol + +- **Future upstream**: Memos 使用 Markdown，flomo 或其他 memo products 可能有不同 authored + representation/fidelity。 +- **Pressure**: 直接复制某一产品 shape 会污染 canonical；开放 `format + arbitrary payload` + 又把分支成本扩散到 resolver/use。 +- **Direction**: product-generation adapter 负责 normalize；canonical generation 固定其 body + representation。真实 fidelity failure 再推动新 generation 或 component graph。 +- **Status**: Canonical v1 当前采用 Markdown semantic minimum；flomo 已延期，不能驱动当前 + wire 扩张。 + +### P-007 — External ID needs its native uniqueness scope + +- **Future upstream**: source 重建、重复配置或 instance locator 变化后仍希望识别同一 memo。 +- **Pressure**: `source_id` 不是 external provenance truth，mutable URL/username 也不能冒充 + immutable namespace。 +- **Direction**: 只使用产品能证明的 stable scope；没有时接受 local best-effort scope，匹配 + 不足宁可产生 duplicate，再由 organization 改善 use。 +- **Status**: policy confirmed；Memos 0.30 evidence 仅为 research，backend MVP 自己是 memo + authority，不需要 cross-system reconciliation。 diff --git a/tasks/knowledge-lifecycle-capabilities/terminology-audit.md b/tasks/knowledge-lifecycle-capabilities/terminology-audit.md new file mode 100644 index 0000000..daa0c14 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/terminology-audit.md @@ -0,0 +1,274 @@ +# InKCre Terminology Audit + +> Repository evidence snapshot，最初采集于 2026-07-29；不是 durable product truth，也不维护 +> program phase 或当前问题。后续决定以 [decision register](decisions/index.md) 为准。 + +## 1. 方法与约束 + +本审计只记录已经能从以下来源证明的语言和行为： + +1. Sir 给出的产品语言； +2. Hub `10-prd` 中的产品术语与可观察行为； +3. Hub `20-product-tdd` 中的跨单元合同； +4. core-py / client-web 的本地文档、schema、代码路径和测试； +5. git 历史中可以证明的旧设计含义。 + +其中 Hub 只能证明“现有文档写了什么”，不能单独证明设计正确；Sir 的产品判断按高置信度 +输入记录，也不是无需核验的绝对事实。如果这些来源不一致，本审计记录冲突，不用新术语掩盖冲突。描述外部系统返回的 +tweet、image、text、mail 等对象时，暂用“source 原生对象”这一解释性短语；它不是建议加入 +glossary 的新领域对象。 + +`resource` 可以出现在 Memos 等外部 API 的原生语言中，但不是 InKCre 已确认的 +通用概念。`source_key` 也已撤回：如果候选模型无法说清 key 在世界中指向什么， +就应回到 source-native identity 与 source instance 的语义，而不是换一个 opaque 名字。 + +## 2. 当前相对稳定的名词 + +| 术语 | Hub 产品含义 | 当前实现中的实际指代 | 审计结论 | +|---|---|---|---| +| `info-base` | 存储并链接可复用信息单元的共享记忆中心 | 没有单一 `InfoBaseModel`；由 blocks、relations 及其持久化协调构成 | 产品边界，不是表或 Python package 的同义词 | +| `block` | info-base 中一个持久化信息单元 | `BlockModel`；`content` 是 inline 内容或 storage pointer，`resolver` 指定解释方式 | 当前唯一明确的通用持久信息单元 | +| `relation` | 两个 block 之间有向、有类型的链接 | `RelationModel(from_, to_, content)`；所谓类型目前只是自由文本 `content` | 图的持久边；“typed”尚无独立 schema/catalog 保证 | +| `source` | 从外部系统采集数据的 capability | 又被用于 source class、runtime instance 和 `sources` 表记录 | 产品能力含义稳定，代码/UI 层级严重重载 | +| `source type` | Hub 使用但未定义 | source class 的注册标识及 `sources_types` catalog row | 可复用实现/配置 schema 层 | +| `source instance` | Hub 使用但未定义 | `sources` 表中的一条 type/config/collect_at/state 配置记录 | 一个已配置的采集入口 | +| `collect job` | 一次 source collection 的执行记录 | `sources_collect_jobs` 的 PENDING/RUNNING/FINISHED/FAILED row | 与 source type、source instance 明确不同 | +| `resolver` | 解释 block 及其局部图上下文，产生可用意义 | resolver type string、class、instance；还实际承担去重、graph factory、breakdown | “解释”是已确认职责，其余职责是现状压力，不是已确认公共定义 | +| `storage` | block 未内联内容时，按 pointer 取得 actual content 的组件 | storage type、storage row、handler、`Block.storage` FK；内置实现是 HTTP fetch | 它是 pointer-based content 访问能力，不等于 PostgreSQL、bucket，也不等于“把信息持久化”这一动作 | +| `sink` | 检索或索引 info-base 内容供 downstream use 的 capability | core-py 的 embedding/reasoning/RAG；client-web 的 graph 可视化代码也放在 sink package | 产品能力词，不是持久对象 | +| `extension` | 可安装并增加 source/resolver/sink 行为的 capability | Python package/class/DB install row/runtime；Web 还有 Module Federation module/runtime | 共享产品词存在，但不同 runtime 的 artifact 与运行模型尚未统一 | +| `peer` | 一个 deployment 中参与同一 info-base 的 InKCre runtime node | 现有 `clients` record、core-py runtime、client-web runtime | D-109 选定的 technical domain term；现有技术 `client` 名称是迁移地址 | +| `client` | 用户安装、访问或使用的 InKCre 应用 | marketing/landing、non-technical docs、`client-web` 等产品/仓库名 | 保留为 user-facing product term；不用于表达 peer-to-peer technical authority | + +`client` 仍可描述真实的 client/server 或外部协议角色，例如 HTTP client、Memos-compatible client、OAuth +`client_id` 和第三方 SDK client。D-109 只迁移 technical runtime-node domain，不授权机械替换这些词或 +user-facing Client product language。 + +### 关键内容词 + +| 词 | 当前可证实含义 | +|---|---| +| `block content` | `BlockModel.content` 持久化字符串；可能是 inline 内容，也可能只是 pointer | +| `hydrated content` | `block.get_hydrated_content()` 返回的 actual content；可能直接来自 inline `content`，也可能由 storage 按 `content` pointer 取得。`raw content` / `real content` 不再作为 canonical terms | +| `solved content` | resolver 解释后的 runtime 表示 | +| `text for embedding` | resolver 为 embedding 生成的文字表示；不必等同 solved content 或 block content | + +因此，后续不能笼统地用 “content” 指代以上所有对象。 + +### Block 职责与 resolver 版本语言 + +| 词 | 稳定含义 | 不表示 | +|---|---|---| +| `metadata block` | canonical content 拥有与关联 semantic content block 有关的 protocol/source-authored identity、role、declaration 或 lifecycle 事实的普通 block；RSS Enclosure 与 MemosAttachment 是当前例子 | 不是新 table/base class，不是 `blocks.metadata` JSON，也不是 source module/runtime wrapper | +| `semantic content block` | resolver 根据 hydrated content 与 graph 解释信息的普通 block；image/audio/video/PDF/EPUB/ZIP/file 是当前例子 | 不表示其 `block.content` 一定 inline；storage-backed 时持久字段仍是 pointer | +| `exact resolver ID` | persisted `block.resolver` 的精确值，选择一个 decoder/solved/graph contract | 不是 MIME、Python/TypeScript class name 或 extension runtime state | +| `resolver contract version` | exact resolver ID 中只在 InKCre persisted/solved/graph contract 不兼容变化时推进的版本轴，例如 `core.image.v1` 的 `v1` | 不是 parser package 版本或文件格式小版本 | + +resolver 的 `v1` / `v2` 轴使用一般开发者更熟悉的 `version`，具体称 `resolver contract version`。 +这不表示 `generation` 在所有上下文都错误，也不需要为它构造“语义过载”理由；此处只是 +`version` 更符合惯例、更自解释。类似地，`source wrapper` / `protocol wrapper` 不再作为 graph 名词，因为对象 +实际是 block；按职责叫 `metadata block`，其关联的内容 block 叫 `semantic content block`。 + +### 稳定的 effect / selection 参数语言 + +D-074 已固定以下跨 peer 语义；这里只记录命名证据与边界，决定 authority 仍是 +[decision register](decisions/index.md)： + +| 参数/动作 | 稳定语义 | 明确不表示 | +|---|---|---| +| `refresh` | 绕过并替换一个可复用的 instance/local snapshot，重新读取当前 authority | 不自行授权 AI/graph write，也不重新生成已有 derivation | +| `materialize_missing` / `materializeMissing` | 允许 resolver 在所需派生图缺失时创建它；ordinary default 为允许 | 不替换已经存在的派生结果 | +| `recompute` | organization 显式重新生成已有 derived representation | 不是普通 resolver cache refresh | +| `invalidate` | 丢弃缓存，不在同一动作中读取替代值 | 不是 refresh 或 recompute | +| `include_in` / `includeIn` | 选择 subject block 为 `to_` 的 direct relations | 不表示 recursive traversal | +| `include_out` / `includeOut` | 选择 subject block 为 `from_` 的 direct relations | 不表示 recursive traversal | + +当前 client-web resolver 的 `getRelations()`、`getRawContent()`、`getSolvedContent()` 以 `force` 表达 cache +bypass/replacement；这是迁移到 `refresh` 的直接证据。`ResolverCache.invalidate()` 已经正确表达只丢弃缓存。 +core-py 与 client-web 的 relation direction selectors 已语义对齐，只需保留各语言惯用 casing。 + +`force` 不成为新的 InKCre-owned 通用 boolean；第三方协议原生参数不因此改名。`reload` 不作为 +`refresh` 同义词，若未来使用，应指 runtime/config/module lifecycle。现有 source job `full` 也不获得 +common-contract 地位：RSS/Atom、Mail、GitHub Stars 与 Twitter Bookmark 的实现把 scan breadth、incremental +cutoff bypass、ordering 和 pagination 等不同效果压在同一名字下，必须按各 source 的产品合同拆解。 + +其余命中没有形成新的跨边界参数合同：SQLAlchemy/SQLModel `Session.refresh()` 与 OAuth +`refresh_token` 都是 dependency/protocol-owned vocabulary；`limit/offset/cursor` 与 Memos +`pageSize/pageToken` 的 ordering、continuation identity 和 invalidation 规则由各 query/protocol boundary +拥有，不能仅因它们都做 pagination 就提升为一个 InKCre-wide options contract。 + +### 四者的联合语义 + +分别定义 block、relation、resolver、storage 仍然不够。当前代码和历史共同支持： + +1. block 是持久锚点，不等于完整的 runtime 信息对象； +2. `block.content` 没有固定的自解释格式：它可以是 inline source-specific payload，也可以是 + 交给 storage 的 pointer； +3. storage 只回答“怎样从 pointer 取得 actual content”，不解释其含义； +4. block hydration 隐藏 inline/pointer 分支，resolver 解释 hydrated content，并可读取该 block 的 local relations； +5. resolver 还可以取得 relation 另一端 block 的 resolver output，因此 relation 可以成为当前 + block 的动态内容，而不只是查询或可视化时使用的边； +6. use 消费的往往是 resolver 生成的 solved content / text / embedding text，未必是 + `block.content`。 + +直接证据包括： + +- Tweet 的基础 JSON 放在根 block，attachments 放在相邻 blocks；`TweetResolver` 沿 + `attachment:*` relations 调用相邻 resolver，重建带附件的 Tweet； +- HTML block 以 URL 为 content、HTTP storage 取得 HTML，resolver 优先使用 + `text content` relation，否则把 hydrated HTML 转成 text； +- Image block 以 URL 为 content、image storage 取得 image data，`alt:text` relation 可直接成为 + 它的 text 表示； +- Mail 把 Email 与 EmailAddress 分成 blocks，以 `from` / `to` / `cc` relations 表达组合结构。 + +所以 `SubGraphForm` 只是 source / resolver 向 info-base 提交递归写入的表单；它不能替代上述 +联合信息模型，也不能直接升格为 collection 的产品输出类型。 + +## 3. 当前动作语言 + +### Collection / `collect` / `record` + +Hub 当前承诺： + +- source 从外部采集数据； +- collection 产生可复用 block 或 relation； +- collect job 记录一次执行； +- observable outcome 是新增或更新的可复用信息单元。 + +core-py 当前行为： + +1. source adapter 从远端 API/协议得到 connector-specific model，例如 `Tweet`、`GithubRepo`、 + `FeedItem`、`Email`、`TelegramMessage`； +2. source 或 resolver factory 将其转换成 `SubGraphForm`； +3. `InfoBaseManager` 持久化其中的 blocks 和 relations； +4. 没有另一种通用、独立持久化的“采集所得对象”。 + +`collect()` 是主动拉取，`record(data)` 是 webhook 等被动接收；二者在产品语言中都属于收集。 + +### Organization / `organize` + +这个动作在现有文档与实现中没有统一含义： + +1. Hub 说 organization 将 collected information normalize 成 blocks/relations； +2. source 的 `collect()` 已经在创建并持久化 blocks/relations； +3. `SourceBase._organize(block_id)` 是 source-specific post-collection hook； +4. `BlockManager.organize(block)` 调用 `Resolver.breakdown()`； +5. 历史实现曾在 source 收集并持久化 block 后，再异步调度 `_organize`； +6. client-web 没有 organize 产品/API/UI 表面。 + +Sir 已给出新的高置信度产品定义：organization 是打理已经存在的 info-base，使后续 use +效果更好；它不是 collection 的一部分。breakdown、merge、linking 是当前已知能力，不构成 +organization 的完备枚举。当前任何一个 `organize` 实现都不应作为新设计约束，但仍可作为 +历史证据或可复用实现接受审视。 + +### Breakdown / Linking / Merge + +| 动作 | 当前证据 | +|---|---| +| `breakdown` | Resolver 有 generator contract；只有 ImageResolver 原型会派生 blocks/relations | +| `linking` | 没有同名 domain API；目前通过创建 `RelationModel` / graph arcs 表达 | +| `merge` | 没有 block/relation consolidation 行为；SQLAlchemy `session.merge` 与产品 merge 无关 | + +这三个词来自本任务的产品需求，尚未进入 Hub glossary 或跨单元合同。 + +### Retrieval / Query / Search / Application + +当前存在至少四种不同动作，不能混称： + +1. 按 ID 取得 block/relation； +2. block 按需 hydrate inline/storage-backed content； +3. sink 用 embedding/reasoning 找相关 blocks； +4. client-web 读取全图并做布局、community detection 和导航。 + +`search` 在 core-py 还常指 IMAP/Twitter 等外部协议查询；client-web 首页的 search 只是占位。 +“应用”是本任务的上位能力语言，但当前没有对应的领域对象或统一 API。 + +## 4. 当前实际拓扑 + +```text +extension artifact/class + -> 注册 source / resolver / storage / sink runtime capability + +source type + -> 配置 source instance + -> collect job + -> source collect() / record() + -> source 原生对象 + -> 编码为 blocks + relations（当前常由 SubGraphForm 承载写入） + -> InfoBaseManager 持久化 + -> info-base graph + +block + -> get_hydrated_content() 隐藏 inline content / storage pointer 分支 + -> resolver 结合 hydrated content + local relations + 相邻 resolver output + -> solved content / text + -> sink 做 embedding / reasoning / RAG + +block + -> BlockManager.organize() + -> Resolver.breakdown() + -> derived blocks / relations +``` + +client-web 大量通过 PostgREST 直接读写 source、job、extension、block、relation；这与 core-py +manager/REST 命令并存，属于跨单元 authority 冲突，不是术语定义本身。 + +## 5. 已证实的主要冲突 + +### C1 — source-specific 对象与 graph 的关系已经确认 + +- 已确认 block 继续是信息进入 info-base 后的基本持久信息单元； +- 已确认不应发明通用“采集所得对象”； +- Tweet、GithubRepo、FeedItem 等是 collection 的 source-specific 输入形状，不是 block + 之外的持久对象； +- collection 通过把 source-specific 数据保存为 block/relation graph 完成收集。 + +具体 graph 映射、identity 与更新语义仍需讨论，但不再存在并列 source object store 的歧义。 + +### C2 — “存储”自然语言与 `storage` 组件不是一回事 + +Sir 所说“收集就是将某物采集（存储）到系统中”的“存储”是持久化动作；InKCre 的 +canonical `storage` 却是按 pointer 取得 actual content 的组件。后续中文讨论应使用“持久化到 +InKCre”描述前者，保留反引号 `storage` 指后者。 + +### C3 — Organization 的产品方向已澄清，具体动作尚未定义 + +source graph construction、source `_organize`、resolver `breakdown` 和 Hub normalization 彼此 +重叠但不等价。新设计以“打理已有 info-base、改善 use”为起点，现有实现不构成约束； +linking、breakdown、merge 的具体操作数、结果与正确性仍未定义。organization 不能先被假定 +为单纯 graph rewrite：它可能需要 block hydration 取得 actual content、resolver 取得 solved content， +再形成新的 blocks/relations。 + +### C4 — Resolver 的核心联合职责与附加实现职责尚未分开 + +“解释 block”实际至少包括 hydrated content 与 local relations 的联合解释；这不是普通 blob parser。 +代码还把 block identity/dedup、graph construction、breakdown 放进 resolver。后三者是否属于 +resolver 必须由具体动作合同推导，不能因现有代码位置而默认接受。 + +### C5 — Retrieval 既是产品能力又被用于内部取数 + +content hydration、graph读取、embedding retrieval、reasoning retrieval、RAG 和 UI graph +navigation 目前缺少上位关系；“特征/语义/图导航检索”需要建立在明确的查询对象和结果之上。 + +### C6 — Extension 是共享记录，但不是共享 artifact/runtime 模型 + +core-py 的 Python artifact 与 client-web 的 Module Federation remote 使用同一 extension row； +`installed/enabled/running` 的共享语义已有合同，但 registry、artifact identity 和两个 runtime +如何共同实现它尚未定义。 + +## 6. Audit Conclusions + +- `block` 是 source information 进入 info-base 后的基本持久信息单元；Tweet、GithubRepo、 + FeedItem 等只是 source-native input shapes，collection 通过 graph 持久化它们。 +- block row 不独自拥有完整可用含义；block hydration 隐藏 inline/pointer 分支，resolver 联合 hydrated + content 与 local relations 解释，storage 只在需要时按 pointer 取得 actual content。 +- collection 可以为了正确持久化而拆分信息；organization 则打理已存在的 info-base、改善 + use。二者都可能读取 resolver/storage，不应用“是否从 graph 开始”做机械边界。 +- `resource`、`observation`、audit、replay、通用 collected object 等没有获得项目事实支持， + 不应凭讨论便利引入领域语言。 +- relation predicate、resolver 附加职责、extension artifact/runtime 与 retrieval 上位合同仍需 + 由具体 implementable unit 证明，而不能从现有代码命名直接推导。 + +program 拆分与讨论顺序现在由 [capability-map.md](capability-map.md) 统一维护；当前 active unit 是 +[Mail extension](units/mail-extension/packet.md)。Memos backend MVP、RSS hardening 与 semantic retrieval 已完成； +后续 scope 不继承其实现批准。 diff --git a/tasks/knowledge-lifecycle-capabilities/tracks/application.md b/tasks/knowledge-lifecycle-capabilities/tracks/application.md new file mode 100644 index 0000000..dee0634 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/tracks/application.md @@ -0,0 +1,40 @@ +# Application Track + +## Objective + +让用户或下游能力通过 info-base 有效找到、导航和使用信息。 + +## Known Capability Slices + +- 特征检索 +- 语义检索 +- 图导航检索 + +## Design Card + +每种检索依次说明： + +1. 用户问题与查询输入； +2. 可检索对象和返回结果； +3. 消费 block content、raw、solved、relation 或 resolver projection 的方式； +4. 排序、过滤、路径、上下文与组合语义； +5. 所需索引或其他派生结构及其 owner； +6. core-py API 与 client-web 交互； +7. 精确性、质量、性能和失败验收； +8. 最小迭代与 Hub / 跨仓影响。 + +## Guardrails + +- point lookup、storage raw fetch 与产品检索不能混称。 +- indexing 是 application / retrieval 的实现支撑，不是 organization。 +- sink 或 client 不接管 info-base 的 graph authority。 + +## Active Slice + +当前 active application Unit 是 +[Graph navigation retrieval](../units/graph-navigation-retrieval/packet.md)。它从已定位 Block/Relation 取得 bounded、 +direction-preserving 的既有 graph facts；Resolver 仍围绕 focal Block 解释/投影局部图，InfoBase View 只负责消费与 +presentation。Product/Technical/Acceptance/implementation plan/preflight 已关闭，当前等待 Impact Handshake 后的新 +明确实施授权。 +[Feature retrieval](../units/feature-retrieval/packet.md)、[Semantic retrieval](../units/semantic-retrieval/packet.md) 与 +[Mail extension](../units/mail-extension/packet.md) 已完成；它们的 corpus/use pressure 可被复用，但不接管本 unit。 diff --git a/tasks/knowledge-lifecycle-capabilities/tracks/collection.md b/tasks/knowledge-lifecycle-capabilities/tracks/collection.md new file mode 100644 index 0000000..de39322 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/tracks/collection.md @@ -0,0 +1,49 @@ +# Collection Track + +## Objective + +增加和完善 sources，使 source-specific 信息能够被可靠地收集为 InKCre 的 blocks / relations， +并在需要时推动 extension、resolver、storage 与 registry 能力演进。 + +## Slices + +| Slice | Role | Status | +|---|---|---| +| Existing sources | 兼容性与回归基线：Twitter、GitHub、RSS、Mail、Telegram | Evidence collected | +| Memos extension | memo-like 的首个可实现单元；当前 scope 是 backend MVP | Complete — implementation/E2E and owner commits complete | +| RSS extension hardening | 首个传统 source-path reference unit；固定 RSS/Atom identity、change、graph、job/state 与 failure contract | Complete — human-accepted 2026-08-03 | +| Mail extension | 以高价值真实邮箱建立可信 collection baseline，并向 organization/query/client-web 传导真实压力 | Complete — implementation/J1–J4/promotion complete 2026-08-11 | +| CalDAV | 日历协议与结构化同步压力 | Queued | +| Nextcloud Files | 文件层级、binary、pointer/storage 压力 | Queued | +| Apple Notes | macOS local runtime 与受限访问压力 | Queued | + +## Per-Source Design Card + +每个 source 依次填写： + +1. **Product**: 用户为什么收集它；纳入和排除什么；用户可观察结果与失败。 +2. **Access**: 外部系统、认证、主动 collect / 被动 record、平台限制。 +3. **Native shape**: source-specific objects 与保留的信息。 +4. **Info-base expression**: 根 block、相关 blocks、relations、inline / pointer。 +5. **Resolver / storage**: persisted、hydrated、solved、text/use 表示与外部内容访问。 +6. **Change behavior**: identity、新增、更新、删除、移动、重复执行与冲突。 +7. **Extension pressure**: artifact、registration、installation、runtime、client-web。 +8. **Acceptance**: static proof boundary、black-box protocol/service input、错误、兼容性、重复执行与 + 端到端 durable effects；white-box test 需满足 D-049 exception。 +9. **Iteration**: 最小 thin slice 与后续增量。 +10. **Hub projection**: PRD、Product TDD 与 claim realization 影响。 + +## Abstraction Rule + +不先设计通用 collected object 或万能 source framework。通常只有两个真实 source units 重复 +出现同一压力，且统一不会抹去 source-specific 语义时，才进入公共合同；若单个 unit 已证明 +现有机制无法正确交付，则只允许解决该 blocker 的最小横切改造。 + +## Active Slice + +当前没有 active collection slice。[Mail extension](../units/mail-extension/packet.md) 已通过 real +Dovecot/PostgreSQL J1–J3、built-browser/Peer J4、durable promotion 与 post-bump repository gates；下一 slice 由 +program selection gate 决定。 +它不以协议完整性或 feature checklist 判定 MVP / MLP；先固定低成本持有邮件信息的真实 job、collection +boundary 与可接受代价。RSS hardening 与 Memos backend MVP 均已完成；future RSS hardening、Memos +collectors/products 仍需以新 scope 过 gate，不能继承既有 approval。 diff --git a/tasks/knowledge-lifecycle-capabilities/tracks/organization.md b/tasks/knowledge-lifecycle-capabilities/tracks/organization.md new file mode 100644 index 0000000..c79ff93 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/tracks/organization.md @@ -0,0 +1,34 @@ +# Organization Track + +## Objective + +打理 info-base，使 use 的效果更好。 + +## Known Capability Slices + +- breakdown +- merge +- linking + +这些是当前已知能力，不是 organization 的完备定义。只有能说明怎样改善 use 的新能力，才可 +进入讨论队列。 + +## Design Card + +每项 organization 能力依次说明： + +1. 用户问题与期望改善的 use； +2. 输入的 blocks / relations / raw / solved 表示； +3. 对 info-base 的可观察改变； +4. resolver、storage、LLM、extension 与人工判断的职责； +5. 正确性、不变量、失败和部分结果； +6. 自动化验收 fixture 与质量指标； +7. 最小迭代及与其他 organization 能力的关系； +8. Hub 与跨仓影响。 + +## Guardrails + +- organization 不是 collection 的一部分。 +- indexing 不属于 organization。 +- 现有 `SourceBase._organize`、`BlockManager.organize` 和 resolver `breakdown` 不约束新设计。 +- 不因为已知能力列表存在，就假定每个 organization 场景都必须归入其中之一。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/acceptance.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/acceptance.md new file mode 100644 index 0000000..eb1cabe --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/acceptance.md @@ -0,0 +1,201 @@ +# Lexical Retrieval Acceptance（Preflight-refined Review Contract） + +## Authority + +Acceptance uses graph state created through ordinary real producer paths，not rows shaped to satisfy the search +implementation。Readable aliases live only in the harness and resolve to actual Block IDs after collection。The corpus reuses +the proven Memos API、RSS/Atom HTTP protocol harness、Mail IMAP harness、PostgreSQL binary storage and the pinned public-domain +SQLite Architecture article where useful。 + +J6 uses one coherent real engineering-media authority rather than hand-authored OCR/ASR strings：NASA asset +`GSFC_20140121_GPM_m11457_Dave_McComas`，“GPM: Meet the Team: Dave McComas”。NASA's asset service supplies the real MP4 and +authored VTT discussing flight-software requirements、implementation、tests、simulation and spacecraft integration。The explicit +acceptance harness pins the origin ID/URLs + observed ETags，creates a bounded clip，extracts its real audio and an on-screen-text +frame，and remuxes the authored VTT into a standard subtitle stream。Derived local files remain ignored acceptance artifacts； +production code sees only ordinary stored bytes/Resolvers and never the corpus ID、aliases or expected phrases。NASA's official +media-usage guidance is the provenance/license authority。 + +AI is absent from J1–J5/J7 and required only by J6's real textualization/interpretation paths。Deterministic unit tests may +verify ranking mechanics，but they do not replace the following black-box journeys。 + +## J1 — Exact Technical Identifier + +Collect a real technical article containing a distinctive identifier and a semantically related distractor that lacks the +identifier。Query the exact identifier through the public lexical capability。 + +- the target existing Block is returned before the distractor； +- evidence names literal/term matching and contains a bounded plain excerpt； +- no answer or transient entity is generated。 + +## J2 — Chinese Fragment Without Tokenization Assumption + +Create a Memo through the real Memos endpoint containing a distinctive Chinese phrase and another conceptually related Memo +without that phrase。Query a contiguous fragment without spaces。 + +- the literal-bearing Memo Block is returned； +- the semantic-only distractor is not promoted merely for conceptual similarity； +- the journey does not depend on a Chinese-specific PostgreSQL dictionary。 + +## J3 — Media Metadata Before Materialization + +Collect a Mail message with an attachment metadata Block whose filename/description/media type is distinctive，while leaving +the attachment bytes unmaterialized。Query that lexical feature。 + +- the MIME-part metadata Block is returned without unnecessary byte download or semantic content child creation，even though + the maintenance call permits missing materialization； +- the result label/excerpt explains the match； +- opening the result can continue through existing graph/Resolver behavior independently。 + +## J4 — Maintenance、Update And Deletion + +Create and index a Block，change the authoritative Block so its lexical projection changes，then run bounded maintenance；also +delete another indexed Block。 + +- stale records are excluded before maintenance； +- maintenance makes the new projection searchable and the old clue non-matching； +- Block deletion removes its derived lexical record through FK cascade； +- repeated/concurrent maintenance converges without duplicate records。 +- direct and Cron-created exact lexical maintain/rebuild Jobs use the same Handler path and expose bounded reports in Job state； +- semantic maintain/rebuild uses its exact typed Jobs after the peer-local interval scheduler is removed，without changing + retrieval results or profile selection semantics。 + +An additional controlled Resolver case proves that when lexical text is genuinely absent and an exact materializer is +available，maintenance may create the derived graph and then index its text。The assertion is on the ordinary graph/Resolver +effect，not a lexical-manager-owned OCR implementation。 + +## J5 — PDF Body Recall Without Metadata Substitution + +Store a real PDF with a text layer through ordinary Storage/Resolver paths。Give its body one distinctive phrase that is absent +from title、author、filename and other metadata，then run lexical maintenance and query that phrase。 + +- an existing Block carrying the PDF body evidence is returned，whether that is the root projection or a materialized semantic- + content child； +- the result does not succeed merely because the phrase was copied into fixture metadata； +- when a semantic-content child owns the body，the PDF root record does not recursively index the same body and produce a + mechanical parent/child duplicate； +- metadata-only、encrypted and scanned-without-OCR branches report their actual capability boundary rather than claiming body + completeness。 + +## J6 — Multimodal Textualization Recall + +Through ordinary PostgreSQL binary Storage and exact core Resolvers，persist one image carrying visible text，one audio recording +carrying spoken text and one video carrying subtitle/spoken/on-screen text。Run credentialed lexical maintenance with exact +configured multimodal models/extractors，then query one distinctive phrase from each medium。 + +- each query returns the materialized existing text Block，not a transient OCR/transcript object or the parent copied body； +- each child remains connected to its media Block through the exact `text`、`transcript` or `subtitle` role；a video with + distinct signals keeps distinct children rather than one aggregate text； +- the parent and child do not mechanically duplicate the complete derived text in their lexical records； +- rerunning maintain reuses an existing derivation under `materialize_missing` semantics； +- image/audio/video exact Resolver configs select their role-scoped Models，and the same Model may be reused across fields； +- an absent/dangling/disabled/incapable Model makes only its exact derivation unavailable，without preventing source-native or + other-role children from being created and indexed。 + +For the same real media corpus，run the system-driven Organization command with a configured multimodal model/Agent，then query +a distinctive concept present only in its submitted description/summary rather than OCR/transcript text。 + +- the system selects the media candidate without a per-Block user request and persists an additive `interpretation` graph； +- the selected Agent is the deployment's independent media-interpretation Agent，not the rumination Agent by implicit fallback； +- image、audio and video candidates route through their exact configured Agent references，with one deliberately unusable slot + proving it does not block another modality； +- the Agent receives actual media through the canonical UserMessage content part，while the real provider request proves dialect- + local image/audio/video wire translation and joint modality + Tool-calling support； +- lexical maintenance indexes the resulting text Block and the concept query returns that Block； +- no Resolver `materialize_missing` call authors or replaces the interpretation； +- faithful text and interpretation remain distinguishable graph facts rather than one ambiguous combined projection； +- a second automatic Job skips the now-present interpretation without claiming freshness，while one controlled failed/no-output + candidate does not block another missing candidate。 + +## J7 — Browser Recall Journey + +From a built client-web runtime，submit a lexical query in the first real InfoBaseListView，observe the bounded result list， +select a target and return with browser Back。 + +- the browser delegates the exact capability through a live eligible Peer； +- result ordering/evidence survives transport validation； +- selection uses `InfoBaseRouter` and opens the target Block Inspector in the List view's destination outlet，without navigating + away to GraphSurface； +- opening solved content remains in the same List host and closing either popup uses browser Back； +- Back restores the query/results rather than turning the lexical result view into a second history authority。 + +## Quality Gate + +- every exact unique clue must return its target in the first position； +- a full literal phrase match must outrank a terms-only distractor； +- every returned row must be an existing fresh Block lexical record and an existing Block； +- no journey may depend on direct `Block.content` parsing、fixture-only production rows、AI answer quality or graph test labels + leaking into production implementation。 + +No million-row performance gate or large-file corpus is added without measured product pressure。Schema verification confirms +the GIN indexes exist；the black-box journeys prove behavior rather than asserting one planner implementation。 + +## Delivery Closure Gate + +Local J1–J7 and static checks are necessary but insufficient because this increment changes PostgreSQL extensions、schema、 +migrations、generated peer projections and two independently deployed Spokes。After Sir separately authorizes commits and +pushes： + +- the core-py PR's exact-head CI and preview delivery must pass against its fresh Neon preview branch，including database init、 + migration/readiness verification and deployed `/readyz` probe； +- the client-web PR's exact-head CI and Cloudflare preview must pass； +- J7 must exercise that built client against the matching core-py PR preview。A client CI run against the stable core release is + useful regression evidence but cannot substitute for this cross-branch integration proof。 + +The lexical increment is marked accepted only after these gates are green；a pushed commit or locally migrated database alone +does not close the Unit。 + +## Local Execution Evidence（2026-08-13） + +- J1–J5/J7 deterministic producer、Resolver、record、Job、Peer and UI contracts pass in their owning suites。 +- J6 passed against real PostgreSQL binary Storage、the pinned NASA asset and real + `qwen3.5-omni-flash` through `core.alibaba-model-studio.v1`。Observed outputs include image-frame OCR + `GPM / nasa.gov/gpm`，audio/video transcript facts about flight software、requirements、testing、simulation and + Tanegashima，then an actual Agent ToolCall submitted an interpretation graph that became lexically recallable。 +- the harness pins the MP4/VTT origin URLs、ETags and SHA-256 values；derived frame/audio/subtitle/video files stay under the + ignored `tests/lexical_retrieval/acceptance/.assets/` boundary。 +- a fresh disposable Neon branch migrated from empty state to `1e4c7a9b2d5f`。The complete database-enabled non-migration suite + passed 414 tests with 5 skips；upgrade/downgrade acceptance passed separately，and RSS real HTTP-double journeys passed after + Job catalog isolation was corrected。 +- the exact PDM 2.27.0 hermetic core contract passes after the `origin/main` extension-registry integration with 445 tests、41 + explicitly external skips and zero lint/type diagnostics；a fresh PostgreSQL database also passes migration、init and + development readiness against the merge revision。 +- client-web's complete test/lint/type/build contract passes with the new facade and InfoBaseListView。At this local checkpoint， + generated cross-repo database/OpenAPI artifacts intentionally awaited an immutable pushed core source；the delivery evidence + below closes that boundary rather than fabricating artifacts from a dirty worktree。 +- Render service/API behavior first passed a mocked official-wire controller test and actionlint；real provider execution then + proceeded under the separate delivery gate recorded below rather than being inferred from those checks。 + +## Delivery Execution Evidence（2026-08-13—15） + +- core-py PR #45 and its delivery follow-ups are merged。The final delivery-race fix is PR #63 at merge commit + `4b180467dd8ca79a28a241fa5e38333692bcb4d3`；its exact-head repository、portable-runtime、Neon preview and Heroku preview + checks passed。The same merge commit passed push CI、runtime/Extension publication and canonical production deployment；the + immutable runtime is `ghcr.io/inkcre/core-py@sha256:d990badb4ce140fef6b13c73e802caeb6b6e7651eaa13719d62cfce527ccb33f`。 +- client-web PR #50 exact head `6d9611bb69c1d8633408b1a1507f69b99732c526` passed all eight owning CI jobs；Cloudflare + preview run `31728559690` remains live at `https://preview-client-web-pr-50.inkcre-client-web.pages.dev/`。The manual J7 + journey used that built client、a live delegated lexical capability and ordinary Block navigation；query/result state survived + Inspector and solved-content navigation plus two browser Back operations，with no console errors。 +- Sir explicitly authorized merging core before final integrated acceptance，so the original matching core PR preview no longer + existed when J7 ran。J7 therefore used the production-admitted core feature line while the core PR preview、core production and + client PR preview were each proven independently。This named execution-order substitution does not claim that one matching- + preview session occurred。 +- the real fork `xiaoland/core-py` fast-forwarded to the exact core merge and，after provider settings were made internally + consistent，workflow run `31875608739` created a fresh PostgreSQL 17 Neon project plus two new Render Free services in one + 4m44s run。It migrated from empty state to `3f7a9c2d5e1b`，published one live four-capability Peer and completed the one-shot + PostgREST read/write/deny verifier。A preceding run failed at Neon project lookup before database、Render or controller mutation + because its API key belonged to a different organization；the onboarding guide now makes that cross-setting invariant explicit。 +- independent deployed probes returned Core `200 ready` and PostgREST `401` from `postgrest/14.15`。After more than 16 minutes + without traffic，the same Core request cold-started to ready in 54.086s；the still-independent PostgREST service woke to its + expected `401` in 12.927s。The Core wake renewed Peer `8c77be1b-b030-5f03-a80f-c75eb78cdede` to a live database lease，which + proves the documented wake-Core-first journey without representing the Free profile as continuously available。 +- workflow logs expose only masked GitHub secrets and non-authorizing service coordinates；no database URL、role password、JWT + signing key or provider authorization appeared in the inspected output。 + +All product and technical acceptance surfaces are green；the authorized core-first execution-order substitution is recorded +above rather than hidden。Client-web PR #50 later passed fresh required checks and squash-merged as `9b5c870`；its main CI also +passed。The first Pages attempt then exposed a delivery-boundary permission defect while installing private `@inkcre/ui-web` +（run `32017501336`）。Client-web PR #75 added only the missing job-level `packages: read`；exact-main Client checks run +`32024516290` and Pages delivery run `32024731957` both passed for merge +`17160ae5e9a49d89fa60d35cee86223f41972c0b`。The deployed `https://app.inkcre.dev/` returned HTTP 200 and referenced the +new static entry `/assets/index-4dhPC3GP.js`。This closes app delivery without folding the defect into Feature retrieval or +the independently completed native Extension release。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/deployment-readiness.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/deployment-readiness.md new file mode 100644 index 0000000..c1d2649 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/deployment-readiness.md @@ -0,0 +1,181 @@ +# Feature Retrieval Deployment Readiness（Implemented Baseline） + +## Product Pressure + +This increment needs a real preview because it changes migrations、PostgreSQL extensions、runtime readiness、Peer capabilities +and the built browser journey。Sir also wants the resulting repository to support a GitHub-only self-host trial：a friend forks +`core-py`、creates provider accounts、sets a small documented input set and deploys without cloning locally。 + +This is a delivery-supporting slice of the active increment，not feature-retrieval product semantics。`client-web` fork delivery is +out of scope；the public client remains a consumer of an owner-selected deployment profile。 + +Two journeys must remain separate： + +1. **Fork owner**：sets their own private JWT signing secret and receives full deployment-Peer authority； +2. **Canonical public demo participant**：may receive intentionally public full-Peer write authority only when every credential + reachable through that authority is also deliberately public/disposable。This is a mutable scratch deployment，not a reader。 + +## Correct Credential Boundary + +The current `JWT_SECRET` is an HS256 signing key，not a role-scoped password。A holder can choose every claim before signing； +publishing it as an “anonymous secret” therefore lets an untrusted caller mint the exact admitted +`role=authenticated, iss=inkcre-peer, aud=inkcre-api` token and become a full Peer。The intended read-only result cannot be +obtained by documenting a different role next to the same public key。 + +Actor/boundary analysis： + +| Value | Actor capability if public | Protected asset/harm | Classification | +| --- | --- | --- | --- | +| Provider API token、Neon/API host authorization | anonymous infrastructure/provider operation | deployment takeover、data deletion、external cost | boundary violation | +| database owner/runtime-role password | anonymous direct database connection | bypass application admission、full data/config control | boundary violation | +| current HS256 Peer signing secret | mint any accepted Peer claims | read credentials、mutate/delete info-base、invoke charged work | boundary violation | +| fixed already-signed read-only token | only its immutable claims until expiry | bounded by the admitted read-only principal | potentially valid，but it is a token，not a public signing secret | +| project IDs、service names、base URLs、model IDs | discovery only | no admission by itself | public variable | + +The current `anonymous` database role deliberately has no privileges and `inkcre_internal.check_jwt` admits only +`authenticated`。A real public-read journey therefore needs an explicit guest/anonymous product surface and safe projections； +it is not a README-only change。In particular，lexical `materialize_missing` may mutate graph and incur provider cost，so a public +reader cannot merely receive the existing full retrieval command。 + +Sir instead selected an intentionally public-writable canonical demo。That removes graph integrity/availability as protected demo +assets，but it does not automatically make unrelated provider/source credentials public：`authenticated` currently has `GRANT +ALL` over the complete `inkcre` schema，and `ai_providers.config` plus Source/Extension configs can contain raw API keys、mail +passwords and tokens。A published full-Peer key is therefore valid only for a credential-free scratch database or one containing +credentials deliberately treated as public and loss-bounded。Fork deployments remain owner-private by default。 + +## Deployment Secret Set + +`INKCRE_DEPLOYMENT_SECRET` is withdrawn。It had been proposed only as a derivation root for database-role passwords。On first +deployment，the trusted controller generates both role passwords in memory、initializes the database and creates Render services +with complete runtime URLs。On later runs，Render's owner-authorized API returns the existing service environment values；the +controller recovers the runtime-role passwords from those URLs and converges the database before deploying。This avoids both an +extra derivation secret and needless per-deploy credential rotation。 + +Fork owner inputs should therefore be： + +- private repository secrets：host API authorization、`NEON_API_KEY`、`JWT_SECRET`； +- public repository variables：Neon project ID、unique service names/prefixes and other non-authorizing deployment identity； +- generated first-run values：database role passwords，then retained only in Render runtime configuration and PostgreSQL role + state；masked before any command can emit them。 + +AI provider keys remain deployment config entered after admission，not GitHub delivery secrets。 + +## Cloudflare Python Worker Fit + +Cloudflare now officially supports FastAPI through its Python ASGI bridge，so route compatibility is real。A complete current +`core-py` Peer is nevertheless not a drop-in Python Worker： + +- Python Workers run in Pyodide and accept pure/PyEmscripten packages；the fixed core artifact includes native/runtime-heavy + dependencies such as `psycopg[binary]`、PyAV、lxml and Pillow，while Cloudflare's documented Hyperdrive drivers are currently + JavaScript/TypeScript and Rust oriented。The existing SQLAlchemy + psycopg database boundary cannot consume Hyperdrive by + changing a URL。 +- current `run.py` owns process-lifetime APScheduler loops for Peer lease renewal、Cron checks、Job checks and maintenance。 + Workers execute request/scheduled invocations；post-response work is bounded，and their Scheduled handler has a 15-minute wall + bound。A Worker host would need an event-driven runtime adapter，not merely another ASGI entry point。 +- current Peer discovery uses an expiring live lease。A platform service can be logically routable while no isolate is resident， + but the present process-owned renewal stops and the Peer disappears from routing。Scale-to-zero hosting therefore exposes a + real missing availability/activation contract rather than a TTL tuning problem。 +- the Free plan's 10 ms CPU and 3 MB Worker bundle bounds are not credible for the current full artifact。Cloudflare Containers + can run the existing Docker image，but Containers require the $5/month Workers Paid plan and therefore do not remove the + cost barrier that motivated this branch。 + +Cloudflare remains valuable as a future specialized/event-driven Peer target and as a cheap external scheduled wake-up，but +porting full `core-py` there would be a separate implementable unit with runtime-host、database-port、dependency-profile and +scale-to-zero discovery work。It must not be smuggled into feature-retrieval implementation readiness。 + +## Neon Data API / PostgREST Spike + +Neon's managed Data API is included in its Free plan and presents a PostgREST-compatible HTTPS surface。A disposable real Neon +branch was used on 2026-08-12 to test the exact InKCre protocol rather than infer compatibility from marketing： + +- exposed `inkcre` schema table read：passed； +- `create_storage_blob(bytea)` raw `application/octet-stream` request：passed； +- `read_storage_blob(uuid)` custom octet-stream response domain：passed； +- pointer-stable `storage_blobs` PATCH and DELETE：passed。 + +The transport can therefore replace the standalone PostgREST binary in principle。The current deployment contract cannot adopt +it unchanged： + +- Data API refuses enablement when `authenticator`、`authenticated` or `anonymous` already exist，then creates and controls those + exact roles itself；this conflicts with `db provision-roles` and InKCre's executable role authority。 +- Data API validates Managed Better Auth or an external JWKS。It does not accept the current raw HS256 secret configuration，and + the available settings do not expose the current `inkcre_internal.check_jwt` pre-request contract。 +- Managed Better Auth can issue a short-lived `role=anonymous` token without login，which is a promising public-demo mechanism， + but adopting it for owner/Peer writes would introduce a different admission model。Publishing an HS256 key through JWKS would + simply recreate the original signing-authority leak。 +- Data API cannot be enabled on an expiring Neon branch，which conflicts with current TTL preview branches unless preview cleanup + becomes controller-owned rather than Neon expiration-owned。 + +Conclusion：keep standalone PostgREST for the immediate implementation/fork path。Treat Neon Data API as a proven transport and +a later deployment/auth migration，not as a protocol risk or an immediate switch。 + +The disposable Data API and branch were deleted after the spike；no production data or contract was changed。 + +## Immediate Free-Host Direction + +A conventional free Docker web host is a better near-term fit than Python Workers because it can run the checked core and +PostgREST artifacts without changing their domain/runtime boundaries。Render currently supports free Docker web services and a +repository Blueprint，but free services sleep after 15 minutes and share 750 running hours per workspace。That is acceptable for +interactive use with cold starts，not for reliable scheduled collection or continuously live Peer leases。 + +The accepted target is a **Render + Neon self-host deployment profile with demo-grade Free-plan availability**，not a projection +of InKCre's canonical production environment： + +```text +fork GitHub workflow + -> validate exact commit and required GitHub inputs + -> obtain the Neon owner URL only inside the controller + -> recover or first-run generate runtime-role passwords + -> converge database before service deployment + -> create/converge two auto-deploy-disabled Render Free Docker web services + -> deploy the exact commit and wait for terminal status + -> publish non-secret deployment profile + -> owner wakes core before capability discovery and configures the public client +``` + +Render's API can create Docker services with full environment、select `Dockerfile` vs `Dockerfile.postgrest` and trigger a +specific commit。Render also translates configured `SOURCE_REVISION` into the existing Docker build argument，preserving the +artifact's source evidence without another Dockerfile。The current process-import baseline is about 147 MiB RSS before live +database bootstrap against a 512 MiB Free instance；this is sufficient preflight headroom but not a substitute for the required +real Render startup/operation probe。The 0.1 CPU allocation primarily predicts cold-start/Job latency，not a semantic change。 + +After 15 minutes without user traffic，both services may sleep。A browser request wakes PostgREST automatically，but an expired +core Peer lease means database discovery cannot itself activate core；the documented demo journey must first open/wake core and +wait for `/readyz`，then enter client retrieval。Inventing activation-aware discovery is outside this delivery slice。 + +Heroku remains the exact always-on/reference delivery until a second host passes the same black-box contract。A second conventional +container Peer can later be attached to the same Neon database for multi-Peer claim/scheduling evidence；Cloudflare Python Worker +is not required to obtain that evidence。 + +## Acceptance Direction + +- use a disposable real GitHub fork，not a renamed local checkout； +- perform onboarding only through provider and GitHub web surfaces； +- one manual workflow starts checked artifact -> database -> services -> readiness； +- the public client consumes a non-secret profile and the fork owner's locally supplied JWT secret；the canonical public demo may + separately document its intentionally public signing key only after credential sanitation； +- no provider/source credential、database credential or host authorization appears in README、workflow summary、logs、artifacts、 + deployment profile or client build； +- missing inputs fail once with exact GitHub setting names before provider mutation； +- free-host cold start and missed scheduled work are stated limits，not silently represented as full Peer equivalence； +- canonical InKCre deployment retains its stricter exact identity/lineage checks。 + +## Operational Execution Evidence + +Public signing-key publication is withdrawn for this increment。Self-host deployments and the canonical public-demo environment +both keep signing authority private by default；an owner may still share a JWT privately during a live demonstration。Public +read-only admission and Cloudflare-native Peer execution remain separate follow-up design problems。 + +The controller、workflow、README journey and simulated Render API contract are implemented。A real public fork at exact core +commit `4b180467dd8ca79a28a241fa5e38333692bcb4d3` completed workflow run `31875608739` against fresh Neon project +`patient-sky-13885177` and the new `inkcre-fork-xiaoland-v2` Render namespace。Fresh migration、exact service creation、Core +readiness、Peer publication and the one-shot PostgREST contract passed。The host returned Core `200 ready` and PostgREST's +expected anonymous `401` independently。 + +After more than 16 minutes without traffic，Core's first `/readyz` request took 54.086s and returned ready；PostgREST then took +12.927s and returned its expected `401` from `postgrest/14.15`。Core renewed the previously sleeping Peer's lease after wake。 +This closes the operational execution gate while confirming，rather than weakening，the documented Free-host availability limit。 + +One earlier run stopped at Neon project lookup before provider mutation because the configured API key and project belonged to +different organizations。The user-facing guide now states the cross-setting invariant。The successful run's logs expose only +masked secrets and non-authorizing deployment coordinates。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/implementation-plan.md new file mode 100644 index 0000000..30ec9af --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/implementation-plan.md @@ -0,0 +1,106 @@ +# Lexical Retrieval Implementation Plan（Provisional） + +> This is an execution preflight，not implementation authorization。The plan is revised after product/technical review and an +> Impact Handshake。 + +## I0 — Freeze Contracts + +- confirm product matching/evidence semantics and the Resolver projection-context choice； +- reconcile both current feature branches with their latest `main` before layering new implementation，resolving ownership + conflicts rather than assuming the old PR heads remain executable baselines； +- promote accepted shared PRD/Product-TDD deltas through the Hub workflow at the implementation/promotion stage； +- freeze exact schema、capability、route and cross-peer names。 + +## I1 — Resolver Projection Parity + +- add stable lexical text context to Python and TypeScript Resolver contracts； +- implement bounded lexical projections across built-in and in-scope extension Resolvers，with media metadata remaining plain + derived text； +- close the current PDF metadata-only gap so a real text-layer PDF body remains searchable，while semantic-content child Blocks + are indexed independently under a Block-local、non-recursive projection contract； +- remove client-web `getStrForEmbedding()` and keep semantic consumers on default `getText()`； +- verify no direct `Block.content` retrieval path；lexical maintenance permits only Resolver-owned missing materialization and + does not construct organization graph itself。 + +## I2 — Provider-Neutral Multimodal Textualization + +- extend canonical AI User messages with non-empty text/image/audio/video content parts carrying actual bytes + MIME；derive + requested modalities in AIManager，verify model/dialect support，and translate bytes only inside the exact dialect adapter； +- extend `core.openai-compatible.v1` only with OpenAI-standard Chat image/base64-audio parts，and add the proposed + `core.alibaba-model-studio.v1` cross-capability dialect for documented Alibaba video/audio-URL extensions；do not route + audio/video through Responses or silently widen the generic dialect； +- add an optional Storage-owned transfer-URL hint without changing lazy hydrated bytes as the semantic/default input；prefer + bounded inline bytes，then use an accepted URL only when inline transfer is unavailable，with no hidden retry/staging； +- aggregate Alibaba streaming multimodal text/ToolCall deltas behind the existing complete AssistantMessage result； +- preserve the Thread message-only contract and keep Block/Storage refs、provider URLs and Resolver-specific solved types out of + AI/Agent schemas； +- register exact image/audio/video Resolver deployment configs with role-scoped direct AI Model references；keep prompts + code-owned，references use-time validated and AIManager free of automatic Model selection/failover； +- implement image、audio and video missing-text materialization as separate ordinary `core.text.v1` child graphs using exact + `text`、`transcript` and `subtitle` information-role Relations，with idempotent reuse per role； +- page maintenance by increasing Block ID without a frozen upper bound，while enforcing the invocation's hard `max_records` + limit so newly materialized child Blocks can be indexed immediately without an unbounded self-extending scan； +- implement a system-driven media-interpretation Organization path that consumes Resolver solved content，runs a configured + per-modality independent Agent，selects missing-only candidates and persists additive interpretation graphs through existing + validated Tools，without routing that effect through `materialize_missing` or adding attempt/freshness records； +- register one parameterless `core.organization.media_interpretation.v1` convergence Job；keep candidate/modality selection + inside Organization，keep candidate/diagnostic bounds code-owned，and use Job state only for bounded outcome reporting； +- add best-effort AIModel/Agent local-executability predicates for Job `can_handle` without remote probes or fallback； +- verify a credentialed real-media path and bounded unavailable outcomes without adding perceptual matching。 + +## I3 — Derived Record And Migration + +- add `pg_trgm` plus `block_lexical_records` table、FK/timestamps and GIN indexes； +- update SQLModel/Alembic metadata、application-table/readiness manifest and generated client database projection； +- keep one Block-keyed record with no profile/config/independent sequence。 + +## I4 — Local Maintenance、Jobs And Retrieval + +- implement bounded lexical maintain/rebuild、freshness and diagnostics behind exact + `core.feature_retrieval.lexical.{maintain,rebuild}.v1` Job Handlers； +- add `core.semantic_retrieval.{maintain,rebuild}.v1` Handlers around existing profile/options contracts，project reports to Job + state，and delete the peer-local direct timer； +- implement escaped literal + plain all-term matching、evidence classification、ranking and plain excerpt construction； +- verify missing/stale/oversized/unknown projection branches without creating a retry/job/dirty lifecycle。 + +## I5 — Exact Peer Capability + +- add `core.feature_retrieval.lexical.v1` typed request/result and fixed non-delegating core inbound； +- register/publish the inbound with existing runtime lifecycle； +- add `@inkcre/core` facade and `routeToPeer` delegation while keeping capability payload opaque to PeerManager。 + +## I6 — Client-Web User Journey + +- turn the start placeholder into the first URL-backed InfoBaseListView with loading/empty/error/result states； +- render label、plain excerpt and match reason； +- host BlockInspectorPopup/SolvedContentPopup as the List view's route-destination outlet； +- extend the app's InfoBaseRouter projection so Block navigation preserves the active List/Graph host and browser Back semantics。 + +## I7 — Acceptance、Hardening And Promotion + +- build J1–J7 from real Memos/RSS/Mail/Storage producers、pinned article and real multimodal corpus； +- run focused checks，then full `pdm run check` and `pnpm check`； +- after separately authorized owner-scoped commits/pushes，wait for the exact core-py PR database/application preview and + client-web PR preview to pass，then run J7 against the matching core preview； +- update core/client local architecture and exact shared Hub owners，publish Hub first，then bump each Spoke separately； +- close the lexical increment without claiming perceptual or hybrid retrieval complete。 + +## Concrete Change Map（Preflight） + +- **Resolver contract**：`app/business/info_base/resolver/main.py` + every Python exact Resolver；client-web + `packages/core/src/info-base/resolvers/**`。Add lexical context consistently and hard-cut TypeScript embedding-only projection。 +- **AI/Agent seam**：`app/schemas/ai/chat.py`、`app/business/ai/{main,contracts}.py`、generic OpenAI translation helpers、 + `app/business/ai/dialects/{openai_compatible,alibaba_model_studio}.py` and Agent Thread tests。No InfoBase imports + enter AI/Agent contracts；Alibaba protocol extensions do not leak into the generic dialect ID。 +- **Organization**：split the growing `app/business/organization.py` only if implementation size justifies a package；retain + `OrganizationManager` as public owner and reuse existing graph Tools。Register the parameterless Handler through ordinary + runtime import/bootstrap。 +- **Lexical capability**：new schema/business package + fixed route/inbound，then register table/readiness/capability facts in + existing catalog owners；do not put ranking in a PostgreSQL RPC。 +- **Semantic cutover**：wrap existing manager methods with exact Job Handlers，remove `run.py`'s direct timer and its Settings/ + runtime-composition tests，preserving direct manager calls for internal/test use。 +- **Database**：one append-only migration for `pg_trgm`、`block_lexical_records` and indexes；update metadata/readiness/role + projection and client-web generated database types through existing generators。 +- **Browser**：add `@inkcre/core` lexical facade/Peer codec；replace the Start placeholder with InfoBaseListView and let that + navigation host own its popup outlet。Selection continues through the bound stateless `InfoBaseRouter`，with no second + navigation/history authority。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/packet.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/packet.md new file mode 100644 index 0000000..7968e57 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/packet.md @@ -0,0 +1,62 @@ +# Feature Retrieval + +- **Unit ID**: `feature-retrieval`（产品与技术 ownership boundary 已成立；内部按可独立交付的 increment 推进）。 +- **State**: **Complete**。 +- **Objective**: 从真实 information-finding jobs 倒推 semantic similarity 与 graph navigation 之外缺少的 retrieval + capability，使用户或 Agent 能从持续增长的 info-base 中定位有用的 Blocks/Relations，并得到足以理解“为什么命中” + 的 evidence。文本、图像、音频、视频、文件、source 与 graph facts 都可以进入发现视野。 +- **Guardrails**: 不把“特征”先验定义为 metadata filters、全文搜索、embedding 之外的剩余集合或 modality operator + checklist；不为了宣称 multimodal 而预先批准 OCR、object detection、transcription、EXIF、perceptual hash 或新 + indexes。先区分 evidence authority、matching/ranking mechanism、query composition 与 result explanation。 +- **Representation-sensitive ownership**: retrieval ownership 取决于 evidence 当前在 info-base 中的表示，而不是其 + 现实世界分类。仍嵌在 PDF content / Resolver lexical projection 中的 filename、page count、MIME 等只能作为 + feature evidence 被发现；Organization 将有 use value 的 facts 拆成 Blocks/Relations 后，后续定位与探索由 + graph-navigation-retrieval 承担。Feature retrieval 不因此扩张为任意 schema-aware field-query engine。 +- **Current Truth**: core-py now owns Resolver lexical projection、`block_lexical_records`、bounded maintain/rebuild Jobs、 + exact local/delegated capability、multimodal faithful materialization、Alibaba Model Studio dialect and system-driven media + interpretation。client-web now owns the peer facade、InfoBaseListView and List-hosted Inspector/SolvedContent outlets。The + real NASA image/audio/video → PostgreSQL Storage → Resolver → provider → graph → lexical → Organization Agent journey passed； + core's final hermetic contract passes with 458 tests and 41 external skips；the exact merge `4b180467` passed preview、artifact + publication and production delivery。A fresh fork workflow migrated Neon from empty state，created exact Render services，passed + Core/PostgREST probes and the authenticated contract，then passed a measured Free cold wake while renewing its Peer lease。 +- **Verification Direction**: Acceptance 最终必须以真实 collected graph 和用户可判断的 finding tasks 证明新增能力 + 补足了 semantic/graph retrieval 不能合理完成的工作；不能以 operator 数量、schema round-trip 或人工构造的 query + fixture 代替产品价值。J1–J7 已冻结 exact identifier、中文片段、Mail attachment metadata、freshness、PDF body、 + multimodal textualization/interpretation 与 browser recall actors；具体合法媒体 asset provenance 在 execution + preflight 固定，不进入生产实现。 +- **Internal decomposition**: lexical retrieval 与 perceptual retrieval 都属于 feature retrieval；前者是必做的第一 + increment，后者后续按真实模态场景继续切分。Graph 中的 fact/relationship 定位由 graph-navigation-retrieval + 承担；hybrid recall 是各基础 retrieval 能力成立后的组合层，不反向模糊本 Unit 的边界。 +- **Multimodal lexical scope**: perceptual matching 延后不代表媒体文本化延后。Image OCR、audio speech transcription、 + video subtitles/transcription/on-screen text 等 faithful text derivations 必须能通过 Resolver materialization 变成 + ordinary text Blocks，并进入同一 lexical-record/query path；模型生成的描述/摘要由 system-driven Organization + 主动产生为 interpretation graph，再进入同一 lexical path，而不是被 `materialize_missing` 偷渡成 Resolver effect。 + Organization 通过 existing `get_solved_content()` 理解媒体，不引入平行的 Resolver media projection。忠实文本化按 + 信息角色分别形成 `text`、`transcript`、`subtitle` child Blocks，不合并为来源不明的聚合文本。 +- **Current Question**: none。Public JWT publication is deferred under D-340；D-341 keeps fork self-hosting separate from the + canonical demo。Core PR #45 and client-web PR #50 are merged，their exact preview/main checks are green，the real + fork/cold-start acceptance is complete，and client-web PR #75 restored the independent Pages controller's private-package + permission。Exact-main Client checks run `32024516290` and Pages delivery run `32024731957` both passed for + `17160ae5e9a49d89fa60d35cee86223f41972c0b`；`https://app.inkcre.dev/` returned HTTP 200 with the new static entry。 +- **Next Step**: return to implementable-unit selection。The completed native Extension release unit and its deferred + independent-token improvement do not reopen or redefine Feature retrieval。 +- **Delivery Gate**: closed under Sir's explicitly approved core-first sequence。Core exact-head preview proved fresh migration and + readiness before merge；the merge then passed artifact and production delivery；client exact-head CI/preview and manual J7 passed。 + Because the core PR preview had already been removed when J7 ran，that journey used the production-admitted core feature line。 + Acceptance records this execution-order substitution explicitly and does not claim a matching-preview session occurred。 + Client-web PR #50 later squash-merged as `9b5c870` and its main CI passed；source promotion is complete。Client-web PR #75 + then fixed the Pages controller at its permission boundary，and exact-main delivery plus the public app smoke passed。 +- **Decision Authority**: D-316—D-342；D-336—D-338 close Alibaba dialect、Storage transfer hints and AI ContentPart ownership， + D-339 fixes Render Free as the sleeping self-host profile，D-340 keeps signing authority private，and D-341/D-342 separate + self-host identity from canonical production plus manual black-box acceptance from automated regression checks。 + 后续决定继续追加到 program decision register，不在本 packet 重复。 + +## Working Proposal Navigation + +- [Product design](product-design.md)：user job、feature/semantic/graph boundary、plain-query matching and result meaning。 +- [Technical design](technical-design.md)：Resolver projection context、derived records、PostgreSQL execution、Peer and UI topology。 +- [Preflight](preflight.md)：observed code facts、rejected alternatives、failure branches and remaining review pressure。 +- [Acceptance](acceptance.md)：real producer corpus and seven end-to-end journeys。 +- [Implementation plan](implementation-plan.md)：approved-boundary execution order；not implementation authorization。 +- [Deployment readiness](deployment-readiness.md)：accepted GitHub-only Render + Neon self-host contract，implemented controller + and completed live-host/cold-start probes。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/preflight.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/preflight.md new file mode 100644 index 0000000..3fd8b50 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/preflight.md @@ -0,0 +1,190 @@ +# Lexical Retrieval Preflight + +## Observed Repository Facts + +- core-py semantic retrieval already proves use-owned projection maintenance、derived rows、freshness filtering、bounded exact + ranking、fixed Peer inbound and caller-local `route_to_peer`。 +- Python Resolver has `get_text()` and `get_label()`；direct content fallback is explicitly forbidden。RelationManager owns one + directed text projection。 +- client-web still declares and implements `getStrForEmbedding()` even though D-095/D-096 and core-py hard-cut that contract。 +- client-web can delegate semantic retrieval but has no retrieval UI；the start view contains only a console-log placeholder。 +- client-web's app-bound `InfoBaseRouter` currently projects every Block destination into GraphSurface even though the shared + InfoBase contract already defines future ListSurface as a peer navigation host。GraphSurface itself owns + `BlockInspectorPopup` and `SolvedContentPopup`。Lexical results are therefore the first concrete pressure to implement + InfoBaseListView and preserve the active host during route projection instead of navigating away from result context。 +- client-web already runs the shared Job worker but has no local semantic/lexical Handler；capability-specific `can_handle` + remains the correct eligibility boundary rather than disabling its global worker。 +- GraphSurface currently loads all Blocks/Relations and can focus only after it receives a known Block reference。 +- core media solved values already expose useful metadata，while their general `get_text()` correctly remains unsupported in the + absence of content extraction。Mail MIME-part resolver already demonstrates a bounded metadata text projection。 +- PostgreSQL supplies literal matching、`tsvector/tsquery`、cover-density ranking、headline extraction and preferred GIN text + indexes。`pg_trgm` supplies indexed `LIKE/ILIKE` and similarity as separable operations。PostgREST supports FTS filtering but + does not by itself own the required cross-signal ranking/evidence contract。 + +Primary references： + +- PostgreSQL text parsing/ranking/headline：https://www.postgresql.org/docs/current/textsearch-controls.html +- PostgreSQL preferred GIN/GiST indexes：https://www.postgresql.org/docs/current/textsearch-indexes.html +- PostgreSQL text-search limits：https://www.postgresql.org/docs/current/textsearch-limitations.html +- PostgreSQL `simple` dictionary：https://www.postgresql.org/docs/current/textsearch-dictionaries.html#TEXTSEARCH-SIMPLE-DICTIONARY +- PostgreSQL `pg_trgm`：https://www.postgresql.org/docs/current/pgtrgm.html +- PostgREST FTS filters：https://docs.postgrest.org/en/v14/references/api/tables_views.html#full-text-search + +The local SVC database provisioner was inspected but not repaired：its existing image build lacks +`release/database-contract/`。SVC cleaned the failed attempt。The installed `neonctl` shim is also broken by a missing global +module。Neither tool failure is evidence against the retrieval design，and neither side branch is expanded in this Unit。 + +The legacy local `.env` is not a current application runtime credential：it still names `DB_CONN_STRING` and the stored Neon +owner password now fails authentication，while current Settings require `DATABASE_URL` + `JWT_SECRET`。Implementation setup must +obtain/rebind a current development/preview database credential rather than silently treating this file as runtime truth。 + +The same file's DashScope key remains usable for a read-only OpenAI-compatible catalog probe。The current catalog exposes exact +candidates including `qwen-vl-ocr-latest`、`qwen-audio-3.0-asr-flash`、`qwen3-asr-flash-2026-02-10`、`qwen3-omni-flash`、 +`qwen3.5-omni-flash` and `qwen3.5-omni-plus`。Catalog visibility proves naming/access only；real content-part + Tool-call +journeys remain Acceptance evidence and no model ID becomes a code/config default。 + +On 2026-08-12，minimal streaming text requests with `modalities=["text"]` completed successfully through the current workspace +for `qwen3.5-omni-plus`、`qwen3.5-omni-flash` and `qwen3-omni-flash`。The earlier HTTP 403 policy gate is therefore resolved。 +Separate real image、audio and public-MP4 video calls with the accepted function Tool all returned a `submit_graph` ToolCall +through Alibaba's OpenAI Chat endpoint for each candidate Model。A generated valid H.264 MP4 sent as actual base64 bytes also +returned the ToolCall，so the production shape is not justified only by public-URL success。No Model ID becomes a code/config +default or automatic fallback。 + +The transport facts are now separated precisely。OpenAI's own Chat Completions contract defines `image_url` with URL/base64 data +and `input_audio` with base64 data + `wav|mp3`；it does not define `video_url`。Alibaba's OpenAI Chat surface adds `video`/ +`video_url` and permits audio URL/Data URL，alongside function calling for the relevant Omni family。The current repository +adapter is exactly `core.openai-compatible.v1` and uses `openai.AsyncOpenAI.chat.completions`；the SDK is only a client，not +evidence that Alibaba extensions belong to the generic protocol。Under D-157/D-158's accepted cross-capability dialect topology， +the current proposal is `core.alibaba-model-studio.v1` rather than a new capability/endpoint-prefixed `core.openai-chat.v1`。 +Native DashScope remains dominated until the documented Chat surface fails a required behavior。 + +The bytes spike exposed one real product bound：the 16,242,564-byte NASA MP4 expands past the endpoint's 20 MiB data-URI item +limit，whereas a bounded valid H.264 clip succeeds。Alibaba recommends small original inputs for base64 and provides temporary +upload URLs only for development/testing with region、model and lifetime constraints。The review proposal is therefore a +conservative 7 MiB original-byte maximum per inline media part in this MVP。A larger part may use an optional Storage-owned +origin transfer URL when the exact dialect supports it；otherwise its exact derivation/interpretation is unavailable。Adding +Alibaba temporary upload or OSS staging here would add a provider-specific media lifecycle and contradict the earlier decision +to defer S3/object storage until real storage pressure。 + +The multimodal corpus is now concrete：NASA's official asset API returns MP4 + authored VTT for +`GSFC_20140121_GPM_m11457_Dave_McComas`；the small MP4 was observed as `video/mp4` with ETag +`6a03adc2cf2e56872d639459541c7533-2`，and its captions contain meaningful flight-software requirements/test language。A bounded +acceptance setup can derive audio/frame and remux subtitles outside production code。NASA's current media guideline says its +content is generally available for educational/informational use while protecting insignia/logotype/identifiers；the acceptance +bundle retains provenance and does not present NASA endorsement。 + +Current branch delivery is not a clean implementation baseline。After fetching latest heads on 2026-08-12，core-py's +`feat/synchronized-client-v3-restacked` is 6 commits behind / 17 ahead of `origin/main`，and client-web's +`feat/synchronized-core-v3` is 14 behind / 6 ahead。A merge-tree preview exposes overlapping changes in both repos，so I0 must +reconcile the branches before implementation instead of leaving conflict resolution to preview delivery。 + +The current core PR #45 preview failure is an orchestration precondition failure，not a migration failure：`preview-verify` +waited for required exact-head checks that never became green and delivery never reached database initialization。The checked-in +preview workflow later creates/resolves a data-free Neon branch，runs `db init --environment preview` and database-contract +verification，deploys exact-head images and probes `/readyz`。That makes a green core preview an appropriate final proof for the +large migration。client-web PR #50 builds its own static Cloudflare preview but CI normally resolves a stable core release，so +the final cross-repo J7 must explicitly bind the built client to PR #45's live core preview。 + +Primary protocol references： + +- Alibaba OpenAI Chat：https://www.alibabacloud.com/help/en/model-studio/qwen-api-via-openai-chat-completions +- Alibaba Responses：https://www.alibabacloud.com/help/en/model-studio/qwen-api-via-openai-responses +- Alibaba API interface comparison：https://www.alibabacloud.com/help/en/model-studio/qwen-api-reference/ +- Alibaba temporary upload：https://www.alibabacloud.com/help/en/model-studio/get-temporary-file-url +- Alibaba error/data-size guidance：https://www.alibabacloud.com/help/en/model-studio/error-code +- OpenAI Chat Completions create contract：https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create +- OpenAI image input guide：https://developers.openai.com/api/docs/guides/images-vision +- OpenAI audio input guide：https://developers.openai.com/api/docs/guides/audio + +## Alternatives Rejected Before Human Review + +| Alternative | Why dominated | +| --- | --- | +| Search `blocks.content` directly | treats storage pointers/source JSON as content，bypasses Resolver，misses solved metadata | +| Resolve every Block during each query | O(N) hydration/network/CPU per read，no scalable entry path，evidence changes during ranking | +| Put ranking in a PostgREST RPC/computed business function | moves domain behavior into the database and repeats the rejected semantic-retrieval topology | +| External Tantivy/Elasticsearch service | adds operational authority、peer-local persistence and synchronization before PostgreSQL is insufficient | +| One universal Feature table/profile/manager for lexical and perceptual | no shared representation or matching lifecycle has been proved；would become a god abstraction | +| Language stemmer/profile table in V1 | multiple selectable profiles lack current identity/reuse value；stemming damages identifiers and does not solve CJK | +| Trigram fuzzy search by default | introduces a second approximate semantics and opaque threshold when literal+terms already solve the target job | +| Generic solved-content scalar serialization | leaks Resolver schemas、indexes bytes/noise and removes Resolver's judgment over meaningful features | +| Index Relations in lexical V1 | overlaps graph-navigation ownership，creates endpoint-label result floods and lacks a clean current UI destination | + +## Failure-Branch Preview + +| Branch | Proposed behavior | +| --- | --- | +| empty/whitespace query | Pydantic/Zod validation failure | +| punctuation-only query | literal path remains valid；empty `tsquery` is simply omitted | +| Chinese/no-space fragment | escaped literal substring path；no claim of linguistic segmentation | +| unknown Resolver | maintenance diagnostic；other Blocks continue | +| unsupported default text but available lexical metadata | lexical context produces a record without unnecessary materialization | +| lexical text requires an absent derivation | maintenance permits Resolver-owned materialization；query remains read-only | +| no lexical text beyond generic label | label-only record remains searchable | +| record missing/stale | excluded from retrieval；query does not repair it | +| storage/Relation changes without Block timestamp | known best-effort freshness gap；explicit rebuild，no speculative dependency graph | +| projection exceeds PostgreSQL engine bound | diagnostic/unavailable，no silent truncation | +| concurrent maintainers | equivalent upsert；possible duplicate work accepted | +| Resolver materializes a child after the current scan cursor | scan may index it later in the same invocation；otherwise the next Job does；`max_records` prevents unbounded self-extension | +| concurrent missing materialization creates equivalent child Blocks | singular graph reads may use any one；distinct Blocks may both be retrieved；no retrieval-owned dedupe or stability promise | +| no capable Peer from browser | existing capability-unavailable behavior | +| post-dispatch unknown outcome | existing Peer outcome-unknown behavior；read operation is not replayed by generic infrastructure | +| literal text contains `%`/`_` | manager escapes wildcard characters before `ILIKE` | +| excerpt contains authored markup | lexical projection contract is plain text；UI renders interpolation，never `v-html` | +| no locally executable media Agent | parameterless Job remains pending and claimable after configuration/runtime changes | +| only some modality Agents are executable | one Peer may claim and process that subset；other candidates remain missing for a later independent Job | +| provider rejects a statically valid media call | candidate diagnostic，continue bounded scan，no fallback or claim rollback | +| streamed multimodal ToolCall deltas are malformed/incomplete | dialect raises output-contract failure；Agent never receives a partial AssistantMessage | +| inline media exceeds the exact dialect's accepted byte bound | use an accepted Storage transfer URL if present；otherwise unavailable；no hidden provider upload、retry or truncation | + +## Remaining Review Pressure + +The proposal is intentionally opinionated。D-320—D-322 now close primitive matching、derived records、permitted missing +materialization and Block-local/non-recursive lexical projection。 + +D-323/D-324 reopen and close two missing high-level boundaries：multimodal textualization remains in lexical scope，and lexical/ +semantic maintenance use the database-owned Cron → typed Job → one capable Peer claim topology。 + +System-driven media Organization is closed at the topology level。Repository evidence shows image/audio/video solved values +already carry hydrated bytes and typed metadata，so Organization can consume `get_solved_content()` without another Resolver +projection。The AI schema stores modalities while canonical Chat messages and `AIManager.chat()` hard-code text input；the +remaining adapter work belongs to the AI call boundary。D-327—D-331 close missing-only candidates、independent Agent ownership、 +per-modality routing、separate information-role text children and the parameterless convergence Job。The solved-content → Agent +seam has one coherent shape：Organization +builds an AI-owned UserMessage with textual graph context plus actual media bytes/MIME；AIManager validates modalities and the +dialect alone encodes provider content items。Block/Storage references、provider URLs and Resolver-owned AI projections are all +dominated by their cross-domain coupling。 + +Faithful-text selection likewise derives from existing ownership：image/audio/video Resolvers each own their exact config，with +Model references scoped to `text` or `transcript` roles；source-native subtitles need no AI config。A shared media config would +invent an owner，while Agent reuse would duplicate Resolver prompt/write authority。Provider/model/payload support is now proved +for the exact acceptance shape；the remaining review pressure is whether bounded inline media is an acceptable product limitation， +not another configuration topology question。 + +The implementation address audit found no hidden competing path：core-py has one legacy semantic maintenance interval in +`run.py`；client-web's Resolver base still requires `getStrForEmbedding()`；its core image/audio/video/PDF Resolvers currently +stop at metadata/unsupported text；and the start view's submit handler only logs the query。These are direct I1/I4/I6 cutover +surfaces，not new product decisions。 + +Job eligibility needs one shared implementation seam but no new scheduling semantics。Current `JobManager` calls synchronous +`can_handle` before atomic claim；there is no public AIModel/Agent executability predicate yet。AIManager can own static +Model/Provider/dialect/config/modality checks and AgentManager can compose Agent/Tool checks。Neither may call the provider or +promise key/quota health；the resolved earlier Omni 403 remains evidence that static eligibility cannot guarantee execution。 + +The parameterless media Job does not imply all-or-nothing fleet convergence。Requiring every configured modality Agent at +claim time would make one deliberately unavailable slot block useful work，while three Job types would move candidate-local +modality back into Cron templates。The accepted best-effort boundary is therefore: at least one local modality makes the Peer +eligible，successful Relations are progress，and unsupported candidates remain discoverable by the next independent Job。 + +Primary provider evidence confirms that this is implementable through an exact Alibaba OpenAI Chat dialect rather than a +hypothetical API。Because the canonical capability returns one complete text/ToolCall AssistantMessage，that adapter can stream +and internally assemble the same output；no provider streaming state crosses the dialect boundary。Provider limits remain +adapter/provider outcomes rather than lexical query semantics。 + +Primary corpus/provenance evidence： + +- NASA asset API：https://images-api.nasa.gov/asset/GSFC_20140121_GPM_m11457_Dave_McComas +- NASA media-usage guidelines：https://www.nasa.gov/nasa-brand-center/images-and-media/ + +D-325/D-326 require active model-authored interpretation through Organization and define “active” as non-user-driven。This is a +new automatic media-interpretation approach rather than a reason to route media through current text-only focal rumination。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/product-design.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/product-design.md new file mode 100644 index 0000000..d6d9078 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/product-design.md @@ -0,0 +1,140 @@ +# Lexical Retrieval Product Design（Preflight-refined Review Contract） + +## Product Job + +Lexical retrieval serves the case where a person or Agent remembers an explicit textual clue but does not know which +Block contains it or where that Block sits in the graph。The clue may be prose、a phrase、a name、filename、URL、identifier、 +error token or a textual projection of media metadata。The operation locates existing information；it does not answer a +question、interpret synonyms or start graph traversal on the caller's behalf。 + +The source modality is not the boundary。A Mail MIME-part filename、PDF title/page count or audio codec can be lexical +evidence even though the underlying Block is not a text Block。OCR/transcription output can also become lexical evidence once +another owner produces it。Pixel/audio similarity remains perceptual retrieval；cross-modal meaning similarity remains +semantic retrieval。 + +## Retrieval Family Boundary + +| Need | Owner | +| --- | --- | +| Match explicit textual features projected from one Block | lexical increment of `feature-retrieval` | +| Match visual/audio/video perceptual features | later perceptual increment of `feature-retrieval` | +| Match meaning or paraphrase | semantic retrieval | +| Locate/explore graph-visible dynamic properties and paths | graph-navigation retrieval | +| Combine several primitive retrieval modes | later hybrid composition | + +Ownership follows current representation。A PDF page count embedded in Resolver-understood content may participate in a +lexical projection；after Organization externalizes that fact as Blocks/Relations，graph-navigation can locate it。The +lexical projection remains rebuildable support and does not become a second information authority。 + +## MVP Query Semantics + +One request contains one non-empty plain query and a result limit from 1 through 20。It has no filters、query-language mode、 +language/profile selection、threshold or pagination。 + +The fixed V1 matcher uses two explainable signals： + +1. case-insensitive literal occurrence of the complete query in the Resolver label or lexical text； +2. all lower-cased lexical terms from PostgreSQL's `simple` text configuration occurring in the projected record。 + +Literal matches rank before terms-only matches；within those classes，label evidence is stronger than body evidence and +cover-density rank orders the remaining candidates。Stable Block ID is the final tie-breaker。This supports exact phrases、 +punctuation-heavy identifiers and Chinese/no-space fragments without promising stemming、synonyms or spelling correction。 +`pg_trgm` may accelerate literal `ILIKE` matching，but trigram similarity/fuzzy matching is not part of V1 behavior。 + +The plain query deliberately does not expose PostgreSQL `tsquery` or web-search operators。An actor asking for semantic +variation should use semantic retrieval；an actor needing graph facts should use graph-navigation。Hybrid syntax is deferred +until the primitive modes have real use evidence。 + +Lexical indexing may request missing Resolver materialization。This permits retrieval support to drive organization，例如 an +image Resolver may create an OCR text child before returning its lexical projection。The permission does not make every read a +write：a Resolver that already has sufficient filename/MIME/text evidence returns it without creating anything，and ranked +retrieval itself never runs maintenance。 + +Metadata is additive，not a substitute for document body recall。For a PDF with an available text layer，its body remains +searchable。If an existing or newly materialized semantic-content child owns that text，the child Block is indexed independently +and the root PDF record does not recursively duplicate the body；otherwise the exact Resolver may include Block-local available +body text in the root lexical projection。Scanned pages require an available OCR path，and encrypted/unsupported content may +remain metadata-only without pretending that the document was fully projected。 + +## Multimodal Textualization + +Perceptual retrieval is not a prerequisite for recalling language carried by non-text media。The lexical increment includes +faithful modality-to-text paths：visible written text from images、spoken text from audio、and subtitle/spoken/on-screen text from +video。An exact Resolver may materialize these as ordinary `core.text.v1` Blocks；the resulting Blocks then use the same lexical +maintenance and query contract as authored text。 + +The product must distinguish faithful textualization from model-authored interpretation： + +- OCR、speech transcription and source-native subtitle/caption extraction aim to preserve language actually present in the + source，even though their derived text may contain recognition errors； +- a generated scene description、object list or summary adds an interpretation that was not literally authored in the source。 + +Both classes must be produced and become searchable in this increment，but through different lifecycles。Faithful textualization +is Resolver-owned missing materialization；description、summary and other model-authored interpretation are Organization-owned +effects。Lexical maintenance triggers only the former，then independently indexes the output of either lifecycle。The lexical +result claims only that the query matched the derived text Block；it does not claim perfect recognition or direct byte-level +proof of the original media。 + +Faithful signals remain separate graph children rather than being flattened into one aggregate text：visual written language is +linked as `text`，spoken language as `transcript`，and source-native subtitles as `subtitle`。A video may therefore have several +text children when it genuinely carries several kinds of language。These predicates describe the information role，not whether +OCR、ASR or a particular provider produced it。Each child is independently searchable；the parent does not recursively copy its +body，and the retrieval layer does not silently aggregate or deduplicate the signals。 + +Media interpretation is system-driven rather than gated by a per-Block user request。It is a distinct Organization approach +that selects media candidates，consumes their Resolver solved content plus bounded graph context and persists additive +`interpretation` graphs。This supersedes D-184's automatic-execution exclusion only for media interpretation；ordinary focal +rumination remains explicit。The approach is still independent of lexical query/maintenance and collection success。 + +The MVP automatic selector is missing-only：an image/audio/video Block with the approach's outgoing `interpretation` Relation is +not selected again。That Relation is presence evidence，not freshness authority；the system does not claim the interpretation is +current or optimal and does not automatically recompute it。A candidate that fails or produces no graph remains missing and may +be reconsidered by a later scheduled Job without introducing an attempt ledger。 + +Media interpretation uses a dedicated reusable AgentDefinition rather than the explicit rumination Agent。Its prompt can focus +on scene/content explanation and summary while retaining the existing validated graph-submission behavior。Deployments may bind +both Agents to the same model，but changing one approach's prompt/tools/budget does not silently redefine the other。 + +The approach routes by source modality to independent image、audio and video Agents。This permits modality-appropriate models、 +prompts and budgets；a deployment may still reuse one Agent ID in several slots when one model is genuinely suitable。No +cross-modality Agent fallback is implicit。 + +## Result Contract + +V1 returns at most 20 existing Blocks in authoritative order。Each match includes： + +- the actual persisted Block； +- its Resolver-qualified label from the indexed snapshot； +- one bounded plain-text excerpt； +- an exact evidence kind：`label_exact`、`label_substring`、`text_substring` or `terms`； +- the within-class lexical rank used for deterministic ordering。 + +The result does not return Relation matches in V1。Current Relation content is a directed dynamic property，and finding those +facts belongs to graph-navigation。This avoids returning an edge that the current UI cannot route to and avoids flooding a +lexical query with every Relation whose endpoint label contains the term。A later concrete case involving material authored +text on Relations may reopen this boundary。 + +The excerpt is display text，never trusted HTML。A match explains why it appeared without requiring the caller to hydrate or +re-run a Resolver。Numeric rank is ordering detail only，not a cross-query probability or stable threshold。 + +## User Surface + +The existing client-web start-page input becomes a lexical info-base lookup，not Chat InKCre and not a semantic question box。 +Submitting updates the URL query so browser Back restores the result set。That page becomes the first real +`InfoBaseListView`：the lexical result list is its persistent base surface，and its route-destination outlet owns the existing +`BlockInspectorPopup` and `SolvedContentPopup` just as GraphSurface owns its own outlet。Selecting a result therefore opens the +Block inside the current List view；it does not discard the result context by navigating to GraphSurface。 + +`InfoBaseRouter` remains one app-bound、stateless Block/Relation route-to-UI-state mapping。The client-web adapter projects the +same domain `overview`、`block` and `solved-content` routes into whichever InfoBaseView currently hosts navigation。Vue Router +and browser history remain the sole history authority，including the lexical query。This does not create a second router、a +List-owned domain route vocabulary or a Mail/RSS/Memos-specific search page。 + +## Explicit Non-Goals + +- fuzzy spelling、stemming/language dictionaries、synonym expansion or query suggestions； +- semantic/perceptual score fusion、graph predicates or arbitrary JSON field filtering； +- Relations、pagination、Chat/RAG answers or automatic Organization approaches beyond the exact media-interpretation path； +- perceptual similarity or direct object matching； +- claiming OCR/transcription is source authority or error-free； +- claiming that every fact present inside binary/structured content is already lexically projected。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/technical-design.md b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/technical-design.md new file mode 100644 index 0000000..aec173d --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/feature-retrieval/technical-design.md @@ -0,0 +1,365 @@ +# Lexical Retrieval Technical Design（Preflight-refined Review Contract） + +## Topology + +```text +LexicalRetrievalManager.maintain/rebuild + -> Block + exact Resolver lexical projection + -> optional Resolver-owned graph materialization + -> block_lexical_records (Lexical Retrieval-owned derived state) + +Organization media-interpretation Job + -> interpretation graph only + -> [later independent lexical maintain/rebuild] + -> block_lexical_records + +LexicalRetrievalManager.retrieve_local + -> PostgreSQL literal + FTS over fresh block_lexical_records + -> existing Block + lexical evidence + +client-web LexicalRetrievalManager.retrieve + -> PeerManager.delegate(core.feature_retrieval.lexical.v1) + -> core-py fixed inbound + -> retrieve_local +``` + +`block_lexical_records` has exactly one lifecycle owner：Lexical Retrieval。Only +`LexicalRetrievalManager.maintain/rebuild` creates or updates those rows。Resolver materialization and Organization +interpretation may change the authoritative info-base graph，but neither writes lexical records、invokes lexical maintenance or +receives a callback into retrieval。Their new/changed Blocks become candidates only when an independently invoked maintenance +Job later scans current graph state。 + +`feature-retrieval` is the Unit，not a mandatory god class。The Python application module may contain a deep +`LexicalRetrievalManager` because projection maintenance、ranking、evidence and delegation form one lifecycle。Future +perceptual retrieval receives its own manager/records/contracts unless implementation evidence proves a useful shared seam。 + +## Resolver Projection Evolution + +Direct `Block.content` search is forbidden：inline content may be source JSON，storage-backed content is a pointer，and both +bypass exact Resolver interpretation。 + +The existing general `get_text()` remains the sole text projection method，but gains a stable optional `context` parameter。 +The default context preserves current semantic/organization behavior；`context="lexical"` asks for a plain-text projection +optimized for explicit feature recall。Context describes the requested representation，not a query、tokenizer、ranking profile、 +AI model or permission to mutate。This is the evidence-led context evolution anticipated by D-096，not a resurrection of +`get_str_for_embedding()` or a model/provider dependency。 + +The lexical context is Block-local and non-recursive。A Resolver may inspect the focal Block's own canonical/solved content and +metadata needed to express it，but it does not copy lexical text owned by adjacent Blocks merely because a Relation makes that +text reachable。Those Blocks receive their own records。This prevents parent/semantic-child duplication without turning the +retrieval manager into a graph-aware deduplicator。 + +- Text-like Resolvers normally return the same value in both contexts。 +- Media/structured Resolvers may remain unsupported in the default context while formatting bounded lexical metadata in the + lexical context，例如 PDF title/author/page count/media type or image format/dimensions。 +- Resolver owns field selection and human-readable serialization；the retrieval owner must not recursively serialize arbitrary + solved-content fields。 +- The projection is plain text and may use field labels；it is not canonical content、a graph write or UI markup。 +- Metadata projection must not displace available document body text。When semantic body text is represented by child Blocks， + those Blocks are independently maintained and the root lexical projection must not recursively copy their complete text；when + no such graph representation exists，the exact Resolver may project Block-local available body on the root record。 +- `materialize_missing=true` independently permits the Resolver to create a derivation required by the lexical projection。 + The Resolver may still return existing metadata without mutation；when it performs OCR or another graph-producing derivation， + that exact Resolver owns the write。It is a retrieval-triggered Resolver materialization，not an Organization-owned approach； + feature retrieval still does not construct the graph itself。 + +Both Python and TypeScript Resolver contracts adopt the same context vocabulary。client-web's remaining +`getStrForEmbedding()` methods are a stale contract regression and are hard-cut in this increment；semantic embedding continues +to use default `getText()`。 + +The current `core.pdf.v1` is an observed implementation gap：it inspects metadata through `pypdf` but declares text projection +unsupported。The increment must add bounded PDF body projection/materialization using a mature parser path；the metadata-only +example is not sufficient completion evidence。 + +## Multimodal Text Materialization（Proposal） + +```text +image/audio/video Block + -> exact Resolver get_text(context="lexical", materialize_missing=true) + -> configured provider-neutral AI text-capable path or source-native extractor + -> Resolver-owned derived core.text.v1 Block + exact relation + -> later/current bounded lexical maintenance scan + -> child block_lexical_records row +``` + +The parent projection remains Block-local：after creating or finding the child，it returns only parent-owned metadata rather +than copying the derived body。The maintenance scan reaches the new child by ordinary Block ID paging；bounded runs may index it +in the same invocation or the next Job without a Resolver→retrieval callback。 + +The scan does not freeze a starting maximum Block ID。A child created after the current cursor is therefore ordinary later work +in the same scan when the invocation still has capacity；otherwise a later Job sees it。`max_records` remains the hard bound，so +a faulty graph-producing Resolver cannot make one invocation chase its own writes without limit。This is deliberately simpler +than a materialization callback or a second derived-work queue。 + +The AI schema already models model `input_modalities` and `output_modalities`，but current canonical Chat messages and +`AIManager.chat()` hard-code text input。The AI boundary therefore extends only `UserMessage` with a discriminated content-part +union： + +```text +TextContentPart {type: "text", text: str} +ImageContentPart {type: "image", data: bytes, media_type: str} +AudioContentPart {type: "audio", data: bytes, media_type: str} +VideoContentPart {type: "video", data: bytes, media_type: str} + +UserMessage.content: non-empty tuple[UserContentPart, ...] +``` + +Bytes are the provider-neutral actual input；`media_type` is the standard content description needed for exact wire encoding。 +Content parts contain neither Block/Storage references nor provider URLs。Organization converts a core media Resolver's typed +solved value into the initial multimodal UserMessage；a faithful-extraction Resolver constructs the same AI input directly。 +AIManager remains graph-blind and derives the complete requested input-modality set from Message history，then verifies both the +persisted model declaration and peer-local dialect implementation before execution。No redundant `multimodal` feature flag is +added because modalities already own that fact。 + +The exact dialect adapter owns wire translation。The existing `core.openai-compatible.v1` remains the generic OpenAI SDK-backed +multi-capability dialect。OpenAI Chat Completions itself defines `image_url` and base64 `input_audio` content parts，so those +standard shapes belong in the generic adapter when the selected Model declares the modalities。OpenAI Chat does not define +`video_url`。Alibaba adds `video`/`video_url` and widens audio data to URL or Base64 Data URL，so the same SDK's ability to send +that JSON does not make those fields generic OpenAI protocol facts。 + +The reviewed proposal adds `core.alibaba-model-studio.v1` as the exact Alibaba dialect。It may reuse internal OpenAI +message/Tool translation helpers and implement every supported capability exposed by that provider family，but owns the +Alibaba-specific video/audio-URL shapes、Omni streaming assembly and provider input bounds。It is deliberately not named +`core.openai-chat.v1` or `core.alibaba-model-studio-openai-chat.v1`：D-157/D-158 already establish that `chat` is one canonical +AI capability and that an AIDialect may implement it through Chat Completions、Responses or native APIs while spanning other +capabilities。Changing to endpoint-scoped dialect identities would be an explicit topology reversal，not a naming cleanup。 + +OpenAI Responses is not this dialect's transport：the current Alibaba Responses surface does not support audio/video input，so +choosing it would make the accepted multimodal scope impossible。Native DashScope is also not added merely for theoretical +completeness because the documented OpenAI Chat surface passed the exact image、audio、video + function-Tool wire journeys。 + +For bounded media，the exact dialect maps actual bytes to base64 data URLs/data and its supported content items；the canonical +schema preserves none of those protocol names。The Alibaba dialect uses `stream=true` with text-only output modalities +for current Omni calls and assembles content + incremental ToolCall deltas into the same complete `AssistantMessage` contract。 +Streaming is dialect-internal execution，not an Agent/Thread state or a second public result type。System/Assistant text and +ToolResult JSON retain their existing forms。Thread continues to persist one canonical Message history；the current in-memory +backend holds bytes as part of the UserMessage，while a future persistence backend may optimize physical storage internally +without exposing InfoBase references or creating a separate media-input lifecycle。 + +Dialect support is an executable local fact in addition to the persisted AI Model declaration。`AIDialectAdapter` therefore +exposes a static input-modality support predicate used by AIManager's ordinary capability check；no dialect-capability table or +remote provider probe is introduced。The Alibaba dialect supports the exact set proved by its implementation/tests；the generic +dialect does not inherit Alibaba-specific video support。 + +An HTTP Storage pointer is not automatically content authority or a provider-reachable URL。The ordinary Resolver path still +hydrates actual bytes lazily and produces solved content without copying those bytes into PostgreSQL binary Storage。As a +transport optimization，Storage may optionally expose `get_transfer_url(pointer) -> str | None`；HTTP Storage may return its +validated origin URL while PostgreSQL binary Storage returns `None`，and a future object Storage may return a scoped URL。This +method describes an available transfer representation，not a promise that a third-party backend can fetch it。 + +The canonical media content part continues to carry actual bytes + MIME and may additionally carry an optional origin +`transfer_url` hint。It carries neither a Block/Storage reference nor a provider-upload URL。The exact dialect uses a shallow +MVP ladder：prefer inline bytes within its documented stable bound；only when inline transfer is unavailable/oversized may it use +an accepted transfer URL；otherwise report capability unavailable。It does not retry a possibly charged model call after remote +fetch failure，does not silently truncate/transcode media and does not stage objects。Thus small Twitter images remain reliable +through lazy hydration even when their durable Storage is HTTP，while a large public video can receive best-effort direct transfer +without pretending hotlink-protected URLs are reliable。 + +This shape is not promoted into a Resolver projection。Resolver-owned faithful extraction consumes its own solved content； +AIManager remains graph-blind。 + +Faithful extraction uses direct AI Model references，not Agents：the Resolver already owns the extraction prompt、the exact +derived role and the text-Block graph write，so Agent prompt/Tool/Turn lifecycle would duplicate that authority。Configuration +follows the exact behavior owner rather than creating one cross-Resolver “media” owner： + +```text +key: core.resolver.image +schema: core.resolver.image.config.v1 +value: {text_model: int} + +key: core.resolver.audio +schema: core.resolver.audio.config.v1 +value: {transcript_model: int} + +key: core.resolver.video +schema: core.resolver.video.config.v1 +value: {text_model: int, transcript_model: int} +``` + +The same AI Model may fill several fields。Source-native video subtitle extraction needs no Model reference。Each field is a +use-time reference：one dangling、disabled or modality-incapable Model makes only that exact missing derivation unavailable and +does not authorize AIManager to choose/fail over to another Model。Prompts remain code-owned Resolver behavior rather than +deployment config。Provider-backed corpus/model IDs remain acceptance-environment facts，not schema defaults。 + +Faithful materialization writes one `core.text.v1` child per distinct signal： + +```text +image --text-------> text Block +audio --transcript-> text Block +video --subtitle--> text Block +video --transcript-> text Block +video --text-------> text Block +``` + +The predicates express the child's role relative to the media Block；extractor technology and provider identity remain outside +the graph vocabulary。A generated description/summary instead enters the separate `interpretation` graph path。Existing-child +reuse is therefore checked per exact relation role，not against one ambiguous “any derived text” marker。For video，subtitle、 +transcript and visible-text attempts are independent：one unavailable role does not prevent existing/source-native/other-role +children from being returned or created，and the shallow Resolver completion contract does not expose created/existing details。 + +Model-authored interpretation uses a distinct path： + +```text +system-driven Organization media-interpretation approach + -> select candidate media Block + -> Resolver.get_solved_content() + bounded graph context + -> configured multimodal model/Agent + -> additive Graph command + -> additive interpretation Block/Relation + -> ordinary lexical maintenance indexes the new text Block +``` + +`materialize_missing` does not enter this path。Organization consumes the existing typed solved value instead of inspecting +`Block.content` or asking `get_text()` to disguise media as text。It builds one initial UserMessage containing a bounded textual +focal/direct-relation context plus exactly the focal media content part。Agent/model selection remains deployment-owned and exact +graph production remains behind the existing Agent Tool boundary。 + +The automatic command is the exact typed Job `core.organization.media_interpretation.v1` with `{}` parameters，normally created +by one Cron template。At execution time Organization selects deterministic bounded pages across `core.image/audio/video.v1` +Blocks with no outgoing `interpretation` Relation，then routes each candidate through its modality Agent。One candidate failure/ +no-output is diagnostic and does not stop later candidates；successful Relations make those Blocks absent from subsequent +missing-only scans。Job state may expose bounded per-modality counts/diagnostics，but there is no cursor、checkpoint、attempt/ +freshness table、automatic retry/recompute or coupling to lexical Job state。 + +Because the Job parameters are intentionally empty，the candidate and diagnostic limits are implementation-owned bounded +policy，not a hidden dynamic schedule payload。A Peer is statically eligible when at least one configured modality Agent can run +locally。It processes every capable candidate and leaves unsupported-modality candidates missing for a later independent Job。 +The Job does not claim that every currently missing medium converged，and it does not add a partial-progress cursor to simulate +that stronger promise。 + +The approach selects an independent persisted AgentDefinition through its own deployment config。AgentManager remains the owner +of model calls、message history and validated Tool execution；Organization owns candidate/context assembly and completion meaning。 +The Agent may share a model or Tool IDs with rumination，but not its config identity or prompt lifecycle。A direct free-text +`AIManager.chat()` response parsed and persisted by Organization is rejected in favor of the existing graph Tool boundary。 + +Proposed exact config key/schema： + +```text +key: core.organization.media_interpretation +schema: core.organization.media_interpretation.config.v1 +value: {image_agent: int, audio_agent: int, video_agent: int} +``` + +Organization selects one reference from the focal media modality before `AgentManager.run()`。No `_id` suffix is used；the +field names describe Agent references rather than database column implementation。All fields are required，may hold the same +value，and are resolved defensively when used rather than reverse-restricting Agent deletion。 + +## Persisted Derived Record + +One Block has at most one `block_lexical_records` row： + +```text +block integer PK/FK -> blocks.id ON DELETE CASCADE +label text NOT NULL +text text NULL +search_vector tsvector NOT NULL +created_at timestamptz NOT NULL +updated_at timestamptz NOT NULL +``` + +The Block reference is the record identity；there is no independent sequence。No `LexicalProfile` or deployment config is +introduced because V1 has one fixed strategy and no independent selection/lifecycle/reuse pressure。The exact capability ID +and migration own the strategy version。 + +`label` is the concise、Resolver-qualified Block reference used for display and high-value recall。`text` is the fuller lexical +projection and may be null；it need not repeat the label。Both are stored so result evidence agrees with the exact indexed +snapshot。`search_vector` is a PostgreSQL `tsvector` derived from both：label lexemes receive weight A and text lexemes weight D， +with positions retained under the `simple` configuration。It is not an embedding/vector-space value and is never returned as +information authority。A GIN +index supports `@@` term matching。`pg_trgm` GIN indexes on label/text accelerate escaped literal `ILIKE` searches；the extension +is used as an index mechanism only，not as authorization for fuzzy product behavior。 + +The row stores the projection text because retrieval evidence must agree with the indexed snapshot and must not call a Resolver +or external Storage while answering。The duplication is rebuildable application support，not information authority。 + +## Maintenance And Freshness + +`maintain()` scans deterministic Block-ID pages，projects outside a transaction，and upserts bounded complete batches in short +transactions。Unknown Resolver/projection/size failures become bounded diagnostics and do not stop the scan。`rebuild()` uses +its invocation timestamp as a cutoff，matching the proven semantic-retrieval pattern。 + +A record is queryable only when `record.updated_at >= block.updated_at`。The known limit remains explicit：externally mutable +storage bytes or Resolver projections depending on changed Relations do not necessarily advance the focal Block timestamp。 +No trigger cascade、dependency graph、content hash or dirty queue is added before measured harm。Explicit rebuild is the current +recovery path。 + +Lexical maintain and rebuild are exact typed Jobs。A deployment may create them explicitly or use the existing Cron table as a +recurring Job template；one capable Peer claims each Job before a thin Handler calls `LexicalRetrievalManager`。Successful rows +are natural resumable progress，so no separate checkpoint/retry lifecycle is needed。Retrieval never runs hidden maintenance。 + +Semantic maintain/rebuild migrate to the same topology。The legacy `run.py` direct interval call to +`SemanticRetrievalManager.maintain_default()` is removed rather than retained as a second scheduling authority。 + +The exact Job contracts are： + +```text +core.feature_retrieval.lexical.maintain.v1 +core.feature_retrieval.lexical.rebuild.v1 + parameters: {options: LexicalMaintenanceOptions = defaults} + +core.semantic_retrieval.maintain.v1 +core.semantic_retrieval.rebuild.v1 + parameters: {profile: int | null = null, + options: EmbeddingMaintenanceOptions = defaults} +``` + +`profile` is a true creator-selected semantic vector-space reference；`null` deliberately resolves the deployment default at +execution。The bounded options snapshot work/cost policy for one command and may be reused by a Cron template。Candidate IDs、 +scan cursors and the current missing/stale set are manager-derived execution facts and never enter Job parameters。Handlers +project their bounded reports into `job.state` and do not duplicate domain logic。Maintain/rebuild remain separate exact intents； +Cron does not judge whether recurring rebuild is useful。 + +Pre-claim `can_handle` is static/best-effort eligibility，not a provider health probe。AIManager supplies a deep predicate over +persisted Model capability + enabled Provider/Model + peer-local dialect registration/config；AgentManager adds Agent existence、 +bound Tool availability and requested input modalities。Semantic Jobs use the selected Profile's Model；media interpretation is +eligible when at least one configured modality Agent is locally executable，then records per-candidate failures for the rest。 +Remote API policy/quota can still change after the check；that ordinary TOCTOU outcome belongs to Job execution and does not +introduce a preflight network request、fallback or claim rollback。 + +An execution-time provider rejection is therefore a candidate diagnostic rather than a model-selection signal。The Handler may +finish after bounded candidate failures；the graph remains the progress authority，and only successful `interpretation` Relations +remove candidates from later Jobs。If no configured modality is statically executable，`can_handle` is false and the Job stays +pending until some capable Peer/config appears。 + +PostgreSQL's documented `tsvector`/position limits are treated as projection-unavailable diagnostics in V1；the implementation +does not silently truncate content or invent durable Block segments。Organization breakdown remains the route for persistently +smaller information units。Internal derived segmentation is deferred until a real accepted corpus exceeds the engine bound。 + +## Matching And Ranking Execution + +`retrieve_local()` builds an escaped case-insensitive literal predicate plus `plainto_tsquery('simple', query)` when that query +contains lexemes。It selects only fresh records and assigns the best evidence class per Block： + +1. normalized label equals the complete query； +2. label contains the complete query； +3. lexical text contains the complete query； +4. weighted vector contains every parsed query term。 + +The query orders by evidence class，then `ts_rank_cd` over the weighted vector，then Block ID。It calculates a bounded plain +excerpt only for the final result set。No stored procedure/RPC owns this behavior：SQLAlchemy expresses the manager-owned query， +while PostgreSQL supplies indexed operators and ranking primitives。This differs from placing the business capability inside a +database function。 + +## Peer And API Boundary + +- exact capability：`core.feature_retrieval.lexical.v1`； +- fixed inbound：`POST /feature-retrieval/lexical`； +- business facade：`LexicalRetrievalManager.retrieve(request, route_to_peer=None)`； +- provider seam：`retrieve_local(request)`，which never delegates； +- `route_to_peer` remains caller-local policy and never enters the capability payload。 + +Core-py advertises the inbound after route/runtime readiness。The static browser implements only the typed facade and delegates +through existing Peer HTTP。Although PostgREST exposes basic FTS filters，direct browser execution cannot preserve the accepted +ranking/evidence contract without database computed/RPC business logic，so it is not a second implementation path。 + +## Client-Web Boundary + +`@inkcre/core` owns Zod request/result contracts and the delegating facade。The start view owns query state and result +presentation；it does not parse PostgreSQL syntax or hydrate results。The resulting `InfoBaseListView` is an InfoBase navigation +host：it keeps the result list as its persistent base surface and hosts its own `BlockInspectorPopup`/`SolvedContentPopup` +destination outlet。Result selection calls the app-bound `InfoBaseRouter` and stays within that List host。 + +The client-web adapter recognizes both List- and Graph-hosted URL projections of the same domain `InfoBaseRoute`。When pushing a +Block destination，it preserves the active host and current List query instead of hard-coding GraphSurface。The existing +GraphSurface remains the graph host and still loads the graph globally；that scaling pressure belongs to its own +InfoBase/graph-navigation work and is not imposed on List retrieval。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/github-extension/packet.md b/tasks/knowledge-lifecycle-capabilities/units/github-extension/packet.md new file mode 100644 index 0000000..c1f3b36 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/github-extension/packet.md @@ -0,0 +1,342 @@ +# GitHub extension + +## Control + +- **State**: Paused after first implementation/review；do not merge Hub PR #18 or core-py PR #80 as currently written。 +- **Program owner**: [knowledge lifecycle capabilities](../../packet.md) remains the only program control authority；this + file is the GitHub unit's design、preflight、implementation and acceptance record。 +- **Blocking predecessor**: + [`extension-ownership-correction`](../../../extension-ownership-correction/packet.md) must close Hub/Spoke and + core/Extension ownership corrections first。 +- **Resume point**: correct symmetric graph batch persistence and replace the handwritten GitHub transport with the retained + PyGithub client，then repeat the real-account acceptance and close the existing PRs。 + +## Outcome and MVP boundary + +Preserve the `github-extension` identity but treat its current implementation as requirements and failure evidence rather +than an incremental foundation。The MVP synchronizes one configured GitHub account's current Stars and GitHub Lists into +reusable info-base graph facts，including each List's Repository membership。 + +- One Source represents one configured GitHub access context。 +- Source configuration contains the access token only；the API determines the authenticated account and visible data。 +- Ordinary collect performs a complete current-snapshot reconciliation of Stars、Lists and List memberships。 +- There is no `full` mode、backfill command、cursor or collected-item ledger。 +- Token visibility is authority；the Source does not add an `include_private` policy。 +- Blocks previously collected are preserved when remote membership disappears。The Source removes only graph facts that its + complete current snapshot has denied。 +- Specialized client-web presentation is out of the MVP unless preflight finds a concrete acceptance blocker。 + +## Canonical graph + +`GitHubAccount` is the single canonical account/owner entity。Its `kind` distinguishes a user from an organization；the +authenticated viewer is a role expressed by provenance，not a second Block type。 + +```text +Source --collects--> GitHubAccount(kind=user) +GitHubAccount --stars--> GitHubRepository +GitHubAccount --owns--> GitHubList +GitHubList --contains--> GitHubRepository +GitHubAccount(kind=user|organization) --owns--> GitHubRepository +``` + +Canonical contents： + +- `GitHubAccount`: `node_id`、`database_id`、`kind`、`login`、`name`、`url`、`avatar_url`。 +- `GitHubRepository`: `node_id`、`database_id`、`name_with_owner`、`description`、`url`、`homepage_url`、 + `primary_language`、`topics`、`is_private`、`is_archived`。 +- `GitHubList`: `node_id`、`name`、`description`、`slug`、`is_private`。 + +Membership、ownership and provenance exist only as Relations。Volatile counters、repository activity timestamps and +`starred_at` are excluded from the MVP。GraphQL `node_id` is exact identity；`database_id` is retained native metadata，not +an identity fallback。Proposed resolver IDs are： + +- `extensions.github.account.v1` +- `extensions.github.repository.v1` +- `extensions.github.list.v1` + +## Technical topology + +```text +GitHub Source + -> GitHubGraphQLAdapter.fetch_snapshot(token) + -> authenticated viewer + -> paginated starredRepositories + -> paginated viewer.lists and each list.items + -> GitHubGraphRepository.reconcile(source, complete snapshot) + -> SourceManager.ensure_block(source) + -> locate/create/update canonical Blocks by node_id + -> reconcile current Relations + -> remove snapshot-denied GitHub-owned Relations +``` + +- The Adapter owns GitHub GraphQL requests、pagination and conversion into canonical facts。It does not know Block、Relation + or persistence。 +- The Source owns collection orchestration and Job-visible diagnostics。 +- `GitHubGraphRepository` owns GitHub graph grammar、exact identity reconciliation and relation-set replacement。 +- Resolvers own canonical Block serialization、rooted graph drafting and use-time solved/text/label projections。The Repository + calls their `create_block()` methods instead of duplicating content construction；Source-only snapshot deletion and + reconciliation do not move into Resolver。 +- Fetch the complete remote snapshot before persistence。Apply reconciliation in one database transaction so a failed page + cannot be interpreted as an authoritative empty remainder。 +- Reuse the Source anchor、Managers and caller-owned session patterns already established by Mail/RSS。 +- `GraphForm` remains the arbitrary graph producer command，but is not forced onto locate/update/delete reconciliation；the + repository uses the existing Block/Relation Managers within one session。 +- Removing a remote List removes the viewer-to-List `owns` Relation and that List's `contains` Relations while preserving the + List Block。 + +## Acceptance baseline + +Use the configured real GitHub account as authority。Keep acceptance manual or script-driven unless repeated value later +justifies promotion to an automated test。 + +1. Query the live GitHub GraphQL API for the authenticated account's complete Stars、Lists and List memberships。 +2. Dispatch an ordinary Source collect through the real Job boundary。 +3. Compare persisted `node_id` sets and membership Relations with the live snapshot。 +4. Collect again and verify that no duplicate canonical Blocks or Relations appear。 +5. Make one explicitly authorized、reversible remote Star or List-membership change，collect again，and verify both addition + and removal reconciliation。 +6. Recover a sample path through graph navigation：`Source -> Account -> List -> Repository`。 + +Exact iteration order is not an acceptance contract。Acceptance must not introduce fixture-shaped implementation behavior。 + +## Preflight findings + +### External contract and scale + +- Live authenticated GraphQL inspection confirmed `viewer`、`starredRepositories`、`viewer.lists`、`UserList.items` and the + accepted Account/Repository/List fields。The current acceptance account has hundreds of Stars、tens of Lists and hundreds + of memberships；at least one List exceeds one 100-node page。 +- Every connection therefore owns its own cursor。The Adapter must page Stars、Lists and oversized List items until + `hasNextPage == false`；a partial page sequence never reaches reconciliation。 +- GitHub GraphQL can return a response body containing `errors` and partial `data`。The Adapter treats any GraphQL error as a + failed snapshot instead of interpreting omitted members as remote deletion。 +- The expected request count and in-memory snapshot are small relative to the existing five-minute ordinary Source Job；no + streaming persistence、sleep-based throttling or new retry lifecycle is justified。 + +### Reuse and ownership + +- RSS confirms the desired split：Resolver owns canonical Block/StarsGraph construction and use projection；Repository owns + transactional reconciliation。Twitter places the reusable rooted producer directly on its Resolver。GitHub will follow + that seam rather than making its Repository a second serializer。 +- `GitHubAccountResolver`、`GitHubRepositoryResolver` and `GitHubListResolver` each provide `create_block()`、a rooted + `create_graph()` where meaningful、exact `node_id` lookup、solved content、text and resolver-qualified label。 +- `GitHubGraphRepository` uses those Resolver forms but owns set replacement for `collects`、`stars`、List ownership/List + membership and repository ownership。These source synchronization semantics are not promoted into generic Managers。 +- `StarsGraphForm` is still useful for reusable rooted resolver output，but a complete snapshot is not forced into one tree： + shared Repository nodes make that representation duplicate branches。The Repository coordinates the canonical shared graph + through one caller-owned session。 + +### Runtime, release and persistence impact + +- Preserve the public Source type identity `extensions.github.stars.Source`。Remove the extension-specific `/github/stars` + convenience route；generic Source creation/configuration and the ordinary `core.source.collect.v1` Job are canonical。 +- Retain the already-adopted PyGithub dependency and use its supported GraphQL requester。The Source may bridge its synchronous + client through `asyncio.to_thread()`；InKCre continues to own queries、nested pagination orchestration、canonical mapping and + snapshot completeness，but does not reimplement GitHub authentication、HTTP transport、retry or protocol error handling。 +- The behavior rewrite is a new GitHub extension release，expected `0.2.0` with Changie release intent、generated changelog + entry and wheel/distribution verification。 +- Source config becomes `{github_token}` and collect config becomes the existing empty command model。The Extension runtime + publishes these schemas from its Source class。The database contract must not describe this or any other Extension Source + type as built-in；existing Extension entries in `BUILTIN_SOURCE_TYPES` are a pre-existing ownership defect to remove。 +- Source state retains only the accepted authenticated Account `node_id` binding。Changing a token may refresh credentials for + the same Account；a contradictory Account identity does not silently rebind the Source。 +- Resolver IDs hard-cut to `.account.v1`、`.repository.v1` and `.list.v1`。Existing legacy `.user.v1`/`.repo.v1` Blocks are + not migrated or treated as canonical matches。 + +### Failure-branch simulation + +| Branch | Intended effect | +| --- | --- | +| HTTP、auth、GraphQL error or any incomplete pagination | Job fails；database graph and Source binding remain unchanged | +| Complete snapshot with removed Star | Delete the exact Account `stars` Repository Relation；preserve Repository Block | +| Complete snapshot with deleted List | Delete Account `owns` List and that List's `contains` Relations；preserve List/Repository Blocks | +| Repository metadata changes | Update canonical Repository content in place；Block timestamp invalidates derived retrieval records | +| Repository transfers owner | Replace the GitHub Account `owns` Repository fact；preserve Repository identity by `node_id` | +| Same Source token rotates but viewer is unchanged | Accept and reconcile normally | +| Same Source token resolves to another Account | Reject the collection before graph mutation；do not silently rebind | +| Repeated identical snapshot | Reuse Blocks/Relations and report unchanged results | +| Concurrent runs of the same Source | Serialize reconciliation by locking the Source row；the later complete snapshot converges | +| Multiple Sources resolve to different Accounts | Independent Source anchors and Account-owned fact sets | + +### Planned source surfaces + +- `extensions/github/schema.py`: canonical API facts and complete snapshot models。 +- `extensions/github/adapter.py`: async GitHub GraphQL client and connection pagination。 +- `extensions/github/resolver.py`: the three exact decoders、producer forms and use projections。 +- `extensions/github/repository.py`: one-session exact identity and relation-family reconciliation。 +- `extensions/github/stars.py`: thin Source orchestration、binding check and Job report。 +- `extensions/github/__init__.py`: resolver/source publication；remove the obsolete convenience route。 +- `extensions/github/{pyproject.toml,README.md,CHANGELOG.md}`、`.changes/github/**`: dependency、user contract and release + intent。 +- `app/database_contract/profile.py`: remove Extension-contributed Source types from the built-in catalog；Extension runtime + publication remains the only schema owner。 +- Root `pyproject.toml`/`pdm.lock` and the GitHub wheel retain PyGithub rather than introducing a handwritten GraphQL client。 +- Extension-specific product and technical truth belongs in a local GitHub Extension Unit TDD/README。The proposed Hub GitHub + capability/claim/reference integration is rejected；the wider pre-existing Memos/RSS/Mail Hub ownership needs a separate + correction review rather than being expanded by this unit。 + +### Verification plan + +- Static/repository gates：format、lint、typecheck、lock、extension release contract、GitHub wheel build and distribution + verification，then the repository `pdm run check` gate。 +- Script/manual black-box journey：real extension runtime + real GitHub GraphQL + ordinary Job + database graph comparison + + replay + one authorized reversible remote delta + graph-navigation path。 +- Do not add schema/helper/unit tests merely to mirror mappings or control flow。Promote no acceptance automation in this unit。 + +## Accepted duplicate-Source boundary + +Two different Source instances can authenticate as the same GitHub Account while their token scopes expose different subsets。 +The canonical graph currently has one Account and unqualified `stars`/`contains` facts，so a complete snapshot from either +Source cannot both own deletion independently。Adding Source IDs to Relation content would preserve observer provenance but +pollute canonical membership facts and produce duplicate graph edges；duplicating Account Blocks would abandon exact external +identity；enforcing one Source per Account adds a restriction and collision machinery。The accepted MVP deliberately adds no +special mechanism：the last successfully reconciled complete snapshot is the Account's current observed fact set，and the +acceptance journey uses one Source for the Account。 + +## Durable projection correction + +- GitHub Stars/List behavior、canonical graph and acceptance evidence are GitHub Extension truth，not a Hub product capability + merely because the Extension is first-party or important。They belong in a local Unit TDD/README。 +- Hub retains only genuinely cross-unit product truth such as generic collection、info-base authority and Extension-based + capability growth。Memos、RSS、Mail、GitHub and other concrete Extensions do not own Hub capability、claim or normative + contracts；their names may appear only as explicitly non-normative implementation examples。 +- Docs PR #18 and the core-py shared-ref commit must not merge in their current form。 + +## Accepted design reasoning + +- Stars and Lists are current collections，not an event stream；complete snapshot reconciliation is therefore ordinary + collection rather than `full` or backfill semantics。 +- The info-base retains collected entities while Source-owned Relations express changing remote membership。 +- A user that owns repositories must not become both a `GitHubAccount` and a `GitHubOwner` Block。One canonical account entity + plus a `kind` field removes that duplicate identity；Relations express its roles。 +- Fetch-before-apply makes completeness an Adapter/Source boundary and keeps partial remote observations from destructively + shaping the graph。 + +## Implementation and acceptance evidence + +- Implemented the three canonical Resolvers、an async GraphQL Adapter、a thin ordinary-collect Source and transactional + current-snapshot reconciliation。The public Source identity remains `extensions.github.stars.Source`；the obsolete + extension-specific route and PyGithub dependency are gone。 +- Real-account preflight found that GitHub Lists are not necessarily a subset of Stars。The Adapter therefore resolves + list-only Repository node IDs in bounded GraphQL batches instead of dropping memberships or embedding an oversized nested + Repository query in each List page。 +- Remote PostgreSQL acceptance exposed per-entity `flush + refresh` as an unacceptable persistence round-trip multiplier。 + `BlockManager.create_many()` and `RelationManager.create_many()` now provide caller-owned batch persistence while the + GitHub Repository retains all identity and reconciliation semantics。 +- A disposable Neon branch was migrated from the declared preview baseline to revision `50b2c08dd267` before running the + real ordinary Job journey。The live authority contained 473 Stars、19 Lists、395 memberships and 474 unique Repositories。 +- First collection created 942 GitHub canonical Blocks and 1,362 managed Relations。The replay reported zero created、updated + or deleted Blocks/Relations。The journey also recovered `Source -> Account -> List -> Repository` through graph-navigation + retrieval and exercised Repository label/text projections。 +- The real reversible remote-delta step remains intentionally unperformed：the complete set and idempotent replay establish + the implementation baseline without mutating the Human's GitHub account。It is a closure option，not hidden automated + coverage。 + +## Review correction baseline + +The first implementation passed its observed data journey but review rejected four design decisions。Passing acceptance does +not override an incorrect ownership or abstraction boundary。 + +1. **Batch graph persistence**：remote PostgreSQL proved that per-entity `flush + refresh` is too expensive，but adding + leaf-level `create_many()` methods without routing existing graph insertion through them created an adjacent interface。 + Keep symmetric `BlockForm[]` / `RelationCreateForm[]` persistence primitives，make `InfoBaseManager.submit_graph()` consume + them，and document why reconciliation cannot be represented as a pure-new `GraphForm` command。`StarsGraphForm` remains the + resolver-rooted identity-aware path。 +2. **Built-in versus Extension**：checked-in、first-party and enabled-by-default do not make an Extension contribution a core + built-in。`BUILTIN_SOURCE_TYPES` currently contains GitHub、Mail、RSS、Telegram and Twitter Source types；the Extension + runtime already publishes Source schemas and is their correct owner。Remove the whole incorrect category rather than only + reverting GitHub's entry。 +3. **Hub versus Extension-local truth**：importance and successful delivery are not promotion criteria。GitHub-specific PRD + capability/claim and Product TDD reference integration were written to the wrong owner。Audit the same historical promotion + for Memos、RSS and Mail，then retain their protocol/product contracts in Extension-local Unit TDDs。 +4. **Protocol client ownership**：the preflight failed to inspect the already-installed PyGithub release。PyGithub 2.9.x + supports GraphQL queries、mutations、GitHub error handling and pagination；a custom `httpx` GitHub GraphQL transport is + unjustified。Use the mature dependency and keep only source-specific snapshot orchestration and mapping。 + +### Root cause and preventive guidelines + +- The repeated failure is **promotion bias**：current vertical pressure was treated as permission to move behavior into a more + public、durable or core-owned layer without proving its owner、consumers、lifecycle and reuse value。 +- Before promotion，name four independent axes：delivery owner（core/Extension）、durable owner（Hub/Spoke）、interface layer + （domain command/persistence mechanism）and external capability owner（existing dependency/InKCre）。A decision on one axis + is not evidence for another。 +- Before implementing an external protocol client，inspect current dependencies and primary documentation，then record the + exact unsupported behavior that remains。Handwritten transport is allowed only after that gap is demonstrated。 +- A vertical acceptance proves observable behavior；it cannot legitimize a wrong module or documentation owner。 +- Ponytail's ordered ladder is now the default implementation check：need → existing code → stdlib → native platform → + installed dependency → minimum new code。It shortens the solution only after end-to-end ownership is understood。 + +### Paused delivery state + +- Hub PR #18 and core-py PR #80 remain open for review but must not merge as currently written。 +- No corrective source or durable-doc mutation starts until Sir explicitly says “start”。 + +## Accepted correction execution baseline + +### Durable ownership correction + +1. Verify that each concrete Extension's local Unit TDD/README retains a readable normative home before removing duplicate Hub + truth；add a local GitHub Extension Unit TDD for the accepted graph、snapshot and reconciliation contract。 +2. Turn Hub PR #18 into an ownership-correction PR：remove concrete Memos、RSS、Mail and GitHub capability/claim/reference + contracts from PRD/Product TDD。Concrete names may remain only where clearly marked as non-normative implementation examples。 +3. Push the corrected Hub source first，then update core-py's `docs/_shared` ref in its own commit。Do not merge the current + shared-ref commit unchanged。 + +### Runtime catalog correction + +1. Remove every Extension Source profile from the core database contract，including GitHub、Mail、RSS、Telegram and Twitter。 +2. Because no true core built-in Source type remains，delete rather than preserve an empty speculative catalog/fallback where + callers prove it has no other consumer。Extension activation remains the only Source schema publication path and uses the + Source class description/config/collect/backfill schemas。 +3. Verify database init/readiness and Extension cold restore without requiring disabled or unavailable Extension Source rows。 + +### Graph insertion correction + +1. Keep symmetric caller-session batch persistence primitives：`BlockForm[] -> BlockModel[]` and + `RelationCreateForm[] -> RelationModel[]`，each with one flush and no per-row refresh。 +2. Route `InfoBaseManager.submit_graph(GraphForm)` through those primitives：insert the new Blocks as one batch，resolve signed + local IDs，then insert Relations as one batch。Single-item creation should reuse the same primitive where doing so shortens + rather than duplicates persistence behavior。 +3. Keep `add_stars_graph_to_session()` as resolver-rooted identity-aware fetchsert。GitHub snapshot reconciliation continues to + use batch primitives directly because it locates、updates、preserves and deletes existing graph facts，while `GraphForm` is + a producer command for inserting a caller-declared graph。 +4. Record this distinction in the existing local `business-pipeline-and-authority.md` and the public method docstrings；do not + create another graph abstraction or document。 + +### GitHub protocol correction + +1. Restore the existing PyGithub dependency in the root and GitHub wheel。Use public `Github.requester.graphql_query()` for + GitHub authentication、transport、retry and GraphQL error handling，bridged from the async Source through + `asyncio.to_thread()`。 +2. Retain only a source-specific snapshot adapter：queries、nested connection pagination、complete-snapshot checks、list-only + Repository resolution and canonical fact mapping。It is not a general GitHub GraphQL client。 +3. Remove the handwritten `httpx` execution/error layer and its dependency delta。Do not add githubkit/gql while the already + installed PyGithub covers the demonstrated boundary。 + +### Verification and delivery + +- Re-run static/lock/release/wheel gates and the real GitHub ordinary Job + exact graph comparison + idempotent replay journey。 +- Add no helper/unit tests；the existing manual script remains non-durable acceptance tooling。 +- Update the two existing PRs rather than opening replacement PRs。Keep Hub、core implementation and shared-ref commits + separable for review。 + +### Correction Impact Handshake + +- **Exact objects**：Hub PRD/Product TDD concrete Extension sections；local Extension Unit TDDs；Source database-contract + profiles and runtime publication；InfoBase batch/GraphForm insertion；GitHub snapshot adapter/dependencies；PR #18/#80。 +- **From -> To**：importance-based promotion -> owner-based placement；Extension schemas in core built-ins -> runtime + Extension publication；adjacent batch APIs -> shared persistence primitives beneath existing graph commands；handwritten + GraphQL client -> PyGithub-backed source adapter。 +- **Side effects**：database initialization no longer seeds Extension Source types before activation；disabled Extensions do + not leave their Source catalog as artifact-owned truth；Hub diff becomes a net deletion/normalization；GitHub collection + remains behaviorally equivalent。 +- **Blast radius**：core-py database init/readiness、all first-party Source Extensions、InfoBase graph producers、GitHub wheel + and lock、Hub shared docs and shared ref。No client-web runtime or database schema migration is expected。 +- **Invariants**：Extension activation publishes complete Source schemas；GraphForm signed references resolve exactly；snapshot + errors never become deletion authority；canonical Blocks/Relations and accepted Job report remain unchanged；one durable + owner per fact。 +- **Verification**：repository gate、extension release/wheel verification、database reset/readiness、Extension activation + catalog inspection、real-account GitHub journey、Hub/submodule checks and final PR diff review。 +- **Uncertainty**：PyGithub raw GraphQL pagination still requires source-owned nested orchestration，but primary docs and the + installed 2.9.x API prove transport/error support。Historical Hub removal may expose a missing local Extension contract；the + pre-delete owner check resolves that without retaining duplicate Hub truth。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/acceptance.md b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/acceptance.md new file mode 100644 index 0000000..e3792e2 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/acceptance.md @@ -0,0 +1,146 @@ +# Graph Navigation Retrieval Acceptance + +## Status + +- **Status**: accepted and executed；Preview closure completed on 2026-08-23 against core `d2cac7d` and client-web + `5b8071e`。 +- **Evidence principle**: black-box-first。Static checks prove types、migration/index shape and ownership boundaries；they do + not substitute for manager/database or browser journeys。No pixel snapshot becomes visual authority。 + +## Evidence topology + +```text +machine-readable topology corpus + ├─> core-py public manager -> real PostgreSQL/SQLModel + └─> @inkcre/core public manager -> real PostgREST + | + v + semantic parity assertions + +real producer corpus + -> Memos / RSS / Atom / HTML / rumination / Mail graph + -> neighborhood + relation + path operations + -> client-web Graph navigation host + -> Resolver preview / Inspector / Solved Content +``` + +The topology corpus owns graph shape and legal-result properties，not database IDs、row ordering beyond the public cursor +contract or one arbitrary equal-shortest path。Its durable authority is a machine-readable JSON contract adjacent to the Hub +Product TDD；both Spokes consume the same file through `docs/_shared` rather than duplicating independently drifting fixtures。 + +## A — Manager contract and parity + +Run every scenario against a migrated disposable PostgreSQL database through the public Python manager and public TypeScript +manager。No repository helper or private query method is the assertion surface。 + +1. **Block neighborhood** + - existing focal is returned with an endpoint-closed page of Relations/endpoints； + - isolated focal succeeds with one Block and zero Relations；missing focal returns `None`/the peer contract equivalent； + - `in`、`out` and `both` preserve persisted Relation direction； + - exact `contents` selects only exact Relation content values； + - default/explicit limits、Relation-ID-desc ordering and exclusive `next_cursor` produce no repeated Relation across pages。 +2. **Relation neighborhood** + - existing Relation returns itself and exactly both persisted endpoint Blocks；missing Relation returns no result； + - direction/content are unchanged and Relation does not acquire Resolver or solved-content fields。 +3. **Bounded path** + - a unique shortest path returns `found` with endpoint-closed GraphModel and aligned ordered Block/Relation ID paths； + - `from == to` returns one Block and zero Relations；cycles terminate； + - an exhaustively disconnected graph returns `not_found`；hop or explored-graph exhaustion returns `limit_reached` when + completeness has not been proved； + - `in`/`out` and exact contents alter admissible traversal without rewriting stored Relation direction； + - an equal-shortest graph accepts any valid shortest result and never exact-asserts an incidental tie-break。 +4. **Concurrent authority changes** + - a successful neighborhood response remains endpoint-closed if a Relation/endpoint disappears between its internal + reads；the original continuation cursor remains the page cursor； + - path assembly reuses the Relations observed during traversal；if endpoint closure can no longer be assembled after a + concurrent authority change，the result is ordinary `not_found`，with no hidden retry or leaked cross-statement + inconsistency。 +5. **Random Block primitive** + - empty authority returns no Block；non-empty authority returns one existing Block without loading all IDs into the + browser or asserting distribution quality from a tiny sample。 + +Parity compares status、entity sets、endpoint closure、direction、continuation and path validity。It does not require the two +implementations to choose the same member of an equal-shortest result set。 + +## B — Real graph vertical + +Reuse the readable producer authority already accepted by Semantic Retrieval rather than inventing lorem-ipsum graph data： + +- a Memos design-capture note and comment； +- real RSS/Atom protocol doubles with deep-module and Peer-discovery articles； +- the pinned public-domain SQLite Architecture document and its rumination-produced Pager interpretation； +- the Mail acceptance thread with parent/reply、participants、MIME parts and materialized semantic content。 + +Generated IDs and graph rows remain runtime results，not corpus authority。Acceptance aliases resolve only after real producers +write the graph and never enter production models/APIs。 + +Required journeys: + +1. Navigate from each producer root to one direct semantically useful neighbor and confirm the returned Relation identity、 + content and direction match persisted producer authority。 +2. Navigate the SQLite source → interpretation/semantic-content chain without using Resolver-local relation access as the + graph-retrieval implementation。 +3. Find one unique path inside the Mail thread/component graph and verify every path step can be independently followed as a + neighborhood request。 +4. Address one real Relation directly and recover both endpoints；then resolve their labels/previews outside the retrieval + result，proving presentation-free authority。 +5. Use an isolated real text Block to prove that “no Relations” is a successful graph fact，not a missing/failure state。 + +## C — Client-web navigation-host journeys + +Execute against the real development/E2E database and real built/dev client，with browser runtime config injected by the +existing E2E harness rather than committed to the build。 + +1. **Initialization**: opening Graph with no focal chooses one existing random Block and realizes its standard bounded + neighborhood。An empty database shows the application Recall/Search fallback through InkCre feedback presentation。 +2. **Progressive focal navigation**: activating a neighboring Block or Relation updates role-named focal query state、replaces + the bounded active scene and preserves shared entity positions；camera zoom alone issues no retrieval。 +3. **Exploration scale**: compact/standard/broad admits bounded continuation around the same focal；decreasing scale hides + surplus entities without deleting authority or requiring a total-count query。 +4. **Direction emphasis**: all/incoming/outgoing changes opacity/emphasis only。Entity identity/count、layout、camera and + retrieval requests remain unchanged，and dimmed entities remain interactive。 +5. **Inspection**: explicit Inspect opens Block/Relation Inspector without a scrim。The Graph remains pointer-accessible； + closing invokes browser/router back without undoing the already selected focal scene。 +6. **Solved content**: Block Inspector opens Solved Content over the same scene。Closing returns to the Inspector；preview and + full renderer consume the same solved-content authority，but Graph preview remains concise and interaction-free。 +7. **Application Search**: `Ctrl/Meta+K` opens Recall/Search。Recall defaults to List outside an InfoBase View and hands `q` + to the current Graph when it is active；Find path hands `path_from`/`path_to` to Graph，which realizes found/not-found/ + limit-reached without broadening or retrying the query。 +8. **Relation route**: a direct Relation destination seeds relation + endpoints，supports focal navigation to either endpoint + and opens only the Relation Inspector—never a fictional solved-content route。 +9. **History/deep link**: refresh reconstructs focal/path/outlet state from the URL contract；camera、scale、layout、cursor and + session cache remain non-authoritative runtime state。 +10. **Responsive/accessibility**: keyboard focal/Inspect actions work；focus presentation is visible without color alone； + reduced-motion removes spatial travel；narrow Solved Content can use the viewport while route/content semantics remain + unchanged。 + +## Visual acceptance + +Visual review runs in the actual InkCre shell at representative desktop and narrow widths。It judges the accepted state +hierarchy—restrained、cool、professional、sharp；existing palette；minimal chrome；legible focal/context/direction；intrinsic +previews；modeless outlets—rather than comparing pixels to a mock。The rejected visual spike is explicitly excluded。 + +## Executed Preview evidence — 2026-08-23 + +- core-py exact head `d2cac7d`：repository/artifact、Preview database and Preview app workflows succeeded；local + `pdm run check` passed migration integrity、lint、format、type checking and the admitted suites。 +- client-web exact head `5b8071e`：Client checks and Pages Preview workflows succeeded；local `pnpm check` passed。 +- On the actual Pages Preview + Preview PostgREST pair，a temporary four-Block/three-Relation corpus proved random focal、 + focal replacement、soft direction emphasis、Block Inspector、Solved Content、Relation Inspector and a three-hop path。 + The path scene contained exactly four Inspectable Blocks and three Inspectable Relations；browser logs contained no error。 +- Inspector outlets preserved the Graph as navigation host：Block Inspect realized `/blocks/2`，Solved Content realized + `/blocks/2/content`，browser back returned to the Inspector，and Relation Inspect realized `/relations/2` over the same + `focal_block=2` scene state。 +- The temporary Preview corpus was removed after the journey；its generated IDs are evidence only and do not become fixture + or product authority。 + +## Proof allocation + +- Schema、types、migration shape and ownership facts belong to static checks。Public-manager behavior first remains a + manually executed real-PostgreSQL/PostgREST script or journey；automation requires a later，explicit promotion decision。 +- High-value client journeys remain manual/scripted against the database/Core Peer chain；do not create component-helper + tests for behavior already covered by type checking or the browser vertical。 +- Visual calibration、camera feel、drag continuity and reduced-motion quality remain an explicit manual/scripted Acceptance + checklist until repeated regressions prove that a narrower automated mechanism has positive ROI。 +- No negative-path matrix is required merely for completeness；retain only failures that distinguish a public outcome or + protect an accepted invariant。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/implementation-plan.md new file mode 100644 index 0000000..d2a7aab --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/implementation-plan.md @@ -0,0 +1,216 @@ +# Graph Navigation Retrieval Implementation Plan + +## Status + +- **Baseline**: Product / Technical contract and Acceptance are accepted for implementation planning. +- **Mutation state**: G0–G6 are implemented in the four owner worktrees；G7 publication、shared-ref consumption、real-producer + and preview closure remain pending。 +- **Execution shape**: owner-separated increments with public-contract evidence after each meaningful vertical；no large + cross-repository atomic commit is attempted. + +## Runtime topology + +```text +caller + -> GraphNavigationRetrievalManager + -> peer-local Block / Relation query primitives + -> endpoint-closure assembly + -> bounded bidirectional BFS when path is requested + -> GraphModel / operation-specific outcome + +client-web InfoBase View + -> @inkcre/core local manager over PostgREST + -> scene merge + Resolver preview loading + -> measured layout + camera realization + -> modeless route outlets +``` + +The Python and TypeScript managers are equal implementations of one use-domain contract. Neither calls the other，uses Peer +delegation，or introduces an HTTP/database RPC. Retrieval never owns preview、layout、camera or route state. + +## Increment G0 — shared read contract and query foundations + +### Hub owner + +- Add one Product-TDD contract projection for graph-navigation retrieval plus a machine-readable topology corpus adjacent to + it. The JSON owns topology aliases、directed Relations、scenario inputs and semantic result assertions；it does not own row + IDs or one arbitrary equal-shortest path. +- Apply through the shared-doc workflow；Spokes only consume the resulting shared ref in owner-separated commits. + +### core-py owner + +- Add read-only `GraphModel` and operation-specific neighborhood/path models under `app/schemas/`，separate from producer + `GraphForm`. +- Add singular random Block access and bounded Relation query primitives. Remove `BlockManager.iterate_from_block()` rather + than preserving it behind the new manager. +- Add endpoint indexes `(from_, id DESC)` and `(to_, id DESC)` through one Alembic migration. Do not add a content index until + query evidence justifies it. + +### client-web `@inkcre/core` owner + +- Add matching Zod/TypeScript read models and peer-native query primitives over PostgREST. +- Add `Block.getRandom()` using count + stable-order random offset；never transfer all Block IDs to choose one. +- Keep existing broad Active Record methods only where current consumers still require them；the retrieval manager must not + implement its contract by `getAll()`. + +### Proof + +- migration upgrade from current clean baseline；catalog inspection proves both endpoint indexes；static/schema checks prove + write `GraphForm` and read `GraphModel` remain distinct. +- shared corpus loads unchanged in Python and TypeScript runners. + +## Increment G1 — core-py public manager + +- Create `app/business/graph_navigation_retrieval/` as the use-domain owner. +- Implement `get_block_neighborhood()` as direction-specific ordered Relation reads followed by a manager-owned ID-desc merge + and batched endpoint lookup. `both` performs one bounded incoming and one bounded outgoing read，then returns the merged + `limit + 1` page；this preserves the public incident-page abstraction while allowing each branch to use its endpoint index. + Omit Relations whose endpoints no longer resolve so every successful result is endpoint-closed. +- Implement `get_relation_neighborhood()` as Relation + exact endpoints or no result. +- Implement bounded bidirectional BFS for `find_path()` with direction and exact-content pruning during traversal，not after + materializing a broad graph. Assemble and revalidate persisted rows only after a candidate path is found. +- Use the accepted `PathFound | PathNotFound | PathLimitReached` public outcomes. Do not expose search frontiers、tie-breaks、 + retries or snapshot claims. + +### Query sequence + +```text +request + -> locate focal/endpoints + -> query bounded incoming/outgoing Relation branches + -> merge by Relation ID and cut the public page + -> query required endpoint Blocks in batches + -> validate endpoint closure + -> return public model/outcome +``` + +For path search，frontier Relation reads are batched by Block IDs and split into internal chunks when needed；the chunk size is +an implementation limit，not public API. Default/hard budgets remain `4/8` hops and `1000/10000` explored Blocks unless real +PostgreSQL evidence contradicts them. + +### Proof + +- manually executed public-manager journey against real PostgreSQL using the shared corpus；include cursor continuity、cycles、 + direction/content pruning、equal-shortest semantic validity and concurrent-authority endpoint closure. +- `EXPLAIN (ANALYZE, BUFFERS)` on a transaction-local sparse 50k topology confirms each direction-specific query chooses its + `(endpoint, id DESC)` index；failure to choose an index on tiny fixtures alone is not treated as contrary evidence. + +## Increment G2 — `@inkcre/core` public manager + +- Add a presentation-free `graph-navigation-retrieval` domain module beside InfoBase models，not under `sink/graph`. +- Implement the same three public operations locally over PostgREST. Use separate ordered incoming/outgoing page queries and + merge them inside the manager；use `.or(from_.in/to_.in)` only for bounded traversal frontiers，plus exact + `.in(content)` and exclusive cursors. Do not add an RPC. +- Validate every returned row through existing Zod Active Record models and enforce endpoint closure before exposing results. +- Remove Vue Flow、MDS、community and layout ownership from `packages/core/src/sink/graph` once client consumers have moved； + presentation algorithms belong to the app InfoBase View. + +### Proof + +- run the shared corpus through the public TypeScript manager against real PostgREST；compare outcome kind、entity sets、 + direction、cursor and path properties with the Python run，not private query counts or equal-path identity. + +## Increment G3 — proven design-system gaps + +- Extend `InkPopup` with a backward-compatible no-scrim/modeless option. Default behavior remains the current modal scrim； + no-scrim does not install an invisible pointer-blocking overlay. +- Add domain-neutral `InkSearchBar` only after extracting the shared query/submit/clear/loading/accessibility presentation + from its two real consumers. Retrieval mode、shortcut、routing and result ownership remain outside `@inkcre/ui-web`. +- Add focused component/story evidence and a Changeset；publish the design package before final client registry verification. + +No Graph node、edge、toolbar、panel-header or route-outlet component is promoted into the design system in this increment. + +## Increment G4 — Resolver preview contract + +- Add required `previewRenderer` beside `solvedContentRenderer` on the Resolver registration contract. Both consume the same + Resolver instance and solved-content authority；there is no preview projection or layout hint. +- Provide interaction-free bounded previews for every core Resolver and the in-scope Mail/Twitter extension Resolvers. Do not + silently fall back to a full renderer because full rendering may materialize content or expose business actions. +- Split presentation components only where preview/full behavior is genuinely different；do not duplicate Resolver content + acquisition. +- Coordinate core package、host and extension version/Changeset inputs so no published runtime loads an old Resolver contract + as if it supported preview. + +### Proof + +- type/build checks cover every registered Resolver；targeted renderer checks prove both contracts receive the same solved + content and preview contains no business actions. Extension host smoke proves Mail/Twitter remotes register successfully. + +## Increment G5 — application router, Recall/Search and outlets + +- Extend the application implementation of `InfoBaseRouter` and Vue Router mapping for Block/Relation focal destinations、 + entity-local inspectors and Block solved content. Keep router state authoritative in Vue/browser history；do not create a + second history store. +- Normalize mutually exclusive reconstructive query shapes：`focal_block`、`focal_relation`、`path_from + path_to`、`q`. + Scene scale/direction/camera/layout/cursor/cache remain runtime state. +- Add one application-owned Recall/Search singleton opened by `Ctrl/Meta+K`. Recall hands selected results to List by default + or the active InfoBase View；Find path selects endpoints and hands path address state to Graph. +- Make Block/Relation Inspector entity-local. Desktop outlets use modeless `InkPopup`；closing calls router `back()` and does + not rewrite that action as another forward route. + +### Proof + +- router normalization/unit evidence for deep links and conflicting query forms；browser evidence for shortcut、back/forward、 + relation inspection and solved-content return path. + +## Increment G6 — Graph View behavior rewrite + +- Replace full-graph loading/community/layout selection with random focal + bounded standard neighborhood. Hard-cut old focal + community machinery rather than adapting it to partial scenes. +- Maintain a bounded session scene cache keyed by entity identity. Activating a canvas entity changes focal；Inspect is an + explicit secondary action. Scale admits the accepted `8/20/50` Relation budgets without claiming totals. +- Retrieve one `both` neighborhood per focal/scale. Direction is soft presentation state：inactive Relations/endpoints remain + visible and interactive，and changing direction triggers no query、layout or camera work. +- Render structural shells first；resolve focal preview first，then admitted neighbors through a small cancellable concurrency + pool and shared Resolver cache with `materializeMissing=false`. +- Use Vue Flow's actual node dimensions and `nodesInitialized` event/composable. Layout is deterministic around intrinsic + measured sizes；parallel Relations receive deterministic lanes. Existing positions survive shared-entity scene changes and + user drag remains session-only. +- Drive camera only from explicit focal/path/refocus actions. After the relevant nodes are initialized，use Vue Flow's + node-scoped `fitView`；remove delayed repeated fitting. Manual pan/zoom owns the camera until another explicit action. +- Reuse the current InkCre shell、palette、tokens and feedback components. Visual state stays restrained、cool、professional + and sharp；focal is not enlarged，debug/query vocabulary is absent，and the rejected spike is not an implementation input. + +### Proof + +- focused browser journeys from Acceptance C for initialization、focal navigation、scale、direction、inspection、relation + route、path and history；manual/scripted actual-shell review for camera feel、drag continuity、intrinsic preview layout、 + reduced motion and representative narrow width. + +## Increment G7 — closure and promotion + +- Run the real producer vertical across Memos、RSS/Atom/HTML、rumination/SQLite and Mail graphs；aliases remain harness-only. +- Run Python and TypeScript corpus parity against the same migrated PostgreSQL authority，then the built client browser + journeys. Do not replace this with schema/helper tests or pixel snapshots. +- Promote stable product/technical truth to Hub and local implementation truth to Unit TDD only after the implementation has + proved it. Update shared refs and owner repositories in separate commits. +- Verify design package publication/consumption、client extensions build/release inputs and preview deployment before closing + the unit. Release boundaries follow repository owners；the implementable unit itself is not a release unit. + +## Branch simulation and hazards + +1. **A focal disappears**：manager returns no neighborhood；Graph keeps route authority and realizes missing/fallback state， + never chooses an unrelated focal silently. +2. **A Relation disappears between reads**：successful response omits it unless both endpoints survive；cursor still derives + from the ordered Relation page，not the filtered output. +3. **Equal shortest paths**：implementation may pick either；tests assert validity and minimal hop count only. +4. **Preview fails**：Node shell remains navigable and presents concise failure feedback；retrieval success is unchanged and + no hidden retry/materialization occurs. +5. **Preview changes dimensions**：measurements are batched before layout；camera is fitted once for the explicit action，not + once per completion. +6. **Old extension remote lacks preview**：coordinated contract/version release is required；full-renderer fallback is not + introduced to hide the mismatch. +7. **PostgREST URL/frontier grows**：split private frontier batches；do not expose transport chunking or add database RPC. +8. **Scene shrinks from broad to compact**：surplus entities leave the active scene but may remain in bounded session cache； + authority is neither deleted nor described as an undo. +9. **Manual camera gesture races loading**：manual movement cancels pending automatic camera ownership；preview/layout may + settle without stealing the viewport. +10. **Design package source lane differs from registry release**：joint development may use the workspace/source lane，but + final client verification consumes the published package exactly as deployment does. + +## Deliberate exclusions + +- graph-navigation Peer delegation/inbound，database RPC，generic N-hop or pattern language； +- full-graph/community analysis，durable layout/camera/scene state，manual node resizing，inline expansion； +- snapshot isolation、hidden retry、path ranking/tie-break API，negative-path matrices without invariant value； +- new Graph-specific design-system abstractions or a new application shell. diff --git a/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/packet.md b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/packet.md new file mode 100644 index 0000000..539c21e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/packet.md @@ -0,0 +1,574 @@ +# Graph Navigation Retrieval + +## Control + +- **State**: Complete — implementation、owner-separated publication、exact-head CI/CD and actual Preview shell closure + completed on 2026-08-23。 +- **Objective**: 让用户或下游能力从一个已定位的 graph entity 出发，取得可理解、可继续沿方向导航的 + 既有 Blocks/Relations，而不要求加载整个 info-base graph。 +- **Current work**: closed；return to implementable-unit selection。 +- **Decision authority**: [task decision register](../../decisions/index.md)；本文件只投影当前 unit 状态、 + working hypotheses 与 discussion queue。 +- **Execution gate**: passed on 2026-08-18。The approved Impact Handshake covers Hub contract/corpus、peer-local managers、 + endpoint indexes、Resolver preview hard cut、application routing/Search、Graph View rewrite and proportionate design-system + changes。 + +## Implementation checkpoint — 2026-08-18 + +- Hub Product TDD and one machine-readable topology corpus are drafted on the dedicated Hub branch；they remain uncommitted + until owner-separated review/push。 +- core-py owns the presentation-neutral public manager、read models、endpoint query primitives and endpoint indexes。 + PostgreSQL integration passes on migration head `50b2c08dd267`；a transaction-local sparse 50k probe selected + `relations_from_id_desc_idx` and completed the bounded page in under 1 ms on the development runtime。 +- `@inkcre/core` implements the same contract directly over PostgREST。Its unit corpus and a real PostgREST smoke both pass； + the smoke produced a 3-Relation endpoint-closed neighborhood and a 2-hop outgoing path without RPC or Peer delegation。 +- Resolver registration now requires distinct preview/full renderers over the same solved-content authority。Core、Mail and + Twitter in-scope registrations have bounded interaction-free previews；old `sink/graph` and app-owned full-graph/community + machinery are hard-cut。 +- client-web owns role-named scene routing、application Recall/Search、entity-local inspectors and the rewritten bounded + Graph navigation host。The actual local shell now passes random/isolated initialization，4-Block/3-Relation focal + navigation，soft outgoing emphasis，3-Block/2-Relation shortest path，modeless Relation/Block/Solved Content outlets and + browser-back restoration against the converged PostgREST runtime。 +- design adds only the proven generic `InkPopup.scrim` capability (default remains modal) plus focused evidence and a + Changeset。A generic SearchBar was not promoted because the second proven presentation consumer did not justify a stable + abstraction yet。 +- Implementation preflight exposed and fixed three delivery-baseline defects needed to run the actual shell：development + Compose now builds the artifact-free `runtime` stage；development readiness accepts additional advertised capabilities； + readiness/reset explicitly invoke `python scripts/container.py` after the image ENTRYPOINT hard cut。These are deployment + contract fixes, not Graph business adaptations。 +- The browser vertical exposed a fourth distributed-runtime defect：a JWT issued at the caller's exact current second can be + rejected by a slightly slower PostgREST clock as `JWT issued at future`。The signing boundary now backdates `iat` by five + seconds while retaining the existing bounded `exp - iat` contract；path and uncached relational reads no longer leak clock + skew into domain callers。 +- Review rejected the accumulated unit/component/helper test baseline rather than only this unit's new tests。client-web and + design have removed Vitest/component automation and its test-only dependencies/config；core-py has removed ordinary + manager、route、schema、adapter-helper、mock runtime and deployment-helper tests。Only admitted migration integrity，real + integration/acceptance and mature Playwright E2E remain，outside the default repository gate unless their existing owner + explicitly retains them。The organization-wide authority is now `.github/TESTING.md`；each governed repository only + references that policy and records justified local suites/commands。Static enforcement and builds are the default CI + proof；new automation requires explicit Sir approval after a manual/scripted black-box journey has matured。 +- Operational self-review removed a speculative concurrency validation from both peer-local path implementations。Traversal + now retains observed Relation rows；a candidate whose endpoint closure can no longer be assembled returns `not_found` + instead of escalating an internal cross-statement race。Client Graph、Relation Inspector and Recall boundaries retain + contextual diagnostics while presenting shallow completion messages rather than raw internal exceptions。 +- Exact-head closure passed for core `d2cac7d` and client-web `5b8071e`。The actual Pages Preview + Preview PostgREST journey + covered focal navigation、soft direction、Block/Relation inspectors、Solved Content and a four-Block/three-Relation + shortest path without browser errors；the temporary Preview corpus was removed afterwards。 + +## Unit Review — 2026-08-18 + +### What is now stable + +- **Product value**: graph-navigation retrieval 是从一个已定位的 Block/Relation 出发，对既有 graph facts 做 bounded、 + direction-preserving、identity-preserving 的取得；它既不等同于 Resolver 对 focal Block 的解释，也不只服务 Graph + View。 +- **MVP primitives**: Block neighborhood、Relation neighborhood、bounded shortest-by-hop path。Unrestricted N-hop、 + pattern language、community/centrality/ranking 不进入 MVP。 +- **Result authority**: `GraphModel` 只承载 persisted Blocks/Relations，并保持 endpoint closure；operation-specific + result 只补 continuation 或 ordered path evidence，不混入 label、preview、layout、scene delta 等 presentation。 +- **Peer topology**: core-py/SQLModel 与 client-web/PostgREST 各自本地实现同一领域 contract；当前没有 graph- + navigation Peer capability、HTTP inbound 或 database RPC。 +- **Graph product model**: random focal + bounded one-hop 是默认 scene；canvas activation 改变 focal，Inspect 是独立 + secondary action；bounded active scene 覆盖 session cache；exploration scale 与 camera zoom 分离；direction 是 soft + emphasis；Find path 由 application Search 组合、Graph View realize。 +- **Presentation ownership**: retrieval 保持 presentation-free；Resolver registration 提供消费同一 solved-content + authority 的 preview/full renderers；Graph/List 各自拥有加载 orchestration；InfoBase View 是 navigation host， + Inspector/Solved Content 是 modeless route outlets。 +- **Visual constraint**: Graph 必须从现有 InkCre palette、tokens、components 与 application shell 出发。被拒绝的 + full-screen spike 只保留为失败证据，不是实现参考。 + +### What is deliberately deferred + +- general N-hop/ego graph、Cypher/SPARQL-like pattern matching、ranked/alternative paths； +- community analysis 及旧 Graph 全图 community/layout selector； +- graph-navigation Peer delegation、generic database RPC、cross-statement snapshot/retry； +- inline expand/full-content、manual node resizing、durable scene/layout/camera state； +- 将 Graph-specific node、toolbar、panel header 等伪通用组件 promotion 到 design system。 + +### Remaining gates + +1. **Presentation preflight — complete**: [presentation-preflight.md](presentation-preflight.md) inventories the actual + client shell and InkCre authority，closes the narrow Node/Relation/outlet state contract and identifies only two proven + design-system pressures (`InkPopup` no-scrim and domain-neutral `InkSearchBar`)。A new broad visual spike is not a gate。 +2. **Acceptance contract — complete baseline**: [acceptance.md](acceptance.md) separates public-manager parity、real producer + graph vertical、client-web navigation-host journeys and non-snapshot visual review。 +3. **Implementation plan — complete**: [implementation-plan.md](implementation-plan.md) maps the state diff into seven + owner-separated increments，records exact module surfaces、runtime sequences、proof and failure branches。 +4. **Preflight calibration — complete**: endpoint indexes、PostgREST query composition、Resolver preview migration surface、 + shared-corpus owner and Vue Flow measured-dimension/camera APIs are verified。Budgets (`8/20/50` scene scale、`4/8` hops、 + `1000/10000` explored Blocks) remain explicit provisional constants subject to real query/UX evidence，not open design。 +5. **Impact Handshake**: freeze the exact mutation and verification boundary before governed source changes begin。 + +Preflight corrected one private query assumption without changing the public contract：a `both` neighborhood is assembled +from separately bounded incoming/outgoing Relation reads，then merged by ID-desc inside the manager。A single +`OR + ORDER BY id` query can favor the primary-key scan and filter away most rows；the split form used both proposed endpoint +indexes on a transaction-local sparse 50k topology and is directly expressible through PostgREST。The experiment rolled back +all generated rows and indexes。 + +## Why This Unit Exists + +Feature/semantic retrieval 可以帮助定位一个可能有用的 Block 或 Relation，但当前通用 graph surface 会加载整个 +info-base。缺失的是一个 bounded use capability：从已知位置取得有意义的局部 graph，并让 caller 沿 Relation +方向继续探索，而不是把全图可视化当作检索。 + +## Boundary With Resolver + +```text +Resolver + focal Block + hydrated content + relevant local Relations + -> solved/use-facing interpretation + -> may materialize an explicitly owned missing derivation + +Graph navigation retrieval + addressed Block/Relation + navigation request + -> selected existing Blocks/Relations + navigation evidence + -> read-only with respect to graph authority in the current hypothesis +``` + +- `Resolver.get_relations()` 是 Resolver 解释 focal Block 时取得 direct local facts 的内部能力；它不自动成为 + graph-navigation product contract。 +- Resolver 可以把若干 adjacent Blocks/Relations 隐藏在一个 solved-content projection 内；graph navigation + retrieval 反而必须保留 entity identity、Relation direction/content 与为什么该 entity 被返回。 +- Resolver 的 `materialize_missing` 是 lazy interpretation contract；graph navigation retrieval 是否发现已有 + graph、是否触发 organization/materialization 是另一条 effect 边界，不能从 method 复用自然推出。 +- Resolver label/solved content 可以服务结果呈现，但不能未经讨论成为 traversal selection authority。 + +## Working Product Topology + +```text +feature / semantic / exact selection + | + v + focal graph entity + | + v + graph-navigation retrieval + | + v + bounded existing graph result + | + +--> GraphSurface / future ListSurface + +--> Agent or another application capability +``` + +这张图只固定 owner 关系与前后位置，不冻结请求字段、遍历算法或 UI。 + +### InfoBase Graph view pressure + +client-web `InfoBase Graph view` 是本 unit 的明确应用场景。当前实现以 `Block.getAll() + Relation.getAll()` 构造 +全图并在 browser 内做 community detection/layout；新的方向是让 scene 只持有由 recall、exact selection、 +one-hop expansion 或 path result 得到的局部 graph。scene 如何累计、dismiss、layout 和 undo 属于 UI state， +不应被持久化为 retrieval authority。 + +直接打开 `overview` 时怎样取得初始 seed 不能从 one-hop contract 自然推出。D-356 修正了 owner：InfoBase View +不能拥有 lexical recall，但可以在 unresolved/404-like state 组合一个外部 recall surface 并消费其 selected Blocks。 +允许的 initialization modes 是 random focal Block、bounded random Block set 与 recall-backed unresolved state； +D-357 已选择 random focal + bounded one-hop 为 default，另两种保持显式模式。明确 +Block/Solved Content route 的 focal 不在 scene 时，仍可执行默认 bounded one-hop。 +`overview` 只表示当前 accumulated scene 的无 focus 全貌。 + +### Approved Graph view UX scope + +D-345 允许在本 unit 激进重做 Graph view 的视觉与交互，而不是只把全量加载替换成 query： + +- node/edge 的信息层级、形状、颜色、方向、label 与 hover/active/new/focal/dimmed states； +- one-hop/path delta 进入 scene 时的 animation、camera fit 与 focus/defocus； +- dynamic graph 的增量稳定 layout，避免每次 expansion 全图跳位； +- community 从 dropdown 选择升级为 canvas 内可见、可进入、可返回 overview 的空间交互； +- Block Inspector 与 solved-content popup 的定位、尺寸、内容层级及其与 canvas focus 的配合； +- responsive/touch/keyboard/accessibility 与 reduced-motion fallback。 + +GraphCon deck 的可迁移原则是：community 是 canvas 上的空间对象；overview 与 focused scene 共用一张 mental +map；focus 通过 camera + contrast + reversible displacement 呈现。不能直接复制其 authored positions，因为 +InKCre scene 会由 recall/expansion/path 运行时增长，community membership 也可能改变。 + +### Working focus-set model + +Graph scene 持有已取得的 Blocks/Relations、稳定 home positions、selection 与瞬时 focus set。focus set 可以来自 +focal one-hop delta、bounded path、multi-selection 或 computed community；它统一驱动 focal/context 对比、内部与 +boundary Relations、可逆 peripheral displacement 以及 smooth camera fit，而不是让每种交互分别实现一套状态机。 + +增量 expansion 的 working sequence 是： + +```text +graph delta + -> merge into scene + -> preserve existing home positions + -> seed new nodes near the focal entity + -> local settle / collision only + -> focus focal + delta + -> camera fit unless user navigation currently owns the camera +``` + +automatic camera 只响应 explicit selection、community enter、expansion、path 等明确动作；manual pan/zoom 暂停 +automatic camera ownership，background changes 不偷走镜头，explicit refocus 才重新启用。reduced-motion 下用即时 +position/contrast change 代替空间旅行。 + +### InkCre UI integration pressure + +本 unit 不允许 Graph view 另造局部视觉语言。UI responsibility 的 working split 是： + +- `../design` / `@inkcre/ui-web` 拥有跨产品语义可复用的 tokens、buttons、loading/empty/error states、tooltip 与 + generic overlay primitives；只有 Graph view 证明的真实通用缺口才下沉。 +- client-web InfoBase Graph view 拥有 Block/Relation rendering、focus set、community hull、scene controls、camera、 + layout 与 navigation-host composition；不能因为这些元素需要统一风格就把 `InkGraphNode` 等 product-specific + 组件塞进 design system。 +- `BlockInspectorPopup`、`SolvedContentPopup` 继续是 InfoBase route destination outlets；内部内容复用 UI + primitives，GraphSurface 只 realize route，不重新接管它们的 shell。 + +已观察到的具体缺口：当前 Graph view 混用旧 `--ink-*` fallback、literal colors/sizes 与 `sys-var`；empty/error +states 未使用 `InkPlaceholder`；node 显示 exact Resolver ID 与 raw content slice；MiniMap/Background 直接硬编码 +颜色。更重要的是，当前 `InkPopup` 无条件创建 full-screen scrim，因此 right-side Block Inspector 会阻断其背后的 +navigation host。design-system preflight 需要判断最小通用修正是给 low-level popup 增加 modeless/no-scrim 能力， +还是已有 primitive 可以无损组合；不能在 Graph view 内复制一套 popup。 + +D-348 已固定 desktop route outlets 为 no-scrim/modeless，narrow-screen Solved Content 可以占满可用区域。新增的 +Product pressure 是：Popup 不应继续充当“canvas node 固定太小，无法承载 Block 内容”的补丁。Graph nodes 应允许 +content-driven、可变的尺寸；但 D-349 已拒绝 inline expand、inline full-content 与 node 内的业务 actions。node +本体始终是 preview surface，Inspector 保留 metadata/Relations/actions，Solved Content 保留 focused reading 与 +未来 Mail reply 等真实业务交互。manual resize 仅是未选择的候选，不因 Vue Flow 支持而自动进入 MVP。 + +即便只采用 intrinsic variable sizing，incremental layout、collision 与 edge anchoring 也必须消费实际 measured +dimensions，而不是当前固定 `200 × 150`/constant collision radius 假设。当前 renderer contract 还存在直接证据： +Twitter `ContentTweet` 在完整 `SolvedContentRenderer` 内以“for graph view”为由截断内容，core `ContentText` 也固定 +截断 100 字；相反 Mail `ContentEmail` 已含 materialize/download/navigation actions，未来还会增加 reply。说明 +preview presentation 与完整、可交互 Solved Content 已被错误地合并。下一项 Technical question 是 Resolver 是否 +应拥有独立、interaction-free 的 preview renderer contract，还是 GraphSurface 能从现有 Block-local projections +无损组合 preview；不能先假设复用完整 renderer。 + +D-350 已关闭这个问题：Resolver registration 拥有 `previewRenderer` 与 `solvedContentRenderer` 两个 presentation +contracts，但二者消费同一个 solved-content authority，不新增 `previewContent` projection。Graph node intrinsic +dimensions 来源于 preview DOM measurement，不由 Resolver 输出 canvas layout hints；preview loading 保持 lazy、 +bounded 且 `materializeMissing=false`。该 preview contract 属于 InfoBase presentation，可由 future List view 复用， +不是 design-system 或 Graph-only abstraction。 + +D-351 进一步固定 retrieval result 为 presentation-free：只返回 persisted Blocks/Relations 与 operation-specific +navigation evidence。label、preview、solved content、node dimensions、focus/layout 以及相对 caller scene 的 delta +均由 InfoBase View 在 merge 后产生；Peer provider 不理解这些字段。 + +## External Research Synthesis + +### Graph-theory problem families + +- **Adjacency / one-hop**：取得 incident edges 与 neighboring vertices；D-344 已确认。 +- **Traversal / reachability**：从一个或多个 anchors 以 BFS/DFS、方向和 depth bound 取得可达部分。 +- **Path**：在已知 source/target 之间判断 reachability、返回一条/多条 shortest/simple path。 +- **Pattern matching**：声明固定、quantified 或 non-linear graph shape，返回所有 bindings/paths。 +- **Analysis**：degree、centrality、community、similarity 等；它们解释 graph 的全局/统计性质，不自动属于 + navigation retrieval。 + +### Primary product/query evidence + +- [Neo4j Bloom scene](https://neo4j.com/docs/bloom-user-guide/current/bloom-visual-tour/bloom-overview/) 只包含用户经 + search/exploration 找到的 graph 部分，而不是默认加载整个 database。 +- [Neo4j/Aura scene interactions](https://neo4j.com/docs/aura/explore/explore-visual-tour/scene-interactions/) + 把 immediate-neighbor expansion、按 relationship type/direction/target type 的 selective expansion、result limit、 + selected-node relationship reveal 与 path exploration 分成不同交互。 +- [Cypher graph patterns](https://neo4j.com/docs/cypher-manual/current/patterns/) 区分 fixed/variable/non-linear + patterns、shortest paths 与 path uniqueness；pattern 是查询 specification，path 是实际匹配结果。 +- [Cypher variable-length path guidance](https://neo4j.com/docs/cypher-manual/current/patterns/variable-length-paths/) + 明确指出宽泛或上界过大的 quantified traversal 会产生巨大 path cardinality，应以有限上界、relationship/node + predicates 与方向在遍历过程中 prune。 +- [SPARQL property paths](https://www.w3.org/TR/sparql11-property-paths/) 将 sequence、inverse、alternative 与 + repetition 组合为 predicate-path expression，也指出 unanchored path 会搜索全图并产生大量结果。 +- [NetworkX traversal reference](https://networkx.org/documentation/stable/reference/algorithms/traversal.html) 将 + bounded BFS/DFS、distance layers 与 edge traversal 作为不同图论 primitives；这些算法存在不代表都应成为 + InKCre MVP 产品能力。 + +### Current working judgment + +1. D-344 one-hop expansion 是 Graph view 与 Agent 都需要的 atomic primitive。 +2. unrestricted N-hop/ego graph 只是把 fan-out 风险藏进 `depth`，不应成为第二项 MVP primitive。 +3. “连接两个已知 Blocks”有独立用户意图，并天然要求返回 path evidence；bounded shortest-by-hop path 是可解释 + 的最低机制，但 hub shortcut 与 filter semantics 仍需讨论。 +4. fixed/quantified graph pattern matching 很强，但会迅速要求 node predicate、Relation content grammar、variable + binding、path uniqueness 和 result cardinality contract；当前直接实现等同于发明一个小型 Cypher/SPARQL。 +5. centrality/community/recommendation 属于 graph analysis 或 future composite use，不因 Graph view 当前已有 + community detection 就自动进入 graph-navigation retrieval。 + +## Discussion Queue + +Product/Technical discussion is no longer driven by an open-ended list。The next sequence is: + +1. actual-client presentation inventory and narrow visual-state contract； +2. black-box Acceptance contract and authoritative corpus； +3. cross-repo implementation-plan probe with topology/sequence/branch simulation； +4. evidence preflight and calibration； +5. return to Product/Technical review only for a concrete contradiction，then freeze the Execution baseline。 + +This sequence should batch naturally connected findings；it must not manufacture one-at-a-time decisions when the accepted +contract already determines the answer。 + +## Evidence Already Established + +- `RelationManager.get()` 只按一个 Block、direct in/out 与 exact content 查询；它没有 depth、path、result bound + 或 graph-shaped result contract。 +- `Resolver.get_relations()` 缓存并消费相同 direct relations，owner 是 focal Block interpretation。 +- `RelationManager.get_text()` 用两端 Block-local labels 投影一个 directed dynamic property；它证明 Relation + 本身具有可应用语义，但不是 traversal/query implementation。 +- client-web GraphSurface 当前加载全量 Blocks/Relations；这是 visualization baseline 与 scaling/use pressure， + 不是 graph-navigation retrieval 的现成实现。 +- `BlockManager.iterate_from_block()` 不是可保留的 traversal baseline：它只跟随 outgoing Relations、以一个全局 + mutable `depth` 穿过递归分支、没有 visited/cycle guard、没有 per-hop/result bound 或 deterministic ordering， + 并只返回 ID sets。它应作为失败证据 hard-cut，而不是包装成新的 manager。 +- producer `GraphForm` 是 flat graph write command，负 ID 表达同批新实体 references；它不应被复用于 read + result。Graph-navigation response 必须返回 persisted `BlockModel`/`RelationModel` authority，并另外表达 operation- + specific evidence（例如 ordered path 或 continuation）。 +- semantic retrieval 已建立 domain manager local/delegated split、exact capability、fixed inbound 和 validated + Block/Relation DTO 的实现模式；graph navigation 可以复用该 Peer topology，但不能复用 vector-ranking payload。 + +### Working one-hop contract + +- Block expansion returns the focal Block、a bounded deterministic page of incident Relations and every opposite endpoint + Block；Relation expansion returns the addressed Relation and both endpoint Blocks。Every returned Relation therefore has + both endpoints in the returned graph result。 +- Block expansion uses explicit `in` / `out` / `both` direction，default `both`，default limit 20 / hard maximum 100，and + stable Relation-ID-desc ordering with an exclusive nullable `next_cursor`。ID is ordering/continuation identity，not a + claim of semantic recency。A page fetches `limit + 1` to distinguish complete from truncated results。 +- Result does not need `frontier` or `new/existing` flags：all returned entities are addressable next steps，while caller + scene identity merge determines which are new。 +- Missing addressed entity is a meaningful not-found failure；an existing isolated Block succeeds with the focal Block and + no Relations。Concurrent mutation remains best-effort，but one response must be endpoint-closed。 +- An optional exact set of Relation `content` values has low implementation cost and supports selective expansion without + inventing prefix/regex/JSONPath semantics over arbitrary string/JSON content。D-352 accepted this exact-content-only + filter and the rest of the proposed contract。 + +### Working bounded-path contract + +- `find_path(from Block, to Block)` returns at most one shortest-by-hop path under explicit bounds；it does not rank + semantic relevance、return all equal paths or hide a general pattern query。Default traversal direction is `both` while + retaining each Relation's persisted direction in evidence；caller may restrict to `out` or `in` and reuse exact content + filtering。 +- Working bounds are default `max_hops=4` / hard maximum 8 plus an explored-graph budget，tentatively default 1,000 Blocks / + hard maximum 10,000。Hop bound alone cannot protect against one high-degree hub。Exact numbers remain preflight-tunable， + but both dimensions are part of the public boundedness model。 +- Result status is `found`、`not_found` within the requested bounds or `limit_reached` before exhaustive bounded search。 + Budget exhaustion is not silently reported as no path。Only a found result returns the final path graph；search working + sets do not leak into InfoBase View。A source equal to target succeeds with one Block and zero Relations。 +- Ordered evidence is a Block-ID sequence plus Relation-ID sequence with `len(blocks) = len(relations) + 1`，backed by an + endpoint-closed persisted graph result。Actual Relation rows reveal whether a traversal step followed or opposed the + stored direction；no duplicate orientation flag is needed。 +- Do not special-case Source/Mailbox or penalize high-degree nodes in MVP。That would be a graph-ranking policy rather than + path retrieval。Direction、exact content filters and search bounds are the honest controls；future ranked/alternative path + capabilities can compose later。 +- Technical hypothesis：application-owned bounded bidirectional BFS issues batched frontier queries through + RelationManager/graph query helpers。Do not put traversal business logic into a PostgreSQL RPC or retain the broken + `BlockManager.iterate_from_block()` recursion。Required `relations.from_` / `relations.to_` indexes belong to the migration + preflight。 + +### Working manager / peer topology + +```text +client-web InfoBase View + -> @inkcre/core GraphNavigationRetrievalManager + -> PostgREST Block/Relation fact queries + -> local bounded traversal + +core-py domain consumer + -> core-py GraphNavigationRetrievalManager + -> SQLModel Block/Relation fact queries + -> local bounded traversal +``` + +- Previous working hypothesis mechanically copied semantic retrieval's Peer delegation。Repository evidence disproves that + need：`@inkcre/core` already owns direct PostgREST Block/Relation access，and graph navigation requires no provider-local + model、secret connection、background worker or other asymmetric capability。Forcing client-web through core-py would add + network and availability dependencies without changing graph authority。 +- Working correction：implement the same domain manager contract locally in TypeScript/PostgREST and Python/SQLModel。 + Do not add an exact Peer capability or HTTP inbound in MVP unless another actual consumer cannot access database authority。 + A future remote adapter may be added without changing the manager's domain methods。D-355 accepted this topology and + requires parity proof over one shared behavioral corpus。 +- Working common read shape is `GraphModel { blocks: BlockModel[], relations: RelationModel[] }`，the read-side counterpart + to producer `GraphForm`。It enforces unique persisted IDs and endpoint closure；it is not a database row or a claim that the + subset is the whole info-base graph。D-354 accepted this shape。 +- Operation results wrap that graph with only their evidence：Block expansion has nullable `next_cursor`；Relation expansion + needs no continuation；found path adds ordered Block/Relation IDs，while non-found/limit outcomes retain only addressed + endpoint facts。 +- Python request schema uses `from_` only as keyword-safe implementation spelling and serializes/accepts wire key `from`； + TypeScript and product contracts use `from` / `to` directly。 + +### Working initial-scene modes + +```text +open Graph overview + -> mode: random focal -> one Block + bounded one-hop + -> mode: random set -> bounded Blocks as seed focus set + -> mode: unresolved -> compose external recall surface -> selected Blocks as seeds + +open /graph/blocks/{id} or solved-content route + -> if focal Block absent, bounded one-hop seed + -> realize modeless route outlet over that scene +``` + +- Do not auto-load recent Blocks or all graph authority merely to avoid an empty canvas；recency is an arbitrary selection + policy and does not make a graph overview。Random selection is explicit seed policy，not a relevance claim。 +- Recall query/results remain owned by an application/recall component similar to Home。Graph View only consumes selected + Block authority；using the component does not expand the View definition。Clear/remove remain scene-local controls；they + do not delete graph authority。 +- Existing InfoBase List results should gain an application-owned “explore in Graph” destination that opens the Graph Block + route。Do not expand `InfoBaseRouter` into a surface registry merely to express this client-specific navigation。 +- Recall/random seeds can be temporarily disconnected；Relations become visible when returned by expansion/path。Do not add an + induced-subgraph primitive solely to decorate the initial search result。 + +### Working application Recall composition + +```text +App-level Recall launcher (Ctrl/Meta+K) + -> application Recall/Search composition + -> lexical query/result + -> active InfoBase View present/consume + -> otherwise List View fallback + +application Recall/Search + -> composes generic ../design InkSearchBar + -> owns InKCre query/routing behavior outside the design primitive +``` + +- D-358 fixes global launcher、default-List/current-View routing and application component reuse。GraphSurface does not own + recall merely because it can consume selected Blocks in fallback/seed modes。 +- `InkSearchBar` is a genuine promotion：domain-neutral accessible search input/submission/clear/loading presentation。 + It does not register global shortcuts、call retrieval or select an InfoBase View。Home's current raw search input must move + to this visual contract rather than preserving a second local language。 +- D-359 fixes URL `?q=` as the handoff authority。List replaces ranked results；Graph merges/focuses Block seeds；routes that + host an InfoBase View opt in through application metadata。Do not add a synchronized global recall-result store or teach + InfoBaseRouter lexical semantics。 + +### Relation route / Inspector + +D-360 makes Relation an addressable InfoBase route。Graph edge selection and direct Relation routes realize a modeless +`RelationInspectorPopup` over an endpoint-closed scene；List can consume the same route later。Relation Inspector presents +from/content/to plus identity/timestamp and endpoint navigation，but Relation does not gain Resolver or Solved Content。 +Block/Relation Inspectors remain explicit components rather than a speculative EntityInspector abstraction。 + +### Working Graph interaction language + +This remains a review hypothesis，not a confirmed decision： + +- Node preview content remains interaction-free。The node shell owns drag、selection and focus；hover changes visual emphasis + only and never moves the camera。 +- Selecting a Block may open the modeless `BlockInspectorPopup`，but D-361 keeps that Inspector entity-local。Changing focal + entity invokes bounded one-hop exploration from the canvas itself rather than placing expansion actions in Inspectors or + buttons in draggable nodes。 +- The focal-scale hypothesis uses a small number of discrete scene budgets：each scale caps visible incident Relations and + endpoint Blocks，while the camera frames the focal entity and currently admitted direct context。The exploration scale + must remain distinct from free camera zoom so ordinary pan/pinch/scroll does not silently issue retrieval requests。 +- D-362 fixes bounded active-scene replacement over a reusable session cache。Shared entities keep their measured positions； + unrelated visited entities leave visibility instead of accumulating indefinitely。Returning to cached focal context may + reuse retrieval and layout state without turning the cache into a second visible graph authority。 +- D-363 separates activation from inspection。Single activation changes focal；a canvas-level `Inspect` action opens the + entity-local Inspector，with double activation and `Enter` as shortcuts。Inspector close uses router `back()` without + rolling back the already-established Graph scene。 +- D-365 fixes one Graph View-level exploration scale with discrete Relation budgets。Compact/standard/broad begin visual + calibration around `8/20/50` Relations；standard is the default，endpoint closure supplies Blocks，and exact numbers remain + preflight-tunable rather than becoming durable retrieval-contract constants。 +- D-364 removes path discovery from focal-canvas interaction。Application-owned Search may compose the bounded path + primitive as its own operation；Graph may later realize a path result but does not call lexical Recall to manufacture a + second endpoint。`not_found` and `limit_reached` remain retrieval outcomes without implicit retry or query broadening。 + Relation-content filtering remains an exact retrieval contract，but the UI does not need to expose a raw-string control + merely because the API supports it。 +- A client-web scene composable owns loaded entity maps、selection、focus、expansion continuations、layout measurements and + camera intent。It is presentation/runtime state，not an `@inkcre/core` retrieval manager or a new domain manager。 + +### Layout / visual preflight evidence + +- The current Graph view loads `Block.getAll()` and `Relation.getAll()`，runs Louvain community detection，then coordinates + separate all-community MDS and topology-selected force/dagre/circular/radial/grid layouts。It repeatedly calls delayed + `fitView()` after load、community selection、layout selection and force stabilization。 +- The accepted focal one-hop scene has a much stronger topology：every admitted Relation is incident to the focal Block，so + its visible graph is a star/multistar。Community detection and a user-facing generic layout selector do not improve that + scene and should not remain as accidental product concepts merely because the old all-database canvas needed them。 +- Existing radial layout ignores measured node dimensions；existing force collision uses one fixed radius；both conflict with + intrinsic resolver previews。Layout must wait for Vue Flow/DOM measurements and admit variable node bounds rather than + treating every Block as the same circle。 +- Existing Relation Bezier edges share endpoint geometry，so parallel Relations can overlap labels and hit targets。The new + edge presentation needs deterministic sibling lanes plus separate hover/active emphasis。 +- Working direction：hard-cut the current focal-mode community/layout machinery，use a deterministic focal radial layout with + one or more rings chosen by admitted density and measured node bounds，and animate only semantic scene changes。Manual drag + overrides layout within the current session scene key instead of becoming durable Block authority。 +- D-366 confirms that direction。Manual positions are cached per focal scene/exploration scale for the current Graph session， + not persisted globally；parallel Relations receive sibling lanes，and reduced-motion users receive non-animated state + transitions。 + +#### Rejected visual spike — do not use as implementation baseline + +The first static full-screen spike was rejected by Sir and must not be referenced as the desired visual direction。It did +demonstrate structural focal/context、intrinsic node、Relation emphasis and modeless-outlet ideas，but failed the visual-system +contract for more fundamental reasons： + +- it invented a cold developer-tool palette instead of beginning with InkCre's existing color palette、tokens and public + components，violating the established design authority； +- it misread “restrained、cool、professional、sharp” as permission to replace the visual language，rather than criteria for + refining the existing language； +- it exposed implementation/debug vocabulary (`focal_block`、direction status、modeless explanation) and redundant eyebrow/ + status copy that users did not need，making the interface explain its architecture instead of serving the interaction； +- it widened the experiment into invented app chrome、toolbar and copy，so state validation became a noisy redesign。 + +Corrective preflight must first inventory representative current client surfaces and the design repo's actual palette、type、 +spacing、radius、icon and component grammar。A later study must be narrow—Node/Relation/focal/context/outlet states inside the +existing shell—with no explanatory/debug copy and no new token unless an observed generic gap earns it。 + +### Route-state conflict discovered by preflight + +The current concrete `/info-base/graph/blocks/:block` route is read as the Graph focal Block and simultaneously mapped to +`InfoBaseRoute {name: "block"}`，which immediately mounts `BlockInspectorPopup`。After D-363，scene focal navigation and an +Inspector outlet are distinct states and cannot continue sharing one route authority。The route topology must preserve +surface-owned focal history while letting `InfoBaseRouter` realize an optional entity-local outlet over that scene。 + +D-367 resolves the conflict：Graph focal uses one role-named query reference (`focal_block` or `focal_relation`) on a stable Graph page path，while +concrete Block/Relation/Solved Content outlet paths preserve that query。Only focal identity is reconstructive URL authority； +scale、layout、camera、cursor and cache remain runtime scene projections。 + +### Resolver-preview loading pressure + +- Current client Resolver exposes lazy cached `getSolvedContent()` and a presentation-neutral `solvedContentRenderer`，but no + preview renderer yet。Current Graph bypasses this contract and truncates `Block.content` directly。 +- A scale-bounded one-hop `GraphModel` is structural evidence，not a declaration that every focal or neighbor Relation has + been loaded。Passing its Relation subset into Resolver as a complete relation cache would corrupt solved-content behavior。 + Graph preview Resolver instances must retain their own complete-context loading semantics。 +- Working loading model：render structural Node shells immediately；resolve the focal first，then proactively resolve all + admitted neighbors through a small concurrency pool and shared Resolver cache。Cancel queued work when a scene leaves，but + do not add hidden retries or treat preview failure as graph-navigation failure。 +- Resolver preview completion changes intrinsic dimensions。Batch `ResizeObserver` measurements per animation frame and + recompute the deterministic scene layout；do not issue a camera fit for every Node completion。The shell has bounded min/max + dimensions so progressive content does not make the canvas unbounded。 +- D-368 confirms this Graph-specific orchestration without turning it into a shared List/Graph wrapper。Only the Resolver + renderer/solved-content contract is shared。Graph retrieval itself is described as returning bounded structural evidence； + navigability and loading speed remain consumer objectives rather than retrieval ownership claims。 +- D-369 makes Graph direction a soft visual emphasis over the same scale-bounded `both` scene。Inactive-direction Relations + and their otherwise-unemphasized endpoints remain clickable but dimmed；changing emphasis does not retrieve、relayout、fit + camera or fork cache identity。Hard `in/out/both` remains available to other one-hop API consumers。 +- D-370 keeps focal and context Node sizes intrinsic。Focus is established primarily through restrained hairline/optional + halo、crisper incident Relations and reduced context contrast；elevation and z-index are not accumulated as ceremonial + state signals。The target visual character is restrained、cool、professional and sharp，with exact tokens left to visual + prototyping instead of frozen in prose。 +- D-371 names role-specific `get_block_neighborhood` / `get_relation_neighborhood` manager methods and keeps UI `expand` + outside the retrieval vocabulary。The public split exposes two coherent semantic operations while hiding endpoint closure、 + pagination and query mechanics；deep-interface quality is clarity and low caller knowledge，not minimal method count。 +- D-372 fixes `find_path` as a discriminated `PathFound | PathNotFound | PathLimitReached` union。Only success carries final + GraphModel and ordered Block/Relation paths；negative outcomes expose no working graph and cannot form nullable-field + contradictions。 +- D-373 keeps random-focal policy in Graph initialization and adds a singular random-row primitive to existing peer-native + Block access (`BlockManager.get_random` / `Block.getRandom`)。Both use count + stable-order random offset；no all-Block + browser transfer、Graph-retrieval method or database RPC is introduced。 +- D-374 removes a public tie-break among equal shortest paths。Stable ID iteration may remain an implementation detail； + Acceptance asserts exact paths only for unique-shortest graphs and otherwise validates semantic path properties，preventing + UI or fixture convenience from promoting database identity into path-ranking authority。 +- D-375 puts `Find path` in application-owned Search。Search selects exact/lexically suggested endpoint Blocks and hands off + `path_from` + `path_to` as reconstructive Graph query state；Graph realizes the path outcome but owns neither the operation + nor endpoint picking。Focal、path and `q` seed address forms remain mutually exclusive。 +- D-376 fixes best-effort concurrent-read semantics：successful GraphModels stay endpoint-closed without a cross-statement + snapshot claim。Neighborhood omits now-unresolvable Relations；path assembly reuses traversal-observed Relations and maps + an endpoint-closure race to ordinary `not_found`，with no hidden retry or caller-visible internal inconsistency。 + +D-353 accepted this contract and corrected public endpoint names to `from` / `to`，matching Relation direction and avoiding +ambiguity with the upstream `Source` domain。Python-only syntax may use `from_`/aliases without changing the public term。 + +## Confirmed Decision References + +- D-073/D-074：Resolver application-facing interpretation、controlled lazy materialization 与 effect vocabulary。 +- D-075/D-196：exact Resolver、`get_text`/`get_label` 与 Block-local label boundary。 +- D-078：semantic retrieval 返回持久 Blocks/Relations，不引入 transient chunks。 +- D-094：Relation 作为 directed dynamic property 的 semantic projection。 +- D-343：本 unit 选择以及与 Resolver interpretation 的明确区分。 +- D-344：one addressed graph entity / one-hop atomic navigation primitive。 +- D-345：bounded Block connection 与 aggressive Graph-view UI/UX scope。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/presentation-preflight.md b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/presentation-preflight.md new file mode 100644 index 0000000..53ce1b0 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/graph-navigation-retrieval/presentation-preflight.md @@ -0,0 +1,68 @@ +# Graph Navigation Presentation Preflight + +## Purpose + +Record evidence from the actual client-web and `@inkcre/ui-web` owners before implementation。This is not a visual spec or +a replacement mock；it bounds the presentation state diff and prevents the rejected full-screen spike from becoming an +implicit implementation reference。 + +## Actual application evidence + +- The real shell is `InkHeader` over one content region with an on-demand right-side `AppSidePanel`。Its visual character is + already sparse、black/white、square-edged and content-led；Graph must not invent another rail、persistent status strip、 + debug vocabulary or explanatory chrome。 +- The current empty Graph page confirms that the application shell itself does not need redesign。Its local italic + `No blocks to display` copy is the defect；the existing `InkPlaceholder` already owns the correct generic empty/error + treatment。 +- Current Graph loads every Block/Relation，runs browser community detection，offers community/layout selectors and repeatedly + schedules `fitView()`。This is a full-graph scene architecture，not a presentation layer that can be retained around the new + bounded focal retrieval。 +- Current Graph/Inspector styles mix `sys-var` with retired `--ink-*` fallbacks and literal colors、spacing、radius and type。 + `BlockNode` mostly uses current tokens but spends elevation、translation and border simultaneously for hover/selection； + `RelationEdge` already has a usable token-based baseline but lacks focal/context/direction and parallel-lane states。 +- The current Graph node reads/truncates persisted `Block.content` directly and displays exact Resolver IDs。This is both a + semantic and presentation defect；the accepted Resolver preview contract replaces it rather than restyling it。 + +## Existing design authority + +- The current token system already provides the required surface、text、border、brand、spacing、radius and restrained + elevation vocabulary。No new color、shadow or graph-specific token is justified by current evidence。 +- `InkButton` covers canvas commands；`InkPlaceholder` covers empty/error states；existing typography/spacing mixins cover + Node、Relation and outlet composition。 +- `InkPopup` always teleports a full-screen scrim。`closeOnScrim=false` only changes dismissal and does not preserve pointer + access to the navigation host。A backward-compatible generic ability to omit the scrim is therefore a proven design-system + gap；Block/Relation Inspector and desktop Solved Content consume it。 +- A domain-neutral `InkSearchBar` remains a valid promotion。The existing List search is a raw local input，while the accepted + application Recall/Search introduces another consumer with the same accessible query/submit/clear/loading presentation。 + Global shortcut、retrieval mode、result routing and InfoBase View selection remain application-owned。 + +## Narrow presentation contract + +- Graph keeps the actual application shell。Its own persistent controls are limited to canvas navigation、exploration scale、 + soft direction emphasis and the contextual Inspect action；only controls justified by those states appear。 +- Block preview is intrinsically sized within scene bounds and uses Resolver `previewRenderer` over the same solved-content + authority as full rendering。It contains no business action、inline expansion、exact Resolver ID or debug state。 +- Focal emphasis does not enlarge a Node。Use the minimum sufficient combination of a crisp border and reduced context + contrast；a halo is optional and only retained if real implementation evidence shows the border alone is insufficient。 + Elevation and z-index are not default state signals。 +- Relation presentation preserves direction、supports deterministic parallel lanes and exposes hover/focal/incident/ + inactive-direction states。Inactive direction remains legible and interactive rather than disabled。 +- Inspector/Solved Content remain independent modeless route outlets over desktop InfoBase Views。They use no scrim，do not + dismiss on outside click and close through `InfoBaseRouter.back()`。Narrow Solved Content may occupy the available viewport。 +- Loading、empty、missing and error states use existing InkCre feedback primitives and concise product copy。No state explains + architecture、query parameters or implementation vocabulary。 + +## Environment observation + +- The client-web local runtime correctly attaches to the core-py-owned development database after the owner runtime reaches a + converged descriptor。The committed/local Portless access projection currently reports an explicit `:1355` URL while the + live Portless proxy serves the hostname on standard HTTPS；this did not change Graph design but must be accounted for in + manual/browser Acceptance setup rather than mistaken for a product failure。 +- A small development-only graph was inserted into the disposable development database for visual/runtime investigation。 + It is not an Acceptance fixture、does not shape implementation and may be removed by the ordinary development reset。 + +## Closed result + +Presentation preflight is complete enough to design Acceptance and implementation sequencing。No replacement full-screen +visual spike is required before implementation；visual verification belongs to the actual client shell during the client-web +implementation loop。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/acceptance.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/acceptance.md new file mode 100644 index 0000000..c6d5e36 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/acceptance.md @@ -0,0 +1,133 @@ +# Mail Extension Acceptance + +- **Status**: Acceptance frozen through D-314 and passed on 2026-08-11。 +- **Purpose**: prove the Mail Extension's observable vertical behavior，not the incidental implementation of schemas、parser + helpers or Adapter internals。 + +## Execution Evidence + +- J1–J3 pass against a real ephemeral Dovecot 2.4.4 and disposable PostgreSQL database using + `tests/extensions/mail/acceptance/test_mail_vertical.py`；the harness owns the `.eml` corpus and provisions source state + only through production database/IMAP boundaries。 +- J4 passes in `client-web/tests/e2e/mail-info-base.spec.ts` against the graph produced by J1–J3，built host/remote assets， + PostgREST and a live core-py Peer。The journey dynamically locates its collected MIME part and does not depend on fixed + production IDs。 +- Core static/unit gate：Ruff clean，Pyrefly 0 diagnostics，`376 passed, 35 skipped`。Client gate：full `pnpm check` passed。 +- No deferred negative-path suite or fake browser Mail Source handler was introduced。 + +## Evidence Authority + +### Blocking real-IMAP harness + +- Start an ephemeral Dovecot instance and interact with it exclusively through real IMAP sockets from outside the server。 +- Install acceptance-owned `.eml` artifacts using IMAP `APPEND`。SMTP is outside this unit：the Mail Source's world begins with + an existing mailbox，so `APPEND` establishes state without replacing any behavior under test。 +- Run the production Mail extension、Source/Job、Resolver、InfoBase and Storage paths against that server。A protocol double or + direct parser call cannot stand in for this gate。 +- Corpus messages must be useful and readable technical material that Sir could reasonably keep，while still deliberately + exercising the accepted MIME、participant、thread and HTML boundaries。Aliases、expected IDs and judgments belong only to + Acceptance and are statically forbidden from production surfaces。 + +### Optional external-provider smoke + +- Use explicit environment-supplied credentials for a dedicated external IMAP account；never embed secrets or depend on a + personal mailbox's uncontrolled contents。 +- Probe real TLS login、mailbox discovery、capability degradation、flags and lazy MIME materialization。 +- Treat network/provider unavailability as unavailable diagnostic evidence rather than a product failure。Any repeatable + standards-compatible discrepancy discovered here must become a Dovecot corpus/journey regression before it can block + ordinary delivery。 + +### Browser journey + +- Playwright opens client-web over the database produced by the real-IMAP collection path；it does not insert a hand-authored + solved DTO to bypass Source/graph/Resolver behavior。 +- Browser assertions cover route realization、focal loading、semantic rendering、navigation、explicit materialization and + untrusted-HTML execution/network effects rather than CSS snapshots。 + +## Test Allocation + +- Prefer static typing、Pydantic schema generation and lint for shape-only facts。 +- Retain narrow unit tests only where a pure algorithm has meaningful branches that the vertical journeys cannot diagnose + economically。 +- Delete the PoC Mail tests that merely prove schema construction/serialization or isolated parser helpers；do not preserve + them as a coverage target。 + +## Blocking Journeys + +### J1 — Ordinary collection becomes a useful、incrementally maintained Mail graph + +1. Before Source creation，the Dovecot mailbox contains one historical message。After Source creation，install a current + reply whose referenced parent is not yet present，with realistic From/To/Cc occurrences、Message-ID/In-Reply-To/ + References、plain + HTML alternatives、mailbox flags and one excluded mailbox copy。 +2. Execute `core.source.collect.v1` through the production Job Handler。The historical occurrence and excluded Mailbox are + absent；current occurrences produce the accepted Source → Mailbox → Email provenance/membership graph、independent body + Blocks、EmailAddress Relations、reply/reference anchors and mailbox-scoped MailFlag graph。 +3. Assert that ordinary default `mark_as_seen` happens only after graph acceptance and is reflected both remotely and in the + graph。Repeat collection and prove the same occurrences reconcile rather than duplicate。 +4. Append the missing parent and mutate the existing occurrence's flags。A later ordinary Job completes the exact-one sparse + reference anchor in place and，when the server advertises the corresponding capability，replaces the old occurrence's + flag snapshot without scanning semantics leaking into Source state。 + +This journey proves initial horizon、ordinary checkpoint、canonical graph production、reference completion、flag authority、 +remote seen action and repeatability together。Exact row counts are asserted only where the product owns uniqueness；the +test does not force Organization-clean graph multiplicity where the accepted model permits benign duplicates。 + +### J2 — Explicit backfill and collection policies remain separate from ordinary synchronization + +1. Run `core.source.backfill.v1` over an exact `[since,before)` range containing the historical message from J1。It is + collected without advancing/regressing the ordinary checkpoint and remains unseen under the default independent backfill + policy。 +2. Repeat the same range and prove reconciliation rather than a second collected occurrence。A valid empty range result is + an ordinary finished no-op。 +3. Materialize an inherited extension mailbox-exclusion default onto a Source，change the extension default and prove the + Source remains unchanged；reset the Source field to null and prove the next validated Mail command materializes the new + default once。Previously excluded graph/checkpoint state is neither deleted nor rewritten。 +4. Exercise prospective remote-removal policy with two occurrences：a removal observed while synchronization is disabled + remains in the graph；after enabling the policy，a newly and reliably observed removal deletes only that occurrence's + membership/flags，not its canonical Email content。No retroactive scan is inferred from enabling the setting。 + +### J3 — MIME metadata is cheap to collect and explicit materialization adds semantic content + +1. Collect a multipart Email containing plain/HTML bodies、a CID inline image and at least one typed attachment。Collection + stores both text representations and MIME metadata/relations but creates no binary semantic child or Storage blob for + non-text parts。 +2. Resolve the Email and each MIME-part read-only。`SolvedEmail` exposes graph-aware bodies、participants、membership、flags、 + references and nullable component content without network materialization；opening/solving the Email does not fetch all + remote parts。 +3. Invoke explicit MIME materialization。The Resolver derives provenance/locator and the effective writable Storage，fetches + the exact `part_id` through the production IMAP Adapter，classifies bytes through the accepted Mail evidence ladder and + adds a semantic media/file Block plus `content` Relation。A second invocation returns usable solved content without + requiring a created/existing status or exact-one graph promise。 +4. After successful materialization，disable/remove remote access and prove the existing child still solves locally without + traversing Source/Storage-routing policy。A different unmaterialized part reports materialization unavailable rather than + guessing another occurrence。 + +### J4 — client-web realizes Mail through generic InfoBase destinations + +1. Playwright opens the collected Email through the accepted GraphSurface Block route，observes graph focus and + `BlockInspectorPopup`，then uses “view solved content” to reach `SolvedContentPopup` without a Mail-specific browsing page。 +2. The Email renderer receives the complete exact Resolver + `SolvedEmail`，presents participants、bodies、mailbox/flags and + MIME metadata，and navigates a reply/reference target through `InfoBaseRouter` so the graph focuses the target Block。 +3. The HTML body is preferred，sanitized and rendered in the isolated iframe。A deliberate script cannot affect the host；a + deliberate remote image/tracker causes no request。A normalized user-clicked HTTP(S) link may leave the app explicitly。 +4. CID content is local only after explicit materialization；an unresolved inline part and attachment present metadata plus + action instead of auto-fetch。The action traverses the same J3 Resolver command and updates the solved projection。 +5. Popup close calls literal back and restores actual Vue/browser history；it does not push a guessed overview or parent + Block route。Direct navigation to a syntactically valid missing Block remains an InfoBase destination with a local missing + state。 + +## Deliberately Unproved Negative Paths + +This unit does not build a focused failure-injection suite for partial Mailbox graph failure、checkpoint CAS loss、post-commit +Seen failure、access-binding mismatch、Cron/Job races、timeout recovery or optional OBJECTID negotiation。Their frozen design +semantics remain implementation constraints，not additional Acceptance gates。Likewise，do not create an acceptance-only +Source handler merely to make client-web claim a collect Job；the generic worker/registration surface remains in implementation +scope and receives static/build/code-path verification，while an unsupported browser IMAP Job is not claimed。 + +## External-Provider Smoke Horizon + +The optional smoke runs a bounded subset of J1/J3 against one dedicated provider account：connect/login、discover Mailboxes、 +collect one known occurrence、observe advertised capability behavior、round-trip one reversible flag mutation and materialize +one known MIME part。It does not delete personal mail、depend on uncontrolled mailbox ordering or assert provider-specific +folder names。A discovered provider discrepancy becomes blocking only after it is understood and reproduced as controlled +Acceptance evidence。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/design-closure.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/design-closure.md new file mode 100644 index 0000000..5b68b58 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/design-closure.md @@ -0,0 +1,61 @@ +# Mail Extension Design Closure Ledger + +> Compact steering surface for finishing design。Decision shards remain the authority；technical-design files explain the +> current contracts。This ledger distinguishes unresolved product judgment from implementation-owned detail。 + +## Frozen Design Surfaces + +| Surface | Status | Authority span | +| --- | --- | --- | +| Delivery scope and user value | Frozen | D-201–D-220：communication-record baseline，ordinary collect/backfill，remote deletion，jobs，lazy attachments，reply facts，generic use surface | +| InfoBase client rendering/navigation | Frozen | D-221–D-238、D-312：SolvedContentRenderer、BlockInspector/Popups、InfoBaseRouter port、GraphSurface realization、route/loading ownership、HTML isolation/privacy | +| Source provenance and Mail graph | Frozen | D-239–D-259：lazy Source anchor，Mailbox scope，Email/body/MIME/address/reply graph，source-native decomposition | +| Email identity and mutable state | Frozen | D-260–D-270：linear reconciliation，exact-one reuse，flags，QRESYNC/CONDSTORE degradation，checkpoint/locator split | +| Mail protocol runtime topology | Frozen | D-271–D-281：no-guess MIME access，Source/Resolver sibling Adapter callers，public protocol config，factory，async scope，Adapter ownership | +| Writable Storage policy | Frozen | D-282–D-285：Source → deployment → PostgreSQL fallback，`sources.storage`，registry-owned `storage_types.writable` | +| MIME durable completion authority | Frozen | D-286：metadata `--content-->` semantic child | + +## Remaining Design Closure + +| ID | Surface | What remains | Human discussion need | +| --- | --- | --- | --- | +| R1 | MIME materialization command | [Approved](technical-design/mime-materialization.md)：existing-child short circuit、singular direct solved content、non-stable singular graph read、concurrent create、failure/refresh | Closed through D-291 | +| R2a | Common Source foundation/config | [Approved](technical-design/runtime-closure.md)：Source/storage policy、global Job/Cron、distributed occurrence materialization、core-py/client-web eligible workers、timeout and no-misfire semantics | Closed through D-306；exact implementation mechanics delegated to plan/preflight | +| R2b | Mail collect/backfill command | [Approved](technical-design/runtime-closure.md)：command forms、exclusion materialization、checkpoint merge、partial failure、remote-action timing and access-context continuity | Closed through D-311；Adapter DTO detail stays preflight-owned | +| R3 | Client-web delivery | [Approved](technical-design/block-rendering.md)：route destination lifecycle、SolvedEmail projection、renderer navigation/materialization actions、HTML isolation and remote-resource policy | Closed through D-312；exact component/library seams stay preflight-owned | +| R4 | Acceptance | [Approved](acceptance.md)：four vertical Dovecot/client-web journeys、acceptance-owned useful corpus and optional external-provider smoke；no focused negative-path suite | Closed through D-314 | +| R5 | Implementation plan/preflight | [Approved plan](implementation-plan.md) + [completed preflight](implementation-preflight.md)：behavior-rewrite slices，database/client-web/shared-contract blast radius，migration/reset strategy，branch simulation、exact MIME Peer command and Impact Handshake draft | Closed through D-315 | + +## Delegated Detail + +- Exact MailAdapter request/result names、pagination/batching and protocol checkpoint serialization are implementation-owned + under D-281 unless they pressure a frozen boundary。 +- No organization、retrieval or generic query increment is assumed。Real Mail Acceptance may reveal one；only that observed + blocker can authorize a minimal horizontal change。 +- Durable PRD/Product TDD、core-py Unit TDD and client-web architecture updates follow implementation evidence and owner + boundaries；the promotion candidates already live in the program documentation ledger。 + +## Adaptive Discussion Batches + +- This is a temporary `mail-extension` design-closure tactic，not a task/program-wide workflow rule and not a precedent that + automatically governs later units。Each later unit chooses its collaboration granularity from its own uncertainty and risk。 +- Batch size follows one coherent dependency/risk closure，not a fixed number of fields or decisions。Closely coupled + config、command、effect and outcome choices should be reviewed together when separating them would repeat context。 +- Escalate only choices that materially change observable product behavior、domain ownership、durable authority、public + contracts、irreversible data effects or a high-cost failure path with more than one credible answer。 +- The main agent derives low-risk consequences、exact names、mechanical validation、ordinary error mapping and other natural + implementation details，records them in the task packet and summarizes them at the batch boundary instead of asking for + item-by-item approval。 +- Implementation evidence may reopen a frozen boundary only when it demonstrates real pressure；the mere existence of + another possible design does not restart discussion。 +- Before escalating any question，apply the program [design-taste filter](../../design-taste.md)。Do not ask Sir to choose a + dominated option merely to keep discussion moving；only credible non-dominated forks survive to human review。 + +## Shortest Closure Path + +1. Close R1 MIME materialization with full input/effect/failure information。 +2. Close R2a Source foundation/config，then R2b Mail collect/backfill；combine only mechanics that have no independent + product/authority judgment。 +3. **Use + Acceptance closure batch（closed）**：R3 is closed through D-312 and R4 through D-314。 +4. **Delivery batch（closed）**：R5 and the derived exact MIME materialization Peer command are closed through D-315。 +5. **Current gate**：perform the final Impact Handshake and wait for Sir's explicit start。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/evidence.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/evidence.md new file mode 100644 index 0000000..57d19d8 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/evidence.md @@ -0,0 +1,193 @@ +# Mail Extension Evidence + +## Historical client-web Source-runtime baseline + +- The preflight `@inkcre/core` exposed persisted `Source`、`SourceType` and legacy `SourceCollectJob` models plus database helpers， + while client-web exposed Source/job management UI。Repository search found no executable Source manager、handler registry + or Source implementation in client-web；the Twitter extension contained Resolver/rendering behavior only。 +- Therefore preflight could not name an existing client-web Source handler。The accepted client-web Job-worker capability was + proven through the production registration contract，while production browser runtime correctly declines IMAP jobs it + cannot transport。This keeps test aliases out of production and does not expand the Mail Source into a browser IMAP + implementation。 + +## Acceptance-server capability coverage + +- Dovecot's own 2.0.19 release notes already discuss fixes to `ENABLE CONDSTORE/QRESYNC`，and the Dovecot imaptest + compatibility matrix lists both CONDSTORE and QRESYNC support。The blocking Dovecot harness can therefore provide real + MODSEQ/flag-delta/VANISHED evidence rather than testing only new-message polling。 +- The same compatibility matrix does not list RFC 8474 OBJECTID support for Dovecot。Adding a second production IMAP server + solely for this optional rung has poor MVP return；a focused socket-level scripted protocol scenario may exercise the + production Adapter's OBJECTID capability/response branch without replacing Dovecot as the vertical authority。 +- Sources：Dovecot [2.0.19 release notes](https://dovecot.org/list/dovecot-news/2012-March/000218.html) and the + Dovecot imaptest [server capability matrix](https://github.com/dovecot/imaptest/wiki/Specs)。 + +> Read-only product/technical evidence retained from the completed Mail unit。Decisions remain in the program decision register。 + +## Attachment Fetch Behavior + +- [IMAP4rev2 RFC 9051](https://www.rfc-editor.org/rfc/rfc9051.html) allows clients to fetch envelope/body structure and + individual MIME body parts through `BODY.PEEK[...]` / `BINARY.PEEK[...]` rather than retrieving the complete message。 +- [Apple Mail account settings](https://support.apple.com/en-euro/guide/mail/cpmlprefacctinfo/mac) state that media + attachments are always downloaded，while other attachment types can be configured as All、Recent or None。 +- [Gmail desktop attachment help](https://support.google.com/mail/answer/30719) presents attachment download as an explicit + action after opening a message。 +- [Gmail Offline help](https://support.google.com/mail/answer/1306849) allows users to disable attachment download to reduce + local storage，and [Gmail Android settings](https://support.google.com/mail/answer/6562) expose automatic attachment + download on Wi-Fi as a setting。 + +### Product inference + +There is no reliable cross-client rule that complete mail synchronization automatically downloads and durably stores every +attachment。Selective/lazy/configured fetching is supported by the protocol and mainstream clients。The current Mail scope +therefore does not need collection-time attachment bytes merely to behave like a credible email client。 + +## Message Identity Facts + +- [Internet Message Format RFC 5322 §3.6.4](https://www.rfc-editor.org/rfc/rfc5322.html#section-3.6.4) defines Message-ID as + the unique identifier for one particular version of one particular message；transport-added trace fields do not normally + change that identity。 +- [IMAP4rev2 RFC 9051 §2.3.1.1](https://www.rfc-editor.org/rfc/rfc9051.html#section-2.3.1.1) defines UID as mailbox-scoped。 + The tuple mailbox name + UIDVALIDITY + UID must refer to one immutable or expunged message on that server and detect UID + regeneration across sessions。 +- [IMAP implementation recommendations RFC 2683 §3.4.4](https://www.rfc-editor.org/rfc/rfc2683.html#section-3.4.4) warns that + UIDVALIDITY does not itself identify a mailbox and UIDs are not unique across mailboxes。 +- [IMAP4rev2 RFC 9051 §5.1.2](https://www.rfc-editor.org/rfc/rfc9051.html#section-5.1.2) allows one authenticated connection + to expose Personal、Other Users' and Shared namespaces。Distinct authentication credentials are therefore not a protocol + proof that their visible mailbox stores are disjoint。 +- [IMAP OBJECTID RFC 8474](https://www.rfc-editor.org/rfc/rfc8474.html) defines optional server-allocated `MAILBOXID` and + `EMAILID` values for servers advertising the `OBJECTID` capability。Base IMAP does not provide an equivalent mandatory + stable account identifier。 + +### Technical inference awaiting decision + +Message-ID is the strongest source-native cross-mailbox/cross-account reconciliation candidate when present。The +server/account instance + mailbox identity + UIDVALIDITY + UID tuple is an exact IMAP remote occurrence locator，not a global +Email identity or a fallback reconciliation key。The current implementation's bare `uid` field cannot locate a message +outside one selected mailbox and UID epoch。 + +A Mail Source can durably identify one configured IMAP access context，not a protocol-proven unique remote account。Different +Sources may expose overlapping mailboxes through aliases、delegation or shared namespaces。Without an optional server-native +identifier such as OBJECTID，cross-Source equality remains best-effort。 + +## Mailbox Discovery and Object-ID Facts + +- [IMAP4rev2 RFC 9051 §7.3.1](https://www.rfc-editor.org/rfc/rfc9051.html#section-7.3.1) defines each `LIST` response using + mailbox attributes、a hierarchy delimiter that may be `NIL`，and a mailbox name。The protocol does not expose one portable + filesystem-like mailbox path field。 +- [IMAP4rev2 RFC 9051 §5.1.2](https://www.rfc-editor.org/rfc/rfc9051.html#section-5.1.2) permits multiple personal、other-user + and shared namespace prefixes and delimiters。Namespace classification is therefore additional discovery context，not the + Mailbox object's universal identity。 +- [IMAP OBJECTID RFC 8474 §4](https://www.rfc-editor.org/rfc/rfc8474.html#section-4) defines `MAILBOXID` as stable across + ordinary mailbox rename and unique only within the mailboxes exposed to one client login on one server hostname。A bare + `MAILBOXID` is not globally comparable。 +- The same RFC requires `SELECT` / `EXAMINE` to return `MAILBOXID` when `OBJECTID` is advertised，so the accepted MVP can + consume the value without one extra query per selected mailbox。 + +### Product/technical inference + +The later collection-value audit supersedes the earlier inference that every stable LIST fact should persist。Canonical +Mailbox calls the protocol field `name`、retains only adapter-understood special-use roles with product meaning and keeps a +nullable bare `mailbox_id` for rename continuity inside its owning Source。LIST delimiter、generic structural/subscription +attributes、message counts and duplicated access scope remain transient Source evidence。Permanent Source scoping plus the +`manages` relation already provides the only comparison scope this unit admits。 + +## Message Content and Envelope Facts + +- [Internet Message Format RFC 5322 §3.6.1](https://www.rfc-editor.org/rfc/rfc5322.html#section-3.6.1) defines `Date` as the + time the creator considered the message complete and ready for delivery，not transport or mailbox-arrival time。 +- [RFC 5322 §3.6](https://www.rfc-editor.org/rfc/rfc5322.html#section-3.6) requires origin date and originator fields for a + conforming message but makes the other header fields syntactically optional。Real clients must still tolerate malformed + or draft messages with missing values。 +- [RFC 5322 §3.6.4](https://www.rfc-editor.org/rfc/rfc5322.html#section-3.6.4) defines Message-ID as identifying one version of + one message；the surrounding angle brackets are syntax rather than part of the semantic identifier。In-Reply-To and + References carry identifiers of other messages and therefore naturally pressure graph relationships/unresolved refs。 +- [IMAP4rev2 RFC 9051 §7.5.2](https://www.rfc-editor.org/rfc/rfc9051.html#section-7.5.2) exposes an `ENVELOPE` parsed from RFC + 5322 headers and a separate `INTERNALDATE` message attribute。The latter belongs to one stored occurrence and must not be + substituted for the message-authored Date fact。 +- [MIME RFC 2046 §5.1.4](https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.4) defines multipart/alternative parts as + representations ordered by increasing faithfulness/preference。Plain text and HTML can therefore be retained together as + authored alternatives rather than collapsing one into the other during collection。 +- [MIME RFC 2046 §5.1](https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1) defines a multipart body as one or more body + parts，each with its own header area and body area；`multipart/mixed` ordering is significant while + `multipart/alternative` means interchangeable representations whose order conveys preference。 +- [Content-Disposition RFC 2183 §2](https://www.rfc-editor.org/rfc/rfc2183.html#section-2) makes disposition optional and + defines `inline` / `attachment` as presentation semantics for a MIME entity/body part。Attachment is therefore not a + standalone media type；filename and disposition remain source-authored metadata about separately typed content。 +- [IMAP4rev2 RFC 9051 §7.5.2](https://www.rfc-editor.org/rfc/rfc9051.html#section-7.5.2) defines `BODYSTRUCTURE` as a + server-parsed MIME structure。A non-multipart part exposes media type/subtype、parameters、Content-ID、description、 + transfer encoding and encoded octet size；extension fields can expose disposition、language and content location。 +- The RFC explicitly defines BODYSTRUCTURE body size as transfer-encoded octets。`BINARY.SIZE[section]` is the separate + decoded size returned only through the corresponding decoded-fetch capability。A canonical pre-download metadata field + must therefore not claim to be actual semantic-content byte size。 +- [IMAP4rev2 RFC 9051 §6.4.5.1](https://www.rfc-editor.org/rfc/rfc9051.html#section-6.4.5.1) defines `section` as positional + part specifiers assigned from MIME occurrence order。A canonical `part_id` such as `2.1` therefore has stable structural + meaning relative to one exact Email/MIME tree：it identifies、orders and remotely locates that part。It is not global + content identity，so its natural owner is the Email → component Relation rather than intrinsic MIME-part Block content。 +- [MIME RFC 2045 §8](https://www.rfc-editor.org/rfc/rfc2045.html#section-8) defines Content-Description specifically as + optional descriptive information for a body（for example，a human description of an image）。It is ordinary authored + semantic metadata and can support label/text retrieval even before bytes are materialized。 +- RFC 2045 calls a Content-Type value a `media type` and defines it through type/subtype identifiers；IMAP BODYSTRUCTURE + returns those as separate `body type` and `body subtype` strings。Canonical `media_type = "type/subtype"` is therefore + standards-aligned terminology，though not one literal IMAP response field name。 +- [CID URL RFC 2392 §2](https://www.rfc-editor.org/rfc/rfc2392.html#section-2) defines Content-ID as a body-part identifier used + by `cid:` references，while [MHTML RFC 2557 §4.2](https://www.rfc-editor.org/rfc/rfc2557.html#section-4.2) defines + Content-Location as another body-part label。The label belongs to the MIME-part metadata；a resolved HTML-body occurrence + referring to that label is a distinct contextual graph edge。 + +### Current implementation evidence + +- `extensions/mail/schema.py` places mailbox-scoped `uid` and derived `has_attachments` in Email root content。 +- `extensions/mail/imap.py` fetches full RFC822 bytes，uses only the first matching plain/HTML part，skips a message without + both From and To，and substitutes local `datetime.now()` when Date is absent or invalid。 +- These behaviors are PoC evidence，not accepted contracts：the new collection boundary already places UID in membership、 + attachments in graph metadata and Block timestamps solely in InKCre persistence lifecycle。 + +## Address Identity and Display-Name Facts + +- [Internet Message Format RFC 5322 §3.4](https://www.rfc-editor.org/rfc/rfc5322.html#section-3.4) models a mailbox as an + addr-spec optionally accompanied by a display name。The display name is presentation supplied in that address-field + occurrence；the same addr-spec may appear without it or with another phrase in another message。 +- [SMTP RFC 5321 §2.4](https://www.rfc-editor.org/rfc/rfc5321.html#section-2.4) requires preserving mailbox local-part case， + even while discouraging servers from exploiting case sensitivity；mailbox domains follow case-insensitive DNS rules。 +- [SMTPUTF8 RFC 6531 §3.2](https://www.rfc-editor.org/rfc/rfc6531.html#section-3.2) permits UTF-8 local parts and requires + internationalized DNS names to use a Unicode-aware resolver or A-label transformation。Canonical identity must therefore + preserve local-part Unicode/case while choosing one domain representation。 +- Provider-specific equivalences such as plus-address removal or dot folding are not protocol identity rules and cannot be + applied by a generic Mail adapter without explicit provider authority。 + +### Current implementation evidence + +- `extensions/mail/schema.py` lowercases the complete addr-spec and stores one display name on the EmailAddress Block。 +- `EmailAddressResolver.get_existing()` reconciles solely by that lowercased value，so independent messages can silently + compete for a contextual name while protocol-significant local-part case has already been erased。 + +## Remote Mail State and Flag Facts + +- [IMAP4rev2 RFC 9051 §2.3.2](https://www.rfc-editor.org/rfc/rfc9051.html#section-2.3.2) defines flags as a mutable list on a + message in IMAP mailbox context。System flags are Seen、Answered、Flagged、Deleted and Draft；Recent is deprecated。 +- The same section distinguishes system flags from server-defined keywords and standard `$...` keywords，but both remain + members of the same FLAGS attribute and both may be added/removed。Different display/action semantics do not create a + different protocol persistence category。 +- [RFC 9051 §6.4.6](https://www.rfc-editor.org/rfc/rfc9051.html#section-6.4.6) changes flags through STORE against message + sequence/UIDs in the selected mailbox。If a local model reconciles several remote occurrences into one Email，global + unscoped Email → Flag edges erase the exact operation scope。D-262 instead scopes each MailFlag through its owning + Mailbox and permits only one live Mailbox/Email occurrence，so a plain tag edge derives one exact UID locator without + copying it into Relation content。 +- The same RFC defines `\Deleted` as “marked for removal by later EXPUNGE”，so it is still a flag while membership exists； + actual EXPUNGE is the separate evidence that removes the occurrence。`\Recent` is deprecated and session-derived，so it + does not earn durable graph persistence。 +- Base IMAP exposes flag names、current FLAGS and mailbox PERMANENTFLAGS，but no per-flag description field。A persisted + MailFlag description can still have the same retrieval/interpretation value as MIME Content-Description，while its + authority must be documented as standards/provider/adapter semantic metadata rather than a wire-returned IMAP fact。 +- A FETCH `FLAGS` data item is the message's current complete flag list，while STORE supports replace/add/remove forms。 + Therefore a collected full FLAGS response is replacement authority for that occurrence's local `tags` set，not merely + another append event。Discovering which previously collected occurrences changed is a separate synchronization problem。 +- [CONDSTORE/QRESYNC RFC 7162](https://www.rfc-editor.org/rfc/rfc7162.html) defines HIGHESTMODSEQ as a mailbox sync checkpoint + whose validity depends on the mailbox UIDVALIDITY。QRESYNC can return changed FLAGS and VANISHED UIDs；CONDSTORE alone can + fetch CHANGEDSINCE metadata but still requires UID FETCH/SEARCH to discover expunges。An empty Mailbox has no occurrence + Relation from which a prior synchronization epoch can be recovered，so checkpoint placement cannot be hand-waved as a + duplicate occurrence locator。 +- A Relation can own occurrence fields but cannot itself be the endpoint of Mail Flag Relations。This is topology evidence， + not proof that an occurrence must become an association Block。A Mailbox-scoped MailFlag plus unique Mailbox/Email + membership preserves exact scope without a MailOccurrence Block、locator-qualified tag or global one-Email-per-locator + rule。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-plan.md new file mode 100644 index 0000000..b64df9d --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-plan.md @@ -0,0 +1,206 @@ +# Mail Extension Implementation Plan + +## Execution Status — 2026-08-11 + +- Slices 0–6 are implemented and accepted。Global Job/Cron、Source anchor/writable-Storage policy、protocol-neutral IMAP + Mail collection/reconciliation、exact MIME materialization Peer capability、generic InfoBase routes/popups and the Mail + remote have passed their full static/type/build repositories gates。 +- A disposable PostgreSQL baseline passed fresh head upgrade、head→previous→head and DB-owned Job lifecycle timestamp + checks。The published migration digest is + `8eda54520a7099c957ee6a1b5e6e48d5ee2b3dbe18b0dea4b6a5246dbef04bd2`。A full downgrade through the oldest historical + revision still exposes that revision's pre-existing enum-cleanup defect；the accepted unpublished-schema gate remains fresh + baseline plus head↔previous rather than expanding this unit into historical migration repair。 +- The repeatable blocking harness starts a real Dovecot 2.4.4 built under WorkSSD，installs only acceptance-owned `.eml` + artifacts through IMAP APPEND and passes J1–J3 through production Source/Job/InfoBase/Resolver/Storage paths。 +- J4 passes with the graph produced by J1–J3 through built client-web、built Mail remote、PostgREST、core-py Peer HTTP and a + real browser。It exposed and fixed an iframe browsing-context history bug rather than weakening the literal-back contract。 +- Slice 7 is complete：code/local docs are committed per Spoke，Hub PR #15 is merged as `067c60a`，and each Spoke consumes + that published truth through a pure shared-ref commit。Post-bump full repository gates pass。 + +- **Status**: Complete；R5 execution、J1–J4、durable promotion and owner-separated delivery passed。 +- **Authority**: D-201–D-315、the frozen technical-design files in this unit and the four blocking journeys in + [Acceptance](acceptance.md)。 +- **Delivery shape**: retain the `mail-extension` identity and replace its PoC behavior。This is one implementable unit，not + a release unit；cross-cutting Source、Job/Cron、Storage、Peer and InfoBase changes remain in scope only where the Mail + vertical has already proved their need。 + +## Derived Seams Exposed By Preflight + +### Exact remote MIME materialization capability + +client-web cannot open an IMAP socket，and the accepted Peer architecture removed `Client(rest_api_url).request()`。The only +non-duplicative path from `SolvedContentRenderer` to the Resolver-owned remote materialization command is therefore one exact +request-response Peer capability： + +```text +client-web MailMimePartResolver + -> PeerManager.delegate("extensions.mail.mime_part.materialize.v1") + -> POST /mail/mime-parts/materialize on one live provider Peer + -> provider-local MailMimePartResolver.get_solved_content(materialize_missing=true) + -> content Block + Relation committed + -> caller resolves the returned child Block through its own database/Storage peer +``` + +- Request body carries only the MIME-part metadata `BlockRef`。 +- Success returns the resulting semantic child `BlockModel`，not bytes and not `created/existing` mechanics。The caller then + solves that Block locally，including PostgreSQL binary hydration through PostgREST。 +- The provider inbound calls a non-delegating local path，so recursion is impossible。 +- This is not generic Resolver delegation、`/capabilities/{id}/invoke` or a Job。It is one Mail-owned exact command whose + need is already observable in J3/J4。 +- `ExtensionBase` gains one minimal declarative Peer-inbound hook。`ExtensionManager` registers/unregisters those inbounds + with extension start/close and republishes the current Peer capability snapshot after the matching routes have changed。 + No separate registry service or extension lifecycle is introduced。 + +### IMAP implementation boundary + +- Use `IMAPClient` 3.1.x over the standard-library email parser。It already owns UID operation、SELECT response、recursive + BODYSTRUCTURE、ENVELOPE/FETCH parsing、ENABLE and CONDSTORE modifiers。 +- Keep the synchronous client behind one fresh async-context `IMAPAdapter` per domain command。Blocking calls run outside the + event loop and remain serialized per adapter instance；Job cancellation is best effort under D-306。 +- QRESYNC SELECT is a narrow adapter-internal extension because IMAPClient has no public QRESYNC selector。It may use the + pinned library's low-level command/response seam，but it must continue to use IMAPClient/imaplib parsing rather than + implementing an IMAP wire parser。The Dovecot J1 journey proves this seam black-box。 +- If the server advertises no QRESYNC，the adapter follows the already frozen CONDSTORE then new-occurrence-only degradation； + capability absence is not parser failure or guessed deletion evidence。 + +## Implementation Slices + +### Slice 0 — Restore the already implemented client-web peer/content baseline + +1. Selectively transplant `93090d2`、`c25b0c7` and `3d95b03` from `feat/synchronized-core-v3` onto the current + `feat/organization-git-workflow-phase-6` branch。 +2. Preserve the current governance commits and current `docs/_shared` reference；do not transplant `41137d5` or remove the + completed governance packet。 +3. Verify PostgreSQL binary CRUD/hydration、exact semantic Resolvers、Peer HTTP delegation and removal of legacy Client HTTP + execution before adding Mail changes。 + +### Slice 1 — Evolve the shared PostgreSQL contract + +1. Append a reviewed Alembic revision to the retained chain；do not squash the baseline。 +2. Add `storage_types.writable` and the D-290 constraint-trigger closure for nullable `sources.storage -> storages.id` + (`RESTRICT`)。Registry bootstrap derives writable from the concrete Storage class once。 +3. Add database-owned `sources.created_at` / `updated_at`、nullable unique `sources.block -> blocks.id` and nullable + `sources.storage`；remove `sources.collect_at` and its process-timezone representation。 +4. Extend `sources_types` with independently generated ordinary-collect and nullable backfill parameter schemas。 +5. Replace `sources_collect_jobs` with: + - `job_types(id, description, parameters_schema, default_timeout_seconds)`； + - `jobs(id int8, type, parameters, state, timeout_seconds, status, created_at, started_at, closed_at)`； + - `crons(id int8, schedule, enabled, job_type, job_parameters, job_timeout_seconds, last_job, + last_scheduled_for, created_at, updated_at)`。 +6. Preserve creator direction：Cron references its last Job；Job has no Cron provenance。Use positive-timeout checks、terminal + status/timestamp checks and ordinary FKs，without a retry/attempt table。 +7. Register `core.source.v1` and project the Source deployment config `core.source / core.source.config.v1` with + `default_storage=-4`。Register `core.cron` timezone config with UTC fallback。 +8. Update the database contract profile/catalog/readiness、migration integrity manifest、PostgREST schema artifact and + synchronized client-web generated database types in the same schema wave。 + +### Slice 2 — Build deep Source、Job and Cron runtimes in both peers + +1. Replace `SourceCollectJobManager` with a global `JobManager` whose local Handler Registry owns exact parameter validation、 + `can_handle`、async handling and default timeout projection。Claim remains a single conditional + `pending -> running` update after validation/eligibility；close and timeout recovery conditionally affect only `running`。 +2. Register separate `core.source.collect.v1` and `core.source.backfill.v1` handlers。They resolve the Source row/type，then + validate command config through that Source class；backfill eligibility requires an explicit implementation。 +3. Refactor `SourceBase`/`SourceManager` so setup、ordinary collect and optional backfill schemas are independent。Remove the + obsolete `_organize` obligation and legacy cached schedule setup。Existing RSS/Memos/Twitter/GitHub/Telegram Sources are + mechanically migrated to the new ordinary command without changing their closed product behavior。 +4. Add `SourceManager.ensure_block(source, session)` and `core.source.v1`；the locked Source row creates/reuses one lazy + anchor and refreshes only the `{id,type,nickname}` projection in the caller transaction。 +5. Add Python `CronManager.check()`：application code evaluates the current deployment-timezone minute；a locked Cron row + compares `last_scheduled_for` and terminality of `last_job`，then creates one Job and advances both fields atomically。 + Missed minutes remain missed and run-now bypasses Cron progress。 +6. APScheduler remains only a local wake-up timer for `CronManager.check()` / `JobManager.check()` and unrelated existing + maintenance。It no longer interprets persisted per-Source schedules。 +7. Add equivalent `JobManager`、handler registration、Source runtime registry and conditional claim/close/timeout paths to + `@inkcre/core`。An open client-web starts/stops its worker with application lifecycle；Mail registers no browser Source + implementation，so the browser never claims IMAP Jobs。 +8. Replace source-specific client models/views with generic Job/Cron models and schema-driven Source collect/backfill forms。 + A Source view may present Crons whose typed template references that Source，but Source persistence does not own or point + to them。 + +### Slice 3 — Rewrite Mail collection around one protocol-neutral adapter + +1. Delete the PoC Newsletter Source/Resolver and `/mail/imap` / `/mail/newsletter` creation shortcuts。Retain one Source type + `extensions.mail.source.Source` created through the generic shared-database Source surface。 +2. Add canonical Mail schemas and exact Resolvers: + - `extensions.mail.email.v1`； + - `extensions.mail.mailbox.v1`； + - `extensions.mail.email_address.v1`； + - `extensions.mail.flag.v1`； + - `extensions.mail.mime_part.v1`。 +3. Add `MailProtocol = Literal["imap"]`、typed IMAP parameters and the shallow + `create_mail_adapter(protocol, parameters)` factory。Both Mail Source and remote-I/O MIME Resolver open their own fresh + adapter context；neither calls the other。 +4. Adapter outputs protocol-neutral mailbox、message/header、participant、MIME-tree、flag、change/removal and exact-part + facts plus typed next-checkpoint proposals。It owns protocol capabilities/checkpoint interpretation but no Source state、 + Block/Relation/GraphForm、Storage or transaction。 +5. Implement source-owned reconciliation in one linear utility-assisted path: + exact local locator -> comparable cross-Source locator -> scoped EMAILID -> Message-ID -> create；every rung is + `zero continue / one reuse / many stop-and-create`，with null completion and non-null contradiction rules unchanged。 +6. Persist the accepted graph in bounded per-occurrence transactions：Source anchor `manages` Mailbox；Mailbox `contains` + Email with UIDVALIDITY/UID；independent text/HTML Blocks；MIME metadata；EmailAddress occurrences；reply/reference anchors； + Mailbox-owned MailFlags and plain `tags`。 +7. Ordinary collect uses Source creation time only for the first horizon，then QRESYNC -> CONDSTORE -> new-only checkpoints。 + It advances a Mailbox checkpoint only across accepted graph facts，merges state without overwriting newer progress and + performs configured Seen mutation after commit。Backfill uses an exact date range and never reads/writes ordinary + checkpoints。 +8. Materialize a null Source exclusion policy once from current extension defaults；later extension changes do not mutate the + Source snapshot。Persist/validate the non-secret access binding before Mail graph effects and reject silent rebind。 + +### Slice 4 — Complete Resolver-owned MIME materialization and Peer delivery + +1. Add `InfoBaseManager.get_related_block(...) -> BlockModel | None` as the non-stable singular graph read frozen by D-291。 +2. Make the Resolver base docstring explicit：solving returns semantic completion，not created/reused/raced mechanics。 +3. Implement `MailMimePartResolver` existing-child short circuit；only absence derives owner Email、eligible exact occurrence、 + live Source binding and effective writable Storage。 +4. Fetch/decode the exact `part_id` through `IMAPAdapter`，classify by declared media type -> byte signature -> + `core.file.v1` fallback，then lock/recheck and atomically write PostgreSQL bytes、semantic child and `content` Relation。 +5. Register/unregister the exact Peer inbound with Mail extension lifecycle。The client-web Resolver delegates only when its + local graph has no child and `materializeMissing` permits creation，then resolves the returned child locally。 + +### Slice 5 — Realize Mail through the generic client-web InfoBase UI + +1. Rename resolver rendering to `SolvedContentRenderer` and pass the complete exact Resolver plus solved content。Move + generic persistence/rumination facts into `BlockInspector`；remove its unused relations prop and embedded solved content。 +2. Add the `InfoBaseRoute` contract and singleton-bound `InfoBaseRouter(current, push, back)` to `@inkcre/core`。client-web + implements it with Vue Router/browser history and the accepted three GraphSurface URLs。 +3. GraphSurface becomes the route realizer and removes local selected-Block authority。Its outlet mounts + `BlockInspectorPopup` or `SolvedContentPopup`，each owning Block load/missing/error and literal-back close；the solved popup + additionally owns Resolver generation guards、refresh and disposal。 +4. Add the client-web Mail extension package/remote。Its Resolvers assemble `SolvedEmail`/`SolvedMimePart` from graph facts； + its Email renderer presents bodies、participants、Mailboxes/flags、references and MIME actions，with all cross-Block + navigation routed through `InfoBaseRouter`。 +5. Sanitize authored Email HTML with DOMPurify，strip passive remote-fetch surfaces and style URL authority，rewrite only + already-materialized CID parts to owned object URLs，and render inside a sandboxed iframe without script/form/same-origin + capability。Only normalized user-initiated HTTP(S) navigation remains active。 + +### Slice 6 — Prove the four accepted journeys + +1. Replace PoC Mail helper/schema tests with `tests/extensions/mail/acceptance/` and a small acceptance-owned `.eml` corpus； + corpus content remains useful professional reading and production code contains no fixture aliases/IDs。 +2. Run a temporary real Dovecot instance on loopback，install the corpus only through IMAP APPEND，and exercise production + Adapter/Source/Job/InfoBase/Resolver/Storage paths for J1–J3。 +3. Run client-web Playwright against the database produced by J1–J3 and the built Mail remote for J4，including route + history、target navigation、explicit materialization、script isolation and zero passive remote-resource requests。 +4. Keep the optional provider smoke environment-gated and diagnostic。Do not add the explicitly deferred negative-path suite + or a test-only browser Source handler。 + +### Slice 7 — Close ownership and remove obsolete surfaces + +1. Remove `CollectAt`、`sources_collect_jobs` models/routes/components、legacy Client execution remnants、PoC Mail schemas/ + parsers and schema/helper-only Mail tests。No compatibility aliases or dual persistence remain。 +2. Update core-py Unit TDD/deployment/security docs and client-web local architecture from implemented truth。 +3. Promote stable business and cross-unit technical truths through the Hub shared-doc workflow，then update each Spoke's + shared reference separately。Do not mix Hub docs、shared-ref bumps and code/local-doc commits。 +4. Reset disposable local/preview/production application data only through the D-195 guarded workflow when required；take + the authorized external WorkSSD dump/digest and Neon recovery branch before production reset。 + +## Verification Ladder + +1. Static boundaries：Ruff/format/Pyrefly，TypeScript/Vue type-check，JSON/Pydantic/Zod schema checks，migration metadata and + generated database contract drift。 +2. Focused implementation checks：Job claim/close algorithms、Cron current-minute materialization、Source-anchor transaction、 + Resolver singular relation query and extension inbound lifecycle。These prove deep common seams，not Mail negative paths。 +3. Core black box：J1–J3 against real Dovecot and a disposable PostgreSQL baseline。 +4. Browser black box：J4 with built client-web + Mail remote + PostgREST + core-py Peer。 +5. Repository gates：full `pdm run check` and client-web `pnpm check` after all vertical journeys pass。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-preflight.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-preflight.md new file mode 100644 index 0000000..c032c3c --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/implementation-preflight.md @@ -0,0 +1,146 @@ +# Mail Extension Implementation Preflight + +- **Status**: Completed and accepted as the R5 execution preflight through D-315；no product code changed。 +- **Date**: 2026-08-10。 + +## Repository And Branch Evidence + +- core-py is on `feat/synchronized-client-v3-restacked` at `9dea682`。Its dirty worktree is confined to the active task + packet；`git diff --check` is clean。 +- client-web is on `feat/organization-git-workflow-phase-6` at `bd08c2d` with a clean worktree。 +- Required preceding client implementation is not missing：it exists on `feat/synchronized-core-v3` as + `93090d2`、`c25b0c7` and `3d95b03`。A temporary local clone cherry-picked those three commits in order with zero conflicts。 + The later `41137d5` shared-ref commit is deliberately excluded so current governance/shared truth remains authoritative。 +- Current untouched baselines pass：core-py `385 passed, 34 skipped`；client-web full `pnpm type-check` passes workspace、 + runtime、database-contract and every package/application type-check。 + +## Codebase Consequences Confirmed + +- Current `SourceBase` couples collection to `SourceCollectJobModel`、requires obsolete `_organize` and installs + `collect_at` through process-local APScheduler。`SourceCollectJobManager` has a hard-coded five-minute timeout and only a + Python execution path。This is a behavior-rewrite boundary，not a safe incremental Mail patch。 +- Current Mail fetches full RFC822 messages，stores UID/attachment summary in Email root，swallows failures and retains a + separate Newsletter Source。It is requirements/failure evidence only。 +- PostgreSQL binary Storage and exact core semantic Resolvers already exist in core-py。Their client-web peer implementation + is exactly the code recovered in Slice 0。 +- `storage_types` does not yet project `writable`；`sources` lacks Storage/Block/timestamps；Source types expose setup schema + only；database contract/profile/readiness and client generated types all name `sources_collect_jobs`。The schema wave must be + atomic across these owners。 +- Existing `InfoBaseManager.submit_graph(GraphForm)` supports signed local Block IDs and arbitrary edges，while + reconciliation/update paths still need Mail-owned queries and ordinary Managers。The plan does not force update/reconcile + behavior through an insert-only GraphForm。 +- Current extension runtime can hot-add/remove FastAPI routes，but it has no extension-owned Peer-inbound hook and does not + republish capabilities on dynamic start/close。That is the only common extension-runtime change required by Mail + materialization。 +- client-web has no executable Source/Job registry，uses source-specific Job/UI models and still keeps solved rendering inside + `BlockDetailsPanel`。The accepted InfoBase UI work is therefore a real refactor rather than a component rename。 + +## IMAP Library And Protocol Evidence + +- IMAPClient 3.1.x is the selected mature client。Its public code supports UID operations、parsed SELECT response including + UIDVALIDITY/HIGHESTMODSEQ、recursive BODYSTRUCTURE、ENABLE and FETCH modifiers such as CHANGEDSINCE。 +- It has no public QRESYNC SELECT API；aioimaplib exposes no comparable QRESYNC/CONDSTORE/BODYSTRUCTURE support。A thin pinned + adapter-internal QRESYNC extension is lower risk than replacing mature UID/MIME parsing or implementing a protocol parser。 +- Dovecot supports CONDSTORE/QRESYNC but not RFC OBJECTID。Therefore J1 can prove the high-value synchronization path；the + optional provider smoke observes OBJECTID when available，and D-314 correctly avoids a bespoke fake server merely for that + optional rung。 + +## Runtime And Tooling Evidence + +- The repository physically lives under `/Volumes/WorkSSD/Development` through the user's Development symlink；generated + corpus、PostgreSQL cluster、Dovecot state and browser acceptance artifacts can remain on WorkSSD。 +- No Docker CLI/runtime is currently available。The Acceptance contract requires ephemeral Dovecot，not a container。The + smallest local harness is Homebrew Dovecot 2.4.4 plus a process-owned loopback config/maildir under the WorkSSD workspace。 + Local PostgreSQL 17 is installed but stopped；PostgREST 16 and Dovecot are available as bottled packages but not installed。 + Their package footprint is small；all material test data remains on WorkSSD。 +- The shell has PDM 2.28.0 while repository/CI deliberately pin 2.27.0，so `pdm run doctor` alone reports that mismatch。 + Implementation must use/restore the pinned 2.27.0 toolchain rather than weakening the repository check。Ordinary tests and + lock verification currently run successfully under the installed environment。 +- The retained Alembic strategy is already frozen by D-195：append reviewable revisions，then reset disposable databases from + the full chain。Do not rewrite the migration baseline or retain `collect_at` compatibility merely because current local rows + exist。 + +## Branch And Failure Simulation + +| Branch | Expected action | Bounded outcome | +| --- | --- | --- | +| existing MIME child | solve locally before Source/Storage routing | no IMAP、no Peer dispatch、no new bytes | +| browser missing child | exact Mail Peer delegation | provider materializes；browser reloads child from shared DB | +| no eligible Mail provider | delegation unavailable surfaced by explicit action | existing graph unchanged | +| QRESYNC | changed flags + VANISHED + new UIDs under one typed checkpoint | reliable prospective deletion when enabled | +| CONDSTORE only | CHANGEDSINCE flags + new UIDs | no deletion inference | +| base UID only | new occurrence progression | no flag/deletion completeness claim | +| ordinary first run | filter transient INTERNALDATE against `sources.created_at` | pre-setup history skipped without persisting INTERNALDATE | +| backfill | exact `[since,before)` scan | ordinary checkpoint untouched | +| source storage absent | deployment default，then `-4` | ordinary fallback | +| source/deployment storage explicitly invalid | configuration error | no hidden fallback | +| duplicate MIME materialization race | lock/recheck reduces duplicate；any child is usable | no uniqueness/stability API | +| multiple capable Job workers | eligibility before conditional claim | one runner；losers leave row alone | +| multiple Cron checkers | current-minute evaluation + locked Cron transaction | one Job/current occurrence，no overlap/misfire debt | + +## Impact Handshake Draft + +### Address and Object + +- core-py：`app/schemas/source`、new Job/Cron schemas/business/routes、Source/Storage/InfoBase/Extension runtime、 + `extensions/mail`、database contract/migrations and Mail acceptance assets。 +- client-web：three recovered prerequisite commits，`packages/core` Source/Job/Cron/Peer/InfoBase contracts，client routes/ + popups/views，new `extensions/mail` remote，generated database types and browser acceptance。 +- durable docs：core-py local Unit TDD/deployment/security projection，client-web local architecture，then Hub PRD/Product TDD + through the canonical shared-doc workflow。 + +### State Diff + +```text +PoC full-message IMAP + Source-specific jobs/schedules + inline Block content panel + -> typed incremental Mail graph + global Job/Cron + lazy Resolver materialization + + exact Peer command + route-realized solved-content UI +``` + +### Operation + +- Append schema migration and hard-cut obsolete unreleased runtime surfaces。 +- Rewrite Mail behavior around a mature protocol client and canonical graph。 +- Restore already completed client peer/content work，then add the accepted generic Job/InfoBase and Mail UI layers。 +- Add four black-box acceptance journeys；do not add the deferred negative suite。 + +### Blast Radius Forecast + +- High but bounded to two Spokes plus later Hub documentation。Every existing Source implementation must adopt the new + ordinary Job signature；Source UI/routes and database artifacts must move together。RSS/Memos/Twitter/GitHub/Telegram + product behavior and already closed acceptance contracts must remain unchanged。 +- Extension lifecycle changes affect capability publication and hot enable/disable，so current extension runtime tests and + Peer capability snapshots are regression surfaces。 +- InfoBase UI changes affect every solved renderer，including Twitter and all core semantic content renderers；their content + projections remain unchanged while shell/navigation ownership moves。 + +### Invariants Check + +- One authority per fact：Mail root stays small；graph owns participants、bodies、membership、flags、references and MIME + structure；Source state owns only validator/checkpoint；Storage owns bytes。 +- Peer subsystem understands only discovery/delegation，not Mail payload；no generic invoke or delegation Job。 +- Resolver owns graph interpretation/materialization；Adapter owns protocol access only；Source owns collection/state。 +- `refresh`、`materialize_missing`、Job terminality and Cron occurrence retain their frozen independent meanings。 +- No hidden retry、rollback、checkpoint campaign、misfire catch-up、eager attachment download or Mail-specific browse page。 +- No shared docs are edited from `docs/_shared` inside a Spoke，and no commit/push occurs without separate explicit authority。 + +### Verification + +- Static/type/migration/contract checks after each coherent schema/runtime wave。 +- Existing core and client baselines preserved after Slice 0–2。 +- J1–J3 real Dovecot/PostgreSQL black box；J4 built-browser black box；then full repository checks。 +- Database reset/dump workflow proves the exact new migration head and contract revision rather than relying on local rows。 + +### Uncertainty + +- IMAPClient's QRESYNC bridge and non-root Dovecot 2.4 configuration are the only library/environment-specific seams。Both + are isolated behind the Adapter/Acceptance harness and will be proven before Mail collection is considered complete。 +- The exact private helper names、batch size、checkpoint JSON field names、CSS/component filenames and diagnostic strings are + implementation-owned。Evidence that changes domain ownership、observable behavior or the four journeys reopens R5；ordinary + mechanical variation does not。 + +## R5 Conclusion + +Sir accepted the derived exact MIME materialization capability as the first extension-owned Peer delegation。No remaining +evidence requires another product-design round before implementation。The next state transition is the final Impact Handshake +confirmation followed by an explicit `开始`。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/packet.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/packet.md new file mode 100644 index 0000000..f7316e8 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/packet.md @@ -0,0 +1,303 @@ +# Mail Extension + +- **Unit ID**: `mail-extension`。 +- **State**: **Complete — implemented、accepted and promoted 2026-08-11**。 +- **Objective**: 保留 Mail extension identity，把现有 PoC 当作需求与失败证据，先建立可信且尽可能完整的 + communication-record baseline，再让真实邮件场景推动 organization、info-base basic use/query 与 + client-web 的必要演进，使用户能以足够低的成本持有和打理来自多个邮箱的信息。长期目标是让 InKCre + 成为完整的 email client/agent；本轮 delivery scope 仍需单独界定，不把终局愿景一次性塞入实现。 +- **Guardrails**: MVP / MLP 由用户 job、所得价值与可接受代价界定，不以邮件协议完整性、字段数量或 + feature checklist 判定；mail identity reconciliation、reply/reference graph、MIME attachment materialization + 等只是候选 mechanism/pressure，不是预设 gate。InKCre 服务生产与创造，不是只读归档镜像；对 source + 状态的改变可以是有意产品行为，但必须服从清晰、可配置的用户选择。Mail vertical 可以推动 + organization、retrieval 与 client-web，但不借此宣称完成这些 capability trunks。 +- **Verification**: Product、Technical、Acceptance 与 R5 plan/preflight 已冻结。2026-08-11 已用 WorkSSD 上从 + official source 构建的 Dovecot 2.4.4、disposable PostgreSQL 和 acceptance-owned `.eml` corpus 通过 J1–J3； + built client-web + Mail remote + PostgREST + core-py Peer 通过 J4。core-py 通过 Ruff、Pyrefly 与 + `376 passed, 35 skipped`；client-web 完整 `pnpm check` 通过。core-py aggregate gate 只被已定位的 shared-doc + fenced-Python formatting 阻塞，Hub source correction 已纳入本次 owner-separated promotion。 +- **Current Truth**: 旧 PoC Mail behavior 已 hard-cut。当前实现拥有 global typed Job/Cron、Source anchor 与 writable + Storage policy、protocol-neutral IMAP Adapter、ordinary/backfill collection、canonical Mail graph、exact MIME + materialization Peer capability、五个 Mail Resolvers，以及 client-web generic InfoBase route/popup/Mail solved-content + journey。J1–J4 已证明 real protocol → graph → Resolver/Storage → browser，而不是 schema/helper proxy。 +- **Next Step**: 本 unit 已关闭。Hub `067c60a`、core-py implementation `d3cded7`、client-web implementation + `1e69938` 以及两个纯 shared-ref commits（core-py `8e07da8`、client-web `056c265`）已按 owner 分离完成； + post-bump `pdm run check` / `pnpm check` 均通过。Program 返回 unit-selection gate，不在本 packet 内预选下一个 + active unit。 + Adaptive batching 仅是本 unit 收口阶段的临时协作策略，低风险自然推论不再逐项请求批准。 +- **Decision History**: Product scope 已形成完整 candidate；D-263 已冻结 linear Email ladder，D-264 已冻结每一 rung 的 + `zero → continue / one → reuse / many → stop-and-create`，D-265 已冻结 null identity completion 与 non-null + contradiction rejection，D-266 已据此恢复 Message-ID reference anchor 的 locate/reuse-or-create 与 later + completion。Mail identity edge 当前关闭；D-268 已冻结 MailFlag canonical content、description authority、name + normalization 与 observed-FLAGS replacement semantics。D-269 又冻结 scheduled sync 为 QRESYNC → CONDSTORE → + new-occurrence-only，且禁止用 full UID scan 冒充增量 removal sync。当前澄清 occurrence locator UIDVALIDITY 与空 + Mailbox 也需要的 sync-checkpoint UIDVALIDITY 已由 D-270 以不同 scope/lifecycle 分别冻结在 Relation 与 Source + state。D-271 又冻结 remote MIME reconciliation safety：Message-ID-only match 一旦涉及 attachment/inline metadata + 就 lazy-duplicate，只有 exact occurrence/scoped EMAILID 或 sparse anchor completion 可以复用；Resolver 不再通过 + metadata 猜测 UID。D-272/D-273 已冻结单一 Mail Source / protocol-neutral Mail Resolver family 作为 + 同级调用者依赖 Source config 选择的 Mail protocol adapter，Resolver 不调用 operational Source； + IMAP 是当前 concrete adapter，未来 POP3 不产生第二套 Source/Resolver domain。D-274 进一步 + 冻结 adapter 解释 typed protocol checkpoint 并提出 next-state，Mail Source 独自拥有持久 state、推进时机 + 与 accepted-effect boundary。D-275 又明确每个 Source instance 对应一个公开标准 protocol；Source 配置 + 不持久内部/versioned adapter ID，当前也不引入 MailManager/adapter registry/catalog。D-276 已冻结 + `protocol` + typed `parameters` + outer common Mail policy 的 config 形状，并将其与 Peer inbound 的同类经验记为 + U-038，但不把公开 protocol 强行变成 InKCre ID。D-277 又将当前 exact `MailProtocol` 收窄为 + `Literal["imap"]`；已知但未实现的 POP3 不进入当前 config validity。D-278 保留极浅的 + `create_mail_adapter(protocol, parameters)` 共享构造 seam，但不引入 Manager/registry/catalog。D-279 已冻结 + factory 无 I/O、每个 Source collect / Resolver materialization command 使用一个 fresh async-context adapter， + 并将语言原生 resource scope 提炼为 U-039。D-280 已冻结 Adapter 对上暴露 canonical Mail + remote-access/materialization operations 而不是 IMAP primitives，且不生产 graph 或持久 state。D-281 + 又纠正 Adapter `collect()` 草案：`Source.collect()` 独占 collection 语义，Adapter 只提供 canonical Mail + remote read/change/part-fetch operations；exact interface 已委托到 plan/preflight，越过已冻结边界才回讨论。先前的 + [solved-content rendering](technical-design/block-rendering.md) edge 已完成当前设计讨论：`BlockInspector` 与 resolver-selected + `SolvedContentRenderer` 的 ownership 已分开；generic render context 已撤回。`SolvedContentPopup`、InfoBaseRouter 与 + surface route realization 以及完整 Resolver + solved-content renderer props 已确认。GraphSurface 只是当前 realizer， + route 不固化 surface；最小 domain routes 已冻结为 `overview | block | solved-content`。InfoBaseRouter 不建立 + 第二套 history；它通过内部可替换 adapter 使用 Vue Router/browser 这一既有 authority，且 `back` 必须保留真实 + history traversal 语义；MVP public interface 已冻结为 `current + push + back`。下一步冻结 adapter/URL mapping + boundary。GraphSurface application URLs 已冻结为 `/info-base/graph`、`/info-base/graph/blocks/:block` 与 + `/info-base/graph/blocks/:block/content`，并直接投影三种 domain routes。下一步冻结 adapter exact contract、 + initialization 与 malformed/unmapped parameters。InfoBaseRouter 已重新定位为由 `@inkcre/core` 提供 contract + + singleton binding、由各 client 完整实现的 capability port；共享 history adapter/codec 候选已撤回。下一步冻结 + binding/init exact semantics。binding 复用现有 MFImplementation 的 module-scoped set/get/fail-fast 模式，不抽 + generic runtime-binding module。malformed route 由 app not-found 处理；合法 BlockRef 的实体缺失由 surface/view + 加载后呈现，Router 不查数据库。GraphSurface 始终保留 graph；`block` 与 `solved-content` 分别打开 + `BlockInspectorPopup` / `SolvedContentPopup`，popup 自己将 close 解释为 literal back。GraphSurface/ListSurface + 稳定统称 `InfoBaseView` navigation hosts，并通过 route destination outlet 实现 routes。下一步冻结 focus 与 loading + owner。InfoBaseRoute 已冻结为 GraphSurface 唯一 focal-Block authority；GraphSurface 合理理解 stable route.name 并 + 实现 focus/outlet，删除本地 selected-Block identity。两个 popup 只接收 BlockRef 并各自拥有 Block loading/missing + lifecycle；SolvedContentPopup 还拥有 Resolver/solving/refresh/dispose。下一步冻结 route-ref change lifecycle。 + route-ref change lifecycle。Mail identity 已转入 + [mail identity and remote occurrence](technical-design/mail-identity.md)：协议不能保证一 Source 对应一个独立 remote + account，只能保证它代表一套 local IMAP access context。D-262 已恢复 best-effort canonical Email，同时保留 exact occurrence + locator 作为 collection idempotency 与 remote-access authority；未知 locator 可以 best-effort 选择现有 canonical + endpoint。D-263 已冻结 local exact locator → comparable cross-Source exact occurrence → scoped EMAILID → Message-ID + → create 的线性顺序。`Block.id` 仍是 local identity。D-239 已冻结 + MVP `OBJECTID/MAILBOXID` consumption：authentication 后至多一次 CAPABILITY query，MAILBOXID 随既有 + SELECT/EXAMINE 返回；bare value 不跨无法证明 comparable scope 的 Sources 比较。D-240 确立了 Source graph anchor、 + Mailbox identity 在 Mailbox Block、occurrence UIDVALIDITY/UID 在 Email–Mailbox membership 以及 Source state 不持有 + collected-item ledger；其 mandatory Source Block timing 已由 D-245 放宽为 lazy anchor。 + D-241 已冻结 canonical chain 为 `Source --manages--> Mailbox --contains {UIDVALIDITY, UID}--> Email`；直接采集对象 + 使用 `Source --collects--> item`。active direction 是 representational-normalization common pattern，不是 Relation + validation rule。D-242 已冻结 operational Source 删除后保留 Source Block、provenance relations 与 collected graph； + relation 不证明 live credentials/readiness。D-244 经 ROI 复审撤回 D-243 的 shared identity：`sources.id` 保留， + Source Block 使用独立 BlockRef。D-245 随后将 `sources.block` 冻结为 nullable unique FK，只有 producer 首次需要 + provenance endpoint 时才由 SourceManager 并发安全地创建 `core.source.v1` anchor；Source 创建继续是普通单表 + 操作，不引入 `create_source` RPC。D-246 又纠正了 D-244 的 authority inversion：SourceModel 始终拥有 + id/type/nickname 等 Source facts；Block content 的 + `{id,type,nickname}` 只是为 `core.source.v1.get_label/get_text` 与历史可读性服务的 projection，不接管 authority。 + D-247 已冻结 `SourceManager.ensure_block(source, session)`：锁 Source row，在 caller transaction 内创建/复用 anchor + 并同步当前 projection，不另加 refresh flag。D-248 已将 Mailbox 永久定义为 Source-scoped observed Block；不同 + Sources 不合并 Mailboxes；D-262 允许不同 Mailboxes 的 occurrence locators 复用 canonical Email Block，但同一 + Mailbox 内的多个 live UIDs 不合并，也不弱化 Mailbox 的 permanent Source scope。 + D-249 的 Mailbox shape 已由 D-258 按 collection-value audit 收窄：`extensions.mail.mailbox.v1` 只保留 + `name/special_uses/mailbox_id`，删除 transient delimiter/generic attributes 与由 `manages` 重复表达的 access + scope。D-250 将 Email root 收窄为 authored scalars；D-260 加入、D-262 保留 optional server evidence `email_id`， + 形成 `{message_id,email_id,subject,authored_at}`。text/HTML body 变为 semantic content Blocks， + attachment/inline MIME parts 变为 + metadata Blocks 并在 materialize 后指向 semantic content；同时冻结 source-native decomposition / “collect graph， + not just Block” common pattern。D-251 冻结 text/HTML body 直接复用 `core.text.v1` / `core.html.v1`，Email-body 角色 + 只由 Relation 表达；复用只是边界正确性的信号而非拆分理由。D-259 已按 source-order authority 纠正 D-252/D-253： + Email → MIME component 统一使用 `{role,part_id}`，MIME tree path 同时拥有结构位置、顺序与 IMAP fetch locator； + MIME-part Block content 不再保存 part_id/disposition，只保留 + `media_type/charset/filename/content_id/description/transfer_encoding/encoded_size/content_location`。HTML body 对 + Content-ID/Location 的实际引用另建 `{type:"embeds",reference}` Relation。D-254–D-255 冻结纯地址 EmailAddress 与 + `{role,order,display_name}` participant Relations。D-261 在实证比较后撤回 D-260 的 one-Email-per-locator hard cut； + D-262 又以 plain MailFlag Relations 取代 locator-qualified tags：不同 Mailboxes 可以共享 canonical Email，但同一 + Mailbox/Email pair 至多一个 live `contains`，同-Mailbox duplicate 创建另一个 Email Block。D-263 保留既有 canonical + reconciliation ladder 并补入 scoped EMAILID rung；D-264 又冻结 exact-one reuse，D-265 冻结 identity + compatibility 并记录 Source-domain ladder utility 的高 ROI 实现压力。D-266 已恢复 D-256 Message-ID-only + incomplete Email anchor：zero/many 创建普通 sparse Email，exact-one 复用，later collection 走同一 ladder；不新增 + placeholder lifecycle。 + +## Confirmed Product Foundation + +- InKCre 的根本目标不是预先回答每条信息最终产生什么具体知识，而是显著降低收集、整理和基本使用 + info-base 的成本，让信息规模积累有机会产生质变。 +- info-base 持有的是 information；knowledge 只能存在于用户脑内。只有被用户理解并用于价值落地的 + information 才成为 knowledge，PRD 不应混用二者。 +- Mail 相比继续枚举低优先级新 source 具有更高的真实用户价值。Sir 已拥有多个待管理、收集的邮箱，能够为 + collection、organization、query 与 client-web 提供真实 corpus 和产品压力。 +- Mail 的长期产品终点不是只读 collector。InKCre 应尽可能持有完整通信记录，并最终成为完整的 email + client/agent，形成收集、理解、组织、查询与邮件行动的闭环。incoming、sent、archive 等具体纳入范围和 + remote actions 仍由每轮 delivery scope 逐步确定。 +- 已批准的纵切是： + + ```text + real mail sources + → trustworthy collection baseline + → persisted mail graph + resolver/use representation + → mail-demanded organization and info-base query improvements + → necessary client-web journey + ``` + +- 旧实现只提供 evidence，不约束 incrementally patch 还是 behavior rewrite；该选择等待 Technical/preflight + evidence。 + +## Accepted Current Delivery Scope + +- 尽可能完整地持有多个邮箱账号的 communication records，而不是只做 INBOX intake。 +- Source setup 默认不自动收集历史邮件；ordinary collect 只面向 setup boundary 之后产生的信息。用户通过 + 显式、可指定边界的 `backfill` collect 收集历史记录；backfill 是 collect 的特殊 intent，不是平行能力， + 也不复用含混的 legacy `full` 语义。 +- 让邮件通过 resolver/use、邮件 journey 所必需的 minimum basic query 与 client-web surface 被实际使用；这些 + 横向实现不自动关闭完整 feature-retrieval 或 graph-navigation units。 +- 保留并验证可配置 `mark_as_seen`，把它作为本轮有意的 remote Mail action。 +- 暂不实现 compose、reply、send、draft lifecycle 或 agent outbound execution；它们作为同一 Mail ownership + unit 的 future delivery scopes 重新过 gate。 +- 设计可以保留通往完整 email client/agent 的路径，但不得以 future compatibility 为由提前创建泛化 action + framework。 + +## Accepted Access-Context and Mailbox Boundary + +- 一个 Mail Source 表示一套 configured IMAP access context/credentials，用户界面可以把它呈现为一个邮箱账号； + 不按 folder 拆成多个 Sources，也不把多套独立配置隐藏在一个 Source identity 后面。但协议不能证明两个 + Source 看见的 remote account/mailbox namespaces 不重叠，因此 Source identity 不参与跨 Source remote identity + 证明；D-239 是对早期“一 Source 一 protocol account”措辞的技术纠正。 +- 默认追求完整 communication record：纳入 inbox、sent、archive 与用户创建的 folders；drafts、spam、trash + 是默认排除类别。具体 provider 的 label/folder 映射属于 adapter。 +- Mail extension config 持有 deployment-wide 默认排除规则；Source config 可以配置自己的排除规则，其初始 + 默认值来源于 extension config。create-time defaulting、reset-to-default 与 update merge 语义留到 Technical + gate；默认排除不等于永久不支持。 + +## Accepted Remote Deletion Boundary + +- Mail Source 拥有远端删除策略；默认只删除 Email–Mailbox membership relation，不向 InfoBase 提交删除 email + graph 的命令。InfoBase 不理解邮件或自行判断保留。 +- graph 只表达当前已知 membership，不新增 `has been deleted from` tombstone relation。Source sync state 负责记录 + change processing progress；exact relation direction/content 留到 Technical gate。 +- 可选 synchronized deletion 默认关闭，并且只有协议/server 能提供可信增量删除证据时才考虑；不靠周期性 + 全 mailbox traversal/diff 实现。 + +## Accepted Mutable Mail Facts + +- ordinary collection 持续同步 Email–Mailbox membership 与相关 remote state；同步失败不使已收集 Email content + 失效。 +- Mailbox 是独立 Block；membership 由 Relation 表达。move 删除旧 relation 并增加新 relation。 +- `\Seen`、`\Answered`、`\Flagged`、`\Deleted`、`\Draft` 与 observed keywords 都是独立 MailFlag Blocks，通过 + plain `tags` Relations 连接 canonical Email，不进入 Email root content 或 `contains`。owning Mailbox + unique + Mailbox/Email membership derives the exact locator；它们的产品行为不同， + 但共享 IMAP FLAGS/STORE authority 与 persistence shape；deprecated `\Recent` 不持久化。 +- `\Deleted` 存在时仍有 membership；可靠 EXPUNGE 后才删除 exact `contains` 与同 locator 的 flag Relations，不创建 + tombstone。 + +## Accepted Collection Freshness + +- 本轮提供 manual 与 scheduled collection；scheduled collection 通过创建 ordinary collect job 实现，不存在另一个 + scheduler-owned collection semantic。 +- collection frequency 可配置；当前不要求 IMAP IDLE 或 near-real-time long-lived connection。 +- global Cron 在 deployment timezone 中按当前 minute 由任意 Cron-capable Peer 检查，并以 Cron row lock、 + `last_scheduled_for` 与 `last_job` 保证同一 occurrence 最多创建一个 Job 且不堆叠未完成执行。错过 occurrence 即 + 错过，不补跑；run-now 直接创建普通 Job。 +- core-py 与打开中的 client-web 都可作为 Job worker，但只 claim 本地 Handler `can_handle(parameters)` 的 Job； + atomic pending-to-running update 决定唯一执行者。Mail contract 不锁死到 core-py-only，也不引入 request-response + Peer delegation、generic invoke 或 Job retry。 + +## Accepted Collect Job Boundary + +- collect job 是 manual/scheduled Source collection 的 CronJob-like execution envelope，不是 mailbox、page、item 或 + remote source completeness contract。 +- Source 内部拥有 traversal、partial effects、failure isolation 与 continuation；checkpoint 是建议的 interruption- + resilience 行为而非 Source capability 要求。generic Job 不理解这些 source-native units，也不检查 checkpoint。 +- Source invocation 按其 shallow public completion semantics 正常返回，Job 即 `finished`；异常逃逸或 runtime failure + 为 `failed`；执行预算耗尽为 `timed_out`。这些终态都不承诺同步了多少邮件或完成了哪个 horizon。 +- mailbox 局部失败可以在 Source 内部隔离并继续其他工作；不为此给 generic job 增加 mailbox-scoped transaction + 或 `completed_with_errors` outcome。 +- Job 是 one-shot；`finished`、`failed`、`timed_out`、`aborted` 都是 terminal，不 retry/reopen。下一次 + manual/scheduled run 创建新 Job；支持 checkpoint 的 Source 可从自己的 state 继续，较弱的 Source 可以重扫并 + 依赖 identity/reconciliation。不新增 attempt、backoff 或 retry lineage。 + +## Accepted Attachment Boundary + +- collection 创建 attachment metadata graph，但默认不下载实际 bytes，也不写入 Storage。exact metadata/remote + reference shape 已冻结到 D-259/D-271；不新增 per-part fetch binding，弱 Message-ID reconciliation 通过 lazy + duplication 保持 remote access exactness。 +- 对既有 Email/Attachment graph 持久增加 semantic content Block，可以分类为 Organization enrichment；该分类不 + 拥有实现。Mail attachment Resolver 执行/封装 materialization，并与 Mail Source 一样依赖 extension-owned + shared protocol adapter 取得远端 bytes，再使用 Storage 与 InfoBase 的通用能力；Resolver 不调用 Source + instance。 +- 用户打开/下载时临时读取或 stream attachment 是 use；只有 durable graph augmentation 才是 enrichment。 +- 当前官方协议/客户端 evidence 不支持“可信 mail client 必然默认持久下载全部附件”的前提；详见 + [evidence](evidence.md)。 + +## Accepted Body and Inline MIME Boundary + +- collection 保存 `text/plain` / `text/html` authored body；exact alternative/canonical shape 留到 Technical gate。 +- CID image 等非文本 inline MIME parts 只收 metadata/reference，默认不下载或持久化 bytes。查看时可以按需读取； + durable materialization 由对应 Mail Resolver 执行，并可分类为 enrichment。 +- HTML 引用的远程资源不在 collection 时自动抓取。邮件文本语义可离线使用，但完全视觉 fidelity 可能依赖按需 + 网络读取；这是已接受的 trade-off。 + +## Accepted Reply and Thread Boundary + +- collection 将 source-native reply/reference facts 表达为 Email Blocks 之间的有向 Relations；方向是 reply Email + → replied-to parent Email，exact predicate 留到 Technical gate。 +- 不创建 generic Thread Block；thread/conversation view 从 Email relation graph 派生。未来只有 source-native + thread object 证明独立 information value/identity 时才重新讨论。 +- collection 不通过 subject/participants/time 猜测缺失关系；best-effort inferred links 属于 Organization linking。 +- D-262 恢复 best-effort canonical Email 后，“Message-ID-only incomplete Email 之后被 occurrence 补全”的 D-256 mechanism + 再次可行。D-266 已明确恢复并约束它：In-Reply-To/References 必须采集；exact-one target 复用，zero/many 创建 + ordinary incomplete Email；later collection 只在 D-263–D-265 允许时原位补全。 + +## Accepted Client-Web Boundary + +- 不建立 Mail 专用 inbox/folder/message-list 浏览页，也不增加 Mail-only query/filter product。 +- client-web Mail extension 参考现有 Twitter extension：注册 Email Resolver 与 resolver-owned content component； + 通用 `BlockContent`、Block details、graph/query result surfaces 负责呈现任意 Block。 +- Email component 展示正文、participants、mailbox membership、flags、reply/reference navigation actions 与 attachment + metadata。reply/reference 不渲染为 ordinary links；以“查看回复”等 action 请求 generic UI 跳转到目标 Email Block。exact + parent/replies labels 与 multi-target interaction 留到 client-web design。 +- `SolvedContentRenderer` 是 resolver-selected presentation contract，取代 `contentComp` 与曾提议的 + `BlockRenderer`。`BlockInspector` 提供“查看内容”（即 view solved content）与 rumination 等 current-Block actions； + 它不拥有 domain rendering 或 cross-Block navigation。Email renderer 可以给出 target Block action，但只有 + GraphSurface 决定如何选择、聚焦、定位或 route 到该 Block。 +- `BlockInspector`、GraphSurface、solved-content view 与 cross-Block navigation 同属 InfoBase domain；不使用 generic + render-context callback bag 掩盖这套 domain navigation。InfoBaseRouter 拥有 route/history operations，GraphSurface + 是当前 route realizer，未来 ListSurface 可以实现相同 locations；route 本身不出现 graph/list surface。 + `overview` 表示无 focal entity 的全局视野。Solved content 是由 `SolvedContentPopup` 实现的一等 MVP destination； + exact surface-independent route vocabulary 是 `overview | block | solved-content`，两个 focal routes 都携带 + `BlockRef`。InfoBaseRouter 不拥有独立 history stack/`InfoBaseHistory` domain module，而通过 router-internal + replaceable adapter 映射到 Vue Router/browser history；不得用 `push(block)` 模拟 `back`。 + MVP public interface 仅为 nullable read-only `current`、`push(route)` 与 literal `back()`；`current = null` 只表示 + 当前 app location 在 InfoBase surface 之外，不是第四种 route；不公开无真实 caller 的 `replace()`。 + GraphSurface application route 通过 `/info-base/graph[/blocks/:block[/content]]` 选择 surface 并投影 domain route； + adapter 直接从 Vue Router 派生 `current`，不保存 location mirror。 + `@inkcre/core` 只拥有 `InfoBaseRouter` contract 与 singleton implementation binding；client-web 完整实现 + Vue-backed `current + push + back`，GraphSurface/ListSurface/renderers 都是 consumer。共享层没有 state/history/ + route registry，也不引入 `InfoBaseRouterHistoryAdapter` 或 `InfoBaseRouteCodec`。 + binding 使用 `setInfoBaseRouter/getInfoBaseRouter`，未配置 get 时 fail-fast；这是复用 singleton-binding pattern， + 不新增 `createRuntimeBinding<T>()`、registry、optional initialization check 或 hot-swap lifecycle。 + malformed/unmapped app route 不产生 InfoBaseRoute；合法 BlockRef 即使 row 不存在也先产生 route，由 + GraphSurface/`SolvedContentPopup` 加载并呈现 missing outcome，InfoBaseRouter 不访问 persistence。 + `overview` 仅 graph；`block` 为 graph + `BlockInspectorPopup`；`solved-content` 为 graph + `SolvedContentPopup`。 + 两个 popup 自己调用 Router.back 关闭，GraphSurface 不把 close 改写成 push 或猜测目标。 + GraphSurface/ListSurface 稳定统称 vocabulary-level `InfoBaseView` navigation host；route-owned destination 因 + dismiss 本身具有 back 语义而例外地自带 popup shell。`SolvedContentView` 已撤回；`SolvedContentRenderer` 保持 + presentation-neutral。 + GraphSurface 直接理解 `InfoBaseRoute.name` 并从 `router.current` 派生 focal Block；`block`/`solved-content` 聚焦同一 + referenced node，`overview` 清除 focus。node click 只 push route，route realization 不反向 push。 + `BlockInspectorPopup`/`SolvedContentPopup` 只接收 BlockRef，各自执行 Block.get 并拥有 loading/missing/error； + GraphSurface 偶然已有的 Block 不作为 destination input。重复读取若成为真实成本再由 Block cache/manager 解决。 + `SolvedContentRendererProps` 同时提供完整 Resolver 与 typed solved content。 +- 完整 email client/agent 是能力方向，不要求复制传统邮箱 UI information architecture。 +- `contentComp` 当前名称/接口错误地暗示只渲染 `Block.content`；实际上 renderer 围绕 focal Block，消费 Resolver + 联合 hydrated content 与 local Relations 产生的 solved projection。hydrated content 只隐藏 inline/Storage branch； + solved content 是 derived use-facing projection，不是 canonical/durable root content。Graph-aware solved projection + 以 `.root` 持有 focal Block canonical parsed content，relation-derived fields 只作为 siblings；exact props 与 + navigation bridge 留在 Technical gate。 +- 本轮不预设 generic query increment；只有 preflight 证明现有 generic surfaces 无法合理触达 Email Blocks 时，才 + 提议消除该 blocker 的最小 generic query change。 + +## Gate Status + +| Gate | State | Exit condition | +| --- | --- | --- | +| Product | **Closed for current delivery** | 新 implementation evidence 或真实 use pressure 才重新打开 | +| Technical | **Closed for implementation** | 新 evidence 若改变 domain ownership、observable behavior 或 accepted graph/runtime contract 才重新打开 | +| Acceptance | **Closed** | D-313/D-314：Dovecot real-IMAP hard gate + 四条纵向 journey + optional provider smoke；本 unit 不增加 focused negative-path suite | +| Implementation Plan + Preflight | **Closed** | D-315：R5 slices、preflight and first extension-owned exact Peer delegation accepted | +| Impact Handshake + Start | Pending | durable/code state diff 获批且 Sir 明确开始 | +| Execute / Verify / Promote | Pending | 实现、证据与 owner-specific durable projection 完成 | + +完整决定由 [program decision authority](../../decisions/index.md) 的 D-198–D-315 拥有；本 packet 只保留 + unit control 与 approved implications。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/block-rendering.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/block-rendering.md new file mode 100644 index 0000000..a3cc7bd --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/block-rendering.md @@ -0,0 +1,231 @@ +# Solved-Content Rendering + +- **Status**: R3 product/technical boundary frozen；exact implementation seams remain plan/preflight-owned。 +- **Decision authority**: [D-220–D-238、D-312](../../../decisions/index.md)。 +- **Problem**: rich focal Blocks require resolver-owned local graph interpretation，while client-web currently calls the + resolver-selected component `contentComp`、renders it inline under `BlockDetailsPanel` and lets only the graph page own + cross-Block selection/navigation。 + +## Current Evidence + +```text +graph.vue + owns selectedBlock + selectedBlockRelations + └─ BlockDetailsPanel(current Block only) + ├─ persistence facts: id / resolver / timestamps / storage + ├─ BlockContent + │ └─ resolverCls.contentComp(resolver, solvedContent) + └─ current-Block rumination action +``` + +- `TweetResolver` already reads attachment Relations/Blocks before its component renders，so literal-content rendering is a + false contract。 +- `BlockDetailsPanel` does not own graph selection and currently emits only close/ruminated。Adding target-Block lookup or + route behavior there would invert ownership。 +- Vue component events do not automatically bubble through arbitrary component layers，so “renderer emits an event” is not + by itself a complete topology。 +- `contentComp` and its Resolver prop were introduced together；current simple renderers are not evidence against the prop。 + The historic Module Federation runtime problem was duplicate `@inkcre/core` instances/registries and was repaired by + singleton sharing。The Resolver's dynamic Relation import addresses a separate model-module cycle。 +- Built-in core Resolver renderers are assigned by the host so the shared core package does not import app components；an + extension Resolver can import its co-owned renderer because both depend in the allowed direction on the shared singleton + core package。 +- `@inkcre/core/extension/module-federation` already defines the closest runtime-binding precedent：a contract owned by the + shared package、one module-scoped nullable implementation、host `setMFImplementation()`、consumer + `getMFImplementation()` and fail-fast access before bootstrap。Its exported `isMFInitialized()` has no repository caller。 +- `@inkcre/ui-web` defines the closest contract/provider precedent：`InkRouter` is implemented by client-web's Vue Router + adapter and injected under `INK_ROUTER_KEY`。That validates “shared contract，host implementation”，but its Vue component + scope is intentionally different from the Module-Federation-singleton scope required here。 +- `configStore.initializeMeta(adapter)` is not the same binding lifecycle：the Store owns reactive config state、loading、 + persistence and an unconfigured null-object adapter。It should not be forced through a generic implementation binding。 + +## Confirmed Invariants + +1. `block.resolver` selects an exact behavior contract；client-web presents its semantic projection through the Resolver's + `SolvedContentRenderer`。The earlier exact name `BlockRenderer` is withdrawn。 +2. `BlockDetailsPanel` becomes `BlockInspector`：it owns generic persistence facts and current-Block commands，not semantic + rendering or graph navigation。 +3. “查看内容” means view solved content，not inspect literal `block.content` or a Storage pointer。 +4. Graph surface exclusively owns current-Block selection、cross-Block navigation/focus and any route consequence。 +5. BlockInspector acts only on its current focal Block and does not know/open another Block。Its unused `relations` prop and + caller binding must be removed。 +6. Hydrated content and solved content remain distinct；graph-aware solved content keeps canonical focal content at `.root` + and relation-derived values as siblings。 +7. BlockInspector、GraphSurface、solved-content viewing and cross-Block navigation are InfoBase-domain concerns。A generic + render-context callback bag is not their owner。 +8. `SolvedContentRendererProps<SolvedContentT, ResolverT>` carries both typed solved content and the complete exact Resolver。 +9. InfoBaseRouter owns current InfoBase location/history operations；an InfoBase surface realizes routes。GraphSurface is the + current realizer，not a route or permanent default；future ListSurface may realize the same locations。 +10. Solved content is a first-class MVP destination realized by `SolvedContentPopup`，not an ambient callback、page or + BlockInspector implementation detail。 +11. MVP routes cover only overview、inspect-one-Block and view-one-Block's-solved-content。No arbitrary extension route + registration or speculative Relation/surface-specific routes。 +12. The exact surface-independent route vocabulary is `overview | block | solved-content`；both focal routes carry a + `BlockRef` under `block`。 +13. InfoBaseRouter does not own a second history stack or delegate to a separately meaningful `InfoBaseHistory` domain + module；an internal replaceable history adapter maps its operations to the existing Vue Router/browser history authority。 +14. `back` retains literal history traversal semantics and must not be simulated by pushing a guessed Block route。 +15. InfoBaseRouter's MVP public interface is exactly read-only `current`、`push(route)` and literal `back()`；public + `replace()` remains out of scope absent a real domain caller。 +16. `current` is `InfoBaseRoute | null`；`null` exclusively means the current application location is outside any InfoBase + surface，not another domain route or retained last-known location。 +17. The accepted GraphSurface web mapping is `/info-base/graph`、`/info-base/graph/blocks/:block` and + `/info-base/graph/blocks/:block/content` for `overview`、`block` and `solved-content` respectively；the surface prefix + belongs to the application route，not `InfoBaseRoute`。 +18. The adapter derives `current` from Vue Router and encodes `push` back into named Vue routes；it stores no mirrored + location state。 +19. InfoBaseRouter is a singleton client capability port，not a shared implementation of history/routing。`@inkcre/core` + owns the fixed contract and one implementation binding；each client implements nullable `current + push + back` against + its own navigation authority。 +20. GraphSurface/ListSurface and renderers are Router consumers；surfaces realize the current domain route into UI state。 + The shared layer owns no navigation state、history stack or route registry。 +21. The shared `InfoBaseRouterHistoryAdapter`、generic Location and `InfoBaseRouteCodec` candidates are withdrawn。Any such + factoring remains private to a client implementation and requires its own evidence。 +22. Singleton binding follows the existing `MFImplementation` pattern：one module-scoped nullable implementation、host + set、consumer get and fail-fast before configuration。Do not extract a generic `createRuntimeBinding<T>()` or registry。 +23. Malformed/unmapped client routes project `current = null` and belong to app-level not-found behavior；a syntactically + valid route with a missing Block remains an InfoBase route，and its surface/view owns loading and missing-entity UI。 +24. InfoBaseRouter never queries Block persistence to determine route validity。 +25. GraphSurface keeps the graph as its surface for all three routes：`overview` has no focal popup，`block` adds + `BlockInspectorPopup` and `solved-content` adds `SolvedContentPopup`。A first-class destination is not + synonymous with a page or replacement body。 +26. Each popup owns close and interprets it as literal `InfoBaseRouter.back()`；GraphSurface does not convert close into an + explicit overview/Block push or guessed parent destination。 +27. GraphSurface and future ListSurface are `InfoBaseView` navigation hosts with a `route destination outlet`；these terms do + not require a shared base component/class。 +28. Presentation-neutral content is normally wrapped by its parent container owner。A route destination exceptionally owns + its shell only when the container lifecycle is part of the destination behavior contract，as with dismiss → back。 +29. Exact shell-owning names are `BlockInspectorPopup` and `SolvedContentPopup`；`SolvedContentView` is withdrawn。 + `SolvedContentRenderer` remains presentation-neutral inside `SolvedContentPopup`。 +30. `InfoBaseRouter.current` is GraphSurface's only focal-Block authority；delete local selected-Block identity。Both focal + routes select/focus their referenced node，while overview clears focus。 +31. GraphSurface correctly understands stable `InfoBaseRoute.name` semantics because an InfoBaseView is the route realizer； + it does not understand Vue route names/paths and does not push while realizing an observed route。 +32. `BlockInspectorPopup` and `SolvedContentPopup` accept only `BlockRef` and each owns `Block.get()` plus loading/missing/ + error lifecycle。SolvedContentPopup additionally owns Resolver acquisition、solving、refresh and disposal。 +33. GraphSurface's graph projection may contain the same Block but is not a resource provider for route destinations；a + future shared cache may remove proven duplicate-read cost without changing ownership。 + +## Candidate Responsibility Topology + +```text +InfoBaseRouter ── current InfoBase location/history ──> selected InfoBase surface realizer + ├─ GraphSurface (current) + └─ ListSurface (future evidence example) + │ + overview / focal destination <──────┘ + ├─ BlockInspector + │ └─ “view content” command + └─ SolvedContentPopup + ├─ Resolver lifecycle + └─ SolvedContentRenderer(resolver, solvedContent) + └─ target-Block navigation command +``` + +This topology and the following domain route shape are accepted： + +```ts +type InfoBaseRoute = + | { name: 'overview' } + | { name: 'block'; block: BlockRef } + | { name: 'solved-content'; block: BlockRef } +``` + +InfoBaseRouter owns these domain locations and public navigation commands，while a router-internal replaceable history +adapter maps them onto the one existing Vue Router/browser history authority；there is no second InfoBase history stack or +independent `InfoBaseHistory` domain abstraction。The selected surface owns its +loading/arrangement/focus mechanics and realizes the location。 +GraphSurface is the current realizer，not the route authority or `overview` synonym。The router may adapt to Vue Router/URL +state without exposing app paths to Module Federation renderers。This is materially different from a render context：the +router models stable InfoBase navigation state and makes `SolvedContentPopup` addressable rather than forwarding one callback。 + +The prior `SolvedContentRenderContext` proposal is withdrawn。Explicit event forwarding through BlockInspector and direct +extension access to the app's Vue Router/routes also remain disfavored。A router shared through the Module Federation singleton +`@inkcre/core` is a plausible mechanism，but its global-instance/test/multi-app consequences still require design。 + +The accepted renderer boundary is: + +```ts +interface SolvedContentRendererProps< + SolvedContentT, + ResolverT extends Resolver<unknown, SolvedContentT>, +> { + resolver: ResolverT + solvedContent: SolvedContentT +} +``` + +Exact generic parameter ordering must follow the final Resolver declaration rather than copying this illustrative snippet +blindly。 + +## Derived Destination Lifecycle（R3 closure candidate） + +The existing client confirms that route changes cannot rely on component construction：`BlockContent` creates one Resolver +from setup-time props，while GraphSurface owns a separate selected-Block object。The accepted Router topology therefore implies +the following explicit `SolvedContentPopup` lifecycle without another product choice： + +1. `block: BlockRef` is watched immediately。Every route-ref change increments a local load generation，clears the previous + destination state and best-effort disposes its Resolver before loading the new Block。 +2. The Popup loads the current Block row，constructs the exact registered Resolver and calls solved-content retrieval。Only + the latest generation may publish Block、Resolver、solved content or error state；a stale completion disposes any Resolver + it created and otherwise has no UI effect。This prevents async route races without requiring every Resolver to implement + cancellation。 +3. Explicit refresh keeps the same InfoBase route but reruns the complete destination load from Block persistence，including + Resolver selection，with the stable `{refresh:true}` cache-replacement semantic。It does not depend on mutating an old + Resolver in place，component remount keys or a new navigation entry。 +4. Popup unmount invalidates the current generation and best-effort disposes the live Resolver。Disposal failure is internal + diagnostic residue；it does not block close/back or the next destination，and no retry/checkpoint lifecycle is added。 +5. Loading、missing-Block、solve failure and refresh failure remain Popup states。The presentation-neutral + `SolvedContentRenderer` receives only the successful exact Resolver plus typed solved content and owns none of this + controller lifecycle。 + +The same lifecycle shape applies to `BlockInspectorPopup` without Resolver/solve/refresh：watch BlockRef，generation-guard +`Block.get()`，own loading/missing/error and invalidate stale completions。Whether `Block.get` gains a nullable lookup seam or +the Popup maps an existing not-found result is implementation-plan detail，not a second route contract。 + +## Solved Email Projection Direction（R3 closure candidate） + +`SolvedEmail` remains a read projection of the accepted graph rather than a second canonical Email DTO。Its structural +families are： + +- `.root`: canonical Email root content； +- body representations：body Block、MIME `part_id` and the child Resolver's solved content； +- participant occurrences：EmailAddress Block/content plus role、order and occurrence display name； +- mailbox memberships：Mailbox Block/content plus UIDVALIDITY/UID locator； +- mailbox-scoped flags：MailFlag Block/content and its owning Mailbox reference； +- reply/reference navigation targets：normalized relation role/direction、order and target Email BlockRef； +- MIME components：Email-relative role/`part_id`、metadata Block and read-only `SolvedMimePart`，whose semantic child remains + nullable and does not trigger automatic attachment download。 + +The Resolver may use narrower exact helper types for these families during implementation。It must not copy graph-owned facts +back into `.root`，make the renderer parse Relation strings or expose loading/created/existing mechanics。 + +## Email HTML Presentation and Remote-Resource Boundary + +The sender-authored HTML body remains raw collected authority in its body Block。Its transition from passive data into a +browser-renderable projection is the concrete security boundary；sanitizing during collection or Storage write would damage +authority while failing to place responsibility where execution capability is introduced。 + +`SolvedContentRenderer` applies the following frozen behavior： + +1. Prefer an available HTML body for faithful Email reading；fall back to the plain-text body。This does not merge or rewrite + the independently collected body Blocks。 +2. Pass HTML through a mature maintained sanitizer such as DOMPurify，then render the result in a sandboxed iframe without + scripts、forms or same-origin capability。The application does not implement its own parser/filter，and CSP is not the sole + XSS defense。 +3. Disable automatic external resource loading by default，including remote images and tracking pixels。Normalize and allow + only user-initiated `http`/`https` link opening。 +4. Rewrite CID references only to client-local object URLs backed by an already materialized semantic content child。If the + inline MIME part is still remote-only，show metadata/placeholder and an explicit materialization action instead。 +5. Opening an Email never implicitly materializes an attachment。Attachment and unresolved inline-part actions retain the + accepted Resolver-owned explicit command boundary。 + +This is the baseline contract at an identified untrusted-HTML/browser boundary，not an open-ended hardening backlog。Exact +sanitizer configuration、iframe construction and object-URL cleanup belong to implementation plan/preflight，provided they +prove these effects and do not invent a second HTML authority。 + +## R3 Closure + +The destination lifecycle、SolvedEmail projection、navigation actions、attachment action boundary and HTML presentation policy +are now product/technical-design complete through D-312。Repository preflight may select exact helpers and expose a concrete +blocker，but does not reopen this topology merely because another component factoring is possible。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/collection-value-audit.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/collection-value-audit.md new file mode 100644 index 0000000..568fc59 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/collection-value-audit.md @@ -0,0 +1,63 @@ +# Mail Collected-Graph Value Audit + +- **Status**: accepted Block/Relation fields and Email identity/reference behavior are frozen through D-266。MailFlag exact + content/normalization remains under review；the rest of this audit is closed unless new implementation evidence reopens it。 +- **Test**: a persisted field/grammar must serve identity、reconciliation、later on-demand access、an accepted use path or + independently valuable structure。Protocol availability、low fetch cost and possible future interest are not sufficient。 + +## Block Content + +| Owner | Fact | Current judgment | Reason | +| --- | --- | --- | --- | +| Source anchor | `id` | keep | stable operational reference and historical descriptor after Source deletion | +| Source anchor | `type` | keep | exact Source identity/projection and readable provenance | +| Source anchor | `nickname` | keep | human label owned by Source and useful after operational deletion | +| Mailbox | `name` | keep | remote operation、label and source-scoped fallback identity | +| Mailbox | `delimiter` | remove — D-258 | no accepted hierarchy renderer/query；Source can use LIST delimiter transiently | +| Mailbox | generic `attributes` | narrow — D-258 | only recognized special-use role has communication/use value；subscription/selectability/tree hints do not earn persistence | +| Mailbox | `mailbox_id.value` | keep | rename continuity inside one Source when OBJECTID is available | +| Mailbox | `mailbox_id.access_scope` | remove — D-258 | permanent Source-scoped Mailbox plus `Source --manages--> Mailbox` already owns comparison/access scope | +| Email | `message_id` | keep — D-263 | authored message-version evidence、reply/reference input and best-effort reconciliation after scoped EMAILID；not exact occurrence identity | +| Email | `email_id` | keep — D-263 | optional server-native immutable-content evidence and scoped reconciliation before Message-ID；not unconditional uniqueness | +| Email | `subject` | keep | direct authored semantic content、label and retrieval | +| Email | `authored_at` | keep | authored chronology distinct from collection time | +| EmailAddress | `address` | keep | exact useful shared identity and participant navigation | +| MIME part | `part_id` | move to owner Relation — D-259 | MIME tree path identifies/orders the part relative to Email and serves remote fetch；it is not intrinsic Block identity | +| MIME part | `media_type` | keep | classification、rendering and semantic resolver selection | +| MIME part | `charset` | keep — D-259 | required to transcode an on-demand text/HTML part into exact core semantic-content encoding when MIME is the only declaration | +| MIME part | `disposition` | remove — D-259 | canonical owner Relation already owns the usable role | +| MIME part | `filename` | keep | user-facing label/download name | +| MIME part | `content_id` | keep + graph edge — D-259 | intrinsic part label；resolved body reference also earns contextual `embeds` Relation | +| MIME part | `description` | keep | MIME-authored basic semantic description with label/text retrieval value | +| MIME part | `transfer_encoding` | keep — D-259 | required to decode a later IMAP BODY section without re-fetching MIME structure；serves the accepted remote-access path | +| MIME part | `encoded_size` | keep | pre-download display/policy bound without fetching bytes；must remain explicitly encoded/estimated semantics | +| MIME part | `content_location` | keep + graph edge — D-259 | intrinsic part label；resolved body reference also earns contextual `embeds` Relation | +| MailFlag | `name` | keep — D-268 | mailbox-scoped case-insensitive flag identity、remote operation token and generic label | +| MailFlag | `description` | keep — D-268 | adapter-owned provider/standards semantic metadata for resolver text、retrieval and graph interpretation；nullable when no authoritative mapping exists | + +## Relation Content + +| Grammar | Judgment | Reason | +| --- | --- | --- | +| `manages` | keep | normalized provenance/access path and Source-scoped Mailbox ownership | +| `{type:"contains", uid_validity, uid}` | keep | exact occurrence idempotency/membership locator and later on-demand remote access | +| `{role:"body|attachment|inline", part_id}` | replace role/order strings — D-259 | one MIME tree path owns component identity、source order and IMAP fetch location without duplicate order authority | +| participant `{role, order, display_name}` | keep | complete contextual communication roles/names without corrupting shared EmailAddress | +| `parent:<order>` | keep | direct source-native reply structure and reverse reply navigation | +| `reference:<order>` | keep | ordered native ancestry/reference evidence without inferred thread entity | +| body → part `{type:"embeds", reference}` | add — D-259 | exact collected HTML reference occurrence and direct render/navigation edge | +| `Mailbox --has--> MailFlag` | add — D-262 | mailbox-scoped flag vocabulary shared by exact applications | +| MailFlag → Email `tags` | add — D-262 | ordinary semantic Relation for all durable IMAP flags；owning Mailbox + unique membership derives exact occurrence | + +## Candidate Corrections + +1. D-258 reduces Canonical Mailbox to `name`、nullable bare `mailbox_id` and a deliberately narrow special-use projection； + delimiter、generic attributes and duplicated access scope are removed。 +2. D-259 re-opens D-252/D-253 narrowly：remove disposition、move `part_id` from MIME-part Block content into a unified Email → + component Relation and let it replace the separate order；retain semantic Content-Description and retain charset/ + transfer_encoding as compact on-demand materialization inputs。 +3. D-259 keeps Content-ID/Location as intrinsic target labels while adding exact HTML body → MIME-part `embeds` Relations for + resolved reference occurrences，rather than hiding useful graph topology in metadata lookup alone。 +4. D-261 supersedes D-260 after duplicate-risk review by restoring best-effort canonical Email and retaining exact locator + Relations。D-262 then removes locator duplication from MailFlag edges：plain `tags` stays exact because Collection forbids + multiple live `contains` Relations for one Mailbox/Email pair。`contains` owns membership/locator only and no flag fields。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/email-graph.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/email-graph.md new file mode 100644 index 0000000..ad2a0b3 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/email-graph.md @@ -0,0 +1,208 @@ +# Canonical Email and Graph Boundary + +- **Status**: Email schema/body/component graph and participant graph frozen through D-266；best-effort canonical Email uses + the complete linear reconciliation ladder、one live occurrence per Mailbox/Email pair、exact-one reuse and + non-destructive identity compatibility；reply/reference anchors now follow the same locate/reuse-or-create model。 +- **Protocol/code evidence**: [Message Content and Envelope Facts](../evidence.md#message-content-and-envelope-facts)。 +- **Resolver**: retain exact ID `extensions.mail.email.v1`；old schema/behavior is not a compatibility authority。 + +## Canonical Email Root Content + +```json +{ + "message_id": "1234@local.machine.example", + "email_id": "M6d99ac3275bb4e", + "subject": "Saying Hello", + "authored_at": "1997-11-21T09:55:06-06:00" +} +``` + +- `message_id`、`email_id`、`subject`、`authored_at` are nullable。Do not reject malformed/draft messages merely + because one is absent。 +- `message_id` stores the semantic msg-id without surrounding CFWS/angle brackets。`email_id` stores the bare server-native + OBJECTID EMAILID when available。Both are authored/server identity evidence useful to reference/use；D-263 places scoped + EMAILID before Message-ID in best-effort reconciliation。D-264 owns candidate cardinality；neither field receives a + database uniqueness constraint。 +- `authored_at` means RFC 5322 origination time。Never synthesize it from collect time、Block.created_at、IMAP + INTERNALDATE or local clock；those are different facts/lifecycles。 +- This root is the Email's intrinsic scalar/header content needed for identity、label and chronology。It is not a wire + archive and does not copy every arbitrary/transport header。An empty-body message is still collectible。 + +## Source-Native Body Graph + +```text +Email Block + ├─ role-bearing relation ─> core.text.v1 semantic content Block + ├─ role-bearing relation ─> core.html.v1 semantic content Block + └─ attachment/inline relation ─> Mail MIME-part metadata Block + └─ content ─> core.<semantic-kind>.v1 Block (when materialized) + └─ Storage pointer -> bytes +``` + +- Decoded authored plain-text and HTML alternatives are independent Blocks。Their exact resolver already expresses content + semantics；the relation expresses that the Block is one representation/body of this Email。 +- Do not add `extensions.mail.text_body.*` / `html_body.*` metadata wrappers。Reuse of the core deep modules is supporting + evidence，while independent semantic/use value remains the actual decomposition reason。Future independently valuable + native metadata must justify its own Block rather than changing body-content authority。 +- Attachment and non-text inline MIME parts do have independent protocol facts before bytes exist，so collection creates + Mail-owned metadata Blocks for them。D-259 places their Email-relative MIME-tree `part_id` on the owner Relation rather + than intrinsic Block content。 +- A pure multipart container is not automatically a Block。It earns representation only when its ordering、alternative/ + related grouping、disposition or another structure fact is required by a real renderer/retrieval/materialization path。 + +## Graph-Owned Facts + +- EmailAddress Blocks and directional originator/destination relations own From、Sender、Reply-To、To、Cc and Bcc。 +- Email-to-Email relations/unresolved-reference representation own In-Reply-To and References。 +- Source/Mailbox relations own provenance and membership occurrence facts，including UIDVALIDITY/UID。IMAP INTERNALDATE is + intentionally not persisted absent a proven use path；Source may consume it transiently for collection bounds。 +- Mailbox-scoped MailFlag Blocks and plain `tags` Relations uniformly own `\Seen`、`\Answered`、`\Flagged`、`\Deleted`、 + `\Draft` and observed keyword state。The owning Mailbox plus its unique Email membership derives the exact occurrence； + behavior differs，but persistence shape does not。Deprecated `\Recent` is excluded。 +- Attachment/inline-part metadata Blocks and relations own MIME component role/tree position、filename、declared media type、 + description、content labels、encoded size and later materialized semantic content links。 +- Therefore root content excludes `uid`、`has_attachments`、body representations、participants、reply/reference IDs、flags、 + mailbox、Source and attachment arrays。Their presence there would create duplicate authority or a domain god object。 + +## MIME Component Relation Grammar + +| From | To | `relation.content` | Meaning | +| --- | --- | --- | --- | +| Email | `core.text.v1` / `core.html.v1` | `{role:"body", part_id:"1.1"}` | selected authored body at this MIME-tree position | +| Email | Mail MIME-part metadata | `{role:"attachment", part_id:"2"}` | attachment at this MIME-tree position | +| Email | Mail MIME-part metadata | `{role:"inline", part_id:"1.2"}` | inline component at this MIME-tree position | + +`part_id` is a canonical numeric MIME-tree path relative to the owning Email。It identifies the component position、preserves +source order and maps directly to IMAP section fetch；compare parsed numeric path segments rather than lexicographic strings。 +It replaces each earlier independent role order，so there is one structural-order authority。`(Email Block, part_id)` is the +bounded identity；MIME-part Blocks do not reconcile globally by this value。No MIME tree table/container Blocks are introduced。 + +When collected HTML contains a reference resolved to a MIME part，Collection also creates an exact HTML body → MIME-part +metadata Relation：`{type:"embeds", reference:"<authored URI>"}`。Content-ID/Content-Location remain intrinsic target +labels；the Relation owns the contextual reference occurrence and enables rendering/navigation without graph lookup guesses。 + +## Canonical MIME-Part Metadata + +- exact resolver ID：`extensions.mail.mime_part.v1`。 +- exact content： + + ```json + { + "media_type": "image/png", + "charset": null, + "filename": "logo.png", + "content_id": "logo@example.com", + "description": "Company logo", + "transfer_encoding": "base64", + "encoded_size": 18342, + "content_location": null + } + ``` + +- `part_id` is required on the owning Email Relation，not in this content。The MIME-part Resolver obtains its exact fetch + locator from that relation；the metadata Block owns only facts intrinsic to the MIME body part。 +- `media_type` is the normalized effective MIME type reported/defaulted by BODYSTRUCTURE；the Mail extension owns later + resolver-classification policy。IMAP returns body type/subtype separately while MIME names their normalized `type/subtype` + value a media type；this field is not byte-signature evidence。 +- `charset` is the optional normalized MIME charset promoted from content-type parameters。It is independently required to + transcode materialized text/HTML into the semantic resolver's supported encoding；this does not justify retaining every + arbitrary MIME parameter。 +- `disposition` is excluded after collection classifies the usable role into the owning Relation。Persisting the source wire + value as well would duplicate authority without another accepted consumer。 +- `filename` is the optional canonical filename selected by the adapter from disposition/content-type parameters；raw + parameter bags are not copied merely for wire fidelity。 +- `content_id` stores the semantic ID without surrounding angle brackets；`content_location` retains the normalized + source-authored body-part label。`description` is decoded MIME-authored semantic description，not generated summary，and + contributes to label/text/retrieval before bytes are materialized。 +- `transfer_encoding` is nullable at the canonical level but the IMAP adapter records BODYSTRUCTURE's effective value so it + can decode a `BODY[section]` response when decoded `BINARY` fetch is unavailable。`encoded_size` is nullable、non-negative； + for IMAP it is BODYSTRUCTURE's transfer-encoded octet count，not actual decoded byte size。 +- Exclude BODYSTRUCTURE MD5、language、line count、arbitrary parameters and disposition timestamps until a real use path + earns them。Exclude checksum/detected MIME/decoded size/dimensions/duration because those are byte-derived Resolver facts。 +- On materialization，the metadata resolver uses `media_type` through ResolverManager，writes decoded bytes through a + configured WritableStorage and adds one `content` relation to the exact core semantic content Block。 +- **Status**: D-253's wider shape and D-252's role-order strings are superseded by D-259。 + +## Canonical EmailAddress + +- retain exact resolver ID `extensions.mail.email_address.v1`。 +- exact content：`{"address":"Local.Part@example.com"}`。 +- EmailAddress Blocks reconcile across messages and Sources by exact canonical addr-spec because that shared graph entity + has concrete navigation/retrieval value。A display name is not part of this Block's content or identity。 +- Canonicalization parses one valid addr-spec、preserves local-part Unicode/case、normalizes equivalent quoted forms to the + minimally quoted serialization，and stores a lowercase IDNA A-label DNS domain；address literals retain a canonical + bracketed representation。It applies no Unicode normalization to the local part。Do not strip plus tags、fold dots or + apply provider-specific alias policy。 +- Message-authored display name and participant role/order belong to the Email → EmailAddress Relation。This prevents + `Alice`、`Support` and no-name occurrences from competing for one Block property while retaining the useful shared address + node。 +- `get_label()` / `get_text()` use the address only。Solved Email rendering gets the contextual display name from the + participant Relation rather than mutating or looking up a preferred global name。 +- From、Sender、Reply-To、To、Cc and Bcc are all Email participant facts represented by EmailAddress Blocks plus directional + Email → EmailAddress Relations。None is copied into Email root content；the relation owns role、message occurrence order + and the optional display name authored for that occurrence。 +- **Status**: frozen by D-254。 + +## Email Participant Relations + +Every observed participant occurrence is a directional Email → EmailAddress Relation whose content is the canonical compact +JSON serialization of： + +```json +{ + "role": "from", + "order": 0, + "display_name": "Alice" +} +``` + +- exact `role` values：`from`、`sender`、`reply_to`、`to`、`cc`、`bcc`。 +- `order` is a required non-negative、zero-based sequence owned independently by each role，preserving source occurrence + order。One EmailAddress may have multiple occurrence Relations when the message gives it multiple roles。 +- `display_name` is nullable and preserves the decoded occurrence-local phrase；it never mutates the shared EmailAddress。 +- From and Sender remain distinct：From identifies authors while Sender identifies the transmitting mailbox when supplied。 + To、Cc and Bcc all remain destination participation facts；the adapter records only observed Bcc values and does not infer + stripped/undelivered recipients。 +- Use structured JSON rather than delimiter escaping because display names freely contain punctuation and Unicode。The Mail + extension owns parsing and canonical compact serialization of this Relation content。 +- RFC address-group labels are not retained in the MVP；only their actual addr-spec members produce participant Relations。 + A group label is not a communication endpoint and currently has insufficient rendering、navigation or retrieval return to + earn a Block/hyperedge representation。Empty groups therefore produce no participant relation。 +- **Status**: frozen by D-255。 + +## Reply / Reference Graph + +- In-Reply-To produces reply Email → referenced Email Relations with `parent:<order>`，where each header owns a contiguous + zero-based order。Incoming `parent:*` Relations are the target Email's replies；outgoing Relations are the current Email's + explicitly authored parents。 +- References produces referencing Email → referenced Email Relations with `reference:<order>`，preserving header order。 + This is ancestry/reference evidence rather than an inferred thread shortcut；a target may also have a `parent:*` Relation + when both source headers name it。 +- D-262 permits different Mailboxes' locators to share one best-effort canonical Email，so a Message-ID-only Email anchor + can be the same domain node later completed by a collected occurrence。A second matching UID in the same Mailbox must + instead create another Email Block。D-264 owns candidate cardinality，D-265 owns identity compatibility and D-266 restores + D-256's anchor mechanism under those rules。 +- The accepted product intent remains：preserve source-native In-Reply-To/References evidence and support reply navigation + without inventing subject/participant/time inference。The same canonical identity rule now chooses/creates the target and + later completes it；an occurrence never needs a separate reference entity merely for this purpose。 +- Only parsed semantic msg-id values produce anchors/Relations。Malformed residue is not promoted to an identity-bearing + Block merely for wire fidelity。Self-links、cycles and repeated protocol facts are not reinterpreted by Collection；normal + exact-relation idempotency applies。 +- Message-ID candidate resolution uses `zero → create anchor / one → reuse / many → create anchor`。The incomplete anchor is + an ordinary Email with only `message_id` known，not a placeholder type or persisted lifecycle state。Later exact-one + compatible collection completes that Block；ambiguity never authorizes rewriting one existing reference target。 +- **Status**: D-256 relation intent and anchor mechanism are restored and refined by D-262–D-266。 + +## Explicit Non-Goals at This Edge + +- Do not introduce a Mail Thread entity/table merely to group reply/reference edges；thread views are graph-derived unless a + later use path proves otherwise。 +- Do not preserve complete RFC822 bytes by default：that would also download attachment/inline bytes and contradict the + accepted lazy materialization boundary。 +- Do not introduce a generic arbitrary-header bag without a demonstrated use case。Specific valuable fields can later earn + canonical root fields or graph representation through their own product pressure。 + +## Status + +Canonical Email、body/component decomposition、participants and reply/reference graph are frozen for implementation +planning through D-266。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-identity.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-identity.md new file mode 100644 index 0000000..9cbe437 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-identity.md @@ -0,0 +1,236 @@ +# Mail Identity and Remote Occurrence + +- **Status**: D-263 freezes the complete linear Email reconciliation ladder；D-264 freezes zero/one/many candidate behavior + under D-262's same-Mailbox guard；D-265 freezes identity completion/contradiction behavior；D-266 restores reference-anchor + creation and later completion under the same locate/reuse model。This identity edge is closed for implementation planning。 +- **Protocol evidence**: [Message Identity Facts](../evidence.md#message-identity-facts)。 +- **Correction pressure**: 不把 local Email identity、跨 occurrence reconciliation 与 IMAP remote occurrence + locator 误认为同一种 identity；但 collection 可以把后两者组织成有明确 scope 的 reconciliation ladder。 + +## Distinct Concepts + +1. **Local Email identity**：`Block.id` 是 InKCre 内唯一无条件成立的 Email identity。 +2. **Authored/server evidence**：`Message-ID` 表达 authored message-version identity；`EMAILID` 表达可选 + server-native immutable-content identity。D-262 允许它们帮助一个新 locator 选择 canonical Email endpoint， + D-263–D-265 已冻结 exact scope、precedence、cardinality 与 compatibility behavior。 +3. **Remote occurrence locator**：IMAP 操作需要 authenticated message-store/source namespace + mailbox + + `UIDVALIDITY` + `UID`，用来再次读取或改变一个远端 occurrence。它不是 Email identity 的 fallback。 + +更稳定的逻辑表达是 `remote mailbox identity + UIDVALIDITY + UID`。`account + mailbox` 只有在 remote account +identity 可知、且 mailbox 确实位于该账号自身 namespace 时，才是 `remote mailbox identity` 的一种 concrete +representation；它不是 Base IMAP 的普遍事实。 + +## Source / Access-Context Boundary + +- Email address 是消息参与者/路由地址，不能证明一个独立 mailbox store；登录名也未必是 Email address。 +- Base IMAP 只能让我们确认一个 Source 持有一套 connection/authentication config。一次 authenticated connection + 可以看到 personal、other-user 与 shared namespaces；不同凭据暴露的 mailbox 集合可能重叠。因此撤回“一 Source + 必然对应一个协议可识别账号”的承诺。 +- Source instance 只能代表这个 local access context。它不会证明两个 Sources 指向不同 remote account，也不支持 + 用 address/password 推导 account identity；credentials 不是 identity。一个 Source instance 在首次需要 graph + provenance 时拥有至多一个 lazy Source Block anchor；Mailbox 通过 graph relation 连接该 Block，而不是在 Mailbox + content 中嵌入 `SourceRef` 作为唯一 provenance。 +- 即使获得 authenticated-user identity，一个 connection 仍可能暴露 delegated/shared mailbox；这类 mailbox 的 + identity 属于 mailbox/store，而不属于当前登录账号。因而 `account + mailbox name` 不能普遍解决跨 Source merge。 +- 当前仍没有足够价值引入 `MailAccount` Block。支持 `OBJECTID` 的服务器可提供更稳定的 `MAILBOXID` / `EMAILID`； + MVP 消费 `MAILBOXID` 支持 owning Source 内的 rename continuity，并将 optional `EMAILID` 保存为 Email + content evidence。D-262 允许 EMAILID 参与 best-effort canonical reconciliation，但不自动证明任意 Sources + 之间的 comparison scope；`THREADID` 仍不采集。 + +## UIDVALIDITY Boundary + +- `UID` 只在一个 mailbox 的某次 UID epoch 内有意义，不能自证它仍指向原来的 remote message。 +- `UIDVALIDITY` 变化后，相同数值的 `UID` 可以指向另一个 message；凡是持久保存 UID 并在以后据此读取/改变远端 + occurrence 的路径，都必须保留并核对它所属的 epoch。 +- tuple 的 durable placement 已冻结：Mailbox Block 持有 remote mailbox identity evidence，Email–Mailbox + membership fact 持有 occurrence-local UID + UIDVALIDITY snapshot；Mailbox–Source Block relation 提供 access/ + provenance path。D-241/D-257 已冻结 exact direction/content 为 + `Source --manages--> Mailbox --contains {type:"contains",uid_validity,uid}--> Email`。 +- Base IMAP 的安全 concrete fallback 是 Source-scoped mailbox binding + `UIDVALIDITY` + `UID`。如果 adapter 获得 + exact provider-native mailbox identity（例如受支持且可正确限定 server/store scope 的 `MAILBOXID`），它可以 + 支持当前 Source 内的 Mailbox rename continuity。D-248 仍保留 Source-scoped Mailbox Blocks；D-262 允许不同 + Mailboxes 的 occurrence locators 指向同一 canonical Email，但不因 bare provider value 自动跨不可比 Sources + merge，也不合并同一 Mailbox 内的多个 live UIDs。 +- MVP 在 authentication 后执行一次 CAPABILITY，发现 `OBJECTID` 时解析随后 SELECT/EXAMINE 必须携带的 + `MAILBOXID`。不为每个已选择 mailbox 追加 STATUS query。RFC 只保证 MAILBOXID 在 single client login + single + server hostname 的可见范围内唯一，因此 adapter 必须先证明比较 scope，不能跨任意 Sources 比较 bare value。 + +## Collection Identity and Canonical Reconciliation + +- local Email identity 始终是 `Block.id`。`(Source-scoped Mailbox Block, UIDVALIDITY, UID)` 是 exact occurrence + idempotency 与 remote-access authority；它不等于 canonical Email identity。 +- 已知 exact locator 必须复用其现有 `contains` Relation 与 Email endpoint。对于未知 locator，Collection 才使用 + source-native identity evidence best-effort 选择或创建 canonical Email；不使用 content fingerprint。 +- 多个 Mailboxes 的 locators 可以指向一个 canonical Email；同一 Mailbox 内的第二个 live UID 即使匹配 canonical + evidence 也创建另一个 Email Block。这样 Mailbox-scoped ordinary flag Relations 仍能精确映射 occurrence-local + mutable facts。 +- UIDVALIDITY reset 使旧 epoch locators 失效；后续收集重建 occurrence Relations，并通过 D-263–D-265 的 canonical + reconciliation rule 选择现有或新的 Email endpoint。它不因 UID epoch 改变而必然复制 canonical Email。 +- D-260 temporarily superseded canonical reconciliation but did not erase its decision lineage。D-261/D-262 restore the + earlier ladder；do not redesign already accepted rungs merely because the current projection had called them pending。 + +## Recovered Accepted Ladder and Narrow Delta + +The accepted D-239/D-248/D-256 order before D-260 was： + +1. **Known local occurrence**：same Source-scoped Mailbox + UIDVALIDITY + UID reuses its bound Email endpoint。This is + idempotency，not logical fallback。 +2. **Comparable exact remote occurrence**：when the adapter can prove comparable OBJECTID/server-login scope，the same + MAILBOXID + UIDVALIDITY + UID observed through another Source-scoped Mailbox reuses that Email endpoint。 +3. **Message-ID**：a valid semantic Message-ID is the best-effort logical Email reconciliation rung。It may also identify + an incomplete reply/reference anchor created before content collection。 +4. **Create**：without usable exact/logical evidence，create another Email Block；do not use a content fingerprint。 + +D-262 adds one eligibility guard to every cross-occurrence rung：a candidate already connected to the current Mailbox by a +different live UID is ineligible，so the new occurrence safely creates or selects another Email Block。 + +The only identity rung not present in that earlier ladder was optional OBJECTID EMAILID，which D-260 later added to Email +content。D-263 freezes it after exact occurrence matching and before Message-ID，because within a proven OBJECTID namespace +it is a server assertion of identical immutable message content and survives COPY/MOVE，while Message-ID is authored +best-effort evidence。Bare EMAILID remains incomparable across arbitrary Sources。 + +The resulting exact ladder is therefore `known local occurrence → comparable exact cross-Source occurrence → scoped +EMAILID → Message-ID → create`。It remains linear：the first usable rung wins，without scoring、cross-rung voting or content +fingerprints。D-264 defines one rung as zero → continue、one → reuse、many → stop and create；contradictory facts remain a +separate compatibility check，not a new identity level。 + +## Exact-One Resolution + +- Apply comparison scope and D-262's current-Mailbox eligibility guard before counting candidates。 +- Zero means this rung found no existing endpoint；continue to the next rung。One means reuse。More than one means the value is + non-unique in the current scope；stop reconciliation and create another Email Block。 +- Do not select by persistence accident such as minimum/oldest Block ID，and do not use a weaker rung to vote among stronger + ambiguous candidates。The identifier value can remain evidence even when it cannot authorize reuse。 +- This is the Collection shallow outcome because the observed occurrence still requires an Email endpoint。Reference-only + observations use the same locate/reuse safety rule and create an incomplete Email on zero or ambiguity per D-266。 + +## Identity Compatibility After Exact-One Location + +- For `message_id` and scope-comparable `email_id`：existing null is filled by an incoming non-null fact；incoming null does + not erase an existing fact；equal non-null values are compatible。 +- Different non-null comparable identity values reject a cross-occurrence reconciliation candidate。The newly observed + occurrence gets another Email endpoint；Collection neither overwrites the old identity nor falls through to weaker rungs。 +- EMAILID and Message-ID cross-check one another only where their comparison scope makes that meaningful。In particular， + bare EMAILID values from adapter-unproven scopes neither match nor contradict one another。 +- A known/comparable exact occurrence locator is a stronger remote-occurrence authority，not a logical reconciliation + candidate。Identity conflict on that exact path does not fork the occurrence；it exposes a producer/data-integrity + inconsistency and must not be hidden by destructive overwrite。 +- `subject` and `authored_at` are not identity evidence。Their completion/update policy remains separate and cannot silently + turn into content-fingerprint reconciliation。 + +## Future Ladder Utility Boundary + +The repeated mechanics now justify a Source-domain utility，subject to implementation preflight：ordered async rung +execution、scope-filtered candidate cardinality、short-circuiting and rung-labelled typed outcomes。Mail adapter logic still +owns candidate queries、OBJECTID comparison scope、same-Mailbox eligibility、identity compatibility and the decision to +create an Email。Other Sources likewise own their evidence semantics and command effects。The shared utility must not own a +universal identity ladder or hide domain policy behind callbacks so generic that the abstraction has no meaningful contract。 + +## Canonical Provenance Graph + +```text +Source Block --manages--> Mailbox Block +Mailbox Block --contains { UIDVALIDITY, UID }--> Email Block +``` + +- Mailbox content 不保存 `source`；每个 Source instance 由 `SourceModel.block` 唯一映射到一个 Source Block。 +- `manages` 表示当前 Source access context 管理/同步该 Source-scoped Mailbox；每个 Mailbox Block 恰有一个 + Source Block provenance path。即使另一 Source 暴露相同 remote mailbox，也保留另一个 Mailbox Block。 +- `contains` 是当前已知 membership fact，其 structured content 同时保存 occurrence-local UID epoch/UID。每个 + exact locator 只有一条 live occurrence Relation；不同 Mailboxes 的 locators 可以共享 canonical Email endpoint， + 但一个 Mailbox/Email pair 至多一条 live Relation。远端 membership 消失时删除 exact relation，不创建历史 + tombstone。 +- `manages` / `contains` 使用 normalized active direction。反向遍历可以描述为 `managed by` / `contained in`，但 + 不为此重复持久化 inverse relations；这是 producer common pattern，不是 Relation validation rule。 +- 删除 operational SourceModel 不删除 Source Block、`manages`/`collects` relations 或 collected graph。Relation + 表达 provenance/binding，不证明 live credentials 或 executor；远端操作必须另外解析仍存在的 operational Source。 +- Mailbox Block 永久保持 Source-scoped：每个 Mailbox 只有一个 `manages` 来源；不同 Sources 即便暴露同一个 remote + mailbox 也保留不同 Blocks。MAILBOXID 在一个 Source 内支持 Mailbox rename continuity；EMAILID remains Email + content evidence and may participate in D-262 canonical reconciliation only under an explicitly supported comparison + scope。 +- `sources.id` 继续拥有 Source identity；`sources.block` 是无独立 sequence/default 的 nullable `UNIQUE` Block FK。 + Relation 始终只是 Block-to-Block；Source producer 读取自己的 `source.block` 并用该 BlockRef 写入 provenance。 + +## Source Anchor Contract + +- exact resolver ID：`core.source.v1`。 +- canonical content：`{ "id": SourceRef, "type": exact Source type, "nickname": string | null }`。 +- `sources.block` 是 nullable unique Block FK。Source 可以在尚未需要 provenance endpoint 时没有 anchor；首次 + producer 写入 `collects`/`manages` 前由 SourceManager 并发安全地创建并绑定，之后不替换。 +- SourceModel 始终拥有 id、type、nickname、config、state、collect_at；Block content 中的 id/type/nickname 只是为 + resolver `get_label/get_text` 与 Source 删除后的历史可读性服务的 projection，不接管 authority。 +- Source Block 是否 operationally active 由 unique `sources.block` binding 是否存在派生，不进入 canonical content。 +- Source Block 与 SourceModel 的 identity/lifecycle 不同；Source 删除后 content 中的 SourceRef 作为历史 descriptor + 保留，不被新 Source 根据相似 config 自动复用。 +- `SourceManager.ensure_block(source, session)` 锁定 Source row，在同一 caller transaction 内创建或复用 anchor， + 并让 `{id,type,nickname}` projection 与本次观察到的 SourceModel 一致。该 postcondition 不另加 `refresh` 参数； + Resolver label/text 只读 Block content。 + +## Canonical Mailbox Content + +- exact resolver ID：`extensions.mail.mailbox.v1`。 +- exact content： + + ```json + { + "name": "Sent", + "special_uses": ["\\Sent"], + "mailbox_id": "F123456" + } + ``` + +- `mailbox_id` 可为 null。它只在 owning Source 内支持 rename continuity；D-248 已永久禁止跨 Source 合并 + Mailboxes，且 `Source --manages--> Mailbox` 已提供 comparison/access scope，因此不复制 host/port/username。 +- `special_uses` 是去重、稳定排序后的 adapter-understood standards-backed user-role attributes。保留例如 + `\\Sent`、`\\Drafts`、`\\Junk`、`\\Trash`、`\\Archive`；不保存 generic subscription、selectability、child hints、 + transient marked state 或 unknown extension attributes。 +- 不包含 SourceRef（由 `manages` relation 表达）、UIDVALIDITY（属于 `contains` occurrence）、LIST delimiter、generic + attributes、message counts、namespace classification 或派生 path。Source 可以在 discovery/config filtering 时临时 + 使用这些 protocol facts，而不复制到 info-base。 +- `get_label()` 返回当前 mailbox name；`get_text()` 只投影 name 与 recognized special uses。 +- **Status**: D-249 的较宽 shape 已由 D-258 收窄。 + +## Mailbox Occurrence Relation + +One current remote occurrence is a Mailbox → Email Relation whose content is canonical compact JSON： + +```json +{ + "type": "contains", + "uid_validity": 3857529045, + "uid": 42 +} +``` + +- `uid_validity` and `uid` are required positive IMAP integers and jointly identify the occurrence within the Source-scoped + Mailbox。 +- A Mailbox/Email pair has at most one live `contains` Relation。When another UID in the same Mailbox matches the same + canonical evidence，Collection creates another Email Block；cross-Mailbox locators may still share one endpoint。The + Relation remains the exact external locator and remote-access authority。 +- Reliable remote removal deletes only the exact matching occurrence Relation。No tombstone or Email deletion follows；a + mailbox move is removal of the old occurrence plus addition of the new occurrence。 +- On UIDVALIDITY change，all old-epoch occurrence locators for that Mailbox are invalid。The Source deletes those stale + Relations、resets its permitted sync cursor/validator state and incrementally rebuilds current occurrences through + ordinary bounded collect jobs；it does not retry or require one job to finish the mailbox。 +- `INTERNALDATE`、`EMAILID` / `THREADID`、flags、mailbox sequence numbers and message body/content do not enter this + Relation。EMAILID persists in Email root content as optional server evidence；all durable flags use separate plain + MailFlag Relations whose owning Mailbox and unique Mailbox/Email membership derive the locator；UIDVALIDITY + UID remains + the remote-access authority。 +- **Status**: D-257 locator/reset retained；D-262 permits cross-Mailbox endpoint reuse while rejecting same-Mailbox + multiplicity，superseding both D-260's global non-reconciliation and D-261's locator-qualified flags。 + +## Reply / Reference Anchor Resolution + +- A parsed Message-ID resolves eligible Email candidates。Exactly one is reused；zero or more than one creates a new ordinary + Email Block containing only that Message-ID and null remaining root facts。 +- Creating on ambiguity preserves the exact authored reference without falsely selecting one candidate or multiplying the + relation across all candidates。The accepted cost is bounded best-effort duplicate anchors，not incorrect graph facts。 +- Later Collection applies the same D-263–D-265 path。One compatible anchor is completed in place and retains inbound + Relations；multiple anchors remain ambiguous and therefore do not authorize mutation of any one of them。 +- “Incomplete anchor” is not persisted state or a separate resolver/lifecycle。It is an ordinary Email whose currently known + content/graph is sparse。 + +## Status + +Mail identity、occurrence、canonical reconciliation and reference-anchor behavior are frozen through D-266。Reopen only for +implementation evidence or a new product use path，not for speculative deduplication completeness。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-state.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-state.md new file mode 100644 index 0000000..b48a2e5 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mail-state.md @@ -0,0 +1,127 @@ +# Mail Occurrence and Remote State + +- **Status**: D-262 best-effort canonical Email + plain MailFlag topology accepted；D-263–D-266 have since closed Email + reconciliation/reference behavior。Canonical MailFlag content、normalization and synchronization remain the active edge。 +- **Protocol evidence**: [Remote Mail State and Flag Facts](../evidence.md#remote-mail-state-and-flag-facts)。 + +## Accepted D-262 Topology + +```text +Source --manages--> Mailbox +Mailbox --contains {type:"contains", uid_validity, uid}--> canonical Email +Mailbox --has--> MailFlag +MailFlag --tags--> canonical Email +``` + +- One Email Block is a best-effort canonical authored Email endpoint。Several exact remote occurrence locators may point + to it across Mailboxes，but one Mailbox never points multiple live UIDs to the same Email Block。 +- `contains` is the current membership fact and exact remote-access locator only。It does not own mutable flag fields。 +- MailFlag is scoped by its owning Mailbox。A plain `tags` Relation therefore identifies one exact occurrence through the + unique Mailbox/Email `contains` edge without repeating the locator。 +- Removing one flag deletes only that `tags` Relation。Reliable EXPUNGE/removal deletes the occurrence's `contains` + Relation and every plain tag between that Mailbox's MailFlags and the Email，while preserving collected Email content。 +- The Mail Source maintains at most one live `contains` Relation for each `(Mailbox, Email)` pair。A second same-Mailbox UID + creates another Email Block even when canonical evidence matches；generic InfoBase does not own this producer invariant。 + +## Unified Flag Boundary + +- Persist IMAP `\Seen`、`\Answered`、`\Flagged`、`\Deleted`、`\Draft` and observed keywords through the same MailFlag + Block + plain `tags` Relation topology。 +- These flags have different use/action semantics，but they are all entries in IMAP's FLAGS message attribute and are all + changed through STORE against a mailbox occurrence。UI meaning does not create a second persistence category。 +- `\Deleted` is a flag meaning “marked for later removal”，not the absence of membership。Until EXPUNGE，both `contains` and + the `\Deleted` tag exist。After reliable EXPUNGE evidence，neither occurrence fact remains；no deletion tombstone is added。 +- Do not persist deprecated `\Recent`。It is session-derived rather than durable state and has no accepted use value。 +- Canonical MailFlag content includes `name` plus nullable `description`。Description has independent resolver/retrieval/LLM + value，but Base IMAP does not supply it；D-268 freezes provider-native → standards-backed → null adapter authority。Name + normalization and mailbox-scoped identity are also frozen there；no flag enters Email root content or `contains` merely + to simplify reading。 + +## Canonical MailFlag + +- exact resolver ID：`extensions.mail.flag.v1`。 +- exact content：`{"name":"\\Seen","description":"The message has been read."}`；`description` is nullable。 +- `name` is ASCII case-insensitive identity inside one owning Mailbox。Known names use standards spelling；an unknown + keyword retains first-observed spelling and later casing-only variants reuse it。 +- Description selection is provider-native adapter metadata when actually available，otherwise stable non-localized + standards-backed prose for known names，otherwise null。It is semantic canonical metadata rather than an IMAP wire fact。 +- Do not persist `kind`/`is_system`（derivable from name）、`permanent`/`mutable`（operational mailbox/session capability）、 + scope references（owned by graph）or `description_source`（derivable from the owning adapter）。 +- When one exact occurrence's complete FLAGS list is observed，replace its graph state：ensure present durable tags and + remove absent ones。Do not treat a complete snapshot as an append-only event。 + +## Resolved Comparison — Locator-Qualified versus Plain `tags` + +The Mailbox already scopes each MailFlag through `Mailbox --has--> MailFlag`。Therefore locator qualification does **not** +add precision across different Mailboxes；their flag Blocks are already distinct。It only distinguishes multiple current +UID occurrences that reconcile to the same canonical Email inside one Mailbox。 + +### Locator-qualified `tags {uid_validity, uid}` + +- **Benefit**：preserves exact flags for arbitrary many `contains` Relations between one Mailbox/Email pair。Remote STORE + and EXPUNGE cleanup can target the locator carried by the flag Relation without deriving it from another Relation。 +- **Damage**：duplicates the occurrence locator into every flag edge and creates a cross-Relation invariant：each tag must + match one `contains` edge。Flag add/remove、UIDVALIDITY reset and occurrence removal must update both representations + coherently，while generic FK/schema cannot enforce the Mail-owned JSON reference。 +- **Semantic damage**：Relation.content is the dynamic-property text consumed by generic graph projection and semantic + retrieval。Embedding UID numbers and locator JSON with the predicate adds operational noise to an otherwise clear + `MailFlag tags Email` fact，unless another protocol-specific interpretation layer is introduced。 +- **Return boundary**：the added precision has value only when same-Mailbox duplicate occurrences share one Email endpoint + and can hold divergent flags。 + +### Plain `MailFlag --tags--> Email` + +- **Benefit**：one relation owns one semantic fact with no duplicated locator。Graph reading、Relation text projection、 + semantic retrieval and idempotent add/remove all use the ordinary generic Relation shape。 +- **Derivable operation target**：the owning Mailbox is found through `Mailbox --has--> MailFlag`；its one live + `Mailbox --contains--> Email` edge supplies UIDVALIDITY + UID。Remote action remains exact if that edge is unique。 +- **Damage without an invariant**：if one Mailbox has two locators pointing to the same Email，plain `tags` cannot express + divergent flag state or multiplicity。Removing one occurrence's flag can erase the other occurrence's state；a write may + target the wrong UID。This is most harmful for `\Deleted` and any future remote mutation。 +- **Low-cost safety condition**：Collection can maintain at most one live `contains` Relation per `(Mailbox, Email)` pair。 + When a second UID in the same Mailbox matches the same canonical evidence，it creates a separate Email Block instead of + reconciling to that endpoint。Cross-Mailbox reconciliation—the common and high-value deduplication case—remains intact。 + +### Current ROI Reading + +- The Enron directional sample observed zero strict duplicate-content groups inside the same owner/folder，while cross-folder + duplicates were material。This is not a protocol guarantee，but it places the qualifier's protected case at the narrow + edge and the cross-Mailbox canonicalization benefit in the common path。 +- Under the one-live-contains-per-Mailbox/Email invariant，plain `tags` has the same operational precision for all accepted + graph states and fails safely by declining same-Mailbox reconciliation rather than by collapsing mutable state。 +- D-262 therefore supersedes D-261's locator-qualified tag content with plain `tags` plus that reconciliation invariant。 + Sir accepted the explicit loss：same authored Email duplicated inside one Mailbox becomes multiple best-effort canonical + Email Blocks。 + +## Superseded D-260 Alternative + +- D-260 made every exact locator create a separate Email Block，which avoided locator-qualified flag Relations but caused + systematic duplicate semantic content across folders/providers。 +- D-261 restored the canonical/occurrence split；D-262 removes locator qualification after proving Mailbox-scoped MailFlags + plus the one-live-contains invariant retain exact mutable state。No MailOccurrence association Block is introduced。 + +## Empirical Duplicate-Risk Calibration + +- Dataset：[CMU Enron Maildir](https://www.cs.cmu.edu/~enron/) through the + [column-preserving mirror](https://huggingface.co/datasets/dpdl-benchmark/enron-maildir)，517,401 records preserving + owner/folder structure。The mirror's Message-ID values are all unique and therefore unusable for measuring Message-ID + reconciliation。A strict proxy signature over authored date、sender、To/Cc、subject and body was used instead。The corpus + has no IMAP UID or flags and has known export/integrity transformations，so these are directional sample measurements，not + universal mailbox probabilities。 +- Same owner + same folder + same strict signature：0 observed。This supports treating same-Mailbox duplicate-occurrence + flag ambiguity as uncommon，while D-262 removes the ambiguity by declining that exact reconciliation when it occurs。 +- Same owner across folders，all folders：359,308 records belong to duplicate groups and 228,226/517,401 (44.1%) are excess + copies relative to one canonical record。This upper bound is dominated by `all_documents` and `discussion_threads` + aggregate folders。 +- Excluding those two aggregate folders：85,713/330,689 (25.9%) records belong to duplicate groups，and 46,636/330,689 + (14.1%) are excess copies。Restricting to six common mailbox families gives 60,018/254,847 (23.5%) exposed and + 30,024/254,847 (11.8%) excess。 +- Provider topology can be worse：[Gmail explicitly exposes labels as IMAP folders and provides X-GM-MSGID](https://developers.google.com/workspace/gmail/imap/imap-extensions) + to identify one message across multiple folders。Collecting All Mail plus Inbox/labels without reconciliation can + therefore duplicate a large share of one Source by construction。 + +## Active Question + +The scheduled sync ladder and UIDVALIDITY's two distinct placements are frozen through D-270。Select the next Mail unit +edge from the remaining collection/action、MIME materialization、client rendering and Acceptance gaps；do not expand +checkpoint serialization without implementation evidence。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mime-materialization.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mime-materialization.md new file mode 100644 index 0000000..e66b494 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/mime-materialization.md @@ -0,0 +1,169 @@ +# Mail MIME Materialization + +- **Status**: D-271 freezes reconciliation eligibility required for exact remote-part access；D-272/D-273 freeze one Mail + Source and one protocol-neutral Mail Resolver family as sibling clients of a selected extension-owned protocol adapter； + D-274 freezes adapter-owned checkpoint interpretation/proposal versus Source-owned durable lifecycle；D-275 separates + the public one-protocol-per-Source fact from its code adapter；D-276 separates `protocol` from typed `parameters`。 + D-277 narrows the current protocol type to `Literal["imap"]`；D-278 retains one small shared + `create_mail_adapter(protocol, parameters)` construction seam；D-279 freezes one async-context adapter instance per domain + command；D-280 freezes canonical Mail operations instead of protocol primitives。Collection streaming/checkpoint + details are delegated by D-281 after reserving `collect` for Source。D-282 promotes writable materialization Storage + selection to a Source-wide explicit → deployment default → built-in PostgreSQL fallback policy。D-283 persists the + explicit selection as nullable `sources.storage`。D-284 corrects the tentative defensive getter：Storage + registry/bootstrap owns code/catalog capability consistency；D-285 places its derived projection on + `storage_types.writable` and constrains `sources.storage` to writable target types。D-286 freezes one outgoing `content` + Relation as the initial durable materialization authority；D-287 corrects exact-one enforcement—any existing content child + short-circuits remote routing and benign duplicates remain usable graph facts。D-288 promotes shallow semantic completion + to the Resolver base contract。D-289 keeps SolvedMimePart's content singular；D-290 freezes database enforcement of + writable Source targets。D-291 corrects D-289's tentative stability rule：InfoBaseManager returns any one matching child + without uniqueness、ordering or repeat-read stability guarantees。This surface is closed for implementation planning。 +- **Product boundary**: Collection persists text/HTML bodies and attachment/inline metadata but not non-text bytes。A later + durable semantic content child is additive enrichment behavior executed by the Mail MIME-part Resolver；transient + streaming remains use。 + +## No-Guess Occurrence Invariant + +Remote MIME `part_id` is positional inside one occurrence's MIME tree。A Message-ID alone does not prove that two +occurrences have identical bytes or structure，so materialization must not compare metadata and silently choose the first +plausible UID。 + +Collection therefore applies this candidate guard before canonical Email reuse： + +1. known/comparable exact occurrence or scoped EMAILID may reuse； +2. a sparse Message-ID reference anchor may be completed by its first occurrence； +3. any other Message-ID-only match involving attachment/inline MIME metadata creates another Email Block。 + +Consequently，a canonical Email with remote MIME components can have several live locators only when collection evidence +already establishes equivalent immutable content。Resolver may select an operational one for availability，not as a semantic +guess。No per-part remote binding、metadata fingerprint or user-facing “try another UID” recovery interaction is introduced。 + +## Frozen Dependency Boundary + +```text +Mail Source ----------------------> selected protocol adapter -----> remote Mail server +Mail Resolvers needing remote I/O -> selected protocol adapter -----> remote Mail server + | + +--------------------------> WritableStorage + InfoBaseManager +``` + +Both callers resolve and validate typed access config before constructing/calling the adapter。Resolver may resolve the +live Source row behind provenance as data，but does not call the Source behavior。There are no parallel protocol-specific +Source/Resolver domain families。The adapter remains free of graph and Storage ownership。IMAP is the current concrete +protocol；the boundary permits a future POP3 adapter without pre-designing its locator grammar now。The selected adapter +declares/interprets typed protocol checkpoint state and returns a next-state proposal；Mail Source alone decides whether and +when to persist it。The Source persists public protocol choice，not an internal/versioned adapter ID；a shared explicit code +factory derives the corresponding implementation。 + +## Approved Command + +Exact Adapter request/result/batching shapes are implementation-owned under D-281's boundary。 + +## Command Input and Graph Context（proposal） + +The domain input is one MIME-part metadata BlockRef whose resolver is `extensions.mail.mime_part.v1`。The Resolver first +reads outgoing `content` Relations。When at least one exists，it resolves those children immediately and does **not** traverse +Email/Mailbox/Source provenance or choose a target writable Storage。Each child may still hydrate its own persisted +`block.storage` through its ordinary Resolver；that is content reading，not materialization routing。 + +Only when no child exists does the Resolver derive，rather than ask callers to supply： + +1. the unique owning Email → MIME-part Relation and its `{role, part_id}`； +2. eligible exact Email occurrences from Mailbox `contains` Relations； +3. each Mailbox's owning Source anchor/binding and live protocol config； +4. the effective Source-wide writable Storage policy。 + +This keeps exact locator、credentials and target Storage routing out of the public call。An existing semantic child can be used even +after the Source is deleted/disabled because no remote access is then required。If creation is needed，one or more +D-271-eligible，proven-equivalent exact occurrences may supply an operational locator；unresolved owner/identity ambiguity or +absence of live Source config is materialization-unavailable。The Resolver never guesses from metadata；exact failover order +among equivalent locators is implementation-owned。 + +## Read versus Materialize（proposal） + +```text +SolvedMimePart { + root: CanonicalMimePart + content: SolvedContentChild | null +} + +SolvedContentChild { + block: BlockModel + solved_content: object +} +``` + +The child wrapper carries the actual result of the child Resolver，not only a BlockRef。Keeping its Block alongside the +solved content preserves resolver identity、graph navigation and future actions without confusing storage-backed +`Block.content`（an opaque pointer）with hydrated/solved content。The Resolver asks InfoBaseManager for one matching related +Block；the manager may implement this as one graph join/filter plus `LIMIT 1` without `ORDER BY`。When redundant Relations +exist，the chosen child need not be stable across reads。Graph multiplicity remains available to ordinary all-Relations +queries and Organization but does not change the singular use-facing meaning。 + +- `materialize_missing=false` reads graph state only and may return null `content`。 +- `materialize_missing=true` permits creating the missing child；it does not require recreation when a child exists。 +- `get_text` and `get_label` use filename、description、media type and other semantic metadata without materializing bytes。 +- Email Resolver asks component Resolvers for read-only projections，so opening an Email does not download every attachment。 +- `refresh` bypasses/replaces local Block hydration、Relation and solved snapshots only。It neither redownloads remote bytes nor + migrates/replaces an existing child；that would require a future explicit command with different effects。 + +Per D-288，the success result omits `created/existing` status because Resolver solving returns semantic completion，not +command-mechanics status。This belongs in the Resolver base docstring and peer-equivalent contract，not a repeated Mail-only +rule。 + +## Concurrent Creation Sequence（proposal） + +```text +Resolver MailAdapter PostgreSQL / Storage + |-- read content edges ----------------------------->| + |<-- existing children? -----------------------------| + | get/solve any one if present | + |-- fetch exact part ------->| | + |<-- decoded bytes ----------| | + |-- lock metadata Block ----------------------------->| + |-- re-read content edges -------------------------->| + |<-- concurrent children? ----------------------------| + | get/solve any one if present | + |-- write bytes + child + content Relation ---------->| + |<-- committed SolvedMimePart ------------------------| +``` + +- Remote I/O happens outside the row lock；slow IMAP cannot block graph writers。A rare race may download the same transient + bytes twice，but the losing command discards them before Storage write。 +- `SELECT ... FOR UPDATE` on the metadata Block reduces cooperating-producer duplicates。The generic Relation table is not + distorted with a Mail-specific unique index and correctness does not rely on exact-one enforcement。 +- After the lock，any existing Relation short-circuits creation regardless of which Storage the current Source policy would + now choose。If a race still leaves several children，all remain ordinary usable graph facts；the Resolver uses any one + without detecting multiplicity or promising stable selection，and Organization may later remove redundancy。 + Materialization does not fail or create another child。 +- If still missing，resolve the selected exact occurrence's Source policy：`sources.storage` → deployment default → `-4`。 + Changing that policy affects future materializations only。 +- Current PostgreSQL binary Storage writes bytes、semantic child and Relation through one caller transaction。This current + guarantee does not invent cross-system rollback/exactly-once semantics for a future S3/Nextcloud Storage。 + +## Classification and Content Effect（proposal） + +- Mail owns the evidence ladder：protocol-declared MIME type，then byte signature，then `core.file.v1` fallback。Existing + `ResolverManager.match_media_type()` maps each candidate to an installed resolver and common `detect_media_type()` supplies + signature evidence。RSS already composes the same primitives in its own protocol order and Memos calls the matcher during + graph creation；there is no universal ordering method because evidence precedence remains extension-owned。 +- image/audio/video/PDF/EPUB/ZIP select their exact existing core semantic Resolver；unknown content remains a file and keeps + source-authored MIME metadata on the root。 +- text/plain and text/html use the declared charset to transcode into the encoding expected by the core semantic Resolver； + Storage still owns only actual bytes and never owns MIME parsing。 +- Materialization adds only the semantic child and `content` Relation。It does not mutate CanonicalMimePart、Email identity、 + occurrence locator or Source checkpoint。 + +## Failure Boundary（proposal） + +| Failure | Durable effect | Public meaning | +| --- | --- | --- | +| no exact eligible occurrence / Source binding | none | materialization unavailable | +| remote authentication/fetch/part missing | none | materialization failed with domain cause | +| multiple existing `content` Relations | none | get/solve any one；no uniqueness/order/stability promise；Organization may later reduce redundancy | +| explicitly selected/configured Storage is missing or read-only | none | Source/deployment configuration failure；do not hide the bad choice | +| Source and deployment Storage both absent | use built-in `-4` | ordinary fallback，not failure | +| PostgreSQL bytes/graph write failure | transaction rolls back | materialization failed | +| concurrent command created child first | winner remains；loser writes nothing | ordinary success with existing child | + +There is no internal retry、alternate-part guessing、created/existing public status or eager duplicate repair。Errors matter to +an explicit download/view operation，so they surface to the calling capability/UI rather than becoming a silent no-op。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/runtime-closure.md b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/runtime-closure.md new file mode 100644 index 0000000..f8126ee --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/mail-extension/technical-design/runtime-closure.md @@ -0,0 +1,325 @@ +# Mail Runtime Closure + +- **Status**: implementation synthesis only；withdrawn as one review batch because it compressed too many independent + judgments。Its sections are reviewed through smaller MIME、Source-foundation and Mail-collection batches before freezing。 +- **Boundary**: closes Mail materialization、Source/config、ordinary/backfill collection and intentional remote effects。 + Exact MailAdapter DTO names、wire batching and checkpoint serialization remain implementation-owned under D-281。 + +## Common Source Contract Pressure（pending separate batch） + +Mail requires four additive common Source fields/contracts rather than hiding them in its protocol config： + +1. `sources.storage`: nullable writable-target reference frozen by D-283。 +2. `sources.created_at` / `updated_at`: database-owned Source lifecycle timestamps。Mail uses `created_at` as the exact local + setup boundary for its first ordinary collection；the timestamp is not external message time。 +3. replace legacy `collect_at` with nullable `collect_cron`: one standard five-field cron expression；null disables scheduled + job creation。This admits minute/hour/day cadence without adding interval/schedule-type abstractions。 +4. `sources_types.collect_config_schema` and nullable `backfill_config_schema`: Source registration derives independent JSON + Schemas for ordinary collect and optional backfill commands，just as it already owns setup `config_schema`。Absence of the + latter means the Source type does not support backfill；client-web does not accept blind JSON。 + +Source registration/runtime validates Source、ordinary-collect and optional backfill configs independently。This is framework- +boundary validation；generic Job remains an execution envelope and does not interpret source-specific intent。 + +### Scheduled command coordination（open pressure） + +Repository evidence invalidates treating `collect_cron` as only a field rename： + +- `SourceManager.set_up_collect_jobs()` currently reads `sources.collect_at` only during runtime bootstrap and installs + process-local APScheduler jobs；client-web writes the Source row directly，so a schedule edit does not reconfigure the + running scheduler until restart。 +- every core-py Peer that boots the Source type can install and fire its own schedule。Each firing creates a different collect + job，so the existing atomic `PENDING → RUNNING` claim prevents two runners executing one row but cannot prevent duplicate + scheduled rows。 +- every Peer currently scans all pending jobs；without local Source-type eligibility in claim，a Peer lacking the extension + implementation can claim a job it cannot execute。This conflicts with the already accepted job-table delegation model。 + +D-293/D-294 correct the first proposal：the Collect-domain Job must not become a generic scheduling ledger，but a Cron +abstraction alone is also insufficient。The exact requirement is distributed occurrence materialization：several capable +Peers may observe one due schedule，yet collectively create at most one durable Collect Job for that canonical occurrence。 + +The active topology candidate is： + +```text +Cron domain + recurring definition + timezone + due/misfire/firing coordination + | + +-- invokes one exact registered command-creation target + | + +-- Collect-owned target creates ordinary SourceCollectJob + | + +-- capable Peer claims and runs Collect-domain Job +``` + +The topology remains a useful ownership sketch，not yet a solution：without a shared atomic election/materialization protocol， +each Peer can still invoke the target once。PostgreSQL may serialize competing application attempts，but it must not evaluate +the schedule or implement domain Job creation through triggers/procedures。The winning application Peer must create the +Collect-owned command；normal Collect runner delegation remains unchanged except that claim must admit only locally registered +Source types。 + +D-295 places the primary arbitration in the invariant that one Cron may associate with at most one non-terminal +`SourceCollectJob`。The Cron cursor is nullable `last_scheduled_for`，and the winning Job insertion plus cursor advance commit +atomically。This both converges ordinary simultaneous attempts and prevents overlap accumulation。 + +Do not prematurely realize that invariant as `SourceCollectJob.cron` plus a partial unique index。A Job is an independently +executable Collect command after creation and must not depend on its optional creation mechanism；manual、Cron and future +producers must not become nullable provenance columns on the command model。The physical association belongs on the creation +side or an exact Collect–Cron binding seam。Its smallest shape and whether a locked creator-owned `last_job` pointer dominates +accepting the benign terminal-transition race remain the active review question。 + +One narrow edge remains before persistence design：a Peer may retain an old cursor snapshot until after the winning Job has +already become terminal，at which point partial active uniqueness alone no longer rejects the same occurrence。Freeze the +smallest exact occurrence fence for this stale-reader case；do not depend on Job duration making it unlikely。Generic Job +extraction is no longer the leading question。Schedule persistence、target binding、timezone and misfire semantics follow only +after this protocol is closed；they must not be silently derived from each Peer process's local timezone。 + +### Global Job + global Cron premise（active evaluation，not frozen） + +Sir reopened the design under a stronger premise：promote the existing Source Collect Job lifecycle into a global durable Job +module，and make Cron global rather than Collect-owned。Under that premise the prior binding candidate changes shape： + +```text +Global Cron + ├── job_type: core.source.collect.v1 + ├── job_parameters: {source, config} + └── last_job ──> Global Job envelope +``` + +- `crons.last_job` becomes a correctly directed creator-owned reference to `jobs.id`。It holds the most recently materialized + Job even after terminal completion；Job completion never calls back into Cron。 +- Cron-row locking、terminal-state recheck、Job creation、`last_job` replacement and `last_scheduled_for` advancement share one + transaction。This closes the fast-terminal stale-reader race without putting `cron` or `scheduled_for` on Job。 +- **Binding correction**：do not add `SourceCollectCronBinding`。The recurring command is a user-authored composition of a + global Cron and a typed Job template；`job_parameters.source` already names the called Source。Source neither owns nor needs + awareness of whether users scheduled its collect behavior。A binding table would duplicate that composition、imply reverse + Source ownership and require lifecycle/query machinery without adding required behavior。 +- Cron remains narrower than a generic invocation system：one firing copies its exact `job_type` and typed + `job_parameters` into one persisted global Job。It does not execute the Job or understand Source collection。 +- `last_scheduled_for` and `last_job` have separate jobs。Under the locked transaction，`last_scheduled_for` proves that the + same canonical occurrence was already dispatched；`last_job` only tells whether the previously produced Job remains + non-terminal and therefore whether later occurrences must be coalesced rather than stacked。Neither field alone provides + both guarantees。 + +The global Job module is justified only as a deep durable background-command lifecycle：exact registered Job types、common +pending/running/terminal timestamps and atomic capable-Peer claim。It must not become `type + arbitrary unvalidated JSON + +universal behavior`，must not absorb request-response Peer delegation，and adds no retry。Whether typed Job input is stored in +the global envelope or an exact domain detail table remains a later persistence decision；the latter preserves FK authority but +costs an additional joined lifecycle。 + +`sources.collect_at` is removed as an authority，not retained as a projection or dual-written compatibility field。A non-null +legacy value maps conceptually to one global Cron whose `job_type` is `core.source.collect.v1` and whose typed parameters carry +that Source ID；null maps to no Cron。The old process-local `SourceManager.set_up_collect_jobs()` is deleted；APScheduler may +remain only as a Peer-local wake-up timer for `CronManager.check()` and `JobManager.check()`。client-web edits the global Cron +surface instead of `sources.collect_at`。Exact legacy-row conversion is blocked by the old schedule's implicit process timezone；under +the already accepted clean-baseline policy，reset is preferable to inventing a timezone，unless preflight discovers an explicit +deployment authority that makes lossless conversion possible。 + +### Cron schedule representation（frozen by D-298） + +- `crons.schedule` is one five-field UNIX cron expression。 +- deployment config `core.cron` owns one IANA `timezone`；absence falls back to `UTC`。 +- Peer-local process/browser/OS timezone never interprets durable Cron。 +- no `CRON_TZ`-style persisted dialect、per-row timezone override、seconds field or interval union enters the MVP。 + +### Global Job persistence（frozen by D-299；handler race open） + +- `job_types` projects exact IDs、descriptions and parameter JSON Schemas from the runtime Handler Registry。 +- `jobs` uses an `int8` identity and persists type、typed JSONB parameters/state、status and lifecycle timestamps；the old + Source-specific Job table is hard-cut rather than retained as a detail table。 +- `core.source.collect.v1` parameters carry Source and collect config；the Handler owns their semantics。 +- Job remains a durable background-command lifecycle，not a generic invocation/retry/delegation mechanism。 +- a boolean `can_handle(parameters)` is only a candidate。The design must close the interval between capability inspection and + atomic pending-to-running claim，including extension/Source handler deactivation，before freezing the Handler interface。 + +### Handler eligibility and claim（frozen by D-300） + +```text +pending candidate + -> validate parameters + -> local can_handle(parameters) + -> atomic PENDING-to-RUNNING conditional update + -> run only when the update returns the claimed Job +``` + +False eligibility leaves the Job untouched；a lost conditional update ends that Peer attempt。`can_handle` is a side-effect-free +local implementation check，not an external readiness probe。Capability disappearance after the check is an accepted narrow +TOCTOU rather than pressure for leases、draining、reverse status transitions or retries。 + +### Job execution budget and resumable Mail progress（persistence frozen by D-305） + +- Job owns an explicit configurable execution timeout；remove the current universal five-minute running timeout。 +- do not add `jobs.peer` or use Peer discovery lease expiry as Job abandonment detection。Job execution must not depend on + Peer delegation/discovery lifecycle merely because both are distributed capabilities。 +- Mail collection/backfill treats one Job as a bounded opportunity to advance，not as a promise to finish an entire historical + horizon。Partial graph effects remain durable；ordinary synchronization may continue from its protocol checkpoint，while + backfill deliberately has no durable continuation checkpoint under D-307。 +- reaching an execution/load boundary neither rolls back prior graph writes nor creates retry/attempt lineage。Cron remains + unaware of progress and simply materializes future Jobs under its ordinary schedule rules。 +- timeout conditionally closes an overdue `running` Job as terminal `timed_out`。That outcome describes only this invocation； + a normal bounded return is `finished` and an escaping execution/domain error is `failed`。A late worker close cannot overwrite + an already-terminal row。 +- incremental Source checkpointing is recommended because collect may be interrupted by timeout or process/system failure， + but is not a required Source capability。A weaker Source may rescan and rely on its own identity/reconciliation；JobManager + does not enforce checkpointing、resume traversal、rollback or retry。 +- `job_types.default_timeout_seconds` owns the non-null exact-type default。Direct/manual creation and + `crons.job_timeout_seconds` may override it；`jobs.timeout_seconds` snapshots the resolved non-null value at creation。 + Later type/Cron edits do not reinterpret existing Jobs。 +- the positive integer seconds representation is portable across PostgreSQL/PostgREST、Python and TypeScript。The budget + begins at successful claim，so pending time does not consume it；expiration derives from `started_at + timeout_seconds`。 +- the executing worker establishes a local deadline and best-effort cancels the Handler。Independently，any Job worker may use + database time to conditionally close an overdue `running` row as `timed_out`，covering original-worker/process loss without + storing an execution Peer。The database terminal state wins over every late close。 +- cancellation-insensitive external operations may rarely produce effects after the terminal timeout；those remain valid + partial effects。MVP does not add Peer-heartbeat coupling、execution leases、requeue、retry or per-Job process isolation。 + +### Cron missed-occurrence semantics（corrected and frozen by D-302） + +- Cron only examines the canonical current minute in the deployment timezone。If no capable checker observes a matching + minute，that occurrence is lost；there is no catch-up、coalescing debt、misfire option or `active_from`。 +- under the locked Cron row，equal `last_scheduled_for` suppresses duplicate creation for the current occurrence；a + non-terminal `last_job` suppresses overlap。Successful creation atomically writes both fields。 +- creation、re-enable、schedule/template edits and timezone edits simply affect subsequent current-minute checks；they do not + create historical debt。 +- run-now directly creates a Job from the Cron template and does not update Cron progress。 + +## Deployment Source Policy（pending separate batch） + +- deployment config key/schema：`core.source` / `core.source.config.v1`。 +- model：`SourceDeploymentConfig(default_storage: StorageID = -4)`。 +- effective target：non-null `sources.storage` → configured deployment default → hard-coded built-in `-4` when the config + record is absent。A present invalid reference does not fall through。 +- Storage registry derives `storage_types.writable` from registered `WritableStorage` implementations；database integrity + prevents `sources.storage` from selecting a read-only type。Deployment JSON references remain use-time defended and + readiness-visible under the existing config-reference policy。 + +## Mail Configuration（pending separate batch） + +### Extension default + +`MailExtensionConfig.default_excluded_mailboxes` owns one `MailboxExclusionPolicy`： + +- exact normalized mailbox names； +- canonical special-use roles，defaulting to `drafts`、`junk` and `trash`。 + +No glob/regex/provider-label language is introduced before real pressure。 + +### Source config + +```text +protocol: Literal["imap"] +parameters: IMAPParameters +excluded_mailboxes: MailboxExclusionPolicy | null +ordinary_mark_as_seen: bool = true +backfill_mark_as_seen: bool = false +synchronize_deletions: bool = false +``` + +- `protocol` discriminates typed parameters；IMAP parameters contain endpoint/security/login values only。 +- null exclusions are a transient materialization request，not dynamic inheritance。At Source creation or the first validated + Mail command，whichever has the necessary extension-config snapshot，the runtime resolves the current extension default and + compare-and-set persists that complete policy into Source config；the command then uses one concrete Source-owned snapshot。 + Later extension-default changes affect only future/unmaterialized Sources。Explicitly resetting the field to null requests + one new materialization on next use；it does not permanently couple that Source to the extension default。 +- a non-null Source value replaces the complete policy；there is no field-level merge authority。Exact-name and special-use + role exclusions are ORed when evaluating one Mailbox，but their persisted lists do not merge with the extension default。 +- ordinary/backfill seen mutation are independent linear Source policies。Each command reads only its matching field；there is + no Job-local override、inheritance or fallback between them。 +- synchronized deletion consumes only trustworthy incremental removal evidence；when QRESYNC/VANISHED is unavailable，the + Source leaves membership untouched and records the unavailable behavior in diagnostics rather than scanning all UIDs。 +- Newsletter Source/Resolver and `/mail/imap` creation shortcuts are PoC remnants and are hard-cut。Generic shared-database + Source creation plus one exact Mail Source type remains the authority。 + +## Independent Mail Collect Commands（pending separate batch） + +Mail registers two exact Job command paths rather than a discriminated intent union： + +```text +core.source.collect.v1: + parameters: { source, config: {} } + +core.source.backfill.v1: + parameters: { + source, + config: { + since: date, # required, inclusive remote INTERNALDATE boundary + before: date | null # optional, exclusive remote INTERNALDATE boundary + } +} +``` + +- `core.source.collect.v1` is the ordinary manual/scheduled Job type；empty Mail config is valid。Cron templates use this exact + type without learning Mail schema。 +- `core.source.backfill.v1` is a distinct Job type。There is no persisted `intent` discriminator，and a Source type with no + backfill config/implementation cannot handle it。The ordinary product journey presents it as an explicit action，but Cron + remains free to materialize any valid Job template and does not judge recurrence value。 +- Backfill is an explicit historical collect，not a parallel lifecycle or legacy `full`。Its boundaries use transient remote + occurrence INTERNALDATE because that is the server query authority；INTERNALDATE is not copied into canonical Email。 +- A required `since` prevents an accidental unbounded history traversal。Source mailbox exclusions apply equally to both + intents；a job-level mailbox selector/override is not introduced without a user path。 +- Backfill uses only `Source.backfill_mark_as_seen`（default false）；ordinary collection uses only + `Source.ordinary_mark_as_seen`（default true）。Changing either is an explicit persisted Source-config update，not a per-Job + override。 +- typed validation requires `since < before` when `before` is present。A valid range with no matching occurrences finishes as + an ordinary no-op；neither condition earns a domain-specific Job outcome。 +- Backfill never reads、advances or regresses the ordinary continuous-sync checkpoint。A timed-out/failed backfill is followed + only by an explicit new one-shot Job，normally with a narrower range or larger timeout，and relies on graph reconciliation； + no durable backfill cursor、campaign or retry lifecycle is added。The ordinary product UI does not encourage a fixed range + as recurring work，but a valid global Cron template remains valid under D-309。 + +## Ordinary Collection and Checkpoint Effects（pending separate batch） + +1. Validate Source and typed job config，resolve live exclusion policy，open one async-context MailAdapter。 +2. Discover eligible mailboxes and ensure Source/Mailbox provenance graph。 +3. For each mailbox，read its adapter-typed checkpoint from Source state。If absent，limit initial ordinary acquisition by + `sources.created_at`；later jobs use the accepted checkpoint and D-269 capability ladder。 +4. Persist each occurrence graph independently，including its authoritative membership、observed flag snapshot and any exact + incremental-removal effect enabled by Source policy。Partial prior effects remain valid and later observation reconciles + them。 +5. A primary graph failure stops traversal of that mailbox before its checkpoint can cross the unaccepted occurrence；the + Source records a bounded diagnostic and continues other mailboxes。It does not invent per-occurrence rollback、retry or a + generic partial-success status。 +6. After each accepted graph commit，perform the matching configured seen action best-effort。On success，also add the local + Seen tag fact； + on failure，record bounded Job diagnostics。The action does not gate accepted collection progress or checkpoint advance。 +7. Advance only successful mailboxes。Under overlapping jobs，briefly lock/merge Source state and accept a proposed mailbox + checkpoint only if the stored checkpoint still equals the command's observed base；never overwrite newer progress。 + +The Source may isolate one mailbox failure and finish the generic job normally，recording bounded diagnostics/counters in +job state。There is no completed-with-errors status、job retry or mailbox-completeness promise。 + +Mailbox exclusion is collection scope，not a graph-deletion command。Excluding a previously observed Mailbox leaves its graph +and checkpoint intact；removing the exclusion lets the adapter continue from that checkpoint。Likewise，enabling synchronized +deletion later is prospective：while it is disabled，trusted remote-removal observations do not mutate the graph and ordinary +checkpoint progress is not held for possible future policy changes。Retroactive deletion reconciliation would require a +separate explicit operation and evidence source，not reinterpretation of an already-accepted checkpoint。 + +## Access-context continuity（frozen by D-311） + +An in-place edit from one configured remote access context to another cannot be treated as proven continuity merely because +the `sources.id` row is unchanged。Existing Mailbox scope、ordinary checkpoints and lazy MIME references all depend on the old +context；blindly resetting only the checkpoint would still mix provenance and can make old remote content unreachable。 + +The selected Adapter projects a non-secret access binding persisted by the Mail Source as validator state after the first +successful adapter entry and before Mail graph effects。For IMAP it includes the public protocol plus normalized host、port、 +security mode and login name，while excluding rotatable password/credential secrets。A later binding mismatch fails use with a +repairable configuration error instead of guessing continuity、deleting old graph or silently rebinding it；the user restores +the old binding fields or creates a new Source for a different context。A remote-I/O Mail Resolver validates an existing +binding but does not initialize or replace Source-owned state。Exact Pydantic field names and client-web edit guidance remain +implementation-owned under this frozen authority rule。 + +## MIME-Part Materialization（closed separately） + +Closed by D-286–D-291 and [Mail MIME materialization](mime-materialization.md)：existing content short-circuits +Source/target-Storage routing；`SolvedMimePart.content` contains any one matching child's Block and direct solved content； +the singular InfoBase read promises no uniqueness、order or stable selection；redundant graph children remain benign and +Organization-repairable；lock/recheck reduces duplicates without making exact-one a correctness condition；Resolver base +solving exposes semantic completion rather than created/existing mechanics。 + +## Implementation-Owned Consequences + +- Mail Source state is a typed map from stable local Mailbox BlockRef to adapter checkpoint；it remains cursor/validator + state，not a collected-item ledger。 +- The generic Source UI projects `config_schema` and `collect_config_schema` through its existing JSON-schema editor；a new + Mail-only setup/backfill page is unnecessary。 +- Exact Pydantic class names、diagnostic field names、IMAP TLS enum spelling、checkpoint DTOs、batch sizes and SQL helper names + are finalized during implementation planning/preflight provided the above contracts remain unchanged。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/acceptance.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/acceptance.md new file mode 100644 index 0000000..926e1f5 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/acceptance.md @@ -0,0 +1,96 @@ +# Memos Extension — Backend MVP Acceptance Contract + +当前 MVP 验收的是 Memos `0.29.1` wire subset 与 MoeMemosAndroid `2.0.4` journey 的可证明闭环。 +每个通过项给出适用的 HTTP、持久 graph/resolver 和 native response 证据；mutation 不得以单个 +200 response 代替 primary info-base mutation 已持久化。 + +## Fixture contract + +- **Server fixture**：以 MoeMemos base URL `https://<deployment>/memos/` 运行 Memos-compatible + endpoint，`instance/profile` 报告 `0.29.1`，投影 + 一个 deployment-scoped profile/creator，并使用一个无时间到期点、可替换/撤销的 Bearer + credential 和至少两页 + NORMAL/ARCHIVED memo 数据。 +- **Memo fixture**：Markdown body（含空白与 hashtag）、memo-side `createTime/updateTime`、 + visibility/state/pinned，以及至少两个可重排 attachments；root content 只保存 + `CanonicalMemo`，附件由 component block/relation 表达。 +- **Graph proof**：create/update/delete 后检查 root block、component block、relation 和 + resolver solved result；禁止引入并行 Memos memo store。成功证明 primary mutation 已持久且 + response 与实际 committed state 一致；D-041 不要求 failure 后没有 orphan/stale components。 + +## Approved primary client matrix + +下表覆盖 MoeMemos APK journey，并由 D-047 固定为 executable fixture contract。 + +| ID | Stimulus | Required proof | Execution | Decision prerequisite | +| --- | --- | --- | --- | --- | +| U-01 | APK 2.0.4 以 `/memos/` base URL 和 Bearer token 登录；version detection 经过 profile | `/memos/api/v1/auth/me`、GENERAL settings 成功；profile 为 `0.29.1`；单一 creator 稳定 | APK + unit | D-039, D-047 | +| U-02 | 同步 NORMAL 与 ARCHIVED | 两次 state 分流；每次 `pageSize=200`、creator filter、page token 直到末页；response 可解析 | APK + unit | D-042, D-047 | +| U-03 | 新建含 body、visibility、时间和 attachments 的 memo | MoeMemos 可先提交 unattached attachment；`POST /api/v1/memos` 成功后 primary root 已持久；resolver response 可被 APK 展示，允许 graph residue/incompleteness | APK + unit | D-042, D-043, D-044, D-047 | +| U-04 | 编辑 body/visibility/state/pinned/attachments | 缺失 mask 按 D-034 从原始 JSON key presence 推导；显式合法 `updateMask` query 保持 upstream 语义；两者都只改变选定 root fields；attachment omission/present-empty/set 被区分 | unit + APK evidence | D-042, D-043, D-044, D-046, D-047 | +| U-05 | 删除 memo | success 后 root 不再被 list/read；owned cleanup 遵循 D-046，允许 residue但不得误删 shared target | APK + unit | D-046, D-047 | +| U-06 | 上传、列举、下载、删除 attachment | `memo=null` orphan 可列举/删除；raw `/memos/file/...` 需 Memos Bearer；attach/reorder 保留 D-040 顺序；failure residue 可诊断 | APK + unit | D-039, D-043, D-044, D-046, D-047 | +| U-07 | resolver/native round-trip | graph → resolver → Memos response 保留 D-042 root facts 与 ordered attachment references；graph-owned fields 不复制进 CanonicalMemo | unit | D-042, D-040, D-044 | +| U-09 | 未列入最小集合的调用 | Explore/users、relations、reactions 等不作为 APK 必需条件；unsupported behavior 不伪装成成功 | unit negative | D-047 | +| U-10 | 经现有 extension API 执行 enable → disable → re-enable | route availability 在同一进程依次为 available → 404 → available；re-enable 无 duplicate routes，core routes 不受影响；decoder remains available for existing blocks | ASGI lifecycle | D-038, D-039 | +| U-11 | 不带 token、带 peer JWT、带 Memos token 分别访问 public detection、core API、Memos protected API | public detection 无凭据可达；core API 只接受 peer JWT；Memos protected API 只接受有效 Memos credential；两种 token 不可互换，普通 extension 仍默认 peer-protected | ASGI auth matrix | D-036, D-039, D-047 | +| U-12 | 经 peer-auth config surface 建立、读取、替换、撤销 PAT，并发送 omitted/invalid updates | persisted/runtime/read PAT config 一致；replace 后 old 立即 `401`、new 成功；revoke 后 protected `401`；omitted 保持、invalid/DB failure 不改变 persisted/runtime state；validation 发生在 persistence 前且不新增 Memos-specific config hooks | config + ASGI | D-039 | + +## Approved packet-required protocol matrix + +Comments 是首版 unit contract，但 MoeMemos 2.0.4 core sync 不调用它。它单独作为 protocol + +graph gate，不计入 APK E2E： + +| ID | Stimulus | Required proof | Execution | Decision prerequisite | +| --- | --- | --- | --- | --- | +| U-08 | `POST/GET /api/v1/memos/{parent}/comments`，再以普通 memo endpoint 更新/删除 comment | comment 是独立 memo root，并以 parent relation 连接；list/read 可由 resolver 还原，update/delete 有 graph 断言 | protocol + unit | D-042, D-044, D-046, D-047 | + +## Exact fixture architecture — D-047 / D-048 + +“Exact” means version-pinned and executable，not complete-server coverage： + +1. **Product-generation fixtures**：Memos 0.29.1 JSON/query/header/status/error examples for only the + approved endpoint subset。 +2. **Client-compatibility fixtures**：MoeMemos 2.0.4 call ordering and deviations，especially public v1 + detection、missing `updateMask`、page token loop and pre-memo attachment upload。 +3. **Family fixtures**：native input → D-042 CanonicalMemo / D-044 graph → solved resolver output，without + importing Memos DTOs into the family layer。 +4. **Runtime/integration fixtures**：route auth/hot lifecycle、PostgreSQL binary storage、residue/delete + safety and client-web config request shape。 + +Candidate test ownership mirrors D-048： + +```text +tests/extensions/memos/family/ +tests/extensions/memos/products/memos/v0_29_1/fixtures/ +tests/extensions/memos/products/memos/v0_29_1/test_adapter.py +tests/extensions/memos/backend/ +tests/extensions/memos/integration/ +``` + +Future flomo adapters add their own product-generation fixtures and must satisfy the reusable family +contract suite。Future collectors reuse family/product mapping tests where applicable but own separate +transport、cursor、reconciliation and partial-result fixtures。 + +## APK/tag evidence + +- 验收 APK 必须是官方 `2.0.4` tag（commit + [`9bfc6517`](https://github.com/mudkipme/MoeMemosAndroid/tree/2.0.4)）构建/下载的 + `moememos-v2.0.4.apk`，并记录 SHA-256 + `5043f14d27c4cc283cb1507a23a84f251e159ab8d3937da9842f2060bd7fe8fa`。 +- 服务端必须固定在 Memos `v0.29.1` tag（commit + [`5f194da`](https://github.com/usememos/memos/tree/v0.29.1)）；测试报告附 profile 返回的 + generation，不能只记录镜像的 `latest` 标签。 +- 证据包至少包含：脱敏 HTTP transcript、APK tag/digest、持久 graph 快照、resolver solved + result 与 create/update/delete 的客户端可见结果。 + +## Execution mapping ownership + +- Acceptance gate 已批准 behavior/fixture contract；candidate test paths表达 owner，不假称文件已实现。 +- [Implementation baseline](implementation-plan.md) 已把 U-ID 映射到 candidate test address、runner、 + command 与实现增量。 +- Preflight 在 packet 中附上已核实的版本、地址、环境与可重复命令；Impact Handshake 再引用 + 这些证据决定是否进入 Execute。 + +候选 test root 是 `tests/extensions/memos/`，按上面的 family/product/backend/integration ownership +分层。ASGI routes、PostgreSQL integration 和 APK E2E 是不同证据，任何一层都不能替代其他层。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/auth-contract.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/auth-contract.md new file mode 100644 index 0000000..eaa940b --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/auth-contract.md @@ -0,0 +1,169 @@ +# Memos Backend MVP — Authentication Contract + +> Confirmed task-state contract（D-039）。This is not durable truth or implementation authorization。 + +## Contract Summary + +- The backend accepts exactly one deployment-scoped Memos-compatible Personal Access Token (PAT)。It + authenticates access to this InKCre deployment，not a terminal user、tenant、session or `ClientModel`。 +- The deployment owner supplies the token through the existing peer-authenticated + `PUT /extensions/memos/config` surface，then enters the same token in MoeMemos。The MVP does not add a + Memos administration endpoint or a one-time secret issuance response。 +- The token is long-lived until replacement or revocation。Replacement has no overlap window；the old + token becomes invalid when the update succeeds。 +- The PAT is ordinary Memos extension configuration。Its raw value is persisted in + `extensions.config`、loaded into runtime config and returned by the existing peer-authenticated config + surface。The current deployment treats database/config peers as trusted operators；the MVP does not + create a Memos-only secret boundary that the rest of the config system does not have。 +- Only `GET /memos/api/v1/instance/profile` is public。`GET /memos/api/v1/status` intentionally remains + unimplemented (`404`) so MoeMemos falls through from its v0 probe to v1 detection。Every other + implemented Memos route，including `/memos/file/*`，requires the Memos PAT。 + +## Evidence Behind the Boundary + +- MoeMemos 2.0.4 calls v0 `status` without a token and treats any successful version-bearing response as + a v0 server；only after failure does it call v1 `instance/profile`。See the tagged + [`detectAccountCaseAndVersion`](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/service/AccountService.kt#L496-L511)。 +- MoeMemos then validates the supplied Bearer token through tagged + [`GET api/v1/auth/me`](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/api/MemosV1Api.kt#L17-L20)。 +- Memos 0.29.1 exposes `GetInstanceProfile` publicly and treats PATs as long-lived tokens。Its tagged + implementation generates `memos_pat_` plus 32 cryptographically random alphanumeric characters and + stores SHA-256，see + [`token.go`](https://github.com/usememos/memos/blob/v0.29.1/server/auth/token.go#L189-L203) and + [`acl_config.go`](https://github.com/usememos/memos/blob/v0.29.1/server/router/api/v1/acl_config.go#L11-L47)。 +- Current core-py persists extension config as JSONB and returns it as part of `ExtensionModel`。Existing + extension/source configs already keep recoverable Twitter password/TOTP、Telegram bot token、IMAP + password and GitHub token values。Hashing only the Memos PAT would reduce one credential's exposure while + leaving the same database/config trust boundary intact，and would add a second update/read lifecycle。 +- The external update currently saves an unvalidated raw dict before applying runtime config。D-039 + therefore requires a generic validate-before-persist update mechanism，not Memos-specific transform or + projection hooks。 + +Memos upstream evidence informs the compatible external shape；it does not require copying upstream's +multi-user PAT storage boundary、list、expiry、last-used tracking or token-management endpoints。 + +## Route Authentication Matrix + +| Route class | Credential | Result | +| --- | --- | --- | +| `GET /memos/api/v1/instance/profile` while extension is enabled | none、peer JWT、Memos PAT or malformed header | `200` with at least `{"version":"0.29.1"}`；authentication is not evaluated | +| `GET /memos/api/v1/status` | any | `404`；do not return a version-bearing v0 status object | +| Implemented `/memos/api/v1/*` except profile | valid active Memos PAT | route handler runs | +| Implemented protected Memos route | missing、malformed、old、revoked or unknown token；peer JWT | `401` with `WWW-Authenticate: Bearer` | +| `/memos/file/*` | same as protected protocol routes | valid PAT required；attachment knowledge does not grant access | +| Core/default-extension protected route | Memos PAT | `401`；only peer JWT is accepted | +| Any Memos route while extension is disabled | any | `404` because D-038 removes the route set | + +Upstream Memos makes additional read routes public for its multi-user visibility model。The backend MVP +does not inherit that larger public surface：MoeMemos sends its token after login，and InKCre has no public +memo-browsing product requirement。 + +## Configuration Update and Persistence + +### Config update + +The existing peer-authenticated extension config update accepts one ordinary config field： + +```json +{ + "personal_access_token": "memos_pat_0123456789abcdefghijklmnopqrstuv" +} +``` + +- The exact accepted format is `^memos_pat_[0-9A-Za-z]{32}$`，matching the selected Memos generation。 +- Field present with a valid string means establish or replace。 +- Field present as `null` means revoke。 +- Core shallow-merges the update with the current config before validation；field absence therefore + preserves the current token and lets other fields change without re-sending it。 +- Invalid type/format returns `422` and leaves persisted and running state unchanged。 + +The deployment owner is responsible for generating the 32-character suffix with a cryptographically +secure generator。Server-side issuance is excluded from the MVP because it would add a parallel token +management API without improving MoeMemos compatibility。 + +### Persisted and read authority + +`extensions.config`、runtime config and peer-authenticated config reads share one shape： + +```json +{ + "personal_access_token": "memos_pat_0123456789abcdefghijklmnopqrstuv" +} +``` + +Authentication compares the presented Bearer value with the configured PAT using constant-time string +comparison。A separate verifier、credential table、password KDF、token ID、expiry、description、last-used +timestamp and history are not justified for one deployment token。 + +This explicitly accepts that a database reader or peer authorized to read extension configuration can +recover the Memos PAT。That is the current config trust boundary，not an accidental omission。If InKCre +later creates a generic encrypted/redacted secret-config facility，Memos should adopt it together with +other credential-bearing configs rather than inventing a private exception now。The current client-web +config path mismatch still needs correction if this GUI is included，but it can otherwise edit the same +config shape。 + +## State Transitions and Atomicity + +| Current state | Command | Successful next state | Observable consequence | +| --- | --- | --- | --- | +| unconfigured | valid token | configured(new) | new token works immediately if running | +| configured(old) | valid different token | configured(new) | old fails immediately；new works；no overlap | +| configured | `null` | unconfigured | all protected Memos routes return `401`；public profile remains available | +| unconfigured | `null` | unconfigured | idempotent success | +| either | field absent | unchanged | generic config merge has no credential effect | +| either | invalid input or persistence failure | unchanged | error；no partially applied runtime state | + +The update order is merge complete next config → validate → persist → assign the already-validated runtime +config。There is no `await` or fallible external work between persistence and assignment。Configuration is +allowed while the extension is disabled；the PAT is loaded on the next enable。Enabling without a +configured PAT is also allowed but fails closed：only the public profile can be used until configuration。 + +## Minimal Core Change + +No Memos-specific config hooks are required。Core needs one generic config-update operation：merge the +patch into current state、validate through the target's existing `config_cls`、persist the validated shape， +then update the live object。Extension is the first required target；the same operation should later be +reused by source and other configurable runtime owners once their addressing and lifecycle are designed。 + +```python +class MemosConfig(SQLModel): + personal_access_token: str | None = Field( + default=None, + pattern=r"^memos_pat_[0-9A-Za-z]{32}$", + ) + + +# ExtensionManager config update, before any database write. +candidate = {**extension.config, **request_body} +normalized = extension_class.__configcls__(**candidate) +persist(normalized.model_dump(mode="json")) +if extension_is_running: + extension_class.config = normalized +``` + +Core owns update ordering but not Memos semantics。The Memos config schema owns the PAT's nullable type and +format；the Memos auth dependency owns comparison。Source/storage reuse must not be claimed merely because +they also expose `config_cls`：their durable owner、instance address and live-reconfiguration consequences +must first be traced。 + +## Explicit Exclusions + +- `/memos/admin/*` and Memos PAT list/create/delete endpoints； +- login/password、short-lived access JWT、refresh cookie or session； +- multiple simultaneous tokens、rotation overlap or grace period； +- time expiry、automatic rotation、last-used/audit history or rate limiting； +- reusing peer JWT as a Memos credential or creating a core User； +- generic extension secret vault/table、auth-specific config hooks or config projection framework in this + MVP。 + +## Acceptance Obligations + +- U-01：public v1 detection followed by PAT-authenticated `auth/me` and GENERAL settings； +- U-06：raw attachment download uses the same PAT dependency； +- U-11：public、peer and Memos auth matrix，including cross-token rejection； +- config fixture：establish、read-back、hot replace、revoke、omitted-field preservation、invalid input + rollback、validation-before-persistence and persisted/runtime consistency； +- lifecycle fixture：configured while disabled，enable without config fails closed，disable returns `404`。 + +O-018 is closed by D-039。Technical/Acceptance are now complete；implementation still waits for the Impact +Handshake and Sir's explicit start。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/design.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/design.md new file mode 100644 index 0000000..38d83c4 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/design.md @@ -0,0 +1,272 @@ +# Memos Extension — Backend MVP Design + +本文把已经确认的产品边界与待审查的技术合同放在同一条端到端链路上。决定 authority 仍是 +[decision register](../../decisions/index.md)；协议和代码事实见 [evidence.md](evidence.md)。 + +## 1. Product Shape + +Memos extension 是 memo-like collection 的可实现 ownership unit；Memos-compatible backend +只是它的首个 MVP delivery scope。该 MVP 复用市面上的 MoeMemos Android 客户端作为低摩擦、 +多端记录入口，但不把该客户端扩张成 InKCre 的 use surface，也不复刻整个 Memos 产品。 + +首版目标固定为： + +- protocol authority：released Memos 0.29.1 generation； +- compatibility client：released MoeMemos Android 2.0.4； +- persistence authority：InKCre block / relation graph； +- canonical boundary：memo extension-owned CanonicalMemo； +- read boundary：memo resolver solved result； +- account boundary：deployment-scoped single-user；Memos profile 是 protocol projection，不是 + core User。 + +MoeMemos 只证明“用户旅程需要哪些协议行为”；字段和错误语义仍以选定 Memos release 为 +authority。为 MoeMemos 所做的偏离必须作为狭窄 compatibility shim 命名和测试。 + +## 2. End-to-End Topology + +```text +MoeMemos 2.0.4 + ↕ base URL /memos/ + Memos 0.29.1 relative HTTP paths + Bearer token +generation-specific API adapter + ↕ native request / native response +memo application service + ↕ CanonicalMemo + graph commands / solved memo +graph transaction ── block.content / relations / component blocks ── storage + ↕ +versioned memo resolver +``` + +写路径的成功边界是 primary memo mutation 已持久；D-041 不保证 graph 完整或 failure 后无残留。 +读路径从 root block 进入 resolver；adapter 不直接查询并拼装 block/relation rows。这样 native +API、canonical memo semantics 和 graph persistence 各自只有一个 owner。 + +## 3. CanonicalMemo and Graph Authority + +### CanonicalMemo owns root facts + +CanonicalMemo 是 memo-family 的 durable content contract，序列化后直接成为 memo root +`block.content`。它不是 Memos DTO，也不是与 graph 并列的 object store。 + +D-042 已确认 CanonicalMemo v1 exact root wire： + +```json +{ + "body": "今天想到…… #idea", + "created_at": "2026-07-30T10:20:30Z", + "updated_at": "2026-07-30T10:20:30Z", + "archived": false, + "visibility": "private", + "pinned": false +} +``` + +- `body` 是用户 authored Markdown；允许空字符串以支持 attachment-only memo，不 trim 或 + 重写用户 whitespace。 +- 两个时间是 memo-side authored/source times，不是 block row 的 persistence timestamps； + 使用 timezone-aware RFC 3339 UTC，未知时不可用 collection time 猜测。 +- info-base local identity 只使用 `block.id`，CanonicalMemo 不复制 local `id`。 +- Canonical generation 由 versioned resolver identity 选择，不写入 payload；unknown resolver + identity 明确失败。 +- `archived`、`visibility`、`pinned` 是需要 list/update/read round-trip 的 memo facts，且没有独立 + identity；D-042 因而由 root content 唯一持久。它们不是 graph component 或 adapter-only + projection。 + +Unknown keys are rejected；deterministic serialization and product-native casing/timestamps remain +fixture obligations rather than new ownership questions。 + +### Relations own independently meaningful structure + +| Information | Authority | Consequence | +| --- | --- | --- | +| authored body and memo-side time | root CanonicalMemo content | resolver 的主要 text / time 输入 | +| local identity | root `block.id` | Memos-facing stable name 如何编码需协议 fixture 证明，但不再造一份 local ID | +| attachment | component block + root relation + storage when needed | memo-family 默认无序；Memos 0.29.1 是 D-040 source-defined ordered exception | +| comment | independent memo root + parent relation | comment 可有自己的 body、attachments、time 与 lifecycle | +| memo reference | relation to target memo block | 不复制进 root content；target 不是 parent-owned component | +| tags/title/snippet/property | resolver-derived projection when derivable | 不建立第二份 durable authority | +| creator/location | 首版确实采集时使用 graph entity/relation | 只有证明独立寻址/use value 后才建模 | +| state/visibility/pinned | D-042 root CanonicalMemo | 没有独立 identity；不得塞入 `extras` 或 adapter-only state | + +D-013 的无序默认仍成立；preflight 已证明 Memos 0.29.1 会刻意持久并返回 request order，因此 +D-040 要求该 adapter 作为例外保留顺序。D-044 在现有 `relation.content` 中使用 +`attachment:<zero-based-order>`，不改通用 relation schema；parent/reference 分别使用 `parent` / +`reference`。 + +## 4. Adapter Boundary + +API adapter 只承担： + +- 验证并转换 Memos 0.29.1 request； +- 调用 memo application service 的 canonical/graph command； +- 把 resolver solved result 转换成 0.29.1 response； +- 在 compatibility contract 内处理命名、pagination、filters、field masks 与 HTTP errors。 + +它不得： + +- 直接把 Memos DTO 序列化为 block content； +- 绕过 resolver 读取 graph rows； +- 建立 Memos-specific durable memo table； +- 把未知字段塞进无边界 `extras`； +- 为通过客户端启动而返回与 persisted graph 不一致的假数据。 + +首版 HTTP surface 的证据分类见 [evidence.md](evidence.md)。D-047 将其冻结为三组： + +1. MoeMemos startup/auth/settings 必需； +2. memo sync/write 与 attachment journey 必需； +3. Sir 已明确要求但不由当前 APK 覆盖的 comment contract。 + +除此之外默认 unsupported，不能从 upstream server 的完整路由表反推首版范围。 + +D-048 additionally separates three axes：memo-family core（canonical/graph/service/resolver）、product +generation mapping（Memos 0.29.1、future flomo）and access mode（current backend、future collector）。 +Family code never imports product DTOs or transport；adapter mapping does not own DB sessions/cursors； +backend/collector orchestration invokes the same family service。This dependency direction is required now， +while a generic registry/framework waits for a second concrete adapter/access mode。 + +## 5. Identity, Account, and Compatibility + +### Backend identity + +本单元中 InKCre 自己是 memo authority，不存在需要与另一个 Memos server reconciliation 的 +第二份 memo。`block.id` 因而是 local memo identity；Memos resource name 的可逆编码由 adapter +负责。source-native provenance 和 `source_id` fallback 主要是未来 collector 的压力，不应为 +当前 backend 提前建立 source binding table。 + +### Deployment and protocol identity — D-033 + +MoeMemos 需要 current user、general settings、Bearer auth 和 creator-scoped list；这些是目标 +protocol shape，不证明 InKCre 需要 core User。当前 InKCre 的 `ClientModel` 表示围绕同一 +info-base 的 runtime peer，业务 rows 没有 terminal-user/tenant/owner ACL。 + +首版因此只投影一个 deployment-scoped Memos-compatible profile 与 default settings；所有 memo +graph 隐式处于同一 owner context，不增加 User/tenant tables，不把 `ClientModel` 冒充人，也不 +做 per-user row filtering。外部 source account 仍可作为 connector configuration/provenance， +但不是 InKCre user。 + +Memos Bearer credential 的生成、寿命、撤销和它与现有 short-lived peer JWT 的关系已由 D-039 +关闭；single-user 决定本身不等于直接复用当前 JWT。Memos `creator` / `visibility` 可以作为 +协议 round-trip facts，但不能因此暗中引入多用户 ownership model。 + +D-036 已确认 auth ownership：core routes 和普通 extensions 默认使用 peer auth；Memos extension +拥有自己的 protocol auth。它不实现成 `ExtensionBase` 三态 mode：peer +verification 变成 route dependency，core protected router tree 与普通 extension router 默认挂载 +它；需要 mixed/public/custom policy 的 extension 显式移除 base dependency，并以 FastAPI child +routers 组合。Memos protected child router 使用自己的 credential dependency，version detection +只有经 D-047 证明的 child routes 才无 dependency。这样不在 catch-all JWT middleware 中维护 +`/memos` override，也不需要 extension sub-app/middleware。 + +必须保留的 minimal implementation shape（最终 symbol/module 名由 preflight 核实）： + +```python +# Core protected route tree. +peer_api = APIRouter(dependencies=[Security(require_peer_jwt)]) +peer_api.include_router(block_router) +peer_api.include_router(extension_management_router) +api_app.include_router(peer_api) + + +class ExtensionBase: + @classmethod + def _api_dependencies(cls): + return (Security(require_peer_jwt),) + + @classmethod + def _build_router(cls): + return APIRouter( + prefix=f"/{cls.__extid__}", + dependencies=list(cls._api_dependencies()), + ) + + +class MemosExtension(ExtensionBase): + @classmethod + def _api_dependencies(cls): + return () # The root only composes explicitly classified child routers. + + @classmethod + def _register_apis(cls, root): + root.include_router(public_detection_router) + root.include_router( + protected_protocol_router, + dependencies=[Security(require_memos_token)], + ) +``` + +这段 example 说明 ownership 与 dependency topology，不冻结尚未 preflight 的文件名或完整 API。 +最终实现必须保持 ordinary extensions fail closed，并让 Memos public/protected routes 在代码结构上 +可直接区分。 + +不新增 `/memos/admin/*`。Credential 的建立、替换和清除复用现有 peer-authenticated extension +config surface：PAT 作为 ordinary Memos extension config 被验证、持久化、加载并通过 trusted +config surface 读取，不建立 Memos-only digest/projection lifecycle。Exact update behavior 见 +[auth-contract.md](auth-contract.md)：一个 deployment-scoped +`memos_pat_` PAT，只有 v1 instance profile public，v0 status 保持 `404`，replacement 无 overlap。 + +D-037 固定 lifetime：credential 默认无时间到期点，直到通过 config 显式替换或撤销。首版不 +引入 refresh token、session、automatic rotation 或周期性重新登录。数据库或 peer config reader +可恢复 PAT 是当前 config trust boundary 的显式取舍；未来若建立 generic secret config，Memos 与 +其它 credential-bearing configs 一并迁移。 + +### Generation compatibility + +- 首版只暴露 Memos 0.29.1 generation；0.30 与更旧 generation 明确 unsupported。 +- future breaking release 使用 generation-specific adapter；相邻 generations 可在迁移窗口 + 并存，但不能移除仍服务 live route/config 的 adapter。 +- product API generation 与 CanonicalMemo resolver generation 正交。协议变更但 canonical + semantics 不变时，新的 adapter 可继续写相同 canonical generation。 +- `latest` 文档或 main branch schema 不能替代 release-tag contract。 + +## 6. Mutation and Failure Contracts Still to Freeze + +### PATCH compatibility — D-034 + +Memos 0.29.1 要求 non-empty `updateMask`，MoeMemos 2.0.4 不发送它。只在此 generation +adapter 对缺失 mask 启用 shim：从原始 JSON 中实际出现的可更新 keys 推导 mask；显式合法 +`updateMask` query 仍按上游语义处理。presence 判断保留 `false`、空字符串与空列表；未知/不可更新 key、空 mask +或不被字段合同允许的 `null` 明确失败。不得把这一宽松规则下沉成通用 update behavior。 + +### Coordinated writes — D-041 + +一次 native mutation 可能同时改变 root、attachments/comments/relations 与 storage。它们必须 +由一个 application service 协调，避免 convenience paths 隐式决定业务顺序；但不对外承诺 +transactional graph completeness。共享 PostgreSQL session 在实现更简单时可以使用，failure 后 +orphan/stale components 允许保留。MoeMemos 先前成功上传的 unattached attachment 更是独立资源。 + +### Delete ownership — D-046 + +推荐 delete success 以 parent root 不再 list/read 为 primary effect，并 best-effort 清理 parent +relation、comment subtree 与 exclusively-owned attachment components/raw storage。shared reference +targets 保留；D-041 允许 residue；repeated unknown delete returns `404`。 + +## 7. Implementation-Plan Findings and Cross-Cutting Boundaries + +早期 [implementation plan](implementation-plan.md) 已用真实代码地址和 MoeMemos 2.0.4 client +code 检验这条链路。它不是 implementation approval，却把“现有机制能否承载”从原则问题 +收敛成了可定位的事实： + +- **可以复用**：checked-in extension artifact discovery/install、built-in profile、 + generation-specific adapter package、`/{extension_id}` route namespace、exact-key resolver + registration。MoeMemos 的 relative endpoints 与 path-preserving host 可直接使用 `/memos/` 作为 + base URL；当前没有证据需要 top-level mount、downloader、artifact registry 或 resolver registry + redesign。 +- **确定不能直接承载**：global peer-JWT middleware 无法验证 Memos Bearer credential；read-only + `Storage` 无法完成 attachment upload/delete/download。 +- **需要最小工程演进**：namespaced Memos route 的 auth dispatch、现有 extension start/close/ + disable correctness、session-aware graph mutations、writable storage，以及把 embedding 等 derived + side effects 与 native commit 分开。D-038 要求 same-process hot enable/disable；由 + `ExtensionManager` 直接发布/撤下 retained extension-owned route set，不增加 persistent per-route + running dependency、request-drain generation 或 isolated dispatcher。 +- **仍由 extension 拥有**：Memos wire、CanonicalMemo、memo graph predicates、resolver solved + model、query/native projection 与 owned deletion semantics。不得为了公共 helper 把这些语义 + 下沉进 `InfoBaseManager`。 + +这些 pressure 进入 [pressure-ledger.md](../../pressure-ledger.md)，但不自动批准某一种方案。 +D-039–D-045 已关闭 credential、failure boundary、CanonicalMemo、writable storage、relation +grammar 与 client-web path；具体改动仍需在 Execution baseline 和 Impact Handshake 中限定 blast +radius。 + +## 8. Remaining Review Sequence + +Technical/Acceptance decisions、preflight and Execution baseline are complete。The next step is the Impact +Handshake for exact code/client-web/migration/doc state diff，then Sir's explicit “开始”。Any newly discovered +owner/observable-behavior branch returns to design review。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/evidence.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/evidence.md new file mode 100644 index 0000000..8be453e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/evidence.md @@ -0,0 +1,179 @@ +# Memos Extension — Backend MVP Evidence + +本文只记录 backend MVP 可复核的版本锚点、MoeMemos 实际最小调用、core fit 证据和已知的 +`updateMask` 偏差。产品/技术草案见 [design.md](design.md)，控制状态见 +[unit packet](packet.md)，决定 authority 仍是 [decision register](../../decisions/index.md)。 + +## Version anchors + +- **Protocol target**：Memos `v0.29.1`，官方 tag 指向 commit `5f194da`； + [release](https://github.com/usememos/memos/releases/tag/v0.29.1)、 + [tagged memo proto](https://github.com/usememos/memos/blob/v0.29.1/proto/api/v1/memo_service.proto)。 +- **Acceptance client**：MoeMemosAndroid `2.0.4`，官方 tag 指向 commit + `9bfc6517f981a05b67ffcac20fe1a908e659f815`； + [release/tag](https://github.com/mudkipme/MoeMemosAndroid/releases/tag/2.0.4)。 + release APK 为 `moememos-v2.0.4.apk`，SHA-256 + `5043f14d27c4cc283cb1507a23a84f251e159ab8d3937da9842f2060bd7fe8fa`； + [APK asset](https://github.com/mudkipme/MoeMemosAndroid/releases/download/2.0.4/moememos-v2.0.4.apk)。 +- MoeMemos `2.0.4` 的 release note 明确声明支持 Memos `0.27.0`–`0.29.1`；本单元取其中 + `0.29.1` generation，不把更高 generation 的 wire shape 混入首版。 + +## Actual minimum wire calls + +以下集合来自 MoeMemosAndroid `2.0.4` 的 v1 Retrofit interface、repository 和 account +version detection，而不是完整 upstream API 的猜测（[client API source](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/api/MemosV1Api.kt)、 +[repository source](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/repository/MemosV1Repository.kt)）。 + +这些 Retrofit annotations 使用不以 `/` 开头的 relative paths；`Retrofit.Builder.baseUrl(host)` +直接消费登录页保留 path 的 host。attachment URI 也在该 host 后追加 `file/...`。因此用户配置 +`https://<deployment>/memos/` 时，请求分别落到 `/memos/api/v1/*` 与 `/memos/file/*`，可以直接 +复用现有 extension namespace；此前把 annotations 中的 `api/v1` 误读为必须占用 server root， +已经撤回。 + +| Journey phase | Calls and required shape | +| --- | --- | +| Detect generation | `GET /api/v1/status` may fail; client then falls through to `GET /api/v1/instance/profile`. The profile version must identify supported `0.29.1`, and is fetched again before sync. | +| Authenticate | User supplies a Bearer token; client sends `Authorization: Bearer <token>`. `GET /api/v1/auth/me` resolves the user, then `GET /api/v1/users/{user}/settings/GENERAL` supplies default visibility. | +| Sync | `GET /api/v1/memos` for `state=NORMAL` and `state=ARCHIVED`, `pageSize=200`, repeated `pageToken`, and `filter=creator == "users/{username}"`; continue until `nextPageToken` is empty. | +| Memo writes | `POST /api/v1/memos`, `PATCH /api/v1/memos/{id}`, and `DELETE /api/v1/memos/{id}`. Core journey does not call `GetMemo`; MoeMemos reads detail from its local database. | +| Attachments | relative `GET/POST/DELETE api/v1/attachments...`，plus authenticated raw download by appending `file/{attachment-name}/{filename}` to the configured host path。Upload is streaming JSON with base64 `content` and nullable `memo`；With `/memos/` base，raw names become `/memos/file/attachments/{id}/{filename}`. | +| Deliberately absent from core set | Explore/users, comments, relations and reactions are not called by the APK core sync. Comments remain a packet-level semantic fixture, covered outside the APK gate. | + +### Authentication-specific findings + +- MoeMemos generation detection is unauthenticated and ordered。It first calls v0 + `GET api/v1/status`；a successful version-bearing response selects the v0 repository。Only failure falls + through to v1 `GET api/v1/instance/profile`。The Memos extension must therefore leave `/api/v1/status` + unimplemented (`404`) and publicly serve only the v1 profile，rather than implementing both detection + endpoints。 +- Memos 0.29.1 PATs use `memos_pat_` plus 32 cryptographically random alphanumeric characters，persist + only a SHA-256 hash and may have no expiry。This proves the compatible token shape and lifetime；its + multi-user storage boundary does not override InKCre's existing ordinary-config trust boundary or + require copying upstream's PAT list、expiry or management APIs。 +- The selected MoeMemos client treats the token as an opaque Bearer value，validates it with + `GET /api/v1/auth/me` and attaches it to same-origin attachment downloads。 + +Memos v0.29.1 本身提供 `GET/POST /api/v1/memos/{memo}/comments`；comment response 仍是 Memo， +后续 update/delete 复用普通 memo endpoints（[tagged proto](https://github.com/usememos/memos/blob/v0.29.1/proto/api/v1/memo_service.proto)）。 +因此 comments 可以作为明确的 protocol fixture 实现，同时诚实地标记为 MoeMemos APK 不覆盖。 + +## Core fit evidence + +- Current JWT authenticates InKCre peer deployments; request state has no Memos terminal-user + principal or PAT owner ([JWT contract](../../../../app/middleware.py#L24-L64)、[middleware gate](../../../../app/middleware.py#L141-L181))。 + `ClientModel` is a peer deployment, not a Memos user ([model](../../../../app/schemas/client/main.py#L13-L18))。 +- Blocks and relations have no owner/tenant field ([block](../../../../app/schemas/info_base/block.py#L18-L53)、 + [relation](../../../../app/schemas/info_base/relation.py#L10-L41))。Multi-user Memos therefore needs + separate identity, token, ownership, visibility and ACL authority; adding endpoints alone is insufficient. +- Native Memos data must map to `CanonicalMemo` root content plus component blocks/relations; + a parallel Memos memo object store would violate the graph authority boundary. +- Memo, attachment, comment and relation changes need one explicit application-service coordinator so + convenience paths do not silently choose business order。D-041 does not require a single transaction or + a no-partial-graph guarantee。 + +## Implementation-address audit + +以下是为 implementation-plan probe 做的本仓只读核验；它们说明 plan pressure，不自动决定 +最终方案： + +### Extension host and authentication + +- [`ExtensionBase.on_start`](../../../../app/business/extension/main.py) always creates + `APIRouter(prefix=f"/{extid}")` and only then calls extension `_register_apis`。For ext id `memos`, + this produces the intended `/memos/...` namespace；MoeMemos can select it through its pathful base + URL, so no root-route exception is required。 +- Extension routes are added only when an installed extension is enabled and runtime bootstrap starts + it；current disable/close never removes them。Pinned FastAPI 0.139.2 keeps an included child + `APIRouter` as a live route branch and versions its effective-route/OpenAPI caches。It has no public + symmetric remove API，but a retained child router can be cleared/repopulated if the runtime host also + performs the required version invalidation；this framework-specific operation must stay localized。 +- A disposable FastAPI 0.139.2 in-memory probe confirmed this exact branch：a route added to a retained + child after inclusion became dispatchable；clearing the child plus `_mark_routes_changed()` removed both + dispatch and OpenAPI output。This is pinned-framework evidence，not a public API guarantee。 +- `RUNNING_EXTENSIONS` is keyed by ext id, but current `start/close` membership checks use the + extension class。This makes duplicate start and ineffective close a confirmed local defect, not a + product-design alternative。 +- [`JWTMiddleware`](../../../../app/middleware.py) runs before route handlers and accepts only canonical + peer JWT claims。A Memos Bearer token cannot be implemented solely inside the extension handler； + D-036 therefore replaces the catch-all gate with route-tree dependencies；D-039 defines the + Memos PAT/config contract and exact public subset。 +- Extension `config` and `config_schema` are JSONB and the ordinary extension endpoints return config。 + Existing configs already carry recoverable Twitter password/TOTP、Telegram bot token、IMAP password + and GitHub token values；there is no current system-wide hidden-secret config boundary。 +- client-web passes `config_schema` to its JSON editor，but the editor also works without a schema；core can + import the extension `config_cls` when updating a disabled extension。Persisting schema before enable is + therefore optional UI metadata，not a PAT lifecycle requirement。 +- Current external config update passes the raw dict directly to `save_config()`，which persists before + runtime validation；`ExtensionBase.on_close()` also uses the same persistence method。A config update + therefore needs validation before commit。The existing extension `config_cls` is sufficient；the + evidence justifies a shared update ordering，not Memos-specific transform/projection hooks。 +- Current client-web reads extension rows/config directly from the database and its config update path is + `/{extension_id}/config`，while core-py's management route is `/extensions/{extension_id}/config`。The + Memos PAT can follow that same trusted-config boundary。The existing generic editor still cannot be + claimed as a working Memos credential UI until its path and update behavior match the management + contract。 + +### Graph, resolver and storage + +- [`BlockManager.fetchsert`](../../../../app/business/info_base/block.py) uses resolver equality and then + performs synchronous embedding work in the caller's session。Default resolver equality is + `(resolver, content)`，which would merge two independently-created identical memos；backend create + must not use content dedup as identity。 +- `BlockManager.create` can flush in a caller session, but `edit_block` owns and commits a new session； + `RelationManager.create` also commits independently, and no block/relation delete API exists。 + `InfoBaseManager.add_subgraph_to_session` proves caller-owned graph transactions are possible, but + mutable memo commands need bounded session-aware primitives/application orchestration。 +- `RelationManager.get` combines incoming/outgoing filters with `AND` when both are requested, and + resolver relation results are cached without a direction-specific key。Memo resolver cannot assume + two arbitrary direction calls reconstruct a complete graph without a query correction/explicit + one-shot classification。 +- Current [`Storage`](../../../../app/business/info_base/storage/main.py) contract only reads raw + content；built-ins fetch remote URLs。There is no upload, raw file persistence, delete or file-serving + contract, and no existing multipart/upload route。This is direct evidence behind D-043。 +- Relation FK cascade removes relation rows when a block is deleted, but does not delete component + blocks or backing raw content。Owned subtree cleanup therefore cannot be inferred from database FK + behavior。 + +### Verification environment and blast radius + +- Ordinary [`tests/conftest.py`](../../../../tests/conftest.py) uses an unreachable Postgres URL and + `SKIP_EXTENSIONS_SYNC=1` so pure tests remain hermetic。Transaction/FK/JSONB/runtime route proof needs + a disposable PostgreSQL integration fixture rather than SQLite or existing pure tests。 +- Adding a new SQLModel table affects schema discovery, Alembic metadata/revision, the exact + application-table manifest/readiness tests, and the client-web database contract projection。The + D-043 PostgreSQL raw-storage decision therefore makes this a known blast radius，not an + implementation-time surprise。 +- No Android/Gradle/ADB/MoeMemos harness exists in core-py。APK compatibility must be executed in an + external controlled runner and retained as an evidence bundle；ASGI `TestClient` remains necessary but + is not sufficient。 + +## Known `updateMask` deviation + +- Memos `v0.29.1` declares `update_mask` required in `UpdateMemoRequest` and the server rejects a + nil or empty mask ([proto](https://github.com/usememos/memos/blob/v0.29.1/proto/api/v1/memo_service.proto#L2067-L2077)、 + [implementation](https://github.com/usememos/memos/blob/v0.29.1/server/router/api/v1/memo_service.go#L3005-L3018)). +- MoeMemos `2.0.4` serializes `UpdateMemoRequest` with only nullable memo fields and sends no + `updateMask` ([request model](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/api/MemosV1Api.kt#L97-L106)、 + [update call](https://github.com/mudkipme/MoeMemosAndroid/blob/2.0.4/app/src/main/java/me/mudkip/moememos/data/repository/MemosV1Repository.kt#L146-L164)). +- Strict upstream behavior therefore breaks the selected client's edit journey. The compatibility + shim is therefore a deliberate deviation (D-034): infer paths from raw JSON key presence when the + mask is absent, while accepting an explicit valid mask under upstream semantics. This shim is not + strict Memos parity. +- The tagged HTTP annotation declares PATCH `body: "memo"` while `update_mask` is a sibling request + field。Therefore an explicit REST mask belongs to the `updateMask` query parameter，not the JSON body。 + +## Attachment order and unattached lifecycle + +- Tagged Memos 0.29.1 `setMemoAttachmentsInternal` treats the request attachment list as the complete set， + deletes omitted owned attachments，reverses the normalized request and assigns increasing `updated_ts`。 + The store then lists attachments by `updated_ts DESC`。This deliberately reconstructs the original + request order，so D-013's “preserve only when source-defined” condition is met for this adapter（D-040）。 +- MoeMemos uploads resources before memo create/update。For a new memo its `remoteId` is null，so + `POST /api/v1/attachments` carries `memo=null`；only after uploads succeed does `POST /api/v1/memos` + carry attachment names。A successful upload is therefore an independently committed、listable、deletable + resource even if the later memo create fails。 +- In this protocol generation an attachment can be unattached or owned by one memo。PATCH attachment + arrays are set semantics，not append semantics；`attachments: []` is observably different from omission。 +- The tagged server uses a 32 MiB upload buffer as its fallback when no instance-level maximum is + configured；the value is not a protocol-wide immutable limit。The MVP may select the same fixed cap + because it intentionally excludes the administration/settings surface。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/impact-handshake.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/impact-handshake.md new file mode 100644 index 0000000..02efe34 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/impact-handshake.md @@ -0,0 +1,159 @@ +# Memos Extension Backend MVP — Impact Handshake + +> Prepared from the approved D-029–D-048 Execution baseline。This bounds implementation but does not +> authorize it；Sir must explicitly say “开始”。 + +## Address and Object + +### core-py shared surfaces + +- `run.py`、`app/middleware.py`：replace catch-all peer JWT enforcement with protected router-tree + dependencies while retaining public health/readiness/docs surfaces。 +- `app/business/extension/main.py`、`app/routes/extension.py`：default extension auth dependency hook， + retained route-set hot publish/unpublish，running-state fixes and validated config update ordering。 +- `app/database_contract/profile.py` and catalog/readiness tests：register the checked-in Memos artifact and + PostgreSQL binary storage profile/instance without redesigning the artifact registry。 +- `app/business/info_base/block.py`、`relation.py`、resolver internals：minimal caller-session mutation + primitives，incoming/outgoing relation correctness and direction-safe resolver relation caching。 +- `app/business/info_base/storage/`、`app/schemas/info_base/`：generic PostgreSQL binary storage and one + raw `BYTEA` table/model。 +- `migrations/`、metadata/application-table/readiness tests：one schema migration and exact catalog/schema + projection updates。 + +### Memos extension owner + +- new `extensions/memos/` checked-in artifact and metadata/config。 +- family-owned CanonicalMemo、graph mapping/repository、application commands、versioned resolver and + attachment handling。 +- product-generation-owned Memos 0.29.1 wire models、mapping and backend routes。 +- candidate dependency layout follows D-048 `family/` and `products/memos/v0_29_1/` boundaries；final + filenames may tighten without changing ownership direction。 + +### Tests and external evidence + +- new `tests/extensions/memos/family/` reusable canonical/graph/resolver contract tests。 +- new `tests/extensions/memos/products/memos/v0_29_1/fixtures/` exact bounded wire/error fixtures and + adapter tests。 +- new `tests/extensions/memos/backend/` auth/route/config/lifecycle tests。 +- new `tests/extensions/memos/integration/` PostgreSQL graph/storage/delete/residue tests。 +- external pinned MoeMemos 2.0.4 APK runner/evidence bundle；no Android harness is invented inside core-py。 + +### client-web sibling repository + +- `packages/core/src/extension/base.ts` and focused tests：change extension config save to + `/extensions/{extension_id}/config`。 +- generated database contract projection only if the accepted raw table is included in that repository's + full application-schema type surface。 + +### Deferred documentation owners + +- No Hub/shared/local durable documentation is edited in implementation batches。Approved task truth stays + in the promotion queue until verified implementation evidence exists，then follows owner-specific Hub / + shared-ref / Spoke-local workflows and separate commits。 + +## State Diff + +1. **Auth**: global peer-JWT request gate → core/default-extension router dependencies + Memos public/PAT + child-router composition。 +2. **Extension runtime**: one-way/duplicating router inclusion and defective running membership → one + retained route-set handle with idempotent hot enable/disable/re-enable and localized FastAPI cache + invalidation。 +3. **Extension config**: persist raw dict then validate → shallow merge、`config_cls` validation、normalized + persistence、live assignment；disabled update imports the config class without publishing routes。 +4. **Resolver lifetime**: decoder availability coupled to live API activation → installed decoder remains + usable for persisted blocks after API disable。 +5. **Graph primitives**: independently committing/incomplete mutation helpers → the minimum caller-session + operations and correct full-star relation reads needed by the family service；no graph-completeness + product guarantee is added。 +6. **Storage**: read-only remote-pointer storage → generic PostgreSQL binary put/get/delete backed by one raw + bytes table，while attachment identity/metadata remain in graph blocks。 +7. **Product capability**: no memo backend → bounded Memos 0.29.1 backend for MoeMemos 2.0.4，including + profile/auth/settings、list/create/PATCH/delete、attachments/raw download and comment fixtures。 +8. **Client operator flow**: broken generic config request path → working peer-authenticated extension config + save and hot apply through core API。 + +## Operation and Expected Side Effects + +- Logic-altering refactor of auth routing and extension runtime lifecycle。 +- New checked-in extension package and tests。 +- Additive PostgreSQL schema migration and built-in storage catalog entry。 +- Local correctness changes in relation retrieval/resolver caching and graph mutation APIs。 +- Separate sibling-repo client-web request-path/test change。 +- No commit、push、Hub edit、shared-ref bump or deployment mutation is implied by implementation start；each + requires its normal scope/command discipline。 + +## Blast Radius Forecast + +- Every core API route and ordinary extension route is sensitive to the auth topology move。 +- All extensions are sensitive to default dependency、start/close and config-update changes。 +- Existing resolvers are sensitive to relation query/cache corrections but should only gain the behavior + their current interface already claims。 +- Database schema/catalog/readiness and client-web generated types are sensitive to the raw table。 +- Existing storages remain readable；only the new database-binary implementation is writable unless future + pressure extends other storage classes。 +- The new Memos semantics remain inside `extensions/memos`；flomo、collectors、organization、retrieval and + complete Memos administration are outside the implementation blast radius。 + +## Invariants Check + +- Core protected routes and ordinary extensions remain fail-closed under peer JWT；Memos PAT never authenticates + them。 +- Only Memos v1 instance profile is public；v0 status stays `404`；all other implemented Memos/file routes + require the configured PAT。 +- `/{extension_id}` remains the protocol namespace；no top-level mount or extension sub-app is added。 +- No parallel Memos memo table/object store；CanonicalMemo remains root `block.content` and graph components + remain relations/blocks/storage。 +- Backend reads are resolver-mediated；product adapters do not interpret raw graph rows。 +- Equal memo bodies remain distinct block identities；backend create does not use content fetchsert。 +- Canonical generation is the resolver identity；payload has no local id/schema version/attachments/parent/ + references。 +- D-041 remains explicit：success guarantees the primary mutation，not complete graph atomicity；residue is + allowed and no compensation/replay subsystem is introduced。 +- D-046 deletion never removes reference targets or components lacking proven exclusive ownership。 +- D-048 seams leave room for flomo/collectors but no empty generic adapter registry、collector framework or + speculative native DTOs are created。 +- Indexing/query projections are not added to the memo collection authority。 +- Existing unrelated worktree changes remain untouched。 + +## Verification + +1. Pure family/product fixtures：D-042 serialization、D-044 graph mapping、resolver result、wire/error + mapping、PATCH presence and cursor behavior。 +2. ASGI：public/peer/PAT cross-auth matrix，OpenAPI/route availability，enable/disable/re-enable，config + establish/read/replace/revoke/invalid update。 +3. PostgreSQL：migration/catalog/readiness，BYTEA storage，graph/resolver queries，orphan attachments， + ordered relations，best-effort deletion residue and no shared-target over-delete。 +4. Repository：`pdm run lint`、`pdm run typecheck`、`pdm run test` and applicable migration checks。 +5. client-web：focused request-path test and repository checks in the sibling repo。 +6. External E2E：pinned APK tag/digest、desensitized HTTP transcript、committed graph snapshot、resolver + output and client-visible login/sync/write/attachment behavior。 + +Each implementation increment reruns its focused tests plus earlier slice regressions；pure tests、ASGI、 +PostgreSQL and APK evidence do not substitute for one another。 + +## Uncertainty + +- FastAPI 0.139.2 route removal depends on private route-version invalidation；the helper is pinned and + tested，but a future FastAPI upgrade must reopen it。 +- Final raw-table/type/instance symbol names may change during implementation for local readability，without + changing D-043 ownership or schema blast radius。 +- The migration head may advance because of unrelated work before execution；generate against the then-current + head rather than the preflight head。 +- Existing extension behaviors may reveal an undocumented reliance on `on_close()` persisting mutated config； + current search found no such writer。If found，return to design rather than silently discard it。 +- client-web generated database types may be produced by an existing schema workflow instead of hand edits； + follow that repository's local instructions during its separate batch。 +- APK automation environment is external and still needs concrete runner setup；this does not change the + bounded server contract。 + +## Entry Gate + +- Product：approved。 +- Technical：approved。 +- Acceptance：approved as D-047 fixture contract。 +- Execution baseline/preflight：complete。 +- Impact Handshake：**approved by Sir**。 +- Explicit start：**granted；Sir said “批准，开始”**。 + +Execution has entered I-01 under this approved state diff。Any newly discovered owner or observable behavior +outside this boundary must return to the corresponding design gate before implementation continues。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/implementation-plan.md new file mode 100644 index 0000000..ea8827e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/implementation-plan.md @@ -0,0 +1,368 @@ +# Memos Extension — Backend MVP Implementation Plan + +## Control Surface + +- **Maturity**: **Execution baseline**。The plan has approved Technical/Acceptance contracts plus verified + addresses、versions、runtime assumptions and failure branches。 +- **Delivery scope**: `memos-extension` ownership unit under the `memos-backend` MVP。 +- **Objective**: make MoeMemos Android 2.0.4 use InKCre as a minimal Memos 0.29.1-compatible backend，with + every successful native write committed as info-base graph/raw storage and every read reconstructed + through the memo resolver。 +- **Guardrails**: no parallel Memos store；no direct adapter reads of block/relation rows；no unproved API + surface；no Memos semantics in generic graph managers；no durable/shared-doc mutation in implementation + batches。 +- **Execution entry**: the remaining Impact Handshake must bound the final code/client-web/migration/doc + state diff，then Sir must explicitly say “开始”。 + +Verified evidence and branch traversal live in [preflight.md](preflight.md)。Addresses below are either +**existing**（already inspected）or **candidate**（the intended owner，final symbol name may change at the +Impact Handshake）。 + +## Slicing and Ordering Basis + +The plan is vertically sliced by an observable behavior crossing protocol → canonical/graph command → +transaction → resolver → native response。Shared changes are extracted only when the next slice cannot be +implemented safely without them。 + +The dependency order is: + +```text +runtime/auth/config safety + → bootable Memos protocol + → text CanonicalMemo round-trip + → list/PATCH/pagination + → writable storage + attachments + → comments + owned deletion + → compatibility hardening + → pinned APK proof +``` + +This is not directory-by-directory work。Each increment has an observable exit proof and reruns the +previous increment's focused tests。 + +## Target Topology + +```text +peer operator ── PUT /extensions/memos/config ──→ validated raw PAT config + +MoeMemos 2.0.4 + → base URL https://<deployment>/memos/ + → retained extension route host /memos + → public profile child router | PAT-protected protocol/file child routers + → Memos 0.29.1 adapter + → memo application service (coordinated PostgreSQL writes without a completeness guarantee) + → CanonicalMemo root + attachment/comment blocks + typed relations + DB raw storage + → versioned memo/attachment resolvers + → Memos 0.29.1 DTO +``` + +Installed resolver decoders remain available for persisted blocks even when the extension's live API and +source surfaces are disabled。Protocol generation and canonical resolver generation remain orthogonal。 + +## Extension Seam for Future Products and Access Modes — D-048 + +Dependency direction is fixed now，without building empty flomo/collector frameworks： + +```text +family semantics + CanonicalMemo + graph mapping + commands + solved resolver model + ↑ consumed by +product-generation adapter + Memos 0.29.1 wire ↔ canonical/solved translation + ↑ invoked by +access mode + current backend routes/auth | future collector client/cursor/reconciliation +``` + +Candidate package ownership： + +```text +extensions/memos/ + config.py + family/ + schema.py + graph.py + service.py + resolver.py + products/ + memos/ + v0_29_1/ + wire.py + adapter.py + backend.py + attachment.py +``` + +The exact folder names may be tightened during the Impact Handshake，but the dependency boundary may not +collapse：family modules never import Memos/flomo DTOs or backend/collector transports；product mapping does +not open DB sessions or own scheduling；access modes call the family application service and read solved +results。A future flomo generation adds a sibling under `products/`。A future collector adds transport/ +cursor/reconciliation orchestration without forking CanonicalMemo or graph mapping。 + +Tests mirror the boundary： + +```text +tests/extensions/memos/ + family/ # reusable canonical/graph/resolver contract suite + products/memos/v0_29_1/ + fixtures/ # exact product-generation JSON/query/error cases + test_adapter.py + backend/ # route/auth/lifecycle behavior + integration/ # PostgreSQL graph/storage behavior +``` + +Future product adapters run the same family contract suite against their mappings，while keeping their own +native fixtures。Backend and collector orchestration never share a test merely because both use the same +product API generation。 + +## Preflight Corrections Incorporated + +1. Memos attachment order is real product semantics（D-040），not hypothetical order；D-044 fixes the exact + relation grammar。 +2. An attachment upload with `memo=null` is a valid standalone operation；D-041 more generally rejects + “no partial graph” as an observable guarantee。 +3. Explicit `updateMask` is a query parameter；missing-mask inference remains adapter-local D-034。 +4. Writable storage is a required implementation slice。D-043 selects PostgreSQL BYTEA-backed + storage，not an optional mid-flight schema choice。 +5. Existing client-web config save is broken by a path mismatch；D-045 includes its repair as a separate + sibling-repo batch。 +6. FastAPI 0.139.2 live child-router mutation works，but route removal/cache invalidation relies on private + framework behavior and must be encapsulated behind one tested runtime host helper。 +7. Extension config schema is currently persisted only from `on_start()`，but this is not a disabled-config + blocker：client-web uses it only as an optional editor aid，while core can import `config_cls` to validate。 +8. Current relation retrieval uses AND for simultaneous incoming/outgoing lookup，and the resolver caches + a direction-agnostic result；both would misresolve a memo star graph and need generic correctness fixes。 +9. `BlockManager.fetchsert()` is forbidden for backend create because equal memo content is not identity and + the path also mixes synchronous embedding work into the native transaction。 + +## Execution Increments + +### I-00 — Freeze design and executable fixtures — completed + +- **From → To**: reviewed design probe → approved Technical/Acceptance contracts and Execution baseline。 +- **Completed evidence**: D-039–D-048、bounded API/fixture contract、pagination/relation/storage/delete + semantics and preflight branch traversal。 +- **Remaining preparation**: the Impact Handshake must enumerate the final core-py/client-web/migration/doc + addresses before execution。 + +I-01 onward changes product code and remains blocked until Sir's explicit start。 + +### I-01 — Core route/auth/config/lifecycle safety + +- **Observable slice**: existing core routes remain peer-JWT protected；a test extension can expose public + and custom-auth child routers；enable → disable → re-enable changes availability without duplicates； + config updates cannot persist invalid state。 +- **Existing addresses**: + - `run.py`，`app/middleware.py`； + - `app/business/extension/main.py`，`app/routes/extension.py`； + - focused extension/auth/runtime tests。 +- **Work**: + 1. reuse peer JWT verification as a router dependency and attach it to the core protected tree；keep + health/readiness/docs/openapi public；ordinary extensions default to the same peer dependency； + 2. add the one overridable dependency hook retained in D-036；do not add an auth enum、path exceptions、 + sub-app or middleware registry； + 3. centralize a retained extension host route set and FastAPI invalidation；fix `RUNNING_EXTENSIONS` + membership、duplicate start and ineffective close； + 4. unpublish before close，retain fail-closed state on close failure，and make cleanup retry idempotent； + 5. separate installed decoder availability from live API/source activation；disable does not unregister a + decoder needed by existing blocks； + 6. import `config_cls` for config updates while disabled without publishing routes；do not make persisted + `config_schema` a lifecycle gate； + 7. implement current + shallow patch → `config_cls` validate → persist normalized JSON → assign live + config；remove `on_close()` stale config persistence。 +- **Failure branches**: construction/start failure publishes no routes；DB config failure changes no runtime + value；a crash after config commit is healed from DB authority at restart；repeated toggles converge。 +- **Exit proof**: peer/public/custom auth matrix、config rollback and route availability lifecycle tests；all + existing core route-auth behavior remains covered。 + +### I-02 — Bootable namespaced Memos protocol + +- **Observable slice**: MoeMemos can select v1 via public profile，authenticate with the configured PAT and + read current profile/GENERAL settings；disabled extension returns `404`。 +- **Existing addresses**: `app/database_contract/profile.py` and extension discovery/sync。 +- **Candidate addresses**: + - `extensions/memos/__init__.py`，`pyproject.toml`，`config.py`； + - `extensions/memos/products/memos/v0_29_1/` adapter-owned wire/backend modules； + - `tests/extensions/memos/test_route_mount.py`，`test_protocol_auth.py`。 +- **Work**: + - add checked-in Memos artifact/profile without redesigning artifact registry； + - compose an auth-neutral root with public detection and PAT-protected protocol/file children； + - compare the ordinary raw PAT using constant-time comparison； + - expose only profile、auth/me and GENERAL settings for this slice；leave v0 status unregistered； + - keep the minimal D-036 code-shaped dependency example in implementation/local technical docs later。 +- **Failure branches**: malformed/missing/peer/old/revoked PAT → `401` + Bearer challenge；Memos PAT on core + routes → `401`；malformed auth never blocks public profile；unconfigured extension exposes only profile。 +- **Exit proof**: U-01、U-10、U-11 and U-12 startup/auth/config subsets。 + +#### Separate client-web batch + +Fix `../client-web/packages/core/src/extension/base.ts` to call +`/extensions/{extension_id}/config` and cover the request。This is a separate repository/change batch；it +must not be folded into a core-py or shared-doc commit。Core HTTP acceptance can proceed without claiming +the GUI works，but productized operator setup cannot。 + +### I-03 — Text CanonicalMemo create and resolver read-back + +- **Observable slice**: create a text-only top-level memo，commit one root block，resolve it and return the + native response；two equal bodies create two identities。 +- **Existing addresses**: + - `app/schemas/info_base/block.py`； + - `app/business/info_base/block.py`，`relation.py`，resolver base/manager。 +- **Candidate Memos owners**: + - `extensions/memos/family/schema.py` — CanonicalMemo v1 and solved memo； + - `extensions/memos/family/resolver.py` — versioned decoder； + - `extensions/memos/family/service.py` — application command coordinator； + - `extensions/memos/family/graph.py` — mapping/repository； + - reusable family canonical、round-trip and write tests。 +- **Core work**: only caller-session block edit/delete and relation create/update/delete/query correctness； + no Memos predicates or owned traversal in generic managers。 +- **Memos work**: deterministic JSON，UTC RFC3339 serialization，explicit block create，resolver-mediated + native projection and clear unknown-generation/invalid-content failures。 +- **Failure branches**: invalid DTO/canonical → `400` before primary write；root failure → non-2xx but may + leave attempted component residue under D-041；resolver failure → no fake success/fallback；embedding is + not invoked inside the native write。 +- **Exit proof**: U-03 text subset and U-07 root round-trip，including equal-body identities and diagnosed + root-write failure behavior。 + +### I-04 — Sync, PATCH and pagination + +- **Observable slice**: MoeMemos syncs NORMAL and ARCHIVED pages；PATCH changes only selected facts and + returns resolver-derived native state。 +- **Candidate address**: `extensions/memos/query.py` or memo-owned repository in `graph.py`，plus sync/mask + tests。 +- **Work**: + - query memo resolver roots，exclude comments by parent relation，enforce the exact deployment creator + filter and state； + - default order by canonical `created_at DESC, block.id DESC`；use an opaque versioned keyset token bound + to state/filter/cursor； + - support client pageSize 200 and return `nextPageToken` including empty terminal value； + - parse explicit query `updateMask` under 0.29.1 semantics；otherwise infer from raw body keys only； + - keep state/visibility/pinned authority at the D-042 root location。 +- **No initial projection/index**: personal-scale JSON/graph query is accepted until measurement proves a + query projection；any future index is derived retrieval support，not a second memo authority。 +- **Failure branches**: invalid creator filter/page token/mask → `400`；false/empty/list presence updates； + concurrent inserts do not shift later pages；top-level list never leaks comments。 +- **Exit proof**: U-02 and U-04 with at least two pages，NORMAL/ARCHIVED and explicit/missing-mask fixtures。 + +### I-05 — PostgreSQL writable storage and attachment graph + +- **Observable slice**: upload unattached or memo-owned attachment → list → attach/reorder → authenticated + raw download → delete，with resolver-only attachment reconstruction。 +- **Existing addresses**: + - `app/business/info_base/storage/`，`app/schemas/info_base/storage.py`； + - `app/database_contract/profile.py`，schema discovery/application-table manifest； + - `migrations/` and migration/readiness tests。 +- **Candidate addresses**: + - generic database-binary storage implementation and a `storage_blobs`-like SQLModel/table； + - Memos attachment model/resolver/service/router； + - attachment/storage/transaction tests。 +- **Core work under D-043**: + - add caller-session put/get/delete to the storage contract； + - store only generated pointer + raw `BYTEA` in the raw table； + - add one built-in database-binary storage profile/instance and make built-in storage setup consume the + profile instead of duplicating literals； + - add migration、metadata、manifest and generated client-web DB-contract pressure。 +- **Memos work under D-044**: + - attachment block owns filename/media type/size/storage pointer；raw bytes remain in storage； + - allow zero-or-one owning memo relation；`memo=null` creates a valid orphan； + - preserve request order in `attachment:<position>` relations；PATCH attachment lists have set semantics； + - enforce decoded size limit、base64/filename/media validation and PAT raw download。 +- **Failure branches**: raw or graph failure returns the exact result for the primary command and may leave + diagnosable residue；memo create failure leaves an earlier successful orphan valid；missing bytes/filename + mismatch is `404`；empty attachment list attempts to remove the owned set。 +- **Exit proof**: U-03 attachment subset and U-06，plus migration/readiness and the selected 32 MiB + MVP boundary。 + +### I-06 — Comments and owned deletion + +- **Observable slice**: comment create/list/update/delete as independent memo roots；parent delete follows + D-046 primary-success/best-effort cleanup and preserves referenced memo targets。 +- **Candidate addresses**: reuse Memos service/graph repository and add comment/delete/transaction tests。 +- **Work**: + - comment → parent relation，top-level exclusion and resolver projection； + - comment visibility follows the parent at creation/update according to exact fixture； + - remove the root from subsequent list/read，then attempt parent/comment relations、exclusively-owned + attachment blocks and raw cleanup； + - reject multiple attachment owners and bound traversal with a visited set。 +- **Failure branches**: shared reference target survives；corrupt cycles stop without over-delete；mid-delete + residue is permitted；repeated unknown delete returns the approved `404` behavior。 +- **Exit proof**: U-05 and U-08，including graph snapshots before/after and residue/over-delete fixtures。 + +### I-07 — Compatibility and lifecycle hardening + +- **Observable slice**: all unsupported、auth、invalid filter/token/mask、unknown resolver、missing raw and + hot lifecycle branches return exact non-2xx without false success；D-041 explicitly permits graph residue。 +- **Work**: finish protocol `400` error translation，unsupported route assertions，OpenAPI/route-set + invalidation proof，batch/list performance smoke and all previous regression suites。 +- **Exit proof**: U-09 plus the full failure matrix；lint、typecheck、unit/integration and migration checks + pass。 + +### I-08 — Pinned MoeMemos APK proof and promotion handoff + +- **Observable slice**: official MoeMemos 2.0.4 APK logs in，syncs，creates，edits，archives/deletes and + handles attachments against InKCre；comments remain a separate protocol fixture。 +- **Runner boundary**: core-py has no Android/ADB harness，so a controlled external runner produces the + evidence bundle；ASGI tests do not substitute for the APK。 +- **Evidence**: APK tag/commit/digest、desensitized HTTP transcript、profile version、committed graph + snapshot、resolver result and client-visible outcome。 +- **Exit proof**: all accepted U-IDs pass；implementation facts are ready for a separately reviewed durable + documentation promotion batch。 + +## Dependency and Review Shape + +```text +D-036/D-038/D-039 ─→ I-01 ─→ I-02 +D-042 ──────────────────────→ I-03 ─→ I-04 +D-043 + D-040 + D-044 ─────────────────→ I-05 +D-046 ───────────────────────────────────────→ I-06 +I-01…06 ─→ I-07 ─→ I-08 +``` + +All Technical/Acceptance decisions are closed by D-039–D-048，the Impact Handshake is approved，and Sir has +granted explicit start。Execution begins at I-01。If final address exploration reveals a new owner or observable +behavior，the plan returns to the relevant gate instead of silently expanding during execution。 + +## Change Batches and Commit Boundaries + +These are review/verification batches，not authorization to commit： + +1. core-py route auth + extension runtime/config safety； +2. core-py Memos artifact/startup surface； +3. client-web generic config-path fix（separate repo）； +4. core-py CanonicalMemo text graph + query/PATCH； +5. core-py writable storage schema/migration + attachments； +6. core-py comments/delete/hardening； +7. external APK evidence； +8. Hub/shared/local durable-doc promotion through owner-specific workflows and separate commits。 + +Hub source edits、shared-ref bumps、Spoke code/local docs and sibling client-web changes must never be +collapsed into one commit merely because they belong to one product unit。 + +## Shared-Surface Budget + +| Surface | Allowed state diff | +| --- | --- | +| Artifact registry | add checked-in `memos` catalog entry only；no redesign | +| Extension runtime | route dependency hook、retained route host、correct hot lifecycle/config ordering | +| Resolver registry | reuse exact versioned keys；separate decoder availability from API activation | +| Info-base managers | caller-session mutation + relation query/cache correctness only | +| Storage | D-043 generic DB binary put/get/delete + one raw table/profile/instance | +| Database/query | storage migration required；no memo table/projection/index initially | +| Sink/embedding | no authority change；native writes avoid synchronous fetchsert embedding | +| Memos extension | all canonical、predicate、resolver、query、adapter and owned traversal semantics | +| client-web | generic extension config path and generated DB projection only | +| Durable docs | no implementation-batch edit；promotion later by owner | + +## Verification Ladder + +1. **Pure contract**: canonical serialization、DTO mapping、mask parsing、cursor、resolver and PAT compare。 +2. **ASGI**: core/public/Memos auth matrix，route table，enable/disable/re-enable and config lifecycle。 +3. **PostgreSQL integration**: graph/FK/JSONB/BYTEA behavior，pagination，storage and deletion；test actual + local transaction choices without promoting them to a graph-completeness guarantee。 +4. **Repository checks**: `pdm run lint`、`pdm run typecheck`、`pdm run test` plus migration commands when + the new table lands。 +5. **Runtime**: healthy SVC worktree database，migration/readiness/catalog reconciliation。 +6. **APK E2E**: pinned released APK and retained evidence bundle。 + +Existing ordinary pytest deliberately avoids PostgreSQL and skips extension sync，so layers 2/3 require an +explicit harness。A green pure suite cannot be reported as graph transaction or hot-runtime proof。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/packet.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/packet.md new file mode 100644 index 0000000..8b5cc23 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/packet.md @@ -0,0 +1,301 @@ +# Memos Extension + +- **Unit ID**: `memos-extension`。 +- **Unit State**: **Complete**；backend MVP implementation 与 verification 已完成，durable owner projections 已随 + RSS/common promotion batch 提交。Push/Hub publication/shared-ref bump 仍是独立授权操作。 +- **Active delivery scope**: `memos-backend` MVP。 +- **Objective**: 建立拥有 memo-family CanonicalMemo、graph mapping、resolver 与 + product-generation adapters 的 core extension；首个 MVP 让 MoeMemos Android 2.0.4 把 + InKCre 当作 Memos 0.29.1-compatible backend 使用。 +- **Guardrails**: unit boundary 是 Memos extension，不是当前接入关系；backend MVP 只实现经 + 上游合同与真实客户端调用共同证明的 API 子集；MoeMemos 是验收 client，不是协议 authority； + 不得建立并行 Memos object store；backend read 必须经过 resolver。collectors、flomo、完整 + Memos server、旧 generations、社交/分享能力不在当前 MVP，但可作为 extension 后续 scope。 +- **Verification**: 以 [acceptance.md](acceptance.md) 的 HTTP → committed graph → resolver + output → Memos response fixtures 为合同，并用发布版 MoeMemos APK 完成真实客户端 E2E。D-041 + 明确不承诺“无部分 graph”；验证只证明成功响应对应的 primary mutation 已持久，且响应与实际 + committed state 一致。 +- **Current Truth**: backend MVP 的 Product gate 已通过。InKCre deployment 是 + single-user/owner context； + Memos profile 只是协议兼容 projection，不引入 core User、tenant 或 per-row ownership。 + Technical 与 Acceptance 已冻结；[implementation-plan.md](implementation-plan.md) 已经过 + implementation-address preflight 并成为 Execution baseline。MoeMemos 可通过 pathful base URL 复用现有 + `/{extension_id}` routes；没有证据需要重做 artifact/resolver registry。已确认需要的公共演进是 + route-auth composition、hot lifecycle repair、validated config update、session-aware graph mutation + 与 writable storage。D-039–D-048 已关闭 PAT/config、attachment order、partial-graph boundary、 + CanonicalMemo v1、PostgreSQL binary storage、relation grammar、client-web config path、owned deletion、 + exact API fixtures 和 future adapter/access-mode seams。Technical/Acceptance questions 已由 + D-039–D-048 关闭；完整排障见 [preflight.md](preflight.md)。I-01–I-08 均已关闭；官方 MoeMemos + 2.0.4 APK 已证明登录、双 state 多页同步、create、attachment upload/read、pin/edit、archive/delete，且 + 真实调用发现并修复了显式空 `pageToken=` 的首页兼容缺口。E2E 数据、PAT 与 runner 已精确清理。 +- **Next Step**: none for backend MVP。Future collector/versioned-product work 必须作为新的 delivery + scope 重新通过 gate；remaining publication/shared-ref operations follow their owner workflows。 + +## Lifecycle Gates + +| Gate | State | Exit evidence | +| --- | --- | --- | +| Product | **Approved for backend MVP** | D-029–D-033, D-035；用户旅程、范围、single-user boundary、成功与失败语义获批 | +| Technical | **Approved** | D-034–D-048；owner、wire、auth、canonical、storage、relations、failure/delete 与 extension seams 获批 | +| Acceptance | **Approved as fixture contract** | D-047 + [acceptance.md](acceptance.md)；每个必要行为映射到分层 executable fixture | +| Implementation Plan | **Execution baseline** | [implementation-plan.md](implementation-plan.md) 已吸收 preflight 与全部获批决定；state diff 等待 Impact Handshake | +| Preflight | **Completed for current probe** | [preflight.md](preflight.md) 固定 pinned upstream/client、实现地址、运行环境、blast radius 与失败分支 | +| Impact Handshake | **Approved** | Sir 已批准 [impact-handshake.md](impact-handshake.md) 所界定的 address/object、state diff、blast radius、invariants、verification 与 uncertainty | +| Explicit Start | **Granted** | Sir 已明确说“批准，开始” | +| Execute / Verify | **Completed** | I-01–I-08 已通过 unit、repository-wide、migration/readiness、PostgreSQL HTTP→graph→resolver/storage/delete 与官方 APK journey；E2E residue 为 0 | +| Unit Close | **Completed** | implementation/verification evidence 已满足 unit acceptance；durable projection 已提交，publication 仍按 owner 独立 | + +只有本文件维护 Memos extension 当前 delivery scope 的 phase、gate 和 next step。supporting +documents 只保存设计、证据和验收内容，不另设控制状态；未来 scope 不继承 backend MVP 的 +approval。 + +## Approved Backend MVP Contract + +### User journey + +1. 用户在 MoeMemos 配置 InKCre endpoint 与 Bearer token。 +2. MoeMemos 完成启动探测、读取当前用户/设置并同步 memo 列表。 +3. 用户通过 MoeMemos 创建、编辑、归档或删除 memo，并处理附件。 +4. 独立 protocol fixture 覆盖 comment create/list/update/delete；不假称 MoeMemos 2.0.4 会 + 调用它。 +5. write success 表示 primary memo mutation 已持久；随后 sync/read 从 resolver 的实际 committed + state 重建相容响应，不额外保证 graph completeness。 + +### Success and observable failure + +- 客户端看到 create/update/delete 成功，表示该 command 的 primary effect 已持久。 +- 认证失败、unsupported behavior、unknown resolver generation、storage failure 或 graph + mutation failure 必须返回明确 non-2xx；failure 后允许留下 orphan/stale graph components，不增加 + compensation/replay 机制只为恢复完整 graph。 +- 对兼容范围之外的 endpoint 或 generation 明确拒绝，不以空响应伪装兼容。 + +### Included + +- Memos 0.29.1 generation 的最小 startup/auth/settings、memo list/write 与必要 read-back。 +- MoeMemos 2.0.4 实际依赖的 attachment upload/list/delete/download。 +- comment 作为独立 memo root，并以 parent relation 连接；这是明确产品要求，即使 MoeMemos + 核心同步目前不调用 comment API，也要有独立协议 fixture。 +- `NORMAL` / `ARCHIVED`、visibility 等进入已证明客户端旅程的必要 Memos 行为；其持久 owner + 已由 D-042 冻结。 + +### Not included in the MVP + +- flomo backend；官方客户端尚无已证明的 replaceable-backend path，可在 extension 后续 scope + 发现新证据时重开。 +- Memos 或其他产品 collector、webhook、export / backup ingestion；它们是 extension 的后续 + delivery scopes，不是另一个 canonical ownership unit。 +- Memos 0.30 或 0.29.1 之前的 generation；未来 breaking generation 使用独立 adapter。 +- 完整 Memos administration、explore/social、reaction、share 或与本旅程无关的 endpoint。 +- InKCre info-base 的浏览、检索或 organization UI。 + +## Closed Technical Contracts + +| Contract | Confirmed result | +| --- | --- | +| Canonical/graph | D-042/D-044：exact root wire；ordered attachment、parent、reference relation grammar | +| Storage/failure/delete | D-041/D-043/D-046：PostgreSQL binary storage；不保证 graph completeness；primary delete + best-effort cleanup | +| Protocol fixtures | D-047：只覆盖 bounded 0.29.1 subset、MoeMemos deviations 和 InKCre graph/resolver effects | +| Extensibility | D-048：family core、product/generation adapters、backend/collector access modes 分层；不提前造 registry/framework | + +上述合同已获批。若 Impact Handshake 或实现证据发现新的 owner/behavior 分叉，必须退回对应 +gate，而不是在 Execute 中临时决定。 + +## Confirmed Decision References + +- Program/graph boundary: [D-001–D-007](../../decisions/D001-D010.md)。 +- Memo role and graph mapping: D-008–D-013、D-017。 +- Canonical/resolver boundary: D-019–D-028、D-032。 +- Unit identity, current delivery, deployment, PATCH, auth、hot lifecycle、canonical/storage/relation、 + failure/delete、fixtures and extensibility boundary: D-029–D-048。 + +`decisions/` 是本任务唯一决定登记册。本文件只描述这些决定如何约束当前 implementable +unit，不复制完整 rationale。 + +## Supporting Material + +- [design.md](design.md): 当前产品与技术合同的内聚草案。 +- [auth-contract.md](auth-contract.md): D-039 的 exact credential、public-route、state transition 与 + minimal core seam contract。 +- [evidence.md](evidence.md): released upstream、MoeMemos 调用面与现有 core 的可复核证据。 +- [acceptance.md](acceptance.md): 实现前要冻结的行为/graph/resolver/E2E 合同。 +- [implementation-plan.md](implementation-plan.md): 已冻结的端到端增量、代码地址、依赖与验证 + execution baseline。 +- [preflight.md](preflight.md): pinned upstream/client、实现地址、纠偏结论、失败分支与 blast radius。 +- [impact-handshake.md](impact-handshake.md): execution state diff、blast radius、invariants、verification and uncertainty。 +- [Program packet](../../packet.md): program 范围、单元路由与交付循环。 +- [Documentation promotion](../../documentation-promotion.md): 讨论产生的 durable-doc pressure。 +- [Pressure ledger](../../pressure-ledger.md): 本单元传导出的横切机制压力。 + +## Gate Discipline + +- Product、Technical、Acceptance 分别由 Sir 审查；提前形成的 Acceptance 与 implementation-plan + probe 是发现问题的工具，不构成隐式 approval。 +- implementation plan 可以在 Technical 阶段先形成 probe，并由 preflight 核实版本、地址、环境 + 与失败分支；只有上游合同获批并关闭其暴露的分叉后，才成为可执行 baseline。 +- 若 Preflight 新证据改变已获批行为、owner 或 blast radius，退回相应 gate，而不是在 Execute + 中临时改设计。 +- durable docs promotion 仍延后到实现证据齐备后的独立批次；业务代码已在获批 Impact + Handshake 与 Sir 明确开始后进入 Execute。task packet 在当前任务边界内持续记录执行状态。 + +## Execution Evidence + +### I-01 — completed + +- Core peer JWT 从全局 catch-all middleware 移到显式 protected route tree；health/docs 与 unmatched + paths 不再被全局认证拦截。 +- Extension API 默认继承 peer dependency；需要 public/self-auth 的 extension 通过唯一 + `api_dependencies()` hook 返回 auth-neutral root，再在内部组合自己的 dependencies。 +- Extension route 以 retained child router 发布；disable/close 先撤销 routes，并用 FastAPI 0.139.2 + 的 child + app route invalidation 行为测试固定热启停。 +- 修复 running membership、duplicate start、close retry；close failure 后 routes 保持 fail-closed， + runtime entry 保留用于重试。 +- installed extension decoders 在 live API/source 之外加载；disable 不移除 persisted block 所需 decoder。 +- Config update 采用 current + shallow patch → config model validate → normalized DB commit → live assign； + invalid config 不写 DB/runtime，`on_close()` 不再回写 stale runtime config。 +- Verification：`30` 个 focused tests、`159` 个 repository-wide tests 通过；I-01 touched files 的 Ruff + 与 Pyrefly 均通过。 + +### I-02 — completed + +- 新增 checked-in `memos` artifact/profile 与 auth-neutral extension root；Memos 0.29.1 product-generation + backend 将 public profile 与 PAT-protected routes 组合在自己的 child routers 中。 +- `GET /memos/api/v1/instance/profile` 固定返回 `0.29.1`；v0 status 保持 unregistered；current user + 固定投影为 `users/inkcre`，其 role 使用 0.29.1 proto authority 的 `ADMIN`（而不是旧 generation 的 + `HOST`），GENERAL settings 固定默认 `PRIVATE`。 +- PAT 是 ordinary nullable extension config，严格匹配 `memos_pat_` + 32 alphanumeric，request-time + constant-time comparison 支持 hot replace/revoke；public profile 完全不评估 Authorization。 +- Pinned JSON fixtures 固定 profile/current-user/settings wire；auth matrix、unknown user、hot revoke、 + disable/re-enable 与 route ownership 均有 ASGI tests。 +- client-web 已在独立 repo batch 将 config request 从 `/{id}/config` 修正为 + `/extensions/{id}/config`，并增加 focused request-shape test。 +- Verification：core-py `178` tests、Ruff、Pyrefly 通过；client-web focused Vitest、Oxfmt、Oxlint 与 + `@inkcre/core` TypeScript check 通过。 + +### I-03 — completed + +- 冻结 exact CanonicalMemo v1 root content、UTC-aware timestamps 与 deterministic JSON；unknown root + facts 和 naive timestamps 明确拒绝，attachments/parent/references 只从 relations 解出。 +- 新增 family-owned graph repository、application command service 与 + `extensions.memos.memo.v1` resolver；product adapter 只做 0.29.1 wire ↔ family 映射，family 不导入 + product DTO/transport。 +- `POST /memos/api/v1/memos` 的 text-only slice 已接通，Memos validation 以 `400` 返回；non-empty + attachments 在 I-05 前明确拒绝，不能静默丢失。成功响应来自 committed root 的 resolver solved + value；相同 body 不去重。 +- Core graph primitives 增加 caller-session block get/edit/delete、relation create/update/delete；修复 + relation 双向查询从错误 intersection 为 union，并使 resolver relation cache 按 requested direction + 区分。 +- Verification：core-py `194` tests、full Ruff/Pyrefly 通过；worktree PostgreSQL 分别证明 equal-body + distinct roots 与真实 PAT HTTP create → committed block → resolver → native response，并精确清理测试 + roots，未 reset development database。 + +### I-04 — completed + +- Family query 只扫描 `extensions.memos.memo.v1` roots、解析 CanonicalMemo 并从 `parent` relation + 排除 comments；无 memo object store、projection 或 index。 +- NORMAL/ARCHIVED 分流按 canonical `created_at DESC, block.id DESC`；opaque token 绑定 generation、 + exact creator filter、state 与 keyset cursor，terminal token 为 MoeMemos 所需空字符串。两页之间的 + 新插入不会造成 offset shift/duplicate。 +- 0.29.1 PATCH adapter 在 `updateMask` 缺失时只从 raw JSON key presence 推导；显式 mask 不增加 + inferred fields，并保留 `false`、`""`、`[]`。selected null、empty/unknown mask 在 primary write 前 + `400`；attachments selected 在 I-05 前明确拒绝。 +- Family application service 的 update 只改 selected canonical root facts，commit 后经 resolver 返回 + native response；unknown memo `404`。 +- Verification：core-py `219` tests、full Ruff/Pyrefly 通过；worktree PostgreSQL HTTP 证明 2+1 + NORMAL pages、ARCHIVED page、comment exclusion、query-bound token 与 inferred PATCH committed + resolver round-trip，并精确清理测试 roots。 + +### I-05 — completed + +- 新增 generic `WritableStorage` caller-session read/write/delete capability，以及 built-in + `postgresql_binary` storage type/instance；raw table 只保存 opaque UUID + `BYTEA`，attachment identity、 + filename/media type/size/time 与 pointer 仍由 attachment block content 拥有。 +- 新增 `f2c8a6d1e4b7` migration、metadata/application-table/database-contract projection、production profile + head 与 migration integrity entry；built-in setup 改为消费同一 profile authority，不再复制 storage + literals。 +- 新增 exact CanonicalAttachment v1、`extensions.memos.attachment.v1` resolver 与 orphan/zero-or-one owner + graph semantics；MemoResolver 按 ordered relations hydrate attachment solved values，CanonicalMemo root + content 仍不复制 attachment facts。 +- Memos 0.29.1 backend 支持 PAT-protected upload/list/delete/raw download；严格 base64、non-path filename、 + media type 与 32 MiB decoded cap。memo create/PATCH 接收 existing attachment identities，relation + `attachment:<position>` 保留请求顺序；omission/present-empty/set 区分并删除 omitted owned components/raw。 +- Exact Memos attachment fixture 与 ASGI tests 固定 orphan/owned upload、auth、validation、cap、list/download/ + delete、create attach 与 PATCH reorder；opt-in PostgreSQL integration test 固定 orphan → attach → reorder → + resolver/download → set removal → block/BYTEA cleanup，并精确清理测试 identities。 +- client-web database-contract generated projection 暂不从未提交 core worktree 同步：development descriptor + 仍以 build source revision 为 provenance authority；待形成 coherent core commit/image 后再通过既有 workflow + 同步，避免伪造 cross-repo revision provenance。该压力不影响当前 runtime/API 行为。 +- Verification：`check:migrations`、worktree database readiness（repository head `f2c8a6d1e4b7`、catalog/ + privileges/seed all ok）、PostgreSQL attachment integration、core-py `234 passed, 1 skipped`、full Ruff 与 + Pyrefly 全部通过。 + +### I-06 — completed + +- Comment 沿用 CanonicalMemo root generation，以 comment → parent 的 `parent` relation 表示 owner； + `POST /memos/api/v1/memos/{parent}/comments` 将请求 visibility 归一为 parent visibility，resolver/native + response 从 graph 还原 `parent`，不复制进 CanonicalMemo content。 +- `GET .../comments` 只沿 incoming parent relations 读取 independent memo roots，以 block identity + descending keyset 分页；opaque token 绑定 parent，default/max page size 与 pinned generation 一致。 + top-level memo list 继续排除所有拥有 parent relation 的 comments。 +- Ordinary memo PATCH 可更新 comment body/attachments；任何 root patch 都重新应用 parent visibility， + 防止 comment 漂离当前 parent policy。ordinary DELETE 同时适用于 top-level memo 与 comment；unknown/ + repeated delete 为 `404`。 +- Owned deletion 在 primary transaction 前构造有限 visited traversal plan，只沿 exclusive parent 与 + attachment ownership，明确不沿 reference；multiple-parent/multiple-owner corruption 被跳过。primary root + 先 commit，随后 comment/attachment/raw cleanup best-effort 执行，failure 留 residue 但不撤销 primary + success。 +- Pinned protocol fixture 与 ASGI tests 固定 comment create/list token/parent response 和 ordinary delete； + pure traversal fixtures 固定 nested postorder、multiple owner/parent skip、reference preservation 与 cycle + termination。 +- PostgreSQL proofs 覆盖真实 comment HTTP create → committed graph → resolver list/PATCH/delete、parent + visibility、nested owned cleanup、reference/shared attachment target survival，以及注入 cleanup failure 后 + primary root removed + component/BYTEA residue retained；全部测试 identities 均精确清理。 +- Verification：opt-in PostgreSQL `4 passed`；`check:migrations`、worktree readiness、core-py + `240 passed, 4 skipped`、full Ruff 与 Pyrefly 全部通过。 + +### I-07 — completed + +- Bounded protocol matrix 固定 public profile、PAT route、peer-token rejection、unknown user/resource、 + unsupported users/reactions/relations/admin surfaces、non-canonical identities、invalid JSON/filter/token/mask + 与 not-yet-supported input 均返回明确 non-2xx，不以空 response 伪装兼容。 +- Route lifecycle proof 固定 disable 只撤销 extension-owned route set，re-enable 不重复注册 route/OpenAPI + path；PAT replace/revoke 不重建 routes 即刻生效。Memos error translation 保持 validation `400`、unknown + root `404` 与 unsupported route `404/405` 的边界。 +- Verification：core-py `258 passed, 6 skipped`；Ruff 全量通过；Pyrefly `0 errors`；migration integrity + `22 passed`、head `f2c8a6d1e4b7`；development database catalog/contract/migration/privileges/roles/seed + readiness 全部为 `ok`。 + +### I-08 — completed + +- Runner 使用官方 MoeMemos Android `2.0.4` release APK；SHA-256 为 + `5043f14d27c4cc283cb1507a23a84f251e159ab8d3937da9842f2060bd7fe8fa`。为避免占用主盘，本任务新增的 + Android 14 AOSP system image 与专用 AVD 实际放在 `/Volumes/WorkSSD/Android/inkcre-e2e/`，主盘只保留 + SDK 可发现它们所需的小型 link/index；runner 结束后整体清理。 +- APK 以 pathful `/memos/` endpoint 和 synthetic PAT 登录；实际 call graph 依次经过 unregistered v1 + status fallback、0.29.1 instance profile、current user、GENERAL setting，然后分别完成 NORMAL 与 + ARCHIVED 的两页同步。为强制证明 cursor loop，数据库临时写入带唯一 marker 的 201+201 roots。 +- 真实 MoeMemos 请求在第一页显式携带 `pageToken=`；原 parser 将它误判为 malformed token 并返回 + `400`。adapter 现将 empty token 与 absent token 同义解释为 first page，同时继续拒绝 malformed opaque + token；exact MoeMemos query、comment empty token 与 route fixture 已回归固定。修复后四个 list calls + 均为 `200`，第二页使用 query-bound opaque token。 +- APK 创建正文 `InKCre MoeMemos E2E create #inkcre-e2e` 并选择两张图片；committed root `475` 的 + attachment relations 为 `475 -> 473 attachment:0`、`475 -> 474 attachment:1`。resolver 以相同顺序 + 组装两张附件；PostgreSQL `BYTEA` 大小分别为 `98981`、`100322`，declared/stored size 一致。客户端 + 可显示正文与两张图片。 +- APK 随后 pin 并将正文编辑为 `InKCre MoeMemos E2E edited #inkcre-e2e`；真实 PATCH 返回 `200`，resolver + projection 保持 `pinned=true`、`visibility=PRIVATE` 与 `[473, 474]` attachment order。Archive 再次通过 + PATCH 完成；ARCHIVED list 读回 exact edited projection。客户端在 edit 后曾短暂保留本地 + `Memo not synced` 标记，但服务端 `200`、committed graph 与 resolver/API read-back 一致，故只作为 + client-local observation 记录，不改变 backend success 结论。 +- 经 action-time confirmation，APK archived menu 的 Delete → Confirm 发出真实 DELETE 并回到列表； + `memos/475` 在 NORMAL/ARCHIVED 都为 0 matches，blocks `473/474/475`、相关 relations 与 owned + attachment list 都为 0，两个 raw URLs 均返回 `404`。删除前数据库有四个 blobs；删除后只剩与 + orphan blocks `471/472` 一一对应的两个 blobs，证明 owned deletion 精确移除了 `473/474` 的 raw， + 没有把 orphan 上传误判为 owned component。 +- Cleanup 先经 attachment API 删除 `471/472`，再以 exact resolver + unique marker guard 删除 + `402` 个分页 fixture roots；最终 marker roots、E2E blocks `471–475` 与 `storage_blobs` 全部为 0。 + Synthetic PAT 已设为 null，extension enabled clients 为空，热撤销后 profile route 返回 `404`；清理后 + opt-in PostgreSQL suite `6 passed`，再次清理后 E2E blocks/blobs 仍为 0，database readiness 全部为 + `ok`。 +- 专用 emulator 已正常停止；`/Volumes/WorkSSD/Android/inkcre-e2e/`、主盘 system-image link、AVD index + 与临时 APK 目录均已移除。没有移动或删除既有 emulator/platform-tools/其他 AVD；清理后主盘约 + `15 GiB` free、WorkSSD 约 `737 GiB` free。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/memos-extension/preflight.md b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/preflight.md new file mode 100644 index 0000000..ede0a13 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/memos-extension/preflight.md @@ -0,0 +1,248 @@ +# Memos Extension — Backend MVP Preflight + +> Completed read-only preflight for the Execution baseline on 2026-08-01。This file records evidence and +> failure branches；D-039–D-048 own the approved decisions，and this file does not authorize code changes。 + +## Verdict + +The unit topology is sound：`memos-extension` owns CanonicalMemo、graph mapping、resolver and protocol +adapters，while `memos-backend` is its first delivery scope。The MVP does **not** need a new extension +artifact registry、top-level protocol mount、resolver registry or parallel memo store。 + +The implementation plan does need five material corrections： + +1. Memos 0.29.1 deliberately preserves attachment order，so this adapter is the source-defined exception + allowed by D-013；order must live on relations。 +2. MoeMemos can upload an attachment before a memo exists。A committed unattached attachment is a valid + independent protocol state；more generally D-041 does not require failed commands to leave no partial + graph。 +3. `updateMask` is outside the HTTP body because the proto annotation declares `body: "memo"`；the + explicit mask is an optional query parameter，while MoeMemos sends none。 +4. client-web currently sends extension config to `/{extension_id}/config`，but core exposes + `/extensions/{extension_id}/config`。A GUI-configurable PAT therefore requires a small client-web fix； + the HTTP management API itself is sufficient for core-only acceptance。 +5. Attachment bytes require a writable storage implementation and database blast radius。Treating schema + work as merely conditional would defer the largest design choice until mid-implementation。 +6. Extension `config_schema` is currently persisted only during `on_start()`。client-web uses it as an + optional JSON-editor aid，but the editor works without it and the core API can import `config_cls` for + authoritative validation。It is therefore not a disabled-extension lifecycle requirement。 + +## Evidence Anchors + +| Surface | Pinned evidence | Consequence | +| --- | --- | --- | +| Memos protocol | tag `v0.29.1`，commit `5f194da` | exact generation authority；main/latest excluded | +| Acceptance client | MoeMemos Android tag `2.0.4`，commit `9bfc6517`，release APK SHA-256 `5043f14d…fe8fa` | real call graph and final E2E client | +| FastAPI | pinned `0.139.2` | included child routers are live/versioned；removal needs localized private invalidation | +| Extension runtime | `app/business/extension/main.py` | namespace already `/{extension_id}`；running-membership and close logic are defective | +| Auth | `app/middleware.py` + `run.py` | global peer-JWT middleware blocks extension-owned PAT before routing | +| Config | `app/routes/extension.py` + `ExtensionManager.save_config` | current PUT persists before validation；schema is optional UI metadata | +| Graph/resolver | block/relation managers + resolver base | caller-owned create is possible；mutable operations and relation reads have gaps | +| Storage | storage base/profile | read-only pointer → raw contract；no put/delete/raw serving | +| Database runtime | `svc dev status database --repo . --json` | worktree-scoped PostgreSQL runtime is healthy and migration head is known | +| client-web | `../client-web/packages/core/src/extension/base.ts` | generic config editor calls the wrong core route | + +The FastAPI route assumption was also exercised in a disposable in-memory app：a retained included child +router accepted routes added after inclusion；clearing its route set plus `_mark_routes_changed()` removed +both dispatch and OpenAPI entries。There is no public symmetric removal API，so one runtime-host helper +must encapsulate this pinned-framework dependency and lifecycle tests must guard it。 + +## Capability Classification + +| Capability | Verdict | Reason | +| --- | --- | --- | +| Checked-in extension discovery/catalog | Reuse | `extensions/<id>` + built-in profile already carries the artifact | +| `/{extension_id}` HTTP namespace | Reuse | MoeMemos Retrofit paths are relative and preserve `/memos/` base path | +| Exact-key resolver registration | Reuse with lifecycle correction | a versioned memo key fits；decoder must remain available for persisted blocks after API disable | +| Route auth composition | Minimal core evolution | peer auth moves to router dependencies；Memos composes public + PAT child routers | +| Hot enable/disable | Minimal core repair | one retained host route set，single writer，localized cache invalidation | +| Config update | Minimal generic repair | import `config_cls` while disabled；merge → validate → persist → live assign；remove stale config save on close | +| Mutable graph commands | Minimal generic primitives | caller-session edit/delete/relation operations；Memos traversal remains extension-owned | +| Memo query | Extension-owned query | no generic memo table/index；query canonical root blocks and graph predicates | +| Writable raw storage | New core capability | DB-backed put/get/delete is required by proven attachment journey | +| client-web config save | Cross-repo compatibility fix | current path mismatch blocks the existing JSON config editor | + +## Exact MVP HTTP Surface + +D-047 fixes the following bounded endpoint surface。Anything else remains unregistered and returns `404`； +it is not implemented as an empty success。 + +| Auth | Method and path under `/memos` | Required behavior | +| --- | --- | --- | +| public | `GET /api/v1/instance/profile` | version reports `0.29.1` | +| absent | `GET /api/v1/status` | `404` so MoeMemos selects v1 | +| PAT | `GET /api/v1/auth/me` | one stable deployment-scoped `users/inkcre` projection | +| PAT | `GET /api/v1/users/{id}/settings/GENERAL` | stable default visibility，initially PRIVATE | +| PAT | `GET /api/v1/memos` | NORMAL/ARCHIVED，exact creator filter，pageSize/pageToken，create-time-desc order；exclude comments | +| PAT | `POST /api/v1/memos` | create one independent memo root and attach referenced existing attachments | +| PAT | `PATCH /api/v1/memos/{id}` | explicit `updateMask` query semantics；otherwise D-034 body-key inference | +| PAT | `DELETE /api/v1/memos/{id}` | remove primary memo resource；owned cleanup follows D-046 | +| PAT | `GET /api/v1/attachments` | list attached and unattached resources visible in this deployment | +| PAT | `POST /api/v1/attachments` | streaming JSON/base64 upload；`memo` may be null | +| PAT | `DELETE /api/v1/attachments/{id}` | remove the primary attachment resource and attempt associated cleanup | +| PAT | `GET /file/attachments/{id}/{filename}` | authenticated raw bytes with content metadata；filename mismatch is `404` | +| PAT | `POST/GET /api/v1/memos/{id}/comments` | independent comment roots connected to parent | +| PAT | ordinary memo PATCH/DELETE | update/delete a comment by its memo resource name | + +Protocol validation/auth/resource failures map to stable `400`/`401`/`404`；unexpected database、resolver +or storage failures return `500`。Memos routes should translate Pydantic request errors to protocol `400`； +the peer-authenticated core config endpoint keeps its ordinary `422` validation contract。 + +## Proposed Canonical and Graph Shape + +D-042 confirms this exact CanonicalMemo v1 root content： + +```json +{ + "body": "Markdown text", + "created_at": "2026-08-01T12:00:00Z", + "updated_at": "2026-08-01T12:00:00Z", + "archived": false, + "visibility": "private", + "pinned": false +} +``` + +- `created_at` / `updated_at` remain nullable for the memo-family canonical contract，but backend-created + memos always establish them；block timestamps remain info-base persistence time。 +- `archived`、`visibility` and `pinned` are explicit memo facts needed for update/list/read round-trip。 + They have no independent identity，so the root is their only sensible authority。 +- identity is `block.id`；generation is the resolver id，proposed + `extensions.memos.memo.v1`；there is no content-level id/schema version。 +- attachments、parent and references remain graph-only。Proposed predicates are root → attachment + `attachment:<zero-based-order>`，comment → parent `parent` and memo → target `reference`。 +- backend create uses explicit block creation，never resolver-content `fetchsert`：equal bodies are still + different memos。 + +Attachment block content is a separate versioned memo-extension contract containing only stable metadata +and a storage pointer（for example blob id、filename、media type and decoded size）。Raw bytes do not enter +CanonicalMemo or attachment JSON。 + +## Writable Storage Decision + +D-043 selects a generic PostgreSQL-backed binary storage： + +- a small `storage_blobs`-like table owns only a generated pointer and `BYTEA` raw bytes； +- a built-in database-binary storage type/instance implements caller-session put/get/delete； +- the attachment block owns attachment identity and metadata，and points to that storage instance； +- one PostgreSQL transaction can cheaply cover raw bytes、block and relation where the implementation + chooses；D-041 does not expose this as a completeness guarantee； +- decoded upload size is capped at 32 MiB for the MVP，matching the tagged server's fallback upload buffer + when no instance-level limit is configured；the API streams base64 decoding rather than materializing + multiple copies where practical。 + +This adds one table/migration but avoids the more expensive DB + filesystem compensation protocol。Inline +base64 would avoid a table only by bloating graph content and bypassing the storage abstraction，which is +the wrong owner boundary。The exact table name/columns remain implementation addresses for the Impact +Handshake。 + +## Main-Path and Failure-Branch Walkthrough + +### Runtime and config + +| Branch | Required outcome | +| --- | --- | +| extension import/config validation fails | no routes published；enable returns non-2xx；durable enabled state is compensated/converges disabled | +| route construction fails | retained host remains empty；no partial public/protected surface | +| enable called twice | second call is idempotent；no duplicate routes or resolver registrations | +| disable | unpublish exact route set first，then close runtime resources；all Memos routes become `404` | +| close fails after unpublish | remain fail-closed/unpublished；retain enough runtime state for idempotent cleanup retry，report non-2xx | +| re-enable | repopulate the same retained host once；decoder registry is not duplicated or retired | +| invalid config patch | schema validation before DB write；persisted/runtime values unchanged | +| config/update while disabled | import `config_cls` without publishing routes，validate/persist config，apply on next enable；persisting `config_schema` is optional UI work | +| DB config write fails | runtime value unchanged | +| process dies after config commit before assignment | restart reloads DB authority；the accepted narrow crash window is self-healing | +| extension closes | it must not persist its possibly stale runtime config over the DB authority | + +Resolver decoders interpret durable blocks，so API disable must not unregister a decoder needed by existing +info-base content。Resolver artifact loading and live endpoint/source activation are separate lifecycle +concerns even if both remain coordinated by the extension package。 + +### Memo and query + +| Branch | Required outcome | +| --- | --- | +| create same body twice | two root block ids；no content dedup | +| primary root mutation fails | non-2xx；D-041 permits residual component rows from attempted work | +| resolver unknown/invalid canonical JSON | explicit `500`/unsupported decoder failure；never fallback decode | +| list filter differs from exact deployment creator expression | `400`，not silently ignored | +| invalid/foreign page token | `400`；token binds version/filter/state and cursor | +| inserts occur between pages | opaque keyset cursor on `(created_at, block.id)` prevents shifting duplicates | +| comment exists | top-level memo list excludes roots with a `parent` relation；comment endpoint includes them | +| resolver relation read | incoming + outgoing uses OR/full star graph；direction cache cannot reuse an incomplete result | + +No query projection/index is introduced in the MVP。For personal-scale data，resolver-key + canonical JSON +query and relation exclusion are sufficient；add an index only after measured pressure，because indexing is +an application/retrieval support concern rather than memo collection authority。 + +### PATCH, attachments and delete + +| Branch | Required outcome | +| --- | --- | +| no `updateMask` | infer only from raw JSON keys；`false`、`""` and `[]` count as present | +| explicit `updateMask` | parse query value under 0.29.1 field-mask semantics；do not add inferred fields | +| empty/unknown/unupdatable mask | `400`，no mutation | +| attachment upload with `memo=null` | commit an independently addressable orphan attachment + raw bytes | +| later memo create fails | uploaded attachment remains valid/listable/deletable；do not call it partial memo graph | +| attach/reorder | request list is the complete set；rewrite positions with the simplest local DB operation | +| PATCH omits attachments | preserve current set；PATCH includes `attachments: []` deletes current owned attachments | +| raw write or graph mutation fails | return the exact non-2xx when the primary operation fails；orphan/raw/relation residue is permitted and diagnosable | +| attachment raw lookup missing/filename mismatched | `404` without altering graph | +| delete parent memo | root must disappear from memo list/read；attempt owned comment/attachment/raw cleanup；keep referenced memo targets | +| corrupted ownership cycle | visited-set traversal terminates and avoids over-deleting；cleanup residue is acceptable | +| repeated delete | `404`；no fabricated idempotent success unless exact upstream fixture proves otherwise | + +An attachment has zero or one owning memo attachment relation in this MVP。Orphan is valid；multiple memo +owners are rejected。Reference relations do not imply ownership。 + +## Implementation Addresses and Blast Radius + +### core-py + +- `run.py` / `app/middleware.py`：peer auth dependency topology；public health/docs remain public。 +- `app/business/extension/main.py`：retained route host、running map、disabled config validation、config + update ordering、decoder/live lifecycle separation；private FastAPI invalidation localized here。 +- `app/routes/extension.py`：generic patch-like config update behavior behind the existing management path。 +- `app/database_contract/profile.py`：Memos artifact and database-binary storage catalog entries；remove + duplicated built-in storage literals while touching that authority。 +- `app/business/info_base/block.py` / `relation.py`：caller-session mutable primitives；relation OR/cache + correctness。No Memos predicates in core managers。 +- `app/business/info_base/storage/` + `app/schemas/info_base/`：writable database binary storage and raw + bytes table/model。 +- `extensions/memos/`：config、protocol DTO/adapter、service、canonical models、graph repository、resolver、 + query and attachment handling。 +- `migrations/`：one storage migration；application-table manifest/readiness/migration tests updated。 +- `tests/extensions/memos/` plus focused existing subsystem tests：pure、ASGI lifecycle and PostgreSQL + integration evidence。 + +### client-web + +- `packages/core/src/extension/base.ts`：send config updates to `/extensions/{id}/config` and test the + request shape。This is a separate repo/change batch；it does not authorize shared-doc or core commits。 +- a new SQL table also changes the generated database contract projection if client-web tracks the full + application schema。 + +### durable/shared documentation + +No durable doc is edited during this unit design。Accepted product/cross-unit/local/runtime truths continue +to accumulate in the program promotion packet and must later be applied by owner；Hub source edits、shared +ref bumps and Spoke implementation remain separate commits/workflows。 + +## Verification Readiness + +- SVC 10.0.1 status is healthy；the worktree database target is ready，with a known profile and migration + head。No environment mutation was needed for preflight。 +- Existing pure pytest intentionally uses unreachable PostgreSQL and skips extension sync；new transaction + tests must explicitly use the worktree-scoped PostgreSQL runtime or a disposable schema。 +- Verification remains four distinct layers：pure contract tests、ASGI auth/lifecycle tests、PostgreSQL + graph/storage/migration tests and the pinned MoeMemos APK evidence bundle。No layer substitutes another。 + +## Remaining Execution Gate + +Technical、Acceptance and preflight are complete，but execution is not authorized yet： + +1. Prepare/review the Impact Handshake for core-py、client-web、migration and later durable-doc state diff。 +2. Sir explicitly says “开始”。 + +Any new evidence that changes owner、schema or observable behavior returns to its design gate；ordinary +symbol names and local implementation mechanics are resolved during the final Impact Handshake。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-plan.md new file mode 100644 index 0000000..c14c0f3 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-plan.md @@ -0,0 +1,555 @@ +# RSS Extension Hardening Implementation Plan + +## Control + +- **Status**: complete and human-accepted；Sir authorized implementation on 2026-08-02，B0–B8 completed the same + day，and accepted the final verification horizon on 2026-08-03。Durable owner projections and owner-specific + commits are complete；push、Hub publication、shared-ref bump and production migration remain separate gates。 +- **Unit**: `rss-extension-hardening`，including the accepted horizontal core/storage/resolver and Memos propagation + required by the RSS vertical。 +- **Inputs**: D-049–D-076 and the accepted + [semantic-content resolver contracts](semantic-content-resolver-contracts.md)。 +- **Validation evidence**: this plan was written first，then its addresses、dependency order、migration branches and + runtime assumptions were replayed in the plan-specific + [preflight report](implementation-preflight.md)。Preflight may reject or revise this plan，but is not an input that + invents its steps。 +- **Start gate**: satisfied；each batch still receives its repository-specific Impact Handshake before mutation。 +- **Durable gate**: unresolved discussion pressure stays in the packet；verified stable contracts are projected to + their durable owners during unit completion。Hub publication、shared-ref bump、commit/push and production migration + remain independent operations。 + +## Outcome And Completion Boundary + +The unit is complete when one configured RSS 2.0 or Atom source can travel through the accepted runtime path： + +```text +source schedule/manual command + -> traceable collect job + -> bounded HTTP fetch + feedparser adapter + -> exact feed/item reconciliation + -> committed feed/item/enclosure graph + -> resolver-instance use projection + -> deterministic source state advance + -> optional full-text and enclosure materialization +``` + +At the same time，the nine common semantic content resolver contracts work in core-py and client-web，PostgreSQL +binary storage is a complete peer-local CRUD capability，Memos attachments use metadata → semantic content → storage， +and no current producer emits retired bare resolver or content-specific HTTP storage IDs。 + +This plan does **not** deliver a feed-reader UI、OPML、S3、organization child expansion、OCR/STT、semantic retrieval or +production deployment。 + +## Dependency Topology + +```text +B0 freeze contracts / baselines + ├─> B1 database protocol + storage + hydration ─┐ + └─> B2 resolver base + exact registry/bootstrap ├─> B3 nine semantic resolvers + │ ├─> B4 Memos attachment v2 + │ ├─> B5 existing producer cut-over + │ └─> B6 RSS primary collection + │ └─> B7 enrichment/materialization + └────────────────────> B8 integrated acceptance +``` + +No batch is a publication or commit boundary。An edit pass inside B1/B2 may temporarily make the worktree non-green， +but every batch must end green before its dependent batch starts；B2 therefore updates all concrete resolver classes +before completion，rather than deferring breakage to B3。B4–B7 are separately verifiable verticals and must not be +merged into one debugging pass。 + +## Plan-wide Invariants + +1. `block.content` remains actual inline content or an opaque storage pointer；hydrated content never overwrites it。 +2. A configured storage returns actual bytes and never decides image/video/HTML/PDF semantics；inline block content + may already be a Unicode string without passing through storage。 +3. Resolver capability execution occurs on a resolver instance；Manager owns exact registration/selection and shared + matching helpers only。 +4. Protocol/source-authored facts remain on metadata/root blocks；byte-derived facts remain solved projections unless + a separately justified organization command materializes them。 +5. `refresh` replaces a local snapshot；`materialize_missing` permits an absent derivation；neither is a synonym for + recompute、redownload or `force`。 +6. Existing user-authored/unrelated worktree changes are preserved。Cross-repo edits are reviewed and verified in + their owning repository。 +7. Implementation and tests use disposable PostgreSQL/Neon-compatible databases。Canonical production remains + read-only evidence until a separately authorized delivery/migration operation。 +8. Runtime compatibility is explicit：Memos v1 receives the accepted one-time migration；retired bare core IDs do not + receive aliases、decoders or row migration。 + +## Frozen Interfaces Used By The Plan + +### Core-py block and resolver surface + +```python +class BlockModel: + async def get_hydrated_content(self, *, refresh: bool = False) -> str | bytes: ... + + +class Resolver(ABC, Generic[SolvedT]): + async def get_solved_content( + self, + *, + refresh: bool = False, + materialize_missing: bool = True, + ) -> SolvedT: ... + + @abstractmethod + async def get_text( + self, + *, + refresh: bool = False, + materialize_missing: bool = True, + ) -> str | None: ... + + @abstractmethod + async def get_str_for_embedding( + self, + *, + refresh: bool = False, + materialize_missing: bool = True, + ) -> str | None: ... +``` + +- `UnsupportedResolverCapability` means the resolver contract does not provide the requested projection。 +- `UnknownResolverError` means no exact decoder is installed/registered for the block's resolver ID。 +- `None` means the capability exists but this block has no meaningful result。 +- Hydration caches `(storage, content) -> hydrated value` in Pydantic private state。A changed pointer/key misses the cache + naturally；`refresh=True` bypasses and replaces it。No ORM event hook or cross-instance invalidation is added。 +- Resolver solved/relation caches follow the same `refresh` spelling。The old raw-content cache disappears；resolver + hydration delegates to its block。 +- `ResolverManager.match_media_type()` normalizes a candidate media type and returns an exact registered semantic + resolver ID or `None`。Extensions still own evidence order and fallback policy；`core.file.v1` is selected by the + extension only after its own ladder fails。 + +### Client-web peer projection + +```ts +abstract class Resolver<RawT, SolvedT> { + abstract getText(options?: ProjectionOptions): Promise<string | null> + abstract getStrForEmbedding(options?: ProjectionOptions): Promise<string | null> +} + +type ProjectionOptions = { + refresh?: boolean + materializeMissing?: boolean +} +``` + +- `Block.getHydratedContent({ refresh })` returns inline `string` or storage-backed `ArrayBuffer`/`Uint8Array`。 +- Browser `Blob`、Object URL and renderer handles are private runtime state，revoked on refresh/dispose/cache eviction。 +- Unknown exact resolver and unsupported capability are different typed errors；there is no default resolver fallback。 +- Parser-derived solved fields may remain `null` when the browser peer lacks a proportionate local parser。Open/render/ + download remains a real local capability rather than a call to core-py。 + +### Shared relation vocabulary introduced by this unit + +| From | Relation content | To | Meaning | +| --- | --- | --- | --- | +| feed item | `feed` | feed | exact feed membership and identity scope；not deletion ownership | +| feed item | `enclosure` | enclosure metadata | unordered native enclosure component | +| feed item | `full_text` | `core.text.v1` | optional fetched main-text enrichment | +| enclosure metadata | `content` | semantic content | downloaded bytes interpreted by an exact core resolver | +| Memos attachment metadata | `content` | semantic content | uploaded bytes interpreted by an exact core resolver | + +RSS enclosure order is not promoted because no current use requires it。Memos keeps its already accepted ordered +`attachment:<order>` owner relation。 + +## B0 — Freeze Working Baseline And Contract Cases + +### Changes + +- Pin the selected dependency ranges in the root and RSS extension manifests before implementation：Pillow、PyAV、 + pypdf、puremagic、feedparser and Trafilatura；regenerate the owning PDM locks through PDM。 +- Add an exact resolver-ID/type case table consumed by core registration/matching tests and static retired-ID checks。 +- Record licensed provenance for repository-generated real-format acceptance samples（image、audio、video、PDF、 + EPUB、ZIP）without committing derived outputs。 +- Capture current green commands and the existing dirty-worktree boundary in the packet；do not stage unrelated files。 + +### Primary addresses + +- core-py：`pyproject.toml`、`pdm.lock`、`extensions/rss/pyproject.toml`、`extensions/rss/pdm.lock` +- tests：`tests/assets/semantic-content/` source manifest/generator、ignored generated real-format files and shared + on-demand pytest fixture +- client-web：no parser dependency is added merely to make nullable solved facts non-null。 + +### Verification + +- `pdm run check:lock` +- import probe for each selected direct dependency under Python 3.12 +- container build/import probe proving PyAV's selected wheel works in the production Python image +- license/source manifest contains no copied sample with unknown redistribution status + +### Execution evidence — complete 2026-08-02 + +- Root and RSS locks resolved with prior pins reused where compatible；the selected exact versions are Pillow 12.3.0、 + PyAV 18.0.0、pypdf 6.14.2、puremagic 2.2.0、feedparser 6.0.14 and Trafilatura 2.2.0。 +- Repository-generated、Git-ignored real-format assets cover the nine-ID case table with no third-party payload + provenance；the shared pytest fixture rebuilds them from a clean checkout。 +- Local Python 3.12 and the production `python:3.12-slim` Dockerfile both imported every selected direct dependency。 +- New B0 code/task-plan surfaces pass targeted Ruff lint/format and lock checks；the repository-wide formatter reports + unrelated pre-existing Markdown/Memos-test drift recorded in the unit packet。 + +## B1 — Database Protocol、Storage Mechanics And Block Hydration + +### State diff + +```text +content-kind HTTP storages + C/R/D writable storage + resolver-owned raw cache + -> one bytes-only HTTP storage + C/R/U/D writable storage + block-owned hydration cache +``` + +### Core-py changes + +1. Make `HTTPStorage` the concrete `http` bytes handler；remove semantic subclasses from current exports/registration。 + Built-in `-1` becomes the generic HTTP instance。Its config owns timeout/redirect/maximum-response-byte mechanics + only，and its chunked HTTP read enforces the byte limit without creating a streaming storage contract。Retired + `-2/-3` records may remain historical database rows but no current code emits them。 +2. Add `WritableStorage.update_raw_content()` and implement pointer-stable PostgreSQL byte replacement。 +3. Add `BlockModel.get_hydrated_content(refresh=False)` with private value+source-key cache；move resolver hydration to + that method and remove `real/raw` ambiguity from current code/docstrings。 +4. Change `blocks.storage -> storages.id` deletion to `RESTRICT` in SQLModel metadata and an append-only Alembic + revision after `f2c8a6d1e4b7`。 +5. Add admitted `inkcre.create_storage_blob(bytea) -> uuid` and + `inkcre.read_storage_blob(uuid) -> bytea` functions for raw PostgREST transport。They are invoker-rights functions； + role reconciliation grants only the normal authenticated peer surface。 +6. Extend the core protocol document with function signatures and ensure `storage_blobs` remains bytes-only。 + +### Client-web changes + +1. Extend database-contract generation for `Functions` instead of hand-editing `generated.ts`。 +2. Add a narrow authenticated raw PostgREST fetch utility adjacent to `DBAPIClient`；it reuses the current dynamic JWT + and origin config but returns `ArrayBuffer` for octet-stream calls。 +3. Implement PostgreSQL binary create/read/update/delete：raw RPC create/read，exact UUID `bytea` PATCH update and + exact UUID delete。Centralize `{"blob_id":"..."}` pointer parsing/serialization。 +4. Move hydration to `Block.getHydratedContent()` and make the private cache non-enumerable/non-transported。 + +### Primary addresses + +- core-py：`app/schemas/info_base/block.py`、`app/business/info_base/storage/{main,http,postgresql,__init__}.py`、 + `app/database_contract/{profile,protocol,roles}.py`、new migrations、migration integrity/tests +- client-web：`packages/core/src/{base/db-api,info-base/block,info-base/storages/*,database/*}`、 + `scripts/database-contract-lib.mjs`、generated contract artifacts、peer-database E2E + +### Acceptance + +- PostgreSQL bytes survive create → hydrate → same-pointer update → refresh → delete byte-exactly。 +- Default hydration reuses its instance snapshot；refresh observes the updated blob；a second block instance is not + promised invalidation。 +- Missing blob/storage handler is explicit；deleting a referenced storage catalog row is RESTRICTed。 +- Storage CRUD never queries or rewrites blocks and never owns MIME/filename/semantic facts。 + +### Execution evidence — complete 2026-08-02 + +- Both peer implementations now own block-instance hydration and generic HTTP/PostgreSQL byte mechanics；the complete + client-web check proves the storage hard cut does not leave application imports or distribution output broken。 +- Disposable PostgreSQL/PostgREST reached exact v2 readiness at `d0e3f4a5b6c7`，then passed authenticated raw + C/R/U/D with byte-exact reads before and after pointer-stable update。 +- Runtime replay rejected two plausible-but-wrong initial assumptions and fixed them append-only：the existing trigger + helper was an internal function leaked into the exposed schema，and PostgREST 14 requires an explicit media-type + domain for raw `bytea` responses。Readiness now checks function signatures as well as names/ACLs so the second defect + cannot hide behind a green catalog projection。 +- Verification：core-py `check:migrations`、typecheck、266 passed/6 skipped；client-web full `pnpm check`、47 tests and + all builds。The client delivery pin remains unchanged until the B8 artifact-ordering step。 + +## B2 — Resolver Base、Exact Registry And Bootstrap + +### Changes + +1. Introduce shared Python/TypeScript error、options and exact core resolver-ID types。 +2. Keep Python and make TypeScript resolver bases abstract for text/embedding methods；update every current concrete + resolver in both repositories in the same batch so missing capability declarations are statically/runtime visible。 +3. Add exact duplicate-registration checks and explicit unknown-ID errors；re-registering the same class is idempotent， + while a different class claiming the same ID fails。Remove client-web's first/default resolver fallback。 +4. Replace `force` with `refresh` on InKCre-owned resolver/relation/cache calls；keep third-party protocol parameters + unchanged。 +5. Add explicit `register_core_resolvers()` bootstrap before extension loading and outside `SKIP_EXTENSIONS_SYNC`。 + Core resolver availability no longer depends on importing an arbitrary extension package。 +6. Make embedding/context consumers skip supported-null and handle unsupported capability explicitly rather than + embedding `""` or aborting the whole missing-embedding scan。The periodic scanner also records/skips unknown retired + IDs，while direct application resolution still raises the exact unknown-ID error；this prevents historical hard-cut + rows from repeatedly failing the whole scheduler job。 + +### Primary addresses + +- core-py：`app/business/info_base/resolver/{main,__init__,bootstrap,contracts}.py`、`run.py`、all concrete extension + resolvers、`app/business/sink/{embedding,main}.py`、`app/schemas/info_base/block.py` +- client-web：`packages/core/src/info-base/resolvers/{base,cache,contracts,index}.ts`、all extension resolvers and + resolver call sites + +### Acceptance + +- An unregistered ID fails as unknown in both peers；retired IDs are not reinterpreted。 +- Every concrete resolver explicitly implements both abstract methods。 +- unsupported、supported-null and authored-empty are three observable outcomes。 +- Application boot with extension sync disabled still resolves all nine core IDs。 + +## B3 — Nine Semantic Content Resolvers And Peer-local Use + +### Core-py changes + +- Replace the old text/HTML/image/video implementations and delete the image resolver's import-time AI credential/ + remote side effect。 +- Add exact `core.text/html/image/audio/video/pdf/epub/zip/file.v1` modules with the accepted solved shapes。 +- Text uses inline Unicode or BOM/strict UTF-8 bytes；HTML additionally honors a bounded in-document charset + declaration and exposes decoded source plus a derived text projection。 +- Pillow reads image format/dimensions/frame count without pixel decode；PyAV selects the first default-disposition + stream，or otherwise the first non-attached-picture stream of the requested kind，without frame decode；pypdf reads bounded root/page metadata without text extraction；EPUB and + ZIP inspect only bounded central-directory/package metadata and never extract；puremagic supplies optional bounded + detected MIME。 +- Invalid claimed format is a resolution error；encrypted/protected valid content returns encryption facts and null + for inaccessible optional facts。 +- CPU/native parser inspection runs through bounded worker-thread calls so one Pillow/PyAV/pypdf/ZIP operation does + not synchronously block the event loop；the parser does not gain an unbounded process/thread pool。 +- `core.text.v1` and `core.html.v1` provide text/embedding projection。Image/audio/video/PDF/EPUB/ZIP/file v1 explicitly + raise `UnsupportedResolverCapability` for those methods until a real caption/transcript/text-extraction capability + exists；metadata/title facts in solved content are not misrepresented as the file's textual content。 + +### Client-web changes + +- Register all nine exact IDs and provide local safe handles：text/HTML preview，image/audio/video/PDF object URLs or + native elements，EPUB/ZIP/file open/download。 +- Never use `v-html` without a separately admitted sanitizer；the MVP HTML component renders a text preview/source + action。 +- Remove raw `block.content` fallbacks from BlockContent、graph preview and editors；storage-backed blocks never expose + pointer JSON as authored content。 +- Revoke object URLs on refresh、resolver disposal and resolver-cache eviction。 + +### Acceptance + +- Each ID resolves a real sample persisted through PostgreSQL binary storage in core-py。 +- Browser peer hydrates and offers a usable local handle for every ID；unsupported parser-derived facts remain null。 +- No resolver-specific solved model duplicates storage pointer、source filename or declared MIME authority。 +- OCR、STT、PDF text、EPUB chapters and ZIP member graphs remain absent rather than faked。 + +## B4 — Memos Attachment V2 And One-time Migration + +### Changes + +1. Append the D-076 reversible data revision：for each exact v1 attachment，validate canonical JSON，extract `blob_id` + into minimal pointer JSON on a new semantic child selected from normalized Memos MIME，rewrite the same metadata + block ID to inline v2 content，and create one `content` relation。One migration transaction prevents partial row + conversion。Downgrade is guarded：it reconstructs v1 for every v2 attachment only when each has exactly one + exclusive PostgreSQL semantic child with the reversible pointer shape；otherwise it refuses before mutation rather + than deleting shared or post-upgrade information。 +2. Remove `blob_id` from `CanonicalAttachment` v2；add `content_block_id` only to the solved/runtime projection。 +3. Rewrite attachment create/list/solve/download/delete to traverse metadata → semantic content through exact + resolvers。Memos wire 0.29.1 continues to expose the metadata block ID/filename/type/size/create time。 +4. Exclusive deletion removes blob + semantic child only when no other metadata block points to it；shared semantic + content survives。Memo ordered ownership and unattached attachment behavior remain unchanged。 + +### Primary addresses + +- migration + integrity entry +- `extensions/memos/family/{schema,graph,attachment,attachment_resolver,resolver}.py` +- Memos backend attachment adapter/tests and local Unit TDD candidate list（not durable mutation in this batch） + +### Acceptance + +- Fresh upload、unattached list、attach/reorder、native read/download and deletion traverse the new graph through real + PostgreSQL bytes。 +- Seeded v1 upgrade preserves metadata block ID、blob UUID/bytes and memo slot relations；a guarded downgrade + round-trips every reversible v2 shape and rejects a non-exclusive/shared shape before mutation。 +- A disposable database beginning at the current production head `d9f4e2a1b7c3` proves the empty v1 path through + repository head；canonical production itself is not migrated by acceptance。 + +## B5 — Existing Producer/Consumer Hard Cut + +### Changes + +- Twitter Python collection emits exact core text/HTML/image/video IDs and uses generic HTTP bytes storage。Repair the + API DTO → canonical root → attachment relation gap exposed by preflight；version the repaired root as + `extensions.twitter.tweet.v1` and mirror it in the client-web Twitter resolver。 +- Keep Twitter attachment source facts relation-owned rather than copying URL arrays into canonical Tweet content。 +- Update webext Taking Note and Arcs Editor producers to `core.text.v1` / `core.html.v1` as appropriate；remove the + unregistered `url` pseudo-resolver。 +- Update all current tactical guides/examples and replace `tests/test_resolver_breakdown.py` with behavior that belongs + to the new resolver/materialization contracts；do not retain the old AI-breakdown prototype as compatibility proof。 + +### Acceptance + +- Twitter collection → graph → TweetResolver regression covers text、photo、video、link and reply without accessing + dropped DTO fields。 +- Repository-wide static scans find no current producer/example emitting bare `text/html/image/video` or + `http_image/http_video/http_html/http_json/http_text`。 +- Direct reads of old IDs fail explicitly in both peers。 + +## B6 — RSS Primary Collection Rewrite + +### Product/graph shape frozen by this plan + +- Keep extension ID `rss` and the two durable source type IDs `extensions.rss.rss.Source` and + `extensions.rss.atom.Source`。Their modules become thin protocol-expectation wrappers over one shared source/service； + parsing、HTTP、reconciliation and graph logic are not duplicated。 +- New exact resolver IDs：`extensions.rss.feed.v1`、`extensions.rss.feed_item.v1` and + `extensions.rss.enclosure.v1`。 +- Canonical feed content owns source instance ID、optional source-native feed ID、declared self URL、configured URL + and feed-authored title/home/description/language/update facts。The identity ladder derives from those exact facts； + it is not copied into a second generic identity-value field。 +- Canonical item content owns optional source-native ID + kind（Atom ID or scoped RSS GUID），optional alternate link， + title/summary/feed-authored content、published/updated times、authors and categories。When native ID is absent，the + alternate link itself is the fallback identity evidence；it is not duplicated into an identity-value field。The + item excludes source instance ID because its `feed` relation provides the scope，and excludes enclosures/fetched + full text because graph relations own them。 +- Canonical enclosure metadata owns URL、declared media type/length and optional title；download result is graph-only。 + +### Source config/state + +- Config validates non-empty HTTP(S) `feed_url`、timeouts/body limits、`fetch_full_text=True`、 + `download_enclosures=False`、enclosure size limit and target writable storage ID（default PostgreSQL `-4`）before + effects。The initial bounded defaults are 30 seconds，8 MiB feed body，8 MiB article body and 64 MiB enclosure； + source config may deliberately raise the byte limits without changing the resolver contract。 +- Job config is typed and contains only source-specific supported overrides；legacy generic `full` is rejected rather + than silently changing reconciliation semantics。 +- State owns conditional HTTP ETag/Last-Modified and one last successful contentful snapshot observation time。That + same timestamp is D-056's next unidentified-item admission watermark；it is not copied into a second state field。 + It does not retain unordered `seen_ids`。 + +### Collection command sequence + +1. A manual request or schedule creates an ordinary PENDING collect job。The scheduler never calls `source.collect` + directly；a shared `SourceCollectJobManager.create()` seam feeds the existing runner。Pending dispatch uses the + database job ID as the deterministic scheduler job ID and an atomic claim/status transition，so repeated polling + cannot run one collect job twice。 +2. The source validates config/job config，captures `snapshot_observed_at` when the complete response is received，and + sends response bytes/effective URL/headers to feedparser in a bounded worker-thread call。 +3. Fatal transport、unsupported feed family or unusable feed document fails the job and does not advance source + state。A parseable `bozo` feed may continue with a diagnostic；one malformed item is skipped with a diagnostic。 +4. Feed reconciliation creates/updates the exact feed block。Each valid item primary graph is committed in its own + transaction：same exact identity updates the existing root，new identity creates，missing old items are untouched。 +5. Unidentified items obey create/discard config。Create uses D-056's source-time watermark only as an admission + filter；it never becomes identity or document-order short-circuit。 +6. After every valid primary item has been considered without a primary persistence failure，advance conditional/ + observation state once。Previously committed items may remain after a later primary failure；retry exact + reconciliation makes that residue safe。 +7. Job status is FINISHED when fetch/parse and all admitted primary item writes succeed。Skipped malformed items and + optional-enrichment failures are structured diagnostics in `job.state`，not silent success；primary failure marks + FAILED and preserves diagnostics/residue facts。 + +### Primary addresses + +- shared runtime：`app/business/source/{main,collect_job}.py` and source tests +- RSS rewrite：`extensions/rss/{rss,atom,source,http,adapter,schema,repository,resolver,service,__init__}.py` +- replace `tests/extensions/test_rss.py` with black-box suites under `tests/extensions/rss/` + +### Acceptance + +- Hermetic HTTP double serves actual RSS 2.0 and Atom bytes through real transport into a real PostgreSQL graph。 +- Cover create、same-content replay、same-ID update、new item、missing old item、unidentified create/discard/watermark、 + 304、malformed item、fatal feed and process retry residue。 +- Resolver text prefers feed-authored content in the primary slice；feed/item/enclosure authorities remain separately + inspectable。 +- Scheduled trigger produces one traceable job and follows the same runner/state semantics as manual collection。 + +## B7 — Default Full-text And Enclosure Materialization + +### Full-text enrichment + +- After primary item commit，default-on enrichment fetches the item link with the same bounded HTTP policy and passes + already-downloaded HTML to Trafilatura in a bounded worker-thread call。 +- Successful main text is an inline `core.text.v1` block related from the item by `full_text`。No extra metadata block + is added because the item link already owns the source URL and the enrichment has no independent protocol identity。 +- Existing full text is reused for unchanged item/link。An item/link change may replace the relation target in a new + transaction。Failure records a diagnostic and does not fail primary collection or advance/rollback primary state。 +- FeedItemResolver prefers the related full-text block for text/embedding use，then falls back to feed-authored + content/summary/title；the feed-authored root remains authority and inspectable。 + +### Enclosure materialization + +- `POST /rss/enclosures/materialize` accepts exact enclosure metadata block IDs plus target writable storage ID and + returns one result per input (`enclosure_id`、existing/new semantic block ID or explicit error)。Each enclosure is an + independent command transaction，so partial results are observable rather than disguised as all-or-nothing。 +- The service obtains the exact enclosure resolver instance，derives a download command，performs bounded HTTP，runs + the RSS/Atom-specific classification ladder，writes bytes to storage，creates the exact semantic block and one + `content` relation。 +- If a valid existing content relation resolves，manual/automatic materialization returns it idempotently；the MVP has + no redownload/recompute flag。Concurrent attempts re-check under enclosure-row lock before creating the child。 +- Automatic policy invokes the same application service after primary collection。Unavailable storage、download or + resolver failure becomes an item/job diagnostic and does not erase enclosure metadata or fail primary collection。 + +### Acceptance + +- Manual and automatic paths materialize real image/audio/video/PDF/EPUB/ZIP and unknown file samples through the + same service，not test-only parser helpers。 +- RSS declaration and Atom HTTP/advisory precedence follow D-070–D-072；observed/detected MIME never overwrites metadata。 +- Replay/concurrency creates at most one current `content` relation/materialized semantic child per enclosure。 +- Failed download leaves the enclosure metadata readable and no success relation；per-input API results expose any + prior committed siblings。 + +## Execution Evidence — B2–B7 complete 2026-08-02 + +- B2/B3：exact resolver registry/bootstrap、typed capability outcomes、nine Python/TypeScript semantic resolvers、 + block-owned hydration and client render/open/disposal paths pass static and real-format acceptance。 +- B4/B5：Memos attachment v2 migration/runtime and Twitter/webext exact producer cut-over pass their targeted and real + PostgreSQL suites；the writable storage seam now returns storage-owned opaque pointer text without exposing + PostgreSQL pointer grammar to RSS or Memos callers。 +- B6：the scheduler and manual route share `SourceCollectJobManager.create()`，pending claims are atomic，and RSS/Atom + source modules are durable-identity wrappers over one HTTP/parser/reconciliation service。State scopes conditional + headers to the configured URL and the source-time watermark to an exact persisted feed root，so config/feed identity + changes cannot reuse unrelated cursors。 +- B7：default full text、resolver-preferred use projection、manual API、automatic enclosure policy and concurrent + idempotency are implemented。The PostgreSQL black box materializes and resolves real image/audio/video/PDF/EPUB/ZIP + and unknown file bytes；RSS declaration and Atom observed HTTP precedence remain inspectable without rewriting + enclosure metadata。 + +## B8 — Integrated Verification And Promotion Preparation + +### Core-py verification + +- `pdm run check:lock` +- `pdm run check` and the repository's static/type/migration checks +- targeted semantic-content、storage、Memos、Twitter、RSS black-box/integration suites +- fresh database `base -> head` and seeded `f2c8... + Memos v1 -> head -> downgrade` migration journeys +- opt-in live RSS and Atom smoke against replaceable public endpoints；only stable collection invariants are asserted + +### Client-web verification + +- root `pnpm check`，package/app type-check and targeted Vitest resolver/storage tests +- PostgREST browser E2E for byte-exact PostgreSQL CRUD、JWT denial and missing UUID +- component proof for unknown resolver、unsupported renderer、object URL disposal and pointer non-disclosure + +### Cross-repo completion review + +- static retired-ID/storage-ID scan in both repositories +- code review of exact diff ownership and generated artifacts +- reconcile task-packet candidates into their Hub/core-py/client-web durable owners after implementation evidence +- no production migration、commit or push without a new explicit instruction + +### Execution evidence — complete 2026-08-02 + +- core-py：Pyrefly zero diagnostics；293 passed/19 environment skips；migration suite 22 passed/2 skipped at + `e1f4a5b6c7d8`；real PostgreSQL Memos+RSS run 15 passed。Repository lint and implementation-owned Ruff format/ + retired-ID scans are green。The repository-wide formatter retains four unrelated pre-existing Markdown guide + drifts，recorded rather than silently edited。 +- client-web：complete `pnpm check` passed all 56 unit/runtime tests，workspace type checks and production builds。 +- live protocol acceptance：replaceable opt-in RSS/Atom tests consume URLs selected through + `INKCRE_LIVE_RSS_URL` / `INKCRE_LIVE_ATOM_URL`；they skip rather than pinning an external endpoint when none is + selected。 +- delivery boundary after acceptance：Hub `48b069f`、core-py `835f89a` and client-web `765b22f` record the + owner-specific batches；no production mutation、push or shared-ref bump was performed。 +- durable validation：Hub `git diff --check` + SVC `init` noop；45 relative links resolved；core-py owner docs targeted + Ruff format and repository lint passed；client-web complete `pnpm check` passed 56 tests、types and builds。 + +## Implementation Loop Per Batch + +For each B1–B7 batch： + +1. restate a batch-specific Impact Handshake against the addresses above； +2. make one coherent edit pass，preserving unrelated worktree changes； +3. run static checks before adding tests that merely repeat types； +4. run the smallest black-box/integration scenario that proves the changed behavior； +5. inspect the diff and update this packet with new evidence/branch changes； +6. continue to the dependent batch only when the batch's acceptance is green or a newly exposed design decision has + returned to Sir。 + +## Known Stop Conditions + +Return to discussion instead of improvising if implementation evidence shows： + +- the raw PostgREST byte RPC cannot provide the accepted octet-stream contract without a materially larger server + extension； +- PyAV's selected wheel cannot run in the production image or metadata inspection requires frame decode； +- an existing Memos v1 row cannot be losslessly mapped to the accepted v2 graph； +- RSS exact reconciliation requires a generic binding table or a new persistent field not approved here； +- client-web needs a server delegation to satisfy a capability that was accepted as peer-local； +- a shortcut would reintroduce semantic storage types、pointer disclosure、silent resolver fallback or duplicate + authority。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-preflight.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-preflight.md new file mode 100644 index 0000000..25cbcea --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/implementation-preflight.md @@ -0,0 +1,180 @@ +# Semantic Content / RSS Implementation Preflight + +## Control + +- **Status**: replayed against the now-approved + [implementation plan](implementation-plan.md)；D-076 closes the Memos data branch。This file does not authorize or + sequence implementation。 +- **Input**: D-057–D-076， + [semantic-content-resolver-contracts.md](semantic-content-resolver-contracts.md) and + [implementation-plan.md](implementation-plan.md)。 +- **Purpose**: challenge the implementation plan's addresses、dependencies、runtime assumptions、migration paths and + acceptance seams before it becomes an execution baseline。The plan owns intended steps；this file owns evidence and + early fault discovery。 + +## Resolver Capability Boundary + +Capability execution belongs to a resolver instance： + +```python +resolver = ResolverManager.get(block) +text = await resolver.get_text(materialize_missing=False) +``` + +`ResolverManager` owns exact-ID registration/selection and optional shared MIME normalization/detection helpers。It +does not proxy `get_text()`、`get_str_for_embedding()` or other instance capabilities。The Python and TypeScript base +classes keep those two methods abstract；every concrete resolver implements them explicitly： + +- unsupported capability raises `UnsupportedResolverCapability`； +- supported capability with no meaningful value for this block returns `None` / `null`； +- authored empty text remains `""` only when the exact content contract permits it； +- embedding callers skip `None` and do not translate unsupported into an empty embedding input。 + +The new common implementations are imported/registered by explicit core bootstrap，not accidentally through an +extension package import。Extension resolver registration remains extension-owned and installed-lifetime scoped。 + +## Proposed Core Runtime Dependencies + +These are implementation choices，not shared protocol fields。Peer runtimes may use different parsers while +preserving the exact resolver IDs and nullable solved-fact meanings。 + +| Resolver | core-py implementation | client-web implementation | +| --- | --- | --- | +| `core.text.v1` | inline Unicode；storage-backed bytes use deterministic Unicode BOM handling then strict UTF-8 | `TextDecoder` with the same minimum policy | +| `core.html.v1` | decoded source；Unicode BOM / bounded in-document charset declaration / strict UTF-8；existing HTML-to-text library only for text projection | decoded source and text-only preview；no unsanitized `v-html` authority | +| `core.image.v1` | direct Pillow dependency；header/verification without pixel decoding | Blob/Object URL plus browser image metadata；dispose URLs on refresh/eviction | +| `core.audio.v1` | direct PyAV dependency；inspect stream/container metadata without decoding frames | Blob/Object URL + native media metadata；unavailable codec/container facts stay null | +| `core.video.v1` | the same PyAV dependency；inspect the deterministic primary video stream without decoding frames | Blob/Object URL + native media metadata；unavailable codec/frame-rate facts stay null | +| `core.pdf.v1` | direct pypdf dependency；root metadata/page tree only，no text/OCR | native open/render handle；unavailable parser facts stay null in the MVP peer | +| `core.epub.v1` | `zipfile` + existing hardened lxml configuration；read only container/OPF/navigation metadata | open/download handle；a future direct parser may populate more nullable facts | +| `core.zip.v1` | stdlib `zipfile` central directory only；never extract | open/download handle；parser facts may be null until a direct bounded runtime dependency is justified | +| `core.file.v1` | direct puremagic dependency for optional bounded byte-signature detection | generic Blob/open/download；detected MIME may remain null | + +PyAV is preferred over a new `ffprobe` deployment dependency in this project: one Python dependency covers audio and +video，Python 3.12 wheels are available，and the implementation can consume hydrated bytes directly。It still carries +native FFmpeg parser risk，so resolution inspects metadata only and source/application download limits remain the first +resource boundary。This choice does not make PyAV or FFmpeg part of the persisted resolver contract。 + +No compatible mature EPUB library satisfies the current Python 3.12 and licensing boundary：current `epublib` +requires Python 3.13，while EbookLib adds an AGPL boundary。The narrow ZIP/container/OPF reader is therefore the +smaller long-term dependency，and must not grow into an EPUB object model or chapter extractor。 + +Statistical charset detection is intentionally excluded from the v1 authority path。A protocol/source adapter may +decode bytes before persistence when it owns a trustworthy external charset declaration；otherwise storage-backed +text uses the deterministic minimum above and fails explicitly when it cannot be decoded。This avoids introducing a +generic metadata map or making a heuristic detector cross-peer authority。 + +## Bounded Inspection And Failure Semantics + +- Storage hydration returns actual `bytes` / `ArrayBuffer` and never decoded semantic objects。 +- Source/application download limits bound whole-content residency；resolver parsing adds format-specific work bounds + only where a central directory、page tree or native parser can amplify work。 +- Image inspection does not decode pixels；audio/video inspection does not decode frames；PDF does not extract text； + ZIP/EPUB never extract members。 +- Encrypted PDF/ZIP is valid solved content with encryption facts and nullable inaccessible facts。Malformed claimed + content raises an explicit resolution error；missing local parser/runtime raises unsupported capability；an + unavailable optional fact is null。 +- Embedded raw EXIF、media tags、PDF/XMP maps and archive member lists are not exposed by v1。Only the accepted bounded + typed facts leave the parser boundary。 + +Exact numeric feed、article and enclosure download limits remain source-config fields/defaults in the RSS slice，not +resolver contract versions。Parser helpers may apply generous defense-in-depth caps without introducing streaming or +S3 acceptance into this unit。 + +## Storage Cut-over + +The five current HTTP storage types (`http_image`、`http_video`、`http_html`、`http_json`、`http_text`) incorrectly own +semantic decoding/Accept behavior。The proposed hard cut replaces their in-repo semantic uses with one mechanics-only +`http` storage that returns bytes；resolver ID owns interpretation。PostgreSQL binary keeps its opaque JSON pointer +shape (`{"blob_id":"..."}`) and storage row owns bytes only。 + +The same implementation pass must： + +1. add `BlockModel.get_hydrated_content(refresh=False)` and the corresponding client-web method/cache； +2. add writable-storage update and change `blocks.storage -> storages.id` deletion from `SET NULL` to `RESTRICT`； +3. implement client-web PostgreSQL binary create/read/update/delete through the admitted PostgREST/RPC surface； +4. remove client-web raw-pointer fallbacks from block content、graph preview and editor； +5. make unknown resolver/storage IDs explicit errors in both peers。 + +Storage deletion still does not infer block ownership from its pointer。Memos/RSS application services prove +exclusive graph ownership before deleting a semantic block/blob；the storage handler itself does not query or mutate +blocks。 + +## Propagated Hard-cut Surface + +This is one coherent cut-over，not nine isolated class additions： + +- core-py：replace the four retired resolver implementations and their import-time image AI side effect；make core + registration explicit；update text/embedding callers for unsupported/null； +- client-web：exact registry with no default fallback，abstract instance capabilities，runtime content-handle disposal， + nine IDs and safe unavailable/open/render states； +- Twitter：replace bare image/video/html/text producers，repair its currently untested API/persisted attachment-shape + gap，and add collection → graph → resolver regression coverage； +- RSS/Atom：new namespaced versioned feed/feed-item/enclosure contracts，real enclosure metadata blocks and + idempotent metadata → semantic materialization； +- Memos：attachment metadata remains the Memos protocol identity，but its storage pointer moves to one related + semantic content block；download/delete/list/read assemble through resolvers and relations； +- tactical docs、fixtures and static retired-ID checks change in the same pass。 + +Proposed exact extension resolver IDs are： + +- `extensions.memos.attachment.v2`，because the existing `v1` persisted/graph contract contains `blob_id` and + `storage=-4` on the metadata block； +- `extensions.rss.feed.v1`、`extensions.rss.feed_item.v1` and `extensions.rss.enclosure.v1`，because the current RSS + IDs are unversioned rather than an existing v1 contract。 + +Twitter's resolver version is intentionally not frozen here。Its root can remain relation-oriented without copying +attachment metadata into Tweet content；the RSS implementation plan should not redesign that extension beyond the +hard-cut producer/consumer regression needed for the shared IDs。 + +## Confirmed Memos V1 Data-preservation Branch + +Sir accepts the one-time atomic data migration。A later read-only query through the local Neon CLI credential found +that canonical production is still at Alembic `d9f4e2a1b7c3`，has no `storage_blobs` table and has zero +`extensions.memos.attachment.v1` rows。The current public demo will therefore take the empty migration path，but the +migration still protects another database that has already run the Memos/PostgreSQL-binary implementation。 + +The confirmed migration is： + +1. keep the existing attachment block ID as Memos protocol identity； +2. extract `blob_id` into the storage handler's minimal opaque pointer JSON on a new `core.<kind>.v1` semantic child， + without changing the UUID or copying blob bytes； +3. rewrite the attachment root to inline `extensions.memos.attachment.v2` metadata without `blob_id`； +4. add one `content` relation from metadata to semantic block；existing `attachment:<order>` owner relations remain； +5. do not register the v1 decoder after migration；migration failure rolls the row conversion back。 + +This preserves actual user content while still ending with one current contract and no permanent compatibility +decoder。A clean-database/historical-row-loss branch is no longer part of the implementation plan。 + +The same production snapshot contains retired bare resolver rows：`html=1`、`image=28`、`text=8` and `video=3`。 +D-075's accepted hard cut applies：they receive no compatibility decoder or data migration and become unsupported。 + +## Plan Replay + +| Plan batch | Preflight result | Exposed correction / branch | +| --- | --- | --- | +| B0 dependencies | viable | PyAV is preferred to a new ffprobe deployment；EPUB uses a narrow ZIP/lxml reader because current compatible library candidates fail Python-version、maturity or license return | +| B1 storage/protocol | viable with an explicit generator change | core protocol currently publishes `functions: {}` and client generation hardcodes empty Functions；the plan now names both owners and does not hand-edit generated TypeScript | +| B2 resolver base | viable only as a whole-repository batch | making methods abstract affects every Python/TypeScript extension resolver；the plan updates all concrete classes before treating the batch as green | +| B3 nine resolvers | viable | core registration currently relies on incidental package imports；the plan adds explicit bootstrap outside extension sync and removes the image resolver's import-time AI credential side effect | +| B4 Memos v2 | viable and approved | production has no v1 rows，but another database may；D-076 keeps the reversible one-time migration and no permanent decoder | +| B5 producer cut-over | viable with a Twitter repair | Twitter's API DTO and persisted Tweet schema currently drop attachment fields and lack collection graph coverage；the plan makes this a regression prerequisite rather than blaming the new resolvers | +| B6 RSS primary | viable after plan review | production has no RSS/Atom source instances，but both source type catalog IDs exist；thin wrappers preserve those durable IDs while sharing the rewrite，avoiding an unnecessary source-row migration | +| B7 enrichment/materialization | viable | random storage pointers defeat content-based fetchsert；the plan uses enclosure identity + existing `content` relation + row re-check for idempotency | +| B8 acceptance | viable | current white-box RSS tests are replaced only after real HTTP→job→PostgreSQL→resolver scenarios exist；client-web already has a PostgREST E2E seam but no resolver/storage unit baseline | + +The replay found no reason to split a separate foundation unit，add S3/streaming acceptance，retain old resolvers or +delegate client-web capability to core-py。It did show that the earlier document's seven-line “execution order” was not +an implementation plan；[implementation-plan.md](implementation-plan.md) now owns the real sequence。 + +## Verification Baseline + +- Current read-only baseline：PDM lock is current；legacy resolver/storage targeted tests pass（3）；Memos attachment + unit/backend tests pass（19）；client-web core and app type-check pass。 +- New behavior is accepted through real-format samples and black-box/runtime journeys，not schema/helper tests that + merely repeat static types。 +- Required scenarios include PostgreSQL byte-exact CRUD/hydration，inline and storage-backed text/HTML，real image/ + audio/video/PDF/EPUB/ZIP samples，malformed/protected samples，Memos upload/read/download/delete，RSS and Atom + collect/update/retry，enclosure manual/automatic materialization and missing local capability。 +- A final repository-wide static assertion proves no current producer/documentation example emits retired resolver or + content-specific HTTP storage IDs；historical task evidence may name them only as explicitly retired facts。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/library-evidence.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/library-evidence.md new file mode 100644 index 0000000..93c1cce --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/library-evidence.md @@ -0,0 +1,75 @@ +# RSS Extension Library Evidence + +> Research snapshot for `rss-extension-hardening` Product/Technical discovery。It records evidence and +> recommendations，not durable dependency authority；the frozen implementation plan and lockfile own the +> selected exact versions after approval。 + +## Feed parser + +### Recommended：Universal Feed Parser (`feedparser`) + +- official repository：[kurtmckee/feedparser](https://github.com/kurtmckee/feedparser) +- selected release：`6.0.14`，published 2026-07-30；the earlier research snapshot saw `6.0.12` before the newer + release reached the package index +- official package metadata：[feedparser on PyPI](https://pypi.org/project/feedparser/) +- official capability docs：[Advanced Features](https://feedparser.readthedocs.io/en/latest/advanced/) +- relevant evidence： + - RSS and Atom normalization across multiple generations； + - namespace handling and non-standard prefixes； + - character encoding and date parsing； + - HTML sanitization and content type/detail fields； + - relative-link resolution； + - enclosure/link/category/source projections； + - feed-version detection and malformed-feed `bozo` signal。 + +`feedparser` should receive response bytes from InKCre's HTTP boundary。Its normalized mapping is adapter +input，not CanonicalFeedItem authority；known ambiguity such as RSS content-type guessing must remain visible +in product/adapter decisions rather than being treated as truth merely because the library produced it。 + +### Not selected as owner：`reader` + +- official docs：[reader](https://reader.readthedocs.io/en/stable/) +- evidence：it deliberately provides a “fat model” with feed/entry storage、read/important state、tags、search、 + update scheduling、OPML and plugins。 + +Those capabilities are mature，but adopting the whole model would create competing feed persistence、source +state and application/search authority。It may be behavior research，not an InKCre runtime/domain dependency。 + +### Not selected：`atoma` + +- latest researched PyPI release is `0.0.16` from 2018。 +- It offers typed RSS/Atom parsing，but its maintenance/compatibility evidence is materially weaker than + `feedparser` for this unit's tolerant real-world feed goal。 + +## Full-text extraction + +### Recommended if admitted：Trafilatura + +- official repository：[adbar/trafilatura](https://github.com/adbar/trafilatura) +- official API：[Core functions](https://trafilatura.readthedocs.io/en/latest/corefunctions.html) +- selected release：`2.2.0`，published 2026-07-31；the earlier research snapshot saw documentation/release `2.1.0` +- official package metadata：[Trafilatura on PyPI](https://pypi.org/project/trafilatura/) +- relevant evidence：main-text extraction、precision/recall modes、plain-text/Markdown/HTML/JSON output and + optional metadata extraction。 + +InKCre should provide already-downloaded HTML plus effective URL and consume only extraction output。Do not +delegate fetching、retry、identity、storage or graph ownership to Trafilatura。 + +## Boundary Summary + +```text +InKCre HTTP client + -> bytes + effective URL + headers + -> feedparser + -> normalized third-party parse result + -> InKCre RSS/Atom adapter + -> CanonicalFeed / CanonicalFeedItem commands + -> InKCre graph + resolver + source state + +optional item-link HTTP + -> Trafilatura extraction + -> independent full-text enrichment component +``` + +The libraries remove protocol/parser reinvention。They do not decide source identity、canonical facts、graph +shape、reconciliation、job success、state advance、partial effects or use-facing authority。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/media-storage-evidence.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/media-storage-evidence.md new file mode 100644 index 0000000..1654e64 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/media-storage-evidence.md @@ -0,0 +1,238 @@ +# Media Resolver And Storage Evidence + +## Purpose + +记录 RSS enclosure vertical 暴露的横向 current-state evidence。这里不是独立 unit，也不是 durable +owner；confirmed cross-unit contracts 仍由 program decisions 与 later Product TDD promotion 拥有。 + +## Resolver Evidence + +- Core `Resolver` 要求 `get_text()` 与 `get_str_for_embedding()`；runtime introspection 证明现有 + `TextResolver`、`VideoResolver` 仍是 abstract class，不能形成完整 resolver path。 +- `ImageResolver` 把 Tencent LKE client/credential construction 放在 module import，且只接受固定 + `http_image` storage ID；stored PostgreSQL/S3 bytes 无法走同一路径。 +- `ImageResolver.get_text()` 会执行 remote AI workflow 并直接写 derived blocks/relations；read/resolve 与 + enrichment/organization side effects 没有清晰边界。 +- `VideoResolver` 只提供 URL graph factory；没有 solved/text/embedding contract。Audio、PDF、EPUB、ZIP、 + generic file resolver 不存在。 +- `HTMLResolver` 把固定 HTTP storage、HTML fetch、text conversion 与已有 text relation fallback 混在一起； + 它不是 RSS full-text extraction 的可信 reference implementation。 +- Existing resolver IDs `image` / `video` / `html` / `text` 未 version/namespace；Twitter bookmark source + 直接依赖 image/video/html factories，因此 common media rewrite 需要 Twitter regression proof。 + +## Storage Evidence + +- `HTTPStorage._fetch_url()` 从 `ClientSession` / response context 返回 response object，调用方在 context + 已退出后读取 body；response lifetime ownership 不成立。 +- HTTP variants 重复 transport code 并混入 media `Accept` preference；`HTTPJsonStorage` 对任意 decoded + JSON 调用 `.strip()`，image docstring 声称 base64 但返回 bytes，HTML docstring/return type 也不一致。 +- HTTP storage 没有 response-size、streaming、content-type/provenance 或 partial-download contract。 +- `WritableStorage.write_raw_content()` 接受完整 `bytes`、是 synchronous、并强制 caller DB session；这能 + 支撑 PostgreSQL small blobs，但不能证明 S3/multipart/large enclosure lifecycle。 +- Writable storage 返回任意 pointer value，block caller 自行把 pointer 编入 resolver content；没有 + stable pointer envelope/capability contract。 +- `blocks.storage` foreign key 当前 `ON DELETE SET NULL`。删除 storage 会让原 pointer string 被 resolver + 当成 inline content，保存 block row 却破坏 raw-content interpretation。 +- Storage type/instance tables 与 config schema 已存在，但 core 没有 storage create/update route；若加入 + S3-compatible storage，需要明确 provisioning/config authority。 + +## Memos Attachment Evidence + +- `extensions.memos.attachment.v1` block 同时承担 Memos attachment identity、filename/media-type/size/ + created time 与 PostgreSQL `blob_id` pointer。 +- Repository hard-codes built-in storage ID `-4` for create/read/delete；canonical schema 直接依赖 UUID blob + pointer，不能换 target storage。 +- Memo graph、owned deletion、list/download/delete 和 v0.29.1 adapter 都要求 exact attachment resolver。 + 直接把 resolver 字符串改成 `image` 会破坏 unattached uploads 与 Memos attachment listing。 +- Memos protocol permits upload before memo ownership；therefore attachment role/identity must remain + recoverable even when no `memo --attachment:<order>--> ...` relation exists。 + +## Confirmed Product Pressure + +- Materialized information uses exact semantic content resolver IDs：image、audio、video、PDF、EPUB、ZIP； + unknown/unsupported falls back to file with best-effort MIME type。 +- RSS enclosure remains source-authored authority；download creates a related local materialization。 +- Memos attachment behavior must migrate to the same semantic media/storage path within the RSS unit loop。 +- Actual-content bytes remain storage-owned；storage representation never determines block information kind。 + +## Open Design Branches + +1. Parser/runtime dependency，charset-authority and bounded-inspection preflight for D-075's nine exact IDs。 +2. Resolver-version/hard-cut-off consequences for existing `extensions.memos.attachment.v1`；the accepted common + bare-ID cut-off retains neither compatibility decoders nor a migration。 + +## Closed Design Branch + +- D-075 fixes nine exact `core.<kind>.v1` resolver IDs，resolver-instance capability calls，mandatory abstract + text/embedding methods with explicit unsupported errors，and hard cut-off of bare `text/html/image/video`。 +- D-059 keeps a MemosAttachment metadata block and relates it to media/file semantic content；direct media identity was + rejected because unattached protocol resources，list/delete/download identity and ordered ownership would + otherwise require hidden provenance/index conventions。 +- D-060 keeps one persisted `content` column：inline blocks store actual content there，storage-backed blocks + store an opaque pointer。`BlockModel.get_hydrated_content()` hides that conditional read and caches actual + content in non-mapped `_hydrated_content` without replacing the persisted pointer；storage deletion is + RESTRICT while referenced。 +- D-067 closes media metadata authority without adding `blocks.metadata`；protocol/source facts stay on the metadata block，storage + mechanics stay in pointer/config，semantic and byte-derived facts stay with resolver projection，and useful + durable derivations may become organization graph enrichment。 +- D-068 closes the current S3/streaming scope branch：S3-compatible storage is sequenced with the future Nextcloud + Files extension；RSS uses PostgreSQL writable storage with explicit bounds and does not add speculative streaming + abstraction or very-large-file acceptance。 +- D-069 closes the global-ladder branch：extensions own classification policy，while core may offer reusable + `ResolverManager` mechanisms without adding a media module。Memos type，RSS enclosure type and Atom link type keep + their protocol-specific semantics。 +- D-070 closes Memos classification：required normalized `Attachment.type` selects the exact resolver ID， + unknown MIME falls back to file，and byte sniffing neither gates upload nor replaces the metadata-block declaration。 +- D-071 closes RSS classification：valid，specific `enclosure.type` is primary；fallback evidence participates only + when the declaration is unusable and never rewrites the metadata-block field。 +- D-072 closes Atom classification：specific dereferenced HTTP type precedes advisory link type，then adapter-owned + fallback；no observed/detected result rewrites the metadata-block declaration。 +- D-073 corrects the earlier pure-read inference：resolver is the application-facing graph interpretation boundary， + may lazily materialize missing derived graph through AI/organization，and exposes optional text/embedding + projections。The missing contract is caller control plus effect correctness，not a universal side-effect ban。 +- D-074 makes missing materialization the ordinary resolver default with an explicit read-only override。`refresh` + only bypasses/replaces a reusable local snapshot；`materialize_missing`，organization `recompute` and cache + `invalidate` retain orthogonal effects。Existing client-web resolver `force` cache options become migration + pressure，not durable vocabulary。 + +## Client-web Cross-runtime Evidence + +`client-web/packages/core` independently implements the same domain seam rather than merely consuming generated +database types： + +- `src/info-base/block.ts` models the conditional `storage` / string `content` row but has no content-read + method or hydrated cache。 +- `src/info-base/resolvers/base.ts` owns `_rawContent`，branches on `block.storage` and dynamically instantiates a + browser `Storage` handler。This duplicates the responsibility D-060 moved onto block and preserves the retired + raw/real terminology。 +- `src/info-base/storages/base.ts` and `storages/http.ts` expose `getRawContent(block)` and interpret + `block.content` as the pointer。The HTTP handlers are browser implementations，not a portable contract for + PostgreSQL binary or future S3-compatible server-owned storage。 +- The canonical topology makes core-py and client-web equal peers over the admitted database protocol；core-py + schema/migration ownership does not make its REST API the content data plane for other peers。No existing + core-py storage-backed-content API contract was found。 +- The generated client-web database types currently include `storage_types` and `storages` but omit + `storage_blobs`，while the current core-py executable peer contract admits `storage_blobs`。That is a concrete + contract-sync/handler prerequisite if client-web must support PostgreSQL binary hydration，not a reason to + proxy through core-py。 +- Resolver-level caching is timestamp-invalidated through `ResolverCache`，but actual-content cache is per + resolver instance。Multiple resolvers for one `Block` can therefore repeat hydration；moving the cache to + `Block._hydratedContent` aligns the two runtimes and leaves `ResolverCache` responsible only for resolver + interpretation/relations。 +- `Resolver.getRelations()`、`getRawContent()` and `getSolvedContent()` currently name cache bypass `force`，while + `ResolverCache.invalidate()` only deletes an entry。This is concrete evidence for distinct `refresh` versus + `invalidate` vocabulary；it is not evidence for preserving an unqualified `force` boolean。 +- Current graph preview and fallback UI render `block.content` directly，so storage-backed blocks visibly expose + pointers；the generic block editor can also present a pointer as editable text。These are downstream use/UI + consequences，not reasons to change the persisted contract。 + +### Cross-runtime implementation pressure + +- Add `Block.getHydratedContent()` and an unloaded-`Symbol`-backed ECMAScript private + `#hydratedContent` cache in `@inkcre/core`；invalidate it on controlled `content`/`storage` updates。A normal + TypeScript `private _hydratedContent` property is insufficient because it remains enumerable at runtime and + `Block.update()` serializes the instance for PostgREST writes。 +- Make browser resolvers consume block hydration and remove their `_rawContent` cache/storage branching。 +- Preserve the generated `blocks.content: string` shape；the core-py FK action change does not alter TypeScript + generated row fields。 +- Keep hydration local to the peer：`Block.getHydratedContent()` selects a locally registered storage handler。 + A missing handler is an explicit unsupported-capability failure。If the product later needs one peer to execute + hydration on behalf of another，design generic capability discovery and explicit peer delegation；do not + privilege core-py or hide network forwarding inside the ordinary block-read contract。 +- Implement browser handlers only for storage types required by this unit's accepted client-web behavior。 + PostgreSQL binary can use the admitted database protocol once generated types/coverage are aligned；future S3 + requires its own browser/runtime feasibility and deployment contract rather than automatic server fallback。 +- Treat graph preview，fallback rendering and storage-backed edit affordances as acceptance surfaces when the + client-web slice executes；do not eagerly hydrate every graph node merely to avoid displaying a pointer。 + +### Corrected inference + +An earlier probe proposed a core-py block-content endpoint primarily to avoid storage-capability divergence。 +That rationale is rejected：it imported a conventional client/server hierarchy that the peer topology explicitly +denies。Peer equality concerns authority and admitted protocol；runtime capability availability may differ and +must be represented honestly。 + +## PostgreSQL Binary / PostgREST Evidence + +- `storage_blobs` is not a storage type。`storage_types.id = postgresql_binary` names the implementation family； + `storages.id = -4` is its configured built-in instance；`storage_blobs(id, data)` is that implementation's + current backing object relation；a block selects instance `-4` and carries the blob UUID in its opaque pointer。 +- The current core contract exposes `inkcre.storage_blobs(id uuid, data bytea)` and describes `data` as protocol + `string/bytea`。The client-web contract pin predates that migration，so `contract:sync --local-core ...` would + add the relation to generated TypeScript rather than hand-editing `generated.ts`。 +- The deployed/test PostgREST artifact is pinned to `v14.15` and exposes the `inkcre` schema to authenticated + peers。No custom raw-media configuration is currently present。 +- PostgREST 14 officially accepts `application/octet-stream` request bodies only through an RPC function with one + unnamed `bytea` parameter。This gives create a natural raw path，but update also needs the target `blob_id`；an + all-raw update would therefore require a custom request header，a binary envelope or another InKCre-specific wire + convention。 +- PostgREST 14 media-type handlers provide a native raw read path：an RPC function can take a normal UUID query + parameter and return an `application/octet-stream` domain over `bytea`。The browser can then consume the response + as `ArrayBuffer` without passing it through the normal JSON decoder。 +- Older official PostgREST documentation explicitly supports selecting one `bytea` column with + `Accept: application/octet-stream`；current v14 documentation instead emphasizes database media handlers。 + Therefore direct table raw-binary response is a candidate to prove against the pinned v14.15 image，not a + contract to assume from old documentation。 +- A stable v14-native alternative is an admitted PostgreSQL function/media handler that takes a blob UUID and + returns an `application/octet-stream` domain over `bytea`。That remains PostgREST peer transport，not a core-py + service endpoint，but it would require migration/ACL/contract-function generation work。 +- JSON row selection would expose PostgreSQL's textual `bytea` representation and require browser decoding with + roughly two hex characters per byte。It is the smallest schema change but a poor default for media-sized data。 +- Direct relation C/U/D remains technically available because authenticated peers already have complete protocol + table privileges。Create/update would send PostgreSQL's accepted `\\x...` hex `bytea` string through JSON，while + delete can filter by UUID；this avoids a custom binary-update protocol at the cost of roughly 2× upload + representation size。A hybrid of direct relation C/U/D plus a media-handler RPC for raw read is therefore the + smallest coherent candidate，not yet a confirmed decision。 +- `@supabase/postgrest-js` 2.110.8 allows setting request headers but its response path reads successful bodies as + text and JSON-decodes them except for a few named formats。A binary handler therefore needs a small authenticated + raw PostgREST transport in `packages/core` rather than pretending the normal typed row decoder can return bytes。 +- Read hydration can return browser bytes (`ArrayBuffer`/`Uint8Array`) without storage owning MIME。MIME remains + a semantic-media metadata decision。 +- Sir has confirmed complete client-web PostgreSQL bytes lifecycle ownership。Raw upload via PostgREST generally + requires an admitted bytea RPC with `Content-Type: application/octet-stream`，and deletion requires a narrowly + admitted backing-object operation；their exact v14.15 shapes still require black-box proof。 +- The current `WritableStorage` contract has create/read/delete but no update。Complete CRUD therefore requires a + real pointer-scoped update operation in core-py and client-web；for PostgreSQL，updating `storage_blobs.data` under + the same UUID is the natural implementation。 +- `blocks.updated_at` only records mutation of the block row。A storage-backed object may be external and mutable， + so neither this timestamp nor an instance-local hydrated-content cache can claim generic content freshness。 + Storage remains independent of block；refresh/reconciliation must be expressed above the CRUD contract。 +- D-065 closes the local cache branch：ordinary hydration reuses the block instance snapshot，while an explicit + refresh option bypasses and replaces it。No TTL，polling or cross-peer invalidation is inferred。 +- D-066 closes the browser wire branch：Create and Read use admitted raw octet-stream RPC/media handlers；Update + PATCHes hex bytea on the exact relation row；Delete uses exact relation DELETE。No custom update header or binary + envelope is added。 + +## Opaque Bytes / Semantic Resolver Consequence + +- Storage-backed actual content is logically bytes regardless of whether the physical mechanism is PostgreSQL + `bytea`，S3，HTTP or another store。Streaming is an execution representation of those bytes，not a new semantic + content kind。 +- Resolver owns decoding/parsing and use-facing representation。Image/video/audio/PDF/EPUB/ZIP/file are resolver + exact resolver IDs or graph information kinds，not storage types。 +- Current `BlockModel` has no generic metadata column；its persisted information surface is resolver，conditional + content/pointer，storage reference and record timestamps。The absence of a metadata field does not by itself prove + that one should be added。 +- Current Memos `CanonicalAttachment` combines filename，declared media type，size，source-created time and + `blob_id` inside the same storage-backed block content。That conflicts with D-059/D-060：the protocol attachment + attachment metadata block should retain protocol/source facts，while a related media/file semantic content block owns only its + storage pointer as persisted `content`。 +- Current image/video/html resolvers choose `http_image` / `http_video` / `http_html` storage instances and branch + on those storage IDs。That makes storage selection carry semantic information-kind meaning and is direct rewrite + evidence under D-062。 +- Candidate authority split for review：source-declared filename/MIME/length/URL/timestamps stay in the canonical + metadata block；storage retrieval/version mechanics stay private to its pointer/config；exact resolver ID plus + hydrated bytes owns detected media kind and byte-derived solved content；organization may persist useful derived + facts as graph enrichment。This candidate would not add `blocks.metadata` in the MVP。 +- Memos v0.29.1 `Attachment.type` is the protocol MIME field。The upstream server accepts a provided normalized + value and only falls back to filename extension then content detection when it is empty；the current MoeMemos MVP + upload contract requires it。This is a stable Memos declaration，not a universal byte-truth rule。 +- RSS 2.0 requires enclosure `url`，`length` and MIME `type`。Atom permits link `type` as an advisory MIME hint and + says the media type returned on dereference is authoritative。The current RSS schema's URL-only enclosure tuple + loses these protocol distinctions and cannot survive the rewrite。 +- Current Python and TypeScript HTTP storage families both mix transport with content parsing/shape：JSON storage + parses JSON，HTML storage sanitizes/extracts title，video storage returns a URL/MIME projection instead of bytes， + and type IDs are content-kind-specific。They are rewrite evidence，not the target abstraction。 +- The generic target is a small number of mechanics-named storages plus semantic resolvers。Exact HTTP storage + split（for example one generic byte-fetcher versus policy-specific transports），PostgreSQL naming and legacy + compatibility remain open until implementation preflight。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/packet.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/packet.md new file mode 100644 index 0000000..e160a27 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/packet.md @@ -0,0 +1,101 @@ +# RSS Extension Hardening + +- **Unit ID**: `rss-extension-hardening`。 +- **State**: **Complete**；Sir 于 2026-08-03 接受最终验收方案，B0–B8 implementation、verification、 + durable reconciliation 与 owner-specific commits 均已完成。 +- **Objective**: 让一个配置好的 RSS 2.0 / Atom source 能可靠地把 feed-native information 收集为 + resolver-readable graph，并建立可信的 + `source instance → collect job → graph → resolver → source state` reference contract。 +- **Guardrails**: feed-authored information 保持 authority；full text 与 downloaded enclosure 是独立 + enrichment/materialization；不把 feed reader、organization、retrieval、所有协议 edge cases、S3 或 + generic source framework 偷渡进本 unit。 +- **Verification**: 以真实 HTTP transport double + migrated PostgreSQL 驱动 source/job/graph/resolver/state； + 横向以真实格式 bytes、migration、PostgREST probe、core-py/client-web 全仓 static/runtime checks 证明。 +- **Current Truth**: RSS/Atom behavior rewrite、shared hydration/storage/resolver contracts、Memos attachment v2、 + producer cut-over 与 durable owner projection 已落地。core-py `835f89a`、client-web `765b22f`、Hub docs + `48b069f` 已提交；未 push、未 bump Spoke shared refs、未执行 production migration。 +- **Next Step**: none for this unit。Program 重新选择下一个 implementable unit；future RSS hardening 只有在新 + product pressure 或已接受的 non-blocking gap 变成实际问题时才重新进入 gate。 + +## Completion Outcome + +- RSS 与 Atom 保留既有 source type identity，内部共享 bounded HTTP/feedparser collection service。 +- manual/scheduled path 产生普通 collect job；pending claim atomic，job diagnostics/status 与 source-state + advance 对齐。 +- exact feed/item identity、same-ID update、idempotent replay、conditional HTTP、unidentified + create/discard/watermark、missing-old-item retention 与 retry residue 已形成明确行为。 +- feed、item、enclosure 是 versioned canonical blocks；components/associations 只通过 relations 表达。 +- feed-authored content 保持 authority；默认 full-text extraction 形成独立 `full_text` block，失败不改变 + primary collection contract。 +- manual/automatic enclosure materialization 通过 resolver-instance command 与 configured writable storage + 形成唯一 semantic content child；image/audio/video/PDF/EPUB/ZIP/file 使用 exact `core.<kind>.v1` resolver。 +- common horizontal result 包括 block-owned hydration、generic HTTP bytes、PostgreSQL bytes CRUD、九种 exact + semantic resolvers、Memos attachment v2 与 client-web peer-local parity。 + +完整决定由 [program decision authority](../../decisions/index.md) 的 D-049–D-078 拥有；本 packet 不再复制逐条 +decision text。稳定技术合同已投影到 [RSS Unit TDD](../../../../docs/30-unit-tdd/rss-extension.md) 与 Hub +PRD/Product TDD。 + +## Accepted Verification Horizon + +### Primary product authority + +[PostgreSQL integration suite](../../../../tests/extensions/rss/integration/test_feed_collection.py) 使用真实 +loopback HTTP server 提供 RSS 2.0 / Atom bytes，并执行真实 parser、source instance、collect job、committed +PostgreSQL graph、storage hydration、resolver solved value 与 source state。它覆盖： + +- atomic claim、success/failure status、structured diagnostics 与 legacy job-config rejection； +- first collect、304、same-ID update、new item、unchanged replay、missing old item 与 feed identity/config change； +- unidentified item create/discard/watermark； +- per-item partial persistence failure、state non-advance 与 retry convergence； +- separate full-text enrichment； +- manual/automatic enclosure materialization、real semantic bytes、MIME evidence precedence、unknown file fallback + 与 concurrent idempotency。 + +### Cross-cutting and regression authority + +- Generated-on-demand real PNG/WAV/MP4/PDF/EPUB/ZIP/text/HTML/file samples prove Python semantic resolvers；derived + outputs remain Git-ignored。 +- A disposable migrated runtime passed authenticated PostgREST byte-exact create/read/same-pointer update/read/delete。 +- Seeded Memos attachment v1 → v2 → downgrade and real PostgreSQL Memos graph journeys passed。 +- core-py：Pyrefly zero diagnostics；293 passed / 19 environment skips；migration suite 22 passed / 2 skipped； + real PostgreSQL Memos+RSS run 15 passed。 +- client-web：complete `pnpm check` passed 56 tests、workspace type checks and production builds。 +- repository lint、implementation-owned formatting、retired-ID scans、Hub SVC/doc-link validation passed；four + unrelated pre-existing Markdown format drifts remain outside this unit。 + +### Accepted non-blocking limits + +Sir 于 2026-08-03 明确接受以下 verification horizon，不把它们作为 unit-close blockers： + +- 主验收是 business-runtime vertical integration，不是启动完整 deployment 后从外部 source API 配置并运行的 + process-level black box；source setup 与 job execution 部分直接使用 database/manager boundary。 +- [live protocol smoke](../../../../tests/extensions/rss/integration/test_live_feed_protocols.py) 是显式 URL 驱动的 + optional fetch/parse check；最终验收未选择公网 endpoint，因此它被 skip，也不声称证明完整 collection graph。 +- transient HTTP timeout、malformed whole feed、enrichment/storage/resolver failure、process interruption 与 + scheduler exact-one-job 的额外 failure probes 可作为未来 hardening evidence，但当前核心 MVP 后果已经具有 + 足够可信度。 + +## Test Infrastructure Extraction Review + +本 unit 没有再新增 generic test harness： + +- 已有可复用基础设施是 `tests/conftest.py` 的 hermetic environment 与 on-demand semantic asset fixture，以及 + `tests/assets/semantic-content/` 的 source case table/generator。 +- RSS HTTP routes、protocol revisions、feed identity、job/state assertions 与 graph cleanup 都携带 source-specific + semantics；把它们抽成通用 source framework 会降低测试可读性并提前约束 CalDAV/Nextcloud 等不同协议。 +- `INKCRE_TEST_DATABASE_URL` gate 的少量重复不足以支撑新的 abstraction。 +- 当第二个 external-source unit 重复需要 loopback protocol-server lifecycle、ordinary collect-job journey 或 + graph cleanup contract 时，再以两个真实 pressure 提取最小 helper；RSS suite 将作为第一份 reference consumer。 + +## Supporting Evidence + +- [Implementation plan](implementation-plan.md)：B0–B8 addresses、dependency order 与 execution evidence。 +- [Implementation preflight](implementation-preflight.md)：library/runtime/migration/branch replay。 +- [Library evidence](library-evidence.md)：parser/extractor selection。 +- [Media/storage evidence](media-storage-evidence.md)：横向 storage/resolver/Memos pressure。 +- [Semantic-content resolver contracts](semantic-content-resolver-contracts.md)：exact resolver IDs 与 capability + semantics。 + +这些 supporting files 保存 expensive-to-recover evidence，不维护独立 active state。Commit、push、Hub publication、 +shared-ref bump 与 production mutation 是 owner-specific delivery operations，不改变本 unit 的完成状态。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/semantic-content-resolver-contracts.md b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/semantic-content-resolver-contracts.md new file mode 100644 index 0000000..2348378 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/rss-extension-hardening/semantic-content-resolver-contracts.md @@ -0,0 +1,203 @@ +# Semantic Content Resolver Contracts And Solved Content + +## Control + +- **Status**: confirmed by D-075；this supporting design records the accepted boundary but is not a durable owner。 +- **Pressure**: D-057–D-075 已固定 content kinds、metadata block/semantic content block/storage/resolver + authority、per-extension + classification、optional text capability 与 resolver effect vocabulary；本设计关闭 exact resolver IDs、 + resolver contract version 与 minimum solved/use-facing shape。 +- **Scope**: shared persisted resolver identities and solved-content contracts across peers。Python/TypeScript + class hierarchy、rendering component、parser dependency and transport implementation remain peer-local。 + +## Recommendation + +### Exact resolver IDs and version axis + +New text/HTML/image/audio/video/document/file semantic content blocks use nine flat，versioned core resolver IDs： + +| Information kind | Resolver ID | +| --- | --- | +| Plain text | `core.text.v1` | +| HTML document | `core.html.v1` | +| Image | `core.image.v1` | +| Audio | `core.audio.v1` | +| Video | `core.video.v1` | +| PDF document | `core.pdf.v1` | +| EPUB publication | `core.epub.v1` | +| ZIP archive | `core.zip.v1` | +| Unknown/unsupported file | `core.file.v1` | + +`core` means InKCre-owned shared semantics，not “execute in core-py” and not a client/server hierarchy。The IDs stay +flat because `media`、`document` and `archive` are useful conceptual groupings but do not currently own shared +persisted union schemas。A second format may later prove a reusable runtime helper；it must not retroactively turn +these exact decoder identities into a god object。 + +Resolver contract version tracks a persisted decoder/solved/graph contract change，not a file-format minor version，parser +package release or newly populated nullable fact。For example，`core.epub.v1` may interpret supported EPUB 3.x files +and expose their actual EPUB version；only an incompatible InKCre contract creates `v2`。 + +### Persistence and authority + +The semantic content block persists only the already-confirmed block shape： + +- `resolver` is one exact resolver ID above； +- inline `content` is actual content，or storage-backed `content` is the storage's opaque pointer； +- `storage` selects access mechanics； +- filename、protocol-declared MIME/length/URL/timestamps remain on the related metadata block； +- byte-derived facts remain resolver projections unless organization has a proven reason to materialize them。 + +The metadata block and semantic content block remain connected by the accepted `content` relation。A metadata block +is an ordinary block whose canonical content owns protocol/source-authored facts about related semantic content；it is not a wrapper +runtime type and is unrelated to a source module abstraction。Neither block duplicates the other's authority merely +to make a standalone DTO convenient。 + +## Solved Content Model + +### One resolver contract，peer-local runtime representation + +A resolver exposes typed，resolver-specific solved content。The stable cross-peer part is the meaning and +nullable facts。`core.text.v1` solves to a Unicode string；`core.html.v1` solves to an HTML source string。For +byte-oriented content，the actual-content handle is peer-local：Python may retain bytes/memoryview，while a browser may create +a `Blob`/object URL and own its disposal。Object URLs，Vue components and Python Pydantic class names are not shared +protocol fields and are never persisted。 + +The seven byte-oriented solved-content shapes（image/audio/video/PDF/EPUB/ZIP/file）expose these base facts： + +- `byte_size: int`； +- `detected_media_type: str | None`，which never overwrites a metadata block's declared media type。 + +They do **not** include a generic metadata map，filename，source declaration，storage pointer，checksum，parse-status +object or duplicated `kind` field。Kind is already exact in the resolver ID；invalid content raises an explicit +resolution error。Checksum remains a future opt-in projection until a real identity/invalidation/use requirement +justifies its full-byte cost。 + +### Minimum resolver-specific facts + +All parser-derived fields are nullable unless the format contract and a successful bounded parse prove them。 +Unknown is not encoded as zero，empty string or `False`。 + +| Resolver ID | Minimum solved content/facts | Explicitly outside the minimum | +| --- | --- | --- | +| `core.text.v1` | Unicode text string | formatting/layout model，pretend byte size for inline text | +| `core.html.v1` | decoded HTML source string | persisted DOM，render component，sanitized-output authority，fetched source URL | +| `core.image.v1` | `format`，`width`，`height`，optional `frame_count` | raw EXIF map，GPS/capture-time exposure，OCR/vision caption | +| `core.audio.v1` | `container`，`codec`，`duration_ms`，`channels`，`sample_rate_hz`，optional `bitrate_bps` | lyrics，transcript，cover extraction，unbounded tags | +| `core.video.v1` | `container`，`video_codec`，`duration_ms`，`width`，`height`，optional `frame_rate` | full stream manifest，subtitle/chapter graph，transcript，poster generation | +| `core.pdf.v1` | `pdf_version`，`page_count`，`is_encrypted`，optional typed `title` / `author` | raw PDF/XMP map，full text，OCR，page graph | +| `core.epub.v1` | `epub_version`，`title`，`creators`，`languages`，`modified_at`，`manifest_count`，`spine_count`，`has_navigation` | raw package map，resource graph，chapter text/cover extraction | +| `core.zip.v1` | `member_count`，`total_compressed_bytes`，`total_uncompressed_bytes`，`compression_methods`，`encrypted_member_count` | extraction，unbounded member list，filesystem paths，member child graph | +| `core.file.v1` | no additional required fact | guessed filename，pretend text，format-specific optional fields | + +This is deliberately not a universal media DTO。A shared runtime helper may carry the two base facts，but each persisted +resolver contract retains its own exact typed solved content and evolves independently。 + +## Application Capabilities + +Solved content is not synonymous with text： + +1. **Open/render**: every successfully hydrated block can expose a peer-local actual-content handle。A peer may render + image/audio/video/PDF or only offer open/download for EPUB/ZIP/file；missing local capability is explicit。 +2. **Text**: optional as an outcome，but explicit in the class contract。`ResolverBase.get_text()` remains abstract。 + Every concrete resolver must implement it；a resolver without text capability raises + `UnsupportedResolverCapability`，while a capable resolver may return `None` when this particular block has no + meaningful text。It never returns fake `""` merely to satisfy a base class。Protocol-authored alt/caption/title facts are preferred through graph + relations。OCR，vision caption and speech transcription are derived capabilities，not root solved facts。 +3. **Embedding text**: optional and distinct from solved content。The sink skips blocks with no embedding projection； + it must not treat an empty string as successful indexing。A typed feature/index may consume dimensions，duration， + page count or other solved facts without converting them to prose。 +4. **Derived graph**: a resolver may support materializing transcript，caption，page/chapter/member or other graph。 + D-074 applies only when that capability exists：ordinary resolution may create a missing derivation， + `materialize_missing=False` is read-only，and `refresh` alone never requests regeneration。 + +Capability is requested directly on a resolver instance，not through `ResolverManager` and not through a persisted +capability/status field： + +- `ResolverBase` declares `get_text()` and `get_str_for_embedding()` as abstract；this makes every concrete + resolver state its behavior explicitly and lets type/static checks enforce the method surface； +- an unsupported implementation raises `UnsupportedResolverCapability`（the Python error may derive from + `NotImplementedError`）； +- a supported implementation may return `None` / `null` when this particular block has no meaningful value； +- `ResolverManager` only selects/constructs a resolver by exact resolver ID and may offer shared registration or MIME + matching mechanisms；it does not own instance capability dispatch； +- `""` remains a valid authored empty string only where the owning content contract permits it；it is never a + substitute for unsupported or absent output。 + +The minimum RSS/Memos vertical therefore does not have to implement OCR/STT or explode PDF/EPUB/ZIP into child blocks +to truthfully support these resolver contracts。It must implement bounded root inspection，actual-content use and honest +absence of unsupported projections。 + +## Why The Nine Resolver IDs Stay Separate + +- Plain text and HTML remain distinct because HTML owns markup/document semantics and a derived text projection； + silently treating HTML source as plain text would leak markup into use/embedding。 +- Image/audio/video have different parsers，rendering behavior and typed facts even though all are commonly called + media。 +- PDF and EPUB are both documents but have incompatible document structures：PDF is page-oriented and may be scanned + or encrypted；EPUB is a publication package with manifest，navigation and an ordered spine。 +- EPUB is physically a specialized ZIP container，but its member order is not its reading order；the EPUB spine is。 +- Generic ZIP owns archive/member/compression semantics，not document semantics。ZIP permits duplicate member names， + so any future member materialization must use a central-directory entry identity/ordinal，not path alone。 +- `core.file.v1` is the honest fallback for bytes whose stronger semantic decoder is unknown or unsupported。It is not + a persistent parent class of every specialized resolver。 + +Primary evidence： + +- [W3C EPUB 3.3](https://www.w3.org/TR/epub-33/) — current Recommendation；package metadata，manifest，navigation and spine/default reading + order； +- [Python 3.12 zipfile](https://docs.python.org/3.12/library/zipfile.html) — ordered central-directory entries， + duplicate names and decompression/resource hazards； +- [Python 3.12 mimetypes](https://docs.python.org/3.12/library/mimetypes.html) — suffix mapping is filename/URL evidence， + not byte detection； +- [Pillow concepts](https://pillow.readthedocs.io/en/stable/handbook/concepts.html) — image size/mode and + format-dependent auxiliary metadata； +- [Mutagen MP3 info](https://mutagen.readthedocs.io/en/latest/api/mp3.html) — duration/channels/bitrate/sample rate； +- [pypdf text extraction](https://pypdf.readthedocs.io/en/latest/user/extract-text.html) — scanned/OCR distinction and + potentially high extraction cost； +- [IANA media types](https://www.iana.org/assignments/media-types/media-types.xhtml) — registered PDF/EPUB/ZIP and + image/audio/video media types。 + +## Hard Cut-off And Peer Compatibility + +- Existing bare `image`、`video`、`html` and `text` implementations are removed rather than retained or + reinterpreted。Their current contracts mix URL/content/storage/AI concerns and are not a compatibility baseline worth + extending，or lack the methods currently forced by the abstract base。The same coherent implementation pass updates + every in-repo producer，consumer and test to the exact replacement IDs above。Old persisted rows become explicitly + unsupported；D-075 accepts this hard cut-off without legacy decoders or a data migration。 +- core-py already fails an unknown resolver ID explicitly；client-web's current fallback to the first/default resolver + must be removed。Unavailable/unknown resolver ID becomes an explicit unsupported state，never silent text rendering。 +- Installed resolver lifetime remains separate from extension enabled/running lifetime。The shared IDs above are core + resolvers available in each capable peer；a peer without one reports unsupported capability rather than delegating + implicitly to core-py。 +- client-web graph preview and fallback must stop displaying a storage pointer as content。It may request a bounded + text/feature projection or show a typed unavailable state，but must not eagerly hydrate every graph node。 +- Twitter's image/video use and Memos attachment metadata-block behavior are rewritten and regression-tested in the + same pass；they are not reasons to retain the old IDs or weaken the new contract。 + +## Technical Decisions Intentionally Deferred + +D-075 accepts this Product/Technical boundary。Preflight still decides： + +- exact parser/runtime dependencies and licenses（for example HTML text extraction，charset handling，Pillow，audio + probing，ffprobe/pure-library trade-off，pypdf，EPUB parser）； +- bounded parse limits，encrypted/protected input behavior and proportional ZIP/PDF resource controls； +- exact Python/TypeScript type names and actual-content handles； +- optional embedded metadata privacy/exposure policy； +- derived relation grammar，materialization idempotency/provenance and child cleanup； +- an operational preflight proving that the accepted hard cut-off does not hide a concrete data-preservation need； +- browser render/open components and PostgreSQL binary transport execution details。 + +## Acceptance Consequences + +The future execution baseline must prove at least： + +- each new byte-oriented resolver ID resolves a storage-backed real sample through PostgreSQL bytes into its typed + minimum facts and peer-local usable content；plain text and HTML also prove inline content，and storage-backed + decoding is accepted only after its charset authority is frozen； +- Memos/RSS/Atom metadata-block facts remain authoritative and are not copied into semantic-content-block content； +- declared/observed MIME conflicts follow D-070–D-072 without rewriting the metadata block； +- unsupported MIME selects `core.file.v1`；unknown resolver ID and missing local storage/resolver fail explicitly； +- no-text blocks are skipped by embedding rather than indexed as empty strings； +- client-web does not expose opaque pointers and does not silently render unknown resolver IDs as text； +- no in-repo producer emits a retired resolver ID，and reads of those IDs fail explicitly rather than silently + changing meaning。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/delivery-map.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/delivery-map.md new file mode 100644 index 0000000..870b187 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/delivery-map.md @@ -0,0 +1,66 @@ +# Semantic Retrieval Delivery Map + +- **Purpose**: keep one semantic-retrieval product vertical understandable while separating its implementation dependency + surfaces。 +- **Status**: design probe only。The increments below are not an approved implementation plan，do not authorize code or + migrations and may change when Acceptance or preflight exposes a different dependency。 +- **Rule**: the active unit stays one Acceptance vertical。An increment is a reversible verification boundary，not a new + product/unit identity or an excuse to create additional runtime services。 + +## Dependency Topology + +```mermaid +flowchart TD + S0["S0 shared protocol foundation"] --> S1["S1 AI registry / execution"] + S0 --> S7["S7 Peer delivery"] + S1 --> S3["S3 Agent / Thread"] + S1 --> S5["S5 semantic projection / records / maintenance"] + S2["S2 graph producer forms"] --> S4["S4 rumination"] + S3 --> S4 + S5 --> S6["S6 local SemanticRetrievalManager"] + S6 --> S7 + S4 --> S8["S8 vertical Acceptance / promotion"] + S6 --> S8 + S7 --> S8 +``` + +S1 and S2 may proceed independently after their own code-address preflight。S3 depends on chat/tool-calling execution but +not on GraphForm；it can verify its generic Tool runtime with a test Tool。S4 is the join of S2 and S3。S7's generic peer +schema/protocol groundwork may proceed early，but its semantic-retrieval inbound/outbound proof depends on S6's local +implementation。 + +## Provisional Increments + +| Increment | Coherent state diff | Depends on | Verification boundary | +| --- | --- | --- | --- | +| S0 — shared foundation | database-owned timestamps；generic ConfigContract；deployment `configs`；required peer/config schema groundwork | approved common contracts | migration/static contract + real PostgreSQL CRUD/trigger probes | +| S1 — AI module | hard-cut process-global AI config；add AIDialect/Provider/Model and AIManager `embed`/`chat` with OpenAI-compatible adapter | S0 persistence/config | typed adapter tests + real configured provider smoke when credentials exist | +| S2 — graph producer forms | add BlockForm/RelationForm、retained StarsGraphForm authoring and flat GraphForm；replace SubGraphForm persisted-model leakage；add InfoBaseManager normalizer/write path；correct audited relations | approved D-179 producer grammar | form/static validation + real PostgreSQL arbitrary graph and targeted migration tests | +| S3 — Agent/Thread | Agent definitions、decorator Tool registry、canonical tool-call loop、in-memory thread persistence backend and structured-concurrent Tool batch | S1 | fake dialect state-machine tests + real tool-calling provider smoke | +| S4 — rumination | focal resolver/direct-relation context、deployment-config Agent selection、on-demand Resolver draft schemas、rooted GraphForm drafting、organization-owned `submit_graph(GraphForm)` and shallow completion semantics | S2、S3 | real focal Block → Agent → draft/commit graph journey；no-op/cannot-understand/failure/repetition/explicit-trigger cases | +| S5 — semantic records | Resolver/Relation projections、EmbeddingProfile/records、freshness、maintain/rebuild and default config | S0、S1 | real Memos/RSS graph → projection → vector rows；stale/unavailable/failure and repeat maintenance | +| S6 — local retrieval | SemanticRetrievalManager、exact cosine ranking、bounded filters/result and local domain route | S5 | direct local request → ranked real Block/Relation identities and scores | +| S7 — peer delivery | Peer persistence/lease、capability advertisement、HTTP protocol/outbound/inbound、any-provider/exact-target routing and bounded failover；technical Client→Peer migration；exact Extension-management consumer replacement | S0、S6 | two real peer runtimes；online selection、target constraint、non-execution failover、uncertain-outcome stop and Extension config/hot lifecycle journey | +| S8 — acceptance/promotion | approved corpus and quality thresholds across rumination、freshness、local/delegated retrieval；durable owner reconciliation | S4–S7 | black-box-first suite + repository/cross-repo static/runtime checks | + +## Cross-Repository And Runtime Surfaces + +- **core-py**: migrations、schemas/managers/routes、AI/Agent/config/Peer domains、info-base forms、resolver/relation + projection、semantic maintenance/retrieval、producer corrections and test infrastructure。 +- **client-web**: peer terminology and shared database DTO/protocol projection，PostgreSQL peer/config/AI/Profile rows and + any locally implemented AI/retrieval capability demanded by parity。Exact implementation scope requires preflight；peer + equality does not mean every runtime implements every capability。 +- **Hub/shared docs**: Product TDD topology for AI/config/Peer/Agent/semantic retrieval and PRD-observable retrieval/ + organization behavior，promoted only after implementation evidence and through the shared-doc workflow。 +- **Database/deployment**: migration chain and configured AI provider/model/profile/Agent/config/Peer facts。Production is a + public demo，but migrations still require real forward/readiness verification and no silent credential/config invention。 + +## Plan-Building Rules + +- Convert these increments into an implementation plan only after closure items 1–3 are approved。 +- Preflight must verify exact code addresses and decide whether S0 needs smaller migration commits；the conceptual grouping + here does not require one migration or one commit。 +- Prefer parallel execution only where the dependency graph permits and verification remains attributable。Do not parallelize + two increments that both rewrite the same manager/schema authority。 +- Every increment must leave the repository internally coherent and must not preserve compatibility wrappers for rejected + legacy embedding、RAG、Client or SubGraphForm authorities unless preflight finds a still-supported consumer。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/evidence.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/evidence.md new file mode 100644 index 0000000..78bdf47 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/evidence.md @@ -0,0 +1,162 @@ +# Semantic Retrieval Evidence + +本文记录 Product/Technical 讨论所需的现状事实与失败证据，不把既有实现提升为设计 authority。 + +## Current Topology + +```text +text query / block ID + → core-py /blocks/query/by_embedding + → hard-coded embedding provider/model call + → block_embeddings cosine-distance query + → raw BlockModel list + +core-py /sink/rag + → same embedding candidate query + → optional nominal rerank + → resolver.get_text() for prompt context + → hard-coded chat model + → generated answer + +block fetchsert / 60s scheduler + → resolver.get_str_for_embedding() + → hard-coded embedding provider/model call + → one block_embeddings row per block +``` + +client-web `packages/core` mirrors the resolver `getStrForEmbedding()` contract but does not own a shared retrieval +operation。client-webext separately calls core-py HTTP through `Block.vectorSearch()`；its local `Root.RAG()` is already +marked deprecated in favor of client-local generation。 + +## Proven Existing Surfaces + +| Concern | Current evidence | Pressure, not decision | +| --- | --- | --- | +| User/API query | core-py accepts text or block ID and returns raw blocks | no explicit result contract、score、matched projection、graph context or stable pagination | +| Semantic projection | every resolver must implement `get_str_for_embedding()` in Python and TypeScript | resolver contract is coupled to one application mechanism；many implementations merely duplicate `get_text()` | +| Projection identity | embedding row is keyed only by block/relation ID | model、provider、dimensions、projection contract version and source snapshot are not represented；only one vector can exist | +| Model selection | query and write paths instantiate `text-embedding-v3` directly；database vector dimension is fixed at 1024 | runtime compatibility and migration behavior are implicit | +| Lifecycle | fetchsert embeds eagerly；ordinary create relies on a 60-second scan；edit leaves an existing embedding row in place | missing-row scan cannot repair stale rows；storage-backed bytes can change independently | +| Unsupported content | scheduler process-local set quarantines unsupported/unknown block versions | policy is not durable、peer-shared or tied to a named projection/index | +| Relations | every missing relation row embeds `relation.content` directly | no proven retrieval consumer or semantic-selection contract | +| Query parameters | server owns `max_distance`；client-webext sends `distance_threshold` | client control is silently ineffective under ordinary FastAPI extra-query handling | +| Reranking | current reranker re-queries the same vector distance and sorts it again | it is not independent reranking evidence | +| RAG | core-py joins retrieval、resolver text projection、prompt assembly and answer generation | client-webext already treats core generation as deprecated；Hub only claims retrieval/RAG as possible downstream uses | +| Acceptance | resolver tests cover unsupported/unknown skip behavior | no real semantic query corpus、relevance judgments、ranking/freshness contract or peer-level black-box acceptance | + +## Hub Evidence Boundary + +Current shared PRD/Product TDD only says collected information should support retrieval/indexing/embedding/downstream +use，and that embedding remains sink-owned even when ingestion triggers it。It does not define semantic retrieval as RAG， +does not require a resolver method named for embedding，and does not define a result or index lifecycle contract。 + +## AI Provider / Model Split-Brain Evidence + +core-py `libs/ai.py` constructs one OpenAI-compatible client from `settings.llm_sp_base_url` and `llm_sp_ak` at import +time。`Chat(provider, model)` and `Embedding(provider, model)` retain `provider` in their object shape but execution uses +the same global client regardless。Embedding and language call sites then hard-code `text-embedding-v3` and `qwen-plus`。 + +client-web independently persists browser-local `LLMProviderConfig` values with provider `type`、API key、base URL and a +flat `models[]` list。client-webext has the more useful separation of `ProviderFactory` dialect strategies and an AI SDK +provider registry，but that registry is local to the browser and LLM-named；it is not a shared provider/model protocol and +does not describe model capabilities or modalities。 + +Mature AI SDK registry behavior supplies useful library evidence rather than InKCre authority：one registry can resolve +provider-qualified model IDs and exposes capability-specific language、embedding and image model lookup；an +OpenAI-compatible provider adapter owns base URL、API key、headers/query parameters and capability-specific model +factories。This supports separating provider instance、dialect adapter and typed model capability，but does not decide +InKCre persistence or peer ownership。 + +## Current `updated_at` Ownership Is Split + +The database currently does not provide one general row-update timestamp contract： + +- `server_default=CURRENT_TIMESTAMP` initializes values on insert only；it does not touch later updates。 +- SQLAlchemy `onupdate=datetime.now` appears on blocks、relations and embedding models，but applies only to ORM-authored + update statements and therefore cannot govern equal peers writing through PostgREST/direct SQL。 +- one PostgreSQL trigger exists only on `blocks` and only changes `updated_at` when `content` changes。It does not cover + resolver/storage changes、relations、embedding records or future AI registry/profile relations。 +- the helper is now internal-schema runtime machinery，but its generic name obscures its block-content-specific behavior。 + +New shared protocol relations therefore need an explicit choice。For database-row mutation time across equal peers，a +selected-table PostgreSQL `BEFORE UPDATE` touch trigger is the smallest common authority；source-authored timestamps、job +event times and storage-content freshness remain separate semantics and must not receive that trigger mechanically。 + +## Existing Application Scenarios + +| Existing surface | Actual user/downstream action | Existing expected result | +| --- | --- | --- | +| client-web start view | deployment owner could type a query into “Find information helps you here” | business logic is still a placeholder；Sir does not use InKCre this way and rejected it as the unit's primary journey | +| client-webext Explain agent | agent asks the info-base vector-search tool for knowledge relevant to an explanation | current tool expects knowledge-base `Block` values；answer generation stays in the client-local agent | +| client-webext Writing Assist | each replaceable word is queried within `learn_english.lexical` | current consumer expects ordered lexical `Block` values and reads the first block's canonical content | + +Memos and RSS add realistic candidate shapes：a memo/comment is a block；feed、feed item、enclosure metadata、full-text +semantic content and attachment semantic content are blocks connected by relations。A hit on a child block can remain that exact block； +following an incoming/outgoing relation is graph navigation，not evidence for an invented retrieval “subject” layer。 + +Relations are also authoritative info-base entities，but these scenarios alone did not prove that every relation—or any +relation in the MVP—needed semantic indexing。That pressure drove the later D-097–D-102 review：MVP now permits Relation +matches through a RelationManager-owned endpoint-aware projection，while blind embedding of raw `relation.content` remains +rejected。 + +## Client-Web Rumination Trigger Evidence + +The current client-web graph view already loads real Blocks/Relations，lets the user select one Block and opens a right-side +`BlockDetailsPanel` containing its ID、resolver、timestamps、storage and rendered content。That panel is the smallest coherent +UI point for an explicit focal-Block organization action；a global toolbar would first need another selection/context model。 +On completion the parent graph view already owns one `loadData()` boundary capable of refreshing newly added graph facts。 + +The current `@inkcre/core` still has legacy `Client(rest_api_url).request(...)` and no approved Peer capability routing +implementation。It also assumes successful responses contain JSON，so it is failure evidence rather than a reusable path +for a `204` rumination result。The accepted Client→Peer hard cut and PeerHTTPOutbound must supply discovery、peer JWT、 +normalized envelope、execution-marker and outcome-unknown semantics；the Block panel must call an organization-domain +facade rather than select a Core URL/provider itself。 + +Core-py's corresponding legacy publication path is `Settings.client_base_url / CLIENT_BASE_URL` → +`ClientManager.initialize()` → `clients.rest_api_url`。Local Compose currently supplies it from `CORE_PUBLIC_URL` with a +localhost fallback。The value demonstrates a real deployment-owned public-address requirement，but its Client/global-Peer- +field projection is rejected；the Peer HTTP protocol needs a renamed peer-local runtime authority from which concrete +inbounds derive their advertised absolute URLs。 + +Sir corrected the replacement authority：the needed public base belongs in the already approved owner-specific +`peers.config`，which deployment or client-web administration can edit directly。The runtime's capability snapshot remains +the derived routing projection。No new environment-only public-address authority is needed。 + +## Chunk And Breakdown Pressure + +No current InKCre path creates a transient search chunk。The earlier proposal incorrectly added one as a result-layer +concept。Sir clarified that semantic granularity is nevertheless central to vector retrieval quality and that organization +breakdown partly exists to solve this problem without a second `block segment` information model。 + +The accepted direction is therefore： + +- breakdown creates reusable ordinary blocks/relations at useful semantic granularity； +- semantic retrieval ranks and returns blocks/relations； +- this unit must exercise both sides through a long/compound corpus rather than assume one embedding per collected root is + sufficient。 + +D-081 closes the earlier owner question：organization owns the graph breakdown；embedding records and vector retrieval +remain use-side derived support。 + +## Candidate Eligibility Correction + +“Select which blocks/relations are searchable” was too close to an allowlist。The stronger default is to consider every +persisted block/relation as a graph entity candidate。For one embedding profile，an entity may still have no useful +embedding input：for example untranscribed audio/image bytes or structural relation labels such as `attachment:0` and +`full_text`。That is profile capability/availability，not exclusion from the info-base or from every future retrieval mode。 + +The later design closes this question through Resolver-owned general Block projections and RelationManager-owned directed +Relation projection，without restoring a resolver method coupled to embedding。 + +## Initial Diagnosis + +`get_str_for_embedding()` is not an isolated naming defect。It is the visible seam where four currently unnamed +responsibilities meet： + +1. **semantic selection**：which authored/solved/graph information represents a candidate； +2. **projection policy**：how that information is shaped for one retrieval strategy； +3. **model execution**：which provider/model turns an input into a vector； +4. **embedding-record lifecycle**：which projection/profile/source snapshot one durable derived row represents。 + +This diagnosis supplied the review order rather than a still-open question。D-079 onward fixed the result/consumer +boundary，and the current contracts place these responsibilities without preserving the old coupling。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/impact-handshake.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/impact-handshake.md new file mode 100644 index 0000000..7a0f779 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/impact-handshake.md @@ -0,0 +1,150 @@ +# Semantic Retrieval Implementation Impact Handshake + +> **State**: approved for implementation on 2026-08-07。Sir requested a clean task-state commit before the first code、 +> schema or client-web mutation；artifact publication、shared runtime reset and Hub promotion retain their later gates。 + +## Address And Object + +### core-py + +- `app/configuration/`、`app/schemas/configuration.py` and deployment-config routes：generic `ConfigContract` plus + deployment-scoped `ConfigManager` mechanics。 +- `app/schemas/info_base/**`、`app/business/info_base/**` and every repository-owned graph producer：database-state-free + forms、StarsGraphForm/GraphForm、normalize/submit、Resolver text/labels and Relation projection。 +- `app/schemas/ai.py`、`app/business/ai/**` and removal of `libs/ai.py`/legacy RAG/query paths：Provider/Model/Profile、 + OpenAI-compatible dialect and canonical embedding/chat contracts。 +- `app/schemas/agent.py` and `app/business/agent/**`：persisted definitions、in-memory Thread persistence backend、async + turn lifecycle and runtime Tool registry。 +- `app/schemas/semantic_retrieval.py`、`app/business/semantic_retrieval/**`、routes and scheduler：records、explicit + maintenance、local retrieval and capability codec/inbound。 +- `app/business/organization/**` and organization routes：explicit focal-Block rumination using the configured Agent and + Graph tools；no periodic organization runner。 +- `app/schemas/peer/**`、`app/business/peer/**`、middleware、settings、composition root and database-contract machinery： + technical Client→Peer hard cut、lease/discovery、HTTP outbound and three fixed business inbounds。 +- exact built-in Resolver/producer surfaces under `extensions/{memos,rss,mail,github,telegram,twitter,learn_english}/`。 +- `migrations/` and database-contract projection/readiness/profile surfaces：three append-only structural revisions，new + clean shared-database target and a breaking peer runtime contract revision。 +- `tests/`：domain-focused suites、real PostgreSQL integration、real RSS/Atom/Memos journeys and pinned SQLite Architecture + corpus authority；generated graph/vector artifacts remain ignored。 + +### client-web + +- `packages/core/src/{peer,extension,organization,semantic-retrieval,ai/info-base}` and generated database/runtime contract： + Peer routing、removal of `Client.request()`、exact Extension management and typed rumination/retrieval facades。 +- technical database/admin surfaces migrate Client→Peer；product/repository identity and ordinary user-facing copy keep + “client”。 +- the selected Block details surface receives one explicit Ruminate action with pending/success/error handling。 +- final generated contract pin requires an exact core commit plus matching digest-pinned OCI artifact。 + +### Documentation And Runtime + +- core-py Unit TDD、deployment/security/runtime docs and nearest AGENTS are reconciled with implemented evidence。 +- shared PRD/Product TDD changes use the Hub workflow and a separate shared-doc/ref operation；`docs/_shared/**` is never + edited directly in this Spoke change。 +- canonical production and the data-free preview baseline are rebuilt only at the later delivery gate under D-195；there + is no active staging target。 + +## State Diff + +```text +legacy split AI/RAG + one embedding row per entity + -> Provider/Model/Profile + profile-scoped records + explicit maintain/retrieve + +embedding-specific Resolver strings + isolated Relation content + -> one general Block text projection + Block-local endpoint labels + -> directed subject/property/value Relation projection + +SubGraphForm over persisted models + -> StarsGraphForm producer authoring + signed-ID flat GraphForm command + +one-call reasoning helpers + -> persisted reusable AgentDefinition + async in-memory Thread runtime + typed Tools + +unused organization hooks + -> explicit focal-Block rumination that may submit an ordinary graph or complete with no write + +Client endpoint shortcuts + -> Peer capability advertisement + lease + exact HTTP protocol + local-or-delegate domain facades +``` + +The peer runtime wire hard cut includes technical profile/database names and JWT issuer +`inkcre-client -> inkcre-peer`。Audience `inkcre-api`、shared-secret trust model、maximum token lifetime and user-facing +client product terminology remain unchanged。 + +## Operation And Expected Side Effects + +- Add new deep domain modules and tables，hard-delete superseded internal mechanisms and rename technical Client symbols。 +- Append structural Alembic revisions；do not create Resolver-row、Mail-edge or legacy vector compatibility migrations。 +- Regenerate the language-neutral database contract and client-web TypeScript projections from one exact core artifact。 +- After verified delivery artifacts exist，make one recoverable destructive rebuild of canonical production application + schemas and advance/sanitize preview-base。The dump、digest and Neon recovery branch precede reset；archived staging + lineage is untouched。 +- Promote proven durable truth only after Acceptance；do not mix Hub edits/shared-ref bumps with Spoke implementation + commits。 + +## Blast Radius Forecast + +- **Very high inside core-py**: schema imports、composition/bootstrap、Resolver abstract methods、all graph producers、AI + calls、scheduler and route assembly must move coherently across implementation increments。 +- **High protocol impact**: PostgREST relation names/types、JWT issuer、peer runtime profile and client-web generated + contracts break together。There is intentionally no compatibility interval because shared databases are rebuilt。 +- **Bounded product impact**: new direct retrieval/rumination capability and one Block action；no Chat InKCre product、 + automatic organization、generic capability console or new release unit。 +- **No source/storage authority change**: collected information remains Blocks/Relations；Storage owns bytes/pointers and + Resolver owns interpretation。 + +## Invariants Check + +- Blocks/Relations remain info-base authority；embedding rows、capability snapshots and corpus aliases are derived/support + state，never new information entities。 +- Organization may materialize ordinary graph improvements but retrieval/indexing remains use-owned。`retrieve()` never + repairs candidate embeddings implicitly。 +- AIManager stays graph-blind；Resolver stays model/profile-blind；PeerManager stays business-capability-blind。 +- No generic `/capabilities/{id}/invoke`、generic delegation job、readiness advertisement or automatic replay after + outcome-unknown dispatch。 +- Agent runtime validates Tool input once through the registered Pydantic schema；InfoBase write relies on database FK for + positive references and adds no duplicate existence layer。 +- `get_label()` is deterministic and Block-local；Relation freshness depends exactly on Profile、Relation and two endpoint + Blocks。 +- Technical/database/domain names use Peer；marketing、landing、external-app and product/repository “client” vocabulary is + not mechanically renamed。 +- No S3 storage、persistent Agent threads、graph-reading Agent Tools、automatic rumination trigger、pagination or HNSW is + introduced。 + +## Verification + +1. Static/lint/type checks plus structural retired-symbol/ID searches after each coherent increment。 +2. Deterministic fake dialect/Agent/Peer protocol state-machine tests for lifecycle、validation、parallel Tools、failover、 + exact-target routing and outcome-unknown behavior。 +3. Disposable PostgreSQL base→head and empty current-head→new-head migrations；ACL、sequence、trigger、PostgREST binary、 + config and generated protocol checks。 +4. Black-box Memos、RSS/Atom、graph producer、Resolver/Relation projection and explicit rumination journeys through real + domain boundaries。 +5. Real provider credentialed embedding quality and tool-calling Acceptance over entity judgments，including SQLite + Architecture rumination；provider responses/scores are not committed authority。 +6. Two real ASGI/HTTP Peer runtimes prove local bypass、delegation、non-execution failover、target constraint、JWT/envelope + and all three fixed capability inbounds。Standards evidence replaces an unnecessary real reverse-proxy smoke test。 +7. client-web generated contract check、package tests/build and BlockDetailsPanel behavior after exact artifact sync。 +8. Before shared reset，verify dump digest/recovery branch/targets；after reset，readiness and production discovery must + report the same new contract revision/head before the operation is considered complete。 + +## Uncertainty And Execution-Time Checks + +- Exact generated DDL、index names、PostgREST grants/sequences and Alembic split are inspected after target SQLModel schema + exists；generation before implementation would be fictitious。The three ownership boundaries remain review-visible even + if autogenerate requires regrouping。 +- Provider semantic quality varies；deterministic lifecycle tests own CI，while a named credentialed provider owns the + empirical Acceptance run。Failure leads to retrieval/projection diagnosis，not test-shaped production code。 +- client-web final pin cannot close before a matching digest-pinned core OCI artifact exists。Publication is separately + authorization-gated after a core commit；local-core generation supports implementation beforehand。 +- canonical production currently disagrees with its checked-in profile（v1/d9 versus v2/d0）。The clean rebuild closes the + discrepancy only after target runtime evidence；this unit does not mutate production during ordinary implementation。 +- The worktree already contains task-packet/local-doc changes from prior units。Implementation preserves them and stages or + commits only explicitly authorized scope。 + +## Execution Order + +Follow [implementation-plan.md](implementation-plan.md) I0→I8，with I1 Graph forms and I3 Agent runtime allowed to proceed +independently after I0/I2 prerequisites and joining at I5 rumination。Each increment must satisfy its local verification +before the next dependency consumes it。Cross-repository artifact publication、shared database reset and Hub promotion are +later explicit gates，not hidden side effects of beginning core implementation。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/implementation-plan.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/implementation-plan.md new file mode 100644 index 0000000..80c0ae5 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/implementation-plan.md @@ -0,0 +1,421 @@ +# Semantic Retrieval Implementation Plan + +- **Status**: closed；I0–I8 implementation、credentialed Acceptance and owner-separated shared-truth promotion complete。 +- **Inputs**: approved Product/Technical/Acceptance decisions through D-196，current core-py/client-web code and delivered + core-py migration head `c0d1e2f3a4b5`。 +- **Delivery rule**: one product unit，multiple coherent increments。Each increment must leave its repository internally + checkable；compatibility aliases for rejected Client、SubGraph、legacy AI/embedding/RAG authorities are forbidden。 + +## Dependency Order + +```mermaid +flowchart TD + I0["I0 config + timestamps"] --> I2["I2 AI domain"] + I0 --> I4["I4 semantic records/retrieval"] + I2 --> I3 + I2 --> I4 + I1 --> I5["I5 organization rumination"] + I3 --> I5 + I4 --> I6["I6 core Peer delivery"] + I5 --> I6 + I6 --> I7["I7 client-web Peer consumers"] + I4 --> I8["I8 vertical Acceptance"] + I5 --> I8 + I7 --> I8 +``` + +I1 may proceed independently of I0/I2。I3 and I4 share AI schema/types but otherwise may be implemented in either order +after I2；do not parallel-edit their shared AI Manager/schema files。I1 and I3 join at I5；I5 and I4 join only at +Acceptance，while I6 must see both +local business capabilities before publishing their inbound advertisements。 + +## I0 — Configuration Mechanics、Deployment Configs And Row Time + +### State diff + +- Extract model-driven complete validation、normalized JSON、shallow patch preparation and JSON Schema projection into + `ConfigContract[Model]` under `app.configuration`。 +- Add deployment-scoped `configs` persistence、schema registry、DeploymentConfigManager and honest PUT/PATCH routes。 +- Refactor Extension config update to consume ConfigContract without moving Extension lifecycle into the generic module。 +- Replace application-owned `onupdate` behavior with no-op-aware PostgreSQL `BEFORE UPDATE` triggers on approved shared + rows，initially Blocks、Relations and all new mutable protocol rows introduced by this unit。 + +### Primary addresses + +- new `app/configuration.py`； +- new `app/schemas/deployment_config.py`、`app/business/deployment_config.py`、`app/routes/deployment_config.py`； +- `app/business/extension/main.py`； +- `app/schemas/{__init__.py,info_base/block.py,info_base/relation.py}`； +- `run.py` route/config-contract registration composition； +- first new Alembic revision plus metadata/readiness/privilege surfaces。 + +### Verification + +- static ConfigContract tests for complete/patch/schema behavior and collision-free owner boundaries； +- real PostgreSQL config CRUD/unknown-schema/invalid-persisted-value journeys； +- trigger tests proving changed row advances `updated_at`、no-op update does not and PostgREST/SQL writes behave equally； +- Extension config regression through its real Manager and running instance apply path。 + +## I1 — Producer Forms、Graph Commands And Relation Corrections + +### State diff + +- Add database-state-free `BlockForm` / `RelationForm` and rename recursive producer authoring from `SubGraphForm` to + `StarsGraphForm` in one hard cut across every repository-owned producer。 +- Add flat `GraphForm` with command-local signed Block IDs、intrinsic Pydantic structural validation and arbitrary connected + graph support。A small Graph-owned wrapper may carry signed `id` beside BlockForm fields；it is not a persisted-model + identity or a new information entity。 +- Add `InfoBaseManager.normalize_graph(stars, id_start)` and `submit_graph(graph)`；PostgreSQL FK remains positive-reference + existence authority。The public graph command accepts GraphForm；Resolver/extension internals keep StarsGraphForm。 +- Correct the evidenced Mail sender direction in the producer；hard-cut Twitter's unused legacy + `_organize()` writer to no-op。Do not reinterpret relations in retrieval or organization runtime。 + +### Primary addresses + +- split `app/schemas/info_base/main.py` into a forms-owned surface and update `app/schemas/info_base/__init__.py`； +- rewrite `app/business/info_base/main.py` around normalize/submit while retaining caller-session transaction discipline； +- update `app/business/info_base/resolver/**` and all `extensions/{mail,github,rss,telegram,twitter}/**` consumers； +- check Memos' repository-owned graph helpers even where they already write directly； +- update `run.py` graph route and `app/routes/AGENTS.md`。 + +### Verification + +- Pydantic form invariants：negative create IDs、positive references、zero rejection、unique declared IDs and known negative + endpoints； +- real PostgreSQL arbitrary connected graph insert and negative→positive mapping； +- StarsGraphForm normalization parity over representative core/extension producers； +- producer-level black-box assertions for every audited relation，including Mail's corrected sender direction； +- structural search proving `SubGraphForm` and persisted BlockModel/RelationModel producer forms are gone。 + +## I2 — AI Registry、Canonical Capability Contracts And Legacy Hard Cut + +### State diff + +- Add AIDialect、AIProvider、AIModel and EmbeddingProfile shared models/catalogs with bigint identities and database-owned + timestamps。Register exact `core.openai-compatible.v1` with config model containing only evidenced OpenAI-compatible + connection values；credentials remain ordinary provider config persisted in AIProvider。 +- Add graph-blind AIManager with `embedding` and `chat` capability modules、canonical Messages/Tools/ToolCalls/results and + one OpenAI-compatible dialect adapter。Validate model capability/modalities/features、provider/model enabled state、tool + choice support、batch order/cardinality and vector dimensions。 +- Remove process-global `libs/ai.py` authority and all obsolete users：legacy Sink/RAG、Block embedding/reasoning helpers、 + block-route semantic shortcuts and eager/background legacy EmbeddingManager behavior。No wrapper preserves old APIs。 + +### Primary addresses + +- new `app/schemas/ai/` and `app/business/ai/` deep modules，exported through package `__init__.py`； +- `app/settings.py` removes `llm_sp_*` after no consumer remains； +- delete/rework `libs/ai.py`、`app/business/sink/**`、`app/schemas/sink/**` and `/sink/rag` composition； +- trim `app/business/info_base/block.py` and `app/routes/block.py` to Block-owned behavior； +- AI/profile/record schema revision recreates legacy embedding tables rather than attempting false provenance migration。 + +### Migration rule + +Legacy block/relation vectors are derived and cannot name their Provider、Model、Profile or semantic input。Drop and +recreate the embedding relations in their approved composite-profile shape；do not synthesize a legacy Profile。Preserve +graph entities and rebuild vectors only after an explicit/default Profile is configured。 + +### Verification + +- model/capability/config validation and catalog-registration collision tests； +- fake OpenAI-compatible transport tests for embed/chat/tool wire translation and all validation failures； +- optional real configured provider smoke for embedding and tool calling，never required merely to type-check the module； +- structural proof that no process-global AI client、legacy RAG/EmbeddingManager or `get_str_for_embedding()` consumer + remains。 + +## I3 — Persisted Agent Definitions And In-Memory Thread Runtime + +### State diff + +- Add `agents` rows with exact approved fields and bigint identity；no Agent seed or default Rumination Agent。 +- Add AgentManager decorator-owned exact Tool registry、Pydantic input binding and runtime Tool schema projection。 +- Implement replaceable thread persistence backend contract with only an in-memory backend，Thread snapshots of Agent + definition behavior and cancellable per-turn asyncio Task。 +- Execute ToolCalls concurrently inside the Turn structured-concurrency scope；commit one closed AssistantMessage + + ToolResultMessage pair atomically，isolate ordinary per-call failures and preserve completed effects on budget/cancel。 + +### Primary addresses + +- new `app/schemas/agent.py` and `app/business/agent/` package（manager、contracts、thread persistence/runtime）； +- AI canonical Messages/Tools from I2 are reused rather than copied； +- Agent schema belongs in the AI/semantic foundation migration or a following append-only revision。 + +### Verification + +- deterministic fake dialect state machines for natural completion、budget exhaustion、multi-call concurrency、validation + error、handler error、unexpected exception、cancellation and incomplete trailing-message recovery； +- registry collision、missing persisted Tool、tool set canonicalization and snapshot isolation； +- optional real tool-calling provider smoke separate from deterministic lifecycle authority。 + +## I4 — Semantic Projection、Embedding Records、Maintenance And Local Retrieval + +### State diff + +- Remove Resolver `get_str_for_embedding()` and make complete `get_text()` the text projection。Add required deterministic + `get_label()` to all core and in-scope extension Resolvers；unsupported/null/empty remain distinct。 +- Hard-cut legacy extension registrations to the exact IDs approved by D-194；retain no aliases or compatibility + decoders。The Resolver execution checklist owns the per-ID text/label/create-graph obligations。 +- Correct unreleased `extensions.rss.feed_item.v1` in place：its complete projection concatenates title、summary and full + text/authored content，whereas the current local implementation selects only one best body。Because v1 has not reached + any retained staging/preview deployment，reset disposable local data rather than minting v2 or a compatibility migration + (D-193)。 +- Add RelationManager subject/property/value projection using exact endpoint labels and no RelationResolver。 +- Add composite-profile Block/Relation EmbeddingRecords with variable dimensions and timestamp/dimension freshness filters。 +- Implement one SemanticRetrievalManager owning default Profile resolution、maintain/rebuild、query embedding、exact cosine + comparison and one global bounded Block/Relation result。`retrieve()` never repairs candidate records。 +- Register `core.semantic_retrieval.config.v1` and the default Profile config；replace the old scheduler hook with bounded + scheduled `maintain` for only the configured default Profile。 +- Add the typed local/non-delegating execution path and fixed HTTP inbound codec required later by Peer composition。 + +### Primary addresses + +- resolver files under `app/business/info_base/resolver/**` and exact in-scope extension resolvers； +- [Resolver execution checklist](resolver-execution-checklist.md) for the complete exact text/label/producer matrix； +- `app/business/info_base/relation.py`； +- new `app/schemas/semantic_retrieval.py` and `app/business/semantic_retrieval/`； +- new `app/routes/semantic_retrieval.py`； +- `run.py` scheduler/route composition； +- embedding/profile schema and timestamp triggers from I2/I0。 + +### Verification + +- real Memos/RSS graphs through Resolver/Relation projection； +- maintain scan pagination past unavailable candidates、provider failure、batch validation、atomic upsert and interruption； +- Block/Relation/Profile mutation makes stored rows immediately stale and excluded until maintain； +- exact mixed global ranking、tie order、limit、threshold、entity filter、dimension and default/dangling config cases； +- direct local Manager and route black boxes return real graph entities，not internal vector rows。 + +## I5 — Organization Rumination And Graph Tools + +### State diff + +- Add OrganizationManager with only `ruminate(block_id)` and a private non-delegating local execution seam；no + RuminationManager、job、scheduler or run record。 +- Register `core.organization.rumination.config.v1` and resolve the configured Agent at use time。 +- Build the focal Resolver text + bounded direct-relation initial UserMessage；cannot-understand completes shallowly without + starting Agent。 +- Register `get_draft_graph_schema`、`draft_graph` and `submit_graph` Tools。Agent runtime owns one validation pass；Resolver + owns `create_graph(input) -> StarsGraphForm`；InfoBase owns normalize/submit；submit returns negative→persisted ID mapping。 +- Await the active Turn and map natural completion/no-write to `None`、budget exhaustion to one organization failure and + caller cancellation to Turn cancellation without retry/rollback。 +- Add fixed `POST /organization/ruminate` body `{block: int}` and `204` local inbound。 + +### Primary addresses + +- new `app/business/organization.py`、`app/routes/organization.py` and owner config model； +- ResolverManager runtime schema/description snapshot and Resolver create-graph contract； +- Agent Tool registration composition； +- delete `BlockManager.query_by_reasoning()` if not already removed in I2。 + +### Verification + +- deterministic focal context construction and direct direction rendering； +- real Resolver schema discovery/draft → GraphForm → submit journey； +- useful write、honest no-op、cannot-understand、validation/Tool failure、budget、cancel、repeat and concurrent additive cases； +- explicit route only；structural proof of no periodic/collection hook or persistent run entity。 + +## I6 — Core Peer Protocol、Routing And Three Capability Inbounds + +### State diff + +- Hard-cut technical Client→Peer：rename table/model/manager/settings/types；the new Peer shape retains UUID identity、name、 + labels、config and config_schema semantics，drops `rest_api_url` and adds capabilities snapshot、lease expiry and + updated_at。D-195 removes retained-row/Extension-array migration compatibility。 +- Add exact `renew_peer_lease(peer, ttl_seconds)` database-time helper and peer-local renewal schedule。TTL remains owner- + supplied；unrelated row updates never imply liveness。 +- Add one PeerManager with local capability/inbound registry、complete advertisement publication、database-time candidate + selection and `delegate(capability, payload, route_to_peer=None)`。 +- Add `core.peer.protocol.http.v1` outbound with peer JWT、normalized envelope and exact + `InkCre-Peer-Execution: not-executed` handling。No generic invoke route or readiness advertisement。 +- Compose fixed inbounds for `core.semantic_retrieval.v1`、`core.organization.rumination.v1` and exact-target + `core.extension.management.v1`。Each enters only the local non-delegating domain path。 +- Extension management uses one fixed POST command with a discriminated enable/disable/patch-config body and returns the + updated ExtensionModel。Remove the three legacy remote-management routes once every consumer is migrated。 +- Obtain absolute inbound bases only from owner-specific `peers.config.http_public_base_url`；publish no HTTP inbound when + absent。Remove CLIENT_BASE_URL/CORE_PUBLIC_URL projection and update CORS expose headers at the application boundary。 + +### Primary addresses + +- rename `app/schemas/client` → `app/schemas/peer` and `app/business/client` → `app/business/peer`，adding protocol/outbound + implementation inside the deep Peer package； +- update `app/settings.py` to peer terms and operational lease/maintenance bounds； +- new/reworked domain routes and `run.py` composition/bootstrap/shutdown； +- `app/middleware.py` and database JWT contract change the technical issuer from `inkcre-client` to `inkcre-peer`； +- migration renames `clients`→`peers` in place and extends it；database catalog/readiness/roles/seed/manifest follows the new + protocol； +- core deployment profiles remove legacy URL projection。 + +### Failure semantics + +- no candidate / invalid or expired target / unsupported protocol → pre-dispatch delegation unavailable； +- `route_to_peer` non-null never substitutes another Peer； +- only pre-dispatch or exact `not-executed` walks another candidate in any-provider mode； +- any domain response or outcome-unknown post-dispatch failure stops；both MVP capabilities remain conservatively + non-replayed； +- local domain implementation bypasses advertisement/outbound entirely；provider inbound cannot delegate recursively。 + +### Verification + +- empty current-head→new-head structural migration plus fresh-base equivalence；no retained Client row or Extension UUID- + array continuity claim； +- database-time lease/renew/clear and per-Peer TTL；capability snapshot replacement and config-derived URL composition； +- HTTP/JWT/envelope/header/CORS tests at application boundary；no real reverse-proxy smoke test； +- local bypass、two-Peer delegation、two-provider pre-dispatch/`not-executed` failover、target constraint and + outcome-unknown stop； +- Extension config validation/live apply and hot enable/disable through exact-target delegation。 + +## I7 — client-web Peer Protocol And Product Consumers + +### State diff + +- Sync the new core-owned database/runtime contract，then replace technical Client Active Record/module/config names with + Peer while retaining user-facing “client” wording where appropriate。 +- Delete `Client.request()`、ping/path construction and `rest_api_url` completely。Move JWT HTTP mechanics into + PeerHTTPOutbound and implement the same opaque delegate pipeline with optional `routeToPeer`。 +- Add remote-capable SemanticRetrieval and Organization domain facades；no local AI/scheduler implementation is required。 +- Migrate Extension config/enable/disable to `core.extension.management.v1` with the selected Peer reference。 +- Change the Clients administration implementation into a Peer view：edit owner config under config_schema and show + capability/lease facts without inventing another health endpoint。 +- Add the BlockDetailsPanel Ruminate action with pending/success/error；204 reloads graph through its existing owner， + outcome unknown asks the user to inspect/refresh and never auto-retries。 + +### Primary addresses + +- `packages/core/src/peer/` replacing `client/`；new business protocol/domain packages for Peer、organization and semantic + retrieval；generated database/runtime contract； +- `packages/core/src/config/**` technical `ClientConfig`/`INKCRE_CLIENT_ID` → Peer equivalents； +- `packages/core/src/extension/base.ts` and known consumer components； +- `apps/client-web/src/components/client/**` technical file/component rename as useful，while locale/product labels may stay + “client”； +- `BlockDetailsPanel`、graph view reload event and locales； +- update client-web ARCHITECTURE/FILESYSTEM/nearest AGENTS after structural changes。 + +### Verification + +- package-level Peer selection/envelope/JWT/failover/targeted-delegation tests； +- Extension target command tests with no arbitrary URL/path escape； +- component test for Ruminate pending/204 reload/error/outcome-unknown； +- generated database contract check、`pnpm check` and required builds； +- structural search for technical ClientRef/rest_api_url/Client.request leftovers，excluding intentional user-facing copy。 + +## I8 — Vertical Acceptance、Runtime Projection And Promotion + +### Corpus + +- real Memos API journey for short professional notes/comments/attachments； +- real RSS and Atom documents served by a controllable protocol double，including a real enclosure graph； +- a pinned repository snapshot of SQLite's official **Architecture of SQLite** document，served by the Acceptance HTTP + double and collected through the real HTML storage/resolver path before rumination。The official document is public + domain，professionally useful and structurally rich enough to expose compiler、VM、B-tree、pager、VFS and testing + sub-entities；the corpus manifest records source URL、retrieval date、digest and public-domain provenance； +- symbolic corpus aliases live only in the Acceptance manifest/harness and resolve real producer outputs/IDs after ingest。 + +### Gates + +- every accepted query has a `primary` in global top three and above every explicit distractor； +- one rumination journey creates a specific primary that enters top three and outranks the coarse source； +- fresh/stale/unavailable/failed maintenance and local/delegated runtime journeys satisfy D-189/D-190； +- deterministic control-flow tests remain ordinary CI；real provider semantic-quality/tool-calling journeys are explicit + credentialed Acceptance commands and never commit credentials or provider responses as authority； +- production/public-demo database is supplemental migration/exploration evidence only，not automated pass/fail authority。 + +### Promotion + +- update core-py local Unit TDD、security/deployment/runtime docs and nearest AGENTS to match implementation； +- capture shared PRD/Product-TDD changes in the packet，then use the canonical Hub shared-doc workflow after evidence； +- update client-web local architecture and shared-ref only through its own repository workflow； +- no Hub edit、shared-ref bump and Spoke code are combined in one commit。 + +### Execution evidence + +- the manifest pins the official SQLite Architecture HTML by digest and keeps all readable aliases inside the Acceptance + harness；a structural check rejects those aliases from production modules、schemas and migrations； +- the real producer vertical exercises Memos memo/comment/attachment、RSS and Atom collection、full-text hydration、RSS + enclosure download and PostgreSQL binary storage，then removes every produced graph/config/catalog row； +- the deterministic AI control vertical exercises the real Resolver draft Tools、Agent Tool-call loop、GraphForm submit、 + embedding maintenance and global Block/Relation ranking without replacing any producer、graph or retrieval boundary； +- the disposable PostgreSQL integration selection passes 31 tests with three intentional environment/protocol skips；the + complete Acceptance suite passes six tests with real DashScope `qwen3.6-plus` tool calls and `text-embedding-v4` + vectors，including all four quality judgments； +- the full core-py check passes 377 tests with 34 intentional skips；client-web's built browser artifact passes all four + real database/Core Peer journeys； +- deployment delivery now writes Peer-local public-base config into the database and waits for the exact three live + capability advertisements。This repairs stale `CLIENT_*`/`LLM_SP_*` deployment inputs discovered by the I8 runtime + rehearsal； +- core-py local Unit TDD and runtime/development docs now project the implemented topology。Hub source PRD/Product TDD + passes relative-link、diff and SVC-noop checks and is published as `95c4023`；core-py `cc8f90a` and client-web `8324293` + consume that exact commit through owner-separated pure shared-ref changes； +- the first restored-key run exposed a malformed `qwen-plus` ToolCall and plaintext provider-key representation in the + traceback。The adapter continued to reject malformed JSON；runtime config now models the credential with Pydantic's + secret type and unwraps it only at SDK construction。No adapter-specific test repeats the library's `repr` behavior，and + this local correction does not pretend to supply a repository-wide observability redaction boundary。The accepted run + uses the provider's current function-calling model rather than adding retry/fallback behavior； +- bounded ranking treats an omitted distractor as below the returned bound only after independently proving every judged + Block owns a fresh compatible embedding。This closes the harness error without enlarging the product result limit or + shaping production retrieval for the fixture。 + +## Migration And Cross-Repository Sequence + +1. Append a config/timestamp revision。 +2. Append AI/Agent/Profile/Embedding schema revision，dropping/recreating only legacy derived embedding relations。 +3. Append Peer protocol hard cut：rename `clients` in place、extend Peer fields、drop endpoint、add lease helper and update + protocol ACL/default-ACL/readiness contract。 +4. Verify both a fresh base→head build and an empty current-head→new-head transition on disposable PostgreSQL。No retained + data migration or downgrade is promised；the reviewed existing chain remains append-only rather than being squashed + into a second hard-cut baseline (D-195)。 +5. Commit/publish the core-owned database contract only when Sir authorizes。client-web then syncs generated types/runtime + contract from that exact artifact and implements I7；do not hand-edit generated PostgREST types。 +6. At delivery，dump canonical production outside the repository on WorkSSD，record its digest and verify a Neon recovery + branch；then rebuild only the exact canonical application schemas through the normal migration/contract-init path。 + Advance and sanitize the data-free preview baseline；there is no active staging target。Reconcile the production profile + only after runtime evidence reports the new exact contract/head。 + +The migrations may be regrouped if Alembic/autogenerate evidence shows one split would create an invalid intermediate +metadata state，but the three ownership boundaries above must remain visible in review and verification。 + +## Preflight Findings And Execution Resolution + +### Already established + +- current migration head is singular at `e1f4a5b6c7d8` and `pdm run check:migrations` passes； +- existing legacy vectors have no trustworthy migration identity and are safe only to rebuild； +- existing Client rows already carry UUID/name/labels/config/config_schema and therefore should be renamed/preserved，not + recreated； +- known `Client.request()` consumers are Extension config/enable/disable，now covered by D-191/D-192； +- `SubGraphForm` has broad extension/core use，so rename and producer conversion must be one coherent increment； +- local SVC database target is currently unavailable；provisioning it is an execution/preflight action after explicit + start，not a reason to change the design； +- installed SVC 11.0.1 is newer than adopted 10.0.1，but this unit does not require adoption or generated-guidance changes。 +- Resolver inventory confirms only exact `extensions.rss.feed_item.v1` has an approved breaking `get_text()` delta。 + Memos Attachment intentionally keeps filename-only text；the extra MIME decoration is retired with the embedding-only + method。Legacy Mail/GitHub/Telegram embedding augmentations do not automatically become general text semantics。D-193 + closes the version branch：the unreleased FeedItem v1 is corrected in place and disposable local data is reset。 +- installed OpenAI SDK 1.109.1 exposes dimensions-aware embeddings、Chat Completions/Responses tool inputs、nullable + `tool_choice` and parallel-tool-call controls。The provider-neutral `chat` operation remains adapter-owned；the initial + OpenAI-compatible translation can use the broadly compatible Chat Completions surface without making it a domain + contract。 +- canonical Neon production read-only evidence contains no RSS、Memos or Mail Blocks，so neither FeedItem nor Mail has + retained data to migrate。Mail is a producer-only correction。Production still contains legacy bare resolver IDs，which + also confirms the RSS/semantic-content contract has not been delivered there。 +- checked-in production discovery says contract v2 / migration `d0e3f4a5b6c7`，while the canonical production database + currently reports contract v1 / `d9f4e2a1b7c3`。This is a pre-existing deployment-truth discrepancy，not authorization + to mutate production or expand this unit。 +- client-web's local-core sync can generate the protocol snapshot and TypeScript projection before publication，but the + final canonical pin must pair the source commit with its digest-pinned GHCR image。OCI artifact publication is a + cross-repository reproducibility step，not a product release；it remains separately authorization-gated after the core + commit rather than weakening the pin or hand-editing generated types。 +- the Acceptance compound-document corpus is SQLite's official Architecture document。Its source and documentation are + declared public domain by SQLite；a pinned source snapshot，not generated graph/vector output，is committed as corpus + authority and served locally through the real collection boundary。 +- the current nine-revision migration chain is about 1,400 lines and contains verified database-runtime behavior；the + integrity checker is presently built around one linked hard cut。D-195 therefore chooses a clean shared-database rebuild + without squashing the chain into a risky monolith or expanding migration-history protocol solely for aesthetics。 + +### Closed during execution + +- generated Alembic DDL、protocol grants/sequences、schema-qualified PostgREST exposure and a clean disposable runtime were + verified before I7/I8； +- the Resolver execution checklist records exact `get_text()`/`get_label()`/create-graph obligations and the implemented + code uses D-194 identities； +- the repository/cross-repository Impact Handshake bounded core-py、client-web、runtime and durable-owner mutations； +- remaining work is evidence/promotion only：run the credentialed provider journey，then apply Hub/shared operations under + their own workflow and authorization boundaries。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/packet.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/packet.md new file mode 100644 index 0000000..3e7f678 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/packet.md @@ -0,0 +1,235 @@ +# Semantic Retrieval + +- **Unit ID**: `semantic-retrieval`。 +- **State**: **Closed；I0–I8、Acceptance and shared-truth promotion complete**。The implementation closure is core-py + `b80e5fd` plus client-web `ca4899c`；Hub `95c4023` is published，then core-py `cc8f90a` and client-web `8324293` + consume it through pure shared-ref commits。I0 provides + ConfigContract、deployment-scoped `configs`、honest PUT/PATCH routes、Extension config reuse and no-op-aware database-owned + timestamps for Blocks/Relations/Configs。I1 hard-cuts producer Forms from database Models，preserves recursive + `StarsGraphForm` authoring and adds signed-ID flat `GraphForm` normalization/submission。I2 adds shared AI facts、one + graph-blind AIManager and a real OpenAI-compatible embedding/chat/tool adapter while removing legacy sink/RAG coupling。 + I3 adds persisted Agent definitions and the replaceable in-memory Thread/Turn/Tool runtime。I4 hard-cuts exact Resolver + IDs/labels，adds directed Relation projection，profile-scoped maintenance/rebuild and exact globally ranked local + Block/Relation retrieval。I5 adds explicit focal-Block rumination，Resolver-owned on-demand draft contracts，three bounded + Agent Tools and additive GraphForm submission while removing the obsolete implicit Block organize hook。I6 hard-cuts + technical Client→Peer，adds structured capability snapshots、database-time leases、opaque exact-capability routing、the + normalized Peer HTTP/JWT protocol and three fixed business inbounds。I7 syncs the v3 contract into client-web，removes + technical Client request shortcuts，adds typed Peer HTTP/delegation、Extension exact-target commands and explicit + BlockDetailsPanel rumination。The client-web full check is green at 68 tests，the real browser/database/Core Peer journey + passes 4/4，and the built Chromium extension smoke passes 1/1。I8 now adds the pinned authoritative corpus、real + producer/storage/rumination/retrieval vertical、deployment Peer convergence and local durable projections。The complete + credentialed journey passes with DashScope `qwen3.6-plus` + `text-embedding-v4`，including real Agent tool calls、graph + submission and all four semantic judgments。The run also exposed and closed API-key repr disclosure plus one bounded- + ranking harness error；Hub PRD/Product-TDD truth and both Spoke references now match the verified implementation。 +- **Objective**: let a deployment owner submit a natural-language intent and receive ranked existing Block/Relation + matches，while a minimum organization rumination path can improve an info-base whose collected roots are too coarse for + useful retrieval。 +- **Guardrails**: graph rows remain information authority；retrieval returns Blocks/Relations rather than transient chunks + or generated answers；organization stops at ordinary graph materialization；embedding/profile/records are use-owned + derived support。Existing embedding/query/RAG code is evidence and failure material，not a compatibility surface。 +- **Primary evidence**: real Memos and RSS graphs，exact resolver/hydration/storage contracts，current pgvector and AI + split-brain implementation，and the relation-producer audit。 +- **Active mode**: closed。No product、technical、implementation、verification or promotion work remains in this unit。 +- **Next step**: return to the program packet and select another implementable unit。core-py branch push and production + mutation remain separately authorized delivery operations，not semantic-retrieval design work；client-web remote was + observed at `8324293` during final verification without a push issued by this workflow。 + +## Control Gates + +| Gate | State | Exit condition | +| --- | --- | --- | +| Product | Approved through D-185 | explicit BlockDetailsPanel action and empty completion response | +| Technical | Approved through D-196 | exact Resolver text/label matrix and clean shared-database rebuild close compatibility pressure | +| Acceptance | Approved through D-190 | corpus、quality、freshness and local/delegated Peer journeys closed | +| Implementation Plan | Approved | [exact dependency-ordered implementation plan](implementation-plan.md) maps code、migrations、cross-repo consumers and verification | +| Preflight | Closed for start | topology、SDK、Resolver、production、artifact、corpus and reset branches inspected；generated DDL is an execution-time evidence gate | +| Impact Handshake | Approved | [bounded cross-repo/runtime state diff](impact-handshake.md) names objects、blast radius、invariants、verification and uncertainty | +| Explicit Start / Execute | Granted | Sir said “开始”；execution resumes immediately after the requested clean baseline commit | +| Verify / Promote | Closed | core 377-test check and six-test credentialed Acceptance pass；Hub `95c4023` published first，then exact pure ref commits landed in both Spokes | + +Only this packet owns unit phase、gate and next step。Topic files own current task-state contracts；the +[decision register](../../decisions/index.md) alone owns decision history。 + +## Implementation Progress + +| Increment | State | Evidence / next boundary | +| --- | --- | --- | +| I0 config + timestamps | Closed | full repository check；real PostgreSQL config resource + timestamp tests；migration downgrade→upgrade；dev runtime metadata regression fixed | +| I1 producer forms + graph commands | Closed | full repository check；real PostgreSQL arbitrary/cyclic GraphForm、fresh negative-ID creation and recursive StarsGraph reconciliation/direction tests | +| I2 AI domain | Closed | full repository check；real PostgreSQL AI catalog/profile invariants；real OpenAI SDK protocol-double embedding/chat/tool wire proof；legacy split-brain/static remnants absent | +| I3 Agent runtime | Closed | deterministic completion/budget/concurrency/failure/cancellation/recovery checks；real PostgreSQL Agent definition、timestamp、constraint and active Thread snapshot proof | +| I4 semantic records/retrieval | Closed | exact resolver ID/label structural proof；real PostgreSQL unavailable scan、atomic batch、freshness、dimension、global ranking/default and route evidence；local DB reset removed unreleased legacy IDs | +| I5 organization rumination | Closed | full check；real PostgreSQL context + draft/submit and complete Organization→Agent→Tools repeated-additive journeys；route/no-op/config/budget/cancel proofs | +| I6 core Peer delivery | Closed | full check；migration downgrade→upgrade + roles/readiness；real PostgreSQL snapshot/lease/discovery；JWT/envelope/failover/target/CORS and three fixed inbound proofs | +| I7 client-web Peer consumers | Closed | v3 contract sync；technical Peer hard cut；68-test full check；real Web Peer 4/4 and Chromium extension 1/1 E2E | +| I8 vertical Acceptance | Closed | pinned corpus + alias isolation；real Memos/RSS/Atom/HTML/PostgreSQL producer vertical；real DashScope Agent/GraphForm/embedding quality 6/6；377-test full check；31-test real-DB suite；real Peer Web E2E 4/4；Chromium 1/1；Hub source projection validates | + +## Design Discussion Taste（guidelines，not contracts） + +These are local steering reminders extracted from repeated corrections。They help frame discussion but cannot override +evidence、an approved contract or a concrete requirement。 + +- **Authority and coordination before transport**: first identify the shared fact、its owner and how equal Peers coordinate + through the database。HTTP is an optional projection or an explicitly required request-response delegation protocol，not + the default architecture frame for creating/configuring durable facts。 +- **Approved signatures are anchors**: later diagrams and prose must reuse exact confirmed names/signatures，not casually + paraphrase them。The current Agent entry is `AgentManager.run(agent_id, initial_message)`。 +- **Separate owner schema from generic config mechanics**: an owner-defined Pydantic model owns the config value's shape。 + ConfigContract supplies generic model-driven mechanics；DeploymentConfigManager resolves the exact schema contract and + persists deployment-scoped rows。Do not describe the generic manager as owning the business schema。 +- **Do not strengthen invariants without value/evidence**: input constraints should match the actual authority and useful + product invariant。A referenced bigint Agent identity is `int` here；do not invent `PositiveInt` when neither the + database contract nor behavior requires that stronger rule。Existence remains a use-time reference check。 +- **Decide proof ownership before adding a test**: do not turn every incident into a regression test。Test behavior owned + by InKCre when a legitimate implementation change could violate a valuable contract；prefer black-box evidence for that + boundary。When an incident came from bypassing or misusing a mature library/abstraction and the corrected type、module + boundary or static mechanism already makes the rule structural，repair that baseline instead of retesting the dependency。 + If the missing owner is shared infrastructure such as observability redaction，record that pressure and eventually test + the shared boundary once，not each secret field or adapter。 + +## Unit-local Anti-patterns + +- Do not create a `RuminationManager` or organization-approach registry for the currently small rumination operation；keep + it as `OrganizationManager.ruminate()`。This is a judgment about this unit's present topology，not a general discussion or + module-design rule。 + +## Product Contract + +### Semantic retrieval + +- `SemanticRetrievalManager.retrieve()` is the single domain entry。It returns one bounded、globally ranked list of real + Blocks/Relations plus score metadata；it does not generate an answer。 +- Direct search、Chat InKCre or another Agent may consume the same capability。The client-web landing-page natural-language + search is not the primary journey or Acceptance authority。 +- Query is text。MVP uses one explicitly selected or deployment-default EmbeddingProfile、exact cosine comparison、optional + score threshold、a maximum result bound and no pagination。 +- A local peer may execute exact capability `core.semantic_retrieval.v1`；a peer without a local implementation delegates + that same capability through PeerManager。Discovery and invocation remain separate，and no generic capability invocation + endpoint or delegation job is introduced。 + +### Organization pressure + +- Semantic granularity matters，but this unit does not create a BlockSegment/chunk information layer。A concrete + `rumination` approach reconsiders one focal Block in its direct graph context and may add useful ordinary Blocks and + Relations。 +- The source Block and existing Relations remain unchanged。MVP does not replace/delete the source，retry、roll back、run a + fallback pipeline or promise that every rumination mutates the graph。 +- `breakdown` is historical/product shorthand，not a code/domain abstraction。`interpretation` is only the fallback + persisted relation from the source to a representative derived-graph entry。 +- MVP rumination uses resolver `get_text()` plus a bounded direct-relation snapshot，Resolver-owned on-demand schema / + rooted GraphForm drafting Tools and one mutating `submit_graph(GraphForm)` Tool。It does not expose graph-reading/ + navigation Tools。 +- `OrganizationManager.ruminate()` awaits the active Agent Turn and exposes shallow `None` completion rather than the + internal Thread。Cannot-understand/no-write completes normally；budget exhaustion is one high-level organization failure； + caller cancellation propagates to the Turn without a job/run entity。 +- Repeated rumination is an independent additive attempt over the latest direct-relation snapshot and may duplicate graph + facts。That snapshot may reveal a prior result but cannot prove its freshness or whether rerumination is needed；MVP adds + no run record、fingerprint、freshness dependency or pre-execution skip。 +- Rumination is triggered only by an explicit request for one focal Block。MVP has no collection hook、new-Block event、 + periodic scan、batch candidate selector or organization job。 + +## Confirmed Technical Topology + +```text + deployment configs + | +AIProvider -> AIModel -> EmbeddingProfile + | | | + | +-> AIManager.chat/embed() + | | \ + | | -> AgentManager -> Thread -> rumination -> GraphForm -> InfoBaseManager + | | +Block/Resolver + RelationManager projection + -> EmbeddingRecord maintenance + -> SemanticRetrievalManager local retrieve + | + +-> PeerManager.delegate(exact capability) + -> Peer Protocol outbound -> provider inbound -> local retrieve +``` + +| Design surface | Confirmed owner/boundary | Current source | +| --- | --- | --- | +| AI routing | shared AIProvider/AIModel facts；peer-local AIManager + dialect adapters；typed `embedding`/`chat` capabilities | [AI and projection](technical-design/ai-and-projection.md) | +| Semantic projection | Resolver owns Block `get_text()`/`get_label()`；RelationManager owns directed endpoint-label + content projection | [AI and projection](technical-design/ai-and-projection.md) | +| Resolver execution | exact in-scope IDs、complete text、Block-local label and producer obligations | [Resolver execution checklist](resolver-execution-checklist.md) | +| Vector space/records | mutable EmbeddingProfile + Block/Relation EmbeddingRecords；timestamp-based best-effort freshness | [Embedding profiles](technical-design/embedding-profiles.md) | +| Maintenance/config | SemanticRetrievalManager maintain/rebuild；`ConfigContract` mechanics + deployment-scoped `configs`/DeploymentConfigManager | [Maintenance and config](technical-design/maintenance-and-config.md) | +| Retrieval | ranked Block/Relation result、score、filters and bounded top-k contract | [Retrieval contract](technical-design/retrieval-contract.md) | +| Peer delivery | Peer capability snapshot + per-peer lease + protocol descriptor + one-shot HTTP delegation/failover | [Peer delegation](technical-design/peer-delegation.md) | +| Exact-target consumer migration | `PeerManager.delegate(..., route_to_peer=PeerRef)` constrains one exact Peer；`core.extension.management.v1` replaces known remote `Client.request()` consumers without alternate-Peer failover | [Peer delegation](technical-design/peer-delegation.md#exact-target-delegation-and-extension-management-approved-through-d-192) | +| Graph commands / rumination | Resolver/extension StarsGraphForm authoring；Agent runtime validates once，`draft_graph` adapts，Resolver creates，InfoBase normalizes/submits，PostgreSQL owns FK integrity | [Rumination and graph boundary](technical-design/rumination-agent-graph.md#current-boundary-ledger) | +| Agent runtime | persisted Agent definition、in-memory Thread persistence backend、cancellable per-turn Task、concurrent ToolCalls and single-writer closed message pairs | [Agent runtime](technical-design/agent-runtime.md) | +| Rumination selection | organization-owned deployment config references one AgentDefinition；`OrganizationManager.ruminate()` resolves it at use time and calls exact `AgentManager.run(agent_id, initial_message)` | [Rumination and graph boundary](technical-design/rumination-agent-graph.md#rumination-selection) | +| Shared timestamps | selected shared protocol rows use PostgreSQL `BEFORE UPDATE` timestamps | [Row timestamps](technical-design/shared-row-timestamps.md) | + +The dependency-ordered implementation decomposition is tracked separately in the +[delivery map](delivery-map.md)；it is a design probe，not an execution baseline。 + +## Active Execution Queue + +Execute the approved dependency order，keeping each increment independently reviewable and evidenced。 + +1. **Closed**: repository ownership was preserved：Hub `95c4023` was published first，then each Spoke committed only its + gitlink to that exact pushed hash。No further product/technical implementation increment remains in this unit。 + +Producer grammar is closed through D-179。Base Forms omit database-managed state；flat GraphForm uses signed IDs for one- +command creation/reference；StarsGraphForm remains the recursive Resolver/extension authoring representation；the current +[boundary ledger](technical-design/rumination-agent-graph.md#current-boundary-ledger) owns validation/create/normalize/write +responsibilities。D-175–D-177 are correction history，not contracts to merge。 + +Thread/Tool lifecycle is no longer open：D-171 makes the Turn Task the structured-concurrency owner；individual ToolCalls +run concurrently，ordinary failures are isolated，abort cancels unfinished calls，and only a closed AssistantMessage + +ToolResultMessage pair is atomically committed。 + +## Acceptance Direction + +- Corpus authority is deterministic source data exercised through real producers/runtime into disposable PostgreSQL：real + Memos API journeys，real RSS/Atom served by a protocol double and a real Resolver/rumination compound document。Do not + hand-insert the target graph as an internal fixture；production/demo data is supplementary exploration only。 +- Prefer authentic、substantive and approachable software/AI/knowledge-systems material that the intended user might + genuinely save；toy prose is not the primary semantic-quality authority。 +- Invoke semantic retrieval directly，not through generated Chat/RAG answers。 +- Use committed PostgreSQL Memos/RSS graphs plus a long/compound source that requires rumination to expose useful semantic + units。Expected matches are judgments about entity identity/rank/coverage，not one provider's exact floating score。 +- Each query classifies explicitly named real entities as `primary`、`relevant` or selected `distractor`；unjudged entities + remain unjudged。Readable corpus references are Acceptance-harness aliases resolved to actual IDs only after real + ingestion；they must not appear in production schemas、models、payloads、domain APIs or runtime code paths。 +- Every accepted query must place at least one `primary` in the global top three and above every explicit `distractor`。 + `relevant` entities are not mandatory recall。Aggregate metrics and exact provider scores may diagnose but do not own + pass/fail；one rumination journey additionally proves that its new specific entity enters the top three and outranks the + coarse source。 +- Candidate maintenance is explicit：after real Block/Relation dependency updates，stored stale records must be excluded + until `maintain` replaces them；`retrieve` never repairs candidate records implicitly。Unavailable projection must not + starve later candidates，and failed provider execution must not write an invalid record。Scheduler evidence verifies + wiring to the same maintain operation rather than waiting on wall-clock time。 +- Peer Acceptance proves local bypass and exact-capability delegation through real HTTP/JWT/codecs，plus expired/ineligible、 + pre-dispatch、`not-executed` and outcome-unknown branches。Both capabilities conservatively stop after possible execution； + no retrieval-specific replay policy is added。Standards evidence plus application-boundary header/CORS assertions are + sufficient；do not add a real reverse-proxy smoke deployment solely for this unit。 +- Prove resolver projection、record freshness/invalidation、maintenance and retrieval through real runtime boundaries；use + static checks for schema/type/registration invariants。 +- Exercise both local execution and one delegated peer path through the real protocol/proxy boundary，including the exact + protocol-guaranteed non-execution failover distinction。 +- Keep the result bound small。A need for pagination or routinely useful results beyond the bound is retrieval-quality or + graph-preparation pressure，not an automatic pagination requirement。 + +## Supporting Material + +- [Evidence](evidence.md): current code/runtime facts and failure topology。 +- [Technical design index](technical-design/index.md): current topic contracts and active technical edge。 +- [Relation producer audit](relation-producer-audit.md): repository-owned directed-relation corrections that must occur at + producer/migration boundaries，never inside retrieval or organization runtime。 +- [Delivery map](delivery-map.md): dependency topology and provisional implementation increments。 +- [Implementation plan](implementation-plan.md): exact dependency-ordered increments、migration/cross-repo sequence and + preflight ledger；draft until final review。 +- [Program packet](../../packet.md): program boundary and delivery loop。 + +## Explicit Non-Decisions + +- No transient chunks/segments、ANN/HNSW index、cross-profile fusion、generic AI service proxy or global registry service。 +- No answer generation、Chat InKCre product behavior、feature retrieval or graph-navigation retrieval in this unit。 +- No generic `/capabilities/{id}/invoke`、delegation job、readiness advertisement or capability-aware PeerManager。 +- No persistent Thread backend、Turn/ToolCall/ToolResult tables、checkpoint/resume or Agent-owned exactly-once guarantee。 +- No claim that every Block/Relation can produce a text embedding；availability is profile/executor-relative。 +- No legacy `Client(rest_api_url).request()` escape path after Peer delegation lands；direct database Active Records may + remain where they own shared facts，but callable business capabilities route through their domain facade + PeerManager。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/relation-producer-audit.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/relation-producer-audit.md new file mode 100644 index 0000000..e207a6e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/relation-producer-audit.md @@ -0,0 +1,68 @@ +# Relation Producer Audit + +- **Purpose**: check repository-owned Relation writers against D-097's directed dynamic-property contract before their + rows become semantic-retrieval candidates。 +- **Invariant**: `from` is subject、`content` is property/behavior/role、`to` is value/object。Persisted direction is graph + authority；SemanticRetrievalManager does not reinterpret or repair it。 +- **Correction owner**: original collection/graph producer；a narrowly evidenced schema migration is added only when + retained historical rows actually exist and can be identified exactly。Never organization、resolver read、retrieval + query or a background repair loop。 +- **Search evidence**: ast-grep inventory of `RelationModel(...)`、`RelationManager.create/update(...)` and + `OutArcForm`/`InArcForm` calls，plus raw migration SQL search。Generic persistence APIs are separated from business + producers below。 + +## Confirmed Business Producers + +| Producer address | Persisted assertion | Verdict | Implementation consequence | +| --- | --- | --- | --- | +| `extensions/memos/family/graph.py` | comment memo → `parent` → parent memo | correct | none；parent is the comment's parent | +| `extensions/memos/family/attachment.py` | memo → `attachment:<order>` → attachment metadata | correct | none；slot content remains extension-owned grammar | +| `extensions/memos/family/attachment.py` | attachment metadata → `content` → semantic content | correct | none | +| Memos reference grammar | memo → `reference` → referenced block | correct grammar；no current product writer | preserve direction when a writer is added | +| `extensions/rss/repository.py` | feed item → `feed` → feed root | correct | none；feed is item's membership/identity scope | +| `extensions/rss/repository.py` | feed item → `enclosure` → enclosure metadata | correct | none | +| `extensions/rss/enrichment.py` | feed item → `full_text` → text block | correct | none | +| `extensions/rss/enrichment.py` | enclosure metadata → `content` → semantic content | correct | none | +| `app/business/info_base/resolver/image.py` | image → `alt:text` → text block | correct | none | +| `extensions/github/resolver.py` | GitHub user → `owns` → repository | correct direction | keep predicate wording；Relation projection is structured subject/property/value，not forced English possessive prose | +| `extensions/mail/resolver.py` | email address → `from` → email | **reversed** | producer must emit email → `from` → sender address；no retained production row exists to migrate | +| `extensions/mail/resolver.py` | email → `to` / `cc` → address | correct | none | +| `extensions/twitter/bookmark.py::tweet_to_graph` | tweet → attachment / URL role → media/HTML | correct | none | +| `extensions/twitter/bookmark.py::_organize` | tweet → `bookmarked for` → reply text | **wrong owner/vocabulary** | do not retain graph mutation in legacy organization hook；future note collection needs an explicit collection/command owner | + +## Generic Writers + +- `app/routes/relation.py` and `RelationManager.create/update()` persist explicit caller-authored from/to/content。They + have no source semantics from which to infer reversal；validate referential/schema integrity only。 +- `InfoBaseManager` maps `OutArcForm` and `InArcForm` exactly as declared。Its direction mechanics are correct；business + producers own the chosen arc form。 +- `Resolver.breakdown()` currently has no concrete Relation-producing implementation。Future organization breakdown + must emit the same invariant，but this audit does not create a runtime admission/repair layer。 + +## Planned Corrections（not implemented） + +### Mail sender + +1. Change `EmailResolver.create_graph()` from sender `InArcForm` to sender `OutArcForm`。 +2. Update its Mermaid/docstring and add graph-shape verification，not schema/helper-only tests。 +3. Do not add a speculative data migration：the canonical production read-only sample contains no Mail Blocks and local + development data is disposable。If another retained deployment is later evidenced，inspect it before choosing a + migration rather than assuming the current zero-row fact applies globally。 + +### Twitter bookmark note legacy hook（approved） + +`SourceBase._organize()` is documented as a legacy no-op hook and has no production caller；only a direct unit test invokes +this implementation。Hard-cut the graph-writing body back to an explicit no-op and remove the test that treats it as a +feature。Do not move reply fetching into organization or add a semantic-retrieval repair path。Whether a future Twitter +collector persists bookmark notes—and under which exact relation vocabulary—is owned by a later Twitter source design。 + +Historical `bookmarked for` rows cannot be safely renamed merely from the generic relation string。A migration may act +only if endpoint resolver evidence and the intended new Twitter note grammar are approved together；this unit currently +does not invent that future grammar。 + +## Verification Boundary + +- static/structural inventory proves no repository-owned constructor path was omitted； +- focused graph tests prove exact from/content/to for corrected producers； +- black-box semantic retrieval later returns stored Relation identity/direction unchanged；it does not serve as the + mechanism that repairs producer defects。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/resolver-execution-checklist.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/resolver-execution-checklist.md new file mode 100644 index 0000000..f8f38ac --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/resolver-execution-checklist.md @@ -0,0 +1,113 @@ +# Resolver Execution Checklist + +> **Status**: implemented through I5。D-075、D-096、D-102、D-193 and D-194 own the underlying decisions；this matrix is +> now enforced by exact-ID、abstract capability、projection、producer and draft-capability tests。 + +## Shared Label Contract + +`get_label()` returns a concise、stable、non-localized graph reference，not a type ontology、UI title or full semantic +projection。 + +```text +<resolver-owned kind> <identifier> + +feed <Hacker News> +github user <octocat> +text <A bounded first-line excerpt…> +``` + +- Every concrete Resolver implements the method；it never raises unsupported-capability merely because an identifier is + absent。`<resolver-owned kind>` alone is the valid fallback。 +- Label evaluation is Block-local。It may hydrate/decode/inspect the focal Block but cannot traverse Relations、invoke AI + or materialize another Block。A Resolver whose ordinary solved projection is graph-aware must provide a local label path。 +- Optional identifiers normalize whitespace。Free text uses the first non-empty logical line，collapses internal + whitespace and truncates after 96 Unicode code points with one `…`。Structured identifiers such as a repository full + name、email address、source-native ID or filename are not truncated unless they exceed the same 96-code-point safety + bound。 +- Exact resolver IDs do not enter the label。The readable kind below is directly owned by each Resolver；there is no + resolver-type table、friendly-name registry or localization layer。 +- Changing a retained label format incompatibly advances the exact Resolver contract because Relation embedding freshness + otherwise cannot observe the semantic-input change。 + +## Core Semantic Content + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `core.text.v1` | hydrated Unicode text | `text <first-line excerpt>`；fallback `text` | StarsGraph root from text | +| `core.html.v1` | rendered document text | `html <title>`；fallback first heading，then `html` | StarsGraph storage-backed root | +| `core.image.v1` | unsupported | `image` | optional `alt:text` child | +| `core.audio.v1` | unsupported | `audio` | storage-backed root | +| `core.video.v1` | unsupported | `video` | storage-backed root | +| `core.pdf.v1` | unsupported | `PDF <metadata title>`；fallback `PDF` | storage-backed root | +| `core.epub.v1` | unsupported | `EPUB <package title>`；fallback `EPUB` | storage-backed root | +| `core.zip.v1` | unsupported | `ZIP` | storage-backed root | +| `core.file.v1` | unsupported | `file` | storage-backed root | + +Unsupported text remains an explicit `UnsupportedResolverCapability`。Typed byte facts do not pretend to be extracted +document text；PDF/EPUB title metadata is useful for a concise label without claiming full-text capability。 + +## Memos + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `extensions.memos.memo.v1` | memo body | `memo <body excerpt>`；fallback `memo` | StarsGraph root；parent/reference/attachment remain graph relations | +| `extensions.memos.attachment.v2` | filename only | `memo attachment <filename>` | metadata → `content` → semantic content | + +Attachment MIME is intentionally absent from both general text and label。It remains canonical metadata/resolver-selection +evidence rather than an embedding-specific decoration。 + +## RSS / Atom + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `extensions.rss.feed.v1` | title + description | `feed <title>`；fallback configured URL | feed root | +| `extensions.rss.feed_item.v1` | title + summary + full text or authored content | `feed item <title>`；fallback native ID、alternate URL or authored excerpt | item → feed/enclosure/full_text | +| `extensions.rss.enclosure.v1` | enclosure title or `None` | `feed enclosure <title>`；fallback URL | enclosure metadata → optional `content` | + +FeedItem v1 is corrected in place before release。Its label decodes only `CanonicalFeedItem` from the focal Block and does +not call the graph-aware solved projection。 + +## Mail + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `extensions.mail.email.v1` | subject + plain-text/HTML body | `email <subject>` | email → from/to/cc → address | +| `extensions.mail.newsletter.v1` | subject + body | `newsletter <subject>` | newsletter root | +| `extensions.mail.email_address.v1` | display name + address，or address | `email address <name / address>`，or address | address root with source-owned reuse | + +Subject is part of the generally useful Email/Newsletter projection，not a model-specific augmentation。Mail sender +direction is corrected only in the producer because no retained malformed rows exist。 + +## GitHub + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `extensions.github.repo.v1` | full name、description、language and topics when present | `github repository <full_name>` | user → `owns` → repository | +| `extensions.github.user.v1` | display name + login，or login | `github user <login>` | user root with source-owned reuse | + +Language/topics are stable repository facts with general use value；they move into the single text projection rather than +surviving as an embedding-only hook。 + +## Other Built-in Extensions + +| Exact ID | `get_text()` | `get_label()` | Producer obligation | +| --- | --- | --- | --- | +| `extensions.telegram.message.v1` | text/caption plus media kind when present | `telegram message <text/caption excerpt>`；fallback native message ID | message root | +| `extensions.twitter.tweet.v1` | tweet text | `tweet <text excerpt>`；fallback native tweet ID | tweet → attachment / URL entity；legacy `_organize()` is no-op | +| `extensions.learn_english.lexical.v1` | lexical text | `lexical item <text>` | lexical root | + +## Structural Verification + +- static registry table proves the exact ID set and rejects every retired ID； +- abstract-method/type checks prove every concrete Resolver implements `get_text()` and `get_label()` explicitly； +- focused black-box cases prove text、label and unsupported/null distinction from hydrated inline and storage-backed Blocks； +- graph-aware Resolvers receive direct Relations that would alter solved content，while label remains unchanged，proving + Block-local evaluation； +- structural search proves `get_str_for_embedding()` has no declaration、implementation or consumer； +- producer parity runs every StarsGraphForm through `InfoBaseManager.normalize_graph()` and checks accepted relation + direction/grammar。 +- graph drafting is opt-in rather than inferred from decode ability；`core.text.v1` and any loaded explicit extension draft + capability publish code-owned description/input schema，while source-native RSS/Memos/GitHub decoders are not automatically + exposed as LLM authoring contracts； +- bound Agent Tool schemas freeze the exact draft Resolver ID set，keep native input details out of `draft_graph`，and expose + them only through `get_draft_graph_schema`；Pydantic performs the selected nested input validation before handler entry。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/agent-runtime.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/agent-runtime.md new file mode 100644 index 0000000..9e0f7ce --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/agent-runtime.md @@ -0,0 +1,134 @@ +# Agent Definition、Thread And Runtime + +> [Technical design index](index.md) + +AgentManager owns FastAPI-like typed Tool input validation。The decorator derives the LLM-visible JSON schema from the +handler's Pydantic input annotation and runtime parses call arguments before invoking the function。Invalid input becomes a +tool-result message by serializing Pydantic's own `ValidationError`，with only a thin transport wrapper if necessary；do not +invent another error schema。Tool return annotations are enforced by static checking only。AgentManager does not perform +runtime output validation or publish an output schema；serialization failure is an implementation failure。 + +## T-019 Agent Definition、Thread And Runtime Tool Binding（approved through D-172） + +```text +persisted Agent definition + id + name + system_prompt + tools + nullable tool_choice + model + max_model_calls_per_turn + | + | AgentManager.run(agent_id, initial_message) + v +SystemMessage(agent.system_prompt) + | + v +Thread.from_agent_definition(definition, messages=[SystemMessage]) + snapshots model + tools + tool_choice + max_model_calls_per_turn + stores canonical Messages through thread persistence backend + | + v +start_turn(input) -> asyncio.Task[TurnTermination] + appends UserMessage and runs model/tool loop + +peer-local AgentManager tool registry + exact tool ID -> code-owned input-contract binder + handler +``` + +### Agent creation and per-turn bound + +Agent definition owns required positive `max_model_calls_per_turn`。Each `start_turn()` resets that budget，and every +attempted `AIManager.chat()` consumes one unit；ToolCalls、tool executions and Messages do not。An AssistantMessage without +ToolCalls ends the turn naturally even when it has no text。When AssistantMessage has ToolCalls，execute that complete batch +and make another model call only when budget remains。 + +`AgentManager.run(agent_id, initial_message)` loads one AgentDefinitionModel，materializes +`SystemMessage(agent.system_prompt)` before entering Thread，constructs Thread from the complete definition model and starts +the first turn。It returns the active Thread immediately rather than awaiting that turn。 + +Per-turn outcome is owned by the Task： + +```text +asyncio.Task[TurnTermination] + completed | max_model_calls + cancelled Task = caller abort +``` + +`completed` means an AssistantMessage returned without ToolCalls。When another model call would exceed the budget，return +`max_model_calls` without an exception or cleanup call。Preserve completed Tool side effects；do not retry、roll back or +compensate。A cancelled Task represents abort。Thread is the history/result handle；there is no AgentRunResult or persisted +Turn entity。 + +One model turn may emit multiple ToolCalls。They form an order-independent execution batch correlated by call ID；their +presentation order is not an execution dependency。The Turn schedules one child Task per call and executes them +concurrently。Each call converts ordinary validation/handler failure into its own result，so siblings continue；Turn +cancellation propagates to unfinished calls。Do not `shield()`、retry or roll back the batch or its successful effects。 +There is no redundant batch-level child Task：the Turn Task already owns the structured-concurrency scope and directly +awaits the complete batch。 + +After the execution barrier，construct exactly one `ToolResultMessage(results=[...])`。Its nested ToolResult call IDs are +unique and exactly cover the preceding AssistantMessage's ToolCalls；array order has no meaning。Tool execution never +writes history。The Turn runtime is the sole writer and asks the thread persistence backend to atomically append the +AssistantMessage + ToolResultMessage closed pair before any next model call。Abort before that commit persists neither +half；completed Tool side effects remain。ToolCalls stay nested in AssistantMessage and ToolResults stay nested in +ToolResultMessage，so neither becomes an independent persisted history entity。Dialect adapters own provider-specific +splitting/grouping。 + +ToolResult has no common error DTO beyond `is_error` and JSON `content`。Pydantic argument failure uses ValidationError's +native JSON value。A handler may raise thin `ToolExecutionError(content)` for Tool-owned actionable failure content。An +unexpected ordinary Exception becomes stable generic error content while its complete exception/traceback is logged；never +make provider-facing behavior depend on database/framework error strings。Successful content is the handler return value's +JSON serialization。All three failure paths remain per-call and do not stop the rest of the batch。 + +Thread snapshots nullable protocol-level `tool_choice` from AgentDefinitionModel and offers no per-turn override。`null` +means unspecified and causes the dialect to omit the provider control；a non-null unsupported value fails before the +provider request。The rumination system prompt owns the product instruction：use schema discovery/drafting only to construct +a potentially useful interpretation，call `submit_graph` only when that result is meaningful，and otherwise end honestly +without a graph write。No ToolCall remains a valid no-op and natural completion，not an error；discovery/draft calls followed +by no submit also leave the info-base unchanged。 + +When AssistantMessage has ToolCalls，execute the complete batch；text presence is irrelevant to that condition。Append the +complete ToolResultMessage first。Only after it is ready may a UserMessage be appended for an actual new-user-input need； +the ordinary loop inserts no synthetic UserMessage before its next model call。 + +Agent definition remains system-prompt authority。AgentManager.run converts it to the first SystemMessage before Thread +construction；Thread does not retain another system_prompt field。Rumination-owned focal/context data enters only through +the UserMessage passed to `start_turn()`，and AIManager has no separate prompt parameter。 + +- Agent definitions are reusable database facts。The exact persisted `agents` fields are database-generated bigint `id`、 + required descriptive `name`、required `system_prompt`、`tools: text[]`、nullable `tool_choice`、required `model`、required + positive `max_model_calls_per_turn` and database-owned timestamps。Do not add enabled、description or generic config。 +- `AgentManager.run()` takes only `agent_id` and one `initial_message: UserMessage`。Thread is an internal Agent module，so its constructor accepts + the complete AgentDefinitionModel and owns the snapshot projection；external callers do not pass flattened model/tool/ + policy parameters。 +- Thread snapshots model、tools、tool_choice and max_model_calls_per_turn。System prompt is not a snapshot field because run + has already materialized it into the initial SystemMessage。 +- The **thread persistence backend** is replaceable and owns whole Thread state。MVP provides only an in-memory + implementation；do not add database Thread/Message persistence now。 +- Thread exposes runtime-only `current_turn`。`start_turn(input)` schedules the whole turn coroutine as one asyncio Task， + appends the UserMessage and returns the Task。Run starts the first Task and returns Thread immediately。One Thread rejects a + second active turn until `current_turn.done()`。 +- Before appending a new UserMessage，`start_turn()` may remove exactly one trailing AssistantMessage whose ToolCalls have + no following ToolResultMessage。This is defensive persistence recovery，not normal loop state。Never erase only the + ToolCalls or rewrite an older ambiguous history segment。 +- Thread history supports atomic multi-Message append。AssistantMessage + ToolResultMessage is the minimum closed Tool + interaction commit；individual Tool execution tasks return values and never become history writers。 +- Tool IDs have set semantics。Persistence order is non-authoritative and duplicates are rejected。 +- Canonically sort tool IDs before persistence so semantically identical sets do not create false row updates。 +- Agent behavior includes prompt、tools、tool_choice、model and per-turn model-call budget。Existing Threads do not reread + later Agent-definition changes。 +- Schema and handler form one runtime Agent Tool contract。The callable remains code-owned；the persisted definition only + references its exact ID。 +- Most Tool binders simply derive one static input model/schema from the decorated handler annotation。A bounded Tool may + instead materialize a code-owned Pydantic input contract from current domain-manager facts when the schema itself is a + runtime projection。D-172's exact available Resolver-ID enum is the first pressure；this is not permission for arbitrary + prompt-time schema mutation or a universal reflection registry。 +- `AgentManager.run()` resolves the persisted Tool IDs and freezes their bound contracts for the new Thread。Later registry + or extension changes do not rewrite that Thread's Tool schemas；a now-unavailable handler execution fails as an ordinary + Tool call rather than silently selecting another Resolver。 +- AgentManager owns a decorator for function-based Agent Tool registration。Other registry-owning Managers should converge + on the same visible decorator pattern without creating a global Registry service or erasing their domain-specific rules。 +- A future persistent **thread persistence backend** stores the AI-module-owned canonical Message union as part of Thread + state，including nested tool calls/results；it does not create independent ToolCall、ToolResult or Turn persistence + entities。 +- No checkpoint/resume model is introduced or reserved。AgentManager offers no exactly-once promise；a handler owns any + tool-specific idempotency/exactly-once mechanism。 +- A run uses the loaded definition snapshot。A missing runtime binding for one persisted exact tool ID ends the run with one + high-level Agent failure；there is no readiness、fallback or hidden tool substitution。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/ai-and-projection.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/ai-and-projection.md new file mode 100644 index 0000000..2f28bcc --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/ai-and-projection.md @@ -0,0 +1,393 @@ +# AI Routing And Semantic Projection + +> [Technical design index](index.md) + +## T-006 AIDialect Type And AIProvider Instance（approved by D-091） + +```text +AIDialect shared type catalog + id: exact versioned text + description + config_schema: JSON Schema + ↑ AIProvider.dialect FK +AIProvider configured instance + id: database-generated bigint + name: text + dialect: exact dialect ID + config: typed JSON + enabled: boolean + created_at / updated_at: database-owned +``` + +- This deliberately follows source type → source instance and storage type → storage instance。 +- The peer-local `AIManager` binds an installed adapter implementation to the same exact dialect ID。A shared + catalog row says the dialect contract exists；it does not prove every peer implements it。 +- `enabled=false` rejects new operations through this provider while preserving provider config、models、profiles and + existing embedding records。 +- Delete is restricted while AIModel children exist。Name is administrative，not identity；duplicate display names do + not define routing。 +- `config_schema` belongs to the dialect type；validated config belongs to the provider instance。Exact registration/ + schema-authority behavior across peer implementations remains implementation preflight，not a reason to duplicate + schema on each provider。 + +## T-007 AIModel Shape And Identity（approved by D-092） + +```text +AIModel + id: database-generated bigint + provider: AIProvider bigint FK + native_model_id: text + name: nullable text + capabilities: typed JSON + enabled: boolean + created_at / updated_at: database-owned + + UNIQUE (provider, native_model_id) +``` + +Each typed capability item has this common shape（D-159）： + +```json +{ + "type": "chat", + "input_modalities": ["text"], + "output_modalities": ["text"], + "features": ["tool_calling"] +} +``` + +`features` is persisted as a JSON string array with unordered、duplicate-free set semantics and canonical ordering。Feature +IDs are capability-scoped exact strings；MVP adds only `tool_calling`。Do not use `extra_supports` or create a feature +registry/table。 + +`tool_calling` has one provider-neutral meaning：accept caller-supplied Tools with structured input schemas，emit +structured ToolCalls，then consume ToolMessages containing caller-side execution results。`function` remains available as +a concrete provider/dialect tool kind；it is not the canonical feature name。This feature does not assert support for +provider-managed built-in tools、MCP or every future Tool kind。 + +- Shared PostgreSQL is the creation/concurrency authority，so UUID supplies no required distributed property。Identity + uses `bigint GENERATED BY DEFAULT AS IDENTITY` or the migration-equivalent sequence-backed mechanism。 +- Nullable `name` is descriptive only；UI falls back to `native_model_id`。 +- Provider/native-model identity is immutable。Changing the actual model creates a new AIModel and updates the consuming + Profile，whose timestamp invalidates older records。 +- Capabilities/name/enabled are mutable model administration/declaration。Disabling rejects new calls while preserving + profiles and embedding records；delete is restricted while profiles refer to the model。 + +## T-008 AIManager Boundary And EmbeddingProfile Shape（approved by D-093/D-094） + +- `AIManager` is the sole domain-level manager。Its internal responsibilities include dialect-adapter registration、type + catalog sync、Provider/Model management and typed capability routing/execution。 +- Do not expose AIDialectManager or a parallel AIService。An internal map or client-web AI SDK ProviderRegistry remains an + implementation detail until it proves independent lifecycle/policy/reuse pressure。 +- EmbeddingProfile exact MVP fields are bigint identity、nullable `name`、`ai_model` FK、required positive + `dimensions` and database-owned timestamps。 +- `dimensions` is deterministic configuration，not nullable provider-default behavior。The adapter validates returned + vector length before persistence；provider error responsibility does not justify admitting a cheaply detectable broken + record。 + +### Capability modules versus dialect adapters（approved by D-157） + +AI implementation is partitioned by two orthogonal axes： + +```text +capability modules + embedding + chat + +dialect adapters + core.openai-compatible.v1 + -> implements every installed capability supported by that dialect/model +``` + +Do not create capability-prefixed dialect adapters such as `chat-openai-compatible` or +`embedding-openai-compatible`。Provider config/client construction and wire conventions belong to one dialect adapter； +typed embedding/message inputs and outputs belong to their capability modules。 + +The exact second capability/module/operation name is `chat`。It accepts canonical Messages plus optional Tools and returns +an AssistantMessage。It does not imply Chat InKCre product semantics or require a Chat Completions endpoint；the selected +dialect owns native endpoint translation。Do not use `llm` as an operation name because it describes a model category。 + +The AI module owns a provider-neutral discriminated Message union：SystemMessage、UserMessage、AssistantMessage and +ToolResultMessage。AssistantMessage contains optional text and canonical `tool_calls: ToolCall[]` from one model call；do +not split these into independent ToolCallMessages。One ToolResultMessage immediately follows and owns the complete +`results: ToolResult[]` batch，where each nested result contains `tool_call_id`、JSON content and `is_error`。Result IDs are +unique and exactly cover the preceding call-ID set；result order is non-authoritative。AIManager dialects translate native +wire values to/from these types，including one-to-many message/item mappings。AgentManager consumes and stores the same +union；AIManager never imports Agent domain types。 + +### Tool-calling support is a joint contract（approved by D-159） + +Tool calling is neither purely a model fact nor purely a dialect fact： + +```text +effective feature support + = provider-bound AIModel declaration + ∩ selected peer's dialect-adapter implementation + ∩ provider configuration accepted by that adapter +``` + +AgentManager requires `tool_calling` from the selected model's `chat.features`。AIManager then verifies peer-local adapter +support and its provider configuration before making the provider request。Because AIModel is already a child of one +AIProvider，the declaration describes that configured provider + native-model offering rather than claiming an eternal +property of a model family。 + +Canonical chat tool selection uses the established protocol term `tool_choice`，but it is nullable because not every +dialect/provider/model offering exposes this control。`null` means unspecified：the dialect omits the provider option and +must not claim that the Agent requested `auto`。A non-null value is an exact Agent requirement and unsupported mapping fails +before the provider request rather than silently degrading。This remains wire/execution control，not the semantic criterion +for whether one organization approach should use a Tool；the rumination system prompt owns that meaningful-use criterion。 + +The distinction is observable in mature serving stacks：vLLM exposes an OpenAI-compatible API but requires +model-specific `--tool-call-parser` and sometimes `--chat-template` configuration for automatic tool calling；Hugging Face +documents that tool schemas are rendered by model-specific chat templates whose format must match model training。Thus +wire-level support alone cannot establish usable tool calling。 + +Evidence： + +- <https://docs.vllm.ai/en/latest/features/tool_calling/> +- <https://huggingface.co/docs/transformers/main/chat_template_tools_and_documents> +- <https://huggingface.co/docs/transformers/en/chat_templating_writing> + +## T-009 Graph Projection Versus AI Execution（approved by D-095/D-096） + +```text +Block / Relation + → embedding/retrieval-owned semantic input projection + → typed capability input + → AIManager(model, capability input, dimensions) + → embedding +``` + +- AIManager does not import or accept Block、Relation、Resolver or graph traversal types。It owns only AI registry values、 + typed capability inputs/options and outputs。 +- Resolver does not receive AIModel/EmbeddingProfile and does not expose `get_str_for_embedding()` or another renamed + embedding-specific method。 +- The embedding/retrieval owner coordinates graph entity → generic resolver capability → typed AI input，then owns + EmbeddingRecord freshness/persistence。 +- MVP Block text uses the resolver's general `get_text()` capability。Relation representation is the separate + RelationManager-owned contract below；multimodal declarations do not justify prematurely inventing a universal + graph-to-AI payload。 + +### Current block evidence + +- Memos、text and HTML resolvers duplicate `get_text()` in `get_str_for_embedding()`。 +- FeedItem is the important exception：`get_text()` chooses one best body，while its embedding method concatenates title、 + summary and body/full text。 +- Attachment metadata also diverges：`get_text()` returns filename，while embedding text adds media type。 +- Image/audio/video/PDF/EPUB/ZIP/file currently reject both text and embedding-text capabilities；their solved values are + typed content/facts，not automatically an AI modality payload。 + +This evidence supports reviewing whether `get_text()` itself should become the one complete reusable textual projection， +rather than preserving two near-duplicate use-specific methods。 + +### Approved Block text contract + +- Remove `get_str_for_embedding()`。For text-modality Profiles，the embedding owner calls only Resolver `get_text()`。 +- `get_text()` means one complete、generally useful textual projection of the Block，not a UI label and not model-specific + prompt text。 +- FeedItem folds title、summary and full text/authored content into `get_text()`；Memo uses body；HTML uses rendered text； + attachment metadata uses filename without embedding-specific MIME decoration。 +- Resolver type without text capability raises `UnsupportedResolverCapability`；a supported resolver whose particular + Block has no meaningful text returns `None`。 +- No scenario parameter is added now。Future retrieval evidence may justify a general use-side projection context without + reversing Resolver → AI dependencies；the concrete pressure must define it first。 +- Organization breakdown owns semantic granularity；`get_text()` does not secretly chunk long content。 + +## T-010 Relation Vector-Search Alternatives（approved by D-098） + +Persisted relation contents currently observed include `attachment:<order>`、`content`、`parent`、`reference`、`feed`、 +`enclosure`、`full_text`、`alt:text`、`from`、`to`、`cc`、`owns` and source-specific attachment/entity labels。These are +graph grammar or endpoint roles，not standalone authored semantic text。Embedding their raw strings would create records +without useful information content and contradict D-082's per-profile availability rule。 + +No current relation owns a resolver/capability contract that could assemble meaningful endpoint-aware text。The dynamic- +property model makes the edge useful，but does not by itself prove per-edge vector retrieval。 + +### Dynamic-property model proposed by Sir + +```text +Relation(from, content, to) + subject = from Block + property = relation.content + value = to Block + +semantic reading: + <value> is <subject>'s <property> +``` + +Direction is therefore semantic，not graph-storage metadata。A Relation expresses one runtime-extensible property of an +object without requiring core to enumerate every possible object class/property schema。This resembles a directed +subject/property/value assertion but introduces no RDF/type hierarchy or new info-base entity。 + +The generic projection belongs to RelationManager，not AIManager or an embedding-specific resolver method： + +```text +subject:\n<Resolver(from).get_label()> +property:\n<relation.content> +value:\n<Resolver(to).get_label()> +``` + +- preserve exact direction and exact property content；do not embed the property token alone； +- RelationManager exposes the complete Relation text；SemanticRetrievalManager consumes it and owns profile/record + execution； +- return the existing Relation identity when matched，so consumers can graph-navigate to both endpoint Blocks； +- tentatively require meaningful concise labels from both endpoints；one-sided/zero-sided relations remain unavailable rather than + pretending a partial assertion is complete； +- do not add a RelationResolver/type hierarchy solely for this projection。 + +`get_label()` is a required Resolver capability for a concise resolver-qualified endpoint reference。The concrete +Resolver owns the complete label，including its readable self-name and any optional instance title/name/identifier，for +example `feed <title>` or `github user <username>`。There is no separate `semantic_kind`、`friendly_name` registration +metadata or world-object type。The label must not expose an exact implementation ID such as +`extensions.rss.feed.v1` into semantic input。 + +This creates a freshness consequence：a Relation EmbeddingRecord depends on relation content/direction **and** both +endpoint label projections。`relation.updated_at` alone is insufficient；endpoint changes must cause recomputation or make +the record's exact input snapshot stale。The record snapshot/lifecycle review must carry this dependency rather than add +trigger cascades from Block to Relation。 + +### Alternative A — endpoint-aware Relation embeddings（selected） + +**Benefit**：one natural-language query can directly rank a directed subject/property/value assertion，例如 “which user +owns repository X”。The property is not actually redundant：two endpoint titles alone do not state their relationship。 + +**Costs**：endpoint semantics duplicate Block records；input/freshness depends on three entities；common structural edges +create many near-duplicate results；generic text embeddings are not guaranteed to model relational composition well。 + +### Alternative B — raw relation-content embeddings + +**Benefit**：maps natural language approximately to property vocabulary，which may help an Agent discover that “owned by” +corresponds to stored `owns` before graph traversal。 + +**Failure**：every edge with the same content receives the same vector and ties，while endpoint relevance is absent。 +`attachment:<order>` and arbitrary/JSON content further fragment the vocabulary。This is property discovery，not relation- +instance retrieval；embedding every edge is redundant representation。 + +If property discovery becomes real，a distinct/normalized property vocabulary plus exact directed graph query is a better +owner than `relation_embeddings` per row。 + +### Alternative C — no Relation embeddings in MVP（rejected after review） + +```text +semantic query + → SemanticRetrievalManager ranks Blocks + → graph-navigation operation follows exact incoming/outgoing Relations + → optional exact property filter +``` + +- semantic retrieval would own content similarity；Relations would own exact dynamic properties/direction in graph navigation； +- Agent relational questions first find a known endpoint，then traverse properties； +- future graph-aware retrieval may combine endpoint scores/property discovery without committing one vector per edge； +- no `get_label()` is added solely for an unproven Relation vector path； +- if approved，the obsolete `relation_embeddings` relation should be removed rather than retained empty “for future”。 + +Sir instead selected Alternative A so semantic retrieval can remain deep across both object content and directed dynamic +properties。Alternative C remains useful cost evidence，not the MVP contract。 + +## Manager Boundary（approved by D-099） + +- `SemanticRetrievalManager` is the public use-level manager：profile selection、Block projection、record maintenance、 + query embedding、vector comparison and ranked result contract。 +- AIManager owns raw model embedding execution only and remains graph-blind。 +- Do not expose a generic `EmbeddingManager` merely because semantic retrieval has internal embedding mechanics。Keep that + support internal first；if organization linking or another real consumer needs the same graph/profile/record lifecycle， + extract the proven shared boundary as `InfoBaseEmbeddingManager`。 + +## T-011 EmbeddingRecord Snapshot（approved by D-101） + +```text +block_embeddings + profile: EmbeddingProfile FK + block: Block FK + embedding: variable-dimension vector + created_at / updated_at: database-owned + PRIMARY KEY (profile, block) + +relation_embeddings + profile: EmbeddingProfile FK + relation: Relation FK + embedding: variable-dimension vector + created_at / updated_at: database-owned + PRIMARY KEY (profile, relation) +``` + +No `input_digest` is currently proposed。With deterministic resolver contracts and database-owned timestamps，freshness +can be derived without persisting another hash： + +```text +Block record fresh iff + record.updated_at >= max(profile.updated_at, block.updated_at) + +Relation record fresh iff + record.updated_at >= max( + profile.updated_at, + relation.updated_at, + from_block.updated_at, + to_block.updated_at + ) +``` + +This requires `get_label()` to be deterministic and Block-local：it may decode/hydrate that Block，but does not traverse +Relations、invoke AI or depend on another Block。Its output formatting is part of the exact resolver contract；a breaking +output-contract change advances the resolver ID version rather than silently changing one implementation。 + +If Profile dimensions changes，old vectors may have another length even though timestamp filters make them stale。The +retrieval candidate subquery must exclude rows whose actual vector length differs from Profile dimensions before applying +the distance operator，so stale mixed-dimension rows cannot fail the query while background/lazy refresh replaces them。 + +The already accepted late-write race remains。An input digest would detect exact projection differences only after the +projection is recomputed and still would not by itself schedule the row；without another proven diagnostic/consistency use， +it does not currently repay its storage and lifecycle semantics。 + +### Direction-integrity pressure + +The dynamic-property reading makes direction objectively reviewable。For example current mail code persists +`EmailAddress --from--> Email`，which reads “Email is EmailAddress's from” and appears reversed；the intended property is +normally `Email --from--> EmailAddress`。Implementation preflight must audit every in-scope producer against subject/ +property/value semantics rather than embedding malformed assertions。 + +Here, **in-scope graph producers** means every code path owned by this repository that creates or changes Relations which +the new SemanticRetrievalManager may index。The current concrete audit set is: + +- Memos memo、attachment、content、parent and reference graph builders； +- RSS feed membership、enclosure、full-text and semantic-content graph builders； +- core image alternate-text resolution； +- mail sender/recipient/cc construction； +- GitHub ownership construction； +- Twitter attachment、URL-entity and bookmark-context construction； +- migrations and tests that create or preserve any of those assertions。 + +This term does not claim authority over arbitrary graph-writing code in another peer，and it does not mean every existing +database row can be inferred or repaired automatically。For external or historical rows，the persisted direction remains +authority unless an explicit migration has stronger evidence。Implementation preflight must map each repository-owned +producer to subject/property/value，fix a reversed producer when intent is provable，and name any required data migration +rather than silently reinterpret stored direction。 + +## T-012 Resolver-Owned Endpoint Label（approved by D-102） + +The endpoint qualifier needs no new `semantic_kind` or registry metadata。The minimum contract is: + +```text +exact resolver id + extensions.rss.feed.v1 # dispatch + compatibility identity + +Resolver.get_label(block) + -> "feed <Hacker News>" # complete resolver-owned label + -> "feed" # valid fallback when no useful identifier exists +``` + +- Current core-py Resolver has no domain `name` property；it only has exact `__rsotype__`。Python `__name__` is an + implementation symbol。client-web likewise exposes JavaScript constructor `.name`，not a domain contract；known + peer spellings already differ，for example `HTMLResolver`/`HtmlResolver` and `EPUBResolver`/`EpubResolver`。 +- Because no current consumer needs a Resolver name independently of its endpoint label，do not add `name` or + `friendly_name` solely to decompose one method。Each concrete `get_label()` directly owns the complete stable，non- + localized reference text。A later second consumer may prove a shared name capability。 +- Exact resolver ID remains dispatch/version identity but does not enter semantic text。A breaking `get_label()` output + change advances that exact resolver contract rather than leaving existing Relation embeddings apparently fresh。 +- Do not add `friendly_name` to ResolverManager registration and do not create a shared `resolver_types` table。The + executing peer already has the concrete Resolver，and semantic retrieval proves no independent metadata discovery、FK + or catalog lifecycle。 + +Repository-owned Relation producer findings are tracked in +[relation-producer-audit.md](../relation-producer-audit.md)。Producer corrections belong to their collection/graph writers +and targeted schema migration，never to organization、read-time normalization or SemanticRetrievalManager。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/embedding-profiles.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/embedding-profiles.md new file mode 100644 index 0000000..0248eb0 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/embedding-profiles.md @@ -0,0 +1,162 @@ +# Embedding Profiles And Vector Records + +> [Technical design index](index.md) + +## Stable Vocabulary + +```text +semantic representation of Block / Relation + → EmbeddingProfile-compatible input + → embedding + → EmbeddingRecord(entity reference, profile, input snapshot, vector) + → exact or ANN-accelerated vector retrieval + → ranked Blocks / Relations +``` + +- **Embedding**: one vector produced from one profile-compatible input。 +- **EmbeddingProfile**: the durable compatibility contract for one vector space；exact MVP shape is approved。 +- **EmbeddingRecord**: one durable derived mapping from a graph entity/profile/input snapshot to an embedding；exact MVP + shape is approved。 +- **ANN index**: optional physical PostgreSQL acceleration，not domain identity or graph authority。 + +## Current Schema Failure + +Current `block_embeddings` / `relation_embeddings` are keyed one-to-one by entity ID and store fixed `vector(1024)` plus +one timestamp。They cannot name model/provider/input contract、coexist across profiles、prove which semantic input was +embedded or distinguish stale/unavailable/error。They are migration evidence，not the new shape。 + +## T-001 — Profile Persistence And Mutable Freshness（D-083 persistence；immutability superseded by D-089） + +### Current contract + +- Persist EmbeddingProfile in the shared peer database protocol；an EmbeddingRecord is not interpretable without the + profile that defines its vector space。 +- Allow vector-contract fields to change in place。A database-owned Profile `updated_at` watermark makes older + EmbeddingRecords candidates for rebuild；the accepted late-write race does not justify another version field。 +- Allow multiple independently addressable Profiles and their records to coexist when consumers genuinely need distinct + vector spaces、evaluation or migration；ordinary Profile edits do not require a new row。 +- Keep active/default profile selection outside Profile definition，as mutable deployment/application configuration。 +- Keep credentials outside the profile。A profile names a non-secret execution contract；each peer separately proves it + has an executor/config capable of realizing it。 + +### Why persistence is proposed + +- records remain explainable after runtime config changes； +- peers agree which vectors can be compared； +- rebuild/switch can be staged instead of corrupting one global vector space； +- profile deletion can be referentially restricted while records still depend on it。 + +EmbeddingProfile remains an independent typed relation but is mutable。A monotonic `version` would close the rare old- +execution/late-write race，but Sir explicitly judged its marginal harm too low to justify another field。Profile and +record database-owned timestamps therefore provide best-effort freshness rather than an exact execution proof。Global +config owns only use-specific selected/default profile references。 + +## T-002 — EmbeddingProfile Versus Retrieval Options（approved by D-084） + +### Proposed ownership rule + +EmbeddingProfile owns only selections and operations that determine the generated vector space： + +- `ai_model`； +- dimensions； +- vector normalization。 + +AIModel owns intrinsic capability and modality declarations。EmbeddingProfile does not duplicate modalities merely to +describe which inputs its referenced model can accept。 + +MVP assumes a symmetric input contract and does not add candidate/query transformations。A future model with proven +asymmetric task/instruction/prefix requirements must evolve the profile contract explicitly。 + +Query-scoped `VectorRetrievalOptions` own how already-generated compatible vectors are searched： + +- distance/similarity metric； +- top-k、maximum distance/minimum similarity and filters； +- exact versus approximate search； +- HNSW/IVFFlat query/build tuning； +- optional reranking/fusion strategy。 + +Metric therefore leaves EmbeddingProfile identity。MVP does not create a RetrievalPolicy table；runtime config may provide +defaults，and a request may supply admitted overrides。 + +## EmbeddingProfile Fields（approved by D-093/D-094） + +| Field | Intended meaning | Current concern | +| --- | --- | --- | +| `id` | stable profile reference | database-generated bigint | +| `name` | optional descriptive label | UI falls back to model identity + dimensions | +| `ai_model` | stable shared executable-model reference | bigint FK；role name needs no `_id` suffix | +| `dimensions` | exact vector shape | required positive integer；adapter verifies every result length | +| `created_at` | ordinary row creation time | database-owned | +| `updated_at` | row mutation time and best-effort invalidation watermark | database-owned；rare late-write race accepted | + +MVP omits `enabled` because use-owned references select active Profiles；omits normalization because no actual model/ +acceptance pressure requires changing model output；and omits a generic config/options bag because dimensions is the only +proven profile parameter。 + +## EmbeddingRecord Shape（approved by D-101） + +```text +block_embeddings: (profile, block, embedding, timestamps...) +relation_embeddings: (profile, relation, embedding, timestamps...) +``` + +Composite profile/entity identity；real FK cascade on entity deletion；restrict profile deletion while either table has +records。Unavailable/error/job state is not automatically an EmbeddingRecord and remains a separate lifecycle question。 +Freshness is derived from database-owned timestamps；MVP does not persist `input_digest`。 + +## PostgreSQL / pgvector Evidence + +- pgvector performs exact nearest-neighbor search without a physical ANN index；HNSW/IVFFlat trade recall/resources for + speed。 +- a dimension-unspecified `vector` column can store different dimensions；ANN indexes must target rows of one dimension， + for example through profile-filtered partial/expression indexes。 +- therefore MVP record persistence need not prematurely choose one HNSW index per profile；quality/scale evidence can + select physical acceleration after the logical contracts are stable。 + +## HNSW Cost Boundary + +HNSW（Hierarchical Navigable Small World）builds a multi-layer proximity graph over vectors。A query navigates from +sparse upper layers toward denser neighbors instead of calculating distance against every record。It is approximate： +speed improves，but some true nearest neighbors may be missed。 + +For variable-dimension records and runtime-created profiles，a correct pgvector HNSW surface would require profile- +filtered、dimension-cast、metric-specific indexes on each record table，conceptually： + +```sql +CREATE INDEX ... ON block_embeddings +USING hnsw ((embedding::vector(1024)) vector_cosine_ops) +WHERE profile_id = '<profile>'; +``` + +and an equivalent relation index。Each dimension/metric/profile combination changes DDL。Ordinary peers currently own +data protocol operations，not migration-owner DDL，so automatic index creation for arbitrary runtime profiles would add a +privileged index-management lifecycle。 + +**Benefit**：lower query latency and fewer distance calculations at larger record counts。 + +**Costs**：build time、additional disk/memory、slower writes、vacuum/reindex maintenance、approximate recall、profile/ +metric-specific DDL and tuning (`m`、`ef_construction`、`ef_search`)。Filtering by profile also interacts with ANN recall。 + +**Approved MVP**：use exact pgvector comparison。Add HNSW only when a representative corpus violates an accepted latency/ +scale threshold；then compare ANN recall against exact search before selecting parameters。 + +## Review Order + +1. profile persistence（approved；initial immutability superseded by D-089）； +2. profile-generation fields versus query retrieval options（approved）； +3. global AI provider/model/dialect topology and profile model reference（approved）； +4. AIModel capability/modality declarations and remaining Provider/Model fields（approved）； +5. dedicated mutable profile、consumer-owned selection and database timestamp invalidation（approved）； +6. AIDialect type / AIProvider instance shape（approved）； +7. AIModel exact fields and identity（approved）； +8. profile remaining exact fields and identity（approved）； +9. Block text representation ↔ profile input contract（approved）； +10. Relation representation and availability（approved）； +11. record entity-reference and source-snapshot shape（approved）； +12. Resolver-owned endpoint label and persistence boundary（approved）； +13. unavailable/error、derived freshness、maintenance/rebuild、config and concurrency（approved）； +14. AIManager capability implementation scope（approved）； +15. retrieval result/score/filter contract（approved）； +16. heterogeneous Peer capability delegation、API and migration（approved through D-136）； +17. minimum rumination/Agent approach required by retrieval quality（Product/Technical contract approved through D-182； + Acceptance remains active）。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/index.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/index.md new file mode 100644 index 0000000..a456148 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/index.md @@ -0,0 +1,39 @@ +# Semantic Retrieval Technical Design + +- **Status**: Product/Technical contracts，both capability landings and HTTP inbound address authority are approved through + D-186；Acceptance is approved through D-190；implementation planning remains。 +- **Current review**: exact implementation decomposition and preflight evidence。 +- **Rule**: each topic becomes a contract only after explicit review；later findings revise it through a visible decision， + not silent prose drift。 + +## Topic Navigation + +| Topic | Ownership / contents | +| --- | --- | +| [Embedding profiles and vector records](embedding-profiles.md) | Stable vocabulary、Profile/Record schema、pgvector and ANN boundary、review order | +| [AI routing and semantic projection](ai-and-projection.md) | AIDialect/Provider/Model/Manager、Block/Relation projection、Resolver labels | +| [Embedding maintenance and deployment config](maintenance-and-config.md) | Freshness/outcomes、maintenance、ConfigContract and DeploymentConfigManager | +| [Semantic retrieval contract](retrieval-contract.md) | Request、score、ranked Block/Relation result and bounded filters | +| [Peer capability delegation](peer-delegation.md) | Discovery、lease、protocol inbound/outbound、HTTP delegation and failover | +| [Rumination and graph forms](rumination-agent-graph.md) | Rumination semantics、Resolver draft Tools、StarsGraphForm/GraphForm and graph-command authority ledger | +| [Agent definition、Thread and runtime](agent-runtime.md) | AgentManager、persisted Agent definitions、Thread persistence boundary、Tool binding/execution and Message lifecycle | +| [Shared row timestamp contract](shared-row-timestamps.md) | Database-owned `updated_at` boundary | +| [Delivery map](../delivery-map.md) | Dependency-ordered implementation increments；design probe only | + +## Current Technical Edge + +Producer Forms、Agent/Thread/Tool execution and Agent-validate → Resolver-create → InfoBase-normalize ownership are closed +through D-186。Acceptance is approved through D-190；the active design edge is implementation planning and preflight。 +Approved upstream +contracts remain linked by decision ID through the [decision register](../../../decisions/index.md)。 + +For the current graph-command boundary，use the topic file's [boundary ledger](rumination-agent-graph.md#current-boundary-ledger) +as the active contract。D-175–D-177 document how the correction was reached；they must not be merged as simultaneous +responsibilities。 + +## File Boundary + +- Topic files own detailed task-state technical contracts；this index owns only navigation and the current edge。 +- Splitting by technical owner does not create additional runtime services or persistence authorities。 +- New material goes to the smallest owning topic。Create another topic only when an existing file gains a second independent + review/navigation pressure，not merely because of line count。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/maintenance-and-config.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/maintenance-and-config.md new file mode 100644 index 0000000..1742b5c --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/maintenance-and-config.md @@ -0,0 +1,249 @@ +# Embedding Maintenance And Deployment Config + +> [Technical design index](index.md) + +## T-013 Embedding Maintenance Outcomes And Freshness（approved by D-104–D-111） + +### Do not persist `dirty` + +Record presence and freshness are derived facts，not another mutable state machine: + +```text +missing = no (profile, entity) record +fresh = record exists + D-101 timestamp dependencies satisfied + vector dimensions match +stale = record exists but one of those predicates fails +``` + +Keeping an old stale record is harmless when every retrieval candidate query applies those predicates。Maintenance may +replace it after success；garbage collection is a storage concern，not required for correctness。A trigger-written +`dirty` flag would duplicate the same dependency facts，need endpoint cascades for Relation and create cross-peer update +ordering that timestamps deliberately avoid。 + +### Attempt outcomes are not EmbeddingRecords + +One maintenance attempt over one entity/profile has three semantic outcomes: + +| Outcome | Examples | Record effect | +| --- | --- | --- | +| `embedded` | projection available，AI response valid and dimensions match | upsert successful record | +| `unavailable` | exact Resolver lacks requested capability；supported projection returns `None`；this peer lacks the exact Resolver | leave missing/stale record unchanged | +| `failed` | corrupt content/storage read；provider disabled/unreachable；adapter/model error；dimension mismatch | leave missing/stale record unchanged and expose diagnostic | + +`UnknownResolver` is executor-local availability，not a global property of the Block；another peer may implement the +same exact Resolver。Likewise a provider/network failure is temporal execution evidence。Persisting either as a shared +entity state could incorrectly suppress a capable peer or outlive the failure。 + +The current proposal therefore persists only successful EmbeddingRecords。`unavailable` / `failed` belong to the result +and diagnostics of the maintenance attempt that observed them。This does **not** yet decide whether that attempt is a +durable job or a synchronous/bounded command；if a durable job is justified，its item diagnostics remain execution facts +owned by that job，not status columns on embedding tables。 + +### Failure and replacement boundary + +- projection and AI execution happen without holding one database transaction across network/storage work； +- success uses atomic upsert on `(profile, entity)` and receives database-owned record `updated_at`； +- unavailable/failure never deletes a previously successful record，but retrieval excludes that record when stale； +- concurrent executors may duplicate work；last successful compatible upsert wins。The already accepted late-write race + remains；MVP does not add leases、revision fields or distributed locks solely to eliminate duplicate AI calls； +- `refresh` remains the established cache-snapshot option and is not reused to mean durable embedding regeneration。The + operation vocabulary for ordinary stale/missing maintenance versus forced rebuilding of already-fresh records remains + part of this review。 + +### Existing failure evidence + +Current `EmbeddingManager._skipped_block_versions` is process-local，Block-only and cleared on restart。It conflates +unsupported projection with unknown local Resolver，while provider errors escape。The 60-second scanner selects only the +first ten missing rows；without its skip set，a stable unavailable prefix can starve all later entities。These are failure +samples，not a reason to persist a global unavailable state。 + +### Maintenance execution contract（approved） + +Do not create an EmbeddingMaintenanceJob relation in MVP。A successful EmbeddingRecord is already an idempotent progress +marker；after interruption，the next ordinary scan naturally selects the remaining missing/stale entities。No separate +job cursor、claim state or item ledger is required to resume correctness。 + +```text +SemanticRetrievalManager.maintain(profile, bounded execution options) + -> scan deterministic pages of candidate Blocks and Relations + -> derive projection outcome for each entity + -> continue past unavailable entities until an available batch is filled or scan ends + -> AIManager.embed(model, ordered text batch, dimensions) + -> validate count/order/dimensions + -> short atomic upsert transaction per successful batch + -> return attempt report (embedded / unavailable / failed counts + bounded diagnostics) +``` + +- the scheduled path and explicit maintenance command call the same manager operation；collection/organization never + invoke provider-specific embedding code。The old eager `BlockManager.fetchsert()` call and old process-local skip set are + removed。 +- database page limit is not the success batch limit；the scan advances past unavailable rows so a low-ID unsupported + prefix cannot starve later candidates。A cursor is local to one invocation，not durable authority。 +- projection/storage reads and provider calls occur outside a database transaction。Only successful batch upsert holds a + short transaction。Batch response cardinality/order and every vector dimension are validated before any row is written。 +- ordinary `maintain` selects only missing/stale records。A separate administrative `rebuild` operation includes records + that were fresh at operation start and uses that start timestamp as its cutoff，so its own writes are not selected again + in the same run。`refresh` is not an alias for either operation。 +- interruption after any committed batch is safe。Repeating ordinary maintenance does not repay completed work； + repeating a full rebuild may regenerate some already rebuilt vectors，which is accepted low-harm duplicate work rather + than a reason to add durable job identity。 +- two peers may select the same entity and duplicate an AI call。The final valid upsert wins；MVP adds no lease/advisory + lock/claim table solely to save that occasional call。 + +Automatic scheduling and batch/config ownership are closed below；they do not change the approved absence of a durable +job relation。 + +### Automatic maintenance scope（approved） + +- SemanticRetrievalManager owns one nullable deployment-default EmbeddingProfile reference。A retrieve/maintain/rebuild + request may explicitly select another Profile，but there is no implicit cross-profile execution or rank fusion。 +- automatic maintenance processes only that default Profile。Scanning every defined Profile would spend provider calls + on dormant evaluation/migration vector spaces merely because their definitions exist。When no default is configured， + the scheduled operation is a no-op and defaulted retrieval reports configuration unavailable。 +- switching the default does not delete the old Profile or records。The next scheduled pass begins maintaining the newly + selected space；old records remain available for explicit evaluation/rollback and normal retention decisions。 +- schedule participation/interval and maximum work per invocation are peer-local operational settings，not shared + EmbeddingProfile fields。Equal peers need not all run a worker，and client-web does not acquire a background scheduler + merely because core-py has one。 +- SemanticRetrievalManager sends an ordered logical batch；AIManager/dialect adapter owns provider request translation and + any provider-limit chunking while preserving one-output-per-input order。Database commit batch size is a local + maintenance resource bound，not a vector-space contract or shared Profile setting。 + +The deployment-default reference is persisted through the approved deployment-scope config contract below；it is not +hidden on EmbeddingProfile as a generic `is_default` flag。 + +### Rejected singleton proposal + +The proposed one-row `semantic_retrieval_configs` relation does not meet the independent identity/lifecycle test。A +deployment can own only one such value，and its whole lifecycle is the owner-shaped value at one deployment config key。 +An ordinary FK would be useful，but that alone does not prove a dedicated singleton domain relation has sufficient value。 + +### Deployment-scope config direction（approved） + +Use one simple shared deployment-scoped config relation，distinct from per-peer config currently stored on legacy +`clients` rows（future `peers.config`）: + +```text +configs + key: text primary key + schema: exact schema-contract ID + value: JSONB + created_at / updated_at: database-owned + +example + key = "semantic_retrieval" + schema = "core.semantic_retrieval.config.v1" + value = {"default_profile": 42} +``` + +- `key` addresses one deployment-owned config value；`schema` selects its exact decoder/validator contract；`value` + remains owner-shaped JSON rather than becoming a universal god-object schema。 +- The concise table name `configs` is sufficient because this relation is the deployment-scoped shared config authority； + the config on each Peer row remains explicitly peer-owned。 +- core-py DeploymentConfigManager locally maps exact schema ID to a Pydantic model。Other equal peers register an equivalent local + validator under the same ID；the durable schema value therefore cannot be a Python import path or class name。 +- schema registration collision/unknown schema is explicit。Config reads and writes validate the complete value before + exposing/persisting it；a schema-breaking change uses a new exact ID and an explicit value migration。 +- `SemanticRetrievalConfig.default_profile` remains a typed Profile ID in that model。Config write/read validates only + the complete JSON structure；it does not pass through SemanticRetrievalManager or query Profile existence。A JSON value + cannot receive an ordinary PostgreSQL FK，so this preserves protocol/application typing but not referential integrity。 + SemanticRetrievalManager resolves the reference only when used and explicitly distinguishes dangling from unconfigured。 +- DeploymentConfigManager owns exact schema-model registration/resolution and deployment config persistence；it uses the generic + ConfigContract for value validation/normalization。SemanticRetrievalManager owns the config key's use semantics and + behavioral consequences，not its write path。The deployment manager does not import EmbeddingProfile or scheduling + policy。 +- peer-local schedule interval/work limits remain outside deployment config。The semantic value initially needs only + nullable `default_profile`。 + +### DeploymentConfigManager read/update contract（approved） + +DeploymentConfigManager is the total manager for the shared `configs` relation and its local schema-model registry: + +```text +register_schema(exact_schema_id, local_model) +get(key) -> load row -> resolve schema -> validate complete value -> typed model +replace(key, schema, complete_value) +patch(key, partial_value) +``` + +- schema registration is idempotent for the same model and rejects a different model claiming the same exact ID。An + unknown schema or persisted value that fails its registered model is explicit config failure，not a raw-dict fallback。 +- `replace` is an upsert and validates one complete value under the supplied schema before the database changes。It is + the only operation that may change a row's schema，so a schema migration and its new complete value are atomic。 +- `patch` requires an existing row，keeps its schema，shallow-merges current object + patch，validates the complete next + model，persists normalized JSON and then exposes success。This matches the already accepted extension-config update + order without making DeploymentConfigManager own extension/source/storage row lifecycles。 +- public HTTP semantics should name these honestly；`PUT /configs/{key}` performs complete replace/upsert and + `PATCH /configs/{key}` performs partial update。The existing extension endpoint's PUT-as-patch behavior is historical + evidence for a later correction，not the new generic contract。 +- DeploymentConfigManager does not own semantic side effects or live caches。An owner reads the newly validated model when acting； + if future config changes require a live callback，that pressure must define an owner-specific notification mechanism + rather than a generic hook registry now。 + +### Generic configuration mechanics vs deployment-scope configs（approved） + +The abstraction threshold is now met by real code: + +- shared `configs` needs exact schema registry，replace/patch and typed JSON persistence； +- ExtensionManager already implements current + shallow patch → complete model validation → normalized persistence → + owner-specific live apply； +- SourceBase and StorageBase separately retain config model/schema and validate persisted JSON on read/construction。 + +Do not collapse these pressures into one nominally "generic" business manager。They form two modules with different +owners: + +```text +generic configuration mechanics（no deployment scope，no persistence） + complete validation + normalized JSON + shallow patch preparation + JSON Schema projection when requested + +deployment-scope configs domain + configs relation / DeploymentConfig row + exact schema ID -> local model registry + CRUD for deployment-scoped configs table + DeploymentConfigManager + +owner managers + ExtensionManager: ExtensionModel persistence + live apply + SourceManager: SourceModel persistence + source consequences + StorageManager: StorageModel persistence + storage consequences + SemanticRetrievalManager: use-time default Profile resolution + scheduling +``` + +- generic configuration mechanics accept a model/contract directly。They do not know `configs`、deployment、keys、exact + schema IDs、database sessions or owner lifecycles。Existing Extension/Source/Storage config therefore does not need an + invented persisted exact schema ID merely to reuse validation logic。 +- the deployment-scope configs module uses those mechanics but additionally owns exact schema registration/resolution and + `configs` persistence。Only rows whose protocol stores `schema`—initially `configs`—require registry lookup。 +- DeploymentConfigManager must not become a polymorphic updater for ExtensionModel、SourceModel and StorageModel。Their address， + transaction，live replacement and failure semantics remain with their owner managers。 +- implementation should move ExtensionManager's already-proven shallow-patch preparation onto the common primitive as the + second executable consumer。Source/Storage adopt it only when their update operations actually exist；their current + validation can reuse the complete-value primitive without adding speculative CRUD。 +- no callback/hook bus is introduced。The common result is a validated model + normalized JSON；the owner decides what + happens after persistence。 + +### Technical naming contract（approved by D-111） + +Use names that expose the ownership split rather than two similarly named managers: + +```text +generic mechanism + module/package: app.configuration + primary abstraction: ConfigContract[Model] + +deployment-scope configs domain + schema: DeploymentConfigModel -> table configs + business: DeploymentConfigManager + HTTP: /configs/{key} +``` + +- `ConfigContract` wraps one local Pydantic model's complete validation、normalized JSON、shallow patch preparation and + JSON Schema projection。It has no registry、database、deployment key or live lifecycle。 +- `DeploymentConfigManager` explicitly names the scope that the earlier `ConfigManager` placeholder obscured。It owns + exact schema-ID registry and `configs` row CRUD，and calls ConfigContract for value mechanics。 +- `SemanticRetrievalConfig` remains an owner-defined Pydantic config model registered under + `core.semantic_retrieval.config.v1`；it is neither DeploymentConfigModel nor ConfigContract。 +- the physical split follows current repository convention：generic support lives outside `app.business`；the persisted + domain uses explicit `deployment_config` schema/business module names even though its table and HTTP resource remain the + concise plural `configs`。Exact files remain an implementation-plan concern after the names are approved。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/peer-delegation.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/peer-delegation.md new file mode 100644 index 0000000..b83de2e --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/peer-delegation.md @@ -0,0 +1,446 @@ +# Peer Capability Delegation + +> [Technical design index](index.md) + +## T-016 Heterogeneous Peer Capability Delegation（approved through D-136） + +### Rejected database-business implementation + +- PostgREST/pgvector can technically execute the proposed mixed comparison，but feasibility does not assign ownership。 + Storage blob RPCs are narrow atomic helpers over opaque bytes；a semantic retrieval RPC would instead own Profile、 + freshness、ranking and result business behavior in SQL，making PostgreSQL a hidden SemanticRetrievalManager。 +- withdraw the entire shared retrieval RPC topology。Embedding record tables remain shared state，but an application + capability provider reads/compares them through maintainable peer-local implementation code。 + +### Product model + +```text +shared database authority: Peers are equal +runtime execution ability: Peers are heterogeneous + +asynchronous delegated work + caller writes domain job -> capable worker Peer claims/executes -> durable outcome + +synchronous delegated capability + caller Peer -> selected provider Peer HTTP request -> typed response +``` + +- collect-job persistence is collection-domain behavior，not a generic delegation queue。Semantic retrieval's natural + work model is request-response and receives no job table。 +- C/S is a role on one interaction edge。A Peer may provide semantic retrieval while consuming another capability；this + does not turn the whole deployment into one fixed client/server hierarchy。 + +### Existing code pressure + +- the current legacy Client/ future Peer row already owns a nullable `rest_api_url`，while some Peers such as browsers are + intentionally unreachable。It has labels but no exact capability declaration。 +- client-web's current Client active record already performs peer-JWT HTTP requests to a selected Peer，but it conflates a + hard-coded Core endpoint with generic peer communication and cannot discover/route by capability。 +- source collect jobs already prove shared-job claim semantics；they do not solve synchronous provider discovery or typed + request-response invocation。 + +### Approved manageability boundary + +Introduce one exact versioned **Peer Capability** contract，for example `core.semantic_retrieval.v1`，and keep its registry +inside the total PeerManager rather than creating a separate CapabilityRegistry/Manager: + +```text +Peer + id / name / labels / config + capabilities: exact capability + inbound descriptor snapshot + lease_expires_at + +PeerManager + local exact-capability-ID registry + self-advertisement + provider discovery / routing + authenticated connection/transport to a selected Peer + +SemanticRetrievalCapability + exact capability ID + domain-owned typed request/response schemas and inbound codec + domain-owned request-response behavior + +provider Peer + typed POST /semantic-retrieval + -> local SemanticRetrievalManager +``` + +- PeerManager does not interpret protocol parameters、request/response schema or interaction shape。Its common boundary + stops at exact-ID advertisement/discovery and authenticated connectivity；the owning domain provides its typed remote + adapter and ordinary OpenAPI route。 +- do not create a generic `/capabilities/{id}/invoke` raw-JSON endpoint or generic job relation。Semantic retrieval owns + its request-response protocol；each job domain keeps its own state/claim/result semantics。 +- advertise exact capability IDs as Peer-owned runtime state；do not repurpose human/admin `labels`。Persistence、lease and + randomized provider selection are approved below。 +- SemanticRetrievalManager is the unified typed facade on every implementing caller。Only a Peer with a non-delegating + local implementation advertises the capability and serves its inbound；other callers delegate through PeerManager。 + +### Approved discovery semantics + +Capability routing distinguishes three facts: + +- **support**: this Peer runtime has an implementation of the exact capability contract； +- **liveness**: this Peer is currently online； +- **readiness**: the implementation's current dependencies/configuration allow it to serve a particular request。 + +D-119 limits discovery to support + Peer liveness。PeerManager routes an exact capability to an online provider so domain +callers do not inspect liveness；readiness remains service-internal and never becomes discovery metadata。Routing may not +blindly replay a post-dispatch failure because retry safety remains operation-specific。 + +### Approved liveness mechanism + +- persist capability declarations as Peer-owned exact IDs rather than independent service rows； +- persist an expiry/lease fact rather than an `online` boolean，so abrupt process loss becomes offline without a graceful + shutdown write； +- derive online using database time，avoiding caller/provider clock disagreement； +- keep the lease Peer-scoped rather than capability-scoped。A running Peer updates its declaration when a capability is + enabled/disabled；readiness does not create capability heartbeats。 + +The caller-local renewal schedule is an implementation/runtime-config concern；the shared renewal and routing contracts +are approved below。 + +### Approved Peer persistence shape + +```text +peers + id uuid primary key + name text not null + labels text[] not null default [] + config jsonb not null default {} + config_schema jsonb not null default {} + capabilities jsonb not null default [] + lease_expires_at timestamptz null + created_at timestamptz not null + updated_at timestamptz not null +``` + +`capabilities` is one validated full snapshot，not a child relation。`labels` remain administrative metadata and never +participate in capability matching or provider selection。`config/config_schema` keep the existing per-Peer owner scope； +they do not become deployment-scoped `configs`。There is no global endpoint、`online`、`last_seen` or readiness column。 +The database-time renewal contract below prevents unrelated Peer-row updates from implying liveness。 + +### Approved lease renewal contract + +```text +renew_peer_lease(peer: uuid, ttl_seconds: positive integer) -> timestamptz +``` + +The `SECURITY INVOKER` database helper requires an existing Peer，calculates expiry from `statement_timestamp()` and +returns the stored value。TTL is supplied by the lease owner because always-on and scale-to-zero deployments have +different renewal models；no fixed protocol duration or duplicate persisted TTL is added。A lease means that the +advertised inbound remains routable，not that one application process is continuously resident，so a deployment control +plane may renew for a wakeable scale-to-zero endpoint。Ordinary Peer-row updates never renew the lease。Graceful shutdown +sets the expiry to null；abrupt loss waits for expiry。 + +### Approved candidate selection + +For one delegation，PeerManager loads candidates whose exact capability matches and whose lease is unexpired by database +time，then excludes the caller itself、malformed inbound descriptors and protocols missing from the caller-local outbound +registry。It randomly orders the remaining Peers once and walks that sequence only when D-129 proves non-execution。UUID、 +labels、snapshot order and lease duration/expiry are not routing scores。No eligible candidate raises +`CapabilityDelegationUnavailable`。MVP has no shared round-robin state、weight、priority、load、stickiness or circuit +breaker。 + +### Approved module topology and delegate contract + +The common surface is one deep logical `peer` module，not a new ServiceRegistry/CapabilityManager hierarchy: + +```text +provider runtime + SemanticRetrievalPeerInbound + capability = exact SemanticRetrieval ID + interface = { + protocol: core.peer.protocol.http.v1, + parameters: { method, absolute url } + } + controller -> local SemanticRetrievalManager + -> PeerInboundRegistry + -> PeerManager publishes capability + inbound-interface snapshot + +caller without local SemanticRetrieval implementation + SemanticRetrievalManager.retrieve(typed request) + -> local implementation when registered；otherwise: + -> PeerManager.delegate(exact capability ID, protocol payload) + -> candidates advertising capability + live lease + -> failover/select Peer and its advertised inbound interface + -> PeerOutboundRegistry.resolve(interface.protocol) + -> construct one-shot PeerHTTPOutbound(peer, inbound.parameters) + -> execute and release outbound + -> SemanticRetrieval validates typed result/domain failure +``` + +Dependency rules: + +- PeerManager imports no Extension、SemanticRetrieval or other capability owner。It sees opaque capability IDs、provider + candidates and discriminated inbound-interface protocol descriptors only。 +- PeerManager may own internal lease/routing components，but they do not become separate public Managers。PeerConnection + and PeerTarget are removed；neither has remaining distinct responsibility。 +- outbound implementations are protocol-specific，not domain-specific。PeerHTTPOutbound is an explicit class/module + implementing `core.peer.protocol.http.v1`：HTTP plus peer-JWT Authorization、wire encoding and protocol/connection + failures。It receives the protocol-owned parameter object from the advertised HTTP inbound interface and does not + interpret SemanticRetrieval payload meaning/domain failures。 +- the semantic-retrieval domain owns typed request/result semantics and a provider inbound/controller，but no longer needs + a SemanticRetrievalPeerOutbound merely to duplicate HTTP mapping。SemanticRetrievalManager delegates its serialized + request and validates the returned payload when no local implementation is registered。 +- D-126 fixes SemanticRetrievalManager as the unified facade。A provider inbound is registered only with a local + implementation and invokes a non-delegating local execution path；it never re-enters local-or-delegate selection。 +- Peer retains an intentional protocol-level relationship to opaque exact capability IDs。No concrete capability module + is imported or interpreted by PeerManager；dependencies continue from concrete inbound/outbound toward the Peer module。 +- core-py `run.py` remains a composition root only：mount/setup providers，start enabled Extensions，then publish the + complete Peer capability snapshot and begin lease renewal。It does not implement discovery or maintain a manual service + catalog。 +- ExtensionManager integrates capability publication with existing hot lifecycle：start must mount/setup successfully + before advertising；close withdraws advertisement before unmounting。Extension code does not mutate Peer rows directly。 +- client-web may retain Active Record where useful，but arbitrary capability invocation stays off the Peer entity。Its + peer module implements the same delegate pipeline and protocol-specific outbound registry；domain packages retain + typed capability facades/contracts rather than HTTP path knowledge。 + +D-123 supersedes D-120's `capabilities: text[]` projection with structured Peer-owned snapshot entries shaped as: + +```json +{ + "id": "core.semantic_retrieval.v1", + "inbound": { + "protocol": "core.peer.protocol.http.v1", + "parameters": { + "method": "POST", + "url": "https://example.com/semantic-retrieval" + } + } +} +``` + +The protocol ID discriminates/owns validation of `parameters`；outbound remains caller-local code selected through its +registry。Parameters are published by inbound specifically for construction/configuration of the paired outbound；they +are not generic capability fields。D-124 fixes delegate as one-shot and defers long-lived WebSocket/session models。The +MVP delegation payload/result boundary is one normalized JSON value in each direction；the capability owner performs +typed conversion/validation outside PeerManager，and `core.peer.protocol.http.v1` owns JSON wire encoding。No generic +network invoke route or generic delegation job is introduced。 + +The advertised parameters and per-call payload are different values。For current SemanticRetrieval: + +```json +{ + "parameters": { + "method": "POST", + "url": "https://example.com/semantic-retrieval" + }, + "payload": { + "query": null, + "body": { + "query": "...", + "options": {} + } + } +} +``` + +`parameters` are static outbound-construction facts published by the inbound。`payload` is the one-shot protocol JSON +produced by the capability owner；an HTTP payload may contain `query` and `body` simultaneously，for example: + +```json +{ + "query": { + "trace": "compact" + }, + "body": { + "query": "..." + } +} +``` + +PeerManager does not inspect either member；PeerHTTPOutbound interprets the HTTP envelope，while the domain-owned inbound +codec decides how typed values map to/from it。Do not add `request.location`。If future evidence requires representation +metadata，use the standard `content_type` name and decide from that use whether it is static parameters or per-call +payload；the current SemanticRetrieval contract needs neither an extra field nor an HTTP query。 + +The argument passed to `PeerManager.delegate()` is therefore the already-encoded protocol payload，not the typed +SemanticRetrieval request。`SemanticRetrievalManager` first validates and normalizes the domain request，then uses its +inbound-owned codec to produce the payload above；the provider inbound uses the matching contract to reconstruct the typed +request before entering the explicit non-delegating local path。`PeerManager` keeps that payload opaque，and the selected +outbound only interprets its Peer Protocol envelope。A future protocol needs a capability-owned codec，not a generic +mapping language inside peer routing。 + +The complete v1 normalized envelope is: + +```text +request = { query?: map<lowercase name, string[]>, + headers?: map<lowercase name, string[]>, + body?: JSON value } +response = { status: integer, + headers: map<lowercase name, string[]>, + body?: JSON value } +``` + +Query、headers and body may coexist。`PeerHTTPOutbound` owns peer-JWT Authorization、authority/framing and hop-by-hop +fields，so the capability payload cannot override them。It consumes the exact D-134 non-execution field before producing +an ordinary response envelope。All other status/header/body values remain available to the domain-owned codec。The current +SemanticRetrieval codec uses only a JSON request body；binary/streaming requires another exact Peer Protocol version。 + +The HTTP `url` is absolute。It replaces the previous method/path-plus-Peer-`rest_api_url` split and gives +`PeerHTTPOutbound` every static construction parameter through one protocol-owned descriptor。The Peer persistence model +therefore has no global HTTP endpoint field；repeated origins inside a small capability snapshot are accepted derived +state，and a future non-HTTP Peer Protocol does not force a new field onto Peer identity。 + +### Two orthogonal inbound/outbound views + +Inbound/outbound is used at two different architectural projections，not as a one-box-one-class rule: + +```text +SemanticRetrieval Business outbound + caller SemanticRetrievalManager + capability-owned codec + -> PeerManager.delegate + -> selected PeerHTTPOutbound + +SemanticRetrieval Business inbound + provider Peer HTTP/FastAPI boundary + -> semantic-retrieval route + capability-owned codec + -> SemanticRetrievalManager.execute_local + +Peer HTTP outbound role + caller-local PeerHTTPOutbound implementation + +Peer HTTP inbound role + reusable peer JWT + envelope + execution-marker + CORS behavior + composed into concrete Business routes +``` + +`SemanticRetrievalOutbound` and `SemanticRetrievalInbound` are therefore useful topology names but need not become public +classes。The Manager's protocol-codec responsibility belongs to the Business edge and explains its intentional contact with +transport-facing values；business ranking/embedding logic still does not move into HTTP。Likewise Peer HTTP inbound may be +implemented through FastAPI dependencies/helpers rather than a nominal `PeerHTTPInbound` object。Here “HTTP” identifies a +concrete Peer Protocol implementation；it does not restore a generic persisted transport domain or TransportManager。 + +Generic failover stops at provable non-execution。PeerManager may skip an ineligible candidate or try another Peer after +an outbound proves that nothing was dispatched。The Peer Protocol may also carry an explicit non-execution result after +contact，allowing another candidate without exposing service readiness。Once a domain response/error exists，return it； +once dispatch may have occurred but the outcome is unknown（for example a response timeout or post-write reset），surface +that failure without automatic replay。A future capability may deliberately add replay-safe behavior，but PeerManager +does not infer it from HTTP method or opaque payload。 + +### Rumination as the second capability landing（approved through D-185） + +The accepted explicit single-Block rumination trigger creates a second concrete pressure for the same generic delegation +machinery。Use exact capability `core.organization.rumination.v1` with one fixed +business inbound： + +```text +POST /organization/ruminate +body: { "block": <int> } +success: 204 No Content +``` + +A fixed route is important because `core.peer.protocol.http.v1` advertises one absolute inbound URL rather than a path- +template language。The Block identity therefore belongs in the capability-owned JSON body。The route is an action endpoint， +not a durable Rumination/run resource；its empty success mirrors `OrganizationManager.ruminate()`'s approved `None` +completion。 + +```text +client-web BlockDetailsPanel + -> client-web OrganizationManager.ruminate(block_id) + -> PeerManager.delegate(core.organization.rumination.v1, encoded HTTP payload) + -> PeerHTTPOutbound + -> provider fixed inbound / codec + -> provider OrganizationManager non-delegating local path +``` + +The public domain facade retains the same local-or-delegate shape as SemanticRetrievalManager；the provider inbound always +uses a non-delegating local path to prevent loops。PeerManager sees only the opaque exact capability and normalized protocol +envelopes。This does not yet freeze another public method name；the exact private implementation shape belongs to planning。 +A provider advertises support when its runtime contains the local implementation；Agent/provider/config +feasibility remains internal readiness。 + +This mutating capability is a stronger D-129 proof than retrieval。A pre-dispatch or exact +`InkCre-Peer-Execution: not-executed` outcome may select another provider。A normal `204` is executed completion，including +cannot-understand/no-write。Budget/failure responses after execution are returned without failover；timeout/reset after +possible dispatch is outcome-unknown and must stop，because replay could duplicate graph effects。 + +The proposed client-web action lives in the selected Block's existing `BlockDetailsPanel`。It is one explicit、non- +destructive “Ruminate” action with pending/success/error state，no confirmation、automatic retry、progress protocol or +cancel API。Success reloads the graph through its existing owner；the response cannot highlight exact new entities because +the approved organization result exposes none。Outcome-unknown tells the user to refresh/inspect rather than retrying +automatically。The UI/domain package never chooses a provider URL directly。 + +The Peer hard cut deletes legacy `Client(rest_api_url).request()` and its convenience methods rather than leaving a second +way for new capabilities to select a Core endpoint directly。The global `rest_api_url` field is already rejected by the +approved Peer persistence shape。Direct database Active Records remain legitimate for shared facts；callable business +capabilities use their domain facade、exact capability ID and PeerManager。Current client-web `Client.request()` additionally +assumes every success has a JSON body，so it is not adapted for rumination's `204` response。 + +### HTTP inbound public-address acquisition（approved through D-186） + +D-130 requires every advertised HTTP inbound to carry an absolute URL and removes a global endpoint from Peer identity。 +The provider runtime obtains its public base from its owner-specific persisted `peers.config`，initially exact field +`http_public_base_url` in the core-py Peer config model。Deployment may edit that shared row directly；client-web's existing +Client administration surface hard-cuts into a Peer view that edits the same owner config under its published +`config_schema`。This is per-Peer configuration and does not enter deployment-scoped `configs` or its schema registry。 + +At advertisement publication/refresh，the provider combines this base with each domain-owned fixed inbound path and writes +the resulting absolute URLs into its full capability snapshot。Config is authority；the snapshot is a routable derived +projection，so the repeated origin does not create a second independently authored address。Exact refresh cadence/change- +detection mechanics belong to implementation planning；no separate environment setting or public-address table is added。 + +The legacy `settings.client_base_url / CLIENT_BASE_URL`、Compose `CORE_PUBLIC_URL` projection and +`clients.rest_api_url` are deleted。Do not infer a replacement from bind host/port or an incoming request：`0.0.0.0` is not +a public address，TLS/proxy/path-prefix rewriting is invisible to Uvicorn，and advertisement exists before a request +arrives。The configured base is an absolute HTTP(S) URL and may include a deployment path prefix；normalization rejects +query/fragment/credentials before appending fixed paths。When absent，local execution remains available but no HTTP inbound +for those capabilities is advertised。 + +The exact HTTP representation that distinguishes a domain response from a protocol-guaranteed non-execution response is +one protocol response header with value `not-executed`。PeerHTTPOutbound alone interprets it and returns a protocol-neutral +internal outcome；absence means potentially executed regardless of HTTP status。The exact field is +`InkCre-Peer-Execution: not-executed`。RFC 6648 deprecates newly minted `X-*` parameters and RFC 9205 recommends a specific +application prefix。Browser callers require this field in CORS `Access-Control-Expose-Headers`。Acceptance crosses the +real deployment proxy；an intermediary that strips the field causes conservative no-failover，never unsafe replay。 + +The earlier rejected shape was: + +```json +{ + "method": "POST", + "path": "/semantic-retrieval", + "request": { + "target": "body", + "media_type": "application/json" + }, + "response": { + "media_type": "application/json" + } +} +``` + +It remains here only as failure evidence：query/body are not exclusive，and `media_type` needlessly renamed an existing +HTTP concept。 + +### Exact-target delegation and Extension management（approved through D-192） + +The legacy client-web Extension domain is an evidenced consumer of `Client.request()`：remote config update、enable and +disable currently select one Client by identity and construct Core HTTP paths from its `rest_api_url`。D-185 requires that +generic escape path to disappear，but deleting it without replacement would regress Extension administration and hot +lifecycle behavior。 + +PeerManager therefore exposes one routing entry over the same opaque exact-capability/protocol machinery： + +```text +delegate(capability, payload, route_to_peer: PeerRef | null = null) + null -> randomized eligible provider sequence + non-null -> exactly that eligible Peer or explicit unavailability +``` + +Exact-target delegation applies the ordinary database-time lease、exact capability、valid inbound descriptor and local +outbound-protocol checks，but never substitutes another Peer。`route_to_peer` is caller-local routing policy and never +enters the protocol payload or advertisement。The target identity is a routing constraint supplied by the business owner； +PeerManager still does not import or interpret that business domain。Its type is UUID-backed `PeerRef`，not legacy +`ClientRef` or an integer identity。 + +The first target-specific consumer is exact `core.extension.management.v1`。client-web's Extension domain addresses the +selected Peer and sends a capability-owned command；the provider's fixed inbound validates/decodes it and calls the target +runtime's non-delegating local ExtensionManager。This preserves Extension-owned config validation and hot enable/disable +effects without a generic capability-invoke endpoint。Exact request/action/error shape remains implementation-plan review， +not a reason to restore arbitrary `Client.request()`。 + +Do not replace this synchronous target command with database desired-state polling in the current unit。That alternative +would introduce observation cadence、invalid-config handling、live reconciliation and failure-recovery semantics that are +not present merely because Extension rows are shared facts。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/retrieval-contract.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/retrieval-contract.md new file mode 100644 index 0000000..0c51085 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/retrieval-contract.md @@ -0,0 +1,72 @@ +# Semantic Retrieval Contract + +> [Technical design index](index.md) + +## T-015 Retrieval Result、Score And Filters（approved by D-113–D-116） + +### Manager input + +```text +SemanticRetrievalManager.retrieve( + query: non-empty text, + profile: EmbeddingProfile reference | null, + options: VectorRetrievalOptions, +) -> SemanticRetrievalResult + +VectorRetrievalOptions + limit: positive integer = 20（public maximum 100） + min_score: finite float [-1, 1] | null = null + entity_types: set["block" | "relation"] = both +``` + +- null `profile` selects the deployment default through D-110；an explicit reference bypasses only default selection，not + Profile/model/provider validation。 +- query is text only。The legacy block-ID “more like this” path is a distinct similarity operation and has not been proven + by the semantic-query/Agent journey。 +- MVP exposes only filters shared by the graph result contract。Resolver/source/time/property filters remain future + pressure；do not mix feature retrieval or source-specific predicates into the first semantic API。 +- D-115 freezes `entity_types` as the only MVP candidate-filter dimension；the set is non-empty and defaults to both。 +- D-116 freezes bounded top-k retrieval，optional-null score threshold and no cursor/offset pagination。More than the + useful top results is query/semantic-preparation quality pressure，not a traversal use case；concurrent graph or embedding + maintenance may also legitimately change later rankings。 + +### Comparison and score（approved by D-114） + +- MVP performs exact cosine comparison over fresh，dimension-compatible records in one Profile。It does not expose a + one-value metric selector merely to claim extensibility。 +- public `score = 1 - cosine_distance`，so larger is better and `min_score` is caller-readable。Score is not confidence or + probability and is comparable only within the same query/Profile/metric execution。 +- candidate vectors and the query vector must be finite and non-zero for cosine comparison。Invalid provider output fails + before persistence/query rather than producing NaN ordering。 +- Block and Relation candidates are compared in the same vector space，merged into one global descending score order and + then globally limited。Exact ties use deterministic `entity_type` then entity ID ordering only for repeatability，not as + relevance evidence。 + +### Result（approved by D-113） + +```text +SemanticRetrievalResult + profile: EmbeddingProfile reference + metric: "cosine" + matches: SemanticRetrievalMatch[] + +BlockSemanticRetrievalMatch + type: "block" + entity: BlockModel + score: float + +RelationSemanticRetrievalMatch + type: "relation" + entity: RelationModel + score: float +``` + +- the result returns existing graph entities，not chunks、segments or a new target domain object。The discriminated match + wrapper carries use-derived ranking metadata without taking identity/authority away from Block or Relation。 +- return the full ordinary entity row so Relation matches are immediately navigable through `from_` / `to_` and Block + matches retain resolver/storage/content references。Do not duplicate resolver `get_text()` / Relation projection in the + result；a consumer that needs solved text resolves the returned entity through the ordinary graph capability。 +- expose one score authority，not redundant score + distance fields。The result-level Profile/metric makes that score + interpretable without repeating them per match。 +- missing/stale/unavailable records are simply absent from comparison。A successful query with no qualifying candidates + returns an empty `matches` list；configuration/provider/query-input failures remain explicit errors。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/rumination-agent-graph.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/rumination-agent-graph.md new file mode 100644 index 0000000..c07c7ea --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/rumination-agent-graph.md @@ -0,0 +1,257 @@ +# Rumination And Graph Forms + +> [Technical design index](index.md) + +## T-017 Rumination Approach And Interpretation Relation（approved through D-145） + +`rumination` is the concrete organization approach in which one focal Block is reconsidered in its graph context and a +more useful ordinary graph is materialized。An implementation may compose parsing、LLM、resolver materialization and linking +mechanics。Output relations should express discovered information value；a mandatory `part:<order>` lineage is rejected。 +A composition/order relation remains legitimate only when document structure itself is useful，not as proof that an +algorithm split text。 + +`breakdown` is retired from the technical vocabulary。It may remain historical/product shorthand for the observable case +where information aggregated in one Block becomes a richer graph，but it does not name a runtime abstraction、command、 +class、registry、DTO or job。 + +The operation may also materialize embedded information：for example，an authored URL can become an independently resolved +Block connected by a supported semantic relation。Extractive/source-faithful output remains the default evidence rule； +generated summaries or paraphrases require a distinguishable future graph grammar。 + +The focal Block is the true organization input。Its direct incoming/outgoing Relations form the initial graph-position +context；they do not turn the operation into a relation-list transformation。The concrete rumination implementation declares +the understanding capability required by its reasoning core，and the Block resolver supplies that capability from hydrated +actual content。The MVP text-LLM implementation therefore requests resolver `get_text()` even when the hydrated value happens +to be a string；runtime representation is not semantic permission to bypass the resolver。A future multimodal implementation +may require image、audio or structured understanding through the same pattern。At the product level，“Block content” means +the actual content after any storage pointer has been resolved；at the storage contract level，the persisted field remains a +pointer until hydration；at the application boundary，the resolver owns understanding。 + +If the resolver cannot supply the understanding capability required by this implementation，the implementation cannot +understand the Block；this does not mean rumination is inapplicable。The organization-facing contract exposes shallow +best-effort completion semantics：one consideration is completed with currently available capabilities，without promising +graph mutation。Cannot-understand and no-useful-output therefore complete silently rather than becoming public typed +outcomes。An execution that cannot complete may surface one high-level rumination failure，without leaking resolver、storage、 +AI-provider or persistence exception taxonomies。 + +MVP performs one attempt。Do not add internal retry、compensating rollback、degradation or fallback policy。A rich internal +outcome model is allowed only if concrete control flow needs it；module depth is not a reason to prebuild unused machinery。 +This follows the shared design pattern that a deep module may own rich internal distinctions while exposing the shallowest +completion semantics that preserve its abstraction-level promise。 + +The MVP text-LLM context is one bounded snapshot。It contains focal resolver `get_text()` output plus every direct +Relation's direction and exact content。The other endpoint is projected as an opaque Block reference、resolver name and +resolver `get_label()` output；its complete text/content is not recursively loaded。The projection preserves the dynamic- +property semantics `to is from's <relation-content>` while keeping the focal Block as the sole substantive input。 + +The approved MVP uses an Agent loop for native structured Resolver schema discovery、GraphForm drafting and eventual +`submit_graph` calls。It still has no graph-reading or navigation Tools and therefore cannot recursively explore neighbors； +the bounded D-145 snapshot remains the complete info-base read-side context。This introduces reusable Tool orchestration +without changing focal-Block authority or importing a future exploration policy。 + +MVP retains the input Block and all existing relations unchanged，adding only useful ordinary graph facts。It does not +delete、replace or generically rewire the target。A future explicit owner-approved replacement contract may reopen that +boundary；this unit does not spend design complexity on a low-benefit/high-risk destructive path。 + +Additive output need not expose a concrete world-property relation directly from the source。When evidence supports one， +use it（for example `highlight`、`need_adjustment` or `reference`）。Otherwise the source may point to one representative +entry Block of a valuable derived subgraph through an interpretation anchor。For a Markdown document，the title Block can +be that entry while quote/section relations form the internal graph。The anchor preserves source-versus-interpretation +authority and navigation without a mandatory direct edge to every output。The active exact relation content is +`interpretation`。 + +## T-018 Agent、Resolver Draft And GraphForm Boundary（approved through D-179） + +The desired authority direction is sound：organization adds connected Blocks/Relations through `GraphForm` and +`InfoBaseManager`，so an LLM should not produce a second graph proposal model merely for code to translate it back into the +same command。Modern typed tool calls can validate JSON arguments directly and avoid text parsing。 + +### Current boundary ledger + +This table is the current-state reasoning authority for the graph-command path。D-175–D-177 remain correction history；do +not compose superseded responsibilities from those snapshots into another topology。 + +| Boundary | Receives / produces | Owns | Explicitly does not own | +| --- | --- | --- | --- | +| Agent Tool runtime | untrusted ToolCall JSON → bound Pydantic Tool arguments | deserialization and one runtime validation pass，including the selected Resolver's code-owned input model；Pydantic failure → per-call ToolResult | Resolver semantics、graph normalization、persistence | +| `draft_graph` Tool handler | already validated Tool arguments → GraphForm | exact Resolver lookup，passing ordinary nested `input` to that Resolver，and adapting StarsGraphForm through the shared normalizer | repeated validation、Resolver-specific construction policy、persistence | +| `Resolver.create_graph(input)` | resolver-native ordinary input → StarsGraphForm | resolver-specific Block content、supporting graph and Relation grammar | Agent/runtime state、details of where/how its caller validated input、signed-ID allocation、persistence | +| `InfoBaseManager.normalize_graph(stars, id_start)` | StarsGraphForm → GraphForm | recursive traversal and deterministic command-local signed-ID allocation | Resolver selection、Resolver semantics、Tool validation、persistence | +| `submit_graph` Tool handler | already validated GraphForm → persisted-ID mapping | narrow Agent effect adapter into `InfoBaseManager.submit_graph()` | repeated structural validation、positive-ID existence probes、graph semantics | +| `InfoBaseManager.submit_graph(graph)` | ordinary GraphForm command → persisted Blocks/Relations | graph insertion coordination and negative→positive Block-ID mapping | Agent validation、Resolver construction、pre-querying positive endpoint existence | +| GraphForm / PostgreSQL | constructed Form / database write | Pydantic model owns intrinsic no-I/O structural invariants；PostgreSQL FK owns persisted endpoint existence and referential integrity | duplicate handler/Manager validation layers | + +“Validated” therefore describes how the Agent runtime obtained the Tool handler arguments；it is not a domain value、a +wrapper type or part of Resolver/InfoBase method names。Internal callers may construct the concrete resolver-native input +or GraphForm through ordinary typed/domain paths without entering AgentManager。 + +Producer grammar is closed。`BlockForm` / `RelationForm` omit database-generated identity、timestamps and other database- +managed state。Flat GraphForm adds the approved command-local signed Block-ID namespace only where one batch must declare +new Blocks and connect Relations to new/existing Blocks：negative means create，positive means persisted reference，zero is +invalid。StarsGraphForm continues to compose the same base Forms recursively for Resolver/extension authoring。The exact +Python container projection is implementation-plan work，not another design question unless preflight reveals a material +contract change。 + +The current `SubGraphForm` implementation is failure evidence rather than a compatibility surface： + +- `SubGraphForm.block` is a persisted `BlockModel`，whose generated schema exposes `id`、timestamps、storage、resolver and + content； +- each arc embeds a persisted `RelationModel`，exposing relation ID/timestamp and endpoint columns； +- recursive tree shape cannot cleanly express an existing Block by ID alone，shared references or arbitrary connected + graph edges； +- the model is therefore forced to understand persistence placeholders even though rumination owns those facts。 + +Do not solve that mismatch by adding an LLM-only graph domain。Producer-facing `BlockForm` / `RelationForm` remove +table-owned fields。Retain the recursively authored representation as `StarsGraphForm` for extension/source Resolver +construction，while flat `GraphForm` expresses arbitrary connected Blocks/Relations through signed references and is the +only Agent-visible graph command。 + +Resolver construction is semantically rooted：a Resolver interprets one resolver-native value as one subject Block and may +add supporting Blocks/Relations；nested Resolver calls make the complete result more than one mathematical star。 +The Agent Tool `draft_graph` accepts negative `id_start=-1` by default，invokes Resolver-owned StarsGraphForm construction， +normalizes it to GraphForm，guarantees that exact ID belongs to the subject/star Block and allocates remaining draft-local +IDs below it。This preserves rooted Resolver authoring without asking the LLM to translate representations。 + +`GraphForm` is self-contained and uses one non-zero signed-ID namespace。Positive IDs refer to existing persisted Blocks； +negative IDs identify new Blocks only within the current form；zero is invalid。Rumination includes real focal/direct- +neighbor IDs in its initial user message，and the LLM assigns unique negative IDs to proposed Blocks。After insertion， +InfoBaseManager maps negative IDs to database-generated positive IDs。The exact field remains `id`：Form semantics already +exclude database-managed values unless explicitly stated，and the negative value is a legitimate command-local identity。 + +Pydantic `GraphForm` validation owns structural invariants that require no I/O：new declarations use negative IDs，endpoints +are non-zero，new IDs are unique and every negative endpoint resolves inside the form。InfoBaseManager directly executes +the graph command without pre-querying positive IDs；PostgreSQL foreign keys remain the sole existence/integrity authority +for persisted Relation endpoints。The `submit_graph` Agent Tool handler does not repeat either layer。 + +Draft-capable Resolvers own a compact description、one Pydantic draft-input model and rooted StarsGraphForm construction。 +Do not expose persisted `block.content` schemas or pretend relation content is one globally closed enum。The rumination +Agent instead uses three bounded Tools： + +```text +get_draft_graph_schema(resolvers: exact Resolver IDs[]) + -> selected descriptions + Resolver-owned draft-input JSON Schemas + +draft_graph(resolver: exact Resolver ID, input: JSON, id_start: negative int = -1) + -> GraphForm whose star Block ID is id_start + +submit_graph(graph: GraphForm) + -> {blocks: [{local_id: negative int, id: persisted positive int}]} +``` + +The run's initial context lists only currently available draft-capable Resolver IDs and compact descriptions。AgentManager +binds `get_draft_graph_schema` / `draft_graph` argument enums from that same ResolverManager snapshot，so the detailed schemas +are fetched only after the model selects likely Resolvers。Agent runtime uses the selected Resolver's Pydantic model to +validate generic `draft_graph.input` before handler invocation；do not add handwritten parallel validation。Schema +discovery and draft construction perform no +info-base/storage mutation，but this is an effect-boundary promise rather than mathematical purity：Resolver-owned external +reads or computation remain possible。`submit_graph` is the sole graph-write Tool and receives only GraphForm。 + +Existing extension/source Resolvers keep ownership of native/canonical input → one subject Block plus supporting graph and +migrate their current recursive `SubGraphForm` call sites to `StarsGraphForm` using BlockForm/RelationForm。One reusable +InfoBaseManager normalizer owns StarsGraphForm traversal and signed-ID allocation，then produces GraphForm for Agent results +and graph-write coordination。Do not duplicate that conversion inside individual extensions、ResolverManager or Agent Tool +handlers。 + +The internal semantic method remains concrete `Resolver.create_graph(input) -> StarsGraphForm`。Extension/source +and other Managers may call it directly；it owns resolver-specific Block content and Relation grammar。The separately +registered Agent Tool `draft_graph(...)` receives arguments already validated by Agent runtime against its bound Tool +contract and the selected Resolver's Pydantic input model。Its handler obtains the exact Resolver through ResolverManager， +passes ordinary nested `input` to that same `create_graph()` method and asks InfoBaseManager to normalize the result with +caller-supplied `id_start` before returning GraphForm。It is a thin model-facing wrapper because it adds no graph- +construction or validation policy。InfoBaseManager owns normalization and `submit_graph(GraphForm)` persistence only；it +does not select Resolvers or expose `create_graph()`。 + +The approved reusable topology is： + +```text +Rumination / another organization approach + owns prepared UserMessage + selected persisted Agent + graph-aware Tool handler/mutation policy + -> AgentManager.run(agent_id, initial_message) -> active Thread + owns bounded model/tool turn orchestration，but no graph semantics + -> AIManager.chat(model, messages, tools, tool_choice) + owns model -> provider -> dialect routing and wire translation +``` + +This does not make AIManager graph-aware。A separate AgentManager is justified because system-prompt/tool composition and +typed tool dispatch have reuse across organization approaches and a stable caller-supplied-tool boundary；it does not own +provider/model registry or graph semantics。 + +### Rumination selection（approved through D-180） + +Rumination remains one function on the enclosing organization domain，`OrganizationManager.ruminate(...)`；the current +behavior does not justify a `RuminationManager` class or a registry of organization approaches。 + +The reusable AgentDefinition is an ordinary shared-database fact and may be provisioned or edited directly through that +database authority。An HTTP Agent CRUD surface may be added as a convenience projection，but is not part of the +coordination topology and is not required by this contract。The withdrawn seed/default proposal stays withdrawn。 + +The deployment selects its rumination Agent through the existing deployment-scoped `configs` relation： + +```text +key = core.organization.rumination +schema = core.organization.rumination.config.v1 +value = {"agent": <int>} +``` + +The organization domain owns the Pydantic value model under that exact schema contract。ConfigContract supplies generic +model-driven validation/normalization；DeploymentConfigManager resolves the registered schema contract and owns the shared +row lifecycle。Neither generic module invents the organization schema。`agent` is an `int` reference，not a strengthened +positive-integer contract。 + +`OrganizationManager.ruminate()` is the reference-use owner：it reads the deployment config，resolves the AgentDefinition +when invoked，constructs the approved focal/direct-relation `initial_message` and calls the exact stable entry +`AgentManager.run(agent_id, initial_message)`。Missing config and missing referenced Agent remain distinct explicit、 +repairable configuration failures。Agent deletion receives no reverse restriction；a dangling config is allowed until use。 +The invocation does not accept a per-call Agent override in the MVP。 + +### Organization-facing completion（approved through D-181） + +`OrganizationManager.ruminate(block_id)` is an async completion-oriented function。After preparing context and resolving +the configured Agent，it calls `AgentManager.run(agent_id, initial_message)`，awaits that Thread's active Turn Task and does +not return the Thread to its caller。This preserves AgentManager's approved Thread-returning contract while keeping Agent +execution state behind the shallower organization surface。 + +A naturally `completed` Turn maps to `None` regardless of whether any graph was submitted。A focal Block that the selected +implementation cannot understand also completes as `None` without starting an Agent run。When the Turn reaches +`max_model_calls` before natural completion，OrganizationManager exposes one organization-level failure；already completed +Tool side effects remain and no retry、rollback or compensation is added。Cancelling the caller's rumination coroutine +propagates cancellation to the awaited Turn Task and likewise preserves completed effects。No organization job、run entity、 +abort endpoint or Thread projection is introduced。 + +### Repeated execution（approved through D-182） + +Every invocation is an independent additive attempt built from the latest focal understanding and bounded direct-relation +snapshot。No run record、`last_ruminated_at`、content fingerprint、idempotency key or per-Block lock is introduced。A +submitted negative GraphForm ID always requests a new Block；only explicit positive IDs reuse existing Blocks。Repeated or +concurrent attempts may therefore create semantically or structurally duplicate Blocks/Relations，including after an +uncertain caller retry。MVP accepts that low-harm side effect；future merge/linking/reconciliation may address demonstrated +use degradation without turning rumination into an implicit deduplicator。 + +The current direct-relation snapshot may show that an `interpretation` or another derived edge probably came from an earlier +rumination，but it is not a freshness authority。It does not carry a run/dependency identity or prove whether focal content、 +Resolver behavior、Agent definition、neighbor facts or the deeper derived graph have changed。OrganizationManager must not +skip execution or declare the prior result current merely because such a direct edge exists。The Agent may still choose a +no-op from its limited context，but that is best-effort semantic judgment rather than an update guarantee。A future unit may +add explicit reevaluation/freshness semantics when real use demonstrates the need。 + +### Trigger policy（approved through D-184） + +MVP rumination runs only after an explicit request naming one focal Block and enters through +`OrganizationManager.ruminate(block_id)`。There is no collection-completion hook、new-Block event trigger、periodic scan、 +batch candidate selection or organization job。Collection success remains independent of organization capability、AI +availability and cost。 + +Automatic organization requires evidence and a separate policy for candidate eligibility、reevaluation/freshness、AI +budget/concurrency and Peer-safe claiming/execution。The current direct-relation snapshot cannot supply those decisions，so +the unit does not disguise a low-information repeated scan as scheduling。A future trigger design may reuse the same +ruminate function without changing its explicit single-Block contract。 + +MVP rumination provides Resolver-owned schema discovery/rooted GraphForm drafting plus one organization-owned +`submit_graph(GraphForm)` mutation Tool。It keeps the D-145 prepared focal text/direct-relation snapshot as ordinary model +input and provides no +graph-reading/navigation Tool。The early handwritten +`BlockManager.query_by_reasoning()` string-command pseudo-agent is deleted，not migrated。Agent definition、Thread and Tool +execution semantics are closed in T-019；rumination Agent provisioning/selection is closed by D-180，organization-facing +completion by D-181 and repeated execution by D-182。The generic Agent/Thread/Tool lifecycle +is owned by the separate +[Agent runtime contract](agent-runtime.md)。 diff --git a/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/shared-row-timestamps.md b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/shared-row-timestamps.md new file mode 100644 index 0000000..5af94a5 --- /dev/null +++ b/tasks/knowledge-lifecycle-capabilities/units/semantic-retrieval/technical-design/shared-row-timestamps.md @@ -0,0 +1,18 @@ +# Shared Row Timestamp Contract + +> [Technical design index](index.md) + +## Database-Owned `updated_at` Contract（approved by D-090） + +- Shared protocol row-mutation time must not depend on SQLAlchemy `onupdate` because PostgREST and other equal peers can + write the same relations。 +- A reusable internal-schema PostgreSQL `BEFORE UPDATE` trigger touches `updated_at` with statement time when the row + actually changes。No-op updates should not create false invalidation pressure。 +- The trigger is shallow：it does not delete、mark or rebuild embeddings。Block/Profile freshness consumers compare + database timestamps and schedule/perform derived maintenance separately。 +- For blocks，changes to content pointer、storage selection or resolver identity all change the semantic input boundary + and therefore touch `updated_at`。The same generic row-mutation behavior also covers relation endpoint/content changes。 +- Mutating bytes behind an unchanged storage pointer remains outside block row freshness；D-062/D-066 storage authority + boundaries are unchanged。 +- Do not apply this trigger to source-authored timestamps、job event columns or other values whose semantics are not + “database row last changed”。