Wiki-Link Retrieval

Rank-budgeted authored-hyperlink candidate expansion for curated knowledge bases.

Target venue: The ACM Web Conference 2027 (WWW 2027)

Current venue facts checked on 2026-07-14: the official ACM Web 2027 dates list Dublin, Ireland, May 10--14, 2027, with research/industry abstracts due October 11 and full papers due October 18, 2026 (AoE). The research-track CFP and final paper format are not yet published, so the repository follows the latest available research-track guidance with ACM sigconf,anonymous,review and a 12-page PDF audit as a planning constraint.

make verify-venue-metadata keeps these confirmed dates synchronized with the paper header and keeps inherited format constraints explicitly provisional until the WWW 2027 CFP is published.

Idea

Most LLM-powered knowledge systems retrieve documents by vector similarity (ANNS), ignoring the explicit link structure between documents. GraphRAG-style approaches build entity graphs from unstructured text, adding model-based extraction and graph-processing stages before retrieval.

We observe that curated knowledge bases (team wikis, documentation sites) already contain explicit, human-authored links between pages. Instead of extracting a graph from text, we parse the existing link structure and use it for post-retrieval expansion:

Query → ANNS retrieves top-k documents
      → optional 1-hop expansion via pre-built adjacency list
      → caller-selected direct injection, rank budget, or reranking

This reuses the collection's existing inter-page structure with a small post-retrieval lookup step in the current hot-path microbenchmark, without entity extraction, community detection, or an extracted entity graph. The SAGE integration keeps expansion disabled by default because uncalibrated injection can displace stronger first-stage hits; deployments must opt in and choose the rank policy appropriate for their downstream consumer.

WWW 2027 Positioning

The paper should be framed as a Web retrieval systems paper: curated Web knowledge bases and documentation sites already contain human-authored hyperlinks, but modern RAG pipelines often flatten those pages into independent chunks. Wiki-Link Retrieval restores that Web-native signal at retrieval time without LLM-based graph construction.

The current submission bar is tracked in roadmap.md. The main freeze risks are publishing the submodule feature branches and replacing legacy artifacts with clean canonical reruns. The unreachable 2Wiki hyperlink archive remains an external-validity warning, not a current-paper blocker. Standard-IR transfer evidence is required only if the paper restores a transfer claim beyond multi-hop QA. Strict DPR-only, BM25+DPR RRF, and rank-normalized hybrid artifacts are now checked in. The protected-prefix and rank-normalized calibrations treat links as candidate generation with explicit score budgets, not as a learned ranking improvement. The current paper deliberately does not make a headline GraphRAG/LightRAG quality claim; the checked-in GraphRAG scaffold artifact is kept only as a sanity check for the data path, while the paper compares published pipeline stages and representations without claiming measured cost.

Architecture

┌──────────────────┐     ┌──────────────┐     ┌────────────────────┐
│ Docusaurus wiki  │────>│  Ingestion   │────>│  JSON KB entries   │
│ (authoring)      │     │  + link parse│     │  + adjacency list  │
└──────────────────┘     └──────────────┘     └─────────┬──────────┘
                                                        │
                                              ┌─────────▼──────────┐
                                              │  ANNS retrieval    │
                                              │  (sagevdb + HNSW) │
                                              └─────────┬──────────┘
                                                        │
                                              ┌─────────▼──────────┐
                                              │  Lexical reranking │
                                              │ + optional links   │
                                              └─────────┬──────────┘
                                                        │
                                              ┌─────────▼──────────┐
                                              │  Final top-k hits  │
                                              │  with linked context│
                                              └────────────────────┘

Key Properties

Property	Vanilla ANNS	GraphRAG	Wiki-Link Retrieval
Inter-linking	None	Auto-extracted	Human-authored
Additional indexing	Embeddings	LLM extraction + graph stages	Embeddings + O(E) link parse
Query operation	ANNS + rerank	Graph/vector retrieval and reasoning	Retrieval + O(kd + c log c) expansion
Link source	N/A	Model-extracted relations	Human-authored links (quality is corpus-dependent)
Added graph stages	None	LLM extraction + graph processing	Markup parsing + adjacency list
Update path	Index changed documents	Refresh affected extraction/graph stages	Re-parse links after content sync

Here d is the average degree of retrieved nodes and c is the number of unique unseen link candidates considered for ordering.

Auto-Sync (CI/CD)

When wiki pages are added or updated in sage-wiki, the KB automatically absorbs the new knowledge:

sage-wiki push → systemd timer (every 30min)
                → git pull sage-wiki
                → ingest_wiki.py (upsert + rebuild link graph)
                → KB is live with new content

• Sync script: tools/sync_wiki_kb.sh (in sage-faculty-twin)
• Timer: sage-faculty-twin-wiki-sync.timer (systemd, 30-min interval)
• Idempotent: safe to run even if wiki hasn't changed (skips ingest)
• Manual trigger: bash tools/sync_wiki_kb.sh for immediate sync

Repository Structure

wiki-link-retrieval/
├── paper/          # LaTeX paper drafts, figures
├── experiments/    # Benchmark scripts comparing retrieval quality
├── results/        # Experiment result data (JSON/CSV)
├── third_party/    # Pinned SAGE/runtime submodules
└── README.md

Reproducibility

Canonical first-stage frontier trace gate

Future query-local storage studies must use the actual ranked document frontier produced by experiments/benchmark_multihop.py, not natural-language search actions or answer-side labels. Opt in with --frontier-trace-output PATH. The benchmark records public dataset and authored-graph SHA-256 identities, original query order, query/request identity, and only the existing pre-expansion rank/document/score fields. It also binds the producer commit and configuration.

The JSONL trace is published with a same-filesystem temporary file, fsync, and atomic rename; duplicate queries, cross-request identity, stale schema, missing source digests, noncanonical link controls, and partial writes fail closed. Gold labels, qrels, supporting facts, answers, future hops, answer-derived edges, and artificial graph mutations are excluded.

This gate only supplies future workload provenance. Trace generation is not a benchmark result, and the artifact always carries performance_claim=false. The all-cold snapshot and fixed 64-entry SGLang-trace hot-tier configurations remain closed negative results.

Fast local check:

bash scripts/reproduce.sh fast

This verifies repo-local SAGE/runtime submodules, checks that experiment entry points expose safe --help CLIs, inspects local dataset inputs when they are present, checks the checked-in result artifacts with scripts/verify_results.py, verifies results/CHECKSUMS.sha256, guards headline paper numbers against those artifacts, regenerates table fragments, regenerates deterministic figure PDFs, and rebuilds paper/main.pdf. The submission audit also regenerates figures in a temporary directory and checks them byte-for-byte against the checked-in PDFs. It does not download datasets or rerun large benchmarks.

The figure steps can also be run directly:

make generate-figures
make audit-figures

Canonical rerun and publication:

bash scripts/reproduce.sh full

Full mode fixes both multi-hop and hybrid subsamples at zero, runs the complete paper-facing grids, and publishes canonical artifacts only after all freeze checks pass. It requires clean, published submodule pointers, but it does not require the unavailable 2Wiki authored-link graph because no 2Wiki result or cross-dataset claim appears in the current paper. The optional extended suite still fails rather than falling back to the quarantined question-context graph.

Dataset input check:

make verify-dataset-inputs
scripts/run_in_conda.sh python scripts/verify_dataset_inputs.py \
  --required --datasets hotpotqa

The default check is advisory so a fresh clone can still verify checked-in paper artifacts without downloading public datasets. The --required mode is intended for full reruns and fails when local paper-facing dataset files or required parsers are missing. experiments/benchmark_multihop.py first looks for data/hotpot_dev_distractor.parquet and data/2wikimultihop_dev.json; if they are absent, it falls back to public HuggingFace dataset loaders for HotpotQA (hotpotqa/hotpot_qa) and 2WikiMultiHopQA (Alabaster/2wikimultihop_dev, then voidful/2WikiMultihopQA).

The benchmark files do not directly provide a complete authored-hyperlink graph. Paper-facing runs therefore require provenance-bearing graph JSONL files. The canonical HotpotQA graph is checked in as data/hotpotqa_authored_links.jsonl; 2Wiki remains a diagnostic until data/2wikimultihop_authored_links.jsonl can be extracted from its official hyperlink-bearing archive. Each graph starts with provenance metadata, followed by adjacency records:

{"type":"metadata","schema_version":1,"source_type":"wikipedia-authored-hyperlinks","wikipedia_snapshot":"DATE_OR_DUMP_ID","extraction_method":"PARSER_AND_VERSION","link_scope":"retrieved-document-intro-paragraph"}
{"source":"Article title","targets":["Linked article","Another article"]}

--link-source authored is the default and refuses a missing graph. context-cooccurrence is retained only for diagnostics; its outputs are labeled diagnostic-context-cooccurrence and are not publication evidence.

Use hyperlink-bearing releases, not stripped question contexts. The canonical HotpotQA extractor reads the pinned hotpot_qa_wiki.abs_adj transport from TIGER-Lab/LongRAG at revision d3983d83a3bf90cd3c489303fe392f0a426b3b73. The matching upstream LongRAG preprocessor at revision ba92f957f4e39b98f9c79c42f751228b95ee086f defines this field as the bidirectional closure of links from the October 1, 2017 introductory paragraphs. Because the transport is already a closure, it cannot support an outbound-only ablation. The canonical extractor uses HotpotQA context titles only to intersect that independently constructed Wikipedia closure with the pooled retrieval corpus; it does not use questions, supporting-fact annotations, or qrels to create edges. verify_authored_link_evidence.py pins both upstream revisions and rejects duplicate targets or a claimed closure with any missing reciprocal edge.

2WikiMultiHopQA publishes para_with_hyperlink.zip with a para_with_hyperlink.jsonl member containing ref_url/ref_ids mention metadata. The source declaration is pinned to official repository revision 13800e5be57df1b4040b9b1588c6c811779e69e9. The official share currently redirects to the scl URL below; Dropbox may still be unreachable in restricted execution environments.

Before starting or resuming the large transfer, diagnose DNS resolution and redirect reachability without downloading the archive:

make diagnose-2wiki-archive

The diagnostic accepts either the official legacy share or its current SCL target, rejects HTTP errors, and reports every locally resolved address. When local DNS is suspect, force one or more independently obtained Dropbox addresses without changing the TLS hostname:

scripts/run_in_conda.sh python scripts/diagnose_2wiki_archive_access.py \
  --resolve-ip 162.125.6.18 \
  --resolve-ip 162.125.248.18

This is only a network preflight. A successful response, including one through forced resolution, does not replace the complete member, schema, record-count, and ZIP CRC checks below.

HF_ENDPOINT=https://hf-mirror.com \
scripts/run_in_conda.sh python scripts/extract_hotpot_intro_links_longrag.py \
  --output data/hotpotqa_authored_links.jsonl

curl -L -C - \
  -o data/para_with_hyperlink.zip \
  'https://www.dropbox.com/scl/fi/p6xcpt4a7wxzqsa58kkko/para_with_hyperlink.zip?rlkey=tzei8xc346a8e2dx8h934p7t1&dl=1'

scripts/run_in_conda.sh python scripts/extract_benchmark_authored_links.py \
  --dataset 2wikimultihop \
  --source data/para_with_hyperlink.zip \
  --validate-only

scripts/run_in_conda.sh python scripts/extract_benchmark_authored_links.py \
  --dataset 2wikimultihop \
  --source data/para_with_hyperlink.zip \
  --output data/2wikimultihop_authored_links.jsonl

make verify-authored-link-evidence

The v5 extractor streams the official JSONL member instead of loading all 5,989,847 paragraph records into memory. --validate-only checks the unique archive member, every JSON record and mention, the exact record count, and the ZIP member CRC before a graph is written. The extraction path repeats these checks before publication. Its graph metadata records the official repository and revision, share URL, archive filename and SHA-256, and archive member, record count, and validated schema. It records both the historical URL committed in the pinned official README and the current SCL redirect target. It also records benchmark-title matches, raw authored targets, linked sources, and directed in-corpus edges. The verifier cross-checks those counts against the graph and rejects missing or malformed source metadata, empty graphs, duplicate source records, and invalid adjacency lists. The expected record count is independently pinned to the 2Wiki loader in datastax/graph-rag at revision f092b3fc56d02f83b7a4bc7892c506251f2d3ea6; it is not inferred from the local question contexts.

Raw archives and cached shards stay ignored; compact canonical graph JSONL files are versioned so their checksums and benchmark artifacts can be reproduced.

An exploratory HotpotQA graph can also be extracted from the full-page Wikipedia records mirrored by ParthMandaliya/hotpotqa-wiki. This graph is useful for testing sensitivity to document scope, but it is deliberately rejected by canonical --link-source authored runs because the benchmark retrieval documents contain only introductory paragraphs:

scripts/run_in_conda.sh python scripts/extract_hotpot_fullpage_links.py \
  --output data/hotpotqa_fullpage_authored_links.jsonl

source scripts/activate_conda_env.sh
scripts/run_in_conda.sh python experiments/benchmark_multihop.py \
  --datasets hotpotqa \
  --retrievers bm25 \
  --alpha 0.3 0.5 0.7 1.0 \
  --max-expansion 1 3 5 10 \
  --protected-prefix 0 \
  --seeds 42 43 44 45 46 \
  --subsample 0 \
  --link-source authored-fullpage-diagnostic \
  --link-graph hotpotqa=data/hotpotqa_fullpage_authored_links.jsonl \
  --analyze-coverage \
  --require-pyserini \
  --output results/multihop_hotpotqa_fullpage_authored_diagnostic.json

The checked-in full-page diagnostic reaches 94.5% one-hop coverage of missed gold documents, yet its best tested expansion setting reduces nDCG@10 from 0.6634 to 0.5091. It is a scope-mismatch stress test; the canonical introductory-paragraph graph must be used for main results.

Degree-preserving shuffled-link null control:

scripts/run_in_conda.sh python experiments/benchmark_multihop.py \
  --datasets hotpotqa \
  --retrievers bm25 \
  --alpha 0.3 \
  --max-expansion 10 \
  --protected-prefix 0 \
  --seeds 42 43 44 45 46 \
  --bootstrap-samples 10000 \
  --bootstrap-seed 2027 \
  --subsample 0 \
  --link-source authored \
  --link-graph hotpotqa=data/hotpotqa_authored_links.jsonl \
  --link-control degree-preserving-shuffle \
  --link-control-seed 2027 \
  --link-control-swaps-per-edge 5 \
  --analyze-coverage \
  --require-pyserini \
  --output results/multihop_hotpotqa_degree_null_TIMESTAMP.json

The control performs deterministic simple-undirected double-edge swaps while preserving every node's degree, the node set, and the edge count. Its output is always labeled diagnostic-degree-preserving-link-null, records the source graph checksum and before/after degree-sequence hashes, and cannot be published as an authored-link canonical artifact. A clean freeze rerun is required before using this control in the paper.

Submission-facing audit:

make audit-submission
scripts/run_in_conda.sh python scripts/audit_submission.py --strict-pages
make audit-final-preview

The audit enforces at most 8 main-content pages and 12 total pages by locating the rendered References heading. If references begin on page 9, it rejects any non-header body text before that heading, preventing a conclusion spill from masquerading as an eight-page main body. It also checks anonymous-review hygiene, ACM placeholder warnings, high-risk unsupported claims, and TeX overfull boxes larger than 1pt in the rendered PDF. make audit-final-preview builds the same anonymous draft without ACM review-mode line numbers so layout can be inspected before the final submission branch changes author/copyright metadata. The result manifest records the current artifact structure, evidence labels, and known non-determinism for checked-in result files. Update paper-facing checksums only when intentionally replacing those artifacts:

scripts/run_in_conda.sh python scripts/audit_run_metadata.py
scripts/run_in_conda.sh python scripts/verify_artifact_checksums.py --update

The run-metadata audit verifies that paper-facing artifacts carry evidence labels and provenance. Strict mode additionally requires every offline retrieval artifact to identify a local dataset file by SHA-256, record evaluated corpus, query, and relevance-judgment counts in dataset_stats, agree with every result row's query count, and come from a clean parent commit. It also requires the actual .conda/env interpreter and key package versions, plus the commit, feat/* branch, pointer state, and clean-worktree state of all 12 recursive source submodules. Before freezing the submission branch, replace older artifacts with fresh runs until the strict gate passes:

scripts/run_in_conda.sh python scripts/audit_run_metadata.py --strict
make verify-hybrid-artifact
make verify-calibrated-hybrid-artifact
make audit-www-readiness

Full rerun entry point:

make bootstrap-full-rerun-env
source scripts/activate_conda_env.sh
make cache-dpr-models
make verify-dpr-model-cache
make verify-full-rerun-prereqs
bash scripts/reproduce.sh full

The bootstrap creates both the Conda prefix and package cache under the ignored .conda/ directory in this checkout. Tests and experiment launchers use scripts/run_in_conda.sh, so they do not fall back to a shared Python or Conda environment when the activation step is omitted. SAGE, sageVDB, and Faculty Twin are installed editable from their pinned submodules; the aarch64-incompatible sage-anns source checkout is pinned for feature work while its compatible 0.2.0 release wheel supplies the current runtime backend.

The full mode runs the benchmark suite before verification and paper rebuild. Paper-facing reruns require real retriever dependencies: Pyserini and OpenJDK 21+ for BM25, pyarrow for the local HotpotQA parquet file, and torch, transformers, plus cached DPR encoder models for the BM25+DPR RRF hybrid rerun. make cache-dpr-models populates the HuggingFace cache when the network can reach HuggingFace; make verify-dpr-model-cache checks the same DPR cache in offline mode and prints the cache root being used. Add --include-colbert to the cache/preflight scripts only if restoring optional ColBERT boundary claims. When the primary HuggingFace endpoint is unavailable, cache the same pinned models through the mirror used by the current reproducibility setup:

HF_ENDPOINT=https://hf-mirror.com make cache-dpr-models
make verify-dpr-model-cache

The scripts still allow lightweight fallbacks for local smoke tests, but canonical publication uses strict flags so fallback results cannot silently become paper evidence.

Deterministic BM25, DPR, and RRF rankings and their deterministic link expansions are computed once and reused across requested seed labels. Per-query metrics for a shared run are likewise computed once and weighted by the number of labels that reference it. The strict result schema reports deterministic 10,000-resample paired-query bootstrap 95% intervals for full-set mean metric deltas. Reachability-stratum summaries remain descriptive and do not receive subgroup intervals. Statistical inference uses queries as paired units; when a smoke-test fallback is randomized, distinct runs are expanded separately and repetitions are averaged within each query before effect estimation. Query-effects schema v2 sets sweep_hypothesis_tests: none and rejects treatment-row p_value fields, because the alpha/budget/prefix grid is descriptive rather than a family of pre-registered hypotheses. Legacy seed-level p-values remain in old JSON files for provenance but are not used in the paper. The semantic-reranking freeze artifact additionally requires the mean paired nDCG@10 delta, a 10,000-resample paired-query bootstrap 95% interval, and a two-sided Wilcoxon signed-rank test with Pratt zero handling. Its old paired t-test value is retained only in the pre-freeze JSON and is not cited by the paper. Strict DPR and hybrid artifacts also record the resolved 40-character HuggingFace commit hash for each DPR encoder; model names without pinned cached revisions are rejected at freeze time. The canonical CPU command uses --dpr-device cpu --dpr-cpu-threads 32; scripts/full_rerun_env.sh applies the same OMP/MKL/OpenBLAS limits before Python starts. This avoids oversized native thread pools on many-core aarch64 hosts, and prevents a CPU run from importing and registering torch_npu implicitly.

Containerized runs:

bash scripts/run_docker.sh --build
bash scripts/run_docker.sh hotpotqa
RUN_IMAGE=wiki-link-exp bash scripts/run_docker.sh

The default Docker runner still uses the Ascend base image directly and installs paper-facing Python dependencies plus OpenJDK 21 at container start. Building the wiki-link-exp image moves those dependencies into the image instead. The runner mounts the host HuggingFace cache into the container so DPR encoder downloads can be reused across reruns; build with CACHE_DPR_MODELS=1 only when network access to HuggingFace is available.

Full runs write timestamped result files by default. They deliberately do not overwrite the canonical JSON files used by the paper, because some suite entries use subsamples for turnaround. After a clean full-size rerun intended to replace the paper evidence, publish canonical artifacts explicitly:

MULTIHOP_SUBSAMPLE=0 \
HYBRID_SUBSAMPLE=0 \
HOTPOT_ALPHA="0.3 0.5 0.7 1.0" \
HOTPOT_MAX_EXPANSION="1 3 5 10" \
SUITE_SCOPE=paper \
PUBLISH_CANONICAL=1 \
bash scripts/run_experiments.sh

SUITE_SCOPE=paper is the default and runs the seven evidence-producing steps used by the submission. SUITE_SCOPE=extended additionally runs the quarantined 2Wiki authored-link extension, standard-IR scaffold, and GraphRAG scaffold; it requires their independent inputs and does not weaken provenance checks.

The script refuses canonical publication if these paper-facing settings are not present. Before the suite writes anything, it captures the parent commit and requires a clean worktree; later steps reuse that snapshot while allowing only the suite's own results/ writes. It also fails early if strict BM25 and BM25+DPR hybrid dependencies are missing. Canonical publication queues use scripts/publish_artifact.py as the final copier after rejecting mismatched source paths, conflicting evidence labels, dirty snapshots, or incomplete run metadata. Publication is two phase: every timestamped source first passes its specialized verifier and a validate-only publisher check, and canonical paths are updated only after all selected suite steps succeed. During reproduce.sh full, inventory, suite, RM3, full-page, and structural artifacts share one persistent queue under the checkout-owned .conda/ directory; the entire queue is revalidated after the last experiment before any canonical path is updated. Queue validation includes the generic strict metadata audit, so the clean parent snapshot, project Conda interpreter, all 12 recursive feat/* submodules, dataset identity, and query-effect schema are checked on each timestamped source rather than only after publication. A failed experiment or validation therefore leaves the previous canonical generation intact. Before the final write loop, the queue snapshots every existing destination under the checkout-owned .conda/ directory. If any publisher fails, it restores all destinations and removes any newly created ones, so a handled write failure cannot leave a mixed canonical generation. The retained queue can be rerun after resolving the underlying storage error.

A standalone paper suite owns and publishes a seven-entry queue; the extended suite owns a nine-entry queue because its standard-IR scaffold has no canonical publication target. Standalone run_rm3.sh owns a one-entry queue. When these scripts run under reproduce.sh full, they append to its shared 12-entry queue and cannot publish early.

bash scripts/reproduce.sh full additionally requires DEPLOYMENT_KB_DIR and refreshes the RM3, full-page scope diagnostic, and deployment-case artifacts. This is the complete current-paper freeze path; it fails early instead of silently retaining old canonical files.

See results/MANIFEST.md for table-to-artifact provenance and known gaps.

To print the exact remaining freeze commands from the current checkout:

make print-freeze-handoff

make audit-references additionally rejects undefined or duplicate citation keys and pins the title, author list, and DOI/arXiv identity of the closest prior work used to define the novelty boundary, including authored-link, induced-link, multi-hop, and adaptive graph--text retrieval alternatives.

The pre-rerun command uses verify_authored_link_evidence.py --inputs-only so stale canonical artifacts do not create a bootstrap deadlock. The final freeze gate reruns the verifier without that flag to require graph-to-artifact checksum agreement. make print-freeze-handoff first prints HotpotQA-only dataset and graph preflights plus an independent semantic-reranking refresh; this work can proceed before the official 2Wiki graph is available. The later full-suite phase repeats both preflights without --datasets, so it still requires every paper-facing dataset. Its selective commands use strict publication queues too: the six dense/calibration/control artifacts publish as one group, while full-page, case-study, and inventory reruns use exact-size queues of their own. The optional ColBERT command keeps only its timestamped output because ColBERT is not currently registered as paper-facing evidence.

The implementation code lives in the sage-faculty-twin repository (knowledge_base.py, tools/ingest_wiki.py), pinned here as third_party/sage-faculty-twin, as the feature is tightly coupled with the existing KB search pipeline. Wiki source content is pinned as third_party/sage-wiki.

Initialize submodules before running implementation-backed experiments:

git submodule update --init --recursive
make verify-submodules
make verify-submodules-remote

All implementation and runtime changes for this paper should be made inside the repo-local submodules on feat/wiki-link-retrieval-www2027, not in sibling checkouts outside this repository.

The recursive checkout includes accelerator and vendor dependencies even when a particular experiment does not exercise that backend. make verify-submodules checks the complete SAGE, vLLM-HUST, Ascend, Metal, CATLASS, dev-hub, and runtime-manager submodule topology used by this project.

Before changing a dependency, check out the feature branch inside the submodule, commit the dependency change there, then update the parent submodule pointer. The parent repository should stage explicit paths only: the submodule pointer, paper/experiment code that consumes it, and documentation that explains it. make verify-submodules fails dirty submodule worktrees and reports missing upstreams as warnings; make verify-submodules-remote turns missing upstream or origin/feat/wiki-link-retrieval-www2027 tracking refs into hard failures before pushing a stable branch.

Research Questions

1. RQ1: What recall-ranking behavior does authored-link expansion exhibit on all HotpotQA distractor-dev queries over their pooled 66.6K-document context corpus?
2. RQ2: Can protected-prefix or rank-normalized calibration retain expanded recall while bounding top-rank disruption?
3. RQ3: In a heterogeneous KB where imported documents lack explicit links, can concept-keyword linking bridge the one-hop structural gap without LLM-based graph extraction?

Status

• Research repo created
• Initial architecture designed
• Wiki ingestion pipeline implemented (tools/ingest_wiki.py)
• Link graph builder implemented (bidirectional adjacency in knowledge_base.py)
• Post-retrieval expansion in search pipeline (local + sagevdb + neuromem)
• Experiment: retrieval quality baseline (vanilla ANNS)
• Historical controlled-corpus link-expansion probe retained as non-paper-facing design context
• Ablation studies (decay, max_expansion, direction)
• GraphRAG scaffold sanity artifact (not a WWW headline quality result)
• Paper draft (paper/main.tex)
• Structural-gap ablation for a fingerprinted 646-document heterogeneous deployment
• Complete distractor-dev HotpotQA BM25, DPR, BM25+DPR RRF, protected-prefix, rank-normalized fusion, and fixed zero-shot DPR reranking results over the pooled context corpus
• Replace the historical PRF prototype artifact with an Anserini/Lucene RM3 rerun before restoring the RM3 comparison to the paper
• Regenerate those paper-facing retrieval artifacts from a clean parent commit with pinned local dataset SHA-256 provenance
• Fast result verifier and reproduction entry point
• SAGE implementation/content dependencies pinned as submodules on project feature branches
• Core result tables generated from JSON artifacts
• Submission audit enforces explicit first-page Web relevance, a repository-level 250-word abstract budget, 8 main-content pages, and 12 total pages
• WWW-oriented introduction and contribution rewrite
• GraphRAG/LightRAG quality comparison removed from headline claims
• Optional full LLM-based GraphRAG/LightRAG comparison for a stronger baseline section
• Strict DPR-only and hybrid sparse+dense boundary artifacts checked in
• Protected-prefix rank-budget calibration checked in
• Rank-normalized hybrid calibration checked in
• Scoped public documentation link-density inventory for motivation
• Canonical 2Wiki authored-link rerun after acquiring the complete official archive