Skip to content

fix(workspace): keep real native benchmark results distinguishable - #8025

Merged
huangruiteng merged 1 commit into
mainfrom
codex/research-native-reference-20261104
Nov 4, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/research-native-reference-20261104

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

The real Ad placement capture has a best value of 49,653,326,363 and a final value of 49,650,446,168. Both previously appeared as 4.965E10, hiding their difference in the default research view. Scientific notation now adds only enough digits to keep distinct captured measurements distinguishable; the original values, source owner and comparison semantics are preserved.

The core research-real-benchmark journey now reads the actual registered Portfolio/Ad native-result captures alongside Graph's saved analysis and the recent complete-history references. Portfolio's initial 0 stays measured zero. Both new captures have no saved analysis job/result or retained result; the reader leaves those unknown. Graph's fixed baseline, retained agent-16, final agent-18 and terminal retention citation can all be read directly.

  • Outcome basis: maintainer request to use authorized real benchmark display state as the core frontend reference.
  • Intended base: main at a0130633723210a5406dca61ee27aa33fe48b3f3.

Author Declaration

Criterion Disposition Implementation and decisive evidence
Read real measured results without hiding differences implemented Existing NativeResults formatter; the real Ad best/final labels must differ before opening evidence
Preserve source identity, zero, unknown and independent analysis cost implemented Unchanged captured HTTP responses with independently pinned byte hashes; Portfolio zero, missing retention/job/result and original Graph usage checks
Read full saved judgments and recover from unavailable evidence implemented All 36 Graph explanations in both interfaces; uncaptured citation error followed by a successful captured read
Review populated whole-page desktop/narrow behavior implemented Packaged core journey and screenshots below, both interface languages at 1440/390px

Self-check: reviewed the actual native results and saved analysis before editing; reused the existing typed owner and source reader. The small local formatting refinement creates no protocol, score, state vocabulary or execution path.

Scope And Continuation

This fixes the demonstrated presentation defect and makes real native results an executable core reference. The documented Graph result is a +0.032297817031212595 raw retained comparison from agent-2 to agent-16, not an isolated causal contribution of LoopX. Its saved negative/unknown judgments remain unchanged. Portfolio and Ad do not acquire an adoption claim from their best or final scores.

The broader research-history work remains open for corresponding complete trajectories, additional saved analysis where absent, complete-cost/input attribution and installed-product qualification. This PR does not run a model, solver, evaluator, install or release, and does not establish those outcomes. No new successor is created just for this bounded fix.

Validation

  • Tested revision: f3ec9eb99c0884f0e849ee0fed3729405554978a.
  • Run state: finished.
  • Input classes: public_fixture, synthetic, authorized_private_read_only.
Check kind Result Evidence / limitation
static passed Chat production build/typecheck; touched Python Ruff, JavaScript syntax and Git whitespace
unit passed uv run --extra test python -m pytest tests/presentation/test_research_history_fixture.py -q: 7 passed; original HTTP hashes, zero/unknown semantics, exact citations, denied mutation verbs and source preservation
real_entrypoint passed Packaged research-real-benchmark: 17.357s; real histories plus three scoped native-result cases, all captured citations, Graph's 36 judgments, unavailable-source recovery, Chinese/English and desktop/390px
real_backend passed Production HTTP reader of three already registered read-only benchmark analysis Goals; actual saved results/native measurements, execution disabled and zero mutation attempts. After the fix, Ad's real best/final read back as 4.9653E10 / 4.965E10
regression_parity passed Before capture shows both distinct Ad values as 4.965E10; head makes them distinct. Existing Grid/Ad/Schema histories and original Graph response hashes are preserved
manual passed Inspected actual packaged desktop/narrow screenshots; full source values remain one click away; unknown retention stays visible

The source build was regenerated after updating the branch to the new base; the final packaged check passed on the stated revision. CI and independent review remain separate. No installed App, model dispatch, full research cost or causal effectiveness claim is qualified here.

Frontend / Visual Evidence

UI impact: changed. Source data: public_fixture. After views use the production packaged UI at the tested revision; they replay authorized captured GET responses, without restoring execution state. No public preview is deployed. A disposable read-only preview is reproducible with:

uv run --extra test python examples/personal-workspace-browser/research-history-backend.py --fixture apps/presentation/dashboard/src/data/fixtures/edgebench-ad-native-results.json

Before — Ad desktop first viewport, original formatter:

ad-before-en-1440

After — Ad desktop first viewport; best and final now differ:

research-real-native-ad-en-1440

Ad narrow reading view (390px, scrolled to native results):

research-real-native-ad-en-390

Graph detail: fixed baseline, retained versus final result, and unknown process judgments:

research-real-native-graph-en-1440

Portfolio desktop first viewport: measured zero and unknown retention, no fabricated analysis:

research-real-native-portfolio-en-1440

Attention/craft review: keep measured results prominent and source/cost detail on demand. The extra numeric precision communicates an actual distinction without adding a label, card or control. Equal values still display equally; unavailable evidence has an explicit error and recovery. The narrow view wraps within its viewport. Existing hierarchy, status glyphs, colors and motion are retained.

Type / Area / Boundary

  • Bug fix, test and documentation update; public presentation surface.
  • Direction: operator research reader and long-horizon evidence.
  • Shared-authority RFC fixture impact: N/A; no authority/provider/promotion change.
  • Public boundary reviewed: bounded display responses and native citations only; no private context, keys, raw sessions, hidden evaluator internals or local machine paths.
  • Existing typed owner reused; no benchmark execution duplicated.
  • No paired RFC changed.
  • Every commit has a DCO sign-off; first-screen evidence is embedded here.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机 / Motivation

阅读已保存研究结果的用户,需要在打开原始证据前区分最佳值、最后提交和实际保留结果。Ad 的最佳值49,653,326,363与最后值49,650,446,168都显示为4.965E10,用户仅看结果区会把不同测量当成相同;缺失分析和保留证据的真实样本也缺少完整阅读回归。现在结果区按需要增加科学计数法精度,两个值直接可辨,完整原值仍可点开;真实零值、未知保留和缺失分析继续各自表达。这次只交付现有阅读流程的显示修复及真实捕获回归,不执行模型分析或 solver,不重算原始成绩,不把原生比较当作 LoopX 因果贡献。

改动思路 / Approach

复用既有测量投影和引用阅读器,只在四个已显示测量中去重并找出足够区分的有效位数;精度为即时派生值,不写回分数或新增持久化配置。当前 PR 仅触及已有 NativeResults 的格式化和只读样本/验证;数据、采用、分析及权限 owner 均未改变。不改成绩、不推断保留决策,也不为了显示而增加模型分析步骤。固定四位在真实 Ad 样本上不足;把所有数值永久展开到十七位又会增加阅读负担。当前实现在既有结果区局部派生所需精度,普通数值、签名增量和完整原文保持既有 owner。

实际捕获 旧主值 当前主值/来源
Ad best / final 4.965E10 / 4.965E10 4.9653E10 / 4.965E10,完整原值可点开
Portfolio initial 实测0 仍是0,缺失retained仍未知
Graph retained / final agent-16 / agent-18 原始投影与分析不变,原保留依据可读

这是一项可独立验收的既有阅读流程修复;真实 capture 的重复响应保留原 HTTP 字节身份,避免用自动更新的 fixture 或新合成解释掩盖源数据变更。

具体改动 / Changes

关键代码讲解

  1. formatNativeMeasurement 复用 Intl 格式器,只给科学计数法接收有效位参数,默认仍为4。普通小数和既有 signed improvement 调用没有被另写一套规则,原始分数也没有被回写。
  2. NativeResults 对四个当前 point 的科学数值按原值去重,再从4增加到最多17位,直到显示字符串可区分。相同原值仍相同;null 仍未知,aria-label 与引用保留完整值。这不是对所有成绩、单位或采用关系作新判断。
  3. readNativeResults 把真实 Portfolio/Ad 无分析结果的样本接入已有阅读回归,检查零值、未知、原始引用及窄屏;Graph 仍保留36条原解释,另外覆盖未捕获引用后的恢复。
  4. test_captured_analysis_fixture_serves_real_saved_results_without_source_mutation 固定各语言原 HTTP 摘要,逐一读取新增的原始引用,确认 job/result 缺失、保留结果未知及原来源不变。Graph 的全部原语言 readback 与 base 相同,新增的是可移植引用范围,没有改写旧判断。

规格依据为 docs/development/design.md,固定修订 a0130633723210a5406dca61ee27aa33fe48b3f3 的 结果层级、完整来源及 populated page review,并核对 task-first frontend delivery。对无编号段落使用描述性检查键:primary-result、truthful-missing、complete-source、populated-responsive 均 implemented,分别由实际前后主值、实测零/未知、完整原解释/引用恢复和长内容桌面/390px 人工检查支持。这里的产物是阅读器;实时模型派发、科学校准或 installed 应用不在本次验收声明内。

对主干的风险 / Risks

只验证 API 里的分数正确会漏掉本次问题,因为旧 renderer 的确把两个不同原值显示成同一个短数字。我用同一份冻结 Ad GET 数据,在受影响 UI 源码与 base 相同的旧打包页面复现两个4.965E10,再在最终 head 的真实打包页面确认中文和英文均为4.9653E10/4.965E10,且原始best引用仍为49,653,326,363。Portfolio 的首次提交引用仍是0,best/final同为10.86仍相等,缺失分析用量显示Not captured;没有借用 Graph 的 job、判断或 retained 值。

PASS:pytest -q tests/presentation/test_research_history_fixture.py 7 passed、0 skipped;npm run build:chat 的类型检查、打包与 bundle 验证通过;对确切 base 的 quick premerge 通过,direct/catalog/boundary 无失败。已有 maintainability advisory 和 bundle-size warning 保留,不据此声称全仓静态清洁或取消其他合并门槛。远端 CI 未查询。

独立浏览器使用 Ego 实际操作原页面,未用 Playwright 替代。检查了 Ad 前后桌面、中文390px,Graph1440px/390px长内容与完整展开项,以及英文窄屏和 Portfolio 的真实稀疏结果。两种界面语言下全部36条原理由完整呈现,原文未翻译的提示保留。未捕获 native-report-1 时,页面明确报错且没有pre原文,关闭后能正常读出captured terminal的agent-16保留依据。最后只读 owner 回读为 0 次 mutation attempts、source_unchanged=true。这验证实际打包阅读/HTTP组合;capture回放不认证实时分析执行或原科学解释正确。

我的整体评价 / Verdict

APPROVE。long_horizon=preserved:原始任务结果、负例、分析解释、执行与调度均未改变;user_experience=improved:真实近似大数可直接区分,零值、未知、原文与失败恢复仍在同一阅读路径。复用既有测量投影和引用阅读器,只在四个已显示测量中去重并找出足够区分的有效位数;精度为即时派生值,不写回分数或新增持久化配置。当前 PR 仅触及已有 NativeResults 的格式化和只读样本/验证;数据、采用、分析及权限 owner 均未改变。同域整理已复用原格式器和引用组件,没有必要为这个缺口新增精度配置、数据 owner 或后端能力。

visual_review=inspected:默认视图以结果为主,方法/hash藏在可达详情;Graph完整长解释在桌面分列、窄屏单列,正文和引用未靠删减换取整洁。Portfolio英文窄屏需继续向下读次要值,是完整内容的正常滚动,零值已通过原文点击核对。未验证范围:installed应用、实时模型/分析派发和独立科学因果结论;这些不能从图片、字段或已保存分数推断。现有 vocabulary/typed projection 复用,无新增通用语义,只有已披露的默认数字表示变化。

结论仅对应 f3ec9eb99c0884f0e849ee0fed3729405554978a。GitHub author-owned 情况以 COMMENTED 承载完整批准结论,不执行自批或合并。

English verdict: APPROVE this exact-head native-result display fix; original evidence, unknowns and read-only recovery remain intact.

@huangruiteng
huangruiteng merged commit 60d755d into main Nov 4, 2026
31 of 42 checks passed
@huangruiteng
huangruiteng deleted the codex/research-native-reference-20261104 branch November 4, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants