Skip to content

build(zip): make the canonical archive reproducible and atomic - #119

Merged
tt-a1i merged 6 commits into
tt-a1i:mainfrom
sunsunsun-java:codex/fix-zip-reproducibility
Aug 27, 2026
Merged

build(zip): make the canonical archive reproducible and atomic#119
tt-a1i merged 6 commits into
tt-a1i:mainfrom
sunsunsun-java:codex/fix-zip-reproducibility

Conversation

@sunsunsun-java

@sunsunsun-java sunsunsun-java commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

问题与价值

修复规范 archify.zip 缺少确定性构建、输入完整性检查和原子发布保障的问题。

Fixes #116

范围

  • 新增不依赖系统 zip 实现的确定性 ZIP 写入器。
  • 将 Node 22 定义为规范 ZIP 构建工具链;其他 Node 版本在写入前失败,运行时仍保持 node >=18
  • ZIP32 的条目数、大小、偏移量和中央目录值严格排除 ZIP64 哨兵。
  • 大 Buffer 字节比较使用 .equals(),失败时不再展开巨大 diff。
  • 路径和模式来自 Git 索引,文件字节来自已跟踪工作树,以保留提交前本地构建能力。
  • 拒绝包含 stage 1/2/3 的未合并索引。
  • 写入同目录临时文件,完成同步后再原子替换目标文件。
  • 在写入前检查 ZIP32 条目数、名称长度、文件大小、偏移量和中央目录大小。
  • Release 先构建并冒烟验证临时包,再与已提交规范包比较。
  • DeepSeek 验收契约明确:规范 Linux 工具链逐字节一致,跨平台只承诺解压内容一致。
  • 未改变 Skill 的运行时渲染或用户 JSON 契约。
  • 无无关改动。

稳定性影响

  • 兼容性与迁移风险:低;发布包的解压内容保持兼容。
  • 包与生成物风险:构建现在对索引冲突和 ZIP32 越界失败关闭。
  • 失败行为与回滚:失败不会覆盖现有可信 ZIP;可通过回退本 PR 的两个提交恢复旧流程。

已运行测试

cd archify
node --test test/release-package-gates.test.mjs test/route-share-card.test.mjs
# 16 passed, 0 failed

cd integrations/deepseek-harness
node --test test/*.test.mjs
# 19 passed, 0 failed

另外完成:

cd archify && npm test
# 731 tests, 714 passed, 17 environment skips, 0 failed

env PATH=/opt/homebrew/Cellar/node/25.6.0/bin:/usr/bin:/bin \\
  /opt/homebrew/Cellar/node/25.6.0/bin/node --test \\
  --test-name-pattern='archive build rejects non-canonical Node versions' \\
  test/release-package-gates.test.mjs
# 1 passed, 0 failed

npm_config_cache=/private/tmp/archify-dsh-npm-cache-5af9 \\
  node --test integrations/deepseek-harness/test/*.test.mjs
# 20 passed, 0 failed

规范 ZIP 已在 Node 22 下重新构建,并使用 cmp 验证字节一致。

可视化证据

不适用;本 PR 不改变页面视觉效果。

生成物

  • 重新生成 archify.zip
  • Release 和 CI 使用同一规范构建入口检查其新鲜度。
  • DeepSeek distribution receipt 契约同步更新。

Checklist

  • 使用聚焦改动,未修改已有 typed JSON 行为
  • archify/ 运行完整 npm test
  • 为行为变化添加或更新回归测试
  • 检查生成物和包新鲜度
  • 测试夹具不包含密钥、私有仓库或用户数据

@sunsunsun-java
sunsunsun-java requested a review from tt-a1i as a code owner August 26, 2026 13:09
@YunyueLi

Copy link
Copy Markdown
Collaborator

方向是对的,但我本地复现到一个需要在合并前处理的兼容性问题:

  1. 规范 ZIP 仍依赖 Node/zlib 版本。 项目当前声明 node >=18。同一份输入在 Node 24 下可复现已提交的 archify.zip,但 Node 26 会稳定生成另一组 ZIP 字节;因此新版 Node 用户运行构建后,CI 会把结果判为 stale。对应测试使用 assert.deepEqual(Buffer, Buffer),字节不同时还会展开很大的 diff,我这里最终被 SIGKILL(exit 137)。建议二选一:

    • 明确并在构建脚本中校验唯一的 canonical Node 版本,只在该工具链断言容器字节一致;或
    • 使用不随 zlib 版本变化的 ZIP 编码方式。

    测试里的大 Buffer 比较建议改为 .equals() / assert.ok(...),让失败保持轻量、可读。

  2. ZIP32 极限值要严格排除。 0xffff0xffffffff 本身是 ZIP64 哨兵值;当前使用 <= 会允许写入这些值,但又没有 ZIP64 record。条目数、大小、offset 和 central-directory size 的相应判断应使用严格 <

另外,当前 DSH distribution acceptance 连续两次失败都是安装 @deepseek-ai/dshETIMEDOUT,不是 ZIP 比较失败,代码修正后重跑即可。

@tt-a1i

tt-a1i commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Thanks for continuing to improve this PR. We rechecked the current revision and found no blocking issue in the reproducible ZIP implementation, cross-platform package smoke tests, ZIP freshness check, or focused local tests.

One CI reliability issue still needs to be addressed before merge. DSH distribution acceptance has now failed twice at the same stage, both after the pinned DSH runtime installation ran for about 300 seconds:

spawnSync npm ETIMEDOUT
npm install @deepseek-ai/dsh@0.1.0-rc.6 exited null

The receipt shows that pack and tarball-inspect completed successfully, so this currently looks like a transient external npm download failure rather than an error in the ZIP contents.

Please add one bounded retry around the real DSH runtime installation, limited to clearly transient network failures such as ETIMEDOUT, ECONNRESET, and EAI_AGAIN. Each attempt should use a clean installation directory, and the final receipt should preserve the attempt count and the last error. Other installation or acceptance failures should still fail immediately; please do not skip the real DSH installation or the subsequent skill-discovery checks.

Once that is updated, please rerun the full checks and we can do the final merge verification.

@tt-a1i

tt-a1i commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Thanks again for the original implementation and for keeping this branch updated.

Because this PR comes from a fork with maintainer edits disabled, we could not push the small CI reliability fix directly to your branch. We therefore preserved all commits and authorship from this PR in repository-local PR #129 and added the bounded transient-network retry there.

We will use #129 for the final CI verification and merge. Your original contribution remains fully credited in the preserved commit history.

@tt-a1i
tt-a1i merged commit dcf8148 into tt-a1i:main Aug 27, 2026
13 of 15 checks passed
@tt-a1i

tt-a1i commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Thank you again for the original reproducible and atomic ZIP implementation.

This work has now been merged through #129 as merge commit 585be4c3c9755b7e5ec0ac38f481a6c1a46d4d29. The repository-local continuation preserved all commits and authorship from this PR, and added the DSH acceptance reliability fix discussed above.

The final GitHub checks were all green, including the complete DSH distribution acceptance, which now finishes in 28 seconds instead of timing out during runtime installation.

Closing this PR as superseded by the merged #129. Thanks for the contribution.

@sunsunsun-java
sunsunsun-java deleted the codex/fix-zip-reproducibility branch August 27, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 完善规范 ZIP 的确定性构建与原子发布保障

3 participants