Skip to content

fix(grpo): preserve remote adapter snapshots during async rollout - #10608

Open
werwrewe wants to merge 1 commit into
modelscope:dev-swift-v5from
werwrewe:fix/cs-grpo-async-snapshots
Open

werwrewe wants to merge 1 commit into
modelscope:dev-swift-v5from
werwrewe:fix/cs-grpo-async-snapshots

Conversation

@werwrewe

Copy link
Copy Markdown

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

远程 GRPO 原来每次都用同一个名称保存 LoRA 权重。单步异步和全异步允许采样与训练重叠。新版发布时,旧请求可能仍在读取旧权重,同名保存会提前删除旧文件,导致采样找不到 adapter_config.json。

改动

  • 远程 one_step_off 和 fully_async 下,每次发布使用独立快照。保存成功后才把新请求切换到新版本。
  • 记录快照的在途请求。等待旧请求结束后再回收,回收期间拒绝显式引用旧 URI。删除超时时也维持这一限制,直到后续回收完成。
  • 发布或退出时查询状态不明的任务。已结束的采样释放保护,已保存但未发布的快照进入回收流程。同步模式和 Ray 模式沿用原有路径。

Checklist

  • I have linked the related issue above, if there is one.
  • I have reviewed and understand every changed line, and I can explain how the change works and why it is correct.
  • I have run the relevant tests or reproductions locally, and included the commands and results above (if applicable).
  • I am responsible for the content of this PR, regardless of how it was produced.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant