Skip to content

fix(dev): align remote multimodal prompt inputs - #10602

Open
werwrewe wants to merge 2 commits into
modelscope:dev-swift-v5from
werwrewe:fix/cs-grpo-multimodal
Open

werwrewe wants to merge 2 commits into
modelscope:dev-swift-v5from
werwrewe:fix/cs-grpo-multimodal

Conversation

@werwrewe

Copy link
Copy Markdown

Use the preprocessed image for rollout and restore visual tensors received as JSON lists before collation.

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

背景

在 client-server 模式下运行图文 GRPO 时,Swift 训练模板会按 max_pixels 缩放图片,采样请求却可能仍携带原图,导致 vLLM 生成回答时使用的视觉输入与训练侧不一致。另外,视觉张量经 HTTP JSON 传输后会成为嵌套列表,服务端直接拼批会报错。

改动

  • encode_prompt() 优先使用引擎编码阶段保留的预处理图片,让采样和训练基于同一张缩放后的图。
  • 视觉字段在拼批前统一通过 torch.as_tensor() 转为张量,兼容 HTTP 列表和本地 Tensor。
  • _data_collator_mm_data() 对 pixel_values、image_sizes 和 pixel_values_videos 逐项转为张量后再拼批。

Checklist

  • I have linked the related issue above, if there is one.
  • I have reviewed and understand every changed line, and I can explain how the change works and why it is correct.
  • I have run the relevant tests or reproductions locally, and included the commands and results above (if applicable).
  • I am responsible for the content of this PR, regardless of how it was produced.

Use the preprocessed image for rollout and restore visual tensors received as JSON lists before collation.

@kevssim kevssim left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 本 review 由 AI 生成,仅供参考,请结合实际情况判断。

collator 用 torch.as_tensor 兼容 HTTP 传来的列表,这部分没有问题,行为与原来一致。encode_prompt 优先取引擎编码后的图片,在 vllm/sglang/lmdeploy 下是正确的,但在 transformers 后端下会取到处理后的张量,建议按后端区分(见行内评论)。

Comment thread swift/dev/template/base.py Outdated
media = MediaBundle.from_sources(sample, feature)
# Engine encoding retains the images after the same resize used for
# the expanded training feature.
media = MediaBundle.from_sources({"images": engine_feature.get("images")}, sample, feature)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 以下为 AI review 意见,供参考。

[P2] backend="transformers" 时,engine_feature["images"] 可能是处理过的张量,不是图片

只有在 vllm、sglang、lmdeploy 模式下,_encode_truncated 才会把预处理后的 inputs.images 写入 encoded["images"](base.py:2347-2357)。backend="transformers" 时,render_mode 返回 "transformers",走的是模型自己的 _encode。这时 glm(glm.py:615)、internlm(internlm.py:112)、llava(llava.py:386)、yi(yi.py:47)、deepseek(deepseek.py:383,单个 [(crop, ori)])等模板写入的都是处理后的张量。twinkle 的 generation.py:102 和 transformers_sampler.py:298 都会以 backend="transformers" 调用 encode_prompt。结果是:RolloutState.media.images 变成张量;twinkle 的 _media_ref 遇到张量会抛 TypeError;deepseek 多图时 _check_engine_placeholders 只看到 1 张图而报错。

建议只在引擎模式下使用 engine_feature 里的图片:

Suggested change
media = MediaBundle.from_sources({"images": engine_feature.get("images")}, sample, feature)
engine_images = engine_feature.get("images") if backend in ("vllm", "sglang", "lmdeploy") else None
media = MediaBundle.from_sources({"images": engine_images}, sample, feature)

@werwrewe
werwrewe force-pushed the fix/cs-grpo-multimodal branch from a0bdb25 to a1df7ca Compare November 20, 2026 08:03
@werwrewe
werwrewe force-pushed the fix/cs-grpo-multimodal branch from a1df7ca to 8ad3623 Compare November 20, 2026 08:16

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants