Question: Support for Cosmos 3 Reasoner Post-training

[Ethan-Lee-Sunghoon]

Hi, thank you for open-sourcing this great project!

I have a question regarding the post-training/SFT support for the Cosmos 3 Reasoner.

In the previous Cosmos Reason2, there were guidelines on performing LoRA SFT using trl and cosmos-rl. For Cosmos 3 Reasoner, I noticed that SFT is now supported through the cosmos-framework.

While reviewing the training documentation (https://github.com/NVIDIA/cosmos-framework/blob/main/docs/training.md), I had a question about the starting weights used in the examples:

1. In the "Reasoner Alignment SFT with LLaVA-OneVision (vfm-vlm)" example, the backbone used is Qwen/Qwen3-VL-8B-Instruct. Why does this process not start directly from the Cosmos 3 Reasoner weights? Additionally, what does "vfm-vlm" at the end of this example's title stand for/mean?
2. In contrast, the "Reasoner Alignment SFT with VideoPhy-2 (Cosmos3-Nano)" example seems to start with the Cosmos 3 Nano weights. Could you please explain the key differences between these two examples and the reasoning behind using different starting weights for them?
   Additionally, in this setup, Qwen's vision encoder is frozen and only the LM of Cosmos3 is used—could you share the reasoning behind this design choice?
3. Lastly, similar to previous Cosmos models, are there plans to release recipes utilizing the cosmos-framework in the cosmos-cookbook?

Thank you so much for your time and support!

[foreverlms]

Thanks for this issue.

For 1, this is just an example for how to use cosmos-framework to do SFT on open-source or user-side dataset. It's just a toy example so we used Qwen/Qwen3-VL-8B-Instruct. This toy-example used a dataloader that we draft for open-source community. vfm-vlm is our inner convention, means that VLM training is under VFM folder, we carried out an unification of training infra of vfm and vlm, all the vlm training infra is moved into vfm folder. I think this is a misunderstanding title? @lfengad or we could make it the same as VideoPhy-2?

For 2, it is the same as LLaVA-OneVision experiment but with different datasets. But it specially add an env called VLM_SAFETENSORS_PATH, you could check how is this env set. If it is set, the standard Qwen3-VL weight will be populated with Cosmso3-Nano reasoner weight. For the vision encode weight if frozen, I think @tylin could share the details about this.

For 3, the recipes you mean are training recipes? I thinks the recipes in cosmos-cookbook are mainly for inference, for finetuning, it will be redirected to cosmos-framework. https://github.com/NVIDIA/cosmos#finetune

[lfengad]

@foreverlms Yeah, we could refine the naming and make it the same as VideoPhy-2

[foreverlms]

@foreverlms Yeah, we could refine the naming and make it the same as VideoPhy-2

will do it