Skip to content

Update rope_scaling to rope_parameters in preparation for Transformers v5 #28542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Update rope_scaling to rope_parameters in preparation for Transformers v5 #28542

wants to merge 31 commits into from
+546 −503

Conversation

@hmellor
Copy link
Member

@hmellor hmellor commented Nov 12, 2025 •
edited by github-actions bot
Loading

In Transformers v5:

  • rope_scaling is now called rope_parameters
  • rope_theta now lives inside rope_parameters
  • rope_parameters may be nested for models which have different RoPE parameters for each layer type (i.e. Gemma & ModernBERT)

This PR adds forward compatibility for Transformesr v5 RoPE config by:

  • Moving any found config.rope_scaling to config.rope_parameters
  • Moving any found config.rope_theta to config.rope_parameters.rope_theta
  • Performs patch_rope_parameters_dict on all nested RoPE parameters if present
  • Globally renaming rope_scaling to rope_parameters
  • Updating the logic for retrieving base and rope_parameters for the get_rope method

@mergify
Copy link

mergify bot commented Nov 12, 2025

Documentation preview: https://vllm--28542.org.readthedocs.build/en/28542/

@mergify mergify bot added documentation Improvements or additions to documentation llama Related to Llama models performance Performance-related issues qwen Related to Qwen models gpt-oss Related to GPT-OSS models speculative-decoding labels Nov 12, 2025
hmellor added 11 commits November 12, 2025 11:12
@hmellor
More renaming in transformers utils
889b900 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix patch_rope_parameters for when rope_scaling was explicitly `N…
50b1a87 
…one`

Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Gemma3 and Gemma3n
bd182e0 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Merge branch 'main' into update-rope-config
4c61e2e
@hmellor
Get rope_theta from the new location too
65c8658 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix condition for non gemma3 models
5d65739 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Make Transformers backend torch compile check work with new rope params
b4e1967 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Re-enable a load of Transformers nightly tests which are now fixed
ee77bd7 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update the custom configs
df4c007 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Make sure scaling factor always exists
325ff8d 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
A couple more models that now init on v5
11c23a7 
Signed-off-by: Harry Mellor <[email protected]>
@mergify mergify bot added the ci/build label Nov 13, 2025
@hmellor
Update Commandr
4ea113c 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Qwen3Next
59b0f27 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update Olmo2
064441b 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
rope_parameters always present because of rope_theta
bdd0e6c 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update LFM2MoE
f224ef4 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update LFM2
19dcc18 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update the rest
2eecd31 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 13, 2025
@mergify mergify bot added the deepseek Related to DeepSeek models label Nov 13, 2025
@hmellor
update tests
e95ccd4 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Update configs
f2bac15 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Missed 2
36e8a1f 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor marked this pull request as ready for review November 13, 2025 12:38
@hmellor
Copy link
Member Author

hmellor commented Nov 13, 2025

(marking as ready so that I can run CI)

chatgpt-codex-connector[bot]
chatgpt-codex-connector bot reviewed Nov 13, 2025
View reviewed changes
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

@noooop noooop Awaiting requested review from noooop noooop is a code owner

@patrickvonplaten patrickvonplaten Awaiting requested review from patrickvonplaten patrickvonplaten is a code owner

@sighingnow sighingnow Awaiting requested review from sighingnow sighingnow is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@houseroad houseroad Awaiting requested review from houseroad houseroad is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

ci/build deepseek Related to DeepSeek models documentation Improvements or additions to documentation gpt-oss Related to GPT-OSS models llama Related to Llama models performance Performance-related issues qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed speculative-decoding

Projects

gpt-oss Issues & Enhancements
Status: To Triage

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hmellor