-
-
Notifications
You must be signed in to change notification settings - Fork 11.2k
[V0 deprecation] Deprecate use_v1 parameter #28112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request deprecates the use_v1 parameter in get_attn_backend_cls across various platform implementations. The changes are consistent and well-implemented. To provide backward compatibility for plugins, a try-except block is introduced in vllm/attention/selector.py. My review includes a suggestion to make this backward compatibility logic more robust by using inspect.signature instead of catching a broad TypeError, which will prevent masking potential bugs in plugins.
b009b12 to
9dd3b38
Compare
|
@hmellor what's the suggested way of marking an argument as will be deprecated? |
|
Ordinarily it's more user facing APIs that we are deprecating (config fields, CLI arguments, API fields). We don't really have a lot of prescedent for the plugin interface. I think the approach that's been taken is reasonable as it ensures backwards compatibility. The only notes I would make are:
|
9dd3b38 to
a84cbc1
Compare
|
@hmellor added the removal version to For plugin doc, I just notice that there no clear doc for hardware plugin. Only one I found is this: https://docs.vllm.ai/en/latest/design/plugin_system/#compatibility-guarantee I'd like to write a detail one for hardware plugin, include interface design, how it works, how to implement one and so on in a following PR. |
|
This pull request has merge conflicts that must be resolved before it can be |
vllm/attention/selector.py
Outdated
| if "use_v1" in sig.parameters: | ||
| logger.warning_once( | ||
| "use_v1 parameter for get_attn_backend_cls is deprecated and will " | ||
| "be removed in v0.12.0. Please remove it from your plugin code." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deprecation message must be present in at least one whole minor revision. (i.e. it will be present for the entirety of v0.12.0)
| "be removed in v0.12.0. Please remove it from your plugin code." | |
| "be removed in v0.13.0 or v1.0.0, whichever is soonest. " | |
| "Please remove it from your plugin code." |
For now, could you please add the warning to the section you linked. And then it can be superseded with the new document you write. |
a84cbc1 to
5b6f280
Compare
Signed-off-by: wangxiyuan <[email protected]>
5b6f280 to
c88daf6
Compare
hmellor
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I'll wait to get some more eyes on this because it's a significant change
Yikun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I search all get_attn_backend_cls in here all is changed.
BTW,
@yaochengji tpu need to remove V0 support:
https://github.com/vllm-project/tpu-inference/blob/015cb2b2dc3b8fb2fe8d2e2037f2d63034daf64b/tpu_inference/platforms/tpu_platform.py#L65
@xuechendi Feel free to remove assert:
https://github.com/vllm-project/vllm-gaudi/blob/e3c20ec22d556af414fabbff12c20dc8070ec14f/vllm_gaudi/platform.py#L46
And other plugin just need remove use_v1 args
| use_sparse, | ||
| ) | ||
| sig = inspect.signature(current_platform.get_attn_backend_cls) | ||
| if "use_v1" in sig.parameters: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
++! This really a nice practice to help plugin smoothly complete interface changes. Let's remove if "use_v1" in sig.parameters in next release
- vllm-project#28112 didn't cause conflicts but main still contained a `use_v1` - Also fixes a pre-commit warning Signed-off-by: Harry Mellor <[email protected]>
Purpose
use_v1forget_attn_backend_clsis useless now. Since this is an interface impact change, let's deprecate it and remove it in the next release.Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.