[SW-228042] Add support for dynamic vLLM kv-cache quantization

[dudilester]

No description provided.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[Copilot]

Pull Request Overview

This PR adds support for dynamic vLLM kv-cache quantization by introducing scaling tensors alongside the existing key and value cache tensors throughout the codebase.

Key changes:

• Extended KV cache tuple structure from 2 elements (key, value) to 4 elements (key, value, key_scale, value_scale)
• Added conditional logic to create scale tensors when FP8 quantization with dynamic scaling is enabled
• Updated all cache operations (copy, swap, fetch) to handle the new scale tensors

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vllm_gaudi/v1/worker/hpu_worker.py	Added scale tensor initialization to empty cache placeholders
vllm_gaudi/v1/worker/hpu_model_runner.py	Added conditional creation of scale tensors based on dtype and QUANT_CONFIG environment variable
vllm_gaudi/extension/utils.py	Updated cache operations to accept optional scale parameters and modernized super() calls
vllm_gaudi/extension/unified.py	Extended CacheUtils to store and manage scale tensors
vllm_gaudi/extension/ops.py	Updated flat_pa to unflatten and pass scale tensors to fetch functions
vllm_gaudi/extension/cache_ops.py	Modified copy_blocks to handle scale tensors with null checks
vllm_gaudi/attention/ops/hpu_paged_attn.py	Updated method signatures and implementations to handle 4-tuple cache structure
vllm_gaudi/attention/backends/hpu_attn.py	Threaded scale tensors through attention forward passes and helper methods

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[github-actions]

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

[michalkuligowski]

e2e tests fail with:

PatchedVLLMKVCache.fetch_from_cache() takes 3 positional arguments but 4 were given

[dudilester]

Thanks @michalkuligowski , there is a related PR in INC to support this feature.
https://github.com/habana-internal/neural-compressor-fork/pull/322
I guess I need the INC PR to be merged first to make this PR pass the tests.