[None][perf] Optimize Video Hashing Speed

[aswinvisva]

Summary by CodeRabbit

• New Features
  • Added source hashing for video inputs across local files, downloaded content, in-memory bytes, and data URIs.
  • Video metadata now preserves the source hash for consistent content identity.
  • Video hashing can use the original source data instead of processing decoded frames, improving efficiency while maintaining stable identification.

Description

Current Video hashing iterates over every decoded frame's pixel data (~100 MB for a 30s @ fps=8 374x374 request) inside apply_mm_hashes, adding ~145 ms per request to the OpenAI serve entry path.

The decoded frame content is a deterministic function of (source mp4 bytes) + (media IO num_frames/fps kwargs),
and the latter is already captured in metadata["frames_indices"] (a lossless projection of the kwargs into the effective sampling result). So the same content identity can be obtained by hashing the source bytes once at media-load time and using them as the anchor in update_hash.

Measured on a real 30s @ fps=8 374x374 mp4 (147 KB, 240 decoded frames) inside the tensorrt-llm/release:1.3.0rc20 container with the patch overlaid, 50 iterations per path:

Coverage preserved:

• metadata["frames_indices"] is still hashed on both paths, so media IO num_frames/fps kwargs land in the hash implicitly.
• Slow-path frame-walk fallback stays for direct-frame-list callers.
• Audio hashing (AudioData.update_hash) is unchanged.

Known pre-existing gap NOT addressed here: HF processor kwargs (mm_processor_kwargs) are still not in the hash. Two requests with identical decoded frames but different per-request HF kwargs share a cache entry, which can produce wrong embeddings. Tracked separately.

Perf improvement

e2e benchmarks

Benchmark is with the cosmos3-reasoner Nano FP8 model ran on a single H200 node. 256 concurrent requests each with 128x128 30s @ 8 fps mp4 videos + 50 token text prompt. KV reuse enabled: enable_block_reuse: true

Metric	Baseline	PR	Δ
Wall-clock	43.6 s	36.2 s	−16.9%
TTFT avg	18,557 ms	15,306 ms	−17.5%
TTFT p50	16,045 ms	13,542 ms	−15.6%
TTFT p90	26,203 ms	22,399 ms	−14.5%
TTFT p95	30,297 ms	25,348 ms	−16.3%
TTFT p99	30,416 ms	25,582 ms	−15.9%
Request latency avg	19,021 ms	15,872 ms	−16.6%

Isolated test for just the hashing function

Before (apply_mm_hashes frame walk):        144.99 ms / request
After  (load-time hash + apply_mm_hashes):    0.12 ms / request

Total hashing cost drops ~1250x. Load-time hashing runs inside the existing async media-I/O threadpool, so almost all of the 145 ms comes off the synchronous request-handling path when KV-reuse is enabled.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.