Text to Video Reference Implementation #2413
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
hvagadia
force-pushed
the
text_to_video
branch
from
0718a2f to
4401a5a
Compare
|
|
||
| - Model: [Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers) | ||
| - VBench: [GitHub](https://github.com/Vchitect/VBench) | ||
| - MLPerf: [Inference](https://github.com/mlcommons/inference) |
There was a problem hiding this comment.
Missing:
- Reference accuracy score section for VBench at original accuracy (BF16)
- Supported hardware (I suppose this is Hopper and pre-hopper based on the torch/cuda version)?
- mlperf.conf change for performance sample count and related settings
- submission checker changes
There was a problem hiding this comment.
Submission checker can go into a separate PR.
There was a problem hiding this comment.
1, 2 done. 3 and 4 can likely be clubbed with Loadgen changes Pablo is helping us with.
| @@ -0,0 +1,46 @@ | |||
| FROM pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel | |||
There was a problem hiding this comment.
Seems like this is pre-Blackwell, recommend adding the supported hardware in the README
|
|
||
| try: | ||
| print("Starting download...") | ||
| snapshot_download( |
There was a problem hiding this comment.
Can we ping to a certain commit of the model? (or at least note it in the readme)
There was a problem hiding this comment.
Noted commit id in the Readme. There isn't much activity in huggingface model repo, so we should be fine.
| @@ -0,0 +1,39 @@ | |||
| # Text-to-Video Benchmark | |||
There was a problem hiding this comment.
Regarding the folder name, it should be text_to_video/Wan2.2-T2V-A14B so we can add new models in the future. (We didn't catch it in the text_to_image)
There was a problem hiding this comment.
Moved to model specific folder.
3 participants
Checkout README.md for instructions.