text to video ref implementation

Author: hvagadia

.gitmodules

@@ -10,3 +10,6 @@
 [submodule "language/deepseek-r1/submodules/LiveCodeBench"]
 	path = language/deepseek-r1/submodules/LiveCodeBench
 	url = https://github.com/LiveCodeBench/LiveCodeBench
+[submodule "text_to_video/submodules/VBench"]
+	path = text_to_video/submodules/VBench
+	url = https://github.com/Vchitect/VBench

text_to_video/.gitignore

@@ -0,0 +1,38 @@
+# Model weights (large files)
+models/
+
+# Cache directories
+.cache/
+
+# Generated outputs and evaluation results
+data/outputs/
+data/outputs_*/
+data/results/
+data/evaluation_results/
+data/*_eval*/
+data/prompt_mapping.json
+
+# Python cache
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.Python
+
+# Jupyter
+.ipynb_checkpoints/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Logs and results
+*.log
+results/
+outputs/

text_to_video/README.md

@@ -0,0 +1,39 @@
+# Text-to-Video Benchmark
+
+Text-to-video generation using Wan2.2 T2V-A14B-Diffusers model and VBench evaluation.
+
+## Quick Start
+
+```bash
+# Clone with submodules
+git clone --recurse-submodules https://github.com/mlcommons/inference.git
+cd inference/text_to_video
+
+# Build Docker
+./launch.sh --build
+
+# Download model
+./launch.sh python3 download_model.py
+
+# Run inference (supports data parallel)
+./launch.sh python -m torch.distributed.run --nproc_per_node=8 run_inference.py
+
+# Evaluate
+./launch.sh python run_evaluation.py
+```
+
+## Files
+
+- `inference_config.yaml` - Generation parameters (resolution, fps, seed, etc.)
+- `download_model.py` - Model download
+- `run_inference.py` - Video generation  
+- `run_evaluation.py` - VBench evaluation
+- `launch.sh` - Docker launcher
+- `data/vbench_prompts.txt` - Input prompts
+- `data/fixed_latent.pt` - Optional fixed latent tensor for deterministic generation
+
+## References
+
+- Model: [Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers)
+- VBench: [GitHub](https://github.com/Vchitect/VBench)
+- MLPerf: [Inference](https://github.com/mlcommons/inference)

text_to_video/data/calibration_prompts.txt

@@ -0,0 +1,50 @@
+a shark is swimming in the ocean, black and white
+A person is shuffling cards
+palace
+aquarium
+pharmacy
+an umbrella and a handbag
+waterfall
+A person is washing dishes
+A person is playing chess
+desert
+a teddy bear and a frisbee
+A person is grooming dog
+a shark is swimming in the ocean, surrealism style
+a cow and an elephant
+A boat sailing leisurely along the Seine River with the Eiffel Tower in background, in cyberpunk style
+an elephant taking a peaceful walk
+A person is shaking hands
+courtyard
+A person is digging
+an apple and a cell phone
+a shark is swimming in the ocean, in cyberpunk style
+A boat sailing leisurely along the Seine River with the Eiffel Tower in background, watercolor painting
+a truck accelerating to gain speed
+a horse and a sheep
+An astronaut flying in space, in cyberpunk style
+alley
+A person is pushing cart
+a car slowing down to stop
+A person is playing trumpet
+A person is skydiving
+A person is shining shoes
+an airplane soaring through a clear blue sky
+sky
+campus
+An astronaut flying in space, surrealism style
+a knife and a tv
+a toilet and a hair drier
+An astronaut flying in space by Hokusai, in the style of Ukiyo
+cafeteria
+volcano
+a car stuck in traffic during rush hour
+a skateboard and a surfboard
+A person is knitting
+A person is push up
+a bear catching a salmon in its powerful jaws
+A person is counting money
+a cat drinking water
+A beautiful coastal beach in spring, waves lapping on sand by Hokusai, in the style of Ukiyo
+A person is ice skating
+a remote and a keyboard