📚 docs(granite-speech): add comprehensive usage examples #42125
+101
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resolves the TODO (@alex-jw-brooks) by adding complete usage documentation for Granite Speech model now that it's released and compatible with transformers. Added examples for: - Basic speech transcription - Speech-to-text with additional context - Batch processing multiple audio files - Tips for best results (audio format, LoRA adapter, memory optimization) This helps users get started with the Granite Speech multimodal model by providing practical, copy-paste-ready code examples for common use cases. Replaces TODO comment on line 44 with ~100 lines of comprehensive documentation following the patterns used in other multimodal model docs.
Comment on lines
90
to
92
| # Prepare inputs with text prompt | ||
| text_prompt = "Transcribe the following audio:" | ||
| audio_input = "path/to/audio.wav" |
Member
There was a problem hiding this comment.
prob we need to apply a chat template and format the prompt
Comment on lines
142
to
148
| ### Tips for Best Results | ||
|
|
||
| - **Audio Format**: The model expects 16kHz sampling rate audio. The processor will automatically resample if needed. | ||
| - **LoRA Adapter**: The LoRA adapter is automatically enabled when audio features are present, so you don't need to manage it manually. | ||
| - **Memory Usage**: For large models, use `torch.bfloat16` or quantization to reduce memory footprint. | ||
| - **Temperature**: Use lower temperatures (0.1-0.5) for accurate transcription, higher (0.7-1.0) for more creative responses. | ||
| - **Batch Size**: Adjust batch size based on available GPU memory. Larger batches improve throughput but require more memory. |
Member
There was a problem hiding this comment.
imo most of the tips are general knowledge about LLMs. If it is really important for GraniteSpeech, we can move it under Usage Tips
…fic tips - Added proper chat template formatting in the second example (per @zucchini-nlp feedback) - Removed generic LLM tips (temperature, batch size, memory) - Moved Granite Speech-specific tips (audio format, LoRA adapter) to Usage tips section This keeps the documentation focused on model-specific features rather than general LLM knowledge.
Contributor
Author
|
Thanks for the review @zucchini-nlp! I've addressed both points:
The documentation now focuses on model-specific features. Let me know if you'd like any other adjustments! cc @eustlb as requested |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
eustlb
reviewed
Nov 13, 2025
|
|
||
| # Prepare audio input (16kHz sampling rate required) | ||
| # audio can be a file path, numpy array, or tensor | ||
| audio_input = "path/to/audio.wav" |
Contributor
There was a problem hiding this comment.
let's comment this and use datasets for have something that works out of the box copy pasted
from datasets import load_dataset, Audio
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
audio = ds['audio'][0]['array']
Comment on lines
+98
to
+108
| text = processor.tokenizer.apply_chat_template( | ||
| messages, | ||
| tokenize=False, | ||
| add_generation_prompt=True | ||
| ) | ||
|
|
||
| inputs = processor( | ||
| text=text, | ||
| audio="path/to/audio.wav", | ||
| return_tensors="pt" | ||
| ).to(model.device) |
Contributor
There was a problem hiding this comment.
why can't we do apply_chat_template with tokenize=True directly? That would be the standard way of doing it
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
docs(granite-speech): add comprehensive usage examples
Resolves the TODO (@alex-jw-brooks) by adding complete usage documentation for Granite Speech model now that it's released and compatible with transformers.
Added examples for:
This helps users get started with the Granite Speech multimodal model by providing practical, copy-paste-ready code examples for common use cases.
Replaces TODO comment on line 44 with ~100 lines of comprehensive documentation following the patterns used in other multimodal model docs.
What does this PR do?
This PR resolves an existing TODO in the Granite Speech documentation by adding comprehensive usage examples now that the model is released and compatible with transformers.
Problem: The documentation contained a placeholder TODO comment asking for usage examples once the model was released. Since
ibm-granite/granite-3.2-8b-speechis now available, users need practical examples to get started.Solution: Added a complete "Usage example" section with three practical code examples:
Impact: Users can now quickly get started with Granite Speech without searching external resources. The examples follow the same structure as other multimodal models (SAM2, SmolVLM, LLaVA) for consistency.
Before submitting
Who can review?
@alex-jw-brooks (original TODO author)
@stevhliu (documentation)
@zucchini-nlp (multimodal models)
Additional Context:
This PR adds ~105 lines of documentation to replace a single TODO comment. All code examples:
The Granite Speech model is a multimodal speech-to-text model from IBM that's now fully integrated into transformers, so users need these examples to get started effectively.