Automatically use zero tolerance for bitwise comparison for fp8 dtypes during autotuning #1158
+146
−9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meta-cla
bot
added
the
CLA Signed
gmagogsfm
force-pushed
the
auto-fp8-tolerance
branch
2 times, most recently
from
b5bbd30 to
fa64d26
Compare
Contributor
Author
This change automatically sets atol=0.0 and rtol=0.0 for autotuning accuracy checks when fp8 dtypes are detected in kernel outputs, while respecting user-specified tolerance values. Changes: - Added _user_set_atol and _user_set_rtol flags to Settings to track whether users explicitly set tolerance values - Added _compute_effective_tolerances() method to BaseSearch that detects fp8 dtypes and automatically sets tolerances to 0.0 when user hasn't explicitly specified them - Updated _validate_against_baseline() to use effective tolerances instead of settings values directly Behavior: - If user sets neither tolerance: automatic 0.0 for fp8 dtypes - If user sets either tolerance: both values are respected (no override) - Supports all fp8 variants: float8_e4m3fn, float8_e5m2, float8_e4m3fnuz, float8_e5m2fnuz, float8_e8m0fnu Tests: - test_autotune_fp8_automatic_tolerance: verifies automatic detection - test_autotune_fp8_explicit_tolerance_override: verifies user values respected - test_autotune_fp8_explicit_default_tolerance: verifies explicit 1e-2 respected - test_autotune_fp8_partial_tolerance_override: verifies partial specification respected Fixes issue where fp8 kernels would fail autotuning with: "Rtol=0.01 and atol=0.01 are not supported for bitwise comparison of low dimensional floats"
gmagogsfm
force-pushed
the
auto-fp8-tolerance
branch
from
fa64d26 to
de76da9
Compare
yf225
reviewed
Nov 21, 2025
yf225
reviewed
Nov 21, 2025
helion/runtime/settings.py
Outdated
| autotune_baseline_rtol: float = dataclasses.field(default=1e-2) | ||
| # Internal fields to track if user explicitly set tolerance values | ||
| _user_set_atol: bool = dataclasses.field(default=False, init=False, repr=False) | ||
| _user_set_rtol: bool = dataclasses.field(default=False, init=False, repr=False) |
Contributor
There was a problem hiding this comment.
Not sure if it's possible, but would really love to avoid needing to have these private fields on the settings class, to keep things more simple
Contributor
Author
There was a problem hiding this comment.
Good point, I changed the implementation a bit. Now the autotune_baseline_atol field is an optional[float]. None value represents unset tolerance level.
gmagogsfm
commented
Nov 21, 2025
helion/runtime/settings.py
Outdated
| autotune_baseline_rtol: float = dataclasses.field(default=1e-2) | ||
| # Internal fields to track if user explicitly set tolerance values | ||
| _user_set_atol: bool = dataclasses.field(default=False, init=False, repr=False) | ||
| _user_set_rtol: bool = dataclasses.field(default=False, init=False, repr=False) |
Contributor
Author
There was a problem hiding this comment.
Good point, I changed the implementation a bit. Now the autotune_baseline_atol field is an optional[float]. None value represents unset tolerance level.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change automatically sets atol=0.0 and rtol=0.0 for autotuning accuracy checks when fp8 dtypes are detected in kernel outputs, while respecting user-specified tolerance values.
Behavior:
Fixes issue where kernels returning fp8 types would fail autotuning with: "Rtol=0.01 and atol=0.01 are not supported for bitwise comparison of low dimensional floats", due to pytorch's bitwise comparison