DirectML EP: add Tier-2 op+activation fusion, fp16 exec flags, and ORT parity fixes - #52
Merged
Conversation
…T parity fixes
Add a generic Tier-2 BaseOp+Activation fusion rule and align several
DirectML plugin behaviors with ONNX Runtime's DirectML execution provider.
Op+Activation fusion (new):
- Add op_activation_ep_fusion.{h,cc} and fusion_activation_helper.h,
fusing an optional downstream activation into Add, Sum, Gemm, Conv,
ConvTranspose, BatchNormalization, InstanceNormalization, and
MeanVarianceNormalization via the DML operator's FusedActivation field,
with allowed activations filtered per op and per device to match ORT's
fusability tables.
- Fix a bug where the eager compile path never recorded compiled_slot_sizes,
causing the fused kernel to be needlessly recompiled on every inference
instead of once (~2x perf regression on fused Conv/ConvTranspose models).
Runtime (non-constant) weight support:
- Bind Conv/ConvTranspose/BatchNorm/InstanceNorm weight, scale, bias,
mean, and variance inputs per slot, either directly from a runtime
OrtValue or via a one-time static upload for true constants, and defer
kernel compilation until all weight shapes are known.
ORT parity fixes:
- BatchNormalization epsilon now defaults to 0.0 (InstanceNormalization
keeps 1e-5), matching ORT's per-op defaults.
- MeanVarianceNormalization explicit axes are normalized for negative
values, matching ORT's GetDmlAdjustedAxis.
fp16 execution flags:
- Set DML_EXECUTION_FLAG_ALLOW_HALF_PRECISION_COMPUTATION when a graph is
entirely fp16, in full-graph/per-partition compilation and in the
QuickGelu, BiasGelu, and FusedMatMul fusion kernels.
tperry-amd
previously approved these changes
Jul 21, 2026
…dels The owned_graph_input_indices construction used raw packed DML slot indices into the ONNX-ordered input_names array, causing initializer weights to be looked up under wrong names for operators with input_name_reorder (e.g. ConvInteger). This left OWNED_BY_DML initializers unbound in the init binding table, triggering a DirectML BindingValidator E_INVALIDARG and device removal during session init. Additional fixes: - Resolve passthrough aliases in consumed_initializer_names tracking so initializers consumed through renamed edges get DML graph input slots - Use min(actualTensorBytes, totalTensorSizeInBytes) for constant node DataSize to avoid reading past ORT tensor allocations - Add dml_input_slot_indices to ConvInteger, QLinearConv, MatMulInteger, QLinearMatMul, and MatMulIntegerToFloat for correct edge wiring when optional inputs are absent - Fix filter zero-point and scale axis placement from 0 to 1 (C-axis) in ConvInteger and QLinearConv per DML spec - Guard InitializeOperator/Flush with ORT_TRY/ORT_CATCH so a single bad partition falls back instead of aborting the process - Extract DrainDmlDebugMessages helper and add device-removed HRESULT logging in CloseAndExecute - Rename ambiguous variables (cn, sg, im, npi, ni, ib, ci) for readability
apwojcik
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a generic Tier-2 BaseOp+Activation fusion rule and align several DirectML plugin behaviors with ONNX Runtime's DirectML execution provider.
Op+Activation fusion (new):
Runtime (non-constant) weight support:
ORT parity fixes:
fp16 execution flags:
Added a follow up
Fix Tier-0 graph fusion binding validation failure for ConvInteger models
The owned_graph_input_indices construction used raw packed DML slot
indices into the ONNX-ordered input_names array, causing initializer
weights to be looked up under wrong names for operators with
input_name_reorder (e.g. ConvInteger). This left OWNED_BY_DML
initializers unbound in the init binding table, triggering a DirectML
BindingValidator E_INVALIDARG and device removal during session init.
Additional fixes:
so initializers consumed through renamed edges get DML graph input slots
DataSize to avoid reading past ORT tensor allocations
QLinearMatMul, and MatMulIntegerToFloat for correct edge wiring when
optional inputs are absent
in ConvInteger and QLinearConv per DML spec
bad partition falls back instead of aborting the process
logging in CloseAndExecute
readability