graph: backend: dnnl, tests: benchdnn: support sdpa / gqa training with gradients for mask #4404
Conversation
| data_type::s8, data_type::u8, data_type::s32, | ||
| data_type::undef})) | ||
| data_type::undef}) | ||
| .set_shape_inference_function(infer_dummy_output_shape)) |
There was a problem hiding this comment.
Why do we need this? By definition, there is no output for an End op.
There was a problem hiding this comment.
The graph's infer shape function will iterate every op in the graph, we can certainly check if one op has output or not to avoid adding a dummy infer shape function for End op here, but I think it's not a better choice because in that way, every op will have to check that condition.
|
make test |
TaoLv
left a comment
TaoLv
left a comment
There was a problem hiding this comment.
Any changes required for document? What's the main difference in the fusion graph?
| pgraph->create_input_port(2, matmul_dv, 1); | ||
| pgraph->create_input_port(2, matmul_v_do, 0); | ||
| }) | ||
| .set_attr<FCreatePattern>("FCreatePattern", |
There was a problem hiding this comment.
Just curious why a new pattern and not incorporating into existing one?
There was a problem hiding this comment.
It's because incorporating into existing one requires the pattern matcher's optional subgraph supports two consumers, which is currently not supported
| if (cur_op_refs.size() == 2 && cur_op_refs[0].kind_ == "End") { | ||
| matmul_idx = 1; | ||
| } |
There was a problem hiding this comment.
Would it be better if:
while (cur_op_refs[matmul_idx].kind_ != "MatMul") {
matmul_idx++;
}
There was a problem hiding this comment.
Not really, becuase matmul_idx (maybe it's not a good name) doesn't mean the consumer must be matmul (below find_next_until indicates that), it means matmul is in the chain of that consumer. On the other hand, the End op is determined.
w.r.t gradients for mask
w.r.t gradients for mask
for sdpa/gqa training backward w.r.t gradients for mask
w.r.t. gradients for mask
ElaineBao
force-pushed
the
yixin/sdpa_mask_grad
branch
from
597aa04 to
e6644ff
Compare
github-actions
bot
added
the
documentation
Added document changes, please take another look @TaoLv |
3 participants
Description
MFDNN-14010.