Skip to content

[XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu #76362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 24, 2025
Merged

[XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu #76362

merged 1 commit into from
Nov 24, 2025

Conversation

@jianingyu-ustc
Copy link
Contributor

@jianingyu-ustc jianingyu-ustc commented Nov 11, 2025 •
edited
Loading

PR Category

Custom Device

PR Types

New features

Description

[XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu

@CLAassistant
Copy link

CLAassistant commented Nov 11, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLAassistant commented Nov 11, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ZibinGuo
❌ jianingyu-ustc
You have signed the CLA already but the status is still pending? Let us recheck it.

@paddle-bot
Copy link

paddle-bot bot commented Nov 11, 2025

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Nov 11, 2025
@jianingyu-ustc jianingyu-ustc changed the title [XPU] [DEEP_EP 2/4] DeepEP support xpu [XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu Nov 19, 2025
@jianingyu-ustc jianingyu-ustc force-pushed the develop branch 4 times, most recently from 6ff38bb to 21fb80c Compare November 20, 2025 15:21
@jianingyu-ustc jianingyu-ustc marked this pull request as draft November 21, 2025 02:49
@jianingyu-ustc jianingyu-ustc marked this pull request as ready for review November 21, 2025 02:50
@jianingyu-ustc jianingyu-ustc changed the title [XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu [XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu Nov 21, 2025
@jianingyu-ustc jianingyu-ustc changed the title [XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu [XPU] [DEEP_EP 2/4] DeepEP normal intranode / internode support xpu Nov 21, 2025
@jianingyu-ustc jianingyu-ustc marked this pull request as draft November 21, 2025 02:59
@jianingyu-ustc jianingyu-ustc marked this pull request as ready for review November 21, 2025 03:00
ZibinGuo
ZibinGuo reviewed Nov 21, 2025
View reviewed changes
paddle/fluid/distributed/collective/deep_ep_xpu/config.hpp

// Message sizes
EP_HOST_ASSERT(num_scales * sizeof(float) <= hidden);
EP_HOST_ASSERT(num_scales * sizeof(float) <= static_cast<size_t>(hidden));
Copy link
Contributor

@ZibinGuo ZibinGuo Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里直接判断会有问题吗？

Copy link
Contributor Author

@jianingyu-ustc jianingyu-ustc Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

应该没有吧...，两边都会转换为size_t

ForFishes
ForFishes approved these changes Nov 21, 2025
View reviewed changes
Copy link
Member

@ForFishes ForFishes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

qingqing01
qingqing01 approved these changes Nov 21, 2025
View reviewed changes
Copy link
Contributor

@qingqing01 qingqing01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for library size, since this PR is unrelated to GPU.

ZibinGuo
ZibinGuo approved these changes Nov 21, 2025
View reviewed changes
Copy link
Contributor

@ZibinGuo ZibinGuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ZibinGuo
Copy link
Contributor

ZibinGuo commented Nov 21, 2025

deep_ep 功能实现，差单测

@ZibinGuo
Copy link
Contributor

ZibinGuo commented Nov 21, 2025

/re-run all-failed

1 similar comment
@jianingyu-ustc
Copy link
Contributor Author

jianingyu-ustc commented Nov 22, 2025

/re-run all-failed

dynamicheart
dynamicheart reviewed Nov 24, 2025
View reviewed changes
paddle/fluid/distributed/collective/process_group_bkcl.cc
calc_event_ = std::make_shared<XPUEventManager>();
auto* calc_ctx = static_cast<phi::XPUContext*>(
phi::DeviceContextPool::Instance().Get(place));
calc_ctx->CreateStream();
Copy link
Contributor

@dynamicheart dynamicheart Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calc_ctx似乎已经默认创建四个流了，这里去给calc_ctx创建一个新的流的目的是什么？

Copy link
Contributor

@ZibinGuo ZibinGuo Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

XPU由于历史原因，一直使用默认流，只有手动调用CreateStream()才会创建新流，但GPU每当创建一个CUDAStream, 就会创建一个新流。这里deep_ep为了建立通信流和计算流，且和GPU对齐，故此修改。

Copy link
Contributor

@dynamicheart dynamicheart Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle/paddle/phi/backends/xpu/xpu_context.cc

Lines 250 to 257 in f9062d5

void CreateStream() {
if (context_->xpu_stream) {
VLOG(3) << "xpu stream is already created for current context";
return;
}
PADDLE_ENFORCE_XPU_SUCCESS(xpu_stream_create(&context_->xpu_stream));
stream_owned_ = true;
}

如果CDNN_CLUSTER_PARALLEL已经创建了流，调用这个接口无效

paddle/phi/core/distributed/comm_context_manager.cc
if (CommContextManager::device_id != -1) {
std::unique_ptr<phi::XPUContext> dev_ctx(new phi::XPUContext(
phi::XPUPlace(CommContextManager::device_id), true));
dev_ctx->CreateStream();
Copy link
Contributor

@dynamicheart dynamicheart Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

涉及通信库Stream的修改，请找 @lj970926 来review下

Copy link
Contributor

@ZibinGuo ZibinGuo Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改的时候就和lj970926还有XiaociZhang对过

dynamicheart
dynamicheart approved these changes Nov 24, 2025
View reviewed changes
Copy link
Contributor

@dynamicheart dynamicheart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dynamicheart dynamicheart merged commit 15301e0 into PaddlePaddle:develop Nov 24, 2025
121 of 135 checks passed
lj970926
lj970926 approved these changes Nov 24, 2025
View reviewed changes
Copy link
Contributor

@lj970926 lj970926 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for comm stream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qingqing01 qingqing01 qingqing01 approved these changes

@zyfncg zyfncg zyfncg approved these changes

@ForFishes ForFishes ForFishes approved these changes

@dynamicheart dynamicheart dynamicheart approved these changes

@sneaxiy sneaxiy Awaiting requested review from sneaxiy sneaxiy is a code owner

+2 more reviewers

@ZibinGuo ZibinGuo ZibinGuo approved these changes

@lj970926 lj970926 lj970926 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

contributor External developers

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@jianingyu-ustc @CLAassistant @ZibinGuo @qingqing01 @zyfncg @ForFishes @dynamicheart @lj970926