File tree Expand file tree Collapse file tree 2 files changed +6
-5
lines changed
Expand file tree Collapse file tree 2 files changed +6
-5
lines changed Original file line number Diff line number Diff line change @@ -59,7 +59,6 @@ MC_TE_METRIC=true \
5959SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE=100000 \
6060SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT=100000 \
6161SGLANG_DISAGGREGATION_WAITING_TIMEOUT=100000 \
62- SGLANG_MOONCAKE_CUSTOM_MEM_POOL=True \
6362MC_FORCE_MNNVL=1 \
6463NCCL_MNNVL_ENABLE=1 \
6564NCCL_CUMEM_ENABLE=1 \
@@ -129,10 +128,10 @@ python3 -m dynamo.sglang \
129128 --disaggregation-mode decode \
130129 --dist-init-addr ${HEAD_DECODE_NODE_IP} :29500 \
131130 --disaggregation-bootstrap-port 30001 \
132- --nnodes 12 \
131+ --nnodes 2 \
133132 --node-rank 0 \
134- --tp-size 48 \
135- --dp-size 48 \
133+ --tp-size 8 \
134+ --dp-size 8 \
136135 --enable-dp-attention \
137136 --host 0.0.0.0 \
138137 --decode-log-interval 1 \
@@ -157,4 +156,4 @@ python3 -m dynamo.sglang \
157156 --disaggregation-transfer-backend nixl
158157```
159158
160- On the other decode nodes (this example has 2 total decode nodes), run the same command but change ` --node-rank ` to 1
159+ On the other decode nodes (this example has 2 total decode nodes), run the same command but change ` --node-rank ` to 1.
Original file line number Diff line number Diff line change @@ -39,6 +39,7 @@ python3 -m dynamo.sglang \
3939 --disaggregation-mode prefill \
4040 --disaggregation-transfer-backend nixl \
4141 --disaggregation-bootstrap-port 30001 \
42+ --load-balance-method round_robin \
4243 --host 0.0.0.0 \
4344 --mem-fraction-static 0.82
4445```
@@ -60,6 +61,7 @@ python3 -m dynamo.sglang \
6061 --disaggregation-transfer-backend nixl \
6162 --disaggregation-bootstrap-port 30001 \
6263 --host 0.0.0.0 \
64+ --load-balance-method round_robin \
6365 --mem-fraction-static 0.82
6466```
6567
You can’t perform that action at this time.
0 commit comments