rocm: sync branch with main

[24601]

Summary

This brings the rocm branch forward to current main (8d576642) while keeping the ROCm patch scoped to the existing community ROCm backend work.

Current state before this change:

• rocm was based at 7a751eb
• rocm was 85 commits behind main and 1 commit ahead

What changed:

• merge current main into rocm with conflicts resolved
• keep the ROCm backend support buildable against the current DS4 GPU code
• add a make rocm target that builds the current main binaries through hipcc
• keep HIP out of CUDA WMMA kernels and use the generic indexer-score fallback on ROCm
• explicitly reject known wave64 GCN/CDNA ROCm targets for now, because DS4 GPU kernels still assume CUDA-style 32-lane warp math

This intentionally does not include the user-facing --rocm naming/MTP mapping work from #156 or the ROCm WMMA/indexer optimization work from #180.

Relation to #16

This is related to #16, but it should not close #16.

The goal here is to make the existing upstream rocm branch current and testable again on current main. It is not the later ROCm optimization work discussed in #16, and the speed numbers below show why this should stay draft until the remaining correctness/performance gaps are understood.

Scope notes

I checked prior ROCm-related PRs before keeping this narrowly scoped:

• Add AMD ROCm/HIP Support and Strix Halo Optimizations #118 was broader than a branch refresh and overlapped with the existing rocm branch
• rocm: rebased on top of the current main, removed unnecessary (previously introduced) changes to ds4_cuda.cu #133 attempted a prior ROCm branch update and surfaced remaining long-context risk
• Fix ROCm build/runtime naming and MTP model mapping #156 is a separate CLI/runtime naming and MTP mapping fix
• rocm: add wmma indexer support #180 is a separate ROCm WMMA/indexer optimization effort

This PR is therefore only a branch sync plus the minimum build fixes needed for current main.

Validation environment

• AMD Ryzen AI Max+ 395 / Radeon 8060S
• ROCm target: gfx1151
• HIP/ROCm toolchain: HIP 7.2.53211-364a905
• Backend selected by DS4: cuda API path over ROCm/HIP
• Model: official project q2-imatrix GGUF, DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf
• Model size reported by DS4: 80.76 GiB, 284.33B logical parameters
• PR head tested: e13de731787799310e53bffe98b1f077f7c96b50

Build and unit validation

Commands run:

make clean
make rocm ROCM_ARCH=gfx1151 -j$(nproc)
make ds4_test GPU_BACKEND=rocm ROCM_ARCH=gfx1151 -j$(nproc)
./ds4_test --server
make clean
make cpu -j$(nproc)

Results:

• ROCm build: PASS
• ROCm-linked ds4_test --server: PASS (server: OK, ds4 tests: ok)
• CPU build: PASS

Negative target guard check:

make clean
make ds4_test GPU_BACKEND=rocm ROCM_ARCH=gfx90a -j$(nproc)

Result: expected compile failure with the new wave64 GCN/CDNA unsupported-target error.

Existing warnings observed during the builds:

• ds4_server.c: const-discard warning in stop_list_find_from
• ds4_agent.c: existing snprintf truncation warnings

Model-backed validation

Model load / inspect:

./ds4 --inspect --cuda -m ds4flash.gguf

Result: PASS. DS4 loaded the official q2-imatrix GGUF and initialized the ROCm/HIP backend.

Short deterministic generation:

./ds4 --cuda -m ds4flash.gguf --ctx 4096 --nothink --temp 0 -n 32 -p 'Reply exactly: ROCm OK'

Result: PASS. Output was ROCm OK.

Reported speed for this tiny prompt:

prefill: 0.46 t/s, generation: 8.96 t/s

Tool-call quality:

DS4_TEST_MODEL=ds4flash.gguf ./ds4_test --tool-call-quality

Result: PASS in both fast and exact paths.

Long-context story recall:

DS4_TEST_MODEL=ds4flash.gguf ./ds4_test --long-context

Result: PASS (long-context: OK, ds4 tests: ok).

Runtime note: this run took about one hour wall time on the validation machine. Progress markers reached 8192/30474, 16384/30474, 24576/30474, and 30474/30474; correctness passed, but throughput is not yet competitive with the best ROCm numbers discussed in #16.

Official logprob vectors:

DS4_TEST_MODEL=ds4flash.gguf ./ds4_test --logprob-vectors

Result: FAIL with one mismatch:

ds4-test: vector short_code_completion step 1 selected token mismatch
logprob-vectors: ERR
ds4 tests: 1 failure(s)

Manual dump for the failing prompt showed ROCm selecting uppercase C after the opening code fence while the fixture expects lowercase c:

step 1 selected 'C'
top candidates: 'C' logprob -0.226568833, 'c' logprob -1.59836829

The exact/quality ROCm path also selected uppercase C for the same prompt:

step 1 selected 'C'
top candidates: 'C' logprob -0.495503277, 'c' logprob -0.941311657

Speed smoke

Command:

./ds4-bench \
  -m ds4flash.gguf \
  --cuda \
  --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 2048 \
  --ctx-max 8192 \
  --step-incr 2048 \
  --gen-tokens 128

Result:

ctx_tokens,prefill_tokens,prefill_tps,gen_tokens,gen_tps,kvcache_bytes
2048,2048,57.76,128,7.63,52184460
4096,2048,46.47,128,7.59,80373132
6144,2048,39.49,128,7.37,108561804
8192,2048,34.26,128,7.41,136750476

These are usable but below the better Strix Halo ROCm numbers reported in #16, which is expected for this branch-sync PR because it deliberately does not include the later ROCm optimization work.

Additional review

Read-only adversarial reviews were run with DeepSeek V4, Cursor Composer 2.5, and Gemini 3.5 Flash. They did not identify a branch-sync merge blocker. Their actionable feedback is reflected here:

• added the explicit wave64 GCN/CDNA compile guard
• documented the HIP shared-memory opt-in semantic difference in the shim
• made the model-dependent test evidence and remaining gaps explicit in this PR body

Draft blockers / remaining gaps

• ./ds4_test --logprob-vectors does not fully pass on ROCm because of the short_code_completion C vs c mismatch.
• ROCm performance is below the stronger issue Support for AMD GPU (ROCm/HIP) backend #16 benchmark reports.
• CUDA regression testing on NVIDIA hardware was not run.
• MTP/speculative decoding was not validated here; that is intentionally separate from this branch sync and overlaps with Fix ROCm build/runtime naming and MTP model mapping #156.
• Full 65K benchmark sweep was not run; the speed smoke covers 2K-8K plus the passing 30K-token long-context correctness test.

[WebReflection]

On average I have ~9 t/s and prefill up to double that, but while on DGX Spark I can see the whole RAM being used, I wonder if I need to force UMA buffer up to 96GB or if leaving it auto is good enough, and these are just the performance this 395+ Max can produce, thanks in advance for any sort of clarification. (ArchLinux btw)