[demandal25]

Summary

The README now states what was validated at the release head (the full suite on both architectures, in the supported image), what torch.compile does per op, and the one torch.compile known issue left after #397, with its workaround. The ROCm example scripts and tutorial notebook are removed: they duplicated the README's quick start and upstream's documentation, and nothing tested them. The README is also cut from 397 to 292 lines: it keeps quick start, usage, the supported configuration, the full support matrix and what upstream has that this does not, and moves the torch.compile and environment-variable reference into docs/rocm/.

What changed

• README.md, Supported hardware and toolchain: the paragraph saying gfx942 had only run the prefill, arch-caps and AITER-routing suites is replaced by the full-suite result on both arches, anchored to the v0.6.18+amd.2 tag.
• docs/rocm/torch-compile.md (new; the README keeps a summary and a one-line warning): adds an append row (append_paged_kv_cache and append_paged_mla_kv_cache compile with the flag on, under fullgraph too except backend="aiter" append), adds the fp8-quantizing norms to the ops the flag cannot register, and replaces "raises a clear error" with a measured per-op table (flag off, flag on, fullgraph=True), a footnote that AITER decode with return_lse=True needs backend="fa2" under fullgraph (Compile AITER decode with return_lse=True under custom ops #388), the host-side flag-on dispatch cost, and what the flag does not cover: plan() stays outside any compiled region, and sampling ops taking a generator, jit_args custom variants and AITER's opt-in norm/RoPE/activation shims are not registered. With the flag off the error is usually torch's RuntimeError when making fake tensor call.
• docs/rocm/torch-compile.md, Known issue: merges Stop declaring the paged KV cache mutated on read-only attention ops #397's append note into one rule: give every tensor an op writes its own storage, because PyTorch 2.12 can miscompile a write into a view of a larger tensor under every dynamic setting (tables below). That includes append's own single [pages, 2, ...] cache, which the wrapper splits into views, so append users are told to pass two separate caches. Four review rounds each found a layout or mode the previous statement of torch's trigger missed, so the README states the rule rather than the trigger. For reduce-overhead it keeps only Stop declaring the paged KV cache mutated on read-only attention ops #397's measured advice: register the KV cache as a module buffer, since AITER decode copies an argument cache on every replay. The paragraph this branch carried before Stop declaring the paged KV cache mutated on read-only attention ops #397, which scoped the issue to dynamic=True, is dropped.
• README.md, Basic usage: the pointer to examples/ is now a pointer to upstream's plan()/run() documentation, flagged as tracking upstream's latest release, plus the ROCm list of refused CUDA-only arguments.
• README.md, the rest (third commit): quick start keeps its commands and links CONTRIBUTING.md for the container gotchas; the two gfx950 routing paragraphs become a pointer to the per-op notes in docs/rocm/backends.md, which already hold them; benchmark caveats are left to benchmarks/rocm/README.md; the env-var table keeps the four variables a user typically sets.
• docs/rocm/environment.md (new): the full runtime env-var table and the runtime-helper snippet, moved verbatim. CONTRIBUTING.md's link to the README table now points here. The doc-ownership table in .claude/skills/pr-workflow/SKILL.md names both new pages.
• examples/: removes single_prefill_example.py, batch_prefill_example.py, batch_decode_example.py, amd_flashinfer_rocm_tutorial.ipynb and run_jupyter_server.sh. All five were fork-added (none exists at v0.6.18), so the additive-only rule is untouched; upstream's examples/pytorch/ stays.

Benchmark results

None: documentation and deleted example scripts only, so no kernel or dispatch path changes.

Test plan

Full suite at e1054f3, the release head (this PR is docs and deletions only), in docker/Dockerfile.rocm, one allocated card per arch (bus id checked on gfx950):

	gfx942 (MI300X)	gfx950 (MI355X)
-n auto --reruns 2 -m "not slow"	33867 passed, 4024 skipped, 0 failed	33875 passed, 4016 skipped, 0 failed¹
-n 1 --reruns 2 -m slow	72 passed	72 passed
Wall time	87 min, warm JIT cache	89 min, cold JIT cache

¹ Four git-dependent tests (test_amd_coverage's two manifest tests, test_redistribution_licences' sdist and wheel exclusions) failed on gfx950 because that copy of the tree had no .git. Both files pass, 125 of 125, in a git checkout of the same commit, and are counted as passed. No other test needed a rerun on either arch.

Flag-on dispatch at afb6440, single prefill qo16/kv128, minimum of 3 interleaved rounds on an idle allocated card:

	Flag off	Flag on	Delta	fa2 control delta
gfx942	21.61 µs	42.42 µs	+20.8 µs	+18.7 µs
gfx950	12.65 µs	23.38 µs	+10.7 µs	+8.2 µs

• torch.compile matrix in bf16 at e1054f3 on gfx942 and gfx950, identical cell for cell (fa2 and aiter; single/paged/ragged prefill, decode, MLA, MoE, append, MLA append; flag 0/1; default and fullgraph=True), every compiled result compared to eager: all match. The README table is that result.
• Writes into views at e1054f3, max |compiled − eager|, identical on gfx942 and gfx950:

Written arguments	dynamic=False	default	dynamic=True
Append into kv[0], kv[1] passed in	4.594	4.594	–
fused_add_rmsnorm(pool[1,0], pool[1,1]) passed in	8.25	8.25	–
Append into one [pages, 2, ...] cache	0	0	5.484
Append into separate K and V	0	0	0

One default-compiled function called at T = 32/48/64/80, once a size goes dynamic:

Written arguments	T=32	48	64	80
fused_add_rmsnorm(pool[1,0], pool[1,1]), pool shaped by T	0	12.1	12.4	13
rmsnorm(out=o[1]), o shaped by T	0	8.94	12.5	9.44
Either, on a buffer allocated once at the largest T	0	0	0	0

In-place RoPE on qkv.split views and MLA append on torch.split(buf, [512, 64]) were exact with dynamic=None, False and True on both arches.

• No remaining reference to the removed files (git grep for the file names, ipynb, jupyter; the matches left are upstream's).
• No Python or C++ changes, so no tests beyond the suite above apply.
• /code-review xhigh on origin/amd-integration...HEAD
• pre-commit run -a

[Copilot]

🟢 Approval recommended

The documentation matches the supplied validation and implementation evidence, and the example deletions leave no identified dangling references.

No open issues 🎉

What changed in this PR

Prepares the ROCm port’s documentation for v0.6.18+amd.2 by recording the validated configuration and compilation limits, while removing duplicated, untested examples.

Changes:

• Records full-suite results on gfx942 and gfx950 in the supported development image.
• Documents measured torch.compile support, dispatch overhead, and remaining workarounds following #397 and #388.
• Replaces ROCm example links with upstream API documentation and ROCm-specific restrictions, then removes five fork-added example files.

File	Description
README.md	Updates release validation, compilation guidance, and API documentation links.
examples/​single_prefill_example.py	Removes the standalone single-prefill example.
examples/​run_jupyter_server.sh	Removes the tutorial’s Jupyter launcher.
examples/​batch_prefill_example.py	Removes the paged and ragged prefill examples.
examples/​batch_decode_example.py	Removes the standalone batch-decode example.
examples/​amd_flashinfer_rocm_tutorial.ipynb	Removes the duplicated ROCm tutorial notebook.

_{Review effort: Balanced}

[github-actions]

Security scan found no critical or high findings on lines changed by this pull request

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

[Copilot]

🟡 Changes recommended

The installation guidance incorrectly promises fallback without AITER for APIs that require it.

1 open issue

• Limit fallback claim to AITER routes that support fallback · New

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No unresolved blockers remain in these documentation updates and example removals, and the prior AITER fallback concern is addressed.

No open issues 🎉

1 resolved since last review

• Limit fallback claim to AITER routes that support fallback

_{Review effort: Balanced}