Repository navigation
[26.10] [DOC] FlashInfer: documentation updates for release - #389
Open
anisha-amd wants to merge 25 commits into
Open
anisha-amd wants to merge 25 commits into
anisha-amd wants to merge 25 commits into
Conversation
demandal25
requested review from
demandal25
and
a balanced review from Copilot
October 20, 2026 17:35
There was a problem hiding this comment.
🟡 Changes recommended
Installation guidance, runtime controls, and capability claims conflict with the supported implementation.
11 open issues
What changed in this PR
Adds ROCm-focused FlashInfer documentation for the 26.10 release.
Changes:
- Documents installation, examples, capabilities, and environment variables.
- Adds site navigation, licensing, and Sphinx/Read the Docs configuration.
| File | Description |
|---|---|
| amd-docs/what-is-flashinfer.rst | Overview and kernel support matrix. |
| amd-docs/sphinx/requirements.txt | Documentation dependencies. |
| amd-docs/sphinx/_toc.yml.in | Site navigation. |
| amd-docs/reference/environment-variables.rst | Runtime settings and helpers. |
| amd-docs/install/flashinfer-install.rst | Installation requirements and commands. |
| amd-docs/index.rst | Documentation landing page. |
| amd-docs/examples/flashinfer-examples.rst | AITER prefill example. |
| amd-docs/conf.py | Sphinx theme and release metadata. |
| amd-docs/about/license.rst | License inclusion. |
| amd-docs/.readthedocs.yaml | Documentation build configuration. |
Review effort: Balanced
…only AITER claim' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…from AITER reinstallation' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
… index' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…ant directory variable' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…e fallback claim' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…mbly environment variables' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…ctions to supported APIs' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…he claims' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
…pport' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
Clears the pre-commit failures on #389: trailing-whitespace in four amd-docs files and ruff-format on conf.py. No content change. Co-Authored-By: Claude <noreply@anthropic.com>
1 task done
…-up-tag Bring release/0.6.18+amd.1 up to the v0.6.18+amd.1 tag
There was a problem hiding this comment.
🟡 Changes recommended
Previously identified dependency mismatches and unsupported runtime controls remain unresolved in the supplied documentation.
1 open issue
11 resolved since last review
Review effort: Balanced
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
demandal25
reviewed
Oct 21, 2026
yugang-amd
reviewed
Oct 21, 2026
Co-authored-by: yugang-amd <yugang.wang@amd.com> Signed-off-by: anisha-amd <anisha.sankar@amd.com>
Drop the AITER install section. The development image builds AITER from the
pinned v0.1.21.post2 tag, so neither recipe applies, and the pip one named a
0.1.20 wheel that the shim's AITER_MIN_VERSION = "0.1.21" floor rejects.
Narrow FLASHINFER_AITER_STRICT. plan() gates the probe on routes_natively and
_aiter_paged_route_page_sizes() is `native & {1024}` for fp16/bf16, so every
other page size takes flat-gather whether or not strict is set.
Add AITER's fp8 paged route (head dim 128 or 256) to the batch-prefill row,
per the batch_prefill/aiter capability note in arch_caps.py.
Co-Authored-By: Claude <noreply@anthropic.com>
Add MI350X to the GPU lists. gfx950 is MI350X and MI355X, which the
environment-variables page already said in two places while the support matrix
and the prerequisites named only MI355X.
Correct FLASHINFER_TRACE_APPLY. flashinfer/__init__.py gates on
`not in ("0", "", "false", "False")`, so "off", "no" and "FALSE" all enable
kernel substitution rather than being warned about and ignored.
Drop --ipc=host from the docker run. It shares the host IPC namespace, which
makes --shm-size a no-op; the README and CONTRIBUTING recipes set --shm-size
without it.
Co-Authored-By: Claude <noreply@anthropic.com>
demandal25
previously approved these changes
Oct 22, 2026
demandal25
changed the base branch from
release/0.6.18+amd.1
to
release/0.6.18+amd.2
October 26, 2026 21:16
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
demandal25
self-requested a review
October 28, 2026 16:24
demandal25
previously approved these changes
Oct 28, 2026
Six claims in amd-docs/what-is-flashinfer.rst disagreed with the generated matrix in README.md, which is emitted from flashinfer/rocm/arch_caps.py. The lead paragraph said operations run on AITER wherever it is supported. That is false for five of the eight routed ops: rmsnorm, fused_add_rmsnorm, silu_and_mul, rope and append_paged_kv_cache always resolve to the in-tree kernel, with AITER opt-in. Batch decode and append paged KV cache advertised an fp8 KV cache without naming a backend. AITER's PA v1 raises on anything but fp16/bf16 (flashinfer/rocm/decode.py:539-543) and the AITER append path is fp16/bf16 too, so fp8 is the HIP kernel in both rows. Cascade's default was listed as HIP. HIP supplies only the merge kernels; each level runs through the ordinary prefill and decode entry points at backend="auto" and can reach AITER, which no cascade wrapper exposes a backend= to override. POD omitted that an fp8 KV cache is refused (docs/rocm/backends.md:865), while every neighbouring attention row advertises fp8. Fused MoE omitted that weights must be pre-shuffled with shuffle_moe_weight (flashinfer/rocm/fused_moe.py:10,218). Unshuffled weights are not rejected, they are large relative error, so this is the one omission that costs a wrong answer rather than an exception. No rows were added or removed: all 19 ops in the generated matrix already map onto the 16 existing rows. Per-backend constraints stay out of the table -- the closing paragraph now names the categories (dtypes, layouts, page sizes, fp8 spellings, when auto declines AITER) and points at the README, which is generated and pre-commit checked, rather than restating them here where they would drift. Two fixes outside the table. The install page's docker run was missing --group-add for the host's render GID, so the documented command yields a container that cannot open a GPU: the image ends on USER $USERNAME (docker/Dockerfile.rocm:215), and --privileged gives a non-root uid no effective CAP_DAC_OVERRIDE, so the DAC check on the 0660 root:render node still runs. Measured on a gfx942 host, flags otherwise identical: without --group-add the open fails EACCES; with the host's numeric render GID it passes and fails EPERM from the DRM driver instead, which only a successful open can reach. The examples page said backend="aiter" selects AITER for any operation; single_decode_with_kv_cache, aiter_fused_moe and the cascade wrappers accept no backend argument. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation updates for release.