Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .claude/skills/code-coverage/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,8 @@ docker exec <container> git config --global --add safe.directory '*'
```

There is **no AMD GPU CI** in this repo, so no automation produces this number
— the workflows that name GPUs are upstream's, on NVIDIA runners, and every
other one runs on `ubuntu-latest` or a CPU runner. The classifier's own
tests (`tests/rocm/test_amd_coverage.py`) do run there, via
— no workflow has a GPU runner. The classifier's own
tests (`tests/rocm/test_amd_coverage.py`) do run in CI, on a CPU runner, via
`arch-caps-conformance.yml`.

## Failure modes worth recognising
Expand Down
88 changes: 5 additions & 83 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,84 +1,6 @@
# CODEOWNERS — manually maintained, not auto-generated.
# Learn about CODEOWNERS file format:
# https://help.github.com/en/articles/about-code-owners
#
# Rules:
# - Last matching pattern wins (GitHub semantics).
# - Catch-all ensures every PR gets at least one core-maintainer review.
# - Prefer @flashinfer-ai/<team> over individual users. (currently working around this)

# ── Catch-all: core maintainers review everything not covered below ──
* @yzh119 @sricketts @aleozlx @yongwww @cyx-6 @saltyminty @bkryu @yyihuang @kahyunnam @jimmyzho @nv-yunzheq @samuellees @dhiraj113 @qsang-nv @jiahanc @qiching @feih-nv @Aneureka

# ── Attention ──
csrc/fmha_v2/ @saltyminty @yzh119 @nv-yunzheq @bkryu @qsang-nv @jimmyzho
csrc/xqa/ @saltyminty @yzh119 @nv-yunzheq @bkryu @qsang-nv
flashinfer/attention.py @saltyminty @yzh119 @nv-yunzheq @bkryu @qsang-nv
include/flashinfer/attention/ @saltyminty @yzh119 @nv-yunzheq @bkryu @qsang-nv
tests/attention/ @saltyminty @yzh119 @nv-yunzheq @bkryu @qsang-nv @jimmyzho

# ── GEMM ──
flashinfer/gemm/ @dhiraj113 @yzh119 @aleozlx @bkryu @jiahanc @StudyingShao
include/flashinfer/gemm/ @dhiraj113 @yzh119 @aleozlx @bkryu @jiahanc @StudyingShao
tests/gemm/ @dhiraj113 @yzh119 @aleozlx @bkryu @jiahanc @StudyingShao

# ── Grouped GEMM ──
flashinfer/grouped_mm/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @StudyingShao
include/flashinfer/grouped_mm/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @StudyingShao
tests/grouped_mm/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @StudyingShao

# ── MOE ──
csrc/fused_moe/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @feih-nv @Aneureka @StudyingShao
flashinfer/fused_moe/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @feih-nv @Aneureka @StudyingShao
include/flashinfer/trtllm/fused_moe/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @feih-nv @Aneureka @StudyingShao
tests/moe/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @feih-nv @Aneureka @StudyingShao

# ── MOE_EP ──
flashinfer/moe_ep/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @Anerudhan @mhoqueanik @feih-nv @Aneureka
tests/moe_ep/ @aleozlx @yzh119 @jiahanc @IwakuraRein @nv-yunzheq @samuellees @Anerudhan @mhoqueanik @feih-nv @Aneureka

# ── Communication ──
flashinfer/comm/ @aleozlx @yzh119 @jimmyzho @bkryu @nv-yunzheq @Anerudhan
include/flashinfer/comm/ @aleozlx @yzh119 @jimmyzho @bkryu @nv-yunzheq @Anerudhan
tests/comm/ @aleozlx @yzh119 @jimmyzho @bkryu @nv-yunzheq @Anerudhan

# ── Norm ──
flashinfer/norm.py @kahyunnam @yzh119 @bkryu @yongwww
tests/norm/ @kahyunnam @yzh119 @bkryu @yongwww

# ── GDN ──
benchmarks/gdn/ @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
csrc/gdn_prefill_launcher.cu @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
csrc/gdn_prefill_sm90_kernel_inst.jinja @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
flashinfer/gdn_decode.py @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
flashinfer/gdn_kernels/ @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
flashinfer/gdn_prefill.py @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
flashinfer/jit/gdn.py @kahyunnam @yzh119 @bkryu @yongwww @jiahanc
tests/gdn/ @kahyunnam @yzh119 @bkryu @yongwww @jiahanc

# ── MAMBA ──
csrc/checkpointing_ssu* @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
csrc/selective_state_update* @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
csrc/seq_chunk_cumsum* @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
flashinfer/jit/mamba/ @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
flashinfer/mamba/ @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
include/flashinfer/mamba/ @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun
tests/mamba/ @kahyunnam @yzh119 @bkryu @jimmyzho @ishovkun

# ── KDA ──
benchmarks/bench_recurrent_kda* @kahyunnam @yzh119 @bkryu @jimmyzho
csrc/kda/ @kahyunnam @yzh119 @bkryu @jimmyzho
docs/api/kda.rst @kahyunnam @yzh119 @bkryu @jimmyzho
docs/api/kda_decode.rst @kahyunnam @yzh119 @bkryu @jimmyzho
docs/api/kda_prefill.rst @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/jit/flash_kda.py @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/jit/flash_kda_decode.py @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/kda.py @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/kda_decode.py @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/kda_kernels/ @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/kda_prefill.py @kahyunnam @yzh119 @bkryu @jimmyzho
flashinfer/trace/templates/kda.py @kahyunnam @yzh119 @bkryu @jimmyzho
tests/jit/test_flash_kda* @kahyunnam @yzh119 @bkryu @jimmyzho
tests/kda/ @kahyunnam @yzh119 @bkryu @jimmyzho

# ── Autotuner ──
flashinfer/autotuner/ @qiching @yzh119 @bkryu @aleozlx
tests/autotuner/ @qiching @yzh119 @bkryu @aleozlx
# Replaced by Advanced Micro Devices, Inc.: upstream's owners have no access
# to this fork. Kept AMD-only on upstream syncs.
* @demandal25
15 changes: 6 additions & 9 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Modified by Advanced Micro Devices, Inc. for the ROCm port (Apache-2.0 sec. 4(b)).
name: 🐛 Bug Report
description: Report something broken, wrong, or slower than it used to be.
labels: ["bug"]
Expand All @@ -14,16 +15,12 @@ body:
attributes:
label: Environment
description: >
Run `python -m flashinfer.collect_env` (or `flashinfer collect-env`)
in the exact environment (same venv/container) where the bug
reproduces and paste the full output below. If `import flashinfer` itself is broken, download the
standalone script and run it with plain Python:
`curl -OL https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/flashinfer/collect_env.py && python collect_env.py`.
The report includes which cuDNN/cuBLAS/CUDA libraries are *actually
loaded* vs merely installed — this resolves most "works on my machine"
divergence, so please don't trim it.
From the exact environment (same venv/container) where the bug
reproduces: the GPU (`amd-smi static --asic`), the ROCm version
(`cat ${ROCM_PATH:-/opt/rocm}/.info/version` -- `torch.version.hip`
reports HIP, not ROCm), the OS, and
`python -m pip list | grep -Ei '^(torch|amd-aiter|amd-flashinfer)'`.
render: text
placeholder: Paste the full output of `python -m flashinfer.collect_env` here.
validations:
required: true
- type: textarea
Expand Down
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Modified by Advanced Micro Devices, Inc. for the ROCm port (Apache-2.0 sec. 4(b)).
name: 🚀 Kernel / Feature Request
description: Request a kernel, hardware enablement, framework integration, API, or performance improvement.
title: "[Feature]: "
Expand Down Expand Up @@ -107,15 +108,14 @@ body:
label: Workload and configuration
description: >
The single most useful section for kernel/performance requests — enough
detail to identify the operator path and representative shapes. For the
environment line, paste the output of `python -m flashinfer.collect_env`.
detail to identify the operator path and representative shapes.
value: |
- Data type and quantization:
- Batch size or request concurrency:
- Sequence lengths or token counts:
- Parallelism: TP / EP / DP / PP / disaggregated
- Relevant shapes: heads, head dimension, hidden size, experts, top-k, page size, or other
- Environment: output of `python -m flashinfer.collect_env`
- Environment: GPU, ROCm, torch and AITER versions
validations:
required: false

Expand Down
22 changes: 22 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Copyright Advanced Micro Devices, Inc.
# SPDX-License-Identifier: MIT
#
# Dependabot version updates. See
# https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference

version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
# Wait a week before adopting newly released action versions.
cooldown:
default-days: 7
exclude:
- "ROCm/rocm-security-gh*"
groups:
rocm-security-gh:
patterns: ["ROCm/rocm-security-gh*"]
github-actions:
patterns: ["*"]
37 changes: 37 additions & 0 deletions .github/scan_tools_configs/bandit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Copyright Advanced Micro Devices, Inc.
# SPDX-License-Identifier: MIT
#
# bandit configuration
# Reference: https://bandit.readthedocs.io/en/latest/config.html
# Local scan: bandit --recursive . --configfile .github/scan_tools_configs/bandit.yml
#
# Bandit configuration for this repository's security scans.
#
# This file defines the complete scanner policy;
# Keep repository-specific suppressions and tuning here.

# Environment and generated-output exclusions support local scans; these
# paths are absent from clean CI checkouts.
# Bandit also matches each entry as a plain substring of the path, so a bare
# `build` would drop build_backend_rocm.py and `.git` all of .github/; anchor
# every entry as a */dir/* glob.
exclude_dirs:
- "*/.git/*"
- "*/.venv*/*"
- "*/venv*/*"
- "*/build/*"
- "*/dist/*"
- "*/node_modules/*"
- "*/__pycache__/*"
# Reviewed HIGH findings in upstream files, which take no `# nosec` here:
# constant-command shell=True, md5/sha1 as cache and store keys, and an
# extractall that passes filter="data" wherever tarfile.data_filter exists.
- "*/flashinfer/collect_env.py"
- "*/flashinfer/deep_gemm.py"
- "*/flashinfer/moe_ep/core/bootstrap_utils.py"
- "*/scripts/check_pr_document.py"

# pytest asserts would bury the weekly code-scanning results; library asserts
# stay reported.
assert_used:
skips: ["*/tests/*", "*/test_*.py"]
105 changes: 105 additions & 0 deletions .github/scan_tools_configs/gitleaks.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Copyright Advanced Micro Devices, Inc.
# SPDX-License-Identifier: MIT
#
# gitleaks.toml
# Reference: https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml
# Local scan: gitleaks detect --source . --config .github/scan_tools_configs/gitleaks.toml
#
# Gitleaks configuration for this repository's security scans.
#
# This file defines the repository's complete Gitleaks policy. Recurring
# exclusions belong in this file.
#
# The `[extend]` section inherits Gitleaks' built-in rules, and the
# `[[allowlists]]` below apply to every rule.
#
# Use `.gitleaksignore` for one-off finding fingerprints. Add recurring path
# or regex exclusions to `[[allowlists]]` in this file.

title = "gitleaks config"

[extend]
# Inherit gitleaks' built-in ruleset (rotations, AWS, GCP, Slack, ...).
# Setting this to false would mean rolling our own detections, which
# is almost never what you want.
useDefault = true



# Allowlist field-vs-target reference:
# stopwords -> always matched against the SECRET value.
# regexes -> matched against `regexTarget` (default: "secret").
# regexTarget -> "secret" | "match" | "line".
# Use "match" or "line" when the "tell" is the variable name or
# surrounding syntax, not the value itself.

[[allowlists]]
# Allowlist by location only where a real first-party secret structurally
# can't live. Dependency lock files qualify: their contents are generated
# from a manifest, and the high-entropy strings they carry are artifact
# digests rather than credentials. Add vendored third-party trees here as
# callers bring them in, one explicit path per entry.
description = "Generated dependency lock files"
paths = [
'''.*\.lock$''',
]

[[allowlists]]
description = "Test fixtures using well-known dummy variable names"
# The "tell" here is the variable name on the same line, not the value
# (the value is a random-looking string by design). Match against the
# full rule match (variable + value) so we anchor on the identifier.
regexTarget = "match"
regexes = [
'''(?i)\bINVALID_TOKEN\s*=''',
'''(?i)\bwrong_secret\s*=''',
# Explicit test-only secret identifiers
'''(?i)\btest_secret\s*=''',
'''(?i)\bdummy_secret\s*=''',
'''(?i)\bfake_secret\s*=''',
'''(?i)\bexample_secret\s*='''
]

[[allowlists]]
description = "Public artifact digests (variable names ending in SHA256/512/MD5)"
# Scoped here: a line-target allowlist with no targetRules would also hide a
# provider token sharing the line.
targetRules = ["generic-api-key"]
# SHA-256/512/MD5 hashes of public artifacts are not secrets.
regexTarget = "line"
regexes = [
'''[A-Z][A-Z0-9_]*SHA256\s*[=:]''',
'''[A-Z][A-Z0-9_]*SHA512\s*[=:]''',
'''[A-Z][A-Z0-9_]*MD5\s*[=:]''',
]

# --- amd-flashinfer tuning ---------------------------------------------------
# Kernel code is full of names next to "token"/"key" that clear the generic
# rule's entropy bar. Scoped to generic-api-key, so provider rules still fire;
# GitHub secret scanning with push protection is also on.

[[allowlists]]
description = "Dtype names, keyword arguments and kernel vocabulary"
targetRules = ["generic-api-key"]
regexes = [
# float8_e4m3fn, torch.float4_e2m1fn_x2, int64
'''^(?:torch\.)?(?:b?float|u?int)\d+(?:_[a-z0-9]+)*$''',
# vector_size=1, l2_scale_shift=5, flag=skip_corr_threshold; not a long token
'''^[a-z_][a-z0-9_]*=(?:[a-z0-9.]{1,4}|[a-z0-9]+(?:_[a-z0-9]+)+)$''',
]
# Substrings of kernel/backend names (cutlass_nvfp4, gemm2_dense_finalize_n256);
# words rather than name shapes, which random keys also match; none is all hex.
stopwords = [
"chunk", "coop", "cudnn", "cutlass", "dense", "finalize", "fmha",
"int32", "int64", "interleave", "mxfp", "mxint", "nopdl", "nvfp4", "partial",
"prefill", "quantize", "repeat", "residual", "routed", "segmented",
"softmax", "softplus", "stride", "swizzle", "threshold", "topk", "trtllm",
]

[[allowlists]]
description = "Lower-case *_sha256 digest fields in generated kernel metadata"
targetRules = ["generic-api-key"]
regexTarget = "line"
regexes = [
'''_sha256"?\s*[=:]''',
]
31 changes: 31 additions & 0 deletions .github/scan_tools_configs/trivy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Copyright Advanced Micro Devices, Inc.
# SPDX-License-Identifier: MIT
#
# trivy configuration
# Reference: https://trivy.dev/latest/docs/references/configuration/config-file/
# Local scan: trivy fs --config .github/scan_tools_configs/trivy.yml .
#
# Trivy configuration for this repository's security scans.
#
# This file defines the complete scanner policy;
# Keep repository-specific suppressions and tuning here.

# Environment and generated-output exclusions support local scans; these
# paths are absent from clean CI checkouts.
scan:
skip-dirs:
- .git
- .venv*
- venv*
- build
- dist
- node_modules
- __pycache__
# Upstream's CUDA images, never built here; fixing their root-user / apt
# findings would only add sync conflicts. docker/Dockerfile.rocm, the image
# this fork ships, stays scanned.
skip-files:
- docker/Dockerfile.ci
- docker/Dockerfile.flashinfer-ep-pytorch
- docker/Dockerfile.flashinfer-nvep
- docker/Dockerfile.vllm-flashinfer-ep
15 changes: 15 additions & 0 deletions .github/scan_tools_configs/zizmor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Copyright Advanced Micro Devices, Inc.
# SPDX-License-Identifier: MIT
#
# zizmor configuration
# Reference: https://docs.zizmor.sh/configuration/
# Local scan: zizmor --config .github/scan_tools_configs/zizmor.yml .
#
# Zizmor configuration for this repository's security scans.
#
# This file defines the complete scanner policy;
# Keep repository-specific suppressions and tuning here.
#
# Add rule-specific settings and suppressions under `rules:`.

rules: {}
Loading