[Issue]: AttributeError: module 'torch._C' has no attribute '_PYBIND11_COMPILER_TYPE'

[fxmarty-amd]

Problem Description

Hi,

The following line fails

aiter/aiter/jit/utils/cpp_extension.py

Line 467 in 6d0beab

val = getattr(torch._C, f"_PYBIND11_{name}")

when using vllm + aiter + gpt-oss, see below. You can also reproduce with: getattr(torch._C, "_PYBIND11_COMPILER_TYPE" using latest pytorch+rocm release at https://pytorch.org/get-started/locally/

(base) root@felix-marty-job-torch-vllm-1-rmfqs:/shared_volume/repos# export VLLM_ROCM_USE_AITER=1
export VLLM_USE_AITER_UNIFIED_ATTENTION=1
export VLLM_ROCM_USE_AITER_MHA=0
(base) root@felix-marty-job-torch-vllm-1-rmfqs:/shared_volume/repos# ^C
(base) root@felix-marty-job-torch-vllm-1-rmfqs:/shared_volume/repos# vllm serve /models/openai_gpt-oss-20b --tensor-parallel
-size 1 --enforce-eager
(APIServer pid=93425) INFO 11-12 15:30:30 [api_server.py:1897] vLLM API server version 0.11.1rc7.dev67+g1761dea1a
(APIServer pid=93425) INFO 11-12 15:30:30 [utils.py:253] non-default args: {'model_tag': '/models/openai_gpt-oss-20b', 'model': '/models/openai_gpt-oss-20b', 'enforce_eager': True}
(APIServer pid=93425) INFO 11-12 15:30:30 [model.py:630] Resolved architecture: GptOssForCausalLM
(APIServer pid=93425) ERROR 11-12 15:30:30 [config.py:306] Error retrieving safetensors: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/models/openai_gpt-oss-20b'. Use `repo_type` argument if needed., retrying 1 of 2
(APIServer pid=93425) ERROR 11-12 15:30:32 [config.py:304] Error retrieving safetensors: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/models/openai_gpt-oss-20b'. Use `repo_type` argument if needed.
(APIServer pid=93425) INFO 11-12 15:30:32 [model.py:1951] Downcasting torch.float32 to torch.bfloat16.
(APIServer pid=93425) INFO 11-12 15:30:32 [model.py:1728] Using max model len 131072
(APIServer pid=93425) [aiter] start build [module_aiter_enum] under /shared_volume/repos/aiter/aiter/jit/build/module_aiter_enum
(APIServer pid=93425) [2025-11-12 15:31:03] INFO core.py:270: start build [module_aiter_enum] under /shared_volume/repos/aiter/aiter/jit/build/module_aiter_enum
(APIServer pid=93425) Successfully preprocessed all matching files.
(APIServer pid=93425) [aiter] failed build jit [module_aiter_enum]↓↓↓↓↓↓↓↓↓↓
(APIServer pid=93425) -->[History]: Traceback (most recent call last):
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 513, in wrapper
(APIServer pid=93425)     module = get_module(md)
(APIServer pid=93425)              ^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 221, in get_module
(APIServer pid=93425)     __mds[md_name] = importlib.import_module(f"{__package__}.{md_name}")
(APIServer pid=93425)                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/root/miniforge3/lib/python3.12/importlib/__init__.py", line 90, in import_module
(APIServer pid=93425)     return _bootstrap._gcd_import(name[level:], package, level)
(APIServer pid=93425)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
(APIServer pid=93425) -->ModuleNotFounderror: No module named 'aiter.jit.module_aiter_enum'
(APIServer pid=93425) -->
(APIServer pid=93425) During handling of the above exception, another exception occurred:
(APIServer pid=93425)
(APIServer pid=93425) -->Traceback (most recent call last):
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 363, in MainFunc
(APIServer pid=93425)     _jit_compile(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1223, in _jit_compile
(APIServer pid=93425)     _write_ninja_file_and_build_library(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1333, in _write_ninja_file_and_build_library
(APIServer pid=93425)     _write_ninja_file_to_build_library(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1547, in _write_ninja_file_to_build_library
(APIServer pid=93425)     common_cflags += [f"{x}" for x in _get_pybind11_abi_build_flags()]
(APIServer pid=93425)                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1086, in _get_pybind11_abi_build_flags
(APIServer pid=93425)     pval = getattr(torch._C, f"_PYBIND11_{pname}")
(APIServer pid=93425)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->Attributeerror: module 'torch._C' has no attribute '_PYBIND11_COMPILER_TYPE'
(APIServer pid=93425) failed build jit [module_aiter_enum]↑↑↑↑↑↑↑↑↑↑
(APIServer pid=93425) [2025-11-12 15:31:14] ERROR core.py:386: failed build jit [module_aiter_enum]↓↓↓↓↓↓↓↓↓↓
(APIServer pid=93425) -->[History]: Traceback (most recent call last):
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 513, in wrapper
(APIServer pid=93425)     module = get_module(md)
(APIServer pid=93425)              ^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 221, in get_module
(APIServer pid=93425)     __mds[md_name] = importlib.import_module(f"{__package__}.{md_name}")
(APIServer pid=93425)                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/root/miniforge3/lib/python3.12/importlib/__init__.py", line 90, in import_module
(APIServer pid=93425)     return _bootstrap._gcd_import(name[level:], package, level)
(APIServer pid=93425)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
(APIServer pid=93425) -->  File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
(APIServer pid=93425) -->ModuleNotFounderror: No module named 'aiter.jit.module_aiter_enum'
(APIServer pid=93425) -->
(APIServer pid=93425) During handling of the above exception, another exception occurred:
(APIServer pid=93425)
(APIServer pid=93425) -->Traceback (most recent call last):
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/core.py", line 363, in MainFunc
(APIServer pid=93425)     _jit_compile(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1223, in _jit_compile
(APIServer pid=93425)     _write_ninja_file_and_build_library(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1333, in _write_ninja_file_and_build_library
(APIServer pid=93425)     _write_ninja_file_to_build_library(
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1547, in _write_ninja_file_to_build_library
(APIServer pid=93425)     common_cflags += [f"{x}" for x in _get_pybind11_abi_build_flags()]
(APIServer pid=93425)                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->  File "/shared_volume/repos/aiter/aiter/jit/utils/cpp_extension.py", line 1086, in _get_pybind11_abi_build_flags
(APIServer pid=93425)     pval = getattr(torch._C, f"_PYBIND11_{pname}")
(APIServer pid=93425)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=93425) -->Attributeerror: module 'torch._C' has no attribute '_PYBIND11_COMPILER_TYPE'
(APIServer pid=93425) failed build jit [module_aiter_enum]↑↑↑↑↑↑↑↑↑↑
(APIServer pid=93425) [aiter] finish build [module_aiter_enum], cost 41.49313282s
(APIServer pid=93425) [2025-11-12 15:31:14] INFO core.py:399: finish build [module_aiter_enum], cost 41.49313282s

using the recommended AITER_BRANCH: https://github.com/vllm-project/vllm/blob/54aecd9ed55d65d1f7be9d14e4e82751663438c8/docker/Dockerfile.rocm_base#L10-L11

Operating System

Ubuntu 24.04

CPU

AMD EPYC 9575F 64-Core Processor

GPU

AMD Instinct MI325X

ROCm Version

6.4.4

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

[valarLip]

you are using old ater version? latest one should had this supported

[fxmarty-amd]

I am using aiter commit recommended by https://github.com/vllm-project/vllm/blob/54aecd9ed55d65d1f7be9d14e4e82751663438c8/docker/Dockerfile.rocm_base#L10-L11 & torch==2.9.0+rocm6.4

[dllehr-amd]

@fxmarty-amd are you using the pytorch in that base image as well?

Essentially torch removed the pybind11. Aiter eventually moves the pybind requirement to 3.0.1 when they address this issue after pytorch moves. But I thought the combination of pytorch and aiter in that dockerfile was correct, and the torch in there still had pybind flags.

Can you just confirm real quick for me?

[fxmarty-amd]

@dllehr-amd Thank you! No, I am not using pytorch pulled from ARG PYTORCH_REPO="https://github.com/ROCm/pytorch.git", I am using pre-compiled upstream pytorch from https://pytorch.org/get-started/locally/