[WIP] feat: enable pip install primus_turbo with wheel publishing#239
[WIP] feat: enable pip install primus_turbo with wheel publishing#239xiaobochen-amd wants to merge 4 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates Primus-Turbo’s Python packaging to better support modern build flows by introducing a pyproject.toml/MANIFEST.in and aligning documented + CI install commands around framework-specific extras.
Changes:
- Add PEP 517/518 build metadata (
pyproject.toml) and an sdist manifest (MANIFEST.in). - Introduce framework extras in
setup.pyand adjust dependency placement (e.g., moving Triton intoinstall_requires). - Update README + GitHub workflows to install with extras (
.[pytorch]/.[jax]) and document wheel/sdist behaviors.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| setup.py | Adds repo-root path injection, defines extras_require, and updates runtime/install requirements. |
| requirements.txt | Removes triton==3.5.1 from the dev requirements list. |
| pyproject.toml | Adds build-system requirements and enables setuptools PEP517 builds. |
| README.md | Updates installation commands to use extras; adds packaging guidance and sdist fallback instructions. |
| MANIFEST.in | Defines files to include in sdists (tools, sources, headers, etc.). |
| .github/workflows/ci.yaml | Updates editable installs to use framework extras in CI. |
| .github/workflows/benchmark.yaml | Updates editable installs to use the PyTorch extra for benchmarks. |
Comments suppressed due to low confidence (1)
setup.py:404
is_package_installed()usesimportlib.util.find_spec(), which checks importable module names; passing a distribution name with a hyphen ("amd-aiter") will never be found. This likely causes the build to always append the VCSamd-aiterdependency even when the package is already installed. Consider checking for theaitermodule instead, or useimportlib.metadata.distribution("amd-aiter")to test the installed distribution.
"triton==3.5.1",
]
# Conditionally add aiter if torch_ext is being built and aiter is not already installed
if torch_ext is not None and not is_package_installed("amd-aiter"):
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
102ec90 to
ed8ddbf
Compare
ed8ddbf to
2eb9058
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #### Install from PyPI (Recommended) | ||
|
|
||
| ```bash | ||
| # Prerequisite for --no-build-isolation builds | ||
| pip3 install "hipify_torch @ git+https://github.com/ROCm/hipify_torch.git" | ||
|
|
||
| # PyTorch backend | ||
| pip3 install --no-build-isolation "primus-turbo[pytorch]" | ||
| ``` |
There was a problem hiding this comment.
The “Install from PyPI (Recommended)” section currently instructs installing hipify_torch and using --no-build-isolation. If a compatible wheel is available (the goal of this PR), neither should be required and this may confuse users. Consider splitting the instructions into (1) simple wheel install (pip install primus-turbo[pytorch]) and (2) explicit source-build fallback steps (including hipify_torch and --no-build-isolation).
2eb9058 to
cbd6e7b
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
setup.py:430
is_package_installed("amd-aiter")will always returnFalsebecauseimportlib.util.find_spec()expects an importable module name, and hyphenated distribution names are not valid module names. This makes theamd-aiter @ git+...requirement get baked intoinstall_requiresfor most wheel builds, forcing all installs to pull from Git (often undesirable/blocked) even whenaiteris already available. Consider checking for the actual importable module (likelyaiter) or usingimportlib.metadatato detect the distribution, and avoid dynamic Git URL injection ininstall_requires(e.g., move to an extra likeprimus-turbo[aiter]or document it as an optional dependency).
# Conditionally add aiter if torch_ext is being built and aiter is not already installed
if torch_ext is not None and not is_package_installed("amd-aiter"):
print("[Primus-Turbo Setup] amd-aiter not found, will be installed automatically.")
install_requires.append(f"amd-aiter @ git+https://github.com/ROCm/aiter.git@{AITER_COMMIT}")
else:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def write_build_info(): | ||
| build_info_path = PROJECT_ROOT / "primus_turbo" / "_build_info.py" | ||
| commit = get_commit() | ||
| if commit == "unknown" and build_info_path.exists(): | ||
| return | ||
| build_time = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") |
There was a problem hiding this comment.
write_build_info() embeds the current wall-clock time into primus_turbo/_build_info.py, which makes wheel/sdist builds non-reproducible and will rewrite the file every build invocation (e.g., metadata generation vs wheel build). Consider honoring SOURCE_DATE_EPOCH (or an env toggle) and/or only recording build time when explicitly requested, so published artifacts can be deterministic.
| def write_build_info(): | |
| build_info_path = PROJECT_ROOT / "primus_turbo" / "_build_info.py" | |
| commit = get_commit() | |
| if commit == "unknown" and build_info_path.exists(): | |
| return | |
| build_time = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") | |
| def _get_build_time() -> str: | |
| """ | |
| Return a deterministic build time string. | |
| If SOURCE_DATE_EPOCH is set, use it as a Unix timestamp to support | |
| reproducible builds; otherwise, fall back to the current UTC time. | |
| """ | |
| source_date_epoch = os.environ.get("SOURCE_DATE_EPOCH") | |
| if source_date_epoch is not None: | |
| try: | |
| timestamp = int(source_date_epoch) | |
| dt = datetime.fromtimestamp(timestamp, tz=timezone.utc) | |
| except (ValueError, OSError, OverflowError): | |
| dt = datetime.now(timezone.utc) | |
| else: | |
| dt = datetime.now(timezone.utc) | |
| return dt.replace(microsecond=0).isoformat().replace("+00:00", "Z") | |
| def write_build_info(): | |
| build_info_path = PROJECT_ROOT / "primus_turbo" / "_build_info.py" | |
| commit = get_commit() | |
| if commit == "unknown" and build_info_path.exists(): | |
| return | |
| build_time = _get_build_time() |
| name="primus-turbo", | ||
| use_scm_version=True, |
There was a problem hiding this comment.
PR description mentions enabling pip install primus_turbo[...], but the distribution name here is set to primus-turbo. If the intent is to keep the published package name aligned with the documented install command/legacy name, please confirm and make the PR description/docs consistent (or revert this if the PyPI project name must remain primus_turbo).
Add pyproject.toml, MANIFEST.in, and extras_require to support pip install primus_turbo[pytorch] / [jax].
Align README and CI install commands. Build and publish wheels to enable direct pip install primus_turbo.