Skip to content

fix(ci): force python3 symlink in Velox deps image - #417

Closed
Avinash-Raj wants to merge 1 commit into
mainfrom
fix/velox-deps-python-symlink
Closed

fix(ci): force python3 symlink in Velox deps image#417
Avinash-Raj wants to merge 1 commit into
mainfrom
fix/velox-deps-python-symlink

Conversation

@Avinash-Raj

@Avinash-Raj Avinash-Raj commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Patch upstream centos-multi.dockerfile after checkout so ln -s $(which python3) /usr/bin/python becomes ln -sfn $(which python3) /usr/bin/python.
  • Unblocks every velox-deps matrix cell; without a deps image, velox-build, tests, and benchmarks never start.

Motivation

Nightly velox-nightly failed on all four deps jobs (cuda12.9/13.1 x amd64/arm64):

https://github.com/rapidsai/velox-testing/actions/runs/34085672229

RUN ln -s $(which python3) /usr/bin/python
ln: failed to create symbolic link '/usr/bin/python': File exists

That line is in facebookincubator/velox scripts/docker/centos-multi.dockerfile after install_build_prerequisites && install_velox_deps_from_dnf. On current CentOS Stream 9, /usr/bin/python can already exist (the RPM that owns it is python-unversioned-command, not python3-pip / python3-devel). Plain ln -s then fails.

ln -sfn creates the python -> python3 link when missing and replaces it when present. Same pattern as the existing ARM_BUILD_TARGET checkout patch. Drop this step once upstream makes the symlink idempotent.

Test plan

  • velox-deps succeeds for cuda12.9/13.1 on amd64 and arm64
  • Confirm the build log shows ln -sfn $(which python3) /usr/bin/python rather than ln -s
  • Downstream velox-build is no longer skipped

@Avinash-Raj
Avinash-Raj requested a review from a team as a code owner September 7, 2026 14:32
@Avinash-Raj
Avinash-Raj requested a review from devavret September 7, 2026 14:32
run: |
sed -i 's/ARM_BUILD_TARGET:-"local"/ARM_BUILD_TARGET:-"generic"/' velox/scripts/setup-helper-functions.sh

# TODO: Remove once facebookincubator/velox makes the python symlink idempotent.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare this to prestodb/presto#28434, it seems simpler.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but we make sure that the base image symlinks python3 to /usr/bin/python

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified that the base image contain the /usr/bin/python symlink by default and it points to the python3 -> python3.9

$ docker run --rm quay.io/centos/centos:stream9 ls -la /usr/bin/python
lrwxrwxrwx 1 root root 9 Aug 14 13:30 /usr/bin/python -> ./python3

@Avinash-Raj

Copy link
Copy Markdown
Contributor Author

Closing this since the fix was already landed on upstream https://github.com/facebookincubator/velox/blob/main/scripts/docker/centos-multi.dockerfile#L110

@Avinash-Raj Avinash-Raj closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants