Skip to content

Conversation

@markovejnovic
Copy link
Collaborator

@markovejnovic markovejnovic commented Nov 6, 2025

What does this PR do?

Throws more jobs at cmake than cmake thinks it should use. Since cmake is oblivious to the fact that it's very likely going to hit a 100% hit rate, we need to throw more cores at it so that all the cache requests are made in parallel.

How did you verify your code works?

I ran CI with different job multipliers and compared the performance of each:

Job Multiplier x64 mac aarch64 mac x64-asan x64-musl x64-baseline win-x64-baseline aarch64-musl win-x64 x64 aarch64 x64-musl-baseline
1 7m21s 2m48s FTC 5m31s FTC 4m55s 4m58s 5m0s FTC FTC 5m41s
2 12m53s 2m46s FTC 4m57s FTC 4m45s 4m54s 5m1s FTC FTC 5m38s
4 12m48s 3m43s FTC 5m30s FTC 4m56s 4m46s 4m42s FTC FTC 5m14s
8 6m50s 2m47s FTC 4m56s FTC 4m46s 4m33s 4m43s FTC FTC 5m25s
16 8m51s 4m7s FTC 5m3s FTC crash 4m51s crash FTC FTC 5m2s
32 7m51s 2m50s FTC 5m33s FTC crash 4m55s crash FTC FTC 5m26s

FTC - Fail-to-cache https://linear.app/oven/issue/ENG-21502/x64-and-aarch64-linux-targets-failing-to-use-sccache
At high job multipliers, windows crashed due to high contention on the sccache lock-file in S3. I don't know why that happens.

job_multiplier_final

Based on this plot, there is no appreciable difference in parallelizing the job, unfortunately

@linear
Copy link

linear bot commented Nov 6, 2025

@robobun
Copy link
Collaborator

robobun commented Nov 6, 2025

Updated 11:47 AM PT - Nov 11th, 2025

@markovejnovic, your commit 242917c has 5 failures in Build #31516 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 24452

That installs a local version of the PR into your bun-24452 executable, so you can run:

bun-24452 --bun

@markovejnovic markovejnovic force-pushed the user/markovejnovic/ENG-21467-more-build-cpp-jobs branch from 583c359 to 7c77d4b Compare November 6, 2025 23:00
@markovejnovic markovejnovic force-pushed the user/markovejnovic/ENG-21471 branch 6 times, most recently from f6ec64d to 0f4a337 Compare November 7, 2025 00:07
Base automatically changed from user/markovejnovic/ENG-21471 to main November 7, 2025 01:26
@markovejnovic markovejnovic marked this pull request as draft November 7, 2025 19:33
@markovejnovic markovejnovic force-pushed the user/markovejnovic/ENG-21467-more-build-cpp-jobs branch 7 times, most recently from 384b1d1 to 9601585 Compare November 11, 2025 17:56
@markovejnovic markovejnovic force-pushed the user/markovejnovic/ENG-21467-more-build-cpp-jobs branch from 9601585 to 242917c Compare November 11, 2025 18:05
@markovejnovic
Copy link
Collaborator Author

As per the discussion in the PR, there is no appreciable difference with parallelization so I think it safe to reject this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants