CUDAARCHS should be NATIVE when building locally

[etseidl]

I'm doing a devcontainer build without GitHub auth. In ~coder/.bashrc SCCACHE_BUCKET and SCCACE_REGION are unset, but CUDAARCHS is left set as "RAPIDS" causing my build to target all architectures. I think CUDAARCHS should either be unset or set to "NATIVE" in this case.

[vyasr]

I'd be in favor of this. There's no need to build all architectures by default IMO.

[trxcllnt]

Yes there is. When you change the list of archs, the sccache key changes. Since most devs have different GPUs than CI/other devs, the likelihood the build cache is populated for your flavor of compile flags/git commit/gpu versions etc. becomes less likely.

This issue is about setting CUDAARCHS to native when the SCCACHE_BUCKET is unset. This should be fine, if a little dirty since it means setting an envvar in one feature to benefit a different feature.

[trxcllnt]

@etseidl you could also set the envvar to native here. Don't unset it, because the default is all-major if unset.

[etseidl]

@etseidl you could also set the envvar to native here. Don't unset it, because the default is all-major if unset.

@trxcllnt Yes, I figured that out after I filed this. But it's still a little dicey since I'd have to remember to not commit that change 😄.

Yes there is. When you change the list of archs, the sccache key changes. Since most devs have different GPUs than CI/other devs, the likelihood the build cache is populated for your flavor of compile flags/git commit/gpu versions etc. becomes less likely.

This is true for NVIDIA devs, but not for plebes like me. I was thinking that since there's already special handling for the sccache envvars, wherever that logic sits it could also modify CUDAARCHS. Right now I just change my bashrc to set CUDAARCHS, so I've got a fix that works for me. My main concern now is just that it took me a pretty long time to figure out that CUDAARCHS even existed 😅

[vyasr]

Yes there is. When you change the list of archs, the sccache key changes. Since most devs have different GPUs than CI/other devs, the likelihood the build cache is populated for your flavor of compile flags/git commit/gpu versions etc. becomes less likely.

That's true, there are certainly some tradeoffs there. It probably varies on a case-by-case basis whether or not your sccache hit rate is going to be good enough to be worth the extra compilation you do locally once you start changing things if you're only compiling for the native architecture.

[trxcllnt]

It probably varies on a case-by-case basis whether or not your sccache hit rate is going to be good enough to be worth the extra compilation you do locally once you start changing things if you're only compiling for the native architecture.

It's not just about your hit rate, you're also populating the cache for CI/other devs :-)

[jrhemstad]

Yes there is. When you change the list of archs, the sccache key changes. Since most devs have different GPUs than CI/other devs, the likelihood the build cache is populated for your flavor of compile flags/git commit/gpu versions etc. becomes less likely.

100% agreed. If a user is using sccache, then they should be building for all archs by default.

I agree with @etseidl that if you're not using sccache than using native makes sense.

[vyasr]

I wasn't thinking about the fact that sccache doesn't cache granularly enough to share artifacts between users who compile for all archs and those who compiles for a specific subset of them. Agreed, compiling for all archs makes sense if you have access to the shared cache.