Skip to content

Conversation

@johannes-graner
Copy link
Contributor

Proposed changes

This PR migrates CK profilers from CPU-side to GPU-side tensor initialization, eliminating a performance bottleneck. The convolution profilers now use the DeviceMemory API methods for on-device data generation, removing expensive CPU-GPU transfers.

Profiler Before After Speedup
Grouped Conv Forward 45.0 s 16.0 s 2.8x
Grouped Conv Backward Data 15.0 s 8.8 s 1.7x
Grouped Conv Backward Weight 20.0 s 6.9 s 2.9x

This saves in total ~50 s per CI run.

Checklist

  • I have added tests relevant to the introduced functionality, and the unit tests are passing locally
  • I have added the test to REGRESSION_TESTS list defined at the top of CMakeLists.txt in tests/CMakeLists.txt, IF the test takes more than 30 seconds to run.
  • I have added inline documentation which enables the maintainers with understanding the motivation
  • I have removed the stale documentation which is no longer relevant after this pull request
  • (If this change is user-facing) I have added release notes which provide the end users with a brief summary of the improvement from this pull request
  • I have run clang-format on all changed files
  • Any dependent changes have been merged

Copy link
Collaborator

@AviralGoelAMD AviralGoelAMD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement!

@johannes-graner johannes-graner merged commit 3ccb15e into develop Jan 14, 2026
21 checks passed
@johannes-graner johannes-graner deleted the jograner/gpu-tensor-init branch January 14, 2026 15:04
shumway pushed a commit that referenced this pull request Jan 15, 2026
* Initialize tensors on GPU in CK profiler

* Kick CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants