Skip to content

Conversation

@a10y
Copy link
Contributor

@a10y a10y commented Jan 12, 2026

Fleshes out the BufferHandle type more.

I hid the inner enum and introduced some extra methods to build/unwrap BufferHandles.

This removes the PrimitiveArray::as_slice method, replacing it with an into_buffer::<T> method which may allocate and copy a new buffer (if the handle points to device memory), or else provides a new Buffer pointing to the existing host memory.

This PR is large but a lot of it is just adding &'s and updating test code to use the buffer! macro

@a10y a10y force-pushed the bufferhandles branch 3 times, most recently from 67fabe3 to 5d5797b Compare January 12, 2026 23:26
@a10y a10y added the chore Release label indicating a trivial change label Jan 12, 2026
@a10y a10y marked this pull request as ready for review January 12, 2026 23:32
@codspeed-hq
Copy link

codspeed-hq bot commented Jan 12, 2026

Merging this PR will degrade performance by 66.22%

⚡ 83 improved benchmarks
❌ 55 regressed benchmarks
✅ 1116 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
bench_compare_primitive[(100000, 128)] 852.4 µs 741.6 µs +14.94%
bench_compare_primitive[(100000, 4)] 850.1 µs 739.6 µs +14.94%
bench_compare_primitive[(100000, 32)] 850.8 µs 740.2 µs +14.94%
bench_compare_primitive[(100000, 2)] 849.6 µs 738.9 µs +14.98%
bench_compare_primitive[(100000, 2048)] 929.1 µs 818.7 µs +13.49%
bench_compare_primitive[(100000, 512)] 905.7 µs 794.4 µs +14.01%
bench_compare_sliced_dict_primitive[(2500, 10000)] 91 µs 78 µs +16.61%
bench_compare_sliced_dict_primitive[(20000, 10000)] 317.7 µs 273.1 µs +16.31%
bench_compare_sliced_dict_primitive[(10000, 10000)] 209.3 µs 176.7 µs +18.43%
bench_compare_primitive[(100000, 8)] 849.5 µs 739.2 µs +14.92%
bench_compare_sliced_dict_primitive[(7500, 10000)] 189.5 µs 156.6 µs +21.06%
bench_compare_sliced_dict_primitive[(3333, 10000)] 103.1 µs 85.5 µs +20.59%
bench_compare_sliced_dict_primitive[(5000, 10000)] 126 µs 100.2 µs +25.78%
bench_compare_sliced_dict_primitive[(9999, 10000)] 209.7 µs 176.7 µs +18.69%
bench_compare_varbin[(100000, 128)] 864.8 µs 754.5 µs +14.61%
bench_compare_varbin[(100000, 2048)] 1,027.1 µs 916.7 µs +12.05%
bench_compare_varbin[(100000, 512)] 935.6 µs 824.9 µs +13.42%
bench_compare_varbin[(100000, 2)] 857.2 µs 746.4 µs +14.84%
bench_compare_varbin[(100000, 4)] 857 µs 747 µs +14.72%
bench_compare_varbin[(100000, 8)] 857.8 µs 747.3 µs +14.79%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing bufferhandles (3f5e8b2) with develop (d757be1)

Open in CodSpeed

@joseph-isaacs
Copy link
Contributor

I wonder if we can remove at lot of these buffer calls from tests?

@a10y
Copy link
Contributor Author

a10y commented Jan 13, 2026

I don't think there's another way to get the values out now

@a10y a10y force-pushed the bufferhandles branch 3 times, most recently from cb6f4b0 to 9e2ba3b Compare January 13, 2026 22:04
@codecov
Copy link

codecov bot commented Jan 13, 2026

Codecov Report

❌ Patch coverage is 77.20280% with 163 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.45%. Comparing base (5cd8b29) to head (3f5e8b2).
⚠️ Report is 4 commits behind head on develop.

Files with missing lines Patch % Lines
vortex-array/src/buffer.rs 38.46% 40 Missing ⚠️
vortex-array/src/arrays/varbin/compute/take.rs 0.00% 16 Missing ⚠️
vortex-array/src/patches.rs 81.81% 10 Missing ⚠️
encodings/alp/src/alp/decompress.rs 16.66% 5 Missing ⚠️
vortex-array/src/array/visitor.rs 0.00% 4 Missing ⚠️
vortex-array/src/arrays/listview/tests/filter.rs 0.00% 4 Missing ⚠️
vortex-array/src/arrays/listview/tests/take.rs 0.00% 4 Missing ⚠️
vortex-array/src/arrays/masked/execute.rs 0.00% 4 Missing ⚠️
vortex-array/src/arrays/primitive/array/mod.rs 85.18% 4 Missing ⚠️
...ex-array/src/arrays/primitive/compute/take/avx2.rs 0.00% 4 Missing ⚠️
... and 37 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

a10y added 2 commits January 13, 2026 21:06
Signed-off-by: Andrew Duffy <[email protected]>
Signed-off-by: Andrew Duffy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Release label indicating a trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants