Proposal: opt-in benchmarks/ harness for per-query scan throughput (ThreadPool + SIMD dispatch)

[cluster2600]

Proposal: opt-in benchmarks/ harness for zvec's own primitives

Before opening a PR I'd like to check whether you'd welcome a small,
opt-in benchmark harness under a new top-level benchmarks/ directory. I
have a branch ready and would only send it if this is something you want
upstream — happy to drop it otherwise.

What it is

A single micro-benchmark, distance_scan_bench, that measures flat-scan
throughput the way zvec actually scales it — and nothing more:

• parallelism is applied per query across ailego::ThreadPool;
• each individual query is a serial scan that calls zvec's
  SIMD-dispatched Distance::SquaredEuclidean (runtime CpuFeatures
  dispatch: SSE / AVX / AVX512 / NEON).

No OpenMP, no hand-written intrinsics, no kernel changes. It uses exactly the
two primitives zvec already relies on. This is deliberately aligned with the
architecture (ENABLE_OPENMP off, hot kernels hand-vectorized, throughput from
distributing queries over the pool while the single-query scan stays serial)
rather than cutting across it.

flowchart TD
    subgraph POOL["ailego::ThreadPool — parallel ACROSS queries"]
      direction LR
      q0["query 0<br/>serial scan"]:::par
      q1["query 1<br/>serial scan"]:::par
      qn["query N-1<br/>serial scan"]:::par
    end
    POOL --> K
    subgraph K["each scan: one query vs all DB vectors"]
      direction TB
      loop["for i in 0..num_db&nbsp;&nbsp;(serial)"]:::seq
      kern["Distance::SquaredEuclidean(db_i, q, dim)<br/>SIMD-dispatched: SSE / AVX / AVX512 / NEON"]:::simd
      loop --> kern
    end
    classDef par  fill:#1f7a1f,color:#fff,stroke:#0a0,stroke-width:2px
    classDef seq  fill:#2b2b2b,color:#fff,stroke:#888
    classDef simd fill:#1f3a7a,color:#fff,stroke:#46f,stroke-width:2px

Loading

The pool parallelizes the outer axis (queries); the SIMD kernel vectorizes the
inner axis (one pair's reduction). They compose and neither touches the other's
loop, so the parallel result is bit-identical to serial — each query writes a
disjoint output row and the per-pair reduction order is unchanged. The benchmark
self-validates this on every run (and is registered as a ctest), exiting
non-zero on any mismatch.

Zero impact when off

• New CMake option BUILD_BENCHMARKS, default OFF — the target does not
  exist unless explicitly enabled, so the default build and CI matrix are
  untouched.
• Links only zvec_ailego; adds no third-party dependency.

cmake -S . -B build -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target distance_scan_bench
./build/bin/distance_scan_bench            # default 4096 256 128
ctest -R distance_scan_bench               # self-validation

Measured (illustrative)

num_db=8192, num_query=256, dim=128, Apple Silicon (10 cores), kernel SIMD
path NEON, -O3, best-of-5; serial baseline 0.023 s:

Threads	Speedup	max_abs_diff vs serial
1	1.00×	0.000e+00
2	1.91×	0.000e+00
4	3.76×	0.000e+00
8	4.31×	0.000e+00

xychart-beta
    title "Per-query thread-pool scaling (NEON) — bar = measured, line = ideal"
    x-axis "Threads" [1, 2, 4, 8]
    y-axis "Speedup x" 0 --> 8
    bar [1.00, 1.91, 3.76, 4.31]
    line [1, 2, 4, 8]

Loading

Near-linear to the physical-core count, then flattens as the scan becomes
memory-bandwidth-bound. Inputs are synthetic and deterministic (no RNG);
correctness is established by bit-for-bit comparison against the serial
reference, not a data cohort.

Questions for maintainers

1. Is a top-level benchmarks/ directory welcome, or would you prefer a
   different location / naming?
2. Is the opt-in BUILD_BENCHMARKS=OFF model acceptable, or do you have an
   existing convention for performance tooling I should follow?
3. Any scope preferences (e.g. keep it to this one scan, or none at all)?

If this is of interest I'll open the PR. This follows the same
architecture-alignment approach as the rotator fix in #534.

---

The "parallelize queries, never the reduction" schedule was derived and proven
legal (no carried dependence) with a polyhedral auto-scheduling pass —
cluster_compilot, an
implementation of Agentic Auto-Scheduling (arXiv:2511.00592). The benchmark
implements that schedule with zvec's existing ThreadPool + SIMD-dispatched
kernels; it proposes no kernel change.

[JalinWang]

@cluster2600 Thanks for the interest and the detailed proposal! We'd definitely welcome a PR for this.

If I understand correctly, this is a micro-benchmark at the db layer level?

Regarding your three questions:

1. Location: We'd prefer placing it under tools/db/ — we already have a full-text search benchmark tool there. (If I misunderstood and this targets the core layer instead, then tools/core/ would be more appropriate.)

2. Build flag: You can just reuse the existing BUILD_TOOLS option rather than introducing a new BUILD_BENCHMARKS flag.

3. Scope: Let's see your implementation first, then we can discuss. We're also planning to add more micro-benchmarks on our side, so there's definitely room to grow.

One more thing: since this would live outside the tests/ directory, we'd suggest not registering it as a ctest target. The self-validation logic can stay, but keep it as a standalone executable.

Looking forward to the PR!

[cluster2600]

@JalinWang Implemented and pushed to #534 — answering your three points:

1. Location: tools/core/distance_bench.cc. It benchmarks the core FP32 Distance::SquaredEuclidean kernel, so it belongs at the core layer rather than tools/db/.
2. Build flag: reuses the existing BUILD_TOOLS flag — no new BUILD_BENCHMARKS flag added.
3. Not a test: it's a standalone executable, intentionally not registered as a ctest since the timings are machine-dependent. It does self-validate the SIMD result against a scalar reference and exits non-zero on mismatch.

It scans a synthetic DB with Distance::SquaredEuclidean vs a scalar reference across a range of dims and reports the speedup; flags --num_db, --dims, --reps, --tolerance. Happy to move or adjust scope if you'd prefer it elsewhere.