Repository navigation

Metal: queued rmsq, swq, gdn and atq kernels and a 4-row q4mv, so a decode token is one command buffer with one wait - #1440

Closed

totaland wants to merge 6 commits into

bendlang:mainfrom

totaland:q4-token

Closed

Metal: queued rmsq, swq, gdn and atq kernels and a 4-row q4mv, so a decode token is one command buffer with one wait#1440
totaland wants to merge 6 commits into
bendlang:mainfrom
totaland:q4-token

Commits on Oct 9, 2026
- Array.q4mv: an affine 4-bit matvec primitive, eight rows per pass on CPU and one kernel thread per row on Metal, bit-identical to the Bend body
  James Nguyen
  committed
  8409b3b
- Array.q4mv: one compact CPU loop, a test on every lane, and the JS lane falls back to the body
  James Nguyen
  committed
  507c552
- Array.q4mv hands a product of 4M weights or more to a Metal kernel when a Metal heap is live, one thread per row, bit-identical to the Bend body
  James Nguyen
  committed
  0aef58a
- Array.q4mv sums each row in a fixed tree order, so a NEON 4-row loop and one Metal SIMD group per row give the Bend body's bits on every lane
  James Nguyen
  committed
  7d59003
- Array.q4go and Array.q4wait queue Metal Q4 matvecs in one command buffer and wait once, on a flag kernel
  James Nguyen
  committed
  49848e0
- Array.rmsq, swq, gdn and atq queue on Metal beside q4go, and q4mv does 4 rows per SIMD group, so a model's decode token can be one command buffer with one wait
  James Nguyen
  committed
  dd51ee0