Optimize Windows ARM64 MSVC kernels and enhance cpuinfo ISA detection - #379
Open
pdeep854 wants to merge 1 commit into
Open
Optimize Windows ARM64 MSVC kernels and enhance cpuinfo ISA detection#379pdeep854 wants to merge 1 commit into
pdeep854 wants to merge 1 commit into
Conversation
Build system: - ruy/CMakeLists.txt: restrict /arch:AVX/AVX2/AVX512 flags to x86 MSVC targets; MSVC ARM64 kernels are compiled as part of kernel_arm64.cc (no separate file needed) Platform detection (ruy/platform.h): - Add _M_ARM64/_M_ARM detection alongside GCC __aarch64__/__arm__ macros - Add _M_ARM64 to NEON detection (mandatory on AArch64, MSVC omits __ARM_NEON__) - Add __builtin_expect no-op shim for MSVC (single canonical location) GAS assembly guards: - ruy/kernel_arm64.cc, kernel_arm32.cc: add !defined(_MSC_VER) to top-level #if guards to skip AT&T-style inline asm on MSVC - ruy/pack_arm.cc: same guard for all 5 asm volatile pack function blocks - ruy/kernel_arm.h, pack_arm.h: add !defined(_MSC_VER) to all asm-based function declarations and PackImpl/Kernel template specialization guards Kernel params (ruy/kernel_common.h): - Fix rhs_stride narrowing: add static_cast<int32_t> to suppress C4267 (size_t * int assigned to int32_t member) - Add MakeKernelParams8bitMixed(): variant of MakeKernelParams8bit for int16_t LHS (i16 x i8 case); stores lhs_base_ptr as reinterpret-cast int8_t* and lhs_stride in int16 elements (kernel multiplies by 2) MSVC ARM64 NEON intrinsic kernels (ruy/kernel_arm64.cc, appended): - KernelFloatNeon/A55ish/X1: 8x8 float32 GEMM using NEON FMA intrinsics - Kernel8bitNeon/A55ish/1Col: 4x4 int8 GEMM using vmull/vmlal intrinsics - Kernel8bitNeonDotprod/A55ish/X1/1Col: 8x8 int8 GEMM using vdotq_laneq_s32 - Pack8bitColMajorForNeon/A55ish: col-major int8 packing with vtrnq_s32 - Pack8bitColMajorForNeonDotprod/A55ish: col-major int8 packing for SDOT layout - Pack8bitRowMajorForNeonDotprod: row-major int8 packing for kNeonDotprod - PackFloatColMajorForNeon/A55ish: col-major float32 packing - Kernel8bitNeonMixedInt16Rhs/Lhs: new mixed-precision i8xi16 and i16xi8 kernels (tile 4x4, depth step 8); use RUY_MIX_MAC (vmull_s16 low + vmlal_high_s16 high) for 8 int16 products per accumulator step, with vmovl_s8 sign-extension for whichever operand is int8 Kernel dispatch (ruy/kernel_arm.h): - Add MSVC ARM64 Kernel<> specializations for kNeon and kNeonDotprod (int8 and float variants) inside #if defined(_MSC_VER) && defined(_M_ARM64) - Add mixed-precision Kernel<kNeon, i8, i16, i32, i16> and Kernel<kNeon, i16, i8, i32, i16> specializations (layout kColMajor/8/4); kNeonDotprod inherits both via RUY_INHERIT_KERNEL (no dotprod for int16) Pack dispatch (ruy/pack_arm.h): - Add MSVC ARM64 PackImpl<> specializations calling intrinsic pack functions - Add mixed-precision PackImpl<kNeon, kColMajor/8/4, int16_t, int16_t> and PackImpl<kNeon, kColMajor/8/4, int8_t, int8_t> specializations; int16 variant packs 4 cols x 8 depths = 64 bytes/block via vtrnq_s16/vtrnq_s32; int8 variant packs 4 cols x 8 depths = 32 bytes/block via vtrnq_s8/vtrn1q_s32 Test infrastructure (ruy/test.h): - Use VirtualAlloc on Windows to give each SeparateMappingAllocator allocation a fresh virtual address, preventing stale PrepackedCache hits (Windows new[] reuses addresses; cache key is the source data pointer) cpuinfo submodule: - Bump from 082deff to upstream main 66ee79c, which includes full Windows ARM64 ISA detection (SVE/SVE2, fp16, dotprod, bf16, i8mm), chip registry lookup, init-by-logical-sys-info fallback, and fp16arith fallback path
pdeep854
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimize
ruyto build and run natively on Windows ARM64 under MSVC. The existing GAS inline assembly kernels are skipped on MSVC via!defined(_MSC_VER)guards; a full set of replacement kernels using NEON intrinsics (<arm_neon.h>) is appended to kernel_arm64.cc inside an#if defined(_MSC_VER) && defined(_M_ARM64)block. In addition to the standard float and int8 paths, this PR adds two new mixed-precision kernels (i8×i16andi16×i8 → i16) that have no upstream equivalent.This PR also bumps the
cpuinfosubmodule to upstream commit66ee79c, adding full Windows ARM64 ISA detection support, including SVE/SVE2, FP16, DotProd, BF16, and I8MM capabilities, along with updated chip registry support.All 33 existing tests pass; all 8 benchmark configurations show competitive
kNeonandkNeonDotprodthroughput.Benchmark Environment
Benchmark measurements were collected on the following system:
Benchmark Results
Float GEMM
Int8 Quantized GEMM
i8×i8→i32→i8
i8×i8→i32→i32
Unsigned Quantized GEMM
u8×u8→i32→u8
u8×u8→i32→i16
Mixed Precision GEMM (New Kernels)
i8×i16→i32→i16
i16×i8→i32→i16