[HWToLLVM][ArcToLLVM] Spill array values early

[fzi-hielscher]

Spill hw.array values on the stack at their definition site instead of their use sites to reduce the number of redundant alloca + store ops created. This should mitigate the negative effects of #9172, fixing #9211.

This PR adds the HWToLLVMArraySpillCache helper to the HWToLLVM conversion patterns. Certain operations on !hw.array typed values need to "spill" (i.e., copy into a buffer, usually stack allocated) their input arrays in order to access dynamically indexed elements. The cache is used to avoid spilling the array for each such user separately. Instead,operations that create an array value which needs spilling allocate a buffer that can be shared by the value's users. HW Dialect operations that inherently allocate a buffer for their output arrays (AggregateConstantOp, ArrayInjectOp, ArraySliceOp) use the cache to associate their output value with a pointer to the buffer, so it does not need to be spilled again. Non-HW Dialect producers of HW array values (e.g., arc.state_load) need to be handled pre conversion. This is done by invoking spillNonHWOps.

Use of the cache is optional and can be disabled by passing spill-arrays-early=false to convert-hw-to-llvm or using a nullopt value for the cache argument of populateHWToLLVMConversionPatterns. In fact, conversions passes that do not disable allowPatternRollback must not use the cache since a buffer added to the cache once cannot be removed in case of a rollback.

[fzi-hielscher]

@SimonEbner, could you check if this actually fixes the compile time regression in your example?

[SimonEbner]

Thank you. I tested 120dd53 vs your head:
120dd53: 12.59s, HEAD: 1.86s

[fzi-hielscher]

Thanks for checking.
I suspect the remaining overhead is due to arc.state_load ops now being spilled instead of accessing the heap buffer directly. We could probably implement some heuristic allowing array accesses without spilling if we can be certain that the underlying memory remains unchanged between the arc.state_load and the hw.array_get op. But at the moment I'd rather not add that complexity if it is only to reduce the load on the LLVM optimizations.

[SimonEbner]

Sure, I think this patch is great, more than good enough. Thanks for looking into this

[maerhart]

That's great! Thanks!

[maerhart]

Nit: as this occurs a few times, we could factor it out into a helper function.