bf16 primitive type#3983
Conversation
| ### Literal Suffix | ||
|
|
||
| For a literal suffix `0bf16` may clash with the lexer's integer suffix `0b` for binary numbers thus; | ||
|
|
||
| ```rust | ||
| let a = 0bf16; | ||
| ``` | ||
|
|
||
| Would need to be written; | ||
| ```rust | ||
| let a = 0.0bf16; | ||
| ``` |
There was a problem hiding this comment.
It could also be disambiguated as 0_bf16.
Is the "unresolved" thing about whether to support the literal 0bf16? There is no other ways this conflict may arise. Since this is just a single special case IMO it is not harmful to also support this.
There was a problem hiding this comment.
To be honest, I had no idea that 0f64 and similar worked by itself. I would have expected the decimal part to be required always, as it is when the type isn't specified.
This feels particularly dubious considering how 0x1f64 is acceptable, but refers to the literal 8036 and not 1.0f64; 0b1f64 rightfully errors. Personally, I would just say maybe a good option to consider is just always requiring the decimal part for float literals just to avoid the need to disambiguate.
Or, at least, in this particular case, allow both 0_bf16 and 0.0bf16 but always lint float literals that don't have a decimal part, due to the dubiousness.
There was a problem hiding this comment.
I think 0_bf16 is clear enough to say it is a "0" of type bf16. Use lint clippy::unseparated_literal_suffix against 0bf16. There is no lint requiring like 5f32 must be written as 5.0f32 though.
| On AArch64, `bf16` is supported by Armv8.6-A and later, implementations that include the `bf16` architectural extension, include platforms based on [Neoverse V1][Neoverse V1], [Neoverse N2][Neoverse N2], and [Neoverse V2][Neoverse V2]. On x86_64, `bf16` is supported through `AVX512_BF16` on platforms such as [Intel Cooper Lake][Intel Cooper Lake], [Intel Sapphire Rapids][Intel Sapphire Rapids] and later platforms with `AMX_BF16`, and [AMD Zen 4][AMD Zen 4] and later platforms with `AVX-512 BF16` support. On [RISC-V][RISC_V], `bf16` is supported through the ratified `Zfbfmin`, `Zvfbfmin`, and `Zvfbfwma` extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively. | ||
|
|
||
| Consequently, `bf16` is not limited to accelerator hardware; it is a natively supported scalar/vector format on modern CPUs, with availability discoverable through target-feature mechanisms. |
There was a problem hiding this comment.
I think the chip list can go in an appendix or footnotes (or link to wikipedia instead), IMO that's not especially interesting as long as the extension has been around for a while. Maybe it would be helpful to have something tabular in the reference-level explanation? E.g.:
- aarch64
foofeature: ...- aarch64
barfeature: ...- x86
bazfeature: ...- RISC-V
Zfbfmin: scalarf32<->bf16, ...- RISC-V
Zvfbfmin: vectorf32<->bf16, ...
There was a problem hiding this comment.
I've moved what was written here with a view of following it up shortly with a table as you have suggested
| ### Vector operations | ||
|
|
||
| A first-class `bf16` type lets Rust SIMD, dot-product, and matrix APIs use the natural element type for `bf16` data instead of `u16`, wrapper types, or architecture-specific encodings. This improves clarity, and portability of API surfaces. | ||
|
|
||
| Target-specific vector and matrix operations may still have target-specific numerical behaviour. APIs exposing hardware instructions should be understood as exposing those instructions, not as promising bit-exact results across architectures. | ||
|
|
||
| For more details, see reference-level explanation. | ||
|
|
||
| ### Scalar arithmetic | ||
|
|
||
| The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. | ||
|
|
||
| Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. | ||
|
|
||
| For more details, see reference-level explanation. |
There was a problem hiding this comment.
For the guide-level explanation I think it's more relevant to say:
- What math traits will be implemented?
- What
ascasts will be supported? - There will be vector versions in
core::arch - Do we guarantee IEEE semantics?
|
|
||
| On AArch64, [the AAPCS64 specification defines `__bf16` as the Brain floating-point format with byte size 2 and natural alignment 2][aapcs64-bf16]. On x86_64, `bf16` ABI compatibility should likewise follow the platform psABI and toolchain support for `__bf16`, which is stated as having ["the same size, alignment, parameter passing and return rules as _Float16"][x86_64-bf16]. | ||
|
|
||
| Within Rust, `bf16` shall have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding described above. These are properties of the Rust type itself, not a new foreign calling convention. |
There was a problem hiding this comment.
Is this something we want/need to guarantee? I think it's fine to say it will usually size/alignment 2, but not tie ourselves into the alignment in case some platform does something different.
Also the last sentence seems a bit redundant, I'm not sure "not a new foreign calling convention" is needed
There was a problem hiding this comment.
I've softened the language on this
| Primitive types;`f16/f32/f64/f128`, `bool`, `char` etc... lives in the global namespace and are always in scope without an import. This RFC proposes the primitive `bf16` would by default do the same, reserving the bare name `bf16` in every Rust program. | ||
|
|
||
| An alternative is to make `bf16` a compiler-known primitive that is reached by path, as `core::num::bf16`, and is not added to the prelude. This mirrors the placement proposed for `Complex` in [RFC 3892][RFC-3892], which positions a numeric interchange type in `core::num` rather than the global namespace. | ||
|
|
||
| #### The Tradeoffs | ||
|
|
||
| A path only `core::num::bf16` does not consume a globally reserved name which signals that this is a specialist interchange type rather than a general-purpose arithmetic type which arguably a `bf16` could be. This could also set a precedent for future numeric formats without committing prelude real estate each time. If we want `bf16` to be something that is only used as an exchange format then `use core::num::bf16;` could be a more natural fit. | ||
|
|
||
| However a global namespace is consistent with `f16/f128` and the rest of the primitives, requires no import, and is what users would need if they are wanting to suffix floating point numbers with `bf16` to define `bf16`'s. Though whether this is desirable needs to be ironed out. |
There was a problem hiding this comment.
Counter: bf16 is kind of a weird type and probably shouldn't be preferred over one of the f* types unless a specific usecase is met. Keeping it from being available by default guides users in the right direction.
I believe core::arch:bf16 was proposed before as well, along with not allowing the bf16 suffix.
There was a problem hiding this comment.
I feel the same way with the other proposed floating point types (f64f64 and f80). f16 is much more likely to be supported (x64 F16C (2009!), universal GPU support, RISC-V Zfh, ARM(?)). no comment on f128 though, haha.
If bf16 and its oddball friends become new "primitive" types in the prelude I'll make a (joke) RFC to add f24.
There was a problem hiding this comment.
@ds84182 As noted in the RFC, bf16 is supported by multiple hardware vendors and is widely used in ML and AI workloads. Intel is also adding further support in upcoming instruction-set extensions; see Chapter 7 of the Intel® Advanced Vector Extensions 10.2 Architecture Specification.
For the other types you mentioned, to the best of my knowledge:
f64f64has native hardware support only on PowerPCf80has native hardware support only on x86f24has no native support from major hardware vendors
In my view, the combination of broad hardware support and substantial real-world use is enough to distinguish bf16 from those types. C++ has reached a similar conclusion by standardising std::bfloat16_t.
The RFC covers these arguments in more detail, so I will avoid repeating all of them here.
There was a problem hiding this comment.
For the other types you mentioned, to the best of my knowledge:
f64f64has native hardware support only on PowerPC
it's not really native support, it's just calling standard library functions that use the normal f64 instructions to implement f64f64
f80has native hardware support only on x86
iirc it's also supported on m68k
There was a problem hiding this comment.
Counter: bf16 is kind of a weird type and probably shouldn't be preferred over one of the f* types unless a specific usecase is met. Keeping it from being available by default guides users in the right direction.
I believe core::arch:bf16 was proposed before as well, along with not allowing the bf16 suffix.
I've added having no bf16 suffix to Rationale/Alternatives
|
|
||
| ### Scalar Arithmetic Alternatives | ||
|
|
||
| Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic for a `bf16` means that the rather complicated and, depending on support, possibly inefficient scalar operations could be omitted for `bf16` which makes the type simpler and prevents a user being confused with the resultant code of widening to `f32`, performing the operations and narrowing to `bf16` along with preventing misuse of the type. It seems reasonable a user could utilise the `From` and `as` casts themselves, perform scalar arithmetic in an `f32` and convert back to `bf16` without relying on built in support. Moreover vendor specific intrinsics could be used, where available, to perform scalar operations. |
There was a problem hiding this comment.
However omitting scalar arithmetic for a
bf16means that the rather complicated and, depending on support, possibly inefficient scalar operations could be omitted forbf16
Omitting scalar arithmetic means scalar operations could be omitted - huh?
There was a problem hiding this comment.
Honestly I'm really struggling to understand this paragraph and think it could be rewritten
There was a problem hiding this comment.
I've reworded it. Somehow it made sense to me when I initially wrote it 😅
| Rust does already accept target-dependent behaviour in a primitive: `usize` differs in size between 32-bit and 64-bit targets, so identical source an overflow on one target and not another. This could be seen as a precedent for letting `bf16` scalar arithmetic follow whatever the hardware supports, thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which the portable specification would forbid. Ostensibly this should, theoretically, be more performant that converting to `f32` doing the operation and narrowing back to `bf16`. | ||
|
|
||
| The `usize` precedent is weaker than first appears as `usize` is deterministic per target: given a target triple its behaviour is fully specified and stable. Its variation is structural 32-bits vs 64-bits and the type system prevents it being mixed with fixed sized integers. However `bf16` would vary on the same target triple depending on what target features have been made available. This problem is described in the [Prior Art](#c-compiler-support). | ||
|
|
||
| Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type. |
There was a problem hiding this comment.
I don't think usize is any kind of precedent here - a different type size isn't remotely close to the complexities introduced by IEEE vs. FTZ or other similar variation.
There was a problem hiding this comment.
True, it is however a primitive type that has different behaviours depending on the target. However if you think this is too much of a reach, I'm happy to remove this point
There was a problem hiding this comment.
usize just has a target-dependent size, the behavior is otherwise fully portable and identical to u32 or u64.
| Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic for a `bf16` means that the rather complicated and, depending on support, possibly inefficient scalar operations could be omitted for `bf16` which makes the type simpler and prevents a user being confused with the resultant code of widening to `f32`, performing the operations and narrowing to `bf16` along with preventing misuse of the type. It seems reasonable a user could utilise the `From` and `as` casts themselves, perform scalar arithmetic in an `f32` and convert back to `bf16` without relying on built in support. Moreover vendor specific intrinsics could be used, where available, to perform scalar operations. | ||
|
|
||
| Rust does already accept target-dependent behaviour in a primitive: `usize` differs in size between 32-bit and 64-bit targets, so identical source an overflow on one target and not another. This could be seen as a precedent for letting `bf16` scalar arithmetic follow whatever the hardware supports, thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which the portable specification would forbid. Ostensibly this should, theoretically, be more performant that converting to `f32` doing the operation and narrowing back to `bf16`. | ||
|
|
||
| The `usize` precedent is weaker than first appears as `usize` is deterministic per target: given a target triple its behaviour is fully specified and stable. Its variation is structural 32-bits vs 64-bits and the type system prevents it being mixed with fixed sized integers. However `bf16` would vary on the same target triple depending on what target features have been made available. This problem is described in the [Prior Art](#c-compiler-support). | ||
|
|
||
| Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type. |
There was a problem hiding this comment.
Why was it determined that we should have bf16->f32->bf16 semantics rather than bf16 semantics? I would expect that most users either want:
- Do all operations at
f32without spending time at intermediate rounding, which I believe C does - Correct bf16 semantics because you're doing something mathy
f32 with intermediate rounding gives predictable results but I'm not sure it brings an advantage over either of those two. (edit: does the difference even matter for the specified ops? #3983 (comment))
My preference would be to require bf16 semantics by default, then add methods like algebraic_add that can do entire operations as f32. That is, a + b + c only ever gives you a correctly rounded bf16 result, but a.algebraic_add(b).algebraic_add(c) could get lowered to any of the following:
// simplest, fastest on soft float targets and if there is hardware supporting
// bf16 arithmetic directly
addbf16(addbf16(a, b), c);
// semantics defined in this RFC: allowed but unlikely
addf32(addf32(a as f32, b as f32) as bf16 as f32, c as f32) as bf16;
// fastest with hardware f32
addf32(addf32(a as f32, b as f32), c as f32) as bf16;There was a problem hiding this comment.
Why was it determined that we should have bf16->f32->bf16 semantics rather than bf16 semantics
From trawling around I could not find any FADD BF16_REG, BF16_REG of BF16ADD REG, REG type instruction on any hardware. I subsequently looked at what GCC and Clang do. They convert the operands to f32 and cast back to bf16, how they do this differs and is described in the C++ section.
So I went down the avenue of bf16->f32->bf16.
I've tried to clarify the point I'm now making that if a FADD BF16_REG, BF16_REG exists and it has IEEE-754 semantics, we would use that. Otherwise we would do bf16->f32->bf16.
With regard to making a a+b+c+d expression all f32 and then converting the result back to bf16 I'd have thought the results could possibly differ from bf16->f32->bf16?
There was a problem hiding this comment.
The question was about the semantics, not about how it is compiled. So I am quite confused by the answer. We are not specifying our operations in terms of assembly instructions after all, so sentences like "if instruction X exists and has behavior Y then it may be used" are entirely out of place here.
What I hope the RFC will say is just: All scalar operations behave according to the IEEE 754-2019 specification when instantiated with the parameters for the bf16 format. In particular, they will return the value representable in the bf16 format that is closest to the infinite-precision result, using roundTiesToEven and fully making use of subnormal numbers. End of spec, that already says everything there is to say in the reference section about the semantics of the scalar operations.
In some other section (appendix?) it may then be worth talking about how those semantics could actually be implemented on the most relevant architectures. For all operations except FMA, it can be implemented by casting to f32 and doing the operations there. (That was the conclusion, right?) FMA will need a more complicated fallback if there is no hardware support. But people like me that care primarily about the opsem and not the asm can then just skip that other section. ;)
There was a problem hiding this comment.
Right 👍, I've shortened this point and moved the bf16->f32->bf16 to an appendix while also showing that there are alternative valid lowering schemes. And where that is not valid, as with the case of FMA, emit a library call; 5321ed6
Pinging @ecnelises who authored that RFC |
| On AArch64, [the AAPCS64 specification defines `__bf16` as the Brain floating-point format with byte size 2 and natural alignment 2][aapcs64-bf16]. On x86_64, `bf16` ABI compatibility should likewise follow the platform psABI and toolchain support for `__bf16`, which is stated as having ["the same size, alignment, parameter passing and return rules as _Float16"][x86_64-bf16]. | ||
|
|
||
| Within Rust, `bf16` shall have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding described above. These are properties of the Rust type itself, not a new foreign calling convention. | ||
|
|
||
| For `extern "C"` and other foreign ABIs, Rust should not independently choose how `bf16` is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ `__bf16` type, Rust's `bf16` should be ABI-compatible with that type. On platforms where the ABI is undefined, see [Unresolved Questions](#unresolved-questions). |
There was a problem hiding this comment.
Do GCC and Clang agree on targets they support? This kind of thing has been most of the f16 bugs preventing stabilization thus far.
|
|
||
| ### Usage | ||
|
|
||
| This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. |
There was a problem hiding this comment.
| This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. | |
| This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations (albeit with a non-standard binary format) and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. |
That binary format should be described in the Reference-level section.
|
|
||
| ## Reference-level explanation | ||
|
|
||
| ### ABI |
There was a problem hiding this comment.
Given that ABI is a very niche aspect that we usually make basically no guarantees about, this is an odd way to start the reference-level section. I would have expected it to start by saying:
- There's a new type, this is its representation
- This are the main (scalar) operations on the type, and these are their specs
Start with the basics, only then descend into the weeds by discussing vector operations and ABI concerns.
There was a problem hiding this comment.
I've moved the section headers around, I hope that addresses your concern
| For narrowing an `f32` to `bf16`, the conversion shall use the IEEE-754-2019 [default rounding-direction attribute for binary floating-point results, namely `roundTiesToEven`][ieee-754]. For subnormals the default delivers a rounded result that may or may not be zero. In other words subnormals are not flushed to zero by default. | ||
|
|
||
| This is a semantic specification, not an implementation requirement. An implementation may use any instruction sequence, including native `bf16` instructions, only when the observable result is identical to the specified model for all non-NaN inputs. The result of scalar `bf16` arithmetic must not depend on whether the target has native `bf16` arithmetic, on optimisation level, or on whether the expression is evaluated at compile time or run time. |
There was a problem hiding this comment.
Also this goes without saying, that's how our spec works everywhere.
| - The narrowing step always rounds to nearest, ties-to-even. Truncation is not a conforming narrowing. | ||
| - The arithmetic performs two IEEE rounding steps: the inner operation rounds to `f32` precision, and the narrow then rounds that result to `bf16` precision. | ||
|
|
||
| The two-step rounding may produce results that differ by one [ULP][ULP_WIKI] from a hypothetical single-rounded native `bf16` operation for a small fraction of inputs. This is a deliberate trade-off in favour of portable, bit-exact semantics; targets without native `bf16` arithmetic could not otherwise be supported uniformly. Code that requires single-rounded native semantics should use target-specific intrinsics. |
There was a problem hiding this comment.
Oh wow. This is extremely surprising. Does the double rounding theorem not apply here? bf16 has less than half the mantissa of f32 so I would have expected that it applies.
Do you have a concrete example of inputs where the two specs differ?
There was a problem hiding this comment.
I thought this is only referring to FMA and sanity checked that indeed that is the only one that fails https://gist.github.com/tgross35/a91c60bd655e24f912c7c7659a53cefd. But hm - I'm reading again and mul_add doesn't seem to be mentioned.
There was a problem hiding this comment.
Oh wow. This is extremely surprising...
I need to check this, for some reason I thought it wouldn't however I think I'm wrong.
There was a problem hiding this comment.
imo mul_add should be defined to work directly on bf16 so you get the extra precision people expect, the fallback implementation is just (edit: turns out to be more complicated than that for (a as f64).mul_add(b as f64, c as f64) as bf16mul_add). defining all of add/sub/mul/div/sqrt to work directly on bf16 is equivalent to defining them to convert to f32 before the op and round back to bf16 after using nearest-ties-to-even mode.
for the other ops (except maybe reciprocal sqrt), defining them as using f32 arithmetic and convering back to bf16 after every op seems fine to me, since those aren't generally expected to provide bit-exact answers anyway (and where they are, bf16 can generally also represent those exact answers)
There was a problem hiding this comment.
Yeah fully agreed, if it's just mul_add then we should be consistent with the other float types and require maximal precision. sqrt should also get maximal precision since it is a "core" IEEE operation. For transcendental functions we already make no precision guarantees so there's no reason to do anything else here.
Overall then this becomes an entirely normal IEEE-style float type, except that the binary format is not listed in the standard. But most of the standard is parametric over the format so that's not a big deal.
There was a problem hiding this comment.
First relevant LLVM bug I noticed, probably of many: incorrect bf16 fma llvm/llvm-project#131531.
I assume there's an easy fix by changing that to a libcall, which could likely use the same algorithm as https://github.com/rust-lang/compiler-builtins/blob/dfe80e1d870b56f34a52bd8b3ce47479cdb95a2d/libm/src/math/generic/fma_wide.rs (for fmaf32 via f64 unfused ops)
There was a problem hiding this comment.
thanks for spotting the fma bug, apparently I didn't think hard enough when I said f64 is enough.
There was a problem hiding this comment.
For this RFC, are the complexities surrounding mul_add something that would belong in the appendix section as opposed to the main body? Where the resolution would be to emit a library call for this op?
There was a problem hiding this comment.
(This thread overlaps a lot with #3983 (comment) )
are the complexities surrounding mul_add something that would belong in the appendix section as opposed to the main body?
It depends. What is the spec for mul_add? If it is "we require the result to be as-if the mul and add are done in infinite precision with a single roundTiesToEven step to bf16 at the end" then there is no complexity for mul_add on the spec/Reference level and that discussion should be moved to a clearly separate section that discusses implementation concerns.
|
I think should be an unification of float types names: thy should all starts with "f" letter. let a: fxb16 = 6.7;
let b: fxb16 = 42.69;
let c = a + b;
println!("{}", c); |
| ### Vector operations | ||
|
|
||
| A first-class `bf16` type is primarily useful for target-specific SIMD and Matrix APIs. These APIs commonly operate on vectors or tiles of `bf16` values and accumulate into `f32`, reflecting the way `bf16` is typically used in machine learning and numerical code. | ||
|
|
||
| For example, Arm exposes instructions that multiply vectors of `bf16` values and accumulate into `f32` results. x86_64 platforms expose similar functionality through features such as `AVX512_BF16` and, on some Intel platforms, `AMX_BF16`. | ||
|
|
||
| With a primitive `bf16` type, Rust APIs can model these operations using the natural element type of the source operands: | ||
|
|
||
| ```rust | ||
| // Example Neon intrinsic that would be added to `std::arch::aarch64` | ||
| fn vbfdot_lane_f32<const LANE: i32>(r: float32x2_t, a: bfloat16x4_t, b: bfloat16x4_t) -> float32x2_t; | ||
| ``` | ||
|
|
||
| For intrinsics that live in stdarch that are hardware-specific, some `bf16` instructions can have different numerical semantics across architectures. For example, Arm [`BFDOT`][ARM_ARM] and related `bf16` dot-product instructions operate on `bf16` inputs and accumulate into `f32` results, but their pseudocode specifies architecture-defined intermediate rounding behaviour, and this behaviour can differ further when enhanced `bf16` support is enabled. | ||
|
|
||
| By contrast, x86_64's [`VDPBF16PS`][intel_a64] specifies `bf16` dot products accumulated into `f32` using round-to-nearest ties to even for the FMA accumulation, while also treating input denormals as zero, flushing output denormals to zero, and ignoring `MXCSR`. | ||
|
|
||
| Therefore, target-specific SIMD, dot-product, and matrix APIs should be understood as exposing the hardware operation, not as promising target-independent bit-exact results. Portable Rust APIs may use `bf16` as the element type for clarity and type safety, but portability of the source-level API should not be read as portability of every numerical detail of the underlying target-specific instruction. This is true today with intrinsics for "packed/vector float min" for example; | ||
|
|
||
| ``` | ||
| x86_64: _mm_min_ps(-0.0, +0.0) -> +0.0 bits 0x00000000 | ||
| AArch64: vminq_f32(-0.0, +0.0) -> -0.0 bits 0x80000000 | ||
| ``` | ||
|
|
||
| While ostensibly similar they produce different results. |
There was a problem hiding this comment.
This entire paragraph is basically just saying "the only types and operations for vectors of bf16 are provided by core::arch (for now), and as usual for those operations, their semantics are architecture-specific", right?
I feel like that point can be made in fewer than 6 paragraphs. ;)
… `FMA` as an op where a library call should be emitted
|
|
||
| `bf16` is a 16-bit floating-point format derived from `f32`. It keeps the same 1-bit sign and 8-bit exponent layout as `f32`, but stores only the top 7 fraction bits. This gives it approximately the same dynamic range as `f32`, but much lower precision. The primary role of `bf16` is representing bfloat16 values in memory, in APIs, in FFI signatures where the target ABI supports bfloat16, and as an element type for SIMD and matrix operations. It is expected to be uncommon for a `bf16` to be used as a general-purpose arithmetic type in the same way as `f32` or `f64`. | ||
|
|
||
| `bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for conversions and scalar arithmetic. These are specified as widening to `f32`, performing the corresponding `f32` operation, and narrowing back to `bf16` using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. |
There was a problem hiding this comment.
"These are specified as widening to ..." is outdated now, that's not how they are specified any more.
|
|
||
| ```rust | ||
| // Widen to `f32` first | ||
| let d: bf16 = (f32::from(a) + f32::from(b) + f32::from(c)) as bf16; |
There was a problem hiding this comment.
when doing bf16 addition, it isn't correct to convert 3 values to f32, do 3 f32 adds, and then convert the result to bf16 -- you have to convert back to bf16 after every addition.
There was a problem hiding this comment.
I think I perhaps got the wrong end of the stick with; #3983 (comment)
I've removed this line.
There was a problem hiding this comment.
If this comes from #3983 (comment), I was talking specifically about algebraic_add there
8 participants
View all comments
Add a
bf16primitive type to Rust, with support for portable scalar arithmetic that follows IEEE-754 semantics for rounding and subnormals. This has been spun off from the RFC Addbf16,f64f64andf80typesImportant
Since RFCs involve many conversations at once that can be difficult to follow, please use review comment threads on the text changes instead of direct comments on the RFC.
If you don't have a particular section of the RFC to comment on, you can click on the "Comment on this file" button on the top-right corner of the diff, to the right of the "Viewed" checkbox. This will create a separate thread even if others have commented on the file too.
Rendered