Expose the spans that widths are additive over
Hi, and thanks for maintaining this crate — unicode-truncate has been built on it from the start, and the contextual width rules in 0.2 were a clear correctness win.
They also surfaced a gap I don't think a downstream crate can close on its own, so I'd like to propose an addition. Feel free to push back on any of it.
unicode-truncate cuts strings to a display width budget. Anything that cuts text needs grapheme clusters as the unit, and therefore sums cluster.width(). That sum is not str::width():
let lam_alef = "\u{0644}\u{0627}"; // لا
assert_eq!(lam_alef.width(), 1);
assert_eq!(lam_alef.graphemes(true).map(|g| g.width()).sum::<usize>(), 2);The contextual rules documented in the README are exactly the cases where the two differ, and each spans a cluster boundary, so no per-cluster width assignment can close the gap.
This is not a rounding error downstream (Aetf/unicode-truncate#40): unicode-truncate 3.0.0 mixed the two models and could return a slice wider than the requested budget ("لا".repeat(8) centered to 1 column returned a 5-column slice), and its padding came up short of the target width. 3.1.0 fixes that by scoring candidates consistently in the per-cluster sum model and measuring the returned slice, which is the best a caller can do today: never over budget, honestly reported, but conservative — on strings containing these ligatures it keeps less than would fit, and centering drifts.
Request
An iterator over the spans widths are additive over:
/// Contiguous byte ranges covering the string, each boundary a grapheme cluster boundary.
fn width_spans(&self) -> impl Iterator<Item = (Range<usize>, usize)>;
with the guarantee that for every span boundary i, s[..i].width() + s[i..].width() == s.width().
That is the whole ask: the positions where cutting does not change what the pieces measure. Truncation, padding, wrapping and cursor placement all become exact on top of it. In shaping terms these are the safe-to-break positions (the complement of HarfBuzz's HB_GLYPH_FLAG_UNSAFE_TO_BREAK), a correctness constraint, unlike UAX #14's break opportunities.
This is deliberately not the truncation API of #84: no position on what to keep, no claim that cutting text is meaningful in general — only surfacing what the crate already computes, since the spans are where WidthInfo carries state across a cluster boundary. I'd rather not reimplement that detection downstream: it would mean copying the tables here and inevitably drifting out of sync with them.
On direction: str_width folds right to left and the Arabic rule allows unbounded transparent characters between lam and alef, so a forward iterator would need unbounded lookahead. An iterator that yields spans back to front is fine for my use.
No attachment to the exact shape — a different signature achieving the same guarantee works just as well for me. Happy to implement it and send a PR if this direction seems reasonable to you.
Expose the spans that widths are additive over
Hi, and thanks for maintaining this crate — unicode-truncate has been built on it from the start, and the contextual width rules in 0.2 were a clear correctness win.
They also surfaced a gap I don't think a downstream crate can close on its own, so I'd like to propose an addition. Feel free to push back on any of it.
unicode-truncate cuts strings to a display width budget. Anything that cuts text needs grapheme clusters as the unit, and therefore sums
cluster.width(). That sum is notstr::width():The contextual rules documented in the README are exactly the cases where the two differ, and each spans a cluster boundary, so no per-cluster width assignment can close the gap.
This is not a rounding error downstream (Aetf/unicode-truncate#40): unicode-truncate 3.0.0 mixed the two models and could return a slice wider than the requested budget (
"لا".repeat(8)centered to 1 column returned a 5-column slice), and its padding came up short of the target width. 3.1.0 fixes that by scoring candidates consistently in the per-cluster sum model and measuring the returned slice, which is the best a caller can do today: never over budget, honestly reported, but conservative — on strings containing these ligatures it keeps less than would fit, and centering drifts.Request
An iterator over the spans widths are additive over:
with the guarantee that for every span boundary
i,s[..i].width() + s[i..].width() == s.width().That is the whole ask: the positions where cutting does not change what the pieces measure. Truncation, padding, wrapping and cursor placement all become exact on top of it. In shaping terms these are the safe-to-break positions (the complement of HarfBuzz's
HB_GLYPH_FLAG_UNSAFE_TO_BREAK), a correctness constraint, unlike UAX #14's break opportunities.This is deliberately not the truncation API of #84: no position on what to keep, no claim that cutting text is meaningful in general — only surfacing what the crate already computes, since the spans are where
WidthInfocarries state across a cluster boundary. I'd rather not reimplement that detection downstream: it would mean copying the tables here and inevitably drifting out of sync with them.On direction:
str_widthfolds right to left and the Arabic rule allows unbounded transparent characters between lam and alef, so a forward iterator would need unbounded lookahead. An iterator that yields spans back to front is fine for my use.No attachment to the exact shape — a different signature achieving the same guarantee works just as well for me. Happy to implement it and send a PR if this direction seems reasonable to you.