Compare exact values in DoubleValidator range checks#410
Conversation
garydgregory
left a comment
There was a problem hiding this comment.
Hello @sahvx655-wq
Please rebase on git master and see my comments.
|
|
||
| private static final FloatValidator VALIDATOR = new FloatValidator(); | ||
|
|
||
| private static boolean isFinite(final Number value) { |
There was a problem hiding this comment.
Rebase on git master and reuse AbstractNumberValidator.isFinite(Number).
There was a problem hiding this comment.
Dropped the FloatValidator change from this PR, so this thread is moot. Reasoning in the PR comment: the float override can't be pinned by a JDK-portable test at 2^53.
|
|
||
| private static final DoubleValidator VALIDATOR = new DoubleValidator(); | ||
|
|
||
| private static boolean isFinite(final Number value) { |
There was a problem hiding this comment.
Rebase on git master and reuse AbstractNumberValidator.isFinite(Number).
There was a problem hiding this comment.
Done. Rebased on master and dropped the local helper in both DoubleValidator and FloatValidator so they use the inherited isFinite(Number).
| final Double value = Double.valueOf(maxExactInt); | ||
| final BigInteger above = BigInteger.valueOf(maxExactInt).add(BigInteger.ONE); // 2^53 + 1 | ||
| final BigInteger below = BigInteger.valueOf(maxExactInt).subtract(BigInteger.ONE); // 2^53 - 1 | ||
| final BigDecimal justBelow = new BigDecimal(BigInteger.valueOf(maxExactInt)).subtract(BigDecimal.valueOf(0.5)); // 2^53 - 0.5 |
There was a problem hiding this comment.
Why is this not just BigDecimal.valueOf(maxExactInt) instead of new BigDecimal(BigInteger.valueOf(maxExactInt)) but since this is Double validator instead of a "Big" validator is might be OK...
There was a problem hiding this comment.
Simplified to BigDecimal.valueOf(maxExactInt) in the Double test. Same value, less noise.
| */ | ||
| @Override | ||
| public boolean maxValue(final Number value, final Number max) { | ||
| return isFinite(value) && isFinite(max) ? BigDecimal.valueOf(value.doubleValue()).compareTo(toBigDecimal(max)) <= 0 : value.doubleValue() <= max.doubleValue(); |
There was a problem hiding this comment.
Reuse the superclass' compareTo(Number, Number).
There was a problem hiding this comment.
Switched over. Both minValue and maxValue in both validators now call the superclass compareTo(value, bound) instead of building the BigDecimal by hand.
| */ | ||
| @Override | ||
| public boolean minValue(final Number value, final Number min) { | ||
| return isFinite(value) && isFinite(min) ? BigDecimal.valueOf(value.doubleValue()).compareTo(toBigDecimal(min)) >= 0 : value.doubleValue() >= min.doubleValue(); |
There was a problem hiding this comment.
Reuse the superclass' compareTo(Number, Number).
| */ | ||
| @Override | ||
| public boolean maxValue(final Number value, final Number max) { | ||
| return isFinite(value) && isFinite(max) ? BigDecimal.valueOf(value.doubleValue()).compareTo(toBigDecimal(max)) <= 0 : value.doubleValue() <= max.doubleValue(); |
There was a problem hiding this comment.
It looks like BigDecimal.valueOf(value.doubleValue()) looses precision.
There was a problem hiding this comment.
Right, that was the wrong basis. Going through compareTo(value, max) runs the value through toBigDecimal, which for a Float uses Float.toString, so the float is compared at its own precision (9.007199E15) rather than the extra digits doubleValue() invents when it widens the float. I adjusted the FloatValidatorTest expectations to that value.
There was a problem hiding this comment.
Removed the FloatValidator override entirely, so this is no longer in play. The precision point is exactly why it can't be tested portably at that magnitude; details in the PR comment.
6c209e2 to
abd6bba
Compare
|
Rebased on master and reused the pulled-up helpers throughout: both validators drop the local isFinite copy for the inherited isFinite(Number), and minValue/maxValue now defer to the superclass compareTo(Number, Number) rather than assembling the BigDecimal locally. The test bound is BigDecimal.valueOf(...) now too. On the float precision point: you were right that BigDecimal.valueOf(value.doubleValue()) was wrong. compareTo runs the value through toBigDecimal, and for a Float that uses Float.toString, so the comparison sits at the float's own decimal value (9.007199E15) instead of the trailing digits doubleValue() fabricates when it widens the float. That moved the FloatValidatorTest expectations onto 9.007199E15, which is the honest bound for a float at that magnitude. mvn clean verify is green. |
abd6bba to
88399fc
Compare
|
Sorry this sat red for a week. Chased down the CI failure: only the JDK 8 job actually failed, the rest of the matrix was cancelled by fail-fast, so it looked worse than it was. The culprit was my own testFloatNumberRangeExactBound. A float at 2^53 has a round-trip interval about a float ULP wide (~10^9 there), and compareTo runs the value through toBigDecimal, which for a Float goes via Float.toString. JDK 19+ emits the shortest form (9.007199E15) while JDK 8's older algorithm emits the exact 2^53, and my bound sat between the two, so the assertion flipped by JDK. The deeper problem is that any bound precise enough to trigger the double-narrowing bug has to fall within one double ULP of the value, which at that magnitude is always inside the float's round-trip interval, so there's no portable way to pin the float override in a test. I've dropped FloatValidator and scoped the PR to Double, where doubleValue() is exact and the test is stable (it already passed on JDK 8). The Double overrides defer to the superclass compareTo(Number, Number) and the inherited isFinite(Number) as you asked, and the test bound is BigDecimal.valueOf(...). mvn clean verify is green locally. |
The
Number-typed range overloads onDoubleValidatorare inherited fromAbstractNumberValidator, which testsvalue.doubleValue()againstbound.doubleValue(). That narrows the bound to adoublebefore comparing, so aBigDecimalorBigIntegerbound carrying more significant digits than adoublecan hold is rounded onto the value first.minValue(2^53, BigInteger 2^53 + 1)returnstrueandmaxValue(2^53, 2^53 - 0.5)returnstrue, both wrong: a value that sits outside the bound is reported as in range, and it then slips through anyisInRangeguard built on these methods.The two overrides defer to the superclass
compareTo(Number, Number), so the bound's precision survives; this mirrors thecompareTocomparisons already used inBigIntegerValidatorandBigDecimalValidator. A non-finite operand keeps the olddoubleValue()comparison so the documented infinity and NaN behaviour is unchanged. Placing the check in the callee meansisInRange(Number, Number, Number)picks it up through virtual dispatch rather than each caller needing its own guard.This was originally paired with the same change to
FloatValidator, but I've dropped that half: at the magnitude where the bound-narrowing bug bites (around 2^53) afloatspans roughly 10^9 per step, so the bound that triggers the bug always lands inside the value's round-trip interval, whereFloat.toString(which drivescompareTo) differs between JDKs. That made the float regression test pass on JDK 21 but fail on JDK 8, and there's no portable way to pin it.DoubleValidatorhas no such window, so I've scoped this PR to it.mvn; that'smvnon the command line by itself.