Repository navigation
Read JSON numbers as Decimal so long-precision values aren't changed - #69
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #60.
Problem
rune_deserializeparsed JSON with plainjson.loads, andmodel_validate_jsonwith pydantic's parser. Both turn an unquoted number with a fraction or exponent into a binaryfloatbefore it becomes aDecimal, so values with more than about 15 to 17 significant digits changed with no error, and a trailing zero was lost:12345678901234567.891234567890123456812345678901234567.890.10000000000000000000010.10.100000000000000000000150000.1050000.150000.10Java writes numbers unquoted, so this affected JSON written by Java. Quoted numbers and whole numbers were already read exactly.
Change
rune_deserializeparses withjson.loads(rune_data, parse_float=Decimal).BaseDataClass.model_validate_jsonis overridden to parse the same way, then callmodel_validate. Pydantic 2.14 has no setting to read JSON floats asDecimal. Withstrict=True, pydantic's own parsing is kept, so strict validation behaves as before; validating parsed values strictly would reject JSON forms that strict mode accepts today, such as an enum's value or a UUID string.test/test_number_precision.py, 25 tests, using the classesrune-python-generatorproduces fornumber, list,[metadata scheme],int,dateand enum attributes.What this closes in #60
numbervalues differently #60 from 2026-11-09, which superseded the original recommendation, for bothrune_deserializeandmodel_validate_json.numbervalues as strings ("50000"). Java reads quoted numbers exactly, and RFC 7493 recommends strings for long precision, so the writer change first proposed in Python and Java write Runenumbervalues differently #60 isn't needed.model_validate_json(..., strict=True)keeps pydantic's parsing and still rounds long numbers. OnBaseDataClass, strict mode also rejects dates, times and numbers in their JSON forms, onmainas well.With the writer change dropped, nothing else in #60 remains, so the issue can be closed when this merges.
Related
PR #62 also makes the
parse_float=Decimalchange inrune_deserialize, together with the writer change that #60 no longer needs.Testing
test/test_number_precision.py: 25 passed. Onmain, 19 of the 25 fail.pip install .[dev]pluspytest,pytest-covandpytest-mock, as inrun-tests.yml): 379 passed, 20 skipped. The skips are the tests that need the generatedserializationpackage or a CDM package.