Skip to content

Conversation

@jasperpotts
Copy link
Member

This is prototyping work on performance gains in varint and strings that can get us between 2x and 7x faster writing of PBJ objects. There are 3 key ideas:

  1. Faster varint to bytearray code from a blog
  2. New write methods direct to bytearray, bypassing streaming concepts. This needs to be fully finished off eveywhere but almost all use cases we have in the consensus node revolve around writing a PBJ model object into a Bytes or bytearray. Even the streaming use cases are object by object with BufferedStreams so we can just write object by object into a reused bytearray then send that to stream without BufferedSteam. Will be much faster and save multiple array copies.
  3. Changing internal storage of String fields from String to byte[]. This helps performance a lot as we rarely need to do UTF8 <-> String conversions. Most of the time we just read, modify and write. And most of the modifications are of non-string fields. So with model objects being classes not records now we can change the internal format while keeping getters, constructor and builders still in String format. So the internal bytearray is hidden from public API. Even copy builder can just reuse bytearray. Only pain is Codecs need direct access to bytearray and are in different package so no easy way to keep 100% private :-( .

Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
…bunch more cleaning up to be production ready but shows performance promise.

Signed-off-by: Jasper Potts <1466205+jasperpotts@users.noreply.github.com>
@github-actions
Copy link

Integration Test Report

1 files   -     403  1 suites   - 403   1s ⏱️ - 18m 26s
1 tests  - 114 834  0 ✅  - 114 835  0 💤 ±0  1 ❌ +1 
1 runs   - 115 075  0 ✅  - 115 076  0 💤 ±0  1 ❌ +1 

For more details on these failures, see this check.

Results for commit 505af05. ± Comparison against base commit faee5de.

This pull request removes 114834 tests.
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [1] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [2] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [3] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [4] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [5] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [6] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ [7] NoToStringWrapper{com.hedera.hapi.block.stream.input.EventHeader}
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ testCodecGetDefaultValueMethod()
com.hedera.hapi.block.stream.input.tests.EventHeaderTest ‑ testTestEqualsAndHashCode()
com.hedera.hapi.block.stream.input.tests.RoundHeaderTest ‑ [1] NoToStringWrapper{com.hedera.hapi.block.stream.input.RoundHeader}
…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants