feat(#115): chunked values — setValueChunked/getValueChunked/deleteValueChunked - #118
Merged
Conversation
…lueChunked - Chunk\ChunkKeyCodec (@internal): \x00-namespaced key space — meta key (sorts first) + chunk keys Tuple::pack([generation, index]); magic-signed 22-byte metadata record (FDBCK1 | total_length | chunk_count | generation) - Transaction::setValueChunked(key, value, chunkSize = 100000): atomic single-transaction write (namespace clear + chunks + meta), capped at MutationBudget::SPLIT_TARGET_BYTES (8 MB) with ChunkedValueTooLargeException (readonly valueSize/maxSize) thrown before any mutation - ReadTransaction::getValueChunked(): one range read of the active generation; null for missing key, '' for empty value, ChunkedValueCorruptedException on malformed metadata / missing chunks / wrong total length; snapshot reads supported - Transaction::deleteValueChunked(): one range clear removes meta + all chunks + all generations - Database::setValueChunked(..., atomic = true): atomic mode via transact(); atomic: false — generation scheme: budget-sized chunk groups across multiple transactions, then an atomic meta swap + clear below the new generation (self-cleaning orphaned chunks, no size cap) - Unit tests (ChunkKeyCodecTest): namespace ordering, generation boundaries incl. the 255->256 tuple-int encoding size change, meta record round trip and tamper detection - Integration tests (ChunkedValuesTest): round trips, chunk-boundary edge, stale-tail overwrite, snapshot read, atomic cap before mutation, non-atomic 12 MB write, generation swap cleanup, corruption detection - docs/chunked-values.md + CHANGELOG entry
5 tasks
Base key 2 bytes below the limit: the metadata key fits exactly at 10,000 bytes but every chunk key (base + separator + packed [generation, index] suffix) exceeds the limit; set() rejects it before the transaction queues mutations, nothing is written and the database stays usable. Closes the last gap versus the literal acceptance criteria of #115.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the chunked-values feature designed in #115 (see the design comment on the issue):
Chunk\ChunkKeyCodec(@internal) — the frozen storage format:$key . "\x00" . Tuple::pack([$generation, $index])$key . "\x00" . "\x00"(sorts before all chunks; unreachable through the public Tuple/Subspace/Directory API)[$key . "\x00", $key . "\x01")— no collision with user text suffixes"FDBCK1" | total_length (8 B) | chunk_count (4 B) | generation (4 B)Transaction::setValueChunked(key, value, chunkSize = 100000)— atomic single-transaction write: namespace clear (stale-tail removal) + chunk writes + metadata record. Capped atMutationBudget::SPLIT_TARGET_BYTES(8,000,000 B) — above the capChunkedValueTooLargeException(readonlyvalueSize/maxSize) is thrown before any mutation.ReadTransaction::getValueChunked(key): ?string— one range read of the active generation; snapshot reads work for free.nullfor a missing key,""for an empty value (meta withchunk_count = 0),ChunkedValueCorruptedExceptionon malformed metadata / missing chunks / wrong total length (loud, never garbage).Transaction::deleteValueChunked(key)— one range clear removes meta + all chunks + all generations.Database::setValueChunked(..., atomic: true)(default) — atomic mode viatransact().atomic: false— generation scheme, no size cap: chunk groups (≤SPLIT_TARGET_BYTES) committed in their own retried transactions, then a final micro-transaction atomically swaps the metadata and clears everything below the new generation (previous generation + orphaned chunks from interrupted attempts). Readers always see either the whole old or the whole new value.Shared
@internal MutationBudgetaccounting from feat(#116): setBatch() helper with mutation-budget enforcement #117; the\x00codec stays private to the chunked layer (setBatchnever touches it).Closes #115
Testing
composer lint— PHPCS + Rector + PHPStan level 9: cleancomposer test:unit— 610 tests OK, incl. newChunkKeyCodecTest: namespace ordering, generation range boundaries (incl. the tuple-int encoding size change at 255 → 256), meta record round trip + tamper detectioncomposer test:integration— 308 tests OK, incl. newChunkedValuesTest(16 tests): round trips (small / empty / multi-chunk > 100 kB / chunk-boundary edge),KeyConvertiblekeys, snapshot read, overwrite larger→smaller with no stale tail, delete removes everything, atomic cap rejected before mutation + accepted exactly at cap, non-atomic 12 MB write, generation swap cleanup, foreign meta data and missing chunks detectedDocs & changelog
docs/chunked-values.md(new) — layout, both write modes, read/delete semantics, caveats (mixing with plainset()unsupported, no watch/atomic-op visibility, no TTL)CHANGELOG.md—[Unreleased] / Addedentry referencing [[Feature] Chunked values: setValueChunked() / getValueChunked() #115]