Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Changelog

## [Unreleased]

### Added
- [#116] Batch write helpers with up-front mutation-budget accounting:
`Transaction::setBatch(iterable $pairs)` (queues `[key, value]` pairs —
`string` or `KeyConvertible` keys — into the current transaction; the
total key+value size is validated against FoundationDB's 10,000,000 B
per-transaction mutation budget *before* any mutation is queued, throwing
the new `CrazyGoat\FoundationDB\BatchTooLargeException` (public readonly
`batchSize` / `maxBatchSize`) at the call site instead of an opaque
server-side error during commit) and `Database::setBatch(iterable $pairs,
bool $split = false)` (single-transaction mode with retries via
`transact()`, or `split: true` mode that groups entries under the new
`MutationBudget::SPLIT_TARGET_BYTES` (8,000,000 B) and commits each group
in its own retried transaction — per-key atomicity, no cross-key snapshot
consistency, documented). The shared `MutationBudget` accounting class is
`@internal`; generator-backed `iterable` input is consumed lazily.
Integration tests in `tests/Integration/SetBatchTest.php`; documented in
`docs/batch-writes.md`.

## [1.0.0] - 2026-09-09

### Added
Expand Down
103 changes: 103 additions & 0 deletions docs/batch-writes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Batch writes

Multi-key write helpers built on top of a single transaction primitive, with
up-front accounting of FoundationDB's per-transaction [mutation budget]
(https://apple.github.io/foundationdb/known-limitations.html) of 10,000,000
bytes (keys + values + overhead).

## `Transaction::setBatch()`

Queues multiple writes into the current transaction:

```php
use CrazyGoat\FoundationDB\KeyConvertible;
use CrazyGoat\FoundationDB\Subspace;

$db->transact(function ($tr) use ($subspace): void {
$tr->setBatch([
['users/alice', $payloadA],
['users/bob', $payloadB],
[$subspace->pack(['stats', 7]), $stats], // KeyConvertible supported
]);
});
```

- Entries are `[key, value]` pairs (list form, so `KeyConvertible` keys are
supported). Keys and values are validated individually exactly like
`set()` (key ≤ 10,000 B, value ≤ 100,000 B).
- The total batch size is computed **before any mutation is queued**. An
oversized batch throws `CrazyGoat\FoundationDB\BatchTooLargeException`
(public readonly fields: `$batchSize`, `$maxBatchSize`) at the call site,
instead of an opaque server-side error surfacing at `commit()` after the
mutations were already queued. Nothing is written.
- The writes are **blind** (no reads), so two concurrent `setBatch()` calls
do not conflict with each other — the last commit wins. Conflict detection
for read-modify-write patterns comes from the reads performed in the same
transaction.
- If a key appears multiple times, the last entry wins; ordering within the
batch is not guaranteed.

Typical read-modify-write flow:

```php
$db->transact(function ($tr): void {
$updates = [];
foreach ($keys as $key) {
$updates[] = [$key, $tr->get($key)->await() . '-appended'];
}
$tr->setBatch($updates);
});
```

## `Database::setBatch()`

Convenience wrapper that runs the batch inside a retried transaction
(`transact()`), so retryable FDB errors are retried with the standard
`onError()` backoff, bounded by the process-wide retry-limit / timeout
settings:

```php
$db->setBatch([
['cache/alpha', '1'],
['cache/beta', '2'],
]);
```

### Split mode

```php
$db->setBatch($hugeBatch, split: true);
```

Groups the batch into chunks of at most `MutationBudget::SPLIT_TARGET_BYTES`
(8,000,000 B) and commits each group in its own retried transaction. This
removes the size ceiling, at a documented cost:

- each **individual key write** stays atomic,
- there is **no cross-key snapshot consistency** — while the batch is in
flight, readers may see a mixture of old and new keys,
- there is **no all-or-nothing commit** — a failure mid-batch leaves part of
the keys updated.

This is deliberately not symmetric with the chunked-values mode (#115),
where a metadata swap hides the multi-transaction split behind an atomic
pointer update: a batch of *independent keys* has no such pointer to swap.
Split mode therefore suits bulk loads and independent key refreshes, but not
keys that must stay mutually consistent (e.g. data + index pairs).

Split mode accepts any `iterable`, including generators — entries are
consumed lazily and grouped on the fly, so oversized batches do not need to
be materialized twice in memory.

## Relationship to chunked values (#115)

Complementary features: `setBatch()` writes **many small values**, while
`setValueChunked()` (issue #115) writes **one value larger than the 100 kB
per-value limit** by splitting it across ordered sub-keys. Framework cache
adapters typically use both — small entries via `setBatch()`, oversized
entries via chunked writes.

Both share the internal `MutationBudget` accounting; the chunked key-space
layout is private to the chunked layer and `setBatch()` performs no
`\x00`-prefix validation on user keys (keys remain arbitrary binary, with
the same freedom as plain `set()`).
31 changes: 31 additions & 0 deletions src/BatchTooLargeException.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?php

declare(strict_types=1);

namespace CrazyGoat\FoundationDB;

/**
* Thrown when a batch write exceeds the per-transaction mutation budget
* enforced by `MutationBudget`.
*
* Unlike the opaque `value_too_large` (2103) / `too_many_mutations` server
* errors, this exception is raised synchronously at the call site — before
* any mutation has been queued on the transaction — and carries the measured
* batch size alongside the enforced limit so the caller can react precisely
* (e.g. fall back to Database::setBatch(..., split: true)).
*/
final class BatchTooLargeException extends \RuntimeException
{
public function __construct(
public readonly int $batchSize,
public readonly int $maxBatchSize,
) {
parent::__construct(sprintf(
'Batch exceeds the FoundationDB transaction mutation budget: %d bytes of key+value data '
. '(limit is %d bytes). Reduce the batch size, or use Database::setBatch(..., split: true) '
. 'to commit it across multiple transactions.',
$batchSize,
$maxBatchSize,
));
}
}
89 changes: 89 additions & 0 deletions src/Database.php
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,95 @@ public function set(string|KeyConvertible $key, string $value): void
});
}

/**
* Write multiple key/value pairs in batched transactions.
*
* Each entry is a `[key, value]` pair (list form, so `KeyConvertible`
* keys are supported) — the same shape as `Transaction::setBatch()`.
*
* - `split: false` (default): the whole batch is committed inside a
* single retried transaction (`transact()`). The batch size is checked
* against FoundationDB's per-transaction mutation budget *before* any
* mutation is queued; an oversized batch throws
* `BatchTooLargeException` and nothing is written.
* - `split: true`: the batch is grouped into chunks of at most
* `MutationBudget::SPLIT_TARGET_BYTES` and each group is committed in
* its own retried transaction. This removes the size ceiling, at a
* documented cost: each *individual key write* stays atomic, but there
* is no cross-key snapshot consistency and no all-or-nothing commit —
* a failure mid-batch leaves part of the keys updated, and readers
* may see a mixture of old and new keys while the batch is in
* flight. Suitable for bulk loads and independent key refreshes;
* not suitable for keys that must stay mutually consistent (e.g.
* data + index pairs).
*
* Retry behaviour: every group goes through `transact()`, so retryable
* FDB errors are retried with the standard `onError()` backoff bounded
* by the process-wide retry-limit / timeout settings.
*
* @param iterable<array{0: string|KeyConvertible, 1: string}> $pairs
*
* @throws \InvalidArgumentException when a key or value violates the
* FDB size limits
* @throws BatchTooLargeException when `split` is false and the batch
* exceeds the per-transaction
* mutation budget
*/
public function setBatch(iterable $pairs, bool $split = false): void
{
if (!$split) {
$this->transact(static function (Transaction $tr) use ($pairs): void {
$tr->setBatch($pairs);
});

return;
}

$group = [];
$groupBytes = 0;

foreach ($pairs as $pair) {
// Runtime guard, deliberately silenced against the docblock shape
// (see Transaction::setBatch() for the rationale).
if (!is_array($pair)) { // @phpstan-ignore function.alreadyNarrowedType
throw new \InvalidArgumentException(
'Each setBatch() entry must be a [key, value] pair',
);
}

if (count($pair) !== 2) { // @phpstan-ignore notIdentical.alwaysFalse
throw new \InvalidArgumentException(
'Each setBatch() entry must be a [key, value] pair',
);
}

$size = MutationBudget::entrySize($pair);

if ($group !== [] && $groupBytes + $size > MutationBudget::SPLIT_TARGET_BYTES) {
$this->setBatchGroup($group);
$group = [];
$groupBytes = 0;
}

$group[] = $pair;
$groupBytes += $size;
}

if ($group !== []) {
$this->setBatchGroup($group);
}
}

/**
* @param list<array{0: string|KeyConvertible, 1: string}> $group
*/
private function setBatchGroup(array $group): void
{
$this->transact(static function (Transaction $tr) use ($group): void {
$tr->setBatch($group);
});
}

public function clear(string|KeyConvertible $key): void
{
$this->transact(function (Transaction $tr) use ($key): void {
Expand Down
84 changes: 84 additions & 0 deletions src/MutationBudget.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<?php

declare(strict_types=1);

namespace CrazyGoat\FoundationDB;

/**
* Mutation-budget accounting shared by the batch-write helpers
* (`Transaction::setBatch()` / `Database::setBatch()`).
*
* FoundationDB enforces a hard limit of 10,000,000 bytes on the total size
* of a committed transaction (keys + values + overhead). The native client
* rejects oversized commits with an opaque server-side error at commit time,
* *after* the application has already queued its mutations. Pre-flight
* accounting at the PHP trust boundary surfaces the overflow at the call
* site instead, as a named `BatchTooLargeException`.
*
* Split mode (`Database::setBatch(..., split: true)`) groups entries under
* {@see self::SPLIT_TARGET_BYTES} — a deliberately conservative margin below
* the hard limit so that key/tuple overhead, conflict-range bookkeeping and
* the read-write sets of the surrounding transaction cannot push a group
* over the server-side limit during commit.
*
* This type intentionally knows nothing about the chunked-values key space
* (see issue #115): it is pure byte accounting, shared by every multi-key
* write helper.
*
* @internal Shared infrastructure; not part of the public API.
*/
final class MutationBudget
{
/**
* FoundationDB's server-side per-transaction mutation budget
* (https://apple.github.io/foundationdb/known-limitations.html).
*/
public const TRANSACTION_MUTATION_LIMIT = 10000000;

/**
* Target group size when a batch is split across multiple transactions.
*
* Comfortably below the hard limit so that per-key overhead (conflict
* ranges, commit bookkeeping) can never push a group past the server-side
* check. Each individual entry is bounded by `KeyValueLimits::MAX_KEY_SIZE
* + MAX_VALUE_SIZE` (20 kB), so a group can only overshoot the target by a
* single entry and always stays far below the hard limit.
*/
public const SPLIT_TARGET_BYTES = 8000000;

private function __construct()
{
}

/**
* Assert that the measured batch fits inside a single transaction's
* mutation budget.
*
* @throws BatchTooLargeException when `$totalBytes` exceeds the budget
*/
public static function assertWithinTransactionLimit(int $totalBytes): void
{
if ($totalBytes > self::TRANSACTION_MUTATION_LIMIT) {
throw new BatchTooLargeException($totalBytes, self::TRANSACTION_MUTATION_LIMIT);
}
}

/**
* Byte size of a single `[key, value]` batch entry, with the key resolved
* to its final binary form (`KeyConvertible` keys are packed via
* `asFoundationDbKey()`).
*
* @param array{0: string|KeyConvertible, 1: string} $pair
*/
public static function entrySize(array $pair): int
{
[$key, $value] = $pair;

return strlen(self::resolveKeyForSizing($key)) + strlen($value);
}

private static function resolveKeyForSizing(string|KeyConvertible $key): string
{
return $key instanceof KeyConvertible ? $key->asFoundationDbKey() : $key;
}
}
Loading
Loading