Every Magazine<T> instance exposes these operations:
| Method | Description |
|---|---|
load(T data) |
Enqueue data into the magazine. |
fire() |
Dequeue and return the next item. |
reload(T data) |
Re-enqueue data (decrements fire counter instead of incrementing load counter). |
delete(MagazineData<T>) |
Remove a specific record from the backend. |
deleteAll(Collection<MagazineData<T>>) |
Remove a batch of records in one round trip. |
getMetaData() |
Retrieve per-shard counters and pointers. |
getShards() |
This magazine's persisted shard count. |
peek(Map<Integer, Set<Long>>) |
Read specific shard/pointer records without consuming. |
getMagazineIdentifier() |
The identifier this magazine is bound to. |
IAerospikeClient client = new AerospikeClient("localhost", 3000);
AerospikeStorageConfig config = AerospikeStorageConfig.builder()
.namespace("test")
.dataSetName("mag_data")
.metaSetName("mag_meta")
.shards(8)
.recordTtl(30 * 24 * 60 * 60)
.metaDataTtl(2 * 30 * 24 * 60 * 60)
.build();
AerospikeStorage<String> storage = AerospikeStorage.<String>builder()
.aerospikeClient(client)
.storageConfig(config)
.enableDeDupe(true)
.farmId("dc1")
.clazz(String.class)
.clientId("my-service")
.scope(MagazineScope.LOCAL)
.build();
Magazine<String> magazine = Magazine.<String>builder()
.baseMagazineStorage(storage)
.magazineIdentifier("orders")
.build();Enqueue data into the magazine. Each successful call advances the load pointer and, after storing the data, increments the load counter for the selected shard.
boolean success = magazine.load("order-12345");When de-duplication is enabled, the write is issued as a create-only operation. If the data already exists, the call returns true without writing a duplicate.
!!! info "De-duplication support"
De-duplication is supported only for String, Long, and Integer data types.
Dequeue the next item. The library picks an active shard and claims a slot with a guarded atomic increment of that shard's fire pointer.
!!! danger "fire() is at-most-once"
The claim is applied server-side before the payload reaches you. If the client times out after the server applied the claim, the pointer has already advanced and that record will never be delivered again. Callers that cannot tolerate loss must persist their own idempotency record before acting on the result. See Delivery Semantics.
MagazineData<String> fired = magazine.fire();
String value = fired.getData(); // the payload
long pointer = fired.getFirePointer(); // position in shard
Integer shard = fired.getShard(); // shard index
String magId = fired.getMagazineIdentifier();!!! info "Two distinct failure modes"
fire() throws a MagazineException with one of two error codes, and treating them alike is a bug:
- `NOTHING_TO_FIRE` — the magazine is drained. There is nothing to consume; back off and retry later.
- `RETRIES_EXHAUSTED` — the library gave up while skipping consecutive pointer holes (slots whose pointer was allocated but whose data write failed), bounded by `maxFireHoleSkips`. This is *not* an empty queue: data may still exist. Alerting on it, rather than silently treating it as "empty", is the correct response.
The scan is bounded — there is no retry loop over contention, because the claim is a guarded atomic increment and can never be lost. Exhausted shards are suppressed in the process-local active-shard cache, so loads from another instance can take up to `activeShardRefreshSeconds` (default 5) to become visible.
Re-enqueue data that was previously fired. This increments the load pointer but decrements the fire counter instead of incrementing the load counter — preserving accurate total-load counts.
boolean reloaded = magazine.reload("order-12345");Remove a specific record from the backend. Requires the MagazineData object returned by fire() or peek().
MagazineData<String> fired = magazine.fire();
// Process the data...
magazine.delete(fired);!!! warning
delete() throws a MagazineException with INVALID_CONFIGURATION if the supplied MagazineData belongs to a different magazine than the one you call it on. Do not route records returned by one magazine into another's delete().
`deleteAll()` applies the same rule to every record in the batch, and validates the whole batch
before deleting any of it - so a batch carrying one foreign record is rejected without
retiring the records it did own.
Read records from specific shards and pointers without modifying any counters or pointers. Useful for inspection, debugging, or replay scenarios.
Map<Integer, Set<Long>> shardPointers = Map.of(
0, Set.of(1L, 2L, 3L),
1, Set.of(1L)
);
Set<MagazineData<String>> results = magazine.peek(shardPointers);
results.forEach(md ->
System.out.printf("Shard %d, Pointer %d → %s%n",
md.getShard(), md.getFirePointer(), md.getData()));Retrieve per-shard counters and pointers to understand magazine state.
Map<String, MetaData> meta = magazine.getMetaData();
meta.forEach((shard, m) -> {
System.out.printf("%-20s load=%d/%d fire=%d/%d%n",
shard,
m.getLoadCounter(), m.getLoadPointer(),
m.getFireCounter(), m.getFirePointer());
});Example output:
SHARD_0 load=5/5 fire=2/2
SHARD_1 load=3/3 fire=1/1
SHARD_2 load=0/0 fire=0/0
...
Answers "where had each shard's fire pointer reached at time T?" — which is how a caller can tell an abandoned in-flight record from one that was only just claimed, without stamping a timestamp on every message.
Off by default. The checkpoint map rides the shard's pointer record and every claim rewrites that record, so a caller that never asks should not pay for it.
Once on, checkpoints are recorded automatically while the magazine is consumed: at most one per shard per window, none for a shard whose pointer has not moved, and one batch call per magazine. See Fire history for how.
AerospikeStorageConfig.builder()
.dataSetName("my_data")
.metaSetName("my_meta")
.namespace("test")
.shards(8)
.fireHistoryEnabled(true) // default false
.fireHistoryWindowSeconds(60) // resolution of every answer
.fireHistoryEntries(32) // checkpoints retained per shard
.build();Map<String, FireCheckpoint> watermarks =
magazine.firePointerBefore(Instant.now().minus(Duration.ofMinutes(20)));
watermarks.forEach((shard, checkpoint) ->
System.out.printf("%s: everything at or below pointer %d was claimed 20+ minutes ago%n",
shard, checkpoint.firePointer()));Every slot at or below the returned pointer was claimed at or before that instant. Slots above it might have been claimed a moment ago.
A checkpoint is labelled with the end of the window it was observed in, so it answers questions about instants from one window later onwards. Precision is therefore one to two windows.
!!! note "An absent shard means 'do nothing', not 'empty'" A shard is missing from the result when no checkpoint reaches back that far yet. That is normal for a young magazine. Treat it as "nothing is provably old enough here this round" — never as "this shard has nothing in flight".
| Setting | Meaning |
|---|---|
fireHistoryWindowSeconds |
Seconds covered by one checkpoint, and therefore the resolution of every answer. At most one entry per shard per window, whatever the throughput, and none while the shard's pointer does not move. A wider window means fewer writes |
fireHistoryEntries |
Checkpoints retained per shard. Eviction is by count, not age, so an intermittently used magazine reaches further back than window x entries rather than having its history wiped by the first write after an idle period. The map rides the pointer record every claim rewrites, so raise this only as far as you need |
Asking about an instant further back than the retained history throws INVALID_REQUEST, and asking
at all with the feature disabled throws NOT_ENABLED.
This is deliberate and must not be caught and ignored. Returning an empty map would be
indistinguishable from "nothing is old enough", and a caller using this to recover abandoned messages
would silently stop recovering them. Size window x entries to comfortably exceed the oldest age you
intend to ask about.
try {
magazine.firePointerBefore(cutoff);
} catch (MagazineException e) {
if (e.getErrorCode() == ErrorCode.INVALID_REQUEST) {
// History no longer spans the cutoff. Skip this round loudly - do not treat as "nothing to do".
}
}Map<String, List<FireCheckpoint>> history = magazine.fireHistory(); // newest first per shardMagazineManager acts as a registry for multiple Magazine instances, potentially of different types.
MagazineManager manager = new MagazineManager("my-service");
Magazine<String> ordersMag = Magazine.<String>builder()
.baseMagazineStorage(stringStorage)
.magazineIdentifier("orders")
.build();
Magazine<Long> idsMag = Magazine.<Long>builder()
.baseMagazineStorage(longStorage)
.magazineIdentifier("id-pool")
.build();
// Register all magazines
manager.replaceAll(List.of(ordersMag, idsMag));
// Retrieve by identifier
Magazine<String> orders = manager.getMagazine("orders");
orders.load("new-order-456");
Magazine<Long> ids = manager.getMagazine("id-pool");
MagazineData<Long> id = ids.fire();!!! warning "replaceAll evicts"
replaceAll() atomically replaces the internal magazine map, so every magazine absent from the list is unregistered. Calling it with a single magazine drops all the others, and the next getMagazine() for one of them throws MAGAZINE_NOT_FOUND.
To add one magazine without disturbing the rest, use `register(magazine)`. On a per-request path use `getOrRegister(identifier, supplier)`: constructing a `Magazine` resolves its persisted shard configuration over the network, and `getOrRegister` skips that entirely when the handle is already held.
If you want to own the miss path yourself, find() is the non-throwing lookup:
Magazine<String> orders = manager.<String>find("orders")
.orElseGet(() -> {
Magazine<String> built = Magazine.<String>builder()
.baseMagazineStorage(stringStorage)
.magazineIdentifier("orders")
.build();
manager.register(built);
return built;
});When enableDeDupe is set to true on the storage, each load() call writes the de-dup marker with a create-only write, so the server — not a client-side lock — resolves concurrent attempts. The data is written only if no marker already exists.
AerospikeStorage<String> storage = AerospikeStorage.<String>builder()
.aerospikeClient(client)
.storageConfig(config)
.enableDeDupe(true) // ← enable de-duplication
.farmId("dc1")
.clazz(String.class)
.clientId("my-service")
.scope(MagazineScope.LOCAL)
.build();
magazine.load("unique-item");
magazine.load("unique-item"); // no-op — already exists| Scope | Reach | Set Name Resolution | Status |
|---|---|---|---|
LOCAL |
Single data-centre | {farmId}_{setName} |
✅ Supported |
GLOBAL |
Cross-DC | {setName} (no prefix) |
❌ Not yet implemented |
// Local scope (recommended)
.scope(MagazineScope.LOCAL)
// Global scope — rejected with NOT_IMPLEMENTED while the storage is
// being constructed, so the application fails at boot.
.scope(MagazineScope.GLOBAL)// ── Setup ──
IAerospikeClient client = new AerospikeClient("localhost", 3000);
AerospikeStorageConfig config = AerospikeStorageConfig.builder()
.namespace("production")
.dataSetName("notification_data")
.metaSetName("notification_meta")
.shards(32)
.recordTtl(7 * 24 * 60 * 60) // 7 days
.metaDataTtl(30 * 24 * 60 * 60) // 30 days
.build();
AerospikeStorage<String> storage = AerospikeStorage.<String>builder()
.aerospikeClient(client)
.storageConfig(config)
.enableDeDupe(true)
.farmId("dc1")
.clazz(String.class)
.clientId("notification-service")
.scope(MagazineScope.LOCAL)
.build();
Magazine<String> magazine = Magazine.<String>builder()
.baseMagazineStorage(storage)
.magazineIdentifier("push-notifications")
.build();
// ── Producer ──
magazine.load("user:1001:Welcome to our app!");
magazine.load("user:1002:Your order has shipped.");
// ── Consumer ──
MagazineData<String> notification = magazine.fire();
sendPushNotification(notification.getData());
magazine.delete(notification); // clean up after processing
// ── Missed delivery? Reload ──
magazine.reload("user:1001:Welcome to our app!");
// ── Monitoring ──
Map<String, MetaData> meta = magazine.getMetaData();
long totalPending = meta.values().stream()
.mapToLong(m -> m.getLoadPointer() - m.getFirePointer())
.sum();
System.out.println("Pending notifications: " + totalPending);