You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: en_US/design/durable-storage.md
+2-6Lines changed: 2 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -111,12 +111,12 @@ Transactions rely on **Optimistic Concurrency Control (OCC)**, assuming that cli
111
111
**Transaction Flow:**
112
112
113
113
1.**Initiation:** A client process (Tx) requests the Leader node to create a transaction context (containing the Leader's term and last committed serial number).
114
-
2.**Operations:** Durable storage transaction is represented by an Erlang function. In the body of the function the client can read data (information about accessed topics and time ranges is added to the transaction context), schedule writes and deletes. The client can also set commit preconditions (checks for the existence/non-existence of specific TTVs). Reads are executed immediately, while scheduled writes/deletes only materialize upon full commitment and replication.
114
+
2.**Operations:** Durable storage transaction is represented by an Erlang function. Inside this function, the client can read data (which adds the information about accessed topics and time ranges to the transaction context) and schedule writes or deletes. The client can also set commit preconditions (checks for the existence/non-existence of specific TTVs). Reads are executed immediately, while scheduled writes/deletes only materialize upon full commitment and replication.
115
115
3.**Submission & Verification:** The client sends the list of operations to the Leader.
116
116
- The Leader checks the preconditions against the latest data snapshot.
117
117
- It verifies that the reads do not conflict with recent writes.
118
118
4.**"Cooking (preparing)" and Logging:** If successful, the Leader "cooks" the transaction:
119
-
- It assigns each written TTVs to on of the streams, creating new streams if necessary.
119
+
- It assigns each written TTV to one of the streams, creating new streams if necessary.
120
120
- It creates a list of low-level storage mutations that can be applied to all replicas in a deterministic manner.
121
121
5.**Commit:** A batch of "cooked" transactions is added to the Raft log (`builtin_raft`) or the RocksDB write-ahead log (WAL).
122
122
6.**Outcome:** Upon successful completion, the transaction process is notified. Conflicts result in the transaction being aborted and retried.
@@ -160,10 +160,6 @@ Both pools group subscribers by stream and topic, reusing resources to serve mul
## Conclusion: The Foundation of High-Reliability MQTT
164
-
165
-
The Optimized Durable Storage in EMQX 6.0 is the resilient foundation for high-reliability MQTT messaging. By re-engineering RocksDB and embedding concepts like TTVs and Streams, DS provides a purpose-built, highly available, and persistent internal database. This architecture, coupled with sophisticated features like the LTS algorithm and Raft replication, ensures lossless message delivery and optimal retrieval for complex wildcard and shared subscriptions, solidifying EMQX's position as a leading solution for demanding IoT infrastructure.
166
-
167
163
## More Information
168
164
169
165
Durable Storage serves as the core data foundation for several high-reliability and persistence-related features in EMQX, providing unified storage, replay, and consistency guarantees for upper-layer functionality, including:
0 commit comments