Add journalEntries view - #2697
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a journalEntries SQLite view that unions all physical journal shard tables, providing a stable way to query journal data without having to dynamically build UNION statements across journal* tables. It updates tests and documentation to use the new view.
Changes:
- Add automatic creation/update of a
journalEntriesview during SQLite journal initialization. - Update cluster tests to query
journalEntriesinstead of enumeratingjournal*tables. - Update documentation to describe querying journal data via
journalEntries.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
sqlitecluster/SQLite.cpp |
Creates/refreshes journalEntries view to UNION ALL all existing journal shard tables. |
test/clustertest/tests/CompressionTest.cpp |
Switches journal reads in tests to use journalEntries. |
test/clustertest/tests/BoundParametersTest.cpp |
Switches journal reads in tests to use journalEntries. |
docs/blockchain.md |
Documents the new journalEntries view as the supported way to query across journal shards. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| SQResult journalEntriesView; | ||
| SASSERT(!SQuery(db, "SELECT type, sql FROM sqlite_master WHERE name = 'journalEntries'", journalEntriesView)); | ||
| if (journalEntriesView.empty()) { | ||
| SASSERT(!SQuery(db, journalEntriesViewQuery)); |
There was a problem hiding this comment.
I assume this is instant and thus safe to run when we deploy this code?
There was a problem hiding this comment.
That is my understanding
tylerkaraszewski
left a comment
There was a problem hiding this comment.
I'm approving this because I think this code is fine, though I wonder if anything that actually uses it will have performance issues.
| SASSERT(journalEntriesView.size() == 1); | ||
| SASSERT(journalEntriesView[0][0] == "view"); | ||
| if (journalEntriesView[0][1] != journalEntriesViewQuery) { | ||
| SASSERT(!SQuery(db, "BEGIN IMMEDIATE")); |
There was a problem hiding this comment.
Seems like it starts the transaction as write transaction from the start. Without it SQlite would wait to see a write statement before making it a write transaction. That is what I understood from the documentation at least: https://sqlite.org/lang_transaction.html
I'm guessing this would still work if you drop IMMEDIATE.
Same issues we would have with |
Details
Adds a view
journalEntriesto UNION all journal tables so it is easier to write custom queries targeting all journal tablesThe view (click to view it)
The code here should keep the view up to date with all journal tables if the table number ever changes.
Currently this is done in the readdb.sh here: https://github.com/Expensify/Salt/blob/554c37ac5d10959f8d2ae6890a925c50ec614f13/shared_pkgs/bedrock/files/readdb.sh#L309-L355
I think we can drop that special code and param
-jfrom readdb.sh if we create this view.Fixed Issues
Fixes https://github.com/Expensify/Expensify/issues/661721
Tests
journalEntriesview is created correctly in the bedrock dbjournalEntriesview is created correctly in the auth dbInternal Testing Reminder: when changing bedrock, please compile auth against your new changes