fix(bay): 改善 SQLite 并发访问 - #27
Conversation
There was a problem hiding this comment.
Code Review
This pull request configures persistent SQLite connections for concurrent tasks by setting a busy timeout, enabling WAL journal mode, and configuring synchronous mode, along with adding a unit test to verify these settings. The review feedback recommends checking if WAL mode is successfully enabled to prevent silent failures on unsupported filesystems, and suggests changing the synchronous mode from FULL to NORMAL to optimize concurrent write performance, with corresponding updates to the unit test.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| def _configure_sqlite_connection(dbapi_connection, _connection_record) -> None: | ||
| """Configure persistent SQLite connections for concurrent Bay tasks.""" | ||
| cursor = dbapi_connection.cursor() | ||
| try: | ||
| cursor.execute(f"PRAGMA busy_timeout={_SQLITE_BUSY_TIMEOUT_MS}") | ||
| cursor.execute("PRAGMA journal_mode=WAL") | ||
| cursor.execute("PRAGMA synchronous=FULL") | ||
| finally: | ||
| cursor.close() |
There was a problem hiding this comment.
在通过 PRAGMA journal_mode=WAL 启用 WAL 模式时,如果数据库文件位于不支持共享内存(mmap)的文件系统(例如 NFS、CIFS/SMB 等网络共享卷,这在 Kubernetes 环境中很常见)或只读文件系统上,SQLite 会静默失败,并返回之前的日志模式(如 delete),而不会抛出异常。
为了避免这种静默失败并便于在容器化/Kubernetes 环境中排查问题,强烈建议检查执行 PRAGMA journal_mode=WAL 后返回的实际模式,并在其不为 wal 时记录警告日志。
此外,对于 WAL 模式,SQLite 官方强烈推荐使用 PRAGMA synchronous=NORMAL(值为 1)而不是 FULL(值为 2)。在 WAL 模式下,NORMAL 已经能够完全保证数据库不会损坏,且仅在极罕见的双重电力故障时可能丢失最后一两个事务。相比之下,FULL 会在每次事务提交时强制同步 WAL 文件到磁盘,这会显著降低并发写入性能,从而增加在高并发场景下触发 database is locked 的概率。
| def _configure_sqlite_connection(dbapi_connection, _connection_record) -> None: | |
| """Configure persistent SQLite connections for concurrent Bay tasks.""" | |
| cursor = dbapi_connection.cursor() | |
| try: | |
| cursor.execute(f"PRAGMA busy_timeout={_SQLITE_BUSY_TIMEOUT_MS}") | |
| cursor.execute("PRAGMA journal_mode=WAL") | |
| cursor.execute("PRAGMA synchronous=FULL") | |
| finally: | |
| cursor.close() | |
| def _configure_sqlite_connection(dbapi_connection, _connection_record) -> None: | |
| """Configure persistent SQLite connections for concurrent Bay tasks.""" | |
| cursor = dbapi_connection.cursor() | |
| try: | |
| cursor.execute(f"PRAGMA busy_timeout={_SQLITE_BUSY_TIMEOUT_MS}") | |
| cursor.execute("PRAGMA journal_mode=WAL") | |
| journal_mode = cursor.fetchone()[0] | |
| if journal_mode.lower() != "wal": | |
| logger.warning( | |
| "sqlite.wal_mode_failed", | |
| requested="WAL", | |
| actual=journal_mode, | |
| detail="WAL mode might not be supported on this filesystem (e.g., network shares).", | |
| ) | |
| cursor.execute("PRAGMA synchronous=NORMAL") | |
| finally: | |
| cursor.close() |
| assert journal_mode == "wal" | ||
| assert synchronous == 2 | ||
| assert busy_timeout == 30_000 |
There was a problem hiding this comment.
修改内容
busy_timeout=30000、journal_mode=WAL和synchronous=FULL。问题原因
Bay 的调度器、垃圾回收任务和 API 请求会并发访问同一个 SQLite 文件。默认回滚日志与较短等待时间容易在写事务重叠时返回
database is locked。验证
1 passed525 passedjournal_mode=wal、synchronous=2、busy_timeout=30000Summary by Sourcery
Improve SQLite configuration for concurrent Bay tasks while keeping other database backends unchanged.
Bug Fixes:
database is lockederrors by increasing connection timeout and configuring write-ahead logging with appropriate sync settings.Enhancements:
connect_argsfrom settings while leaving non-SQLite drivers with existing parameters.Tests: