feat(compat): add Knex compatibility layer for MySQL and PostgreSQL#174
Merged
Conversation
The file had drifted from the source after the compat-layer, retry, and ORM work landed. Corrected/added: - Default engine documented as 'pg' (was wrongly 'mysql'; client.ts:65) - New sections: Driver Compatibility Layers (compat/pg, compat/mysql2, compat/errors) and Retry Behavior (scale-to-zero wake-up retries) - Config block: added retryOptions and namedPlaceholders - Testing: replaced dead test:integration* scripts with the real test:int:core / :compat / :orm:* suites; listed compat + ORM test files - Dependencies bumped to match package.json (SDK 3.1048, vitest 4, @types/pg) - Key Files tree: added retry.ts, src/compat/, new integration tests, infra/, and the package subpath exports - Project Status: 2.0 beta -> 2.x stable with feature summary
3e90501 to
eacb1c2
Compare
Knex can't take an injected pool like Drizzle/Kysely — it builds its own
driver via Client._driver(). We subclass the mysql2 and pg dialects and
override _driver() to hand Knex a Data API-backed connection, so Knex
keeps full ownership of SQL generation while queries run over the Data API.
- src/compat/knex.ts: createKnexMySQLClient / createKnexPgClient
(lazy-require knex so it stays an optional peer dependency)
- Export both from compat/index.ts; add ./compat/knex subpath export
- knex added as optional peerDependency (^3.0.0)
- Replace the all-skipped knex-mysql.int.test.ts with a real suite and
add knex-pg.int.test.ts (8 live tests each: insert/returning, select,
where, parameterized bindings, update, count, orderBy+limit, delete)
- test:int:orm:knex now runs both engines (+ :pg / :mysql variants)
- README: document Knex usage for both engines
Also fixes a latent compat/pg binding bug surfaced by Knex: the
query({ text, values }, cb) callback form left params defaulting to [],
which clobbered config.values in executeQuery — so parameterized queries
lost their bindings. Default params to undefined so config-embedded
values are honored (client and pool paths).
Limitation: Knex transactions (db.transaction()) are not supported — Knex
issues literal BEGIN/COMMIT/ROLLBACK SQL the Data API can't honor; use the
native client.transaction() instead. Documented in README and as skipped
tests.
Tests: 157 unit + 16 Knex live (2 skipped) + 102 pg compat/ORM regression
all pass. Requires Node 22 (repo's vitest 4).
eacb1c2 to
c28f261
Compare
The merged CLAUDE.md refresh predated the Knex compat layer and said 'Knex is NOT supported'. Update it to reflect compat/knex: - Add a compat/knex bullet to Driver Compatibility Layers - Rewrite ORM support status: Knex supported via _driver() override, both engines tested; keep the db.transaction() limitation note - Fix Project Status and Testing suite lines
The repo's vitest 4 / rolldown toolchain requires Node 20.19+ (Node 18 fails on node:util styleText). CI integration tests already run on Node 20; codify that floor: - engines.node: >=20 (consumer-facing; matches the lowest Node CI tests) - .nvmrc: 20 (contributor default)
6affc1b to
7070d85
Compare
Resolves 3 transitive devDependency advisories (2 moderate, 1 high), all ReDoS/hang issues in tooling deps (eslint/typescript-eslint chain): - ajv 6.12.6 -> 6.15.0 (GHSA-2g4f-4pwh-qvx6) - brace-expansion -> 1.1.15 / 2.1.1 (GHSA-v6h2-p8h4-qcjw, GHSA-f886-m6hf-6m8v) - cross-spawn -> 7.0.6 (GHSA-3xgq-45jj-v275) Dev-only; does not affect the published package (ships dist/ only). package.json unchanged; npm audit now reports 0 vulnerabilities.
Note the shell-persistence gotcha: env vars from `source .env.local` do not carry across separate commands, and nothing auto-loads the file, so it must be chained (`source .env.local && npm run test:int:...`).
ORMs/Knex drive transactions by issuing literal BEGIN/COMMIT/ROLLBACK SQL on the connection. Intercept and map these to the Data API transaction lifecycle, threading transactionId onto subsequent queries. - src/compat/transaction-sql.ts (new): classifyTransactionControl() normalizes a statement (strips trailing ';', collapses whitespace) and recognizes BEGIN/BEGIN TRANSACTION/START TRANSACTION, COMMIT, ROLLBACK, SAVEPOINT/RELEASE/ROLLBACK TO SAVEPOINT, and SET TRANSACTION. - pg compat: replace the inline matcher (which used === 'BEGIN' and missed Knex's 'BEGIN;') with the shared classifier. - mysql2 compat: add the same interception (it previously had only lifecycle methods, never intercepting the SQL strings). - mysql2 compat: fix query() arg-parsing so query(config, undefined, cb) picks up the 3rd-arg callback. Knex sends this form for bindings-less transaction statements; missing it caused transactions to hang. Fixed in all four query parsers (connection + pool, object + string forms). - Nested transactions need SAVEPOINTs, which the Data API has no primitive for, so the classifier throws a clear error; documented and tested. Replace the skipped 'not supported' tests with real commit/rollback/nested tests in both Knex suites (6 new). README and CLAUDE.md updated. Tests: 22/22 Knex (both engines), 157 unit + 223 compat/ORM regression, all pass.
…ence Add query-builder coverage suites for both engines (26 tests each) exercising the common syntax from knexjs.org/guide/query-builder.html: select/distinct/ pluck/first, the where family (whereIn/whereNull/whereBetween/whereExists/ whereRaw/...), joins, groupBy/having/aggregates, orderBy/limit/offset, unions, subqueries, CTEs, insert/update/del, returning (pg), increment/decrement, and onConflict().merge() upserts. Fix surfaced by the CTE tests: inferCommand() returned 'QUERY' for statements starting with WITH, so Knex's pg processResponse didn't treat WITH...SELECT as a SELECT and returned the raw response instead of rows. Resolve the primary verb at parenthesis depth 0 (CTE bodies are nested, so their verbs are skipped). Benefits all pg compat callers doing CTEs, not just Knex. Notes: Knex's whereLike() appends COLLATE utf8_bin which MySQL rejects on utf8mb4 columns (a Knex/MySQL quirk, not a compat issue); the portable where(col,'like',val) form is used. Streaming (.stream()) is unsupported. test:int:orm:knex now runs the query-builder suites too. Tests: 74 Knex (both engines) + 157 unit + pg regression all pass.
Minor bump for the new Knex compatibility layer (compat/knex) with transaction support and query-builder coverage.
- Drop the outdated v2.1.0 'Note' callout; keep only the v1.x pointer at the top - Remove the inline 'Version 2.1/2.0' intro paragraphs (superseded below) - Add a 'What's New in v2.3' section for the Knex compat layer - Collapse v2.2/v2.1/v2.0 into a consolidated Changelog - Move Knex out of the v2.1 entry (it lands in v2.3)
Mention Knex, Drizzle, and Kysely by name in the opening paragraph, and give it a bit more energy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Knex compatibility layer so the Data API Client can back Knex for both MySQL and PostgreSQL, closing the last gap among the supported query builders (Drizzle and Kysely already work). CRUD, the common query-builder surface, transactions, and upserts all work.
Client._driver(). We subclass the mysql2 and pg dialects and override that single method to hand Knex a Data API-backed connection. Knex keeps full ownership of SQL generation; queries execute over the Data API.data-api-client/compat/knexsubpath exportingcreateKnexMySQLClient(config)andcreateKnexPgClient(config).knexis lazy-required and declared as an optional peer dependency (^3.0.0).BEGIN/COMMIT/ROLLBACKSQL Knex issues (src/compat/transaction-sql.ts) and maps it to the Data API transaction lifecycle, threadingtransactionIdonto subsequent queries on that connection.db.transaction()commits on success and rolls back on a thrown callback.distinct/pluck/first, thewherefamily, joins,groupBy/having/aggregates, ordering/pagination, unions, subqueries, CTEs,insert/update/del,returning,increment/decrement, andonConflict().merge()upserts.Bugs fixed along the way
These were latent in the shared compat layers and benefit non-Knex callers too:
compat/pgbinding clobber: thequery({ text, values }, cb)callback form leftparamsdefaulting to[], which overrodeconfig.valuesinexecuteQuery, so parameterized queries using node-postgres's config-object form lost their bindings (bind message supplies 0 parameters).compat/mysql2callback parsing:query(config, undefined, cb)(the form Knex uses for bindings-less statements likeBEGIN;) never picked up the third-argument callback, so the callback never fired and transactions hung. Fixed in all four query parsers.compat/pgCTE command inference:inferCommand()keyed off the first SQL word, soWITH ... SELECTwas taggedQUERYinstead ofSELECT. node-postgres-driven callers (Knex/Kysely/Drizzle) read result shape offcommand, so a CTE returned the raw response object instead of rows. Now resolves the primary verb at parenthesis depth 0.Test Plan
npm run test:unit(157 passed)npm run test:int:orm:knex(74 passed)compat/pgandcompat/mysql2fixes do not break existing callers)tscbuild emitsdist/compat/knex.{js,d.ts}anddist/compat/transaction-sql.js; ESLint cleanLimitations
SAVEPOINTs, which the Data API has no primitive for, so a nestedtrx.transaction(...)throws a clear error. A single top-level transaction works..stream()) is unsupported, since the Data API has no cursor API.whereLike()appendsCOLLATE utf8_bin, which MySQL rejects onutf8mb4columns. This is a Knex/MySQL quirk that affects realmysql2too, not the compat layer; the portablewhere(col, 'like', val)form works.Notes
knex/lib/dialects/{mysql2,postgres}is Knex's documented-but-internal extension point, so a Knex major version could move it, which is why the peer range is^3.0.0.npm audit fixfor three dev-dependency advisories, and a note that.env.localmust be sourced in the same shell command as the test run.