Skip to content

Fix string parameters losing apostrophes when interpolated into SQL - #1674

Open
zajacmp3 wants to merge 2 commits into
databricks:mainfrom
zajacmp3:fix-interpolator-quote-escaping
Open

Fix string parameters losing apostrophes when interpolated into SQL#1674
zajacmp3 wants to merge 2 commits into
databricks:mainfrom
zajacmp3:fix-interpolator-quote-escaping

Conversation

@zajacmp3

@zajacmp3 zajacmp3 commented Sep 3, 2026

Copy link
Copy Markdown

Description

ixed string parameters silently losing apostrophes when supportManyParameters=1 interpolates them into SQL. SQLInterpolator escaped ' as the SQL-standard '', which Databricks does not implement: 'O''Brien' is read as the adjacent literals 'O' and 'Brien' and concatenated, so O'Brien was stored as OBrien. Quotes are not escaped as \', consistent with the backslash escaping already applied to \, \n, \r and \t. Data written through this path before the upgrade still holds the stripped values, so lookups that now send the apostrophe (WHERE name = ? bound to O'Brien) no longer match those rows. Only values written with supportManyParameters=1 are affected; correcting them requires a data fix.

Why this does not change behaviour for anything that worked before

  • Inputs without an apostrophe emit byte-identical SQL. The case '\'' branch is the only thing that
    changed; every other input takes exactly the path it took before. The \, \n, \r, \t and
    supplementary-plane branches, the BINARY/hex-literal path guarded by HEX_LITERAL_PATTERN (SEC-20590),
    NULL handling, TIMESTAMP/DATE quoting and numeric pass-through are all untouched.
  • Inputs with an apostrophe had no correct behaviour to preserve. The previous output was never
    interpreted as the caller intended — the value arrived at the server with its quotes removed. So the change
    is a no-op for inputs that worked and a fix for inputs that did not; there is no third category.
  • Both forms keep the literal balanced, so this is a fidelity fix, not a security fix. '' contributes an
    even number of quotes; \' is a single escaped quote inside one literal. Backslashes in the input are still
    escaped first (\ -> \\), so an input ending in a backslash cannot close the literal early: a\'b emits
    'a\\\'b', which the server reads back as a\'b. Statement structure is unchanged either way; only the
    value the server stores differs.

Testing

Running batch inserts with supportManyParameters=1 against real databricks instance

Telemetry Errors

  • Not applicable — this PR does not add or change a telemetry-visible error.
  • Applicable — the error uses DatabricksDriverErrorCode where appropriate, and any
    new code is uniquely numbered and tested.
  • Applicable — its driver/server/user classification is linked, or maintainer help is
    requested because the author cannot access the classification.

Additional Notes to the Reviewer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant