Emit a leaner stack_trace field (emit frames, not the message) - #1675
Draft
samikshya-db wants to merge 1 commit into
Draft
Emit a leaner stack_trace field (emit frames, not the message)#1675samikshya-db wants to merge 1 commit into
samikshya-db wants to merge 1 commit into
Conversation
The telemetry stack_trace error field was populated with the exception message (reason), which routinely embeds user data — SQL text, hostnames, literal filter values — so error telemetry could leak PII. Populate the field with the exception's actual stack frames instead (class, method, file, line via a new TelemetryHelper.formatStackTrace), which are PII-free by construction. exportFailureLog now takes the Throwable, and each driver exception passes `this` so the captured throw-site frames are recorded. error_name/sqlState are unchanged. Add tests asserting the field contains frames and never the message. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
samikshya-db
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The telemetry
stack_traceerror field was populated with the exception message (reason), which despite the field name is not a stack trace at all. Exception messages in the driver routinely embed user data — SQL text, hostnames, literal filter values — so error telemetry could leak PII.This change populates the field with the exception's actual stack frames (
class.method(File:line)) via a newTelemetryHelper.formatStackTrace(Throwable), which are PII-free by construction.exportFailureLognow takes theThrowable, and each driver exception passesthisso the captured throw-site frames are recorded. Frames are capped at 100.error_name/sqlStateand all error codes are unchanged.Affected exceptions:
DatabricksSQLException,DatabricksDriverException,DatabricksTimeoutException,DatabricksTransactionException,DatabricksBatchUpdateException.Testing
TelemetryHelperTest#testFormatStackTraceIncludesFramesButNotMessage, which asserts the formatted field contains code-location frames and never contains the exception message (including a sample PII string).testFormatStackTraceWithNullReturnsNull.exportFailureLogtest call sites to pass aThrowable.Telemetry Errors
DatabricksDriverErrorCodewhere appropriate, and anynew code is uniquely numbered and tested.
requested because the author cannot access the classification.
Additional Notes to the Reviewer
No error names or numeric codes are added or changed — only the content of the existing
stack_tracefield changes (real frames instead of the message string). No telemetry taxonomy/dashboard classification update is required. Any downstream consumer that parsed the message text out ofstack_traceshould be aware it now contains stack frames.Tradeoff: the human-readable exception message is no longer present in telemetry;
error_name+sqlState+ the stack frames carry the debugging signal.NO_CHANGELOG=true