Skip to content

feat: prepare client-side delegation and telemetry for pandas-gbq - #17704

Open
shuoweil wants to merge 12 commits into
mainfrom
shuowei-gbq-delegation-telemetry-mvp
Open

feat: prepare client-side delegation and telemetry for pandas-gbq#17704
shuoweil wants to merge 12 commits into
mainfrom
shuowei-gbq-delegation-telemetry-mvp

Conversation

@shuoweil

@shuoweil shuoweil commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Injects pandas-gbq/<version> into the client user-agent when query result conversion to DataFrame / GeoDataFrame is executed and supported by pandas-gbq.

This PR enables backend metrics and telemetry to distinguish between client-side conversion and delegated conversions without altering existing DataFrame output or behavior.

Key Changes

  • Telemetry Injection: Appends pandas-gbq/<version> to ClientInfo.user_agent in RowIterator.to_dataframe() and to_geodataframe().
  • Idempotency: Ensures the user-agent token is not duplicated across multiple calls on the same client connection.
  • Graceful Fallback: Handles missing dependencies or unsupported environments without raising errors.

Fixes #<540939659> 🦕

@shuoweil
shuoweil requested a review from tswast July 13, 2026 21:38
@shuoweil shuoweil self-assigned this Jul 13, 2026
@shuoweil
shuoweil requested review from a team as code owners July 13, 2026 21:38

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces query delegation support to the pandas-gbq package when retrieving dataframes via to_dataframe. It adds version helper utilities to check if delegation is supported, issues a deprecation warning if the core client fallback is used, and delegates the dataframe creation to pandas_gbq when available. The review feedback suggests making the pandas-gbq import and version checks more robust by catching general exceptions rather than just ImportError, using getattr to safely retrieve the package version, and ensuring that pyarrow is skipped in tests when not installed to prevent test failures.

Comment on lines +260 to +268
if self._installed_version is None:
try:
import pandas_gbq # type: ignore

self._installed_version = packaging.version.parse(
getattr(pandas_gbq, "__version__", "0.0.0")
)
except ImportError:
self._installed_version = packaging.version.parse("0.0.0")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues (e.g., compiled extension load failures, AttributeError, or TypeError from broken dependencies). Additionally, packaging.version.parse can raise InvalidVersion if the version string is malformed. Catching Exception ensures a robust fallback to version 0.0.0.

Suggested change
if self._installed_version is None:
try:
import pandas_gbq # type: ignore
self._installed_version = packaging.version.parse(
getattr(pandas_gbq, "__version__", "0.0.0")
)
except ImportError:
self._installed_version = packaging.version.parse("0.0.0")
if self._installed_version is None:
try:
import pandas_gbq # type: ignore
self._installed_version = packaging.version.parse(
getattr(pandas_gbq, "__version__", "0.0.0")
)
except Exception:
self._installed_version = packaging.version.parse("0.0.0")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not want to hide errors for users.

Comment on lines +275 to +283
if self._delegation_api_version is None:
try:
import pandas_gbq # type: ignore

self._delegation_api_version = getattr(
pandas_gbq, "_internal_delegation_api_version", 0
)
except ImportError:
self._delegation_api_version = 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to installed_version, catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues. Catching Exception ensures a robust fallback to 0.

Suggested change
if self._delegation_api_version is None:
try:
import pandas_gbq # type: ignore
self._delegation_api_version = getattr(
pandas_gbq, "_internal_delegation_api_version", 0
)
except ImportError:
self._delegation_api_version = 0
if self._delegation_api_version is None:
try:
import pandas_gbq # type: ignore
self._delegation_api_version = getattr(
pandas_gbq, "_internal_delegation_api_version", 0
)
except Exception:
self._delegation_api_version = 0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not want to hide errors for users.

Comment thread packages/google-cloud-bigquery/google/cloud/bigquery/table.py Outdated
Comment thread packages/google-cloud-bigquery/tests/unit/test_table.py Outdated
@parthea
parthea marked this pull request as draft July 14, 2026 14:11
@parthea

parthea commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Converting to draft until presubmits are green

@shuoweil
shuoweil removed the request for review from tswast July 21, 2026 00:15
@shuoweil
shuoweil marked this pull request as ready for review July 30, 2026 20:44
@shuoweil
shuoweil requested review from GarrettWu, sycai and tswast July 30, 2026 20:45
@shuoweil
shuoweil marked this pull request as draft July 30, 2026 20:46
@shuoweil
shuoweil force-pushed the shuowei-gbq-delegation-telemetry-mvp branch from 5965e97 to 5d80f9a Compare August 4, 2026 00:06
@shuoweil
shuoweil marked this pull request as ready for review August 4, 2026 00:12
@shuoweil
shuoweil requested review from GarrettWu, sycai and tswast August 4, 2026 00:14
@sycai

sycai commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Is this PR a functional duplicate of #17957? They look quite similar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants