Skip to content

test(bigquery): add long-running heavy poller test - #6406

Open
haphungw wants to merge 2 commits into
googleapis:mainfrom
haphungw:bq-job-poller-heavy-test
Open

test(bigquery): add long-running heavy poller test#6406
haphungw wants to merge 2 commits into
googleapis:mainfrom
haphungw:bq-job-poller-heavy-test

Conversation

@haphungw

Copy link
Copy Markdown
Contributor

Add an integration test that proves the JobPoller LRO backoff logic is invoked.

Typical testing queries (e.g., SELECT 1) execute so fast that the initial insert_job REST call fast-paths and returns DONE instantly. This suppresses the inner .poll() loop from back-off sleeping, leaving the LRO logic untested.

Follow-up to #6232

@haphungw
haphungw marked this pull request as ready for review August 14, 2026 20:02
@haphungw
haphungw requested a review from a team as a code owner August 14, 2026 20:02
@product-auto-label product-auto-label Bot added the api: bigquery Issues related to the BigQuery API. label Aug 14, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a new integration test, job_service_poller_heavy, to test the BigQuery job poller with a delayed query. Feedback was provided regarding the use of a tight busy loop in the BigQuery script, which can exceed the statement execution limit and cause test failures; using a temporary JavaScript UDF is recommended instead.

Comment thread tests/bigquery/src/job.rs Outdated
Comment on lines +222 to +233
let query = r#"
DECLARE DELAY_TIME DATETIME;
DECLARE WAIT STRING;
SET WAIT = 'TRUE';
SET DELAY_TIME = DATETIME_ADD(CURRENT_DATETIME, INTERVAL 5 SECOND);

WHILE WAIT = 'TRUE' DO
IF (DELAY_TIME < CURRENT_DATETIME) THEN
SET WAIT = 'FALSE';
END IF;
END WHILE;
"#;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using a tight busy loop in BigQuery scripting (e.g., WHILE WAIT = 'TRUE' DO ...) will quickly exceed the BigQuery scripting limit of 1,000 executed statements per query, causing the test to fail with a Resources exceeded error.

To implement a reliable delay without hitting the statement limit, you can use a temporary JavaScript UDF that performs the busy-wait. This executes as a single statement in BigQuery.

    let query = r#"
        CREATE TEMP FUNCTION wait(ms INT64) RETURNS STRING LANGUAGE js AS '
          const start = Date.now();
          while (Date.now() - start < ms) {}
          return "done";
        ';
        SELECT wait(5000);
    "#;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/datetime_functions#current_datetime - Based on this, I think current datetime will not update over the course of the function so this will not work as expected

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, TIL. I borrowed this from the Python SDK 😢

To make the execution time deterministic, I think we can switch to a massive Cartesian cross-join (which is what the Go team does).

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.18%. Comparing base (6f373a5) to head (4cf63e7).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6406      +/-   ##
==========================================
- Coverage   96.18%   96.18%   -0.01%     
==========================================
  Files         288      288              
  Lines       75424    75424              
==========================================
- Hits        72548    72547       -1     
- Misses       2876     2877       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the BigQuery API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants