Skip to content

[CF-4208] Add --page-size to on-prem Flink list commands - #3424

Draft
Paras Negi (paras-negi-flink) wants to merge 2 commits into
mainfrom
cli-cf-4208-flink-list-limit-filter
Draft

[CF-4208] Add --page-size to on-prem Flink list commands#3424
Paras Negi (paras-negi-flink) wants to merge 2 commits into
mainfrom
cli-cf-4208-flink-list-limit-filter

Conversation

@paras-negi-flink

@paras-negi-flink Paras Negi (paras-negi-flink) commented Jul 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Release Notes

New Features

  • Added a --page-size flag to the on-prem (Confluent Platform / CMF) confluent flink ... list commands, to fetch large result sets in fewer, larger API requests. Defaults to 100 — existing behavior is unchanged.

Checklist

  • I have successfully built and used a custom CLI binary, without linter issues from this PR.
  • I have clearly specified in the What section below whether this PR applies to Confluent Cloud, Confluent Platform, or both.
  • I have attached manual CLI verification results in the Test & Review section below.
  • I have added appropriate CLI integration or unit tests for any new or updated commands and functionality.
  • I confirm that this PR introduces no breaking changes or backward compatibility issues.
  • I have indicated the potential customer impact if something goes wrong in the Blast Radius section below.
  • I have put checkmarks below confirming that the feature associated with this PR is enabled in:
    • Confluent Cloud prod
    • Confluent Cloud stag
    • Confluent Platform
    • Check this box if the feature is enabled for certain organizations only

What

Confluent Platform (CMF on-prem) only — part of CF-4208; Confluent Cloud flink commands are untouched.

On-prem Flink list commands paginate at a hardcoded size=100 and always fetch every page, so large environments cost many serial round trips (~51 requests for 5000 applications — CF-4202).

  • Adds --page-size to all paginated on-prem list commands (application / event / instance / catalog / catalog database / compute-pool / detached-savepoint / environment / savepoint / secret / secret-mapping / statement list). Shared addPageSizeFlag/getPageSize in command.go validate 0..MaxInt32; 0 = the default 100.
  • Refactors the 12 duplicated pagination loops in pkg/flink/cmf_rest_client.go into one generic listAllPages helper (net −84 lines). statement exception list is untouched (non-paginated).

Blast Radius

  • Additive optional flag; --page-size 0 reproduces today's output exactly. Non-breaking, easy to revert.
  • The shared pagination refactor is guarded by a listAllPages unit test plus multi-page integration coverage.

References

Test & Review

  • Unit TestListAllPages: default / custom / large size (fewer round trips), size > total, empty, error propagation.
  • Integration: --page-size on application / statement / detached-savepoint list (full list fetched across multiple pages) + invalid --page-size -1; help goldens regenerated. make lint clean.
$ confluent flink application list --environment default --page-size -1
Error: `--page-size` must be between 0 and 2147483647

# small or large page size -> identical output, just more/fewer round trips
$ confluent flink application list --environment default --page-size 2
          Name          | Environment |     Job Name      | Job Status
------------------------+-------------+-------------------+--------------
  default-application-1 | default     | State machine job | RECONCILING
  default-application-2 | default     | State machine job | RECONCILING
  default-application-s | default     | State machine job | RECONCILING

Copilot AI review requested due to automatic review settings July 31, 2026 15:26
@confluent-cla-assistant

confluent-cla-assistant Bot commented Jul 31, 2026

Copy link
Copy Markdown

❌ Error getting contributor login(s).
Please ensure the email address associated with this commit is added to your Github account.

Copilot AI reviewed Jul 31, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR closes CF-4208 by adding user-facing pagination control (--limit) across on-prem Flink “list” commands and introducing flink application list filtering (--name, --status). It also refactors the CMF REST client to centralize paging behavior (including limit short-circuiting) and updates test server behavior plus integration fixtures accordingly.

Changes:

  • Add a shared --limit flag (default 0 = unlimited) to on-prem Flink list commands, applying it server-side where endpoints are paginated and client-side for non-paginated exception listing.
  • Add --name / --status filtering to flink application list by composing CMF’s generic filter query parameter.
  • Refactor duplicated CMF paging loops into a generic listAllPages helper and add unit + integration coverage (with updated goldens).

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
test/test-server/flink_onprem_handler.go Adds CMF applications filter query emulation in the test server.
test/flink_onprem_test.go Adds integration coverage for --limit and application list filtering flags.
test/fixtures/output/flink/statement/list-limit-json.golden New golden output for statement list with --limit.
test/fixtures/output/flink/statement/list-help-onprem.golden Updates help golden to include --limit.
test/fixtures/output/flink/statement/list-env-missing-failure.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/statement/exception/list-help-onprem.golden Updates help golden to include --limit for exception list.
test/fixtures/output/flink/secret/list-help-onprem.golden Updates help golden to include --limit for secret list.
test/fixtures/output/flink/secret-mapping/list-help-onprem.golden Updates help golden to include --limit for secret-mapping list.
test/fixtures/output/flink/savepoint/list-help-onprem.golden Updates help golden to include --limit for savepoint list.
test/fixtures/output/flink/savepoint/list-fail-both.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/environment/list-help-onprem.golden Updates help golden to include --limit for environment list.
test/fixtures/output/flink/detached-savepoint/list-limit-json.golden New golden output for detached-savepoint list with --limit.
test/fixtures/output/flink/detached-savepoint/list-help-onprem.golden Updates help golden to include --limit.
test/fixtures/output/flink/compute-pool/list-missing-env-flag-failure.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/compute-pool/list-help-onprem.golden Updates help golden to include --limit.
test/fixtures/output/flink/catalog/list-help-onprem.golden Updates help golden to include --limit.
test/fixtures/output/flink/catalog/database/list-help-onprem.golden Updates help golden to include --limit.
test/fixtures/output/flink/application/list-status-no-match-json.golden New golden output for application list status filter no-match case.
test/fixtures/output/flink/application/list-status-filter-json.golden New golden output for application list status filter match case.
test/fixtures/output/flink/application/list-name-filter-json.golden New golden output for application list name filter case.
test/fixtures/output/flink/application/list-limit-json.golden New golden output for application list with --limit.
test/fixtures/output/flink/application/list-limit-invalid.golden New golden output for invalid --limit value.
test/fixtures/output/flink/application/list-help-onprem.golden Updates help golden to include --name, --status, and --limit.
test/fixtures/output/flink/application/list-env-missing.golden Updates env-missing output to include new flags in help text.
test/fixtures/output/flink/application/instance/list-help-onprem.golden Updates help golden to include --limit for instance list.
test/fixtures/output/flink/application/instance-list-env-missing.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/application/instance-list-app-missing.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/application/event/list-help-onprem.golden Updates help golden to include --limit for event list.
test/fixtures/output/flink/application/event-list-missing-flags.golden Updates failure/help golden to include --limit.
test/fixtures/output/flink/application/event-list-app-missing.golden Updates failure/help golden to include --limit.
pkg/flink/test/mock/cmf_client_mock.go Updates generated CMF client mock for the new ListStatements(..., limit) signature.
pkg/flink/cmf_rest_client.go Refactors paging into listAllPages and threads limit through CMF list calls.
pkg/flink/cmf_rest_client_test.go Adds unit tests for listAllPages behavior across limits and paging scenarios.
internal/flink/command.go Adds shared --limit flag registration and parsing helper (getLimit).
internal/flink/command_statement_list_onprem.go Wires --limit into on-prem statement list via CMF client.
internal/flink/command_statement_exception_list_onprem.go Adds --limit and truncates exceptions client-side for non-paginated endpoint.
internal/flink/command_secret_mapping_list.go Wires --limit into secret-mapping list.
internal/flink/command_secret_list.go Wires --limit into secret list.
internal/flink/command_savepoint_list.go Wires --limit into savepoint list.
internal/flink/command_environment_list.go Wires --limit into environment list.
internal/flink/command_detached_savepoint_list.go Wires --limit into detached-savepoint list.
internal/flink/command_compute_pool_list_onprem.go Wires --limit into compute-pool list.
internal/flink/command_catalog_list.go Wires --limit into catalog list.
internal/flink/command_catalog_database_list.go Wires --limit into catalog database list.
internal/flink/command_application_list.go Adds --name/--status filtering and wires --limit into application list.
internal/flink/command_application_instance_list.go Wires --limit into application instance list.
internal/flink/command_application_event_list.go Wires --limit into application event list.
Files not reviewed (1)
  • pkg/flink/test/mock/cmf_client_mock.go: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/flink/command.go Outdated
Comment thread pkg/flink/cmf_rest_client_test.go Outdated
Paras Negi @claude
[CF-4208] Add --page-size to on-prem Flink list commands
1ecbfcb 
The on-prem Flink (CPF/CMF) list commands loop over a hardcoded 100-item
page size with no way to control it, so listing a large environment costs
many serial round trips (~51 requests for 5000 applications). Add a
--page-size flag to every paginated on-prem list command so callers can
fetch the full list in fewer, larger requests. It defaults to 100, which
preserves today's behavior and output.

Collapse the 12 duplicated pagination loops in the CMF client into a single
generic listAllPages helper that takes the page size. The requested size is
passed through to the CMF page/size API; the loop still terminates on the
first empty page.

Server-side filtering for `application list` is deliberately left out and
will follow in a separate PR stacked on this one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@airlock-confluentinc
airlock-confluentinc Bot force-pushed the cli-cf-4208-flink-list-limit-filter branch from bba9ac2 to 1ecbfcb Compare August 2, 2026 08:38
@paras-negi-flink Paras Negi (paras-negi-flink) changed the title [CF-4208] Add --limit pagination and application list filtering to on-prem Flink [CF-4208] Add --page-size to on-prem Flink list commands Aug 2, 2026
@paras-negi-flink Paras Negi (paras-negi-flink) mentioned this pull request Aug 2, 2026
Draft
11 tasks
Paras Negi @claude
[CF-4208] Clarify --page-size help, cover the upper bound, drop tauto…
600dd7d 
…logical pagination cases

- Reword the --page-size help so it reads as a per-request batch size, not a
  cap on total results (regenerate the affected help/usage goldens).
- Add an integration case for --page-size > MaxInt32 (3000000000), exercising the
  previously untested upper-bound branch of getPageSize (reuses the invalid golden).
- Drop the statement/detached-savepoint --page-size cases: those mock handlers
  ignore the size param, so the cases only verified flag plumbing (already covered
  by the help goldens) rather than pagination. The applications case (mock genuinely
  pages) plus TestListAllPages remain the real coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@paras-negi-flink