Skip to content

testserver: round-trip all catalog fields on create and update - #6158

Open
radakam wants to merge 2 commits into
mainfrom
testserver-catalog-echo-create-fields
Open

testserver: round-trip all catalog fields on create and update#6158
radakam wants to merge 2 commits into
mainfrom
testserver-catalog-echo-create-fields

Conversation

@radakam

@radakam radakam commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Changes

CatalogsCreate now echoes connection_name, custom_max_retention_hours and managed_encryption_settings. CatalogsUpdate now applies custom_max_retention_hours, managed_encryption_settings, options and properties.

Why

RemapState reads all of these back off CatalogInfo, so any field the fake dropped showed up as a change on the next plan and never converged.

Found by fuzz testing.

Tests

New no_drift config catalog_optional_fields.yml.tmpl covers custom_max_retention_hours and managed_encryption_settings.

connection_name and the update path are covered by unit tests only — no_drift deploys once, so no acceptance test exercises the update round-trip.

CreateCatalog accepts connection_name, custom_max_retention_hours and
managed_encryption_settings, but the fake server dropped them from the response
it stored. The CLI then read back a catalog missing fields it had just sent, so
the next plan saw a change that does not exist.
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 079c2e9

Run: 30917670041

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 5 3 591 1066 52:44
💚​ aws windows 5 3 593 1064 68:16
💚​ azure linux 5 3 590 1066 50:21
🔄​ azure windows 2 5 3 590 1064 66:04
💚​ gcp linux 2 4 591 1066 57:31
💚​ gcp windows 2 4 593 1064 77:53
10 interesting tests: 5 RECOVERED, 3 SKIP, 2 flaky
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/invariant/no_drift 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestFsCpFileToDirWithOverwriteFlag ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestFsCpFileToDirWithOverwriteFlag/local_to_uc-volumes ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
11:07 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
10:53 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
10:43 aws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
10:37 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
10:35 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
10:01 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
9:26 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
8:31 gcp windows TestAccept
8:16 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:45 azure windows TestAccept
6:38 aws windows TestAccept
6:38 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
6:19 aws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:17 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:15 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:08 aws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:05 aws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:57 gcp linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
5:56 gcp linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
5:52 gcp windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:52 gcp linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:41 gcp windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
5:37 azure linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:37 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
5:35 gcp linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
5:34 gcp linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
5:33 gcp windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:31 gcp linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:26 gcp linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:24 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
5:24 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:23 gcp linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:19 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:17 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:15 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
5:15 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:14 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
5:13 azure windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
5:09 azure linux TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:09 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
5:00 azure linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
4:59 gcp windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
4:58 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
4:58 azure linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
4:56 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
4:54 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
4:53 azure windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
4:50 azure windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
4:50 aws windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
4:42 azure windows TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=

CatalogsUpdate had the same gap the create handler did: it applied only comment,
owner and new_name, so custom_max_retention_hours, managed_encryption_settings,
options and properties silently reverted to their create-time values. Changing
any of them on a deployed catalog produced drift that never converged.

Add a catalog config to the no_drift invariant so the phantom-change symptom is
guarded where users hit it, and assert in the unit test that the create request
still covers every field CreateCatalog accepts, so a new SDK field fails there
rather than as unexplained drift.
@radakam radakam changed the title testserver: echo back every field catalog create accepts testserver: round-trip all catalog fields on create and update Aug 4, 2026
@radakam
radakam marked this pull request as ready for review August 4, 2026 15:24
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

7 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @andrewnester, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

Files: libs/testserver/catalogs.go, libs/testserver/catalogs_test.go
Based on git history:

  • @denik -- recent work in libs/testserver/, acceptance/bundle/invariant/configs/, acceptance/bundle/invariant/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.
See OWNERS for ownership rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants