Repository navigation
feat(semantic): accept Cloud SQL sources, with the dialect read from Knowledge Catalog - #539
Merged
Merged
Conversation
feixufeixu
approved these changes
Oct 21, 2026
feixufeixu
left a comment
feixufeixu
left a comment
Collaborator
There was a problem hiding this comment.
Approving, since the required comments are small. Please apply them before you merge.
merr1988
force-pushed
the
feat/sources/cloudsql
branch
from
October 21, 2026 23:16
5a68c7c to
c645e63
Compare
…Knowledge Catalog Part 5 of b/573288219. A Cloud SQL table or view can be a source, written as its GCP resource name: .../instances/<i>/databases/<db>/tables/<t> for MySQL, or .../databases/<db>/schemas/<s>/tables/<t> for PostgreSQL and SQL Server. The name cannot tell PostgreSQL from SQL Server, so kcmd reads the dialect from the table's entry in the @cloudsql entry group: cloudsql-mysql-table gives MYSQL, cloudsql-postgresql-table POSTGRES and cloudsql-sqlserver-table SQLSERVER, and each matching -view type gives the same dialect. Without a lookup, a Cloud SQL resource name implies no dialect. The commands always look up; code that loads a model without the looked-up dialects gets none for these sources, and every expression text counts. - cloudsql_dialect.ts collects the Cloud SQL resource names in the model and profile files and looks each table up once, in parallel, with lookupEntry. Dataplex answers 403 for a missing entry, a name it cannot read and a misspelled one alike, so the error says to turn on Dataplex integration for the instance, check access, or check the name. - push, kcmd profiles, profiles --print-store and skills-generate look them up before anything loads, and stop on an entry they cannot read. A model with no such source makes no call. - LoadOptions.cloudSqlDialects carries the dialects. The loader and a profile's rebinding record them on the entity, sourcesDialect uses them, and profileForDocument and the completeness check take them. The recorded dialect is never written back. - databaseOf accepts both shapes. For the one-database rule a MySQL or SQL Server table's database is its instance, since one query reaches every database on it, and a PostgreSQL table is in its database. - profiles.md and reference.md document the forms, the entry types, the Dataplex requirement and that kcmd profiles makes the lookup. Bug: b/573288219
merr1988
force-pushed
the
feat/sources/cloudsql
branch
from
October 21, 2026 23:24
c645e63 to
37f2cb6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 5 of b/573288219. A Cloud SQL table or view can be a source, written as its resource name (with or without
//):The name can't tell PostgreSQL from SQL Server, so kcmd reads the dialect from the table's entry in
@cloudsql(
lookupEntry, entry id = resource name):cloudsql-mysql-table→MYSQL,cloudsql-postgresql-table→POSTGRES,cloudsql-sqlserver-table→SQLSERVER, and each matching-viewtype gives the same dialect.Important
No lookup, no dialect. Code that loads a model without the looked-up dialects gets none for these sources, and
every expression text counts. The commands always look up.
push,kcmd profiles(incl.--print-store) andskills-generatelook each Cloud SQL table up once,before anything loads, and pass the dialects to every load. No Cloud SQL resource names, no call.
number, so the error says to turn on Dataplex integration for the instance (with the
gcloud sql instances patchcommand), check access, or check the name, and the command stops.
LoadOptions.cloudSqlDialects→Entity.sourceDialect(set by the loader and by a profile'srebinding) →
sourcesDialect. Never written to a model file or the catalog.database on it (a MySQL name keys the same as
cloudsql_mysql:); a PostgreSQL table is in its database.Verified live with the real
kcmd profiles: MySQL, PostgreSQL and SQL Server tables readMYSQL,POSTGRESandSQLSERVER, and a missing table stopped the command. Notes: the catalog syncs about once a day; a SQL Server Expressdatabase needs
AUTO_CLOSE OFFto sync; entries outlive a deleted instance until the next sync.Limitations:
cloudsql_postgresql:and the resource name of the same database count as two databases (catalognames go away in b/573772860); kcmd trusts the catalog, so a stale entry gives a stale dialect; no
cloudsql_sqlserver:catalog name.Testing:
tscclean; semantic 1866 → 1883, tool 363 → 370, others unchanged (onmainatf024085). Eachplace that carries the dialect fails a test when removed: push applying a profile, the report's legacy-wrapper load,
and a vanilla model's GOOGLE-block profile included.
Bug: b/573288219