Context
We use graphify to build knowledge graphs over a set of GCP data/analytics repos (Terraform infra + BigQuery pipelines). A good chunk of our SQL pipeline logic lives in Google Cloud Dataform, which uses .sqlx files rather than plain .sql.
Right now .sqlx isn't in the supported file type table, so these files are silently skipped during extract (same bucket as e.g. .gitignore — "no supported extension"). For repos where Dataform is the primary way SQL/pipeline logic is defined, that's a meaningful blind spot in the graph.
What a .sqlx file looks like
A Dataform .sqlx file is SQL plus a Dataform-specific config block and templating, e.g.:
config {
type: "table",
schema: "analytics",
tags: ["daily"],
dependencies: ["stg_events"]
}
js {
const cutoffDays = 30;
}
SELECT
user_id,
COUNT(*) AS events
FROM ${ref("stg_events")}
WHERE event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL ${cutoffDays} DAY)
GROUP BY user_id
This is why treating .sqlx as plain SQL (even with the [sql] extra / tree-sitter-sql) doesn't really work — the config {} block and ${ref(...)} / js {} templating aren't valid SQL syntax and would likely break or silently produce nothing useful.
What would be useful to extract
ref("other_model") / ${ref(...)} calls → depends_on edges between Dataform models, similar to how package manifests currently get a canonical node + depends_on edges
- The
config {} block metadata (type, schema, tags, dependencies) as node attributes
- Treating each
.sqlx file as a node representing a Dataform model/table, so it shows up in the graph the same way a .py/.tf file does
Even a first pass that just parses the config {} block and ref()/self() calls (without fully understanding the SQL body) would already make Dataform-heavy repos much more useful to query — that's the same value proposition as the existing [terraform] extra for .tf files.
Suggested approach
Given the pattern already used for other niche formats, an opt-in extra along the same lines as [terraform]/[sql] would make sense, e.g.:
uv tool install "graphifyy[dataform]"
with .sqlx added to the file type table.
Happy to test against a real-world corpus of .sqlx files if that's useful for building/validating an extractor.
Context
We use graphify to build knowledge graphs over a set of GCP data/analytics repos (Terraform infra + BigQuery pipelines). A good chunk of our SQL pipeline logic lives in Google Cloud Dataform, which uses
.sqlxfiles rather than plain.sql.Right now
.sqlxisn't in the supported file type table, so these files are silently skipped duringextract(same bucket as e.g..gitignore— "no supported extension"). For repos where Dataform is the primary way SQL/pipeline logic is defined, that's a meaningful blind spot in the graph.What a .sqlx file looks like
A Dataform
.sqlxfile is SQL plus a Dataform-specific config block and templating, e.g.:This is why treating
.sqlxas plain SQL (even with the[sql]extra / tree-sitter-sql) doesn't really work — theconfig {}block and${ref(...)}/js {}templating aren't valid SQL syntax and would likely break or silently produce nothing useful.What would be useful to extract
ref("other_model")/${ref(...)}calls →depends_onedges between Dataform models, similar to how package manifests currently get a canonical node +depends_onedgesconfig {}block metadata (type,schema,tags,dependencies) as node attributes.sqlxfile as a node representing a Dataform model/table, so it shows up in the graph the same way a.py/.tffile doesEven a first pass that just parses the
config {}block andref()/self()calls (without fully understanding the SQL body) would already make Dataform-heavy repos much more useful to query — that's the same value proposition as the existing[terraform]extra for.tffiles.Suggested approach
Given the pattern already used for other niche formats, an opt-in extra along the same lines as
[terraform]/[sql]would make sense, e.g.:uv tool install "graphifyy[dataform]"with
.sqlxadded to the file type table.Happy to test against a real-world corpus of
.sqlxfiles if that's useful for building/validating an extractor.