Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions docs/docs/development/type-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,30 @@ Pass `--wait` for CI and production builds, where accurate types must be present
npx @databricks/appkit generate-types --wait
```

In blocking mode the generator starts a stopped warehouse, waits (bounded) for it to reach `RUNNING`, and then describes your queries. It fails only when the configured warehouse no longer exists (deleted/deleting), so a transient outage or a cold warehouse degrades gracefully rather than breaking the build. The app template wires this up for you: `postinstall` and `predev` run the non-blocking default, while `prebuild` runs `--wait`.
#### CI resilience: committed types as fallback

In blocking mode (`--wait`), the generator attempts to fetch real types from your warehouse, but delegates to **committed `.d.ts` files** (`shared/appkit-types/analytics.d.ts`, `metric-views.d.ts`) as the fallback when the warehouse is unreachable. These committed files should be part of your repository. On a fresh CI checkout, every build attempts to DESCRIBE against the warehouse; the committed types are used only when that cannot complete.

The generator **never overwrites committed types with degraded (`result: unknown`) types** — it writes real types, or it does not write at all.

A **two-bucket failure taxonomy** determines whether the build crashes or falls back to committed types:

- **Deterministic failures (always crash):** SQL syntax errors in your queries (genuine DESCRIBE failure against a reachable warehouse), HTTP 404 (bad or unknown warehouse ID), HTTP 400 (malformed request). These are developer or configuration errors that committed types must not hide.
- **Environmental failures (gate on committed types):** Authentication failures (401/403), network unreachability, warehouse unavailability (cold, deleting, or deleted), wait timeout on `RUNNING`, or any unrecognized failure. If committed types exist, the build **keeps them, emits a loud warning to stderr, and succeeds (exit 0)**. If no committed types exist, the build **crashes** with a message instructing you to run `npx @databricks/appkit generate-types --wait` locally (against a reachable warehouse) and commit the `.d.ts` files.

The loud warning is a single greppable stderr line naming the coarse cause (auth blocked / warehouse unreachable / warehouse unavailable) and the warehouse ID, so CI logs surface that the build fell back to committed types.

**Note:** If your app declares only metric views and no `config/queries/`, the first build still writes an empty `analytics.d.ts`, which counts as "committed types present" for the gate. An environmental failure will then fall back and warn rather than crash, even on a first build — an accepted v1 simplification.

The app template wires this up for you: `postinstall` and `predev` run the non-blocking default, while `prebuild` runs `--wait`.

## Metric-view types

`generate-types` (and the Vite plugin) emit metric-view types **additively** — there is no separate command. When a `config/metric-views/definitions.json` file is present, the same run that generates your query types also DESCRIBEs each declared [UC Metric View](../plugins/analytics.md) and writes `metric-views.d.ts` into `shared/appkit-types/`:

- `metric-views.d.ts` — augments the `MetricRegistry` interface so `useMetricView('<key>', …)` is autocompleted and type-checked. Each view's measures, dimensions, and their semantic metadata (SQL type, display name, format, time grains) are encoded at the type level.

If `config/metric-views/definitions.json` is absent the metric path stays dormant (nothing is emitted). When present it follows the **same** warehouse-readiness contract as query types: in the default non-blocking run a view that can't be described yet — a cold warehouse, or a bad/unreachable source — is written with permissive types and a warning, while under `--wait` that same situation fails the build so CI never ships incomplete metric types. A malformed `definitions.json` (invalid JSON, or a source that isn't a three-part UC FQN) fails fast in every mode.
If `config/metric-views/definitions.json` is absent the metric path stays dormant (nothing is emitted). When present it follows the **same** warehouse-readiness contract as query types: in the default non-blocking run a view that can't be described yet — a cold warehouse, or a bad/unreachable source — is written with permissive types and a warning, while under `--wait` metric views obey the [two-bucket taxonomy](#ci-resilience-committed-types-as-fallback) (environmental failures gate to committed `metric-views.d.ts` + warn; deterministic failures like malformed definitions crash the build). A malformed `definitions.json` (invalid JSON, or a source that isn't a three-part UC FQN) fails fast in every mode.

`definitions.json` is keyed by metric key; each entry names the three-part UC FQN of the view and, optionally, the executor it runs as (`app_service_principal`, the default, or `user`):

Expand Down
88 changes: 88 additions & 0 deletions packages/appkit/src/type-generator/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,91 @@ export function isConnectivityError(error: unknown): boolean {

return false;
}

const AUTH_ERROR_STATUSES = new Set([401, 403]);

/**
* Classifies a thrown failure into one of two buckets: deterministic failures
* that must be surfaced (bad warehouse id, malformed request) or environmental
* issues (connectivity, auth, deleted warehouse, timeouts) that the has-types
* gate will handle later.
*
* Returns:
* - "deterministic": HTTP 404 (bad warehouse id) or 400 (malformed request).
* The build must fail.
* - "environmental": Everything else — auth (401/403), connectivity errors,
* warehouse state changes (DELETED/DELETING), wait-for-RUNNING timeouts,
* unrecognized failures. Default = environmental.
*
* Walks `cause`/`AggregateError` chains when checking for deterministic status,
* so a wrapped 404 is still recognized as deterministic.
*/
export function classifyBlockingFailure(
error: unknown,
): "deterministic" | "environmental" {
// Deterministic: check first so they're never swallowed by environmental rules.
// Walk the error chain to find any deterministic status.
const seen = new Set<unknown>();
const stack = [error];

while (stack.length > 0) {
const current = stack.pop();
if (current === undefined || seen.has(current)) continue;
seen.add(current);

const status = getErrorStatus(current);
if (status === 404 || status === 400) {
return "deterministic";
}

stack.push(...getErrorChildren(current));
}

// Environmental: auth, connectivity, unrecognized, default.
const topLevelStatus = getErrorStatus(error);
if (topLevelStatus !== undefined && AUTH_ERROR_STATUSES.has(topLevelStatus)) {
return "environmental";
}

if (isConnectivityError(error)) {
return "environmental";
}

// Default: any unrecognized failure or no status (DELETED/DELETING messages,
// timeout messages, plain Error objects) → environmental.
return "environmental";
}

/**
* Coarse cause label for an environmental failure, used by the `--wait`
* committed-types warning so the log says *why* generation fell back.
*
* Returns:
* - "unreachable": transport/connectivity failure (see {@link isConnectivityError}).
* - "auth": HTTP 401/403, including a status carried on `response.status` or
* wrapped in a `cause`/`AggregateError` chain.
* - "unavailable": everything else (DELETED/DELETING, wait timeouts, degraded
* DESCRIBEs).
*/
export function classifyEnvironmentalCause(
error: unknown,
): "auth" | "unreachable" | "unavailable" {
if (isConnectivityError(error)) return "unreachable";

// Walk the error chain so a wrapped 401/403 is still labeled as auth.
const seen = new Set<unknown>();
const stack = [error];

while (stack.length > 0) {
const current = stack.pop();
if (current === undefined || seen.has(current)) continue;
seen.add(current);

const status = getErrorStatus(current);
if (status !== undefined && AUTH_ERROR_STATUSES.has(status)) return "auth";

stack.push(...getErrorChildren(current));
}

return "unavailable";
}
Loading
Loading