diff --git a/spec/language.md b/spec/language.md new file mode 100644 index 000000000..585c50eb3 --- /dev/null +++ b/spec/language.md @@ -0,0 +1,508 @@ +# The OpenUI Language + +**1.0-beta, community review draft** + +The exact rules of OpenUI Lang. Start with [overview.md](./overview.md) for the guided tour; [prompt.md](./prompt.md) covers what is sent to the model. MUST, MUST NOT, SHOULD, and MAY are used as in RFC 2119, and *(proposed)* marks designed but unshipped behavior. + +Three words used throughout: the **generator** is whatever writes OpenUI Lang, in practice a model. The **client** parses and renders it. The **host** is the application embedding the client. + +## 1. Grammar + +### 1.1 Notation + +The grammar is given in EBNF. `=` defines a production, `|` separates alternatives, `[ ]` marks an optional part, `{ }` marks zero or more repetitions, `( )` groups, and terminals appear in double quotes. Lexical tokens (`identifier`, `state_name`, `string`, `number`, `newline`) are defined in prose with patterns in section 1.3; syntactic productions build on them. The complete grammar is collected in section 1.6. + +### 1.2 Source text + +Before parsing, the input text is preprocessed in order: + +1. **Fence extraction.** If the input contains fenced code blocks (three backticks, with or without a language tag), the contents of all fences are extracted and joined with newlines; text outside fences is ignored by the parser (in inline mode, where the model mixes prose with fenced code, it is treated as prose for the conversation). Backtick sequences inside double-quoted strings do not open or close fences; single-quoted strings do not shield them. Comment stripping runs after extraction, so backtick runs inside comments do open and close fences. An unterminated fence extends to the end of the input, which keeps extraction stable while a fence is still streaming. *(proposed)* A host that adopts the multi-library segments engine ([overview.md](./overview.md), section 4.6) supersedes this joined extraction: only fences whose info string starts with `openui-lang` are programs, each fence is its own program, and a fence tagged `text` stays prose. Single-segment hosts keep the joined behavior above. +2. **Comment stripping.** `//` and `#` begin a comment that runs to the end of the line. Comment markers inside string literals are not comments, and string state carries across lines, so a marker inside a multi-line string is preserved. +3. Leading and trailing whitespace of the whole extracted text is trimmed. + +### 1.3 Lexical elements + +#### Identifiers + +Identifiers match `[a-zA-Z_][a-zA-Z0-9_]*`. The first letter carries meaning: an identifier starting with an uppercase letter is a component name; starting with a lowercase letter or underscore, a reference. + +```ebnf +identifier = ( letter | "_" ) { letter | digit | "_" } ; +state_name = "$" identifier ; +function_name = "@" identifier ; +``` + +`$` followed by an identifier is a state variable; the `$` is part of the name. `@` followed by an identifier is a built-in or registered function call. The `$` and `@` prefixes are reserved: clients MUST NOT assign other meanings to them. + +#### Keywords + +The keyword literals are `true`, `false`, and `null`. There are no other keywords. + +#### Predeclared and reserved names + +The full set of names the language claims for itself: + +| Group | Names | +| --- | --- | +| Keyword literals | `true`, `false`, `null` | +| Reserved call forms | `Query`, `Mutation`, `Action` | +| Built-in functions | `@Count`, `@First`, `@Last`, `@Sum`, `@Avg`, `@Min`, `@Max`, `@Filter`, `@Sort`, `@Round`, `@Abs`, `@Floor`, `@Ceil`, `@Each` | +| Action steps | `@Set`, `@Reset`, `@Run`, `@ToAssistant`, `@OpenUrl`, declared actions *(proposed)* | +| State prefix | every `$name` | + +`Query` and `Mutation` are statement forms and `Action` is an expression form (section 2.1); none of the three is a component. Built-in semantics are in section 3.4, action steps in section 6.3. Registered functions *(proposed)* extend the `@` namespace; libraries MUST NOT shadow built-ins or define components with reserved names ([prompt.md](./prompt.md), library rules). + +#### Operators and punctuation + +```text += == != > < >= <= + - * / % && || ! ? : . , ( ) [ ] { } +``` + +A single `&` or `|` is accepted with the meaning of `&&` or `||`. + +#### String literals + +Strings are double-quoted or single-quoted. Double-quoted strings use JSON escape sequences (`\n`, `\t`, `\"`, `\\`, `\uXXXX`). Single-quoted strings support only `\'`, `\\`, `\n`, `\t`; any other escaped character is kept as the bare character. An unterminated string at the end of a streaming buffer is closed implicitly (section 4). + +#### Number literals + +Numbers match `-?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?`. An `e` or `E` after the digits is consumed as an exponent marker even when no digits follow it; the resulting token is not a usable number. A `-` starts a number literal only when the previous token is not a value (not a literal, reference, closing parenthesis, closing square bracket, or call; a closing brace does not count) and the next character is a digit; otherwise it is the subtraction or negation operator. With optional commas this matters: `[1 -2]` is one element, the subtraction `1 - 2`; a generator that means two elements writes the comma. A `.` is part of a number only when a digit follows; `1.` is the number `1` followed by member access. + +#### Whitespace and other characters + +Spaces and tabs separate tokens. The `newline` token is `\n`; an immediately preceding `\r` is consumed with it, so CRLF input lexes identically. A newline ends a statement when it occurs at bracket depth zero (section 1.4). Any character that matches no rule is skipped as if it were whitespace (`a;b` lexes as the two tokens `a`, `b`); lexing never fails. + +### 1.4 Statements + +A program is a sequence of statements: + +```ebnf +program = statement { newline statement } ; +statement = ( identifier | state_name ) "=" expression ; +``` + +The left side is a reference name, a component name, or a state variable. A line that does not have this shape is skipped without error in the default mode; in strict mode *(proposed)* it MUST produce an `invalid-statement` diagnostic. + +A statement continues past a newline in three cases: + +1. The newline occurs inside unclosed `(`, `[`, or `{`. +2. The newline occurs inside an unterminated string literal; strings may contain raw newlines. +3. The newline occurs while a ternary is open at bracket depth zero, or the first non-whitespace token of the following line is `?`. This lets a ternary span lines. A ternary is **open** from its `?` until the first complete operand after its matching `:` has been consumed; a newline arriving while either branch is still empty continues the statement. + +These three rules are the complete definition of a statement boundary. Both the batch parser and the streaming parser MUST agree on them byte for byte. Because rule 3 depends on the following line, a statement boundary at a newline is provisional during streaming: the statement is complete only once the first non-whitespace character of the next line has arrived and is known not to be `?` (section 4, rule 1). + +### 1.5 Expressions + +#### Operands + +```ebnf +primary = literal | array | object | call | reference | "(" expression ")" ; +literal = string | number | "true" | "false" | "null" ; +array = "[" [ arguments ] "]" ; +object = "{" [ key ":" expression { [ "," ] key ":" expression } ] "}" ; +reference = identifier | state_name ; +``` + +Object keys may be names, strings, numbers, component names, or `$`-prefixed names (the `$` is stripped), and are read as strings; any other token is read as the key `?`. `name` denotes a lowercase-initial identifier; `ComponentName` an uppercase-initial one. A bare identifier of either case in operand position is a reference: hoisting is case-blind, and only the call-head position gives an uppercase identifier component meaning. + +Inside an expression, `$name = expression` parses as a binding assignment, the form the runtime uses for two-way binding. Generators are not taught this form and SHOULD NOT produce it. + +#### Calls + +```ebnf +call = ( ComponentName | function_name ) "(" [ arguments ] ")" ; +arguments = expression { [ "," ] expression } ; +``` + +Commas between call arguments, array elements, and object entries are optional separators, and trailing commas are ignored. `function_name` is defined in section 1.3; the identifier after `@` is conventionally uppercase, matching the built-ins, and the grammar accepts either case. Built-ins require the `@` prefix: `Count(x)` without `@` is not a call, it parses as the bare reference `Count`, and the arguments are lost. Generators are taught the prefix; clients SHOULD surface a diagnostic *(proposed)* when an uppercase call matches a built-in name without `@`. + +#### Member access and indexing + +```ebnf +postfix = primary { "." field | "[" expression "]" } ; +``` + +Member-access fields accept the same token set as object keys. Only component names and `@` names head calls; a member access is never callable (`foo.bar(x)` is not a call form). + +#### Operators and precedence + +Precedence from lowest to highest; binary operators are left-associative: + +1. `? :` (ternary, right-associative) +2. `||` +3. `&&` +4. `==` `!=` +5. `>` `<` `>=` `<=` +6. `+` `-` +7. `*` `/` `%` +8. unary `!` and `-` (they stack: `!!x` is valid) +9. postfix: member access and indexing (calls are primary forms, not postfix operators) + +### 1.6 Complete grammar + +```ebnf +program = statement { newline statement } ; +statement = ( identifier | state_name ) "=" expression ; +expression = ternary ; +ternary = or [ "?" ternary ":" ternary ] ; +or = and { "||" and } ; +and = equality { "&&" equality } ; +equality = comparison { ( "==" | "!=" ) comparison } ; +comparison = additive { ( ">" | "<" | ">=" | "<=" ) additive } ; +additive = multiplicative { ( "+" | "-" ) multiplicative } ; +multiplicative = unary { ( "*" | "/" | "%" ) unary } ; +unary = { "!" | "-" } postfix ; +postfix = primary { "." field | "[" expression "]" } ; +primary = literal | array | object | call | reference + | "(" expression ")" ; +call = ( ComponentName | function_name ) "(" [ arguments ] ")" ; +arguments = expression { [ "," ] expression } ; +array = "[" [ arguments ] "]" ; +object = "{" [ key ":" expression { [ "," ] key ":" expression } ] "}" ; +literal = string | number | "true" | "false" | "null" ; +reference = identifier | state_name ; +``` + +The lexical tokens and the token sets for `key` and `field` are defined in sections 1.3 and 1.5. + +A few corners are implementation-defined until the conformance fixtures pin them: trailing tokens after a complete expression in one statement, the value of a number token with a dangling exponent, partially received unicode escapes under implicit closing (section 4), and whether an unchanged query re-fetches after a merge (section 7). + +## 2. Program model + +### 2.1 Statement kinds + +Each statement is classified by its right side, in this order: + +1. `Query(...)` at the top level: a **query statement**. +2. `Mutation(...)` at the top level: a **mutation statement**. +3. Left side is a `$` variable: a **state declaration**; the right side is its default value. +4. Anything else: a **value statement** (components, literals, derived expressions). + +The order matters: `$x = Query(...)` is a query whose id is `$x`, not a state declaration. `Query` and `Mutation` are valid only as the entire right side of a statement. Two terms recur in the rules that follow: a **value position** is a direct argument of a component call or a direct element of an array literal, before any operator applies; a **computed expression** is any other expression context (an operand of an operator, a ternary branch, an object value). `Query` and `Mutation` used in a value position produce an `inline-reserved` error and evaluate to nothing; nested inside a computed expression they evaluate to null, currently without the error. `Action(...)` is an ordinary expression, not a statement kind: it may appear inline in a component argument or be bound to its own statement and referenced from one. + +### 2.2 Entry resolution + +A program's entry SHOULD be a statement named `root` whose value is a single component call, and generated prompts teach exactly that form. The fallback chain below exists as error recovery for a generator that forgot; it is not an alternative authoring style. + +When no statement is named `root`, clients MUST recover by choosing the entry in this order, so that recovery renders identically everywhere: + +1. The statement whose name equals the library's root component name. +2. The first value statement whose component call matches the library's root component. +3. The first value statement that is a component call. +4. The first statement. + +If the chosen entry does not resolve to a component, nothing renders and the client reports it once the stream is complete (a dedicated `no-root` code is proposed; the reference client currently reports `parse-failed`). During streaming, an absent root is not an error; the program may not have arrived yet. + +*(proposed)* 1.0-beta simplifies recovery to a pure function of the program text: steps 1 and 2 are dropped, so the chosen entry never depends on which library renders the text, and the fallback is the first value statement that is a component call. A program recovered this way renders and reports a non-fatal `no-root`, teaching the generator to name its entry. A program with no component statement at all reports `no-root` with nothing rendered, and a `root` bound to a non-component recovers the same way, with a hint that `root` must be a component. + +*(proposed)* `createLibrary` requires the `root` field, and it accepts one component name or several (`root: ["Card", "Dashboard"]`), letting the model pick the right top level per request. The field guides prompt generation only; under the simplified recovery above, entry resolution never consults it, so what renders is always decided by the program text. + +### 2.3 Reference resolution and hoisting + +A bare name in an expression refers to the statement with that name, wherever it appears in the program. Statements form a graph, not a sequence. Resolution rules: + +- A reference to a statement that does not exist is **unresolved**. Unresolved references are recorded in the parse metadata, not reported as errors, because during streaming they usually mean "not yet". +- An unresolved reference evaluates as null in an expression and renders nothing in a component position. During streaming, clients SHOULD NOT surface required-prop errors caused only by unresolved references; when the stream ends, the rules of section 8.4 apply. +- A reference that participates in a cycle is unresolved at the point that closes the cycle. +- A statement referenced from two places is evaluated independently at each site. Shared references are copies, not instances; copies duplicate only the tree, and the single page state store is shared. Runtime results are the exception: a query referenced twice still fetches once, and both sites read the same result. +- References to query or mutation statements resolve to their runtime results (section 6), not to the statements themselves. + +Value statements not reachable from the entry are **orphans**, reported in parse metadata and not rendered. State, query, and mutation statements are never reported as orphans, and a query statement executes whether or not it is reachable from the entry (section 6.1). + +### 2.4 Duplicate names + +If two statements bind the same name, the later one wins. During streaming, a pending parse never replaces a completed statement (section 4, rule 3). One exception to later-wins: when a mutation and a query bind the same name, references to that name resolve to the mutation result (section 6.2). The names the language reserves are listed in section 1.3; the rules libraries must follow around them live in [prompt.md](./prompt.md). + +## 3. Evaluation + +### 3.1 Values and coercion + +Values are strings, numbers, booleans, null, arrays, objects, and component instances. Numeric coercion (`toNumber`) is used by arithmetic and comparisons: a number is itself; a string converts via number parsing, or 0 if it does not parse, so NaN never enters the value domain; `true` is 1 and `false` is 0; everything else is 0. Truthiness follows JavaScript ToBoolean; arrays, objects, and component instances are truthy. A component instance in an expression behaves like an object: `toNumber` 0, truthy, member access yields null. + +For cross-platform determinism, string-to-number parsing, loose equality, and number-to-string formatting follow the JavaScript algorithms (ECMA-262 ToNumber, IsLooselyEqual, and Number::toString); non-JavaScript clients MUST reproduce them, and the fixture suite exercises them. + +### 3.2 Operators + +- `+`: if either operand is a string, string concatenation, with null and undefined becoming the empty string (so `"Total: " + missing` is `"Total: "`, never `"Total: null"`). Otherwise numeric addition via `toNumber`. +- `-`, `*`: numeric via `toNumber`. +- `/`, `%`: numeric via `toNumber`; when the divisor is 0 the result is 0, not an error and not Infinity. +- `==`, `!=`: loose equality per IsLooselyEqual (`5 == "5"` is true). +- `>`, `<`, `>=`, `<=`: both sides via `toNumber`. +- `&&`, `||`: short-circuit and return the deciding operand's value, as in JavaScript. +- `!`: negates truthiness. Unary `-` negates `toNumber` of its operand. +- `cond ? a : b`: chooses by truthiness of `cond`. + +A value of null in a component position renders nothing. This makes `cond ? panel : null` the conditional rendering idiom. Strings, numbers, and booleans in a component position render as text. + +### 3.3 Member access and pluck + +`a.b` on an object reads the field. `a.b` on an array plucks: it produces the array of `b` values of every element, with null for elements that lack the field. As a special case, `.length` on an array is its element count. Member access on null is null. `a[i]` indexes arrays by number and objects by string key; a null object or index gives null. + +### 3.4 Built-in function reference + +| Built-in | Signature | Semantics | +| --- | --- | --- | +| `@Count` | `(array) → number` | Element count; 0 for non-arrays. | +| `@First`, `@Last` | `(array) → value` | First or last element; null for empty or non-arrays. | +| `@Sum` | `(array) → number` | Sum via `toNumber`; 0 for non-arrays. | +| `@Avg` | `(array) → number` | Mean via `toNumber`; 0 for empty or non-arrays. | +| `@Min`, `@Max` | `(array) → number` | Minimum or maximum via `toNumber`; 0 for empty or non-arrays. | +| `@Filter` | `(array, field, op, value) → array` | Keeps elements whose `field` satisfies `op value`. Ops: `==`, `!=` (loose), `>`, `<`, `>=`, `<=` (numeric), `contains` (case-sensitive substring of the stringified field). An empty or absent `field` tests the elements themselves; an absent op defaults to `==`; a present but unrecognized op matches nothing. Empty array for non-arrays. | +| `@Sort` | `(array, field, direction?) → array` | Stable sort on `field` (empty string sorts the elements themselves). Pairs where both sides are numeric compare numerically; otherwise as locale-compared strings, so identical programs can sort differently across platforms until the pinned collation lands (proposed: code-point order, fixed by the fixture suite). Descending only when the third argument is `"desc"`. Returns the input unchanged for non-arrays. | +| `@Round` | `(number, decimals?) → number` | Round to `decimals` places, default 0, with JavaScript `Math.round` semantics (half rounds toward positive infinity). | +| `@Abs`, `@Floor`, `@Ceil` | `(number) → number` | Via `toNumber`. | +| `@Each` | `(array, varName, template) → array` | Section 3.5. | + +Field arguments accept dot paths (`"customer.name"`). + +### 3.5 Iteration with `@Each` + +`@Each(array, varName, template)` evaluates `template` once per element with `varName` bound to the element. The loop variable exists only inside the template and shadows a statement of the same name there; a nested `@Each` reusing the same `varName` shadows the outer binding. No index variable is provided. The element's value is substituted into the template before deferred evaluation, so an action inside the template captures the element it was created with: + +```openui-lang +rows = @Each(tickets, "t", Row(t.title, Button("Close", Action([@Run(close), @Set($selected, t.id)])))) +``` + +### 3.6 Purity requirements + +Registered functions and validators MUST be pure and deterministic. The runtime MAY cache their results and MAY invoke them any number of times in any order. Anything that needs application state or produces effects belongs in an action handler, not a function. + +## 4. Streaming + +A client MUST accept input incrementally and produce a valid render after every chunk. The rules: + +1. **Statement completion.** A statement is complete when its terminating newline (per section 1.4) has arrived and the continuation rules cannot extend it; for the ternary rule this means the first non-whitespace character of the following line has arrived and is not `?`. Completed statements are parsed once and their results are stable across subsequent chunks. +2. **The pending tail.** The text after the last completed statement is parsed on every chunk after implicit closing: an unterminated string is closed (with a `\` appended first if the text ends mid-escape), then unclosed brackets are closed in reverse order of opening. Mismatched closing brackets are skipped and do not change the bracket stack. A pending statement whose expression still fails to parse after implicit closing is not produced. +3. **Pending never overwrites completed.** A statement parsed from the pending tail is discarded if a completed statement with the same name exists. +4. **Reconciliation.** If preprocessing of the fuller text no longer begins with the previously completed prefix (for example, a fence opener arrives and retroactively changes what the program text is), the client MUST discard its cache and reparse from the start. +5. **No placeholders.** An array element that is an unresolved reference is omitted, not rendered as a hole or skeleton. The element appears when its statement arrives. +6. **Interactive features wait.** Queries and mutations MUST NOT execute while streaming is in progress. State declarations initialize as they arrive, but a default value recovered from a truncated statement MUST be replaced when the full statement arrives, unless the user has already edited that state. *(The reference client does not yet implement this replacement; it is a known defect.)* +7. **The host ends the stream.** The language has no in-band terminator; the host signals end of stream to the client (in the React client, the `isStreaming` prop). The signal finalizes the pending tail: implicit closing applies, and the resulting statements become completed, including a redefinition of an earlier name, which then wins per section 2.4. After finalization the streaming parser's result MUST equal the batch parse of the same bytes. *(The reference streaming parser currently drops a final-line redefinition; known defect.)* Chunks arriving after the signal are a new stream. + +```mermaid +sequenceDiagram + participant S as Stream + participant P as Parser + participant R as Rendered page + S->>P: root = Card([header, chart]) ⏎ + P->>R: empty Card (header, chart unresolved, omitted) + S->>P: header = Header("Monthly Rev + P->>R: Card with Header("Monthly Rev") via implicit close + S->>P: enue", "Last 6 months") ⏎ + P->>R: Card with full Header + S->>P: chart = BarChart(labels, [series]) ⏎ ... + P->>R: chart appears, fills as data lines land +``` + +## 5. State and forms + +### 5.1 The state store + +State is a flat map of `$name` to value, scoped to one rendered program. A state declaration provides the default. Any `$name` referenced anywhere without a declaration is auto-declared with default null. Defaults apply only when the variable has no value yet: re-parses during streaming and edits MUST NOT overwrite a value the user has produced. Host-persisted state, when supplied, is applied over defaults at initialization. Keys are never deleted by re-parsing. A query or mutation bound to a `$` name (section 2.1) is classified as a query or mutation, not a state declaration, but expressions cannot reach its result: `$name` always reads the state store, which holds the auto-declared null. Generators SHOULD NOT produce this form. + +### 5.2 Two-way binding + +Passing a `$variable` as the argument for a prop the library marks as bindable creates a two-way binding: the component displays the current value and writes user changes back. Which props are bindable is part of the component's library definition, not the language; the prompt renders such props as `$binding`, for example `$binding`, so the model knows where state can be attached, and the LibrarySpec marks them with a `bindable` flag *(proposed)* so prompts generated from the spec print them the same way. A `$variable` passed to a non-bindable prop evaluates to its current value. + +### 5.3 Form state + +Fields group under the nearest enclosing form component, one that provides a form name to its subtree the way the reference library's `Form` does; how a component provides a form name is part of its library definition (a LibrarySpec marker for it is proposed alongside `bindable`). Inputs outside any form write into **page-level state**, the unnamed default scope. Form values persist across re-parses. When an action sends a message to the model, the current form state travels with the event so the model sees what the user entered. Hosts MAY persist form state and restore it when re-rendering a stored program. + +### 5.4 Validation rules and named validators + +Validation rules are an object argument on input components: `{ required: true, email: true, url: true, numeric: true, minLength: 3, maxLength: 80, min: 0, max: 100, pattern: "^[A-Z]" }`. `pattern` is an ECMAScript regular expression evaluated without flags. Rules other than `required` skip empty values, where empty means null, undefined, the empty string, an empty array, or an object with no keys. The first failing rule produces the field error. A form submits only when every field passes. + +Named validators *(proposed)* extend the set: a validator declared in the library joins the rules object as its own key, `{ required: true, corporateEmail: true }`, indistinguishable from a built-in rule. The prompt lists registered names next to the built-ins. Like every rule other than `required`, a custom validator skips empty values; emptiness always belongs to `required`. A rule key that is neither a built-in rule nor a declared validator MUST NOT block the field: the client ignores that rule, applies the remaining rules, and emits an `unknown-validator` diagnostic. + +## 6. Data and actions + +### 6.1 Query lifecycle + +`name = Query(tool, args, defaults, refreshSeconds?)`. Arguments by position: the tool name, the argument object, the default result rendered until data arrives, and an optional refresh interval in seconds. + +- A query executes when streaming ends, and again whenever a `$variable` referenced in its `args` changes. Dependencies are the `$variables` written literally in the `args` expression. State reached indirectly, through a referenced statement, is not a dependency and does not trigger a re-fetch; generators are taught to place `$variables` directly in `args`. +- While a re-fetch driven by changed arguments is in flight, the previous result remains visible; the result of a fetch whose arguments are no longer current is discarded. +- References to a query resolve to its latest result, or its defaults before the first result. The reserved keys `__openui_loading`, `__openui_refetching`, and `__openui_errors` expose fetch state on the query result object to the host; they are not readable from expressions. +- A refresh interval re-executes the query on a timer. + +### 6.2 Mutation lifecycle + +`name = Mutation(tool, args)`. A mutation never runs on load; it runs only through `@Run`. Its `args` are evaluated at invocation time with current state. References to a mutation resolve to `{ status, data, error }`, where `status` is `idle`, `loading`, `success`, or `error`. A mutation whose status is `loading` rejects a second invocation. When a mutation and a query are bound to the same name, references to that name resolve to the mutation result. + +### 6.3 Action plans and steps + +`Action([step, step, ...])` builds a plan; a component's action prop triggers it. Steps run in order; a mutation run is awaited, while query re-fetches and host events are dispatched without waiting: + +- `@Set($var, value)`: evaluates `value` at click time and writes it. +- `@Reset($a, $b, ...)`: restores declared defaults (null if none). +- `@Run(ref)`: runs a mutation, or re-fetches a query. A failed mutation halts the remaining steps. +- `@ToAssistant(message, context?)`: emits a `continue_conversation` event to the host, carrying the message, an optional context string in `params.context`, and the current form state. +- `@OpenUrl(url)`: emits an `open_url` event to the host. + +`@Run`, `@Set`, and `@Reset` name their targets rather than evaluate them: `@Run` takes a reference to a query or mutation statement, `@Set` takes the state variable itself, and `@Reset` takes state variables. The template argument of `@Each` is deferred the same way (section 3.5). `@Run` on a query requests a re-fetch and never halts the plan; only a failed mutation halts. + +Components MAY define a default action; the reference library's button with no action emits `continue_conversation` with its label as the message. Events cross to the host in one field set. `formName` is the name of the nearest form enclosing the triggering component, absent otherwise, and form field values travel in the store shape, each wrapped as an object with `value` and `componentType` keys: + +```json +{ "type": "continue_conversation", + "humanFriendlyMessage": "Ticket closed", + "params": { "context": "..." }, + "formName": "ticket", + "formState": { "ticket": { "status": { "value": "closed", "componentType": "input" } } } } +``` + +`open_url` events carry the same field set with `params.url` and an empty `humanFriendlyMessage`. Custom action events *(proposed)* carry the declared action name as the event type and the validated arguments as `params` (section 6.4). + +```mermaid +flowchart TD + btn(["User clicks"]) --> plan[Action plan] + plan --> s1["@Set($busy, true)"] + s1 --> s2["@Run(save)"] + s2 -- success --> s3["@ToAssistant('Saved')"] --> host[Host onAction] + s2 -- failure --> halt([Remaining steps halted]) +``` + +### 6.4 Custom actions *(proposed)* + +An action declared in the library with `defineAction` is invoked as a named step, exactly like a built-in: `@ApproveInvoice("inv_42")`. Positional arguments map to the declaration's params schema by key order, the same rule component calls follow, and the client validates them against that schema before dispatching. The step then dispatches to the host through the same action-event channel the built-in effect steps use, with the action name as the event type and the validated arguments as its params; implementations stay host-side and never serialize. An unknown name in action-step position emits an `unknown-action` diagnostic and only that step is skipped; the remaining steps run, and the plan MUST NOT halt silently. + +### 6.5 Tool resolution + +The host supplies tools as a map of async functions keyed by tool name, or as an MCP client. Tool names are case-sensitive. A tool that is not found fails the query or mutation with `tool-not-found` and a hint listing available tools; it MUST NOT crash the host application. + +## 7. Incremental editing + +In edit mode the model receives the current program with the conversation and responds with only the statements that change. The client merges by statement name: + +- A patch statement whose name exists replaces the original. +- A new name appends. +- Names absent from the patch are kept. +- `name = null` deletes the statement. +- After merging, statements no longer reachable from the `root` statement are removed; a program with no statement named `root` is not garbage collected. State declarations are always kept. + +The parser does not treat `name = null` specially: inside one program it is an ordinary binding and later-wins applies. Deletion is the merge routine's interpretation of a top-level `name = null` in a patch, and the host decides when a text is merged as a patch rather than parsed as a program. A state declaration assigned null in a patch has its default set to null; the key is kept. + +Merging composes with streaming through the streaming parser, which applies each patched line as it completes; merging two complete texts is a batch operation. Inline mode composes with editing: the model may reply with prose plus a fenced patch, and only the fenced part is merged. + +```mermaid +flowchart LR + user(["User: weekly instead"]) --> model[Model] + model -- "2 changed lines" --> merge[Merge by name] + current[Current program] --> merge + merge --> gc[Drop unreachable] --> page[Updated page] +``` + +## 8. Errors and recovery + +### 8.1 Drop and render + +The invariant behind every rule in this section: **a mistake removes the smallest possible unit, and everything else renders.** An invalid argument degrades the prop, an invalid component drops that component, an invalid statement drops that statement. Nothing a model can emit crashes the client, and every removal is reported. + +### 8.2 Error codes + +| Code | Source | Meaning | Recovery | +| --- | --- | --- | --- | +| `unknown-component` | parser | Component not in the library | Statement dropped (see 8.4 for computed-expression positions) | +| `missing-required` | parser | Required prop absent | Filled from schema default if present, else component dropped | +| `null-required` | parser | Required prop is null | Same as missing | +| `excess-args` | parser | More arguments than props | Extras dropped, component renders | +| `inline-reserved` | parser | `Query`/`Mutation` in a value position | Expression evaluates to nothing | +| `parse-failed` | parser | Response yielded no renderable program | Nothing renders | +| `invalid-statement` *(proposed)* | parser | Line is not a valid statement (strict mode) | Line skipped | +| `parse-exception` | parser | Parser failure | Nothing renders; MUST be caught | +| `no-root` *(proposed)* | parser | No statement named `root`, or `root` is not a component call (stream complete) | Non-fatal when a component statement is recovered as the entry (section 2.2); nothing renders when none exists. Reported as `parse-failed` today | +| `runtime-error` | runtime | Expression evaluation threw | Prop falls back to the value as parsed, unevaluated | +| `render-error` | runtime | Component renderer threw | Contained; last successful render kept | +| `tool-not-found` | query/mutation | Unknown tool name | Query keeps defaults; hint lists tools | +| `tool-error`, `mcp-error` | query/mutation | Tool invocation failed | Mutation result carries the error; a query keeps defaults or last good data | +| `unknown-function` *(proposed)* | parser | `@Name` in expression position is neither built-in nor registered | Statement dropped | +| `unknown-action` *(proposed)* | parser | Step name in an action plan is neither built-in nor declared | Only that step skipped; remaining steps run | +| `unknown-validator` *(proposed)* | parser | Rule key is neither a built-in rule nor a declared validator | That rule ignored; remaining rules apply | +| `constraint-violation` *(proposed)* | parser | Argument violates a schema constraint ([prompt.md](./prompt.md), section 3) | Value renders as-is; warning-level diagnostic, MUST NOT drop the component | + +### 8.3 The error object + +Errors cross the wire in one shape, designed to be pasted into a model conversation: + +```json +{ + "source": "parser", + "code": "missing-required", + "statementId": "chart", + "component": "BarChart", + "path": "/labels", + "message": "Missing required prop labels", + "hint": "Signature: BarChart(labels*, series*, variant) — * marks required" +} +``` + +`hint` carries a compact signature built from the JSON Schema, prop names only with required props starred, or the available options when the problem is an unknown name. The reference client emits the error list when a completed render produces a different list than the last emission; clients MUST NOT re-emit an unchanged error list and MUST signal recovery by emitting an empty list. + +```mermaid +sequenceDiagram + participant M as Model + participant C as Client + M->>C: program, one statement broken + C->>C: drop it, render the rest + C->>M: error list (code, statement, hint) + M->>C: one-line patch + C->>C: merge, re-render, errors clear +``` + +### 8.4 Component validation rules + +Arguments map to props positionally against the library schema. Validation then runs per prop: a missing or null required prop uses the schema default when one exists, otherwise the component is invalid and is dropped. Excess arguments are dropped with a diagnostic while the component still renders. Inside arrays, invalid components and unresolved references are omitted; explicit `null` literals are kept (and render nothing). Unknown components in a value position are dropped; inside a computed expression they are kept in the tree so the error can point at them, but they render nothing. + +## 9. The program tree and serialization + +### 9.1 The tree + +Parsing produces a tree: each statement's name and kind, and for component calls the component type with arguments mapped to named props against the schema. The tree, not the text, is what evaluation and rendering consume. A canonical JSON interchange shape for this tree is under exploration and not part of this specification; it would give hosts a schema-independent storage format and give the conformance fixtures their expected-output format. + +### 9.2 Serialization + +A rendered tree serializes back to source text: each component with a statement name becomes a statement, children are emitted before their parents, props emit positionally in schema order with `null` holding unfilled required positions, and trailing nulls for optional props are trimmed. Object keys serialize unquoted, so keys must be valid names to round-trip. Serialization is currently defined for component trees only; query, mutation, and state statements have no serialization rules yet and do not round-trip. Serialization output SHOULD reparse to a tree equivalent up to the recovery rules of section 8, and serializers SHOULD parenthesize nested expressions whenever the reparse would otherwise change grouping; the reference serializer currently parenthesizes only lower-precedence binary children, so expressions like `a - (b - c)` and `-(a + b)` do not yet round-trip. + +## 10. Client checklist + +A conforming client: + +- MUST parse the complete grammar of section 1, including features it does not implement, and drop the statements that depend on a missing feature rather than failing. +- MUST re-render correctly after every streamed chunk and follow all seven rules of section 4. +- MUST apply the recovery table of section 8 exactly: same drops, same fallbacks. +- MUST NOT let any generated content crash the host application, including component renderers that throw. +- MUST report errors in the wire shape of section 8.3 and clear them on recovery. +- MUST resolve tools case-sensitively and, when custom actions ship, validate dispatch payloads against declared schemas. +- MUST keep user-entered state across re-parses and edits. +- SHOULD provide hooks for unknown components and actions so hosts can render diagnostics instead of blank space. +- MAY pace the visual reveal of streamed content, provided parsing itself follows section 4. + +## Appendix A. A worked streamed example + +The response arrives in four chunks. After each chunk, the client state: + +**Chunk 1**: `root = Card([header, kpis])\nheader = Head` +Parse: one completed statement (`root`); the pending tail parses as `header` bound to the bare reference `Head`. Render: an empty Card. `Head` and `kpis` are unresolved. + +**Chunk 2**: `er("Tickets", "Today")\nkpis = Stack([open` +Parse: `header` completes. Pending: `kpis = Stack([open` autocloses to `Stack([open])`; `open` unresolved, omitted. Render: Card with Header, empty Stack. + +**Chunk 3**: `, closed])\nopen = Metric("Open", @Count(@Filter(tickets.rows, "status", "==", "open")))\n` +Parse: `kpis` and `open` complete. `closed` and `tickets` unresolved. Render: Card, Header, Stack with one Metric (its count is 0 until `tickets` arrives). + +**Chunk 4**: `closed = Metric("Closed", 8)\ntickets = Query("listTickets", {}, { rows: [] })\n` +Parse: complete; stream ends; the query executes and the metrics recompute. Render: the full dashboard, live. + +Every intermediate frame was a valid page. The client showed no errors and no placeholders. + +## Appendix B. Conformance fixtures + +A conformance fixture suite is planned to accompany this specification: pairs of input program and expected parse result (rendered tree, metadata, errors), plus streaming fixtures given as chunk sequences with the expected state after each chunk. A client claims conformance against a tagged fixture release. The fixtures are the operational definition of this document: where prose and fixtures disagree, that is a specification bug, and the next tagged release fixes both. + +## Appendix C. Changelog + +- **2026-08-12**: Fixes from three independent review passes. Grammar: the call production uses `function_name` (either case after `@`), `reference` admits bare identifiers of either case, matching the prose and Appendix A. Streaming: end-of-stream finalizes the pending tail with batch-parse equality required (rule 7); "open ternary" defined precisely (1.4). Entry: the required `root` field and its array form stated (2.2). Fence extraction gains the segments-engine supersession note (1.2); `newline` pinned with CRLF tolerance (1.3); `constraint-violation` added to the error table (8.2); Appendix A example corrected to `tickets.rows`. +- **2026-08-05**: Draft renamed from 0.9 to 1.0-beta; earlier entries keep the old name. +- **2026-08-04**: Proposed designs sharpened after a second review round: entry recovery proposed as a pure function of the program text with a non-fatal `no-root` (2.2); custom actions became direct named steps dispatched through the existing action-event channel (6.4); named validators became rules-object keys (5.4); the `unsupported-feature` code was withdrawn; error-table recoveries made per-code (8.2). +- **2026-08-03**: Grammar restructured with notation, lexical elements, and predeclared names up front; conformance profiles removed (the parse-everything rule moved into the client checklist); draft split into overview, language, and prompt documents. Review fixes from four review passes: grammar corrections (state-name statements, stacked unary, underscore in identifiers, binding assignments), streaming boundary made precise for ternary continuation, unresolved-reference evaluation specified, orphan and event wire shapes corrected to shipped behavior, validator list completed, terminology defined (value position, computed expression, page-level state). +- **2026-07-22**: First community review draft (0.9). Covers the implemented language plus proposed extensions: registered functions, named validators, custom actions, strict parse mode, and the LibrarySpec registries. diff --git a/spec/overview.md b/spec/overview.md new file mode 100644 index 000000000..2417d2765 --- /dev/null +++ b/spec/overview.md @@ -0,0 +1,586 @@ +# OpenUI Overview + +**1.0-beta, community review draft** + +The OpenUI specification is split into three documents: + +- **overview.md** (this file): what OpenUI is, its features in detail, and how to build with it. +- **[language.md](./language.md)**: the normative language specification, for client implementers. +- **[prompt.md](./prompt.md)**: the LibrarySpec and everything that is sent to the model. + +This is the final specification draft for community review until 1.0. Shipped behavior and proposed extensions are kept apart: sections 1 to 3 describe what works today, and section 4 collects the proposed features, where feedback is especially welcome. + +## What's new in 1.0-beta + +v0.1 covered the static core (syntax, positional mapping, streaming, hoisting) and v0.5 added the interactive layer (reactive state, built-ins, queries, mutations, actions, prompt flags). 1.0-beta changes no shipped syntax; the one proposed change to shipped behavior is the simplified entry recovery ([language.md](./language.md), section 2.2). The theme of 1.0-beta is running OpenUI in production: making a library yours, shipping more than one surface, and keeping persisted UIs working as your product evolves. These are the proposed additions, each tagged *(proposed)* where it appears: + +- **Extending a library.** `library.extend({ add, remove, override })` derives a new library from a base: add your own components to the default library, remove what your product does not use, or override a component with your own. Added components can appear anywhere general content goes, with no extra wiring (section 4.5). +- **Multiple libraries in one response.** Fenced blocks tagged with `library=` let one generation carry programs from different libraries: a chat answer and a slides artifact in one response, each routed to its own surface through the segments API (section 4.6). +- **Library versioning and backward compatibility.** Libraries carry a stable `id` and a `version`, and stored UIs survive library evolution: props can be reordered, removed, or added and persisted pages still render correctly. The mechanism is one metadata line appended to stored messages, carrying the key orders the message was written against, the same writer-schema idea Avro uses for positional data (section 4.7; full protocol in [prompt.md](./prompt.md), section 7). Renames, component successors, and serving older clients a matching library build on the same protocol and are not yet specified. +- **Registered functions.** Libraries declare pure, typed functions the model calls like built-ins: `price = TextContent(@FormatCurrency(total, "USD"))`. Declared with `defineFunction`, advertised in the prompt, implemented in each client (sections 4.1 and 4.2). +- **Named validators.** Custom validation checks join the built-in rules object as their own keys, with no new syntax: `{ required: true, corporateEmail: true }`. Declared with `defineValidator` (section 4.3). +- **Custom actions.** `defineAction` declares the name, params schema, and description; the model invokes it as a named step like any built-in, `@ApproveInvoice("inv_42")`; the host handles it in the existing `onAction` callback, whose event becomes typed over the declared set (section 4.4). +- **Library registries.** `createLibrary` accepts `functions`, `validators`, and `actions` next to `components`, with one rule for all three: declarations serialize into the contract, implementations stay local (section 4.1). +- **The unified LibrarySpec.** One JSON document bundling the validation schema and the new registries; the prompt's signature strings are derived from it, so the two can never drift. It is the interchange format that makes native Kotlin and Swift clients and gateway integrations work from one contract ([prompt.md](./prompt.md), section 2). +- **Richer component docs in prompts.** JSDoc blocks over signatures: `@param` lines from prop descriptions and `@example` lines from per-component usage examples, emitted only where authored ([prompt.md](./prompt.md), section 5). +- **A required, flexible root.** `createLibrary` now requires `root`, and it takes one component name or several: with `root: ["Card", "Dashboard"]` the model picks the right top-level per request. The field guides generation only; what renders is always decided by the program text ([language.md](./language.md), section 2.2). +- **Data components.** `defineComponent({ schemaOnly: true })` declares positional data shapes, `Series("Revenue", [10, 20])`, that materialize as plain validated values in the parent component's props: no null renderers, no unwrapping helpers, and the wire format is unchanged (section 4.8). +- **Schema constraints that teach and check.** Prop constraints like `.min`, `.max`, and `.regex` render into the prompt automatically and are validated at parse time: a violation still renders and reports a warning the model can fix. The supported set is limited to JSON-Schema-mappable keywords, so every platform enforces the same rules from the schema document alone ([prompt.md](./prompt.md), sections 3 and 5). +- **New error codes.** `no-root`, `unknown-function`, `unknown-action`, and `unknown-validator` complete the recovery table, and entry recovery becomes a pure function of the program text ([language.md](./language.md), sections 2.2 and 8.2). + +Where the old v0.1 and v0.5 pages and this specification disagree, this specification is correct; those pages will be retired when 1.0 lands. + +## 1. Introduction + +OpenUI Lang is a small language that a model writes to describe a user interface. When your application asks a model for UI, it does not return a JSON tree or raw HTML. It writes a short program, one statement per line, and the client renders the program while it streams in: + +```openui-lang +root = Card([header, chart]) +header = Header("Monthly Revenue", "Last 6 months") +chart = BarChart(labels, [series]) +labels = ["Jan", "Feb", "Mar", "Apr", "May", "Jun"] +series = Series("Revenue", [12000, 15000, 14000, 18000, 21000, 25000]) +``` + +Every line binds a name to a value: a component, a list, a piece of data. A name can be used before it is defined (the language calls this hoisting), so a program can stream top-down. The first line already declares the shape of the whole page, and every line after it fills that shape in. The renderer draws what it has and refines as more arrives. + +The whole loop: + +```mermaid +flowchart LR + user([User]) -- "What did I spend last month?" --> app[Your app] + app -- "system prompt from your library + message" --> llm["Model (any provider)"] + llm -- "openui-lang stream" --> app + app --> ren[OpenUI renderer] + ren -- "live UI, streamed" --> user + ren -- "queries and mutations" --> tools[Your data and tools] + tools -- "live data" --> ren +``` + +Your app holds the data and the component library. The library generates a system prompt that teaches any model provider the language and your components. The model streams OpenUI Lang back, the renderer draws it as it arrives, and the rendered page talks to your data directly through queries and mutations, with no model in that path. + +## 2. Features + +This is the whole surface of OpenUI Lang in one place. The precise rules for each feature live in the [language specification](./language.md). Component names in the examples are illustrative; every library defines its own vocabulary (section 3.2). + +### 2.1 Components + +The model composes UI from components you define. Arguments are positional and map onto your component's schema in declaration order, so if `Button` declares `label`, `action`, `variant` in that order: + +```openui-lang +btn = Button("Save changes", saveAction, "primary") +``` + +the renderer receives `{ label: "Save changes", action: saveAction, variant: "primary" }`. Trailing optional arguments can be omitted. There is no keyword syntax; position is the contract. + +Rendering starts at one entry statement, by convention named `root`. Everything reachable from the entry renders; value statements nothing reaches are orphans, reported in metadata and skipped. When a response forgets to name `root`, the client recovers by picking the most plausible entry through a deterministic order defined in the language spec, but generators are always taught to write one. + +### 2.2 References and hoisting + +Every statement names one part of the page, and any statement can reference another by name, wherever it appears. The program is a graph, not a sequence; the line order is presentation, nothing else: + +```openui-lang +root = Card([title, kpis, refreshBtn]) +title = Header("Support Overview") +kpis = Stack([openCount, closedCount]) +openCount = Metric("Open", 12) +closedCount = Metric("Closed", 8) +refreshBtn = Button("Refresh") +``` + +The first line uses `title`, `kpis`, and `refreshBtn` three lines before any of them exists. That is hoisting. It lets the model write the layout first and the data last, which is also the order a reader wants to see the page appear in: frame first, numbers filled in. + +A reference to a statement that does not exist is unresolved, and unresolved means not yet, not error. If the stream stopped before `closedCount` arrived, the `Stack` would render with one metric and no hole where the second should be; the moment the statement lands, the metric appears. During streaming this is the normal state of the program, so clients never treat it as a failure. + +One statement can be referenced from many places: + +```openui-lang +badge = Tag("Beta") +header = Header("Reports", badge) +footer = Footer([badge, legal]) +``` + +`badge` renders in both places as independent copies, like a value assigned into two variables. Runtime results are the one exception: a query (section 2.5) referenced from five components still fetches once, and all five read the same result. + +### 2.3 Streaming + +The program is re-parsed and re-rendered as tokens arrive, so a partially received response is always a valid page. Here is the introduction's program arriving chunk by chunk, and what the user sees at each moment: + +| Arrived so far | On screen | +| --- | --- | +| `root = Card([header, chart])` | An empty card. `header` and `chart` are unresolved, so nothing marks their place. | +| `header = Header("Monthly Rev` | The card gains a header reading "Monthly Rev". The half-finished string renders as received, so text fills in the way a person types. | +| `enue", "Last 6 months")` | The header completes: "Monthly Revenue", with its subtitle. | +| `chart = BarChart(labels, [series])` | An empty chart appears; its data lines have not arrived. | +| `labels = [...]` then `series = Series(...)` | The bars fill in. The page is done before the stream formally ends. | + +The renderer re-renders on every chunk, not every line. The first paint happens on the first chunk whose accumulated text yields a renderable entry, and thanks to implicit closing that is usually midway through line one: `root = Card([header` renders an empty card before its bracket ever closes. A line the parser cannot use is skipped, so what matters is the first valid statement, not literally the first line. From there, elements appear when they arrive, in a frame declared at the start, with no spinners, skeletons, or layout jumps. + +### 2.4 Reactive state + +Variables prefixed with `$` hold client-side state: + +```openui-lang +$query = "" +search = Input("search", $query) +label = TextContent("Searching for " + $query) +``` + +Passing `$query` to an input binds it both ways: typing updates the variable, and every expression that reads it re-evaluates. The label above updates on each keystroke. (This example library declares `Input(name, value)`; your library's signatures may differ.) + +The result is interaction without the model. Without client-side state, every interaction is another model round trip: seconds of latency and tokens spent to switch a tab. In OpenUI the model writes the behavior once and the client runs it locally, with no added latency or token cost: + +```openui-lang +$tab = "overview" +tabs = Tabs($tab, [Tab("overview", "Overview"), Tab("billing", "Billing")]) +body = $tab == "overview" ? overviewPanel : billingPanel +``` + +Switching tabs swaps the panel instantly; the model is not consulted. (The `? :` conditional is part of the expression layer, section 2.9.) Filters, toggles, live search, conditional sections, derived totals: all of it runs client-side. State also feeds data fetching (next section). + +### 2.5 Queries and mutations + +`Query` and `Mutation` connect the UI to tools you provide, and they turn a generated page into a live application. Letting the model write the data into the page itself would pass every number through the model: it copies figures from whatever is in its context, and each one is a chance to be stale, truncated, or invented. A query is the spreadsheet trick: put the formula in the cell instead of pasting the number. The model writes where the data comes from; your backend supplies what it is. + +```openui-lang +data = Query("list_tickets", {}, { rows: [] }) +``` + +Argument by argument: the tool name (matching what your server exposes), the arguments to pass, and the default result, which renders immediately while the fetch is in flight. Query results are plain data, reachable with dot notation: `data.rows.title` plucks the `title` field from every row. + +Queries are reactive. Put a `$variable` in the args and the query re-fetches whenever it changes: + +```openui-lang +$days = "7" +data = Query("analytics", { days: $days }, { rows: [] }) +filter = Select("days", $days, [SelectItem("7", "7 days"), SelectItem("30", "30 days")]) +``` + +The user picks "30 days", `$days` updates, the query re-fetches, the chart redraws. The model wired this once; from then on the page talks to your backend directly. An optional fourth argument re-fetches on a timer: `Query("get_server_health", {}, { cpu: 0 }, 30)` refreshes every 30 seconds. + +Mutations write. They never run on load, only when an action triggers them (the `Action` wrapper and its steps are covered in section 2.6): + +```openui-lang +tickets = Query("list_tickets", {}, { rows: [] }) +createResult = Mutation("create_ticket", { title: $title, priority: $priority }) +submitBtn = Button("Create", Action([@Run(createResult), @Run(tickets), @Reset($title)])) +feedback = createResult.status == "error" ? Callout("error", "Failed", createResult.error) : null +``` + +The button creates the ticket, re-fetches the tickets query so the table refreshes, and clears the form, in order; a failed mutation halts the steps after it, and its `status` and `error` are readable in expressions for feedback UI. + +On the host side, one renderer prop resolves tool names: `toolProvider` accepts either a map of async functions or an MCP client (any object with `callTool({ name, arguments })`, such as one from the MCP SDK). If you already run an MCP server for your agents, the generated UI can call the same tools with no extra glue: + +```mermaid +flowchart LR + model[Model] -- writes the wiring once --> rt[Renderer runtime] + rt -- callTool --> tp[toolProvider] + tp --> be[Your backend or MCP server] + be -- live data --> rt + rt --> ui[Components update] +``` + +The data path never touches the model: results flow from your backend into the runtime and straight into components. `Query` and `Mutation` are statement forms rather than components: each is valid only as the entire right side of a statement, and a reference to that statement resolves to its runtime result. + +### 2.6 Actions + +An action is a sequence of steps that runs when the user clicks: + +```openui-lang +save = Mutation("updateTicket", { id: $ticketId, status: "closed" }) +closeBtn = Button("Close ticket", Action([@Run(save), @Set($showModal, false), @ToAssistant("Ticket closed")])) +``` + +Steps run in order, and a failed mutation halts the rest, so success messages never fire on failure. `@Set` and `@Reset` change state, `@Run` executes a mutation or re-fetches a query, `@ToAssistant` sends a message back to the model, and `@OpenUrl` opens a link. Actions you declare yourself are a proposed extension (section 4.4). + +### 2.7 Forms and validation + +Inputs carry declarative validation rules: + +```openui-lang +email = Input("email", "you@company.com", "email", { required: true, email: true }) +``` + +The renderer enforces the rules and blocks submission until they pass. The built-in rule set: `required`, `email`, `url`, `numeric`, `minLength`, `maxLength`, `min`, `max`, `pattern`. Inputs group under an enclosing form component and submit together; the exact grouping rules live in the [language spec](./language.md). Custom checks arrive with named validators, a proposed extension that slots into this same rules object (section 4.3). + +### 2.8 Incremental editing + +Redefining a variable behaves as in any other language: `x = 1` near the top, `x = 2` further down, and the later line wins. A single OpenUI program already works this way when two statements bind the same name. Incremental editing extends the rule across responses: the conversation is one long program, and an edit is the model reassigning only the names that change. + +The page currently showing: + +```openui-lang +root = Card([header, chart]) +header = Header("Monthly Revenue", "Last 6 months") +chart = BarChart(labels, [series]) +labels = ["Jan", "Feb", "Mar", "Apr", "May", "Jun"] +series = Series("Revenue", [12000, 15000, 14000, 18000, 21000, 25000]) +``` + +The user says "show it weekly". In edit mode the model responds with only the statements that change: + +```openui-lang +header = Header("Weekly Revenue", "Last 4 weeks") +chart = BarChart(weekLabels, [series]) +weekLabels = ["W1", "W2", "W3", "W4"] +series = Series("Revenue", [4800, 5300, 5100, 6200]) +``` + +The client merges by name: `header`, `chart`, and `series` are replaced, `weekLabels` is new and appended, and `root` was not mentioned so it stays. `labels` is no longer referenced by anything and is garbage collected. Explicit removal is an assignment too: `chart = null` deletes the chart. Four lines instead of regenerating the page, and the patch streams and renders like any other program. + +### 2.9 Expressions + +The language has a small expression layer: arithmetic, comparison, logic, ternaries, member access, and indexing. Expressions are evaluated on the client, so derived values update without a model round trip: + +```openui-lang +sales = Query("getSales", { period: $days }, { amount: [] }) +total = @Sum(sales.amount) +status = TextContent(total > 100000 ? "On track" : "Behind target") +average = @Round(total / @Count(sales.amount), 2) +``` + +Expressions are the glue between queries and components: raw tool data goes in, and the derived numbers, labels, and conditions come out as component props, recomputed live whenever the data or state behind them changes. (The `@Sum` and `@Round` calls are built-in functions, section 2.10.) Member access on an array plucks a field from every element: if `sales.rows` is a list of objects, `sales.rows.amount` is the list of their `amount` values. This one rule covers most of what the model needs to reshape tool data into chart and table inputs. + +### 2.10 Built-in functions + +Built-ins are called with an `@` prefix and cover the data work a dashboard needs: `@Count`, `@Sum`, `@Avg`, `@Min`, `@Max`, `@First`, `@Last` for aggregation, `@Filter` and `@Sort` for reshaping, `@Round`, `@Abs`, `@Floor`, `@Ceil` for math, and `@Each` for iteration. Their main job is turning query results into component props: rows from a tool become the filtered, sorted arrays a table or chart accepts: + +```openui-lang +tickets = Query("list_tickets", {}, { rows: [] }) +urgent = @Filter(tickets.rows, "priority", "==", "high") +sorted = @Sort(urgent, "createdAt", "desc") +rows = @Each(sorted, "t", Row(t.title, Tag(t.priority))) +``` + +Built-ins compose: `@Count(@Filter(tickets.rows, "status", "==", "open"))` is the idiomatic KPI counter. + +### 2.11 Error recovery + +Models make mistakes, and the language is designed around that fact. An invalid statement is dropped, everything else renders, and the client produces a structured error naming the statement, the problem, and a hint: + +```json +{ "source": "parser", "code": "unknown-component", "statementId": "chart", + "message": "Unknown component PieChart", "hint": "Available components: BarChart, LineChart, Table, ..." } +``` + +Fed back to the model, this error is enough to produce a one-line fix through incremental editing. The user sees a page with one missing chart for a moment, not a broken screen. + +## 3. Building with OpenUI + +### 3.1 The end-to-end flow + +```mermaid +sequenceDiagram + participant App as Your app + participant Model + participant Renderer as OpenUI renderer + participant User + App->>Model: system prompt (generated from your library) + user message + Model->>Renderer: openui-lang, streamed token by token + Renderer->>User: page builds up as tokens arrive + User->>App: action event (click, form submit) + alt chat app + App->>Model: follow-up message with context and form state + Model->>Renderer: a new response, rendered as a new message + else editing surface (canvas, dashboard, artifact) + App->>Model: current program + the request + Model->>Renderer: patch with only the changed statements + end +``` + +You define a component library once. From that library you generate a system prompt, which teaches the model your components and the language rules. The model responds to user messages in OpenUI Lang, the renderer draws the stream, and user interactions come back to your code as events. Two packages appear in this chapter: `@openuidev/react-lang`, the React runtime, and `@openuidev/cli`, the prompt generator. + +What happens after an action depends on the product. In a chat app, the interaction becomes a new turn and the model answers with a new message holding a new program; the earlier message stays in the thread. In an editing surface, the app sends the current program along with the request and the model responds with a patch of only the changed statements, merged into the live page. Both are the same language; the difference is what the app asks for and what it does with the reply. + +### 3.2 Component definition + +A component is a name, a schema, a description, and a renderer: + +```tsx +import { defineComponent } from "@openuidev/react-lang"; +import { z } from "zod/v4"; + +const Button = defineComponent({ + name: "Button", + description: "A clickable button", + props: z.object({ + label: z.string(), + action: z.any().optional(), + variant: z.enum(["primary", "secondary"]).optional(), + }), + component: ({ props }) => , +}); +``` + +The schema does double duty. Its key order defines the positional argument order the model uses, and its types are what the prompt generator prints and the parser validates against. Declare required props before optional ones, and treat key order as a public API: reordering keys silently changes the meaning of every existing program and prompt. + +### 3.3 Library creation + +A library groups components and names the root: + +```tsx +import { createLibrary } from "@openuidev/react-lang"; + +export const library = createLibrary({ + root: "Card", + components: [Card, Header, TextContent, Button, Input, Table, Col, BarChart, Series], +}); +``` + +The library is the whole contract. The model can only speak the components in it, the prompt is generated from it, and the parser validates against it. Different products ship different libraries: a support tool and an analytics tool can use the same language with entirely different vocabularies. Registries for functions, validators, and actions are proposed extensions of this same contract (section 4). + +### 3.4 System prompt generation + +The prompt is generated from the library, never written by hand: + +```bash +npx @openuidev/cli generate ./src/library.ts --out ./generated/system-prompt.txt +``` + +or programmatically with `generateSystemPrompt({ library, promptOptions })`. The prompt generator prints the syntax rules, every component signature with its description, and, when enabled, the sections for tools, state, and editing. Feature flags control what the model is taught: `toolCalls` and `bindings` unlock queries, mutations, and state; `editMode` teaches patching; `inlineMode` teaches mixing prose with fenced code. A model is never told about a feature the client does not support, so a simple client stays safe without version checks. The full prompt contract lives in [prompt.md](./prompt.md). + +### 3.5 Rendering + +```tsx + +``` + +The renderer parses the accumulated response on every chunk and renders what is valid. It holds queries and mutations until streaming ends, and the reference components disable form interaction until the stream closes. It never throws into your application: a component that fails to render is contained, and the errors arrive as a structured list on `onError`. + +### 3.6 Action and tool handling + +Renderer props connect generated UI to your code. `toolProvider` resolves the tool names that queries and mutations call. For an in-process backend, pass a map of async functions; for a standing tool server, pass an MCP client, and the generated UI calls the same tools your agents already use: + +```tsx +// Function map +const tools = { + getSales: async (args) => db.sales.forPeriod(args.period), + updateTicket: async (args) => db.tickets.update(args.id, args), +}; + + +// Or an MCP client (anything with callTool({ name, arguments })) + +``` + +This is the MCP flow end to end: the MCP server's tool list is described to the model in the prompt, the model writes `Query`/`Mutation` statements naming those tools, and the runtime calls them through the client. One tool surface serves both your agents and your generated UI. + +`onAction` receives the events that leave the page: `continue_conversation` when the model should be told something and `open_url` for links. + +### 3.7 Native clients and the LibrarySpec + +Everything a client needs to know about a library serializes to a JSON document, the LibrarySpec (defined in [prompt.md](./prompt.md)). A Kotlin or Swift client defines the same components natively, emits the same spec, and renders the same programs. + +```mermaid +flowchart LR + subgraph client [Client app] + lib["Library: schemas + native renderers"] + ren[Renderer] + end + lib -- emits --> spec[LibrarySpec JSON] + spec -- request --> gw["Backend / gateway"] + gw -- "deterministic conversion" --> prompt[System prompt] + prompt --> model[Model] + model -- "openui-lang stream" --> ren +``` + +Two integration shapes are supported. A client can generate the prompt locally and send it as an ordinary system message, which works with any model API today. Or it can pass the LibrarySpec to a backend that converts it to the prompt, the shape used when OpenUI is offered as an output modality by a gateway such as OpenRouter. The conversion from spec to prompt is deterministic, so every platform's generations behave identically. The second shape depends on the unified LibrarySpec, which is *(proposed)* ([prompt.md](./prompt.md), section 2). + +## 4. Proposed extensions + +Everything in this section is designed but not shipped. It is written in the present tense so the design can be judged as it would ship; nothing here is implemented today. + +### 4.1 Declared extensions: functions, validators, actions *(proposed)* + +Beyond components, a library declares three kinds of named extensions, all with the same shape: the declaration (name, typed signature, description) lives in the library and travels to the prompt, while the implementation stays out of the serialized contract. + +```tsx +const FormatCurrency = defineFunction({ + name: "FormatCurrency", + params: z.object({ amount: z.number(), currency: z.string() }), + returns: "string", + description: "Format a number as currency", + fn: ({ amount, currency }) => + new Intl.NumberFormat("en", { style: "currency", currency }).format(amount), +}); + +const CorporateEmail = defineValidator({ + name: "corporateEmail", + description: "Rejects free-mail providers", + validate: (value) => + typeof value === "string" && value.endsWith("@gmail.com") + ? "Use your work email" + : undefined, +}); + +const ApproveInvoice = defineAction({ + name: "ApproveInvoice", + params: z.object({ id: z.string() }), + description: "Approve an invoice by id", +}); + +export const library = createLibrary({ + root: "Card", + components: [/* ... */], + functions: [FormatCurrency], + validators: [CorporateEmail], + actions: [ApproveInvoice], +}); +``` + +Functions and validators are pure, so their implementations sit in the library and stay importable everywhere, including the CLI that generates prompts. Actions touch your application, so `defineAction` declares only the contract and the handler binds at the renderer (section 4.4). The rule of thumb: if it touches the outside world, it is a renderer concern. + +### 4.2 Registered functions in programs *(proposed)* + +The model calls registered functions like built-ins: + +```openui-lang +price = TextContent(@FormatCurrency(total, "USD")) +``` + +The name, signature, and description are declared in the library, so the prompt advertises the function and every client knows the full callable surface ahead of time. Implementations are plain functions in each client, and they must be pure: same inputs, same output, no side effects. + +### 4.3 Named validators *(proposed)* + +Named validators slot into the same rules object as the built-in validation rules, so custom checks cost no new syntax: + +```openui-lang +email = Input("email", "you@company.com", "email", { required: true, corporateEmail: true }) +``` + +`defineValidator({ name, description, validate })` declares one; the registered name becomes a rules-object key, listed in the prompt next to the built-in rules with its description. A custom validator only ever sees non-empty values: emptiness stays owned by `required`, so custom and built-in rules compose the same way. An unknown rule key does not block rendering; that rule is ignored, the remaining rules still apply, and a diagnostic is emitted. + +### 4.4 Custom actions *(proposed)* + +A declared action is invoked as a named step, exactly like a built-in: `defineAction({ name, params, description })` adds `@ApproveInvoice(id: string)` to the step vocabulary, with positional arguments mapped by the params schema's key order, the same rule components follow. The declaration serializes with the library; at the renderer, the action arrives through the existing `onAction` callback. `ActionEvent` becomes generic over the library, so `event.type` narrows to the built-in types plus the declared action names, and `event.params` narrows to that action's schema: + +```tsx + { + switch (event.type) { + case "ApproveInvoice": + api.approve(event.params.id); // params typed from the declaration + break; + // built-in events (continue_conversation, open_url) handled as today + } + }} + ... +/> +``` + +Arguments are validated against the schema before the event is dispatched. An unknown action name skips that step with a diagnostic rather than halting the plan, and the remaining steps still run. + +### 4.5 Extending a library *(proposed)* + +A library derives from another with one method, covering both directions of the common request: use the default library but add your own components, or use it minus the parts your product does not want. + +```tsx +export const library = defaultLibrary.extend({ + add: [ProductCard], // joins every content slot automatically + remove: ["Carousel"], + override: [MyButton], // same name, replaces schema and renderer +}); +``` + +The result is a full library: the prompt, the LibrarySpec, and parser validation all derive from the final component set. An added component joins every content slot by default (every prop typed as a union of components), so the model can place it anywhere general content goes; `{ component, slots: ["Card.children"] }` narrows placement when a component belongs in exactly one place. Placement can also be steered in prose: a component description like "use inside a Card, next to the totals" needs no schema mechanics, the model follows it. Every inconsistency fails at `extend()` time, at the line that caused it: removing a component that a remaining schema still references names the referencer instead of breaking at render time. In managed setups the extension serializes as a delta, so a server-side prompt generator applies the same change to its copy of the base library. An override replaces the full contract, schema and renderer together; for a visual-only swap, reuse the base library's exported schema object in the override so the contract cannot drift. + +Extension also gives library authors an opt-in distribution pattern: defining a component and including it in a library are separate acts, so an author can export fully implemented components without putting them in the base library, and a developer enables them with one `extend({ add })` call. Components left out cost nothing: no prompt tokens, no bundle weight from an extras entry point, and the model never hears of them. + +### 4.6 Multiple libraries in one response *(proposed)* + +The model tags each fenced block with the library it speaks: + +````markdown +Here is the revenue picture: + +```openui-lang library=charts +root = ChartCard([rev]) +rev = BarChart(labels, [series]) +``` + +And I have drafted the slides you asked for: + +```openui-lang library=slides +root = Deck([intro, numbers]) +``` +```` + +Untagged blocks bind to the first library the app passed, so single-library apps and forgetful models keep working unchanged. Every block is its own program with its own state: a response can interleave prose and UI freely (text, chart, more text, form), each block renders independently in document order, and cross-block references are not allowed. + +This is what lets one generation produce chat and artifact together. The user asks for a revenue deck; one model call answers with prose, a chart for the conversation, and the slides content, each in its own tagged block. The host renders the prose and chart in the message and lifts the slides segment into the artifact panel. The routing key is the same tag everywhere: it picks the library that validates and renders the segment, and it tells the host which surface the segment belongs to. Fence tags live in the message text itself, so a persisted message routes every segment the same way on reload. + +Under every layer sits one engine, `parseMessage(text, libraries)`, and one separation rule: a fence whose info string starts with `openui-lang` is a UI segment, and everything else, including any other code fences the model writes, stays prose. The scanner is string-aware (a backtick inside a double-quoted OpenUI string cannot close a fence, [language.md](./language.md), section 1.2), and an unterminated fence during streaming is already a UI segment instead of leaking into the prose. To show OpenUI Lang as an example rather than render it, the model tags the fence `text`; the prompt teaches that rule. The response above parses to: + +```ts +parseMessage(text, [charts, slides]); +// [ +// { kind: "prose", text: "Here is the revenue picture:" }, +// { kind: "program", libraryId: "charts", code: "root = ChartCard([rev])\n..." }, +// { kind: "prose", text: "And I have drafted the slides you asked for:" }, +// { kind: "program", libraryId: "slides", code: "root = Deck([intro, numbers])" }, +// ] +``` + +Apps consume this through the segments API. The zero-config component renders everything in place; the hook hands back the parsed segments plus a default renderer for any of them, so custom placement is plain code: + +```tsx +const { segments, renderSegment } = useOpenUIMessage(response, { + libraries: [standard, slides], isStreaming, onAction, toolProvider, +}); + +segments.map((seg) => + seg.libraryId === "slides" + ? {renderSegment(seg)} + : {renderSegment(seg)} +); +``` + +The hook is the React binding of the engine: it parses incrementally as chunks arrive instead of rescanning the whole message, keeps segment identity stable across renders so completed blocks never re-render or lose input state, and wires each program segment's renderer once with the resolved library, actions, and tools. + +Segmentation also improves streaming: a program whose fence has closed is complete, so its queries fire and its inputs go live while the rest of the message is still arriving. + +### 4.7 Stored messages and the meta line *(proposed)* + +OpenUI Lang is a positional wire: `Button("Save", saveAction, "primary")` carries no prop names, and its meaning depends on the key order of the schema it was written against. Avro's binary encoding makes the same trade, and it forces the same solution: data that outlives its schema must travel with the schema it was written under. For OpenUI the traveling part is tiny. The text already identifies its own values (strings are quoted, numbers are bare), so the only knowledge a stored message loses is the names for its positions. + +When a response is stored, the host appends one metadata line per library used: + +````markdown +Here is the ticket view: + +```openui-lang +root = Card([title, closeBtn]) +title = Header("Ticket #4821") +closeBtn = Button("Close ticket", closeAction, "primary") +``` + +]]>openui:meta library=support@1.2.0 orders={"components":{"Card":["children","sources"],"Header":["title","subtitle"],"Button":["label","action","variant"]}} +```` + +The `orders` attribute is a projection of the LibrarySpec: each component, function, and action used in the message, mapped to its key order at generation time. Reading the message later, the client parses the text with the stored orders, binds every argument to its prop name at every depth of every expression, and re-serializes in the current library's order before rendering it or resending it as history. Reorders and removals resolve mechanically; added props need nothing; renames will use an explicit alias mapping, planned but not yet specified. When the stored orders match the current projection, the line is stripped and nothing else runs. Equality of orders, not of version numbers, is the fast path, so a schema change that forgot to bump the version is still caught. + +The line is host-authored. The model never writes it (the prompt does not teach the syntax) and never reads it (hosts strip it before the model sees history). A message that loses its meta line degrades to today's behavior: the text renders as-is against the current library. Library authors keep that degraded case safe by adding new props at the end of the key order instead of reordering, the append-only discipline every schema-evolution format converges on. + +The full sentinel grammar, the storage utilities, and publish-time compatibility checking live in [prompt.md](./prompt.md), section 7. + +### 4.8 Data components *(proposed)* + +Some components exist only to give structured props a positional, schema-checked shape: `Series("Revenue", [10, 20])` is data for a chart, not a thing that renders. Declaring one with `defineComponent({ schemaOnly: true })` makes that explicit. The call parses and validates exactly like any component (positional arguments by key order, the same error codes, the same participation in the meta line's orders projection), but it materializes as a plain object keyed by its prop names, `{ category: "Revenue", values: [10, 20] }`, inside the parent component's props instead of becoming a renderable element. + +The parent receives ready data with no unwrapping helpers, and the library ships no null renderer for the shape. A schema-only component cannot be a program's entry; a `root` bound to one recovers through the ordinary entry rules ([language.md](./language.md), section 2.2). In the LibrarySpec the component carries the `schemaOnly` marker ([prompt.md](./prompt.md), section 2). The wire format is unchanged: the model writes the same call either way, so flipping a component to `schemaOnly` is never a breaking change to stored programs. + +## 5. Security considerations + +- **The library is the capability boundary.** The model can only invoke components, tools, functions, and actions the library declares. Hosts MUST treat tool implementations and action handlers as the security perimeter and validate their inputs; the model chooses the arguments. +- **No code execution.** The language has no eval, no loops beyond `@Each`, and a closed expression surface. Clients MUST NOT extend evaluation with dynamic code paths. +- **URLs.** `@OpenUrl` payloads are model-authored. Hosts SHOULD restrict schemes to https and validate targets before navigation. +- **Prompt injection shows up as UI.** Injected instructions can produce misleading interfaces (a button labeled "Cancel" that submits, a fake login form). Hosts rendering third-party or multi-agent content SHOULD attribute UI to its source and keep sensitive actions behind their own confirmation surfaces. +- **Form state travels to the model.** `@ToAssistant` events carry form contents into the conversation. Hosts MUST NOT place secrets in form defaults and SHOULD scrub sensitive fields before forwarding. +- **Persisted programs replay.** Stored UIs re-render later, possibly against a changed library. The recovery rules of the [language spec](./language.md) apply; hosts MUST NOT execute stored mutations without a fresh user gesture. Stored messages may also carry host-authored metadata lines ([prompt.md](./prompt.md), section 7); clients MUST strip every such line from display, including kinds they do not recognize. diff --git a/spec/prompt.md b/spec/prompt.md new file mode 100644 index 000000000..cd283f3b5 --- /dev/null +++ b/spec/prompt.md @@ -0,0 +1,188 @@ +# The OpenUI Prompt and LibrarySpec + +**1.0-beta, community review draft** + +This document specifies everything that is sent to the model: the LibrarySpec a library serializes to, the system prompt generated from it, and the shape of the conversation the model sees. MUST, MUST NOT, SHOULD, and MAY are used as in RFC 2119, and *(proposed)* marks designed but unshipped behavior. + +## 1. Overview + +The model's entire knowledge of your UI system comes from two places: + +1. **The system prompt**: a deterministic function of the LibrarySpec, a set of feature flags, and a prompt template version (section 4). It teaches the language rules, the component vocabulary, and only the features the client supports. +2. **The conversation**: user messages, the model's own earlier OpenUI Lang responses, and context the host injects (error reports, form state, the current program in edit mode). + +Nothing else is sent. There is no hidden capability negotiation: if the prompt did not teach it, the model was not told about it, and the prompt MUST NOT teach features the target client does not support. + +## 2. The LibrarySpec + +A library serializes into two documents today. `library.toSpec()` emits the component signatures, for prompt generation: + +```json +{ + "root": "Card", + "components": { + "Button": { + "signature": "Button(label: string, action?: ActionExpression, variant?: \"primary\" | \"secondary\")", + "description": "A clickable button" + } + }, + "componentGroups": [] +} +``` + +`library.toJSONSchema()` emits the validation schema, the machine-readable half a client validates against. It carries one `$defs` entry per component: + +```json +{ + "$defs": { + "Button": { + "type": "object", + "properties": { + "label": { "type": "string" }, + "action": {}, + "variant": { "type": "string", "enum": ["primary", "secondary"] } + }, + "required": ["label"] + } + } +} +``` + +Reading it: the key order of `properties` is the positional argument order, so `Button("Save", a, "primary")` maps label, action, variant in that order; `required` lists the props whose absence invalidates the component; a property's `default` value fills a missing required argument before the component is dropped. Because the positional contract rides on JSON object key order, producers MUST emit `properties` in schema key order and consumers MUST parse with an order-preserving JSON parser. + +The CLI (`npx @openuidev/cli generate`) emits both artifacts by default: the system prompt, and a spec document combining `toSpec()` with a `schema` key holding `toJSONSchema()`. With `--out prompt.txt` the spec lands beside the prompt as `prompt.spec.json`; `--json-schema` prints the validation schema alone, and `--spec` prints the combined document. + +A unified LibrarySpec document *(proposed)* bundles both, adds a library name, and carries `functions`, `validators`, and `actions` registries as declarations only (name, params schema, return type, description); implementations never serialize. The document self-describes its format with `specVersion`, the version of this specification it conforms to, distinct from the library's own `version` (which tracks the library's content). It stores no signature strings: the schema is the single source of truth (its property key order is the positional contract), and the TypeScript-style signature is derived from it by the prompt template, so the two can never drift. A backend that wants the prompt calls `generateSystemPrompt` rather than reading a stored string; the CLI's current combined output still carries legacy signature strings during the transition. It also adds the `bindable` marker for two-way-binding props and the form-component markers; until those land, a prompt generated from the spec alone cannot enable the `bindings` flag faithfully, because bindability lives only in the library definition today. The document further carries `root` as one component name or an array of candidates, and the `schemaOnly` marker for data components ([overview.md](./overview.md), section 4.8). The LibrarySpec is the interchange format between platforms: a Kotlin library and a TypeScript library that emit the same spec are interchangeable. + +The LibrarySpec drives prompt generation and validation. It does not drive rendering: components are implemented natively on each platform, and each platform's library definition owns its components' behavior, including which props are bindable and how inputs attach to forms. A native client is not a generic schema-driven widget engine; it is the same components, written for that platform, agreeing on one contract. + +## 3. Conformance rules for libraries + +- Component names MUST start with an uppercase letter and match the identifier rule. Registered function and action names SHOULD follow the built-ins' uppercase convention (`@FormatCurrency`, `@ApproveInvoice`); validator names MUST start lowercase, since they are rules-object keys. +- Required props MUST precede optional props in schema key order. +- Key order is part of the public contract. A reorder changes the meaning of every generated prompt and of every stored program that lacks its meta line (section 7), so authors SHOULD add new props at the end of the key order and treat reorders and removals as version-bumping changes. A publish-time compatibility check *(proposed)* classifies these changes mechanically (section 7.6). +- Libraries MUST NOT define components named `Query`, `Mutation`, or `Action`, and MUST NOT register functions shadowing built-ins. +- Every component in `root` and in `componentGroups` MUST exist in `components`. +- Argument constraints *(proposed)* are limited to the JSON-Schema-mappable keywords, so any platform can enforce them from the schema document alone: `minLength`, `maxLength`, `pattern`, `format` (`uri`, `email`) on strings; `minimum`, `maximum`, integer type on numbers; `minItems`, `maxItems` on arrays; `default` on any prop. A violation renders the value as-is and reports a warning diagnostic; it MUST NOT drop the component. Custom-function refinements do not serialize and are unsupported; implementations SHOULD ignore them with a definition-time warning. +- Definition-time enforcement of these rules is proposed; the reference `createLibrary` currently checks only that `root` names a member component. + +## 4. Prompt generation + +The system prompt is a deterministic function of the LibrarySpec, a set of flags, and a prompt template. The canonical entry point is `generateSystemPrompt({ library, promptOptions })`; the older flat `generatePrompt(spec)` form is deprecated. The canonical template is the reference implementation's generator at a tagged release; this document does not reprint it, so byte-level determinism is defined against that tagged template, and publishing the template as a normative appendix is planned alongside the fixture suite. Given the same spec, flags, and template version, prompt generators MUST produce the same prompt bytes, whichever platform runs them. This determinism lets a gateway generate the prompt server-side from a client's LibrarySpec and get identical model behavior to a client that generated it locally. + +The flags gate feature sections: `toolCalls` (queries, mutations, tools), `bindings` (state and `$binding` props), `editMode` (patching), `inlineMode` (prose plus fenced code). Built-in function documentation appears only when `toolCalls` or `bindings` is set. The inline-mode section MUST teach two rules explicitly: openui-lang belongs only in fences, with a `text`-tagged fence showing code without rendering it, and independent UI blocks split into separate fences with prose between them. Models follow both reliably when taught, and just as reliably coalesce a whole response into one fence when the second rule is left out. + +The generated prompt contains, in order: the syntax rules (statement shape, positional arguments, the root convention: a single component statement named `root`), the component catalog (section 5), the flag-gated sections for built-ins and tools, the hoisting and streaming guidance, and the flag-gated sections for editing and inline mode. Components render in the spec's key order. `componentGroups` are named groups (`{ name, components, notes? }`) that organize the catalog into titled sections; every listed component must exist in `components`. + +Tool descriptors are supplied to prompt generation as options, not in the LibrarySpec today: each is a name string or a ToolSpec (`{ name, description?, inputSchema, outputSchema, annotations? }`). Tools are described to the model with their names, typed signatures, and default values derived from their output schemas. The unified LibrarySpec *(proposed)* will carry tool declarations so spec-driven generation can build this section. + +## 5. Component signatures and descriptions + +Each component appears in the prompt as a single-line signature joined to its description; the separator in the current template is the em dash character: + +``` +Button(label: string, action?: ActionExpression, variant?: "primary" | "secondary") — A clickable button +``` + +The signature line MUST stay single-line so it can be quoted whole in prompts and logs. Error `hint` fields do not reuse it: they carry a compact signature built from the JSON Schema, prop names only with required props starred ([language.md](./language.md), section 8.3). + +The signature string format is part of the template: primitive types print as `string`, `number`, `boolean`, `any`; enums as quoted alternatives joined by `|`; arrays as `T[]`; inline objects as `{field: type}`; unions joined by `|`; optional props with `?` before the colon; bindable props as `$binding`. Names like `ActionExpression` come from schema id tags the library registers for non-component schemas. The exact grammar of the signature string is pinned by the reference template and will appear in the normative template appendix. + +Per-prop descriptions and usage examples *(proposed)* render as a JSDoc block above the signature. `@param` lines come from prop descriptions in the schema (`.describe()` in the Zod definition, a `description` field in the LibrarySpec); `@example` lines come from the component's `example` field (a string or an array of strings, one `@example` entry each): + +``` +/** + * A clickable button + * @param label - Text shown on the button + * @param variant - Visual weight, defaults to primary + * @example + * btn = Button("Save changes", saveAction, "primary") + */ +Button(label: string, action?: ActionExpression, variant?: "primary" | "secondary") +``` + +Argument constraints (section 3) also render here *(proposed)*: a deterministic suffix on the `@param` line, derived from the schema, so the model learns the constraint without the author restating it: `@param value - Stars filled (integer, 0 to 5)`, `@param items - (min 2 items)`. A constraint with no hand-written description still produces its `@param` line. + +JSDoc is chosen because models require no teaching to read it, and because it costs nothing when unused: the block appears only when a component has prop descriptions, an argument constraint, or an example; otherwise the component keeps the compact single-line form, and a library with none of these produces no boilerplate at all. Descriptions SHOULD add semantics the type does not carry (units, ranges, when to use which enum member), not restate the type. *(Today the reference prompt generator drops per-prop descriptions; this section is the fix for that gap.)* + +## 6. The conversation + +### 6.1 Assistant history + +The model's earlier responses appear in history as the OpenUI Lang text it generated (with surrounding prose when inline mode is on). History is the strongest style signal the model gets: whatever form its earlier messages use is the form it will continue to use, so the stored form of history matters. Section 7 *(proposed)* defines the storage protocol: stored responses carry a metadata line naming the library and key orders they were written against, and hosts re-serialize history into the current dialect, with every metadata line stripped, before the model sees it. + +### 6.2 Error feedback + +When a response produced errors, the host SHOULD include the structured error list (wire shape in [language.md](./language.md), section 8.3) in the next request context. The `hint` field is designed so the model can produce a one-line patch without re-reading the library documentation. + +### 6.3 Action events + +A `continue_conversation` event becomes the next user-side turn: the human-friendly message, the event context, and the current form state travel together, so the model sees what the user entered without asking. + +### 6.4 Edit mode + +With `editMode` on, the host sends the current program with the request, and the prompt teaches the model to respond with only the changed statements. Without it, every response is a complete program. + +## 7. Stored messages and the sentinel protocol *(proposed)* + +OpenUI Lang is a positional format: argument meaning depends on schema key order, and a stored message can outlive the key order it was written against. This is the schema-evolution problem Avro solves for positional binary data, and this section adopts Avro's solution: the writer's schema travels with the data, either embedded (the `orders` attribute below) or by reference to a registry (slim mode). OpenUI needs far less than Avro carries, because the text encodes its own value types; the only writer knowledge a stored message loses is the prop names for its argument positions. Like every proposed section, this one is written in the present tense so the design can be judged as it would ship; none of it is implemented today. + +### 7.1 The sentinel line + +A sentinel line is a line beginning with `]]>openui:` followed by a kind identifier and, optionally, a single space and space-separated `key=value` attributes. Values contain no spaces (identifiers, `id@semver`, or compact JSON), so the line splits on spaces. Sentinel lines MUST be line-anchored; the byte sequence mid-line is content, not a marker. + +Sentinel lines are host-authored. The prompt MUST NOT teach the syntax. The party assembling a model request MUST strip every sentinel line before text reaches the model, and the party rendering MUST strip every sentinel line from display, in both cases including kinds and attributes they do not recognize; unknown attributes on a recognized kind are ignored. During streaming, a client SHOULD withhold from display an incomplete final line that is a prefix of, or begins with, `]]>openui:`, so a marker split across chunks never flashes as text. + +### 7.2 Kinds + +| kind | channel | position | payload | status | +| --- | --- | --- | --- | --- | +| `meta` | assistant message | trailing block, one line per library | `library=id@semver`, `orders={...}` (absent in slim mode) | proposed, defined here | +| `context` | user and assistant messages | opens a section; body runs to the next marker or end of message | JSON (form state, action event context) | in use today; normative | +| `content` | assistant message | opens a section | attributes | legacy, in use today; parsers MUST keep accepting it | +| `end` | assistant message | standalone line, anywhere | none; attributes reserved | in use today; optional | + +A message's trailing block is the maximal run of sentinel lines at its end; `meta` lines are read only there. If a legacy `content` header and a `meta` line disagree, `meta` wins. Only `context` and legacy `content` open sections: every other kind, including kinds defined in the future, MUST be a standalone line, so a parser that does not recognize a kind can always strip exactly one line and never leaks a section body. `end` marks the last stored chunk of a live stream, so its absence from a persisted message indicates a stream that died mid-write; how a response ended on the wire is structural, never in-band. Managed backends define additional kinds on other channels (an artifact carrier on tool results, a configuration block on request instructions); they are profile extensions, they follow the single-line rule, and they impose nothing on clients of this specification. + +### 7.3 The meta line + +When a host stores an assistant response containing OpenUI Lang, it SHOULD append one `meta` line per library used (this is the storage half of the versioning guarantee; without it, messages degrade as described in 7.4): + +``` +]]>openui:meta library=support@1.2.0 orders={"components":{"Card":["children","sources"],"Button":["label","action","variant"]},"actions":{"ApproveInvoice":["id"]}} +``` + +`library` is the library id and version the prompt was generated from. `orders` is the key-order projection of the LibrarySpec, restricted to names the message uses, grouped by registry: `components`, `functions`, and `actions`, each mapping a used name to its params in schema key order, with empty groups omitted. The grouping keeps the projection unambiguous, since functions and actions share the `@` call form and name casing is a convention, not a discriminator. Arrays make the order explicit, so any JSON parser reads the projection correctly; the order-preserving requirement of section 2 does not apply here. In slim mode `orders` is absent and the reader resolves the same projection from a registry by `library`; embedded orders keep the message self-contained where no registry exists. + +Writers MUST be idempotent: an existing trailing `meta` line for the same library id is replaced, never duplicated. Writers SHOULD put one blank line before the trailing block for readability; parsers accept both. + +### 7.4 Reading stored messages + +Before a stored message is rendered or resent as history, the host normalizes it: + +1. If the stored orders equal the current projection for every name used, strip the sentinel lines and use the text as-is. Equality of orders, not of version numbers, is the fast path: a schema change that shipped without a version bump is still caught. The version attribute remains useful as a skew diagnostic. +2. Otherwise, parse the text binding argument positions to prop names through the stored orders, then re-serialize each statement in the current library's key order: repositioned props move, gaps before a later argument are filled with `null`, props absent from the current schema are dropped, and trailing unwritten props are omitted. Renames need an explicit alias mapping, planned alongside this protocol and not yet specified; until it lands, a renamed name behaves as a removal. A name with no current entry is dropped with a diagnostic. + +Re-binding applies to every component, registered-function, and declared-action call at every depth: inside arrays, object values, ternary branches, `@Each` templates, and action plans, not only at the top of a statement. A call whose name is absent from the stored orders is read with the current library's order and reported with a skew diagnostic. Beyond that re-binding, normalization is textual rewriting: expressions, state declarations, and hoisted references re-serialize as written, values are never evaluated or coerced, and line comments are not preserved. The model MUST NOT receive sentinel bytes, and SHOULD receive history in the current dialect, since history is the strongest style signal (section 6.1). The conformance fixtures pin this behavior with nested-call cases. + +A message without a meta line (never enriched, or truncated in storage) is read as-is against the current library, which is exactly the pre-protocol behavior. The append-only discipline of section 3 exists for this case: additions at the end of the key order keep even degraded messages correct, so the meta line only has to earn its keep for reorders and removals. + +### 7.5 The storage boundary + +The reference implementation ships the protocol as two functions: one wraps a completed response with its meta lines, one normalizes a stored message and strips every sentinel for the model. Where they run depends on who owns the prompt. A managed backend that generated the prompt runs both server-side, on the response stream and on incoming history, so clients store and echo plain strings. A self-hosted backend calls the wrap function where it already handles the stream; a client-side stack calls the normalize function at its API boundary before resending history. A client that persists messages itself MAY wrap on the client; such a stamp reflects the client's library, not necessarily the prompt's, and SHOULD be marked so skew warnings can weigh it accordingly. + +### 7.6 Compatibility checking *(proposed)* + +Schema registries for positional formats reject incompatible schemas at publish time rather than letting readers fail later, and the same check applies here. When a new library version is published or a prompt is generated, diff the full component schemas against the previous version, not only the key orders, and classify each component: additions at the end of the key order are compatible even for messages without a meta line, except that a newly required prop MUST carry a `default`, since stored messages cannot supply it (and a required prop added after optionals also violates section 3's ordering rule); reorders and removals are compatible only through the meta line; a removed name alongside a new name suggests a rename and warrants a warning when no alias covers it; a type or enum change on an existing prop warrants a warning, since stored values may now fail validation with no order change to detect it by; an unchanged version with a changed projection is an error, the forgotten version bump. A registry MAY reject on these classes; a generator SHOULD warn. + +### 7.7 Security + +In-band metadata is forgeable in principle: model output or third-party text could contain a marker-shaped line. The mitigations are structural. The syntax is never taught, so the model does not produce it in practice; display parsers strip every sentinel line regardless of kind, so forged lines never render; `meta` is only read from the trailing block, and the blast radius of a forged projection is one message re-binding incorrectly. Hosts MUST NOT derive trust decisions from sentinel attributes. + +## Appendix A. Changelog + +- **2026-08-12**: Fixes from three independent review passes. Naming: registered function and action names follow the built-ins' uppercase convention; the `orders` projection grouped by registry (`components`/`functions`/`actions`) so casing is no longer a discriminator. Section 7 hardened: proposed-tense disclaimer; strip obligations split between the request assembler and the renderer; single-line rule for all future kinds; `end` defined; re-binding stated to apply at every expression depth with fixtures to pin it; rename aliases marked as planned rather than assumed; the compatibility check widened to full-schema diffs with the required-prop-needs-default rule. Section 2 documents `root` plurality and the `schemaOnly` marker. Section 4 pins the two inline-mode teaching rules (fence-only code, fence splitting), the second validated empirically. +- **2026-08-07**: Added section 7, the proposed storage protocol: sentinel line grammar and kind registry, the meta line carrying the key-order projection, normalization with an orders-equality fast path, slim mode, the storage boundary, and publish-time compatibility checking. Section 3's key-order rule gains the append-only guidance and section 6.1 references the protocol. +- **2026-08-05**: Draft renamed from 0.9 to 1.0-beta; earlier entries keep the old name. +- **2026-08-04**: Argument constraints limited to the JSON-Schema-mappable keywords with warn-and-render recovery, rendered as deterministic `@param` suffixes; `example` documented as string or array; CLI section updated to PR #811 behavior (default generate emits prompt plus `.spec.json`, `--json-schema` prints the validation schema, `generateSystemPrompt` canonical); unified LibrarySpec standardized as schema-only with signatures derived by the prompt template. +- **2026-08-03**: Split out of the 0.9 draft as its own document; added the conversation contract and the proposed JSDoc form for per-prop descriptions and examples. Review fixes: determinism scoped to a tagged prompt template, the validation-schema half of the LibrarySpec documented with the order-preserving requirement, CLI output described as shipped, tool descriptor input defined, signature format and the em dash separator documented as emitted.