Skip to content

feat: support VirtualTable Read with inline format#106

Merged
wackywendell merged 2 commits intomainfrom
wendell/inline-virtual-table
Apr 22, 2026
Merged

feat: support VirtualTable Read with inline format#106
wackywendell merged 2 commits intomainfrom
wendell/inline-virtual-table

Conversation

@wackywendell
Copy link
Copy Markdown
Collaborator

@wackywendell wackywendell commented Apr 17, 2026

Description

Depends on #107 (parser refactoring), which must merge first. Factored out of #101 - this is a simpler version of that (only inline parsing, no + Row syntax).

Add support for VirtualTable in ReadRel (closes #56), enabling inline literal data tables in Substrait plans — similar to SQL VALUES clauses.

The virtual read is written as a Read:Virtual operator, with rows inline as positional arguments:

Read:Virtual[(1, 'alice'), (2, 'bob') => id:i64, name:string]

Use _ for an empty virtual table (no rows):

Read:Virtual[_ => id:i64, name:string]

Design decisions

Type of Change

  • New feature
  • Documentation update

Testing

  • Added roundtrip tests (inline, empty table, single-column, VirtualTable in complex plans)
  • All existing tests pass
  • GRAMMAR.md doc tests compile and pass

Checklist

  • Code follows Rust conventions
  • Documentation updated if needed
  • No breaking changes (or breaking changes documented)
  • Pre-commit hooks pass

Related Issues

Closes #56

@wackywendell wackywendell force-pushed the wendell/inline-virtual-table branch from 2d99fa9 to 5f27f0a Compare April 17, 2026 22:40
@wackywendell wackywendell marked this pull request as ready for review April 20, 2026 15:10
@wackywendell wackywendell requested a review from a team as a code owner April 20, 2026 15:10
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5f27f0a6c1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/parser/relations.rs Outdated
Comment thread src/parser/relations.rs Outdated
fix: return parse errors instead of panicking on invalid VirtualTable expressions
@wackywendell wackywendell force-pushed the wendell/inline-virtual-table branch from 62c2b3c to 51ad742 Compare April 20, 2026 22:24
Copy link
Copy Markdown
Contributor

@benbellick benbellick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments but otherwise LGTM

Comment thread tests/plan_roundtrip.rs
let plan = r#"
=== Plan
Root[id, name]
Read:Virtual[_ => id:i64, name:string]"#;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably fine because I think this is the convention for other relations and it is better to be consistent, but I find the usage of _ here a bit unintuitive. To me, _ communicates "I don't care", which doesn't really make sense in this context, but that is where my mind goes.

I think a more explicit syntax would be to use [] in all of the places, e.g.:

  Read:Virtual[[] => id:i64, name:string]"#;

or

  Read:Virtual[[(1), (2), (3)] => id:i64]"#;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point about _; it's used to mean 'unspecified value' in several languages, whereas here it means 'empty', which is quite different.

That said - I'm not really sure what would be a better replacement:

  • [] brackets around args: I'd rather not add extra brackets if we don't need to - the syntax operator[args... => columns] seems fairly unambiguous, and having to put extra brackets around args everywhere adds clutter.
  • () for empty: Nice syntax, but ambiguous: it's not clear if Operator[() => …] means "1 argument, an empty tuple" or "no arguments".
  • empty keyword: E.g. Operator[empty => …]. Right now, there are no keywords, and this would conflict with the possibility of having an identifier - e.g. Read[empty => …] currently means "a read from the table named empty"
  • Nothing: Allow either Operator[columns] like Project and Root(feat: make '=>' optional when relation arguments are absent #104), or Operator[=> columns] / Operator[args =>] for empty columns / args. The first has precedent and should be OK; the second is unambiguous.

So for now, I think we keep it...

Comment thread tests/plan_roundtrip.rs
Comment thread GRAMMAR.md Outdated

`"Read:Virtual" "[" (virtual_row ("," virtual_row)*)? "=>" named_column_list "]"`

Where `virtual_row := "(" expression ("," expression)* ")"` — a parenthesized tuple of expressions forming one row. Use `_` in place of rows for an empty virtual table.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Where `virtual_row := "(" expression ("," expression)* ")"` — a parenthesized tuple of expressions forming one row. Use `_` in place of rows for an empty virtual table.
Where `virtual_row := "(" expression ("," expression)* ")"` — a parenthesized tuple of expressions forming one row. For an empty virtual table, write `_` in place of the entire row list, e.g. `Read:Virtual[_ => id:i64]`. The `_` replaces the whole list and cannot appear alongside rows.

What do you think of something like this?

@wackywendell wackywendell merged commit 5679f9c into main Apr 22, 2026
4 checks passed
@wackywendell wackywendell deleted the wendell/inline-virtual-table branch April 22, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Support Textify Representation of VirtualTable ReadRel Type

2 participants