Top-level list & definition & unspecified fields

[lucendio]

Hi everyone,

I'm probably holding it wrong, but what I gathered from the tutorials the following should not be valid - I thought, that's what Closed structs are for. Anyway, here it goes:

items.cue

#Item: {
    aa: int
    bb: string
    dd?: bool
}

[ ...#Item ]

items.yaml

- aa: 42
  bb: 'foo'
- bb: 'bar'
  aa: 6
  ff: true

I would expect that cue vet items.cue items.yaml fails for the second item, because ff: bool is not part of the #Item definition, but it doesn't. The command executes successful. So, a) I'm doing something wrong, or b) it is expected, or c) #822 is addressing this. Which is it?

Thanks in advance for the help!

[rogpeppe]

There's an interesting and somewhat subtle reason for this: because [ ... #Item ] has been embedded at the top level, it's "opened up" to new fields (as long as they don't conflict).

This is the reason we can add fields when embedding in examples like this:

#Item: {
    aa: int
    bb: string
    dd?: bool
}

#OtherItem: {
	#Item
	ff: bool
}

x: #OtherItem & {
	bb: "bar"
	aa: 6
	ff: true
}

So, technically your example is similar to this CUE, which is legitimate:

#S: {
    a: int
}

[... #S]
[{	a: 1
	other: 2
}]

There does however seem to be a discrepancy between the treatment of YAML and CUE files on the command line. For example (using testscript to demo):

# this is fine
exec cue vet -c a.cue b.cue

# this fails, even though the contents of b.cue and b.yaml
# are ostensibly identical.
exec cue vet -c a.cue b.yaml

-- a.cue --
#S: {
    a: int
}

#S
-- b.cue --
a: 1
other: 2
-- b.yaml --
a: 1
other: 2

Arrays behave a bit differently here, although they probably shouldn't. This seems like a bug.

I don't actually know what the semantics of independent files on the command line are meant to be when there's no package. Do they all exist inside the same package (in which case your example is technically working as intended) or should they be treated as separate packages for the purposes of unification.

I suspect that in your case it was important to verify that the top level YAML corresponded directly to the schema with no spelling mistakes in field names. I'm not aware of a way of doing that currently, but there may well be on that I haven't thought of.

For now, the workaround would be to verify the YAML without the need to embed. One solution might be to use the -l flag to place the YAML inside another field. For example:

# this fails as it should do.
! exec cue vet -l '"yaml"' a.cue x.yaml

-- a.cue --
#Item: {
    aa: int
    bb: string
    dd?: bool
}

yaml: [ ...#Item ]

-- x.yaml --
- aa: 42
  bb: 'foo'
- bb: 'bar'
  aa: 6
  ff: true

[verdverm]

You can use cue vet -d to validate against a specific schema or definition

cue vet schema.cue -d '#schema' data.yaml

-- data.yaml --
- aa: 42
  bb: 'foo'
- bb: 'bar'
  aa: 6
  ff: true
  
-- schema.cue --
#schema: [...{
	aa: int
	bb: string
	dd?: bool
}]

[philippgille]

Hello 👋 , I recently ran into the same issue (see here), and noticed that both proposed workarounds help. But I'm not sure about the advantages/disadvantages of the two.

The -d flag is clear to me: "expression to select schema for evaluating values in non-CUE files"
Not sure about the -l flag though: "CUE expression for single path component"

With the -d workaround the schema remains a "clean" schema, i.e. it's not mixing definitions with data. The official tutorial says:

In CUE, schemas are typically written as Definitions. A definition is a field which identifier starts with # or _#

(source)

Any other points to consider?