Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# hostRequirements.amounts[].max may be a format string under FEATURE_BUNDLE_1 (§3.3.1 L960),
# the same as `min` at §3.3.1 L958. The numeric constraint cannot be checked at decode, so it
# must be checked after resolution at job creation. Here the parameter resolves to a non-numeric
# string and the run must fail.
#
# Companion to 3.3.1--amount-min-format-string-resolves-to-non-number.invalid.test.yaml. `min`
# and `max` carry different types in the spec, <nonnegativefloat> against <positivefloat>, so
# they are covered separately rather than assumed symmetric.
#
# Spec references, Template Schemas 2023-09:
# §3.3.1 L960 amounts[].max may be `<positivefloatstring>` under FEATURE_BUNDLE_1, @fmtstring
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L960
# §3.3.1 L958 the min counterpart, covered separately because the types differ
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L958
template:
specificationVersion: jobtemplate-2023-09
extensions:
- FEATURE_BUNDLE_1
name: TestJob
parameterDefinitions:
- name: CpuMax
type: STRING
default: "abc"
steps:
- name: Step1
hostRequirements:
amounts:
- name: amount.worker.vcpu
max: "{{Param.CpuMax}}"
script:
actions:
onRun:
command: python
args:
- -c
- print(r'RAN')
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# hostRequirements.amounts[].min may be a format string under FEATURE_BUNDLE_1 (§3.3.1 L958).
# The numeric constraint cannot be checked at decode, so it must be checked after resolution at
# job creation. Here the parameter resolves to a non-numeric string and the run must fail.
#
# This is the negative half of the amount-resolution gap. The positive half, asserting that min
# and max resolve to the CORRECT numbers, is not writable today: no openjd CLI surfaces resolved
# host requirements, in summary --output json or anywhere else.
#
# Spec references, Template Schemas 2023-09:
# §3.3.1 L958 amounts[].min may be `<nonnegativefloatstring>` under FEATURE_BUNDLE_1, @fmtstring
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L958
# §7.4 L2017 format strings not annotated @fmtstring[host] resolve at job creation
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L2017
template:
specificationVersion: jobtemplate-2023-09
extensions:
- FEATURE_BUNDLE_1
name: TestJob
parameterDefinitions:
- name: CpuMin
type: STRING
default: "abc"
steps:
- name: Step1
hostRequirements:
amounts:
- name: amount.worker.vcpu
min: "{{Param.CpuMin}}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding 2 (secondary): amount min/max plain string-form numeric comparison is untested.

amounts[].min/.max (§3.3.1 L958/960) are <nonnegativefloat> | <nonnegativefloatstring> and @fmtstring. This fixture covers the @fmtstring→non-number path. The plain string-form path is not covered anywhere: min/max written as floatstrings where min > max must be caught by a numeric comparison (base/job_templates/3.3.1--amount-min-greater-than-max.invalid.yaml uses numeric literals only). Same lexical-vs-numeric class as the INT bounds fixtures. Lower severity than the FLOAT parameter-definition gap, but currently a blind spot. Suggest a min/max floatstring min > max invalid fixture.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I will create more tests to cover this as well.

script:
actions:
onRun:
command: python
args:
- -c
- print(r'RAN')
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# hostRequirements.amounts[].max is typed <positivefloat> (§3.3.1 L959-960), and §3.3.1 confirms
# it separately from min, which is <nonnegativefloat>. So 0 is legal for min and illegal for
# max.
#
# Under FEATURE_BUNDLE_1 max may be a format string, and the positivity constraint cannot be
# checked at decode because the value is not yet known. It must be re-checked after resolution
# at job creation. Here the parameter resolves to 0.
#
# This is the only assertion in the suite that distinguishes max semantics from min semantics
# for a resolved value. A literal max: 0 is already rejected at decode by
# base/job_templates/3.3.1--amount-max-zero.invalid.yaml, so only the resolved path is
# unguarded.
#
# Spec references, Template Schemas 2023-09:
# §3.3.1 L959 amounts[].max is `<positivefloat>`, so 0 is illegal
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L959
# §3.3.1 L957 amounts[].min is `<nonnegativefloat>`, so 0 IS legal there
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L957
# §3.3.1 L960 max may be a format string under FEATURE_BUNDLE_1, so the bound defers past decode
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L960
template:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

specificationVersion: jobtemplate-2023-09
extensions:
- FEATURE_BUNDLE_1
name: TestJob
parameterDefinitions:
- name: CpuMax
type: INT
# <intstring> form, so the positivity check must survive both coercions.
default: '0'
steps:
- name: Step1
hostRequirements:
amounts:
- name: amount.worker.vcpu
max: "{{Param.CpuMax}}"
script:
actions:
onRun:
command: python
args:
- -c
- print(r'RAN')
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Proposed FEATURE_BUNDLE_1 job fixtures: post-resolution bound checks

These fixtures are believed spec-correct and currently fail at least one reference
implementation. They live here rather than in `../` because the runner discovers test
files with a non-recursive glob, so `proposed/` is excluded and the suite stays green.
Promotion is `git mv` up one directory with no edit to the fixture.

| Fixture | Construct | Observed |
|---|---|---|
| `3.3.1--amount-max-format-string-resolves-to-zero.invalid.test.yaml` | `amounts[].max: "{{Param.CpuMax}}"` where the parameter default is `0`, against the `<positivefloat>` type at §3.3.1 L959 | openjd-model Python rejects at job creation. The Rust CLI accepts the resolved `0` and runs the job to completion |

## Classification

Implementation fix.

`max` is `<positivefloat>` while `min` is `<nonnegativefloat>`, so `0` is legal for one and
not the other. A literal `max: 0` is already rejected at decode by
`../../job_templates/3.3.1--amount-max-zero.invalid.yaml`. Under FEATURE_BUNDLE_1 the value
may be a format string, in which case the positivity constraint cannot be checked at decode
and must be re-checked after resolution. The Python path does this; the Rust path defers the
check and never resumes it.

This is the same defect shape as the two fixtures in
`../../../TASK_CHUNKING/jobs/proposed/` and `../../../base/jobs/proposed/`: validation
correctly deferred for a format string, never resumed. Reviewers may prefer to treat all of
them as one issue.

The green twin,
`../3.3.1--amount-max-format-string-resolves-to-non-number.invalid.test.yaml`, covers a
`max` that resolves to a non-numeric string. Both implementations reject that, which is why
it sits in the live suite. The divergence recorded here is specific to the positivity bound
rather than to numeric parsing.

Not covered by any fixture, here or in `../`: the positive case asserting a resolved `max`
is the correct number. No `openjd` CLI surfaces resolved host requirements, and the runner
asserts only on stdout and task status, so a host requirement has no observable effect on a
single-host run. The negative cases are the reachable half.

## Spec references, Template Schemas 2023-09

- [§3.3.1 L959](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L959) amounts[].max is `<positivefloat>`, so 0 is illegal
- [§3.3.1 L957](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L957) amounts[].min is `<nonnegativefloat>`, so 0 IS legal there
- [§3.3.1 L960](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L960) max may be a format string under FEATURE_BUNDLE_1, so the bound defers past decode

Line numbers are a locator for where each claim was verified. The section numbers are the
durable reference if the spec is re-flowed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# chunks.defaultTaskCount is annotated @fmtstring (§3.4.1.5 L1263), so it may be a parameter
# reference and must be resolved at job creation. This case pins the resolved value by its
# observable effect: ChunkSize=3 over the range 1-8 can only produce the boundaries 1-3, 4-6,
# 7-8. An implementation that reads the pre-resolution template value fails here rather than at
# some later stage.
#
# The parameter default is written in the <intstring> string form deliberately. A native
# `default: 3` would exercise only the format-string layer; the string form chains both, and the
# assertion is identical, so it is strictly stronger at no cost.
#
# Status-only by design: a 3-chunk run cannot self-assert, because each task sees only its own
# output (see the conformance-tests README).
#
# Spec references, Template Schemas 2023-09:
# §3.4.1.5 L1263 chunks.defaultTaskCount is `<integer> | <intstring>` and @fmtstring
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1263
# §2.3 L306 INT parameter default is `<integer> | <intstring>`
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L306
# §7.4 L2017 format strings not annotated @fmtstring[host] resolve at job creation
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L2017
template:
specificationVersion: jobtemplate-2023-09
extensions:
- TASK_CHUNKING
name: ChunkTest
parameterDefinitions:
- name: ChunkSize
type: INT
# <intstring> form, with a leading zero. INT parameter defaults are
# `<integer> | <intstring>` in base 2023-09 (§2.3 L306), so this composes the two
# coercions the fixture cares about: the string default must parse to 3, and the
# format string below must then resolve to it. The boundaries assert both at once.
default: '03'
steps:
- name: Step1
parameterSpace:
taskParameterDefinitions:
- name: Frame
type: CHUNK[INT]
range: 1-8
chunks:
defaultTaskCount: "{{Param.ChunkSize}}"
rangeConstraint: CONTIGUOUS
script:
actions:
onRun:
command: python
args:
- -c
- print(r'CHUNK:{{Task.Param.Frame}}')
expected:
output:
- 'CHUNK:1-3'
- 'CHUNK:4-6'
- 'CHUNK:7-8'
forbidden:
- 'CHUNK:1-8'
- 'CHUNK:9'
39 changes: 39 additions & 0 deletions conformance-tests/2023-09/TASK_CHUNKING/jobs/proposed/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Proposed TASK_CHUNKING job fixtures: post-resolution bound checks

These fixtures are believed spec-correct and currently fail at least one reference
implementation. They live here rather than in `../` because the runner discovers test
files with a non-recursive glob, so `proposed/` is excluded and the suite stays green.
Promotion is `git mv` up one directory with no edit to the fixture.

| Fixture | Construct | Observed |
|---|---|---|
| `default-task-count-format-string-resolves-to-zero.invalid.test.yaml` | `chunks.defaultTaskCount: "{{Param.ChunkSize}}"` where the parameter default is `0`, against the documented minimum of 1 (§3.4.1.5 L1276) | openjd-model Python rejects at job creation. The Rust CLI reports `Template ... passes validation checks` and then runs, logging `Frame(CHUNK[INT]) = 1-1`, so the resolved `0` is treated as `1` |

## Classification

Implementation fix.

The minimum-of-1 bound cannot be checked at decode when the value is a format string,
because the value is not yet known, and `TASK_CHUNKING/job_templates/default-task-count-zero.invalid.yaml`
already covers the literal `0` case at decode. The bound must therefore be re-checked
after resolution at job creation. The Python path does this; the Rust path defers the
check and never resumes it.

Cross-check: in `openjd-model/src/template/validate_v2023_09/task_chunking.rs` the
`>= 1` comparison is guarded on the chunk count being a literal integer, which is
correct for decode-time validation. No equivalent check appears on the job-creation
path, so a format-string value reaches the scheduler unvalidated.

A green twin ships in `../default-task-count-format-string.test.yaml`, which asserts
that a format-string `defaultTaskCount` resolves and produces the expected chunk
boundaries. That case passes both implementations, so the divergence recorded here is
specific to the bound check rather than to resolution itself.

## Spec references, Template Schemas 2023-09

- [§3.4.1.5 L1263](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1263) chunks.defaultTaskCount is @fmtstring, so the bound defers past decode
- [§3.4.1.5 L1276](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1276) defaultTaskCount minimum value: 1, the bound not re-checked after resolution
- [§7.4 L2017](https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L2017) format strings not annotated @fmtstring[host] resolve at job creation

Line numbers are a locator for where each claim was verified. The section numbers are the
durable reference if the spec is re-flowed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# chunks.defaultTaskCount has a minimum of 1. When the value is written as a format string the
# bound cannot be checked at decode, because the value is not yet known, so it must be checked
# after resolution at job creation. This case supplies a parameter that resolves to 0 and must
# therefore fail.
#
# .invalid.test.yaml rather than .invalid.yaml: the template is statically valid and the error
# only fires once the expression is evaluated.
#
# Spec references, Template Schemas 2023-09:
# §3.4.1.5 L1263 chunks.defaultTaskCount is @fmtstring, so the bound cannot be checked at decode
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1263
# §3.4.1.5 L1276 defaultTaskCount minimum value: 1
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1276
# §2.3 L306 INT parameter default is `<integer> | <intstring>`
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L306
template:
specificationVersion: jobtemplate-2023-09
extensions:
- TASK_CHUNKING
name: ChunkTest
parameterDefinitions:
- name: ChunkSize
type: INT
# <intstring> form, so the bound check must survive both coercions.
default: '0'
steps:
- name: Step1
parameterSpace:
taskParameterDefinitions:
- name: Frame
type: CHUNK[INT]
range: 1-8
chunks:
defaultTaskCount: "{{Param.ChunkSize}}"
rangeConstraint: CONTIGUOUS
script:
actions:
onRun:
command: python
args:
- -c
- print(r'CHUNK:{{Task.Param.Frame}}')
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# chunks.targetRuntimeSeconds is annotated @fmtstring (§3.4.1.5 L1264), so it may be a parameter
# reference and must be resolved at job creation.
#
# The resolved value is 0, and §3.4.1.5 says that when targetRuntimeSeconds is 0 a scheduler
# should ignore it and use defaultTaskCount for every chunk. That makes the assertion
# deterministic: the boundaries must be exactly those defaultTaskCount implies, with no adaptive
# resizing. A non-zero resolved value would let a conforming scheduler adjust the chunk size, so
# this case pins the one resolved value whose effect is fixed.
#
# An implementation that reads the pre-resolution template value sees the literal
# "{{Param.TargetRuntime}}" instead of 0 here.
#
# Spec references, Template Schemas 2023-09:
# §3.4.1.5 L1264 chunks.targetRuntimeSeconds is @fmtstring
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1264
# §3.4.1.5 L1280 when the value is 0 a scheduler should ignore it and use defaultTaskCount
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L1280
# §2.3 L306 INT parameter default is `<integer> | <intstring>`
# https://github.com/OpenJobDescription/openjd-specifications/blob/mainline/wiki/2023-09-Template-Schemas.md?plain=1#L306
template:
specificationVersion: jobtemplate-2023-09
extensions:
- TASK_CHUNKING
name: ChunkTest
parameterDefinitions:
- name: TargetRuntime
type: INT
# <intstring> form: the string must parse to 0 before the format string can resolve to
# it, so this fixture covers both coercion layers rather than only the second.
default: '00'
steps:
- name: Step1
parameterSpace:
taskParameterDefinitions:
- name: Frame
type: CHUNK[INT]
range: 1-6
chunks:
defaultTaskCount: 2
targetRuntimeSeconds: "{{Param.TargetRuntime}}"
rangeConstraint: CONTIGUOUS
script:
actions:
onRun:
command: python
args:
- -c
- print(r'CHUNK:{{Task.Param.Frame}}')
expected:
output:
- 'CHUNK:1-2'
- 'CHUNK:3-4'
- 'CHUNK:5-6'
forbidden:
- 'CHUNK:1-6'
- 'CHUNK:7'
Loading
Loading