From 0ec477be9f2b2754eac9c366f0d78653b740a946 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Sat, 1 Aug 2026 19:36:01 -0700 Subject: [PATCH 1/3] test: Add EXPR expression-language conformance fixtures for coverage gaps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 27 new fixtures, 1 strengthened, 3 renamed: direct path property access on all four routes (pins PR #292), unresolved[T] progressive evaluation incl. the failing-arm suppression case, let-scope must-rejects, in-expression scope negatives, whole-field null-drop (FEATURE_BUNDLE_1 timeout), operator precedence and associativity pins, string escapes and uppercase numeric prefixes, misplaced-underscore rejects, cross-type ordering/membership/not errors, IEEE float precision and NaN-arithmetic, HasPathMappingRules true branch, let boundary edges, int64-max accept twins, and the supplied-value 2^63 reject verified green on both implementations. Suite: 350 -> 379 passing. Spec-correct fixtures that FAIL today are split to conformance-expr-lang-expected-failures. Review: quorum-review fixes — :END terminators close stem-vs-name and LEN prefix traps in the Env.File/Task.File property fixtures; added 3.6.2--env-script-let-valid-scope.yaml as the accept twin of the env-script let-scope negatives; corrected the garbled operator-precedence comment (2 ** (3 ** 2) = 512); verified and documented that upper(int) has no matching signature (the unresolved-condition fixture's failing arm is real); reworded two headers that referenced a ../proposed/ directory which lives on the expected-failures PR, not this branch. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- .../3.6--let-boundary-edges.yaml | 78 +++++++++++++++++++ ....2--env-script-let-task-param.invalid.yaml | 34 ++++++++ .../3.6.2--env-script-let-valid-scope.yaml | 31 ++++++++ ...tep-let-session-in-expression.invalid.yaml | 20 +++++ ...let-session-working-directory.invalid.yaml | 18 +++++ .../3.6.2--step-let-task-file.invalid.yaml | 23 ++++++ .../3.6.2--step-let-task-param.invalid.yaml | 24 ++++++ ...ion-in-host-requirements-call.invalid.yaml | 23 ++++++ ...rscore-adjacent-decimal-point.invalid.yaml | 16 ++++ ...6--underscore-before-exponent.invalid.yaml | 16 ++++ ...xpr1.1.6--underscore-trailing.invalid.yaml | 15 ++++ ...2.1--unresolved-condition-failing-arm.yaml | 28 +++++++ ...r1.2.1--unresolved-type-error.invalid.yaml | 18 +++++ .../expr1.2.1--unresolved-valid-op.yaml | 27 +++++++ ...> expr1.3.12--float-range-expression.yaml} | 0 ...=> expr1.3.12--path-range-expression.yaml} | 0 ... expr1.3.12--string-range-expression.yaml} | 0 .../expr1.3.4--nan-arithmetic.invalid.yaml | 17 ++++ .../expr2.1.4--order-int-string.invalid.yaml | 17 ++++ .../expr2.1.6--not-non-bool.invalid.yaml | 16 ++++ .../expr2.1.7--path-subscript.invalid.yaml | 17 ++++ ...expr2.2.1--float-from-neg-inf.invalid.yaml | 17 ++++ .../7.3--env-file-property-direct.test.yaml | 41 ++++++++++ ...7.3--has-path-mapping-rules-true.test.yaml | 28 +++++++ .../7.3--task-file-property-direct.test.yaml | 33 ++++++++ ...7.3--task-file-property-embedded.test.yaml | 31 ++++++++ .../7.3--task-file-property-in-call.test.yaml | 29 +++++++ .../expr1.1--operator-precedence.test.yaml | 45 +++++++++++ .../jobs/expr1.1.5--string-escapes.test.yaml | 28 +++++++ ....1.6--numeric-prefixes-uppercase.test.yaml | 36 +++++++++ .../expr1.3.4--float-ieee-precision.test.yaml | 32 ++++++++ ...xpr2.1.1--int64-max-param-values.test.yaml | 40 ++++++++++ ...-overflow-param-supplied.invalid.test.yaml | 24 ++++++ ...xpr2.1.6--and-or-value-returning.test.yaml | 6 ++ .../5--timeout-expr-value-honored.test.yaml | 33 ++++++++ .../5--timeout-null-drops-field.test.yaml | 36 +++++++++ 36 files changed, 897 insertions(+) create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6--let-boundary-edges.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-task-param.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-valid-scope.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-in-expression.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-working-directory.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-file.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-param.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/7.3--session-in-host-requirements-call.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-adjacent-decimal-point.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-before-exponent.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-trailing.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-condition-failing-arm.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-type-error.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-valid-op.yaml rename conformance-tests/2023-09/EXPR/job_templates/{expr1.3.11--float-range-expression.yaml => expr1.3.12--float-range-expression.yaml} (100%) rename conformance-tests/2023-09/EXPR/job_templates/{expr1.3.11--path-range-expression.yaml => expr1.3.12--path-range-expression.yaml} (100%) rename conformance-tests/2023-09/EXPR/job_templates/{expr1.3.11--string-range-expression.yaml => expr1.3.12--string-range-expression.yaml} (100%) create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr1.3.4--nan-arithmetic.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr2.1.4--order-int-string.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr2.1.6--not-non-bool.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr2.1.7--path-subscript.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/job_templates/expr2.2.1--float-from-neg-inf.invalid.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/7.3--env-file-property-direct.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/7.3--has-path-mapping-rules-true.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-direct.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-embedded.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-in-call.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr1.1--operator-precedence.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr1.1.5--string-escapes.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr1.1.6--numeric-prefixes-uppercase.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr1.3.4--float-ieee-precision.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-max-param-values.test.yaml create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-overflow-param-supplied.invalid.test.yaml create mode 100644 conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml create mode 100644 conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6--let-boundary-edges.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6--let-boundary-edges.yaml new file mode 100644 index 00000000..901a3902 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6--let-boundary-edges.yaml @@ -0,0 +1,78 @@ +# Boundary edges for let bindings (Template Schemas §3.6, §3.6.1): +# exactly 50 bindings is the maximum and must be accepted (the 51-binding +# reject twin is 3.6--let-too-many.invalid.yaml); a 512-character +# is the maximum length and must be accepted (the +# 513-character reject twin is parked in a proposed/ directory added by +# the separate expected-failures PR — see its README there; it is not on +# this branch); +# names may start with an underscore; and later bindings may reference +# earlier bindings in the same let block (a 10-deep forward-reference +# chain). +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + let: + - _lead = 1 + - aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = 42 + - chain0 = _lead + 1 + - chain1 = chain0 + 1 + - chain2 = chain1 + 1 + - chain3 = chain2 + 1 + - chain4 = chain3 + 1 + - chain5 = chain4 + 1 + - chain6 = chain5 + 1 + - chain7 = chain6 + 1 + - chain8 = chain7 + 1 + - chain9 = chain8 + 1 + - pad0 = 0 + - pad1 = 1 + - pad2 = 2 + - pad3 = 3 + - pad4 = 4 + - pad5 = 5 + - pad6 = 6 + - pad7 = 7 + - pad8 = 8 + - pad9 = 9 + - pad10 = 10 + - pad11 = 11 + - pad12 = 12 + - pad13 = 13 + - pad14 = 14 + - pad15 = 15 + - pad16 = 16 + - pad17 = 17 + - pad18 = 18 + - pad19 = 19 + - pad20 = 20 + - pad21 = 21 + - pad22 = 22 + - pad23 = 23 + - pad24 = 24 + - pad25 = 25 + - pad26 = 26 + - pad27 = 27 + - pad28 = 28 + - pad29 = 29 + - pad30 = 30 + - pad31 = 31 + - pad32 = 32 + - pad33 = 33 + - pad34 = 34 + - pad35 = 35 + - pad36 = 36 + - pad37 = 37 + script: + actions: + onRun: + command: python + args: + - "-c" + - | + print(r'LEAD:{{ _lead }}') + print(r'LONG:{{ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa }}') + print(r'CHAIN:{{ chain9 }}') + print(r'PAD_LAST:{{ pad37 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-task-param.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-task-param.invalid.yaml new file mode 100644 index 00000000..5aca83c4 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-task-param.invalid.yaml @@ -0,0 +1,34 @@ +# An .let binding may reference only Param.*, RawParam.*, +# Session.*, Env.File.*, Job.Name, and earlier bindings (Template Schemas +# §3.6.2). Task.Param.* is never in scope for an environment script — even +# for a stepEnvironment whose enclosing step defines the task parameter — +# so this template must be rejected. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + parameterSpace: + taskParameterDefinitions: + - name: Frame + type: INT + range: "1-5" + stepEnvironments: + - name: Setup + script: + let: + - f = Task.Param.Frame + actions: + onEnter: + command: python + args: + - "-c" + - "print(r'{{ f }}')" + script: + actions: + onRun: + command: python + args: + - "-c" + - "print()" diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-valid-scope.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-valid-scope.yaml new file mode 100644 index 00000000..42284905 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--env-script-let-valid-scope.yaml @@ -0,0 +1,31 @@ +# Template Schemas §3.6.2: the accept twin of the env-script let-scope +# negatives. An .let may reference Param.*, RawParam.*, +# and Job.Name (plus Session.* and Env.File.* at run time) — a template +# using exactly the allowed compile-time symbols must VALIDATE. Without +# this accept fixture, an implementation lacking `let` support in +# environment scripts entirely would pass every 3.6.2--env-script-let-* +# negative for the wrong reason. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +parameterDefinitions: +- name: Prefix + type: STRING + default: run +jobEnvironments: +- name: Env1 + script: + let: + - banner = Param.Prefix + '-' + Job.Name + actions: + onEnter: + command: python + args: ["-c", "print(r'{{ banner }}')"] +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: ["-c", "print()"] diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-in-expression.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-in-expression.invalid.yaml new file mode 100644 index 00000000..e148c03c --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-in-expression.invalid.yaml @@ -0,0 +1,20 @@ +# Session.WorkingDirectory is out of scope for a .let binding +# (Template Schemas §3.6.2). Here the out-of-scope symbol is nested deep in +# an expression tree rather than being the whole binding — a validator that +# only inspects top-level references in let bindings would incorrectly +# accept this. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + let: + - n = len(string(Session.WorkingDirectory)) + 1 + script: + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ n }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-working-directory.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-working-directory.invalid.yaml new file mode 100644 index 00000000..c90db89b --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-session-working-directory.invalid.yaml @@ -0,0 +1,18 @@ +# A .let binding may reference only Param.*, RawParam.*, +# Job.Name, Step.Name, and earlier bindings (Template Schemas §3.6.2). +# Session.WorkingDirectory is a host-context symbol and must be rejected here. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + let: + - wd = Session.WorkingDirectory + script: + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ wd }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-file.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-file.invalid.yaml new file mode 100644 index 00000000..5e9dcd7c --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-file.invalid.yaml @@ -0,0 +1,23 @@ +# A .let binding may reference only Param.*, RawParam.*, +# Job.Name, Step.Name, and earlier bindings (Template Schemas §3.6.2). +# Task.File.* is only available in .let, so referencing an +# embedded file from the step-level let must be rejected. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + let: + - cfg = Task.File.config + script: + embeddedFiles: + - name: config + type: TEXT + data: "hello" + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ cfg }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-param.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-param.invalid.yaml new file mode 100644 index 00000000..cb9d53c5 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/3.6.2--step-let-task-param.invalid.yaml @@ -0,0 +1,24 @@ +# A .let binding may reference only Param.*, RawParam.*, +# Job.Name, Step.Name, and earlier bindings (Template Schemas §3.6.2). +# Task.Param.* is only available in .let and .let, +# so referencing it from the step-level let must be rejected. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + parameterSpace: + taskParameterDefinitions: + - name: Frame + type: INT + range: "1-5" + let: + - f = Task.Param.Frame + script: + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ f }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/7.3--session-in-host-requirements-call.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/7.3--session-in-host-requirements-call.invalid.yaml new file mode 100644 index 00000000..7e9505f9 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/7.3--session-in-host-requirements-call.invalid.yaml @@ -0,0 +1,23 @@ +# Session.WorkingDirectory is not available in the submission-time host +# requirements context (Template Schemas §7.3.1). Unlike the existing bare +# "{{ Session.WorkingDirectory }}" negative, the symbol here is buried inside +# a function call — a validator that walks bare format-string references but +# not full expression trees would incorrectly accept this. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + hostRequirements: + attributes: + - name: attr.worker.os.family + anyOf: + - "{{ upper(string(Session.WorkingDirectory)) }}" + script: + actions: + onRun: + command: python + args: + - "-c" + - "print()" diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-adjacent-decimal-point.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-adjacent-decimal-point.invalid.yaml new file mode 100644 index 00000000..1c2a2010 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-adjacent-decimal-point.invalid.yaml @@ -0,0 +1,16 @@ +# Underscores in numeric literals cannot appear adjacent to the decimal +# point (Expression Language §1.1.6). "1_.5" must be rejected as a syntax +# error. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ 1_.5 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-before-exponent.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-before-exponent.invalid.yaml new file mode 100644 index 00000000..1fc63a58 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-before-exponent.invalid.yaml @@ -0,0 +1,16 @@ +# Underscores in numeric literals cannot appear adjacent to the exponent +# marker (Expression Language §1.1.6). "1_e10" must be rejected as a syntax +# error. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ 1_e10 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-trailing.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-trailing.invalid.yaml new file mode 100644 index 00000000..48f2dcd5 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.1.6--underscore-trailing.invalid.yaml @@ -0,0 +1,15 @@ +# Underscores in numeric literals cannot appear at the end of a number +# (Expression Language §1.1.6). "123_" must be rejected as a syntax error. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ 123_ }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-condition-failing-arm.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-condition-failing-arm.yaml new file mode 100644 index 00000000..1167ef12 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-condition-failing-arm.yaml @@ -0,0 +1,28 @@ +# When the condition of an if/else is unresolved[bool], both branches are +# evaluated; if one branch succeeds and the other fails, the failing branch's +# error is suppressed and the result takes the succeeding branch's type +# (Expression Language: Static Type Checking via Unresolved Values, +# "Conditional Expressions with Unknown Conditions"). Here upper(int) has no +# signature, so the else-arm fails to type-check — but the template must +# still VALIDATE. An eager checker that reports errors from both arms +# over-rejects. (Verified that upper(int) really is the failing arm: no +# int->string call-position coercion exists — a standalone `upper(5)` let +# binding is rejected with "No matching signature for upper(int)".) +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + parameterSpace: + taskParameterDefinitions: + - name: Frame + type: INT + range: "1-5" + script: + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ Task.Param.Frame if Task.Param.Frame < 100 else upper(Task.Param.Frame) }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-type-error.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-type-error.invalid.yaml new file mode 100644 index 00000000..e403c142 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-type-error.invalid.yaml @@ -0,0 +1,18 @@ +# Session.HasPathMappingRules is unresolved[bool] at template parse time +# (Expression Language §1.2.1; §1.2.2 Session Symbols). Operations on +# unresolved values propagate the constraint type, so bool + int has no +# __add__ signature (§2.1.1) and openjd check must reject this template +# even though the symbol's concrete value is not yet known. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - "-c" + - "print(r'{{ Session.HasPathMappingRules + 1 }}')" diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-valid-op.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-valid-op.yaml new file mode 100644 index 00000000..0854a080 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.2.1--unresolved-valid-op.yaml @@ -0,0 +1,27 @@ +# At template parse time (openjd check), host-context symbols like +# Task.Param.* and Session.* are unresolved[T] placeholders (Expression +# Language §1.2.1; Static Type Checking via Unresolved Values; Template +# Schemas §7.4). Valid operations on unresolved values must type-check and +# the template must validate — a checker that rejects any expression it +# cannot fully evaluate over-rejects. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + parameterSpace: + taskParameterDefinitions: + - name: Frame + type: INT + range: "1-5" + script: + actions: + onRun: + command: python + args: + - "-c" + - | + print(r'NEXT:{{ Task.Param.Frame + 1 }}') + print(r'OUT:{{ Session.WorkingDirectory / "out" }}') + print(r'HASWD:{{ len(string(Session.WorkingDirectory)) > 0 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--float-range-expression.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--float-range-expression.yaml similarity index 100% rename from conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--float-range-expression.yaml rename to conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--float-range-expression.yaml diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--path-range-expression.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--path-range-expression.yaml similarity index 100% rename from conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--path-range-expression.yaml rename to conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--path-range-expression.yaml diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--string-range-expression.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--string-range-expression.yaml similarity index 100% rename from conformance-tests/2023-09/EXPR/job_templates/expr1.3.11--string-range-expression.yaml rename to conformance-tests/2023-09/EXPR/job_templates/expr1.3.12--string-range-expression.yaml diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr1.3.4--nan-arithmetic.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr1.3.4--nan-arithmetic.invalid.yaml new file mode 100644 index 00000000..1c811571 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr1.3.4--nan-arithmetic.invalid.yaml @@ -0,0 +1,17 @@ +# "Operations that would produce NaN (e.g., 0.0 / 0.0) are errors" +# (Expression Language §1.3.4). This exercises the arithmetic NaN guard, +# which is a different code path from the float("nan") conversion guard +# already covered by expr2.2.1--float-from-nan. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ 0.0 / 0.0 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr2.1.4--order-int-string.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.4--order-int-string.invalid.yaml new file mode 100644 index 00000000..0572744b --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.4--order-int-string.invalid.yaml @@ -0,0 +1,17 @@ +# Ordering operators work on int, float, string, path, and bool, and only +# the compatible cross-type pairs int/float and string/path may be mixed; +# "comparing other cross-type pairs is an error" (Expression Language +# §2.1.4). 1 < "a" must be rejected. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ 1 < "a" }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr2.1.6--not-non-bool.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.6--not-non-bool.invalid.yaml new file mode 100644 index 00000000..fa9b7c08 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.6--not-non-bool.invalid.yaml @@ -0,0 +1,16 @@ +# "not remains strictly boolean — it requires a bool operand and returns +# bool" (Expression Language §2.1.6). There is no truthiness concept, so +# "not 1" must be rejected rather than evaluating to false. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ not 1 }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr2.1.7--path-subscript.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.7--path-subscript.invalid.yaml new file mode 100644 index 00000000..0373ad76 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr2.1.7--path-subscript.invalid.yaml @@ -0,0 +1,17 @@ +# The subscript operator is defined only for list, range_expr, and string +# receivers (Expression Language §2.1.7). There is no __getitem__ signature +# for path, so 'path("/a/b")[0]' must be rejected. (Use .parts to index a +# path's components.) +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print(r'{{ path("/a/b")[0] }}') diff --git a/conformance-tests/2023-09/EXPR/job_templates/expr2.2.1--float-from-neg-inf.invalid.yaml b/conformance-tests/2023-09/EXPR/job_templates/expr2.2.1--float-from-neg-inf.invalid.yaml new file mode 100644 index 00000000..82ab0955 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/job_templates/expr2.2.1--float-from-neg-inf.invalid.yaml @@ -0,0 +1,17 @@ +# "float('inf'), float('nan'), and float('-inf') are errors" (Expression +# Language §1.3.4). The '-inf' spelling is a distinct parse branch from +# 'inf' (covered by expr2.2.1--float-from-inf) — an implementation matching +# only the unsigned spellings would let it through. +specificationVersion: jobtemplate-2023-09 +extensions: +- EXPR +name: TestJob +steps: +- name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - "print(r'{{ float(\"-inf\") }}')" diff --git a/conformance-tests/2023-09/EXPR/jobs/7.3--env-file-property-direct.test.yaml b/conformance-tests/2023-09/EXPR/jobs/7.3--env-file-property-direct.test.yaml new file mode 100644 index 00000000..a6c9c805 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/7.3--env-file-property-direct.test.yaml @@ -0,0 +1,41 @@ +# Direct property access on Env.File. (path-typed) as a whole-field format +# string and embedded in surrounding text (Template Schemas §7.3.1; Expression +# Language §2.3.1). Mirrors the Task.File direct-access route for the +# Environment symbol family, which had zero direct-access coverage. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + jobEnvironments: + - name: TestEnv + script: + embeddedFiles: + - name: config + type: TEXT + filename: envconf.txt + data: "hello" + actions: + onEnter: + command: python + args: + - -c + - | + import sys + print('DIRECT_STEM:' + sys.argv[1] + ':END') + print(r'EMBEDDED_NAME:{{ Env.File.config.name }}:END') + - "{{ Env.File.config.stem }}" + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - print('RAN') +expected: + output: + - "DIRECT_STEM:envconf:END" + - "EMBEDDED_NAME:envconf.txt:END" + - "RAN" diff --git a/conformance-tests/2023-09/EXPR/jobs/7.3--has-path-mapping-rules-true.test.yaml b/conformance-tests/2023-09/EXPR/jobs/7.3--has-path-mapping-rules-true.test.yaml new file mode 100644 index 00000000..50e754d3 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/7.3--has-path-mapping-rules-true.test.yaml @@ -0,0 +1,28 @@ +# Session.HasPathMappingRules must be true when path mapping rules are +# supplied to the session (Template Schemas §7.3.1; Expression Language +# §1.2.2 Session Symbols). Every other fixture asserts the false branch, so +# "returns bool" was indistinguishable from "returns constant false". +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'HAS_PM:{{ Session.HasPathMappingRules }}') + print(r'HAS_PM_COND:{{ "yes" if Session.HasPathMappingRules else "no" }}') +pathMapping: + - source_path_format: POSIX + source_path: /mnt/shared + destination_path: /local/cache +expected: + output: + - "HAS_PM:true" + - "HAS_PM_COND:yes" diff --git a/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-direct.test.yaml b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-direct.test.yaml new file mode 100644 index 00000000..c7fef2bb --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-direct.test.yaml @@ -0,0 +1,33 @@ +# Direct property access on a path-typed symbol as a whole-field format string: +# the arg is exactly "{{Task.File.config.name}}" with no surrounding text +# (Template Schemas §7.3.1; Expression Language §2.3.1 path properties). +# This is the exact shape of a known validator bug: the property reached via a +# let binding worked while the direct route was rejected at validation. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + embeddedFiles: + - name: config + type: TEXT + filename: config.txt + data: "hello" + actions: + onRun: + command: python + args: + - -c + - | + import sys + print('NAME:' + sys.argv[1] + ':END') + print('STEM:' + sys.argv[2] + ':END') + - "{{ Task.File.config.name }}" + - "{{ Task.File.config.stem }}" +expected: + output: + - "NAME:config.txt:END" + - "STEM:config:END" diff --git a/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-embedded.test.yaml b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-embedded.test.yaml new file mode 100644 index 00000000..42c2e509 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-embedded.test.yaml @@ -0,0 +1,31 @@ +# Direct property access on a path-typed symbol embedded in a format string +# with surrounding text (Template Schemas §7.3.1; Expression Language §1.3.2, +# §2.3.1). The embedded form takes the natural-result-then-stringify route, +# distinct from the whole-field route. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + embeddedFiles: + - name: config + type: TEXT + filename: config.txt + data: "hello" + actions: + onRun: + command: python + args: + - -c + - | + print(r'NAME:{{ Task.File.config.name }}:END') + print(r'STEM:{{ Task.File.config.stem }}:END') + print(r'SUFFIX:{{ Task.File.config.suffix }}:END') +expected: + output: + - "NAME:config.txt:END" + - "STEM:config:END" + - "SUFFIX:.txt:END" diff --git a/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-in-call.test.yaml b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-in-call.test.yaml new file mode 100644 index 00000000..e563d536 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/7.3--task-file-property-in-call.test.yaml @@ -0,0 +1,29 @@ +# Direct property access on a path-typed symbol as a function-call argument +# (Template Schemas §7.3.1; Expression Language §2.2.1, §2.2.4, §2.3.1). +# The route by which a symbol is reached selects which validator code path +# checks it; the function-call route must accept Task.File... +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + embeddedFiles: + - name: config + type: TEXT + filename: config.txt + data: "hello" + actions: + onRun: + command: python + args: + - -c + - | + print(r'UPPER:{{ upper(Task.File.config.stem) }}:END') + print(r'LEN:{{ len(Task.File.config.name) }}:END') +expected: + output: + - "UPPER:CONFIG:END" + - "LEN:10:END" diff --git a/conformance-tests/2023-09/EXPR/jobs/expr1.1--operator-precedence.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr1.1--operator-precedence.test.yaml new file mode 100644 index 00000000..4a9c5c2f --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr1.1--operator-precedence.test.yaml @@ -0,0 +1,45 @@ +# Operator precedence and associativity pins from the expression grammar +# (Expression Language §1.1). Each assertion selects a different production +# nesting: * binds tighter than +; unary minus binds looser than ** (so +# -2 ** 2 is -(2 ** 2)); ** is right-associative (2 ** 3 ** 2 is +# 2 ** (3 ** 2) = 2 ** 9 = 512, not (2 ** 3) ** 2 = 64); - and // are left-associative; + binds +# tighter than <; not binds looser than ==; and binds tighter than or; +# parentheses override all of it. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'MUL_OVER_ADD:{{ 2 + 3 * 4 }}') + print(r'PAREN_ADD:{{ (2 + 3) * 4 }}') + print(r'NEG_POW:{{ -2 ** 2 }}') + print(r'NEG_PAREN_POW:{{ (-2) ** 2 }}') + print(r'POW_RIGHT_ASSOC:{{ 2 ** 3 ** 2 }}') + print(r'POW_PAREN_LEFT:{{ (2 ** 3) ** 2 }}') + print(r'SUB_LEFT_ASSOC:{{ 10 - 4 - 3 }}') + print(r'FLOORDIV_LEFT_ASSOC:{{ 100 // 10 // 5 }}') + print(r'ADD_OVER_CMP:{{ 1 + 2 < 2 + 3 }}') + print(r'NOT_OVER_EQ:{{ not 1 == 2 }}') + print(r'AND_OVER_OR:{{ true or false and false }}') +expected: + output: + - "MUL_OVER_ADD:14" + - "PAREN_ADD:20" + - "NEG_POW:-4" + - "NEG_PAREN_POW:4" + - "POW_RIGHT_ASSOC:512" + - "POW_PAREN_LEFT:64" + - "SUB_LEFT_ASSOC:3" + - "FLOORDIV_LEFT_ASSOC:2" + - "ADD_OVER_CMP:true" + - "NOT_OVER_EQ:true" + - "AND_OVER_OR:true" diff --git a/conformance-tests/2023-09/EXPR/jobs/expr1.1.5--string-escapes.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr1.1.5--string-escapes.test.yaml new file mode 100644 index 00000000..5c9f146f --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr1.1.5--string-escapes.test.yaml @@ -0,0 +1,28 @@ +# Escape-sequence rows of the string literal table (Expression Language +# §1.1.5): \xhh, \uhhhh, \Uhhhhhhhh, and one \N{name} representative. The +# escapes are processed by the expression language before the command runs, +# so the printed text contains the decoded characters. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'HEX:{{ "\x41\x42" }}') + print(r'U16:{{ "\u0043\u0044" }}') + print(r'U32:{{ "\U00000045\U00000046" }}') + print(r'NAMED:{{ "\N{LATIN SMALL LETTER Z}" }}') +expected: + output: + - "HEX:AB" + - "U16:CD" + - "U32:EF" + - "NAMED:z" diff --git a/conformance-tests/2023-09/EXPR/jobs/expr1.1.6--numeric-prefixes-uppercase.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr1.1.6--numeric-prefixes-uppercase.test.yaml new file mode 100644 index 00000000..61da07d0 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr1.1.6--numeric-prefixes-uppercase.test.yaml @@ -0,0 +1,36 @@ +# Unasserted rows of the numeric and string literal tables (Expression +# Language §1.1.5, §1.1.6): uppercase 0X/0O/0B prefixes, the uppercase R raw +# string prefix, the explicitly-valid literal 00, and 1e10 producing a float. +# The EXPFLOAT assertion discriminates float from int: float pass-through +# (§1.3.4) preserves the literal text "1e10", while an implementation lexing +# it as an int would print 10000000000. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'HEXU:{{ 0X2A }}') + print(r'OCTU:{{ 0O52 }}') + print(r'BINU:{{ 0B101010 }}') + print(r'ZEROZERO:{{ 00 }}') + print(r'EXPFLOAT:{{ 1e10 }}') + print(r'EXPFLOAT_EQ:{{ 1e10 == 10000000000.0 }}') + print(r'RAWU:{{ R"a\nb" }}') +expected: + output: + - "HEXU:42" + - "OCTU:42" + - "BINU:42" + - "ZEROZERO:0" + - "EXPFLOAT:1e10" + - "EXPFLOAT_EQ:true" + - 'RAWU:a\nb' diff --git a/conformance-tests/2023-09/EXPR/jobs/expr1.3.4--float-ieee-precision.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr1.3.4--float-ieee-precision.test.yaml new file mode 100644 index 00000000..377416cd --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr1.3.4--float-ieee-precision.test.yaml @@ -0,0 +1,32 @@ +# Floats are 64-bit IEEE 754 and subject to standard binary floating-point +# precision: "0.1 + 0.2 produces 0.30000000000000004, not 0.3" — the spec's +# own worked example (Expression Language §1.3.4). An implementation using +# decimal arithmetic would produce 0.3 and fail all three SUM assertions. +# Also pins that values near the maximum finite double (1e308) are accepted +# and usable. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'SUM:{{ 0.1 + 0.2 }}') + print(r'SUM_EQ_EXACT:{{ 0.1 + 0.2 == 0.30000000000000004 }}') + print(r'SUM_NEQ_POINT3:{{ 0.1 + 0.2 == 0.3 }}') + print(r'BIG_LIT:{{ 1e308 }}') + print(r'BIG_CMP:{{ 1e308 > 1e307 }}') +expected: + output: + - "SUM:0.30000000000000004" + - "SUM_EQ_EXACT:true" + - "SUM_NEQ_POINT3:false" + - "BIG_LIT:1e308" + - "BIG_CMP:true" diff --git a/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-max-param-values.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-max-param-values.test.yaml new file mode 100644 index 00000000..4550723f --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-max-param-values.test.yaml @@ -0,0 +1,40 @@ +# The int type is a 64-bit signed integer (Expression Language §1.2.1), so +# 2^63-1 = 9223372036854775807 must be accepted on the parameter-value axis: +# as a job parameter default and as a supplied job parameter value. These are +# the accept twins of the 2^63 reject cases (parked in a proposed/ directory +# added by the separate expected-failures PR, not on this branch) — YAML +# parsers yield arbitrary-precision integers silently, making parameter +# values the most likely place for an implementation to mishandle the +# boundary. The task-range-element twin is likewise parked there because of +# a live implementation divergence (ranges reject at >= 2^62). +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + parameterDefinitions: + - name: DefMax + type: INT + default: 9223372036854775807 + - name: SuppliedMax + type: INT + default: 0 + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - | + print(r'DEF:{{ Param.DefMax }}') + print(r'SUPPLIED:{{ Param.SuppliedMax }}') + print(r'DEF_IS_MAX:{{ Param.DefMax == 9223372036854775807 }}') +parameters: + SuppliedMax: 9223372036854775807 +expected: + output: + - "DEF:9223372036854775807" + - "SUPPLIED:9223372036854775807" + - "DEF_IS_MAX:true" diff --git a/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-overflow-param-supplied.invalid.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-overflow-param-supplied.invalid.test.yaml new file mode 100644 index 00000000..74ed0687 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr2.1.1--int64-overflow-param-supplied.invalid.test.yaml @@ -0,0 +1,24 @@ +# The int type is a 64-bit signed integer (Expression Language §1.2.1; +# RFC 0005 "64-bit Signed Integer Type", dropped from the published form). +# A supplied job parameter value of 2^63 = 9223372036854775808 is out of +# range and must be rejected at job creation. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + parameterDefinitions: + - name: TooBig + type: INT + default: 0 + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - -c + - "print(r'{{ Param.TooBig }}')" +parameters: + TooBig: 9223372036854775808 diff --git a/conformance-tests/2023-09/EXPR/jobs/expr2.1.6--and-or-value-returning.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr2.1.6--and-or-value-returning.test.yaml index 8354a375..dc0cd3b8 100644 --- a/conformance-tests/2023-09/EXPR/jobs/expr2.1.6--and-or-value-returning.test.yaml +++ b/conformance-tests/2023-09/EXPR/jobs/expr2.1.6--and-or-value-returning.test.yaml @@ -18,6 +18,9 @@ template: print(r'OR_ZERO:{{ 0 or 5 }}') print(r'OR_EMPTY:[{{ "" or "fallback" }}]') print(r'AND_NULL:[{{ null and "hello" }}]') + print(r'AND_NULL_IS_NULL:{{ (null and "hello") == null }}') + print(r'AND_NULL_SENT:{{ (null and "hello") or "SENT" }}') + print(r'AND_FALSE:{{ false and "hello" }}') print(r'AND_TRUE:{{ true and "hello" }}') print(r'AND_STR:{{ "hello" and "world" }}') print(r'BOOL:{{ true or false }}') @@ -29,6 +32,9 @@ expected: - 'OR_ZERO:0' - 'OR_EMPTY:[]' - 'AND_NULL:[]' + - 'AND_NULL_IS_NULL:true' + - 'AND_NULL_SENT:SENT' + - 'AND_FALSE:false' - 'AND_TRUE:hello' - 'AND_STR:world' - 'BOOL:true' diff --git a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml new file mode 100644 index 00000000..b8863c43 --- /dev/null +++ b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml @@ -0,0 +1,33 @@ +# Control for 5--timeout-null-drops-field.test.yaml: when the conditional +# takes the non-null arm, the computed timeout value must be applied to the +# action (Template Schemas §5 Action timeout @fmtstring under +# FEATURE_BUNDLE_1; Expression Language §1.3.2). A 1-second timeout must kill +# a 10-second command — this catches an implementation that silently drops +# the whole field whenever its value is an expression. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - FEATURE_BUNDLE_1 + - EXPR + name: TestJob + parameterDefinitions: + - name: UseTimeout + type: BOOL + default: true + steps: + - name: Step1 + script: + actions: + onRun: + timeout: "{{ 1 if Param.UseTimeout else null }}" + command: python + args: + - -c + - | + import time + time.sleep(10) + print('SHOULD_NOT_PRINT') +expected: + taskFailure: {} + forbidden: + - SHOULD_NOT_PRINT diff --git a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml new file mode 100644 index 00000000..40b87068 --- /dev/null +++ b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml @@ -0,0 +1,36 @@ +# An EXPR-computed null on an optional scalar field drops the field entirely +# (Expression Language §1.3.2: "For an optional field of type T ... If the +# expression evaluates to None/null, the field is omitted as if it were not +# specified."). The Action timeout is the only optional scalar format-string +# field observable at runtime (it is @fmtstring under FEATURE_BUNDLE_1; base +# EXPR has none). With the field dropped, onRun has no timeout and a 2-second +# command completes. An implementation that keeps the field — coercing null +# to 0/""/"None" — fails validation or kills the command instead. +# See 5--timeout-expr-value-honored.test.yaml for the control proving that an +# expression-computed timeout is honored when it is not null. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - FEATURE_BUNDLE_1 + - EXPR + name: TestJob + parameterDefinitions: + - name: UseTimeout + type: BOOL + default: false + steps: + - name: Step1 + script: + actions: + onRun: + timeout: "{{ 30 if Param.UseTimeout else null }}" + command: python + args: + - -c + - | + import time + time.sleep(2) + print('DONE_NO_TIMEOUT') +expected: + output: + - DONE_NO_TIMEOUT From ee519750e3120d11d723f1cb06c26e6373b4d695 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Fri, 4 Sep 2026 21:08:20 -0700 Subject: [PATCH 2/3] fix: Build the timeout control's forbidden marker at run time 5--timeout-expr-value-honored.test.yaml was the single red test on windows-latest in run 33936907488. The action timeout was never at fault: the action was killed on time on both platforms (1.39s on Linux, 1.51s on Windows, against a 10-second sleep), and the runner's taskFailure check passed, so the run did exit non-zero. Only the forbidden-output check tripped. That check is a plain substring search over the combined stdout+stderr of the whole `openjd run`. openjd-sessions logs the command it is about to run at INFO, and builds that log line from different things on each platform (_subprocess.py, the cmd_line_for_logger branch): POSIX _runner_base.py writes the action into a temp .sh wrapper, so the logged line is shlex.join([]). The command text appears only at DEBUG, which the CLI does not emit. Windows There is no wrapper, so the logged line is list2cmdline(args) -- the full argument list, including the python -c source body. So on Windows the echoed command line carried the fixture's own literal SHOULD_NOT_PRINT, and the check matched the echo rather than the program's output. On POSIX the same marker was invisible, which is why this passed on Linux and failed on Windows. Building the marker as 'SHOULD_NOT' + '_PRINT' keeps it out of the command line while still printing it verbatim if the action is ever allowed to finish, so the assertion is unchanged in substance. Verified with the CI versions (openjd-cli 0.7.7, openjd-model 0.11.9, openjd-sessions 0.12.1): baseline 1 passed, 0 failed mutant: timeout line removed 0 passed, 1 failed mutant: condition inverted to null 0 passed, 1 failed FEATURE_BUNDLE_1/jobs suite 12 passed, 0 failed Both mutants printed SHOULD_NOT_PRINT after the full 10 seconds and exited 0, so the fixture still fails when the timeout is not honored. Windows is verified by construction only -- list2cmdline of the new source does not contain the marker -- and needs a CI run to confirm. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- .../jobs/5--timeout-expr-value-honored.test.yaml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml index b8863c43..f19a2b6b 100644 --- a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml +++ b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml @@ -4,6 +4,13 @@ # FEATURE_BUNDLE_1; Expression Language §1.3.2). A 1-second timeout must kill # a 10-second command — this catches an implementation that silently drops # the whole field whenever its value is an expression. +# +# The marker is built at run time instead of written as a literal: the runner's +# `forbidden` check is a substring search over the whole `openjd run` output, +# and on Windows openjd-sessions logs the resolved command line at INFO, so a +# literal marker trips the check even when the action was killed on time. On +# POSIX the command goes into a temp .sh whose contents log only at DEBUG, +# which is why a literal marker passed on Linux and failed on Windows. template: specificationVersion: jobtemplate-2023-09 extensions: @@ -26,7 +33,7 @@ template: - | import time time.sleep(10) - print('SHOULD_NOT_PRINT') + print('SHOULD_NOT' + '_PRINT') expected: taskFailure: {} forbidden: From 24f63b6c9b34fa37162d172645cf00067ad8afc4 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Fri, 4 Sep 2026 21:49:24 -0700 Subject: [PATCH 3/3] docs: Trim the timeout fixture headers to spec, test, expectation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both headers now run in three paragraphs: the spec rule, what the fixture does, and what must happen. 23 comment lines down to 16. Cut the verbatim §1.3.2 quote, the restatement of what the runner's forbidden check does, and the POSIX .sh-wrapper mechanism. Kept the reason the marker is joined at run time: a reader who folds it back into a literal reintroduces the Windows failure fixed in 374c4d3. Comment-only change. FEATURE_BUNDLE_1/jobs is 12 passed, 0 failed. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- .../5--timeout-expr-value-honored.test.yaml | 19 +++++++------------ .../5--timeout-null-drops-field.test.yaml | 18 ++++++++---------- 2 files changed, 15 insertions(+), 22 deletions(-) diff --git a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml index f19a2b6b..a8059e08 100644 --- a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml +++ b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-expr-value-honored.test.yaml @@ -1,16 +1,11 @@ -# Control for 5--timeout-null-drops-field.test.yaml: when the conditional -# takes the non-null arm, the computed timeout value must be applied to the -# action (Template Schemas §5 Action timeout @fmtstring under -# FEATURE_BUNDLE_1; Expression Language §1.3.2). A 1-second timeout must kill -# a 10-second command — this catches an implementation that silently drops -# the whole field whenever its value is an expression. +# Action timeout is @fmtstring under FEATURE_BUNDLE_1 (Template Schemas §5) and +# Expression Language §1.3.2 requires a non-null expression value to reach it. # -# The marker is built at run time instead of written as a literal: the runner's -# `forbidden` check is a substring search over the whole `openjd run` output, -# and on Windows openjd-sessions logs the resolved command line at INFO, so a -# literal marker trips the check even when the action was killed on time. On -# POSIX the command goes into a temp .sh whose contents log only at DEBUG, -# which is why a literal marker passed on Linux and failed on Windows. +# Control for 5--timeout-null-drops-field.test.yaml: a computed 1-second timeout +# over a 10-second command. The marker is joined at run time because Windows +# logs the resolved command line at INFO and would otherwise match it. +# +# The action must be killed: the task fails and the marker never prints. template: specificationVersion: jobtemplate-2023-09 extensions: diff --git a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml index 40b87068..b63ffee1 100644 --- a/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml +++ b/conformance-tests/2023-09/FEATURE_BUNDLE_1/jobs/5--timeout-null-drops-field.test.yaml @@ -1,13 +1,11 @@ -# An EXPR-computed null on an optional scalar field drops the field entirely -# (Expression Language §1.3.2: "For an optional field of type T ... If the -# expression evaluates to None/null, the field is omitted as if it were not -# specified."). The Action timeout is the only optional scalar format-string -# field observable at runtime (it is @fmtstring under FEATURE_BUNDLE_1; base -# EXPR has none). With the field dropped, onRun has no timeout and a 2-second -# command completes. An implementation that keeps the field — coercing null -# to 0/""/"None" — fails validation or kills the command instead. -# See 5--timeout-expr-value-honored.test.yaml for the control proving that an -# expression-computed timeout is honored when it is not null. +# Expression Language §1.3.2: an optional field whose expression evaluates to +# null is omitted as if it were not specified. +# +# Action timeout is the only such field observable at runtime (@fmtstring under +# FEATURE_BUNDLE_1). Its conditional takes the null arm over a 2-second command. +# +# No timeout reaches onRun and the command completes; coercing null instead +# fails validation or kills it. Control: 5--timeout-expr-value-honored.test.yaml. template: specificationVersion: jobtemplate-2023-09 extensions: