Skip to content

chore(policies): improve error message in policy eval when no branches were executed #2576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 25, 2025
+3 −5
Merged

chore(policies): improve error message in policy eval when no branches were executed #2576

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
133d076
improve error message in policy eval when no branches were executed
jiparis Nov 21, 2025
0c16319
Merge branch 'main' into PFM-3981
jiparis Nov 24, 2025
6a58aa2
use global debug flag
jiparis Nov 24, 2025
45e3fb1
fix docs
jiparis Nov 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions app/cli/cmd/policy_develop_eval.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ func newPolicyDevelopEvalCmd() *cobra.Command {
policyPath string
inputs []string
allowedHostnames []string
debug bool
)

cmd := &cobra.Command{
Expand All @@ -53,7 +52,7 @@ evaluates the policy against the provided material or attestation.`,
PolicyPath: policyPath,
Inputs: parseKeyValue(inputs),
AllowedHostnames: allowedHostnames,
Debug: debug,
Debug: flagDebug,
}

policyEval, err := action.NewPolicyEval(opts, ActionOpts)
Expand All @@ -77,7 +76,6 @@ evaluates the policy against the provided material or attestation.`,
cmd.Flags().StringVarP(&policyPath, "policy", "p", "policy.yaml", "Policy reference (./my-policy.yaml, https://my-domain.com/my-policy.yaml, chainloop://my-stored-policy)")
cmd.Flags().StringArrayVar(&inputs, "input", []string{}, "Key-value pairs of policy inputs (key=value)")
cmd.Flags().StringSliceVar(&allowedHostnames, "allowed-hostnames", []string{}, "Additional hostnames allowed for http.send requests in policies")
cmd.Flags().BoolVarP(&debug, "debug", "", false, "Include detailed evaluation inputs/outputs in JSON output and enable verbose logging")

return cmd
}
Expand Down
2 changes: 1 addition & 1 deletion app/cli/documentation/cli-reference.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2869,7 +2869,6 @@ Options
```
--allowed-hostnames strings Additional hostnames allowed for http.send requests in policies
--annotation strings Key-value pairs of material annotations (key=value)
--debug Include detailed evaluation inputs/outputs in JSON output and enable verbose logging
-h, --help help for eval
--input stringArray Key-value pairs of policy inputs (key=value)
--kind string Kind of the material: ["ARTIFACT" "ATTESTATION" "BLACKDUCK_SCA_JSON" "CHAINLOOP_RUNNER_CONTEXT" "CONTAINER_IMAGE" "CSAF_INFORMATIONAL_ADVISORY" "CSAF_SECURITY_ADVISORY" "CSAF_SECURITY_INCIDENT_RESPONSE" "CSAF_VEX" "EVIDENCE" "GHAS_CODE_SCAN" "GHAS_DEPENDENCY_SCAN" "GHAS_SECRET_SCAN" "GITLAB_SECURITY_REPORT" "HELM_CHART" "JACOCO_XML" "JUNIT_XML" "OPENVEX" "SARIF" "SBOM_CYCLONEDX_JSON" "SBOM_SPDX_JSON" "SLSA_PROVENANCE" "STRING" "TWISTCLI_SCAN_JSON" "ZAP_DAST_ZIP"]
Expand All @@ -2885,6 +2884,7 @@ Options inherited from parent commands
-c, --config string Path to an existing config file (default is $HOME/.config/chainloop/config.toml)
--control-plane string URL for the Control Plane API ($CHAINLOOP_CONTROL_PLANE_API) (default "api.cp.chainloop.dev:443")
--control-plane-ca string CUSTOM CA file for the Control Plane API (optional) ($CHAINLOOP_CONTROL_PLANE_API_CA)
--debug Enable debug/verbose logging mode
-i, --insecure Skip TLS transport during connection to the control plane ($CHAINLOOP_API_INSECURE)
-n, --org string organization name
-o, --output string Output format, valid options are json and table (default "table")
Expand Down
2 changes: 1 addition & 1 deletion app/cli/internal/policydevel/eval.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ func verifyMaterial(pol *v1.Policies, material *v12.Attestation_Material, materi
}

if len(policyEvs) == 0 || policyEvs[0] == nil {
return nil, fmt.Errorf("no execution branch matched for kind %s", material.MaterialType.String())
return nil, fmt.Errorf("no execution branch matched, or all of them were ignored, for kind %s", material.MaterialType.String())
Copy link
Member

@migmartri migmartri Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any better way we could do this? I mean, is there any way to distinguish them? Maybe we can add some debug logging so if you run eval with debug you get info about if the execution path was ignored or executed?

Copy link
Member Author

@jiparis jiparis Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be ideal, yes. It has implications in the engine, though. Let me check them

Copy link
Member Author

@jiparis jiparis Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The message was already there, but a bug prevented it from being shown. Check my comment in the PR

}

// Only one evaluation expected for a single policy attachment
Expand Down
Loading