Skip to content

CLA: use release-workflows v1.3.0 - #3642

Merged
pjhartlieb merged 1 commit into
devfrom
feat/cla-v1.3.0
Nov 13, 2026
Merged

pjhartlieb merged 1 commit into
devfrom
feat/cla-v1.3.0

Conversation

@pjhartlieb

Copy link
Copy Markdown
Contributor

Moves the CLA check from CLA/cla-reusable.yml@4f85d3c5 to release-workflows v1.3.0, and passes only APP_ID and APP_PRIVATE_KEY instead of secrets: inherit.

  • Signatures save again, on CLA's bot-only signatures branch. None have saved since 10-27: CLA main refuses the bot's push.
  • Who signs: on a pull request opened by an org member or named bot, other members and bots are exempt; on one opened by anyone else, the opener and every commit author sign, members included.
  • The App token is minted only for pull request events and sign/recheck comments, and only for the CLA repository.
  • Fails for a maintainer to handle, instead of passing, when the commits can't all be read or a commit's author has no GitHub login.

Comments run the workflow on the default branch (stable), which keeps the old copy until the next promotion. Until then, after a contributor signs, a maintainer re-runs the pull request's CLA check ("Re-run all jobs").

Details: blacklanternsecurity/release-workflows#18 and #20.

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (bde4ba9) to head (3162fcc).
⚠️ Report is 4 commits behind head on dev.

Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #3642   +/-   ##
=====================================
+ Coverage     91%     91%   +1%     
=====================================
  Files        465     465           
  Lines      51928   51928           
=====================================
+ Hits       47014   47024   +10     
+ Misses      4914    4904   -10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@en0f en0f left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 3162fcc.

🔴 Renaming the workflow to CLA breaks the CLA action's self-lookup until promotion

Once every committer has signed, contributor-assistant@ca4a40a7 looks up its own workflow with workflows.find(w => w.name == context.workflow) (dist/index.js:472) and fails the job with "Unable to locate this workflow's ID" when nothing matches. After this merges, dev's copy runs as CLA, but the repository's workflow record is CLA Assistant, taken from stable (version_updater.yml's record only changed name when its rename reached stable). So a signed outside contributor's pull request into dev gets a red CLA check until promotion, including the sign-then-re-run path in the description. Keep name: "CLA Assistant" on line 1; V09 below checks it.

🟢 Secrets narrowed from inherit to the two the job declares

APP_ID and APP_PRIVATE_KEY are exactly what cla-reusable.yml@v1.3.0 declares under workflow_call.secrets, so no other secret reaches a pull_request_target job.

🟢 Pinned to v1.3.0's commit, and the called job is gated

3fe35ff9 is the commit tag v1.3.0 names. The called job mints the App token only for pull request events or a recheck/sign comment, passes event values through env instead of ${{ }} in shell, and never checks out pull request code. The CLA@4f85d3c5 copy had neither the gate nor the env routing.

📝 Test artifact owed: a run of the new caller

pull_request_target runs the base branch's workflow, so the CLA check on this pull request is dev's old copy. The proof is the first run after merge that saves an outside contributor's signature on CLA's signatures branch and concludes success. I'll check for it; nothing is needed from you.

📝 Format: squash title

The single commit's subject becomes the squash commit and isn't conventional (standards/git.md). At merge, use e.g. ci(cla): use release-workflows v1.3.0.

Reproduce this review
# save the script below as verify-3642.sh, then:
bash verify-3642.sh blacklanternsecurity/bbot 3162fcc55576adfa0cbe068e78f864ec618d18bd
# expected: SUMMARY 8 PASS, 1 FAIL, 0 SKIP, 0 BROKEN   (V09 FAIL)
# control:  bash verify-3642.sh blacklanternsecurity/bbot bde4ba9248e6e78a1c147c491cfe5cfecca6eee2   -> V01, V02, V03, V04, V07, V08 FAIL; V09 PASS
#!/usr/bin/env bash
# verify-3642.sh: reproducible checks for blacklanternsecurity/bbot#3642
# Usage: bash verify-3642.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Reads the called reusable workflow and the release-workflows tag via gh api --method GET.
#
# CONTROL: sha bde4ba9248e6e78a1c147c491cfe5cfecca6eee2
# CONTROL-EXPECT: V01=FAIL V02=FAIL V03=FAIL V04=FAIL V07=FAIL V08=FAIL V09=PASS
# AUTHOR-CODE: none
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "gh $(gh --version | head -1 | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import platform; print(platform.python_version())')"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi

CALLER=.github/workflows/cla.yml
[ -f "$CALLER" ] || { broken V00 "caller exists" "$CALLER missing"; summary; exit 1; }

# The called reusable workflow, as the caller pins it: owner/repo/path@ref
USES=$(grep -E '^[[:space:]]+uses:[[:space:]]' "$CALLER" | head -1 | sed -E 's/^[[:space:]]*uses:[[:space:]]*([^ #]+).*/\1/')
CALLED_REPO=$(echo "$USES" | cut -d/ -f1-2)
CALLED_PATH=$(echo "$USES" | cut -d/ -f3- | cut -d@ -f1)
CALLED_REF=${USES##*@}
echo "info caller uses $USES"
gh api --method GET "repos/$CALLED_REPO/contents/$CALLED_PATH?ref=$CALLED_REF" -H 'Accept: application/vnd.github.raw+json' > "$WORK/called.yml" 2>/dev/null \
  || { broken V00 "read called workflow" "gh api GET failed for $USES"; summary; exit 1; }
echo "info called_sha256 $(shasum -a 256 "$WORK/called.yml" | cut -d' ' -f1)"

# V01: the caller pins release-workflows' cla-reusable.yml to the full SHA that tag v1.3.0 names
TAG_SHA=$(gh api --method GET repos/blacklanternsecurity/release-workflows/git/ref/tags/v1.3.0 --jq .object.sha 2>/dev/null)
if [ -z "$TAG_SHA" ]; then skip V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA" "tag read failed"
elif [ "$USES" = "blacklanternsecurity/release-workflows/.github/workflows/cla-reusable.yml@$TAG_SHA" ]; then pass V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA ($TAG_SHA)"
else fail V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA" "uses $USES; v1.3.0 is $TAG_SHA"; fi

# V02: the caller does not hand every secret to the called workflow
if grep -Eq '^[[:space:]]+secrets:[[:space:]]*inherit' "$CALLER"; then fail V02 "caller does not use secrets: inherit" "secrets: inherit present"
else pass V02 "caller does not use secrets: inherit"; fi

# V03: the caller passes exactly the secrets the called workflow declares under workflow_call.secrets
python3 - "$CALLER" "$WORK/called.yml" > "$WORK/v03" <<'PY'
import re, sys
caller, called = open(sys.argv[1]).read().splitlines(), open(sys.argv[2]).read().splitlines()
def block_keys(lines, header_re, indent):
    keys, inside, base = [], False, None
    for l in lines:
        if not inside:
            m = re.match(header_re, l)
            if m: inside, base = True, len(l) - len(l.lstrip()); continue
        else:
            if not l.strip(): continue
            ind = len(l) - len(l.lstrip())
            if ind <= base: break
            if ind == base + indent and re.match(r'\s*[A-Za-z_][A-Za-z0-9_-]*:', l): keys.append(l.strip().split(':')[0])
    return keys
passed = block_keys(caller, r'^\s{4}secrets:\s*$', 2)
declared = block_keys(called, r'^\s{4}secrets:\s*$', 2)
print("passed=" + ",".join(sorted(passed)) + " declared=" + ",".join(sorted(declared)))
sys.exit(0 if passed and sorted(passed) == sorted(declared) else 1)
PY
if [ $? -eq 0 ]; then pass V03 "caller passes exactly the declared secrets ($(cat "$WORK/v03"))"
else fail V03 "caller passes exactly the declared secrets" "$(cat "$WORK/v03")"; fi

# V04: the caller's permissions are exactly what the called job requests, nothing more
python3 - "$CALLER" "$WORK/called.yml" > "$WORK/v04" <<'PY'
import re, sys
caller, called = open(sys.argv[1]).read().splitlines(), open(sys.argv[2]).read().splitlines()
def perms(lines, header_re):
    out, inside, base = {}, False, None
    for l in lines:
        if not inside:
            if re.match(header_re, l): inside, base = True, len(l) - len(l.lstrip()); continue
        else:
            if not l.strip() or l.strip().startswith('#'): continue
            ind = len(l) - len(l.lstrip())
            if ind <= base: break
            k, _, v = l.strip().partition(':'); out[k.strip()] = v.split('#')[0].strip()
    return out
caller_p = perms(caller, r'^permissions:\s*$')
job_p = perms(called, r'^\s{4}permissions:\s*$')
print(f"caller={caller_p} called_job={job_p}")
sys.exit(0 if caller_p and caller_p == job_p else 1)
PY
if [ $? -eq 0 ]; then pass V04 "caller grants exactly the called job's permissions ($(cat "$WORK/v04"))"
else fail V04 "caller grants exactly the called job's permissions" "$(cat "$WORK/v04")"; fi

# V05: the called workflow never checks out or fetches pull request code (pull_request_target safety)
if grep -Eiq 'actions/checkout|git (clone|fetch|checkout)|head\.ref|head\.sha.*(clone|checkout)' "$WORK/called.yml"; then
  fail V05 "called workflow checks out no pull request code" "$(grep -Ein 'actions/checkout|git (clone|fetch|checkout)' "$WORK/called.yml" | head -3 | tr '\n' ' ')"
else pass V05 "called workflow checks out no pull request code"; fi

# V06: every uses: in the caller is a full 40-character SHA (ci.md)
BAD=$(grep -E '^[[:space:]]*(-[[:space:]]*)?uses:[[:space:]]' "$CALLER" | grep -Ev '@[0-9a-f]{40}([[:space:]]|$)' || true)
if [ -z "$BAD" ]; then pass V06 "every uses: in the caller is pinned to a 40-character SHA"
else fail V06 "every uses: in the caller is pinned to a 40-character SHA" "$BAD"; fi

# V07: no run: block in the called workflow interpolates ${{ }} into shell (values go through env)
python3 - "$WORK/called.yml" > "$WORK/v07" <<'PY'
import sys
lines = open(sys.argv[1]).read().splitlines(); hits = []; inside = False; base = 0
for n, l in enumerate(lines, 1):
    s = l.strip(); ind = len(l) - len(l.lstrip())
    if inside and s and ind <= base: inside = False
    if s.startswith('run:') or s.startswith('- run:'):
        inside, base = True, ind
        if '${{' in s: hits.append(n)
        continue
    if inside and '${{' in l: hits.append(n)
print("lines=" + ",".join(map(str, hits)))
sys.exit(1 if hits else 0)
PY
if [ $? -eq 0 ]; then pass V07 "called workflow's run: blocks interpolate no \${{ }} expressions"
else fail V07 "called workflow's run: blocks interpolate no \${{ }} expressions" "$(cat "$WORK/v07")"; fi

# V08: the called job runs (and mints the App token) only for pull_request_target or a recheck/sign comment
if grep -Eq "^[[:space:]]{4}if:.*" "$WORK/called.yml" && grep -q "github.event_name == 'pull_request_target'" "$WORK/called.yml" \
   && grep -q "github.event.comment.body == 'recheck'" "$WORK/called.yml"; then
  pass V08 "called job is gated to pull_request_target and recheck/sign comments"
else fail V08 "called job is gated to pull_request_target and recheck/sign comments" "no job-level event gate"; fi

# V09 (challenge C13): contributor-assistant finds its own workflow by name (w.name == context.workflow) before
# re-running a failed check; a name that differs from the default branch's copy makes that lookup throw and fails
# the job. Desired: the caller keeps the name the default branch's cla.yml has.
ACTION=$(grep -E 'uses:[[:space:]]*contributor-assistant/github-action@' "$WORK/called.yml" | head -1 | sed -E 's/.*github-action@([^[:space:]#]+).*/\1/')
DEF=$(gh api --method GET "repos/$REPO" --jq .default_branch 2>/dev/null)
gh api --method GET "repos/contributor-assistant/github-action/contents/dist/index.js?ref=$ACTION" -H 'Accept: application/vnd.github.raw+json' > "$WORK/ca.js" 2>/dev/null
gh api --method GET "repos/$REPO/contents/$CALLER?ref=$DEF" -H 'Accept: application/vnd.github.raw+json' > "$WORK/def.yml" 2>/dev/null
if [ -z "$ACTION" ] || [ -z "$DEF" ] || [ ! -s "$WORK/ca.js" ] || [ ! -s "$WORK/def.yml" ]; then
  skip V09 "caller's workflow name matches the default branch's (contributor-assistant rerun lookup)" "could not read action source or $DEF copy"
elif ! grep -q 'w.name == github_1.context.workflow' "$WORK/ca.js"; then
  pass V09 "caller's workflow name matches the default branch's (contributor-assistant rerun lookup) -- action at $ACTION does not look up by name"
else
  HN=$(grep -E '^name:' "$CALLER" | head -1 | sed -E 's/^name:[[:space:]]*//; s/^"(.*)"$/\1/')
  DN=$(grep -E '^name:' "$WORK/def.yml" | head -1 | sed -E 's/^name:[[:space:]]*//; s/^"(.*)"$/\1/')
  if [ "$HN" = "$DN" ]; then pass V09 "caller's workflow name matches the default branch's ($DEF: '$DN')"
  else fail V09 "caller's workflow name matches the default branch's (contributor-assistant rerun lookup)" "caller '$HN', $DEF '$DN'"; fi
fi

summary

Comment thread .github/workflows/cla.yml
@@ -1,17 +1,19 @@
name: "CLA Assistant"
name: CLA

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 contributor-assistant finds its own workflow by this name (w.name == context.workflow) once everyone has signed. The workflow record is CLA Assistant from stable, so this fails signed contributors' PRs into dev until promotion. Keep name: "CLA Assistant".

@pjhartlieb

Copy link
Copy Markdown
Contributor Author

pull_request_target runs the default branch's copy of the workflow, not the base branch's, so dev's name: isn't used before promotion. Today's merges show it. After dev's cla.yml became the v1.3.0 caller, each repo's next pull_request_target run on a pull request into dev still ran the default branch's old copy:

  • radixtarget 43451652973: release-workflows/…/cla-reusable.yml@c53c202e (stable's v1.0.0), not @3fe35ff9
  • badsecrets 43470482869: CLA/…/cla-reusable.yml@main (main's copy)
  • blasthttp 43451689241, baddns 43451709861, MANSPIDER 43451730981: stable's inline action
gh api repos/blacklanternsecurity/radixtarget/actions/runs/43451652973 --jq '.event, .referenced_workflows[].path'

At promotion, stable's file and the workflow record both become CLA, so w.name == context.workflow still matches. Keeping CLA, the same as the six callers already merged. This merges with a merge commit, so the squash title doesn't apply.

@pjhartlieb
pjhartlieb requested a review from en0f November 13, 2026 18:02

@en0f en0f left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 3162fcc.

🔴 Renaming the workflow to CLA breaks the CLA action's self-lookup until promotion

Withdrawn. pull_request_target loads cla.yml from stable, so until promotion the action looks itself up as CLA Assistant, which is the workflow record's name. At promotion the file and the record become CLA together: radixtarget made the same rename, and its record and runs switched at promotion with no failures. V10 to V12 below check this.

📝 Test artifact pending: the first CLA run after promotion

Correcting my earlier note: merging into dev changes no CLA run, because every trigger loads stable's copy. The proof is the first run after promotion that saves an outside contributor's signature on CLA's signatures branch and concludes success. I'll check it; nothing is needed from you.

Reproduce this review
# save the script below as verify-3642.sh, then:
bash verify-3642.sh blacklanternsecurity/bbot 3162fcc55576adfa0cbe068e78f864ec618d18bd
# expected: SUMMARY 11 PASS, 0 FAIL, 0 SKIP, 0 BROKEN
# control:  bash verify-3642.sh blacklanternsecurity/bbot bde4ba9248e6e78a1c147c491cfe5cfecca6eee2   -> V01, V02, V03, V04, V07, V08 FAIL
#!/usr/bin/env bash
# verify-3642.sh: reproducible checks for blacklanternsecurity/bbot#3642
# Usage: bash verify-3642.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Reads the called reusable workflow and the release-workflows tag via gh api --method GET.
#
# CONTROL: sha bde4ba9248e6e78a1c147c491cfe5cfecca6eee2
# Round 2 at the round-1 head (3162fcc5, unchanged), so the round-1 head cannot be the control; the base is.
# CONTROL-EXPECT: V01=FAIL V02=FAIL V03=FAIL V04=FAIL V07=FAIL V08=FAIL
# AUTHOR-CODE: none
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "gh $(gh --version | head -1 | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import platform; print(platform.python_version())')"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi

CALLER=.github/workflows/cla.yml
[ -f "$CALLER" ] || { broken V00 "caller exists" "$CALLER missing"; summary; exit 1; }

# The called reusable workflow, as the caller pins it: owner/repo/path@ref
USES=$(grep -E '^[[:space:]]+uses:[[:space:]]' "$CALLER" | head -1 | sed -E 's/^[[:space:]]*uses:[[:space:]]*([^ #]+).*/\1/')
CALLED_REPO=$(echo "$USES" | cut -d/ -f1-2)
CALLED_PATH=$(echo "$USES" | cut -d/ -f3- | cut -d@ -f1)
CALLED_REF=${USES##*@}
echo "info caller uses $USES"
gh api --method GET "repos/$CALLED_REPO/contents/$CALLED_PATH?ref=$CALLED_REF" -H 'Accept: application/vnd.github.raw+json' > "$WORK/called.yml" 2>/dev/null \
  || { broken V00 "read called workflow" "gh api GET failed for $USES"; summary; exit 1; }
echo "info called_sha256 $(shasum -a 256 "$WORK/called.yml" | cut -d' ' -f1)"

# V01: the caller pins release-workflows' cla-reusable.yml to the full SHA that tag v1.3.0 names
TAG_SHA=$(gh api --method GET repos/blacklanternsecurity/release-workflows/git/ref/tags/v1.3.0 --jq .object.sha 2>/dev/null)
if [ -z "$TAG_SHA" ]; then skip V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA" "tag read failed"
elif [ "$USES" = "blacklanternsecurity/release-workflows/.github/workflows/cla-reusable.yml@$TAG_SHA" ]; then pass V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA ($TAG_SHA)"
else fail V01 "caller pins release-workflows cla-reusable.yml at tag v1.3.0's SHA" "uses $USES; v1.3.0 is $TAG_SHA"; fi

# V02: the caller does not hand every secret to the called workflow
if grep -Eq '^[[:space:]]+secrets:[[:space:]]*inherit' "$CALLER"; then fail V02 "caller does not use secrets: inherit" "secrets: inherit present"
else pass V02 "caller does not use secrets: inherit"; fi

# V03: the caller passes exactly the secrets the called workflow declares under workflow_call.secrets
python3 - "$CALLER" "$WORK/called.yml" > "$WORK/v03" <<'PY'
import re, sys
caller, called = open(sys.argv[1]).read().splitlines(), open(sys.argv[2]).read().splitlines()
def block_keys(lines, header_re, indent):
    keys, inside, base = [], False, None
    for l in lines:
        if not inside:
            m = re.match(header_re, l)
            if m: inside, base = True, len(l) - len(l.lstrip()); continue
        else:
            if not l.strip(): continue
            ind = len(l) - len(l.lstrip())
            if ind <= base: break
            if ind == base + indent and re.match(r'\s*[A-Za-z_][A-Za-z0-9_-]*:', l): keys.append(l.strip().split(':')[0])
    return keys
passed = block_keys(caller, r'^\s{4}secrets:\s*$', 2)
declared = block_keys(called, r'^\s{4}secrets:\s*$', 2)
print("passed=" + ",".join(sorted(passed)) + " declared=" + ",".join(sorted(declared)))
sys.exit(0 if passed and sorted(passed) == sorted(declared) else 1)
PY
if [ $? -eq 0 ]; then pass V03 "caller passes exactly the declared secrets ($(cat "$WORK/v03"))"
else fail V03 "caller passes exactly the declared secrets" "$(cat "$WORK/v03")"; fi

# V04: the caller's permissions are exactly what the called job requests, nothing more
python3 - "$CALLER" "$WORK/called.yml" > "$WORK/v04" <<'PY'
import re, sys
caller, called = open(sys.argv[1]).read().splitlines(), open(sys.argv[2]).read().splitlines()
def perms(lines, header_re):
    out, inside, base = {}, False, None
    for l in lines:
        if not inside:
            if re.match(header_re, l): inside, base = True, len(l) - len(l.lstrip()); continue
        else:
            if not l.strip() or l.strip().startswith('#'): continue
            ind = len(l) - len(l.lstrip())
            if ind <= base: break
            k, _, v = l.strip().partition(':'); out[k.strip()] = v.split('#')[0].strip()
    return out
caller_p = perms(caller, r'^permissions:\s*$')
job_p = perms(called, r'^\s{4}permissions:\s*$')
print(f"caller={caller_p} called_job={job_p}")
sys.exit(0 if caller_p and caller_p == job_p else 1)
PY
if [ $? -eq 0 ]; then pass V04 "caller grants exactly the called job's permissions ($(cat "$WORK/v04"))"
else fail V04 "caller grants exactly the called job's permissions" "$(cat "$WORK/v04")"; fi

# V05: the called workflow never checks out or fetches pull request code (pull_request_target safety)
if grep -Eiq 'actions/checkout|git (clone|fetch|checkout)|head\.ref|head\.sha.*(clone|checkout)' "$WORK/called.yml"; then
  fail V05 "called workflow checks out no pull request code" "$(grep -Ein 'actions/checkout|git (clone|fetch|checkout)' "$WORK/called.yml" | head -3 | tr '\n' ' ')"
else pass V05 "called workflow checks out no pull request code"; fi

# V06: every uses: in the caller is a full 40-character SHA (ci.md)
BAD=$(grep -E '^[[:space:]]*(-[[:space:]]*)?uses:[[:space:]]' "$CALLER" | grep -Ev '@[0-9a-f]{40}([[:space:]]|$)' || true)
if [ -z "$BAD" ]; then pass V06 "every uses: in the caller is pinned to a 40-character SHA"
else fail V06 "every uses: in the caller is pinned to a 40-character SHA" "$BAD"; fi

# V07: no run: block in the called workflow interpolates ${{ }} into shell (values go through env)
python3 - "$WORK/called.yml" > "$WORK/v07" <<'PY'
import sys
lines = open(sys.argv[1]).read().splitlines(); hits = []; inside = False; base = 0
for n, l in enumerate(lines, 1):
    s = l.strip(); ind = len(l) - len(l.lstrip())
    if inside and s and ind <= base: inside = False
    if s.startswith('run:') or s.startswith('- run:'):
        inside, base = True, ind
        if '${{' in s: hits.append(n)
        continue
    if inside and '${{' in l: hits.append(n)
print("lines=" + ",".join(map(str, hits)))
sys.exit(1 if hits else 0)
PY
if [ $? -eq 0 ]; then pass V07 "called workflow's run: blocks interpolate no \${{ }} expressions"
else fail V07 "called workflow's run: blocks interpolate no \${{ }} expressions" "$(cat "$WORK/v07")"; fi

# V08: the called job runs (and mints the App token) only for pull_request_target or a recheck/sign comment
if grep -Eq "^[[:space:]]{4}if:.*" "$WORK/called.yml" && grep -q "github.event_name == 'pull_request_target'" "$WORK/called.yml" \
   && grep -q "github.event.comment.body == 'recheck'" "$WORK/called.yml"; then
  pass V08 "called job is gated to pull_request_target and recheck/sign comments"
else fail V08 "called job is gated to pull_request_target and recheck/sign comments" "no job-level event gate"; fi

# V10 (round 2, replaces V09): pull_request_target loads the workflow file from the default branch, not from the
# branch the run is for. Evidence system: radixtarget run 43451652973 (cited on the PR), which ran at dev commit
# 45d7be96 on the closed event of radixtarget PR #128, which merged that commit into dev. Desired (challenge C1): the run
# started after the merge, and the reusable workflow it called is the default branch's pin as of the run, not the pin in
# dev's cla.yml at the merge commit (base-branch sourcing would have loaded that one).
RT=blacklanternsecurity/radixtarget; RUN=43451652973
gh api --method GET "repos/$RT/actions/runs/$RUN" > "$WORK/rt_run.json" 2>/dev/null
RT_EVENT=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["event"])' "$WORK/rt_run.json" 2>/dev/null)
RT_HEAD=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["head_sha"])' "$WORK/rt_run.json" 2>/dev/null)
RT_AT=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["created_at"])' "$WORK/rt_run.json" 2>/dev/null)
RT_REF=$(python3 -c 'import json,sys; r=json.load(open(sys.argv[1])).get("referenced_workflows") or []; print(next((w["sha"] for w in r if w["path"].startswith("blacklanternsecurity/release-workflows/.github/workflows/cla-reusable.yml@")), ""))' "$WORK/rt_run.json" 2>/dev/null)
RT_DEF=$(gh api --method GET "repos/$RT" --jq .default_branch 2>/dev/null)
RT_DEF_SHA=$(gh api --method GET "repos/$RT/commits?sha=$RT_DEF&until=$RT_AT&per_page=1" --jq '.[0].sha' 2>/dev/null)
pin() { gh api --method GET "repos/$RT/contents/.github/workflows/cla.yml?ref=$1" -H 'Accept: application/vnd.github.raw+json' 2>/dev/null | grep -E 'cla-reusable.yml@' | head -1 | sed -E 's/.*cla-reusable.yml@([0-9a-f]{40}).*/\1/'; }
if [ -z "$RT_REF" ] || [ -z "$RT_HEAD" ] || [ -z "$RT_DEF_SHA" ]; then
  skip V10 "pull_request_target loads cla.yml from the default branch" "could not read run $RUN or $RT_DEF"
else
  RT_MERGED=$(gh api --method GET "repos/$RT/pulls?state=closed&base=dev&head=blacklanternsecurity:feat/cla-v1.3.0" --jq '.[0].merged_at // empty' 2>/dev/null)
  RT_MSHA=$(gh api --method GET "repos/$RT/pulls?state=closed&base=dev&head=blacklanternsecurity:feat/cla-v1.3.0" --jq '.[0].merge_commit_sha // empty' 2>/dev/null)
  P_DEF=$(pin "$RT_DEF_SHA"); P_DEV=$(pin "$RT_MSHA")
  if [ -z "$RT_MERGED" ] || [ -z "$P_DEV" ]; then skip V10 "pull_request_target loads cla.yml from the default branch" "could not read PR merge or dev pin"
  elif [ "$RT_EVENT" = "pull_request_target" ] && [[ "$RT_MERGED" < "$RT_AT" ]] && [ "$RT_REF" = "$P_DEF" ] && [ "$RT_REF" != "$P_DEV" ]; then
    pass V10 "pull_request_target loads cla.yml from the default branch (run $RUN at $RT_AT, after merge $RT_MERGED, called @${RT_REF:0:8} = $RT_DEF@${RT_DEF_SHA:0:8}'s pin, not dev@${RT_MSHA:0:8}'s @${P_DEV:0:8})"
  else fail V10 "pull_request_target loads cla.yml from the default branch" "event=$RT_EVENT run=$RT_AT merged=$RT_MERGED called=$RT_REF default_pin=$P_DEF dev_pin=$P_DEV"; fi
fi

# V11 (round 2): until promotion, contributor-assistant's self-lookup (w.name == context.workflow) runs with the
# default branch's name: line, so it needs that name to equal the repository's workflow record for cla.yml.
# Desired: they are equal (the lookup round 1's F7 said would fail finds its workflow).
DEF=$(gh api --method GET "repos/$REPO" --jq .default_branch 2>/dev/null)
DN=$(gh api --method GET "repos/$REPO/contents/$CALLER?ref=$DEF" -H 'Accept: application/vnd.github.raw+json' 2>/dev/null | grep -E '^name:' | head -1 | sed -E 's/^name:[[:space:]]*//; s/^"(.*)"$/\1/')
RN=$(gh api --method GET "repos/$REPO/actions/workflows?per_page=100" --jq '.workflows[] | select(.path==".github/workflows/cla.yml") | .name' 2>/dev/null)
if [ -z "$DN" ] || [ -z "$RN" ]; then skip V11 "default branch's cla.yml name equals the workflow record" "could not read $DEF copy or workflow record"
elif [ "$DN" = "$RN" ]; then pass V11 "default branch's cla.yml name equals the workflow record ('$RN')"
else fail V11 "default branch's cla.yml name equals the workflow record" "$DEF '$DN', record '$RN'"; fi

# V12 (round 2, challenge C7, A8): the workflow record follows the default branch's name: through a promotion.
# radixtarget made this PR's rename (CLA Assistant -> CLA) and promoted it; desired: its record is now 'CLA', equal to its
# default branch's file, and the CLA runs after the rename ran under 'CLA' and none failed.
RTD=$(gh api --method GET "repos/$RT/contents/.github/workflows/cla.yml?ref=$RT_DEF" -H 'Accept: application/vnd.github.raw+json' 2>/dev/null | grep -E '^name:' | head -1 | sed -E 's/^name:[[:space:]]*//; s/^"(.*)"$/\1/')
RTR=$(gh api --method GET "repos/$RT/actions/workflows?per_page=100" --jq '.workflows[] | select(.path==".github/workflows/cla.yml") | .name' 2>/dev/null)
RTN=$(gh api --method GET "repos/$RT/actions/workflows/cla.yml/runs?branch=$RT_DEF&per_page=100" --jq '[.workflow_runs[] | select(.name=="CLA")] | "\(length) \([.[] | select(.conclusion=="failure")] | length) \([.[] | select(.name=="CLA Assistant")] | length)"' 2>/dev/null)
RTO=$(gh api --method GET "repos/$RT/actions/workflows/cla.yml/runs?branch=$RT_DEF&per_page=100" --jq '[.workflow_runs[] | select(.name=="CLA Assistant")] | length' 2>/dev/null)
set -- $RTN
if [ -z "$RTD" ] || [ -z "$RTR" ] || [ -z "${1:-}" ]; then skip V12 "workflow record follows the default branch's name through a promotion (radixtarget)" "could not read radixtarget"
elif [ "$RTD" = "CLA" ] && [ "$RTR" = "CLA" ] && [ "${RTO:-0}" -gt 0 ] && [ "$1" -gt 0 ] && [ "$2" -eq 0 ]; then
  pass V12 "workflow record follows the default branch's name through a promotion (radixtarget: file '$RTD', record '$RTR', $1 runs as CLA after $RTO as CLA Assistant, 0 failed)"
else fail V12 "workflow record follows the default branch's name through a promotion (radixtarget)" "file='$RTD' record='$RTR' cla_runs=${1:-} failed=${2:-} old_runs=$RTO"; fi

summary

@pjhartlieb
pjhartlieb merged commit 3532105 into dev Nov 13, 2026
20 checks passed
@pjhartlieb
pjhartlieb deleted the feat/cla-v1.3.0 branch November 13, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants