Skip to content

Add F* verification tooling, agent guidance, and runnable examples - #75068

Closed
pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/extend-gh-aw-formal-methods-f
Closed

pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/extend-gh-aw-formal-methods-f

Conversation

Copilot AI commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Extend GH-AW’s formal-methods guidance with executable F* verification alongside TLA+/TLC and Z3, including reproducible tooling and agent-assisted proof repair.

  • Tooling: Add checksum-pinned F* 2026.10.25 and bundled Z3 5.3.0 through shared setup hooks, with version checks and smoke verification.
  • Agent guidance: Integrate pinned Proof Copilot skills, document optional MCP support, and define the Specify → Implement → Verify → Diagnose → Repair → Reverify loop.
  • Verification discipline: Preserve acceptance contracts, reject proof bypasses, retain reproducible evidence, and distinguish verified models from production implementations.
  • Runnable examples: Pair an F* counter contract and failing-proof repair with complementary TLC safety/progress checks. Add independent read-only Actions verification and an agentic repair workflow.
  • TLA+ compatibility: Refresh the replaced upstream TLC asset’s checksum and affected locks; retain existing work-queue version pins.
"$FSTAR_BIN" --smt "$FSTAR_Z3" --z3version 5.3.0 \
  --cache_off --z3rlimit 50 specs/formal-methods/Counter.fst

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add first-class F* formal verification guidance and runnable examples Add F* verification tooling, agent guidance, and runnable examples Nov 5, 2026
Copilot AI requested a review from pelikhan November 5, 2026 02:32
Comment on lines +46 to +51
- name: Reverify the candidate repair (feedback, not an independent gate)
run: |
set -euo pipefail
python3 specs/formal-methods/check.py \
--repair /tmp/gh-aw/agent/proof-repair/CounterBroken.fst \
--results /tmp/gh-aw/agent/formal-repair-evidence
tools:
github: false
edit:
bash: ["*"]
Comment thread .github/aw/formal-methods.md Outdated
in workflow frontmatter. Its `steps:` hook runs after checkout and before the
agent, installing the Linux x64 F* **2026.10.25** binary release and selecting
its bundled **Z3 5.3.0**. The archive is SHA256-pinned to
`a716153960545bc26b9d093f55614fedc716a202e4daa1665e5c63e4a2fb04e4`,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot don't be this specific, the agent should resolve the sha on its own and pass the sha as a arg to install.sh

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the guidance to require independently resolving the release asset SHA and passing it to the installer; the installer and workflow call sites now enforce that interface. Committed as 877fb48.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the guidance to say to resolve the archive SHA-256 and pass it to the installer. Commit: 83f74d8.

@pelikhan

pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot keep it minimal

Copilot AI commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor Author

@copilot run pr-finisher skill

Completed one PR-finisher pass. The PR is still draft; current-HEAD completed checks show no failures, with scans still running.

@copilot keep it minimal

Kept this pass minimal: no code changes or empty commit.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot simplify these changes to a minimalistic update. shared/fsahrp.md must be self contained, not scripts

@pelikhan

pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot use env vars for f* version/sha

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor Author

@copilot simplify these changes to a minimalistic update. shared/fsahrp.md must be self contained, not scripts

Made the shared F* hook self-contained and removed the separate installer script, updating the standalone verifier and tests accordingly. This is in local commit e71c305; the remote rejected the push, so it is not yet on the PR.

@copilot use env vars for f* version/sha

FSTAR_VERSION and FSTAR_SHA256 are now environment pins in the shared hook and independent verifier. This is in local commit e71c305; the remote rejected the push, so it is not yet on the PR.

@pelikhan

pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

@copilot do not pollute instructions with how to use f*, focus on installation and how to run f*. the model knows how to use f*. same for other tools.

@pelikhan pelikhan closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants