Skip to content

tests: run-tests.php leaves untracked tests/*.php artifacts after failed runs #194

Description

@s2x

Finding (from #176 follow-up, verified by subagent review)

run-tests.php extracts each .phpt --FILE-- section to tests/<basename>.php and normally unlinks it — but only when the test passes. Any failed test (fatal error, output mismatch, crash) leaves the generated .php behind.

Evidence

  • run-tests.php:2618-2621@unlink($test_file) is inside the if ($passed) block only. Any failure skips it. (Not specific to fatal errors — any failed test leaves the file; that's stock php-src behavior for debugging.)
  • tests/*.php is not git-ignored (unlike tests/*.diff|.exp|.log|.out|.sh which are), so leftovers show up as untracked files in git status.
  • Observed on disk after suite runs: tests/test_query_fp64.{diff,exp,log,out,sh} artifacts from a failed expected-fail test (Aug 2026). The generated .php was manually removed.
  • Relation to tests: running run-tests.php deletes legacy tracked .php test files that share a basename with .phpt #187: that issue tracks the opposite hazard of the same unlink logic (deleting tracked legacy tests/<name>.php sharing basenames). This case (leftover on failure) is distinct.

Suggested fix

Found during #176 implementation; verified by subagent review (CONFIRMED, not tracked).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:lowLow prioritytype:testTest coverage / test infrastructure issues

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions