Skip to content

tests: running run-tests.php deletes legacy tracked .php test files that share a basename with .phpt #187

Description

@s2x

Problem

Running the test suite (php run-tests.php -n tests/) silently deletes tracked legacy test files tests/<name>.php whenever a tests/<name>.phpt with the same basename exists.

Observed during the #177 session: a full suite run deleted 49–50 legacy files from the working tree (restored manually with git checkout -- tests/).

Root cause

run-tests.php (bundled from php-src) derives the executable path from the .phpt basename in the same directory and unlinks it unconditionally — with no guard against deleting pre-existing/git-tracked files:

// run-tests.php:1942 / 1952
$main_file_name = basename($file, 'phpt');
$test_file = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . 'php';

// run-tests.php:1982–1991 — runs before every test, no existence check
// unlink old test results
@unlink($test_file);   // ← deletes a pre-existing tracked file too

The file is then overwritten with generated test code (save_text($test_file, ...), line 2276) and unlinked again after the run (line 2618) unless --keep-php is used. --keep-php does not prevent the pre-test unlink at line 1991.

Impact

Reproduction (empirical, verified)

ls tests/bug_0002.php          # exists (tracked in git)
php -n run-tests.php --no-progress tests/bug_0002.phpt >/dev/null 2>&1
ls tests/bug_0002.php          # No such file or directory
git checkout -- tests/bug_0002.php

Suggested fixes

  1. Recommended: delete the 50 legacy .php files from git in a dedicated migration PR (they are fully superseded by .phpt; task chore: cleanup php/ directory — move examples, remove embeddings.php #24) — removes the root hazard
  2. Add a sanity guard (script or CI pre-check) that refuses to run the suite while tracked .php files overlap with .phpt basenames
  3. Document in AGENTS.md that suite runs mutate tests/ until the migration lands

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:highHigh prioritytype:testTest coverage / test infrastructure issues

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions