Skip to content

fix: age-filter in scanForLeftovers to guard against concurrent builds - #354

Open
gnodet wants to merge 1 commit into
apache:masterfrom
gnodet:fix/353-scan-leftover-age-filter
Open

gnodet wants to merge 1 commit into
apache:masterfrom
gnodet:fix/353-scan-leftover-age-filter

Conversation

@gnodet

@gnodet gnodet commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Problem

BackgroundCleaner.scanForLeftovers() scans the fast staging directory at session start and queues any leftover directories for background deletion. It did not protect against the case where two concurrent builds share the same staging directory: one build's leftover scan could delete directories that are actively being used by the other build.

Fixes #353.

Fix

Introduce a LEFTOVER_AGE_THRESHOLD_MS constant (30 seconds). scanForLeftovers() now reads the last-modified time of each staged directory and skips any directory younger than the threshold.

  • A directory created by an ongoing concurrent build will always be younger than 30 s at the time of the scan (the BackgroundCleaner is initialised very early in the session, so there is no practical way a live build's staged directories are 30 s old at that point).
  • A directory left behind by a killed build will be older than 30 s in virtually all cases.
  • If getLastModifiedTime() fails (unlikely), the directory is also skipped — conservative, safe default.

An age filter was chosen over the alternative PID-marker approach because:

  • It requires no additional file I/O on the hot path (only during the leftover scan at session start).
  • It is robust on all platforms (no /proc or OS-specific PID checks).
  • The 30-second threshold is conservative enough to make false negatives (live directory treated as leftover) virtually impossible.

Also fixed

A pre-existing missing closing } in CleanerTest.batchRetryLogsWarningAfterRetryWhenStillFailing that was causing a compile error on the spotless/checkstyle phase.

Tests

Two new tests in BackgroundCleanerTest:

  • scanForLeftoversSkipsRecentDirectories — a directory with lastModified = now is not deleted.
  • scanForLeftoversDeletesOldDirectories — a directory with lastModified = now − threshold − 60 s is deleted.

The existing scanForLeftoversDeletesOrphanedDirectories test is updated to set an old timestamp on the leftover directory (it would have been skipped by the new age filter with the default lastModified = now from createDirectory).

apache#353)

BackgroundCleaner.scanForLeftovers() iterated over all directories in the
fast staging area and queued them for deletion. In setups where two concurrent
builds share the same staging directory (e.g. the same ${project.build.directory}),
one build's leftover scan could delete directories actively being used by the other.

Fix: introduce a LEFTOVER_AGE_THRESHOLD_MS constant (30 s). scanForLeftovers()
now reads the last-modified time of each staged directory and skips any directory
younger than the threshold. A directory created by an ongoing concurrent build
will always be younger than 30 s at the time of the scan; a directory left behind
by a killed build will be older than 30 s in virtually all cases.

If getLastModifiedTime() fails (unlikely, but defensive), the directory is also
skipped to err on the side of caution.

Also fix a pre-existing missing closing brace in CleanerTest (the test method
batchRetryLogsWarningAfterRetryWhenStillFailing was missing its closing '}',
causing a compile error that blocked spotless/checkstyle).

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid fix. The age-filter approach is well-chosen — simpler than PID markers, no hot-path I/O overhead, and the 30-second threshold is conservative enough to make false positives (deleting a concurrent build's directory) effectively impossible under normal conditions.

The three critical paths are correctly handled:

  • Recent directories (lastModified > threshold) → skipped
  • Old directories (lastModified <= threshold) → queued for deletion
  • getLastModifiedTime() failure → skipped (safe default)

Test coverage is adequate: both the positive and negative age-filter cases are exercised, and the existing scanForLeftoversDeletesOrphanedDirectories test is properly updated to set an old timestamp.

The CleanerTest fix (missing closing }) is a legitimate pre-existing compile error fix.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fast mode: scanForLeftovers does not protect against concurrent builds sharing the same staging directory

2 participants