Skip to content

Carve slow/e2e tests into separate testSlow tasks #700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jbachorik merged 4 commits into from
Aug 4, 2026
Merged

Carve slow/e2e tests into separate testSlow tasks #700

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
30e9a13
Carve slow/e2e tests into separate testSlow tasks, run only nightly/p…
jbachorik Jul 31, 2026
5502908
Fix TagFilter import in ProfilerTestRunner (wrong package)
jbachorik Jul 31, 2026
734072b
Merge branch 'main' into jb/slow-test-suite
jbachorik Jul 31, 2026
0a04cee
Namespace test artifacts by slow/regular, run slow tests under ASan, …
jbachorik Aug 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/scripts/test_alpine_aarch64.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export LIBRARY="musl"
export CONFIG="${3}"
export JAVA_HOME="${4}"
export JAVA_TEST_HOME="${5}"
SLOW_TESTS="${6:-false}"

export PATH="${JAVA_HOME}/bin":${PATH}

Expand All @@ -33,4 +34,9 @@ apk update && apk add curl moreutils wget hexdump linux-headers bash make g++ cl
# Install debug symbols for musl libc
apk add musl-dbg

./gradlew -PCI -PkeepJFRs :ddprof-test:test${CONFIG} --no-daemon --parallel --build-cache --no-watch-fs
TASK_PREFIX="test"
if [ "${SLOW_TESTS}" = "true" ]; then
TASK_PREFIX="testSlow"
fi

./gradlew -PCI -PkeepJFRs :ddprof-test:${TASK_PREFIX}${CONFIG} --no-daemon --parallel --build-cache --no-watch-fs
16 changes: 15 additions & 1 deletion .github/workflows/nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,20 @@ jobs:
# C++ gtests (ASan + TSan) run on every PR via native-sanitizer-tests in ci.yml.
# Skip them here so the nightly focuses on Java functional tests under ASan.
skip_gtest: true
run-slow-test:
uses: ./.github/workflows/test_workflow.yml
with:
configuration: '["debug"]'
skip_gtest: true
slow_tests: true
run-slow-test-asan:
uses: ./.github/workflows/test_workflow.yml
with:
configuration: '["asan"]'
# C++ gtests (ASan + TSan) run on every PR via native-sanitizer-tests in ci.yml.
# Skip them here so the nightly focuses on Java functional tests under ASan.
skip_gtest: true
slow_tests: true
fuzz:
runs-on: ubuntu-latest
continue-on-error: true
Expand Down Expand Up @@ -52,7 +66,7 @@ jobs:
path: ddprof-lib/fuzz/build/fuzz-crashes/
report-failures:
runs-on: ubuntu-latest
needs: run-test
needs: [run-test, run-slow-test, run-slow-test-asan]
if: failure()
steps:
- name: Download all failure artifacts
Expand Down
14 changes: 11 additions & 3 deletions .github/workflows/release-validated.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,17 +169,25 @@ jobs:
with:
configuration: '["debug", "asan"]'

pre-release-slow-tests:
needs: validate-inputs
if: ${{ inputs.dry_run != true && inputs.skip_tests != true && inputs.release_type != 'retag' }}
uses: ./.github/workflows/test_workflow.yml
with:
configuration: '["debug", "asan"]'
slow_tests: true
Comment thread
jbachorik marked this conversation as resolved.

create-release:
needs: [validate-inputs, pre-release-tests]
if: always() && needs.validate-inputs.result == 'success' && (needs.pre-release-tests.result == 'success' || needs.pre-release-tests.result == 'skipped')
needs: [validate-inputs, pre-release-tests, pre-release-slow-tests]
if: always() && needs.validate-inputs.result == 'success' && (needs.pre-release-tests.result == 'success' || needs.pre-release-tests.result == 'skipped') && (needs.pre-release-slow-tests.result == 'success' || needs.pre-release-slow-tests.result == 'skipped')
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
id-token: write # Needed to federate a token for the bump-PR create-release step
steps:
- name: Check test results
if: ${{ inputs.dry_run != true && inputs.skip_tests != true && inputs.release_type != 'retag' && needs.pre-release-tests.result != 'success' }}
if: ${{ inputs.dry_run != true && inputs.skip_tests != true && inputs.release_type != 'retag' && (needs.pre-release-tests.result != 'success' || needs.pre-release-slow-tests.result != 'success') }}
run: |
echo "::error::Pre-release tests failed. Cannot proceed with release."
exit 1
Expand Down
68 changes: 39 additions & 29 deletions .github/workflows/test_workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ on:
required: false
type: boolean
default: false
slow_tests:
description: "Run the slow/e2e test suite (testSlow<Config>) instead of the regular suite"
required: false
type: boolean
default: false

permissions:
contents: read
Expand Down Expand Up @@ -133,7 +138,7 @@ jobs:

for attempt in $(seq 1 $MAX_ATTEMPTS); do
mkdir -p build/logs
${GRADLEW_PREFIX} ./gradlew -PCI -PkeepJFRs ${{ inputs.skip_gtest == true && '-Pskip-gtest' || '' }} :ddprof-test:test${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
${GRADLEW_PREFIX} ./gradlew -PCI -PkeepJFRs ${{ inputs.skip_gtest == true && '-Pskip-gtest' || '' }} :ddprof-test:test${{ inputs.slow_tests && 'Slow' || '' }}${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
| tee -a build/test-raw.log \
| python3 -u .github/scripts/filter_gradle_log.py
EXIT_CODE=${PIPESTATUS[0]}
Expand Down Expand Up @@ -162,13 +167,13 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success()
with:
name: (build) test-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (build) test-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: build/
- name: Upload failures
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: failures-glibc-${{ matrix.java_version }}-${{ matrix.config }}-amd64
name: failures-glibc-${{ matrix.java_version }}-${{ matrix.config }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: failures_glibc-${{ matrix.java_version }}-${{ matrix.config }}-amd64.txt
- name: Prepare reports
if: always() && steps.set_enabled.outputs.enabled == 'true'
Expand All @@ -178,26 +183,26 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success() && matrix.config == 'debug'
with:
name: (unwinding-reports) unwinding-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (unwinding-reports) unwinding-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: unwinding-reports
- name: Upload test reports
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: (test-reports) test-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (test-reports) test-linux-glibc-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: test-reports
- name: Upload signal-safety violation log
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: signal-safety-violation-glibc-${{ matrix.java_version }}-${{ matrix.config }}-amd64
name: signal-safety-violation-glibc-${{ matrix.java_version }}-${{ matrix.config }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/signal-safety-violation.txt
if-no-files-found: ignore
- name: Upload ASan logs
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure() && matrix.config == 'asan'
with:
name: asan-logs-glibc-${{ matrix.java_version }}-amd64
name: asan-logs-glibc-${{ matrix.java_version }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/asan.log.*
if-no-files-found: ignore

Expand Down Expand Up @@ -273,7 +278,7 @@ jobs:
echo "JAVA_VERSION=${JAVA_VERSION}"

mkdir -p build/logs
./gradlew -PCI -PkeepJFRs :ddprof-test:test${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
./gradlew -PCI -PkeepJFRs :ddprof-test:test${{ inputs.slow_tests && 'Slow' || '' }}${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
| tee -a build/test-raw.log \
| python3 -u .github/scripts/filter_gradle_log.py
EXIT_CODE=${PIPESTATUS[0]}
Expand All @@ -295,13 +300,13 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success()
with:
name: (build) test-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (build) test-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: build/
- name: Upload failures
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: failures-musl-${{ matrix.java_version }}-${{ matrix.config }}-amd64
name: failures-musl-${{ matrix.java_version }}-${{ matrix.config }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: failures_musl-${{ matrix.java_version }}-${{ matrix.config }}-amd64.txt
- name: Prepare reports
if: always()
Expand All @@ -311,26 +316,26 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success() && matrix.config == 'debug'
with:
name: (unwinding-reports) unwinding-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (unwinding-reports) unwinding-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: unwinding-reports
- name: Upload test reports
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: (test-reports) test-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (test-reports) test-linux-musl-amd64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: test-reports
- name: Upload signal-safety violation log
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: signal-safety-violation-musl-${{ matrix.java_version }}-${{ matrix.config }}-amd64
name: signal-safety-violation-musl-${{ matrix.java_version }}-${{ matrix.config }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/signal-safety-violation.txt
if-no-files-found: ignore
- name: Upload ASan logs
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure() && matrix.config == 'asan'
with:
name: asan-logs-musl-${{ matrix.java_version }}-amd64
name: asan-logs-musl-${{ matrix.java_version }}-amd64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/asan.log.*
if-no-files-found: ignore

Expand Down Expand Up @@ -387,10 +392,14 @@ jobs:
- name: Setup OS
if: steps.set_enabled.outputs.enabled == 'true'
run: |
# ports.ubuntu.com's IPv6 route is intermittently unreachable from GH-hosted
# aarch64 runners; force IPv4 so apt doesn't stall/fail on a dead IPv6 path,
# and retry on top in case of other transient connection blips.
printf 'Acquire::ForceIPv4 "true";\nAcquire::Retries "5";\n' | sudo tee /etc/apt/apt.conf.d/99force-ipv4 >/dev/null
sudo apt update -y
sudo apt remove -y g++
sudo apt autoremove -y
sudo apt install -y curl zip unzip clang make build-essential binutils gdb
sudo apt install -y curl zip unzip clang make build-essential binutils gdb libgtest-dev libgmock-dev
# Install debug symbols for system libraries
sudo apt install -y libc6-dbg
if [[ ${{ matrix.java_version }} =~ "-zing" ]]; then
Expand Down Expand Up @@ -448,7 +457,7 @@ jobs:

for attempt in $(seq 1 $MAX_ATTEMPTS); do
mkdir -p build/logs
${GRADLEW_PREFIX} ./gradlew -PCI -PkeepJFRs ${{ inputs.skip_gtest == true && '-Pskip-gtest' || '' }} :ddprof-test:test${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
${GRADLEW_PREFIX} ./gradlew -PCI -PkeepJFRs ${{ inputs.skip_gtest == true && '-Pskip-gtest' || '' }} :ddprof-test:test${{ inputs.slow_tests && 'Slow' || '' }}${{ matrix.config }} --no-daemon --parallel --build-cache --no-watch-fs 2>&1 \
| tee -a build/test-raw.log \
| python3 -u .github/scripts/filter_gradle_log.py
EXIT_CODE=${PIPESTATUS[0]}
Expand Down Expand Up @@ -482,13 +491,13 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success()
with:
name: (build) test-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (build) test-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: build/
- name: Upload failures
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: failures-glibc-${{ matrix.java_version }}-${{ matrix.config }}-aarch64
name: failures-glibc-${{ matrix.java_version }}-${{ matrix.config }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: failures_glibc-${{ matrix.java_version }}-${{ matrix.config }}-aarch64.txt
- name: Prepare reports
if: always() && steps.set_enabled.outputs.enabled == 'true'
Expand All @@ -498,26 +507,26 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success() && matrix.config == 'debug'
with:
name: (unwinding-reports) unwinding-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (unwinding-reports) unwinding-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: unwinding-reports
- name: Upload test reports
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: (test-reports) test-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (test-reports) test-linux-glibc-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: test-reports
- name: Upload signal-safety violation log
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: signal-safety-violation-glibc-${{ matrix.java_version }}-${{ matrix.config }}-aarch64
name: signal-safety-violation-glibc-${{ matrix.java_version }}-${{ matrix.config }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/signal-safety-violation.txt
if-no-files-found: ignore
- name: Upload ASan logs
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure() && matrix.config == 'asan'
with:
name: asan-logs-glibc-${{ matrix.java_version }}-aarch64
name: asan-logs-glibc-${{ matrix.java_version }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/asan.log.*
if-no-files-found: ignore

Expand Down Expand Up @@ -565,7 +574,8 @@ jobs:
docker run --cpus 4 --rm -v /tmp:/tmp -v "${GITHUB_WORKSPACE}:${GITHUB_WORKSPACE}" -w "${GITHUB_WORKSPACE}" alpine:3.21 /bin/sh -c "
\"$GITHUB_WORKSPACE/.github/scripts/test_alpine_aarch64.sh\" \
\"${{ github.sha }}\" \"musl/${{ matrix.java_version }}-${{ matrix.config }}-aarch64\" \
\"${{ matrix.config }}\" \"${{ env.JAVA_HOME }}\" \"${{ env.JAVA_TEST_HOME }}\"
\"${{ matrix.config }}\" \"${{ env.JAVA_HOME }}\" \"${{ env.JAVA_TEST_HOME }}\" \
\"${{ inputs.slow_tests }}\"
" 2>&1 \
| tee -a build/test-raw.log \
| python3 -u .github/scripts/filter_gradle_log.py
Expand Down Expand Up @@ -603,13 +613,13 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success()
with:
name: (build) test-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (build) test-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: build/
- name: Upload failures
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: failures-musl-${{ matrix.java_version }}-${{ matrix.config }}-aarch64
name: failures-musl-${{ matrix.java_version }}-${{ matrix.config }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: failures_musl-${{ matrix.java_version }}-${{ matrix.config }}-aarch64.txt
- name: Prepare reports
if: always()
Expand All @@ -619,25 +629,25 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: success() && matrix.config == 'debug'
with:
name: (unwinding-reports) unwinding-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (unwinding-reports) unwinding-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: unwinding-reports
- name: Upload test reports
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: (test-reports) test-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }})
name: (test-reports) test-linux-musl-aarch64 (${{ matrix.java_version }}, ${{ matrix.config }}, ${{ inputs.slow_tests && 'slow' || 'regular' }})
path: test-reports
- name: Upload signal-safety violation log
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: signal-safety-violation-musl-${{ matrix.java_version }}-${{ matrix.config }}-aarch64
name: signal-safety-violation-musl-${{ matrix.java_version }}-${{ matrix.config }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/signal-safety-violation.txt
if-no-files-found: ignore
- name: Upload ASan logs
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure() && matrix.config == 'asan'
with:
name: asan-logs-musl-${{ matrix.java_version }}-aarch64
name: asan-logs-musl-${{ matrix.java_version }}-aarch64-${{ inputs.slow_tests && 'slow' || 'regular' }}
path: /tmp/asan.log.*
if-no-files-found: ignore
Loading
Loading