feat: DH-21975: Gc benchmarking - #421

Draft

stanbrub wants to merge 114 commits into

deephaven:mainfrom

stanbrub:gc-benchmarking

Commits on Mar 13, 2026

Disabled some benchmarks and scaled
stanbrub
committed
9d490f5
Scaled up basic math combo
stanbrub
committed
47f066f

Commits on Mar 19, 2026

Merge branch 'deephaven:main' into gc-benchmarking
stanbrub
authored
dea74d7

Commits on Mar 20, 2026

Added a Local Parquet Generator as opposed to going through Kafka
stanbrub
committed
15cf1f4

Commits on Mar 24, 2026

Added local parquet generator and 1st training test
stanbrub
committed
8604111

Commits on Mar 25, 2026

Added more train benchmarks. Improved Local Parquet Generator
stanbrub
committed
83b1c11

Commits on Mar 26, 2026

Revert BasicMathCombo
stanbrub
committed
c552c01
Revert BasicMathCombo
stanbrub
committed
62aa96a
Reverted scale and disabled for pre-train standard tests used for previous GC pass
stanbrub
committed
f78ca22

Commits on Mar 31, 2026

Parallelized local parquet. worked around directory link failures
stanbrub
committed
e5412e7

Commits on Apr 1, 2026

Added 1st pass at benchmark even retrieval with JFR
stanbrub
committed
ff4d891

Commits on Apr 7, 2026

Merge branch 'deephaven:main' into gc-benchmarking
stanbrub
authored
f35ab4f
Added jfr events
stanbrub
committed
25629cc
Merge branch 'deephaven:main' into gc-benchmarking
stanbrub
authored
254cca0

Commits on Apr 9, 2026

Added UGP events
stanbrub
committed
528c365

Commits on Apr 10, 2026

Rescaled only static trained for 120 secs
stanbrub
committed
bd5ff02
Updated adhoc for local parquet env variables
stanbrub
committed
75449bb
Open up dh data dir so local parquet can work
stanbrub
committed
ec2d95e
More logging for benchmark runs
stanbrub
committed
a402a54
Scaling back AggBy because of system lockup
stanbrub
committed
4cf8357
Restrict the number of parquet threads and memory for the runner
stanbrub
committed
8507794

Commits on Apr 11, 2026

Fixed NaturalJoin OOM
stanbrub
committed
c0b5e7a

Commits on Apr 22, 2026

Added separate scalling for static vs inc
stanbrub
committed
8f1a77f

Commits on Apr 23, 2026

Better separation for running static and inc. Added ugp deltas
stanbrub
committed
2938992
turn on JFR metrics
stanbrub
committed
9b326e0
Turn off Inc runs
stanbrub
committed
7fe14cc

Commits on May 5, 2026

Added ss_log budget metric
stanbrub
committed
5e1d59c

Commits on May 6, 2026

Added runner setting for auto tune cycle factor
stanbrub
committed
a1316d4

Commits on May 15, 2026

G1 inc release max
stanbrub
committed
af3d82b

Commits on May 16, 2026

ParallelGC inc release max
stanbrub
committed
28745a8
Shenandoah GC inc release max
stanbrub
committed
ac0c87f
ZGC inc release max
stanbrub
committed
9d630d9

Commits on May 17, 2026

Inc release filter min
stanbrub
committed
dfa44df

Commits on May 21, 2026

Updated for 100ms cycle at 90% min
stanbrub
committed
3ab1845

Commits on May 28, 2026

Added state log ugp times
stanbrub
committed
e0e763f
Pared down to two Filter/Nat tests
stanbrub
committed
4fd604a

Commits on May 29, 2026

Scaled for 1 sec cycles
stanbrub
committed
07eacdf

Commits on Jun 1, 2026

Set autotune to 90% target release
stanbrub
committed
90e7153

Commits on Jun 2, 2026

User JVM 25 for adhoc
stanbrub
committed
f18c994

Commits on Jun 3, 2026

Added update listener cycle times instead of server state log ones
stanbrub
committed
8240602
Set 20% inc release
stanbrub
committed
9951735
Change inc release to 40%
stanbrub
committed
7c18a2c

Commits on Jun 4, 2026

Change inc release to 60%
stanbrub
committed
ce992b3
Change inc release to 80%
stanbrub
committed
fcde3fa
Change inc release to 100%
stanbrub
committed
ff2795a
Run 50ms against zfc
stanbrub
committed
60839de
Make release filter table smaller based on inc release factor
stanbrub
committed
d18049c
Change inc release to 40%
stanbrub
committed
c43719a
Change inc release to 20%
stanbrub
committed
45d5314
Change inc release to 40%
stanbrub
committed
36b1e83
Change inc release to 60%
stanbrub
committed
d4e0255
Change inc release to 80%
stanbrub
committed
1b351a5

Commits on Jun 5, 2026

Change inc release to 100%
stanbrub
committed
4f4124a
Scale for Java 25
stanbrub
committed
11dbba7
Roll back changes some unwanted changes
stanbrub
committed
c048170

Commits on Jun 8, 2026

Switch to java 17
stanbrub
committed
c9af58a

Commits on Jun 9, 2026

Change to JVM 25
stanbrub
committed
5364601

Commits on Jun 10, 2026

Rescaled for 100ms benchmarks
stanbrub
committed
4af7393
Changed autotune to 90% for testing
stanbrub
committed
920c89c

Commits on Jun 11, 2026

Add 1.10% inc target
stanbrub
committed
37dc336

Commits on Jun 12, 2026

Turn off static for now. Do inc 100p
stanbrub
committed
c6f4a00

Commits on Jun 17, 2026

Do huge mem options for 1 sec
stanbrub
committed
e29b3bb

Commits on Jun 18, 2026

Doing Just Inc for 80% throughput now
stanbrub
committed
83f4c78

Commits on Jun 22, 2026

Doing JVM 17 with p1.0 inc and static
stanbrub
committed
6837afa

Commits on Jun 23, 2026

Scale updateBy for 1sec cycles
stanbrub
committed
e093003
Added sleep for slow DH startups
stanbrub
committed
11f7985
Yet more waiting for ZGC
stanbrub
committed
7ed69b6

Commits on Jun 24, 2026

More p90 100ms j25 huge benchmarks
stanbrub
committed
f68564d
1st try at GC report
stanbrub
committed
0f5c61c

Commits on Jun 25, 2026

Reformatted summary tables
stanbrub
committed
d580534
Fixed static names
stanbrub
committed
390f019
Fixed png names
stanbrub
committed
8989a84

Commits on Jun 30, 2026

Fixed inc memory leak. Made train dashboard.
stanbrub
committed
dd15089
Copyright updates
stanbrub
committed
758520f

Commits on Jul 2, 2026

Got train dashboard working against GCloud with csvs pulled up into parquet files
stanbrub
committed
3b9293b
Added all gc rankings tables
stanbrub
committed
1f092aa

Commits on Jul 6, 2026

Added train dashboard usage
stanbrub
committed
c6190b6

Commits on Jul 7, 2026

Revamped tables images gor gc-report
stanbrub
committed
16d1343
Fixed a mispelling
stanbrub
committed
e7daf30
Renamed docker compose file
stanbrub
committed
146d770
Added more train dashboard usage
stanbrub
committed
b42aff5

Commits on Jul 9, 2026

Udpated compare query
stanbrub
committed
47b579c
Merge branch 'deephaven:main' into gc-benchmarking
stanbrub
authored
b8da078
Added matrix workflow for support
stanbrub
committed
528b741

Commits on Jul 10, 2026

Added matrix framework and tests
stanbrub
committed
eec1413
Fixed missing run dir clear
stanbrub
committed
d82a059

Commits on Jul 13, 2026

Added total matrix combinations to show up in the UI
stanbrub
committed
7a06101
Fixed missing =
stanbrub
committed
008cf06
Fixed test row count
stanbrub
committed
3698323
Sigh
stanbrub
committed
660b5bc
Course grained heap fail static
stanbrub
committed
19b229e
Made expected combos optional
stanbrub
committed
f5df613

Commits on Jul 14, 2026

Skip slack notification step for adhoc
stanbrub
committed
aa62c8f
Do heap failure at real scale
stanbrub
committed
19f54e1

Commits on Jul 15, 2026

Commits on Jul 20, 2026

More finer grained heap tests
stanbrub
committed
c14bb51

Commits on Jul 21, 2026

Added RSS and CPU load events. Try with heap fail
stanbrub
committed
a03bcd7
More heap tests because some failed even though they succeeded last time
stanbrub
committed
3f74500
One more failure
stanbrub
committed
090203b
One more failure
stanbrub
committed
c90728c
Dump MODE because it doesn't work
stanbrub
committed
f818ede
Another heap fail
stanbrub
committed
55c24e9
One more
stanbrub
committed
52cbc8e
One more
stanbrub
committed
c443e17
More heap events on success
stanbrub
committed
ee05c7b
Get More event data
stanbrub
committed
90c5600
Synced from main and dealt with the dreaded whitespace
stanbrub
committed
20d109d

Commits on Jul 22, 2026

Disabled AppArmor. Add ssh retries. More heap fail tests
stanbrub
committed
e5b2468

Commits on Jul 29, 2026

Merge branch 'deephaven:main' into gc-benchmarking
stanbrub
authored
736be56

Commits on Jul 30, 2026

Merge App Armor disable
stanbrub
committed
0049900

Commits on Aug 7, 2026

Added heap dash and heap doc
stanbrub
committed
8d74cde