Add warmup/cooldown offsets to the aperf diff/viewer

[sebpop]

I’m hitting a small snag when profiling Rust benchmarks using cargo bench. Since these benchmarks typically include a warmup period (usually ~5s), the samples collected during that window are skewing the results. APerf’s diagnostics and IPC analysis get a bit "confused" by the low-utilization data from the warmup/cooldown phases, making the final report less representative of the actual steady-state performance.

Proposed Solution
It would be awesome if we could add begin and end offsets to the diff and viewer tools. This would allow us to:

Filter out the noise: Manually exclude the warmup/cooldown windows from the visualization and analysis.

Post-collection slicing: If I collect a large sample set (e.g., scaling from 1 to 16 threads), I could use these offsets to "zoom in" on a specific thread count's execution window without needing to re-run the collection.

I think this would make the tool even more powerful for benchmarkers. Let me know what you think!

[CongkaiTan]

Thanks for creating the issue!

We're aware of the inconveniences caused by the noises from warmups and cooldowns, so we have prioritized a task this year to improve the report's scalability and usability, and that includes the feature where users can adjust the time range in the report UI.

However, since the report is currently in the static HTML format, APerf performs many computations and processing in Rust when generating the report, and the report frontend is only responsible for rendering the results - just to avoid being blocked by browser's resource limitations.

Therefore, we need to come up with a new model to make dynamic computations possible (e.g. a desktop app, maybe). The planned delivery date is by the end of Q3 this year.

Please let us know if you have any questions, suggestions, or concerns!

[arthurpetitpierre]

Another option would be to be able to add warmup/cooldown offsets at report generation time. May not be the definitive fix, but a good intermediate option, much easier to implement. It's not as good as doing it at visualisation time, but it's better than not being able to do it at all.

[sebpop]

I thought you were doing javascript for the fancy browser experience.
JS could dynamically extract data from the static data.tar.gz collected on the SUT.

As Arthur suggests, if the HTML output remains static, allowing users to pass --start-offset and --end-offset flags would enable us to re-generate reports that 'crop out' the warmup noise without needing a full dynamic frontend or a desktop app.

[CongkaiTan]

Of course, as the first step, we can add the new CLI options to crop the report's time range statically. We'll include them in the next release (which should be around mid of this month). Thanks for the suggestions!

[rajeshvandanapu]

One workaround meanwhile: Does starting workload first, let it complete warmup, then launch APerf during steady-state execution. Stop APerf before workload finishes to avoid cooldown noise will work ?

[CongkaiTan]

Starting v1.2.2, APerf supports the --time-range option for report generation (aperf report):

--time-range RUN_NAME=FROM_TIME:TO_TIME

The time range to apply to a run in the report, including its time-series metrics, statistics, and analytical findings. Specify the option > multiple times to apply a time range for multiple runs, or omit the RUN_NAME= part to apply it to all runs. Either bound can be omitted or negative.

You can now regenerate the report using this option to trim the warmup or cooldown time. Also, as mentioned before, we will further improve the report usability and make trimming easier (ideally without needing to regenerate the entire report).