benchmarks: Initial DirectTrajOpt benchmark suite (Ipopt vs MadNLP)#67
Open
jack-champagne wants to merge 8 commits intofeat/madnlp-integrationfrom
Open
benchmarks: Initial DirectTrajOpt benchmark suite (Ipopt vs MadNLP)#67jack-champagne wants to merge 8 commits intofeat/madnlp-integrationfrom
jack-champagne wants to merge 8 commits intofeat/madnlp-integrationfrom
Conversation
Sets up benchmark/ with Project.toml, .gitignore, and three @testitem benchmarks: evaluator micro-benchmarks, Ipopt vs MadNLP comparison, and memory scaling sweep (N × state_dim grid). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Specs for: - Overall HarmoniqsBenchmarks.jl architecture - Altissimo GPU benchmarks (3-way: Ipopt CPU / MadNLP-GPU / Altissimo-GPU) - Implementation plan for DirectTrajOpt + HarmoniqsBenchmarks tasks
- Add .github/workflows/benchmark.yml that runs on PRs touching src/ or benchmark/ - Uses Pkg.add(url=...) to install HarmoniqsBenchmarks (unregistered) - Uploads JLD2 artifacts for 90 days - Add benchmark/README.md with run instructions - Remove empty BenchmarkUtils.jl leftover - Ignore Manifest.toml (regenerated on each CI run)
Uses Julia 1.11+ [sources] section to resolve: - DirectTrajOpt from local path (parent dir) - HarmoniqsBenchmarks from public GitHub URL CI workflow simplified to just Pkg.instantiate (no manual Pkg.add needed).
Remove docs/superpowers/ (specs and plans) from this PR to keep the diff focused on benchmarks and MadNLP integration. Fix stale Pkg.add instruction in benchmark README — deps resolve via [sources] now. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 task
The benchmark @testItems require HarmoniqsBenchmarks which is only available in the benchmark/ project environment, not the test extras. Filter them out so `Pkg.test()` / julia-runtest CI doesn't pick them up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Initial benchmarking infrastructure for DirectTrajOpt.jl comparing Ipopt and MadNLP solvers on the
feat/madnlp-integrationbranch.Depends on: HarmoniqsBenchmarks.jl (new shared package — needs to be created in the org)
What's included
Preliminary Results
Ipopt vs MadNLP (bilinear N=51)
Micro-benchmark hotspot
`eval_hessian_lagrangian` is the clear optimization target — 12.7ms median, 73K allocations, 68MB per call.
Test plan
🤖 Generated with Claude Code