You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Just to double-check, as I understand it, OpenEvolve does not explicitly implement a crossover operator for program mutation, unlike ShinkaEvolve. Am I correct?
Yes, OpenEvolve does not have an explicit crossover operator. It uses an "inspirations" mechanism where the LLM is shown a parent program plus several other high-performing programs as context, and asked to improve/mutate. The LLM might implicitly draw ideas from those programs, but there is no dedicated crossover step or prompt that explicitly instructs combining multiple programs.
ShinkaEvolve does indeed have a dedicated, explicit crossover operator. It has a separate prompts_cross.py (see https://github.com/SakanaAI/ShinkaEvolve/blob/main/shinka/prompts/prompts_cross.py) with its own system prompt that literally tells the LLM: "You are given multiple code scripts implementing the same algorithm. Perform crossover between the code scripts." The sampler in ShinkaEvolve has logic to specifically select the crossover mutation type, distinct from regular mutation.
@codelion Thank you for your reply. Your response resolves my question :)
I have another question. I am currently running all the examples (in the examples folder) to collect actual trajectories from the OpenEvolve agent. I am using the default configuration, but there are many stochastic factors involved, such as how to set the number of iterations, how to set the population size, which LLM to use, and so on.
Is the default configuration in the examples folder the best setting you found?
I assume that using more recent or stronger models could potentially lead to better results. Have you tried such experiments?
Since OpenEvolve adopts a form of test-time scaling, have you tested whether performance continues to improve as the number of iterations increases?
The examples were contributed over time by various community members, so they reflect what worked at the time they were added rather than a carefully tuned baseline. Treat them as starting points rather than optimal configs. The README has guidance on how to approach tuning. The key levers are population size, the ratio of exploration vs. exploitation, and how you structure your evaluator. It does take some experimentation to find what works well for a given problem.
Yes, stronger frontier models generally produce better results. I haven't done exhaustive systematic benchmarking across model generations, but the pattern holds that more capable models explore the solution space more creatively. The UC Berkeley ADRS paper ran OpenEvolve with GPT-5 and Gemini 3, and found it achieved the highest success rate across their 10 case studies (best solution in 9 out of 20 runs). Their configs are worth looking at as a reference for what good settings look like on real research problems: https://github.com/UCB-ADRS/ADRS
Yes, performance generally keeps improving with more iterations, especially for open-ended optimization problems where there's no ceiling on the score. That said, there's a real risk of overfitting to the evaluator, the agent can find reward hacks that look good on your metric but don't generalize. So you need to be careful about evaluator design and it's worth validating winners on held-out workloads. The ADRS paper goes into this in detail and has good practical advice on evaluator construction to avoid this problem.
Activity
codelion commentedon Feb 18, 2026
Yes, OpenEvolve does not have an explicit crossover operator. It uses an "inspirations" mechanism where the LLM is shown a parent program plus several other high-performing programs as context, and asked to improve/mutate. The LLM might implicitly draw ideas from those programs, but there is no dedicated crossover step or prompt that explicitly instructs combining multiple programs.
ShinkaEvolve does indeed have a dedicated, explicit crossover operator. It has a separate
prompts_cross.py(see https://github.com/SakanaAI/ShinkaEvolve/blob/main/shinka/prompts/prompts_cross.py) with its own system prompt that literally tells the LLM: "You are given multiple code scripts implementing the same algorithm. Perform crossover between the code scripts." The sampler in ShinkaEvolve has logic to specifically select the crossover mutation type, distinct from regular mutation.passing2961 commentedon Feb 18, 2026
@codelion Thank you for your reply. Your response resolves my question :)
I have another question. I am currently running all the examples (in the
examplesfolder) to collect actual trajectories from the OpenEvolve agent. I am using the default configuration, but there are many stochastic factors involved, such as how to set the number of iterations, how to set the population size, which LLM to use, and so on.examplesfolder the best setting you found?codelion commentedon Feb 19, 2026
The examples were contributed over time by various community members, so they reflect what worked at the time they were added rather than a carefully tuned baseline. Treat them as starting points rather than optimal configs. The README has guidance on how to approach tuning. The key levers are population size, the ratio of exploration vs. exploitation, and how you structure your evaluator. It does take some experimentation to find what works well for a given problem.
Yes, stronger frontier models generally produce better results. I haven't done exhaustive systematic benchmarking across model generations, but the pattern holds that more capable models explore the solution space more creatively. The UC Berkeley ADRS paper ran OpenEvolve with GPT-5 and Gemini 3, and found it achieved the highest success rate across their 10 case studies (best solution in 9 out of 20 runs). Their configs are worth looking at as a reference for what good settings look like on real research problems: https://github.com/UCB-ADRS/ADRS
Yes, performance generally keeps improving with more iterations, especially for open-ended optimization problems where there's no ceiling on the score. That said, there's a real risk of overfitting to the evaluator, the agent can find reward hacks that look good on your metric but don't generalize. So you need to be careful about evaluator design and it's worth validating winners on held-out workloads. The ADRS paper goes into this in detail and has good practical advice on evaluator construction to avoid this problem.
codelion commentedon Feb 19, 2026
Also the recent ADRS blog post - https://www.sigops.org/2026/let-the-barbarians-in-how-ai-can-accelerate-systems-performance-research/ has some comparision between OpenEvolve, ShinkaEvolve, and GEPA.