More Granular SLURM Configurations in XFEL GUI

[kmdalton]

I'm not sure how to address this, but I think it is a barrier to implementing new tasks in the XFEL GUI. Specifically, we're having trouble figuring out a user-friendly way to specify different SLURM configurations for different tasks. This would be a useful feature for, for instance, incorporating GPU tasks into the GUI. At NERSC, we need to switch (at least) the following sbatch parameters between CPU and GPU jobs

• --account
• --constraint
• --ntasks-per-node
• --cpus-per-task
• --time

[phyy-nx]

Howdy, does the Extra Submission Arguments box work? We put in those kind of arguments in there and they get passed on to the slurm script.

[kmdalton]

Absolutely, that works. But having to edit it prevents chaining tasks. Sorry that wasn't clear from my post!

[phyy-nx]

Oh, is the idea that different tasks would have different slurm submission requirements?

[kmdalton]

Yes indeed. For instance, we would like to run

1. indexing
2. ensemble refinement
3. abismal merging

in a single dataset. In this case 3 needs different slurm job parameters. Let me know if that makes sense!

[phyy-nx]

OK, for that we need to be able to add custom multi processing parameters to individual tasks. The way to do this is to make the multiprocessing phil scope so that it can take multiple entries, with a tag that indicates what task each goes with.  This is all something that’s been low-key on my radar, because the alternative is to make the multi processing part of the gui even hairier.There might be other ways to do this that are easier. Let’s chat about it.-AaronOn May 16, 2025, at 5:06 PM, Kevin Dalton ***@***.***> wrote: kmdalton left a comment (cctbx/cctbx_project#1078) Yes indeed. For instance, we would like to run indexing ensemble refinement abismal merging in a single dataset. In this case 3 needs different slurm job parameters. Let me know if that makes sense! —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[kmdalton]

Yes! We should definitely chat about it. There are some important design decisions. I just wanted to make a note of this before I forgot :)