Support for PBS cluster with Singularity and mpirun

[Pedrexus]

Hello all, thanks for the amazing library!

I am in the process of porting my code to NeMo 2.0 and NeMo Run. I work with a GH200 HPC cluster and run commands via PBS with Singularity with the NeMo container image and mpirun for going multi-node. Could I use some of the executors to run that? Does the DockerExecutor support Singularity/Apptainer? Would it be possible I make my own executor, and is there a tutorial for that?

I appreciate any help!

EDIT 1: I also have access to enroot 3.5, but haven't used it before.