rutabaga virglrenderer fallback when gpu is absent

[valpackett]

Currently if we have the entire GPU setup working and try to launch without /dev/dri present, the entire GPU worker crashes..

DEBUG rutabaga_gfx::virgl_renderer] failed to initialize vrend winsys

thread 'gpu worker' panicked at src/devices/src/virtio/gpu/virtio_gpu.rs:282:14:
Rutabaga initialization failed!: ComponentError(-1)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

causing a 5 second delay in the guest kernel boot process..

[    0.314939] printk: legacy console [hvc0] enabled
[    5.345195] [drm:virtio_gpu_get_capsets] *ERROR* timed out waiting for cap set 0
[    5.346399] [drm] Initialized virtio_gpu 0.1.0 for virtio-mmio.3 on minor 0

and leaving it without cross-domain (Wayland/etc.) functionality.

What would be the best way, API-wise, to make it possible to automatically fall back to a minimal no-gpu flag set (such as just VIRGLRENDERER_NO_VIRGL) that would let cross-domain work?

Currently the error isn't even really catchable by the frontend as it just crashes a secondary thread while the VM keeps running..

[mtjhrc]

What would be the best way, API-wise, to make it possible to automatically fall back to a minimal no-gpu flag set (such as just VIRGLRENDERER_NO_VIRGL) that would let cross-domain work?

Personally I don't like implicit fallbacks like that, it makes you think something is working, when it really isn't. In an end user application I'd be more fine in making judgments for the user, but in a library I don't like that at all.

Currently the error isn't even really catchable by the frontend as it just crashes a secondary thread while the VM keeps running..

Yes, this is a general problem in our devices. The way to fix this would be to setup a channel when starting the worker thread, that waits on initialization success/error msgs. The activate method of each device can wait on this message and forward the error. But currently we would just panic! if the device activation fails (which instead should be propagated to the user).

To let the library fully handle it, but that requires non trivial changes to the architecture, but this should be the option IMO. Namely virglrenderer must be used by the thread that initiated it, which means if we had an explicit function to initialize virglrenderer (or generally Rutabaga), it would have to leave that thread running and blocked on waiting for the device initialization. Upon device activation the device could then pass the memory, queues etc. to the worker thread and then unlock the worker thread to let it work normally.

Ideally I imagine the API looking something like this (but I'm not sure how hard this would be to implement).

// Initializes Rutabaga/virglrenderer thread here
int gpu  = krun_open_gpu(/*GPU arguments, flags*/...); 

if (gpu < 0 ) {
    // If it failed, you can try to initialize GPU with different args
    gpu  = krun_open_gpu(/*Different GPU arguments, flags*/...);
}

if (gpu > 0) {
   err = krun_add_gpu(ctx, gpu, /*SHMEM Size*/ 2048);
} else {
    errno = -gpu;
    perror("Failed to initialize GPU")
    // Here the library user chose to not add the GPU to the VM, 
    // but they could also crash on their end if they wanted to
}

...
// The GPU device is initialized normally on vm startup, 
// but it uses the already started Rutabaga thread 
krun_start_enter(ctx);

(We also need richer error messages than just errno codes, but that is also a general issue not specific to the GPU)

[slp]

I think a reasonable compromise would be dealing with this directly inside virtio_gpu. If rutabaga_gfx initialization fails, we can warn and try again with a minimal capset. This would lead to a barely useful /dev/dri, but at least will not stall the boot for that long.

I'll try to come up with a draft PR later.