Render wgpu frames as webview overlay

[Armaldio]

Hey all 👋

I'm trying to render a wgpu frame over the webview with the goal to support the steam overlay.
Here is my advancement so far:

Kooha-2024-12-09-08-13-17.mp4

The green part is wgpu rendering to the window.
You can see the webview buttons rendering on top of it
What I'm trying to do is to have the webview under the wgpu frames with transparency (to show the overlay over the webview)
Using maybe some sort of frame ordering, I don't know

My work is based on:

• https://github.com/clearlysid/tauri-wgpu-cam/blob/main/src-tauri/src/main.rs
• Render WebView on Top of Native GPU Rendered Content [feat]  #8246
• https://github.com/FabianLars/tauri-v2-wgpu
• [bug] Flickering with raw display handle + winit + wgpu + tauri v2 + transparency #9220
• WebView2 based app and Steam Overlay MicrosoftEdge/WebView2Feedback#4019

Other solutions I've explored but don't know the viability:

• Do some headless wgpu rendering to compute a texture, send the buffer to the frontend and draw it into a canvas, on top of the rest.
  • I've see 300ms rendering on a low-end PC so i'm concerned about performance
  • I don't know how to handle mouse events since only an image of the overlay is transfered
• Use a second window:
  stacking two windows on top of each other
  make the overlay window topmost, transparent, frameless
  when overlay is present disable passthrough, otherwise enable passthrough (to allow mouse clicks and other to hit the correct window)
  • I'm also concerned it doesn't work in practice
  • What about fullscreen ?

What do you think about these solutions ?

[cdbattags]

Any code to review?

[Armaldio]

You can find two of my lib.rs experiments
I don't remember exactly how it works nor how to exactly setup, but you'll find out if you know what you are doing

https://pastebin.com/CnvXj5wF
https://pastebin.com/kWTYN1GX

I'll try to find some time later to properly setup multiple reproduction test cases

[Elanis]

I didn't post any message, but I'm following the post as I initially raised the issue for my game (which is still not solved).
So if you need any help or review, feel free to post indeed.

---

As for your questions, I don't have the answer as I don't know exactly how it works inside, but I'd be apply to try tuning your code and seeing if I find alternative ways to fix the issue :)

[Elanis]

Since I wasn't able to make it work properly after a few tries, and have a few others important bugs. I've given up on the idea of having a Tauri app on Steam for the moment

[Dexus]

Only for Following. Hope someone get a success way to do it.

EDIT:
I would also like to add my two cents, even if it might mean a bit of overhead.

Currently, and purely theoretically, I would do the following:

Give the Tauri app a switch that can only be called up again from within its own programme – with environment or parameter, whatever. Then create a full screen with a logo or whatever, for example with winit and wgpu, create the overlay that is only started on request and provided with the corresponding URL (I'm assuming in-game payment here).

It would also be possible to create a click-through here, but not all platforms would be supported – as far as I know from a former colleague, Wayland and the mobile apps would be left out. Windows, MacOS and X11 under linux should work.

Or ask today a AI to implement something that works.

EDIT:
It would need a sidecar or own app via command parameter. As the Code would need to be on the Main Thread on MacOS. Difficult.