Hardware encoding

[theLucy]

Any plans to add support for hardware encoding?

[valadaptive]

I looked into this a bit ago. GStreamer does support hardware encoding, but there are some reasons it's not straightforward to expose that functionality:

• There could be bugs in the hardware encoders' implementations. I've already had to disable NVDEC decoding in GStreamer due to a bug that would make it randomly fail to load videos.

• I need some way to enumerate and allow the user to select encoding devices. Not sure if GStreamer exposes this functionality.

• Different hardware uses different APIs. For instance, Nvidia uses NVENC, AMD and Intel use VA-API, and everything on macOS uses VideoToolbox.

• These different APIs expose different settings to the end user. The software encoder (x264) uses CRF, a variable-bitrate encoding mode with a simple quality slider. The encoding speed slider selects from x264's presets ranging from "very slow" to "very fast".

  None of the hardware APIs support CRF. All except for VideoToolbox support a variable-bitrate mode, but the quality still varies depending on the hardware vendor. Speed preset selection also varies depending on the API.

• The hardware encoding APIs may not support all resolutions or encoding modes. For instance, none of them support interlaced encoding, and VA-API requires chroma subsampling to be enabled. NVENC has a minimum resolution of 160x64 and a maximum resolution of 4096x4096--a wide enough range that most users are unlikely to go outside it, but they could.

  VA-API claims to support any resolution, but Intel's QuickSync API has a minimum resolution of 16x16, and AMD's AMF has a minimum resolution of 128x128. If you use VA-API on an Intel or AMD card, and go below 16x16 or 128x128 respectively, what will happen? Will it work because the minimum resolutions are artificial limitations in the Intel and AMD-specific APIs? Or will it fail at runtime with a confusing error because they are actual hardware limits? The only way to tell is to try it out on the hardware in question.

Hardware encoding APIs are bespoke, idiosyncratic, and require manual testing to ensure they actually work as intended across all inputs. If I can find a way to expose them without causing weird failure modes or making the settings require a ton of trial and error, I'll do that, but it's a big undertaking.