Repository navigation
Suggestion: List of possible improvements / work聽#19
Description
Doing this list for some nice-to-have's on my side, but anyone is obviously free to do them before me, putting this as issue here may help potential contributors 馃檪
Improvements
-
Add decoding support. (H.264 in progress in feat: add H.264 hardware decoding聽#26)
-
Improve code re-use across encoders. (Some work towards this was done in feat: encode pipelining with "pushification" of outputs聽#21)
-
Add
VkVideoEncodeTuningModeKHR,VkVideoEncodeContentFlagBitsKHRandVkVideoEncodeUsageFlagBitsKHRsupport for tuning encoding latency along other tunables. - feat: Add encoding hints and tuning modes聽#20 -
B-frame support for potentially better encoding efficiency.
-
Video Encode Intra Refresh, reduces bitrate-spikes, adds error resiliency and potentially lowers the decode latency by gradually constructing frame-data instead of having periodic and fat I-frame being encoded and sent.
-
Video Encode Quantization Maps, allow to specify video regions that are potentially higher or lower quality, known as ROI (Region Of Interest) encoding on VA-API. Cool example is Valve using this with their upcoming VR headset, when streaming content from a PC they use eye-tracking to focus bitrate where the person is looking, leaving peripheral area at lower quality.
-
Update Rust edition from 2021 to 2024, will require updating
unsafeusages along possibly other project-wide changes. Benefit is access to improved usage patterns along other niceties. -
Update to latest working
ashcommit, the current used package is 6 months old, Vulkan evolves fast and getting access to latest extensions can open up some niceties (i.e.VK_KHR_video_encode_feedback2was recent addition to the spec) (Work on this done in fix: round color quants, add measurement example and update deps聽#37)
Activity
urwrstkn8mare commentedon Jun 14, 2026
I'm not really an export on encoding but just out of interest to learn would B-frame support increase latency? I'm assuming this would an optional feature for those who really need lower bitrate at the cost of latency.
Also how would ROI be used for game streaming outside of VR? Sounds really cool but I'm curious on what use case you're seeing.
DatCaptainHorse commentedon Jun 14, 2026
B-frames would increase latency yes, but pixelforge isn't targeted at just low-latency streaming so it's a nice-to-have for those that want or need better encoding efficiency with it. So for low-latency streaming no much use.
ROI is in the same bandwagon somewhat as a nice-to-have. Personally I'd like to test how good you can get with machine-learning based eye-tracking, using plain webcam as input - latency wouldn't be as good but for slower-paced game content it could be interesting to see the effect.
urwrstkn8mare commentedon Jun 14, 2026
Oh yeah true. Latency isn't everything having the stream quality almost as good as it is when you're playing directly on your computer is almost as important.
hgaiser commentedon Jun 14, 2026
I agree 馃憤 this is also why I opted to make it a standalone library. I currently only need it for low-latency streaming, but I imagine there'll be other usecases in the future.