Pin uniffi to git rev with Python buffer perf fix#17
Merged
Conversation
Bumps uniffi from crates.io 0.29 to mozilla/uniffi-rs @ 86a8c0041fc58bb43075dfde349a731d11711188 (PR #2824, "Performance improvement by fixing python loop over bytes"). The generated Python `_UniffiRustBufferBuilder.write` and `_pack_into` now use `ctypes.memmove` instead of a per-byte loop that held the GIL for ~tens of ms per video frame. Merged upstream 2026-02-23 but not in a crates.io release yet (latest 0.31.1 branched before the fix). The in-wrapper monkey-patch that previously mitigated this at import time is no longer needed and should be dropped once bindings regenerate.
Bumps uniffi to mozilla/uniffi-rs@bd74c376547c (main HEAD). The previous pin at 86a8c0041f (PR #2824, "Performance improvement by fixing python loop over bytes", 2026-02-23) landed in a main window that contained a second, then-undetected regression: the "Improved pipeline mappings" refactor (42b3b2b5b8, 2025-12-30) switched type_definitions_from_api to rely on `namespace.visit::<Type>` to enumerate every type needing a converter, but `Box<T>::visit_children` at the time delegated straight to `(**self).visit_children`, never calling the visitor on the inner T itself. Practical effect: the inner type of any `Map`/`Sequence`/`Optional` was invisible to the visitor. `_UniffiFfiConverterFloat64` was referenced by the generated map converter but never emitted, because our only f64 appearances are inside `HashMap<String, f64>` (the state/action wire format). Calling `Portal.send_state` / `send_action` from Python therefore raised:: NameError: name '_UniffiFfiConverterFloat64' is not defined. Did you mean: '_UniffiFfiConverterFloat32'? Fixed upstream 2026-03-19 in baf24e77b5 ("Implement bindgen-tests for Python ... Also fixed a bug in the visitor code that this revealed") by making `Box<T>::visit_children` call `visitor(&**self)`. Current main HEAD carries both that fix and PR #2824, so we land both with a single bump rather than cherry-picking. Roundtripped `HashMap<String, f64>` through the regenerated `_UniffiFfiConverterMapStringFloat64` to confirm the class is now emitted and the buffer perf fix (memmove in `_pack_into` / `write`) is still present.
Adds `tests/test_roundtrip.py` covering both sides of the FFI boundary
for every declared dtype (F64, F32, I32/I16/I8, U32/U16/U8, BOOL).
* Send: one test calls `send_action` with a payload that populates every
dtype and confirms the FFI boundary is crossed (domain-level
`PortalError.WrongRole` is acceptable — the portal isn't connected to
a peer). This is the direct regression guard for the uniffi codegen
bug where `_UniffiFfiConverterFloat64` was referenced by the
generated Map converter but not emitted as a class; prior to the pin
bump, `send_action` raised `NameError` on this exact path.
* Receive: a parametrized sweep across every dtype draws 128 random
in-range values, simulates what the core delivers (narrow to dtype,
widen back to f64 via `TypedValue::as_f64`), and asserts the wrapper
reconstructs the original value with the right Python type. F32 is
compared against the IEEE 754 round-trip (`struct.pack('<f',v)`) so
precision loss is accounted for.
Rust-side narrowing and wire serialization are covered by the core
crate's tests; these Python tests stop at the FFI receive boundary.
Adds `.github/workflows/python-tests.yml` covering the `python/packages/livekit-portal/tests` suite. The workflow builds the Rust cdylib, regenerates the UniFFI Python bindings via the repo-root `scripts/build_ffi_python.sh`, installs the uv workspace, and runs pytest. Rust target cache is keyed off the ffi crate's manifest and the lockfile, so a pin bump rebuilds and unrelated Rust edits do not. README picks up the workflow's badge in the top-of-page badge row so the status is visible without digging through Actions.
Splits the workflow into two parallel jobs — `cargo test --workspace --locked` (52 tests across the core crate) and the existing FFI-build + pytest flow. A failure on either side fails the check, and the per-job names let GitHub's UI say which one broke. Renamed the workflow file to `tests.yml` and the top-level name to `tests` so the badge covers both. README badge URL updated to match.
The livekit Rust SDK's transitive WebRTC/media deps need glib-sys (pkg-config → glib-2.0.pc) plus a handful of X11 / DRM / libva headers on Ubuntu. Without them `cargo test --workspace` fails at compile time with "Package glib-2.0 was not found in the pkg-config search path". Same recipe as livekit/rust-sdks' own CI, minus the desktop- capture and NVIDIA pieces we don't pull in. Also installs protoc (v25.2, matching the SDK) since livekit's build.rs drives protobuf codegen. Applied to both jobs — the python job builds the same cdylib via `scripts/build_ffi_python.sh`, so it hits the same link-time deps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
uniffidep inlivekit-portal-ffifrom crates.io0.29to a git source pinned atmozilla/uniffi-rs@86a8c004— the commit that merges uniffi-rs#2824 ("Performance improvement by fixing python loop over bytes"). Resolves to uniffi 0.31.0._UniffiRustBufferBuilder.writeand_pack_into. For a 480x640x3 video frame that is 921,600 Python bytecode iterations persend_video_frame, holding the GIL long enough to starve motor-bus workers and surface as[TxRxResult] There is no status packet!on blockingserial.read().ctypes.memmove. Merged upstream on 2026-02-23 but not in a crates.io release yet — latest 0.31.1 was branched offmainbefore PR #2824 landed, so a git rev is the only way to pick it up.livekit_portal_ffi.pycontainsctypes.memmovein bothwriteand_pack_into. Drop this pin back to a version req once a release ships with the fix.Breaking-change audit for the 0.29 → 0.31 jump
#[uniffi::trait_interface]. Not applicable — this crate is proc-macro only (no.udl)._Dispatcher(_ffi.PortalCallbacks)and_RpcHandlerAdapter(_ffi.RpcHandler).uniffi-bindgen --lib-fileremoved.scripts/build_ffi_python.shalready uses--library.cargo check -p livekit-portal-ffiand the fullbash scripts/build_ffi_python.sh debug(cdylib + bindgen) pass clean.Test plan
cargo build -p livekit-portal-ffi --releasebash scripts/build_ffi_python.sh releasegrep memmove python/packages/livekit-portal/livekit/portal/livekit_portal_ffi.pyshows the fix inwriteand_pack_into