Skip to content
BranchesTags

Repository files navigation

live2d-rs

Rust 中文

Rust bindings and viewer for Live2D Cubism SDK for Native.

Supports Cubism 5.x (V3) and Cubism 2.x (V2) models in a single application.

Workspace Structure

Crate Description
live2d-core-sys Raw -sys FFI declarations generated by bindgen over the Cubism 5.x Core C API (csmMoc, csmModel, etc.)
live2d-core Safe Rust wrapper for V3 — Moc, Model, Parameters, Parts, Drawables, CanvasInfo, OffscreenInfos
live2d-v2-core-sys bindgen FFI for Cubism 2.x C API (built from live2d-py C++ artifacts)
live2d-v2-core Safe Rust wrapper for V2 — model loading, parameter control, rendering
live2d-viewer Desktop viewer binary: winit + glutin + glow (OpenGL 3.3 Core) + egui UI + self-implemented motion system

live2d-py/ is a separate git submodule (not in the workspace) — its built C++ artifacts feed live2d-v2-core-sys.

Quick Start

Prerequisites

  • Rust 1.75+ (MSRV, set per crate)
  • Cubism 5.x SDK for Native — place at workspace root as CubismSdkForNative-5-r.5/, or set LIVE2D_SDK_ROOT to point elsewhere
  • OpenGL 3.3 Core capable GPU (runtime, for viewer)

Setup

# 1. Download Cubism SDK for Native from Live2D:
#    https://www.live2d.com/download/cubism-sdk/download-native/
#    Extract it as CubismSdkForNative-5-r.5/ under the workspace root.

# 2. Build & run with a model directory:
cargo build --release
cargo run --release -p live2d-viewer -- /path/to/model-directory

The build system statically links libLive2DCubismCore.a from the SDK by default. See Linking for alternatives.

Running with sample models

Sample models are bundled in the Cubism SDK under Samples/Resources/:

cargo run --release -p live2d-viewer -- \
    /path/to/CubismSdkForNative-5-r.5/Samples/Resources/Haru

Auto-scan with LIVE2D_SDK_ROOT

When LIVE2D_SDK_ROOT is set and no model path is given on the command line, the viewer automatically scans $LIVE2D_SDK_ROOT/Samples/Resources/ for all V3 and V2 models:

LIVE2D_SDK_ROOT=/path/to/CubismSdkForNative-5-r.5 cargo run --release -p live2d-viewer

Overlay mode

A small always-on-top window positioned at the bottom-right corner of the screen:

cargo run --release -p live2d-viewer -- --overlay /path/to/model-directory

Commands

# Default build (static link)
cargo build --release

# Dynamic linking (load libLive2DCubismCore.so at runtime)
cargo build --release --no-default-features

# Run viewer
cargo run --release -p live2d-viewer -- /path/to/model-dir

# Overlay mode (small window, bottom-right corner)
cargo run --release -p live2d-viewer -- --overlay /path/to/model-dir

# Testing (requires SDK samples available at LIVE2D_SDK_ROOT/Samples/Resources)
LIVE2D_SDK_ROOT=/path/to/CubismSdkForNative-5-r.5 cargo test --release -p live2d-core-sys

V2 model support (optional)

V2 models require building the C++ wrapper from the live2d-py submodule first:

# Build live2d-py (see its own build instructions)
cd live2d-py && mkdir build && cd build && cmake .. && make

# Set the build output directory and run
V2_PY_BUILD_DIR=/path/to/live2d-py/build cargo run --release -p live2d-viewer -- /path/to/v2-model-dir

The viewer auto-detects model format by checking for *.model3.json (V3) vs *model*.json (V2) in the directory.

Viewer Features

Model Management

  • Model list — select, rename (double-click or ✏ button), delete (✖ button), or add new model directories via file picker
  • Model history — previously loaded models are persisted in a SQLite database (~/.local/share/live2d-viewer/) and restored on startup
  • Scan directories — configure folders to scan recursively for models; generic system names (e.g. engineering code names) are auto-skipped
  • Model search — full-text fuzzy search across model names via SQLite FTS5 with similarity scoring
  • Model validation — V3 model files are validated (MOC3 + textures existence) before being added to the list; invalid entries are skipped
  • Async model switching — V3 model files are loaded on a background thread via mpsc channel; the main thread drains the receiver without blocking rendering

Rendering (V3)

  • OpenGL 3.3 Core renderer with glow typed wrapper
  • 3 shader programs: standard (un-masked drawables), mask (FBO), masked (composite with per-pixel mask)
  • Cached uniform locations — queried once at shader init for ~30 fewer driver calls per frame
  • Multiply/screen blending support via uMultiplyColor / uScreenColor uniforms
  • Mask compositing — mask shapes rendered to offscreen FBO, then sampled in the masked fragment shader using gl_FragCoord / uMaskSize

Rendering (V2)

  • Separate GL 2.1-style rendering path via the C++ wrapper
  • Requires a VAO wrapper for core profile compatibility
  • V2 code resets GL state (VAO, program, textures, blend) after drawing because the V2 internals leave GL in an unknown state

Motion System

The motion system is self-implemented (not Cubism Framework), located in live2d-viewer/src/motion/:

  • Custom JSON parser — parses Cubism motion3.json files without the official framework
  • Curve evaluator — supports all interpolation types (linear, bezier, stepped)
  • Queue manager — per-group queues for concurrent independent motions (e.g. idle + expression)
  • Eye blink — automatic eyelid closure with configurable interval and duration
  • Breath — subtle breathing motion on chest/body parameters
  • Look-at-cursor — eye tracking toward mouse position with smooth interpolation
  • Expressions — loaded from expressions/*.exp3.json with fade transitions
  • Physics — physics3.json simulation with spring dynamics (hair/clothing/swing items)
  • Pose — auto-fade between tap motions via pose3.json groups
  • Tap detection — click on model body triggers a random "TapBody" motion (V3: from loaded motions; V2: cycles through internal C++ motions)
  • Audio — V2 motions can trigger sound playback via rodio (OGG/WAV/MP3)

Camera & Layout

  • Interactive pan & zoom — drag to pan, scroll or buttons to zoom
  • Reset — one-click restore default view
  • Layout mode — enter adjustment mode with pan X/Y and zoom sliders for precise positioning
  • Per-model layout persistence — pan position + zoom level saved to DB per model, restored on switch (V2 stores v2_scale; V3 computes from camera with proper Y-flip and aspect ratio)

Pet / Overlay Modes

Three desktop pet modes accessible from the GUI or system tray:

Mode Description
Off Normal windowed viewer
Windowed Pet Decorationless always-on-top window sized to model canvas; behaves like a desktop mascot
AlwaysOnTop Pet On Linux/Wayland: spawns a separate smithay-client-toolkit layer-shell surface (bypasses window manager). On X11: similar to Windowed but with floating circle minimize

Additional pet features:

  • Floating circle minimize — minimize to a small draggable overlay circle (saved as 50×50 on Wayland)
  • Pet toolbar — raw-GL overlay with buttons (prev/next model, zoom in/out, reset camera, search, exit pet)
  • Click-through mode — input passthrough so clicks reach windows underneath
  • Tray icon — system tray menu to toggle pet modes, toggle click-through, show/hide window, or quit

GUI (egui)

The viewer uses egui for all UI overlays:

  • Model List panel — select, rename, delete, add model directories
  • Parameters panel — all model parameters as sliders with live update
  • Search panel — full-text model search with similarity scores
  • Settings panel — configure scan directories, view scan results
  • Motion status — display current motion queue entries, expression state, fade weights
  • Action buttons — replay idle, stop all, tap body
  • Zoom controls — zoom in/out/reset buttons
  • Pet mode buttons — toggle Windowed Pet / AlwaysOnTop
  • Loading indicator — shown during async model switch
  • CJK font support — loads SourceHanSansCN-Medium.otf at startup; without it, Chinese/Japanese labels render as □□□

Other Features

  • System tray — cross-platform tray icon (GTK on Linux via ksni, tray-icon on other platforms) with full menu
  • Wayland handling — auto-detects Wayland, forces X11 backend for winit; on Wayland-only compositors, tray degrades gracefully; pet mode respawns the process with appropriate flags
  • Floating play button — raw GL triangle overlay avoids an egui_glow coordinate bug at small window sizes
  • TextRenderer — atlas-based monospace text renderer used in the pet overlay (raw GL, not egui)
  • Database — SQLite via libsql for model history and settings persistence

Architecture

┌─────────────────────────────────────────────┐
│              live2d-viewer                   │  Desktop viewer binary
│  ┌──────────┐ ┌──────────┐ ┌─────────────┐  │
│  │ Renderer │ │ GUI      │ │ Motion      │  │
│  │ (glow)   │ │ (egui)   │ │ System      │  │
│  │ V3/V2    │ │ Pet Tool │ │ Blink/Breath│  │
│  │ Shaders  │ │ bar      │ │ Look/Phys   │  │
│  └──────────┘ └──────────┘ └─────────────┘  │
│  ┌──────────┐ ┌──────────┐ ┌─────────────┐  │
│  │ Camera   │ │ Database │ │ Pet/Overlay │  │
│  │ (Zoom)   │ │ (SQLite) │ │ (Wayland)   │  │
│  └──────────┘ └──────────┘ └─────────────┘  │
├─────────────────────────────────────────────┤
│   live2d-core  (safe V3 wrapper)            │
│   Moc / Model / Parameters / Drawables       │
├─────────────────────────────────────────────┤
│   live2d-v2-core  (safe V2 wrapper)         │
├──────────────────┬──────────────────────────┤
│ live2d-core-sys  │ live2d-v2-core-sys       │
│ (bindgen V3 FFI) │ (bindgen V2 FFI)         │
├──────────────────┴──────────────────────────┤
│   Cubism Core 5.x (C static lib / .so)      │
│   live2d-py (V2 C++ wrapper, submodule)     │
└─────────────────────────────────────────────┘

Crate Details

live2d-core-sys (live2d_core_sys)

Auto-generated FFI declarations via bindgen. Generates at build time from CubismCore/include/Live2DCubismCore.h.

Key types: csmMoc, csmModel, csmVector2, etc.

live2d-core (live2d_core)

Safe zero-cost wrappers with lifetimes for V3 models:

let moc = Moc::from_bytes(&moc3_data)?;       // load MOC3 data
let mut model = Model::initialize(&moc)?;     // create model instance

let canvas = model.canvas_info();              // pixel size, origin
let params = model.parameters();              // parameter IDs, values, ranges
let parts = model.parts();                    // part opacity
let drawables = model.drawables();            // vertex/index data, textures, masks
let orders = model.render_orders();           // draw order

model.update();                                // run physics/motion

live2d-v2-core (live2d_v2_core)

Safe Rust wrapper for V2 models, wrapping the C++ API from live2d-py:

  • Model loading via Model::from_file() / Model::from_bytes()
  • Parameter get/set, scale and offset control
  • Motion triggering and sound path lookup

live2d-viewer (live2d_viewer)

Full desktop application. See Viewer Features above.

Shaders (V3)

Viewer uses 3 GLSL 330 Core shaders:

Program Uniforms Purpose
program uTexture, uMultiplyColor, uScreenColor, uOpacity Standard un-masked drawables
mask_program uScale, uTranslate, uOpacity Render mask shapes to FBO
masked_program +uMaskTexture, uMaskSize Per-pixel mask compositing via gl_FragCoord

Mask rendering flow:

  1. Bind offscreen FBO, draw mask geometry → mask alpha in FBO texture
  2. Switch to masked program, bind FBO texture as uMaskTexture
  3. Sample mask in fragment shader by computing UV from gl_FragCoord / uMaskSize
  4. Multiply fragment alpha by mask value

Linking

By default, the build uses the static-link feature (enabled via default = ["static-link"] in all crate Cargo.tomls):

  • Static (default): links libLive2DCubismCore.a from the SDK at build time
  • Dynamic (--no-default-features): links libLive2DCubismCore.so and adds -rpath to the binary via live2d-viewer/build.rs

The SDK is located either:

  1. At ../CubismSdkForNative-5-r.5 relative to the workspace root
  2. Via LIVE2D_SDK_ROOT environment variable overriding the path

Build Quirks

  • Release profile (in root Cargo.toml): lto=true, strip=true, panic="abort", codegen-units=1, overflow-checks=false. Always build with --release for actual use — debug is slow.
  • V2 core depends on C++ build artifacts from live2d-py. live2d-v2-core-sys/build.rs links v2_c_api, V2, and glad as static libs. On Linux it also links GL, stdc++fs, stdc++, m.
  • bindgen runs at build time for both -sys crates. Generated code lands in OUT_DIR/bindings.rs.

Wayland Caveat

main.rs detects WAYLAND_DISPLAY and forces WINIT_UNIX_BACKEND=x11 + GDK_BACKEND=x11. The tray icon uses GTK (ksni crate), which also needs X11. On Wayland-only compositors, GTK init may fail → tray icon is disabled (graceful fallback).

Minimize-to-floating-circle uses a different code path on X11 (hide) vs Wayland (resize to 50×50 float overlay). AlwaysOnTop pet mode on Wayland spawns a separate smithay-client-toolkit layer-shell thread instead of relying on winit.

Testing

All tests are integration-style and require the Cubism SDK:

LIVE2D_SDK_ROOT=/path/to/CubismSdkForNative-5-r.5 cargo test --release -p live2d-core-sys
  • live2d-core-sys/src/lib.rs: unit tests for csmGetVersion(), csmGetLatestMocVersion(), log function roundtrip
  • live2d-viewer/src/model_loader.rs: model dump tests that load real SDK sample models (Mao, Rice, Natori) — these are debug/exploration tests that print model structure but don't assert much

Lint & Format

cargo fmt
cargo clippy
cargo check --release

Order: fmt → clippy → test. No custom rustfmt.toml or clippy.toml. Some #[allow(clippy::...)] annotations exist on specific items.

Missing Features / TODO

Status Feature
✅ AI chat companion — configurable provider (OpenAI-compatible), per-model session chat, chat panel UI, settings panel UI
(✅) HitAreas + tap-to-motion — hit-testing from model3.json HitAreas (V3) and C++ hitTest (V2); tap triggers body motions. Done.
✅ UserData (userdata3.json) — parsed into ID→description lookup map; shown in GUI on tap
✅ Save/Load parameter presets — named snapshots stored in DB per model; restore via GUI
✅ Layout settings — per-model pan + zoom saved to DB; layout mode with sliders for precise positioning

License

This project is not affiliated with Live2D Inc.

  • live2d-core-sys contains FFI declarations for the Cubism Core SDK, used under Live2D's license terms.
  • All other code is provided under the MIT License.

About

A trial rust binding for live2d v3 or v2(sure)?

Resources

Readme
Activity

Stars

3 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages