NANAMI

A physically based voice for machines. NANAMI synthesizes speech and singing by simulating how a human voice is made — lungs, vocal folds, and a moving vocal tract — instead of generating audio with a neural network. It is driven by a small markup language (NML) designed so that language models can directly write what to say (text or IPA phonemes) and how (emotion, prosody, voice quality, melody).

cargo run --release -- say "Hello! <happy>I'm Nanami.</happy> My name is /ˈnɑnəmi/." -o hello.wav
cargo run --release -- say "<ja>こんにちは、ななみです。</ja>" --play
cargo run --release -- say '<sing tempo="90">twinkle@C4,C4 twinkle@G4,G4 little@A4,A4 star@G4:2</sing>' -o star.flac
cargo run --release -- midi song.mid --lyrics "la la la" -o song.wav
cargo run --release -- phonemes "I can't believe it's 2026!"

See docs/NML.md for the full markup reference (written to be pasted into an LLM's system prompt).

How it works

NML markup ─▶ G2P ─▶ prosody ─▶ gestural score ─▶ articulators ─▶ aerodynamics ─▶ glottis ─▶ vocal tract ─▶ audio
 (markup.rs)  (g2p/)  (prosody.rs) (score.rs)     (articulation.rs)  (synth.rs)    (glottis.rs)  (tract.rs)

Stage	Model	References
Text → phonemes	CMU Pronouncing Dictionary + letter-to-sound rules (English); kana/romaji rules with ん assimilation, っ gemination (Japanese)	CMUdict; Elovitz et al. 1976
Timing	Klatt's multiplicative duration rules (English); mora timing (Japanese); note-aligned onsets for singing	Klatt 1979
Intonation	Sparse tonal targets: H* accents with downstep, L-L% / H-H% / L-H% boundary tones; Japanese lexical pitch accent	Pierrehumbert 1980; Pierrehumbert & Beckman 1988
Emotion	Offsets in F0 level/range, rate, loudness, laryngeal tension, breathiness, creak, jitter, articulatory precision, smile	Murray & Arnott 1993; Burkhardt & Sendlmeier 2000; Scherer 2003
Gestures	Per-articulator tiers (tongue body, lips, tip, dorsum, root, velum, glottis) as critically damped second-order systems; vowels and consonants on separate tiers → coarticulation	Browman & Goldstein 1992; Saltzman & Munhall 1989
Vocal tract shape	Stevens & House parabolic tongue + asymmetric front channel, pharyngeal expansion and a smooth 8-point area correction; consonant constrictions overlaid; lingual consonants pull the tongue body toward place loci (CV transitions). Vowel and approximant shapes calibrated by acoustic-to-articulatory inversion, per anatomy (male / female sets)	Stevens & House 1955; Hillenbrand et al. 1995; Peterson & Barney 1952; Recasens 1999
Airflow	Quasi-steady Bernoulli flow through glottis and supraglottal constriction, oral pressure integrated through wall compliance → stop bursts, devoicing in closures, aspiration emerge physically	Stevens 1998; Maeda 1982; Birkholz 2007
Turbulence	Jet noise ∝ Δp^1.5·√A gated by Reynolds number, injected where the jet expands into the front cavity and at the teeth obstacle (sibilants); release transients ∝ dU/dt after a full closure	Stevens 1971, 1998; Flanagan & Cherry 1969; Shadle 1985
Voice source	Liljencrants–Fant glottal flow derivative via Fant's Rd parameter, with jitter, shimmer, flutter, creak, pulsatile aspiration	Fant et al. 1985; Fant 1995; Klatt & Klatt 1990
Acoustics	Kelly–Lochbaum waveguide at 96 kHz (one section = 3.6 mm) with area-dependent losses, yielding mass–spring walls, piriform-sinus side branch, glottal termination from the instantaneous Bernoulli glottal resistance (open-phase damping), nasal branch via 3-port junction at the velum, Flanagan piston radiation load	Kelly & Lochbaum 1962; Ishizaka et al. 1975; Flanagan 1972; Story 1995
Singing	Note-timed vowels with consonants placed before the beat, portamento, delayed vibrato, epilarynx narrowing for the singer's formant	Sundberg 1987

The only things "learned from data" are the vowel tract shapes, fitted by examples/calibrate.rs (Nelder–Mead over the articulatory model, using the same waveguide that synthesizes speech). Everything else is physics or published phonetic rules, so every parameter is interpretable and editable.

I/O uses the oxideav crates: WAV (oxideav-basic), FLAC (oxideav-flac), playback (oxideav-sysaudio), MIDI input (oxideav-midi), spectrogram PNGs (oxideav-png).

Voices

nanami (default; young, light, slightly breathy, 14.6 cm tract), female, male (17.5 cm), child (12 cm). A voice is anatomy (tract/nasal length, epilarynx) plus habitual source settings — see src/voice.rs.

Development

cargo test                                # unit tests (LF model, IPA, markup, G2P)
cargo run --release --example calibrate > src/vowel_table.rs   # refit vowels
cargo run --release --example diag -- "/ˈasa ˈaʃa ˈapa/"       # per-segment level/centroid/aero trace
cargo run --release --example f0stats    # F0/duration per emotion
cargo run --release -- say "..." -s spec.png   # spectrogram for visual checks

Measured checks (nanami voice, diag): fricative levels relative to vowels /s/ −15 dB, /ʃ/ −16 dB, /f/ −30 dB, /θ/ −30 dB, /h/ −21 dB; /s/ centroid ≈ 9 kHz, /ʃ/ ≈ 5 kHz; voiceless stops show burst + aspiration, voiced stops keep voicing through closure. Vowel formants are within a few percent of Hillenbrand et al. (1995) means (examples/vowel_check).

Objective listening proxies

Since a synthesizer's author can't always listen, tools/eval/bench.sh renders 40 Harvard sentences and scores them with Whisper (word error rate) and UTMOS (a MOS predictor trained on listener ratings of synthetic speech; natural speech ≈ 4+):

	WER small	WER medium	UTMOS
v0.1 (first release)	84%	—	—
current, nanami	37%	37%	2.42
natural female (CMU ARCTIC slt)	—	—	4.29
natural speech (LibriSpeech)	—	—	3.78

These are proxies, not a substitute for human listening.

Status & known limitations

This is an early research prototype (v0.1):

• Onset clusters (/st str br/) and some stop places are still often misheard.
• /l/ is modeled with a 1-D tube (no true lateral side channels); /ɹ/ uses three constrictions near F3 velocity maxima.
• English intonation is rule-based (no syntax/focus analysis beyond function words and <emph>); Japanese defaults to heiban accent unless ꜜ is given.
• Kanji are not converted; supply kana or IPA.
• A voice change (<voice>) starts a new rendering run (no cross-fade).

Layout

src/
  main.rs          CLI
  lib.rs           render_markup() entry point
  markup.rs        NML parser
  g2p/             English (CMUdict + rules, numbers) and Japanese G2P
  ipa.rs           IPA → segments
  phoneme.rs       inventory, features, articulatory gestures
  utterance.rs     words/phrases/songs, styles, emotion presets
  prosody.rs       durations, intonation, breathing, singing timeline
  score.rs         gestural score → articulator trajectories
  articulation.rs  articulators → area function
  synth.rs         aerodynamics, turbulence, engine
  glottis.rs       LF voice source
  tract.rs         waveguide vocal + nasal tract
  vowel_table.rs   calibrated vowel shapes (generated)
  midi.rs          MIDI → NML
  audio_io.rs      WAV/FLAC/playback/spectrogram via oxideav
data/cmudict.dict  CMU Pronouncing Dictionary (see data/CMUDICT_LICENSE)