Graduate student working on sound and audio foundation models. I build small, readable, open-source tooling for the audio stack — general-purpose embeddings, sound-event detection, text-to-audio generation, and audio question-answering with LLMs — with a bias toward code that is reproducible and offline-by-default: minimal dependencies, no checkpoints to download, and pipelines that run end-to-end on a CPU.
The thread running through my projects is that you should be able to read the machinery. Most audio repositories are either thin FFT wrappers or research dumps wired to multi-gigabyte deep-learning stacks. I aim for the middle — batteries-included but lightweight enough to understand in an afternoon.
- General-purpose audio representations — embeddings for retrieval, clustering, and similarity.
- Sound-event detection — frame-level detection with honest, literature- aligned evaluation (segment- and event-based metrics, collars and all).
- Text-to-audio / foley generation — latent diffusion for sound effects.
- Audio question-answering with large language models — connecting audio representations to LLMs for reasoning over sound.
| Project | What it is | Stack |
|---|---|---|
| resona | Lightweight, NumPy-first toolkit for audio embeddings and sound-event detection: STFT / mel / MFCC / spectral features, three pluggable embedders, threshold + energy-gate detectors, and segment- and event-based metrics — one dependency, runs anywhere Python does. | Python · NumPy |
| foleydiff | Compact latent-diffusion toolkit for text-to-audio and sound-effect (foley) generation in plain PyTorch: a mel-spectrogram VAE, a conditional 1-D U-Net, DDPM/DDIM samplers with classifier-free guidance, and a Griffin-Lim vocoder — the whole stack from scratch, no pretrained weights required. | Python · PyTorch |
| harken | Small PyTorch framework that connects a frozen audio encoder to an LLM for audio question answering: pluggable encoders (CLAP / Whisper / AST), a projector zoo (linear / MLP / temporal-stack / Q-Former-style), LLaVA-style token splicing, and abstention-aware evaluation — with a download-free core that runs the full pipeline on CPU. | Python · PyTorch |
Each project is MIT-licensed, typed, linted, and tested across recent CPython versions, and ships synthetic data so the full pipeline runs with no external downloads.
Tying the three pieces together: better audio representations feeding both sound-event detection and audio question-answering, while keeping the constraints that shape everything above — minimal dependencies, reproducible runs, and components small enough to read.
📍 Seattle, WA · open-source, reproducible audio research tooling.