Skip to content

api-reference.md

Latest commit

 

History

History
71 lines (49 loc) · 2.57 KB

api-reference.md

File metadata and controls

71 lines (49 loc) · 2.57 KB
Raw

API reference

Everything below is importable from the top-level foleydiff package.

Pipeline

TextToAudioPipeline(config=None, *, device=None)

High-level entry point.

  • generate(prompt, *, seconds=4.0, steps=None, guidance_scale=None, sampler="ddim", eta=0.0, seed=None) -> Tensor Generate audio for a string or list of prompts. Returns a 1-D tensor for a single string prompt, otherwise (batch, samples).
  • encode_audio(waveform) -> Tensor — encode a waveform into the diffusion latent.
  • num_parameters() -> int — trainable parameter count of the generative parts.
  • to(device) -> self, eval() -> self.
  • slerp(a, b, alpha) -> Tensor (staticmethod) — spherical latent interpolation.

Presets

  • get_preset(name) -> PipelineConfig — "tiny" (CI-sized) or "small" (default).
  • list_presets() -> tuple[str, ...].

Configuration

Plain dataclasses, all validating in __post_init__:

  • MelConfig, AutoencoderConfig, UNetConfig, DiffusionConfig, TextEncoderConfig, PipelineConfig.

PipelineConfig.to_dict() serialises the whole tree.

Core components

GaussianDiffusion(config=None)

  • q_sample(x_start, t, noise=None) — forward noising.
  • to_x0_eps(model_output, x_t, t) -> (x0, eps) — parametrisation-agnostic.
  • get_velocity, predict_start_from_noise, predict_start_from_v.
  • loss(model, x_start, t, *, context=None, noise=None) — MSE denoising loss.

UNet1D(config=None)

forward(x, timesteps, context=None) -> Tensor. Conditional 1-D denoiser.

AudioAutoencoder(config=None)

  • encode(mel) -> DiagonalGaussian
  • decode(z, length=None) -> Tensor
  • forward(mel) -> (reconstruction, posterior)
  • latent_length(n_frames) -> int

HashingTextEncoder(config=None)

encode(prompts) -> Tensor of shape (B, L, embed_dim); tokenize(text) -> list[int].

Audio

  • MelSpectrogram(config=None) — forward(waveform), compress/decompress staticmethods.
  • GriffinLimVocoder(config=None, n_iter=32, momentum=0.99) — forward(mel, length=None, generator=None).
  • griffin_lim(...), mel_filterbank(config).
  • load_wav(path) -> (Tensor, int), save_wav(path, waveform, sample_rate).

Samplers

  • DDPMSampler(), DDIMSampler(eta=0.0).
  • get_sampler(name, **kwargs), available_samplers().
  • Custom samplers: subclass Sampler and decorate with @register_sampler("name").

Schedules

  • make_beta_schedule(name, num_timesteps, beta_start=0.00085, beta_end=0.012).
  • available_schedules() — linear, scaled_linear, cosine, sigmoid.

Utilities

  • seed_everything(seed).