Doppel

Ask your documents a question. Get the answer back in your own cloned voice.

Upload a PDF, ask about it by voice or text, and Doppel retrieves the source, answers with an LLM, and speaks the response back in a voice cloned from a short reference recording, entirely on infrastructure you control.

No hosted doc-chat tool does both halves of that: NotebookLM and Perplexity answer in a generic voice and run on someone else's servers. Doppel does neither.

What it does

Chat mode: Upload documents (PDF, TXT). Ask questions by voice or text. Doppel retrieves the full source document, generates an answer via LLM, and speaks it back. Source attribution shows exactly which documents grounded each response.

Voice cloning: Clone a voice from a short reference recording. The assistant speaks as you, not as a generic AI voice.

Meeting mode: Record a meeting or conversation. Doppel transcribes the audio and lets you query the transcript alongside your knowledge base. Ask "what did we agree on regarding the timeline?" and get an answer grounded in both the meeting and your uploaded docs.

LLM orchestration: Falls back across Groq, Gemini, and OpenAI. Swap providers or models without changing anything else.

Full document context: When a document is relevant, the entire text is passed to the LLM (stored in PostgreSQL), not just retrieval chunks. This means answers stay grounded in the complete source, not lossy fragments.

Also useful for

Doppel is a foundation, not a finished product for these, but the pieces are there:

• Internal knowledge base: point it at your team wiki, PRDs, and onboarding docs; new hires ask questions and get sourced answers.
• Voice interface for docs: wire the chat API into a Slack bot or support widget so people talk to your documentation instead of filing a ticket.
• Workflow automation: the API is standard REST, so any system that can make an HTTP call (CI/CD, a deploy hook) can trigger a spoken summary.

Architecture

User speech or text
  -> FastAPI assistant endpoint
  -> Whisper transcription (when audio)
  -> FAISS similarity search over uploaded documents
  -> Full document retrieval from PostgreSQL
  -> LLM response generation (Groq / Gemini / OpenAI)
  -> Qwen TTS synthesis (built-in or cloned voice)
  -> Streamed text + audio back to client (SSE)

Key files:

File	Purpose
api_assistant.py	Assistant router: chat streaming, RAG, meeting support
api_combined.py	Core TTS + ASR service
vector_db.py	Document ingestion, FAISS search, PostgreSQL full-text storage
llm_orchestrator.py	Multi-provider LLM fallback
frontend/src/App.tsx	React frontend with Chat and Meeting modes

Running It

Prerequisites

• Python 3.10+
• Node.js 20+ (frontend)
• PostgreSQL (for full document text storage)
• CUDA GPU recommended, CPU fallback available

Backend

pip install -r requirements.txt

Model weights are not committed. Download instructions:

• models/Qwen3-TTS-12Hz-1.7B-CustomVoice/MODEL_DOWNLOAD.md
• models/Qwen3-TTS-12Hz-1.7B-Base/MODEL_DOWNLOAD.md
• models/whisper-medium/MODEL_DOWNLOAD.md

Start the service:

python api_assistant.py

API docs at http://localhost:8002/docs.

Frontend

cd frontend
npm install
npm run dev

Environment

# Required: at least one LLM provider
GROQ_API_KEY=your_key
GEMINI_API_KEY=your_key
OPENAI_API_KEY=your_key

# PostgreSQL for full document storage
DATABASE_URL=postgres://postgres:postgres@localhost:5432/agents

# Optional
TTS_PORT=8002
ASR_MODEL_SIZE=medium
CLONE_MODEL_LAZY_LOAD=true
CLONE_MODEL_DEVICE=cpu
ALLOW_DANGEROUS_DESERIALIZATION=true
CORS_ALLOW_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
ASSISTANT_DEFAULT_SPEAKER=Vivian
ASSISTANT_API_TOKEN=

Voice Cloning

cd voice_cloning
python voice_clone.py

Saves a voice profile to custom_voices/ that the API picks up automatically. See voice_cloning/README.md.

Security Defaults

• Assistant and document endpoints are localhost-only unless ASSISTANT_API_TOKEN is set.
• Existing FAISS indexes are not loaded unless ALLOW_DANGEROUS_DESERIALIZATION=true.
• CORS defaults to common local development origins instead of all origins.

What This Is Not

This is a working prototype, not a production SaaS product. It demonstrates the full pipeline end-to-end and provides a foundation for building on top of. It has not been load-tested, security-hardened, or optimized for multi-tenant deployment.

API Reference

• Full API docs: docs/TTS_API.md
• Postman collection: docs/Qwen_TTS_ASR.postman_collection.json
• Startup helper: start-tts-service.ps1
• Environment verification: verify_imports.py

License

MIT