Unlimited-OCR + Gemma 4 — RunPod Serverless (GitHub build)

A scale-to-zero endpoint: send a PDF/image + a JSON schema, get structured JSON back. RunPod builds this image from the repo; the models are baked into the image at build time, so there's no volume to manage and no per-worker downloads.

client --(base64 PDF + schema)--> RunPod /runsync --> worker --> JSON

File	Purpose
Dockerfile	RunPod builds this; downloads + bakes both models at build time
handler.py	RunPod handler — reuses server.py, loads models once per worker
server.py	the shared OCR + extraction logic
start_worker.sh	container entrypoint (starts Ollama, then the handler)
requirements.txt	Python deps
client_example.py	sample client (/runsync)
templates/	example prompt + schema pairs

Deploy (RunPod dashboard, one time)

1. Serverless → New Endpoint → Import Git Repository (connect GitHub, pick this repo).
2. RunPod detects the Dockerfile. Set:
   • GPU: H100 80GB (pool ADA_80_PRO).
   • Workers: min 0, max 1–3.
   • Idle timeout: 60–120s. FlashBoot: on.
   • (Optional) Env GEMMA_MODEL=gemma4:e4b for a smaller/faster extractor.
3. Deploy. The first build takes a while (it downloads ~15 GB of weights). Later pushes rebuild automatically.

Call it

export RUNPOD_API_KEY=...      # RunPod account key
export ENDPOINT_ID=...         # from the endpoint page
python client_example.py invoice.pdf templates/everrise_schema.json templates/everrise_prompt.txt

Raw HTTP:

curl -X POST "https://api.runpod.ai/v2/$ENDPOINT_ID/runsync" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" -H "Content-Type: application/json" \
  -d "{\"input\":{\"file_b64\":\"$(base64 -w0 invoice.pdf)\",\"filename\":\"invoice.pdf\",\"schema\":$(cat templates/everrise_schema.json)}}"

Job input fields: file_b64 (required), filename, task (extract|ocr), mode (gundam|base), dpi, prompt, schema (JSON object or "json"), include_text.

Notes

• Cold start (after idle): first request ~30–60s while the worker loads ~15 GB into VRAM; subsequent requests ~6–10s on H100. Idle timeout keeps a batch warm.
• Batches: use /run + poll /status/<id> instead of /runsync.
• Change the extractor: set GEMMA_MODEL=gemma4:e4b in the endpoint env (the image already bakes 12b; to bake e4b instead, edit the Dockerfile's GEMMA_MODEL).