A powerful self-hosted web scraping solution
⚠️ Maintained fork. This is a community-maintained fork of the original Scraperr (archived Oct 2025). We keep it alive: refreshed dependencies, Docker builds, and bug fixes.
Scrape websites without writing a single line of code.
- XPath-Based Extraction: Precisely target page elements
- Queue Management: Submit and manage multiple scraping jobs
- Domain Spidering: Option to scrape all pages within the same domain
- Custom Headers: Add JSON headers to your scraping requests
- Media Downloads: Automatically download images, videos, and other media
- Results Visualization: View scraped data in a structured table format
- Data Export: Export your results in markdown and csv formats
- Notification Channels: Send completion notifications through various channels
- Backend: FastAPI (Python 3.10, PDM), SQLAlchemy, SQLite (default,
DATABASE_URLoverridable) - Frontend: Next.js 14 / TypeScript, TailwindCSS, Redux
- Scraping engines: requests-html, selenium-wire, Playwright, Camoufox
- Auth: JWT (email/password) + optional OpenAI (LLM) assistant
docker compose up -d- Frontend: http://localhost:3000
- API docs: http://localhost:8000/docs
The first build takes a while (the API image installs Playwright + Camoufox browsers). To use the pre-built images from Docker Hub instead, run:
docker compose -f docker-compose.hub.yml up -d| Variable | Default | Purpose |
|---|---|---|
NEXT_PUBLIC_API_URL |
http://scraperr_api:8000 |
API URL used by the Next.js server-side proxy |
SERVER_URL |
http://scraperr_api:8000 |
API URL used in server-side props |
DATABASE_URL |
sqlite+aiosqlite:///data/database.db |
SQLAlchemy connection string |
OPENAI_KEY |
(empty) | Enables the AI assistant feature |
DEFAULT_USER_EMAIL / DEFAULT_USER_PASSWORD |
(empty) | Pre-seeded admin user |
This fork is actively maintained. Need a custom scraper/parser for your site, a specific feature, or help with self-hosting?
- 🤖 Telegram: @medbot1_bot
Нужен кастомный парсер или доработка? Напиши нашему боту.
- 🗂 Portfolio: github.com/Ant19801108
- 🌐 Main site: Ant Mystik
❤️ Support this fork (hosting + maintenance are community-funded):
- ETH (Ethereum / ERC-20):
0x8AFC3Cc28fFC4cde92D13Bf6DAf6447f5227CF93 - USDT (TON):
UQAgrUTTUyDnKbC2gQsusk4Fk7n6Nu86Sx8X5rT14FqsrF64
When using Scraperr, please remember to:
- Respect
robots.txt: Always check a website'srobots.txtfile to verify which pages permit scraping - Terms of Service: Adhere to each website's Terms of Service regarding data extraction
- Rate Limiting: Implement reasonable delays between requests to avoid overloading servers
Disclaimer: Scraperr is intended for use only on websites that explicitly permit scraping. The creator accepts no responsibility for misuse of this tool.
This project is licensed under the MIT License. See the LICENSE file for details.
Development made easier with the webapp template. Contributions and bug reports are welcome.