|
3 | 3 | <!-- prettier-ignore --> |
4 | 4 | [)](https://pypi.org/project/context.dev/) |
5 | 5 |
|
6 | | -The Context Dev Python library provides convenient access to the Context Dev REST API from any Python 3.9+ |
7 | | -application. The library includes type definitions for all request params and response fields, |
8 | | -and offers both synchronous and asynchronous clients powered by [httpx](https://github.com/encode/httpx). |
| 6 | +Context.dev is a web scraping API for AI agents and LLMs. This SDK turns any URL into clean, LLM-ready markdown, crawls whole sites, searches the web, takes screenshots and extracts structured JSON against a schema you define, all with one API key. Proxies, JavaScript rendering and anti-bot handling run on Context.dev's side, so there is no headless browser to host. |
9 | 7 |
|
10 | 8 | It is generated with [Stainless](https://www.stainless.com/). |
11 | 9 |
|
@@ -39,6 +37,49 @@ brand = client.brand.retrieve( |
39 | 37 | print(brand.request_id) |
40 | 38 | ``` |
41 | 39 |
|
| 40 | +### Extract structured JSON |
| 41 | + |
| 42 | +```python |
| 43 | +from context.dev import ContextDev |
| 44 | + |
| 45 | +client = ContextDev() |
| 46 | + |
| 47 | +page = client.web.scrape( |
| 48 | + url="https://example.com", |
| 49 | + formats={"json": True}, |
| 50 | + json_params={ |
| 51 | + "schema": { |
| 52 | + "type": "object", |
| 53 | + "properties": { |
| 54 | + "title": {"type": ["string", "null"]}, |
| 55 | + "description": {"type": ["string", "null"]}, |
| 56 | + }, |
| 57 | + "required": ["title", "description"], |
| 58 | + "additionalProperties": False, |
| 59 | + }, |
| 60 | + }, |
| 61 | +) |
| 62 | +print(page.json_.data) |
| 63 | +``` |
| 64 | + |
| 65 | +## What you can do |
| 66 | + |
| 67 | +| Task | Method | |
| 68 | +| --- | --- | |
| 69 | +| Scrape a URL to markdown, HTML, JSON or a screenshot | `client.web.scrape` | |
| 70 | +| Crawl a site and get every page as markdown | `client.web.web_crawl_md` | |
| 71 | +| Map every URL on a domain | `client.web.map_urls` | |
| 72 | +| Search the web | `client.web.search` | |
| 73 | +| Take a screenshot of a page | `client.web.screenshot` | |
| 74 | +| Parse PDFs and documents | `client.parse.handle` | |
| 75 | +| Run thousands of URLs as a batch | `client.batch.submit` | |
| 76 | +| Watch a page for changes | `client.monitors.create` | |
| 77 | +| Look up a company's logo, colors and brand data | `client.brand.retrieve` | |
| 78 | + |
| 79 | +## Use it from an AI agent |
| 80 | + |
| 81 | +Context.dev also ships as a plugin for [Claude](https://github.com/context-dot-dev/claude-plugin), [Cursor](https://github.com/context-dot-dev/cursor-plugin) and [Gemini CLI](https://github.com/context-dot-dev/gemini-cli-context), and as tools for [LangChain](https://github.com/context-dot-dev/langchain-context) and [Haystack](https://github.com/context-dot-dev/context-haystack). |
| 82 | + |
42 | 83 | While you can provide an `api_key` keyword argument, |
43 | 84 | we recommend using [python-dotenv](https://pypi.org/project/python-dotenv/) |
44 | 85 | to add `CONTEXT_DEV_API_KEY="My API Key"` to your `.env` file |
|
0 commit comments