# pagesift

> The web and document layer for AI agents: any URL, PDF or Office file in, clean Markdown or JSON out. One self-hosted Rust binary with a CLI, Firecrawl v1/v2 and Jina Reader HTTP APIs, an MCP server, and Python and Node.js packages. AGPL-3.0 or commercial license.

- Website: https://pagesift.tech
- Version: 1.2.2
- Input: URLs (HTTP, or a browser when rendering adds content), HTML, PDF including scanned pages (OCR on CPU or NVIDIA GPU), DOCX, DOC, XLSX, XLS, XLSB, PPTX, PPT, OpenDocument, RTF, WordPerfect, HWP, EPUB, MOBI, FB2, EML, MBOX, MSG, ZIP, TAR, 7Z, images inside documents (OCR), SQLite, dBASE, CSV, Markdown and other text formats, 373 code languages as fenced blocks, WARC
- Output: Markdown, HTML, raw HTML, links, images, metadata, schema-based JSON; warnings when a result is incomplete
- Install: `cargo install pagesift --locked` · `pip install pagesift` · `npm install @pagesift/node` · `docker run -p 3002:3002 ghcr.io/yolow92/pagesift:1.2.2`
- First call: `pagesift scrape https://example.com` · `pagesift.scrape(url).markdown` · `(await new Scraper().scrape(url)).markdown` · `curl localhost:3002/r/https://example.com`
- MCP: `pagesift mcp` (stdio) or `pagesift mcp http --port 3003`; tools `pagesift_scrape`, `pagesift_batch_scrape`, `pagesift_crawl`, `pagesift_poll_job`, `pagesift_cancel_job`, `pagesift_map`, `pagesift_interact`, `pagesift_interact_stop`, and with a configured model `pagesift_agent`, `pagesift_agent_status`
- Not for: standalone image files (OCR covers scanned pages inside PDFs)

## Website

- [Documentation](https://pagesift.tech/docs/): architecture and how it works, in one page
- [Changelog](https://pagesift.tech/changelog/): release history with highlights per version
- [Articles](https://pagesift.tech/#articles): release posts and engineering deep dives on dev.to, Zenn and Qiita
- [简体中文](https://pagesift.tech/zh/): the landing page in Simplified Chinese
- [日本語](https://pagesift.tech/ja/): the landing page in Japanese
- [Español](https://pagesift.tech/es/): the landing page in Spanish

## Docs

- [Full text for LLMs](https://pagesift.tech/llms-full.txt): install, CLI, HTTP APIs, MCP tools, capabilities, inputs, outputs and measured performance in one file
- [README](https://github.com/YOLOW92/pagesift/blob/main/README.md): what it is, install, examples, capabilities, performance, architecture
- [File upload API](https://github.com/YOLOW92/pagesift/blob/main/docs/parse-api.md): `POST /v2/parse`, parsers and OCR modes
- [Structured extraction](https://github.com/YOLOW92/pagesift/blob/main/docs/structured-extract.md): schemas, selector memory and relocation
- [Agent API](https://github.com/YOLOW92/pagesift/blob/main/docs/agent-api.md): `POST /v2/agent`
- [Browser sessions](https://github.com/YOLOW92/pagesift/blob/main/docs/interact.md): `keepSession` and `/interact`
- [Monitoring](https://github.com/YOLOW92/pagesift/blob/main/docs/monitor.md): `/v2/monitor`
- [Jina Reader compatibility](https://github.com/YOLOW92/pagesift/blob/main/docs/jina-reader.md): `/r/` and `/s/` headers
- [Shared MCP service](https://github.com/YOLOW92/pagesift/blob/main/docs/mcp-service.md): one local MCP server for every agent on a machine
- [Architecture](https://github.com/YOLOW92/pagesift/blob/main/docs/architecture.md)
- [Benchmarks](https://github.com/YOLOW92/pagesift/blob/main/docs/benchmarks.md): methods, datasets and results against other tools
- [Example config](https://github.com/YOLOW92/pagesift/blob/main/docs/pagesift.example.toml)

## Optional

- [Release notes](https://github.com/YOLOW92/pagesift/blob/main/docs/releases/1.2.2.md): changes, migration and known limits
- [Changelog](https://github.com/YOLOW92/pagesift/blob/main/CHANGELOG.md)
- [Commercial license](https://github.com/YOLOW92/pagesift/blob/main/docs/COMMERCIAL.md)