Svipall is a local web layer for AI agents. One Rust binary acts as an MCP server, a command
line tool and a small REST server. It runs on your own machine: there is no scraping API
behind it, no account to create, no API key to paste, no paid captcha service and no
telemetry. Web requests go to the sites you visit, and results go to the agent or client you
connect.
It can:
- Read a page and return clean Markdown, with boilerplate removed and an optional query that
keeps only the relevant passages, so a long page does not arrive as 300,000 tokens of raw
HTML.
- Turn listings and tables into typed rows and write them straight to CSV or JSON, or capture
the JSON the page itself requests while loading, which is usually the site's own API.
- Crawl a site within page, token and traffic limits, with a resumable frontier, near
duplicate labels and llms.txt output, for RAG and dataset work.
- Search the web without an API key across DuckDuckGo, Bing and Brave, and drive a site's own
search box when it has one.
- Let an agent click, type, scroll and wait on a real page through human-like input, keep a
signed-in profile, and watch a page or a single CSS region for changes.
- Read PDF, docx, xlsx, pptx, odt, epub, rtf and csv as Markdown, from the web or from a
local file.
- Route a request automatically from plain HTTP to a browser to a stealth browser, and
attempt supported anti-bot challenges and captchas locally: 15 widget families, 11 answer
modalities, optional local ONNX vision models, and a phone-friendly dashboard where a
person answers what the machine could not.
A "checking your browser" screen comes back labelled as a block, with the reason, instead of
reaching the agent as the article it was hiding. Quality labels annotate pages and never drop
them.
Install with one line on Windows, macOS and Linux, through a shell script, Homebrew, Scoop,
Docker, npx or cargo, then wire it into Claude Code, Claude Desktop, Cursor or any other MCP
client. Any language can call the 19 local REST routes instead.
AGPL-3.0. The benchmarks and their raw run logs are published, including the runs that
improved nothing.