Roadmap
The Crawlingo development roadmap — what's shipped, what's in progress, and what's planned.
4 min read Updated July 2026
This roadmap reflects current development priorities. Have a feature request? Open an issue on GitHub.
Shipped
13✓Rust core engine
HTTP/2 fetching, HTML parsing, selector engine
✓Python SDK (PyO3)
Page, Session, Dataset, Crawl, Watch
✓Node.js SDK (napi-rs)
Full TypeScript support, auto .d.ts
✓Rust native crate
Direct Rust API with tokio async
✓Self-healing auto-match
DOM fingerprinting + Jaro-Winkler scoring
✓Stealth TLS
JA3 fingerprint rotation, browser profiles
✓Proxy rotation
Static pool, round-robin, remote provider
✓Dataset streaming
Constant-memory URL list processing
✓Watch monitors
Typed callbacks, webhook support
✓Export: JSON/CSV/Parquet
One-method structured export
✓TOML + env var config
crawlingo.toml + CRAWLINGO_* vars
✓Middleware stack
Metrics, Cache, Auth (Basic/Bearer/OAuth2)
✓Pre-built wheels
Linux, macOS, Windows binaries
In Progress
4MCP Server
highModel Context Protocol server for AI agent integration
Go SDK
mediumCGo bindings to Rust core for Go developers
Documentation website
highFull docs site at crawlingo-docs.vercel.app
GitHub Actions CI
highAutomated wheel builds and publish pipeline
Planned
10Browser rendering mode
highHeadless Chromium integration for JS-heavy sites
Async Python API
highasyncio-native interface for Python SDK
Distributed crawl
mediumRedis-backed frontier for multi-machine crawls
Visual selector builder
mediumBrowser extension to build selectors by clicking
LLM-assisted extraction
highNatural language field descriptions for schema-less extraction
WASM target
lowCompile Crawlingo core to WebAssembly for browser use
Java / JVM SDK
lowJNI bindings for Java and Kotlin
Dashboard UI
mediumWeb UI for monitoring crawl jobs and watch alerts
Cloud function templates
mediumAWS Lambda, Vercel, Cloudflare Worker starters
Anti-CAPTCHA integration
highPlug-in support for 2captcha, CapSolver