ShadowCrawl

ShadowCrawl (CortexScout) is a self-hostable MCP server that exposes advanced web search, extraction, and browser automation tools, specifically designed to serve AI assistants and autonomous agents. It enables token-efficient web data retrieval, robust anti-bot handling (with CDP and HITL fallback), and persistent stateful browser sessions. Integrating easily with multi-agent architectures, it replaces heavy E2E testing frameworks with a unified API surface and supports advanced workflows like deep research, structured field extraction, and multi-hop automations. Ideal for power users, AI developers, agent builders, and research teams needing reliable web data automation via any MCP-compatible client.