cortex-scout

What does this tool actually do?

CortexScout is a tool that helps AI agents search the web, extract content, and control a browser automatically. It handles tricky things like bot detection and JavaScript-heavy pages, and can even ask a human for help when needed. It's designed to be efficient and work well with large language models.

Deep dive

CortexScout is a self-hostable Rust binary that provides search, extraction, and stateful browser automation capabilities over MCP (stdio) and an optional HTTP server. It is built to handle practical failure modes of web retrieval such as rate limits, bot challenges, and JavaScript-heavy pages through progressive fallbacks: native retrieval → Chromium CDP rendering → stateful E2E testing → Human-in-the-Loop (HITL) workflows. It includes a built-in stateful CDP automation engine designed specifically for AI agents, replacing heavy frameworks like Playwright or Cypress for E2E testing. Key features include a unified tool family for search, fetch, crawl, and extraction; anti-bot handling with proxy rotation and block-aware retries; HITL support for CAPTCHA or complex logins; memory-backed research history via LanceDB; and deep research capabilities for multi-hop search and synthesis. It integrates with the Cortex-Works ecosystem for multi-agent scaling and shared retrieval artifacts.

Additional use cases

• Building knowledge graphs by crawling and extracting content from multiple pages.
• Human-in-the-loop workflows for handling CAPTCHAs or complex authentication during web automation.

Security and Trust Notes

• Proxy rotation and anti-bot bypass techniques may be considered unethical if used for scraping without permission.