MCPs βΊ Data & APIs βΊ Crawlie
**Technical SEO + AI-search audits for agents.** Crawl any website and get a structured report your agent can actually work with β broken links, redirect chains, missing titles/canonicals, duplicate content, structured data, and GEO (generative engine optimization) readiness: is your site answerable by ChatGPT, Claude, and Perplexity? ### What your agent can do - **Audit** a single URL, a list, or crawl an entire site - **Slice saved reports** without re-crawling β top fixes by impact, affected URLs per issue, link graph, GEO gaps - **Explain any issue** β why it matters and how to fix it, with agent-actionable output - **Monitor continuously** β create a project, set a crawl schedule, diff reports to catch regressions after deploys - **Gate releases** β run a pre-launch check and fail on broken links, 5xx, or noindex slips ### Try asking - "Crawl mysite.com and give me the top 10 fixes by impact" - "What changed since the last crawl of my docs site?" - "Is my site ready to be cited by AI search engines?" Sign in with OAuth β no API key to paste. Free to start. Also available as an open-source desktop crawler at [crawlie.co](https://www.crawlie.co).
Not monetized yet
Turn Crawlieβs tool calls into revenue: one disclosed sponsored slot, 70% revenue share, fail-open by design.
Install Crawlie
For anyone using Crawlie β no Lulu account neededclaude mcp add --transport http crawlie https://crawlie.run.tools
Real-time weather for any city, built on the lulu-ads widget gallery. First-party, free forever.
View server βFAQ
Run: claude mcp add --transport http crawlie https://crawlie.run.tools
Yes β itβs a remote MCP server, so no local install is needed.
ChatGPT: Settings β Connectors β Advanced β Developer mode β Add connector, then paste https://crawlie.run.tools.
Claude.ai: Settings β Connectors β Add custom connector, then paste the same URL.
Unrated out of 100, computed from cross-registry traction signals (installs, stars, registry presence) β never influenced by sponsorship.
Similar servers
Works well together