crw

by us · Rust · ★ 223

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

#ai#ai-agents#crawler#data-extraction#docker#firecrawl#firecrawl-alternative#html-to-markdown#llm#markdown#mcp#mcp-server#rust#scraping-api#self-hosted#tavily-alternative#web-crawler#web-scraper#web-scraping#web-search-api

Install

cargo install --git https://github.com/us/crw.git

Auto-generated from the repo's primary language — check the repo's own README for its exact install instructions.

Claude Desktop config

A starting point for your claude_desktop_config.json (auto-generated — the repo's README may specify a published package name or required env vars):

{
  "mcpServers": {
    "crw": {
      "command": "npx",
      "args": [
        "-y",
        "github:us/crw"
      ]
    }
  }
}

From the README

fastCRW Self-hosted, Rust-native web crawler & scraper for AI agents The open-source alternative to Firecrawl. One static binary, ~50 MB RAM idle, Firecrawl-compatible REST API on **both and ** (scrape, crawl, map, search, extract, plus v2 batch & parse) — a drop-in for the official Firecrawl SDKs — plus first-class MCP. Self-host free under AGPL-3.0, or hit our managed API at . Reproducible 63.74% truth-recall on the public 1,000-URL dataset (, 2026-05-08) — see [fastcrw.com/benchmarks](https://fastcrw.com/benchmarks). Built in Rust because every millisecond of agent latency compound…

Read full README on GitHub →

Related cloud & devops servers

TrendRadar

sansan0

★ 59,979

⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载，你的 AI 舆情监控助手与热点筛选工具！聚合多平台热点 + RSS 订阅，支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机，也支持接入 MCP 架构，赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ，数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。

Cloud & DevOps · Python

LibreChat

danny-avila

★ 39,915

Enhanced ChatGPT Clone: Features Agents, MCP, Skills, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active

Cloud & DevOps · TypeScript

PDFMathTranslate

★ 35,253

[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero

Cloud & DevOps · Python

casdoor

★ 13,842

An open-source Agent-first Identity and Access Management (IAM) /LLM MCP & agent gateway and auth server with web UI supporting OpenClaw, MCP, OAuth, OIDC, SAML, CAS, LDAP, SCIM, WebAuthn, TOTP, MFA, Face ID, Google Workspace, Azure AD

Cloud & DevOps · Go

kubeshark

★ 11,975

eBPF-powered network observability for Kubernetes. Indexes L4/L7 traffic with full K8s context, decrypts TLS without keys. Queryable by AI agents via MCP and humans via dashboard.

Cloud & DevOps · Go

pal-mcp-server

BeehiveInnovations

★ 11,614

The power of Claude Code / GeminiCLI / CodexCLI + [Gemini / OpenAI / OpenRouter / Azure / Grok / Ollama / Custom Model / All Of The Above] working as one.

Cloud & DevOps · Python