The short answer
Firecrawl offers a hosted web data API and an AGPL-3.0 core. Crawl4AI combines an Apache-2.0 Python crawler with a hosted API. ScrapeGraphAI offers a managed extraction API and a separate MIT-licensed Python library. Choose by who runs the infrastructure, how you define structured output and what each operation costs. SOTA’s assessment is documentation-based.
| Decision | Firecrawl | Crawl4AI | ScrapeGraphAI |
|---|---|---|---|
| Hosting and licence | Hosted API or self-hosted AGPL-3.0 core; SDKs have separate MIT licences | Hosted API or self-hosted Apache-2.0 Python library and Docker server | Hosted API or separate self-hosted MIT Python library |
| Hosted free allowance | 1,000 credits every month; no card required | 1,000 credits every month after email verification; no card required | 500 one-time API credits |
| Entry paid plan, billed monthly | Hobby: $19/month, 5,000 credits/month | Starter: $10/month, 10,000 credits/month | Starter: $20/month, 10,000 credits/month |
| Hosted limits: Free → entry paid | Scrape: 10 → 100 requests/min; crawl: 2 → 20 requests/min; 2 → 5 concurrent browsers | 10 → 120 requests/min; 2 → 10 concurrent requests | 10 → 100 requests/min; 1 → 3 concurrent crawls |
| Basic page and extraction credits | Basic scrape/crawl: 1/page; LLM JSON format adds 4/page | Page: 0.5; archive: 0.25; browser: 1; hard site: 2; extraction adds token charges | Markdown: 1; Extract: 5; crawl: 2 startup plus per-page formats; stealth adds 5/page |
| Main interfaces | /scrape, /crawl, /map; JSON schema extraction; /agent succeeds /extract | Python arun/arun_many; cloud /scrape, /extract, /site and batch/jobs | Hosted /api/scrape, /api/extract, /api/crawl; library SmartScraperGraph |
| Latest PyPI package snapshot, checked November 7 | firecrawl-py SDK 4.55.0 — November 4, 2026 | crawl4ai library 0.9.5 — October 14, 2026 | scrapegraphai library 2.3.1 — October 6, 2026 |
Separate the hosted service from the code you run
Firecrawl can run through Docker Compose under its AGPL-3.0 core licence; its SDKs use separate MIT licences. The self-hosting guide requires Docker Compose v2 and says the baseline does not include every Cloud capability. LLM-backed extraction needs a configured provider such as an OpenAI-compatible endpoint or Ollama.
Crawl4AI provides an Apache-2.0 Python library, a Docker server and a managed cloud API. Its library supports CSS/XPath extraction without an LLM and LLMExtractionStrategy for schema-based JSON. ScrapeGraphAI’s MIT library builds graph-based extraction pipelines, including SmartScraperGraph for a prompt and a source URL. You supply the library’s model and operate browsers, proxies and scaling; its managed API handles those separately.
Budget for operations, not just pages
The table uses USD monthly billing. Firecrawl’s Hobby also lists $16 per month billed annually. A basic Firecrawl scrape or crawl costs 1 credit per page; LLM JSON extraction adds 4. Hobby offers 1,000 additional credits per $5. Crawl4AI Starter lists extra credits at $1 per 1,000; its page cost depends on the fetch route, and Extract bills the fetch plus model tokens. Crawl4AI labels its hosted pricing a soft launch, so recheck before budgeting.
ScrapeGraphAI’s Free credits are one-time, while Starter includes 10,000 each month. Extract costs 5 credits, with 5 more for stealth. A crawl costs 2 credits to start plus each page’s selected formats; multiple format costs are summed. Markdown costs 1 credit per page and JSON costs 5. Render mode does not change the credit cost. These hosted allowances do not cover infrastructure or model usage for the separate self-hosted libraries.
Match output formats, SDKs and MCP to the workflow
Firecrawl returns markdown, HTML or structured JSON. Its JSON mode accepts a prompt or JSON schema; its extractor guide directs /extract users to /agent. JSON extraction sees the markdown conversion, so HTML attributes are unavailable to the LLM. Crawl4AI’s cloud /scrape returns markdown, /extract accepts an instruction or schema for typed JSON, and /site starts a job that follows links within a site. Batch and bulk jobs handle explicit URL lists.
ScrapeGraphAI’s hosted Scrape supports markdown, HTML and JSON, while Extract uses an LLM with a prompt and optional schema on a URL, HTML or markdown. Crawl runs asynchronously with polling or webhooks. Its official SDKs cover Python and JavaScript/TypeScript. Firecrawl documents Python, Node.js/TypeScript, Go, Java, Ruby, Rust, .NET, PHP and Elixir SDKs. Crawl4AI’s library is Python; cloud examples show HTTP calls from Python and JavaScript. All three document official hosted MCP servers.
Pin the library or SDK you actually deploy
The release row identifies the latest PyPI client or library packages checked November 7, 2026: firecrawl-py 4.55.0, crawl4ai 0.9.5 and scrapegraphai 2.3.1. A client package version does not identify a hosted API or server version. Crawl4AI requires Python 3.10 or later. The ScrapeGraphAI library declares Python >=3.12,<4.0 and documents installing Playwright browsers; its hosted SDK guide requires Python 3.12 or later or Node.js 22 or later. Firecrawl SDK manifests declare Python >=3.8 and Node.js >=22.0.0. Pin the component you actually install and keep its runtime requirements separate from server deployment.
Which should you choose?
SOTA’s documentation-based assessment: consider Firecrawl for a managed scrape/crawl API with a broad SDK selection and JSON extraction on known URLs. Consider Crawl4AI when you want a Python crawler with both selector-based and LLM extraction, with a hosted route available. Consider ScrapeGraphAI for prompt-and-schema extraction from URLs or supplied content, or its MIT graph library when you want to operate the pipeline yourself. Check licence terms, required page fields and the full operation cost against your workload. No hands-on testing or performance ranking is claimed.
A reproducible evaluation plan
SOTA’s proposed evaluation, not benchmark results: freeze a manifest of 30 permitted URLs with 10 static articles, 10 JavaScript-rendered pages and 10 paginated listings. Save timestamped reference content and manually label expected fields. Run separate markdown and structured-extraction tasks using the same JSON schema, prompt and null-for-missing rule. Record the exact endpoint, package version, model where configurable, rendering, proxy and cache settings. Pin those settings for every repeat.
Process the same manifest three times at one request at a time, with a 120-second client timeout and at most one retry after a failure; log retries separately. For asynchronous crawls, apply the same deadline to job completion and set the same page cap. Save raw responses, elapsed time, errors, billed credits and self-hosted model/infrastructure costs. Score schema validity, field correctness, missing and unsupported values, markdown boilerplate and completeness against the labels. Compare total cost per correctly extracted record; publish the manifest and configuration with any results. This plan has not been executed.
Sources & method
Official documentation was checked on 2026-11-07. Selection advice is our assessment. Interfaces and requirements may change; check the linked source for the version you intend to use.
- Firecrawl hosted web data API
- Firecrawl plans, allowances and credit costs
- Firecrawl root AGPL-3.0 licence
- Firecrawl repository and SDK licence distinction
- Firecrawl Docker Compose self-hosting
- Firecrawl endpoint rate and concurrency limits
- Firecrawl scrape formats
- Firecrawl recursive crawling
- Firecrawl LLM extraction and HTML attribute limitation
- Firecrawl scrape JSON mode, extract and agent
- Firecrawl official SDK languages
- Firecrawl hosted and local MCP setup
- Firecrawl Python SDK runtime metadata
- Firecrawl Node SDK runtime metadata
- Firecrawl repository tags
- Crawl4AI Cloud, free allowance and pricing
- Crawl4AI library and hosted API documentation
- Crawl4AI Cloud endpoints, extraction billing and MCP
- Crawl4AI library, extraction strategies and Docker deployment
- Crawl4AI Apache-2.0 licence
- Crawl4AI v0.9.5 Python requirement
- ScrapeGraphAI hosted web extraction service
- ScrapeGraphAI plans, limits and credit costs
- ScrapeGraphAI formats, services and official SDKs
- ScrapeGraphAI self-hosted library versus managed API
- ScrapeGraphAI library MIT licence
- ScrapeGraphAI LLM extraction inputs and billing
- ScrapeGraphAI asynchronous crawling and format billing
- ScrapeGraphAI hosted SDK runtime requirements
- ScrapeGraphAI official hosted MCP server
- ScrapeGraphAI v2.3.1 library Python requirement
- firecrawl-py 4.55.0 on PyPI
- crawl4ai 0.9.5 on PyPI
- scrapegraphai 2.3.1 on PyPI
How SOTA reviews projects · Suggest a correction · Download the evaluation checklist
