Crawl4AI

Development1024MB+ RAM

Open-source, LLM-friendly web crawler and scraper — async, fast, and built for RAG pipelines

Download Server Compass View Documentation

developmentaicrawl4aiopen-sourceself-hosteddocker

Deploy Crawl4AI in 3 Steps

Connect Your VPS

Add your server credentials to Server Compass

Select Crawl4AI

Choose from our template library

Deploy & Configure

Fill in settings and click Deploy

No Docker knowledge required

After Deployment

After deploying Crawl4AI with Server Compass, complete these steps to finish setup

Open http://YOUR_SERVER_IP:{{PORT}}/docs to see the OpenAPI playground

Test a crawl against a public URL with curl or the playground

(Optional) Set CRAWL4AI_API_TOKEN to require Bearer auth on every request

Wire it into your RAG / LLM pipeline as a markdown-extraction backend

Need help? Check out our documentation for detailed guides.

Crawl4AI FAQ

Common questions about self-hosting Crawl4AI

How do I deploy Crawl4AI with Server Compass?

Simply download Server Compass, connect to your VPS, and select Crawl4AI from the templates list. Fill in the required configuration and click Deploy. The entire process takes under 3 minutes.

What are the system requirements for Crawl4AI?

Crawl4AI requires a minimum of 1024MB RAM. We recommend a VPS with at least 2048MB RAM for optimal performance. Any modern Linux server with Docker support will work.

Can I migrate my existing Crawl4AI data?

Yes! Server Compass provides volume mapping that allows you to import existing data. You can also use standard Crawl4AI backup and restore procedures.

How do I update Crawl4AI to the latest version?

Server Compass makes updates easy. Simply click the Update button in your deployment dashboard, and the latest Crawl4AI image will be pulled and deployed with zero downtime.

Is Crawl4AI free to self-host?

Crawl4AI is open-source software. You only pay for your VPS hosting (typically $5-20/month) and optionally Server Compass ($29 one-time). No subscription fees or per-seat pricing.