QUICK ANSWER:
Firecrawl is a service that visits web pages, renders them properly and hands back clean, structured text an AI model can use, either one page at a time or across an entire site. It bills in credits, and the credit rate differs sharply between endpoints: a basic scrape is one credit per page, while asking it to return structured JSON costs five times that. Knowing which endpoint to reach for is the difference between a small bill and a surprising one.
Almost every genuinely useful AI project in a marketing business starts with the same unglamorous problem. The information you need is on a website, and websites are built for eyes rather than for models. Menus, cookie banners, navigation, footers, scripts that load the actual content half a second after the page arrives.
What Firecrawl does
Firecrawl renders the page the way a browser does before it extracts anything, so the JavaScript-heavy sites that defeat ordinary scrapers come back as readable text rather than an empty shell.
You give it a URL and it gives you back the content, stripped of the furniture, as clean markdown or structured data. That sounds simple until you have tried to do it yourself against a modern website.
The distinction that matters is rendering. A basic scraper fetches the raw HTML and takes what it finds, which on a site built with a modern framework is often close to nothing, because the content arrives later via JavaScript. Firecrawl runs an actual browser, waits for the page to settle, and then extracts. That single behaviour is why it works on sites that defeat the simple approach.
The five things it can do, and what each is for
Firecrawl is really five tools wearing one coat, and picking the right one is most of the skill.
Scrape
One page in, clean content out. This is the workhorse and the cheapest thing it does. Use it when you know exactly which page you want.
Crawl
Give it a starting URL and it follows links to pull an entire site or section. This is how you turn a client’s whole website into a knowledge base, or capture every product page a competitor publishes.
Map
Returns the list of URLs on a site without fetching the content. Cheap, fast, and the sensible first step before any crawl, because it tells you how big the job is before you commit to it.
Search
Runs a web search and returns results with the content already extracted, which saves stitching a search API and a scraper together yourself.
Extract and monitor
Extract pulls specific fields into a structured shape you define, rather than handing back prose. Monitor re-checks pages on a schedule and tells you what changed, which is the basis of any competitor or price watching workflow.
How the credits actually work
This is the section worth reading twice, because the rates are not uniform and nothing on the surface warns you.
| What you ask for | Credit cost | Relative to a basic scrape |
|---|---|---|
| Scrape, crawl or map | 1 credit per page | Baseline |
| Search | 2 credits per 10 results | Cheap per result |
| Advanced formats (JSON, question, highlight) | Adds 4 credits per page | 5x baseline |
| Interact (browser actions) | 2 credits per browser minute | Depends on the page |
| Monitor | 7 credits per page per check | 7x baseline, every check |
Three consequences follow, and all three are worth designing around.
First, asking for structured JSON costs five times a plain scrape. Often you do not need it. Pull the clean markdown at one credit and let a cheap model turn it into fields, which for most jobs works out considerably cheaper than paying for structured extraction on every page. A local model through Ollama does that for nothing at all.
Second, monitoring is the expensive endpoint and the easy one to leave running. Seven credits per page per check means a hundred pages checked daily is a serious monthly commitment. Check weekly instead of daily unless the change genuinely matters within a day, and monitor the twenty pages that move rather than the thousand that do not.
Third, map before you crawl. Map is cheap and tells you the size of the site. Crawling blind into a site with tens of thousands of URLs is how people discover their allowance in the least pleasant way.
One helpful detail: a request that fails and returns nothing is not charged. A page that returns an error status such as a 403 or a 404 still costs its credit, because the fetch happened.
The other limit that shapes what you can build
Credits get all the attention, but concurrency is what determines how long a job takes. The free tier runs two requests at a time with low rate limits; the upper tiers run a hundred or more, with rate limits orders of magnitude higher.
That difference is not about cost, it is about feasibility. Crawling a ten thousand page site two requests at a time takes a very long while. If your work involves whole-site jobs on a deadline, concurrency is the number to buy, and it is worth checking before you assume a lower tier will do.
Where it earns its place in a marketing business
Turning a client site into something an AI knows
Crawl the site once, and you have a knowledge base that can answer questions about the client’s own products, services and policies accurately. That is the foundation under every internal assistant and support bot that actually works, as opposed to the ones that invent answers.
Competitor and SERP intelligence at scale
Pulling the content of every page that currently ranks for a target term gives you a genuine picture of what the search engines are rewarding, rather than a guess. That feeds directly into the analysis behind our SEO work, and it pairs naturally with a content optimisation tool like Surfer SEO.
Keeping an eye on things that change
Price pages, stock levels, job ads, council planning notices, tender listings. Anything where knowing early is worth something. Use monitor sparingly and on the pages that matter, and it is the cheapest research assistant you will ever hire.
Feeding an agent
An agent that can read the web properly is dramatically more useful than one that cannot. This is the usual way that capability gets wired into a self-hosted agent such as OpenClaw.
What Australian businesses should know
Two things, and neither is complicated once you have decided them upfront.
The first is that scraping is not a free-for-all. Respect robots directives and site terms, keep request rates polite, and do not collect personal information you have no basis to hold. If a page contains names, contact details or anything else identifying, collecting it brings the Australian Privacy Principles into play, including the requirement to collect only what you actually need for a stated purpose. Scraping a directory of individuals because you can is a different act from crawling a client’s own product pages, and it is worth being clear internally about which you are doing.
The second is where the data goes next. Firecrawl processes offshore, so anything it returns has already crossed a border, and if you then feed it to a hosted model that is a second disclosure. For public marketing content none of that matters. For anything touching customer records, decide the boundary before you build the pipeline.
Pros and Cons
Pros
- Renders JavaScript-heavy pages that defeat simple scrapers.
- Clean markdown output that models read well with no preprocessing.
- Five endpoints covering single pages, whole sites, search and change monitoring.
- A genuinely usable free tier with no card required.
- Failed requests that return nothing are not charged.
Worth knowing before you start
- Structured JSON output costs five times a basic scrape, so use it deliberately.
- Monitoring bills every check, which adds up on a large page set.
- Concurrency on the lower tiers makes big crawls slow rather than impossible.
- Error-status pages still consume a credit.
- Scraping brings terms-of-use and privacy questions worth settling upfront.
The verdict
Firecrawl is the piece most businesses skip and then wish they had started with. If your AI work keeps stalling because the information lives on a website and arrives in an unusable state, this is the fix, and the free tier is enough to prove the case on a real job rather than a demo.
Use map first, scrape at one credit and post-process with a cheap model, and reserve monitoring for the handful of pages where a change genuinely matters. Do that and the running cost stays small while the capability it unlocks does not.
Build the data layer before the clever bit
Most AI projects that disappoint were built on inputs nobody prepared. Getting the data layer right first is unglamorous and it is what separates a working system from a demo. That sequencing is what our AI consulting engagements sort out early, and if you would rather your own team could build it, our AI training workshops cover exactly this ground.
There is a free tier with no card required, so you can test it against a real site this afternoon. You can find it at Firecrawl free.
Frequently asked questions
What is Firecrawl?
A service that fetches web pages, renders them the way a browser does and returns clean, structured content an AI model can use. It works on single pages, whole sites, search results and scheduled change checks.
How do Firecrawl credits work?
A basic scrape, crawl or map costs one credit per page. Search costs two credits per ten results. Advanced formats such as structured JSON add four credits per page, taking it to five times a basic scrape. Monitoring costs seven credits per page per check, and browser interaction bills two credits per browser minute.
Is Firecrawl free?
There is a free tier with a monthly credit allowance, two concurrent requests and low rate limits, and it needs no credit card. Paid tiers raise the credit allowance, the concurrency and the rate limits together.
How do I keep Firecrawl costs down?
Map a site before crawling it so you know its size. Pull plain markdown at one credit and let a cheap or local model turn it into structured fields rather than paying for JSON extraction on every page. And monitor weekly rather than daily, on the few pages where a change actually matters.
Is it legal to scrape websites in Australia?
It depends on what you collect and from where. Respect robots directives and site terms, keep request rates polite, and be careful with personal information: collecting names or contact details engages the Australian Privacy Principles, including collecting only what you need for a stated purpose. Crawling a client’s own site is a very different act from harvesting a directory of individuals.
