Agent research
Use agent or search when the user asks for facts across multiple sources. Keep citations and task prompt with the result.
Use cases
Map AI web data jobs to safer Firecrawl workflows.
The useful question is not whether a page can be scraped. It is what source, output, limit, and audit trail the product needs.
RAG and docs ingestion
Use map, crawl, and scrape to create Markdown chunks from docs pages, then record source URL and update date.
Pricing intelligence
Use search for discovery, scrape for pricing pages, and schema JSON for fields. Avoid presenting stale prices without refresh dates.
Content operations
Use scrape and screenshot outputs to audit landing pages, changelog pages, or competitor messaging.
QA and monitoring
Use map plus targeted scrape checks to detect missing pages, status changes, and critical text drift.
Internal data boundary
Use self-hosting only when security, privacy, or network policy requires controlled infrastructure.
Workflow template
- Define allowed source types.
- Pick endpoint by discovery level and scale.
- Choose Markdown, JSON, screenshot, or HTML output.
- Store prompt, source URL, timestamp, and extraction config.
- Review robots, terms, consent, and data sensitivity.
Success metrics
- Extraction success rate by source type.
- Schema validity and empty-field rate.
- Freshness of source URLs.
- Token savings from cleaner Markdown.
- Manual review rate for sensitive sources.