mcpField
F
firecrawl-knowledge-base

firecrawl-knowledge-base

github.com/firecrawlData & databases1393.7k installsUpdated 6d ago
Repository

About firecrawl-knowledge-base

firecrawl-knowledge-base is a skill for collecting URLs or topics and organizing the material into LLM-ready content. It defines collection plans and output structures for reference, RAG, training, and documentation mirror use.

Key features

  1. 01URL and topic-based collection
  2. 02Firecrawl mapping, search, and scraping
  3. 03Reference, RAG, and training outputs
  4. 04Documentation mirror with table of contents
  5. 05Source metadata and formatting guidance

Use cases

  1. 01Building a RAG corpus from documentation
  2. 02Preparing source material for model training
  3. 03Creating a searchable reference collection

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/firecrawl/skills --skill firecrawl-knowledge-base

After installing, type/firecrawl-knowledge-basein Claude Code to confirm it loaded.

Details

From the project README

Firecrawl Knowledge Base

Use this to turn URLs or topics into organized LLM-ready content.

Onboarding Interview

Infer the source, goal, depth, and output location from context. If the source and goal are clear, proceed immediately.

Ask at most 1-3 concise questions only if blocked, such as the source URL/topic, whether the output is reference/RAG/training/docs, or training format if training is requested.

Firecrawl Collection Plan

Use Firecrawl map for documentation sites, search for topic-based corpora, scrape pages into markdown, and preserve code examples and tables.

For files, follow the Firecrawl download-style convention:

.firecrawl/
  <hostname>/
    <path>/
      index.md

Parallel Work

If appropriate, use sub-agents or equivalent parallel task runners:

  • one docs section per researcher
  • official docs, tutorials, community discussions, and references by source type
  • source scraping vs chunk generation vs manifest generation

Output Modes

  • Reference: markdown files, index.md, and sources.json.
  • RAG: markdown files plus chunk files and manifest.json.
  • Training: scraped source files plus training-data.jsonl and training-metadata.json.
  • Docs mirror: complete markdown mirror with a table of contents.

Final Deliverable

# Knowledge Base: [Source]

## Summary
[What was collected and why]

## Output Structure
[Files/directories created]

## Coverage
[Sections, source types, counts]

## Usage Notes
[How to use in RAG, docs, training, or agent context]

## Sources
[URLs collected]

## Rerun Inputs
workflow: firecrawl-knowledge-base
source: [url/topic]
goal: [reference/rag/train/docs]
depth: [quick/thorough/exhaustive]
output_dir: [.firecrawl/]

Quality Bar

  • Preserve code examples and formatting.
  • Remove boilerplate navigation where possible.
  • Include source URLs in frontmatter or metadata.

Comments

Markdown supported