mcpField
F
firecrawl-knowledge-base

firecrawl-knowledge-base

github.com/firecrawlData & databases21232.6k installsUpdated 1w ago
Repository

About firecrawl-knowledge-base

A Firecrawl skill for turning URLs or topics into organized content for LLM use. It supports reference collections, RAG materials, training data, and documentation mirrors with source information preserved.

Key features

  1. 01Collects documentation and topic sources
  2. 02Preserves code examples and tables
  3. 03Supports reference, RAG, and training outputs
  4. 04Creates documentation mirrors with contents

Use cases

  1. 01Build a sourced reference collection
  2. 02Prepare content for RAG or training
  3. 03Mirror a documentation site

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/firecrawl/firecrawl-workflows --skill firecrawl-knowledge-base

After installing, type/firecrawl-knowledge-basein Claude Code to confirm it loaded.

Details

From the project README

Firecrawl Knowledge Base

Use this to turn URLs or topics into organized LLM-ready content.

Onboarding Interview

Infer the source, goal, depth, and output location from context. If the source and goal are clear, proceed immediately.

Ask at most 1-3 concise questions only if blocked, such as the source URL/topic, whether the output is reference/RAG/training/docs, or training format if training is requested.

Firecrawl Collection Plan

Use Firecrawl map for documentation sites, search for topic-based corpora, scrape pages into markdown, and preserve code examples and tables.

For files, follow the Firecrawl download-style convention:

.firecrawl/
  <hostname>/
    <path>/
      index.md

Parallel Work

If appropriate, use sub-agents or equivalent parallel task runners:

  • one docs section per researcher
  • official docs, tutorials, community discussions, and references by source type
  • source scraping vs chunk generation vs manifest generation

Output Modes

  • Reference: markdown files, index.md, and sources.json.
  • RAG: markdown files plus chunk files and manifest.json.
  • Training: scraped source files plus training-data.jsonl and training-metadata.json.
  • Docs mirror: complete markdown mirror with a table of contents.

Final Deliverable

# Knowledge Base: [Source]

## Summary
[What was collected and why]

## Output Structure
[Files/directories created]

## Coverage
[Sections, source types, counts]

## Usage Notes
[How to use in RAG, docs, training, or agent context]

## Sources
[URLs collected]

## Rerun Inputs
workflow: firecrawl-knowledge-base
source: [url/topic]
goal: [reference/rag/train/docs]
depth: [quick/thorough/exhaustive]
output_dir: [.firecrawl/]

Quality Bar

  • Preserve code examples and formatting.
  • Remove boilerplate navigation where possible.
  • Include source URLs in frontmatter or metadata.

Comments

Markdown supported