Web-Scraping-Agent-with-N8N-and-MCP-Server
LocalAbout Web-Scraping-Agent-with-N8N-and-MCP-Server
Web-Scraping-Agent-with-N8N-and-MCP-Server is a tutorial project showing how to build a chat-triggered scraping agent in n8n. It combines a chat model, simple memory, MCP tools, and Firecrawl to retrieve web content and respond to queries.
Key features
- 01Chat-triggered n8n agent workflow
- 02Firecrawl webpage scraping
- 03MCP client and server setup
- 04Simple memory for conversation context
Use cases
- 01Build an agent that retrieves webpage content from chat requests
- 02Learn to connect an n8n agent to an MCP scraping server
MCP support
The example uses an MCP client to send scraping tasks to an MCP server. The server performs scraping with Firecrawl and returns structured results.
Configuration
Local server config is the same for every client. Copy it into your client’s MCP config file.
We haven’t collected a package or remote endpoint for this server. Follow the instructions in the project README.
Tools
Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.
Details
From the project READMEWeb-Scraping-Agent-with-N8N-and-MCP-Server
Web Scraping Agent with N8N and MCP Server
🤖 AI Scraping Agent in n8n with MCP Server
This project demonstrates how to create a powerful, AI-powered web scraping agent in n8n using MCP (Multi-Component Protocol) and Firecrawl. The agent can receive a chat message, fetch relevant data from the internet via scraping, and respond intelligently using OpenAI or Claude models.
📌 Project Overview
This is the most beginner-friendly and hands-on tutorial for building an autonomous Scraping Agent with tools and memory capabilities using:
- 🧠 OpenAI Chat Model
- 🧰 MCP Tools
- 🗂️ Simple Memory
- 🌐 Firecrawl for web scraping
🧠 Agent Workflow (Chat-triggered)
The core agent is triggered when a chat message is received. It includes:
- Chat Model: Uses OpenAI (via OpenRouter)
- Memory: Context is maintained using Simple Memory
- Tool: Connects to MCP Client which communicates with the scraping server
🔧 Agent Setup
🖥️ MCP Server Workflow (Scraping Handler)
This is the actual scraping backend that receives the agent’s request, performs the scrape using Firecrawl, and sends structured results back.
🌐 MCP Server Setup
🧩 Components Used
| Component | Purpose |
|---|---|
Agent (Tools Agent) | Orchestrates AI, Memory, and Scraping |
OpenAI Chat Model | Understands and responds to user queries |
Simple Memory | Stores previous messages / memory context |
MCP Client | Forwards tasks to the MCP server |
MCP Server Trigger | Starts the scraping backend |
Scrape Tool | Uses Firecrawl to extract webpage data |
📚 What You'll Learn
- How to create and coordinate multiple AI agents in n8n
- Connecting OpenAI models with real-time tools
- Setting up an MCP Server inside n8n
- Ethical scraping using Firecrawl
- Using the agent-as-a-tool pattern for modular AI
⚙️ Technologies Used
- n8n
- OpenRouter
- Supabase (for optional DB storage)
- QuickChart.io (for optional visualization)
- Firecrawl for scraping
- [Claude / GPT-4] (for LLM reasoning)
- MCP Architecture
Comments
New to MCP?
From what the protocol is to installing your first server, these answers cover most first questions.
An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.