mcpField
L
llms.txt Checker - AI Readiness Audit

llms.txt Checker - AI Readiness Audit

Remote
github.com/apifymcpfactory-droidBusiness & Marketing0Updated 14w ago
Repository

About llms.txt Checker - AI Readiness Audit

llms.txt Checker is a pay-per-use MCP tool for auditing a website’s readiness for AI crawlers. It checks llms.txt files, crawler rules in robots.txt, and the sitemap, then provides a 0–100 score and a list of suggested fixes.

Key features

  1. 01Checks llms.txt and llms-full.txt
  2. 02Reviews AI crawler rules in robots.txt
  3. 03Checks for sitemap.xml
  4. 04Returns a 0–100 readiness score
  5. 05Lists issues to fix

Use cases

  1. 01Audit AI crawler access across a group of sites
  2. 02Identify missing files or restrictive crawler rules
  3. 03Prioritize website changes using the readiness score

MCP support

Exposes the website audit as a tool through Apify’s hosted MCP server. Usage is billed through the connected Apify account on a per-event basis.

Configuration

Remote servers need no local install. Pick your client, then copy the config or install in one click.

Endpointhttps://mcp.apify.com?tools=apifmcpfactory/llms-txt-checker
Copy install command
Claude CodeTerminal
Cursor.cursor/mcp.json
VS Code.vscode/mcp.json
Claude Desktopclaude_desktop_config.json
Windsurf~/.codeium/windsurf/mcp_config.json

Run commands in a terminal; paste JSON into the client’s MCP config and restart. Servers that need sign-in will prompt on first connect.

All endpoints
Streamable HTTPhttps://mcp.apify.com?tools=apifmcpfactory/llms-txt-checkerAuth required

Tools

Details

From the project README

MCP Factory: pay-per-use MCP tools for AI agents

MCP Factory builds small, reliable, pay-per-use tools that AI agents can call over the Model Context Protocol (MCP). Each tool runs as an Apify Actor and is exposed through Apify's hosted MCP server — no installation, no infrastructure, no subscription. Your agent calls the tool directly; usage bills through your Apify account at flat, transparent per-result prices.

The tools

🔍 Tech Stack Detector

Find out what any website is built with. Scans a list of URLs and identifies the technologies behind each site — CMS, ecommerce platform, web frameworks, analytics, CDN, web server, and more — using the open-source webappanalyzer fingerprint dataset (7,500+ technologies across 109 categories). A fast, affordable Wappalyzer / BuiltWith alternative at roughly $5 per 1,000 sites.

🧾 EU VAT Validator

Validate EU VAT numbers in bulk against VIES, the European Commission's official VAT validation service — and get the official consultation number on every check as audit evidence for tax inspections. Bulk-safe with automatic retries, tolerant of messy input formats, $5 per 1,000 validations flat.

🤖 llms.txt Checker

A complete AI readiness audit for any website in one pass: checks llms.txt, llms-full.txt, robots.txt AI-crawler policy (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Bytespider), and sitemap.xml, then returns a 0–100 AI readiness score with a plain-English list of exactly what to fix. ~$5 per 1,000 sites.

Connect all three tools at once

One MCP server URL exposes any combination of tools. To give your agent the full MCP Factory toolkit:

https://mcp.apify.com?tools=apifmcpfactory/tech-stack-detector,apifmcpfactory/eu-vat-validator,apifmcpfactory/llms-txt-checker

Claude Desktop (claude_desktop_config.json):

{
    "mcpServers": {
        "mcp-factory": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com?tools=apifmcpfactory/tech-stack-detector,apifmcpfactory/eu-vat-validator,apifmcpfactory/llms-txt-checker",
                "--header",
                "Authorization: Bearer YOUR_APIFY_TOKEN"
            ]
        }
    }
}

Cursor (.cursor/mcp.json):

{
    "mcpServers": {
        "mcp-factory": {
            "url": "https://mcp.apify.com?tools=apifmcpfactory/tech-stack-detector,apifmcpfactory/eu-vat-validator,apifmcpfactory/llms-txt-checker",
            "headers": {
                "Authorization": "Bearer YOUR_APIFY_TOKEN"
            }
        }
    }
}

Replace YOUR_APIFY_TOKEN with your API token from Apify Console. New Apify accounts include free monthly usage credit, so you can try every tool before paying anything.

How pricing works

Every tool is pay-per-event: a small flat fee per run plus a flat fee per result. No subscriptions, no tiers, no minimums. Usage is billed through the Apify account whose token your agent uses — you can see every run, result, and charge in Apify Console.

About

MCP Factory publishes on the Apify platform as apifmcpfactory. Each tool is designed around the same principles: one clear job, bulk-safe execution (one output row per input, always), honest pricing, and output an AI agent can use without post-processing.

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.