fetch-url

Fetch URL

Fetch a single URL and convert its content to clean Markdown. This does not add anything to the documentation index; it is a one-shot read operation.

When to use

• You need to read a single web page and get its content as Markdown.
• You want to inspect a documentation page before deciding whether to index it.
• You need the raw text of a URL for summarisation or analysis.

If you need to index the content for repeated searching, use the docs-manage skill (scrape command) instead.

Command

npx @arabold/docs-mcp-server@latest fetch-url <url> [options]

Flag	Default	Description
--follow-redirects	true	Follow HTTP redirects
--no-follow-redirects		Disable following redirects
--scrape-mode auto|fetch|playwright	auto	HTML processing strategy
--header "Name: Value"		Custom HTTP header (repeatable)
--quiet		Suppress non-error diagnostics
--verbose		Enable debug logging

Scrape modes

Mode	When to use
auto	Default. Tries a simple HTTP fetch first, falls back to Playwright for JS-rendered pages.
fetch	Force a plain HTTP fetch. Fastest, but misses content rendered by JavaScript.
playwright	Force a headless browser. Use for SPAs or pages that require JavaScript to render.

Examples

# Fetch a documentation page
npx @arabold/docs-mcp-server@latest fetch-url https://react.dev/reference/react/useEffect

# Fetch with custom auth header
npx @arabold/docs-mcp-server@latest fetch-url https://docs.internal.com/api \
  --header "Authorization: Bearer tok_xxx"

# Force Playwright for a JS-heavy page
npx @arabold/docs-mcp-server@latest fetch-url https://some-spa.dev/docs --scrape-mode playwright

# Disable redirect following
npx @arabold/docs-mcp-server@latest fetch-url https://example.com/old-page --no-follow-redirects

Output

The command writes the converted Markdown text directly to stdout. The global --output flag is accepted but has no effect because the result is already plain text, not structured data.

Diagnostics and errors go to stderr and are suppressed by default in non-interactive sessions. Use --verbose (or set LOG_LEVEL=INFO) to re-enable them. Use --quiet to suppress all non-error diagnostics regardless of session type.

Tips

• Pipe the output to a file if you want to save it: npx @arabold/docs-mcp-server@latest fetch-url <url> > page.md
• Combine with search: fetch a page to read its full content after search returns a relevant URL.
• For pages behind authentication, use --header to pass cookies or tokens.