Groundhog

Groundhog

Web search, read and research for AI agents, through a real, stealth-patched Chrome. Groundhog is an MCP server that returns clean Markdown a model can trust: text no human could see is stripped before the model reads it and reported, every source comes with a provenance receipt, and a real browser reads pages that block plain fetchers.

agent / crawler  ──MCP──▶  Groundhog (search, read_url, research)  ──CDP──▶  stealth Chrome  ──▶  the web

Measured against other fetch layers

Twenty-eight pages carry text a human can't see, and three controls carry none. A case passes when the payload doesn't reach the model and the article text still does.

	contained	kept the article	reported it	false positives
requests + trafilatura	25/28	28/28	0/28	0/3
Scrapling (HTTP)	16/28	28/28	0/28	0/3
Jina Reader	15/28	28/28	0/28	0/3
Firecrawl	7/28	28/28	0/28	0/3
Groundhog	28/28	28/28	25/28	0/3

Other extractors often drop hidden text by accident, but none of them tells the caller anything was removed. The three cases Groundhog doesn't report (a hidden input value, a meta description, an iframe srcdoc) never reach any extractor. The corpus is public at dmytrome.github.io/groundhog; the harness and its limits are in benchmark/.

Quick start

Claude Code:

claude mcp add groundhog -- uvx groundhog-mcp

Claude Desktop, Cursor, Windsurf:

{
  "mcpServers": {
    "groundhog": {
      "command": "uvx",
      "args": ["groundhog-mcp"]
    }
  }
}

On the first fetch Groundhog starts the Chrome you already have, headless, in its own profile at ~/.groundhog/chrome. Your own browsing, tabs and logins are never touched. If Chrome is missing, status and the error say what to install.

GROUNDHOG_BROWSER=stealth runs the hardened browser container instead (Docker or Podman; the first pull takes a few minutes). To run it yourself:

docker run -d --rm --name groundhog-browser --shm-size 512m \
  -p 127.0.0.1:9222:9222 -- ghcr.io/dmytrome/groundhog:latest

CDP_URL points Groundhog at any browser and is used as given: http:// is probed for /json/version, ws:// or wss:// (what hosted browsers hand out) is connected to directly. Auto-start only runs for a local address that isn't answering, and GROUNDHOG_AUTO_START_BROWSER=false turns it off. From source: cd mcp && uv sync && uv run groundhog-mcp.

All four tools are annotated readOnlyHint: they write nothing and change no remote state. The one side effect is auto-start, which may launch Chrome or, with stealth, run the container after removing an unreachable one named groundhog-browser.

What makes it different

• Hidden text is stripped and reported. A rendered DOM shows what a human would see; everything else goes, and each removal is listed in threats. It's a strong heuristic, not a proof: see the limits.
• Every source carries a receipt: a SHA-256 of the extracted content, canonical URL, language, word count, and author and date when the page declares them.
• Safe by default. An SSRF guard checks every document request, redirects and frames included, with per-domain rate limiting. See Security.
• No automation tell. Groundhog drives Chrome over raw CDP and never enables the Runtime domain that anti-bots detect in Puppeteer, Playwright and Selenium.
• A real fingerprint. Real Chrome means real TLS, HTTP/2, WebGL and canvas. In headless mode the HeadlessChrome token is replaced and client hints are copied from the browser; shared and service workers aren't covered. The stealth image runs headful under Xvfb.
• No model, no API key. research returns extracts and your agent does the synthesis.

Tools

read_url(url, format="markdown", max_tokens=None, query=None, include_hidden=False)

Key	Meaning
markdown	Extracted content, article first, falling back to the rendered text inside a code fence, so none of it reads as markdown; a PDF comes back as one ## Page N section per page with its text fenced. format may be markdown or text
title	Page title
url	The URL you asked for
final_url	The URL after redirects, re-checked by the SSRF guard. If unusable, the requested URL is reported with a final_url_suppressed threat
fetched_at	UTC ISO-8601 timestamp
status	ok, or challenge / blocked / rate_limited / not_found / server_error / unsupported_content when what came back isn't the page you asked for. blocked covers 401, 403 and other 4xx error pages. too_large means the page had over 5 MiB of HTML or 2 MiB of text, or was a PDF over 10 MiB, 100 pages or 20 seconds of parsing, and nothing of it is returned. An encrypted or malformed PDF is unsupported_content. empty means the rendered page showed no text a reader would see (text in closed shadow roots, iframes and alt attributes doesn't count), whatever markdown holds. With include_hidden, text hidden by styling counts as shown; invisible characters never do. unknown means no usable status was observed. Judged on the document the text came from, after client-side redirects
http_status	The top-level HTTP status, or null
truncated	Whether the token budget cut the content
threats	What was stripped; empty when nothing was found
matches	With query: ranked passages with id, heading, offset, end, score and clamped (cut short to fit max_tokens). offset and end index the page's full markdown, and that span is the text returned; a piece of a long code block comes back with its fence lines added around it. Code is never cut mid-piece: a matching code block or piece larger than max_tokens is left out, and when nothing fits markdown is empty with truncated set, so raise max_tokens
set_aside	With query: scoring passages the relevance floor dropped
provenance	Content hash, canonical URL, language, word count, author and date

Stripped by default: display:none, visibility:hidden, content-visibility:hidden, opacity ≤ 0.05, font-size < 4px, zero-size and display:contents boxes that render nothing, .sr-only-style sub-pixel boxes, clip: rect(...), a clip-path that leaves nothing (circle(0), inset(100%), a polygon on one line), transform: scale(0) on either axis, a filter whose opacity is 0.05 or less, zero-height or zero-width boxes that clip their overflow, transparent or background-coloured text, off-page positioning, collapsed <details>, <template> content, and zero-width, bidi and Unicode Tag characters. A box whose only text is an open popover or modal dialog is kept, because that dialog still shows on screen; any other text in the box is stripped as usual. HTML comments are reported too. In a PDF, text drawn in render mode 3 or 7, which paints nothing, is stripped and reported as hidden_pdf. That includes the OCR layer of a scanned page, so a scan whose only text is that layer is empty unless you pass include_hidden. include_hidden=True keeps the text but still reports it. alt, aria-label, aria-description and title text never appears in markdown or the rendered text; values of 125 characters or more are reported as hidden_attribute with a capped excerpt, because long text there is a known carrier.

Treat threats as untrusted: its excerpts are page-authored text, cleaned and length-capped.

type	Carries
hidden_css	The hiding reason, an 80-character excerpt and the DOM location
hidden_attribute	The same, for text in an alt, aria-label, aria-description or title attribute
hidden_template	The same, for text inside a <template>
hidden_pdf	The same, for text a PDF drew invisibly, with the page as its location
zero_width / bidi / tag	A codepoint and count in reason, detected on the served text
report_truncated	How many entries the cap dropped
final_url_suppressed	The page's final URL was unusable and not returned
detection_degraded	Detection ran in the page's own JavaScript world, which the page can tamper with
strip_incomplete	The text was rebuilt from markup rather than layout, usually because of open shadow roots

At most 50 findings per page are returned (10 per source in research), with the two classes capped separately so decoys can't bury a real payload. Notices are appended after the cap.

query swaps head-truncation for BM25 passage ranking on the sanitized content, keeping passages within half the best score up to the budget.

search(query, limit=10)

Returns hits (title, url, snippet, engine, score, published) and the backend that answered. Nothing is fetched. With SEARXNG_URL set it uses your SearXNG instance, which needs formats: [html, json]; otherwise it renders a search page through the browser. Hit text is cleaned and capped, and a hit whose URL would change under cleaning, isn't http(s), carries credentials or exceeds 2048 characters is dropped. A backend that can't answer raises an error instead of returning an empty web.

research(query, max_sources=5, max_tokens=None, include_domains=None, exclude_domains=None, published_after=None, aim=None, follow=None)

Searches, reads the top sources and ranks passages from all of them in one pass.

• passages: id, text, source_url, heading, score, start, end, clamped, fence. text is exactly markdown[start:end]; clamped says it was cut short to fit the budget. A code block is never cut short, only left out. A passage cut from a long code block carries the block's opening line in fence: reopen with it when text doesn't start with it, and close the block when text doesn't end with a closing fence. id stays stable while the page doesn't change.
• sources: url, title, status (ok / blocked / timeout / error), page_status (as in read_url), threats, threat_total, threat_counts (every finding by type, spans the page's detector dropped as hidden_unclassified), provenance, filtered, depth and discovered_via (the candidate it was followed from, with your why). A failed source doesn't fail the call. A challenge, an error page, an empty page or any page whose markdown is blank counts as not_content and contributes no passages.
• One page per registrable domain unless include_domains is set, and then no more pages per site than GROUNDHOG_MAX_CONCURRENT_PAGES; max_sources goes up to 10.

include_domains reads only those domains and their subdomains, exclude_domains never reads them, and published_after (YYYY-MM-DD) drops pages dated earlier. Up to 8 domains can be included and 50 excluded. A page that redirects into an excluded domain or out of the included ones, is dated too early, or repeats the URL or text of a page already used is listed with filtered and contributes no passages. The query sent to the engine names at most eight domains; further excluded ones are filtered after the search. aim (up to 200 characters) says what the evidence must show and is echoed back without changing the search or the ranking.

receipt holds the executed_query, the distinct hits and hits_by_domain, the hits not fetched (hits_excluded, hits_outside_domains, hits_before_date, hits_same_domain, hits_unread past the source limit), each source counted once as failed, not_content, filtered or usable, outside_domains, unknown_dates, and the passages omitted, set_aside or clamped. Read it before concluding the web has no answer.

candidates are links inside the returned passages, at most 12, no more than 4 from one page and 3 to one domain. Each has a token, url, domain, label (the link text, which the page wrote), parent_source_url, parent_passage_id, depth and flags (parent_strip_incomplete, parent_detection_degraded, parent_threats=N, likely_unsupported_format, same_domain_pages=N). Links back to the page, to pages already read in the call, outside the domain filters, or from a page three links deep are not offered. To read some, call again with follow=[{"token": ..., "why": ...}] (up to 8, each why up to 160 characters) and the same query and filters: nothing is remembered between calls. A follow runs no search, and followed says for each entry whether it was read, not_content, failed, filtered, a duplicate, over the same_domain_limit (as many pages per site as can start before the deadline), an invalid_token or expired. Tokens are signed by the server and expire after 30 minutes or on restart.

The JSON text block of a response never exceeds 40,000 characters; the MCP result carries the same object again as structuredContent. When it would, cards go first, then threat excerpts, then passages, then metadata, each counted in the receipt (candidates_omitted, threat_excerpts_omitted, passages_omitted, sources_omitted, metadata_compacted); safety counts from sources left out are kept in omitted_safety, and notes says so: read_url on a source returns its page, and a narrower query or fewer sources leaves more room.

status()

Returns browser_reachable, cdp_url (scheme, host and port only) and a hint when the browser can't be reached.

Configuration

MCP server:

Env var	Default	Purpose
CDP_URL	http://127.0.0.1:9222	CDP endpoint, local or remote. Unauthenticated, so keep it private
GROUNDHOG_BROWSER	chrome	What auto-start launches: chrome or stealth. Defaults to stealth when GROUNDHOG_COMPOSE_FILE or GROUNDHOG_BROWSER_IMAGE is set
GROUNDHOG_CHROME_PATH	(found)	Chrome executable
GROUNDHOG_CHROME_PROFILE	~/.groundhog/chrome	Profile directory; a login made here is reused
GROUNDHOG_BLOCK_PRIVATE_IPS	true	The SSRF guard
GROUNDHOG_MIN_DELAY_MS	5000	Delay between requests to one domain
GROUNDHOG_MAX_TOKENS	20000	Token budget
GROUNDHOG_MAX_CONCURRENT_PAGES	4	Concurrent tabs
SEARXNG_URL	(unset)	SearXNG instance for search
GROUNDHOG_SEARCH_BACKEND	auto	auto, searxng or serp
GROUNDHOG_AUTO_START_BROWSER	true	Start the browser when a local CDP_URL isn't answering
GROUNDHOG_BROWSER_IMAGE	ghcr.io/dmytrome/groundhog:latest	Image for stealth
GROUNDHOG_COMPOSE_FILE	(none)	Start with docker compose -f <file> up -d instead

Browser container:

Env var	Default	Purpose
USER_AGENT	derived from installed Chrome	UA set at launch, clean in every scope
PROXY	(none)	Upstream proxy http://user:pass@host:port; timezone and locale follow the exit IP
TZ	UTC	Fallback timezone
WINDOW_SIZE	1920,1080	Chrome window size
XVFB_WHD	1920x1080x24	Virtual display

The stealth Chrome container

Headful Chrome under Xvfb with a CDP endpoint, usable by any CDP client. It reports Chrome rather than HeadlessChrome, sets navigator.webdriver to false, and sets the UA at launch. With PROXY, timezone and locale match the exit IP, credentials go through a local tinyproxy, and WebRTC is pinned to the proxy. WebGL uses a detected GPU, else Mesa llvmpipe.

Against a fresh container (Chrome 151, no proxy), over raw CDP:

Detector	Result
deviceandbrowserinfo	not a bot (isBot: false, zero flags)
browserscan	Normal
bot.sannysoft.com	31 / 31 checks pass

iphey flags any datacenter IP, so it's tracked but not scored. The live table is in RESULTS.md. Puppeteer, Playwright and Selenium are still flagged because they enable Runtime; examples/ has clients for each, and OTHER_TOOLS.md covers more.

Security

The CDP endpoint is unauthenticated: anyone who reaches the port controls the browser. Keep it on localhost or a private network. The stealth image runs --no-sandbox, so keep the container isolated. Report vulnerabilities via SECURITY.md.

The SSRF guard resolves each host and refuses loopback, private, link-local (including 169.254.169.254), reserved, multicast, unspecified, CGNAT and IPv4-mapped IPv6 addresses, non-http(s) schemes, and URLs with credentials. It checks every document request Chrome is about to send, redirects and cross-site frames included, and final_url once more. Sub-resources are left to Chrome's Local Network Access rule, and the Chrome that Groundhog launches has preloading turned off.

Limits of the guard:

• Only documents are checked. An older Chrome, or a site granted local-network access in the profile, isn't covered for sub-resources.
• Preloading stays on in the stealth container and in a browser you supply.
• Groundhog and Chrome resolve DNS separately, so a short-TTL rebinding window remains.
• Fetches share one profile, so cookies set by one page persist into later fetches, and page JavaScript can make its own requests from it.

GROUNDHOG_BLOCK_PRIVATE_IPS=false turns the guard off; use it only where reaching internal addresses is intended.

Limits of hidden-text detection

An empty threats list isn't a clean bill of health. Detection runs in an isolated world the page can't tamper with, and the live page is never modified. But:

• Thresholds can be sat just inside (opacity: 0.06, transform: scale(0.01)), and a style is only treated as hiding when it provably paints nothing, so these pass: text-indent, inset(50%) (Chrome rounds the value it reports), clip-path in pixels or by url()/path(), 3D rotations, mask-image, text painted over by another element or stacked behind one, and a positioned child placed outside a box that clips its overflow.
• In a PDF only render modes 3 and 7 are detected. White or tiny text, text outside the page, hidden layers and text under an image pass, and every PDF result says so in a detection_degraded notice. A PDF whose invisible text is in a font whose glyph widths can't be measured (vertical writing, a CMap other than Identity-H) is unsupported_content rather than returned with that text in it, unless you pass include_hidden. Parsing runs in a child process that is killed at 20 seconds. On Linux it's also held to 1 GiB of memory, and elsewhere only the time cap and a 256 MiB cap on decoded streams bound it.
• The invisible-character set is a denylist.
• Closed shadow roots can't be read, so their content is left out entirely.
• When the page wins the cascade against the hiding stylesheet or hides its own <body>, text is rebuilt from markup, line breaks are guessed, and strip_incomplete says so.

Limits of block detection

• A challenge is called with certainty from vendor headers (cf-mitigated, x-vercel-mitigated, x-amzn-waf-action, x-dd-b, x-datadome-cid) or challenge assets (Cloudflare, DataDome, PerimeterX, Imperva). Other vendors fall back to English wording on a near-empty page.
• A soft block, a 200 with the data quietly withheld, looks like content.
• unknown isn't ok, but it isn't treated as a block either.
• Vendor presence (server: cloudflare, cf-ray) is deliberately ignored.

A note on "stealth"

Best effort. It beats common open-source detectors, not commercial anti-bot systems that gate on IP reputation and behaviour. Use it for legitimate, authorized automation.

Privacy Policy

Groundhog is self-hosted software and talks to no servers the project runs. It collects nothing: no telemetry, analytics, crash reports or accounts.

Pages are fetched by a browser on your machine and returned to your MCP client. The server keeps no database and writes no logs. The browser profile does persist cookies and storage: ~/.groundhog/chrome for local Chrome (delete the folder to discard it), or the container for stealth (docker rm -f groundhog-browser).

Your traffic reaches the sites you fetch, your search backend (your SearXNG, or a public search engine without one), an IP-geolocation service for the proxy's exit IP only when PROXY is set, and PyPI and GHCR at install time.

Questions: SECURITY.md or https://github.com/dmytrome/groundhog/issues.

License

MIT