mcpField
V
verify-docs-findable

verify-docs-findable

github.com/zernieDocuments15340 installsUpdated 2d ago
Repository

About verify-docs-findable

verify-docs-findable is a skill for engineers maintaining documentation who need to test whether readers can find answers from the docs alone. Fresh Haiku agents start at the index, answer real questions, and report accuracy, navigation steps, and any need to consult source code.

Key features

  1. 01Cold-reads docs starting from the index
  2. 02Runs parallel Haiku checks on real questions
  3. 03Records answers, sources, and navigation steps
  4. 04Finds dead ends, omissions, and contradictions
  5. 05Rechecks findability after documentation fixes

Use cases

  1. 01Test whether contributors can find documented answers
  2. 02Audit findability after reorganizing documentation
  3. 03Find gaps between prose and source code
  4. 04Measure whether revisions reduce navigation steps

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/zernie/vigiles --skill verify-docs-findable

After installing, type/verify-docs-findablein Claude Code to confirm it loaded.

Details

From the project README

verify-docs-findable — cold-read the docs with fresh eyes

The insight this operationalizes: complete ≠ findable. An index that lists every doc (and even passes a "every doc is indexed" check) can still make a reader take 5 hops and fall back to source code to answer a basic question. The only honest test is to give the question to someone who has never seen the repo and watch how far they get from the docs alone. A fresh cheap agent is that someone.

Why HAIKU, and why "docs only"

  • Haiku (a small, cheap model), fresh (no session context). The point is a naive cold reader, NOT a clever one. A strong model with your context will grep the source, reason around gaps, and hide the doc problem. Haiku, told to use only the docs, exposes it.
  • "Use ONLY the docs, start at the index." The moment a verifier has to open source code to answer, the docs have failed — that's the signal. Steps-from-index is the metric.

Procedure

1. Get the questions (3–8)

Test the recurring, load-bearing, and mis-stated questions — the ones that actually cost time. Sources, in order:

  • What the user (or you) just got confused about this session — the highest-signal source.
  • The "canonical answers" the docs claim to provide (a docs corpus usually has ~5–8 facts it exists to convey; test those).
  • If none given: ask the user for the questions, or infer them from the corpus's top-level topics. Don't invent trivia — test what a real contributor needs.

Phrase each as a real question a reader would ask, not a doc title.

2. Spawn one fresh Haiku verifier per question (in parallel)

Use the Agent tool with model: "haiku", run_in_background: true, one per question. Give each the SAME strict template (fill in <QUESTION>):

You are a fresh engineer opening <repo path> for the first time. Answer using ONLY the repo's docs — START at the index (<the index file(s)>) and follow pointers; prefer docs over reading source code. QUESTION: <QUESTION> REPORT: (a) your answer; (b) the exact file(s) where you found it; (c) how many steps from the index (did a pointer take you straight there, or did you dig?); (d) clear/unambiguous or scattered/confusing? (e) did you have to read source code because the docs didn't say? Be honest — this tests whether the docs make this findable.

Run all in one message so they go concurrently. (Haiku + parallel = the audit is cheap.)

3. Score each answer
SignalGoodGap
Correct?matches ground truthwrong/partial → the doc is wrong or missing
Steps from index1–23+ → no signposted pointer
Dead-endsnonelanded on a wrong-but-plausible doc first → title/scoping is misleading
Needed source code?noyes → the docs don't actually state it
Contradictions—cited two docs that disagree → a cohesion bug (fix immediately)

Verifiers surface contradictions for free — a cold reader citing two docs that say different things is the cheapest contradiction-finder you have.

4. Fix the gaps
  • 3+ steps / dead-end → add a question→doc pointer at the index entry point (a "Canonical answers: question → the ONE doc" block). Naming the answer is what turns 5 hops into 1.
  • Needed source code → the fact isn't in prose; write it into the one canonical doc.
  • Wrong answer → the doc is stale/incorrect; fix it (append-don't-erase if it records a decision).
  • Contradiction → reconcile the docs; keep the historical decision, mark it superseded.
  • Scattered across N docs → consolidate into ONE doc per question; make the others point to it.
5. Re-verify (measure, don't assume)

Re-run the worst question(s) against the fixed docs. Confirm the number moved (e.g. 5 steps → 2). A fix you didn't re-measure is a guess.

Output

A short findability scorecard (question | correct | steps | needed-source | verdict) + a prioritized fix list. Report the before/after on any question you fixed and re-verified.

Notes

  • This is orthogonal to "is every doc indexed?" completeness checks — it measures the layer above: can a stranger reach the answer. Run it after any doc reorganization.
  • Keep the question set in the repo (e.g. a docs-findability-questions.md) so the audit is repeatable and the canonical answers stay honest as the corpus grows.
  • Scales down (1 question, 1 agent, to spot-check one fix) and up (the full canonical set).

Comments

Markdown supported