readmeaudit

readmeaudit

READMEs drift. The install command names a package that was renamed, the usage section documents flags the CLI dropped two releases ago, and nobody notices until a user copies a command that does not exist.

readmeaudit compares a package README against the two sources of truth that live next to it: package.json and the CLI source under src/. It checks that the package name used in install commands matches the package.json name, that every long flag the README documents actually appears in the source, that every long flag in the source is documented, and that a package which declares an MCP server says so in its README. The result is a deterministic report with a verdict: IN-SYNC or DRIFT, with per-item evidence (rule, severity, file, line) for every disagreement.

It runs offline, reads only static text, and never executes anything in the target package. It is callable from the CLI or as an MCP server, so the check can run in release workflows and agent sessions, not just by hand.

Install

npx @reneturcios/readmeaudit

That single command runs the tool without a global install. Point it at a package directory with the audit command shown below.

CLI usage

readmeaudit audit [path] [--json]
readmeaudit mcp
readmeaudit --version

• audit checks the package at the given path (default: the current directory) and prints the human readable report. Adding the JSON option prints the same report as JSON instead.
• mcp starts the MCP server on stdio.
• The version option prints the tool version and exits.

Exit codes: 0 when the verdict is IN-SYNC, 1 when the verdict is DRIFT, 2 for usage errors and unreadable targets. A reader that closes the output pipe early makes the process exit 141, never 0.

Example output for a package whose docs match reality:

readmeaudit 0.1.0
Target: .
Verdict: IN-SYNC
Counts: flagsDocumented=3, flagsInSource=3, phantom=0, undocumented=0, nameDrift=0
Findings: none
IN-SYNC: no high or medium findings; 0 low or info finding(s) noted.

The counts in every report tally the --flags documented in the README and the --flags found in source, plus how many documented flags were phantom, how many source flags were undocumented, and how many install commands named the wrong package.

What it checks

rule	severity	what it means
NAME-DRIFT	high	An install command in the README names a different package than the name field in package.json. The finding carries the line and the offending line as evidence.
NO-INSTALL	medium	The README contains no install invocation at all, so a new user has no copyable command that names the package.
FLAG-PHANTOM	medium	A long flag is documented in the README but appears in no source file under src/.
FLAG-UNDOCUMENTED	low	A long flag appears in source under src/ but is never documented in the README.
MCP-UNDOCUMENTED	low	package.json declares an mcpName, so the package ships an MCP server, but the README never mentions mcp.
CMD-UNKNOWN	info	A fenced code block line starts with a bare command word that is not a declared bin name for the package, on a line that also carries a long flag or the word mcp.

How each check works

Names. The name check scopes itself to install lines only: lines that carry a package runner command or an install command from the standard JavaScript package manager. On those lines, the first non-option token after the command word is the invoked package name, with any version suffix stripped. Scoped names and plain names are both understood. An invoked name that differs from the package.json name is NAME-DRIFT. Restricting the check to install lines is deliberate: prose and comparison tables legitimately mention other tools by name, and those mentions are not install commands.

Flags. The flag check collects long flag tokens (two dashes, a letter, then letters, digits, or dashes) from the README and from every TypeScript file under src/, recursively. The same collector runs on both sides; it is the collector that picks up a token such as --flags when that token appears as text. The two sets are then diffed in both directions, one finding per flag.

MCP. When package.json has an mcpName field, the package presents itself to MCP clients under that name, and the README should at least mention the protocol. The check is a case-insensitive search for the word mcp anywhere in the README.

Commands. The command check is the most conservative check in the tool, by design. It only looks inside fenced code blocks, only at the first token of a line, and only when that token is a bare word that is neither a declared bin name nor a well known shell or program word (the package managers, the runtime, the container tool, version control, and basic shell words are all excluded). Even then it reports at info severity, it is counted like every other finding, and it never changes the verdict. An unknown command word in a usage block is often a helper, an alias, or a renamed binary from an older release; the finding exists so a human glances at it, not so the gate fails.

Verdict rules

• IN-SYNC: no high or medium findings. Low and info findings are reported but do not change the verdict. Exit code 0.
• DRIFT: at least one high or medium finding. Exit code 1.
• Findings are ordered by rule, then by the flag or name they concern, then by line. Reports contain no timestamps, so two runs on the same tree produce byte-identical output.

MCP server

Add to your MCP client config:

{
  "mcpServers": {
    "readmeaudit": {
      "command": "npx",
      "args": ["-y", "@reneturcios/readmeaudit", "mcp"]
    }
  }
}

Tool: audit_readme(path) returns the full audit report as JSON, the same object the CLI prints with the JSON option.

Limits, stated plainly

• This is a static text comparison. readmeaudit can check names, flags, and commands, because those are concrete tokens with a source of truth in package.json and src/. It cannot judge whether prose descriptions are true: a README that describes behavior the code does not have, in flowing sentences with no wrong names or flags in it, passes.
• Flags are text tokens. A flag that appears in a source comment or a string literal counts as present in source, and a flag assembled dynamically at runtime (built from pieces, or read from configuration) will be missed.
• Source coverage is the TypeScript files under src/ only. A CLI implemented in another language, or in files elsewhere in the tree, is invisible to the flag check, and its flags will read as phantom.
• The name check sees only install lines. A stale package name in prose, in a badge URL, or in a heading is not flagged.
• The bin names come from the bin field of package.json. A package with no bin field gets no command check protection beyond the known-word exclusions, and CMD-UNKNOWN stays info severity for exactly this reason: the tool would rather under-report commands than accuse a README over a guess.
• Missing inputs are errors, not findings: a target with no package.json, or with neither README.md nor README, fails with a clear message and exit code 2.