mcpField
A
audit

audit

github.com/glebisSecurity40265 installsUpdated 2d ago
Repository

About audit

confide:audit is a security skill for measuring PII across session files, suited to evaluating raw or redacted corpora. Detection runs locally, and reports contain aggregate statistics without transcript text or filenames.

Key features

  1. 01Recursively scans Markdown and text files
  2. 02Layered local PII detection
  3. 03Counts by type and detection layer
  4. 04Measures redaction rates and residuals
  5. 05Creates Markdown, JSON, and optional HTML reports

Use cases

  1. 01Estimate PII volume in raw corpora
  2. 02Check redacted corpora for residual PII
  3. 03Share aggregate results without exposing text

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/glebis/claude-skills --skill audit

After installing, type/auditin Claude Code to confirm it loaded.

Bundled filesscripts/audit.py

Details

From the project README

confide:audit — corpus-scale, stats-only PII audit

Measure how much PII lives across a whole folder of sessions, without ever exposing any of it. The audit runs the layered LOCAL detector stack from shared/confide_core.py (regex → Natasha → local LLM) over each file and emits only aggregates. This mirrors the real_session_eval privacy contract: read text only in-process, emit counts.

Privacy invariants (do not violate)

  • Local-only. No cloud APIs. Raw transcript text never leaves the machine.
  • Stats-only output. The report (markdown + json + optional HTML) contains ONLY counts and rates — never a transcript substring, never a detected PII value.
  • No filenames. Per-file rows are keyed by anonymized ids own-00, own-01, … The original path/name is never written or printed. On an unreadable file, only the index + exception class name is recorded.
  • Safe to surface. Because it is counts-only, the aggregate report can be shared with a cloud agent or pasted into a chat. The PII stays on the machine.

What it reports

  • n_files, total / mean / min / max document chars
  • spans_by_type (PERSON, EMAIL, PHONE, DATE, …) and spans_by_layer (regex / natasha / llm)
  • overall_redaction_rate plus the per-session redaction-rate distribution (min / median / mean / max)
  • a coarse residual proxy: spans still detectable after redaction — ~0 on a clean RED corpus, a leakage signal on a GREEN corpus.

Run it

Point it at a folder (recurses, processes every .md/.txt; skips confide's own *.green.md / *.stats.json outputs):

python3 skills/audit/scripts/audit.py FOLDER

Options:

  • --list paths.txt — also/instead audit absolute paths listed one per line.
  • --layers regex,natasha,llm — choose detection layers (default from config). Use --layers regex for a fully offline, deterministic pass (no models/network).
  • --out report.md — report path; a report.json sibling is written alongside.
  • --html — also write a Tufte-ish dashboard (report.html, counts only).

Writes the markdown + json report (and optional HTML) and prints the aggregate summary — all counts only.

RED vs GREEN

  • RED (raw) corpus: sizes the PII problem before any redaction.
  • GREEN (redacted) corpus: the residual proxy and remaining spans_by_type tell you whether redaction is holding at scale.

After running

  1. Report the aggregate summary (file count, span totals by type/layer, redaction-rate distribution, residual proxy) — never paste PII.
  2. If residual is non-trivial on a GREEN corpus, point the user at confide:anon to re-redact and confide:red to probe re-identification risk.

Setup

Layer availability (Natasha, local LLM via Ollama) comes from config — run confide:setup if they aren't installed. --layers regex always works offline.

Comments

Markdown supported