mcpField
V
validator-help

validator-help

github.com/codagent-aiTesting & quality3063 installsUpdated 1d ago
Repository

About validator-help

validator-help is a diagnosis-only skill for explaining agent-validator behavior from runtime artifacts and CLI output. It routes questions to troubleshooting references and requires evidence, confidence, and next steps in each diagnosis.

Key features

  1. 01Runtime-artifact diagnosis
  2. 02Passive evidence before CLI commands
  3. 03Troubleshooting by issue domain
  4. 04Evidence and confidence reporting
  5. 05Bug-report routing rules

Use cases

  1. 01Investigating failed validation gates
  2. 02Diagnosing configuration or lock problems
  3. 03Checking adapter health and cooldowns

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/codagent-ai/agent-validator --skill validator-help

After installing, type/validator-helpin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

references/
adapter-troubleshooting.md
config-troubleshooting.md
gate-troubleshooting.md
lock-troubleshooting.md

Details

From the project README

/validator-help

Evidence-based diagnosis of validator behavior. This skill is diagnosis-only — it explains what happened and why, but does not auto-fix issues. It operates from runtime artifacts and CLI outputs, not source code.

Diagnostic Workflow

Follow this order for every diagnostic question:

  1. Resolve log_dir: Read .validator/config.yml and extract the log_dir field (default: validator_logs). All log paths below are relative to <log_dir>/.
  2. Passive evidence first: Read files before running commands.
    • <log_dir>/.debug.log — timestamped event log (commands, gate results, state changes, errors)
    • <log_dir>/.execution_state — JSON with last_run_completed_at, branch, commit, working_tree_ref, and unhealthy_adapters (adapter name → {marked_at, reason})
    • <log_dir>/console.*.log — console output per run (highest number = latest)
    • <log_dir>/check_*.log — check gate output
    • <log_dir>/review_*.json — review gate results with violations (file, line, issue, fix, priority, status)
    • .validator/config.yml — project configuration
  3. Active evidence when needed: Run CLI commands only when passive evidence is insufficient for a confident diagnosis.
  4. Explain with evidence: Clearly distinguish confirmed findings from inference.

Evidence Sources

SourceWhat It Confirms
.validator/config.ymllog_dir, base_branch, entry_points, cli.default_preference, max_retries, rerun_new_issue_threshold
<log_dir>/.debug.logTimestamped event history: commands executed, gate results, state transitions, errors
<log_dir>/.execution_stateLast successful run timestamp, branch/commit at that time, working tree stash ref, unhealthy adapter cooldowns
<log_dir>/console.*.logHuman-readable output from each run iteration
<log_dir>/check_*.logRaw output from check gate commands (linters, test runners, etc.)
<log_dir>/review_*.jsonStructured review violations with file, line, issue, priority, and resolution status
<log_dir>/.validator-run.lockLock file (contains PID) — present only during active execution
<log_dir>/.ci-wait-attemptsCI wait attempt counter

CLI Command Quick-Reference

Use these only when passive evidence is insufficient:

CommandWhen to Use
agent-validate listSee configured gates and entry points
agent-validate healthCheck adapter availability and health status
agent-validate detectSee which files changed and which gates would apply
agent-validate validateValidate config.yml syntax and schema
agent-validate cleanArchive current logs and reset state (destructive — confirm with user first)

Routing Logic

Based on the user's question, load the appropriate reference file for detailed guidance:

Question DomainReference File
Missing config, YAML errors, misconfiguration, init problemsreferences/config-troubleshooting.md
Check failures, review failures, no_changes, no_applicable_gates, rerun modereferences/gate-troubleshooting.md
Lock conflict, stale locks, parallel runs, cleanupreferences/lock-troubleshooting.md
Adapter health, missing tools, usage limits, cooldownreferences/adapter-troubleshooting.md

If the question spans multiple domains, load each relevant reference.

Output Contract

Every diagnostic response MUST include these sections:

Diagnosis

What happened and why, stated clearly.

Evidence

Specific files read, field values observed, and command outputs that support the diagnosis. Quote relevant log lines or config values.

Confidence

One of:

  • High — diagnosis is fully supported by direct evidence
  • Medium — diagnosis is likely but some evidence is missing or ambiguous
  • Low — diagnosis is inferred; key evidence is unavailable

Downgrade confidence when:

  • .debug.log or .execution_state is missing or empty
  • Log files referenced in output don't exist
  • Config values can't be verified
  • CLI commands fail or return unexpected output
Next Steps

Actionable recommendations for the user. If confidence is not high, suggest what additional evidence would confirm the diagnosis.

Bug Filing

After completing your diagnosis, apply the following routing logic to determine whether to file a GitHub issue:

  • High confidence + bug indicated (evidence points to a defect in agent-validator, not a configuration issue, user error, or expected behavior): Automatically invoke validator-issue with --auto-file <diagnosis summary> as arguments. This passes the diagnosis summary as the bug description and skips the interactive confirmation — the issue is filed immediately after showing the draft.
  • High confidence + not a bug (diagnosis concludes configuration issue, user error, or expected behavior): Do nothing. Do not invoke validator-issue.
  • Medium confidence + possible bug (evidence suggests a possible validator defect but is not conclusive): Ask the user: "This may be a validator bug. Want me to file a GitHub issue?" If the user confirms, invoke validator-issue with the diagnosis summary as the bug description. If the user declines, exit without filing.
  • Low confidence: Do nothing. Do not prompt the user and do not invoke validator-issue.

Comments

Markdown supported