mcpField
S
skill-eval-designer

skill-eval-designer

github.com/sylphxaiTesting & quality163 installsUpdated today
Repository

About skill-eval-designer

skill-eval-designer helps skill authors test whether a skill changes agent behavior on realistic tasks. It defines positive and negative prompts, behavioral assertions, artifact checks, and a forward-test plan.

Key features

  1. 01Defines behavior boundaries
  2. 02Creates positive and negative prompts
  3. 03Adds behavior assertions and artifact checks
  4. 04Covers edge cases and failure modes
  5. 05Produces fixtures and forward-test plans

Use cases

  1. 01Testing whether a skill triggers in the right situations
  2. 02Checking outputs for unsafe or generic behavior
  3. 03Evaluating skill behavior beyond static examples

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/sylphxai/skills --skill skill-eval-designer

After installing, type/skill-eval-designerin Claude Code to confirm it loaded.

Bundled filesagents/openai.yamlreferences/skill-eval-systems.md

Details

From the project README

Skill Eval Designer

Use this skill to prove a skill changes agent behavior on realistic tasks.

Workflow

  1. Identify skill purpose, trigger boundary, expected artifact, failure modes, and target agent behavior.
  2. Read references/skill-eval-systems.md.
  3. Create positive prompts, negative prompts, expected behavior assertions, and artifact checks.
  4. Add edge cases for over-triggering, under-triggering, unsafe advice, missing references, and generic output.
  5. Produce eval fixture, behavior example, and forward-test plan.

When not to use

  • Do not use to author the skill procedure itself, design a whole marketplace, or claim quality from static examples alone.
  • Do not create an eval when the behavior is fully deterministic and a unit, contract, property, or snapshot test is the stronger oracle.

Guardrails

  • Do not evaluate only whether the skill name appears.
  • Do not leak the expected answer into the prompt.
  • Include negative prompts so the skill does not trigger everywhere.

Output format

Skill under eval:
Behavior boundary:

Positive prompts:
- <prompt> -> expected behavior

Negative prompts:
- <prompt> -> why skill should not trigger

Assertions:
- <observable behavior>

Forward-test plan:
- task, artifact, pass/fail evidence

Comments

Markdown supported