About paperclip-evals
A skill for selecting, interpreting, and running Paperclip evaluations and handling their evidence and history. It distinguishes Runner Evals from Product E2E Evals and emphasizes safe execution and evidence retention.
Key features
- 01Distinguish the two evaluation families
- 02Locate the repository and eval definitions
- 03Follow safe validation and run procedures
- 04Record evidence and run metadata
- 05Apply failure classification and projection limits
Use cases
- 01Choose a Paperclip evaluation family
- 02Interpret results and failure causes
- 03Prepare an authorized live evaluation run
Install
Copy the command and run it in a terminal.
npx skills add https:/After installing, type/paperclip-evalsin Claude Code to confirm it loaded.
Details
From the project READMEPaperclip evals
Use this skill when a request concerns Paperclip evaluation selection,
interpretation, evidence, history, or a live run. Read doc/evals.md in the
Paperclip repository first. It defines the two families and their boundaries.
Discover the repository
Do not assume the skill's installed location is inside a checkout. Locate the
repo explicitly with git rev-parse --show-toplevel from the current directory,
or inspect likely workspace roots and select the checkout containing
package.json, tests/runner-e2e, and packages/paperclip-runner. Locate the
private sibling paperclip-evals only when a Runner Eval needs its definitions;
use an explicit PAPERCLIP_EVALS_ROOT or a discovered sibling checkout. Never
invent a relative path from this copied skill into the repository.
Route the request
Choose Runner Evals for real runner/provider protocol behavior against the
mock control plane. Authoritative details are in
packages/paperclip-runner/docs/runner-protocol-live-evals.md and the sibling
paperclip-evals/evals/paperclip-runner definitions.
Choose Product E2E Evals for real browser/server/database/runner/provider
workflows, including local and Daytona environments. Read
tests/runner-e2e/README.md, then FIXTURES.md, SECURITY.md, or
EVERYDAY-WORKFLOWS.md as relevant. Everyday Workflows remain Product E2E
even when imported into Evalbook. “Headless” is a browser mode, not a family.
Work safely
Start with read-only catalog inspection and credential-free validation. For
Product E2E use pnpm test:e2e:runner:typecheck,
pnpm test:e2e:runner:unit, and pnpm test:e2e:runner -- --list; run one
explicit cell only when the user has authorized a live/paid run and the needed
credentials and immutable Daytona image are configured. For Runner Evals use
the pinned eval revision and the documented workflow/CLI. Never use a partial
selector as evidence of full coverage.
Keep source revisions, definition/catalog fingerprints, model/profile, environment, selected cells, retries, timing, usage/cost coverage, and grader version attached to every interpretation. Preserve partial attempts and classify failures as product, model/provider behavior, grading/evidence, or infrastructure from the observed failure and supported cause. A usable completed behavior failure is not infrastructure; missing provider/profile, transport, startup, or evidence requires examining the evidence before choosing the cause.
Use the existing family generator and viewer. Public projections may contain
sanitized fixture conversation and allowlisted tool outcomes/evidence; follow
the family's projection and publisher checks. Do not expose raw trusted
artifacts, credentials, secrets, private data, provider session IDs, or hidden
reasoning. A refresh from retained evidence has zero provider calls and remains
the original measurement with a new presentation. Link the public histories
and hub from doc/evals.md when reporting results.
For adding a case or fixture, use the narrower add-runner-eval or add-product-e2e-eval skill.