R
representation-probe-designer
About representation-probe-designer
A skill for designing probes that test what a model representation encodes, rather than only inspecting outputs. It specifies controls and interpretation limits to help distinguish competing hypotheses.
Key features
- 01Define the target representation and property
- 02Select minimal probe methods
- 03Add controls for trivial cues and leakage
- 04Compare against negative and shuffled baselines
- 05Set interpretation limits
Use cases
- 01Testing whether an embedding captures a target property
- 02Diagnosing suspected representation collapse
- 03Designing controlled evidence for a paper or debugging
Install
Copy the command and run it in a terminal.
$
npx skills add https:/After installing, type/representation-probe-designerin Claude Code to confirm it loaded.
Details
From the project READMERepresentation Probe Designer
Probe structure, not just outputs.
Intake
Clarify:
- which layer/embedding/state is being probed;
- what property should be represented;
- available labels or proxy labels;
- suspected failure mode;
- whether the probe is for diagnosis, paper evidence, or debugging.
Workflow
- Define the representation and target property.
- Choose minimal probes: linear probe, nearest-neighbor, clustering, retrieval, or collapse diagnostic.
- Add controls for trivial cues.
- Choose negative controls and random/label-shuffled baselines.
- Define what would count as meaningful structure.
- Recommend the smallest informative probe set.
Rules
- A probe is evidence, not full explanation.
- Control for capacity and leakage.
- Prefer probes that distinguish two hypotheses over probes that merely look interesting.
- Do not overinterpret a high-capacity probe as proof the base representation contains a simple feature.
Output
Return:
- target representation and property;
- probe plan;
- controls;
- interpretation rules;
- failure modes and limits;
- smallest next diagnostic.
Comments
Markdown supported