mcpField
E
experiment-design-kit

experiment-design-kit

github.com/gtmagentsMarketing & growth42865 installsUpdated 30w ago
Repository

About experiment-design-kit

An experiment-planning skill for teams turning ideas into testable hypotheses and operational briefs. It covers measurement, variants, rollout, and review, with templates for experiment packets and governance.

Key features

  1. 01Problem framing and hypothesis format
  2. 02Metrics, guardrails, and power planning
  3. 03Control, variant, and targeting strategy
  4. 04Owners, dependencies, QA, and rollback plan
  5. 05Experiment briefs and weekly packets
  6. 06Prioritization and learning-loop guidance

Use cases

  1. 01Turning growth ideas into testable experiments
  2. 02Preparing measurement and variant plans
  3. 03Coordinating experiment launch and rollback steps
  4. 04Sharing weekly results and upcoming launch dependencies

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/gtmagents/gtm-agents --skill experiment-design-kit

After installing, type/experiment-design-kitin Claude Code to confirm it loaded.

Details

From the project README

Experiment Design Kit Skill

When to Use

  • Translating raw ideas into testable hypotheses with clear success metrics.
  • Ensuring experiment briefs include guardrails, instrumentation, and rollout details.
  • Coaching pods on best practices for multi-variant or multi-surface tests.

Framework

  1. Problem Framing – define user problem, business impact, and north-star metric.
  2. Hypothesis Structure – "If we do X for Y persona, we expect Z change" with assumptions.
  3. Measurement Plan – primary metric, guardrails, min detectable effect, power calc.
  4. Variant Strategy – control definition, variant catalog, targeting, and exclusion rules.
  5. Operational Plan – owners, timeline, dependencies, QA/rollback steps.

Templates

  • Experiment brief (context, hypothesis, design, metrics, launch checklist).
  • Guardrail register with thresholds + alerting rules.
  • Variant matrix for surfaces, messaging, and states.
  • GTM Agents Growth Backlog Board – capture idea → sizing → prioritization scoring (ICE/RICE) @puerto/README.md#183-212.
  • Weekly Experiment Packet – includes KPI guardrails, qualitative notes, and next bets for Marketing Director + Sales Director.
  • Rollback Playbook – pre-built checklist tied to lifecycle-mapping rip-cord procedures.

Tips

  • Pressure-test hypotheses with counter-metrics to avoid local optima.
  • Document data constraints early to avoid rework during build.
  • Pair with guardrail-scorecard to ensure sign-off before launch.
  • Apply GTM Agents cadence: Monday backlog groom, Wednesday build review, Friday learnings sync.
  • Require KPI guardrails per stage (activation, engagement, monetization) before authorizing build.
  • If a test risks Sales velocity, include Sales Director in approval routing per GTM Agents governance.

GTM Agents Experiment Operating Model

  1. Backlog Intake – ideas flow from GTM pods; Growth Marketer tags theme, objective, expected impact.
  2. Prioritization – score with RICE + qualitative "strategic fit" modifier; surface top 3 bets weekly.
  3. Design & Instrumentation – reference Serena/Context7 to patch code + confirm documentation.
  4. Launch & Monitor – use guardrail-scorecard to watch leading indicators (churn, complaints, latency).
  5. Learning Loop – run Sequential Thinking retro; document hypothesis, result, decision, follow-up in backlog card.

KPI Guardrails (GTM Agents Reference)

  • Activation rate change must stay within ±3% of baseline for Tier-1 segments.
  • Revenue per visitor cannot drop more than 2% for more than 48h.
  • Support tickets tied to experiment variant must remain <5% of total volume.

Weekly Experiment Packet Outline

Week Ending: <Date>

1. Portfolio Snapshot – tests live, status, KPI trend (guardrail vs actual)
2. Key Wins – hypothesis, uplift, next action (ship, iterate, expand)
3. Guardrail Alerts – what tripped, mitigation taken (rollback? scope adjust?)
4. Pipeline Impact – SQLs, ARR influenced, notable customer anecdotes
5. Upcoming Launches – dependencies, owners, open questions

Share packet with Growth, Marketing Director, Sales Director, and RevOps to mirror GTM Agents's cross-functional communication rhythm.

Comments

Markdown supported