mcpField
A
ab-testing

ab-testing

github.com/gtmagentsMarketing & growth43069 installsUpdated 30w ago
Repository

About ab-testing

A marketing experimentation skill for testing messaging, segmentation, and campaign timing. It provides a structured process for setting hypotheses and metrics, sizing samples, running tests, and documenting results.

Key features

  1. 01Hypothesis and metric definition
  2. 02Sample-size planning
  3. 03Variant isolation and randomization
  4. 04Statistical or Bayesian analysis
  5. 05Experiment and debrief templates

Use cases

  1. 01Testing subject lines or creative
  2. 02Comparing persona and behavior segments
  3. 03Optimizing campaign timing and triggers

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/gtmagents/gtm-agents --skill ab-testing

After installing, type/ab-testingin Claude Code to confirm it loaded.

Details

From the project README

Experimentation & A/B Testing Skill

When to Use

  • Validating new subject lines or creative.
  • Testing segmentation hypotheses (persona vs behavior).
  • Optimizing cadence, timing, or automation triggers.

Framework

  1. Hypothesis – define expected uplift + rationale.
  2. Metric Selection – primary (open/click/conv) + guardrails (unsubs, spam).
  3. Sample Sizing – ensure stat significance (min 500 recipients per variant or use power calculator).
  4. Execution – randomize, keep variants isolated, limit simultaneous tests.
  5. Analysis – use z-test or Bayesian uplift; document learnings.

Templates

  • Experiment brief (hypothesis, segments, KPI, risk guardrails).
  • Variant table (control vs test inputs, creative asset links, owner).
  • Calculator sheet for minimum detectable effect + sample size.
  • Post-test debrief doc capturing learnings + rollout plan.

Experiment Ideas

  • Subject line vs preview text combos.
  • CTA placement (hero vs footer).
  • Personalization depth (basic vs dynamic modules).
  • Wait times between touches.

Tips

  • Run no more than two tests per journey simultaneously.
  • Recycle learnings into playbooks + automation templates.
  • Segment results by persona to catch hidden signals.

Comments

Markdown supported