---

name: test-thinking description: Lightweight reference for designing meaningful test variations.

A/B Test Mindset

Lightweight reference for designing meaningful test variations.

The goal in testing is to build strong variations that create signal. We don't necessarily want to isolate a single variable — we want to create distinct experiences that have a chance in breaking through the noise.

---

Variation Design Framework

Step 1: Identify the Mechanisms at our disposal.

Is the change noticable to users? (above the fold, in the first 5 seconds, different flow, etc.)

Are there usability insights we have discovered?

Is there a psychological lever we are pulling to increase motivation? Name it.

Step 2: Design Distinct Variants

Bad: Control vs slightly different version Good: Control vs fundamentally different approaches to the same goal

Example (Progress Bar):

• Control: No progress indicator
• Variant B: Explicit steps ("Step 2 of 5") — tests clarity/goal gradient
• Variant C: Visual bar only (no numbers) — tests progress without anchoring

Step 3: Validate Each Variant

For each variant, answer:

1. What evidence supports this approach?
2. Why might this win?
3. Why might this lose?

If you can't answer #1, it's not a real test — it's a guess.

---

Behavioral Science Quick Reference

Motivation & Goals

Prospect Theory

Social & Identity

Choice Architecture

Trust & Risk

---

Red Flags in Variation Design

---

Output Format

Use this structure when proposing variations:

Copy| Variant | What User Sees | Principle | Why It Might Win | Why It Might Lose |
|---------|----------------|-----------|------------------|-------------------|
| Control | [Current state] | Baseline | - | - |
| B | [Description] | [Named principle] | [Evidence/logic] | [Risk] |
| C | [Description] | [Named principle] | [Evidence/logic] | [Risk] |

Then ask: "Does this capture meaningful variation? Any variants to add or modify?" and "Do we have enough sample to test this many variatiants in a reasonable timeframe?"