---

name: validation-frameworks description: Problem and solution validation methodologies, assumption testing, and MVP validation experiments

Validation Frameworks

Frameworks for validating problems, solutions, and product assumptions before committing to full development.

When to Use This Skill

Auto-loaded by agents:

• research-ops - For problem/solution validation and assumption testing

Use when you need:

• Validating product ideas before building
• Testing assumptions and hypotheses
• De-risking product decisions
• Running MVP experiments
• Validating problem-solution fit
• Testing willingness to pay
• Evaluating technical feasibility

Core Principle: Reduce Risk Through Learning

Building the wrong thing is expensive. Validation reduces risk by answering critical questions before major investment:

• Problem validation: Is this a real problem worth solving?
• Solution validation: Does our solution actually solve the problem?
• Market validation: Will people pay for this?
• Usability validation: Can people use it?
• Technical validation: Can we build it?

The Validation Spectrum

Validation isn't binary (validated vs. not validated). It's a spectrum of confidence:

CopyWild Guess → Hypothesis → Validated Hypothesis → High Confidence → Proven

Early stage: Cheap, fast tests (low confidence gain) Later stage: More expensive tests (high confidence gain)

Example progression:

1. Assumption: "Busy parents struggle to plan healthy meals"
2. Interview 5 parents → Some validation (small sample)
3. Survey 100 parents → More validation (larger sample)
4. Prototype test with 20 parents → Strong validation (behavior observed)
5. Launch MVP, track engagement → Very strong validation (real usage)
6. Measure retention after 3 months → Proven (sustained behavior)

Problem Validation vs. Solution Validation

Problem Validation

Question: "Is this a problem worth solving?"

Goal: Confirm that:

• The problem exists
• It's painful enough that people want it solved
• It's common enough to matter
• Current solutions are inadequate

Methods:

• Customer interviews (discovery-focused)
• Ethnographic observation
• Surveys about pain points
• Data analysis (support tickets, analytics)
• Jobs-to-be-Done interviews

Red flags that stop here:

• No one cares about this problem
• Existing solutions work fine
• Problem only affects tiny niche
• Pain point is mild annoyance, not real pain

Output: Problem statement + evidence it's worth solving

See assets/problem-validation-canvas.md for a ready-to-use framework.

---

Solution Validation

Question: "Does our solution solve the problem?"

Goal: Confirm that:

• Our solution addresses the problem
• Users understand it
• Users would use it
• It's better than alternatives
• Value proposition resonates

Methods:

• Prototype testing
• Landing page tests
• Concierge MVP (manual solution)
• Wizard of Oz (fake backend)
• Pre-sales or waitlist signups

Red flags:

• Users don't understand the solution
• They prefer their current workaround
• Would use it but not pay for it
• Solves wrong part of the problem

Output: Validated solution concept + evidence of demand

See assets/solution-validation-checklist.md for validation steps.

---

The Assumption-Validation Cycle

1. Identify Assumptions

Every product idea rests on assumptions. Make them explicit.

Types of assumptions:

Desirability: Will people want this?

• "Users want to track their habits"
• "They'll pay $10/month for this"
• "They'll invite their friends"

Feasibility: Can we build this?

• "We can process data in under 1 second"
• "We can integrate with their existing tools"
• "Our team can build this in 3 months"

Viability: Should we build this?

• "Customer acquisition cost will be under $50"
• "Retention will be above 40% after 30 days"
• "We can reach 10k users in 12 months"

Best practice: Write assumptions as testable hypotheses

• Not: "Users need this feature"
• Yes: "At least 60% of users will use this feature weekly"

---

2. Prioritize Assumptions to Test

Not all assumptions are equally important. Prioritize by:

Risk: How uncertain are we? (Higher risk = higher priority) Impact: What happens if we're wrong? (Higher impact = higher priority)

Prioritization matrix:

Riskiest assumptions (test these first):

• Leap-of-faith assumptions the entire business depends on
• Things you've never done before
• Assumptions about user behavior with no data
• Technical feasibility of core functionality

Lower-risk assumptions (test later or assume):

• Based on prior experience
• Easy to change if wrong
• Industry best practices
• Minor features

---

3. Design Validation Experiments

For each assumption, design the cheapest test that could prove it wrong.

Experiment design principles:

1. Falsifiable: Can produce evidence that assumption is wrong 2. Specific: Clear success/failure criteria defined upfront 3. Minimal: Smallest effort needed to learn 4. Fast: Results in days/weeks, not months 5. Ethical: Don't mislead or harm users

The Experiment Canvas: See assets/validation-experiment-template.md

---

4. Run the Experiment

Before starting:

• Define success criteria ("At least 40% will click")
• Set sample size ("Test with 50 users")
• Decide timeframe ("Run for 1 week")
• Identify what success/failure would mean for product

During:

• Track metrics rigorously
• Document unexpected learnings
• Don't change experiment mid-flight

After:

• Analyze results honestly (avoid confirmation bias)
• Document what you learned
• Decide: Pivot, persevere, or iterate

---

Validation Methods by Fidelity

Low-Fidelity (Hours to Days)

1. Customer Interviews

• Cost: Very low (just time)
• Validates: Problem existence, pain level, current solutions
• Limitations: What people say ≠ what they do
• Best for: Early problem validation

2. Surveys

• Cost: Low
• Validates: Problem prevalence, feature preferences
• Limitations: Response bias, can't probe deeply
• Best for: Quantifying what you learned qualitatively

3. Landing Page Test

• Cost: Low (1-2 days to build)
• Validates: Interest in solution, value proposition clarity
• Measure: Email signups, clicks to "Get Started"
• Best for: Demand validation before building

4. Paper Prototypes

• Cost: Very low (sketch on paper/whiteboard)
• Validates: Concept understanding, workflow feasibility
• Limitations: Low realism
• Best for: Very early solution concepts

---

Medium-Fidelity (Days to Weeks)

5. Clickable Prototypes

• Cost: Medium (design tool, 2-5 days)
• Validates: User flow, interaction patterns, comprehension
• Tools: Figma, Sketch, Adobe XD
• Best for: Usability validation pre-development

6. Concierge MVP

• Cost: Medium (your time delivering manually)
• Validates: Value of outcome, willingness to use/pay
• Example: Manually curate recommendations before building algorithm
• Best for: Validating value before automation

7. Wizard of Oz MVP

• Cost: Medium (build facade, manual backend)
• Validates: User behavior, feature usage, workflows
• Example: "AI" feature that's actually humans behind the scenes
• Best for: Testing expensive-to-build features

---

High-Fidelity (Weeks to Months)

8. Functional Prototype

• Cost: High (weeks of development)
• Validates: Technical feasibility, actual usage patterns
• Limitations: Expensive if you're wrong
• Best for: After other validation, final pre-launch check

9. Private Beta

• Cost: High (full build + support)
• Validates: Real-world usage, retention, bugs
• Best for: Pre-launch final validation with early adopters

10. Public MVP

• Cost: Very high (full product)
• Validates: Product-market fit, business model viability
• Best for: After all other validation passed

---

Validation Metrics and Success Criteria

Quantitative Validation Metrics

Interest/Demand:

• Landing page conversion rate
• Waitlist signup rate
• Pre-order conversion
• Prototype test completion rate

Usage:

• Feature adoption rate
• Task completion rate
• Time on task
• Frequency of use

Engagement:

• DAU/MAU (daily/monthly active users)
• Session length
• Return rate
• Feature usage depth

Value Realization:

• Aha moment rate (reached key action)
• Time to value
• Retention curves
• Referral rate

Business:

• Conversion to paid
• Lifetime value (LTV)
• Customer acquisition cost (CAC)
• Churn rate

---

Qualitative Validation Signals

Strong positive signals:

• Users ask when they can buy it
• Users describe it to others excitedly
• Users find use cases you didn't imagine
• Users return without prompting
• Users complain when you take it away

Weak positive signals:

• "That's interesting"
• "I might use that"
• Polite feedback with no follow-up
• Theoretical interest but no commitment

Negative signals:

• Confusion about value proposition
• Preference for existing solution
• Feature requests that miss the point
• Can't articulate who would use it
• Ghosting after initial interest

---

Setting Success Criteria

Before running experiment, define what success looks like.

Framework: Set three thresholds

1. Strong success: Clear green light, proceed confidently
2. Moderate success: Promising but needs iteration
3. Failure: Pivot or abandon

Example: Landing page test

• Strong success: > 30% email signup rate
• Moderate success: 15-30% signup rate
• Failure: < 15% signup rate

Example: Prototype test

• Strong success: 8/10 users complete task, would use weekly
• Moderate success: 5-7/10 complete, would use monthly
• Failure: < 5/10 complete or no usage intent

Important: Decide criteria before seeing results to avoid bias.

---

Teresa Torres Continuous Discovery Validation

Opportunity Solution Trees

Map opportunities (user needs) to solutions to validate:

CopyOutcome
└── Opportunity 1
    ├── Solution A
    ├── Solution B
    └── Solution C
└── Opportunity 2
    └── Solution D

Validate each level:

1. Outcome: Is this the right goal?
2. Opportunities: Are these real user needs?
3. Solutions: Will this solution address the opportunity?

Assumption Testing

For each solution, map assumptions:

Desirability assumptions: Will users want this? Usability assumptions: Can users use this? Feasibility assumptions: Can we build this? Viability assumptions: Should we build this?

Then test riskiest assumptions first with smallest possible experiments.

Weekly Touchpoints

Continuous discovery = continuous validation:

• Weekly customer interviews (problem + solution validation)
• Weekly prototype tests with 2-3 users
• Weekly assumption tests (small experiments)

Goal: Continuous evidence flow, not one-time validation.

See references/lean-startup-validation.md and references/assumption-testing-methods.md for detailed methodologies.

---

Common Validation Anti-Patterns

1. Fake Validation

What it looks like:

• Asking friends and family (they'll lie to be nice)
• Leading questions ("Wouldn't you love...?")
• Testing with employees
• Cherry-picking positive feedback

Fix: Talk to real users, ask open-ended questions, seek disconfirming evidence.

---

2. Analysis Paralysis

What it looks like:

• Endless research without decisions
• Testing everything before building anything
• Demanding statistical significance with 3 data points

Fix: Accept uncertainty, set decision deadlines, bias toward action.

---

3. Confirmation Bias

What it looks like:

• Only hearing what confirms existing beliefs
• Dismissing negative feedback as "they don't get it"
• Stopping research when you hear what you wanted

Fix: Actively seek disconfirming evidence, set falsifiable criteria upfront.

---

4. Vanity Validation

What it looks like:

• "I got 500 email signups!" (but 0 conversions)
• "People loved the demo!" (but won't use it)
• "We got great feedback!" (but all feature requests, no usage)

Fix: Focus on behavior over opinions, retention over acquisition.

---

5. Building Instead of Validating

What it looks like:

• "Let's build it and see if anyone uses it"
• "It'll only take 2 weeks" (takes 2 months)
• Full build before any user contact

Fix: Always do cheapest possible test first, build only after validation.

---

Validation Workflow: End-to-End Example

Idea: Mobile app for tracking baby sleep patterns

Phase 1: Problem Validation

Assumption: "New parents struggle to understand their baby's sleep patterns"

Experiment 1: Customer Interviews

• Interview 10 new parents
• Ask about sleep tracking current methods, pain points
• Success criteria: 7/10 express frustration with current methods

Result: 9/10 keep notes on paper, all frustrated by inconsistency

Conclusion: Problem validated, proceed.

---

Phase 2: Solution Validation (Concept)

Assumption: "Parents would use an app that auto-tracks sleep from smart monitor"

Experiment 2: Landing Page

• Create page describing solution
• Run ads targeting new parents
• Success criteria: 20% email signup rate

Result: 8% signup rate

Conclusion: Interest is lower than hoped. Talk to non-signups to understand why.

Learning: They don't have smart monitors, manual entry would be fine.

---

Phase 3: Solution Validation (Refined)

Assumption: "Parents will manually log sleep times if it's fast"

Experiment 3: Concierge MVP

• Ask 5 parents to text sleep times
• Manually create charts/insights
• Send back daily summaries
• Success criteria: 4/5 use it for 2 weeks

Result: All 5 use it for 2 weeks, ask to keep using

Conclusion: Strong validation, build simple MVP.

---

Phase 4: Build & Validate MVP

Build: Simple app with manual time entry + basic charts

Experiment 4: Private Beta

• 30 parents use MVP for 4 weeks
• Success criteria: 60% retention after 2 weeks, 40% after 4 weeks

Result: 70% 2-week retention, 50% 4-week retention

Conclusion: Exceeded criteria, prepare for launch.

---

This workflow: problem → concept → refined solution → MVP took 8 weeks and cost minimal development before validation.

---

Validation Checklist by Stage

Idea Stage

• Problem validated through customer interviews
• Current solutions identified and evaluated
• Target user segment defined and accessible
• Pain level assessed (nice-to-have vs. must-have)

Concept Stage

• Solution concept tested with users
• Value proposition resonates
• Demand signal measured (signups, interest)
• Key assumptions identified and prioritized

Pre-Build Stage

• Prototype tested with target users
• Core workflow validated
• Pricing validated (willingness to pay)
• Technical feasibility confirmed

MVP Stage

• Beta users recruited
• Usage patterns observed
• Retention measured
• Unit economics validated

---

Ready-to-Use Resources

In assets/:

• problem-validation-canvas.md: Framework for validating problems before solutions
• solution-validation-checklist.md: Step-by-step checklist for solution validation
• validation-experiment-template.md: Design experiments to test assumptions

In references/:

• lean-startup-validation.md: Build-Measure-Learn cycle, MVP types, pivot decisions
• assumption-testing-methods.md: Comprehensive assumption testing techniques

---

Further Reading

Books:

• "The Lean Startup" by Eric Ries
• "The Mom Test" by Rob Fitzpatrick
• "Continuous Discovery Habits" by Teresa Torres
• "Testing Business Ideas" by Bland & Osterwalder

Key Concepts:

• Minimum Viable Product (MVP)
• Build-Measure-Learn
• Validated learning
• Pivot or persevere
• Assumption mapping
• Evidence-based product development

---

Remember: Validation isn't about proving you're right. It's about learning what's true. Seek evidence that you're wrong - that's the fastest path to building something valuable.