Back to list
alexanderop

mutation-testing

by alexanderop

9🍴 0📅 Jan 19, 2026

SKILL.md


name: mutation-testing description: | Mutation testing patterns for verifying test effectiveness. Use when analyzing branch code to find weak or missing tests. Triggers: "mutation testing", "test effectiveness", "would tests catch this bug", "weak tests", "are my tests good enough", "surviving mutants".

Mutation Testing

Mutation testing answers: "Would my tests catch this bug?" by actually introducing bugs and running tests.


Execution Workflow

CRITICAL: This skill actually mutates code and runs tests. Follow this exact process:

Step 1: Identify Target Code

# Get changed files on the branch
git diff main...HEAD --name-only | grep -E '\.(ts|js|tsx|jsx|vue)$' | grep -v '\.test\.' | grep -v '\.spec\.'

Step 2: For Each Function to Test

Execute this loop for each mutation:

1. READ the original file and note exact content
2. APPLY one mutation (edit the code)
3. RUN tests: pnpm test --run (or specific test file)
4. RECORD result: KILLED (test failed) or SURVIVED (test passed)
5. RESTORE original code immediately
6. Repeat for next mutation

Step 3: Report Results

After all mutations, provide a summary table:

| Mutation | Location | Result | Action Needed |
|----------|----------|--------|---------------|
| `>` → `>=` | file.ts:42 | SURVIVED | Add boundary test |
| `&&` → `||` | file.ts:58 | KILLED | None |

Mutation Operators to Apply

Priority 1: Boundary Mutations (Most Likely to Survive)

OriginalMutate ToWhy It Matters
<<=Boundary not tested
>>=Boundary not tested
<=<Equality case missed
>=>Equality case missed

Priority 2: Boolean Logic Mutations

OriginalMutate ToWhy It Matters
&&||Only tested when both true
||&&Only tested when both false
!conditionconditionNegation not verified

Priority 3: Arithmetic Mutations

OriginalMutate ToWhy It Matters
+-Tested with 0 only
-+Tested with 0 only
*/Tested with 1 only

Priority 4: Return/Early Exit Mutations

OriginalMutate ToWhy It Matters
return xreturn nullReturn value not asserted
return truereturn falseBoolean return not checked
if (cond) return// removedEarly exit not tested

Priority 5: Statement Removal

OriginalMutate ToWhy It Matters
array.push(x)// removedSide effect not verified
await save(x)// removedAsync operation not verified
emit('event')// removedEvent emission not tested

Practical Execution Example

Example: Testing a Validation Function

Original code (src/utils/validation.ts:15):

export function isValidAge(age: number): boolean {
  return age >= 18 && age <= 120;
}

Mutation 1: Change >= to >

export function isValidAge(age: number): boolean {
  return age > 18 && age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests PASS → SURVIVED (Bad! Need test for isValidAge(18))

Restore original code immediately

Mutation 2: Change && to ||

export function isValidAge(age: number): boolean {
  return age >= 18 || age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests FAIL → KILLED (Good! Tests catch this bug)

Restore original code immediately


Results Interpretation

Mutant States

StateMeaningAction
KILLEDTest failed with mutantTests are effective
SURVIVEDTests passed with mutantAdd or strengthen test
TIMEOUTTests hung (infinite loop)Counts as detected

Mutation Score

Score = (Killed + Timeout) / Total Mutations * 100
ScoreQuality
< 60%Weak - significant test gaps
60-80%Moderate - improvements needed
80-90%Good - minor gaps
> 90%Strong test suite

Fixing Surviving Mutants

When a mutant survives, add a test that would catch it:

Surviving: Boundary mutation (>=>)

// Add boundary test
it('accepts exactly 18 years old', () => {
  expect(isValidAge(18)).toBe(true);  // Would fail if >= became >
});

Surviving: Logic mutation (&&||)

// Add test with mixed conditions
it('rejects when only one condition met', () => {
  expect(isValidAge(15)).toBe(false);  // Would pass if && became ||
});

Surviving: Statement removal

// Add side effect verification
it('saves to database', async () => {
  await processOrder(order);
  expect(db.save).toHaveBeenCalledWith(order);  // Would fail if save removed
});

Quick Checklist During Mutation

For each mutation, ask:

  1. Before mutating: Does a test exist for this code path?
  2. After running tests: Did any test actually fail?
  3. If survived: What specific test would catch this?
  4. After fixing: Re-run mutation to confirm killed

Common Surviving Mutation Patterns

Tests Only Check Happy Path

// WEAK: Only tests success case
it('validates', () => {
  expect(validate(goodInput)).toBe(true);
});

// STRONG: Tests both cases
it('validates good input', () => {
  expect(validate(goodInput)).toBe(true);
});
it('rejects bad input', () => {
  expect(validate(badInput)).toBe(false);
});

Tests Use Identity Values

// WEAK: Mutation survives
expect(multiply(5, 1)).toBe(5);  // 5*1 = 5/1 = 5

// STRONG: Mutation detected
expect(multiply(5, 3)).toBe(15);  // 5*3 ≠ 5/3

Tests Don't Assert Return Values

// WEAK: No return value check
it('processes', () => {
  process(data);  // No assertion!
});

// STRONG: Asserts outcome
it('processes', () => {
  const result = process(data);
  expect(result).toEqual(expected);
});

Important Rules

  1. ALWAYS restore original code after each mutation
  2. Run tests immediately after applying mutation
  3. One mutation at a time - don't combine mutations
  4. Focus on changed code - prioritize branch diff
  5. Track all results - report full mutation summary

Summary Report Template

After completing mutation testing, provide:

## Mutation Testing Results

**Target**: `src/features/workout/utils.ts` (functions: X, Y, Z)
**Total Mutations**: 12
**Killed**: 9
**Survived**: 3
**Score**: 75%

### Surviving Mutants (Action Required)

| # | Location | Original | Mutated | Suggested Test |
|---|----------|----------|---------|----------------|
| 1 | line 42 | `>=` | `>` | Test boundary value |
| 2 | line 58 | `&&` | `\|\|` | Test mixed conditions |
| 3 | line 71 | `emit()` | removed | Verify event emission |

### Killed Mutants (Tests Effective)

- Line 35: `+` → `-` killed by `calculation.test.ts`
- Line 48: `true` → `false` killed by `validate.test.ts`
- ...

  • systematic-debugging - Root cause analysis
  • testing-conventions - Query priority, expect.poll()
  • vue-integration-testing - Page objects, browser mode
  • vitest-mocking - Test doubles and mocking patterns

Score

Total Score

50/100

Based on repository quality metrics

SKILL.md

SKILL.mdファイルが含まれている

+20
LICENSE

ライセンスが設定されている

0/10
説明文

100文字以上の説明がある

0/10
人気

GitHub Stars 100以上

0/15
最近の活動

3ヶ月以内に更新がある

0/10
フォーク

10回以上フォークされている

0/5
Issue管理

オープンIssueが50未満

+5
言語

プログラミング言語が設定されている

+5
タグ

1つ以上のタグが設定されている

0/5

Reviews

💬

Reviews coming soon