Back to list
5dlabs

property-based-testing

by 5dlabs

Cognitive Task Orchestrator - GitOps on Bare Metal or Cloud for AI Agents

2🍴 1📅 Jan 25, 2026

SKILL.md


name: property-based-testing description: Property-based testing for stronger coverage than example tests - roundtrip, idempotence, invariants.

Property-Based Testing Guide

Use this skill when you encounter patterns where PBT provides stronger coverage than example-based tests.

When to Invoke (Automatic Detection)

Invoke this skill when you detect:

  • Serialization pairs: encode/decode, serialize/deserialize, toJSON/fromJSON
  • Parsers: URL parsing, config parsing, protocol parsing
  • Normalization: normalize, sanitize, clean, canonicalize
  • Validators: is_valid, validate, check_*
  • Data structures: Custom collections with add/remove/get operations
  • Mathematical/algorithmic: Pure functions, sorting, ordering, comparators

When NOT to Use

  • Simple CRUD operations without transformation logic
  • One-off scripts or throwaway code
  • Code with side effects that cannot be isolated
  • Tests where specific example cases are sufficient
  • Integration or end-to-end testing

Property Catalog (Quick Reference)

PropertyFormulaWhen to Use
Roundtripdecode(encode(x)) == xSerialization, conversion pairs
Idempotencef(f(x)) == f(x)Normalization, formatting, sorting
InvariantProperty holds before/afterAny transformation
Commutativityf(a, b) == f(b, a)Binary/set operations
Associativityf(f(a,b), c) == f(a, f(b,c))Combining operations
Identityf(x, identity) == xOperations with neutral element
Inversef(g(x)) == xencrypt/decrypt, compress/decompress
Oraclenew_impl(x) == reference(x)Optimization, refactoring
Easy to Verifyis_sorted(sort(x))Complex algorithms
No ExceptionNo crash on valid inputBaseline property

Strength hierarchy (weakest to strongest): No Exception → Type Preservation → Invariant → Idempotence → Roundtrip

Priority by Pattern

PatternPropertyPriority
encode/decode pairRoundtripHIGH
Pure functionMultipleHIGH
ValidatorValid after normalizeMEDIUM
Sorting/orderingIdempotence + orderingMEDIUM
NormalizationIdempotenceMEDIUM
Builder/factoryOutput invariantsLOW

Examples by Language

Python (Hypothesis)

from hypothesis import given, strategies as st

@given(st.binary())
def test_roundtrip(data):
    assert decode(encode(data)) == data

@given(st.text())
def test_normalize_idempotent(s):
    assert normalize(normalize(s)) == normalize(s)

@given(st.lists(st.integers()))
def test_sort_idempotent(xs):
    assert sorted(sorted(xs)) == sorted(xs)

TypeScript (fast-check)

import fc from 'fast-check';

test('roundtrip', () => {
  fc.assert(fc.property(fc.uint8Array(), (data) => {
    expect(decode(encode(data))).toEqual(data);
  }));
});

test('normalize is idempotent', () => {
  fc.assert(fc.property(fc.string(), (s) => {
    expect(normalize(normalize(s))).toBe(normalize(s));
  }));
});

Rust (proptest)

use proptest::prelude::*;

proptest! {
    #[test]
    fn roundtrip(data: Vec<u8>) {
        prop_assert_eq!(decode(&encode(&data)), data);
    }
    
    #[test]
    fn normalize_idempotent(s: String) {
        let once = normalize(&s);
        let twice = normalize(&once);
        prop_assert_eq!(once, twice);
    }
}

Go (rapid)

func TestRoundtrip(t *testing.T) {
    rapid.Check(t, func(t *rapid.T) {
        data := rapid.SliceOf(rapid.Byte()).Draw(t, "data")
        decoded := Decode(Encode(data))
        if !bytes.Equal(decoded, data) {
            t.Fatalf("roundtrip failed")
        }
    })
}

How to Suggest PBT

When you detect a high-value pattern while writing tests, offer PBT as an option:

"I notice encode_message/decode_message is a serialization pair. Property-based testing with a roundtrip property would provide stronger coverage than example tests. Want me to use that approach?"

If codebase already uses a PBT library (Hypothesis, fast-check, proptest), be more direct:

"This codebase uses Hypothesis. I'll write property-based tests for this serialization pair using a roundtrip property."

If user declines, write good example-based tests without further prompting.

Red Flags

  • Recommending trivial getters/setters
  • Missing paired operations (encode without decode)
  • Ignoring type hints (well-typed = easier to test)
  • Overwhelming user with candidates (limit to top 5-10)
  • Being pushy after user declines

PBT Libraries by Language

LanguageLibraryInstall
PythonHypothesispip install hypothesis
TypeScript/JSfast-checknpm install fast-check
Rustproptestcargo add proptest --dev
Gorapidgo get pgregory.net/rapid
HaskellQuickCheckcabal install QuickCheck
ScalaScalaChecksbt dependency
ElixirStreamDatamix dependency

Attribution

Based on trailofbits/skills property-based-testing skill.

Score

Total Score

65/100

Based on repository quality metrics

SKILL.md

SKILL.mdファイルが含まれている

+20
LICENSE

ライセンスが設定されている

+10
説明文

100文字以上の説明がある

0/10
人気

GitHub Stars 100以上

0/15
最近の活動

1ヶ月以内に更新

+10
フォーク

10回以上フォークされている

0/5
Issue管理

オープンIssueが50未満

+5
言語

プログラミング言語が設定されている

+5
タグ

1つ以上のタグが設定されている

+5

Reviews

💬

Reviews coming soon