Back to list
therealityreport

qa-acceptance

by therealityreport

0🍴 0📅 Jan 23, 2026

SKILL.md


name: qa-acceptance description: Evaluate pipeline runs against ACCEPTANCE_MATRIX.md. Use after feature releases to verify acceptance criteria.

QA Acceptance Skill

Use this skill to validate features against acceptance criteria.

When to Use

  • After implementing a new feature
  • Before promoting from FEATURES/ to production
  • Running acceptance checks in CI
  • Generating QA reports for release
  • Debugging why acceptance checks failed

Key Skills

Run smoke validation against ACCEPTANCE_MATRIX.md

Use the smoke runner to validate required artifacts + basic invariants (and optionally fail if anything is skipped).

# Minimal (will mark missing prereqs as skipped)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --max-frames 300

# Strict (fails if any requested stage is skipped/failed)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --max-frames 300 --strict

# Include body-tracking artifact checks (requires you ran FEATURES.body_tracking separately)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --body-tracking --strict

Run targeted pytest checks

# Face alignment sandbox
pytest FEATURES/face_alignment/tests/test_face_alignment.py -v

# Face alignment integration helpers
pytest tests/integration/test_face_alignment_pipeline.py -v

# Body tracking sandbox
pytest FEATURES/body_tracking/tests/test_body_tracking.py -v

# TensorRT embedding sandbox
pytest FEATURES/arcface_tensorrt/tests/test_tensorrt_embedding.py -v

# ML-gated pipeline embedding invariants (requires RUN_ML_TESTS=1)
RUN_ML_TESTS=1 pytest tests/ml/test_arcface_embeddings.py -v

Acceptance Workflow

  1. Load thresholds from ACCEPTANCE_MATRIX.md
  2. Run relevant tests for the feature
  3. Compare metrics to targets
  4. Generate report with pass/fail/warn status
  5. Flag blockers that must be resolved

Feature Acceptance Criteria

Face Alignment (3.7)

MetricTargetWarning
landmark_jitter_px< 2.0> 5.0
alignment_quality_mean>= 0.75< 0.60
pose_accuracy_degrees<= 5> 10

Alignment Quality Gate (3.8)

MetricTargetWarning
faces_gated_pct10-30%> 50%
false_rejection_rate< 5%> 10%

3D Head Pose (3.9)

MetricTargetWarning
pose_yaw_accuracy<= 5 MAE> 10
pose_pitch_accuracy<= 5 MAE> 10

Body Tracking (3.10)

MetricTargetWarning
person_recall>= 90%< 80%
body_track_fragmentation< 0.15> 0.25

Person Re-ID (3.11)

MetricTargetWarning
reid_mAP>= 0.80< 0.70
reid_rank1_accuracy>= 0.90< 0.80

Track Fusion (3.12)

MetricTargetWarning
association_accuracy>= 95%< 90%
screen_time_gap_reduction>= 30%< 15%

TensorRT Embedding (3.14)

MetricTargetWarning
speedup_vs_pytorch>= 5x< 3x
cosine_sim_mean>= 0.995< 0.990
cosine_sim_min>= 0.990< 0.980

Face Mesh (3.15)

MetricTargetWarning
mesh_stability_px< 3.0> 6.0
visibility_fraction_accuracy>= 90%< 80%

CenterFace (3.16)

MetricTargetWarning
precision>= 0.90< 0.85
recall>= 0.85< 0.80

Report Format

# Acceptance Report: face_alignment

**Date:** 2025-12-11
**Episode:** test-episode-001
**Status:** PASS

## Metrics

| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| landmark_jitter_px | 1.5 | < 2.0 | PASS |
| alignment_quality_mean | 0.78 | >= 0.75 | PASS |
| pose_accuracy_degrees | 4.2 | <= 5 | PASS |

## Tests

- [x] FEATURES/face_alignment/tests/test_face_alignment.py - PASSED
- [x] tests/integration/test_face_alignment_pipeline.py - PASSED

## Notes

All acceptance criteria met. Ready for promotion.

CI Integration

Add to GitHub Actions:

- name: Smoke (fast)
  run: |
    python -m tools.smoke.smoke_run --episode-id $EP_ID --max-frames 300 --strict

Key Files

FilePurpose
ACCEPTANCE_MATRIX.mdAcceptance criteria
tools/smoke/smoke_run.pySmoke runner (artifact checks + invariants)
tests/ml/ML-gated integration tests
tests/integration/Integration tests

Score

Total Score

60/100

Based on repository quality metrics

SKILL.md

SKILL.mdファイルが含まれている

+20
LICENSE

ライセンスが設定されている

+10
説明文

100文字以上の説明がある

0/10
人気

GitHub Stars 100以上

0/15
最近の活動

3ヶ月以内に更新がある

0/10
フォーク

10回以上フォークされている

0/5
Issue管理

オープンIssueが50未満

+5
言語

プログラミング言語が設定されている

+5
タグ

1つ以上のタグが設定されている

0/5

Reviews

💬

Reviews coming soon