← スキル一覧に戻る
Run smoke validation against

qa-acceptance
by therealityreport
⭐ 0🍴 0📅 2026年1月23日
SKILL.md
name: qa-acceptance description: Evaluate pipeline runs against ACCEPTANCE_MATRIX.md. Use after feature releases to verify acceptance criteria.
QA Acceptance Skill
Use this skill to validate features against acceptance criteria.
When to Use
- After implementing a new feature
- Before promoting from FEATURES/ to production
- Running acceptance checks in CI
- Generating QA reports for release
- Debugging why acceptance checks failed
Key Skills
Run smoke validation against ACCEPTANCE_MATRIX.md
Use the smoke runner to validate required artifacts + basic invariants (and optionally fail if anything is skipped).
# Minimal (will mark missing prereqs as skipped)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --max-frames 300
# Strict (fails if any requested stage is skipped/failed)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --max-frames 300 --strict
# Include body-tracking artifact checks (requires you ran FEATURES.body_tracking separately)
python -m tools.smoke.smoke_run --episode-id <EP_ID> --body-tracking --strict
Run targeted pytest checks
# Face alignment sandbox
pytest FEATURES/face_alignment/tests/test_face_alignment.py -v
# Face alignment integration helpers
pytest tests/integration/test_face_alignment_pipeline.py -v
# Body tracking sandbox
pytest FEATURES/body_tracking/tests/test_body_tracking.py -v
# TensorRT embedding sandbox
pytest FEATURES/arcface_tensorrt/tests/test_tensorrt_embedding.py -v
# ML-gated pipeline embedding invariants (requires RUN_ML_TESTS=1)
RUN_ML_TESTS=1 pytest tests/ml/test_arcface_embeddings.py -v
Acceptance Workflow
- Load thresholds from
ACCEPTANCE_MATRIX.md - Run relevant tests for the feature
- Compare metrics to targets
- Generate report with pass/fail/warn status
- Flag blockers that must be resolved
Feature Acceptance Criteria
Face Alignment (3.7)
| Metric | Target | Warning |
|---|---|---|
landmark_jitter_px | < 2.0 | > 5.0 |
alignment_quality_mean | >= 0.75 | < 0.60 |
pose_accuracy_degrees | <= 5 | > 10 |
Alignment Quality Gate (3.8)
| Metric | Target | Warning |
|---|---|---|
faces_gated_pct | 10-30% | > 50% |
false_rejection_rate | < 5% | > 10% |
3D Head Pose (3.9)
| Metric | Target | Warning |
|---|---|---|
pose_yaw_accuracy | <= 5 MAE | > 10 |
pose_pitch_accuracy | <= 5 MAE | > 10 |
Body Tracking (3.10)
| Metric | Target | Warning |
|---|---|---|
person_recall | >= 90% | < 80% |
body_track_fragmentation | < 0.15 | > 0.25 |
Person Re-ID (3.11)
| Metric | Target | Warning |
|---|---|---|
reid_mAP | >= 0.80 | < 0.70 |
reid_rank1_accuracy | >= 0.90 | < 0.80 |
Track Fusion (3.12)
| Metric | Target | Warning |
|---|---|---|
association_accuracy | >= 95% | < 90% |
screen_time_gap_reduction | >= 30% | < 15% |
TensorRT Embedding (3.14)
| Metric | Target | Warning |
|---|---|---|
speedup_vs_pytorch | >= 5x | < 3x |
cosine_sim_mean | >= 0.995 | < 0.990 |
cosine_sim_min | >= 0.990 | < 0.980 |
Face Mesh (3.15)
| Metric | Target | Warning |
|---|---|---|
mesh_stability_px | < 3.0 | > 6.0 |
visibility_fraction_accuracy | >= 90% | < 80% |
CenterFace (3.16)
| Metric | Target | Warning |
|---|---|---|
precision | >= 0.90 | < 0.85 |
recall | >= 0.85 | < 0.80 |
Report Format
# Acceptance Report: face_alignment
**Date:** 2025-12-11
**Episode:** test-episode-001
**Status:** PASS
## Metrics
| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| landmark_jitter_px | 1.5 | < 2.0 | PASS |
| alignment_quality_mean | 0.78 | >= 0.75 | PASS |
| pose_accuracy_degrees | 4.2 | <= 5 | PASS |
## Tests
- [x] FEATURES/face_alignment/tests/test_face_alignment.py - PASSED
- [x] tests/integration/test_face_alignment_pipeline.py - PASSED
## Notes
All acceptance criteria met. Ready for promotion.
CI Integration
Add to GitHub Actions:
- name: Smoke (fast)
run: |
python -m tools.smoke.smoke_run --episode-id $EP_ID --max-frames 300 --strict
Key Files
| File | Purpose |
|---|---|
ACCEPTANCE_MATRIX.md | Acceptance criteria |
tools/smoke/smoke_run.py | Smoke runner (artifact checks + invariants) |
tests/ml/ | ML-gated integration tests |
tests/integration/ | Integration tests |
Related Skills
- pipeline-insights - Pipeline debugging
- pipeline-debug - Job debugging
スコア
総合スコア
60/100
リポジトリの品質指標に基づく評価
✓SKILL.md
SKILL.mdファイルが含まれている
+20
✓LICENSE
ライセンスが設定されている
+10
○説明文
100文字以上の説明がある
0/10
○人気
GitHub Stars 100以上
0/15
○最近の活動
3ヶ月以内に更新がある
0/10
○フォーク
10回以上フォークされている
0/5
✓Issue管理
オープンIssueが50未満
+5
✓言語
プログラミング言語が設定されている
+5
○タグ
1つ以上のタグが設定されている
0/5
レビュー
💬
レビュー機能は近日公開予定です