---

name: alto-test-protocol description: Use when testing ALTO protocol and finding issues. Run protocol tests, analyze artifacts, find issues, classify, and optionally fix. Invoked via /alto-test-protocol in dev mode.

Protocol Test & Analysis

Test ALTO protocol, analyze results, find and fix issues.

Requirement: Run from ALTO source repo OR consumer project with url: path:../alto-2 in devenv.yaml.

Startup

Use AskUserQuestion:

• Header: "Mode"
• Question: "What would you like to do?"
• Options:
  1. Label: "Find issues", Description: "Run tests, analyze artifacts, identify protocol issues"
  2. Label: "Classify + suggest", Description: "Find issues, classify severity, suggest fixes"
  3. Label: "Full fix", Description: "Find, classify, create issues, implement fixes"

Step 1: Run Tests

CRITICAL:

• Use 10 minute timeout
• Do NOT run in background
• WAIT for complete output before proceeding to Step 2

Copydevenv shell -- alto-test-multi --all --json --keep

The command takes ~2-5 minutes for full suite. WAIT for it to finish completely.

Parse JSON from stdout. Structure:

Copy{
  "suite": true,
  "scenarios": 6,
  "passed": 4,
  "failed": 2,
  "total_turns": 16,
  "total_failures": 2,
  "results": [
    {
      "scenario": "setup-new-project",
      "success": true,
      "turns": 4,
      "failures": [],
      "test_dir": "/tmp/alto-multi-abc123"
    },
    {
      "scenario": "build-phase-transitions",
      "success": false,
      "turns": 3,
      "failures": [{"turn": "turn-2", "message": "phase_transition_valid failed"}],
      "test_dir": "/tmp/alto-multi-def456"
    }
  ]
}

Extract test_dir from each result in results[] array.

Step 2: Read Artifacts

Only proceed after Step 1 completes and you have the test_dir paths.

For each test_dir from JSON:

2a. Check directory exists

Copyls -la $TEST_DIR/runs/

2b. Read state.json

CopyRead $TEST_DIR/runs/state.json

Check for:

• Valid phases: ARCHITECTURE, PLANNING, IN_TASK, BETWEEN_TASKS, BLOCKED, COMPLETED
• completed_tasks grows as tasks finish
• current_task set during IN_TASK

2c. Read tool usage

CopyRead $TEST_DIR/runs/tools/usage.jsonl

Each line: {"tool": "Name", "params": {...}, "timestamp": "..."}

Check for:

• Expected tools called (AskUserQuestion for menus, Write for files, Task for agents)
• No forbidden tools for the phase
• Correct tool order

2d. Read handoffs

Copyls $TEST_DIR/runs/handoffs/

Then read each file found.

Required sections: ## Summary, ## Changes, ## Status

Step 3: Analyze

Evaluate against protocol rules:

Valid state transitions:

CopyNone → ARCHITECTURE, PLANNING, IN_TASK
ARCHITECTURE → PLANNING
PLANNING → IN_TASK, BLOCKED
IN_TASK → BETWEEN_TASKS, BLOCKED, COMPLETED
BETWEEN_TASKS → IN_TASK, PLANNING, COMPLETED
BLOCKED → PLANNING, IN_TASK

Issues to find (even if test passed):

• Invalid state transitions
• Missing/incomplete handoffs
• Wrong tools for phase
• Unnecessary tool calls
• Protocol steps skipped

Step 4: Report Issues

Copy## Issue 1: [scenario] - [summary]

**Scenario:** build-simple-feature
**Problem:** Handoff missing ## Changes section
**Evidence:** $TEST_DIR/runs/handoffs/task-001.md has ## Summary but no ## Changes

If Mode 1: STOP here.

Step 5: Classify

For each issue:

• Severity: Critical | High | Medium | Low
• Component: File responsible
• Root cause: Why it happens
• Suggested fix: Specific change

Check duplicates: gh issue list --repo gonzaloetjo/alto --label protocol-test --state open

If Mode 2: STOP here.

Step 6: Create Issues and Fix

Copygh issue create --repo gonzaloetjo/alto \
  --title "[protocol-test] component: summary" \
  --label "protocol-test,automated" \
  --body "..."

Then implement via /alto-self-fix workflow.