Back to list
plaited

headless-adapters

by plaited

Evaluate AI agents with Unix-style pipeline commands. Schema-driven adapters for any CLI agent, trajectory capture, pass@k metrics, and multi-run comparison.

1🍴 0📅 Jan 24, 2026

SKILL.md


name: headless-adapters description: Discover, create, and validate headless adapters for agent integration. Includes scaffolding tools and schema-driven compliance testing. compatibility: Bun >= 1.2.9

Headless Adapters

Purpose

Schema-driven adapter for headless CLI agents. No code required - just define a JSON schema describing how to interact with the CLI.

Use CaseTool
Wrap headless CLI agentheadless command
Create new schemasSchema Creation Guide

Quick Start

  1. Check if a schema exists in schemas/
  2. Run the adapter:
    ANTHROPIC_API_KEY=... bunx @plaited/agent-eval-harness headless --schema .claude/skills/headless-adapters/schemas/claude-headless.json
    

CLI Commands

headless

Schema-driven adapter for ANY headless CLI agent.

bunx @plaited/agent-eval-harness headless --schema <path>

Options:

FlagDescriptionRequired
-s, --schemaPath to adapter schema (JSON)Yes

Schema Format:

{
  "version": 1,
  "name": "my-agent",
  "command": ["my-agent-cli"],
  "sessionMode": "stream",
  "prompt": { "flag": "-p" },
  "output": { "flag": "--output-format", "value": "stream-json" },
  "autoApprove": ["--allow-all"],
  "outputEvents": [
    {
      "match": { "path": "$.type", "value": "message" },
      "emitAs": "message",
      "extract": { "content": "$.text" }
    }
  ],
  "result": {
    "matchPath": "$.type",
    "matchValue": "result",
    "contentPath": "$.content"
  }
}

Session Modes:

ModeDescriptionUse When
streamKeep process alive, multi-turn via stdinCLI supports session resume
iterativeNew process per turn, accumulate historyCLI is stateless

Pre-built Schemas

Tested schemas are available in schemas/:

SchemaAgentModeAuth Env VarStatus
claude-headless.jsonClaude CodestreamANTHROPIC_API_KEYTested
gemini-headless.jsonGemini CLIiterativeGEMINI_API_KEYTested

Usage:

# Claude Code
ANTHROPIC_API_KEY=... bunx @plaited/agent-eval-harness headless --schema .claude/skills/headless-adapters/schemas/claude-headless.json

# Gemini CLI
GEMINI_API_KEY=... bunx @plaited/agent-eval-harness headless --schema .claude/skills/headless-adapters/schemas/gemini-headless.json

Creating a Schema

  1. Explore the CLI's --help to identify prompt, output, and auto-approve flags
  2. Capture sample JSON output from the CLI
  3. Map JSONPath patterns to output events
  4. Create schema file based on an existing template
  5. Test with headless command

See Schema Creation Guide for the complete workflow.

Troubleshooting

Common Issues

IssueLikely CauseSolution
Tool calls not capturedJSONPath not iterating arraysUse [*] wildcard syntax - see guide
"unexpected argument" errorStdin mode misconfiguredUse stdin: true - see guide
401 Authentication errorsAPI key not properly configuredSet the correct API key environment variable (see Pre-built Schemas table)
Timeout on promptJSONPath not matchingCapture raw CLI output, verify paths - see guide
Empty responsesContent extraction failingCheck extract paths - see guide

Complete troubleshooting documentation: Troubleshooting Guide

External Resources

Score

Total Score

75/100

Based on repository quality metrics

SKILL.md

SKILL.mdファイルが含まれている

+20
LICENSE

ライセンスが設定されている

+10
説明文

100文字以上の説明がある

+10
人気

GitHub Stars 100以上

0/15
最近の活動

1ヶ月以内に更新

+10
フォーク

10回以上フォークされている

0/5
Issue管理

オープンIssueが50未満

+5
言語

プログラミング言語が設定されている

+5
タグ

1つ以上のタグが設定されている

+5

Reviews

💬

Reviews coming soon