---

name: add-golden description: Curate and add documents to the golden dataset with multi-agent validation. Use when adding test data, creating golden datasets, saving examples. context: fork version: 1.0.0 author: OrchestKit tags: [curation, golden-dataset, evaluation, testing] user-invocable: true

Add to Golden Dataset

Multi-agent curation workflow for adding high-quality documents.

Quick Start

Copy/add-golden https://example.com/article
/add-golden https://arxiv.org/abs/2312.xxxxx

Phase 1: Input Collection

Get URL and detect content type:

• article (blog post, tech article)
• tutorial (step-by-step guide)
• documentation (API docs, reference)
• research_paper (academic, whitepaper)

Phase 2: Fetch and Extract

Extract document structure:

• Title and sections
• Code blocks
• Key technical terms
• Metadata (author, date)

Phase 3: Parallel Analysis (4 Agents)

Quality Dimensions

Difficulty Levels

• trivial: Direct keyword match (>0.85 score)
• easy: Common synonyms (>0.70 score)
• medium: Paraphrased intent (>0.55 score)
• hard: Multi-hop reasoning (>0.40 score)
• adversarial: Edge cases, robustness

Phase 4: Validation Checks

• URL validation (no placeholders)
• Schema validation (required fields)
• Duplicate check (>80% similarity)
• Quality gates (min sections, content length)

Phase 5: Decision Thresholds

Phase 6: User Approval

Present results for user decision:

• Approve: Add with generated queries
• Modify: Edit details before adding
• Reject: Do not add

Phase 7: Write to Dataset

Update fixture files:

• documents_expanded.json
• source_url_map.json
• queries.json

Validate fixture consistency after writing.

Summary

Total Parallel Agents: 4

• 1 code-quality-reviewer
• 3 Explore agents

Quality Gates:

• Minimum score: 0.55 for review
• No placeholder URLs
• No duplicates (>90% similar)
• At least 2 tags, 2 sections

Related Skills

• golden-dataset-validation - Validate existing golden datasets for quality and coverage
• llm-evaluation - LLM output evaluation patterns used in quality scoring
• test-data-management - General test data strategies and fixture management

Key Decisions

References

• Quality Scoring