---

name: mlflow-evaluation description: "MLflow 3 GenAI evaluation for agent development. Use when (1) writing mlflow.genai.evaluate() code, (2) creating @scorer functions, (3) building evaluation datasets from traces, (4) using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), (5) analyzing traces for latency/errors/architecture, (6) optimizing agent context/prompts/token usage, (7) debugging evaluation failures. Covers the full eval workflow: trace analysis -> dataset building -> scorer creation -> evaluation execution."

MLflow 3 GenAI Evaluation

Before Writing Any Code

1. Read GOTCHAS.md - 15+ common mistakes that cause failures
2. Read CRITICAL-interfaces.md - Exact API signatures and data schemas

End-to-End Workflows

Follow these workflows based on your goal. Each step indicates which reference files to read.

Workflow 1: First-Time Evaluation Setup

For users new to MLflow GenAI evaluation or setting up evaluation for a new agent.

Workflow 2: Production Trace -> Evaluation Dataset

For building evaluation datasets from production traces.

Workflow 3: Performance Optimization

For debugging slow or expensive agent execution.

Workflow 4: Regression Detection

For comparing agent versions and finding regressions.

Workflow 5: Custom Scorer Development

For creating project-specific evaluation metrics.

Reference Files Quick Lookup

Critical API Facts

• Use: mlflow.genai.evaluate() (NOT mlflow.evaluate())
• Data format: {"inputs": {"query": "..."}} (nested structure required)
• predict_fn: Receives **unpacked kwargs (not a dict)

See GOTCHAS.md for complete list.