---

name: model-router description: Build cost-optimized chat applications with Azure OpenAI Model Router that intelligently routes queries to the most appropriate model based on complexity, achieving up to 60% cost savings.

Model Router Skill

Folder Contents

---

Build cost-optimized chat applications with Azure OpenAI Model Router that intelligently routes queries to the most appropriate model based on complexity, achieving up to 60% cost savings.

Overview

Model Router is a deployable AI chat model in Azure OpenAI that dynamically selects the best underlying LLM for each prompt in real-time. It analyzes query complexity, cost, and performance requirements to route requests optimally.

Key Benefits

• Cost Optimization: Up to 60% savings by routing simple queries to cheaper models
• Transparent Routing: API response reveals which model was selected
• No Routing Overhead: Billed only for underlying model usage
• 18 Supported Models: GPT-5, GPT-4.1, o4-mini, Claude, DeepSeek, Llama, Grok families

Routing Modes

Supported Models (2025-11-18 Version)

Quick Start

Environment Variables

CopyAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_API_KEY=your-api-key
MODEL_ROUTER_DEPLOYMENT=model-router
AZURE_OPENAI_API_VERSION=2024-12-01-preview

Basic API Call

Copyfrom openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version=os.environ.get("AZURE_OPENAI_API_VERSION", "2024-12-01-preview")
)

response = client.chat.completions.create(
    model="model-router",  # Deployment name
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is 2+2?"}
    ],
    max_tokens=512,
    temperature=0.7
)

# Response reveals which model was selected
print(f"Model used: {response.model}")  # e.g., "gpt-4.1-nano-2025-04-14"
print(f"Tokens: {response.usage.total_tokens}")

Extract Token Usage

Copy# Full usage details
usage = response.usage
print(f"Prompt tokens: {usage.prompt_tokens}")
print(f"Completion tokens: {usage.completion_tokens}")
print(f"Total tokens: {usage.total_tokens}")

# Detailed breakdowns (when available)
if hasattr(usage, 'completion_tokens_details') and usage.completion_tokens_details:
    print(f"Reasoning tokens: {usage.completion_tokens_details.reasoning_tokens}")
if hasattr(usage, 'prompt_tokens_details') and usage.prompt_tokens_details:
    print(f"Cached tokens: {usage.prompt_tokens_details.cached_tokens}")

Cost Calculation

Pricing Per Million Tokens (January 2026)

CopyPRICING_PER_MILLION = {
    # GPT-5 Series
    "gpt-5": {"input": 1.25, "output": 10.00},
    "gpt-5-mini": {"input": 0.25, "output": 2.00},
    "gpt-5-nano": {"input": 0.05, "output": 0.40},
    # GPT-4.1 Series
    "gpt-4.1": {"input": 2.00, "output": 8.00},
    "gpt-4.1-mini": {"input": 0.40, "output": 1.60},
    "gpt-4.1-nano": {"input": 0.10, "output": 0.40},
    # Reasoning
    "o4-mini": {"input": 1.10, "output": 4.40},
    # Third-party
    "deepseek": {"input": 0.14, "output": 0.28},
    "llama": {"input": 0.20, "output": 0.40},
    "grok": {"input": 3.00, "output": 15.00},
}

Cost Calculation Function

Copydef calculate_cost(model: str, prompt_tokens: int, completion_tokens: int) -> float:
    """Calculate cost for a request based on model and tokens."""
    model_lower = model.lower()
    prices = PRICING_PER_MILLION.get(model_lower, {"input": 2.00, "output": 8.00})

    for key, p in PRICING_PER_MILLION.items():
        if key in model_lower:
            prices = p
            break

    input_cost = (prompt_tokens / 1_000_000) * prices["input"]
    output_cost = (completion_tokens / 1_000_000) * prices["output"]
    return input_cost + output_cost

Deployment

Deploy via Azure Portal

1. Navigate to Azure AI Foundry → Model Catalog
2. Search for "model-router"
3. Select deployment type:
   • Default settings: Balanced mode, all models
   • Custom settings: Choose routing mode and model subset
4. Select region: East US 2 or Sweden Central
5. Configure tokens-per-minute (TPM) rate limit

Deploy via Azure CLI

Copy# Create Model Router deployment
az cognitiveservices account deployment create \
  --resource-group $RESOURCE_GROUP \
  --name $OPENAI_RESOURCE \
  --deployment-name model-router \
  --model-name model-router \
  --model-version 2025-11-18 \
  --model-format OpenAI \
  --sku-capacity 250 \
  --sku-name Standard

Important Notes

Parameter Handling

• Temperature/Top_P: Ignored for reasoning models (o-series)
• reasoning_effort: Supported in 2025-11-18+ for o-series models
• stop, presence_penalty, frequency_penalty: Dropped for o-series

Routing Behavior

• Simple queries (factual, arithmetic) → nano/mini models
• Medium complexity (explanations) → mini models
• Complex queries (code, reasoning) → full models or o4-mini
• Changes to routing mode take up to 5 minutes to take effect

Monitoring

• Performance: Azure Monitor → Metrics → Filter by deployment
• Costs: Resource Management → Cost Analysis → Filter by deployment tag

Building Blocks

This skill includes the following building block scripts:

• scripts/model_router_client.py: Core client for Model Router API
• scripts/cost_tracker.py: Token usage and cost tracking
• scripts/routing_analyzer.py: Analyze routing decisions across queries

Lessons Learned (January 2026 Implementation)

Model Naming Convention

The API returns model names with version dates appended:

• gpt-5-nano-2025-08-07 (not just gpt-5-nano)
• gpt-5-mini-2025-08-07
• gpt-5-2025-08-07

Tip: Use substring matching for cost calculation, not exact key lookup:

Copyfor key, prices in PRICING_PER_MILLION.items():
    if key in model_name.lower():
        return prices

Routing Distribution (Real-World Test)

Testing with 28 varied queries showed:

This confirms significant cost savings - ~97% of queries routed to cheaper models!

Cost Display Best Practices

Avoid ambiguous abbreviations:

• ❌ $2.59m - Could mean "million" or "milli"
• ✅ $2.59×10⁻³ - Clear scientific notation
• ✅ $0.00259 - Full decimal
• ✅ ~0.3¢ - Cents for small values

Routing Behavior Observations

1. Not deterministic: Same query type may route to different models on different calls
2. Complexity assessment: Router analyzes prompt structure, not just length
3. Code queries: Consistently routed to higher-tier models (gpt-5, gpt-5-mini)
4. Factual queries: Usually routed to nano tier

Implementation Recommendations

1. Backend API: Keep Azure credentials server-side, never expose in frontend
2. Batch testing: Include feature to run multiple queries and show distribution
3. Session stats: Track cumulative usage for cost visibility
4. Model tier colors: Visual differentiation helps users understand routing decisions
5. Savings comparison: Show "cost if always using gpt-5" vs actual cost

References

• Model Router How-To
• Model Router Concepts
• What's New in Model Router
• GitHub: guygregory/ModelRouter
• Build 2025 Session BRK178