---

name: gpu-cost-optimizer description: "Analyze GPU workloads and recommend optimal configurations. Get cost estimates, compare GPU options, avoid over-provisioning, and find the cheapest GPU that meets your requirements."

GPU Cost Optimizer

Spend less, compute more.

This skill analyzes workloads and recommends the most cost-effective GPU configuration. Stop guessing and overpaying.

When to Use This Skill

GPU Database (January 2025 Prices)

Consumer/Prosumer GPUs

Professional GPUs

Datacenter GPUs

VRAM Requirements Guide

LLM Inference (Chat/Completion)

Formula: VRAM (GB) = Parameters (B) × Precision (bytes) × 1.2 (overhead)

• FP16: 2 bytes/param
• INT8: 1 byte/param
• INT4: 0.5 bytes/param

LLM Training

Formula: Full FT = Model × 8 (weights + gradients + optimizer states)

Image Models

Video Generation

Decision Trees

LLM Inference Selection

CopyWhat model size?
├── 7-8B (Llama 3.1 8B, Mistral 7B)
│   └── INT4 quantization?
│       ├── Yes → RTX 4070 Ti (12GB) @ $0.25/hr
│       └── No → RTX 4090 (24GB) @ $0.44/hr ✓ Best value
│
├── 13B
│   └── INT4 quantization?
│       ├── Yes → RTX 4090 (24GB) @ $0.44/hr
│       └── No → RTX A6000 (48GB) @ $0.80/hr
│
├── 34B (CodeLlama 34B, etc.)
│   └── RTX A6000 or L40 (48GB) @ $0.80-0.90/hr
│
├── 70B (Llama 3.1 70B)
│   └── Quantization?
│       ├── INT4 → A100 40GB @ $1.29/hr
│       └── FP16 → 2× A100 80GB @ $3.58/hr
│
└── 405B (Llama 3.1 405B)
    └── INT4 → 4× A100 80GB or 2× H200 @ $7-10/hr

Training Selection

CopyWhat are you training?
├── Image LoRA (SDXL)
│   └── RTX 4090 (24GB) @ $0.44/hr ✓
│
├── Image LoRA (FLUX)
│   └── A100 80GB @ $1.79/hr
│
├── LLM Fine-tune (7B)
│   └── Method?
│       ├── QLoRA → RTX 4090 @ $0.44/hr ✓
│       ├── LoRA → A100 40GB @ $1.29/hr
│       └── Full FT → A100 80GB @ $1.79/hr
│
├── LLM Fine-tune (70B)
│   └── Method?
│       ├── QLoRA → A100 80GB @ $1.79/hr
│       ├── LoRA → 2× A100 80GB @ $3.58/hr
│       └── Full FT → 8× A100 80GB @ $14.32/hr
│
└── Custom PyTorch
    └── Profile first, then select

Cost Optimization Strategies

1. Use the Smallest GPU That Works

Bad: Requesting A100 80GB for a task that fits on RTX 4090

Copy// Don't do this
{"gpu_type": "A100 PCIe 80GB"}  // $1.79/hr for SD 1.5? Wasteful!

Good: Match GPU to workload

Copy// Do this
{"gpu_type": "RTX 4090", "min_vram": 12}  // $0.44/hr, plenty for SD 1.5

2. Use Quantization

INT4 quantization reduces VRAM by 4× with minimal quality loss for inference.

Copy# Instead of loading full precision
model = AutoModelForCausalLM.from_pretrained("model")

# Use 4-bit quantization
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(
    "model",
    quantization_config=quantization_config
)

Savings: Llama 70B on 1× A100 ($1.79/hr) instead of 2× A100 ($3.58/hr)

3. Enable Memory Optimizations

Copy# Gradient checkpointing (training)
model.gradient_checkpointing_enable()

# CPU offloading (inference)
pipe.enable_model_cpu_offload()

# VAE tiling (image generation)
pipe.vae.enable_tiling()

# Attention slicing
pipe.enable_attention_slicing()

4. Batch Processing

Process multiple items per GPU session instead of one-at-a-time:

Copy# Bad: Start pod for each image
for image in images:
    gpu_run(f"process {image}")  # Pod startup overhead each time

# Good: Batch process
gpu_run("process_batch images/")  # One pod startup, many images

5. Use Cooldown Wisely

Copy// For interactive work (many short commands)
{"cooldown_minutes": 10}  // Pod stays warm

// For batch jobs
{"cooldown_minutes": 5}   // Stop quickly when done

// For API servers
{"cooldown_minutes": 30}  // Keep serving longer

6. Profile Before Committing

Test with small batch first:

Copy# Test with subset
gpu run python train.py --max-steps 100

# Check VRAM usage
nvidia-smi  # In the output

# Then decide GPU size

Configuration Review Checklist

When reviewing a gpu.jsonc, check:

Cost Estimation Formulas

Total Session Cost

CopyCost = (GPU $/hr) × (Hours Active) + (Download Time × GPU $/hr)

Examples:
- RTX 4090, 2 hours: $0.44 × 2 = $0.88
- A100 80GB, 4 hours: $1.79 × 4 = $7.16
- 2× H100, 8 hours: $6.38 × 8 = $51.04

Training Cost Estimate

CopyHours = (Dataset Size × Epochs × Time per Sample) / 3600

Examples:
- QLoRA 7B, 10K samples, 3 epochs: ~3 hours on RTX 4090 = $1.32
- Full FT 7B, 10K samples, 3 epochs: ~24 hours on A100 = $42.96

Inference Cost per 1K Tokens

CopyCost per 1K tokens = (GPU $/hr) / (Tokens/hr / 1000)

Examples (vLLM):
- Llama 8B on RTX 4090: $0.44/hr ÷ 360K/hr = $0.0012/1K tokens
- Llama 70B on 2× A100: $3.58/hr ÷ 108K/hr = $0.033/1K tokens

Common Mistakes

Over-Provisioning

Under-Provisioning

GPU Comparison Tool

Compare Options for Your Task

Task: Run Llama 3.1 70B for inference

Recommendation: Option B for cost, Option C for speed

Output Format

When analyzing GPU requirements:

Copy## GPU Analysis for [Task]

### Requirements

- **Model size**: [X GB]
- **VRAM needed**: [X GB] ([precision])
- **Task type**: [inference/training]

### Recommended Configuration

```jsonc
{
  "gpu_type": "[GPU]",
  "min_vram": [X],
  // ... rest of config
}

Cost Estimate

Alternatives

Optimization Tips

1. [Tip 1]
2. [Tip 2]
3. [Tip 3]

Copy