---

name: localai description: | LocalAI local inference API management via Podman Quadlet. Provides an OpenAI-compatible API for local model inference with GPU acceleration. Use when users need to configure, start, or manage the LocalAI service.

LocalAI - Local AI Inference API

Overview

The localai command manages the LocalAI service using Podman Quadlet containers. It provides an OpenAI-compatible API for running AI models locally with GPU acceleration.

Key Features:

• OpenAI-compatible API endpoints
• GPU-specific container images (auto-selected)
• Multiple GPU support (NVIDIA, AMD, Intel)
• Cross-pod DNS via bazzite-ai network

Quick Reference

Parameters

GPU-Specific Images

LocalAI uses different container images optimized for each GPU type:

The appropriate image is automatically selected based on detected GPU hardware.

Configuration

Copy# Default configuration (auto-detects GPU, port 8080)
ujust localai config

# Custom port (long form)
ujust localai config --port=8081

# Custom port (short form)
ujust localai config -p 8081

# Network-wide access
ujust localai config --bind=0.0.0.0

# Force CPU image (ignore GPU)
ujust localai config --image=localai/localai:latest

# Combine parameters (long form)
ujust localai config --port=8081 --bind=0.0.0.0

# Combine parameters (short form)
ujust localai config -p 8081 -b 0.0.0.0

Update Existing Configuration

Running config when already configured updates the existing settings:

Copy# Change only the bind address
ujust localai config --bind=0.0.0.0

# Update port without affecting other settings
ujust localai config --port=8082

Lifecycle Management

Copy# Start LocalAI
ujust localai start

# Stop service
ujust localai stop

# Restart (apply config changes)
ujust localai restart

# View logs (default 50 lines)
ujust localai logs

# View more logs (long form)
ujust localai logs --lines=200

# View more logs (short form)
ujust localai logs -l 200

# Check status
ujust localai status

# Show API URL
ujust localai url

Multi-Instance Support

Copy# Start all instances (long form)
ujust localai start --instance=all

# Start all instances (short form)
ujust localai start -n all

# Stop specific instance
ujust localai stop --instance=2

# Delete all instances
ujust localai delete --instance=all

Shell Access

Copy# Interactive shell
ujust localai shell

# Run specific command (use -- separator)
ujust localai shell -- ls -la /models
ujust localai shell -- nvidia-smi

Network Architecture

LocalAI uses the bazzite-ai bridge network for cross-container DNS:

Copy+-------------------+     DNS      +-------------------+
|   Open WebUI      | -----------> |     LocalAI       |
|   (openwebui)     |              |    (localai)      |
|   Port 3000       |              |   Port 8080       |
+-------------------+              +-------------------+
         |                                  |
         +------ bazzite-ai network --------+
                         |
+-------------------+    |    +-------------------+
|     Ollama        |----+----+     Jupyter       |
|    (ollama)       |         |    (jupyter)      |
|   Port 11434      |         |   Port 8888       |
+-------------------+         +-------------------+

Cross-Pod DNS:

• LocalAI accessible as http://localai:8080 from other containers
• Can replace Ollama as backend for OpenWebUI

API Endpoints (OpenAI-Compatible)

Example API Usage

Copy# List models
curl http://localhost:8080/v1/models

# Chat completion
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Model Storage

Models persist across container restarts. Each instance has isolated storage.

Loading Models

Place model files (GGUF, GGML) in the models directory:

Copy# Copy a model
cp my-model.gguf ~/.config/localai/1/models/

# Or download directly
curl -L -o ~/.config/localai/1/models/model.gguf \
  https://huggingface.co/.../model.gguf

Common Workflows

Initial Setup

Copy# 1. Configure LocalAI (auto-detects GPU)
ujust localai config

# 2. Start the service
ujust localai start

# 3. Check the API
ujust localai url
# Output: http://127.0.0.1:8080

# 4. Test the API
curl http://localhost:8080/v1/models

Use with OpenWebUI

OpenWebUI can use LocalAI as an OpenAI-compatible backend:

Copy# Start LocalAI
ujust localai start

# In OpenWebUI settings, add connection:
# URL: http://localai:8080/v1  (cross-pod DNS)
# Or: http://host.containers.internal:8080/v1  (from host)

Remote Access Setup

Copy# Configure for network access
ujust localai config --bind=0.0.0.0

# Start the service
ujust localai start

# Or use Tailscale for secure access
ujust tailscale serve --service=localai

GPU Support

GPU is automatically detected and the appropriate image is selected:

Check GPU in Container

Copy# NVIDIA
ujust localai shell -- nvidia-smi

# Check GPU environment
ujust localai shell -- env | grep -i gpu

Troubleshooting

Service Won't Start

Copy# Check status
ujust localai status

# View logs
ujust localai logs --lines=100

# Check image was pulled
podman images | grep localai

Common causes:

• Port 8080 already in use
• Container image not pulled
• GPU driver issues

GPU Not Detected

NVIDIA:

Copy# Check CDI configuration
nvidia-ctk cdi list

# Regenerate CDI spec
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

AMD:

Copy# Check /dev/kfd exists
ls -la /dev/kfd

# Check ROCm
rocminfo

API Errors

Copy# Test API endpoint
curl http://localhost:8080/v1/models

# Check logs for errors
ujust localai logs --lines=100

Clear Data and Start Fresh

Copy# Delete everything
ujust localai delete --instance=all

# Reconfigure
ujust localai config
ujust localai start

Cross-References

• Network peers: ollama, openwebui, jupyter, comfyui (all use bazzite-ai network)
• Alternative: ollama (simpler model management, different API)
• Client: openwebui (can use LocalAI as backend)
• Docs: LocalAI Documentation

When to Use This Skill

Use when the user asks about:

• "install localai", "setup local inference", "openai-compatible api"
• "configure localai", "change port", "gpu acceleration"
• "localai not working", "api error", "model loading"
• "localai logs", "debug localai"
• "delete localai", "uninstall"