Ollama Optimizer

Ollama Optimizer

Optimize Ollama configuration based on system hardware analysis.

When to Use

Use this skill when the user asks to optimize Ollama, configure Ollama, speed up Ollama, fix Ollama running slow, set up a local LLM, tune inference speed, reduce memory usage, or select models that fit their GPU/RAM. The skill analyzes hardware (GPU, VRAM, RAM, CPU) and produces tailored recommendations.

Do not use for LM Studio, llama.cpp, vLLM, or hosted-API LLM providers (OpenAI, Anthropic) — those use different runtimes and tuning surfaces.

Safety Rules

• Phases 1 and 2 are read-only. They run scripts/detect_system.py, ollama --version, ollama list, and ollama ps, and they change nothing.
• An applied change is any write to the machine: an env-var write, a service or app restart, ollama pull, a Modelfile ollama create, or any sudo command. Before each applied change, show its exact command and wait for the user's yes for that change.
• Saving the guide to the path the user chose is not an applied change.
• If the user asks for recommendations only, apply nothing and deliver the guide. A recommendations-only request is not a decline.
• Back up each config location before the first write to it (Choose where the env vars go).
• Never delete a model, and never run ollama rm. List oversized models in the guide instead.
• Never set OLLAMA_HOST=0.0.0.0 unless the user asks for network access. If they ask, warn that it exposes the Ollama API to the network without authentication.

Workflow

Fast path (opt-in only): only skip full hardware analysis if the user explicitly asks to. Otherwise always run Phases 1-4 and follow the tier-based recommendation — do not apply shortcuts by default, and do not let them override a tier decision already made. For the per-platform shortcut commands and env vars, see Platform-Specific Setup and Environment Variables.

Phase 1: System Detection

Run the detection script to gather hardware information:

python3 scripts/detect_system.py

The script prints JSON on stdout and exits 0. On an unexpected failure it prints Error: ... on stderr and exits 1. If it exits 1, show that message, follow its fix hint, and run it once more. If the second run also exits 1, stop with BLOCKED (Final Report).

Parse the JSON output to identify:

• OS and version
• CPU model and core count
• Total RAM / unified memory
• GPU type, VRAM, and driver version
• Current Ollama installation and environment variables
• hardware_tier — the script's computed category, max_model_size, and recommended_quant

If ollama.installed is false, continue to Phase 2. The plan then starts with the install step from Platform-Specific Setup. Installing Ollama is an applied change.

Phase 2: Analyze and Recommend

Use hardware_tier from Phase 1 as the tier decision. Do not re-derive it; the table below explains what each tier means and which optimizations it implies. Override the script only with an explicit reason (e.g. VRAM shared with a display), and state that reason in the report.

Unknown VRAM. The script tiers a GPU only from vram_gb. An AMD ROCm GPU, an NVIDIA GPU whose VRAM reads [N/A], and the Windows WMIC fallback report no vram_gb, so the script returns low_vram. An Intel Mac with a discrete GPU reports an empty gpu list, so the script returns cpu_only. In these cases, ask the user for the VRAM size. If they give it, override the tier from the table and state the reason. If they do not, keep the script's tier and list it under Uncertainty: as a conservative tier.

Hardware Tier Classification:

Apple Silicon Special Case:

• Unified memory = shared CPU/GPU RAM; the script tiers it directly from total unified memory
• 8GB Mac → entry
• 16GB Mac → prosumer
• 32GB Mac → workstation; 64GB+ Mac → high_end

Nothing to change. current_env_vars shows only the environment the script ran in. For Ollama.app or the systemd service, also read the values Ollama uses (launchctl getenv VAR, systemctl show ollama -p Environment). If every recommended env var already has the recommended value there and every installed model fits the tier, the guide says so and the run proposes no applied change.

Phase 3: Generate Optimization Plan

Read Report Templates, then create the guide with these sections:

1. System Overview

Present detected hardware specs and highlight constraints (e.g., "8GB unified memory limits to 8B models").

2. Dependency Assessment

List what's needed based on the platform:

• macOS: Ollama only (Metal automatic)
• Linux NVIDIA: Ollama + NVIDIA driver 450+
• Linux AMD: Ollama + ROCm 5.0+
• Windows: Ollama + NVIDIA driver 452+

3. Configuration Recommendations

Essential environment variables:

# Always recommended
export OLLAMA_FLASH_ATTENTION=1

# Memory-constrained systems (<12GB)
export OLLAMA_KV_CACHE_TYPE=q8_0  # or q4_0 for severe constraints

Set OLLAMA_KV_CACHE_TYPE when the GPU has under 12GB of VRAM or unified memory: q4_0 for low_vram, q8_0 otherwise. OLLAMA_KV_CACHE_TYPE requires OLLAMA_FLASH_ATTENTION=1.

Model selection guidance:

• Recommend specific models from ollama list output
• Suggest appropriate quantization (Q4_K_M default, Q5_K_M if headroom exists)
• Warn if current models exceed hardware capacity

Modelfile tuning (when needed):

PARAMETER num_gpu <layers>    # Partial offload for limited VRAM
PARAMETER num_thread <cores>  # CPU threads (physical cores, not hyperthreads)
PARAMETER num_ctx <size>      # Reduce context for memory savings

4. Choose where the env vars go

A shell init file reaches only an ollama serve started from that shell. Find how Ollama runs: on macOS, check for a running Ollama.app (pgrep -x Ollama); on Linux, run systemctl is-active ollama; on Windows, use the Windows row; if docker ps lists an Ollama container, use the Docker row. If the checks are inconclusive, ask the user. Then use the matching row; the commands are in Environment Variables → Setting Variables Permanently.

5. Execution Checklist

Provide copy-paste commands in order. Run a command only after the user approves it (Safety Rules):

1. Back up the config location from step 4 and write the env vars. For the shell init file ($SHELL decides: ~/.zshrc, ~/.bashrc, or ~/.bash_profile):

   RC=~/.zshrc  # or ~/.bashrc / ~/.bash_profile, matching $SHELL
   cp "$RC" "$RC.ollama-bak"
   printf '\n# ollama-optimizer start\nexport OLLAMA_FLASH_ATTENTION=1\n<KV cache + other export lines from section 3, per tier>\n# ollama-optimizer end\n' >> "$RC"

2. Restart Ollama the way the step 4 row runs it: restart ollama serve, quit and reopen Ollama.app, run sudo systemctl daemon-reload && sudo systemctl restart ollama, quit and reopen Ollama from the Windows taskbar, or recreate the container
3. Pull recommended models
4. Test with ollama run <model> --verbose
5. Rollback (one command, same location as step 1): for the shell init file, cp "$RC.ollama-bak" "$RC" — then restart Ollama. Other locations use the rollback column in step 4.

If an approved command exits non-zero, stop the checklist, show the error, and keep the backup. Offer the rollback command, and run it only after the user approves it.