Ollama in 2026: Install, Run and Serve Local LLMs (Linux, Windows, Mac)

What you get in one minute#

Ollama is a free program that runs large language models on your own computer and serves them through one HTTP API on port 11434.

Your prompts stay on your machine. There is no per-token bill, and it works offline once a model is downloaded. Apps that speak the OpenAI or Anthropic API can talk to it, so you can point tools like Claude Code at a local model.

At a glance:

• You get: a local model server, a chat in your terminal, and an API other apps can use.
• Time needed: 10 to 20 minutes, plus the model download.
• Difficulty: easy for the native install, medium for Docker with a GPU.
• You need: 8 GB RAM or more, 10 GB of free disk or more, and optionally a GPU.

Version choice: I checked the issue tracker and the release page. v0.40.11 has only positive reactions, and no install failures or data-loss waves are reported. The open problems are tied to specific hardware (some Radeon 890M, Intel Arc iGPU and macOS cases, see Troubleshooting). So this guide uses the latest release. Forums and Reddit could not be read in this check.

How it fits together#

Your apps send requests to the Ollama server, which loads a model from disk and computes the answer on your GPU or CPU.

Not sure which install fits you? Follow the arrows.

Before you start#

You need a 64-bit computer, free disk for models, and either the native installer or Docker.

Check your tools (only needed for the Docker route):

docker --version
docker compose version

Checklist:

• [ ] Port 11434 is free on this machine
• [ ] You have 10 GB or more free disk (big models need tens of GB)
• [ ] Existing ~/.ollama models folder backed up (see Updates and backups)
• [ ] Only for Traefik: a domain, DNS and ports 80 and 443

Checked against the official docs on 2026-10-26. The commands below were not run by DIY Lab Hub; Linux, Windows and Mac are all taken from the current docs.

Step 1: Install Ollama#

Pick one route: native install (simplest) or Docker (best for servers).

Linux#

Native install with the official script, then check the version:

curl -fsSL https://ollama.com/install.sh | sh
ollama -v

The script uses sudo right after the download, so stay at the keyboard. AMD GPUs also need the extra ROCm package:

curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst | sudo tar x -C /usr

The service logs are in journalctl -e -u ollama.

Windows (PowerShell)#

Download OllamaSetup.exe from ollama.com/download and run it. It installs for your user only and starts a tray app plus the server. Optional: choose another install folder.

.\OllamaSetup.exe /DIR="D:\Ollama"
ollama -v

Mac (Terminal)#

Download ollama.dmg from ollama.com/download, open it, drag Ollama to Applications and approve the command-line link. Then:

ollama -v

Check it worked: ollama -v prints a version, and curl http://localhost:11434 answers Ollama is running.

On Mac and Windows the desktop app starts the server at login and updates itself. Click the tray or menu icon and choose "Restart to update".

Step 2: Pull and run your first model#

Download a small model, chat with it, and check where it runs.

Linux#

ollama pull llama3.2
ollama run llama3.2 "Say hello in five words"
ollama ps

Windows (PowerShell)#

ollama pull llama3.2
ollama run llama3.2 "Say hello in five words"
ollama ps

Mac (Terminal)#

Same commands as Linux:

ollama pull llama3.2
ollama run llama3.2 "Say hello in five words"
ollama ps

Check it worked: ollama ps has a PROCESSOR column. 100% GPU is what you want. 100% CPU works but is slower.

Everyday commands:

Step 3: Run Ollama in Docker (optional, for servers)#

Docker keeps the install tidy and pins the version. Data goes under ~/docker_volumes/ollama.

On Mac skip this step: Docker there has no GPU, so use the native app.

Linux#

Create the folder and the Compose file (no version: line, it is obsolete):

mkdir -p ~/docker_volumes/ollama/data
cd ~/docker_volumes/ollama
nano docker-compose.yml

services:
  ollama:
    image: ollama/ollama:0.40.11      # official image; AMD: 0.40.11-rocm
    container_name: ollama
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"       # this machine only
    volumes:
      - ./data:/root/.ollama
    environment:
      - OLLAMA_KEEP_ALIVE=10m
      - OLLAMA_CONTEXT_LENGTH=8192
    deploy:                            # NVIDIA only; delete these lines for CPU
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

For NVIDIA, install the NVIDIA Container Toolkit first, then:

sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker
docker compose up -d
docker exec -it ollama ollama pull llama3.2

For AMD, drop the deploy: block and add devices: ["/dev/kfd:/dev/kfd", "/dev/dri:/dev/dri"] with the -rocm image tag. The GPU block is standard Compose syntax, not taken from the Ollama docs (VERIFY on your hardware).

The official one-liner without Compose is docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:0.40.11.

Windows (PowerShell)#

Needs Docker Desktop with WSL2. Use the same Compose file as above.

mkdir $HOME\docker_volumes\ollama\data
cd $HOME\docker_volumes\ollama
notepad docker-compose.yml
docker compose up -d
docker exec -it ollama ollama run llama3.2

Mac (Terminal)#

Prefer the native app. If you still want Docker for a CPU-only test, the commands are the same as Linux (without the deploy: block):

mkdir -p ~/docker_volumes/ollama/data
cd ~/docker_volumes/ollama
docker compose up -d

Check it worked: docker compose ps shows ollama as running, and curl http://localhost:11434 answers Ollama is running.

Step 4: Change the settings that matter#

Ollama is configured with environment variables. Set them once and restart.

Linux#

Native service: edit it, add the variable under [Service], then restart.

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_KEEP_ALIVE=10m"

sudo systemctl daemon-reload
sudo systemctl restart ollama

In Docker, put the variable under environment: in the Compose file and run docker compose up -d.

Windows (PowerShell)#

Quit Ollama from the tray, then set a user variable and start it again:

[Environment]::SetEnvironmentVariable("OLLAMA_KEEP_ALIVE", "10m", "User")

Start Ollama from the Start menu afterwards.

Mac (Terminal)#

launchctl setenv OLLAMA_KEEP_ALIVE "10m"

Then quit and reopen the Ollama app.

Warning: OLLAMA_HOST=0.0.0.0 opens the API to your whole network and Ollama has no login. Read the security section before you do it.

Step 5: Use the API#

The API listens on port 11434. Ollama also copies the OpenAI and Anthropic styles, so many tools work unchanged.

Linux#

curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Why is the sky blue?","stream":false}'
curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"llama3.2","messages":[{"role":"user","content":"Hello"}]}'

Windows (PowerShell)#

(Invoke-WebRequest -Method POST -Body '{"model":"llama3.2","prompt":"Hi","stream":false}' -Uri http://localhost:11434/api/generate).Content | ConvertFrom-Json

Mac (Terminal)#

Same commands as Linux:

curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Why is the sky blue?","stream":false}'

Check it worked: you get a JSON answer with a response field. That text comes from the docs' format, your wording will differ.

Use Claude Code with a local model#

Point Claude Code at Ollama with two variables and a model you pulled first (ollama pull qwen3-coder):

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coder

Ollama also has a helper, ollama launch claude, for the same job. The exact flags may change between releases, so check ollama launch --help (VERIFY). If a tool hard-codes a Claude model name, ollama cp qwen3-coder claude-3-5-sonnet creates an alias. Coding tools need a long context, so raise it (see the Modelfile below).

Step 6: Make your own model with a Modelfile#

A Modelfile saves a base model plus your settings under a new name. This is also how you give OpenAI-style clients a bigger context, because their API has no field for it.

Linux#

cat > Modelfile <<'EOF'
FROM llama3.2
PARAMETER num_ctx 8192
PARAMETER temperature 0.4
SYSTEM You are a concise assistant.
EOF
ollama create mymodel -f Modelfile
ollama run mymodel

Windows (PowerShell)#

@'
FROM llama3.2
PARAMETER num_ctx 8192
PARAMETER temperature 0.4
SYSTEM You are a concise assistant.
'@ | Set-Content -Encoding ascii Modelfile
ollama create mymodel -f Modelfile
ollama run mymodel

Mac (Terminal)#

Same commands as Linux:

ollama create mymodel -f Modelfile

Check it worked: ollama list shows mymodel.

Reading model names: llama3.2:3b is the family plus size, and suffixes like q4_0 or q8_0 are the quantization (smaller number = smaller file and lower quality). latest means the default tag.

Put it behind HTTPS (with or without Traefik)#

Already run Traefik? Use the first route. No Traefik yet and one small server? Use Caddy.

With Traefik#

This section is only for a server with a domain. If you use Ollama on one computer, skip it.

Ollama has no login. So the Traefik example adds basic auth. Create the hash with htpasswd -nb <user> <password> and put it in a .env file as OLLAMA_BASIC_AUTH (escape each $ as $$).

docker network create proxy

services:
  ollama:
    image: ollama/ollama:0.40.11
    restart: unless-stopped
    volumes:
      - ./data:/root/.ollama
    networks: [proxy]
    labels:
      - traefik.enable=true
      - traefik.http.routers.ollama.rule=Host(`ollama.<your-domain>`)
      - traefik.http.routers.ollama.entrypoints=websecure
      - traefik.http.routers.ollama.tls.certresolver=letsencrypt
      - traefik.http.routers.ollama.middlewares=ollama-auth
      - traefik.http.middlewares.ollama-auth.basicauth.users=${OLLAMA_BASIC_AUTH}
      - traefik.http.services.ollama.loadbalancer.server.port=11434
networks:
  proxy:
    external: true

The entrypoint and resolver names must match your own Traefik setup (VERIFY). This is standard Traefik syntax, not from the Ollama docs. Add the deploy: GPU block from Step 3 if you use an NVIDIA card.

Without Traefik: Caddy#

Choose this when you have no Traefik yet and run one small server. Caddy is one program with a short config, and it handles certificates itself (see caddyserver.com/docs, Automatic HTTPS).

Caddyfile for Ollama, public domain:

ollama.example.com {
    basic_auth {
        <user> <HASH>
    }
    reverse_proxy localhost:11434
}

Caddyfile for Ollama, home only:

ollama.home.arpa {
    tls internal
    basic_auth {
        <user> <HASH>
    }
    reverse_proxy localhost:11434
}

In this guide Ollama listens on port 11434, published on 127.0.0.1, which is where Caddy reaches it when Caddy runs on the same machine. Ollama has no login, so the Caddyfile adds basic auth. Create the hash with caddy hash-password and paste it in place of <HASH>.

Run Caddy

• Linux: install from the official apt repository (caddyserver.com/docs/install), put the Caddyfile at /etc/caddy/Caddyfile, then sudo systemctl reload caddy.
• Windows (PowerShell): winget install CaddyServer.Caddy, then in the folder with your Caddyfile run caddy run.
• Mac: brew install caddy, then in the folder with your Caddyfile run caddy run.
• Docker: add this service to the same docker-compose.yml, so Caddy shares the Docker network with Ollama. Ollama installed natively (not in Docker) cannot be reached from a container on its loopback address, so for a native Ollama use the Linux, Windows or Mac route above.

  caddy:
    image: caddy:2.11.7
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ~/docker_volumes/caddy/data:/data
      - ~/docker_volumes/caddy/config:/config

In the Docker route write reverse_proxy ollama:11434 in the Caddyfile instead of localhost:11434. If only Caddy should reach Ollama, remove its 127.0.0.1 port publish.

Home only: trust Caddy's root certificate (the tls internal certificates are signed by it).

• Caddy machine: caddy trust.
• Linux device: copy pki/authorities/local/root.crt from Caddy's data directory, save it as /usr/local/share/ca-certificates/caddy-root.crt, then sudo update-ca-certificates.
• Windows device: copy root.crt, then in an administrator PowerShell certutil -addstore -f ROOT root.crt.
• Mac device: copy root.crt, then sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain root.crt.

Make ollama.home.arpa resolve to the server (your router's DNS, or each device's hosts file).

Check it worked: curl -I -u <user>:<password> https://ollama.example.com answers HTTP/2 401 without a password and HTTP/2 200 with the password.

Security and testing#

Keep Ollama off the internet. The API has no authentication.

• Never port-forward 11434. For remote use, go through a VPN such as WireGuard or Tailscale, or a reverse proxy with auth.
• Bind to localhost. The default is already 127.0.0.1. The Compose file above also publishes the port on 127.0.0.1 only.
• Secrets in .env. Run chmod 600 .env. Never write keys into the Compose file. This also covers OLLAMA_API_KEY if you use cloud models.
• Your key pair. ~/.ollama/id_ed25519 is private. Back it up privately and never share it.
• Browser access. Allow extra web origins only with OLLAMA_ORIGINS.
• Stay fully local. Set OLLAMA_NO_CLOUD=1 if you never want cloud models or web search.

Tests:

curl http://localhost:11434
docker compose ps
docker logs ollama --tail 100
curl -I https://ollama.<your-domain>

The first answers Ollama is running. The last one (Traefik only) should answer 401 without a password and 200 with it.

Troubleshooting#

Known bugs#

These are open GitHub issues from the last weeks, tied to specific hardware. Your setup may be fine.

• Radeon 890M: garbage output or GPU memory fault on 0.40.6 and 0.40.7 (#19290). The reporter says 0.33.3 works.
• Intel Arc iGPU: ggml_vulkan: device lost on 0.40.x (#19158). Try OLLAMA_VULKAN=0, or an older release (0.34.4 worked for the reporter).
• macOS: intermittent 500 errors "apply-template error ... broken pipe" on 0.40.8 and 0.40.11 (#19293).
• Windows: "model not found" after a pull on 0.40.11 (#19276).

To install an older release on Linux, set the version in the script: curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.34.4 sh (VERIFY on your GPU). Downgrading below 0.40 means re-pulling models that 0.40 upgraded.

Diagnostics:

docker logs ollama --tail 100
docker compose ps
journalctl -e -u ollama

Updates and backups#

Back up the models folder before every upgrade, then prove the backup restores.

Update:

• Linux: run the install script again.
• Mac and Windows: use "Restart to update" in the tray or menu icon.
• Docker: change the tag in the Compose file, then docker compose pull && docker compose up -d.

Where things live:

Linux#

Backup script (stops the container so files are not changing):

cd ~/docker_volumes/ollama && docker compose down
mkdir -p backups
tar -czf backups/ollama-$(date +%F).tar.gz -C ~/docker_volumes ollama/data ollama/docker-compose.yml
docker compose up -d

Schedule it at 03:00 with crontab -e:

0 3 * * * cd ~/docker_volumes/ollama && docker compose down && tar -czf backups/ollama-$(date +\%F).tar.gz -C ~/docker_volumes ollama/data ollama/docker-compose.yml && docker compose up -d

Restore test (never skip it):

mkdir -p /tmp/restore && tar -xzf ~/docker_volumes/ollama/backups/ollama-*.tar.gz -C /tmp/restore
docker run -d --name ollama-test -v /tmp/restore/ollama/data:/root/.ollama -p 127.0.0.1:11435:11434 ollama/ollama:0.40.11
curl http://127.0.0.1:11435/api/tags
docker rm -f ollama-test

Your models must be listed.

Windows (PowerShell)#

cd $HOME\docker_volumes\ollama; docker compose down
New-Item -ItemType Directory -Force backups | Out-Null
Compress-Archive -Path data, docker-compose.yml -DestinationPath "backups\ollama-$(Get-Date -Format yyyy-MM-dd).zip" -Force
docker compose up -d

Schedule it with Task Scheduler (daily 03:00, run the lines above from a .ps1 file). Restore test: expand the zip into a temp folder, start a test container on port 11435 against that folder as in the Linux test, and check http://127.0.0.1:11435/api/tags.

Mac (Terminal)#

Native app: quit Ollama, then copy the folder.

tar -czf ~/ollama-backup-$(date +%F).tar.gz -C ~ .ollama

Cron at 03:00 works the same as on Linux. Restore test: extract into a temp folder and compare ls of the models folder.

Performance and resources#

Fit the model in GPU memory and keep the context modest.

• Check where it runs. ollama ps should say 100% GPU. A split such as 30% CPU / 70% GPU means the model spills into system RAM and gets slower.
• Context costs memory. The default is 4096. Memory grows with OLLAMA_NUM_PARALLEL times the context length, so 4 parallel requests at 2K need as much as one at 8K.
• Shrink the cache. Flash Attention is used when supported. A q8_0 KV cache uses about half the memory of f16, and q4_0 about a quarter.
• Avoid reload delays. Keep models loaded with OLLAMA_KEEP_ALIVE=-1, or preload with ollama run <model> "".
• Several GPUs. Ollama puts a model on one GPU if it fits, otherwise it spreads it across all.
• Watch it. docker stats shows RAM and CPU of the container. Exact VRAM needs per model are on each model's page at ollama.com; I did not measure them here.

What changed since early 2025#

Short answer: more GPU choices, a bigger API surface and a new packaging. Things that were true in the older guides and are now different:

Note: models pulled with an older version are upgraded in the background since v0.40.2, so take that backup.

FAQ#

Does Ollama send my prompts to ollama.com? Not when you run models locally. Cloud models do process your prompts on Ollama's servers. Set OLLAMA_NO_CLOUD=1 to be sure nothing leaves your machine.

Where are my models stored, and how do I move them? Linux service: /usr/share/ollama/.ollama/models. Mac: ~/.ollama/models. Windows: %USERPROFILE%\.ollama\models. Set OLLAMA_MODELS to a new folder and, on Linux, chown it to the ollama user.

Can I use Ollama with Claude Code or other tools? Yes. Set ANTHROPIC_BASE_URL=http://localhost:11434 and ANTHROPIC_AUTH_TOKEN=ollama, or run ollama launch claude. OpenAI-style tools use http://localhost:11434/v1/ with any dummy key.

How do I give an OpenAI-style client a bigger context? Their API has no such field. Make a Modelfile with PARAMETER num_ctx 8192, create a model from it and use that model name.

Why is my model slow? Run ollama ps. If it shows CPU or a split, the GPU is not fully used. Lower the context or parallel requests, pick a smaller model, or fix the driver.

Sources#

Read on 2026-10-26. The docs pages carry no date, so they are treated as the live current docs.

• Ollama releases (v0.40.11, 2026-10-24)
• Ollama issues
• Docker Hub: ollama/ollama tags
• Ollama docs: FAQ
• Ollama docs: Linux
• Ollama docs: Docker
• Ollama docs: Windows
• Ollama docs: macOS
• Ollama docs: OpenAI compatibility
• Ollama docs: Anthropic compatibility

Checked against the official docs on 2026-10-26.

• Caddy docs: automatic HTTPS, reverse_proxy, install (caddyserver.com/docs, Caddy v2.11.7, read 2026-10-26)