Share One NVIDIA GPU Between Ollama, Immich and Jellyfin in Docker (Linux, Windows, Mac)

What you get in one minute#

One NVIDIA graphics card can run your local AI chat (Ollama), your photo search and face recognition (Immich) and your video streaming (Jellyfin) at the same time, as long as you tell Docker to hand the card to each app and you plan who may hold how much video memory.

Logo: the official Docker brand kit (docker.com/company/newsroom/media-resources), shown unchanged. NVIDIA allows its logo only with written permission, so we use the Docker logo instead.

At a glance:

• What you get: three apps on one GPU, each with a known VRAM budget and a test that shows all three busy at once.
• Time needed: about 45 minutes if the three apps already run on the CPU.
• Difficulty: medium on Linux, medium on Windows (WSL2), and not possible with NVIDIA on a Mac.
• You end up with: ollama ps showing 100% GPU, Immich using -cuda and NVENC, Jellyfin transcoding on NVENC, and a nvidia-smi reading you can trust.

The card does not split itself. Every app takes video memory (VRAM) while it works, and the video encoder (NVENC) limits parallel jobs. Most crashes come from one app holding VRAM it no longer needs, so the fix is mostly settings.

How it fits together#

Each container asks Docker for the GPU, the NVIDIA Container Toolkit passes the driver into it, and all of them end up on the same physical card.

Our guides cover the apps: Ollama, Immich and Jellyfin. This page only adds the GPU part. To pick a model that fits your card, read Which local LLM fits your GPU.

Before you start#

You need a NVIDIA card with a current driver, Docker with Compose, and the three apps (or at least a plan for them).

Driver minimums from the official docs:

So a driver of 570 or newer covers all three on any supported card. Check the tools:

docker --version
docker compose version
nvidia-smi

• [ ] nvidia-smi shows your card and its memory
• [ ] Ports 11434, 2283 and 8096 are free
• [ ] Your existing compose files and .env files are backed up
• [ ] Only for the HTTPS section: a domain, DNS and ports 80/443

Tested on: not run by DIY Lab Hub for this page, because the writing machine has no NVIDIA card (Linux = not run, Windows = not run, Mac = not run). Every command was checked against the official docs on 2026-10-30 (see Sources).

Step 1: Make Docker see the GPU#

Install the toolkit on Linux, or switch on GPU support in Docker Desktop on Windows.

Linux#

These are NVIDIA's own commands for Ubuntu and Debian (the docs also give dnf and zypper versions):

sudo apt-get update && sudo apt-get install -y --no-install-recommends ca-certificates curl gnupg2
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.20.1-1
sudo apt-get install -y \
  nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

The restart briefly stops your running containers. The nvidia-ctk command edits /etc/docker/daemon.json.

Windows (PowerShell)#

There is no toolkit to install. Docker Desktop passes the GPU through WSL2. You need a NVIDIA driver for Windows that supports WSL2 GPU, up-to-date Windows, and Docker Desktop with the WSL2 backend switched on.

wsl --update
docker version

Mac (Terminal)#

Not supported with NVIDIA: a Mac has no NVIDIA GPU, and Ollama's macOS page lists only Apple M-series (CPU and GPU) or Intel (CPU only). Skip to Step 2 and run the native apps. A container on a Mac only uses the CPU.

Check it worked. Linux:

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Windows (Docker's own test, the same command works on Linux):

docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

Expected (from the NVIDIA and Docker docs, with their example card): a table naming your GPU and its memory on Linux, and a line like GPU Device 0: "GeForce RTX 2060 with Max-Q Design" with compute capability 7.5 on Windows, with your own card's name.

Tip: if the test says it cannot select a device driver, you skipped nvidia-ctk runtime configure or the Docker restart. Do both before you touch any app file.

Step 2: Give Ollama the GPU#

Run Ollama in a container with one loaded model, a short keep-alive and a fixed context, so it hands VRAM back quickly.

Linux#

mkdir -p ~/docker_volumes/ollama/{data,config,logs,backups}
cd ~/docker_volumes/ollama
cat > docker-compose.yml <<'EOF'
services:
  ollama:
    image: ollama/ollama:0.40.13
    container_name: ollama
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"
    volumes:
      - ./data:/root/.ollama
    environment:
      - OLLAMA_KEEP_ALIVE=2m
      - OLLAMA_MAX_LOADED_MODELS=1
      - OLLAMA_NUM_PARALLEL=1
      - OLLAMA_CONTEXT_LENGTH=4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
EOF
docker compose up -d
docker exec -it ollama ollama pull llama3.2

The deploy block is Docker's documented Compose form of the --gpus=all flag in Ollama's own Docker page. If you have two GPUs and want Ollama on one, use device_ids: ['0'] instead of count: all.

Windows (PowerShell)#

Docker Desktop with WSL2. Same file, written from PowerShell:

$d = "$HOME\docker_volumes\ollama"
New-Item -ItemType Directory -Force "$d\data","$d\config","$d\logs","$d\backups" | Out-Null
Set-Location $d
@'
services:
  ollama:
    image: ollama/ollama:0.40.13
    container_name: ollama
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"
    volumes:
      - ./data:/root/.ollama
    environment:
      - OLLAMA_KEEP_ALIVE=2m
      - OLLAMA_MAX_LOADED_MODELS=1
      - OLLAMA_NUM_PARALLEL=1
      - OLLAMA_CONTEXT_LENGTH=4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
'@ | Set-Content -Encoding ascii docker-compose.yml
docker compose up -d
docker exec -it ollama ollama pull llama3.2

Mac (Terminal)#

Use the native app from ollama.com/download; it uses the Apple GPU. The same two settings work as environment variables (see our Ollama guide):

launchctl setenv OLLAMA_KEEP_ALIVE "2m"
launchctl setenv OLLAMA_MAX_LOADED_MODELS "1"

Restart the Ollama app afterwards.

Check it worked (Linux and Windows; on Mac run ollama ps locally):

docker exec ollama ollama run llama3.2 "Say hello in five words."
docker exec ollama ollama ps

Expected (format from the Ollama FAQ): a row for llama3.2 with 100% GPU in the PROCESSOR column and UNTIL about 2 minutes from now. 100% CPU means the GPU is not used.

Tip: set OLLAMA_CONTEXT_LENGTH yourself. The Ollama FAQ says the default is 4096, but the help text in the v0.40.13 source says it depends on VRAM (4k, 32k or 256k). A larger window costs VRAM you wanted for Immich.

Step 3: Give Immich the GPU#

Immich needs two changes: the machine-learning service gets the -cuda image, and the server gets NVENC for video.

Start from the working Immich folder of our Immich guide (~/docker_volumes/immich, with IMMICH_VERSION=v3.3.5 in .env). Immich's docs allow either the extends helper files or the same blocks written straight into the service ("single compose file"). The second needs no extra download. In docker-compose.yml, change these two services and keep the rest:

  immich-server:
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
                - compute
                - video

  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

Only the machine-learning image has a -cuda tag. The server image does not. Optionally add this line to .env, so the models leave VRAM after two idle minutes instead of five (the default is 300 seconds):

MACHINE_LEARNING_MODEL_TTL=120

Linux#

cd ~/docker_volumes/immich
echo 'MACHINE_LEARNING_MODEL_TTL=120' >> .env
docker compose config --quiet
docker compose pull
docker compose up -d

Windows (PowerShell)#

Our Immich guide keeps the stack inside WSL, so edit the file there (wsl, then nano docker-compose.yml). The NVIDIA driver for Windows is enough, since Immich's docs only ask for the toolkit on Linux outside WSL2.

wsl --exec bash -lc 'cd "$HOME/docker_volumes/immich" && echo "MACHINE_LEARNING_MODEL_TTL=120" >> .env && docker compose config --quiet && docker compose pull && docker compose up -d'

Mac (Terminal)#

Not supported. Immich's hardware docs say only Linux and Windows (through WSL2) servers are supported, and there is no Apple backend. Leave the stack on the CPU. If you still want faster transcoding, put Immich on a Linux or Windows machine.

Then switch NVENC on in the web app: Administration, Settings, Video Transcoding Settings, set Hardware Acceleration to NVENC and save. Hardware decoding is optional. By default only encoding runs on the GPU.

Check it worked: the machine-learning log lists the available providers when a job starts or you search with text.

docker logs immich_machine_learning --tail 100 | grep -i "ORT providers"

You should see CUDAExecutionProvider in the list (the Immich docs name this log line). For video, start a transcode job and watch nvidia-smi (Step 5).

Tip: do not raise Immich's job concurrency while Ollama has a model loaded. The Immich docs warn that higher concurrency raises VRAM use.

Step 4: Give Jellyfin the GPU#

Run Jellyfin on Linux in a container with NVENC, or install it natively on Windows and Mac.

Jellyfin's container page says Windows and macOS containers are not supported and names hardware transcoding as one of the known-broken features there. That is why this step differs by system.

Linux#

mkdir -p ~/docker_volumes/jellyfin/{config,cache,logs,backups}
cd ~/docker_volumes/jellyfin
id -u && id -g
cat > docker-compose.yml <<'EOF'
services:
  jellyfin:
    image: jellyfin/jellyfin:12.4
    container_name: jellyfin
    user: "1000:1000"
    restart: unless-stopped
    ports:
      - "127.0.0.1:8096:8096"
    volumes:
      - ./config:/config
      - ./cache:/cache
      - /path/to/media:/media:ro
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
EOF

Replace 1000:1000 with the two numbers id printed and /path/to/media with your media folder. The runtime and deploy lines follow Jellyfin's own NVIDIA example. The official image already sets the NVIDIA environment variables. Then:

docker compose up -d
docker exec -it jellyfin nvidia-smi

Windows (PowerShell)#

Install Jellyfin natively from jellyfin.org/downloads/server. It then uses the NVIDIA driver directly and shares the card with your Docker containers. Jellyfin 10.11 docs name driver 522.25 as the Windows minimum.

nvidia-smi

Mac (Terminal)#

No NVIDIA. Install Jellyfin natively from the same page; Jellyfin's docs list Apple VideoToolbox as its macOS hardware method.

Finally, in the Jellyfin web page open Dashboard, Playback, Transcoding, choose NVIDIA NVENC as the hardware acceleration and untick the codecs your card does not support. The Jellyfin docs say only "enable NVENC in Jellyfin and uncheck the unsupported codecs", so the menu names may differ slightly in your version.

Check it worked: play a video and lower its quality to force a transcode. nvidia-smi should show GPU use and a jellyfin-ffmpeg process on Linux. In Jellyfin's own docs one such transcode shows 274MiB / 1998MiB used on a small card (their example, not our measurement).

Tip: on Linux, if Jellyfin says CUDA_ERROR_NO_DEVICE, do not guess. Jellyfin's docs list the exact fix: pass the /dev/nvidia* devices into the container (see Troubleshooting).

Step 5: Budget the VRAM and test all three together#

Write down how much VRAM each app holds, who gives it back and when, then run all three at once.

Who holds video memory:

The Ollama FAQ also says that several models load side by side only if each fits completely in VRAM. Otherwise requests wait until an idle model is unloaded.

Measure it, do not guess. Run one app at a time, note the VRAM, and add it up with a margin.

Linux#

nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu,encoder.stats.sessionCount --format=csv -l 5

Windows (PowerShell)#

nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu,encoder.stats.sessionCount --format=csv -l 5

Task Manager, GPU page, also shows Video Encode and Video Decode (Jellyfin's docs use it). Per-process memory may not show under WSL2, so rely on the totals.

Mac (Terminal)#

No NVIDIA, so no nvidia-smi. Use ollama ps to see where a model sits.

The query field names come from nvidia-smi --help-query-gpu. If your driver rejects one, run that command and pick the closest name.

The worksheet:

1. Idle VRAM = number with all apps quiet (desktop included).
2. Load your Ollama model with a prompt. Note the rise. That is chat weight plus context.
3. Start an Immich smart-search or face job. Note the rise.
4. Start one Jellyfin transcode. Note the rise.
5. Sum 2, 3 and 4 on top of 1. If it comes close to memory.total, shorten keep-alive, cut context, or pick a smaller model.

Then run the real test: a chat prompt, an Immich job and a Jellyfin transcode at the same time. A pass means no out of memory in any log and ollama ps still reading 100% GPU.

Encoder sessions on consumer cards#

NVIDIA's own Video Encode and Decode GPU Support Matrix (read 2026-10-30) has a column "Max # of concurrent sessions". It lists 12 for every GeForce row we read (RTX 50, 40, 30 and 20 series, GTX 16 and 10 series). Some small professional cards list 8, and larger professional cards list "Unrestricted". The table does not say whether the cap is counted per container, so treat the number as shared by Jellyfin and Immich. Every transcode takes one session. encoder.stats.sessionCount in the query above shows how many are active. Read the matrix for your exact card.

Tip: run big Immich imports and library scans while nobody is chatting or streaming. A first import queues thousands of search, face and transcode jobs, and they are the usual reason the three apps collide.

Put it behind HTTPS (with or without Traefik)#

Publish Immich and Jellyfin over HTTPS. Never publish Ollama this way: it has no login.

This is for a server with a domain. Local-only users can skip it.

With Traefik#

Assumes Traefik v3 with the Docker provider, an entrypoint called https and a certificate for your hostnames. Create the network once if it is missing, then remove the ports: lines of both apps and add the labels:

docker network create proxy

services:
  jellyfin:
    networks:
      - proxy
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"
      - "traefik.http.routers.jellyfin.rule=Host(`media.<your-domain>`)"
      - "traefik.http.routers.jellyfin.entrypoints=https"
      - "traefik.http.routers.jellyfin.tls=true"
      - "traefik.http.services.jellyfin.loadbalancer.server.port=8096"
networks:
  proxy:
    external: true

For Immich, give immich-server the same pattern with Host(photos.<your-domain>) and port 2283. Immich must sit at the root of its own hostname, and its uploads need longer timeouts; see the Traefik section of our Immich guide.

Check it worked: curl -I https://media.<your-domain> answers with a Jellyfin redirect or 200.

Without Traefik: Caddy#

Choose Caddy when you have no Traefik yet and run one small server. Facts are from caddyserver.com/docs.

Public domain Caddyfile (the apps stay on 127.0.0.1):

photos.example.com {
    reverse_proxy localhost:2283
}

media.example.com {
    reverse_proxy localhost:8096
}

Home only: add tls internal as the first line inside each block and use names such as photos.home.arpa. Trust the root: on the Caddy machine run sudo caddy trust (Linux); on Windows and Mac import pki/authorities/local/root.crt from Caddy's data folder as a trusted root (certificate manager, Keychain Access).

Run Caddy (2.11.7 was current on 2026-10-30): Linux with the official apt repository from the install page, then sudo systemctl reload caddy. Windows: winget install CaddyServer.Caddy then caddy run. Mac: brew install caddy then caddy run. Or use the image caddy:2.11.7 on the same Docker network and write reverse_proxy immich_server:2283.

Immich's docs ask that /.well-known/immich reaches Immich when you use the Let's Encrypt http-01 challenge. A single site block like the one above does that. Jellyfin's docs say reverse-proxy access rules only work if the proxy is set under Known Proxies in its networking settings.

Check it worked: curl -I https://media.example.com (on Windows curl.exe) answers HTTP/2 200 or a redirect.

Security and testing#

GPU access gives containers no extra network power, but each app is a service that must not face the internet unprotected.

• Keep every port on 127.0.0.1. Do not forward 11434, 2283 or 8096 on your router. Jellyfin's docs call opening a port to the internet insecure and not recommended.
• Ollama has no login. Leave it on loopback or behind your own auth.
• Never add privileged: true or pass /dev wholesale to get the GPU working. The device reservation is enough.
• Run Jellyfin as a normal user (user: line), keep the media mount read-only.
• Keep Immich's .env at chmod 600 and pin image tags as shown.

What leaves your machine. Everything runs locally, but some things still talk to the internet:

Quick tests after every change:

docker compose ps
docker exec ollama ollama ps
docker logs jellyfin --tail 50

Common mistakes#

Most GPU-sharing trouble is predictable. Check this table before you open a forum thread.

Troubleshooting#

Read the first error in docker logs <name> --tail 100 before changing anything.

Failed to initialize NVML: Unknown Error#

The container lost its GPU after a unit reload. Delete and recreate it (docker compose up -d --force-recreate). For a lasting fix NVIDIA documents {"exec-opts": ["native.cgroupdriver=cgroupfs"]} in /etc/docker/daemon.json followed by systemctl restart docker.

CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected#

Jellyfin's docs: add /dev/nvidia-caps, /dev/nvidia0, /dev/nvidiactl, /dev/nvidia-modeset, /dev/nvidia-uvm and /dev/nvidia-uvm-tools as devices: entries, run sudo usermod -aG video $USER, then docker exec -it --user root jellyfin ldconfig and sudo systemctl restart docker.

Failed to initialize NVML: Insufficient Permissions#

On SELinux systems (Red Hat style) NVIDIA's docs say to add --security-opt=label=disable. This turns SELinux separation off for that container.

Known bugs#

• Immich issue 32664 (open when we read it): the machine-learning container keeps the GPU devices open while idle, so idle power stays higher. Stopping that container releases the card.
• Jellyfin issue 14463 (closed): NVENC failed with FFmpeg exited with code 187 after some time, and only a container restart helped.
• Ollama issue 18679 (open): the VRAM reserve setting may be ignored.

Updates and backups#

Update one app at a time, then rerun the Step 5 test.

Change the pinned tag in the compose file (or IMMICH_VERSION in .env), then run:

cd ~/docker_volumes/ollama
docker compose pull
docker compose up -d

Repeat for jellyfin and immich (Windows: PowerShell, Immich through wsl --exec bash -lc '...'). After a driver update, restart Docker and rerun the Step 1 test.

Backup script#

The important parts are the compose files, .env files and Jellyfin's config. Models and caches can be downloaded again. Photos and the Immich database need the backup routine from our Immich guide.

#!/usr/bin/env bash
set -euo pipefail
for app in ollama jellyfin; do
  cd ~/docker_volumes/$app
  tar --exclude='data' --exclude='cache' -czf backups/$app-$(date +%F).tar.gz docker-compose.yml config
  find backups -name "$app-*.tar.gz" -mtime +14 -delete
done

Linux and Mac: save as backup.sh, make it executable, add 0 3 * * * /home/<you>/backup.sh to cron. Windows: run the tar lines from Task Scheduler at 03:00.

Restore test#

mkdir -p /tmp/restore-test
tar -xzf ~/docker_volumes/jellyfin/backups/jellyfin-$(date +%F).tar.gz -C /tmp/restore-test
ls -a /tmp/restore-test
rm -rf /tmp/restore-test

You should see docker-compose.yml and config.

Performance and resources#

Sharing works best when the three apps rarely peak together.

Watch it live:

docker stats
nvidia-smi

DIY Lab Hub has not measured speeds on this setup, so we publish no tokens-per-second or frames-per-second numbers. Measure your own card.

FAQ#

Can three containers really use one GPU at once? Yes. Docker lets any number of containers reserve the same device. The limits are VRAM and the encoder session cap.

Do I need a card with a lot of VRAM? More VRAM gives more room, but short keep-alive times let a smaller card work. The apps simply wait if a model must be reloaded.

Can I run it on a Mac? Not with NVIDIA. Use native Ollama and Jellyfin, which use the Apple GPU, and keep Immich on the CPU.

Should I use Docker Desktop on Windows for all three? Ollama and Immich: yes (Immich inside WSL). Jellyfin: install it natively.

Sources#

Read on 2026-10-30:

• Installing the NVIDIA Container Toolkit, sample workload, troubleshooting and release notes (version 1.20.1)
• NVIDIA Video Encode and Decode GPU Support Matrix and nvidia-smi
• Docker Desktop GPU support (Windows) and Compose GPU support
• Ollama Docker, FAQ, GPU and macOS docs, and the environment settings source
• Immich machine-learning acceleration, hardware transcoding, environment variables and reverse proxy, with the v3.3.5 release files
• Jellyfin NVIDIA acceleration, container install and networking
• Issues: Immich 32664, Immich 32375, Jellyfin 14463, Ollama 18679
• Caddy docs and Docker brand guidelines

Checked against the official docs on 2026-10-30. Not run by DIY Lab Hub (no NVIDIA card on the writing machine).