Comprehensive Guide to Deploying Open WebUI with Ollama Using Docker Compose

Introduction

Open WebUI is an extensible, feature-rich web interface for interacting with large language models that runs entirely offline. It provides a ChatGPT-like experience for models running through Ollama, an open-source framework designed for running LLMs locally. This guide covers multiple deployment configurations using Docker Compose, from basic setups to advanced scenarios with GPU support, enhanced security, and specialized integrations.

Key Benefits

Complete Privacy: All data remains locally stored with no external requests
Administrative Control: The first account created automatically receives administrator privileges
Model Privacy: All models are private by default unless explicitly shared
Simplified Deployment: Docker Compose handles container orchestration, networking, and volume management

Prerequisites

Before proceeding with any deployment, ensure your system meets these requirements:

Docker Engine and Docker Compose installed (verify with docker --version and docker-compose --version)
For NVIDIA GPU support:
- Compatible NVIDIA drivers installed
- NVIDIA Container Toolkit configured (verify with nvidia-smi)
For AMD GPU support:
- ROCm drivers installed and configured for container usage

Recommended System Requirements

Modern CPU with at least 4 cores
16GB or more of RAM
NVIDIA GPU with 8GB+ VRAM (for larger models)
SSD storage for faster model loading

Basic Installation with Docker Compose

This configuration provides a quick start with minimal setup while delivering a fully functional environment.

Step 1: Create a Project Directory

mkdir open-webui-ollama
cd open-webui-ollama

Step 2: Create the Docker Compose File

Create a file named docker-compose.yml with the following content:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

Step 3: Deploy the Services

docker-compose up -d

Step 4: Access the Interface

Open your web browser and navigate to http://localhost:3000. Create an account when prompted – the first account will automatically receive administrator privileges.

What Could Go Wrong?

Port Conflicts: If ports 3000 or 11434 are already in use, you’ll see errors. Change the left side of the port mapping (e.g., "3001:8080")
Memory Issues: If your system has limited RAM, larger models may fail to load. Consider using smaller models or increasing your system’s memory
Docker Network Issues: If containers can’t communicate, check if Docker’s bridge network is functioning properly

Advanced Docker Compose Configurations

Configuration with NVIDIA GPU Support

For users with NVIDIA GPUs, this configuration enables hardware acceleration for significantly improved performance:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
    restart: unless-stopped
    environment:
      - OLLAMA_HOST=0.0.0.0:11434

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"

volumes:
  ollama_data:
  open-webui_data:

Configuration with AMD GPU Support

For AMD GPU users, use this specialized configuration:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    restart: unless-stopped
    environment:
      - HSA_OVERRIDE_GFX_VERSION=x.x.x  # Replace with your GPU's GFX version
      - OLLAMA_HOST=0.0.0.0:11434

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

Replace x.x.x with your specific AMD GPU’s GFX version. You can find this information by running rocminfo and looking for the “gfx” value.

What Could Go Wrong with GPU Configurations?

Driver Compatibility: Ensure your GPU drivers are compatible with the container runtime
Missing NVIDIA Container Toolkit: For NVIDIA GPUs, this must be properly installed
ROCm Compatibility: Not all AMD GPUs are supported by ROCm; check compatibility before deployment
Resource Allocation: If multiple applications are using the GPU, performance may be affected

Specialized Network Configurations

Configuration for Ollama Running on Host (Outside Docker)

If Ollama is installed directly on your system rather than in Docker:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://host.docker.internal:11434
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped

volumes:
  open-webui_data:

Configuration for Ollama on a Remote Server

If Ollama is running on a different machine:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=https://your-ollama-server:11434
    restart: unless-stopped

volumes:
  open-webui_data:

Replace https://your-ollama-server:11434 with the actual URL of your remote Ollama server.

Configuration with Bundled Ollama

Open WebUI offers a version with Ollama bundled into the same image for simplified deployment:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:ollama
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - ollama:/root/.ollama
      - open-webui:/app/backend/data
    restart: unless-stopped

volumes:
  ollama:
  open-webui:

For GPU support with this bundled version, add the deploy section as shown in the GPU configurations.

What Could Go Wrong with Network Configurations?

Firewall Issues: Ensure the Ollama server allows incoming connections on port 11434
SSL Certificate Problems: If using HTTPS, ensure certificates are valid
DNS Resolution: The host.docker.internal hostname might not work in all Docker environments
API Version Mismatch: Ensure Open WebUI and Ollama versions are compatible

Enhanced Security Configurations

Configuration with Enhanced Security

Add security-related environment variables for better protection:

version: '3.8'

services:
  ollama:
    # Ollama configuration as in basic setup

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=your_secret_key  # Generate with: openssl rand -hex 32
      - WEBUI_JWT_SECRET_KEY=your_jwt_secret  # Generate with: openssl rand -hex 32
      - WEBUI_AUTH_ENABLED=true  # Enforce authentication
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

Generate secure keys using:

openssl rand -hex 32

Configuration with SearXNG Integration for RAG

For users interested in Retrieval-Augmented Generation (RAG) capabilities:

version: '3.8'

services:
  ollama:
    # Ollama configuration as in basic setup

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=your_secret_key
      - SEARXNG_BASE_URL=http://searxng:8080
    depends_on:
      - ollama
      - searxng
    restart: unless-stopped

  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    ports:
      - "8080:8080"
    environment:
      - SEARXNG_SECRET=your_searxng_secret  # Generate with: openssl rand -hex 32
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

Configuration with OpenAI API Integration

If you want to use Open WebUI with OpenAI’s API:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    environment:
      - OPENAI_API_KEY=your_openai_api_key
    restart: unless-stopped

volumes:
  open-webui_data:

Replace your_openai_api_key with your actual OpenAI API key.

What Could Go Wrong with Enhanced Security?

Secret Key Issues: If the secret key changes, existing sessions will be invalidated
JWT Authentication: If misconfigured, users may be unable to log in
SearXNG Integration: If SearXNG is unreachable, RAG functionality will not work
API Key Exposure: Storing API keys in Docker Compose files poses a security risk; consider using Docker secrets or environment variables from a secure source

Production Deployment Configurations

Configuration with Traefik Reverse Proxy

For production deployments with domain name support and SSL certificates:

version: '3.8'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/traefik.yml:/etc/traefik/traefik.yml:ro
      - ./traefik/acme.json:/acme.json
    networks:
      - traefik-network
    restart: unless-stopped

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama_data:/root/.ollama
    networks:
      - ollama-network
      - traefik-network
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    volumes:
      - open-webui_data:/app/backend/data
    networks:
      - ollama-network
      - traefik-network
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.openwebui.rule=Host(`your-domain.com`)"
      - "traefik.http.routers.openwebui.entrypoints=websecure"
      - "traefik.http.routers.openwebui.tls.certresolver=letsencrypt"
      - "traefik.http.services.openwebui.loadbalancer.server.port=8080"
    depends_on:
      - ollama
    restart: unless-stopped

networks:
  traefik-network:
    external: true
  ollama-network:
    external: true

volumes:
  ollama_data:
  open-webui_data:

Before deploying, create the necessary networks:

docker network create traefik-network
docker network create ollama-network

And create a basic Traefik configuration file at ./traefik/traefik.yml:

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: ":443"

certificatesResolvers:
  letsencrypt:
    acme:
      email: your-email@example.com
      storage: acme.json
      httpChallenge:
        entryPoint: web

providers:
  docker:
    exposedByDefault: false
    network: traefik-network

Also, create an empty acme.json file and set proper permissions:

touch ./traefik/acme.json
chmod 600 ./traefik/acme.json

Configuration with Cloudflare Tunnel

For secure remote access without exposing your server directly to the internet:

version: '3.8'

services:
  ollama:
    # Ollama configuration as in basic or GPU setup

  open-webui:
    # Open WebUI configuration as above

  tunnel:
    image: cloudflare/cloudflared:latest
    container_name: cloudflare-tunnel
    command: tunnel --no-autoupdate run
    environment:
      - TUNNEL_TOKEN=your_cloudflare_tunnel_token
    restart: unless-stopped
    depends_on:
      - open-webui

Replace your_cloudflare_tunnel_token with the token generated from your Cloudflare account.

What Could Go Wrong with Production Configurations?

Certificate Issuance: Let’s Encrypt has rate limits that may be hit during testing
Domain Configuration: DNS must be properly configured to point to your server
Traefik Configuration: Syntax errors in traefik.yml may prevent the service from starting
Network Isolation: Ensure services are on the correct networks to communicate properly
Cloudflare Token: If expired or revoked, the tunnel will stop working

Service Management and Maintenance

Starting and Stopping Services

To start your services:

docker-compose up -d

To stop services while preserving volumes:

docker-compose down

To completely remove everything including volumes:

docker-compose down -v

Viewing Logs

To view logs of all services:

docker-compose logs -f

For a specific service:

docker-compose logs -f open-webui

Updating Services

Pull the latest images and restart:

docker-compose pull
docker-compose up -d

Backing Up Data

Back up the Docker volumes:

docker run --rm -v ollama_data:/data -v $(pwd):/backup alpine tar czf /backup/ollama_backup.tar.gz -C /data .
docker run --rm -v open-webui_data:/data -v $(pwd):/backup alpine tar czf /backup/open-webui_backup.tar.gz -C /data .

To restore from backups:

docker run --rm -v ollama_data:/data -v $(pwd):/backup alpine sh -c "rm -rf /data/* && tar xzf /backup/ollama_backup.tar.gz -C /data"
docker run --rm -v open-webui_data:/data -v $(pwd):/backup alpine sh -c "rm -rf /data/* && tar xzf /backup/open-webui_backup.tar.gz -C /data"

What Could Go Wrong with Maintenance?

Disk Space: Ensuring sufficient disk space before updates and backups is crucial
Inconsistent State: Stopping services unexpectedly may lead to data corruption
Version Incompatibility: New versions might have breaking changes requiring configuration updates
Backup Failures: Always verify backups are complete and restorable

Troubleshooting Common Issues

GPU Not Detected

Verify GPU Drivers: # For NVIDIA nvidia-smi # For AMD rocm-smi
Check Container Toolkit: # For NVIDIA sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Verify Docker Compose Configuration: Ensure the deploy section is correctly formatted.

Connection Issues Between Open WebUI and Ollama

Check Network Configuration: Ensure the OLLAMA_API_BASE_URL is correctly set.
Test Connection: docker exec open-webui curl -s http://ollama:11434/api/version
Check Firewall Rules: If using different servers, ensure ports are open.

Performance Issues

Check System Resources: # CPU and memory usage htop # GPU usage nvidia-smi -l 1
Use a Smaller Model: If resources are limited, try models that require less memory.
Adjust Concurrent Requests: In Open WebUI settings, lower the number of concurrent requests.

Model Loading Failures

Check Disk Space: df -h
Manually Pull Models: docker exec -it ollama ollama pull model_name
Check Ollama Logs: docker-compose logs -f ollama

Environmental Variables Reference

Open WebUI Environment Variables

Variable	Description	Default
`OLLAMA_API_BASE_URL`	URL to the Ollama API	`http://ollama:11434`
`WEBUI_SECRET_KEY`	Secret key for encryption	Random generated
`WEBUI_JWT_SECRET_KEY`	Secret for JWT tokens	Random generated
`WEBUI_AUTH_ENABLED`	Enable authentication	`true`
`WEBUI_ALLOW_REGISTRATION`	Allow new user registration	`true`
`SEARXNG_BASE_URL`	URL to SearXNG for RAG	None
`OPENAI_API_KEY`	OpenAI API key	None
`WEBUI_SERVE_PORT`	Port to serve the UI	`8080`
`WEBUI_MAX_UPLOAD_SIZE`	Max file upload size	`100` (MB)

Ollama Environment Variables

Variable	Description	Default
`OLLAMA_HOST`	Host and port to bind Ollama	`0.0.0.0:11434`
`OLLAMA_MODELS`	Models to pull at startup	None
`OLLAMA_DEBUG`	Enable debug logging	None
`OLLAMA_NUM_GPU`	Number of GPUs to use	All available

Advanced Configuration Examples

Configuration with Multiple Models Loaded at Startup

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
      - OLLAMA_MODELS=llama3,phi3,gemma
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
    restart: unless-stopped

  open-webui:
    # Standard Open WebUI configuration

volumes:
  ollama_data:
  open-webui_data:

Configuration with Resource Limits

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        limits:
          cpus: '4'
          memory: 16G
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              count: 1  # Use only one GPU if multiple are available
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 2G
    environment:
      - OLLAMA_API_BASE_URL=http://ollama:11434
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

Configuration with Persistent Cache Directory

This helps speed up repeated model loading:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
      - ollama_cache:/tmp/ollama_cache
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
      - XDG_CACHE_HOME=/tmp/ollama_cache
    restart: unless-stopped

  open-webui:
    # Standard Open WebUI configuration

volumes:
  ollama_data:
  open-webui_data:
  ollama_cache:

Conclusion

This comprehensive guide covers most deployment scenarios for Open WebUI with Ollama using Docker Compose. By following these configurations and best practices, you can create a robust and secure deployment tailored to your specific requirements.

Remember that the performance of local language models depends significantly on your hardware capabilities. Regular maintenance, including updates and backups, ensures your deployment remains secure and benefits from the latest improvements to both Open WebUI and Ollama.

As these technologies continue to evolve, these configurations may need adjustments to accommodate new features and capabilities. Stay informed about updates to both projects and adapt your configurations accordingly.

Discover more from DIYLABHub.com

Subscribe to get the latest posts sent to your email.