Hermes Agent with a Local Model (Ollama): Your Own Self-Improving AI Agent (Linux, Windows, Mac)

What you get in one minute#

Hermes Agent is a free, open-source AI agent from Nous Research that runs terminal commands, edits files, searches the web and remembers what it learned, and it can use a model that runs on your own computer so nothing leaves your machine.

Logo: the official Hermes Agent project (github.com/NousResearch/hermes-agent, file assets/icon-master.svg at v0.22.5)

Its trick is a learning loop. After a hard task it can write itself a reusable "skill", it keeps notes about you, and it can search its own old chats. You talk to it in a terminal or from Telegram, Discord, Slack, WhatsApp or Signal. This guide uses a local model as the main route and shows OpenRouter as the cloud option.

At a glance#

• What you get: a personal agent with memory and skills, powered by a model on your own hardware, reachable from the terminal and from a chat app.
• Time needed: about 30 minutes, most of it the model download.
• Difficulty: medium. You install two programs and edit one small config file.
• You need: 16 GB RAM or more for a useful agent model (8 GB runs only small chat models), about 30 GB of free disk, and a terminal.
• Cloud is optional: add an OpenRouter key only if you want a fallback for hard tasks.

Why this version? Hermes ships fast: six releases between 14 and 22 October 2026. We read the open GitHub issues for the v0.22.x line. They are narrow bugs (for example gateway sessions, context compaction, a Linux desktop opacity bug). We found no report of a failed install or lost data in those issues. We could not run our forum and Reddit check this time, so treat that part as unchecked. This guide uses the latest release, v0.22.5. The official rollback tag is v0.22.4.

How it fits together#

You send a message, the Hermes gateway runs the tools and asks your model what to do next, and everything it learns is saved in one data folder.

Two rules to remember:

1. The model server (Ollama or llama.cpp) is separate from Hermes. Hermes only needs its OpenAI-compatible address, for example http://localhost:11434/v1.
2. The data folder holds your keys, memory and skills. Protect it like a password manager.

Before you start#

Hermes needs a model that supports tool calling and a context window of at least 64,000 tokens. A model that can only chat will not be able to act.

Rough hardware from the official local-Ollama guide:

On CPU only, the docs say a 9B model gives about 10 tokens per second and a 31B model 2 to 5. It works, but slowly.

Check your tools (Docker is only needed for the server route in Step 6):

curl --version
docker --version
docker compose version

Checklist:

• [ ] You know which machine runs the model and which runs Hermes (they can be the same)
• [ ] Ports 11434 and 9119 are free
• [ ] You will back up ~/.hermes (Windows: %LOCALAPPDATA%\hermes) before replacing anything

Tested on 2026-10-26: not run. Commands checked against the official Hermes docs and release v0.22.5.

Step 1: Install a local model server (Ollama)#

Ollama is the easiest route. It serves an OpenAI-compatible API on port 11434. (Prefer llama.cpp? See "Use llama.cpp instead" below.)

Linux#

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
curl http://localhost:11434/api/tags

Windows (PowerShell)#

winget install Ollama.Ollama
ollama --version

Mac (Terminal)#

brew install ollama
brew services start ollama
ollama --version

Check it worked#

ollama --version prints a version, and curl http://localhost:11434/api/tags (Windows: curl.exe http://localhost:11434/api/tags) answers with a JSON list. Per the Hermes docs a fresh install returns {"models":[]}.

Step 2: Pull a model with tool calling and a big context#

Download a tool-capable model, then make a copy with a 64,000-token context. Ollama's default context is only 2,048 tokens, which is far too small for an agent.

The official Hermes guide uses gemma4:31b (about 20 GB, 24 GB or more RAM) and states it is the model with reliable tool calling that they tested. The Ollama library also lists smaller Gemma 4 sizes such as gemma4:12b and gemma4:26b. Hermes only documents 31b, so if you try a smaller one, check that ollama show <model> lists tools.

Linux#

ollama pull gemma4:31b
printf 'FROM gemma4:31b\nPARAMETER num_ctx 64000\n' > Modelfile
ollama create gemma4-64k -f Modelfile

Windows (PowerShell)#

ollama pull gemma4:31b
"FROM gemma4:31b`nPARAMETER num_ctx 64000" | Set-Content -Encoding ascii Modelfile
ollama create gemma4-64k -f Modelfile

Mac (Terminal)#

Same commands as Linux:

ollama pull gemma4:31b
printf 'FROM gemma4:31b\nPARAMETER num_ctx 64000\n' > Modelfile
ollama create gemma4-64k -f Modelfile

Check it worked#

ollama list

You should see gemma4-64k in the list. Potential issue: a model bigger than your RAM swaps to disk and becomes painfully slow. Pick a smaller one.

Use llama.cpp instead#

If you prefer llama.cpp, start llama-server with these flags (from the Hermes provider docs):

llama-server --jinja -fa -c 64000 -ngl 99 -m <path-to-model>.gguf --port 8080

--jinja is required: without it llama-server ignores tools and you will see raw JSON in the chat. -c 64000 sets the 64K context. Then use http://localhost:8080/v1 as the address in Step 4. Hermes' own desktop app can also install and manage a llama.cpp server for you (Settings > Providers > Local Models).

Step 3: Install Hermes Agent#

One command installs everything Hermes needs. The installer handles Python, Node.js and the other tools itself.

Linux#

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
source ~/.bashrc

Windows (PowerShell)#

Native Windows is supported, no WSL needed. Data lives in %LOCALAPPDATA%\hermes.

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

Mac (Terminal)#

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
source ~/.zshrc

Check it worked#

hermes doctor

It checks your setup and reports problems. Potential issue on Windows: antivirus may quarantine uv.exe in %LOCALAPPDATA%\hermes\bin. The Hermes README says this is a false positive (it is Astral's uv). Exclude the folder, not the file.

Step 4: Connect Hermes to your local model#

Tell Hermes the address of your model server. No API key is needed.

Run the wizard:

hermes model

Choose Custom endpoint, enter the address, leave the key empty and type the model name:

Or write it directly into config.yaml. The content is the same on every system, only the folder differs.

model:
  default: "gemma4-64k"
  provider: "custom"
  base_url: "http://localhost:11434/v1"

On a slow CPU-only machine, give Hermes more time. This is an environment variable in the .env file, not a config.yaml key:

HERMES_API_TIMEOUT=1800

Linux#

The files are in ~/.hermes/ (config.yaml and .env):

nano ~/.hermes/config.yaml

Windows (PowerShell)#

The files are in %LOCALAPPDATA%\hermes\:

notepad "$env:LOCALAPPDATA\hermes\config.yaml"

Mac (Terminal)#

Same as Linux:

nano ~/.hermes/config.yaml

Check it worked#

hermes

Type: List the files in this folder and say what the biggest one is. Hermes should run a terminal command and answer. The first reply can take minutes on CPU: Hermes sends its system prompt and all tool descriptions every time and the model must read them first. This is expected, not a hang.

Step 5: Add OpenRouter as a cloud option (optional)#

OpenRouter gives you one key for hundreds of hosted models. Use it as the main brain, or only as a fallback when the local model struggles.

Open the data folder files the same way on each system:

Linux#

nano ~/.hermes/.env

Windows (PowerShell)#

notepad "$env:LOCALAPPDATA\hermes\.env"

Mac (Terminal)#

nano ~/.hermes/.env

Then:

1. Create a key at openrouter.ai.
2. Add it to the .env file in your data folder (open it as shown in Step 4):

OPENROUTER_API_KEY=<YOUR_KEY>

3. To use it as a fallback only (from the official local-Ollama guide), extend config.yaml:

model:
  default: "gemma4-64k"
  provider: "custom"
  base_url: "http://localhost:11434/v1"

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4

Pick any current model id from the OpenRouter model list (the id above is the docs' example, VERIFY it is still listed). Hermes switches to the fallback when the local model keeps failing. To use OpenRouter as the main route, run hermes model and choose OpenRouter. Remember: with a cloud model, your chats leave your computer.

Check it worked#

Inside hermes, type /model to see the active model and /usage to see what a session used.

Step 6: Talk to it from Telegram (optional)#

The gateway is one process that connects Hermes to chat apps. Hermes has a built-in allowlist, and by default nobody is allowed in.

1. In Telegram, message @BotFather, create a bot and copy the token.
2. Put the token and your own numeric Telegram user ID in .env (open it as shown in Step 5):

TELEGRAM_BOT_TOKEN=<YOUR_BOT_TOKEN>
TELEGRAM_ALLOWED_USERS=<YOUR_TELEGRAM_USER_ID>

Prefer a button? The dashboard and desktop app have Messaging > Telegram > Create with QR, which makes the bot and fills both values for you.

3. Start the gateway. The commands are the same everywhere:

Linux#

hermes gateway setup
hermes gateway start

Windows (PowerShell)#

hermes gateway setup
hermes gateway start

Mac (Terminal)#

Same commands as Linux:

hermes gateway setup
hermes gateway start

Check it worked#

Send your bot a message. Hermes answers using your local model. If the log says No user allowlists configured. All unauthorized users will be denied., the allowlist is empty: add your ID.

Step 7: Run it in Docker on a Linux server (optional)#

Use Docker when Hermes should run all the time on a server. Hermes publishes an official image. On Windows and Mac we recommend the native install above: the docs warn that Docker Desktop bind mounts (virtiofs, 9p) can silently corrupt Hermes' SQLite database. If you still want Docker there, use a named volume (hermes-data:/opt/data) instead of a folder.

This setup runs Hermes and Ollama together. Ollama has no published port, and the Hermes dashboard is bound to 127.0.0.1.

Linux#

mkdir -p ~/docker_volumes/hermes/{data,ollama}
cd ~/docker_volumes/hermes
printf 'HERMES_UID=%s\nHERMES_GID=%s\n' "$(id -u)" "$(id -g)" > .env
printf 'HERMES_DASHBOARD_BASIC_AUTH_USERNAME=admin\nHERMES_DASHBOARD_BASIC_AUTH_PASSWORD=%s\nHERMES_DASHBOARD_BASIC_AUTH_SECRET=%s\n' "$(openssl rand -hex 16)" "$(openssl rand -hex 32)" >> .env
chmod 600 .env
nano docker-compose.yml

services:
  ollama:
    image: ollama/ollama:0.40.11
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ./ollama:/root/.ollama

  hermes:
    image: nousresearch/hermes-agent:v0.22.5
    container_name: hermes
    restart: unless-stopped
    command: ["gateway", "run"]
    shm_size: "1gb"
    env_file: .env
    environment:
      HERMES_DASHBOARD: "1"
    ports:
      - "127.0.0.1:9119:9119"
    volumes:
      - ./data:/opt/data
    depends_on:
      - ollama

Start Ollama, load the model (same Modelfile trick as Step 2), then write the Hermes config:

docker compose up -d ollama
docker exec ollama ollama pull gemma4:31b
docker exec ollama sh -c 'printf "FROM gemma4:31b\nPARAMETER num_ctx 64000\n" > /tmp/Modelfile && ollama create gemma4-64k -f /tmp/Modelfile'
cat > data/config.yaml <<'EOF'
model:
  default: "gemma4-64k"
  provider: "custom"
  base_url: "http://ollama:11434/v1"
EOF
docker compose up -d

Inside a container, localhost is the container itself, so the address uses the service name ollama. The shm_size line is for Hermes' browser tool. For an NVIDIA GPU add the GPU reservation block to the ollama service (see the Ollama Docker docs).

Windows (PowerShell)#

Use the native install from Step 3. The gateway runs from PowerShell with hermes gateway start.

Mac (Terminal)#

Use the native install from Step 3 as well.

Check it worked#

docker compose ps
docker logs hermes --tail 50

Both containers are running. Open http://127.0.0.1:9119 on the server and log in with the username admin and the password from .env. Potential issue: the first start can take a minute while the s6 supervisor sets things up.

Put it behind HTTPS (with or without Traefik)#

With Traefik#

This is for a server with a domain name and Traefik already running, so you can open the dashboard from outside. Running only at home on localhost? Skip this section.

Prerequisites: a domain, DNS pointing to the server, ports 80 and 443 open, Traefik on the external proxy network.

docker network create proxy

Change the hermes service (remove the ports lines, add the network and labels). The labels are our example, not from the official docs (VERIFY against your Traefik setup):

  hermes:
    image: nousresearch/hermes-agent:v0.22.5
    container_name: hermes
    restart: unless-stopped
    command: ["gateway", "run"]
    shm_size: "1gb"
    env_file: .env
    environment:
      HERMES_DASHBOARD: "1"
    volumes:
      - ./data:/opt/data
    networks:
      - default
      - proxy
    labels:
      - traefik.enable=true
      - traefik.docker.network=proxy
      - traefik.http.routers.hermes.rule=Host(`<your-domain>`)
      - traefik.http.routers.hermes.entrypoints=https
      - traefik.http.routers.hermes.tls=true
      - traefik.http.services.hermes.loadbalancer.server.port=9119

networks:
  proxy:
    external: true

The official docs say a proxy in another container is not trusted by default. Tell Hermes the public address and trust only that proxy, in data/config.yaml (never * or 0.0.0.0/0, Hermes rejects those):

dashboard:
  public_url: "https://<your-domain>"
  trusted_proxies:
    - "<traefik-container-ip-or-small-subnet>"

Restart with docker compose up -d. The dashboard keeps its own password login in front of it.

Without Traefik: Caddy#

Choose Caddy if you have no Traefik yet and run one small server.

Caddyfile for a public domain (the dashboard is published on 127.0.0.1:9119 in our compose file):

hermes.example.com {
    reverse_proxy localhost:9119
}

Caddyfile for home only (point the name at the Caddy machine):

hermes.home.arpa {
    tls internal
    reverse_proxy localhost:9119
}

Trust the root certificate (pki/authorities/local/root.crt in Caddy's data directory):

# Linux: on the Caddy machine
sudo caddy trust
# other Linux devices, with root.crt copied over
sudo cp root.crt /usr/local/share/ca-certificates/caddy-root.crt && sudo update-ca-certificates
# Windows (PowerShell as administrator)
certutil -addstore -f ROOT root.crt
# Mac
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain root.crt

Run Caddy (Linux: install from the official apt repository on caddyserver.com/docs/install first):

# Linux
sudo systemctl reload caddy
# Windows (PowerShell)
winget install CaddyServer.Caddy
caddy run
# Mac
brew install caddy && caddy run

Or use Docker on the app's network. Add the proxy network to the hermes service, use reverse_proxy hermes:9119 in the Caddyfile and save this in ~/docker_volumes/caddy (create the network once: docker network create proxy):

services:
  caddy:
    image: caddy:2.11.7
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ./caddy_data:/data
      - ./caddy_config:/config
    networks:
      - proxy
networks:
  proxy:
    external: true

Set dashboard.public_url and trusted_proxies in data/config.yaml as shown in the Traefik part, using the Caddy container's address.

Check it worked: curl -I https://hermes.example.com answers HTTP/2 200 (or a redirect to the login page).

Security and testing#

Hermes can run commands on your machine, so treat access as the whole security story.

• Never expose the dashboard or the API server to the internet without a login. The Hermes docs say an open dashboard was the entry point of a June 2026 campaign: scanners reached exposed dashboards and API servers and made the agent plant an SSH-key backdoor. Hermes now refuses to start a non-loopback dashboard with no auth provider. Prefer SSH tunnel, WireGuard or Tailscale.
• Keep the API server off. It stays off unless you set API_SERVER_ENABLED=true. If you ever turn it on beyond localhost, set API_SERVER_KEY too (at least 8 characters, for example openssl rand -hex 32).
• Approvals stay on. The default approvals.mode: smart lets a helper model auto-approve low-risk commands and block dangerous ones. manual asks every time. Do not use off or --yolo outside a throwaway container.
• Allowlist your chat. Set TELEGRAM_ALLOWED_USERS and never GATEWAY_ALLOW_ALL_USERS=true.
• Sandbox the tools. In config.yaml set terminal.backend: docker so commands run in a container, not on your host.
• Lock the secrets:

chmod 600 ~/.hermes/.env

Tests:

hermes doctor
curl -I https://<your-domain>
docker compose ps

Expected: no problems from hermes doctor, HTTP/2 200 (or a login redirect), containers running.

Hermes Agent or OpenClaw?#

Both are self-hosted agents that can use a local model. Hermes focuses on memory and skills that improve over time. OpenClaw focuses on a Gateway with a web page and chat apps. Our OpenClaw guide shows the other route.

The Hermes column comes from the official docs. The OpenClaw column is a summary from our own guide, not an official comparison. Moving over? hermes import-agent openclaw --dry-run previews what Hermes can import (memories, skills, settings) before you migrate.

Troubleshooting#

Exact error: no endpoint#

provider 'ollama' has no endpoint configured

You used --provider ollama without an address. Add it to config.yaml:

providers:
  ollama:
    base_url: "http://localhost:11434/v1"

Exact error: connection refused#

Ollama is not running. Start it with ollama serve (Linux also sudo systemctl start ollama).

Exact warning: no allowlist#

No user allowlists configured. All unauthorized users will be denied.

Add TELEGRAM_ALLOWED_USERS=<id> to .env and restart the gateway.

Exact warning: PID 1#

this process is PID 1 with no init above it

A Compose entrypoint: override skipped the image's init, so dead processes pile up. Remove the override or add init: true.

Known bugs#

Open on GitHub at the time of writing (read 2026-10-26), all in the v0.22.x line:

• #143402: context compaction write-back can fail on empty message ids, so a compression result is lost.
• #143197: the X-Hermes-Session-Key header is ignored by the API server (the X-Hermes-Session-Id header works).
• #143290: stopping a run that just finished can interrupt the next turn.
• If v0.22.5 misbehaves for you, set the image tag to v0.22.4 and recreate the container.

Diagnostics: hermes doctor, docker logs hermes --tail 100, docker compose ps.

Updates and backups#

Update with one command or one new tag, and back up the data folder before every update.

Native install:

hermes update

Docker: change the image tag, then:

cd ~/docker_volumes/hermes
docker compose pull && docker compose up -d

Your data folder is kept. When a config format change is needed, Hermes writes timestamped backups of config.yaml and .env first.

Backup (Linux / Mac)#

mkdir -p ~/docker_volumes/hermes/backups
tar czf ~/docker_volumes/hermes/backups/hermes-$(date +%F).tgz -C ~ .hermes

For the Docker setup use -C ~/docker_volumes/hermes data instead. Stop the container first (docker compose stop hermes) so the database is consistent, then start it again. The archive contains your keys: keep it private.

Backup (Windows)#

New-Item -ItemType Directory -Force "$HOME\docker_volumes\hermes\backups" | Out-Null
Compress-Archive -Path "$env:LOCALAPPDATA\hermes" -DestinationPath "$HOME\docker_volumes\hermes\backups\hermes-$(Get-Date -Format yyyy-MM-dd).zip"

Schedule: Linux/Mac cron at 03:00 (0 3 * * * <path-to-script>). Windows: Task Scheduler running the PowerShell line.

Restore test#

Do this once, before you need it. Restore into a scratch folder and look inside:

mkdir -p /tmp/restore && tar xzf ~/docker_volumes/hermes/backups/hermes-*.tgz -C /tmp/restore
ls /tmp/restore/.hermes

You should see config.yaml, .env, memories and skills. Then clean up with rm -rf /tmp/restore. To restore for real, stop Hermes, put the folder back and start it.

Performance and resources#

The model is the heavy part, not Hermes. We did not measure RAM or CPU ourselves. The docs give these numbers: about 10 tokens per second for a 9B model on an 8-core CPU, 2 to 5 for a 31B model. Measure your own machine:

docker stats --no-stream hermes ollama
ollama ps

Tips from the docs:

• Keep the model loaded: OLLAMA_KEEP_ALIVE=24h. Ollama unloads idle models after 5 minutes.
• A GPU, even partial offload, helps a lot. ollama ps shows how many layers are on the GPU.
• Hermes sends all tool descriptions on every call. Run hermes prompt-size, then switch off tools you do not use with hermes tools.
• Use /compress in long chats.

FAQ#

Does Hermes work fully offline? Yes for chat and local tools with a local model. Web search and Telegram obviously need the internet.

Which local model should I pick? One with tool calling and 64K context. The Hermes docs tested gemma4:31b. For llama.cpp they list Llama 3.x, Qwen 2.5 (including Coder), Mistral and DeepSeek as having native tool-call support.

Do I need Docker? No. The one-line installer works on Linux, Mac and native Windows. Docker is handy for an always-on Linux server.

Is it safe to let it run commands? Only with care. Keep approvals on, use a Docker terminal backend, allowlist your chat and never expose the dashboard or API without a login.

Can I switch between local and cloud models? Yes. Type /model in a chat, or run hermes model to add a new provider. The OpenRouter fallback in Step 5 does it automatically.

What does the cloud option cost? OpenRouter charges per token and the price depends on the model. The Hermes local guide estimates about $0.60 to $0.80 per heavy coding session on cloud models versus nothing locally (their estimate, not ours).

Sources#

• Caddy docs: automatic HTTPS, tls internal, reverse_proxy, install (Caddy 2.11.7, read 2026-10-26)
• Hermes Agent README, v0.22.5 (read 2026-10-26)
• Run Hermes Locally with Ollama, Providers, Docker setup, Security, Telegram (docs at v0.22.5, read 2026-10-26)
• GitHub releases, v0.22.5 (22 Oct 2026) and issues
• Docker Hub: nousresearch/hermes-agent and ollama/ollama
• Ollama library: gemma4

Checked against the official docs on 2026-10-26. Not run by DIY Lab Hub. Items marked VERIFY are not confirmed in the official docs.