Pipecat: Build a Real-Time Voice AI Agent with Local Models (Linux, Windows, Mac)

What you get in one minute#

Pipecat is a free, open-source Python framework for building real-time voice agents: you chain listening, thinking and speaking steps into one pipeline, and each step can run on your own computer or at a cloud service you choose.

Logo: the official Pipecat project (github.com/pipecat-ai/pipecat)

It is a toolkit, not an app. You write a short Python file, start it, open a page in your browser and talk to your agent. This guide builds the fully local version: Whisper turns speech into text, a model served by Ollama writes the answer, and Kokoro speaks it. No account or API key is needed. If you would rather click than code, our Dograh guide covers a ready-made voice agent platform; its build docs mention a pipecat submodule, so it appears to be built on Pipecat.

At a glance:

• What it is: a pipeline: microphone audio, speech to text (STT), a language model (LLM), text to speech (TTS), speaker audio.
• Local parts we use: Whisper (STT), Ollama (LLM), Kokoro (TTS), plus bundled Silero voice detection and Smart Turn end-of-turn detection.
• Cloud is a swap: Deepgram, OpenAI, Cartesia and many others plug into the same slots.
• The official quickstart is cloud: it needs Deepgram, OpenAI and Cartesia keys, so audio leaves your machine. Our route avoids that.
• Development tool: the built-in web runner is for your own machine; the docs say it is not built for production.
• Not tested by us: built from the official pages and examples, not run.

How it fits together#

Your browser sends microphone audio to the bot over WebRTC. The bot detects when you stop speaking, transcribes, asks the model, speaks the answer and sends the audio back.

Drawn for this guide from the official Pipecat docs and example code. Everything in teal runs inside one Python process; Ollama is a separate local program.

Before you start#

You need Ollama, a browser, a microphone and speakers; uv installs Python 3.12 for you. Pipecat publishes no RAM or GPU minimum, so these figures come from each part's own project.

Checklist:

• [ ] A microphone and speakers that work in your browser
• [ ] About 5 GB free disk space
• [ ] Ports 7860 and 11434 free
• [ ] No VPN or firewall rule blocking UDP (WebRTC uses it)
• [ ] Only for the HTTPS section: a name that points at your machine, and ports 80 and 443 free

Tested by DIY Lab Hub: not yet. Commands come from the official docs, examples and release files read in November 2026. We did not run them.

Step 1: Install uv and create the project#

uv is the Python tool the Pipecat docs use. It installs Python and locks package versions. We pin Pipecat to 1.15.0, because the project releases about weekly and renames things.

Linux#

curl -LsSf https://astral.sh/uv/install.sh | sh

Open a new terminal, then:

uv --version
uv init --python 3.12 my-voice-agent
cd my-voice-agent
uv add "pipecat-ai[runner,webrtc,whisper,kokoro,ollama]==1.15.0"

Windows (PowerShell)#

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Open a new PowerShell window, then:

uv --version
uv init --python 3.12 my-voice-agent
Set-Location my-voice-agent
uv add "pipecat-ai[runner,webrtc,whisper,kokoro,ollama]==1.15.0"

The uv docs also list winget install --id=astral-sh.uv -e.

Mac (Terminal)#

Same commands as Linux. The uv docs also list brew install uv.

The extras are Pipecat's own names: runner (web runner and page), webrtc (browser transport), whisper, kokoro and ollama. The local extra only installs PyAudio for a microphone on the same machine.

Check it worked. On every system, the version prints as 1.15.0 (expected, from the pin):

uv pip show pipecat-ai

Tip: uv creates a main.py you can ignore. Your agent goes in bot.py.

Step 2: Install Ollama and pull a model#

Ollama runs the language model (our Ollama guide goes deeper). Pipecat expects it at http://localhost:11434.

Linux#

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b

Windows (PowerShell)#

Download the installer from ollama.com/download and run it; Ollama then runs from the tray. Then:

ollama pull llama3.2:3b

Mac (Terminal)#

Download the Mac app from ollama.com/download and open it once. Then:

ollama pull llama3.2:3b

llama3.2:3b is a 2.0 GB model with a 128K context window (Ollama library). Any model that fits your RAM works; change the name in the code.

Check it worked. ollama -v prints a version (the Ollama docs use it to verify the install). ollama list shows the model, and a test question gets an answer:

ollama list
ollama run llama3.2:3b "Say hello in five words."

Tip: Ollama keeps a model in memory for 5 minutes after the last request (its FAQ). After a pause, the first reply is slower while the model loads again.

Step 3: Write the agent#

This is the official voice-kokoro.py example from the v1.15.0 tag with two swaps: Whisper instead of Deepgram, and Ollama instead of OpenAI. We did not run this combination, so treat it as a starting point. Save it as bot.py in my-voice-agent:

from loguru import logger

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker, ProcessorUnusablePolicy
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
    LLMUserAggregatorParams,
)
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.kokoro.tts import KokoroTTSService
from pipecat.services.ollama.llm import OLLamaLLMService
from pipecat.services.whisper.stt import Model, WhisperSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.workers.runner import WorkerRunner

# Only the browser (WebRTC) transport is offered.
transport_params = {
    "webrtc": lambda: TransportParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
    ),
}


async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
    logger.info("Starting bot")

    # Listening: Whisper, on this computer
    stt = WhisperSTTService(
        settings=WhisperSTTService.Settings(model=Model.BASE),
    )

    # Thinking: a model served by Ollama on this computer
    llm = OLLamaLLMService(
        base_url="http://localhost:11434/v1",
        settings=OLLamaLLMService.Settings(
            model="llama3.2:3b",
            system_instruction=(
                "You are a helpful assistant in a voice conversation. Your answers are "
                "spoken aloud, so use no emojis, lists or formatting. "
                "Answer in one or two short sentences."
            ),
        ),
    )

    # Speaking: Kokoro, on this computer
    tts = KokoroTTSService(
        settings=KokoroTTSService.Settings(voice="af_heart"),
    )

    context = LLMContext()
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
    )

    pipeline = Pipeline(
        [
            transport.input(),  # microphone audio from the browser
            stt,
            user_aggregator,
            llm,
            tts,
            transport.output(),  # speech audio back to the browser
            assistant_aggregator,
        ]
    )

    worker = PipelineWorker(
        pipeline,
        params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
        idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
        processor_unusable_policy=ProcessorUnusablePolicy.END,
    )

    runner = WorkerRunner(handle_sigint=runner_args.handle_sigint)
    await runner.add_workers(worker)

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        logger.info("Client connected")
        context.add_message(
            {"role": "developer", "content": "Please introduce yourself in one sentence."}
        )
        await worker.queue_frames([LLMRunFrame()])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        logger.info("Client disconnected")
        await runner.cancel()

    await runner.run()


async def bot(runner_args: RunnerArguments):
    """The runner calls this once per session."""
    transport = await create_transport(runner_args, transport_params)
    await run_bot(transport, runner_args)


if __name__ == "__main__":
    from pipecat.runner.run import main

    main()

The pipeline list is the order audio travels. The aggregators keep the conversation history. SileroVADAnalyzer spots speech, and Pipecat's default end-of-turn detector, Smart Turn v3, decides when you have finished a sentence (its weights are bundled). Settings go in settings=...Settings(...); passing model= straight to a service is deprecated.

Linux#

Create the file with any editor, for example nano bot.py, and paste the code.

Windows (PowerShell)#

Run notepad bot.py, choose to create the file, and paste the code. Save it as plain text with the name bot.py, not bot.py.txt.

Mac (Terminal)#

Create the file with nano bot.py or open -e bot.py and paste the code. On a Mac with Apple Silicon you can swap the speech-to-text line for Whisper's MLX version after uv add "pipecat-ai[mlx-whisper]". The docs' snippet:

from pipecat.services.whisper.stt import WhisperSTTServiceMLX, MLXModel

stt = WhisperSTTServiceMLX(
    settings=WhisperSTTServiceMLX.Settings(model=MLXModel.LARGE_V3_TURBO_Q4),
)

Check it worked. The file parses without errors:

uv run python -m py_compile bot.py

No output means success.

Tip: Copy Pipecat code only from the docs or examples at the version you installed. Examples written for older versions use class names that are now deprecated or gone (see Common mistakes).

Step 4: Start the agent and talk to it#

The first start downloads models and can look silent for a while. Keep the terminal open.

Linux#

uv run bot.py -t webrtc

Windows (PowerShell)#

uv run bot.py -t webrtc

Mac (Terminal)#

Same command as Linux.

-t webrtc limits the runner to the browser transport. When the server is up, open http://localhost:7860/client and click Connect. Allow the microphone when asked. Whisper and Kokoro load at connection time, so the first connection can take a minute; the bot then greets you.

Check it worked. The terminal shows output similar to this (expected, from the official quickstart, which adds a rocket symbol before the first line):

WebRTC server starting at http://localhost:7860/client
   Open this URL in your browser to connect!

You hear the greeting, and your sentences get spoken answers. Pipecat's metrics, including time to first byte, appear in the log because we set enable_metrics=True.

Per the docs, Whisper models come from the Hugging Face hub (cache ~/.cache/huggingface) and Kokoro stores its two files in ~/.cache/pipecat/kokoro-onnx/. On Windows ~ is your user folder.

Tip: On the very first run, wait for the downloads to finish in the log before you click Connect. Two silent minutes look like a hang but are usually a download.

Step 5: Choose what runs locally and what runs in the cloud#

Every stage is one line in bot.py. The README and the docs list these services; the docs index names the local ones.

Piper instead of Kokoro. The docs' snippet; the voice downloads on first use. Install with uv add "pipecat-ai[piper]==1.15.0":

from pipecat.services.piper.tts import PiperTTSService

tts = PiperTTSService(
    settings=PiperTTSService.Settings(voice="en_US-ryan-high"),
)

Two warnings. The piper-tts package is GPL-3.0 and runs inside your process, so distributing an app that includes it may bring source-disclosure terms (Pipecat's docs). The old rhasspy/piper repository is archived; the maintained one is OHF-Voice/piper1-gpl, which asks for new maintainers. Kokoro has no such catch (MIT code, Apache 2.0 model).

Whisper models. The default, distil-medium.en, is English only. For other languages use Model.TINY, BASE, SMALL, MEDIUM, LARGE or LARGE_V3_TURBO; language=None turns on auto-detection. Download sizes (Hugging Face): tiny 76 MB, base 145 MB, small 484 MB, medium 1,528 MB, distil-medium.en 789 MB.

A cloud service instead. The quickstart's pattern, with the key in an environment variable and never in the file (add import os at the top):

from pipecat.services.deepgram.stt import DeepgramSTTService

stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))

From then on your audio goes to that provider, and this guide's privacy claim no longer holds for that stage.

Linux#

Edit bot.py, restart with uv run bot.py -t webrtc, and keep the version pin on every extra: uv add "pipecat-ai[extra]==1.15.0".

Windows (PowerShell)#

Same steps. Set a key for this window with $env:DEEPGRAM_API_KEY = "<YOUR_KEY>".

Mac (Terminal)#

Same as Linux. Set a key with export DEEPGRAM_API_KEY="<YOUR_KEY>".

Check it worked. The bot starts without ImportError and answers. A missing extra prints, for example, In order to use Whisper, you need to uv add "pipecat-ai[whisper]": add it and retry.

Transports: how audio reaches the bot#

A transport connects the user and the bot; the client and the bot must use a matching pair. The docs' guidance:

For browser voice the docs call WebRTC the right choice, not a plain WebSocket, because WebSocket runs on TCP, which handles lost audio packets badly. Phone calls need a public webhook address; the docs' examples use a tunnel, which we do not teach. For phone calls without Python, see our Dograh guide.

Put it behind HTTPS (with or without Traefik)#

Skip this section if you only talk to the bot from the same computer. Browsers allow the microphone only in a secure context, and http://localhost counts as one (MDN). Opening http://<your-computer-address>:7860 from a phone blocks the microphone. Pipecat's docs say WebRTC apps must run over HTTPS.

The docs also say the runner is a development tool: POST /start accepts requests from anyone and nothing limits the rate. For a prototype they ask for a reverse proxy with authentication, rate limiting and TLS in front, while the runner stays on localhost. Audio skips the proxy: WebRTC media flows straight between browser and bot over UDP, which works on one local network. Other networks need STUN or TURN servers, which we do not cover.

With Traefik#

This bot is a normal Python program, not a container, so Traefik container labels do not apply, and we give no tested Traefik recipe for it. If you already run Traefik, point a router at http://<machine>:7860 and add authentication, or use Caddy.

Without Traefik: Caddy#

Caddy gets or makes its own certificates and can ask for a password. Install it as in our Caddy guide (the section "Run Caddy without Docker" covers all three systems; its Steps 4 and 5 cover the name and the root certificate). Make a password hash:

caddy hash-password

Save this as Caddyfile for a home-only name. For a public domain with ports 80 and 443 reachable, delete the tls internal line and Caddy uses Let's Encrypt:

voice.home.arpa {
    tls internal
    basic_auth {
        <your-user> <paste-the-hash-here>
    }
    reverse_proxy localhost:7860
}

Start the bot with the docs' --allowed-origins flag, so only your HTTPS page may talk to it.

Linux#

Put the site block in /etc/caddy/Caddyfile (the package's service reads it), then sudo systemctl reload caddy and, once, sudo caddy trust. Start the bot:

uv run bot.py -t webrtc --allowed-origins https://voice.home.arpa

Windows (PowerShell)#

caddy run --config Caddyfile

In a second window:

uv run bot.py -t webrtc --allowed-origins https://voice.home.arpa

Mac (Terminal)#

caddy run --config Caddyfile

In a second tab, start the bot as on Linux.

Check it worked. curl -I https://voice.home.arpa/ returns a status line over HTTPS; a 401 before login means the password check works. Open https://voice.home.arpa/client on your phone, log in and allow the microphone. The phone needs Caddy's root certificate installed (Caddy guide, Step 5).

Tip: A certificate warning on one device means that device has not trusted Caddy's root yet. With a public domain there is no warning.

Security and testing#

A voice agent hears your room. These controls matter most.

Quick tests after a change:

ollama list
uv run python -m py_compile bot.py
curl -I https://voice.home.arpa/

On Windows use curl.exe for the last line. With the local stack, the only network traffic we read about is the first-run downloads. The docs describe no usage telemetry, but we did not audit the code. Google's public STUN server is used only if the client sets enableDefaultIceServers; one network needs none. If you ever record audio, check your local laws first.

Common mistakes#

These come from the changelog, the docs' own warnings and the issue trackers.

Tip: When something breaks after an upgrade, read CHANGELOG.md for the version range before you debug your own code. Releases come about weekly.

Tip: Speak a normal sentence in your first tests. Whisper only transcribes segments the voice detector reports (docs), so a very short, quiet reply can be missed; issue #6539 (open) discusses missed utterances for streaming STT services.

Troubleshooting#

Exact messages from the 1.15.0 source: In order to use Kokoro, you need to uv add "pipecat-ai[kokoro]" and In order to use Whisper, you need to uv add "pipecat-ai[whisper]".

Known bugs and open items (November 2026)#

• kokoro-onnx #187 (open): Python 3.14 is not supported. Stay on 3.12.
• Pipecat #6539 (open): speech that the voice detector misses can leave a turn without a transcript for some streaming STT services.
• Pipecat #6042 (closed): a long-running server grew 20 to 50 MiB per finished SmallWebRTC session; maintainers said they were investigating. Restart long-running bots and watch memory.
• Production example in the docs: its Dockerfile still shows NLTK and a Silero download step, but the 1.15.0 pyproject.toml has no NLTK and the VAD docs say Silero is in the core package. Check before copying it.

Updates and backups#

Update. Read the changelog for every version in between, change the pin, then reinstall and fix any deprecation warnings:

uv add "pipecat-ai[runner,webrtc,whisper,kokoro,ollama]==<new-version>"
uv run bot.py -t webrtc

Back up bot.py, pyproject.toml and uv.lock. uv init also created a git repository, so a commit is a backup too. For a dated archive:

Linux#

cd ~/my-voice-agent
mkdir -p backups
tar -czf "backups/voice-agent-$(date +%F).tar.gz" bot.py pyproject.toml uv.lock

For 03:00 every night, add a crontab entry with the same commands (cron needs % written as \%).

Windows (PowerShell)#

Set-Location "$HOME\my-voice-agent"
New-Item -ItemType Directory -Force backups | Out-Null
Compress-Archive -Path bot.py, pyproject.toml, uv.lock -DestinationPath "backups\voice-agent-$(Get-Date -Format yyyy-MM-dd).zip" -Force

Schedule it in Task Scheduler at 03:00.

Mac (Terminal)#

Same as Linux; schedule with crontab -e or launchd.

Restore test. Unpack into a new empty folder, never over your project, and check the files:

mkdir -p /tmp/restore-test && tar -xzf backups/voice-agent-<date>.tar.gz -C /tmp/restore-test
ls /tmp/restore-test

On Windows use Expand-Archive into a new folder. Model caches need no backup; they download again.

Performance and resources#

The Pipecat docs publish no hardware or latency figure for local models. The docs' "typically under one second" round trip belongs to the cloud quickstart. We did not measure the local stack; measure yours.

To watch it: ollama ps shows loaded models and GPU use, and nvidia-smi, Task Manager, Activity Monitor or top show the rest.

FAQ#

Is Pipecat free? Yes, under the BSD 2-Clause licence. Cloud services you plug in bill you themselves. Pipecat Cloud, the maker's hosting product, is optional and unused here.

Does it need a GPU? No. Whisper and Kokoro run on a CPU, and so can small Ollama models. A GPU only makes it faster.

Does my voice leave my computer? Not with the local stack here. It does once you swap in a cloud service, and the official quickstart uses three.

Can I talk to it from my phone? Yes, on the same network once HTTPS is set up. Over the internet you need STUN or TURN servers, which we do not cover.

How is it different from Home Assistant voice or Speaches? Home Assistant voice is a finished smart-home assistant, and Speaches is a Whisper server with an OpenAI-style API. Pipecat is the framework for your own conversation flow.

Sources#

• Pipecat: release v1.15.0, README, LICENSE, pyproject.toml, CHANGELOG, voice-kokoro example and PyPI - read November 2026
• Pipecat docs: Quickstart, Running Bots Locally, Running Bots in Production, Development Runner, Small WebRTC, Choosing a Transport, Whisper, Ollama, Kokoro, Piper, Silero VAD and Smart Turn - read November 2026
• Parts: kokoro-onnx, faster-whisper, piper1-gpl, the archived rhasspy/piper, Ollama and llama3.2 - read November 2026
• Tools: uv, Caddy docs, Hugging Face Hub environment variables and MDN Secure Contexts - read November 2026
• Issues: Pipecat #4963, #5114, #6042, #6539; kokoro-onnx #187 - read November 2026

Checked against the official docs on 2026-11-05.