karakeep-ollama-router

A tiny reverse proxy that lets Karakeep split its embedding and inference (tagging/summarization) traffic across two different Ollama backends, even though Karakeep itself only supports a single OLLAMA_BASE_URL.

Why this exists

Karakeep's config has one OLLAMA_BASE_URL shared by everything — embeddings, text tagging, and image tagging all hit the same endpoint. That's fine if you run one Ollama instance for all of it, but it stops working the moment you want to put embeddings on a different GPU than tagging (for example: a small dedicated card that keeps the embedding model always warm, while a bigger shared card handles the heavier tagging model on demand).

This proxy sits in front of that single OLLAMA_BASE_URL and picks the real backend per request, by looking at the "model" field in the JSON body:

• request asks for your configured embedding model → forwarded to EMBED_UPSTREAM
• anything else (tagging, summarization, image models) → forwarded to TAG_UPSTREAM

Point Karakeep's OLLAMA_BASE_URL at this proxy instead of at Ollama directly, and both backends stay fully independent — no code changes to Karakeep needed.

How it works

Plain Python standard library, no dependencies. It reads the request body, checks the model field, and forwards the request (headers + body) to whichever upstream matches, streaming the response back to the client as it arrives. Requests with no JSON body or no model field (health checks, /api/tags, etc.) fall through to TAG_UPSTREAM as the default.

Configuration

All via environment variables:

Variable	Description
EMBED_UPSTREAM	Base URL of the Ollama instance serving your embedding model
TAG_UPSTREAM	Base URL of the Ollama instance serving your tagging/summarization models
EMBED_MODEL	The exact model name that should route to EMBED_UPSTREAM (e.g. mxbai-embed-large)
LISTEN_PORT	Port to listen on (default 8000)

Running it

docker build -t karakeep-ollama-router:latest .
docker run -d \
  -e EMBED_UPSTREAM=http://ollama-embed:11434 \
  -e TAG_UPSTREAM=http://192.168.1.196:11434 \
  -e EMBED_MODEL=mxbai-embed-large \
  -p 8000:8000 \
  karakeep-ollama-router:latest

Then in Karakeep's config:

OLLAMA_BASE_URL: http://<this-container>:8000

Scope

This is a narrow, single-purpose routing proxy — it does not repair or modify model output in any way. If you're looking for something that fixes malformed tool-call output from a specific model, that's a different tool with a different job.

License

GPL-3.0 — see LICENSE.

---

🤖 Disclaimer: this project was built with the help of Claude (Sonnet 5), Anthropic's AI assistant.