Skip to main content

Local AI

Campbooks asks for an AI role, never for a vendor. You choose what serves each role. The shipped defaults use open-weight models only, so every model can run on your own hardware.

The roles​

RoleWhat it is for
decideTyped decisions about an item: which option, what score, is this true. Each answer comes with a probability.
completeLonger generation, such as drafts and summaries.
embedSearch vectors.
visionOptional. Images and scanned documents.

A role left unset is simply unavailable. Features that need it do not run.

Start the local services​

The ai profile adds three services, one per role it serves.

docker compose --profile ai up -d
ServiceRoleModelNotes
layadecideconvaiinnovations/laya-multilingual (Apache-2.0, 322M)Built from deploy/laya. About 680 MB download on first start. About 2 GB of RAM.
ollamacomplete (and vision)qwen3:8bPull models with docker compose exec ollama ollama pull …
embeddingsembedBAAI/bge-m3linux/amd64 only. On arm64 use Ollama's bge-m3.

1. Set the decide API key​

The laya service refuses to start without a key, because laya-serve answers anyone when it has none. The server sends the same value. The service has no published port, so only containers on the compose network reach it.

openssl rand -hex 32

Put the result in .env:

CAMPBOOKS_AI__DECIDE__API_KEY=<the value you generated>

2. Pull the complete model once​

Laya and the embeddings server download their models themselves. Ollama needs one pull.

docker compose exec ollama ollama pull qwen3:8b

CPU or GPU​

  • decide (Laya) and embed run fine on CPU. Laya takes a few hundred milliseconds per item on a recent laptop.
  • complete works on CPU but is slow. Use a GPU host, or a hosted provider that serves open-weight models, and set its URL in .env.
  • To use an NVIDIA GPU with Ollama, uncomment the deploy block under the ollama service in compose.yaml and install the NVIDIA Container Toolkit.
  • Laya uses every CPU core by default. Set LAYA_THREADS on the laya service to cap it.

Use another endpoint​

Every role is configured by three settings: CAMPBOOKS_AI__<ROLE>__BASE_URL, CAMPBOOKS_AI__<ROLE>__MODEL and CAMPBOOKS_AI__<ROLE>__API_KEY. Point a role at any OpenAI-compatible endpoint (Ollama, vLLM, a hosted provider serving open-weight models):

CAMPBOOKS_AI__COMPLETE__BASE_URL=https://llm.example.com/v1
CAMPBOOKS_AI__COMPLETE__MODEL=your-model
CAMPBOOKS_AI__COMPLETE__API_KEY=...

For decide, set CAMPBOOKS_AI__DECIDE__PROTOCOL=openai to use an instruct model instead of Laya. It answers the same questions with JSON-schema output. Probabilities come from token log-probabilities when the server returns them. Without them, answers carry no confidence, so nothing that needs one acts on its own. Set CAMPBOOKS_AI__DECIDE__LOGPROBS=false for servers that reject the logprobs parameter.

CAMPBOOKS_AI__DECIDE__PROTOCOL=openai
CAMPBOOKS_AI__DECIDE__BASE_URL=http://ollama:11434/v1
CAMPBOOKS_AI__DECIDE__MODEL=qwen3:8b

On arm64, drop the embeddings service and point embed at Ollama:

CAMPBOOKS_AI__EMBED__BASE_URL=http://ollama:11434/v1
CAMPBOOKS_AI__EMBED__MODEL=bge-m3

About Laya​

  • There is no official image, so compose builds one from deploy/laya/Dockerfile. After an upgrade, rebuild it with docker compose --profile ai build laya.
  • On first start it downloads the multilingual checkpoint into the laya-models volume. Later starts load it from there.
  • CAMPBOOKS_AI__DECIDE__MODEL also accepts convaiinnovations/laya (English only, 512 tokens) and convaiinnovations/laya-typed-decisions. The image preloads only the multilingual one, so set LAYA_MODELS on the service too.
  • Out of the box the models are over-confident. The CAMPBOOKS_AI__DECIDE__CALIBRATION__* settings soften the probabilities. They default to 1.0.

Open-weight models only​

Only open-weight models appear in the shipped catalog. They can be self-hosted (vLLM, llama.cpp, Ollama, Text Embeddings Inference) or used through any hosted endpoint that serves open weights. All roles speak the OpenAI-compatible HTTP API, except Laya's own typed protocol for decide.

If you change the embedding model, stored vectors are tied to the old model. Embeddings record their model id and dimensions, and a re-embed job is part of the design.

Last updated on