Local Models¶

BDPD supports running LLM agents against locally-hosted models via llama-server or Ollama. No external API key required.

llama-server (Recommended)¶

llama-server is bundled with llama.cpp. It exposes an OpenAI-compatible HTTP API on a configurable port.

Setup¶

# Install llama.cpp (system package or from source)
# Arch Linux:
sudo pacman -S llama.cpp

# macOS:
brew install llama.cpp

# From source:
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j

Running the Arena¶

LLM_MODE=local \
  MODEL_PATH=/path/to/model.gguf \
  LLM_PORT=10000 \
  LLAMA_EXTRA_ARGS="-c 4096 -ngl 99" \
  ./scripts/run_arena.sh

Variable	Default	Description
`MODEL_PATH`	(required)	Path to `.gguf` model file
`LLM_PORT`	`10000`	Port llama-server listens on
`LLAMA_EXTRA_ARGS`	`-c 4096`	Extra llama-server flags (`-ngl 99` for GPU offload)

Card Game with Local Model¶

LLM_MODE=local MODEL_PATH=/path/to/model.gguf \
  python agents/cards_ai_play.py \
    --deck1 stranger --deck2 temple \
    --cards cards/cards_v03.json \
    --api-base-url http://localhost:10000/v1 \
    --games 5

Ollama¶

Ollama provides a simplified local model runner. BDPD connects to it via the OpenAI-compatible API endpoint.

Setup¶

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull deepseek-r1:7b
ollama pull qwen2.5:7b

# Start the server (if not already running)
ollama serve

Running with Ollama¶

LLM_MODE=api \
  BDPD_API_KEY=ollama \
  API_BASE_URL=http://localhost:11434/v1 \
  API_MODEL=deepseek-r1:7b \
  ./scripts/run_arena.sh

Ollama's BDPD_API_KEY can be any non-empty string (Ollama ignores it but the OpenAI client library requires one).

Hardware Requirements¶

Model Size	RAM (approx)	GPU VRAM (if GPU offload)	Recommended GPU
7B Q4_K_M	4–6 GB	4 GB	RTX 3060+
13B Q4_K_M	8–10 GB	8 GB	RTX 4070+
34B Q4_K_M	20–24 GB	20 GB	RTX 4090 / A6000
70B Q4_K_M	40–48 GB	40 GB	2× RTX 4090 / A100

Without GPU offload, inference runs on CPU and is significantly slower (30–60 seconds per decision vs 2–5 seconds with GPU).

Performance Notes¶

Factor	Impact
Context size	Platform games run 20–60 turns with history window ≤ 5. Card games run 8–12 rounds with history window ≤ 8. Both fit within 4K context.
Thinking mode	Disable thinking/chain-of-thought for speed (`--no-thinking` in llama-server, or use non-reasoning models).
Decision latency	GPU-offloaded 7B models: 1–3 seconds per turn. CPU-only: 10–60 seconds.
Batching	Not applicable — BDPD queries the LLM sequentially per turn.
Temperature	Use low temperature (0.0–0.3) for reproducible experiments.

Troubleshooting¶

llama-server fails to start on port 10000

# Check if something is already on that port
ss -tlnp | grep 10000

# Use a different port
LLM_PORT=10001 ./scripts/run_arena.sh

Model runs out of context

Increase context size (may need more RAM):

LLAMA_EXTRA_ARGS="-c 8192" ./scripts/run_arena.sh

Ollama connection refused

# Ensure Ollama server is running
ollama serve &

# Verify API is reachable
curl http://localhost:11434/api/tags