Multi-Model Ensemble

Optional per-agent model routing across multiple llama-server instances. Implemented in local-ai-ensemble and wired when local-ai-core[ensemble] is installed and ensemble.enabled: true.

Install

pip install -e "packages/local-ai-core[ensemble]"

Configuration

llama:
  models:
    main:
      path: ...
      port: 8080
    coder:
      path: ...
      port: 8081

ensemble:
  enabled: true
  default_model: main

agents:
  assistant:
    tools: [...]
    model: main
  coder:
    tools: [...]
    model: coder

Each key under llama.models gets its own ModelPlanner pass, ProcessSupervisor, and LocalLLMClient. AgentLoop uses EnsembleRouter as llm_client; routing picks agents.<name>.model or falls back to ensemble.default_model.

Startup

Use await ctx.start_supervisors() (not ctx.supervisor.start() alone) to start every model's llama-server process. ctx.supervisors lists all watchdogs; ctx.shutdown() stops each.

Wiring

When ensemble is enabled, create_app() skips single-model planning and calls build_ensemble_stack(). AppContext.ensemble_stack holds the router, clients, and resolved model configs.

See AGENTS.md for tests: packages/local-ai-ensemble/tests/test_ensemble_router.py.