API Reference: Core Modules

core/llm_client.py - LocalLLMClient

Purpose: HTTP client for communicating with llama-server.

class LocalLLMClient:
    def __init__(
        self,
        base_url: str,
        timeouts: LLMTimeoutConfig | None = None,
        max_retries: int = 3,
        circuit_breaker: CircuitBreaker | None = None,
        model_name: str | None = None,
        *,
        circuit_breaker_scope: str = "global",
    ) -> None

    async def chat(
        self,
        messages: list[dict],
        *,
        temperature: float = 0.7,
        max_tokens: int = 1024,
        scope_key: str | None = None,
    ) -> str

    async def tokenize(self, text: str) -> int
    async def health(self) -> bool
    async def reset_circuit_breakers(self) -> None
    async def close(self) -> None

Implementation Details

HTTP Endpoints:

• POST /v1/chat/completions — chat
• POST /tokenize — token counting (native)
• POST /v1/tokenize — token counting (fallback)
• GET /health — health check

Retry / circuit breaker:

• Up to 3 retries with exponential backoff
• Per-scope or global circuit breaker (circuit_breaker_scope)
• reset_circuit_breakers() called after supervisor restart recovery

Timeouts: LLMTimeoutConfig (connect, read, write, pool); trust_env=False on httpx client.

core/supervisor.py - ProcessSupervisor

Purpose: llama-server lifecycle with health watchdog and automatic restart.

class ProcessSupervisor:
    def __init__(
        self,
        binary_path: str,
        model_config: ResolvedLlamaModelConfig,
        logs_dir: str | Path = "logs",
        lora_manager: LoraManager | None = None,
        on_restart: Callable[[], Awaitable[None] | None] | None = None,
    ) -> None

    @property
    def base_url(self) -> str
    @property
    def restart_count(self) -> int

    async def start() -> None
    async def stop() -> None
    async def is_healthy() -> bool

Delegates process launch to llama_tools.llama_server_process.LlamaServerProcess.

Watchdog: polls every 5s; restarts on crash or >3 consecutive health failures; exponential backoff up to 30s.

on_restart: optional callback after successful health following restart (wired to LocalLLMClient.reset_circuit_breakers() in create_app()).

core/health.py - check_health

@dataclass
class HealthStatus:
    llama_healthy: bool
    circuit_state: str
    memory_index_files: int
    supervisor_restarts: int
    ok: bool

async def check_health(
    *,
    llama_client: Any,
    memory_store: Any | None = None,
    supervisor_restarts: int = 0,
) -> HealthStatus

Used by local-ai health.

llama_tools/llama_server_process.py

Shared llama-server subprocess management:

• ResolvedServerConfig — resolved launch parameters (including MTP flags)
• build_llama_server_args() — CLI argument list
• LlamaServerProcess — start/stop/health polling

Used by ProcessSupervisor (with watchdog) and ServerLauncher (benchmark, no watchdog).

agents/loop.py - AgentLoop

Per-agent system prompt:

• Preferred: agents.<name>.prompt_files — host-owned Jinja-rendered stack via PromptManager.build_prompt() (paths validated at create_app())
• Fallback when prompt_files is empty: bundled prompts/system.md or prompts/system_native.md via PromptManager (selected by app.tool_call_mode)
• Copy package templates next to your config to customize; do not edit installed package files

Registered native tools: report_inability; load_message when archive enabled; load_tool_result / search_tool_results when memory enabled.

Capability tools: mcp_* from host mcp.json (auto-granted). See docs/mcp.md.

For memory, ensemble, skills, monitor, and MCP see package docs in docs/.