Application Bootstrap and Embedding Guide

This document explains when and in what order LevPRO AI runtime components are initialized, and how to embed the runtime into another application. For class-level API details, see docs/api/interfaces.md.

Overview: Two Phases

Do not call agent_loop.run() until llama-server is healthy. Do not skip ctx.shutdown() — it stops the memory index watcher (when enabled), terminates llama-server process(es), closes MCP sessions, and closes the HTTP client pool.

Chat-only vs project mode

Capability tools (filesystem, shell, git, HTTP, …) always come from MCP — not from built-in Core packages. Runtime state (sessions, L0–L3, RAG, mcp.json) uses runtime_data_dir (default ~/.local-ai/).

Quick Start: Full Embed (Recommended)

Canonical pattern from packages/local-ai-core/interfaces/cli.py:

import asyncio

from agents.registry import get_agent_config
from app import create_app, default_agent_name
from config.loader import load_config
from core.shutdown import install_signal_handlers


async def main() -> None:
    config = load_config("config.yaml")
    ctx = create_app(config=config)
    install_signal_handlers(ctx)  # optional
    await ctx.start_supervisors()
    await ctx.startup()
    try:
        agent_name = default_agent_name(ctx.config)
        agent_config = get_agent_config(ctx.config, agent_name)
        result = await ctx.session_runner.run(
            user_id="user1",
            session_id="session_123",
            user_input="Hello",
            agent_config=agent_config,
            agent_name=agent_name,
        )
        print(result.response)
    finally:
        await ctx.shutdown()


asyncio.run(main())

Prerequisites:

• Valid config.yaml with llama.binary_path, llama.models.main.path, and at least one agent
• MCP servers in {runtime_data_dir}/mcp.json (or --mcp-config / LOCAL_AI_MCP_CONFIG) when the agent needs tools
• For RAG/L0–L3: pip install -e "packages/local-ai-core[memory]" and memory.enabled: true
• create_app() raises ModelPlanError if the model cannot fit in available RAM/VRAM

Optional extras (install + enable in config):

local-ai-mcp is a hard dependency of local-ai-core (not an optional extra). Enable via mcp.json / --mcp-config / LOCAL_AI_MCP_CONFIG.

Host injection of MCP: host_integrations.md.

Construction: create_app() Dependency Order

packages/local-ai-core/app.py remains the public composition root and wires dependencies synchronously in this order. Focused construction logic lives under core/bootstrap/: inference creation, model planning, resource/degradation wiring, and tool-registration validation. Runtime startup/shutdown ownership lives in core/app_context.py.

1. load_config() (or use passed config)
2. apply_env_overrides() (LOCAL_AI_MCP_CONFIG)
3. setup_logging()
4. merge_mcp_external() — load {runtime_data_dir}/mcp.json (or mcp.external_path) as sole server source
5. DirectoryGuard(runtime_root) — mkdir sessions/; optional workspace DirectoryGuard
6. Optional ensemble (when local-ai-ensemble installed and ensemble.enabled: true):

• build_ensemble_stack() → EnsembleRouter + one ProcessSupervisor per llama.models entry
• llm_client becomes the router; supervisors list holds all model supervisors

1. Single model (default): planner and/or autotune → ResolvedLlamaModelConfig

• ProcessSupervisor(binary_path, resolved model, …) → base_url
• LocalLLMClient(base_url, timeouts, model_name)

1. TokenCounter(llm_client) → ContextBuilder
2. SessionStore(guard, max_session_size)
3. Optional memory (when local-ai-memory installed and memory.enabled: true):

• build_memory_stack() → MemoryStack (store, retriever, manager, compressor, index_service, tools)
• mkdir memory_data/knowledge/
• memory_retriever = stack retriever; else NullMemoryRetriever

1. MCP (when mcp.enabled and servers from mcp.json):

• build_mcp_stack() → lazy mcp_* tools (stdio and/or HTTP)

1. Optional skills (skills.enabled) → SkillsStack (retriever + capture); else NullSkillsRetriever
2. Message archive tools when message_archive_enabled
3. build_tool_executor(config, …, extra_tools=..., auto_enable=mcp names)
4. PromptManager(base_dir=config directory)
5. AgentLoop(...) with retriever protocols (mcp_* + recovery tools auto-granted)
6. SessionRunner(agent_loop, max_concurrent_sessions)
7. Optional monitor stack when monitor.enabled
8. AppContext(...)

When an optional package is installed but its config flag is false, a warning is logged at startup.

Rules:

• Do not construct ProcessSupervisor before plan/autotune resolution (single-model path).
• Do not call memory_stack.index_service.startup() inside create_app() — use await ctx.startup() (also discovers MCP tools when allowed_tools: ["*"]).
• Prefer create_app() over manual wiring unless injecting mocks.
• Hosts may pass CLI overrides (--mcp-config) before create_app(config=...); env overrides are applied again inside create_app.

Runtime Lifecycle Checklist

start_supervisors()

Starts every supervisor in ctx.supervisors (ensemble mode) or ctx.supervisor (single-model). Use this instead of ctx.supervisor.start() directly so ensemble multi-model setups start all llama-server processes.

AppContext public surface

What ctx.startup() does

1. When memory_stack is wired: builds/refreshes the memory index and starts the watcher
2. When MCP servers request discover-all (allowed_tools: ["*"] or empty): connects and registers tools

Electron / stdio embedding

local-ai serve-stdio --config config.yaml --mcp-config path/to/mcp.json

Protocol: docs/stdio_protocol.md.

For in-process Python embedding, use SessionRunner as above.

Selective Component Embedding

External llama-server

1. Ensure health on config.llama.models.main.port (or config.llama.main.port property)
2. Skip await ctx.start_supervisors() if already healthy
3. TokenCounter still requires /tokenize on the running server

Failure Modes and Pitfalls

Related Documentation

• docs/architecture.md — layer responsibilities
• docs/api/interfaces.md — AppContext API reference
• docs/extension_points.md — adding tools and interfaces
• docs/mcp.md — MCP bridge
• AGENTS.md — AI coding assistant guide