CLI Documentation

Overview

The CLI (Command Line Interface) provides interactive and one-shot query capabilities for LevPRO AI runtime. It supports REPL mode for conversation and one-shot mode for single queries.

Purpose

1. Interactive chat - REPL mode for ongoing conversations
2. One-shot queries - Single input, single response
3. Session management - User and session isolation
4. Agent selection - Choose which agent to use
5. Configurable - Custom configuration per run

Responsibilities

1. Parse arguments - Command-line options
2. Create app context - Initialize app and dependencies
3. Start supervisor - Launch llama-server
4. Run agent - Execute agent via SessionRunner
5. Manage sessions - Create and track sessions
6. Handle shutdown - Clean termination

Dependencies

• argparse - Argument parsing
• asyncio - Event loop
• uuid - Session ID generation
• app.py - AppContext creation
• agents/registry.py - Agent config

Constraints

1. Session isolation - Each session is separate
2. User isolation - User ID separation
3. Single agent - One agent per run
4. Async execution - All agent calls async
5. Proper shutdown - Always call ctx.shutdown()

Extension Points

Adding a New Command

1. Add subparser in packages/local-ai-core/main.py:

sub = parser.add_subparsers()
new_cmd = sub.add_parser("newcommand")
new_cmd.add_argument("--arg", default="default")

1. Create interfaces/newcommand.py:

from app import create_app

async def main_async(args):
    ctx = create_app(args.config)
    await ctx.supervisor.start()
    try:
        # Your logic
    finally:
        await ctx.shutdown()

1. Register in packages/local-ai-core/main.py:

elif args.mode == "newcommand":
    from interfaces.newcommand import main as cmd_main
    cmd_main(args)

1. Add to config if needed

Module Details

interfaces/cli.py - CLI Interface

Methods:

run_repl()

async def run_repl(ctx, user_id, session_id, agent_name) -> None:
    agent_config = get_agent_config(ctx.config, agent_name)
    print(f"Local AI CLI — user={user_id} session={session_id} agent={agent_name}")
    print("Type 'exit' or 'quit' to stop.\n")
    
    while True:
        user_input = await asyncio.to_thread(input, "You> ")
        
        text = user_input.strip()
        if not text:
            continue
        if text.lower() in {"exit", "quit"}:
            break
        
        result = await ctx.session_runner.run(
            user_id, session_id, text, agent_config,
            agent_name=agent_name,
            resume=args.resume,
        )
        print(f"Assistant> {result.response}\n")

run_oneshot()

async def run_oneshot(ctx, user_id, session_id, agent_name, user_input) -> None:
    agent_config = get_agent_config(ctx.config, agent_name)
    result = await ctx.session_runner.run(
        user_id, session_id, user_input, agent_config,
        agent_name=agent_name,
        resume=args.resume,
    )
    print(result.response)
    for warning in result.warnings:
        print(f"Warning: {warning}", file=sys.stderr)

main_async()

async def main_async(args) -> None:
    config = apply_cli_overrides(load_config(args.config), args)
    ctx = create_app(config=config)
    await ctx.supervisor.start()
    try:
        session_id = args.session_id or str(uuid.uuid4())[:8]
        agent_name = args.agent or default_agent_name(ctx.config)
        
        if args.input:
            await run_oneshot(ctx, args.user_id, session_id, agent_name, args.input)
        else:
            await run_repl(ctx, args.user_id, session_id, agent_name)
    finally:
        await ctx.shutdown()

Command-Line Options

cli

local-ai cli [options]

Config overrides (replace matching config.yaml values when set):

Environment overrides (applied in create_app and stdio/CLI hosts):

See host_integrations.md.

Workspace

Optional project root via top-level workspace: in config.yaml (skills storage root when [skills] is enabled). Capability tools come from MCP (mcp.json), not built-in Core FS/git tools. Sessions, memory L0–L3, and RAG live under runtime_data_dir (default ~/.local-ai/), independent of workspace. There is no Core workspace file index.

Relative paths resolve from the process working directory, not from the config file location. serve-stdio also accepts --mcp-config (same as CLI); set workspace in the config passed to --config or via overrides.

Usage Examples

Interactive REPL:

local-ai cli

One-shot query:

local-ai cli --input "Read test.txt and tell me what it contains"

Custom user/session:

local-ai cli --user-id alice --session-id alice_session_1

Specific agent:

local-ai cli --agent coder

With custom config:

local-ai cli --config custom_config.yaml

Resume interrupted session:

local-ai cli --session-id abc12345 --resume

With feedback:

local-ai cli --input "hello" --thumbs up

Session Management

Session ID

• Auto-generated: first 8 characters of UUID
• User-specified: --session-id option
• Persists in: {runtime_data_dir}/sessions/{user_id}_{session_id}.json

User ID

• Isolation key for sessions and memory
• Default: "default_user"
• User-specified: --user-id option

Session File Format

{
  "messages": [
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi there!"},
    {"role": "user", "content": "How are you?"},
    {"role": "assistant", "content": "I'm doing well..."}
  ]
}

Agent Selection

Default: First agent in config.yaml

Specify: --agent <name>

Config:

agents:
  assistant:
    tools: []
  coder:
    tools: []
memory:
  enabled: true
  categories: ["knowledge", "work"]

Usage:

local-ai cli --agent coder

REPL Features

Commands

Output

You> Hello
Assistant> Hi there! How can I help you?

You> Read test.txt
Assistant> The file contains: Hello from test.txt

Error Handling

Errors from agent loop are printed to stderr:

Error: Connection refused - llama-server not healthy

Shutdown

Proper shutdown:

await ctx.shutdown()
    - Stops memory index watcher
    - Cancels watchdog task
    - Terminates llama-server gracefully
    - Closes HTTP client

Signal handlers (core/shutdown.py) set shutdown_event; AgentLoop saves an interrupted L2 checkpoint before exiting. The stdio cancel method sets a per-run cancel_event without triggering app shutdown.

Always in finally block:

try:
    await run_agent(...)
finally:
    await ctx.shutdown()

Testing

Key test cases:

• REPL mode with multiple messages
• One-shot mode
• Exit/quit commands
• Empty input handling
• Agent selection
• Session persistence
• Proper shutdown

Run: python -m pytest packages/local-ai-core/tests/test_cli.py -v

Usage Patterns

Quick Query

# Single question
local-ai cli --input "What's the weather like?"

# Read and process file
local-ai cli --input "Analyze the code in main.py"

Interactive Session

# Start REPL
local-ai cli

# Conversation
You> Tell me about Python
Assistant> Python is...

You> Now explain async/await
Assistant> Async/await is...

You> exit

Custom Agent

# Use coding agent
local-ai cli --agent coder --input "Review this code"

# Use custom config
local-ai cli --config dev_config.yaml

Security

1. Session isolation - Each session independent
2. User isolation - User ID separation
3. Path sandbox - Core runtime I/O through DirectoryGuard (runtime_data_dir; optional workspace for skills only)
4. Capability tools - Filesystem/shell via host MCP (mcp.json), not built-in Core tools
5. Local inference - Agent LLM traffic goes to llama-server; MCP HTTP servers use host-configured endpoints

Integration with App Context

# CLI loads config, applies overrides, then creates AppContext
config = apply_cli_overrides(load_config(args.config), args)
ctx = create_app(config=config)
await ctx.supervisor.start()
    - Starts llama-server
    - Initializes watchdog
try:
    # Run agent
    result = await ctx.session_runner.run(...)
finally:
    await ctx.shutdown()
        - Clean shutdown

Performance

• Async execution - No blocking I/O
• SessionRunner - Semaphore for max_concurrent_sessions
• Efficient memory - Session history limited by config; L1 working memory compression
• Fast startup - Reuse existing llama-server

Architecture Rules for CLI

1. Single SessionRunner / AgentLoop - Don't duplicate agent logic
2. Proper shutdown - Always call ctx.shutdown()
3. Async execution - Use await for agent calls
4. Session management - Isolate by user/session
5. Config loading - load_config() → apply_cli_overrides() → create_app(config=...)
6. Error handling - Safe error messages to user