AI Context Documentation

Overview

This document describes the AI context used by the LevPRO AI runtime agent. It defines the system prompt structure, available tools, memory retrieval, and how the agent interacts with the environment.

Agent System Prompt

Structure

You are {agent_name} with access to the following tools and capabilities:

## Tools

{tool_descriptions}

## Memory

You have access to a knowledge memory system organized by categories.
For each query, relevant context will be provided from memory.

## Memory Categories

{memory.categories}

## Rules

1. Always use tools when needed
2. For tool calls, respond with ONLY a JSON object
3. When finished, respond with plain text only
4. Do not invent capabilities
5. Be honest about limitations

## Guidelines

{additional_guidelines}

Components

Agent Name / base prompt

Default instructional text comes from the bundled templates
packages/local-ai-core/prompts/system.md (JSON tool mode) or system_native.md
(native tool mode) when agents.<name>.prompt_files is omitted.
Customize by copying those templates (or prompts/executor/ / prompts/planner/
stacks) next to your host config and setting prompt_files — do not edit files
inside the installed package.

Tool Descriptions

Each tool is described with:

• Name
• Description
• JSON Schema

Example:

## mcp_filesystem_read_file

Description: Read a text file via a host MCP filesystem server (example name).

Schema:
{"type":"object",
 "properties":{"path":"string"},
 "required":["path"],
 "additionalProperties":false}

Built-in Core file tools were removed; real tool names follow mcp_{server}_{tool} from mcp.json.

Memory Categories

Categories configured per agent:

agents:
  assistant:
memory:
  enabled: true
  categories: ["knowledge", "personal", "work"]

Example:

## Memory Categories

- knowledge: Factual information, reference materials, tutorials
- personal: Personal notes, preferences, reminders
- work: Work-related documents, projects, tasks

Additional Guidelines

Custom guidelines go in your host prompt markdown via agents.<name>.prompt_files:

You are a coding assistant...

Guidelines:
1. Always validate code before suggesting execution
2. Follow Python best practices
3. Provide explanations for complex code
4. Suggest tests for new functionality

Tool Call Format

Expected Format

LLM must emit tool calls as strict JSON:

Format 1: Raw JSON

{"tool": "mcp_filesystem_read_file", "args": {"path": "test.txt"}}

Format 2: Fenced code block

{"tool": "mcp_filesystem_read_file", "args": {"path": "test.txt"}}

Valid Formats

Both formats are accepted by agents/tool_parser.py.

Invalid Formats

• Plain text description
• Non-JSON content
• Missing "tool" key
• Non-dict "args"

Tool Execution Flow

1. LLM emits tool call (JSON or fenced JSON)
2. ToolParser.parse_tool_call() → parsed tool call or None
3. If tool call:
   - ToolExecutor.execute() → ToolResult
   - Large successful output (> app.tool_result_inline_limit UTF-8 bytes)?
     → saved to MemoryManager L3; observation is {result_id, summary, size} JSON
   - Otherwise → full content inline in observation
   - AgentLoop appends observation to history
4. If no tool call or error:
   - Finalize response

Example Interaction

LLM:

{"tool": "mcp_filesystem_read_file", "args": {"path": "test.txt"}}

Tool Result (small file, inline):

Hello from test.txt

Tool Result (large file, externalized):

{"result_id": "a1b2c3...", "summary": "first N chars…", "size": 128450}

(summary length is app.tool_result_summary_max_chars, default 1000)

LLM:

The file contains: Hello from test.txt

(or uses summary / result_id reference for large outputs)

Memory Retrieval

3-Stage Retrieval

Memory retrieval uses a 3-stage process:

Stage 1: BM25 / Hybrid filtering

• bm25_search() ranks index entries (default); optional hybrid_search() when memory.search_mode: hybrid
• No LLM call
• Returns top 5 candidates (TOP_N)

Stage 2: LLM File Selection

• LLM selects from filtered candidates
• Returns up to 5 files
• JSON array of filenames

Stage 3: Content Loading

• Loads selected files
• Tracks token count
• Returns concatenated content

Index Structure

{
  "index_version": 3,
  "last_updated": 1686234567.123,
  "files": [
    {
      "filename": "user1_notes.md",
      "title": "Personal Notes",
      "description": "Collection of personal notes...",
      "keywords": ["notes", "personal", "reminders"]
    }
  ]
}

Working Memory (L1)

WorkingMemoryCompressor maintains a compact markdown summary at scoped {runtime_data_dir}/memory/working/{user_id}_{session_id}.md:

• Read at the start of each AgentLoop.run() and injected as Working memory: in the system prompt
• Written after each run via update_summary(run_messages) with sections: Current Goal, Completed, Important Discoveries, Open Issues, Decisions
• Budget: memory.working_memory_target_tokens (default 768)
• Session compact: when message count exceeds history_limit, older messages are dropped after compression (working_memory_compact_keep_last)

Context Building

ContextBuilder.build() assembles the message context:

system_parts = [system_prompt, f"Available tools:\n{tools}"]
if working_memory:
    system_parts.append(f"Working memory:\n{working_memory}")
if agent_state:
    system_parts.append(f"Agent state:\n{agent_state}")
if rag_memory:
    system_parts.append(f"Memory context:\n{rag_memory}")

messages = [{"role": "system", "content": "\n\n".join(system_parts)}]
messages += history
messages.append({"role": "user", "content": user_input})

Trim priority when over context_token_budget: oldest history → RAG memory → working memory last.

Token Budgeting

Budget Enforcement

ContextBuilder enforces a token budget:

budget = config.app.context_token_budget  # Default: 6000

# Build context
context = assemble(system, tools, memory, history, user_input)

# Check budget
if count_tokens(context) > budget:
    # Trim history from oldest
    context = trim_to_budget(context, budget)

History Limit

History is limited by config.app.history_limit (default: 12 messages):

recent_history = history[-8:]  # Last 8 messages

Memory Token Limit

Memory content is limited by remaining budget:

remaining = budget - count_tokens(context_without_memory)
memory_content = load_memory(user_id, max_tokens=remaining)

Session Management

Session Structure

Sessions are stored in JSON files:

{runtime_data_dir}/sessions/{user_id}_{session_id}.json

Format:

{
  "messages": [
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi there!"},
    {"role": "user", "content": "How are you?"},
    {"role": "assistant", "content": "I'm doing well..."}
  ]
}

Session Flow

1. AgentLoop.run(user_id, session_id, input, config)
2. SessionStore.append(user_id, session_id, {"role": "user", "content": input})
3. Load history from session
4. Run agent loop
5. SessionStore.append(user_id, session_id, {"role": "assistant", "content": response})

Isolation

• Each session is independent
• Each user has isolated sessions
• Sessions persist across app restarts

Agent Config

Config Structure

agents:
  assistant:
    tools: []   # mcp_* from mcp.json are auto-granted
memory:
  enabled: true
  categories: ["knowledge"]

System prompt: host agents.<name>.prompt_files, or bundled system.md /
system_native.md when omitted (see PromptManager).

Tool Configuration

• Native recovery tools — report_inability (always); load_message / load_tool_result / search_tool_results when archive/memory are enabled; auto-granted
• MCP tools — mcp_{server}_{tool} from host mcp.json; auto-granted
• *agents..tools** — optional explicit native names only (usually [])
• Runtime Available tools: comes from ToolExecutor.list_tools()

Memory Configuration

Categories are configured globally in memory.categories. Each category:

• Creates a subdirectory
• Stores Markdown files
• Indexed for retrieval

Error Handling

Tool Errors

Errors are wrapped safely:

# ToolExecutor wraps exceptions
try:
    result = await tool.func(**args)
except Exception as exc:
    log_event("ERROR_STACKTRACE", f"tool={name} error={exc}")
    result = f"Error: tool '{name}' failed during execution"

LLM never sees stack traces.

LLM Errors

If the response is not a valid tool call:

# Plain text → final answer; salvageable JSON failures → re-prompt (up to max_parse_errors)
outcome = parse_tool_call(response)
if outcome.tool_call is None and not outcome.is_salvageable_failure:
    return response  # Final answer

Context Errors

If context exceeds budget:

# Trim history from oldest
context = trim_to_budget(context, budget)

Examples

Example 1: File Reading (via MCP)

Requires a filesystem MCP server in mcp.json. Tool name follows mcp_{server}_{tool}.

User: "Read the contents of main.py"

LLM:

{"tool": "mcp_filesystem_read_file", "args": {"path": "main.py"}}

Tool Result:

File contents:
#!/usr/bin/env python3
import asyncio
...

LLM:

The main.py file contains the main entry point with argument parsing and mode dispatch.

Example 2: Multi-step (one MCP tool)

User: "Read test.txt and tell me how many words it has"

LLM:

{"tool": "mcp_filesystem_read_file", "args": {"path": "test.txt"}}

Tool Result:

Hello world

LLM (final):

The file contains 2 words.

Example 3: Memory Retrieval

User: "What are my project preferences?"

MemoryRetriever:

• Stage 1: Finds "preferences.md" (score: 15)
• Stage 2: LLM selects "preferences.md"
• Stage 3: Loads content

LLM sees:

## user_preferences.md

My preferences:
- Dark mode preferred
- Coffee over tea
- Weekend mornings are for relaxation
...

LLM:

Based on your preferences, you prefer dark mode and coffee over tea.

Best Practices

For Developers

1. Keep system prompts clear - Use bullet points, sections
2. Provide tool schemas - Include JSON Schema for each tool
3. Describe memory categories - Explain what each category contains
4. Test tool calls - Verify LLM follows JSON format
5. Monitor token usage - Adjust budget as needed

For Users

1. Be specific - Clear queries get better results
2. Use memory - Organize knowledge for retrieval
3. Iterate - Multi-turn conversations work best
4. Check memory - Review memory content for accuracy

Testing

Test System Prompts

Verify system prompts are correct:

def test_system_prompt_includes_tools():
    from core.prompts import PromptManager
    prompt = PromptManager().load_system_prompt()
    assert "JSON" in prompt

def test_system_prompt_loads():
    from core.prompts import PromptManager
    prompt = PromptManager().load_system_prompt()
    assert len(prompt.strip()) > 0

Test Tool Calls

Verify LLM emits valid tool calls:

def test_llm_emits_valid_json():
    response = await llm.chat(context)
    # Verify response is valid JSON or final answer
    if "tool" in response:
        assert isinstance(response, dict)
        assert "tool" in response
        assert "args" in response

Test Memory Retrieval

Verify memory retrieval works:

def test_memory_retrieval_finds_relevant():
    memory = await retriever.retrieve(
        query="python basics",
        user_id="test_user",
    )
    assert "python" in memory.lower()

---

Summary

The AI context includes:

1. System prompt - Agent identity and capabilities
2. Tool descriptions - Available tools with schemas
3. Memory categories - Knowledge organization
4. Conversation history - Previous turns
5. Memory content - Retrieved knowledge

All components work together to enable the agent to:

• Understand user intent
• Use tools when needed
• Retrieve relevant knowledge
• Provide helpful responses