LevPRO AI runtime - Planner/Doctor Architecture

Overview

This document shows the architectural changes made to eliminate duplicate calculations between Planner and Doctor modules.

Dependency Diagram

Component Responsibilities

Bootstrap

packages/local-ai-core/app.py and packages/llama-tools/llama_tools/cli/

• ModelPlanner(os_reserve_mb=config.llama.os_reserve_mb) — created inline at bootstrap
• No separate factory module; construction is a one-liner at each entry point

Core Layer (Single Source of Truth)

ModelPlanner (planner.py)

• Owner of all calculations:
• KV cache size estimation
• RAM/VRAM requirements
• Memory fit checks
• Launch parameter optimization
• WARNING generation
• RECOMMENDATION generation

• Plan method signature:

``python
def plan(
self,
model_path: str,
mode: OptimizationMode,
metadata: ModelMetadata | None = None, # Pre-loaded metadata (optional)
overrides: LaunchOverrides | None = None,
system: SystemInfo | None = None,
) -> PlanResult
``

• Returns PlanResult with:
• can_run: bool
• reason: str | None
• recommended: LaunchParams | None
• estimated: PlanEstimate | None
• warnings: list[PlanWarning]
• recommendations: list[Recommendation]
• minimum_required: MinimumResources | None
• metadata: ModelMetadata (input snapshot)
• system: SystemInfo (input snapshot)

Estimator (estimator.py)

• Provides estimation formulas as private functions
• Never called directly by Doctor
• Called only by ModelPlanner.plan()

SystemProbe (system_probe.py)

• Detects system hardware (RAM, CPU, GPU)
• Provides system info for planning

GGUFReader (gguf_reader.py)

• Reads GGUF metadata files
• Called by ModelPlanner.plan() if metadata not provided

Adapter Layer (Passive)

Doctor (doctor.py)

• PASSIVE adapter - NO calculations, NO warnings, NO recommendations generation
• Constructor: __init__(self, planner: ModelPlanner)
• Method: check_model(metadata, system, mode) -> ModelCheckResult
• Only responsibility: Call Planner.plan() and map results
• Returns ModelCheckResult which is a direct mapping of PlanResult

ModelCheckResult

• Fields:
• file_size: int
• quantization: str
• layers: int
• context_length: int
• ram_needed_mb: float
• vram_needed_mb: float
• can_run: bool
• reason: str | None
• warnings: list[PlanWarning] (passed through from Planner)
• recommendations: list[Recommendation] (passed through from Planner)

Interface Layer

llama_tools/cli/doctor.py

• CLI entry point for doctor command
• Uses run_diagnostics() which internally:

1. Creates Planner via factory
2. Creates Doctor with Planner
3. Doctor calls Planner for model checks

• Formats output for display

Types Layer

types.py

PlanResult (from Planner)

• OUTPUT fields (calculations):
• can_run
• reason
• recommended
• estimated
• warnings
• recommendations
• minimum_required
• INPUT fields (parameters, for display):
• metadata
• system

ModelCheckResult (from Doctor)

• Direct mapping of PlanResult fields
• Used for diagnostics display

Recommendation

• Dataclass with:
• code: str (e.g., "RAM_ESTIMATE", "CONTEXT_SUGGESTION")
• message: str
• severity: str ("info"|"suggestion"|"warning"|"error")
• field: str | None (optional field reference)

PlanWarning

• Planner-generated warnings with:
• field: str
• message: str
• requested: Any
• resolved: Any

Data Flow

1. User runs: doctor --config config.yaml
   ↓
2. llama_tools/cli/doctor.py.run_doctor()
   ↓
3. run_diagnostics() creates:
   - planner = ModelPlanner(os_reserve_mb=...)
   - doctor = Doctor(planner)
   ↓
4. For each model in config:
   - metadata = GGUFReader.read(model_path)
   - result = doctor.check_model(metadata, system, mode)
   ↓
5. Doctor.check_model() calls:
   - plan = planner.plan(model_path, metadata, mode, system)
   ↓
6. Planner.plan() does:
   - Reads metadata (if not provided)
   - Detects system (if not provided)
   - Runs estimator functions
   - Generates warnings (PlanWarning objects)
   - Generates recommendations (Recommendation objects)
   - Returns PlanResult
   ↓
7. Doctor.check_model() maps:
   - PlanResult → ModelCheckResult
   - NO modifications, NO calculations
   ↓
8. run_diagnostics() returns SystemDiagnostics
   ↓
9. llama_tools/cli/doctor.py formats and displays

Key Design Decisions

1. Passive Doctor Pattern

Doctor is a COMPLETELY PASSIVE adapter. It does not:

• Call estimator functions directly
• Generate recommendations or warnings
• Perform any calculations
• Create ModelPlanner instances

This ensures:

• Single source of truth (Planner owns all logic)
• Consistent behavior between Planner and Doctor
• Easier to maintain and test

2. Structured Recommendations

Replaced recommendations: list[str] with recommendations: list[Recommendation]:

@dataclass
class Recommendation:
    code: str
    message: str
    severity: str  # "info"|"suggestion"|"warning"|"error"
    field: str | None = None  # Optional field reference

Benefits:

• Type-safe severity levels
• Machine-readable codes
• Optional field references
• Easy filtering and filtering by severity

3. Pre-loaded Metadata

Planner.plan() accepts optional metadata: ModelMetadata | None:

• If None, reads from model_path (original behavior)
• If provided, uses it without re-reading
• Enables caching and reuse of metadata

4. Input Parameter Documentation

PlanResult includes metadata and system as INPUT fields:

• Useful for display/debugging
• Snapshot of parameters used
• Documented clearly in docstrings

5. Inline ModelPlanner construction

ModelPlanner is instantiated directly at bootstrap (app.py, CLI doctor/plan commands). A dedicated planner_factory module was considered but not added to the monorepo; inline construction is sufficient.

Benefits

1. No Duplication: All calculations in Planner only
2. Single Source of Truth: Doctor never makes its own calculations
3. Testability: Doctor is trivial to test (just verify it passes through Planner's results)
4. Extensibility: New calculations only need to be added to Planner
5. Consistency: Planner and Doctor always produce identical results