Вернуться обратно

Resource Monitor

Resource Monitor

Скачать · Исходный код · О проекте

Resource Monitor

Optional background RAM/VRAM polling with graceful degradation. Implemented in local-ai-monitor and wired when local-ai-core[monitor] is installed and monitor.enabled: true.

Works alongside built-in degradation (crash/OOM recovery in GracefulDegradation). Monitor is the proactive path; degradation handles reactive restarts. When built-in degradation is enabled, the monitor does not independently handle the same crash event. Both paths synchronize the active model profile and can restore the original launch profile after a configurable stable period.

RAM pressure uses the same effective budget as the ModelPlanner: physical RAM minus llama.os_reserve_mb, plus SSD virtual RAM (volume free - llama.disk_reserve_mb) when the model path is on an SSD. KV/compute buffers must still fit in physical RAM.

Install

pip install -e "packages/local-ai-core[monitor]"

Configuration

monitor:
  enabled: true
  poll_interval_seconds: 10.0
  auto_degrade: true
  ram_headroom_mb: 1024
  vram_headroom_mb: 512
  min_ctx_size: 512
  degrade_cooldown_seconds: 60
  max_degrade_steps: 3

degradation:
  recovery_enabled: true
  recovery_stable_seconds: 120
  recovery_cooldown_seconds: 60
Field Default Purpose
enabled false Start background RAM/VRAM polling
poll_interval_seconds 10.0 Seconds between memory checks
auto_degrade true Auto CPU fallback / ctx reduction; false = warnings only
ram_headroom_mb 1024 Free RAM buffer before pressure warning
vram_headroom_mb 512 Free VRAM buffer before pressure warning
min_ctx_size 512 Floor for server ctx degradation
degrade_cooldown_seconds 60 Min seconds between degradation steps
max_degrade_steps 3 Max auto-degradation steps per session

Behavior

  • ResourceMonitor consumes the vendor-neutral SystemProbe snapshot (CUDA, ROCm/HIP on Linux or Windows, Vulkan, or Metal) and estimates model usage via split_memory()
  • ResourceMonitorService runs the poll loop; log events RESOURCE_PRESSURE, RESOURCE_DEGRADE
  • DegradationController may apply CPU fallback (gpu_layers=0), reduce ctx, or trim context_token_budget
  • before_restart hook on supervisors supports OOM crash recovery
  • after all monitored models remain healthy for recovery_stable_seconds, the runtime restores the original model and context profiles; a failed restart is rolled back to the last degraded profile

Wiring

packages/local-ai-core/app.py calls build_monitor_stack() when the extra is installed and enabled.

See AGENTS.md for tests: packages/local-ai-monitor/tests/test_monitor.py, test_service.py.