Вернуться обратно

HTTP API (OpenAI-compatible)

HTTP API (OpenAI-compatible)

Скачать · Исходный код · О проекте

HTTP API (OpenAI-compatible)

LevPRO AI runtime can expose an OpenAI-compatible HTTP API for host applications. Chat runs through SessionRunner (same path as CLI / stdio). Streaming uses SSE with AgentLoop events (parity with stdio_protocol.md) plus OpenAI chat.completion.chunk frames for text deltas.

Requires the optional extra:

pip install "local-ai-core[http]"

Launch

local-ai serve-http --config config.yaml

CLI overrides (optional):

--host HOST
--port PORT
--bearer-token TOKEN
--mcp-config PATH

Supervisors and ctx.startup() run eagerly at process start (unlike lazy serve-stdio).

Configuration

http:
  host: 127.0.0.1      # default
  port: 8080           # default
  bearer_token: null   # when set, require Authorization: Bearer <token>

Port collision: http.port must differ from llama.models.*.port (llama-server). The example config uses Core on 8080 and llama-server on 8081.

When bearer_token is set, /v1/* endpoints require a matching Bearer token. /health stays open for probes.

Endpoints

Method Path Auth Description
GET /health no Liveness / readiness (llama health, circuit, degradation)
GET /v1/models if token set Lists config agents as OpenAI model objects
POST /v1/chat/completions if token set Agent turn (OpenAI body + Local AI extensions)

POST /v1/chat/completions

Request (OpenAI + extensions):

Field Type Default Description
model string first agent Agent name from agents:
messages array required OpenAI messages; only the last user message is appended — prior turns come from SessionStore
stream bool false SSE when true
user string default_user Session user_id
session_id string random 8-char Session id
resume bool false Resume L2 checkpoint
approve_execution bool false Auto-approve gated tools for this request

Multimodal: last user content may be a string or an array of {type: text|image_url} parts (same as stdio).

Non-stream response: OpenAI chat.completion object plus:

"local_ai": {
  "session_id": "...",
  "trace_id": "...",
  "warnings": [],
  "degradation_tier": "NORMAL"
}

Stream (stream: true): text/event-stream

  1. Named AgentLoop events (stdio parity), e.g.:
event: run_start
data: {"user_input":"...","trace_id":"..."}

event: llm_chunk
data: {"step":0,"delta":"Hello"}

event: tool_start
data: {"step":1,"tool":"...","args":{...}}

event: final
data: {"step":2,"response":"..."}
  1. For each llm_chunk, also an OpenAI chunk (object: chat.completion.chunk, delta.content) so standard OpenAI clients receive text.
  1. After the run: OpenAI final chunk (finish_reason: stop), event: result (completion + stdio-shaped fields), then data: [DONE].

Client disconnect sets the run cancel_event (same idea as stdio cancel).

AgentLoop event names match stdio — see stdio_protocol.md (run_start, memory_start, llm_start, llm_chunk, llm_done, tool_start, tool_done, final, loop_abort, …).

Errors

OpenAI-style body:

{"error": {"message": "...", "type": "invalid_request_error", "param": null, "code": "..."}}

Example

# Non-stream
curl http://127.0.0.1:8080/v1/chat/completions `
  -H "Content-Type: application/json" `
  -d '{"model":"assistant","messages":[{"role":"user","content":"Hello"}],"session_id":"demo1"}'

# Stream
curl -N http://127.0.0.1:8080/v1/chat/completions `
  -H "Content-Type: application/json" `
  -d '{"model":"assistant","messages":[{"role":"user","content":"Hello"}],"stream":true}'

With auth:

curl http://127.0.0.1:8080/v1/models -H "Authorization: Bearer YOUR_TOKEN"

Out of scope

  • OpenAI embeddings / images / audio endpoints
  • OpenAI function-calling protocol (tools stay inside the ReAct loop; they appear as AgentLoop tool_* SSE events)
  • TLS termination (bind locally; put a reverse proxy in front if needed)

See also bootstrap.md, api/interfaces.md.