Inference¶
Inference is the language-model layer of LogMan.io AI SIEM. It answers prompts, streams reasoning and text, and proposes tool calls. It does not execute shell commands, call LogMan.io APIs, or touch customer filesystems — that is the job of the harness and the on-prem sandbox.
How inference is provided¶
Deployments choose where the models run. The recommended order is:
1. TeskaLabs AI cluster (default)¶
TeskaLabs operates a remote AI cluster. The on-prem harness connects to it through a routing layer, sends conversation context, and receives streamed model output.
This is the primary offering:
- You do not operate GPUs or model servers on site for day-to-day use
- Models and routing stay under TeskaLabs operations
- The harness, sandboxes, Library, and LogMan.io data stay on the customer premises
2. On-premise inference server¶
For customers who need models inside their own network, TeskaLabs can help stand up an on-premise inference server that the harness talks to the same way.
The control plane and agent runtime remain llm-microlink on site; only where the GPU/model process runs changes.
3. Bring your own inference¶
As a last option, the harness can use customer-supplied OpenAI-compatible (or related) endpoints — for example an existing vLLM, TensorRT-LLM, or cloud API that you already operate.
Use this when policy or architecture forbids TeskaLabs-hosted or TeskaLabs-built inference. You are then responsible for model quality, capacity, availability, and API compatibility.
What inference does¶
| Done by inference | Not done by inference |
|---|---|
| Understand the user goal and conversation context | Run shell or code |
| Stream assistant text and (when available) reasoning | Read or write the sandbox filesystem |
| Decide which tools or skills to use next | Call LogMan.io REST APIs or external hosts |
| Draft queries, rules, reports, and explanations | Enforce vetting or human approval |
| Judge progress toward a goal (when the harness asks) | Persist conversations, shares, or audit records |
The model only proposes. The harness executes, records, and gates.
Models in the UI¶
When a conversation starts, the UI shows the models available from the configured providers. You pick a model per conversation. From then on the harness:
- Streams tokens into the conversation timeline
- Turns model function calls into harness tool executions
- Can show approximate token usage when the provider exposes it
- Can keep wire-level traces for evaluation and support
Design note¶
Inference is replaceable; the harness is the product differentiator. The same skills, sandbox, Library content, and human-in-the-loop controls work whether the model runs in the TeskaLabs AI cluster, on an on-prem inference server, or on a bring-your-own endpoint.