Data sources

Data sources are named connections to datasets within a backend. Each source is exposed to the API with a provider prefix:

Prefix	Backend	Example ID
es.	Elasticsearch	es.events
ix.	InfluxDB	ix.cpu
db.	DuckDB	db.ingest

Discovery

Data sources are assembled from several sources:

Library (/DataSources/)

YAML declarations with define.type of datasource/elasticsearch or datasource/influxdb and a discover section. Reloaded on Library change.

InfluxDB auto-discovery

All measurements in the configured database are exposed under the system tenant (no RBAC resources).

DuckDB (built-in)

Per-tenant in-memory sources: db.ingest (ingest rates) and db.eps (events per second). Not configured in the Library.

API properties

Each data source exposes:

Property	Description
type	elasticsearch, influxdb, or duckdb
label	Display name
order	Sort priority (lower = first)
tenant	Owning tenant
resources	RBAC resources required for access (not returned in API; used for filtering)

List data sources: GET /{tenant}/datasources?types=elasticsearch,influxdb

Library layout

Data sources are defined in the Library. Each data source definition must be on global target, not tenant.

Paths are: /DataSources/<tenant>/<datasource>.yaml or /DataSources/<datasource>.yaml.

• Tenant-specific data sources — file under /DataSources/<tenant>/. Available only to that tenant.
• Global data sources — file directly under /DataSources/ (tenant *). These data sources are copied to every tenant. Parameters like {{tenant}} are substituted per tenant.

Only declarations with a discover section are loaded for Chart (Discover/Explore UI).

Elasticsearch

/DataSources/events.yaml

define:
  type: datasource/elasticsearch

discover:
  name: Events
  order: 100
  resources:
    - "lmio:dataset:events"

specification:
  index: "lmio-{{tenant}}-events-*"
  schema: /Schemas/ECS.yaml  # optional; defaults to tenant schema
  restrict:                    # optional static filter on every query
    event.dataset: linux

/DataSources/<tenant>/fortinet-fortigate-1.yaml

define:
  type: datasource/elasticsearch

discover:
  name: Fortinet Fortigate 1
  order: 1100

specification:
  index: "lmio-{{tenant}}-fortinet-fortigate-1-*"
  schema: /Schemas/ECS.yaml

define

• type — must be datasource/elasticsearch

discover

• name — display label (required)
• order — sort order (default 1000)
• resources — list of RBAC resources; user must hold at least one. Users with lmio:dataset:readall see all sources.

Custom RBAC resource

Create a custom resource in SeaCat Auth (for example lmio:dataset:fortinet-fortigate-1) and assign it to roles to control access to a specific data source. List it in discover.resources on the data source declaration.

specification

• index — Elasticsearch index pattern; supports {{tenant}}
• schema — Library schema path; if omitted, tenant default from ZooKeeper is used; fallback /Schemas/ECS.yaml.
• restrict — static field filters applied to every query. Each filter is a key-value pair.

Others data source

Data source Others uses a dedicated schema located in the Library at /Schemas/Others.yaml.

Remote cluster index patterns may include the cluster alias, e.g. {{tenant}}:lmio-{{tenant}}-events*. See Elasticsearch remote cluster.

InfluxDB

/DataSources/metrics/cpu.yaml

define:
  type: datasource/influxdb

discover:
  name: CPU
  order: 200

specification:
  measurement: cpu
  restrict:
    host: "{{tenant}}"

define

• type — must be datasource/influxdb

discover

Same as Elasticsearch (name, order, resources).

specification

• measurement — InfluxDB measurement name
• restrict — static tag/field filters; supports {{tenant}} substitution

Schema (tags and fields) is built automatically from SHOW TAG KEYS / SHOW FIELD KEYS on the measurement.

DuckDB

Built-in per tenant; not declared in the Library.

DuckDB sources expose derived operational metrics computed in memory from InfluxDB commlink data. Use them for platform monitoring charts (ingest throughput and EPS statistics) without storing extra data in Elasticsearch.

ID	Label	Description
db.ingest	Ingest	Ingestion rates (bytes.in, event.in) per stream and partition
db.eps	EPS	Events-per-second aggregates (min, max, mean, median, 95th percentile)

Template variables

• {{tenant}} — replaced with the current tenant identifier in specification.index, specification.restrict, and InfluxDB restrict values.

Schema resolution

For Elasticsearch sources, schema is resolved in order:

1. specification.schema from the data source declaration
2. Tenant default schema from ZooKeeper (/lmio/tenants/{tenant}.yaml)
3. Fallback schema is /Schemas/ECS.yaml

Authorization

Data source list and queries are filtered by RBAC. Configure discover.resources on Library declarations, or grant lmio:dataset:readall for unrestricted access.