Threat intelligence feeds

Threat intelligence feeds keep lookups filled with current indicators of compromise (IOC): IP addresses, domains, URLs, hashes, and similar values. As an analyst you usually choose which feed fills which lookup, then use that lookup in detections (and optionally in risk scoring) so rules stay current without hard coding indicator lists in YAML.

Typical uses:

• Watchlists: alert when traffic or DNS hits a known bad IP or domain.
• Risk boost: raise event.risk_score when a window dimension matches an IOC row.
• Shared intel: keep one maintained list for many correlation rules.

For membership tests and whitelist / blacklist patterns in predicates, see Lookups in detections. For how IOC rows change numeric risk, see Risk scoring. Lookup types and declarations are in the Lookups reference.

How feeds connect to detections

The path is always the same:

1. A feed declaration in the Library (/Feeds/…) pulls data on a schedule (HTTP JSON, CSV, LDAP, and similar sources).
2. The feed writes items into a named lookup (output.lookup).
3. A detection (or risk scoring) reads that lookup with !LOOKUP / !LOOKUP.CONTAINS, or via the ioc lookup group.

External source (TI API, LDAP, …)  →  Feed (lmio-feeds)  →  Lookup items  →  Correlator / enricher / risk score

You can also attach a feed from the Lookups UI (feedPath). The runtime effect is the same: the lookup content is refreshed from the feed.

Who owns what

Analysts maintain which indicators matter (lookup content and feed targets). Detection engineers keep predicate wiring stable. Prefer changing feed / lookup content over editing rule YAML when the indicator set grows.

Prerequisites

• LogMan.io Feeds is deployed (service that runs feed declarations).
• A lookup declaration exists under /Lookups (many IOC lookups ship in the common library with default: true).
• The feed’s output.lookup name matches the lookup define.name (tenant suffix is handled for you).
• Key types align: IP feeds → lookup/ipaddress (or IP range); domains → string keys; and so on.

Shipped IOC oriented lookups include:

Lookup	Typical key	Group	Used for
maliciousipaddresses	IP	ioc	Bad source / destination IPs
maliciousdomains	domain string	ioc	DNS and HTTP hostnames
maliciousurls	URL string	(varies)	Full URL matches
malicioushashes	file hash	(varies)	File / malware hashes
torexitnodes / vpnexitnodes	IP	(none)	Anonymizer exit nodes

Put a lookup in the ioc group when you want Correlator risk scoring to add score automatically for matching window dimensions. See IOC and threat intelligence.

Example: feed that fills a malicious IP lookup

Minimal feed that reads a JSON list of IP objects and updates maliciousipaddresses.

Example response from GET /api/v1/ips (fictional TI API, shortened). The feed iterates results:

{
  "count": 2,
  "results": [
    {
      "ip": "203.0.113.66",
      "risk.score": 25.0,
      "tags": ["botnet", "scanner"]
    },
    {
      "ip": "198.51.100.14",
      "risk.score": 40.0,
      "tags": ["c2"]
    }
  ]
}

How this maps in the feed below:

• iterate: results → one lookup item per object
• key: !ITEM EVENT ip → 203.0.113.66 and 198.51.100.14 become keys in maliciousipaddresses
• risk.score is stored on the item for risk scoring; if missing, the feed defaults to 25.0
• extra fields such as tags are ignored unless you add them to schema / apply

---
define:
  name: Malicious IP Feed
  type: lmio/feed

  schema:
    ip: str
    risk.score: fp64

input:
  source: https://ti.example.com/api/v1/ips
  type: json
  iterate: results
  period: 1h
  headers: '{"Authorization": "Bearer YOUR_TOKEN"}'
  verify: none

apply:
  key: !ITEM EVENT ip
  risk.score:
    !TRY
    - !ITEM EVENT risk.score
    - 25.0

output:
  lookup: maliciousipaddresses

Notes:

• period controls how often the feed refreshes (for example 1h, 1d).
• key becomes the lookup key; extra fields (such as risk.score) become lookup attributes used by risk scoring.
• Replace the URL, headers, and iterate path to match your TI provider’s JSON shape.
• For APIs that return a next URL per page, use pagination (subtype: pagination, pagination_mode: url). Ask your administrator if you need that wiring.

After the feed runs, open Lookups → maliciousipaddresses and confirm new IPs appear.

Example: CSV feed (including headerless files)

Many TI and inventory lists arrive as CSV. Use type: csv on the feed input. By default the first row is the header and column names become event fields for apply.

---
define:
  name: Malicious Domains CSV Feed
  type: lmio/feed

  schema:
    domain: str
    risk.score: fp64

input:
  source: https://ti.example.com/lists/malicious-domains.csv
  type: csv
  delimiter: ","
  period: 1d
  verify: none

apply:
  key: !ITEM EVENT domain
  risk.score:
    !TRY
    - !ITEM EVENT risk.score
    - 20.0

output:
  lookup: maliciousdomains

Headerless CSV

If the file has no header row, set fieldnames to the column names in order. The first data row is then read as values, not as headers.

input:
  source: ./data/dbip-asn-ipv4-num.csv
  type: csv
  delimiter: ","
  fieldnames:
    - start_ip
    - end_ip
    - number
    - organization.name
  period: 1d

Notes:

• fieldnames works for file, HTTP, and OAuth2 CSV sources.
• You can also pass a comma-separated string, for example fieldnames: "start_ip,end_ip,number".
• When creating a feed from the Lookups UI / API template, use the same list as csvFieldnames.
• headers under input are HTTP request headers, not CSV column names.

After the feed runs, open the target lookup and confirm keys match the columns you named in fieldnames / the CSV header.

Example: alert when traffic hits an IOC IP

Same pattern as other watchlists in Lookups in detections: membership of source.ip in the lookup.

predicate:
  !AND
  - !IN
    what: source.ip
    where: !EVENT
  - !IN
    what: !ITEM EVENT source.ip
    where:
      !LOOKUP
      what: maliciousipaddresses

trigger:
  - event:
      source.ip: !ITEM EVENT source.ip
      destination.ip: !ITEM EVENT destination.ip
      threat.indicator.confidence: high
      threat.indicator.ip: !ITEM EVENT source.ip
      threat.indicator.type: malicious_ip

Library rules such as Traffic from Tor Exit Nodes use the same idea with torexitnodes instead of maliciousipaddresses. Point your feed at torexitnodes (or keep a dedicated Tor list) and the shipped rule starts matching as soon as items land in the lookup.

Shorthand

- !LOOKUP.CONTAINS
  what: !ITEM EVENT source.ip
  in: maliciousipaddresses

Example: malicious domains and DNS

1. Feed writes hostnames into maliciousdomains (string keys).
2. Rule tests dns.question.name (or url.domain, depending on your parsers).

predicate:
  !AND
  - !IN
    what: dns.question.name
    where: !EVENT
  - !IN
    what: !ITEM EVENT dns.question.name
    where:
      !LOOKUP
      what: maliciousdomains

Align the feed’s key with the event field (FQDN vs bare domain). Normalize in the feed apply section if the TI source and your DNS logs use different formats.

Example: raise risk without a dedicated rule

If the lookup is in the ioc group, Correlator can add risk.score from matching rows whenever a detection’s window dimensions include that value (for example source.ip or dns.question.name). You do not need a separate “IOC hit” rule for that boost.

key              risk.score
203.0.113.66     25.0
evil.example.com 20.0

A VPN failure rule with base score 45 whose source.ip matches the first row becomes 70 before any asset weight. Details: Risk scoring.

You can still write an explicit watchlist rule (previous sections) when you want a dedicated alert for every IOC sighting, not only a score bump on other detections.

Wiring checklist

1. Lookup exists and has the right type (ip, string, and so on).
2. Optional: set group: ioc on the lookup declaration for automatic risk adds.
3. Feed output.lookup equals the lookup name.
4. Feed key / schema matches lookup keys; cast types in apply when needed.
5. Detection uses !LOOKUP / !LOOKUP.CONTAINS on the same name, or you rely on ioc risk only.
6. After deploy, wait for one feed period, then verify items in the UI and a test event in Discover.

Practical tips

• Prefer one feed per lookup (or a clear merge strategy) so ownership stays obvious.
• Keep base severity on the rule honest; use IOC risk.score and rule tuning to prioritize without rewriting predicates.
• Use global lookups / feeds for shared intel across tenants; use tenant scoped ones for local or customer specific lists.
• Do not paste large static IOC lists into correlation YAML; that is what feeds and lookups are for.
• When testing, insert a single known test IP or domain into the lookup (UI or feed), fire a synthetic event, then remove the test row.

Advanced example: Netbox prefixes as enrichment

Not every feed is a classic IOC watchlist. The same feed → lookup path can keep inventory data current and reuse it in parsing enrichment and in detections. A common case is Netbox IPAM prefixes: each prefix is a CIDR with optional VLAN and description; the feed stores them in an IP range lookup; Parsec then tags events by source.ip.

1. IP range lookup

Declare a lookup with two IP keys (start, end) and the attributes you want on the event:

---
define:
  type: lookup/ipaddressrange
  name: netbox
  label: Netbox prefixes
  description_title: Prefixes and VLANs from Netbox IPAM

keys:
  - name: start
    type: ip
  - name: end
    type: ip

fields:
  prefix:
    type: str
  description:
    type: str
  vlan.id:
    type: str
  vlan.name:
    type: str
  status:
    type: str

2. Feed that pages Netbox and expands CIDR keys

Netbox (and similar Django REST APIs) return an absolute next URL. Use pagination_mode: url. The feed key is the CIDR string (prefix); LogMan.io Feeds expands it to start / end for the IP range lookup.

Example response from GET /api/ipam/prefixes/?limit=100 (shortened). The feed iterates results and follows next until it is null:

{
  "count": 2,
  "next": "https://netbox.example.com/api/ipam/prefixes/?limit=100&offset=100",
  "previous": null,
  "results": [
    {
      "id": 42,
      "prefix": "10.0.112.0/22",
      "description": "Office LAN Prague",
      "status": {
        "value": "active",
        "label": "Active"
      },
      "vlan": {
        "id": 7,
        "vid": 112,
        "name": "OFFICE-PRG"
      }
    },
    {
      "id": 43,
      "prefix": "10.0.200.0/24",
      "description": "Guest WiFi",
      "status": {
        "value": "active",
        "label": "Active"
      },
      "vlan": null
    }
  ]
}

How this maps in the feed below:

• iterate: results → one lookup item per prefix object
• pagination_path: next → second page URL above; last page has "next": null
• key: !ITEM EVENT prefix → for example 10.0.112.0/22 becomes range 10.0.112.0 … 10.0.115.255
• nested vlan.vid / vlan.name and status.label via nested !GET; when vlan is null, !TRY stores an empty string

---
define:
  name: Netbox Prefixes Feed
  type: lmio/feed

  schema:
    prefix: str
    description: str
    vlan: "(id: si64, vid: si64, name: str)"
    status: "(value: str, label: str)"

input:
  source: https://netbox.example.com/api/ipam/prefixes/?limit=100
  type: json
  subtype: pagination
  pagination_path: next
  pagination_mode: url
  iterate: results
  period: 1d
  headers: '{"Authorization": "Bearer NETBOX_TOKEN"}'
  verify: none

apply:
  key: !ITEM EVENT prefix

  prefix:
    !CAST
    what: !ITEM EVENT prefix
    type: str

  description:
    !TRY
    - !CAST
      what: !ITEM EVENT description
      type: str
    - ""

  vlan.id:
    !TRY
    - !CAST
      what:
        !GET
        from:
          !GET
          from: !EVENT
          what: vlan
        what: vid
      type: str
    - ""

  vlan.name:
    !TRY
    - !CAST
      what:
        !GET
        from:
          !GET
          from: !EVENT
          what: vlan
        what: name
      type: str
    - ""

  status:
    !TRY
    - !CAST
      what:
        !GET
        from:
          !GET
          from: !EVENT
          what: status
        what: label
      type: str
    - ""

output:
  lookup: netbox

Nested Netbox objects (vlan, status) need nested !GET (not a flat !ITEM EVENT vlan.vid). Missing VLAN is handled with !TRY and an empty string. A template for this feed ships under /Templates/Feeds/ in the common library.

3. Enricher: map source.ip to ECS fields

Once the lookup is populated, a Parsec enricher can resolve the range that contains source.ip and write ECS fields such as source.vlan.id, source.vlan.name, source.subnet.id, and network.name. Place the enricher in the parser chain after mapping so source.ip already exists.

---
define:
  type: parsec/enricher
  schema: /Schemas/ECS.yaml

enrich:
  source.vlan.id:
    !OR
    - !GET
      what: vlan.id
      from:
        !GET
        from:
          !LOOKUP
          what: netbox
        what: !ITEM EVENT source.ip
    - !ITEM EVENT source.vlan.id

  source.vlan.name:
    !OR
    - !GET
      what: vlan.name
      from:
        !GET
        from:
          !LOOKUP
          what: netbox
        what: !ITEM EVENT source.ip
    - !ITEM EVENT source.vlan.name

  source.subnet.id:
    !OR
    - !GET
      what: prefix
      from:
        !GET
        from:
          !LOOKUP
          what: netbox
        what: !ITEM EVENT source.ip
    - !ITEM EVENT source.subnet.id

  network.name:
    !OR
    - !GET
      what: description
      from:
        !GET
        from:
          !LOOKUP
          what: netbox
        what: !ITEM EVENT source.ip
    - !ITEM EVENT network.name

!OR keeps an existing value when the lookup misses (unknown or public IP). The same pattern works for destination.ip or client.ip if you add parallel blocks.

Schema auto enrich

ECS also documents automatic enrich from lookup subnets into source.vlan.id / source.subnet.id. If you name the lookup subnets and align field names, Parsec can fill those two fields without a custom enricher. The enricher above is for a lookup named netbox and for extra fields such as source.vlan.name and network.name.

4. Optional: detections on enriched or raw lookup data

After enrichment, rules can filter on source.vlan.id or network.name like any other ECS field. You can also test membership in the range lookup directly (same style as zones):

- !IN
  what: !ITEM EVENT source.ip
  where:
    !LOOKUP
    what: netbox

That is useful for “traffic from a Netbox managed prefix” style rules, while the enricher keeps Discover and dashboards readable without repeating the lookup in every query.

Advanced example: Active Directory SID to user ID

Feeds are not limited to threat intel APIs. An LDAP feed can sync accounts from Active Directory into a lookup so parsers and detections resolve Windows SIDs to usernames.

The common library ships the lookup sids2userid (group: asset) and Windows Event Log enrichers that map we.user.security.id (and similar SID fields) through that lookup to user.name / related identity fields.

1. Lookup

sids2userid uses a string SID as the key and stores user.id (typically sAMAccountName):

---
define:
  type: lookup/string
  name: sids2userid
  label: Security Identifiers to User ID
  group: asset
  description_title: List of user security identifiers to user IDs
  default: true

keys:
  - name: sid
    type: str

fields:
  user.id:
    type: str

2. LDAP feed

The feed binds to a domain controller, searches user objects, converts binary objectSid with !LDAP.SID, and writes each row into sids2userid.

Replace host, bind account, base DN, and secret name for your environment. Prefer a Vault placeholder in password so the Library YAML does not store the bind password.

Secure the bind password with Vault

Availability

Requires LogMan.io Feeds v26.35 or newer. See LDAP / AD: Vault password placeholder.

1. Store the password in the Vault (on the LogMan.io host):

curl -X PUT localhost:8891/vault/LDAP_USER_PASSWORD --data 'supersecret'

1. Map it onto Feeds in /Site/model.yaml, then apply:

services:
  lmio-feeds:
    asab:
      config:
        passwords:
          LDAP_USER_PASSWORD: "{{LDAP_USER_PASSWORD}}"

secrets:
  LDAP_USER_PASSWORD: {}

1. In the feed use password: "{{LDAP_USER_PASSWORD}}" (see example below).

Full step by step (including alternative syntax and troubleshooting) is in LDAP / AD user identity feed: Use a Vault password placeholder.

---
define:
  name: Sids to User ID Feed
  type: lmio/feed
  # Define schema from the input fields loaded by the feed
  schema:
    objectSid: binary
    sAMAccountName: str

input:
  source: ldap://10.84.203.57
  type: ldap
  username: adldap@domain.int
  password: "{{LDAP_USER_PASSWORD}}"
  base: dc=DOMAIN,dc=int
  filter: (&(objectCategory=person)(objectClass=user))
  attributes: objectSid sAMAccountName
  period: 1h

apply:
  key:
    !LDAP.SID
    what: !ITEM EVENT objectSid

  user.id:
    !CAST
    what: !ITEM EVENT sAMAccountName
    type: str

output:
  lookup: sids2userid

How this maps:

• type: ldap pulls entries from Active Directory on each period (1h above).

• password: "{{…}}" resolves the bind password from Feeds [passwords] (Vault via Maestro); do not put plaintext passwords in shared Library feeds.

• filter selects person/user objects; attributes limits the LDAP attributes returned.

• !LDAP.SID turns binary objectSid into the string SID used as the lookup key.
• user.id stores sAMAccountName as the lookup attribute (same name as in the lookup declaration).

After the feed runs, open Lookups → sids2userid and confirm SID to username pairs appear.

3. Enrichment and detections

Windows parsers in the common library already use sids2userid (for example SID to user.name). Once the feed keeps the lookup current, enriched events show account names even when the raw log only carried a SID.

For mapping display names (or other username strings) to canonical user.id via usernames2userid, see LDAP / AD user identity feed.

In detections you can also test membership or resolve identity the same way as other lookups (see Lookups in detections). Keep the feed period short enough for joiner/leaver churn in your AD.

Related reading

• Lookups in detections: whitelist and blacklist predicates
• LDAP / AD user identity feed: fill usernames2userid from Active Directory

• LDAP / AD: Vault password placeholder: keep bind passwords out of Library YAML

• Risk scoring: IOC group and score layers

• Predicates: SP Lang filters
• Enricher declarations: adding fields during parsing
• Lookups reference: types, declarations, deployment
• IP address lookups: IP and IP range lookups
• Advanced correlation rules: multi stage rules that may consume threat intel signals