Lookups in detections¶

Lookups are lists (and key/value tables) that detection rules use to decide whether an event should alert. As an analyst you mainly maintain the content of lookups and reference them in predicates so rules stay accurate without hard-coding usernames, IPs, or ports in YAML.

Typical uses:

Whitelists: suppress known-good activity (scanners, jump hosts, allowed network segments, vendor cloud ranges such as Microsoft 365).
Blacklists / watchlists: focus on known-bad or high-interest entities (Tor exit nodes, ransomware extensions, break-glass accounts).
Inventory lists: mark who or what is in scope (admins, service accounts, privileged groups, internal zones).

For creating lookup declarations, types, and deployment, see the Lookups reference. For managing items in the UI, see User Manual: Lookups. Custom geolocation zones are covered in Geolocation lookups. Attaching a MaxMind .mmdb file is covered in Custom MaxMind enricher. Filling IOC lookups from external sources is covered in Threat intelligence feeds. Filling usernames2userid from Active Directory is covered in LDAP / AD user identity feed.

How lookups appear in rules¶

In correlation predicates, a lookup is almost always used as a membership test: is this field value a key in the lookup?

Match if the value is in the lookup (`!IN` + `!LOOKUP`)¶

Use this for watchlists and “in scope” lists (admins, IoCs, unusual ports):

predicate:
  !AND
  - !IN
    what: user.name
    where: !EVENT
  - !IN
    what: !ITEM EVENT user.name
    where:
      !LOOKUP
      what: admins

This pattern is used widely in the common library (for example Administrator Logon Outside Business Hours, Break Glass Account Used, Traffic from Tor Exit Nodes).

Exclude if the value is in the lookup (whitelist)¶

Wrap the membership test in !NOT so listed entities do not trigger:

predicate:
  !AND
  - !IN
    what: source.ip
    where: !EVENT
  - !NOT
    what:
      !IN
      what: !ITEM EVENT source.ip
      where:
        !LOOKUP
        what: excludedipaddresses

Network sweep / port sweep rules use excludedipaddresses this way so scanners and monitoring hosts do not raise false positives.

Use case: exclude Microsoft 365 (or other) IP ranges¶

Goal: a detection should not alert when destination.ip (or source.ip) falls inside published Microsoft 365 / Office 365 service ranges, while still alerting for other destinations.

1. Use an IP address range lookup¶

Yes: store the ranges in a lookup of type lookup/ipaddressrange (two IP keys: range start and range end). A single IP lookup (lookup/ipaddress) is wrong for CIDR or start/end blocks.

Example declaration (Library /Lookups/):

---
define:
  type: lookup/ipaddressrange
  name: m365ipranges
  label: Microsoft 365 IP ranges
  description_title: Published Microsoft 365 service IP ranges used as detection exceptions

keys:
  - name: start
    type: ip
  - name: end
    type: ip

fields:
  name:
    type: str

Create the lookup in the UI with the same name and add items as start/end pairs (from Microsoft’s published IP list). Optional name can label the product or service area.

Full type details: IP address range lookup.

2. Exception in the rule: IP not in the lookup¶

Use the same whitelist pattern as above: !NOT around !IN + !LOOKUP. Example for destination:

predicate:
  !AND
  # … other detection conditions …
  - !IN
    what: destination.ip
    where: !EVENT
  - !NOT
    what:
      !IN
      what: !ITEM EVENT destination.ip
      where:
        !LOOKUP
        what: m365ipranges

For source.ip, change the field names the same way. Shorthand:

- !NOT
  what:
    !LOOKUP.CONTAINS
    what: !ITEM EVENT destination.ip
    in: m365ipranges

Membership against an IP range lookup succeeds when the event IP lies inside any stored start/end range (you do not expand CIDRs in the predicate yourself).

3. Keeping ranges up to date (optional feeds)¶

Manual / CSV: paste or import ranges into the lookup when Microsoft updates the list.
Feed: if you can obtain prefixes as CIDR (or similar) from an API, a feed can refresh the range lookup automatically. See the Netbox prefixes example (CIDR key expanded to start/end): Threat intelligence feeds: Netbox prefixes.

The same pattern works for any vendor or allowlisted subnet set (Azure, AWS, partner networks), not only Microsoft 365.

Shorthand: `!LOOKUP.CONTAINS`¶

Some rules use !LOOKUP.CONTAINS instead of !IN with where: !LOOKUP. Behaviour is the same idea: test whether a value is present in the named lookup.

# Alert only if the user is NOT on the authorized sudo list
- !NOT
  what:
    !LOOKUP.CONTAINS
    what: !ITEM EVENT user.name
    in: sudousers

# Alert if the destination is a known DB server AND the source is not allowed
- !LOOKUP.CONTAINS
  what: !ITEM EVENT destination.ip
  in: databaseservers
- !NOT
  what:
    !LOOKUP.CONTAINS
    what: !ITEM EVENT source.ip
    in: alloweddbsegments

Cast when key types differ¶

Lookup keys have a declared type (str, ip, and so on). If the event field is a number (for example a port) and the lookup stores strings, cast before the check:

- !IN
  what:
    !CAST
    what: !ITEM EVENT destination.port
    type: str
  where:
    !LOOKUP
    what: unusualports

Compound keys (`!TUPLE`)¶

Lookups with more than one key (for example country + date in holidays) need a !TUPLE in the same order as the declaration. See Using a compound key in rules.

Whitelists vs blacklists¶

There is no separate “whitelist” or “blacklist” lookup type. The role comes from how the rule uses the list:

Role	Predicate idea	Library examples
Whitelist (allow / ignore)	`!NOT` + membership in the lookup	`excludedipaddresses`, `sudousers`, `alloweddbsegments`, `safecountries`, custom IP range lists (for example `m365ipranges`)
Blacklist / watchlist (alert when present)	membership in the lookup	`torexitnodes`, `vpnexitnodes`, `ransomwareextensions`, `breakglassaccounts`, `maliciousipaddresses`
Scope / inventory	membership defines who the rule cares about	`admins`, `serviceaccounts`, `admingroups`, `zones`

You can combine both in one rule. Suspicious Connection requires source.ip in zones and destination.ip not in zones (traffic leaving internal ranges).

Practical tips¶

Prefer updating lookup items in the UI over editing the rule when the set of usernames or IPs changes.
Keep whitelist keys accurate; a missing entry often means noise, an extra entry can hide real detections.
Align key values with the event field you test (user.name vs user.id, ISO country codes, dotted IPs).
For IP ranges and private geography, use IP-range / geo lookups. See Geolocation lookups and IP address lookups.

Common lookups used by detection rules¶

These names appear frequently in the TeskaLabs common correlation library. Populate them for your tenant so shipped rules behave as intended.

Lookup	Typical role	Used for
`admins`	Scope	Admin authentication and privilege-related rules
`breakglassaccounts`	Watchlist	Emergency / break-glass account usage
`serviceaccounts`	Scope / watchlist	Interactive or unusual use of service accounts
`defaultaccounts`	Watchlist	Built-in / default account activity
`disabledaccounts`	State list	Failed logons against disabled accounts (also updated by some rules)
`sudousers`	Whitelist	Authorized sudo users (alert when not listed)
`excludedipaddresses`	Whitelist	IPs excluded from sweep / scan detections
`zones`	Inventory / geo	Internal IP ranges; zone-crossing firewall rules
`safecountries`	Whitelist	Expected country ISO codes for logons
`holidays`	Whitelist / calendar	Compound keys (for example country + date) for out-of-hours logic
`unusualports`	Watchlist	Non-standard ports
`torexitnodes` / `vpnexitnodes`	Blacklist	Traffic involving known exit nodes
`ransomwareextensions`	Blacklist	Ransomware-related file extensions
`domainprivilegedgroups` / `admingroups` / `privilegedlocalgroups` / `monitoredgroups`	Scope	Group membership change detections
`criticalservices`	Scope	Critical service stop / change
`activevpnusers`	State list	Who is currently on VPN (written by VPN login/logout rules)

Declarations for these lookups live under /Lookups in the library. Many are created automatically (default: true) with a small set of default_items; replace or extend them for your environment.

Updating lookups from detections¶

Rules can also write lookup content when they fire, using a lookup trigger:

trigger:
  - lookup: activevpnusers
    key: !ITEM EVENT user.name
  - lookup: activevpnusers
    delete: !ITEM EVENT user.name

Examples in the library:

VPN login / logout maintain activevpnusers.
Account disable / enable maintain disabledaccounts.
Scheduled task start / finish maintain scheduledtasks (compound key with !TUPLE).

Use this for short-lived state that other rules consume. For long-lived allow/deny lists that analysts own, prefer the Lookups UI (or CSV import) instead of encoding the list in the rule.

Testing rules that use lookups¶

When you validate a detection rule in the Library or with the correlator builder, predicates that call lookups (!LOOKUP.CONTAINS, !IN with !LOOKUP, and similar) need lookup data at test time. You do not have to populate production lookup files or rely on ZooKeeper for offline rule tests.

Add a lookups block inside each named case under the rule’s top-level test section. The lookup declaration must already exist in the Library under /Lookups/ (name, key types, fields). The test case supplies only the rows used for that run.

test:
  alice_is_allowed:
    lookups:
      allowed_events:
        alice:
          reason: whitelisted
        bob:
          reason: blocked

    input:
      - user.name: alice
        "@timestamp": 142446461673537536
        tenant: default
        _id: 1

    output:
      tenant: default
      # … expected complex / trigger output …

Part	Meaning
Top-level lookup name (`allowed_events`)	Lookup name without tenant suffix
Inner key (`alice`)	Lookup record key
Inner value	Record fields as declared in `/Lookups/`

For compound keys, use a list of records with an explicit _id in the same order as the declaration keys:

lookups:
  UserList:
    - _id: [tenant1, alice]
      role: admin

Behaviour notes:

If a test case does not define lookups, synchronous lookups are loaded from the builder’s production path (lookup.base_path, typically /lookups). Use that when you want to exercise real tenant data.
If any test case in the rule defines lookups, the builder uses a temporary lookup folder for those tests only.
Supported for test seeding: generic lookups with keys and fields (the same shape the lookup builder service uses).
Not supported in test seeding: lookup/ipaddressrange and lookup/macaddressrange (use production lookup files or mock the predicate differently).
Lookup triggers (- lookup: under trigger:) are still mocked during rule tests: writes are logged but not applied to the seeded lookup for later events in the same case.

For the full test section structure (input, output, timestamps, _id), see Test (optional, for rule validation).

Predicates: building SP-Lang filters
Triggers: event and notification actions (lookup triggers are detailed in Reference)
What is a detection?
Lookups reference: types, declarations, builders
Lookup trigger: set / delete / add / sub
Threat intelligence feeds: feeds that populate IOC lookups