Skip to content

Parser expressions¤

Overview¤

Parser expressions are functions for parsing a certain sequence of characters.

Basic parsers can differentiate between digits, letters and spaces:

The following expressions are used for parsing characters from custom set of characters and looking for specific characters in input strings:

The following expressions are used for parsing dates and times in various formats:

The following expressions are used for parsing specific types of strings:

!PARSE.DIGIT¤

Parse a single digit.

Type: Parser.

Synopsis:

!PARSE.DIGIT

Example

Input string: 2

!PARSE.DIGIT

!PARSE.DIGITS¤

Parse a sequence of digits.

Type: Parser.

Synopsis:

!PARSE.DIGITS
min: <...>
max: <...>
exactly: <...>
  • exactly specifies the exact number of digits to parse.
  • min and max specify the minimal and maximal number of digits to parse. They cannot be combined with exactly parameter.
  • If none of fields min, max and exactly is specified, as many digits as possible are parsed.

Warning

exactly field can't be used together with min or max fields. And of course max value can't be less than min value.

Example

Input string: 123

!PARSE.DIGITS
max: 4
More examples Parse as many digits as possible: 
!PARSE.DIGITS
Parse exactly 3 digits: 
!PARSE.DIGITS
exactly: 3
Parse at least 2 digits, but not more than 4: 
!PARSE.DIGITS
min: 2
max: 4

!PARSE.LETTER¤

Parse a single letter.

By letters, we mean latin letters from A to Z, both uppercase and lowercase.

Type: Parser.

Synopsis:

!PARSE.LETTER

Example

Input string: A

!PARSE.LETTER

!PARSE.LETTERS¤

Parse a sequence of letters.

By letters, we mean latin letters from A to Z, both uppercase and lowercase.

Type: Parser.

Synopsis:

!PARSE.LETTERS
min: <...>
max: <...>
exactly: <...>

Fields min, max and exactly are optional.

Warning

exactly field can't be used together with min or max fields. And of course max value can't be less than min value.

Example

Input string: cat

!PARSE.LETTERS
max: 4
More examples Parse as many letters as possible: 
!PARSE.LETTERS
Parse exactly 3 letters: 
!PARSE.LETTERS
exactly: 3
Parse at least 2 letters, but not more than 4: 
!PARSE.LETTERS
min: 2
max: 4

!PARSE.SPACE¤

Parse a single space character.

Type: Parser.

Synopsis:

!PARSE.SPACE

!PARSE.SPACES¤

Parse a sequence of space characters.

Parse as many space symbols as possible:

Type: Parser.

Synopsis:

!PARSE.SPACES

!PARSE.CHAR¤

Parse a single character of any type.

Type: Parser.

Synopsis:

!PARSE.CHAR

Example

Input string: @

!PARSE.CHAR

!PARSE.CHARS¤

Parse a sequence of characters.

Type: Parser.

Synopsis:

!PARSE.CHARS
min: <...>
max: <...>
exactly: <...>

Fields min, max and exactly are optional.

Warning

exactly field can't be used together with min or max fields. And of course max value can't be less than min value.

Example

Input string:_ name@123_

!PARSE.CHARS
max: 8

Tip

Use !PARSE.CHARS with default settings to parse till the end of the string.

More examples Parse as many chars as possible: 
!PARSE.CHARS
Parse exactly 3 chars: 
!PARSE.CHARS
exactly: 3
Parse at least 2 chars, but not more than 4: 
!PARSE.CHARS
min: 2
max: 4

!PARSE.EXACTLY¤

Parse a precisely defined sequence of characters.

Type: Parser.

Synopsis:

!PARSE.EXACTLY
what: <...>

or shorter version:

!PARSE.EXACTLY <...>

Example

Input string:_Hello world!

!PARSE.EXACTLY
what: "Hello"

!PARSE.UNTIL¤

Parse a sequence of characters until a specific character is found.

Type: Parser.

Synopsis:

!PARSE.UNTIL
what: <...>
stop: <before/after>
eof: <true/false>

or shorter version:

!PARSE.UNTIL <...>

  • what: Specifies one (and only one) character to search for in the input string.

  • stop: Indicates whether the stop character should be parsed or not. Possible values: before or after (default).

  • eof: Indicates if we should parse till the end of the string if what symbol is not found. Possible values: true or false (default).

  • escape: Indicates escape character.

Info

Field what must be a single character. But some whitespace characters can also be used such as tab. To search for a sequence of characters, see the expression !PARSE.CHARS.LOOKAHEAD.

Example

Input string: 60290:11

!PARSE.UNTIL
what: ":"
More examples Parse until : symbol and stop before it: 
!PARSE.UNTIL
what: ":"
stop: "before"
Parse until space symbol and stop after it: 
!PARSE.UNTIL ' '
Parse until , symbol or parse till the end of the string if it's not found: 
!PARSE.UNTIL
what: ","
eof: true
Parse until tab symbol: 
!PARSE.UNTIL
what: 'tab'
Parse until vertical slash, escape internal vertical slashes:
Input string:CRED_REFR\|success\|fail| 
!PARSE.UNTIL
what: '|'
escape: '\'

!PARSE.BETWEEN¤

Parse a sequence of characters between two specific characters.

Type: Parser.

Synopsis:

!PARSE.BETWEEN
what: <...>
start: <...>
stop: <...>
escape: <...>

or shorter version:

!PARSE.BETWEEN <...>

  • what - indicates between which same characters we should parse.

  • start, stop - indicates between which different characters we should parse.

  • escape - indicates escape character.

Example

Input string:_ [10/May/2023:08:15:54 +0000]

!PARSE.BETWEEN
start: '['
stop: ']'
More examples Parse between double-quotes: 
!PARSE.BETWEEN
what: '"'
Parse between double-quotes, short form: 
!PARSE.BETWEEN '"'
Parse between double-quotes, escape internal double-quotes:
Input string:"one, \"two\", three" 
!PARSE.BETWEEN
what: '"'
escape: '\'

!PARSE.ONEOF¤

Parse a single character from a selected set of characters.

Type: Parser.

Synopsis:

!PARSE.ONEOF
what: <...>
or shorter version: 
!PARSE.ONEOF <...>

Example

Input strings:

process finished with status 0
process finished with status 1
process finished with status x

!PARSE.KVLIST
- "process finished with status "
- !PARSE.ONEOF
what: "01x"

!PARSE.NONEOF¤

Parse a single character that is not in a selected set of characters.

Type: Parser.

Synopsis:

!PARSE.NONEOF
what: <...>

or shorter version:

!PARSE.NONEOF <...>

Example

Input string:_ Wow!

!PARSE.NONEOF
what: ",;:[]()"

!PARSE.REGEX¤

Parse a sequence of characters that matches a regular expression.

Type: Parser.

Synopsis:

!PARSE.REGEX
what: <...>

Example

Input string:_ FTVW23_L-C: Message...

Output: FTVW23_L-C

!PARSE.REGEX
what: '[a-zA-Z0-9_\-0]+'

!PARSE.DATETIME¤

Parse datetime.

Type: Parser.

Synopsis:

!PARSE.DATETIME
- year: <...>
- month: <...>
- day: <...>
- hour: <...>
- minute: <...>
- second: <...>
- microsecond: <...>
- nanosecond: <...>
- timezone: <...>
  • Fields month and day are required.
  • Field year is optional. If not specified, the smart year function will be used. Both 2 and 4-digit numbers are supported.
  • Fields hour, minute, second, microsecond, nanosecond are optional. If not specified, the default value 0 will be used.
  • Specifying microseconds field like microseconds? allows you to parse microseconds or not, depending on their presence in the input string.
  • Field timezone is optional. If not specified, the default value UTC will be used. Read more about timezone parsing here.

Common datetime formats

Use Shortcuts for parsing datetime formats RFC 3339, RFC 3164 and ISO 8601.

UNIX time

For parsing datetime in UNIX time, use !PARSE.DATETIME EPOCH.

Tip

Use !PARSE.MONTH for parsing a month.

Tip

Use !PARSE.FRAC for parsing microseconds and nanoseconds. Note that this expression consumes . and , as well. Do not parse them separately.

Example

Input string: 2022-10-13T12:34:56.987654

!PARSE.DATETIME
- year: !PARSE.DIGITS
- '-'
- month: !PARSE.MONTH 'number'
- '-'
- day: !PARSE.DIGITS
- 'T'
- hour: !PARSE.DIGITS
- ':'
- minute: !PARSE.DIGITS
- ':'
- second: !PARSE.DIGITS
- microsecond: !PARSE.FRAC
        base: "micro"
Two-digit year

Parse datetime with two-digit year:

Input string: 22-10-13T12:34:56.987654

!PARSE.DATETIME
- year: !PARSE.DIGITS  # Year can be either 4-digit or 2-digit
- '-'
- month: !PARSE.MONTH "number"
- '-'
- day: !PARSE.DIGITS
- 'T'
- hour: !PARSE.DIGITS
- ':'
- minute: !PARSE.DIGITS
- ':'
- second: !PARSE.DIGITS
- microsecond: !PARSE.FRAC
            base: micro
No year, optional microseconds

Parse datetime without a year, with short month form and optional microseconds:

Input strings: 

Aug 17 12:00:00
Aug 17 12:00:00.123
Aug 17 12:00:00.123456

!PARSE.DATETIME
# There is no year in input string, smart year function is used.
- month: !PARSE.MONTH 'short'
- !PARSE.SPACE
- day: !PARSE.DIGITS
- !PARSE.SPACE
- hour: !PARSE.DIGITS
- ":"
- minute: !PARSE.DIGITS
- ":"
- second: !PARSE.DIGITS
- microsecond?: !PARSE.FRAC  # Parsing of microseconds is optional here
                base: "micro"
                max: 6

In this case, year is automatically determined by the smart year function, which basically means that the current year is used.

Milliseconds

Parse datetime with milliseconds:

Input string: 2023-03-23T07:00:00.734

!PARSE.DATETIME
- year: !PARSE.DIGITS
- "-"
- month: !PARSE.DIGITS
- "-"
- day: !PARSE.DIGITS
- "T"
- hour: !PARSE.DIGITS
- ":"
- minute: !PARSE.DIGITS
- ":"
- second: !PARSE.DIGITS
- microsecond: !PARSE.FRAC
            base: milli
            max: 3
Nanoseconds

Parse datetime with nanoseconds:

Input string: 2023-03-23T07:00:00.734323900

!PARSE.DATETIME
- year: !PARSE.DIGITS
- "-"
- month: !PARSE.DIGITS
- "-"
- day: !PARSE.DIGITS
- "T"
- hour: !PARSE.DIGITS
- ":"
- minute: !PARSE.DIGITS
- ":"
- second: !PARSE.DIGITS
- nanosecond: !PARSE.FRAC
            base: "nano"
            max: 9

Timezone¤

Timezone can be either specified in the log or it can be missing. There are two approaches for that:

  1. Timezone is parsed from the input string. In that case, use the suitable parsing expression for the timezone part.

    !PARSE.DATETIME
- timezone: !PARSE.UNTIL " "

    Permissible formats of timezones are: Z, UTC, +02:00, -0600.

  2. Timezone is fixed. In that case, specify it as IANA timezone.

    !PARSE.DATETIME
- timezone: "Europe/Prague"

    This will come handy when the timezone is missing in the log or when it is used incorrectly.

Timezone from input

Parse datetime which contains timezone in input strings.

Input strings:

2024-04-15T12:00:00+04:00 ...(other log content)
2024-04-15T12:00:00+02:00 ...
2024-04-15T12:00:00+00:00 ...
2024-04-15T12:00:00-02:00 ...

!PARSE.DATETIME
- year: !PARSE.DIGITS
- '-'
- month: !PARSE.MONTH 'number'
- '-'
- day: !PARSE.DIGITS
- 'T'
- hour: !PARSE.DIGITS
- ':'
- minute: !PARSE.DIGITS
- ':'
- second: !PARSE.DIGITS
- timezone: !PARSE.UNTIL " "  # Read timezone from '+04:00', '+02:00', etc.

Smart year¤

The smart year function is designed to predict the complete year from a provided month by taking into account the current year and month to determine the most likely corresponding four-digit year.

Shortcuts¤

Shortcut forms are available (in both lower/upper variants):

ISO 8601¤

!PARSE.DATETIME ISO8601

This expression parses datetimes defined by ISO 8601. Timezone can be parsed from the input string or, if not present, it can be set in the lmio-parsec configuration.

Example of datetimes that can be parsed using the shortcut:

2024-04-12T10:16:21Z
20240412T101621Z
2024-12-11T11:17:21.123456+00:00
2024-04-12T03:16:21−07:00
2024-04-12T03:16:21

RFC 3339¤

!PARSE.DATETIME RFC3339

This expression parses datetimes defined by RFC 3339. Timezone is always parsed from the input string.

Example of datetimes that can be parsed using the shortcut:

1985-04-12T23:20:50.52Z
1996-12-19T16:39:57-08:00
2021-06-29 16:51:43.987654+02:00

RFC 3164¤

!PARSE.DATETIME RFC3164

This expression parses datetimes defined by RFC 3164. Year is provided by the smart year function. Timezone must be set in LogMan.io Parsec configuration, otherwise considered as UTC.

Example of datetimes that can be parsed using the shortcut:

Apr 24 15:25:20
Oct  3 20:33:02
AUG  4 10:20:20

Epoch¤

!PARSE.DATETIME EPOCH

!PARSE.DATETIME epoch

This expression parses datetimes defined by Unix time. Expression allows parsing for different Unix datetime representations, such as seconds, milliseconds, microseconds and seconds with micro/milliseconds floating point.

Example of datetimes that can be parsed using the shortcut:

1731410205 - seconds
1727951634.687 - seconds with microseconds floating point
1728564383160 - milliseconds

!PARSE.MONTH¤

Parse a month.

Type: Parser.

Synopsis:

!PARSE.MONTH
what: <...>

or shorter version:

!PARSE.MONTH <...>

Parameter what indicates format of the month name. Possible values are:

  • number: numbered representation, e.g. 01 for January, 12 for December
  • short: three letters representation, e.g. Jan for January, Dec for December
  • full: full name representation, e.g. January, December

Tip

Use !PARSE.MONTH to parse month name as part of !PARSE.DATETIME.

Example

Input string: 10/Jan/2023:08:15:54

!PARSE.MONTH 'short'

Input string: 10/01/2023:08:15:54

!PARSE.MONTH 'number'

Input string: 10/January/2023:08:15:54

!PARSE.MONTH 'full'

!PARSE.FRAC¤

Parse a fraction.

Warn

Fraction parsing includes parsing of a dot (.) and a comma (,) separator.

Type: Parser.

Synopsis:

!PARSE.FRAC
base: <...>
max: <...>

  • base: Indicates a base of the fraction. Possible values are:

    • milli for 10^-3 base
    • micro for 10^-6 base
    • nano for 10^-9 base

  • max: Indicates a maximum number of digits depending on the base value. Default values 3, 6, 9 will be applied if max parameter is not specified.

Tip

Use !PARSE.FRAC to parse microseconds or nanoseconds as part of !PARSE.DATETIME.

Example

Input strings:

Aug 22 05:40:14.264
Aug 22 05:40:14.264023

!PARSE.FRAC
base: "micro"

or the full form:

!PARSE.FRAC
base: "micro"
max: 6

!PARSE.IP¤

Parse IP address in both IPv4 and IPv6 formats.

Returns numeric representation of the IP address.

Type: Parser.

Synopsis:

!PARSE.IP

Example

Input string:_ 193.178.72.2

!PARSE.IP

!PARSE.MAC¤

Parse MAC address in the format XX:XX:XX:XX:XX:XX.

Returns numeric representation of the MAC address.

Type: Parser.

Synopsis:

!PARSE.MAC

Example

Input string: 4d:3b:4c:bc:e5:6d

!PARSE.MAC