Extract fields from the MDM

In Message Query Language (MQL), fields can be extracted from the MDMMDM - Message Data Model by specifying the field name. To get a subfield, combine two fields together with a . in between.

For example, to retrieve the sender's display name:

sender.display_name

Literal values

Strings can be specified in two forms: escaped and unescaped.

Escaped strings are surrounded by double quotes " and special characters can be escaped with \. If an unrecognized character is specified after \, that is treated as an invalid escape sequence and causes a syntax error.

"hello world"

"line 1\nline2\nline3"

"unicode characters like ✉️ are supported"

Unescaped or raw strings are surrounded by single quotes '. No escape sequences are supported within a raw string. However, two single quotes '' can be used to insert a single quote character.

'hello world'

'escaping apostrophes isn''t that difficult'

'this back\slash is interpreted literally'

Escape sequences

  • \r carriage return CR (ASCII 0x0d)
  • \n new line LF (ASCII 0x0a)
  • \t tab (ASCII 0x09)
  • \' single quotes ' (ASCII 0x27)
  • \" double quotes " (ASCII 0x22)
  • \\ backslash \ (ASCII 0x5c)
  • \u{xxxxxxxx} Unicode code point between 0x01 and 0x10ffff.

Unicode escape sequences can include any valid unicode code point, including ASCII characters, non-printable characters or other unicode characters. Between 2 and 8 hex digits are recognized within { and }.

Example unicode escapes:

  • \u{0a} identical to \n for a newline
  • \u{0398} greek capital theta: Θ
  • \u{200f} unicode right-to-left encoding character
  • \u{1f4ec} open mailbox emoji: 📬
  • \u{0001f4ec} open mailbox emoji, with optional leading zeros: 📬

Comparing values

MQL provides eight built-in operators to compare two values. The syntax is <left> <operator> <right>:

sender.email.domain.root_domain == "sublimesecurity.com"

Numbers can be compared with the below operators. MQL can represent unsigned, signed, or floating-point values.

  • <: less than
  • <=: less than or equal to
  • ==: equal to
  • !=: not equal to
  • >=: greater than or equal to
  • >: greater than

Note: When two values are compared of different numeric types, they are automatically converted to have compatible types. For example, in the comparison 1 < 1.5, the integer 1 is compared to a floating point 1.5. First, the 1 is converted to a floating point 1.0, and then the comparison is performed. This means that an expression like 3 == 3.14 will always evaluate false because it's converted to 3.0 == 3.14.

Strings support two additional operators to support case-insensitive equality. Unless explicitly specified, assume that strings are interpreted with case-sensitivity (meaning that "a" and "A" are distinct). Range operators, such as < use lexicographical ordering and are always case-sensitive.

case-sensitive comparisons

  • <: less than
  • <=: less than or equal to
  • ==: equal to ("Abc" == "abc" evaluates as false)
  • !=: not equal to
  • >=: greater than or equal to
  • >: greater than

case-insensitive comparisons

  • =~: case-insensitive equality. ("Abc" =~ "abc" evaluates as true)
  • !~: case-insensitive inequality. ("Abc" !~ "abc" evaluates as false)

Booleans can only be compared with == or !=. Booleans also have dedicated operators for traditional boolean logic, such as and. (see the next section).

Boolean logic

Multiple boolean expressions can be combined with traditional boolean operators. Use the and, or, or not keywords for boolean operations:

sender.email.email == "[email protected]" and subject.subject == "Password reset request"

sender.email.email == "[email protected]" or subject.subject == "Password reset request"

not (sender.email.email == "[email protected]")

Check against multiple values with in

One common pattern that often combines comparisons with boolean logic is to check the same field against multiple literal values. Instead of combining similar equality checks == with a logical or, use the in keyword to check a value against a set of values:

For example, to detect subjects commonly used in BEC attacks:

subject.subject == "Urgent" or
subject.subject == "Can you help" or
subject.subject == "Quick errand"

# more compact form with `in`
subject.subject in ("urgent", "can you help", "quick errand")

To check the inverse and ensure that a value is not in a list of values, use not in:

subject.subject not in ("Urgent", "Can you help", "Quick errand")

in can also be used in a case-insensitive way, using in~:

subject.subject in~ ("Urgent", "Can you help", "Quick errand")

Matching several boolean expressions

Occasionally when writing rules, a minimum amount of boolean clauses must match. With and, all the clauses must be true and with or at least one clause has to be true. Use of for a hybrid operator when checking for a minimum amount of matches.

X of (...) evaluates true if at least X terms evaluate true. The basic structure for of follows:

X of (clause1, clause2, ..., clauseN)

Using 1 of (...) is identical to an or over all of the terms. Similarly, if X is the same as the total number of terms, then it's equivalent to and. The minimum number of clauses to check with of must be between 1 and the total number of clauses between ( and ).

For example,

# returns true
3 of (true, false, true, true)

# returns false
3 of (true, false, false, true)

Checking a named list with in

A value can be checked against a list by using in $list or not in $list syntax. See the full reference of currently available lists.

sender.email.domain.domain in $alexa_1m

sender.email.domain.domain not in $alexa_1m

Arithmetic

All numbers support standard arithmetic operations. When two numbers of different types are used in arithmetic they are first converted to a matching type. That means 1 * 2.0 is automatically converted to 1.0 * 2.0. The supported arithmetic operations are:

  • + Add two numbers
  • - Subtract two numbers
  • * Multiply two numbers
  • / Divide two numbers. If both values are integers, this indicates integer division, meaning that 5 / 2 is 2. To get 2.5, add a decimal point to either side: 5 / 2.0, 5.0 / 2 and 5.0 / 2.0 are all equivalent.
  • % Modulo of two numbers

Order of operations

The precedence of operations, ordered from highest to lowest:

  • ( ... ): parentheses
  • *, / and %: multiplication, division, and modulus
  • + and -: addition and subtraction
  • <, <=, ==, =~, !=, !~, >=, >, in: comparisons
  • of: matching multiple boolean terms
  • not: boolean NOT
  • and: boolean AND
  • or: boolean OR

Comments

MQL supports single-line comments beginning with //.

type.inbound
// check if the sender's domain uses the ru TLD
and sender.email.domain.tld != 'ru'