www.espertech.comDocumentation

Chapter 7. EPL Reference: Patterns

7.1. Event Pattern Overview
7.2. How to Use Patterns
7.2.1. Pattern Syntax
7.2.2. Patterns in EPL
7.2.3. Subscribing to Pattern Events
7.2.4. Pulling Data From Patterns
7.2.5. Pattern Error Reporting
7.2.6. Suppressing Same-Event Matches
7.2.7. Discarding Partially Completed Patterns
7.3. Operator Precedence
7.4. Filter Expressions in Patterns
7.4.1. Filter Expressions and Filter Indexes
7.4.2. Controlling Event Consumption
7.4.3. Use With Named Windows and Tables
7.5. Pattern Operators
7.5.1. Every
7.5.2. Every-Distinct
7.5.3. Repeat
7.5.4. Repeat-Until
7.5.5. And
7.5.6. Or
7.5.7. Not
7.5.8. Followed-By
7.5.9. Pattern Guards
7.6. Pattern Atoms
7.6.1. Filter Atoms
7.6.2. Observer Atoms Overview
7.6.3. Interval (timer:interval)
7.6.4. Crontab (timer:at)
7.6.5. Schedule (timer:schedule)
7.7. Event Pattern Walkthrough

Event patterns match when an event or multiple events occur that match the pattern's definition. Patterns can also be time-based.

Pattern expressions consist of pattern atoms and pattern operators:

  1. Pattern atoms are the basic building blocks of patterns. Atoms are filter expressions, observers for time-based events and plug-in custom observers that observe external events not under the control of the engine.

  2. Pattern operators control expression lifecycle and combine atoms logically or temporally.

The below table outlines the different pattern atoms available:


There are 4 types of pattern operators:

  1. Operators that control pattern sub-expression repetition: every, every-distinct, [num] and until

  2. Logical operators: and, or, not

  3. Temporal operators that operate on event order: -> (followed-by)

  4. Guards are where-conditions that control the lifecycle of subexpressions. Examples are timer:within, timer:withinmax and while-expression. Custom plug-in guards may also be used.

Pattern expressions can be nested arbitrarily deep by including the nested expression(s) in () round parenthesis.

Underlying the pattern matching is a hierarchical finite state machine and behavior tree that allocates, transitions and destroys branch and leaf nodes of state based on arriving events and based on time advancing. A single event or advancing time may cause a reaction in multiple parts of your active pattern state. Patterns are stateful as the engine maintains pattern state. There is a walkthrough of how a sample pattern behaves in Section 7.7, “Event Pattern Walkthrough”.

This is an example pattern expression that matches on every ServiceMeasurement events in which the value of the latency event property is over 20 seconds, and on every ServiceMeasurement event in which the success property is false. Either one or the other condition must be true for this pattern to match.

every spike=ServiceMeasurement(latency>20000) 
 or every error=ServiceMeasurement(success=false)

In the example above, the pattern expression or operator indicates that the pattern should fire when either of the filter expressions fire. The every operator indicates to fire for every matching event and not just the first matching event. The left hand of the or operator filters for events with a high latency value. The right hand of the or operator filters for events with error status. Filter expressions are explained in Section 7.4, “Filter Expressions in Patterns”.

The example above assigned the tags spike and error to the events in the pattern. The tags are important since the engine only places tagged events into the output event(s) that a pattern generates, and that the engine supplies to listeners of the pattern statement. The tags can further be selected in the select-clause of an EPL statement as discussed in Section 5.4.2, “Pattern-Based Event Streams”.

Patterns can also contain comments within the pattern as outlined in Section 5.2.2, “Using Comments”.

Pattern statements are created via the EPAdministrator interface. The EPAdministrator interface allows to create pattern statements in two ways: Pattern statements that want to make use of the EPL select clause or any other EPL constructs use the createEPL method to create a statement that specifies one or more pattern expressions. EPL statements that use patterns are described in more detail in Section 5.4.2, “Pattern-Based Event Streams”. Use the syntax as shown in below example.

EPAdministrator admin = EPServiceProviderManager.getDefaultProvider().getEPAdministrator();

String eventName = ServiceMeasurement.class.getName();

EPStatement myTrigger = admin.createEPL("select * from pattern [" +
  "every spike=" + eventName + "(latency>20000) or every error=" + eventName + "(success=false)]");

Pattern statements that do not need to make use of the EPL select clause or any other EPL constructs can use the createPattern method, as in below example.

EPStatement myTrigger = admin.createPattern(
  "every spike=" + eventName + "(latency>20000) or every error=" + eventName + "(success=false)");

Partially-completed patterns are incomplete matches that are not yet indicated by the engine because the complete pattern condition is not satisfied. Any given event can be part of multiple partially-completed patterns.

For example, consider the following pattern:

every a=A -> B and C(id=a.id)

Given this sequence of events:

A1{id='id1'}   A2{id='id2'}   B1  

According to the sequence above there are no matches. The pattern is partially completed waiting for C events. The combination {A1, B1} is waiting for a C{id='id1'} event before the pattern match is complete for that combination. The combination {A2, B1} is waiting for a C{id='id2'} event before the pattern match is complete for that combination.

Assuming event C1{id='id1') arrives the pattern outputs the combination {A1, B1, C1}. Assuming event C2{id='id2') arrives the pattern outputs the combination {A2, B1, C2}. Note that event B1 is part of both partially-completed patterns.

Use the @DiscardPartialsOnMatch pattern-level annotation to instruct the engine that when any matches occur to discard partially completed patterns that overlap in terms of the events that make up the match (or matches if there are multiple matches).

The same example using the @DiscardPartialsOnMatch pattern-level annotation is:

select * from pattern @DiscardPartialsOnMatch [every a=A -> B and C(id=a.id)]

When event C1{id='id1') arrives the pattern outputs the match combination {A1, B1, C1}. Upon indication of the match the engine discards all partially-completed patterns that refer to either of the A1, B1 and C1 events. Since event B1 is part of a partially-completed pattern waiting for C{id='id2'}, the engine discards that partially-completed pattern. Therefore when C2{id='id2'} arrives the engine outputs no matches.

When specifying both @DiscardPartialsOnMatch and @SuppressOverlappingMatches the engine discards the partially-completed patterns that overlap all matches including suppressed matches.

The operators at the top of this table take precedence over operators lower on the table.


If you are not sure about the precedence, please consider placing parenthesis () around your subexpressions. Parenthesis can also help make expressions easier to read and understand.

The following table outlines sample equivalent expressions, with and without the use of parenthesis for subexpressions.


The simplest form of filter is a filter for events of a given type without any conditions on the event property values. This filter matches any event of that type regardless of the event's properties. The example below is such a filter. Note that this event pattern would stop firing as soon as the first RfidEvent is encountered.

com.mypackage.myevents.RfidEvent

To make the event pattern fire for every RfidEvent and not just the first event, use the every keyword.

every com.mypackage.myevents.RfidEvent

The example above specifies the fully-qualified Java class name as the event type. Via configuration, the event pattern above can be simplified by using the name that has been defined for the event type.

every RfidEvent

Interfaces and superclasses are also supported as event types. In the below example IRfidReadable is an interface class, and the statement matches any event that implements this interface:

every org.myorg.rfid.IRfidReadable

The filtering criteria to filter for events with certain event property values are placed within parenthesis after the event type name:

RfidEvent(category="Perishable")

All expressions can be used in filters, including static method invocations that return a boolean value:

RfidEvent(com.mycompany.MyRFIDLib.isInRange(x, y) or (x<0 and y < 0))

Filter expressions can be separated via a single comma ','. The comma represents a logical AND between expressions:

RfidEvent(zone=1, category=10)
...is equivalent to...
RfidEvent(zone=1 and category=10)

For more information on filters please see Section 5.4.1, “Filter-Based Event Streams”. Contained-event selection on filters in patterns is further described in Section 5.19, “Contained-Event Selection”.

Filter criteria can also refer to events matching prior named events in the same expression. Below pattern is an example in which the pattern matches once for every RfidEvent that is preceded by an RfidEvent with the same asset id.

every e1=RfidEvent -> e2=RfidEvent(assetId=e1.assetId)

The syntax shown above allows filter criteria to reference prior results by specifying the event name tag of the prior event, and the event property name. The tag names in the above example were e1 and e2. This syntax can be used in all filter operators or expressions including ranges and the in set-of-values check:

every e1=RfidEvent -> 
  e2=RfidEvent(MyLib.isInRadius(e1.x, e1.y, x, y) and zone in (1, e1.zone))

An arriving event changes the truth value of all expressions that look for the event. Consider the pattern as follows:

every (RfidEvent(zone > 1) and RfidEvent(zone < 10))

The pattern above is satisfied as soon as only one event with zone in the interval [2, 9] is received.

Important

A detailed description of how filters become active and are indexed engine-wide is provided at Section 2.18.2.2, “Filter Index Pattern Example”.

An expression such as a=A -> B(id=a.id) (A followed-by B with the same id as A) is not just a state change, in fact the engine registers new B-filter instances dynamically and in engine-wide shared filter indexes. This means that while such a pattern seems to be slow if you are sending A-events, the engine can filter, match or discard B-events very fast as for B-events it only needs to perform a lookup in filter indexes. If you are looking for best performance and don't expect to need filter indexes, or if you compare to another technology that doesn't have the concept of filter indexes, please use match-recognize instead.

The engine analyzes all filter expressions within a pattern and determines the filter indexes to use or to create. Indexing filter values to match event properties of incoming events enables the engine to match incoming events faster to pattern subexpressions.

More information on filter indexes in general can be found at Section 2.18.2, “Filter Indexes”.

More information on the operators relevant to filter indexes can be found at Section 5.4.1.2, “Specifying Filter Criteria”.

An arriving event applies to all filter expressions for which the event matches. In other words, an arriving event is not consumed by any specify filter expression(s) but applies to all active filter expressions of all pattern sub-expressions.

You may provide the @consume annotation as part of a filter expression to control consumption of an arriving event. If an arriving event matches the filter expression marked with @consume it is no longer available to other filter expressions of the same pattern that also match the arriving event.

The @consume can include a level number in parenthesis. A higher level number consumes the event first. The default level number is 1. Multiple filter expressions with the same level number for @consume all match the event.

Consider the next sample pattern:

a=RfidEvent(zone='Z1') and b=RfidEvent(assetId='0001')

This pattern fires when a single RfidEvent event arrives that has zone 'Z1' and assetId '0001'. The pattern also matches when two RfidEvent events arrive, in any order, wherein one has zone 'Z1' and the other has assetId '0001'.

Mark a filter expression with @consume to indicate that if an arriving event matches multiple filter expressions that the engine prefers the marked filter expression and does not match any other filter expression.

This updated pattern statement uses @consume to indicate that a match against zone is preferred:

a=RfidEvent(zone='Z1')@consume and b=RfidEvent(assetId='0001')

This pattern no longer fires when a single RfidEvent arrives that has zone 'Z1' and assetId '0001', because when the first filter expression matches the pattern engine consumes the event. The pattern only matches when two RfidEvent events arrive in any order. One event must have zone 'Z1' and the other event must have a zone other than 'Z1' and an assetId '0001'.

The next sample pattern provides a level number for each @consume:

a=RfidEvent(zone='Z1')@consume(2) 
  or b=RfidEvent(assetId='0001')@consume(1) 
  or c=RfidEvent(category='perishable'))

The pattern fires when an RfidEvent arrives with zone 'Z1'. In this case the output event populates property 'a' but not properties 'b' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other than 'Z1' and an asset id of '0001'. In this case the output event populates property 'b' but not properties 'a' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other than 'Z1' and an asset id other than '0001' and a category of 'perishable'. In this case the output event populates property 'c' but not properties 'a' and 'b'.

When your filter expression provides the name of a named window then the filter expression matches each time an event is inserted into the named window that matches the filter conditions.

For example, assume a named window that holds the last order event per order id:

create window LastOrderWindow#unique(orderId) as OrderEvent

Assume that all order events are inserted into the named window using insert-into:

insert into LastOrderWindow select * from OrderEvent

This sample pattern fires 10 seconds after an order event with a price greater then 100 was inserted:

select * from pattern [every o=LastOrderWindow(price >= 100) -> timer:interval(10 sec)]

The pattern above fires only for events inserted-into the LastOrderWindow named window and does not fire when an order event was updated using on-update or merged using on-merge.

If your application would like to have the pattern fire for any change to the named window events including updates and merges, you must select from the named window as follows:

insert into OrderWindowChangeStream select * from LastOrderWindow
select * from pattern [every o=OrderWindowChangeStream(price >= 100) -> timer:interval(10 sec)]

A table cannot be listed as part of a pattern filter, however any filter EPL expressions can have tables access expressions and subqueries against tables.

Assuming that MyTable is a table, the following is not allowed:

// not allowed
select * from pattern [every MyTable -> timer:interval(10 sec)]

The every operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every keyword evaluates to true or false. Without the every operator the pattern sub-expression stops when the pattern sub-expression evaluates to true or false.

As a side note, please be aware that a single invocation to the UpdateListener interface may deliver multiple events in one invocation, since the interface accepts an array of values.

Thus the every operator works like a factory for the pattern sub-expression contained within. When the pattern sub-expression within it fires and thus quits checking for events, the every causes the start of a new pattern sub-expression listening for more occurrences of the same event or set of events.

Every time a pattern sub-expression within an every operator turns true the engine starts a new active subexpression looking for more event(s) or timing conditions that match the pattern sub-expression. If the every operator is not specified for a subexpression, the subexpression stops after the first match was found.

This pattern fires when encountering an A event and then stops looking.

A

This pattern keeps firing when encountering A events, and doesn't stop looking.

every A

When using every operator with the -> followed-by operator, each time the every operator restarts it also starts a new subexpression instance looking for events in the followed-by subexpression.

Let's consider an example event sequence as follows.

A1   B1   C1   B2   A2   D1   A3   B3   E1   A4   F1   B4


The examples show that it is possible that a pattern fires for multiple combinations of events that match a pattern expression. Each combination is posted as an EventBean instance to the update method in the UpdateListener implementation.

Let's consider the every operator in conjunction with a subexpression that matches 3 events that follow each other:

every (A -> B -> C)

The pattern first looks for A events. When an A event arrives, it looks for a B event. After the B event arrives, the pattern looks for a C event. Finally, when the C event arrives the pattern fires. The engine then starts looking for an A event again.

Assume that between the B event and the C event a second A2 event arrives. The pattern would ignore the A2 event entirely since it's then looking for a C event. As observed in the prior example, the every operator restarts the subexpression A -> B -> C only when the subexpression fires.

In the next statement the every operator applies only to the A event, not the whole subexpression:

every A -> B -> C

This pattern now matches for each A event that is followed by a B event and then a C event, regardless of when the A event arrives. Note that for each A event that arrives the pattern engine starts a new subexpression looking for a B event and then a C event, outputting each combination of matching events.

As the introduction of the every operator states, the operator starts new subexpression instances and can cause multiple matches to occur for a single arriving event.

New subexpressions also take a very small amount of system resources and thereby your application should carefully consider when subexpressions must end when designing patterns. Use the timer:within construct and the and not constructs to end active subexpressions. The data window onto a pattern stream does not serve to limit pattern sub-expression lifetime.

Lets look at a concrete example. Consider the following sequence of events arriving:

A1   A2   B1  

This pattern matches on arrival of B1 and outputs two events (an array of length 2 if using a listener). The two events are the combinations {A1, B1} and {A2, B1}:

every a=A -> b=B

The and not operators are used to end an active subexpression.

The next pattern matches on arrival of B1 and outputs only the last A event which is the combination {A2, B1}:

every a=A -> (b=B and not A)

The and not operators cause the subexpression looking for {A1, B?} to end when A2 arrives.

Similarly, in the pattern below the engine starts a new subexpression looking for a B event every 1 second. After 5 seconds there are 5 subexpressions active looking for a B event and 5 matches occur at once if a B event arrives after 5 seconds.

every timer:interval(1 sec) -> b=B

Again the and not operators can end subexpressions that are not intended to match any longer:

every timer:interval(1 sec) -> (b=B and not timer:interval(1 sec))
// equivalent to
every timer:interval(1 sec) -> (b=B where timer:within(1 sec))

This example considers a generic pattern in which the pattern must match for each A event followed by a B event and followed by a C event, in which both the B event and the C event must arrive within 1 hour of the A event. The first approach to the pattern is as follows:

every A  -> (B -> C) where timer:within(1 hour)

Consider the following sequence of events arriving:

A1   A2   B1   C1   B2   C2

First, the pattern as above never stops looking for A events since the every operator instructs the pattern to keep looking for A events.

When A1 arrives, the pattern starts a new subexpression that keeps A1 in memory and looks for any B event. At the same time, it also keeps looking for more A events.

When A2 arrives, the pattern starts a new subexpression that keeps A2 in memory and looks for any B event. At the same time, it also keeps looking for more A events.

After the arrival of A2, there are 3 subexpressions active:

In the pattern above, there is a 1-hour lifetime for subexpressions looking for B and C events. Thus, if no B and no C event arrive within 1 hour after A1, the first subexpression goes away. If no B and no C event arrive within 1 hour after A2, the second subexpression goes away. The third subexpression however stays around looking for more A events.

The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and for combination {A2, B1, C1}, provided that B1 and C1 arrive within an hour of A1 and A2.

You may now ask how to match on {A1, B1, C1} and {A2, B2, C2} instead, since you may need to correlate on a given property.

The pattern as discussed above matches every A event followed by the first B event followed by the next C event, and doesn't specifically qualify the B or C events to look for based on the A event. To look for specific B and C events in relation to a given A event, specify correlating properties of the A event, for example:

every a=A -> (B(id=a.id) -> C(id=a.id)) where timer:within(1 hour)

The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and on arrival of C2 for combination {A2, B2, C2}.

Similar to the every operator in most aspects, the every-distinct operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every-distinct keyword evaluates to true or false. In addition, the every-distinct eliminates duplicate results received from an active subexpression according to its distinct-value expressions.

The synopsis for the every-distinct pattern operator is:

every-distinct(distinct_value_expr [, distinct_value_exp[...][, expiry_time_period])

Within parenthesis are one or more distinct_value_expr expressions that return the values by which to remove duplicates.

You may optionally specify an expiry_time_period time period. If present, the pattern engine expires and removes distinct key values that are older then the time period, removing their associated memory and allowing such distinct values to match again. When your distinct value expressions return an unlimited number of values, for example when your distinct value is a timestamp or auto-increment column, you should always specify an expiry time period.

When specifying properties in the distinct-value expression list, you must ensure that the event types providing properties are tagged. Only properties of event types within filter expressions that are sub-expressions to the every-distinct may be specified.

For example, this pattern keeps firing for every A event with a distinct value for its aprop property:

every-distinct(a.aprop) a=A

Note that the pattern above assigns the a tag to the A event and uses a.prop to identify the prop property as a value of the a event A.

A pattern that returns the first Sample event for each sensor, assuming sensor is a field that returns a unique id identifying the sensor that originated the Sample event, is:

every-distinct(s.sensor) s=Sample

The next pattern looks for pairs of A and B events and returns only the first pair for each combination of aprop of an A event and bprop of a B event:

every-distinct(a.aprop, b.bprop) (a=A and b=B)

The following pattern looks for A events followed by B events for which the value of the aprop of an A event is the same value of the bprop of a B event but only for each distinct value of aprop of an A event:

every-distinct(a.aprop) a=A -> b=B(bprop = a.aprop)

When specifying properties as part of distinct-value expressions, properties must be available from tagged event types in sub-expressions to the every-distinct.

The following patterns are not valid:

// Invalid: event type in filter not tagged
every-distinct(aprop) A
			
// Invalid: property not from a sub-expression of every-distinct
a=A -> every-distinct(a.aprop) b=B

When an active subexpression to every-distinct becomes permanently false, the distinct-values seen from the active subexpression are removed and the sub-expression within is restarted.

For example, the below pattern detects each A event distinct by the value of aprop.

every-distinct(a.aprop) (a=A and not B)

In the pattern above, when a B event arrives, the subexpression becomes permanently false and is restarted anew, detecting each A event distinct by the value of aprop without considering prior values.

When your distinct key is a timestamp or other non-unique property, specify an expiry time period.

The following example returns every distinct A event according to the timestamp property on the A event, retaining each timestamp value for 10 seconds:

every-distinct(a.timestamp, 10 seconds) a=A

In the example above, if for a given A event and its timestamp value the same timestamp value occurs again for another A event before 10 seconds passed, the A event is not a match. If 10 seconds passed the pattern indicates a second match.

You may not use every-distinct with a timer-within guard to expire keys: The expiry time notation as above is the recommended means to expire keys.

// This is not the same as above; It does not expire transaction ids and is not recommended
every-distinct(a.timestamp) a=A where timer:within(10 sec)

The repeat operator fires when a pattern sub-expression evaluates to true a given number of times. The synopsis is as follows:

[match_count] repeating_subexpr

The repeat operator is very similar to the every operator in that it restarts the repeating_subexpr pattern sub-expression up to a given number of times.

match_count is a positive number that specifies how often the repeating_subexpr pattern sub-expression must evaluate to true before the repeat expression itself evaluates to true, after which the engine may indicate a match.

For example, this pattern fires when the last of five A events arrives:

[5] A

Parenthesis must be used for nested pattern sub-expressions. This pattern fires when the last of a total of any five A or B events arrives:

[5] (A or B)

Without parenthesis the pattern semantics change, according to the operator precedence described earlier. This pattern fires when the last of a total of five A events arrives or a single B event arrives, whichever happens first:

[5] A or B

Tags can be used to name events in filter expression of pattern sub-expressions. The next pattern looks for an A event followed by a B event, and a second A event followed by a second B event. The output event provides indexed and array properties of the same name:

[2] (a=A -> b=B)

Using tags with repeat is further described in Section 7.5.4.6, “Tags and the Repeat Operator”.

Consider the following pattern that demonstrates the behavior when a pattern sub-expression becomes permanently false:

[2] (a=A and not C)

In the case where a C event arrives before 2 A events arrive, the pattern above becomes permanently false.

Lets add an every operator to restart the pattern and thus keep matching for all pairs of A events that arrive without a C event in between each pair:

every [2] (a=A and not C)

Since pattern matches return multiple A events, your select clause should use tag a as an array, for example:

select a[0].id, a[1].id from pattern [every [2] (a=A and not C)]

The repeat until operator provides additional control over repeated matching.

The repeat until operator takes an optional range, a pattern sub-expression to repeat, the until keyword and a second pattern sub-expression that ends the repetition. The synopsis is as follows:

[range] repeated_pattern_expr until end_pattern_expr

Without a range, the engine matches the repeated_pattern_expr pattern sub-expression until the end_pattern_expr evaluates to true, at which time the expression turns true.

An optional range can be used to indicate the minimum number of times that the repeated_pattern_expr pattern sub-expression must become true.

The optional range can also specify a maximum number of times that repeated_pattern_expr pattern sub-expression evaluates to true and retains tagged events. When this number is reached, the engine stops the repeated_pattern_expr pattern sub-expression.

The until keyword is always required when specifying a range and is not required if specifying a fixed number of repeat as discussed in the section before.

Similar to the Java && operator the and operator requires both nested pattern expressions to turn true before the whole expression turns true (a join pattern).

This pattern matches when both an A event and a B event arrive, at the time the last of the two events arrive:

A and B

This pattern matches on any sequence of an A event followed by a B event and then a C event followed by a D event, or a C event followed by a D and an A event followed by a B event:

(A -> B) and (C -> D)

Note that in an and pattern expression it is not possible to correlate events based on event property values. For example, this is an invalid pattern:

// This is NOT valid
a=A and B(id = a.id)

The above expression is invalid as it relies on the order of arrival of events, however in an and expression the order of events is not specified and events fulfill an and condition in any order. The above expression can be changed to use the followed-by operator:

// This is valid
a=A -> B(id = a.id)
// another example using 'and'...
a=A -> (B(id = a.id) and C(id = a.id))

Consider a pattern that looks for the same event:

A and A

The pattern above fires when a single A event arrives. The first arriving A event triggers a state transition in both the left and the right hand side expression.

In order to match after two A events arrive in any order, there are two options to express this pattern. The followed-by operator is one option and the repeat operator is the second option, as the next two patterns show:

A -> A
// ... or ...
[2] A

The not operator negates the truth value of an expression. Pattern expressions prefixed with not are automatically defaulted to true upon start, and turn permanently false when the expression within turns true.

The not operator is generally used in conjunction with the and operator or subexpressions as the below examples show.

This pattern matches only when an A event is encountered followed by a B event but only if no C event was encountered before either an A event and a B event, counting from the time the pattern is started:

(A -> B) and not C

Assume we'd like to detect when an A event is followed by a D event, without any B or C events between the A and D events:

A -> (D and not (B or C))

It may help your understanding to discuss a pattern that uses the or operator and the not operator together:

a=A -> (b=B or not C)

In the pattern above, when an A event arrives then the engine starts the subexpression B or not C. As soon as the subexpression starts, the not operator turns to true. The or expression turns true and thus your listener receives an invocation providing the A event in the property 'a'. The subexpression does not end and continues listening for B and C events. Upon arrival of a B event your listener receives a second invocation. If instead a C event arrives, the not turns permanently false however that does not affect the or operator (but would end an and operator).

To test for absence of an event, use timer:interval together with and not operators. The sample statement reports each 10-second interval during which no A event occurred:

every (timer:interval(10 sec) and not A)

In many cases the not operator, when used alone, does not make sense. The following example is invalid and will log a warning when the engine is started:

// not a sensible pattern
(not a=A) -> B(id=a.id)

The followed by -> operator specifies that first the left hand expression must turn true and only then is the right hand expression evaluated for matching events.

Look for an A event and if encountered, look for a B event. As always, A and B can itself be nested event pattern expressions.

A -> B

This is a pattern that fires when 2 status events indicating an error occur one after the other.

StatusEvent(status='ERROR') -> StatusEvent(status='ERROR')

A pattern that takes all A events that are not followed by a B event within 5 minutes:

every A -> (timer:interval(5 min) and not B)

A pattern that takes all A events that are not preceded by B within 5 minutes:

every (timer:interval(5 min) and not B -> A)

The followed-by -> operator can optionally be provided with an expression that limits the number of sub-expression instances of the right-hand side pattern sub-expression.

The synopsis for the followed-by operator with limiting expression is:

lhs_expression -[limit_expression]> rhs_expression

Each time the lhs_expression pattern sub-expression turns true the pattern engine starts a new rhs_expression pattern sub-expression. The limit_expression returns an integer value that defines a maximum number of pattern sub-expression instances that can simultaneously be present for the same rhs_expression.

When the limit is reached the pattern engine issues a com.espertech.esper.client.hook.ConditionPatternSubexpressionMax notification object to any condition handlers registered with the engine as described in Section 14.11, “Condition Handling” and does not start a new pattern sub-expression instance for the right-hand side pattern sub-expression.

For example, consider the following pattern which returns for every A event the first B event that matches the id field value of the A event:

every a=A -> b=B(id = a.id)

In the above pattern, every time an A event arrives (lhs) the pattern engine starts a new pattern sub-expression (rhs) consisting of a filter for the first B event that has the same value for the id field as the A event.

In some cases your application may want to limit the number of right-hand side sub-expressions because of memory concerns or to reduce output. You may add a limit expression returning an integer value as part of the operator.

This example employs the followed-by operator with a limit expression to indicate that maximally 2 filters for B events (the right-hand side pattern sub-expression) may be active at the same time:

every a=A -[2]> b=B(id = a.id)

Note that the limit expression in the example above is not a limit per value of id field, but a limit counting all right-hand side pattern sub-expression instances that are managed by that followed-by sub-expression instance.

If your followed-by operator lists multiple sub-expressions with limits, each limit applies to the immediate right-hand side. For example, the pattern below limits the number of filters for B events to 2 and the number of filters for C events to 3:

every a=A -[2]> b=B(id = a.id) -[3]> c=C(id = a.id)

Esper allows setting a maximum number of pattern sub-expressions in the configuration, applicable to all followed-by operators of all statements.

If your application has patterns in multiple EPL statements and all such patterns should count towards a total number of pattern sub-expression counts, you may consider setting a maximum number of pattern sub-expression instances, engine-wide, via the configuration described in Section 15.4.18.1, “Followed-By Operator Maximum Subexpression Count”.

When the limit is reached the pattern engine issues a notification object to any condition handlers registered with the engine as described in Section 14.11, “Condition Handling”. Depending on your configuration the engine can prevent the start of a new pattern sub-expression instance for the right-hand side pattern sub-expression, until pattern sub-expression instances end or statements are stopped or destroyed.

The notification object issued to condition handlers is an instance of com.espertech.esper.client.hook.ConditionPatternEngineSubexpressionMax. The notification object contains information which statement triggered the limit and the pattern counts per statement for all statements.

For information on static and runtime configuration, please consult Section 15.4.18.1, “Followed-By Operator Maximum Subexpression Count”. The limit can be changed and disabled or enabled at runtime via the runtime configuration API.

Guards are where-conditions that control the lifecycle of subexpressions. Custom guard functions can also be used. The section Chapter 17, Integration and Extension outlines guard plug-in development in greater detail.

The pattern guard where-condition has no relationship to the EPL where clause that filters sets of events.

Take as an example the following pattern expression:

MyEvent where timer:within(10 sec)

In this pattern the timer:within guard controls the subexpression that is looking for MyEvent events. The guard terminates the subexpression looking for MyEvent events after 10 seconds after start of the pattern. Thus the pattern alerts only once when the first MyEvent event arrives within 10 seconds after start of the pattern.

The every keyword requires additional discussion since it also controls subexpression lifecycle. Let's add the every keyword to the example pattern:

every MyEvent where timer:within(10 sec)

The difference to the pattern without every is that each MyEvent event that arrives now starts a new subexpression, including a new guard, looking for a further MyEvent event. The result is that, when a MyEvent arrives within 10 seconds after pattern start, the pattern execution will look for the next MyEvent event to arrive within 10 seconds after the previous one.

By placing parentheses around the every keyword and its subexpression, you can have the every under the control of the guard:

(every MyEvent) where timer:within(10 sec)

In the pattern above, the guard terminates the subexpression looking for all MyEvent events after 10 seconds after start of the pattern. This pattern alerts for all MyEvent events arriving within 10 seconds after pattern start, and then stops.

Guards do not change the truth value of the subexpression of which the guard controls the lifecycle, and therefore do not cause a restart of the subexpression when used with the every operator. For example, the next pattern stops returning matches after 10 seconds unless a match occurred within 10 seconds after pattern start:

every ( (A and B) where timer:within(10 sec) )

The timer:within guard acts like a stopwatch. If the associated pattern expression does not turn true within the specified time period it is stopped and permanently false.

The synopsis for timer:within is as follows:

timer:within(time_period_expression)

The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds as a parameter. The interval expression may contain references to properties of prior events in the same pattern as well as variables and substitution parameters.

This pattern fires if an A event arrives within 5 seconds after statement creation.

A where timer:within (5 seconds)

This pattern fires for all A events that arrive within 5 seconds. After 5 seconds, this pattern stops matching even if more A events arrive.

(every A) where timer:within (5 seconds)

This pattern matches for any one A or B event in the next 5 seconds.

( A or B ) where timer:within (5 sec)

This pattern matches for any 2 errors that happen 10 seconds within each other.

every (StatusEvent(status='ERROR') -> StatusEvent(status='ERROR') where timer:within (10 sec))

The following guards are equivalent:

timer:within(2 minutes 5 seconds)
timer:within(125 sec)
timer:within(125)

The timer:withinmax guard is similar to the timer:within guard and acts as a stopwatch that additionally has a counter that counts the number of matches. It ends the subexpression when either the stopwatch ends or the match counter maximum value is reached.

The synopsis for timer:withinmax is as follows:

timer:withinmax(time_period_expression, max_count_expression)

The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds.

The max_count_expression provides the maximum number of matches before the guard ends the subexpression.

Each parameter expression may also contain references to properties of prior events in the same pattern as well as variables and substitution parameters.

This pattern fires for every A event that arrives within 5 seconds after statement creation but only up to the first two A events:

(every A) where timer:withinmax (5 seconds, 2)

If the result of the max_count_expression is 1, the guard ends the subexpression after the first match and indicates the first match.

This pattern fires for the first A event that arrives within 5 seconds after statement creation:

(every A) where timer:withinmax (5 seconds, 1)

If the result of the max_count_expression is zero, the guard ends the subexpression upon the first match and does no indicate any matches.

This example receives every A event followed by every B event (as each B event arrives) until the 5-second subexpression timer ends or X number of B events have arrived (assume X was declared as a variable):

every A -> (every B) where timer:withinmax (5 seconds, X)

The timer:interval pattern observer waits for the defined time before the truth value of the observer turns true. The observer takes a time period (see Section 5.2.1, “Specifying Time Periods”) as a parameter, or an expression that returns the number of seconds.

The observer may be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value.

After an A event arrived wait 10 seconds then indicate that the pattern matches.

A -> timer:interval(10 seconds) 

The pattern below fires every 20 seconds.

every timer:interval(20 sec)

The next example pattern fires for every A event that is not followed by a B event within 60 seconds after the A event arrived. The B event must have the same "id" property value as the A event.

every a=A -> (timer:interval(60 sec) and not B(id=a.id)) 

Consider the next example, which assumes that the A event has a property waittime:

every a=A -> (timer:interval(a.waittime + 2) and not B(id=a.id))

In the above pattern the logic waits for 2 seconds plus the number of seconds provided by the value of the waittime property of the A event.

The timer:at pattern observer is similar in function to the Unix “crontab” command. At a specified time the expression turns true. The at operator can also be made to pattern match at regular intervals by using an every operator in front of the timer:at operator.

The syntax is: timer:at (minutes, hours, days of month, months, days of week [, seconds [, time zone]]).

The value for seconds and time zone is optional. Each element allows wildcard * values. Ranges can be specified by means of lower bounds then a colon ‘:’ then the upper bound. The division operator */x can be used to specify that every xth value is valid. Combinations of these operators can be used by placing these into square brackets ([]).

The timer:at observer may also be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value. The frequency division operator */x and parameters lists within brackets ([]) are an exception: they may only contain variables, substitution parameters or numeric values.

This expression pattern matches every 5 minutes past the hour.

every timer:at(5, *, *, *, *)

The below timer:at pattern matches every 15 minutes from 8am to 5:45pm (hours 8 to 17 at 0, 15, 30 and 45 minutes past the hour) on even numbered days of the month as well as on the first day of the month.

timer:at (*/15, 8:17, [*/2, 1], *, *)

The below table outlines the fields, valid values and keywords available for each field:


The keyword last used in the days-of-month field means the last day of the month (current month). To specify the last day of another month, you must provide a value for the month field. For example: timer:at(*, *, last,2,*) is the last day of February.

The last keyword in the day-of-week field by itself simply means Saturday. If used in the day-of-week field after another value, it means "the last xxx day of the month" - for example "5 last" means "the last Friday of the month". So the last Friday of the current month will be: timer:at(*, *, *, *, 5 last). And the last Friday of June: timer:at(*, *, *, 6, 5 last).

The keyword weekday is used to specify the weekday (Monday-Friday) nearest the given day. Variant could include month like in: timer:at(*, *, 30 weekday, 9, *) which for year 2007 is Friday September 28th (no jump over month).

The keyword lastweekday is a combination of two parameters, the last and the weekday keywords. A typical example could be: timer:at(*, *, *, lastweekday, 9, *) which will define Friday September 28th (example year is 2007).

The time zone is a string-type value that specifies the time zone of the schedule. You must specify a value for seconds when specifying a time zone. Esper relies on the java.util.TimeZone to interpret the time zone value. Note that TimeZone does not validate time zone strings.

The following timer:at pattern matches at 5:00 pm Pacific Standard Time (PST):

timer:at (0, 17, *, *, *, *, 'PST')

Any expression may occur among the parameters. This example invokes a user-defined function computeHour to return an hour:

timer:at (0, computeHour(), *, *, *, *)

The following restrictions apply to crontab parameters:

  • It is not possible to specify both Days Of Month and Days Of Week.

The timer:schedule observer is a flexible observer for scheduling.

The observer implements relevant parts of the ISO 8601 specification however it is not necessary to use ISO 8601 formats. The ISO 8601 standard is an international standard covering the exchange of date and time-related data. The standard specifies a date format, a format for time periods and a format for specifying the number of repetitions. Please find more information on ISO 8601 at Wikipedia.

The observer takes the following named parameters:


In summary, for example, the below pattern schedules two callbacks: The first callback 2008-03-01 at 13:00:00 UTC and the second callback on 2009-05-11 at 15:30:00 UTC.

select * from pattern[every timer:schedule(iso: 'R2/2008-03-01T13:00:00Z/P1Y2M10DT2H30M')]

The number of repetitions, date and period can be separated and do not have to be ISO 8601 strings, allowing each part to be an own expression.

This example specifies separate expressions. The equivalent schedule to the above example is:

select * from pattern[every timer:schedule(repetitions: 2, date: '2008-03-01T13:00:00Z', period: 1 year 2 month 10 days 2 hours 30 minutes)]

When providing the iso parameter, it must be the only parameter. The repetitions parameter is only allowed in conjunction with other parameters.

This walkthrough discusses the following pattern:

every a=LoginEvent -> (timer:interval(1 min) and not LogoutEvent(userId=a.userId))]

This pattern detects when a LoginEvent is not followed by a LogoutEvent arriving within 1 minute of the LoginEvent.

The engine parses the pattern expression and builds a expression tree. At the root of the expression tree, for this specific pattern, is the followed-by (->) operator as followed-by has the lowest precedence (see precedence).

The pattern expression tree looks like this.

Followed-by operator (->)
  |
  | ---- Every
  |       |
  |       | ---- a=LoginEvent
  |
  | ---- And
          |
          | ---- timer:interval(1 min)
          |                  
          | ---- Not
                 |
                 | ---- LogoutEvent(userId=a.userId)

The followed-by (->) operator has two subexpressions that it manages, the Every expression that itself has the a=LoginEvent subexpression as a child, and the And expression that itself has two subexpressions as children.

When the engine activates a pattern or pattern subexpression, it activates it from a top-down direction. In this example the engine activates the followed-by (->) expression. The followed-by expression activates only its own left-most subexpression, which is the Every expression. The Every expression in turn activates the a=LoginEvent subexpression which registers a filter looking for a LoginEvent in the global filter index (see Section 2.18.2, “Filter Indexes”).

In this example, the And subexpression and its subexpressions do not get activated. Therefore after this pattern initially activates there is no filter active time looking for LogoutEvent event and there is no time tracking of any kind at this point.

Let time t0 be the time of EPL statement creation. As discussed, upon creation of the EPL statement the engine activates the pattern subexpression every a=LoginEvent only. At this time there is 1 active subexpression:

  1. every a=LoginEvent

As you can see, the tree of active subexpressions (the tree of states) is not the same as the tree of expressions. After the pattern activated there are no active subexpressions (no states) for the And-part of the example expression tree.

Let's assume that at time t1 a LoginEvent with user id 10 arrives. This causes the a=LoginEvent subexpression to become true. In turn, this causes the Every subexpression to become true. This causes the left-hand-side of the followed-by subexpression to become true. This in turn causes activation of the And subexpression.

The activation of the And-subexpression causes activation of both timer:interval(1 min) (from t1) and the Not-subexpression. The activation of the And-subexpression registers a filter looking for a LogoutEvent that has a user id value of 10 in the global filter index and registers a timer callback for t1+1 minute.

The engine does not terminate the subexpression a=LoginEvent as it lives under an Every operator which means it should keep looking for more LoginEvent events.

There now are 2 active subexpressions:

  1. every a=LoginEvent

  2. timer:interval(t1+1 min) and not LogoutEvent(userId=10)

Let's assume that at time t2 another LoginEvent with user id 20 arrives (assume t2 – t1 < 1 min). This again causes the a=LoginEvent subexpression to become true. In turn, this causes the Every subexpression to become true. This causes the left-hand-side of the followed-by subexpression to become true. This in turn causes activation of the And subexpression.

The activation of the And-subexpression causes activation of both timer:interval(1 min) (from t2) and the Not-subexpression. The activation of the And-subexpression registers a filter looking for a LogoutEvent that has a user id value of 20 in the global filter index and registers a timer callback for t2+1 minute.

There now are 3 active subexpressions:

  1. every a=LoginEvent

  2. timer:interval(t1+1 min) and not LogoutEvent(userId=10)

  3. timer:interval(t2+1 min) and not LogoutEvent(userId=20)

In this scenario at time t3 one minute has passed since t1 and a matching LogoutEvent was not received.

The subexpression timer:interval(t1+1 min) becomes true. In turn, this causes the And-subexpression to become true as the Not-subexpression is already true on start. In turn, this causes the followed-by subexpression to become true. In turn, this causes an output of the pattern match.

The engine terminates the subexpression not LogoutEvent(userId=10) which unregisters the filter looking for a LogoutEvent that has a user id value of 10 from the global filter index.

There now are 2 active subexpressions:

  1. every a=LoginEvent

  2. timer:interval(t2+1 min) and not LogoutEvent(userId=20)

Let's assume that at time t4 a LogoutEvent with user id 20 arrives (assume t4 – t2 < 1 min). This again causes the LogoutEvent(userId=20) subexpression to become true. In turn, this causes the Not subexpression to become permanently false (as Not reverses the truth value). This causes the And-subexpression to become permanently false. This causes the pattern subexpression timer:interval(t2+1 min) and not LogoutEvent(userId=20) to terminate which causes the timer callback t2+1 minute to unregister.

Finally there is 1 active subexpression remaining:

  1. every a=LoginEvent