esper.codehaus.org and espertech.comDocumentation
Event patterns match when an event or multiple events occur that match the pattern's definition. Patterns can also be time-based.
Pattern expressions consist of pattern atoms and pattern operators:
Pattern atoms are the basic building blocks of patterns. Atoms are filter expressions, observers for time-based events and plug-in custom observers that observe external events not under the control of the engine.
Pattern operators control expression lifecycle and combine atoms logically or temporally.
The below table outlines the different pattern atoms available:
Table 6.1. Pattern Atoms
| Pattern Atom | Example |
|---|---|
| Filter expressions specify an event to look for. | StockTick(symbol='ABC', price > 100) |
| Time-based event observers specify time intervals or time schedules. | timer:interval(10 seconds) timer:at(*, 16, *, *, *) |
| Custom plug-in observers can add pattern language syntax for observing application-specific events. | myapplication:myobserver("http://someResource")
|
There are 4 types of pattern operators:
Operators that control pattern sub-expression repetition: every, every-distinct, [num] and until
Logical operators: and, or, not
Temporal operators that operate on event order: -> (followed-by)
Guards are where-conditions that control the lifecycle of subexpressions. Examples are timer:within, timer:withinmax and while-expression. Custom plug-in guards may also be used.
Pattern expressions can be nested arbitrarily deep by including the nested expression(s) in () round parenthesis.
Underlying the pattern matching is a state machine that transitions between states based on arriving events and based on time advancing. A single event or advancing time may cause a reaction in multiple parts of your active pattern state.
This is an example pattern expression that matches on every ServiceMeasurement events in which the
value of the latency event property is over 20 seconds, and on every ServiceMeasurement event in which the
success property is false. Either one or the other condition must be true for this pattern to match.
every ( spike=ServiceMeasurement(latency>20000) or error=ServiceMeasurement(success=false) )
In the example above, the pattern expression starts with an every operator to indicate that the pattern should fire for every matching events and not just the first
matching event. Within the every operator in round brackets is a nested pattern expression using the or operator.
The left hand of the or operator is a filter expression that filters for events with a high latency value. The right hand of the operator
contains a filter expression that filters for events with error status. Filter expressions are explained in Section 6.4, “Filter Expressions In Patterns”.
The example above assigned the tags spike and error to the events in the pattern. The tags are important since the
engine only places tagged events into the output event(s) that a pattern generates, and that the engine supplies to listeners of the pattern statement. The tags can
further be selected in the select-clause of an EPL statement as discussed in Section 5.4.2, “Pattern-based Event Streams”.
Patterns can also contain comments within the pattern as outlined in Section 5.2.2, “Using Comments”.
Pattern statements are created via the EPAdministrator interface. The EPAdministrator interface allows to create pattern statements in two ways:
Pattern statements that want to make use of the EPL select clause or any other EPL constructs use the createEPL method to create a statement that specifies one or more pattern expressions. EPL statements that use patterns are described in more detail in Section 5.4.2, “Pattern-based Event Streams”. Use the syntax as shown in below example.
EPAdministrator admin = EPServiceProviderManager.getDefaultProvider().getEPAdministrator();
String eventName = ServiceMeasurement.class.getName();
EPStatement myTrigger = admin.createEPL("select * from pattern [" +
"every (spike=" + eventName + "(latency>20000) or error=" + eventName + "(success=false))]");
Pattern statements that do not need to make use of the EPL select clause or any other EPL constructs can use the createPattern method, as in below example.
EPStatement myTrigger = admin.createPattern( "every (spike=" + eventName + "(latency>20000) or error=" + eventName + "(success=false))");
A pattern may appear anywhere in the from clause of an EPL statement including joins and subqueries. Patterns may therefore be used in combination with the where clause, group by clause, having clause as well as output rate limiting and insert into.
In addition, a data window view can be declared onto a pattern. A data window declared onto a pattern only serves to retain pattern matches. A data window declared onto a pattern does not limit, cancel, remove or delete intermediate pattern matches of the pattern when pattern matches leave the data window.
This example statement demonstrates the idea by selecting a total price per customer over pairs of events (ServiceOrder followed by a ProductOrder event for the same customer id within 1 minute), occuring in the last 2 hours, in which the sum of price is greater than 100, and using a where clause to filter on name:
select a.custId, sum(a.price + b.price)
from pattern [every a=ServiceOrder ->
b=ProductOrder(custId = a.custId) where timer:within(1 min)].win:time(2 hour)
where a.name in ('Repair', b.name)
group by a.custId
having sum(a.price + b.price) > 100
When a pattern fires it publishes one or more events to any listeners to the pattern statement.
The listener interface is the com.espertech.esper.client.UpdateListener interface.
The example below shows an anonymous implementation of the com.espertech.esper.client.UpdateListener interface.
We add the anonymous listener implementation to the myPattern statement created earlier.
The listener code simply extracts the underlying event class.
myPattern.addListener(new UpdateListener() {
public void update(EventBean[] newEvents, EventBean[] oldEvents) {
ServiceMeasurement spike = (ServiceMeasurement) newEvents[0].get("spike");
ServiceMeasurement error = (ServiceMeasurement) newEvents[0].get("error");
... // either spike or error can be null, depending on which occurred
... // add more logic here
}
});
Listeners receive an array of EventBean instances in the newEvents parameter.
There is one EventBean instance passed to the listener for each combination of events that matches
the pattern expression. At least one EventBean instance is always passed to the listener.
The properties of each EventBean instance contain the underlying events that caused the
pattern to fire, if events have been named in the filter expression via the name=eventType syntax.
The property name is thus the name supplied in the pattern expression, while the property type is the type of the underlying class,
in this example ServiceMeasurement.
Data can also be obtained from pattern statements via the safeIterator() and iterator() methods on EPStatement (the pull API).
If the pattern had fired at least once, then the iterator returns the last event for which it fired.
The hasNext() method can be used to determine if the pattern had fired.
if (myPattern.iterator().hasNext()) {
ServiceMeasurement event = (ServiceMeasurement) view.iterator().next().get("alert");
... // some more code here to process the event
}
else {
... // no matching events at this time
}Further, if a data window is defined onto a pattern, the iterator returns the pattern matches according to the data window expiry policy.
This pattern specifies a length window of 10 elements that retains the last 10 matches of A and B events, for use via iterator or for use in a join or subquery:
select * from pattern [every (A or B).win:length(10)
While the pattern compiler analyzes your pattern and verifies its integrity, it may not detect certain pattern errors that may occur at runtime. Sections of this pattern documentation point out common cases where the pattern engine will log a runtime error. We recommend turning on the log warning level at project development time to inspect and report on warnings logged. If a statement name is assigned to a statement then the statement name is logged as well.
The operators at the top of this table take precedence over operators lower on the table.
Table 6.2. Pattern Operator Precedence
| Precedence | Operator | Description | Example |
|---|---|---|---|
| 1 | guard postfix | where timer:within and while (expression) (incl. withinmax and plug-in pattern guard) | MyEvent where timer:within(1 sec) a=MyEvent while (a.price between 1 and 10) |
| 2 | unary | every, not, every-distinct | every MyEvent timer:interval(5 min) and not MyEvent |
| 3 | repeat | [num], until | [5] MyEvent [1..3] MyEvent until MyOtherEvent |
| 4 | and | and | every (MyEvent and MyOtherEvent) |
| 5 | or | or | every (MyEvent or MyOtherEvent) |
| 6 | followed-by | -> | every (MyEvent -> MyOtherEvent) |
If you are not sure about the precedence, please consider placing parenthesis () around your subexpressions. Parenthesis can also help make
expressions easier to read and understand.
The following table outlines sample equivalent expressions, with and without the use of parenthesis for subexpressions.
Table 6.3. Equivalent Pattern Expressions
| Expression | Equivalent | Reason |
|---|---|---|
| every A or B | (every A) or B | The every operator has higher precedence then the or operator. |
| every A -> B or C | (every A) -> (B or C) | The or operator has higher precedence then the followed-by operator. |
| A -> B or B -> A | A -> (B or B) -> A | The or operator has higher precedence then the followed-by operator,
specify as (A -> B) or (B -> A) instead.
|
| A and B or C | (A and B) or C | The and operator has higher precedence then the or operator. |
| A -> B until C -> D | A -> (B until C) -> D | The until operator has higher precedence then the followed-by operator. |
| [5] A or B | ([5] A) or B | The [num] repeat operator has higher precedence then the or operator. |
| every A where timer:within(10) | every (A where timer:within(10)) | The where postfix has higher precedence then the every operator. |
The simplest form of filter is a filter for events of a given type without any conditions on the event property values. This filter matches any event of that type regardless of the event's properties. The example below is such a filter. Note that this event pattern would stop firing as soon as the first RfidEvent is encountered.
com.mypackage.myevents.RfidEvent
To make the event pattern fire for every RfidEvent and not just the first event, use the every keyword.
every com.mypackage.myevents.RfidEvent
The example above specifies the fully-qualified Java class name as the event type. Via configuration, the event pattern above can be simplified by using the name that has been defined for the event type.
every RfidEvent
Interfaces and superclasses are also supported as event types. In the below example IRfidReadable is an interface class, and the statement matches any event that implements this interface:
every org.myorg.rfid.IRfidReadable
The filtering criteria to filter for events with certain event property values are placed within parenthesis after the event type name:
RfidEvent(category="Perishable")
All expressions can be used in filters, including static method invocations that return a boolean value:
RfidEvent(com.mycompany.MyRFIDLib.isInRange(x, y) or (x<0 and y < 0))
Filter expressions can be separated via a single comma ','. The comma represents a logical AND between expressions:
RfidEvent(zone=1, category=10) ...is equivalent to... RfidEvent(zone=1 and category=10)
The following set of operators are highly optimized through indexing and are the preferred means of filtering high-volume event streams:
equals =
not equals !=
comparison operators < , > , >=, <=
ranges
use the between keyword for a closed range where both endpoints are included
use the in keyword and round () or square brackets [] to control how endpoints are included
for inverted ranges use the not keyword and the between or in keywords
list-of-values checks using the in keyword or the not in keywords followed by a comma-separated list of values
At compile time as well as at run time, the engine scans new filter expressions for subexpressions that can be indexed. Indexing filter values to match event properties of incoming events enables the engine to match incoming events faster. The above list of operators represents the set of operators that the engine can best convert into indexes. The use of comma or logical and in filter expressions does not impact optimizations by the engine.
For more information on filters please see Section 5.4.1, “Filter-based Event Streams”. Contained-event selection on filters in patterns is further described in Section 5.20, “Contained-Event Selection”.
Filter criteria can also refer to events matching prior named events in the same expression. Below pattern is an example in which the pattern matches once for every RfidEvent that is preceded by an RfidEvent with the same asset id.
every e1=RfidEvent -> e2=RfidEvent(assetId=e1.assetId)
The syntax shown above allows filter criteria to reference prior results by specifying the event name tag of the prior event, and the event property name. The tag names in the above example were e1 and e2. This syntax can be used in all filter operators or expressions including ranges and the in set-of-values check:
every e1=RfidEvent -> e2=RfidEvent(MyLib.isInRadius(e1.x, e1.y, x, y) and zone in (1, e1.zone))
An arriving event changes the truth value of all expressions that look for the event. Consider the pattern as follows:
every (RfidEvent(zone > 1) and RfidEvent(zone < 10))
The pattern above is satisfied as soon as only one event with zone in the interval [2, 9] is received.
An arriving event applies to all filter expressions for which the event matches. In other words, an arriving event is not consumed by any specify filter expression(s) but applies to all active filter expressions of all pattern sub-expressions.
You may provide the @consume annotation as part of a filter expression to control consumption of an arriving event. If an arriving event matches the filter expression marked with @consume it
is no longer available to other filter expressions of the same pattern that also match the arriving event.
The @consume can include a level number in parenthesis. A higher level number consumes the event first. The default level number is 1. Multiple filter expressions with the same level number for @consume all match the event.
Consider the next sample pattern:
a=RfidEvent(zone='Z1') and b=RfidEvent(assetId='0001')
This pattern fires when a single RfidEvent event arrives that has zone 'Z1' and assetId '0001'. The pattern also matches when two RfidEvent events arrive, in any order, wherein one has zone 'Z1' and the other has assetId '0001'.
Mark a filter expression with @consume to indicate that if an arriving event matches multiple filter expressions that the engine prefers the marked filter expression and does not match any other filter expression.
This updated pattern statement uses @consume to indicate that a match against zone is preferred:
a=RfidEvent(zone='Z1')@consume and b=RfidEvent(assetId='0001')
This pattern no longer fires when a single RfidEvent arrives that has zone 'Z1' and assetId '0001', because when the first filter expression matches the pattern engine consumes the event. The pattern only matches when two RfidEvent events arrive in any order. One event must have zone 'Z1' and the other event must have a zone other then 'Z1' and an assetId '0001'.
The next sample pattern provides a level number for each @consume:
a=RfidEvent(zone='Z1')@consume(2) or b=RfidEvent(assetId='0001')@consume(1) or c=RfidEvent(category='perishable'))
The pattern fires when an RfidEvent arrives with zone 'Z1'. In this case the output event populates property 'a' but not properties 'b' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other then 'Z1' and an asset id of '0001'. In this case the output event populates property 'b' but not properties 'a' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other then 'Z1' and an asset id other then '0001' and a category of 'perishable'. In this case the output event populates property 'c' but not properties 'a' and 'b'.
The every operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every keyword evaluates to true or false.
Without the every operator the pattern sub-expression stops when the pattern sub-expression evaluates to true or false.
As a side note, please be aware that a single invocation to the UpdateListener interface may deliver multiple events in one invocation, since the interface accepts an array of values.
Thus the every operator works like a factory for the pattern sub-expression contained within. When the pattern sub-expression within it fires and thus quits checking for events, the every causes the start of a new pattern sub-expression listening for more occurrences of the same
event or set of events.
Every time a pattern sub-expression within an every operator turns true the engine starts a new active subexpression looking
for more event(s) or timing conditions that match the pattern sub-expression. If the every operator is not specified for a subexpression,
the subexpression stops after the first match was found.
This pattern fires when encountering an A event and then stops looking.
A
This pattern keeps firing when encountering A events, and doesn't stop looking.
every A
When using every operator with the -> followed-by operator, each time the every operator restarts it also starts a new subexpression instance looking for events in the followed-by subexpression.
Let's consider an example event sequence as follows.
A1 B1 C1 B2 A2 D1 A3 B3 E1 A4 F1 B4
Table 6.4. 'Every' operator examples
| Example | Description |
|---|---|
every ( A -> B ) |
Detect an A event followed by a B event. At the time when B occurs the pattern matches, then the pattern matcher restarts and looks for the next A event.
|
every A -> B |
The pattern fires for every A event followed by a B event.
|
A -> every B |
The pattern fires for an A event followed by every B event.
|
every A -> every B |
The pattern fires for every A event followed by every B event.
|
The examples show that it is possible that a pattern fires for multiple combinations of events that match a pattern expression.
Each combination is posted as an EventBean instance to the update method in the UpdateListener implementation.
Let's consider the every operator in conjunction with a subexpression that matches 3 events that follow each other:
every (A -> B -> C)
The pattern first looks for A events. When an A event arrives, it looks for a B event. After the B event arrives, the pattern looks for a C event. Finally, when the C event arrives the pattern fires. The engine then starts looking for an A event again.
Assume that between the B event and the C event a second A2 event arrives. The pattern would ignore the A2 event entirely since it's then looking for a C event.
As observed in the prior example, the every operator restarts the subexpression A -> B -> C only when the subexpression fires.
In the next statement the every operator applies only to the A event, not the whole subexpression:
every A -> B -> C
This pattern now matches for each A event that is followed by a B event and then a C event, regardless of when the A event arrives. Note that for each A event that arrives the pattern engine starts a new subexpression looking for a B event and then a C event, outputting each combination of matching events.
A pattern that only has the every operator and a single filter expression is equivalent to selecting the same filter in the from clause:
select * from StockTickEvent(symbol='GE') // Prefer this // ... equivalent to ... select * from pattern[every StockTickEvent(symbol='GE')]
As the introduction of the every operator states, the operator starts new subexpression instances and can cause multiple matches to occur for a single arriving event.
New subexpressions also take a very small amount of system resources and thereby your application should carefully consider when subexpressions must end when designing patterns. Use the timer:within construct and the and not constructs to end active subexpressions. The data window onto a pattern stream does not serve to limit pattern sub-expression lifetime.
Lets look at a concrete example. Consider the following sequence of events arriving:
A1 A2 B1
This pattern matches on arrival of B1 and outputs two events (an array of length 2 if using a listener). The two events are the combinations {A1, B1} and {A2, B1}:
every a=A -> b=B
The and not operators are used to end an active subexpression.
The next pattern matches on arrival of B1 and outputs only the last A event which is the combination {A2, B1}:
every a=A -> (b=B and not A)
The and not operators cause the subexpression looking for {A1, B?} to end when A2 arrives.
Similarly, in the pattern below the engine starts a new subexpression looking for a B event every 1 second. After 5 seconds there are 5 subexpressions active looking for a B event and 5 matches occur at once if a B event arrives after 5 seconds.
every timer:interval(1 sec) -> b=B
Again the and not operators can end subexpressions that are not intended to match any longer:
every timer:interval(1 sec) -> (b=B and not timer:interval(1 sec)) // equivalent to every timer:interval(1 sec) -> (b=B where timer:within(1 sec))
In this example we consider a generic pattern in which the pattern must match for each A event followed by a B event and followed by a C event, in which both the B event and the C event must arrive within 1 hour of the A event. The first approach to the pattern is as follows:
every A -> (B -> C) where timer:within(1 hour)
Consider the following sequence of events arriving:
A1 A2 B1 C1 B2 C2
First, the pattern as above never stops looking for A events since the every operator instructs the pattern to keep looking for A events.
When A1 arrives, the pattern starts a new subexpression that keeps A1 in memory and looks for any B event. At the same time, it also keeps looking for more A events.
When A2 arrives, the pattern starts a new subexpression that keeps A2 in memory and looks for any B event. At the same time, it also keeps looking for more A events.
After the arrival of A2, there are 3 subexpressions active:
The first active subexpression with A1 in memory, looking for any B event.
The second active subexpression with A2 in memory, looking for any B event.
A third active subexpression, looking for the next A event.
In the pattern above, we have specified a 1-hour lifetime for subexpressions looking for B and C events. Thus, if no B and no C event arrive within 1 hour after A1, the first subexpression goes away. If no B and no C event arrive within 1 hour after A2, the second subexpression goes away. The third subexpression however stays around looking for more A events.
The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and for combination {A2, B1, C1}, provided that B1 and C1 arrive within an hour of A1 and A2.
You may now ask how to match on {A1, B1, C1} and {A2, B2, C2} instead, since you may need to correlate on a given property.
The pattern as discussed above matches every A event followed by the first B event followed by the next C event, and doesn't specifically qualify the B or C events to look for based on the A event. To look for specific B and C events in relation to a given A event, the correlation must use one or more of the properties of the A event, such as the "id" property:
every a=A -> (B(id=a.id -> C(id=a.id)) where timer:within(1 hour)
The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and on arrival of C2 for combination {A2, B2, C2}.
This example looks at temperature sensor events named Sample. The pattern detects when 3 sensor events indicate a temperature of more then 50 degrees uninterrupted within 90 seconds of the first event, considering events for the same sensor only.
every sample=Sample(temp > 50) -> ( (Sample(sensor=sample.sensor, temp > 50) and not Sample(sensor=sample.sensor, temp <= 50)) -> (Sample(sensor=sample.sensor, temp > 50) and not Sample(sensor=sample.sensor, temp <= 50)) ) where timer:within(90 seconds))
The pattern starts a new subexpression in the round braces after the first followed-by operator for each time a sensor indicated more then 50 degrees. Each subexpression then lives a maximum of 90 seconds. Each subexpression ends if a temperature of 50 degress or less is encountered for the same sensor. Only if 3 temperature events in a row indicate more then 50 degrees, and within 90 seconds of the first event, and for the same sensor, does this pattern fire.
Similar to the every operator in most aspects, the every-distinct operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every-distinct keyword evaluates to true or false. In addition, the every-distinct eliminates duplicate results received from an active subexpression according to its distinct-value expressions.
The synopsis for the every-distinct pattern operator is:
every-distinct(distinct_value_expr [, distinct_value_exp[...][, expiry_time_period])
Within parenthesis are one or more distinct_value_expr expressions that return the values by which to remove duplicates.
You may optionally specify an expiry_time_period time period. If present, the pattern engine expires and removes distinct key values that are older then the time period, removing their associated memory and allowing such distinct values to match again. When your distinct value expressions return an unlimited number of values, for example when your distinct value is a timestamp or auto-increment column, you should always specify an expiry time period.
When specifying properties in the distinct-value expression list, you must ensure that the event types providing properties are tagged. Only properties of event types within filter expressions that are sub-expressions to the every-distinct may be specified.
For example, this pattern keeps firing for every A event with a distinct value for its aprop property:
every-distinct(a.aprop) a=A
Note that the pattern above assigns the a tag to the A event and uses a.prop to identify the prop property as a value of the a event A.
A pattern that returns the first Sample event for each sensor, assuming sensor is a field that returns a unique id identifying the sensor that originated the Sample event, is:
every-distinct(s.sensor) s=Sample
The next pattern looks for pairs of A and B events and returns only the first pair for each combination of aprop of an A event and bprop of a B event:
every-distinct(a.aprop, b.bprop) (a=A and b=B)
The following pattern looks for A events followed by B events for which the value of the aprop of an A event is the same value of the bprop of a B event but only for each distinct value of aprop of an A event:
every-distinct(a.aprop) a=A -> b=B(bprop = a.aprop)
When specifying properties as part of distinct-value expressions, properties must be available from tagged event types in sub-expressions to the every-distinct.
The following patterns are not valid:
// Invalid: event type in filter not tagged every-distinct(aprop) A // Invalid: property not from a sub-expression of every-distinct a=A -> every-distinct(a.aprop) b=B
When an active subexpression to every-distinct becomes permanently false, the distinct-values seen from the active subexpression are removed and the sub-expression within is restarted.
For example, the below pattern detects each A event distinct by the value of aprop.
every-distinct(a.aprop) (a=A and not B)
In the pattern above, when a B event arrives, the subexpression becomes permanently false and is restarted anew, detecting each A event distinct by the value of aprop without considering prior values.
When your distinct key is a timestamp or other non-unique property, specify an expiry time period.
The following example returns every distinct A event according to the timestamp property on the A event, retaining each timestamp value for 10 seconds:
every-distinct(a.timestamp, 10 seconds) a=A
In the example above, if for a given A event and its timestamp value the same timestamp value occurs again for another A event before 10 seconds passed, the A event is not a match. If 10 seconds passed the pattern indicates a second match.
You may not use every-distinct with a timer-within guard to expire keys: The expiry time notation as above is the recommended means to expire keys.
// This is not the same as above; It does not expire transaction ids and is not recommended every-distinct(a.timestamp) a=A where timer:within(10 sec)
The repeat operator fires when a pattern sub-expression evaluates to true a given number of times. The synopsis is as follows:
[match_count] repeating_subexpr
The repeat operator is very similar to the every operator in that it restarts the repeating_subexpr pattern sub-expression up to a given number of times.
match_count is a positive number that specifies how often the repeating_subexpr pattern sub-expression must evaluate to true before the repeat expression itself evaluates to true, after which the engine may indicate a match.
For example, this pattern fires when the last of five A events arrives:
[5] A
Parenthesis must be used for nested pattern sub-expressions. This pattern fires when the last of a total of any five A or B events arrives:
[5] (A or B)
Without parenthesis the pattern semantics change, according to the operator precedence described earlier. This pattern fires when the last of a total of five A events arrives or a single B event arrives, whichever happens first:
[5] A or B
Tags can be used to name events in filter expression of pattern sub-expressions. The next pattern looks for an A event followed by a B event, and a second A event followed by a second B event. The output event provides indexed and array properties of the same name:
[2] (a=A -> b=B)
Using tags with repeat is further described in Section 6.5.4.6, “Tags and the Repeat Operator”.
Consider the following pattern that demonstrates the behavior when a pattern sub-expression becomes permanently false:
[2] (a=A and not C)
In the case where a C event arrives before 2 A events arrive, the pattern above becomes permanently false.
Lets add an every operator to restart the pattern and thus keep matching for all pairs of A events that arrive without a C event in between each pair:
every [2] (a=A and not C)
Since pattern matches return multiple A events, your select clause should use tag a as an array, for example:
select a[0].id, a[1].id from pattern [every [2] (a=A and not C)]
The repeat until operator provides additional control over repeated matching.
The repeat until operator takes an optional range, a pattern sub-expression to repeat, the until keyword and a second pattern sub-expression that ends the repetition. The synopsis is as follows:
[range] repeated_pattern_expr until end_pattern_expr
Without a range, the engine matches the repeated_pattern_expr pattern sub-expression until the end_pattern_expr evaluates to true, at which time the expression turns true.
An optional range can be used to indicate the minimum number of times that the repeated_pattern_expr pattern sub-expression must become true.
The optional range can also specify a maximum number of times that repeated_pattern_expr pattern sub-expression evaluates to true and retains tagged events. When this number is reached, the engine stops the repeated_pattern_expr pattern sub-expression.
The until keyword is always required when specifying a range and is not required if specifying a fixed number of repeat as discussed in the section before.
In the unbound repeat, without a range, the engine matches the repeated_pattern_expr pattern sub-expression until the end_pattern_expr evaluates to true, at which time the expression turns true. The synopsis is:
repeated_pattern_expr until end_pattern_expr
This is a pattern that keeps looking for A events until a B event arrives:
A until B
Nested pattern sub-expressions must be placed in parenthesis since the until operator has precedence over most operators. This example collects all A or B events for 10 seconds and places events received in indexed properties 'a' and 'b':
(a=A or b=B) until timer:interval(10 sec)
The synopsis for the optional range qualifier is:
[ [low_endpoint] : [high_endpoint] ]
low_endpoint is an optional number that appears on the left of a colon (:), after which follows an optional high_endpoint number.
A range thus consists of a low_endpoint and a high_endpoint in square brackets and separated by a colon (:) characters. Both endpoint values are optional but either one or both must be supplied. The low_endpoint can be omitted to denote a range that starts at zero. The high_endpoint can be omitted to denote an open-ended range.
Some examples for valid ranges might be:
[3 : 10] [:3] // range starts at zero [2:] // open-ended range
The low_endpoint, if specified, defines the minimum number of times that the repeated_pattern_expr pattern sub-expression must become true in order for the expression to become true.
The high_endpoint, if specified, is the maximum number of times that the repeated_pattern_expr pattern sub-expression becomes true. If the number is reached, the engine stops the repeated_pattern_expr pattern sub-expression.
In all cases, only at the time that the end_pattern_expr pattern sub-expression evaluates to true does the expression become true. If end_pattern_expr pattern sub-expression evaluates to false, then the expression evaluates to false.
An open-ended range specifies only a low endpoint and not a high endpoint.
Consider the following pattern which requires at least three A events to match:
[3:] A until B
In the pattern above, if a B event arrives before 3 A events occurred, the expression ends and evaluates to false.
A high-endpoint range specifies only a high endpoint and not a low endpoint.
In this sample pattern the engine will be looking for a maximum of 3 A events. The expression turns true as soon as a single B event arrives regardless of the number of A events received:
[:3] A until B
The next pattern matches when a C or D event arrives, regardless of the number of A or B events that occurred:
[:3] (a=A or b=B) until (c=C or d=D)
In the pattern above, if more then 3 A or B events arrive, the pattern stops looking for additional A or B events. The 'a' and 'b' tags retain only the first 3 (combined) matches among A and B events. The output event contains these tagged events as indexed properties.
A bounded range specifies a low endpoint and a high endpoint.
The next pattern matches after at least one A event arrives upon the arrival of a single B event:
[1:3] a=A until B
If a B event arrives before the first A event, then the pattern does not match. Only the first 3 A events are returned by the pattern.
The tags assigned to events in filter subexpressions within a repeat operator are available for use in filter expressions and also in any EPL clause.
This sample pattern matches 2 A events followed by a B event. Note the filter on B events: only a B event that has a value for the "beta" property that equals any of the "id" property values of the two A events is considered:
[2] A -> B(beta in (a[0].id, a[1].id))
The next EPL statement returns pairs of A events:
select a, a[0], a[0].id, a[1], a[1].id from pattern [ every [2] a=A ]
The select clause of the statement above showcases different ways of accessing tagged events:
The tag itself can be used to select an array of underlying events. For example, the 'a' expression above returns an array of underlying events of event type A.
The tag as an indexed property returns the underlying event at that index. For instance, the 'a[0]' expression returns the first underlying A event, or null if no such A event was matched by the repeat operator.
The tag as a nested, indexed property returns a property of the underlying event at that index. For example, the 'a[1].id' expression returns the 'id' property value of the second A event, or null if no such second A event was matched by the repeat operator.
You may not use indexed tags defined in the sub-expression to the repeat operator in the same subexpression. For example, in the following pattern the subexpression to the repeat operator is (a=A() -> b=B(id=a[0].id)) and the tag a cannot be used in its indexed form in the filter for event B:
// invalid every [2] (a=A() -> b=B(id=a[0].id))
You can use tags without an index:
// valid every [2] (a=A() -> b=B(id=a.id))
Similar to the Java && operator the and operator requires both nested pattern expressions to turn
true before the whole expression turns true (a join pattern).
This pattern matches when both an A event and a B event arrive, at the time the last of the two events arrive:
A and B
This pattern matches on any sequence of an A event followed by a B event and then a C event followed by a D event, or a C event followed by a D and an A event followed by a B event:
(A -> B) and (C -> D)
Note that in an and pattern expression it is not possible to correlate events based on event property values. For example, this is an invalid pattern:
// This is NOT valid a=A and B(id = a.id)
The above expression is invalid as it relies on the order of arrival of events, however in an and expression the order of events is not specified and events fulfill an and condition in any order. The above expression can be changed to use the followed-by operator:
// This is valid a=A -> B(id = a.id) // another example using 'and'... a=A -> (B(id = a.id) and C(id = a.id))
Consider a pattern that looks for the same event:
A and A
The pattern above fires when a single A event arrives. The first arriving A event triggers a state transition in both the left and the right hand side expression.
In order to match after two A events arrive in any order, there are two options to express this pattern. The followed-by operator is one option and the repeat operator is the second option, as the next two patterns show:
A -> A // ... or ... [2] A
Similar to the Java “||” operator the or operator requires either one of the expressions
to turn true before the whole expression turns true.
Look for either an A event or a B event. As always, A and B can itself be nested expressions as well.
A or B
Detect all stock ticks that are either above or below a threshold.
every (StockTick(symbol='IBM', price < 100) or StockTick(symbol='IBM', price > 105)
The not operator negates the truth value of an expression. Pattern expressions prefixed with not are automatically
defaulted to true upon start, and turn permanently false when the expression within turns true.
The not operator is generally used in conjunction with the and operator or subexpressions as the below examples show.
This pattern matches only when an A event is encountered followed by a B event but only if no C event was encountered before either an A event and a B event, counting from the time the pattern is started:
(A -> B) and not C
Assume we'd like to detect when an A event is followed by a D event, without any B or C events between the A and D events:
A -> (D and not (B or C))
It may help your understanding to discuss a pattern that uses the or operator and the not operator together:
a=A -> (b=B or not C)
In the pattern above, when an A event arrives then the engine starts the subexpression B or not C. As soon as the subexpression starts, the not operator turns to true. The or expression turns true and thus your listener receives an invocation providing the A event in the property 'a'. The subexpression does not end and continues listening for B and C events. Upon arrival of a B event your listener receives a second invocation. If instead a C event arrives, the not turns permanently false however that does not affect the or operator (but would end an and operator).
To test for absence of an event, use timer:interval together with and not operators. The sample statement reports each 10-second interval during which no A event occurred:
every (timer:interval(10 sec) and not A)
In many cases the not operator, when used alone, does not make sense. The following example is invalid and will log a warning when the engine is started:
// not a sensible pattern (not a=A) -> B(id=a.id)
The followed by -> operator specifies that first the left hand expression must turn true and only
then is the right hand expression evaluated for matching events.
Look for an A event and if encountered, look for a B event. As always, A and B can itself be nested event pattern expressions.
A -> B
This is a pattern that fires when 2 status events indicating an error occur one after the other.
StatusEvent(status='ERROR') -> StatusEvent(status='ERROR')
A pattern that takes all A events that are not followed by a B event within 5 minutes:
every A -> (timer:interval(5 min) and not B)
A pattern that takes all A events that are not preceded by B within 5 minutes:
every (timer:interval(5 min) and not B -> A)
The followed-by -> operator can optionally be provided with an expression that limits the number of sub-expression instances of the right-hand side pattern sub-expression.
The synopsis for the followed-by operator with limiting expression is:
lhs_expression -[limit_expression]> rhs_expression
Each time the lhs_expression pattern sub-expression turns true the pattern engine starts a new rhs_expression pattern sub-expression. The limit_expression returns an integer value that defines a maximum number of pattern sub-expression instances that can simultaneously be present for the same rhs_expression.
When the limit is reached the pattern engine issues a com.espertech.esper.client.hook.ConditionPatternSubexpressionMax notification object to any condition handlers registered with the engine as described in Section 14.12, “Condition Handling” and does not start a new pattern sub-expression instance for the right-hand side pattern sub-expression.
For example, consider the following pattern which returns for every A event the first B event that matches the id field value of the A event:
every a=A -> b=B(id = a.id)
In the above pattern, every time an A event arrives (lhs) the pattern engine starts a new pattern sub-expression (rhs) consisting of a filter for the first B event that has the same value for the id field as the A event.
In some cases your application may want to limit the number of right-hand side sub-expressions because of memory concerns or to reduce output. You may add a limit expression returning an integer value as part of the operator.
This example employs the followed-by operator with a limit expression to indicate that maximally 2 filters for B events (the right-hand side pattern sub-expression) may be active at the same time:
every a=A -[2]> b=B(id = a.id)
Note that the limit expression in the example above is not a limit per value of id field, but a limit counting all right-hand side pattern sub-expression instances that are managed by that followed-by sub-expression instance.
If your followed-by operator lists multiple sub-expressions with limits, each limit applies to the immediate right-hand side. For example, the pattern below limits the number of filters for B events to 2 and the number of filters for C events to 3:
every a=A -[2]> b=B(id = a.id) -[3]> c=C(id = a.id)
Esper allows setting a maximum number of pattern sub-expressions in the configuration, applicable to all followed-by operators of all statements.
If your application has patterns in multiple EPL statements and all such patterns should count towards a total number of pattern sub-expression counts, you may consider setting a maximum number of pattern sub-expression instances, engine-wide, via the configuration described in Section 15.4.15.1, “Followed-By Operator Maximum Subexpression Count”.
When the limit is reached the pattern engine issues a notification object to any condition handlers registered with the engine as described in Section 14.12, “Condition Handling”. Depending on your configuration the engine can prevent the start of a new pattern sub-expression instance for the right-hand side pattern sub-expression, until pattern sub-expression instances end or statements are stopped or destroyed.
The notification object issued to condition handlers is an instance of com.espertech.esper.client.hook.ConditionPatternEngineSubexpressionMax. The notification object contains information which statement triggered the limit and the pattern counts per statement for all statements.
For information on static and runtime configuration, please consult Section 15.4.15.1, “Followed-By Operator Maximum Subexpression Count”. The limit can be changed and disabled or enabled at runtime via the runtime configuration API.
Guards are where-conditions that control the lifecycle of subexpressions. Custom guard functions can also be used. The section Chapter 17, Integration and Extension outlines guard plug-in development in greater detail.
The pattern guard where-condition has no relationship to the EPL where clause that filters sets of events.
Take as an example the following pattern expression:
MyEvent where timer.within(10 sec)
In this pattern the timer:within guard controls the subexpression that is looking for MyEvent events. The guard terminates the subexpression looking for MyEvent events after 10 seconds after start of the pattern. Thus the pattern alerts only once when the first MyEvent event arrives within 10 seconds after start of the pattern.
The every keyword requires additional discussion since it also controls subexpression lifecycle. Let's add the every keyword to the example pattern:
every MyEvent where timer.within(10 sec)
The difference to the pattern without every is that each MyEvent event that arrives now starts a new subexpression, including a new guard, looking for a further MyEvent event. The result is that, when a MyEvent arrives within 10 seconds after pattern start, the pattern execution will look for the next MyEvent event to arrive within 10 seconds after the previous one.
By placing parentheses around the every keyword and its subexpression, we can have the every under the control of the guard:
(every MyEvent) where timer.within(10 sec)
In the pattern above, the guard terminates the subexpression looking for all MyEvent events after 10 seconds after start of the pattern. This pattern alerts for all MyEvent events arriving within 10 seconds after pattern start, and then stops.
Guards do not change the truth value of the subexpression of which the guard controls the lifecycle, and therefore do not cause a restart of the subexpression when used with the every operator. For example, the next pattern stops returning matches after 10 seconds unless a match occurred within 10 seconds after pattern start:
every ( (A and B) where timer.within(10 sec) )
The timer:within guard acts like a stopwatch.
If the associated pattern expression does not turn true within the specified time period it is stopped and permanently false.
The synopsis for timer:within is as follows:
timer:within(time_period_expression)The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds as a parameter. The interval expression may contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
This pattern fires if an A event arrives within 5 seconds after statement creation.
A where timer:within (5 seconds)
This pattern fires for all A events that arrive within 5 seconds. After 5 seconds, this pattern stops matching even if more A events arrive.
(every A) where timer:within (5 seconds)
This pattern matches for any one A or B event in the next 5 seconds.
( A or B ) where timer:within (5 sec)
This pattern matches for any 2 errors that happen 10 seconds within each other.
every (StatusEvent(status='ERROR') -> StatusEvent(status='ERROR') where timer:within (10 sec))
The following guards are equivalent:
timer:within(2 minutes 5 seconds) timer:within(125 sec) timer:within(125)
The timer:withinmax guard is similar to the timer:within guard and acts as a stopwatch that additionally has a counter that counts the number of matches. It ends the subexpression when either the stopwatch ends or the match counter maximum value is reached.
The synopsis for timer:withinmax is as follows:
timer:withinmax(time_period_expression, max_count_expression)
The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds.
The max_count_expression provides the maximum number of matches before the guard ends the subexpression.
Each parameter expression may also contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
This pattern fires for every A event that arrives within 5 seconds after statement creation but only up to the first two A events:
(every A) where timer:withinmax (5 seconds, 2)
If the result of the max_count_expression is 1, the guard ends the subexpression after the first match and indicates the first match.
This pattern fires for the first A event that arrives within 5 seconds after statement creation:
(every A) where timer:withinmax (5 seconds, 1)
If the result of the max_count_expression is zero, the guard ends the subexpression upon the first match and does no indicate any matches.
This example receives every A event followed by every B event (as each B event arrives) until the 5-second subexpression timer ends or X number of B events have arrived (assume X was declared as a variable):
every A -> (every B) where timer:withinmax (5 seconds, X)
The while guard is followed by an expression that the engine evaluates for every match reported by the guard pattern sub-expression. When the expression returns false the pattern sub-expression ends.
The synopsis for while is as follows:
while (guard_expression)The guard_expression is any expression that returns a boolean true or false. The expression may contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
Each time the subexpression indicates a match, the engine evaluates guard_expression and if true, passes the match and when false, ends the subexpression.
This pattern fires for every A event until an A event arrives that has a value of zero or less for its size property (assuming A events have an integer size property).
(every a=A) while (a.size > 0)
Note the parenthesis around the every subexpression. They ensure that, following precedence rules, the guard applies to the every operator as well.
The timer:within and timer:withinmax guards may be parameterized by an expression that contains one or more references to properties of prior events in the same pattern.
As a simple example, this pattern matches every A event followed by a B event that arrives within delta seconds after the A event:
every a=A -> b=B where timer:within (a.delta seconds)
Herein A event is assumed to have a delta property that provides the number of seconds to wait for B events. Each arriving A event may have a different value for delta
and the guard is therefore parameterized dynamically based on the prior A event received.
When multiple events accumulate, for example when using the match-until or repeat pattern elements, an index must be provided:
[2] a=A -> b=B where timer:within (a[0].delta + a[1].delta)
The above pattern matches after 2 A events arrive followed by a B event within a time interval after the A event that is defined by the sum of the delta properties of both A events.
You can combine guard expression by using parenthesis around each subexpression.
The below pattern matches for each A event while A events of size greater then zero arrive and only within the first 20 seconds:
((every a=A) while (a.size > 0)) where timer:within(20)
Filter atoms have been described in section Section 6.4, “Filter Expressions In Patterns”.
Observers observe time-based events for which the thread-of-control originates by the engine timer or external timer event. Custom observers can also be developed that observe timer events or other engine-external application events such as a file-exists check. The section Chapter 17, Integration and Extension outlines observer plug-in development in greater detail.
The timer:interval observer waits for the defined time before the truth value of the observer turns true.
The observer takes a time period (see Section 5.2.1, “Specifying Time Periods”) as a parameter, or an expression that returns the number of seconds.
The observer may be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value.
After an A event arrived wait 10 seconds then indicate that the pattern matches.
A -> timer:interval(10 seconds)
The pattern below fires every 20 seconds.
every timer:interval(20 sec)
The next example pattern fires for every A event that is not followed by a B event within 60 seconds after the A event arrived. The B event must have the same "id" property value as the A event.
every a=A -> (timer:interval(60 sec) and not B(id=a.id))
Consider the next example, which assumes that the A event has a property waittime:
every a=A -> (timer:interval(a.waittime + 2) and not B(id=a.id))
In the above pattern the logic waits for 2 seconds plus the number of seconds provided by the value of the waittime property of the A event.
The timer:at observer is similar in function to the Unix “crontab” command. At a specified time the
expression turns true. The at operator can also be made to pattern match at regular intervals by using an every operator
in front of the timer:at operator.
The syntax is: timer:at (minutes, hours, days of month, months, days of week [, seconds [, time zone]]).
The value for seconds and time zone is optional. Each element allows wildcard * values. Ranges can be specified
by means of lower bounds then a colon ‘:’ then the upper bound. The division operator */x can be used to
specify that every xth value is valid. Combinations of these operators can be used by placing these into square brackets ([]).
The timer:at observer may also be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value. The frequency division operator */x and parameters lists within brackets ([]) are an exception: they may only contain variables, substitution parameters or numeric values.
This expression pattern matches every 5 minutes past the hour.
every timer:at(5, *, *, *, *)
The below timer:at pattern matches every 15 minutes from 8am to 5:45pm (hours 8 to 17 at 0, 15, 30 and 45 minutes past the hour) on even numbered days of the month as well as on the
first day of the month.
timer:at (*/15, 8:17, [*/2, 1], *, *)
The below table outlines the fields, valid values and keywords available for each field:
Table 6.5. Crontab Fields
| Field Name | Mandatory? | Allowed Values | Additional Keywords |
|---|---|---|---|
| Minutes | yes | 0 - 59 | |
| Hours | yes | 0 - 23 | |
| Days Of Month | yes | 1 - 31 | last, weekday, lastweekday |
| Months | yes | 1 - 12 | |
| Days Of Week | yes | 0 (Sunday) - 6 (Saturday) | last |
| Seconds | no (required if specifying a time zone) | 0 - 59 | |
| Time Zone | no | any string (not validated, see TimeZone javadoc) |
The keyword last used in the days-of-month field means the last day of the month (current month). To specify the last day of another month, a value for the month field has to be provided. For example: timer:at(*, *, last,2,*) is the last day of February.
The last keyword in the day-of-week field by itself simply means Saturday. If used in the day-of-week field after another value, it means "the last xxx day of the month" - for example "5 last" means "the last Friday of the month".
So the last Friday of the current month will be: timer:at(*, *, *, *, 5 last). And the last Friday of June: timer:at(*, *, *, 6, 5 last).
The keyword weekday is used to specify the weekday (Monday-Friday) nearest the given day. Variant could include month like in: timer:at(*, *, 30 weekday, 9, *) which for year 2007 is Friday September 28th (no jump over month).
The keyword lastweekday is a combination of two parameters, the last and the weekday keywords. A typical example could be: timer:at(*, *, *, lastweekday, 9, *) which will define Friday September 28th (example year is 2007).
The time zone is a string-type value that specifies the time zone of the schedule. You must specify a value for seconds when specifying a time zone. Esper relies on the java.util.TimeZone to interpret the time zone value. Note that TimeZone does not validate time zone strings.
The following timer:at pattern matches at 5:00 pm Pacific Standard Time (PST):
timer:at (0, 17, *, *, *, *, 'PST')
Any expression may occur among the parameters. This example invokes a user-defined function computeHour to return an hour:
timer:at (0, computeHour(), *, *, *, *)
The following restrictions apply to crontab parameters:
It is not possible to specify both Days Of Month and Days Of Week.
When using timer:at with the every operator the crontab-like timer computes the next time at which the timer should fire based on the specification and the current time. When using every, the current time is the time the timer fired or the statement start time if the timer has not fired once.
For example, this pattern fires every 1 minute starting at 1:00pm and ending at 1:59pm, every day:
every timer:at(*, 13, *, *, *)
Assume the above statement gets started at 1:05pm and 20 seconds. In such case the above pattern fires every 1 minute starting at 1:06pm and ending at 1:59pm for that day and 1:00pm to 1:59pm every following day.
To get the pattern to fire only once at 1pm every day, explicitly specify the minute to start. The pattern below fires every day at 1:00pm:
every timer:at(0, 13, *, *, *)
By specifying a second resolution the timer can be made to fire every second, for instance:
every timer:at(*, *, *, *, *, *)