www.espertech.comDocumentation
Event patterns match when an event or multiple events occur that match the pattern's definition. Patterns can also be time-based.
Pattern expressions consist of pattern atoms and pattern operators:
Pattern atoms are the basic building blocks of patterns. Atoms are filter expressions, observers for time-based events and plug-in custom observers that observe external events not under the control of the engine.
Pattern operators control expression lifecycle and combine atoms logically or temporally.
The below table outlines the different pattern atoms available:
Table 7.1. Pattern Atoms
Pattern Atom | Example |
---|---|
Filter expressions specify an event to look for. | StockTick(symbol='ABC', price > 100) |
Time-based event observers specify time intervals or time schedules. | timer:interval(10 seconds) timer:at(*, 16, *, *, *) timer:schedule(....) |
Custom plug-in observers can add pattern language syntax for observing application-specific events. | myapplication:myobserver("http://someResource") |
There are 4 types of pattern operators:
Operators that control pattern sub-expression repetition: every
, every-distinct
, [num]
and until
Logical operators: and
, or
, not
Temporal operators that operate on event order: ->
(followed-by)
Guards are where-conditions that control the lifecycle of subexpressions. Examples are timer:within
, timer:withinmax
and while
-expression. Custom plug-in guards may also be used.
Pattern expressions can be nested arbitrarily deep by including the nested expression(s) in ()
round parenthesis.
Underlying the pattern matching is a hierarchical finite state machine and behavior tree that allocates, transitions and destroys branch and leaf nodes of state based on arriving events and based on time advancing. A single event or advancing time may cause a reaction in multiple parts of your active pattern state. Patterns are stateful as the engine maintains pattern state. There is a walkthrough of how a sample pattern behaves in Section 7.7, “Event Pattern Walkthrough”.
This is an example pattern expression that matches on every ServiceMeasurement
events in which the
value of the latency
event property is over 20 seconds, and on every ServiceMeasurement
event in which the
success
property is false. Either one or the other condition must be true for this pattern to match.
every spike=ServiceMeasurement(latency>20000) or every error=ServiceMeasurement(success=false)
In the example above, the pattern expression or
operator indicates that the pattern should fire when either of the
filter expressions fire. The every
operator indicates to fire for every matching event and not just the first
matching event.
The left hand of the or
operator filters for events with a high latency value.
The right hand of the or
operator filters for events with error status.
Filter expressions are explained in Section 7.4, “Filter Expressions in Patterns”.
The example above assigned the tags spike
and error
to the events in the pattern. The tags are important since the
engine only places tagged events into the output event(s) that a pattern generates, and that the engine supplies to listeners of the pattern statement. The tags can
further be selected in the select-clause of an EPL statement as discussed in Section 5.4.2, “Pattern-Based Event Streams”.
Patterns can also contain comments within the pattern as outlined in Section 5.2.2, “Using Comments”.
Pattern statements are created via the EPAdministrator
interface. The EPAdministrator
interface allows to create pattern statements in two ways:
Pattern statements that want to make use of the EPL select
clause or any other EPL constructs use the createEPL
method to create a statement that specifies one or more pattern expressions. EPL statements that use patterns are described in more detail in Section 5.4.2, “Pattern-Based Event Streams”. Use the syntax as shown in below example.
EPAdministrator admin = EPServiceProviderManager.getDefaultProvider().getEPAdministrator(); String eventName = ServiceMeasurement.class.getName(); EPStatement myTrigger = admin.createEPL("select * from pattern [" + "every spike=" + eventName + "(latency>20000) or every error=" + eventName + "(success=false)]");
Pattern statements that do not need to make use of the EPL select
clause or any other EPL constructs can use the createPattern
method, as in below example.
EPStatement myTrigger = admin.createPattern( "every spike=" + eventName + "(latency>20000) or every error=" + eventName + "(success=false)");
A pattern may appear anywhere in the from
clause of an EPL statement including joins and subqueries. Patterns may therefore be used in combination with the where
clause, group by
clause, having
clause as well as output rate limiting and insert into
.
In addition, you may use data window with a pattern. A data window declared for a pattern only serves to retain pattern matches, for use in joins or for iterating via the iterator API. A data window declared onto a pattern does not limit, cancel, remove or delete intermediate pattern matches of the pattern when pattern matches leave the data window.
This example statement demonstrates the idea by selecting a total price per customer over pairs of events (ServiceOrder followed by a ProductOrder event for the same customer id within 1 minute), occurring in the last 2 hours, in which the sum of price is greater than 100, and using a where
clause to filter on name:
select a.custId, sum(a.price + b.price) from pattern [every a=ServiceOrder -> b=ProductOrder(custId = a.custId) where timer:within(1 min)]#time(2 hour) where a.name in ('Repair', b.name) group by a.custId having sum(a.price + b.price) > 100
When a pattern fires it publishes one or more events to any listeners to the pattern statement.
The listener interface is the com.espertech.esper.client.UpdateListener
interface.
The example below shows an anonymous implementation of the com.espertech.esper.client.UpdateListener
interface.
The example adds the anonymous listener implementation to the myPattern
statement created earlier.
The listener code simply extracts the underlying event class.
myPattern.addListener(new UpdateListener() { public void update(EventBean[] newEvents, EventBean[] oldEvents) { ServiceMeasurement spike = (ServiceMeasurement) newEvents[0].get("spike"); ServiceMeasurement error = (ServiceMeasurement) newEvents[0].get("error"); ... // either spike or error can be null, depending on which occurred ... // add more logic here } });
Listeners receive an array of EventBean
instances in the newEvents
parameter.
There is one EventBean
instance passed to the listener for each combination of events that matches
the pattern expression. At least one EventBean
instance is always passed to the listener.
The properties of each EventBean
instance contain the underlying events that caused the
pattern to fire, if events have been named in the filter expression via the name=eventType
syntax.
The property name is thus the name supplied in the pattern expression, while the property type is the type of the underlying class,
in this example ServiceMeasurement
.
Data can also be obtained from pattern statements via the safeIterator()
and iterator()
methods on EPStatement
(the pull API)
If the pattern had fired at least once and the @IterableUnbound
annotation is declared for the statement, then the iterator returns the last event for which it fired.
The hasNext()
method can then be used to determine if the pattern had fired.
if (myPattern.iterator().hasNext()) { ServiceMeasurement event = (ServiceMeasurement) statement.iterator().next().get("alert"); ... // some more code here to process the event } else { ... // no matching events at this time }
Further, if a data window is defined onto a pattern, the iterator returns the pattern matches according to the data window expiry policy.
This pattern specifies a length window of 10 elements that retains the last 10 matches of A and B events, for use via iterator or for use in a join or subquery:
select * from pattern [every (A or B)]#length(10)
While the pattern compiler analyzes your pattern and verifies its integrity, it may not detect certain pattern errors that may occur at runtime. Sections of this pattern documentation point out common cases where the pattern engine will log a runtime error. We recommend turning on the log warning level at project development time to inspect and report on warnings logged. If a statement name is assigned to a statement then the statement name is logged as well.
Any given event can contribute to multiple matches.
For example, consider the following pattern:
every a=A -> B
Given this sequence of events:
A1 A2 B1
When event B1 arrives the pattern matches for both the combination {A1, B1} and the combination {A2, B1}. The engine indicates both matches to the listener or subscriber by delivering an array containing both matches in a single listener or subscriber invocation.
Use the @SuppressOverlappingMatches
pattern-level annotation to instruct the engine to discard all but the first match among multiple overlapping matches.
The same example with the pattern-level annotation is:
select * from pattern @SuppressOverlappingMatches [every a=A -> b=B]
When event B1 arrives the pattern outputs only the first combination that matches, namely the combination {A1, B1}. The engine discards the second combination ({A2, B1}) that matches as it detects that event B1 overlaps between the first and the second match.
Partially-completed patterns are incomplete matches that are not yet indicated by the engine because the complete pattern condition is not satisfied. Any given event can be part of multiple partially-completed patterns.
For example, consider the following pattern:
every a=A -> B and C(id=a.id)
Given this sequence of events:
A1{id='id1'} A2{id='id2'} B1
According to the sequence above there are no matches. The pattern is partially completed waiting for C events. The combination {A1, B1} is waiting for a C{id='id1'} event before the pattern match is complete for that combination. The combination {A2, B1} is waiting for a C{id='id2'} event before the pattern match is complete for that combination.
Assuming event C1{id='id1') arrives the pattern outputs the combination {A1, B1, C1}. Assuming event C2{id='id2') arrives the pattern outputs the combination {A2, B1, C2}. Note that event B1 is part of both partially-completed patterns.
Use the @DiscardPartialsOnMatch
pattern-level annotation to instruct the engine that when any matches occur to discard partially completed patterns that overlap in terms of the events that
make up the match (or matches if there are multiple matches).
The same example using the @DiscardPartialsOnMatch
pattern-level annotation is:
select * from pattern @DiscardPartialsOnMatch [every a=A -> B and C(id=a.id)]
When event C1{id='id1') arrives the pattern outputs the match combination {A1, B1, C1}. Upon indication of the match the engine discards all partially-completed patterns that refer to either of the A1, B1 and C1 events. Since event B1 is part of a partially-completed pattern waiting for C{id='id2'}, the engine discards that partially-completed pattern. Therefore when C2{id='id2'} arrives the engine outputs no matches.
When specifying both @DiscardPartialsOnMatch
and @SuppressOverlappingMatches
the engine discards the partially-completed patterns that overlap all matches including suppressed matches.
not
operator can change truth value to permanently false.
The change in truth-value does not get reversed when a match removes the event that caused the not
operator to become permanently false.
The operators at the top of this table take precedence over operators lower on the table.
Table 7.2. Pattern Operator Precedence
Precedence | Operator | Description | Example |
---|---|---|---|
1 | guard postfix | where timer:within and while (expression) (incl. withinmax and plug-in pattern guard) | MyEvent where timer:within(1 sec) a=MyEvent while (a.price between 1 and 10) |
2 | unary | every, not, every-distinct | every MyEvent timer:interval(5 min) and not MyEvent |
3 | repeat | [num] , until | [5] MyEvent [1..3] MyEvent until MyOtherEvent |
4 | and | and | every (MyEvent and MyOtherEvent) |
5 | or | or | every (MyEvent or MyOtherEvent) |
6 | followed-by | -> | every (MyEvent -> MyOtherEvent) |
If you are not sure about the precedence, please consider placing parenthesis ()
around your subexpressions. Parenthesis can also help make
expressions easier to read and understand.
The following table outlines sample equivalent expressions, with and without the use of parenthesis for subexpressions.
Table 7.3. Equivalent Pattern Expressions
Expression | Equivalent | Reason |
---|---|---|
every A or B | (every A) or B | The every operator has higher precedence then the or operator. |
every A -> B or C | (every A) -> (B or C) | The or operator has higher precedence then the followed-by operator. |
A -> B or B -> A | A -> (B or B) -> A | The or operator has higher precedence then the followed-by operator,
specify as (A -> B) or (B -> A) instead.
|
A and B or C | (A and B) or C | The and operator has higher precedence then the or operator. |
A -> B until C -> D | A -> (B until C) -> D | The until operator has higher precedence then the followed-by operator. |
[5] A or B | ([5] A) or B | The [num] repeat operator has higher precedence then the or operator. |
every A where timer:within(10) | every (A where timer:within(10)) | The where postfix has higher precedence then the every operator. |
The simplest form of filter is a filter for events of a given type without any conditions on the event property values. This filter matches any event of that type regardless of the event's properties. The example below is such a filter. Note that this event pattern would stop firing as soon as the first RfidEvent is encountered.
com.mypackage.myevents.RfidEvent
To make the event pattern fire for every RfidEvent and not just the first event, use the every
keyword.
every com.mypackage.myevents.RfidEvent
The example above specifies the fully-qualified Java class name as the event type. Via configuration, the event pattern above can be simplified by using the name that has been defined for the event type.
every RfidEvent
Interfaces and superclasses are also supported as event types. In the below example IRfidReadable
is an interface class, and the statement matches any event that implements this interface:
every org.myorg.rfid.IRfidReadable
The filtering criteria to filter for events with certain event property values are placed within parenthesis after the event type name:
RfidEvent(category="Perishable")
All expressions can be used in filters, including static method invocations that return a boolean value:
RfidEvent(com.mycompany.MyRFIDLib.isInRange(x, y) or (x<0 and y < 0))
Filter expressions can be separated via a single comma ',
'. The comma represents a logical AND between expressions:
RfidEvent(zone=1, category=10) ...is equivalent to... RfidEvent(zone=1 and category=10)
For more information on filters please see Section 5.4.1, “Filter-Based Event Streams”. Contained-event selection on filters in patterns is further described in Section 5.19, “Contained-Event Selection”.
Filter criteria can also refer to events matching prior named events in the same expression. Below pattern is an example in which the pattern matches once for every RfidEvent that is preceded by an RfidEvent with the same asset id.
every e1=RfidEvent -> e2=RfidEvent(assetId=e1.assetId)
The syntax shown above allows filter criteria to reference prior results by specifying the event name tag of the prior event, and the event property name. The tag names in the above example were e1
and e2
. This syntax can be used in all filter operators or expressions including ranges and the in
set-of-values check:
every e1=RfidEvent -> e2=RfidEvent(MyLib.isInRadius(e1.x, e1.y, x, y) and zone in (1, e1.zone))
An arriving event changes the truth value of all expressions that look for the event. Consider the pattern as follows:
every (RfidEvent(zone > 1) and RfidEvent(zone < 10))
The pattern above is satisfied as soon as only one event with zone in the interval [2, 9] is received.
A detailed description of how filters become active and are indexed engine-wide is provided at Section 2.18.2.2, “Filter Index Pattern Example”.
An expression such as a=A -> B(id=a.id)
(A followed-by B with the same id as A) is not just a state change, in fact the engine registers new B-filter instances dynamically and in engine-wide shared filter indexes.
This means that while such a pattern seems to be slow if you are sending A-events, the engine can filter, match or discard B-events very fast as for B-events it only needs to perform a lookup in filter indexes.
If you are looking for best performance and don't expect to need filter indexes, or if you compare to another technology that doesn't have the concept of filter indexes, please use match-recognize instead.
The engine analyzes all filter expressions within a pattern and determines the filter indexes to use or to create. Indexing filter values to match event properties of incoming events enables the engine to match incoming events faster to pattern subexpressions.
More information on filter indexes in general can be found at Section 2.18.2, “Filter Indexes”.
More information on the operators relevant to filter indexes can be found at Section 5.4.1.2, “Specifying Filter Criteria”.
An arriving event applies to all filter expressions for which the event matches. In other words, an arriving event is not consumed by any specify filter expression(s) but applies to all active filter expressions of all pattern sub-expressions.
You may provide the @consume
annotation as part of a filter expression to control consumption of an arriving event. If an arriving event matches the filter expression marked with @consume
it
is no longer available to other filter expressions of the same pattern that also match the arriving event.
The @consume
can include a level number in parenthesis. A higher level number consumes the event first. The default level number is 1. Multiple filter expressions with the same level number for @consume
all match the event.
Consider the next sample pattern:
a=RfidEvent(zone='Z1') and b=RfidEvent(assetId='0001')
This pattern fires when a single RfidEvent event arrives that has zone 'Z1' and assetId '0001'. The pattern also matches when two RfidEvent events arrive, in any order, wherein one has zone 'Z1' and the other has assetId '0001'.
Mark a filter expression with @consume
to indicate that if an arriving event matches multiple filter expressions that the engine prefers the marked filter expression and does not match any other filter expression.
This updated pattern statement uses @consume
to indicate that a match against zone is preferred:
a=RfidEvent(zone='Z1')@consume and b=RfidEvent(assetId='0001')
This pattern no longer fires when a single RfidEvent arrives that has zone 'Z1' and assetId '0001', because when the first filter expression matches the pattern engine consumes the event. The pattern only matches when two RfidEvent events arrive in any order. One event must have zone 'Z1' and the other event must have a zone other than 'Z1' and an assetId '0001'.
The next sample pattern provides a level number for each @consume
:
a=RfidEvent(zone='Z1')@consume(2) or b=RfidEvent(assetId='0001')@consume(1) or c=RfidEvent(category='perishable'))
The pattern fires when an RfidEvent arrives with zone 'Z1'. In this case the output event populates property 'a' but not properties 'b' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other than 'Z1' and an asset id of '0001'. In this case the output event populates property 'b' but not properties 'a' and 'c'. The pattern also fires when an RfidEvent arrives with a zone other than 'Z1' and an asset id other than '0001' and a category of 'perishable'. In this case the output event populates property 'c' but not properties 'a' and 'b'.
When your filter expression provides the name of a named window then the filter expression matches each time an event is inserted into the named window that matches the filter conditions.
For example, assume a named window that holds the last order event per order id:
create window LastOrderWindow#unique(orderId) as OrderEvent
Assume that all order events are inserted into the named window using insert-into:
insert into LastOrderWindow select * from OrderEvent
This sample pattern fires 10 seconds after an order event with a price greater then 100 was inserted:
select * from pattern [every o=LastOrderWindow(price >= 100) -> timer:interval(10 sec)]
The pattern above fires only for events inserted-into the LastOrderWindow
named window and does not fire when an order event was updated using on-update or merged using on-merge.
If your application would like to have the pattern fire for any change to the named window events including updates and merges, you must select from the named window as follows:
insert into OrderWindowChangeStream select * from LastOrderWindow
select * from pattern [every o=OrderWindowChangeStream(price >= 100) -> timer:interval(10 sec)]
A table cannot be listed as part of a pattern filter, however any filter EPL expressions can have tables access expressions and subqueries against tables.
Assuming that MyTable
is a table, the following is not allowed:
// not allowed select * from pattern [every MyTable -> timer:interval(10 sec)]
The every
operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every
keyword evaluates to true or false.
Without the every
operator the pattern sub-expression stops when the pattern sub-expression evaluates to true or false.
As a side note, please be aware that a single invocation to the UpdateListener
interface may deliver multiple events in one invocation, since the interface accepts an array of values.
Thus the every
operator works like a factory for the pattern sub-expression contained within. When the pattern sub-expression within it fires and thus quits checking for events, the every
causes the start of a new pattern sub-expression listening for more occurrences of the same
event or set of events.
Every time a pattern sub-expression within an every
operator turns true the engine starts a new active subexpression looking
for more event(s) or timing conditions that match the pattern sub-expression. If the every
operator is not specified for a subexpression,
the subexpression stops after the first match was found.
This pattern fires when encountering an A event and then stops looking.
A
This pattern keeps firing when encountering A events, and doesn't stop looking.
every A
When using every
operator with the ->
followed-by operator, each time the every
operator restarts it also starts a new subexpression instance looking for events in the followed-by subexpression.
Let's consider an example event sequence as follows.
A1 B1 C1 B2 A2 D1 A3 B3 E1 A4 F1 B4
Table 7.4. Every
Operator Examples
Example | Description |
---|---|
every ( A -> B ) |
Detect an A event followed by a B event. At the time when B occurs the pattern matches, then the pattern matcher restarts and looks for the next A event.
|
every A -> B |
The pattern fires for every A event followed by a B event.
|
A -> every B |
The pattern fires for an A event followed by every B event.
|
every A -> every B |
The pattern fires for every A event followed by every B event.
|
The examples show that it is possible that a pattern fires for multiple combinations of events that match a pattern expression.
Each combination is posted as an EventBean
instance to the update
method in the UpdateListener
implementation.
Let's consider the every
operator in conjunction with a subexpression that matches 3 events that follow each other:
every (A -> B -> C)
The pattern first looks for A events. When an A event arrives, it looks for a B event. After the B event arrives, the pattern looks for a C event. Finally, when the C event arrives the pattern fires. The engine then starts looking for an A event again.
Assume that between the B event and the C event a second A2 event arrives. The pattern would ignore the A2 event entirely since it's then looking for a C event.
As observed in the prior example, the every
operator restarts the subexpression A -> B -> C
only when the subexpression fires.
In the next statement the every
operator applies only to the A event, not the whole subexpression:
every A -> B -> C
This pattern now matches for each A event that is followed by a B event and then a C event, regardless of when the A event arrives. Note that for each A event that arrives the pattern engine starts a new subexpression looking for a B event and then a C event, outputting each combination of matching events.
A pattern that only has the every
operator and a single filter expression is equivalent to selecting the same filter in the from
clause:
select * from StockTickEvent(symbol='GE') // Prefer this // ... equivalent to ... select * from pattern[every StockTickEvent(symbol='GE')]
As the introduction of the every
operator states, the operator starts new subexpression instances and can cause multiple matches to occur for a single arriving event.
New subexpressions also take a very small amount of system resources and thereby your application should carefully consider when subexpressions must end when designing patterns. Use the timer:within
construct and the and not
constructs to end active subexpressions. The data window onto a pattern stream does not serve to limit pattern sub-expression lifetime.
Lets look at a concrete example. Consider the following sequence of events arriving:
A1 A2 B1
This pattern matches on arrival of B1 and outputs two events (an array of length 2 if using a listener). The two events are the combinations {A1, B1} and {A2, B1}:
every a=A -> b=B
The and not
operators are used to end an active subexpression.
The next pattern matches on arrival of B1 and outputs only the last A event which is the combination {A2, B1}:
every a=A -> (b=B and not A)
The and not
operators cause the subexpression looking for {A1, B?} to end when A2 arrives.
Similarly, in the pattern below the engine starts a new subexpression looking for a B event every 1 second. After 5 seconds there are 5 subexpressions active looking for a B event and 5 matches occur at once if a B event arrives after 5 seconds.
every timer:interval(1 sec) -> b=B
Again the and not
operators can end subexpressions that are not intended to match any longer:
every timer:interval(1 sec) -> (b=B and not timer:interval(1 sec)) // equivalent to every timer:interval(1 sec) -> (b=B where timer:within(1 sec))
This example considers a generic pattern in which the pattern must match for each A event followed by a B event and followed by a C event, in which both the B event and the C event must arrive within 1 hour of the A event. The first approach to the pattern is as follows:
every A -> (B -> C) where timer:within(1 hour)
Consider the following sequence of events arriving:
A1 A2 B1 C1 B2 C2
First, the pattern as above never stops looking for A events since the every
operator instructs the pattern to keep looking for A events.
When A1 arrives, the pattern starts a new subexpression that keeps A1 in memory and looks for any B event. At the same time, it also keeps looking for more A events.
When A2 arrives, the pattern starts a new subexpression that keeps A2 in memory and looks for any B event. At the same time, it also keeps looking for more A events.
After the arrival of A2, there are 3 subexpressions active:
The first active subexpression with A1 in memory, looking for any B event.
The second active subexpression with A2 in memory, looking for any B event.
A third active subexpression, looking for the next A event.
In the pattern above, there is a 1-hour lifetime for subexpressions looking for B and C events. Thus, if no B and no C event arrive within 1 hour after A1, the first subexpression goes away. If no B and no C event arrive within 1 hour after A2, the second subexpression goes away. The third subexpression however stays around looking for more A events.
The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and for combination {A2, B1, C1}, provided that B1 and C1 arrive within an hour of A1 and A2.
You may now ask how to match on {A1, B1, C1} and {A2, B2, C2} instead, since you may need to correlate on a given property.
The pattern as discussed above matches every A event followed by the first B event followed by the next C event, and doesn't specifically qualify the B or C events to look for based on the A event. To look for specific B and C events in relation to a given A event, specify correlating properties of the A event, for example:
every a=A -> (B(id=a.id) -> C(id=a.id)) where timer:within(1 hour)
The pattern as shown above thus matches on arrival of C1 for combination {A1, B1, C1} and on arrival of C2 for combination {A2, B2, C2}.
This example looks at temperature sensor events named Sample. The pattern detects when 3 sensor events indicate a temperature of more then 50 degrees uninterrupted within 90 seconds of the first event, considering events for the same sensor only.
every sample=Sample(temp > 50) -> ( (Sample(sensor=sample.sensor, temp > 50) and not Sample(sensor=sample.sensor, temp <= 50)) -> (Sample(sensor=sample.sensor, temp > 50) and not Sample(sensor=sample.sensor, temp <= 50)) ) where timer:within(90 seconds))
The pattern starts a new subexpression in the round braces after the first followed-by operator for each time a sensor indicated more then 50 degrees. Each subexpression then lives a maximum of 90 seconds. Each subexpression ends if a temperature of 50 degress or less is encountered for the same sensor. Only if 3 temperature events in a row indicate more then 50 degrees, and within 90 seconds of the first event, and for the same sensor, does this pattern fire.
Similar to the every
operator in most aspects, the every-distinct
operator indicates that the pattern sub-expression should restart when the subexpression qualified by the every-distinct
keyword evaluates to true or false. In addition, the every-distinct
eliminates duplicate results received from an active subexpression according to its distinct-value expressions.
The synopsis for the every-distinct
pattern operator is:
every-distinct(distinct_value_expr [, distinct_value_exp[...][, expiry_time_period])
Within parenthesis are one or more distinct_value_expr expressions that return the values by which to remove duplicates.
You may optionally specify an expiry_time_period time period. If present, the pattern engine expires and removes distinct key values that are older then the time period, removing their associated memory and allowing such distinct values to match again. When your distinct value expressions return an unlimited number of values, for example when your distinct value is a timestamp or auto-increment column, you should always specify an expiry time period.
When specifying properties in the distinct-value expression list, you must ensure that the event types providing properties are tagged. Only properties of event types within filter expressions that are sub-expressions to the every-distinct
may be specified.
For example, this pattern keeps firing for every A event with a distinct value for its aprop
property:
every-distinct(a.aprop) a=A
Note that the pattern above assigns the a
tag to the A event and uses a.prop
to identify the prop
property as a value of the a
event A.
A pattern that returns the first Sample event for each sensor, assuming sensor is a field that returns a unique id identifying the sensor that originated the Sample event, is:
every-distinct(s.sensor) s=Sample
The next pattern looks for pairs of A and B events and returns only the first pair for each combination of aprop
of an A event and bprop
of a B event:
every-distinct(a.aprop, b.bprop) (a=A and b=B)
The following pattern looks for A events followed by B events for which the value of the aprop
of an A event is the same value of the bprop
of a B event but only for each distinct value of aprop
of an A event:
every-distinct(a.aprop) a=A -> b=B(bprop = a.aprop)
When specifying properties as part of distinct-value expressions, properties must be available from tagged event types in sub-expressions to the every-distinct
.
The following patterns are not valid:
// Invalid: event type in filter not tagged every-distinct(aprop) A // Invalid: property not from a sub-expression of every-distinct a=A -> every-distinct(a.aprop) b=B
When an active subexpression to every-distinct
becomes permanently false, the distinct-values seen from the active subexpression are removed and the sub-expression within is restarted.
For example, the below pattern detects each A event distinct by the value of aprop
.
every-distinct(a.aprop) (a=A and not B)
In the pattern above, when a B event arrives, the subexpression becomes permanently false and is restarted anew, detecting each A event distinct by the value of aprop
without considering prior values.
When your distinct key is a timestamp or other non-unique property, specify an expiry time period.
The following example returns every distinct A event according to the timestamp property on the A event, retaining each timestamp value for 10 seconds:
every-distinct(a.timestamp, 10 seconds) a=A
In the example above, if for a given A event and its timestamp value the same timestamp value occurs again for another A event before 10 seconds passed, the A event is not a match. If 10 seconds passed the pattern indicates a second match.
You may not use every-distinct with a timer-within guard to expire keys: The expiry time notation as above is the recommended means to expire keys.
// This is not the same as above; It does not expire transaction ids and is not recommended every-distinct(a.timestamp) a=A where timer:within(10 sec)
The repeat operator fires when a pattern sub-expression evaluates to true a given number of times. The synopsis is as follows:
[match_count] repeating_subexpr
The repeat operator is very similar to the every
operator in that it restarts the repeating_subexpr pattern sub-expression up to a given number of times.
match_count is a positive number that specifies how often the repeating_subexpr pattern sub-expression must evaluate to true before the repeat expression itself evaluates to true, after which the engine may indicate a match.
For example, this pattern fires when the last of five A events arrives:
[5] A
Parenthesis must be used for nested pattern sub-expressions. This pattern fires when the last of a total of any five A or B events arrives:
[5] (A or B)
Without parenthesis the pattern semantics change, according to the operator precedence described earlier. This pattern fires when the last of a total of five A events arrives or a single B event arrives, whichever happens first:
[5] A or B
Tags can be used to name events in filter expression of pattern sub-expressions. The next pattern looks for an A event followed by a B event, and a second A event followed by a second B event. The output event provides indexed and array properties of the same name:
[2] (a=A -> b=B)
Using tags with repeat is further described in Section 7.5.4.6, “Tags and the Repeat Operator”.
Consider the following pattern that demonstrates the behavior when a pattern sub-expression becomes permanently false:
[2] (a=A and not C)
In the case where a C event arrives before 2 A events arrive, the pattern above becomes permanently false.
Lets add an every
operator to restart the pattern and thus keep matching for all pairs of A events that arrive without a C event in between each pair:
every [2] (a=A and not C)
Since pattern matches return multiple A events, your select clause should use tag a
as an array, for example:
select a[0].id, a[1].id from pattern [every [2] (a=A and not C)]
The repeat until
operator provides additional control over repeated matching.
The repeat until operator takes an optional range, a pattern sub-expression to repeat, the until
keyword and a second pattern sub-expression that ends the repetition. The synopsis is as follows:
[range] repeated_pattern_expr until end_pattern_expr
Without a range, the engine matches the repeated_pattern_expr pattern sub-expression until the end_pattern_expr evaluates to true, at which time the expression turns true.
An optional range can be used to indicate the minimum number of times that the repeated_pattern_expr pattern sub-expression must become true.
The optional range can also specify a maximum number of times that repeated_pattern_expr pattern sub-expression evaluates to true and retains tagged events. When this number is reached, the engine stops the repeated_pattern_expr pattern sub-expression.
The until
keyword is always required when specifying a range and is not required if specifying a fixed number of repeat as discussed in the section before.
In the unbound repeat, without a range, the engine matches the repeated_pattern_expr pattern sub-expression until the end_pattern_expr evaluates to true, at which time the expression turns true. The synopsis is:
repeated_pattern_expr until end_pattern_expr
This is a pattern that keeps looking for A events until a B event arrives:
A until B
Nested pattern sub-expressions must be placed in parenthesis since the until
operator has precedence over most operators. This example collects all A or B events for 10 seconds and places events received in indexed properties 'a' and 'b':
(a=A or b=B) until timer:interval(10 sec)
The synopsis for the optional range qualifier is:
[ [low_endpoint] : [high_endpoint] ]
low_endpoint is an optional number that appears on the left of a colon (:), after which follows an optional high_endpoint number.
A range thus consists of a low_endpoint and a high_endpoint in square brackets and separated by a colon (:) characters. Both endpoint values are optional but either one or both must be supplied. The low_endpoint can be omitted to denote a range that starts at zero. The high_endpoint can be omitted to denote an open-ended range.
Some examples for valid ranges might be:
[3 : 10] [:3] // range starts at zero [2:] // open-ended range
The low_endpoint, if specified, defines the minimum number of times that the repeated_pattern_expr pattern sub-expression must become true in order for the expression to become true.
The high_endpoint, if specified, is the maximum number of times that the repeated_pattern_expr pattern sub-expression becomes true. If the number is reached, the engine stops the repeated_pattern_expr pattern sub-expression.
In all cases, only at the time that the end_pattern_expr pattern sub-expression evaluates to true does the expression become true. If end_pattern_expr pattern sub-expression evaluates to false, then the expression evaluates to false.
An open-ended range specifies only a low endpoint and not a high endpoint.
Consider the following pattern which requires at least three A events to match:
[3:] A until B
In the pattern above, if a B event arrives before 3 A events occurred, the expression ends and evaluates to false.
A high-endpoint range specifies only a high endpoint and not a low endpoint.
In this sample pattern the engine will be looking for a maximum of 3 A events. The expression turns true as soon as a single B event arrives regardless of the number of A events received:
[:3] A until B
The next pattern matches when a C or D event arrives, regardless of the number of A or B events that occurred:
[:3] (a=A or b=B) until (c=C or d=D)
In the pattern above, if more then 3 A or B events arrive, the pattern stops looking for additional A or B events. The 'a' and 'b' tags retain only the first 3 (combined) matches among A and B events. The output event contains these tagged events as indexed properties.
A bounded range specifies a low endpoint and a high endpoint.
The next pattern matches after at least one A event arrives upon the arrival of a single B event:
[1:3] a=A until B
If a B event arrives before the first A event, then the pattern does not match. Only the first 3 A events are returned by the pattern.
The tags assigned to events in filter subexpressions within a repeat operator are available for use in filter expressions and also in any EPL clause.
This sample pattern matches 2 A events followed by a B event. Note the filter on B events: only a B event that has a value for the "beta" property that equals any of the "id" property values of the two A events is considered:
[2] A -> B(beta in (a[0].id, a[1].id))
The next EPL statement returns pairs of A events:
select a, a[0], a[0].id, a[1], a[1].id from pattern [ every [2] a=A ]
The select
clause of the statement above showcases different ways of accessing tagged events:
The tag itself can be used to select an array of underlying events. For example, the 'a' expression above returns an array of underlying events of event type A.
The tag as an indexed property returns the underlying event at that index. For instance, the 'a[0]' expression returns the first underlying A event, or null if no such A event was matched by the repeat operator.
The tag as a nested, indexed property returns a property of the underlying event at that index. For example, the 'a[1].id' expression returns the 'id' property value of the second A event, or null if no such second A event was matched by the repeat operator.
You may not use indexed tags defined in the sub-expression to the repeat operator in the same subexpression. For example, in the following pattern the subexpression to the repeat operator is (a=A() -> b=B(id=a[0].id))
and the tag a
cannot be used in its indexed form in the filter for event B:
// invalid every [2] (a=A() -> b=B(id=a[0].id))
You can use tags without an index:
// valid every [2] (a=A() -> b=B(id=a.id))
Similar to the Java && operator the and
operator requires both nested pattern expressions to turn
true before the whole expression turns true (a join pattern).
This pattern matches when both an A event and a B event arrive, at the time the last of the two events arrive:
A and B
This pattern matches on any sequence of an A event followed by a B event and then a C event followed by a D event, or a C event followed by a D and an A event followed by a B event:
(A -> B) and (C -> D)
Note that in an and
pattern expression it is not possible to correlate events based on event property values. For example, this is an invalid pattern:
// This is NOT valid a=A and B(id = a.id)
The above expression is invalid as it relies on the order of arrival of events, however in an and
expression the order of events is not specified and events fulfill an and
condition in any order. The above expression can be changed to use the followed-by operator:
// This is valid a=A -> B(id = a.id) // another example using 'and'... a=A -> (B(id = a.id) and C(id = a.id))
Consider a pattern that looks for the same event:
A and A
The pattern above fires when a single A event arrives. The first arriving A event triggers a state transition in both the left and the right hand side expression.
In order to match after two A events arrive in any order, there are two options to express this pattern. The followed-by operator is one option and the repeat operator is the second option, as the next two patterns show:
A -> A // ... or ... [2] A
Similar to the Java “||” operator the or
operator requires either one of the expressions
to turn true before the whole expression turns true.
Look for either an A event or a B event. As always, A and B can itself be nested expressions as well.
A or B
The next EPL outputs all A and B events:
every A or every B
Elaborating further, the expression every A or every B
is equivalent to every (A or B)
.
Prefer every A or every B
as the every
keyword lets the engine know that filters for A and B can remain active.
Consider the expression every A or every timer:interval(10 sec)
which is not equivalent to every (A or timer:interval(10 sec))
.
This is because in the latter expression when an A event arrives the interval restarts.
The not
operator negates the truth value of an expression. Pattern expressions prefixed with not
are automatically
defaulted to true upon start, and turn permanently false when the expression within turns true.
The not
operator is generally used in conjunction with the and
operator or subexpressions as the below examples show.
This pattern matches only when an A event is encountered followed by a B event but only if no C event was encountered before either an A event and a B event, counting from the time the pattern is started:
(A -> B) and not C
Assume we'd like to detect when an A event is followed by a D event, without any B or C events between the A and D events:
A -> (D and not (B or C))
It may help your understanding to discuss a pattern that uses the or
operator and the not
operator together:
a=A -> (b=B or not C)
In the pattern above, when an A event arrives then the engine starts the subexpression B or not C
. As soon as the subexpression starts, the not
operator turns to true. The or
expression turns true and thus your listener receives an invocation providing the A event in the property 'a'. The subexpression does not end and continues listening for B and C events. Upon arrival of a B event your listener receives a second invocation. If instead a C event arrives, the not
turns permanently false however that does not affect the or
operator (but would end an and
operator).
To test for absence of an event, use timer:interval
together with and not
operators. The sample statement reports each 10-second interval during which no A event occurred:
every (timer:interval(10 sec) and not A)
In many cases the not
operator, when used alone, does not make sense. The following example is invalid and will log a warning when the engine is started:
// not a sensible pattern (not a=A) -> B(id=a.id)
The followed by ->
operator specifies that first the left hand expression must turn true and only
then is the right hand expression evaluated for matching events.
Look for an A event and if encountered, look for a B event. As always, A and B can itself be nested event pattern expressions.
A -> B
This is a pattern that fires when 2 status events indicating an error occur one after the other.
StatusEvent(status='ERROR') -> StatusEvent(status='ERROR')
A pattern that takes all A events that are not followed by a B event within 5 minutes:
every A -> (timer:interval(5 min) and not B)
A pattern that takes all A events that are not preceded by B within 5 minutes:
every (timer:interval(5 min) and not B -> A)
The followed-by ->
operator can optionally be provided with an expression that limits the number of sub-expression instances of the right-hand side pattern sub-expression.
The synopsis for the followed-by operator with limiting expression is:
lhs_expression -[limit_expression]> rhs_expression
Each time the lhs_expression pattern sub-expression turns true the pattern engine starts a new rhs_expression pattern sub-expression. The limit_expression returns an integer value that defines a maximum number of pattern sub-expression instances that can simultaneously be present for the same rhs_expression.
When the limit is reached the pattern engine issues a com.espertech.esper.client.hook.ConditionPatternSubexpressionMax
notification object to any condition handlers registered with the engine as described in Section 14.11, “Condition Handling” and does not start a new pattern sub-expression instance for the right-hand side pattern sub-expression.
For example, consider the following pattern which returns for every A event the first B event that matches the id
field value of the A event:
every a=A -> b=B(id = a.id)
In the above pattern, every time an A event arrives (lhs) the pattern engine starts a new pattern sub-expression (rhs) consisting of a filter for the first B event that has the same value for the id
field as the A event.
In some cases your application may want to limit the number of right-hand side sub-expressions because of memory concerns or to reduce output. You may add a limit expression returning an integer value as part of the operator.
This example employs the followed-by operator with a limit expression to indicate that maximally 2 filters for B events (the right-hand side pattern sub-expression) may be active at the same time:
every a=A -[2]> b=B(id = a.id)
Note that the limit expression in the example above is not a limit per value of id
field, but a limit counting all right-hand side pattern sub-expression instances that are managed by that followed-by sub-expression instance.
If your followed-by operator lists multiple sub-expressions with limits, each limit applies to the immediate right-hand side. For example, the pattern below limits the number of filters for B events to 2 and the number of filters for C events to 3:
every a=A -[2]> b=B(id = a.id) -[3]> c=C(id = a.id)
Esper allows setting a maximum number of pattern sub-expressions in the configuration, applicable to all followed-by operators of all statements.
If your application has patterns in multiple EPL statements and all such patterns should count towards a total number of pattern sub-expression counts, you may consider setting a maximum number of pattern sub-expression instances, engine-wide, via the configuration described in Section 15.4.18.1, “Followed-By Operator Maximum Subexpression Count”.
When the limit is reached the pattern engine issues a notification object to any condition handlers registered with the engine as described in Section 14.11, “Condition Handling”. Depending on your configuration the engine can prevent the start of a new pattern sub-expression instance for the right-hand side pattern sub-expression, until pattern sub-expression instances end or statements are stopped or destroyed.
The notification object issued to condition handlers is an instance of com.espertech.esper.client.hook.ConditionPatternEngineSubexpressionMax
. The notification object contains information which statement triggered the limit and the pattern counts per statement for all statements.
For information on static and runtime configuration, please consult Section 15.4.18.1, “Followed-By Operator Maximum Subexpression Count”. The limit can be changed and disabled or enabled at runtime via the runtime configuration API.
Guards are where-conditions that control the lifecycle of subexpressions. Custom guard functions can also be used. The section Chapter 17, Integration and Extension outlines guard plug-in development in greater detail.
The pattern guard where-condition has no relationship to the EPL where
clause that filters sets of events.
Take as an example the following pattern expression:
MyEvent where timer:within(10 sec)
In this pattern the timer:within
guard controls the subexpression that is looking for MyEvent events. The guard terminates the subexpression looking for MyEvent events after 10 seconds after start of the pattern. Thus the pattern alerts only once when the first MyEvent event arrives within 10 seconds after start of the pattern.
The every
keyword requires additional discussion since it also controls subexpression lifecycle. Let's add the every
keyword to the example pattern:
every MyEvent where timer:within(10 sec)
The difference to the pattern without every
is that each MyEvent event that arrives now starts a new subexpression, including a new guard, looking for a further MyEvent event. The result is that, when a MyEvent arrives within 10 seconds after pattern start, the pattern execution will look for the next MyEvent event to arrive within 10 seconds after the previous one.
By placing parentheses around the every
keyword and its subexpression, you can have the every
under the control of the guard:
(every MyEvent) where timer:within(10 sec)
In the pattern above, the guard terminates the subexpression looking for all MyEvent events after 10 seconds after start of the pattern. This pattern alerts for all MyEvent events arriving within 10 seconds after pattern start, and then stops.
Guards do not change the truth value of the subexpression of which the guard controls the lifecycle, and therefore do not cause a restart of the subexpression when used with the every
operator. For example, the next pattern stops returning matches after 10 seconds unless a match occurred within 10 seconds after pattern start:
every ( (A and B) where timer:within(10 sec) )
The timer:within
guard acts like a stopwatch.
If the associated pattern expression does not turn true within the specified time period it is stopped and permanently false.
The synopsis for timer:within
is as follows:
timer:within(time_period_expression)
The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds as a parameter. The interval expression may contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
This pattern fires if an A event arrives within 5 seconds after statement creation.
A where timer:within (5 seconds)
This pattern fires for all A events that arrive within 5 seconds. After 5 seconds, this pattern stops matching even if more A events arrive.
(every A) where timer:within (5 seconds)
This pattern matches for any one A or B event in the next 5 seconds.
( A or B ) where timer:within (5 sec)
This pattern matches for any 2 errors that happen 10 seconds within each other.
every (StatusEvent(status='ERROR') -> StatusEvent(status='ERROR') where timer:within (10 sec))
The following guards are equivalent:
timer:within(2 minutes 5 seconds) timer:within(125 sec) timer:within(125)
The timer:withinmax
guard is similar to the timer:within
guard and acts as a stopwatch that additionally has a counter that counts the number of matches. It ends the subexpression when either the stopwatch ends or the match counter maximum value is reached.
The synopsis for timer:withinmax
is as follows:
timer:withinmax(time_period_expression, max_count_expression)
The time_period_expression is a time period (see Section 5.2.1, “Specifying Time Periods”) or an expression providing a number of seconds.
The max_count_expression provides the maximum number of matches before the guard ends the subexpression.
Each parameter expression may also contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
This pattern fires for every A event that arrives within 5 seconds after statement creation but only up to the first two A events:
(every A) where timer:withinmax (5 seconds, 2)
If the result of the max_count_expression is 1, the guard ends the subexpression after the first match and indicates the first match.
This pattern fires for the first A event that arrives within 5 seconds after statement creation:
(every A) where timer:withinmax (5 seconds, 1)
If the result of the max_count_expression is zero, the guard ends the subexpression upon the first match and does no indicate any matches.
This example receives every A event followed by every B event (as each B event arrives) until the 5-second subexpression timer ends or X number of B events have arrived (assume X was declared as a variable):
every A -> (every B) where timer:withinmax (5 seconds, X)
The while
guard is followed by an expression that the engine evaluates for every match reported by the guard pattern sub-expression. When the expression returns false the pattern sub-expression ends.
The synopsis for while
is as follows:
while (guard_expression)
The guard_expression is any expression that returns a boolean true or false. The expression may contain references to properties of prior events in the same pattern as well as variables and substitution parameters.
Each time the subexpression indicates a match, the engine evaluates guard_expression and if true, passes the match and when false, ends the subexpression.
This pattern fires for every A event until an A event arrives that has a value of zero or less for its size
property (assuming A events have an integer size
property).
(every a=A) while (a.size > 0)
Note the parenthesis around the every
subexpression. They ensure that, following precedence rules, the guard applies to the every
operator as well.
The timer:within
and timer:withinmax
guards may be parameterized by an expression that contains one or more references to properties of prior events in the same pattern.
As a simple example, this pattern matches every A event followed by a B event that arrives within delta
seconds after the A event:
every a=A -> b=B where timer:within (a.delta seconds)
Herein A event is assumed to have a delta
property that provides the number of seconds to wait for B events. Each arriving A event may have a different value for delta
and the guard is therefore parameterized dynamically based on the prior A event received.
When multiple events accumulate, for example when using the match-until or repeat pattern elements, an index must be provided:
[2] a=A -> b=B where timer:within (a[0].delta + a[1].delta)
The above pattern matches after 2 A events arrive followed by a B event within a time interval after the A event that is defined by the sum of the delta
properties of both A events.
You can combine guard expression by using parenthesis around each subexpression.
The below pattern matches for each A event while A events of size greater then zero arrive and only within the first 20 seconds:
((every a=A) while (a.size > 0)) where timer:within(20)
Filter atoms have been described in section Section 7.4, “Filter Expressions in Patterns”.
Observers observe time-based events for which the thread-of-control originates by the engine timer or external timer event. Custom observers can also be developed that observe timer events or other engine-external application events such as a file-exists check. The section Chapter 17, Integration and Extension outlines observer plug-in development in greater detail.
The timer:interval
pattern observer waits for the defined time before the truth value of the observer turns true.
The observer takes a time period (see Section 5.2.1, “Specifying Time Periods”) as a parameter, or an expression that returns the number of seconds.
The observer may be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value.
After an A event arrived wait 10 seconds then indicate that the pattern matches.
A -> timer:interval(10 seconds)
The pattern below fires every 20 seconds.
every timer:interval(20 sec)
The next example pattern fires for every A event that is not followed by a B event within 60 seconds after the A event arrived. The B event must have the same "id" property value as the A event.
every a=A -> (timer:interval(60 sec) and not B(id=a.id))
Consider the next example, which assumes that the A event has a property waittime
:
every a=A -> (timer:interval(a.waittime + 2) and not B(id=a.id))
In the above pattern the logic waits for 2 seconds plus the number of seconds provided by the value of the waittime
property of the A event.
The timer:at
pattern observer is similar in function to the Unix “crontab” command. At a specified time the
expression turns true. The at
operator can also be made to pattern match at regular intervals by using an every
operator
in front of the timer:at
operator.
The syntax is: timer:at (minutes, hours, days of month, months, days of week [, seconds [, time zone]]).
The value for seconds and time zone is optional. Each element allows wildcard *
values. Ranges can be specified
by means of lower bounds then a colon ‘:’ then the upper bound. The division operator */x
can be used to
specify that every xth value is valid. Combinations of these operators can be used by placing these into square brackets ([]).
The timer:at
observer may also be parameterized by an expression that contains one or more references to properties of prior events in the same pattern, or may also reference variables, substitution parameters or any other expression returning a numeric value. The frequency division operator */x
and parameters lists within brackets ([]) are an exception: they may only contain variables, substitution parameters or numeric values.
This expression pattern matches every 5 minutes past the hour.
every timer:at(5, *, *, *, *)
The below timer:at
pattern matches every 15 minutes from 8am to 5:45pm (hours 8 to 17 at 0, 15, 30 and 45 minutes past the hour) on even numbered days of the month as well as on the
first day of the month.
timer:at (*/15, 8:17, [*/2, 1], *, *)
The below table outlines the fields, valid values and keywords available for each field:
Table 7.5. Crontab Fields
Field Name | Mandatory? | Allowed Values | Additional Keywords |
---|---|---|---|
Minutes | yes | 0 - 59 | |
Hours | yes | 0 - 23 | |
Days Of Month | yes | 1 - 31 | last, weekday, lastweekday |
Months | yes | 1 - 12 | |
Days Of Week | yes | 0 (Sunday) - 6 (Saturday) | last |
Seconds | no (required if specifying a time zone) | 0 - 59 | |
Time Zone | no | any string (not validated, see TimeZone javadoc) |
The keyword last
used in the days-of-month field means the last day of the month (current month). To specify the last day of another month, you must provide a value for the month field. For example: timer:at(*, *, last,2,*)
is the last day of February.
The last
keyword in the day-of-week field by itself simply means Saturday. If used in the day-of-week field after another value, it means "the last xxx day of the month" - for example "5 last" means "the last Friday of the month".
So the last Friday of the current month will be: timer:at(*, *, *, *, 5 last)
. And the last Friday of June: timer:at(*, *, *, 6, 5 last)
.
The keyword weekday
is used to specify the weekday (Monday-Friday) nearest the given day. Variant could include month like in: timer:at(*, *, 30 weekday, 9, *)
which for year 2007 is Friday September 28th (no jump over month).
The keyword lastweekday
is a combination of two parameters, the last
and the weekday
keywords. A typical example could be: timer:at(*, *, *, lastweekday, 9, *)
which will define Friday September 28th (example year is 2007).
The time zone is a string-type value that specifies the time zone of the schedule. You must specify a value for seconds when specifying a time zone. Esper relies on the java.util.TimeZone
to interpret the time zone value. Note that TimeZone
does not validate time zone strings.
The following timer:at
pattern matches at 5:00 pm Pacific Standard Time (PST):
timer:at (0, 17, *, *, *, *, 'PST')
Any expression may occur among the parameters. This example invokes a user-defined function computeHour
to return an hour:
timer:at (0, computeHour(), *, *, *, *)
The following restrictions apply to crontab parameters:
It is not possible to specify both Days Of Month and Days Of Week.
When using timer:at
with the every
operator the crontab-like timer computes the next time at which the timer should fire based on the specification and the current time. When using every
, the current time is the time the timer fired or the statement start time if the timer has not fired once.
For example, this pattern fires every 1 minute starting at 1:00pm and ending at 1:59pm, every day:
every timer:at(*, 13, *, *, *)
Assume the above statement gets started at 1:05pm and 20 seconds. In such case the above pattern fires every 1 minute starting at 1:06pm and ending at 1:59pm for that day and 1:00pm to 1:59pm every following day.
To get the pattern to fire only once at 1pm every day, explicitly specify the minute to start. The pattern below fires every day at 1:00pm:
every timer:at(0, 13, *, *, *)
By specifying a second resolution the timer can be made to fire every second, for instance:
every timer:at(*, *, *, *, *, *)
The timer:schedule
observer is a flexible observer for scheduling.
The observer implements relevant parts of the ISO 8601 specification however it is not necessary to use ISO 8601 formats. The ISO 8601 standard is an international standard covering the exchange of date and time-related data. The standard specifies a date format, a format for time periods and a format for specifying the number of repetitions. Please find more information on ISO 8601 at Wikipedia.
The observer takes the following named parameters:
Table 7.6. Timer Schedule Parameters
Name | Description |
---|---|
iso | An expression returning a string-type ISO 8601 formatted date, time period and/or number of repetitions. |
repetitions | An expression returning a numeric value that specifies a number of repetitions. Provide a value of -1 for an unlimited number of repetitions.
If unspecified, the number of repetitions is one. |
date | An expression returning a string-type ISO 8601 formatted date, or an expression that returns any of these types:
long , Date , Calendar , LocalDateTime , ZonedDateTime . |
period | An expression returning a time period, see Section 5.2.1, “Specifying Time Periods” |
In summary, for example, the below pattern schedules two callbacks: The first callback 2008-03-01 at 13:00:00 UTC and the second callback on 2009-05-11 at 15:30:00 UTC.
select * from pattern[every timer:schedule(iso: 'R2/2008-03-01T13:00:00Z/P1Y2M10DT2H30M')]
The number of repetitions, date and period can be separated and do not have to be ISO 8601 strings, allowing each part to be an own expression.
This example specifies separate expressions. The equivalent schedule to the above example is:
select * from pattern[every timer:schedule(repetitions: 2, date: '2008-03-01T13:00:00Z', period: 1 year 2 month 10 days 2 hours 30 minutes)]
When providing the iso
parameter, it must be the only parameter. The repetitions
parameter is only allowed in conjunction with other parameters.
The complete document for ISO 8601, the international standard for the representation of dates and times, can be found at http://www.w3.org/TR/NOTE-datetime.
The supported ISO 8601 date formats are:
Table 7.7. ISO 8601 Period Examples
Description | Format | Example |
---|---|---|
Complete date plus hours, minutes and seconds (zero milliseconds, zero microseconds): | YYYY-MM-DDThh:mm:ssTZD |
(i.e. GMT+00:00, UTC)
(i.e. GMT+01:00)
(i.e. local time zone) |
Complete date plus hours, minutes, seconds and a decimal fraction of a second (zero microseconds) | YYYY-MM-DDThh:mm:ss.sTZD |
(i.e. GMT+00:00, UTC)
(i.e. GMT+01:00)
(i.e. local time zone) |
In ISO 8601, periods are specified by a P
and an optional year, month, week and day count. If there is a time part, add T
and optionally provide the hour, minute and seconds. The format does not have any whitespace. The synopsis is:
P [nY] [nM] [nW] [nD] [T [nH [nM] [nS] ]
The Y
stands for years, the M
for month as well as minutes, the W
for weeks and the D for days.
The H
stands for hours and the S
means seconds.
Table 7.8. ISO 8601 Period Examples
Example | Description |
---|---|
P10M | 10 months |
PT10M | 10 minutes |
P1Y3M12D | 1 year, 3 month and 12 days |
P10DT5M | 10 days and 5 minutes |
P1Y2M3DT4H5M6S | 1 year, 2 month, 3 days, 4 hours, 5 minutes, 6 seconds |
To instruct the engine to observe a date, provide a date to the observer. When time advances to the specified date, the pattern subexpression fires.
For example, this pattern fires once when time reaches 2012-10-01 at 5:52:00 (UTC):
timer:schedule(iso:'2012-10-01T05:52:00Z')
This equivalent pattern specifies separate expressions:
every timer:schedule(date: '2012-10-01T05:52:00Z')
When the observer fires, the pattern subexpression becomes permanently false and the engine does not restart the observer.
If the provided date is a past date as compared to engine time, the pattern subexpression becomes permanently false on start.
To instruct the engine to observe a period starting from the current engine time, provide a period. When time advances to the current engine time plus the specified period, the pattern subexpression fires.
Assuming the current engine time is 2012-10-01 at 5:52:00 (UTC), this pattern fires once when time reaches 5:53:00:
timer:schedule(iso:'PT1M')
This equivalent pattern specifies separate expressions:
every timer:schedule(period: 1 minute)
When the observer fires, the pattern subexpression becomes permanently false and the engine does not restart the observer.
To instruct the engine to observe a period starting from a given date, provide a date and a period. When time advances to the date plus the specified period, the pattern subexpression fires.
Assuming the current engine time is 5:52:00 (UTC), this pattern fires once when time reaches 2012-10-01 at 5:53:00:
timer:schedule(iso:'2012-10-01T05:52:00Z/PT1M')
This equivalent pattern specifies separate expressions:
every timer:schedule(date: '2012-10-01T05:52:00Z', period: 1 minute)
When the observer fires, the pattern subexpression becomes permanently false and the engine does not restart the observer.
To instruct the engine to observe a period starting from the current engine time and repeatedly thereafter anchored to current engine time, provide a number of repetitions and a period (see synopsis provided earlier), like this:
repetitions/period
timer:schedule(iso: 'R2/PT1M')
When time advances to the current engine time plus the specified period, the pattern subexpression fires for the first time. Repeatedly when time advances to the current engine time plus a multiple of the specified period, the pattern subexpression fires, up to the number of repetitions specified (if any).
This pattern specifies a repetition of two. Assuming the current engine time is 2012-10-01 at 5:52:00 (UTC), it fires when time reaches 5:53:00 and again when time reaches 5:54:00:
every timer:schedule(iso: 'R2/PT1M')
This equivalent pattern specifies separate expressions:
every timer:schedule(repetitions: 2, period: 1 minute)
All schedule computations are relative to (i.e. anchored to) current engine time at observer start. Once the number of repetitions is reached relative to the current engine time at observer start, the pattern subexpression becomes permanently false and the engine does not restart the observer.
Please specify the every
operator for repeating schedules.
To instruct the engine to observe a period starting from a given date and repeatedly thereafter anchored to the provide date, provide a number of repetitions and a date and a period (see synopsis provided earlier), like this:
repetitions/date/period
When time advances to the date, the pattern subexpression fires for the first time. You may specify a date older than current engine time as an anchor. Repeatedly when time advances to the date plus a multiple of the specified period, the pattern subexpression fires, up to the number of repetitions specified (if any).
This pattern specifies a repetition of two. The pattern fires when time reaches 2012-10-01 at 5:52:00 (UTC) and again when time reaches 5:53:00.
every timer:schedule(iso: 'R2/2012-10-01T05:52:00Z/PT1M')
This equivalent pattern specifies separate expressions:
every timer:schedule(repetitions: 2, date:'2012-10-01T05:52:00Z', period: 1 minute)
All schedule computations are relative to (i.e. anchored to) the provided date. Once the number of repetitions is reached relative to the provided date, the pattern subexpression becomes permanently false and the engine does not restart the observer.
Please specify the every
operator for repeating schedules.
The pattern below outputs every MyEvent
event after the MyEvent arrived and upon the next round 15 seconds:
select * from pattern[every e=MyEvent -> timer:schedule(iso: 'R/1980-01-01T00:00:00Z/PT15S']
Assuming a MyEvent
event arrives on 2012-10-01 at 5:51:07 the output for that event occurs at 5:51:15.
All parameters can be expressions. The date
parameter could, for example, be used with current_timestamp
to compute a schedule:
select * from pattern[date: current_timestamp.withTime(9, 0, 0, 0)]
The above statement fires only at 9am and not after 9am on the same day (one repetition).
The following EPL is equivalent:
select * from pattern[every timer:schedule(iso: 'R2/2008-03-01T13:00:00Z/P1Y2M10DT2H30M')]
select * from pattern[every (timer:schedule(iso: '2008-03-01T13:00:00Z') or timer:schedule(iso: '2009-05-11T15:30:00Z'))]
select * from pattern[every (timer:schedule(iso: '2008-03-01T13:00:00Z') or timer:schedule(iso: '2008-03-01T13:00:00Z/P1Y2M10DT2H30M'))]
This walkthrough discusses the following pattern:
every a=LoginEvent -> (timer:interval(1 min) and not LogoutEvent(userId=a.userId))]
This pattern detects when a LoginEvent
is not followed by a LogoutEvent
arriving within 1 minute of the LoginEvent
.
The engine parses the pattern expression and builds a expression tree. At the root of the expression tree, for this specific pattern, is the followed-by (->
) operator as followed-by has the lowest precedence (see precedence).
The pattern expression tree looks like this.
Followed-by operator (->) | | ---- Every | | | | ---- a=LoginEvent | | ---- And | | ---- timer:interval(1 min) | | ---- Not | | ---- LogoutEvent(userId=a.userId)
The followed-by (->
) operator has two subexpressions that it manages, the Every
expression that itself has the a=LoginEvent
subexpression as a child, and the And
expression that itself has two subexpressions as children.
When the engine activates a pattern or pattern subexpression, it activates it from a top-down direction. In this example the engine activates the followed-by (->
) expression.
The followed-by expression activates only its own left-most subexpression, which is the Every
expression. The Every
expression in turn activates the a=LoginEvent
subexpression which registers a filter looking for a LoginEvent
in the global filter index (see Section 2.18.2, “Filter Indexes”).
In this example, the And
subexpression and its subexpressions do not get activated. Therefore after this pattern initially activates there is no
filter active time looking for LogoutEvent
event and there is no time tracking of any kind at this point.
Let time t0 be the time of EPL statement creation. As discussed, upon creation of the EPL statement the engine activates the pattern subexpression every a=LoginEvent
only. At this time there is 1 active subexpression:
every a=LoginEvent
As you can see, the tree of active subexpressions (the tree of states) is not the same as the tree of expressions. After the pattern activated there are no active subexpressions (no states) for the And
-part of the example expression tree.
Let's assume that at time t1 a LoginEvent
with user id 10 arrives.
This causes the a=LoginEvent
subexpression to become true. In turn, this causes the Every
subexpression to become true. This causes the left-hand-side of the followed-by subexpression to become true. This in turn causes activation of the And
subexpression.
The activation of the And
-subexpression causes activation of both timer:interval(1 min)
(from t1) and the Not
-subexpression.
The activation of the And
-subexpression registers a filter looking for a LogoutEvent
that has a user id value of 10 in the global filter index
and registers a timer callback for t1+1 minute.
The engine does not terminate the subexpression a=LoginEvent
as it lives under an Every
operator which means it should keep looking for
more LoginEvent
events.
There now are 2 active subexpressions:
every a=LoginEvent
timer:interval(t1+1 min) and not LogoutEvent(userId=10)
Let's assume that at time t2 another LoginEvent
with user id 20 arrives (assume t2 – t1 < 1 min).
This again causes the a=LoginEvent
subexpression to become true. In turn, this causes the Every
subexpression to become true. This causes the left-hand-side of the followed-by subexpression to become true. This in turn causes activation of the And
subexpression.
The activation of the And
-subexpression causes activation of both timer:interval(1 min)
(from t2) and the Not
-subexpression.
The activation of the And
-subexpression registers a filter looking for a LogoutEvent
that has a user id value of 20 in the global filter index
and registers a timer callback for t2+1 minute.
There now are 3 active subexpressions:
every a=LoginEvent
timer:interval(t1+1 min) and not LogoutEvent(userId=10)
timer:interval(t2+1 min) and not LogoutEvent(userId=20)
In this scenario at time t3 one minute has passed since t1 and a matching LogoutEvent
was not received.
The subexpression timer:interval(t1+1 min)
becomes true. In turn, this causes the And
-subexpression to become true as the Not
-subexpression is already true on start. In turn, this causes the followed-by subexpression to become true. In turn, this causes an output of the pattern match.
The engine terminates the subexpression not LogoutEvent(userId=10)
which unregisters the filter looking for a LogoutEvent
that has a user id value of 10 from the global filter index.
There now are 2 active subexpressions:
every a=LoginEvent
timer:interval(t2+1 min) and not LogoutEvent(userId=20)
Let's assume that at time t4 a LogoutEvent
with user id 20 arrives (assume t4 – t2 < 1 min).
This again causes the LogoutEvent(userId=20)
subexpression to become true. In turn, this causes the Not
subexpression to become permanently false (as Not
reverses the truth value). This causes the And
-subexpression to become permanently false.
This causes the pattern subexpression timer:interval(t2+1 min) and not LogoutEvent(userId=20)
to terminate which causes the timer callback t2+1 minute to unregister.
Finally there is 1 active subexpression remaining:
every a=LoginEvent