Esper Reference

If your application uses java.util.Map, Object[] (object-array) or XML to represent events, then event property names may themselves contain the dot ('.') character. The backslash ('\') character can be used to escape dot characters in property names, allowing a property name to contain dot characters.

For example, the EPL as shown below expects a property by name part1.part2 to exist on event type MyEvent:

select part1\.part2 from MyEvent

Sometimes your event properties may overlap with EPL language keywords or contain spaces or other special characters. In this case you may use the backwards apostrophe ` (aka. back tick) character to escape the property name.

The next example assumes a Quote event that has a property by name order, while order is also a reserved keyword:

select `order`, price as `price.for.goods` from Quote

When escaping mapped or indexed properties, make sure the back tick character appears outside of the map key or index.

The next EPL selects event properties that have names that contain spaces (e.g. candidate book), have the tick special character (e.g. children's books), are an indexed property (e.g. children's books[0]) and a mapped property that has a reserved keyword as part of the property name (e.g. book select('isbn')):

select `candidate book` , `children's books`[0], `book select`('isbn') from MyEventType

The key or index expression must be placed in parenthesis. When using an expression as key for a mapped property, the expression must return a String-typed value. When using an expression as index for an indexed property, the expression must return an int-typed value.

This example below uses Java classes to illustrate;The same principles apply to all event representations.

Assume a class declares these properties (getters not shown for brevity):

public class MyEventType {
  String myMapKey;
  int myIndexValue;
  int myInnerIndexValue;
  Map<String, InnerType> innerTypesMap;	// mapped property
  InnerType[] innerTypesArray; // indexed property
}

public class InnerType {
  String name;
  int[] ids;
}

A sample EPL statement demonstrating expressions as map keys or indexes is:

select innerTypesMap('somekey'),  // returns map value for 'somekey'
  innerTypesMap(myMapKey),        // returns map value for myMapKey value (an expression)
  innerTypesArray[1],             // returns array value at index 1
  innerTypesArray(myIndexValue)   // returns array value at index myIndexValue (an expression)
  from MyEventType

The dot-operator can be used to access methods on the value objects returned by the mapped or indexed properties. By using the dot-operator the syntax follows the chained method invocation described at Section 8.6, “Dot Operator”.

A sample EPL statement demonstrating the dot-operator as well as expressions as map keys or indexes is:

 select innerTypesMap('somekey').ids[1],
  innerTypesMap(myMapKey).getIds(myIndexValue),
  innerTypesArray[1].ids[2],
  innerTypesArray(myIndexValue).getIds(myInnerIndexValue)
  from MyEventType

Please note the following limitations:

As outlined earlier, the different property types are supported by the standard JavaBeans specification, and some of which are uniquely supported by Esper:

Assume there is an NewEmployeeEvent event class as shown below. The mapped and indexed properties in this example return Java objects but could also return Java language primitive types (such as int or String). The Address object and Employee can themselves have properties that are nested within them, such as a street name in the Address object or a name of the employee in the Employee object.

public class NewEmployeeEvent {
	public String getFirstName();
	public Address getAddress(String type);
	public Employee getSubordinate(int index);
	public Employee[] getAllSubordinates();
}

Simple event properties require a getter-method that returns the property value. In this example, the getFirstName getter method returns the firstName event property of type String.

Indexed event properties require either one of the following getter-methods. A method that takes an integer-type key value and returns the property value, such as the getSubordinate method, or a method that returns an array-type, or a class that implements Iterable. An example is the getAllSubordinates getter method, which returns an array of Employee but could also return an Iterable. In an EPL or event pattern statement, indexed properties are accessed via the property[index] syntax.

Mapped event properties require a getter-method that takes a String-typed key value and returns the property value, such as the getAddress method. In an EPL or event pattern statement, mapped properties are accessed via the property('key') syntax.

Nested event properties require a getter-method that returns the nesting object. The getAddress and getSubordinate methods are mapped and indexed properties that return a nesting object. In an EPL or event pattern statement, nested properties are accessed via the property.nestedProperty syntax.

All event pattern and EPL statements allow the use of indexed, mapped and nested properties (or a combination of these) anywhere where one or more event property names are expected. The below example shows different combinations of indexed, mapped and nested properties in filters of event pattern expressions (each line is a separate EPL statement):

every NewEmployeeEvent(firstName='myName')
every NewEmployeeEvent(address('home').streetName='Park Avenue')
every NewEmployeeEvent(subordinate[0].name='anotherName')
every NewEmployeeEvent(allSubordinates[1].name='thatName')
every NewEmployeeEvent(subordinate[0].address('home').streetName='Water Street')

Similarly, the syntax can be used in EPL statements in all places where an event property name is expected, such as in select lists, where-clauses or join criteria.

select firstName, address('work'), subordinate[0].name, subordinate[1].name
from NewEmployeeEvent
where address('work').streetName = 'Park Ave'

Property names follows Java standards: the class java.beans.Introspector and method getBeanInfo returns the property names as derived from the name of getter methods. In addition, Esper configuration provides a flag to turn off case-sensitive property names. A sample list of getter methods and property names is:

select price from MyEvent

select NAME from MyEvent

select itemDesc from MyEvent

select q from MyEvent

select QN from MyEvent

select qn from MyEvent

select s from MyEvent

When your getter methods or accessor fields return a parameterized type, for example Iterable<MyEventData> for an indexed property or Map<String, MyEventData> for a mapped property, then property expressions may refer to the properties available through the class that is the type parameter.

An example event that has properties that are parameterized types is:

public class NewEmployeeEvent {
  public String getName();
  public Iterable<EducationHistory> getEducation();
  public Map<String, Address> getAddresses();
}

A sample of valid property expressions for this event is shown next:

select name, education, education[0].date, addresses('home').street
from NewEmployeeEvent

An EPL statement may update indexed or mapped properties of an event, provided the event class exposes the required setter method.

The setter method for indexed properties must be named setPropertyName and must take two parameters: the int-type index and the Object type new value.

The setter method for mapped properties must be named setPropertyName and must take two parameters: the String-type map key and the Object type new map value.

The following is an example event that features a setter method for the props mapped property and for the array indexed property:

public class MyEvent {
  private Map props = new HashMap();
  private Object[] array = new Object[10];

  public void setProps(String name, Object value) {
    props.put(name, value);
  }

  public void setArray(int index, Object value) {
    array[index] = value;
  }
  // ... also provide regular JavaBean getters and setters for all properties

This sample statement updates mapped and indexed property values:

update istream MyEventStream set props('key') = 'abc', array[2] = 100

Esper employs byte code generation for fast access to event properties. When byte code generation is unsuccessful, the engine logs a warning and uses Java reflection to obtain property values instead.

A known limitation is that when an interface has an attribute of a particular type and the actual event bean class returns a subclass of that attribute, the engine logs a warning and uses reflection for that property.

Events can also be represented by objects that implement the java.util.Map interface. Event properties of Map events are the values in the map accessible through the get method exposed by the java.util.Map interface.

Similar to the Object-array event type, the Map event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types, thereby making it easier to change types at runtime or generate type information from another source.

A given Map event type can have one or more supertypes that must also be Map event types. All properties available on any of the Map supertypes are available on the type itself. In addition, anywhere within EPL that an event type name of a Map supertype is used, any of its Map subtypes and their subtypes match that expression.

Your application can add properties to an existing Map event type during runtime using the configuration operation updateMapEventType. Properties may not be updated or deleted - properties can only be added, and nested properties can be added as well. The runtime configuration also allows removing Map event types and adding them back with new type information.

After your application configures a Map event type by providing a type name, the type name can be used when defining further Map or Object-array event types by specifying the type name as a property type or an array property type.

One-to-Many relationships in Map event types are represented via arrays. A property in a Map event type may be an array of primitive, an array of Java object, an array of Map or an an array of Object-array.

The engine can process java.util.Map events via the sendEvent(Map map, String eventTypeName) method on the EPRuntime interface. Entries in the Map represent event properties. Keys must be of type java.util.String for the engine to be able to look up event property names specified by pattern or EPL statements.

The engine does not validate Map event property names or values. Your application should ensure that objects passed in as event properties match the create schema property names and types, or the configured event type information when using runtime or static configuration.

Map event properties can be of any type. Map event properties that are Java application objects or that are of type java.util.Map (or arrays thereof) or that are of type Object[] (object-array) (or arrays thereof) offer additional power:

In order to use Map events, the event type name and property names and types must be made known to the engine via Configuration or create schema EPL syntax. Please see examples in Section 5.16, “Declaring an Event Type: Create Schema” and Section 15.4.2, “Events represented by java.util.Map”.

The code snippet below defines a Map event type, creates a Map event and sends the event into the engine. The sample defines the CarLocUpdateEvent event type via runtime configuration interface (create schema or static configuration could have been used instead).

// Define CarLocUpdateEvent event type (example for runtime-configuration interface)
Map<String, Object> def = new HashMap<String, Object>;
def.put("carId", String.class);
def.put("direction", int.class);

epService.getEPAdministrator().getConfiguration().
  addEventType("CarLocUpdateEvent", def);

The CarLocUpdateEvent can now be used in a statement:

select carId from CarLocUpdateEvent.win:time(1 min) where direction = 1

// Create a CarLocUpdateEvent event and send it into the engine for processing
Map<String, Object> event = new HashMap<String, Object>();
event.put("carId", carId);
event.put("direction", direction);

epRuntime.sendEvent(event, "CarLocUpdateEvent");

The engine can also query Java objects as values in a Map event via the nested property syntax. Thus Map events can be used to aggregate multiple data structures into a single event and query the composite information in a convenient way. The example below demonstrates a Map event with a transaction and an account object.

Map event = new HashMap();
event.put("txn", txn);
event.put("account", account);
epRuntime.sendEvent(event, "TxnEvent");

An example statement could look as follows.

select account.id, account.rate * txn.amount 
from TxnEvent.win:time(60 sec) 
group by account.id

Your Map event type may declare one or more supertypes when configuring the type at engine initialization time or at runtime through the administrative interface.

Supertypes of a Map event type must also be Map event types. All property names and types of a supertype are also available on a subtype and override such same-name properties of the subtype. In addition, anywhere within EPL that an event type name of a Map supertype is used, any of its Map subtypes also matches that expression (similar to the concept of interface in Java).

This example assumes that the BaseUpdate event type has been declared and acts as a supertype to the AccountUpdate event type (both Map event types):

epService.getEPAdministrator().getConfiguration().
    addEventType("AccountUpdate", accountUpdateDef, 
    new String[] {"BaseUpdate"});

Your application EPL statements may select BaseUpdate events and receive both BaseUpdate and AccountUpdate events, as well as any other subtypes of BaseUpdate and their subtypes.

// Receive BaseUpdate and any subtypes including subtypes of subtypes
select * from BaseUpdate

Your application Map event type may have multiple supertypes. The multiple inheritance hierarchy between Maps can be arbitrarily deep, however cyclic dependencies are not allowed. If using runtime configuration, supertypes must exist before a subtype to a supertype can be added.

See Section 15.4.2, “Events represented by java.util.Map” for more information on configuring Map event types.

An event can also be represented by an array of objects. Event properties of Object[] events are the array element values.

Similar to the Map event type, the object-array event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types, thereby making it easier to change types at runtime or generate type information from another source.

A given Object-array event type can have only a single supertype that must also be an Object-array event type. All properties available on the Object-array supertype is also available on the type itself. In addition, anywhere within EPL that an event type name of an Object-array supertype is used, any of its Object-array subtypes and their subtypes match that expression.

Your application can add properties to an existing Object-array event type during runtime using the configuration operation updateObjectArrayEventType. Properties may not be updated or deleted - properties can only be added, and nested properties can be added as well. The runtime configuration also allows removing Object-array event types and adding them back with new type information.

After your application configures an Object-array event type by providing a type name, the type name can be used when defining further Object-array or Map event types by specifying the type name as a property type or an array property type.

One-to-Many relationships in Object-array event types are represented via arrays. A property in an Object-array event type may be an array of primitive, an array of Java object, an array of Map or an array of Object-array.

The engine can process Object[] events via the sendEvent(Object[] array, String eventTypeName) method on the EPRuntime interface. Entries in the Object array represent event properties.

The engine does not validate Object array length or value types. Your application must ensure that Object array values match the declaration of the event type: The type and position of property values must match property names and types in the same exact order and object array length must match the number of properties declared via create schema or the static or runtime configuration.

Object-array event properties can be of any type. Object-array event properties that are Java application objects or that are of type java.util.Map (or arrays thereof) or that are of type Object-array (or arrays thereof) offer additional power:

In order to use Object[] (object-array) events, the event type name and property names and types, in a well-defined order that must match object-array event properties, must be made known to the engine via configuration or create schema EPL syntax. Please see examples in Section 5.16, “Declaring an Event Type: Create Schema” and Section 15.4.3, “Events represented by Object[] (Object-array)”.

The code snippet below defines an Object-array event type, creates an Object-array event and sends the event into the engine. The sample defines the CarLocUpdateEvent event type via the runtime configuration interface (create schema or static configuration could have been used instead).

// Define CarLocUpdateEvent event type (example for runtime-configuration interface)
String[] propertyNames = {"carId", "direction"};   // order is important
Object[] propertyTypes = {String.class, int.class};  // type order matches name order

epService.getEPAdministrator().getConfiguration().
  addEventType("CarLocUpdateEvent", propertyNames, propertyTypes);

The CarLocUpdateEvent can now be used in a statement:

select carId from CarLocUpdateEvent.win:time(1 min) where direction = 1

// Send an event
Object[] event = {carId, direction};
epRuntime.sendEvent(event, "CarLocUpdateEvent");

The engine can also query Java objects as values in an Object[] event via the nested property syntax. Thus Object[] events can be used to aggregate multiple data structures into a single event and query the composite information in a convenient way. The example below demonstrates a Object[] event with a transaction and an account object.

epRuntime.sendEvent(new Object[] {txn, account}, "TxnEvent");

An example statement could look as follows:

select account.id, account.rate * txn.amount 
from TxnEvent.win:time(60 sec) 
group by account.id

Your Object[] (object-array) event type may declare one supertype when configuring the type at engine initialization time or at runtime through the administrative interface.

The supertype of a Object[] event type must also be an object-array event type. All property names and types of a supertype are also available on a subtype and override such same-name properties of the subtype. In addition, anywhere within EPL that an event type name of an Object-array supertype is used, any of its Object-array subtypes also matches that expression (similar to the concept of interface or superclass).

The properties provided by the top-most supertype must occur first in the object array. Subtypes each append to the object array. The number of values appended must match the number of properties declared by the subtype.

For example, assume your application declares the following two types:

create objectarray schema SuperType (p0 string)

create objectarray schema SubType (p1 string) inherits SuperType

The object array event objects that your application can send into the engine are shown by the next code snippet:

epRuntime.sendEvent(new Object[] {"p0_value", "p1_value"}, "SubType");
epRuntime.sendEvent(new Object[] {"p0_value"}, "SuperType");

If you have a XSD schema document available for your XML events, Esper can interrogate the schema. The benefits are:

URL schemaURL = this.getClass().getClassLoader().getResource("sensor.xsd");

epService = EPServiceProviderManager.getDefaultProvider();
ConfigurationEventTypeXMLDOM sensorcfg = new ConfigurationEventTypeXMLDOM();
sensorcfg.setRootElementName("Sensor");
sensorcfg.setSchemaResource(schemaURL.toString());
epService.getEPAdministrator().getConfiguration()
    .addEventType("SensorEvent", sensorcfg);

select ID, Observation.Command, Observation.ID, 
  Observation.Tag[0].ID, Observation.Tag[1].ID
from SensorEvent

InputSource source = new InputSource(new StringReader(xml));
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
builderFactory.setNamespaceAware(true);
Document doc = builderFactory.newDocumentBuilder().parse(source);

epService.getEPRuntime().sendEvent(doc);

select nested.mapped('key').indexed[1].attribute from MyEvent

/n0:rootelement/n0:nested/n0:mapped[@id='key']/n0:indexed[position() = 2]/@attribute

<item>
  <book sku="8800090">
    <author>Isaac Asimov</author>
    <author>Robert A Heinlein</author>
  </book>
</item>

select Origin?.LocationCode from SensorEvent

insert into ObservationStream
select ID, Observation from SensorEvent

select Observation.Command, Observation.Tag[0].ID from ObservationStream

insert into TagListStream
select ID as sensorId, Observation.* from SensorEvent

select sensorId, Command, Tag[0].ID from TagListStream

EventSender sender = epRuntime.getEventSender("SensorEvent");
sender.sendEvent(node);

System.setProperty(DOMImplementationRegistry.PROPERTY, 
  "com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl");

Without a schema document a XML event may still be queried. However there are important differences in the metadata available without a schema document and therefore the property expression results. These differences are outlined below.

All property expressions against a XML type without schema are assumed valid. There is no validation of the property expression other then syntax validation. At runtime, property expressions return string-type values or null if the expression did not yield a matching element or attribute result.

When asked for property names or property metadata, a no-schema type returns empty array.

In all other aspects the type behaves the same as the schema-provided type described earlier.

Regardless of whether or not you provide a XSD schema for the XML event type, you can always fall back to configuring explicit properties that are backed by XPath expressions.

For further documentation on XPath, please consult the XPath standard or other online material. Consider using Jaxen or Apache Axiom, for example, to provide faster XPath evaluation then your Java VM built-in XPath provider may offer.

epService = EPServiceProviderManager.getDefaultProvider();
ConfigurationEventTypeXMLDOM sensorcfg = new ConfigurationEventTypeXMLDOM();
sensorcfg.addXPathProperty("countTags", "count(/ss:Sensor/ss:Observation/ss:Tag)", 
    XPathConstants.NUMBER);
sensorcfg.addNamespacePrefix("ss", "SensorSchema");
sensorcfg.setRootElementName("Sensor");
epService.getEPAdministrator().getConfiguration()
    .addEventType("SensorEvent", sensorcfg);

select countTags from SensorEvent

sensorcfg.addXPathProperty("countTags", "count(/ss:Sensor/ss:Observation/ss:Tag)", 
    XPathConstants.NUMBER, "int");

sensorcfg.addXPathProperty("idarray", "//ss:Tag/ss:ID", 
    XPathConstants.NODESET, "String[]");

sensorcfg.addXPathPropertyFragment("tagOne", "//ss:Tag[position() = 1]", 
    XPathConstants.NODE, "TagEvent");
sensorcfg.addXPathPropertyFragment("tagArray", "//ss:Tag", 
    XPathConstants.NODESET, "TagEvent");

ConfigurationEventTypeXMLDOM tagcfg = new ConfigurationEventTypeXMLDOM();
tagcfg.setRootElementName("//Tag");
tagcfg.setSchemaResource(schemaURL);
epAdministrator.getConfiguration()
    .addEventType("TagEvent", tagcfg);

insert into TagOneStream select tagOne.* from SensorEvent

select ID from TagOneStream

insert into TagArrayStream select tagArray as mytags from SensorEvent

select mytags[0].ID from TagArrayStream

Statements that aggregate events via aggregation functions also post remove stream events as aggregated values change.

Consider the following statement that alerts when 2 Withdrawal events have been received:

select count(*) as mycount from Withdrawal having count(*) = 2

When the engine encounters the second withdrawal event, the engine posts a new event to update listeners. The value of the "mycount" property on that new event is 2. Additionally, when the engine encounters the third Withdrawal event, it posts an old event to update listeners containing the prior value of the count, if specifing the rstream keyword in the select clause to select the remove stream. The value of the "mycount" property on that old event is also 2.

Note the statement above does not specify a data window and thereby counts all arriving events since statement start. The statement above retains no events and its memory allocation is only the aggregation state, i.e. a single long value to represent count(*).

The istream or rstream keyword can be used to eliminate either new events or old events posted to listeners. The next statement uses the istream keyword causing the engine to call the listener only once when the second Withdrawal event is received:

select istream count(*) as mycount from Withdrawal having count(*) = 2

Following SQL (Standard Query Language) standards for queries against relational databases, the presence or absence of aggregation functions and the presence or absence of the group by clause dictates the number of rows posted by the engine to listeners. The next sections outline the output model for batched events under aggregation and grouping. The examples also apply to data windows that don't batch events and post results continously as events arrive or leave data windows. The examples also apply to patterns providing events when a complete pattern matches.

In summary, as in SQL, if your query only selects aggregation values, the engine provides one row of aggregated values. It provides that row every time the aggregation is updated (insert stream), which is when events arrive or a batch of events gets processed, and when the events leave a data window or a new batch of events arrives. The remove stream then consists of prior aggregation values.

Also as in SQL, if your query selects non-aggregated values along with aggregation values in the select clause, the engine provides a row per event. The insert stream then consists of the aggregation values at the time the event arrives, while the remove stream is the aggregation value at the time the event leaves a data window, if any is defined in your query.

The documentation provides output examples for query types in Appendix A, Output Reference and Samples, and the next sections outlines each query type.

select * from Withdrawal.win:time_batch(1 sec)

select sum(amount) 
from Withdrawal.win:time_batch(1 sec)

select account, sum(amount) 
from Withdrawal.win:time_batch(1 sec)

select account, sum(amount) 
from Withdrawal.win:time_batch(1 sec) 
group by account

select account, accountName, sum(amount) 
from Withdrawal.win:time_batch(1 sec) 
group by account

Each of the context declarations makes available a set of built-in context properties as well as initiating event or pattern properties, as applicable. You may select these context properties for output or use them in any of the statement expressions.

Refer to built-in context properties as context.property_name, wherein property_name refers to the name of the built-in context property.

Refer to initiating event or pattern match event properties as context.stream_name.property_name, wherein stream_name refers to the name assigned to the event or the tag name specified in a pattern and property_name refers to the name of the initiating event or pattern match event property.

This context assigns events to context partitions based on the values of one or more event properties, using the value of these property(s) as a key that picks a unique context partition directly. Each event thus belongs to exactly one context partition or zero context partitions if the event does not match the optional filter predicate expression(s). Each context partition handles exactly one set of key values.

The syntax for creating a keyed segmented context is as follows:

create context context_name partition [by]
  event_property [and event_property [and ...]] from stream_def
  [, event_property [...] from stream_def]
  [, ...]

The context_name you assign to the context can be any identifier.

Following the context name is one or more lists of event properties and a stream definition for each entry, separated by comma (,).

The event_property is the name(s) of the event properties that provide the value(s) to pick a unique partition. Multiple event property names are separated by the and keyword.

The stream_def is a stream definition which consists of an event type name optionally followed by parenthesis that contains filter expressions. If providing filter expressions, only events matching the provided filter expressions for that event type are considered by context partitions. The name of a named window is not allowed.

You may list multiple event properties for each stream definition. You may list multiple stream definitions. Please refer to usage guidelines below when specifying multiple event properties and/or multiple stream definitions.

The next statement creates a context SegmentedByCustomer that considers the value of the custId property of the BankTxn event type to pick the context partition to assign events to:

create context SegmentedByCustomer partition by custId from BankTxn

The following statement refers to the context created as above to compute a total withdrawal amount per account for each customer:

context SegmentedByCustomer
select custId, account, sum(amount) from BankTxn group by account

The following statement refers to the context created as above and detects a withdrawal of more then 400 followed by a second withdrawal of more then 400 that occur within 10 minutes of the first withdrawal, all for the same customer:

context SegmentedByCustomer
select * from pattern [
  every a=BankTxn(amount > 400) -> b=BankTxn(amount > 400) where timer:within(10 minutes)
]

The EPL statement that refers to a keyed segmented context must have at least one filter expression, at any place within the EPL statement that looks for events of any of the event types listed in the context declaration.

For example, the following is not valid:

// Neither LoginEvent nor LogoutEvent are listed in the context declaration
context SegmentedByCustomer
select * from pattern [every a=LoginEvent -> b=LogoutEvent where timer:within(10 minutes)]

// Not valid
create context SegmentedByCustomer partition by custId from BankTxn, account from BankTxn

// Invalid: Type mismatch between properties
create context SegmentedByCustomer partition by custId from BankTxn, loginTime from LoginEvent

create context SegmentedByCustomer partition by 
  custId from BankTxn, loginId from LoginEvent, loginId from LogoutEvent

create context SegmentedByCustomer partition by 
  custId from BankTxn, loginId from LoginEvent(failed=false)

create context ByCustomerAndAccount partition by custId and account from BankTxn

context ByCustomerAndAccount select custId, account, sum(amount) from BankTxn

create context ByCustomerAndAccount partition by custId and account from BankTxn

context ByCustomerAndAccount
  select context.name, context.id, context.key1, context.key2 from BankTxn

create context ByCust partition by custId from BankTxn

context ByCust 
  select * from BankTxn as t1 unidirectional, BankTxn.win:time(30) t2 
  where t1.amount = t2.amount

context ByCust
  select * from SecurityEvent as t1 unidirectional, BankTxn.win:time(30) t2 
  where t1.customerName = t2.customerName

This context assigns events to context partitions based on result of a hash function and modulo operation. Each event thus belongs to exactly one context partition or zero context partitions if the event does not match the optional filter predicate expression(s). Each context partition handles exactly one result of hash value modulo granularity.

The syntax for creating a hashed segmented context is as follows:

create context context_name coalesce [by] 
  hash_func_name(hash_func_param) from stream_def
  [, hash_func_name(hash_func_param) from stream_def ]
  [, ...]
  granularity granularity_value
  [preallocate]

The context_name you assign to the context can be any identifier.

Following the context name is one or more lists of hash function name and parameters pairs and a stream definition for each entry, separated by comma (,).

The hash_func_name can either be consistent_hash_crc32 or hash_code or a plug-in single-row function. The hash_func_param is a list of parameter expressions.

The stream_def is a stream definition which consists of an event type name optionally followed by parenthesis that contains filter expressions. If providing filter expressions, only events matching the provided filter expressions for that event type are considered by context partitions. The name of a named window is not allowed.

You may list multiple stream definitions. Please refer to usage guidelines below when specifying multiple stream definitions.

The granularity is required and is an integer number that defines the maximum number of context partitions. The engine computes hash code modulo granularity hash(params) mod granularity to determine the context partition. When you specify the hash_code function the engine uses the object hash code and the computation is params.hashCode() %granularity.

Since the engine locks on the level of context partition to protect state, the granularity defines the maximum degree of parallelism. For example, a granularity of 1024 means that 1024 context partitions handle events and thus a maximum 1024 threads can process each assigned statement concurrently.

The optional preallocate keyword instructs the engine to allocate all context partitions at once at the time a statement refers to the context. This is beneficial for performance as the engine does not need to determine whether a context partition exists and dynamically allocate, but may require more memory.

The next statement creates a context SegmentedByCustomerHash that considers the CRC-32 hash code of the custId property of the BankTxn event type to pick the context partition to assign events to, with up to 16 different context partitions that are preallocated:

create context SegmentedByCustomerHash
  coalesce by consistent_hash_crc32(custId) from BankTxn granularity 16 preallocate

The following statement refers to the context created as above to compute a total withdrawal amount per account for each customer:

context SegmentedByCustomerHash
select custId, account, sum(amount) from BankTxn group by custId, account

Note that the statement above groups by custId: Since the events for different customer ids can be assigned to the same context partition, it is necessary that the EPL statement also groups by customer id.

The context declaration shown next assumes that the application provides a computeHash single-row function that accepts BankTxn as a parameter, wherein the result of this function must be an integer value that returns the context partition id for each event:

create context MyHashContext
  coalesce by computeHash(*) from BankTxn granularity 16 preallocate

The EPL statement that refers to a hash segmented context must have at least one filter expression, at any place within the EPL statement that looks for events of any of the event types listed in the context declaration.

For example, the following is not valid:

// Neither LoginEvent nor LogoutEvent are listed in the context declaration
context SegmentedByCustomerHash
select * from pattern [every a=LoginEvent -> b=LogoutEvent where timer:within(10 minutes)]

create context HashedByCustomer as coalesce
  consistent_hash_crc32(custId) from BankTxn,
  consistent_hash_crc32(loginId) from LoginEvent, 
  consistent_hash_crc32(loginId) from LogoutEvent
  granularity 32 preallocate

create context HashedByCustomer 
  coalesce consistent_hash_crc32(loginId) from LoginEvent(failed = false) 
  granularity 1024 preallocate

create context ByCustomerHash coalesce consistent_hash_crc32(custId) from BankTxn granularity 1024

context ByCustomerHash
  select context.name, context.id from BankTxn

This context assigns events to context partitions based on the values of one or more event properties, using a predicate expression(s) to define context partition membership. Each event can thus belong to zero, one or many context partitions depending on the outcome of the predicate expression(s).

The syntax for creating a category segmented context is as follows:

create context context_name
  group [by] group_expression as category_label
  [, group [by] group_expression as category_label]
  [, ...]
  from stream_def

The context_name you assign to the context can be any identifier.

Following the context name is a list of groups separated by the group keyword. The list of group is followed by the from keyword and a stream definition.

The group_expression is an expression that categorizes events. Each group expression must be followed by the as keyword and a category label which can be any identifier.

Group expressions are predicate expression and must return a Boolean true or false when applied to an event. For a given event, any number of the group expressions may return true thus categories can be overlapping.

The stream_def is a stream definition which consists of an event type name optionally followed by parenthesis that contains filter expressions. If providing filter expressions, only events matching the provided filter expressions for that event type are considered by context partitions.

The next statement creates a context CategoryByTemp that consider the value of the temperature property of the SensorEvent event type to pick context partitions to assign events to:

create context CategoryByTemp
  group temp < 65 as cold,
  group temp between 65 and 85 as normal,
  group temp > 85 as large
  from SensorEvent

The following statement simply counts, for each category, the number of events and outputs the category label and count:

context CategoryByTemp select context.label, count(*) from SensorEvent

context CategoryByTemp
  select context.name, context.id, context.label from SensorEvent

You may declare a non-overlapping context that exists once or that repeats in a regular fashion as controlled by start and end conditions. The number of context partitions is always either one or zero: Context partitions do not overlap.

The syntax for creating a non-overlapping context is as follows:

create context context_name
  start (@now | start_condition) 
  end end_condition

The context_name you assign to the context can be any identifier.

Following the context name is the start keyword, either @now or a start_condition, the end keyword and an end_condition.

Both the start (if specified) and end condition can be an event filter, a pattern, a crontab or a time period. The syntax of start and end conditions is described in Section 4.2.7, “Context Conditions”.

Once the start condition occurs, the engine no longer observes the start condition and begins observing the end condition. Once the end condition occurs, the engine observes the start condition again. If you specified @now instead of a start condition, the engine begins observing the end condition instead.

If you specified an event filter as the start condition, then the event also counts towards the statement(s) that refer to that context. If you specified a pattern as the start condition, then the events that may constitute the pattern match can also count towards the statement(s) that refer to the context provided that @inclusive and event tags are both specified (see below).

At the time of context activation when your application creates a statement that utilizes the context, the engine checks whether the start and end condition are crontab expressions. The engine evaluates the start and end crontab expressions and determines whether the current time is a time between start and end. If the current time is between start and end times, the engine allocates the context partition and waits for observing the end time. Otherwise the engine waits to observe the start time and does not allocate a context partition.

The built-in context properties that are available are the same as described in Section 4.2.6.2, “Built-In Context Properties”.

The next statement creates a context NineToFive that declares a daily time period that starts at 9 am and ends at 5 pm:

create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

The following statement outputs speed violations between 9 am and 5 pm, considering a speed of 100 or greater as a violation:

context NineToFive select * from TrafficEvent(speed >= 100)

The example that follows demonstrates the use of an event filter as the start condition and a pattern as the end condition.

The next statement creates a context PowerOutage that starts when the first PowerOutageEvent event arrives and that ends 5 seconds after a subsequent PowerOnEvent arrives:

create context PowerOutage start PowerOutageEvent end pattern [PowerOnEvent -> timer:interval(5)]

The following statement outputs the temperature during a power outage and for 5 seconds after the power comes on:

context PowerOutage select * from TemperatureEvent

To output only the last value when a context partition ends (terminates, expires), please read on to the description of output rate limiting.

The next statement creates a context Every15Minutes that starts immediately and lasts for 15 minutes, repeatedly allocating a new context partition at the end of 15 minute intervals:

create context Every15Minutes start @now end after 15 minutes

create context MyCtx start MyStartEvent end MyEndEvent

context MyCtx select count(*) as cnt from MyEndEvent output when terminated

This context initiates a new context partition when an initiating condition occurs, and terminates one or more context partitions when the terminating condition occurs. The engine maintains as many context partitions as the initiating condition fired, and discards context partitions that terminate when the termination condition fires.

The syntax for creating an overlapping context is as follows:

create context context_name
  initiated [by] [distinct (distinct_value_expr [,...])] [@now and] initiating_condition
  terminated [by] terminating_condition

The context_name you assign to the context can be any identifier.

Following the context name is the initiated keyword. After the initiated keyword you can optionally specify the distinct keyword and, within parenthesis, list one or more distinct value expressions. After the initiated keyword you can also specify @now and as explained below.

After the initiated keyword you must specify the initiating condition. It follows the terminated keyword followed by the terminating condition.

Both the initiating and terminating condition can be an event filter, a pattern, a crontab or a time period. The syntax of initiating and terminating conditions is described in Section 4.2.7, “Context Conditions”.

If you specified @now and before the initiating condition then the engine initiates a new context partition immediately. The @now is only allowed in conjunction with initiation conditions that specify a pattern, crontab or time period and not with event filters.

If you specified an event filter for the initiating condition, then the event that initiates a new context partition also counts towards the statement(s) that refer to that context. If you specified a pattern to initiate a new context partition, then the events that may constitute the pattern match can also count towards the statement(s) that refer to the context provided that @inclusive and event tags are both specified (see below).

The next statement creates a context CtxTrainEnter that allocates a new context partition when a train enters a station, and that terminates each context partition 5 minutes after the time the context partition was allocated:

create context CtxTrainEnter
  initiated by TrainEnterEvent as te
  terminated after 5 minutes

The context declared above assigns the stream name te. Thereby the initiating event's properties can be accessed, for example, by specifying context.te.trainId.

The following statement detects when a train enters a station as indicated by a TrainEnterEvent, but does not leave the station within 5 minutes as would be indicated by a matching TrainLeaveEvent:

context CtxTrainEnter
select t1 from pattern [
  t1=TrainEnterEvent -> timer:interval(5 min) and not TrainLeaveEvent(trainId = context.te.trainId)
  ]

Since the TrainEnterEvent that initiates a new context partition also counts towards the statement, the first part of the pattern (the t1=TrainEnterEvent) is satisfied by that initiating event.

The next statement creates a context CtxEachMinute that allocates a new context partition immediately and every 1 minute, and that terminates each context partition 1 minute after the time the context partition was allocated:

create context CtxEachMinute
  initiated @now and pattern [every timer:interval(1 minute)]
  terminated after 1 minutes

The statement above specifies @now to instruct the engine to allocate a new context partition immediately as well as when the pattern fires. Without the @now the engine would only allocate a new context partition when the pattern fires after 1 minute and every minute thereafter.

The following statement averages the temperature, starting anew every 1 minute and outputs the aggregate value continuously:

context CtxEachMinute select avg(temp) from SensorEvent

To output only the last value when a context partition ends (terminates, expires), please read on to the description of output rate limiting.

create context OrderContext
  initiated by distinct(orderId) NewOrderEvent as newOrder 
  terminated by CloseOrderEvent(closeOrderId = newOrder.orderId)

context NineToFive 
select context.name, context.startTime, context.endTime from TrafficEvent(speed >= 100)

context CtxTrainEnter
select *, context.te.trainId, context.id, context.name
from TrainLeaveEvent(trainId = context.te.trainId)

Context start/initiating and end/terminating conditions are for use with overlapping and non-overlapping contexts. Any combination of conditions may be specified.

event_stream_name [(filter_criteria)] [as stream_name]

// A non-overlapping context that starts when MyStartEvent arrives and ends when MyEndEvent arrives
create context MyContext start MyStartEvent end MyEndEvent

// An overlapping context where each MyEvent with level greater zero 
// initiates a new context partition that terminates after 10 seconds
create context MyContext initiated MyEvent(level > 0) terminated after 10 seconds

// A non-overlapping context that starts when MyEvent arrives
// and ends when a matching MyEvent arrives (same id)
create context MyContext 
start MyEvent as myevent
end MyEvent(id=myevent.id)

// An overlapping context where each MyInitEvent initiates a new context partition 
// that terminates when a matching MyTermEvent arrives 
create context MyContext 
initiated by MyInitEvent as e1 
terminated by MyTermEvent(id=e1.id, level <> e1.level)

pattern [pattern_expression] [@inclusive]

// A non-overlapping context that starts when either StartEventOne or StartEventTwo arrive
// and that ends after 5 seconds.
// Here neither StartEventOne or StartEventTwo count towards any statements
// that are referring to the context.
create context MyContext 
  start pattern [StartEventOne or StartEventTwo] 
  end after 5 seconds

// Same as above. 
// Here both StartEventOne or StartEventTwo do count towards any statements
// that are referring to the context.
create context MyContext 
  start pattern [a=StartEventOne or b=StartEventTwo] @inclusive 
  end after 5 seconds

// An overlapping context where each distinct MyInitEvent initiates a new context 
// and each context partition terminates after 20 seconds
// We use @inclusive to say that the same MyInitEvent that fires the pattern
// also applies to statements that are associated to the context.
create context MyContext
  initiated by pattern [every-distinct(a.id, 20 sec) a=MyInitEvent]@inclusive
  terminated after 20 sec

// An overlapping context where each pattern match initiates a new context 
// and all context partitions terminate when MyTermEvent arrives.
// The MyInitEvent and MyOtherEvent that trigger the pattern are themselves not included 
// in any statements that are associated to the context.
create context MyContext
  initiated by pattern [every MyInitEvent -> MyOtherEvent where timer:within(5)]
  terminated by MyTermEvent

// A non-overlapping context that starts when either StartEventOne or StartEventTwo arrive
// and that ends when either a matching EndEventOne or EndEventTwo arrive
create context MyContext 
  start pattern [a=StartEventOne or b=StartEventTwo]@inclusive
  end pattern [EndEventOne(id=a.id) or EndEventTwo(id=b.id)]

// A non-overlapping context started daily between 9 am to 5 pm
// and not started outside of these hours:
create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

// An overlapping context where crontab initiates a new context every 1 minute
// and each context partition terminates after 10 seconds:
create context MyContext initiated (*, *, *, *, *) terminated after 10 seconds

after time_period_expression

// A non-overlapping context started after 10 seconds 
// that ends 1 minute after it starts and that again starts 10 seconds thereafter.
create context NonOverlap10SecFor1Min start after 10 seconds end after 1 minute

// An overlapping context that starts a new context partition every 5 seconds
// and each context partition lasts 1 minute
create context Overlap5SecFor1Min initiated after 5 seconds terminated after 1 minute

Context properties of all nested contexts are available for use. Specify context.nested_context_name.property_name or if nested context declaration provided stream names or tags for patterns then context.nested_context_name.stream_name.property_name.

For example, consider the CtxNestedTrainEnter context declared earlier. The following statement selects a few of the context properties:

context CtxNestedTrainEnter
select context.InitCtx.te.trainId, context.HashCtx.id,
  tagId, count(*) from PassengerScanEvent group by tagId

In a second example, consider the NineToFiveSegmented context declared earlier. The following statement selects a few of the context properties:

context NineToFiveSegmented
select context.NineToFive.startTime, context.SegmentedByCustomer.key1 from BankTxn

The following context properties are available in your EPL statement when it refers to a nested context:

context NineToFiveSegmented select context.name, context.id from BankTxn

Time-based windows as well as pattern observers and guards take a time period as a parameter. Time periods follow the syntax below.

time-period : [year-part] [month-part] [week-part] [day-part] [hour-part] 
      [minute-part] [seconds-part] [milliseconds-part]

year-part : (number|variable_name) ("years" | "year")
month-part : (number|variable_name) ("months" | "month")
week-part : (number|variable_name) ("weeks" | "week")
day-part : (number|variable_name) ("days" | "day")
hour-part : (number|variable_name) ("hours" | "hour")
minute-part : (number|variable_name) ("minutes" | "minute" | "min")
seconds-part : (number|variable_name) ("seconds" | "second" | "sec")
milliseconds-part : (number|variable_name) ("milliseconds" | "millisecond" | "msec")

Some examples of time periods are:

10 seconds
10 minutes 30 seconds
20 sec 100 msec
1 day 2 hours 20 minutes 15 seconds 110 milliseconds
0.5 minutes
1 year
1 year 1 month

Variable names and substitution parameters '?' for prepared statements are also allowed as part of a time period expression.

A unit in the month part is equivalent to 30 days.

Comments can appear anywhere in the EPL or pattern statement text where whitespace is allowed. Comments can be written in two ways: slash-slash (// ...) comments and slash-star (/* ... */) comments.

Slash-slash comments extend to the end of the line:

// This comment extends to the end of the line.
// Two forward slashes with no whitespace between them begin such comments.

select * from MyEvent  // this is a slash-slash comment

// All of this text together is a valid statement.

Slash-star comments can span multiple lines:

/* This comment is a "slash-star" comment that spans multiple lines.
 * It begins with the slash-star sequence with no space between the '/' and '*' characters.
 * By convention, subsequent lines can begin with a star and are aligned, but this is 
 * not required.
 */		
select * from MyEvent  /* this also works */

Comments styles can also be mixed:

select field1, // first comment
  /* second comment*/  field2
  from MyEvent

Certain words such as select, delete or set are reserved and may not be used as identifiers. Please consult Appendix B, Reserved Keywords for the list of reserved keywords and permitted keywords.

Names of built-in functions and certain auxiliary keywords are permitted as event property names and in the rename syntax of the select clause. For example, count is acceptable.

Consider the example below, which assumes that 'last' is an event property of MyEvent:

// valid
select last, count(*) as count from MyEvent

This example shows an incorrect use of a reserved keyword:

// invalid
select insert from MyEvent

EPL offers an escape syntax for reserved keywords: Event properties as well as event or stream names may be escaped via the backwards apostrophe ` (ASCII 96) character.

The next example queries an event type by name Order (a reserved keyword) that provides a property by name insert (a reserved keyword):

// valid
select `insert` from `Order`

Escaping event property names is described in Section 2.2.1, “Escape Characters”.

The sample EPL below escapes the single quote in the string constant John's, and filters out order events where the name value matches:

select * from OrderEvent(name='John\'s')
// ...equivalent to...
select * from OrderEvent(name='John\u0027s')

The next EPL escapes the string constant Quote "Hello":

select * from OrderEvent(description like "Quote \"Hello\"")
// is equivalent to
select * from OrderEvent(description like "Quote \u0022Hello\u0022")

When building an escape string via the API, escape the backslash, as shown in below code snippet:

epService.getEPAdministrator().createEPL("select * from OrderEvent(name='John\\'s')");
// ... and for double quotes...
epService.getEPAdministrator().createEPL("select * from OrderEvent(
  description like \"Quote \\\"Hello\\\"\")");

EPL returns double-type values for division regardless of operand type. EPL can also be configured to follow Java rules for integer arithmetic instead as described in Section 15.4.22, “Engine Settings related to Expression Evaluation”.

Division by zero returns positive or negative infinity. Division by zero can be configured to return null instead.

select 'volume' as field1,
   "sleep" as field2, 
  "\u0041" as unicodeA

select true as field1, 
  false as field2

select 1 as field1, 
  -1 as field2, 
  1e2 as field3

select 1L as field1, 
  1l as field2

select 1.67 as field1, 
  167e-2 as field2, 
  1.67d as field3

select 1.2f as field1, 
  1.2F as field2

select 0x10 as field1

For declaring expressions that are visible across multiple EPL statements i.e. globally visible expressions please consult Section 5.19.1, “Declaring a Global Expression” that explains the create expression clause.

An expression declaration follows the lambda-style expression syntax. This syntax was chosen as it typically allows for a shorter and more concise expression body that can be easier to read then most procedural code.

The syntax for an expression declaration is:

expression expression_name { expression_body }

An expression declaration consists of the expression name and an expression body. The expression_name is any identifier. The expression_body contains optional parameters and the expression. The parameter types and the return type of the expression is determined by the engine and do not need to be specified.

Parameters to a declared expression can be a stream name, pattern tag name or wildcard (*). Use wildcard to pass the event itself to the expression. In a join or subquery, or more generally in an expression where multiple streams or pattern tags are available, the EPL must specify the stream name or pattern tag name and cannot use wildcard.

In the expression body the => lambda operator reads as "goes to". The left side of the lambda operator specifies the input parameters (if any) and the right side holds the expression. The lambda expression x => x * x is read "x goes to x times x".

In the expression body, if your expression takes no parameters, you may simply specify the expression and do not need the => lambda operator.

If your expression takes one parameters, specify the input parameter name followed by the => lambda operator and followed by the expression. The synopsis for use with a single input parameter is:

expression_body:   input_param_name => expression 

If your expression takes two or more parameters, specify the input parameter names in parenthesis followed by the => lambda operator followed by the expression. The synopsis for use with a multiple input parameter is:

expression_body:   (input_param [,input_param [,...]]) => expression 

The following example declares an expression that returns two times PI (ratio of the circumference of a circle to its diameter) and demonstrates its use in a select-clause:

expression twoPI { Math.PI * 2} select twoPI() from SampleEvent

The next example declares an expression that accepts one parameter: a MarketData event. The expression computes a new "mid" price based on the buy and sell price:

expression midPrice { x => (x.buy + x.sell) / 2 } 
select midPrice(md) from MarketDataEvent as md

The variable name can be left off if event property names resolve without ambiguity.

This example EPL removes the variable name x:

expression midPrice { x => (buy + sell) / 2 } 
select midPrice(md) from MarketDataEvent as md

The next example EPL specifies wildcard instead:

expression midPrice { x => (buy + sell) / 2 } 
select midPrice(*) from MarketDataEvent

A further example that demonstrates two parameters is listed next. The example joins two streams and uses the price value from MarketDataEvent and the sentiment value of NewsEvent to compute a weighted sentiment:

expression weightedSentiment { (x, y) => x.price * y.sentiment } 
select weightedSentiment(md, news) 
from MarketDataEvent.std:lastevent() as md, NewsEvent.std:lastevent() news

Any expression can be used in the expression body including aggregations, variables, subqueries or further declared expressions. Sub-queries, when used without in or exists, must be placed within parenthesis.

An example subquery within a declared expression is shown next:

expression newsSubq(md) { 
    (select sentiment from NewsEvent.std:unique(symbol) where symbol = md.symbol) 
} 
select newsSubq(mdstream)
from MarketDataEvent mdstream

When using declared expressions please note these limitations:

The following scope rules apply for declared expressions:

Please see Chapter 18, Script Support for more information.

You may refer to a context in the EPL text by specifying the context keyword followed by a context name. Context are described in more detail at Chapter 4, Context and Context Partitions

The effect of referring to a context is that your statement operates according to the context dimensional information as declared for the context.

The synopsis is:

... context context_name ...

You may refer to a context in all statements except for the following types of statements:

The syntax for selecting all event properties in a stream is:

select * from stream_def

The following statement selects StockTick events for the last 30 seconds of IBM stock ticks.

select * from StockTick(symbol='IBM').win:time(30 sec)

You may well be asking: Why does the statement specify a time window here? First, the statement is meant to demonstrate the use of * wildcard. When the engine pushes statement results to your listener and as the statement does not select remove stream events via rstream keyword, the listener receives only new events and the time window could be left off. By adding the time window the pull API (iterator API or JDBC driver) returns the last 30 seconds of events.

The * wildcard and expressions can also be combined in a select clause. The combination selects all event properties and in addition the computed values as specified by any additional expressions that are part of the select clause. Here is an example that selects all properties of stock tick events plus a computed product of price and volume that the statement names 'pricevolume':

select *, price * volume as pricevolume from StockTick

When using wildcard (*), Esper does not actually read or copy your event properties out of your event or events, neither does it copy the event object. It simply wraps your native type in an EventBean interface. Your application has access to the underlying event object through the getUnderlying method and has access to the property values through the get method.

In a join statement, using the select * syntax selects one event property per stream to hold the event for that stream. The property name is the stream name in the from clause.

To choose the particular event properties to return:

select event_property [, event_property] [, ...] from stream_def

The following statement simply selects the symbol and price properties of stock ticks, and the total volume for stock tick events in a 60-second time window.

select symbol, price, sum(volume) from StockTick(symbol='IBM').win:time(60 sec)

The following statement declares a further view onto the event stream of stock ticks: the univariate statistics view (stat:uni). The statement selects the properties that this view derives from the stream, for the last 100 events of IBM stock ticks in the length window.

select datapoints, total, average, variance, stddev, stddevpa
from StockTick(symbol='IBM').win:length(100).stat:uni(volume)

The select clause can contain one or more expressions.

select expression [, expression] [, ...] from stream_def

The following statement selects the volume multiplied by price for a time batch of the last 30 seconds of stock tick events.

select volume * price from StockTick.win:time_batch(30 sec)

Event properties and expressions can be renamed using below syntax.

select [event_property | expression] as identifier [, ...]

The following statement selects volume multiplied by price and specifies the name volPrice for the resulting column.

select volume * price as volPrice from StockTick

Identifiers cannot contain the "." (dot) character, i.e. "vol.price" is not a valid identifier for the rename syntax.

If your statement is joining multiple streams, your may specify property names that are unique among the joined streams, or use wildcard (*) as explained earlier.

In case the property name in your select or other clauses is not unique considering all joined streams, you will need to use the name of the stream as a prefix to the property.

This example is a join between the two streams StockTick and News, respectively named as 'tick' and 'news'. The example selects from the StockTick event the symbol value using the 'tick' stream name as a prefix:

select tick.symbol from StockTick.win:time(10) as tick, News.win:time(10) as news
where news.symbol = tick.symbol

Use the wildcard (*) selector in a join to generate a property for each stream, with the property value being the event itself. The output events of the statement below have two properties: the 'tick' property holds the StockTick event and the 'news' property holds the News event:

select * from StockTick.win:time(10) as tick, News.win:time(10) as news

The following syntax can also be used to specify what stream's properties to select:

select stream_name.* [as name] from ...

The selection of tick.* selects the StockTick stream events only:

select tick.* from StockTick.win:time(10) as tick, News.win:time(10) as news
where tick.symbol = news.symbol

The next example uses the as keyword to name each stream's joined events. This instructs the engine to create a property for each named event:

select tick.* as stocktick, news.* as news 
from StockTick.win:time(10) as tick, News.win:time(10) as news
where stock.symbol = news.symbol

The output events of the above example have two properties 'stocktick' and 'news' that are the StockTick and News events.

The stream name itself, as further described in Section 5.4.5, “Using the Stream Name”, may be used within expressions or alone.

This example passes events to a user-defined function named compute and also shows insert-into to populate an event stream of combined events:

insert into TickNewStream select tick, news, MyLib.compute(news, tick) as result
from StockTick.win:time(10) as tick, News.win:time(10) as news
where tick.symbol = news.symbol

// second statement that uses the TickNewStream stream
select tick.price, news.text, result from TickNewStream

In summary, the stream_name.* streamname wildcard syntax can be used to select a stream as the underlying event or as a property, but cannot appear within an expression. While the stream_name syntax (without wildcard) always selects a property (and not as an underlying event), and can occur anywhere within an expression.

If your statement employs pattern expressions, then your pattern expression tags events with a tag name. Each tag name becomes available for use as a property in the select clause and all other clauses.

For example, here is a very simple pattern that matches on every StockTick event received within 30 seconds after start of the statement. The sample selects the symbol and price properties of the matching events:

select tick.symbol as symbol, tick.price as price
from pattern[every tick=StockTick where timer:within(10 sec)]

The use of the wildcard selector, as shown in the next statement, creates a property for each tagged event in the output. The next statement outputs events that hold a single 'tick' property whose value is the event itself:

select * from pattern[every tick=StockTick where timer:within(10 sec)]

You may also select the matching event itself using the tick.* syntax. The engine outputs the StockTick event itself to listeners:

select tick.* from pattern[every tick=StockTick where timer:within(10 sec)]

A tag name as specified in a pattern is a valid expression itself. This example uses the insert into clause to make available the events matched by a pattern to further statements:

// make a new stream of ticks and news available
insert into StockTickAndNews 
select tick, news from pattern [every tick=StockTick -> news=News(symbol=tick.symbol)]

// second statement to select from the stream of ticks and news
select tick.symbol, tick.price, news.text from StockTickAndNews

The optional istream, irstream and rstream keywords in the select clause control the event streams posted to listeners and observers to a statement.

If neither keyword is specified, and in the default engine configuration, the engine posts only insert stream events via the newEvents parameter to the update method of UpdateListener instances listening to the statement. The engine does not post remove stream events, by default.

The insert stream consists of the events entering the respective window(s) or stream(s) or aggregations, while the remove stream consists of the events leaving the respective window(s) or the changed aggregation result. See Chapter 3, Processing Model for more information on insert and remove streams.

The engine posts remove stream events to the oldEvents parameter of the update method only if the irstream keyword occurs in the select clause. This behavior can be changed via engine-wide configuration as described in Section 15.4.17, “Engine Settings related to Stream Selection”.

By specifying the istream keyword you can instruct the engine to only post insert stream events via the newEvents parameter to the update method on listeners. The engine will then not post any remove stream events, and the oldEvents parameter is always a null value.

By specifying the irstream keyword you can instruct the engine to post both insert stream and remove stream events.

By specifying the rstream keyword you can instruct the engine to only post remove stream events via the newEvents parameter to the update method on listeners. The engine will then not post any insert stream events, and the oldEvents parameter is also always a null value.

The following statement selects only the events that are leaving the 30 second time window.

select rstream * from StockTick.win:time(30 sec)

The istream and rstream keywords in the select clause are matched by same-name keywords available in the insert into clause. While the keywords in the select clause control the event stream posted to listeners to the statement, the same keywords in the insert into clause specify the event stream that the engine makes available to other statements.

Property or column names can optionally be qualified by a stream name and the provider URI. The syntax is:

[[provider_URI.]stream_name.]property_name

The provider_URI is the URI supplied to the EPServiceProviderManager class, or the string default for the default provider.

This example assumes the provider is the default provider:

select MyEvent.myProperty from MyEvent
// ... equivalent to ...
select default.MyEvent.myProperty from MyEvent

Stream names can also be qualified by the provider URI. The syntax is:

[provider_URI.]stream_name

The next example assumes a provider URI by name of Processor:

select Processor.MyEvent.myProperty from Processor.MyEvent

The optional distinct keyword removes duplicate output events from output. The keyword must occur after the select keyword and after the optional irstream keyword.

The distinct keyword in your select instructs the engine to consolidate, at time of output, the output event(s) and remove output events with identical property values. Duplicate removal only takes place when two or more events are output together at any one time, therefore distinct is typically used with a batch data window, output rate limiting, on-demand queries, on-select or iterator pull API.

If two or more output event objects have same property values for all properties of the event, the distinct removes all but one duplicated event before outputting events to listeners. Indexed, nested and mapped properties are considered in the comparison, if present in the output event.

The next example outputs sensor ids of temperature sensor events, but only every 10 seconds and only unique sensor id values during the 10 seconds:

select distinct sensorId from TemperatureSensorEvent output every 10 seconds

Use distinct with wildcard (*) to remove duplicate output events considering all properties of an event.

This example statement outputs all distinct events either when 100 events arrive or when 10 seconds passed, whichever occurs first:

select distinct * from TemperatureSensorEvent.win:time_length_batch(10, 100)

When selecting nested, indexed, mapped or dynamic properties in a select clause with distinct, it is relevant to know that the comparison uses hash code and the Java equals semantics.

For transposing an instance of a Java object returned by an expression to a stream use the transpose function as described in Section 9.4, “Select-Clause transpose Function”.

By default, for certain select-clause expressions that output events or a collection of events, the engine outputs the underlying event objects. With outputs we refer to the data passed to listeners, subscribers and inserted-into into another stream via insert-into.

The select-clause expressions for which underlying event objects are output by default are:

To have the engine output EventBean instance(s) instead, add @eventbean to the relevant expressions of the select-clause.

The sample EPL shown below outputs current data window contents as EventBean instances into the stream OutStream, thereby statements consuming the stream may operate on such instances:

insert into OutStream 
select prevwindow(s0) @eventbean as win 
from MyEvent.win:length(2) as s0

The next EPL consumes the stream and selects the last event:

select win.lastOf() from OutStream

It is not necessary to use @eventbean if an event type by the same name (OutStream in the example) is already declared and a property exist on the type by the same name (win in this example) and the type of the property is the event type (MyEvent in the example) returned by the expression. This is further described in Section 5.10.8, “Select-Clause Expression And Inserted-Into Column Event Type”.

The stream_def syntax for a filter-based event stream is as below:

event_stream_name [(filter_criteria)] [contained_selection] [.view_spec] [.view_spec] [...]

The event_stream_name is either the name of an event type or name of an event stream populated by an insert into statement or the name of a named window.

The filter_criteria is optional and consists of a list of expressions filtering the events of the event stream, within parenthesis after the event stream name.

The contained_selection is optional and is for use with coarse-grained events that have properties that are themselves one or more events, see Section 5.20, “Contained-Event Selection” for the synopsis and examples.

The view_spec are optional view specifications, which are combinable definitions for retaining events and for deriving information from events.

The following EPL statement shows event type, filter criteria and views combined in one statement. It selects all event properties for the last 100 events of IBM stock ticks for volume. In the example, the event type is the fully qualified Java class name org.esper.example.StockTick. The expression filters for events where the property symbol has a value of "IBM". The optional view specifications for deriving data from the StockTick events are a length window and a view for computing statistics on volume. The name for the event stream is "volumeStats".

select * from 
  org.esper.example.StockTick(symbol='IBM').win:length(100).stat:uni(volume) as volumeStats

Esper filters out events in an event stream as defined by filter criteria before it sends events to subsequent views. Thus, compared to search conditions in a where clause, filter criteria remove unneeded events early. In the above example, events with a symbol other then IBM do not enter the time window.

select * from com.mypackage.myevents.RfidEvent

select * from RfidEvent

select * from org.myorg.rfid.IRfidReadable

select * from RfidEvent(category="Perishable")

select * from com.mycompany.RfidEvent(MyRFIDLib.isInRange(x, y) or (x < 0 and y < 0))

select * from RfidEvent(zone=1, category=10)
...is equivalent to...
select * from RfidEvent(zone=1 and category=10)

mypackage.RfidEvent(x in [100:200], y in [0:100])

mypackage.RfidEvent(x between 100 and 200, y between 0 and 50)

mypackage.RfidEvent(x not in [0:100])

mypackage.RfidEvent(x not between 0 and 100)

mypackage.RfidEvent(category in ('Perishable', 'Container'))

mypackage.RfidEvent(category not in ('Household', 'Electrical'))

Event pattern expressions can also be used to specify one or more event streams in an EPL statement. For pattern-based event streams, the event stream definition stream_def consists of the keyword pattern and a pattern expression in brackets []. The syntax for an event stream definition using a pattern expression is below. As in filter-based event streams, an optional list of views that derive data from the stream can be supplied.

pattern [pattern_expression] [.view_spec] [.view_spec] [...]

The next statement specifies an event stream that consists of both stock tick events and trade events. The example tags stock tick events with the name "tick" and trade events with the name "trade".

select * from pattern [every tick=StockTickEvent or every trade=TradeEvent]

This statement generates an event every time the engine receives either one of the event types. The generated events resemble a map with "tick" and "trade" keys. For stock tick events, the "tick" key value is the underlying stock tick event, and the "trade" key value is a null value. For trade events, the "trade" key value is the underlying trade event, and the "tick" key value is a null value.

Lets further refine this statement adding a view the gives us the last 30 seconds of either stock tick or trade events. Lets also select prices and a price total.

select tick.price as tickPrice, trade.price as tradePrice, 
       sum(tick.price) + sum(trade.price) as total
  from pattern [every tick=StockTickEvent or every trade=TradeEvent].win:time(30 sec)

Note that in the statement above tickPrice and tradePrice can each be null values depending on the event processed. Therefore, an aggregation function such as sum(tick.price + trade.price)) would always return null values as either of the two price properties are always a null value for any event matching the pattern. Use the coalesce function to handle null values, for example: sum(coalesce(tick.price, 0) + coalesce(trade.price, 0)).

Views are used to specify an expiry policy for events (data window views) and also to derive data. Views can be staggered onto each other. See the section Chapter 12, EPL Reference: Views on the views available that also outlines the different types of views: Data Window views and Derived-Value views.

Views can optionally take one or more parameters. These parameters are expressions themselves that may consist of any combination of variables, arithmetic, user-defined function or substitution parameters for prepared statements, for example.

The example statement below outputs a count per expressway for car location events (contains information about the location of a car on a highway) of the last 60 seconds:

select expressway, count(*) from CarLocEvent.win:time(60) 
group by expressway

The next example serves to show staggering of views. It uses the std:groupwin view to create a separate length window per car id:

select cardId, expressway, direction, segment, count(*) 
from CarLocEvent.std:groupwin(carId).win:length(4) 
group by carId, expressway, direction, segment

The first view std:groupwin(carId) groups car location events by car id. The second view win:length(4) keeps a length window of the 4 last events, with one separate length window for each car id. The example reports the number of events per car id and per expressway, direction and segment considering the last 4 events for each car id only.

Note that the group by syntax is generally preferable over std:groupwin for grouping information as it is SQL-compliant, easier to read and does not create a separate data window per group. The std:groupwin in above example creates a separate data window (length window in the example) per group, demonstrating staggering views.

When views are staggered onto each other as a chain of views, then the insert and remove stream received by each view is the insert and remove stream made available by the view (or stream) earlier in the chain.

The special keep-all view keeps all events: It does not provide a remove stream, i.e. events are not removed from the keep-all view unless by means of the on-delete syntax or by revision events.

Data window views provide an expiry policy that indicates when to remove events from the data window, with the exception of the keep-all data window which has no expiry policy and the std:groupwin grouped-window view for allocating a new data window per group.

EPL allows the freedom to use multiple data window views onto a stream and thus combine expiry policies. Combining data windows into an intersection (the default) or a union can achieve a useful strategy for retaining events and expiring events that are no longer of interest. Named windows and the on-delete syntax provide an additional degree of freedom.

In order to combine two or more data window views there is no keyword required. The retain-intersection keyword is the default and the retain-union keyword may instead be provided for a stream.

The concept of union and intersection come from Set mathematics. In the language of Set mathematics, two sets A and B can be "added" together: The intersection of A and B is the set of all things which are members of both A and B, i.e. the members two sets have "in common". The union of A and B is the set of all things which are members of either A or B.

Use the retain-intersection (the default) keyword to retain an intersection of all events as defined by two or more data windows. All events removed from any of the intersected data windows are entered into the remove stream. This is the default behavior if neither retain keyword is specified.

Use the retain-union keyword to retain a union of all events as defined by two or more data windows. Only events removed from all data windows are entered into the remove stream.

The next example statement totals the price of OrderEvent events in a union of the last 30 seconds and unique by product name:

select sum(price) from OrderEvent.win:time(30 sec).std:unique(productName) retain-union

In the above statement, all OrderEvent events that are either less then 30 seconds old or that are the last event for the product name are considered.

Here is an example statement totals the price of OrderEvent events in an intersection of the last 30 seconds and unique by product name:

select sum(price) from OrderEvent.win:time(30 sec).std:unique(productName) retain-intersection

In the above statement, only those OrderEvent events that are both less then 30 seconds old and are the last event for the product name are considered. The number of events that the engine retains is the number of unique events per product name in the last 30 seconds (and not the number of events in the last 30 seconds).

For an intersection the engine retains the minimal number of events representing that intersection. Thus when combining a time window of 30 seconds and a last-event window, for example, the number of events retained at any time is zero or one event (and not 30 seconds of events).

When combining a batch window into an intersection with another data window the combined data window gains batching semantics: Only when the batch criteria is fulfilled does the engine provide the batch of intersecting insert stream events. Multiple batch data windows may not be combined into an intersection.

In below table we provide additional examples for data window intersections:

For advanced users and for backward compatibility, it is possible to configure Esper to allow multiple data window views without either of the retain keywords, as described in Section 15.4.12.2, “Configuring Multi-Expiry Policy Defaults”.