www.espertech.comDocumentation

Chapter 2. Event Representations

2.1. Event Underlying Java Objects
2.2. Event Properties
2.2.1. Escape Characters
2.2.2. Expression as Key or Index Value
2.3. Dynamic Event Properties
2.4. Fragment and Fragment Type
2.5. Plain-Old Java Object Events
2.5.1. Java Object Event Properties
2.5.2. Property Names
2.5.3. Parameterized Types
2.5.4. Setter Methods for Indexed and Mapped Properties
2.5.5. Known Limitations
2.6. java.util.Map Events
2.6.1. Overview
2.6.2. Map Properties
2.6.3. Map Supertypes
2.6.4. Advanced Map Property Types
2.7. Object-array (Object[]) Events
2.7.1. Overview
2.7.2. Object-Array Properties
2.7.3. Object-Array Supertype
2.7.4. Advanced Object-Array Property Types
2.8. org.w3c.dom.Node XML Events
2.8.1. Schema-Provided XML Events
2.8.2. No-Schema-Provided XML Events
2.8.3. Explicitly-Configured Properties
2.9. Additional Event Representations
2.10. Updating, Merging and Versioning Events
2.11. Coarse-Grained Events
2.12. Event Objects Instantiated and Populated by Insert Into
2.13. Comparing Event Representations

This section outlines the different means to model and represent events.

Esper uses the term event type to describe the type information available for an event representation.

Your application may configure predefined event types at startup time or dynamically add event types at runtime via API or EPL syntax. See Section 16.4, “Configuration Items” for startup-time configuration and Section 15.3.8, “Runtime Configuration” for the runtime configuration API.

The EPL create schema syntax allows declaring an event type at runtime using EPL, see Section 5.15, “Declaring an Event Type: Create Schema”.

In Section 15.6, “Event and Event Type” we explain how an event type becomes visible in EPL statements and output events delivered by the engine.

An event is an immutable record of a past occurrence of an action or state change. Event properties capture the state information for an event.

In Esper, an event can be represented by any of the following underlying Java objects (NEsper .NET, see Section C.6, “.NET Event Underlying Objects”):


Esper provides multiple choices for representing an event. There is no absolute need for you to create new Java classes to represent an event.

Event representations have the following in common:

  • All event representations support nested, indexed and mapped properties (aka. property expression), as explained in more detail below. There is no limitation to the nesting level.

  • All event representations provide event type metadata. This includes type metadata for nested properties.

  • All event representations allow transposing the event itself and parts of all of its property graph into new events. The term transposing refers to selecting the event itself or event properties that are themselves nestable property graphs, and then querying the event's properties or nested property graphs in further statements. The Apache Axiom event representation is an exception and does not currently allow transposing event properties but does allow transposing the event itself.

  • The Java object, Map and Object-array representations allow supertypes.

The API behavior for all event representations is the same, with minor exceptions noted in this chapter.

The benefits of multiple event representations are:

  • For applications that already have events in one of the supported representations, there is no need to transform events into a Java object before processing.

  • Event representations are exchangeable, reducing or eliminating the need to change statements when the event representation changes.

  • Event representations are interoperable, allowing all event representations to interoperate in same or different statements.

  • The choice makes its possible to consciously trade-off performance, ease-of-use, the ability to evolve and effort needed to import or externalize events and use existing event type metadata.

Event properties capture the state information for an event. Event properties be simple as well as indexed, mapped and nested event properties. The table below outlines the different types of properties and their syntax in an event expression. This syntax allows statements to query deep JavaBean objects graphs, XML structures and Map events.


Combinations are also possible. For example, a valid combination could be person.address('home').street[0].

You may use any expression as a mapped property key or indexed property index by putting the expression within parenthesis after the mapped or index property name. Please find examples below.

The key or index expression must be placed in parenthesis. When using an expression as key for a mapped property, the expression must return a String-typed value. When using an expression as index for an indexed property, the expression must return an int-typed value.

This example below uses Java classes to illustrate;The same principles apply to all event representations.

Assume a class declares these properties (getters not shown for brevity):

public class MyEventType {
  String myMapKey;
  int myIndexValue;
  int myInnerIndexValue;
  Map<String, InnerType> innerTypesMap;	// mapped property
  InnerType[] innerTypesArray; // indexed property
}

public class InnerType {
  String name;
  int[] ids;
}

A sample EPL statement demonstrating expressions as map keys or indexes is:

select innerTypesMap('somekey'),  // returns map value for 'somekey'
  innerTypesMap(myMapKey),        // returns map value for myMapKey value (an expression)
  innerTypesArray[1],             // returns array value at index 1
  innerTypesArray(myIndexValue)   // returns array value at index myIndexValue (an expression)
  from MyEventType

The dot-operator can be used to access methods on the value objects returned by the mapped or indexed properties. By using the dot-operator the syntax follows the chained method invocation described at Section 9.6, “Dot Operator”.

A sample EPL statement demonstrating the dot-operator as well as expressions as map keys or indexes is:

 select innerTypesMap('somekey').ids[1],
  innerTypesMap(myMapKey).getIds(myIndexValue),
  innerTypesArray[1].ids[2],
  innerTypesArray(myIndexValue).getIds(myInnerIndexValue)
  from MyEventType

Please note the following limitations:

  • The square brackets-syntax for indexed properties does now allow expressions and requires a constant index value.

  • When using the dot-operator with mapped or indexed properties that have expressions as map keys or indexes you must follow the chained method invocation syntax.

Dynamic (unchecked) properties are event properties that need not be known at statement compilation time. Such properties are resolved during runtime: they provide duck typing functionality.

The idea behind dynamic properties is that for a given underlying event representation we don't always know all properties in advance. An underlying event may have additional properties that are not known at statement compilation time, that we want to query on. The concept is especially useful for events that represent rich, object-oriented domain models.

The syntax of dynamic properties consists of the property name and a question mark. Indexed, mapped and nested properties can also be dynamic properties:


Dynamic properties always return the java.lang.Object type. Also, dynamic properties return a null value if the dynamic property does not exist on events processed at runtime.

As an example, consider an OrderEvent event that provides an "item" property. The "item" property is of type Object and holds a reference to an instance of either a Service or Product.

Assume that both Service and Product classes provide a property named "price". Via a dynamic property we can specify a query that obtains the price property from either object (Service or Product):

select item.price? from OrderEvent

As a second example, assume that the Service class contains a "serviceName" property that the Product class does not possess. The following query returns the value of the "serviceName" property for Service objects. It returns a null-value for Product objects that do not have the "serviceName" property:

select item.serviceName? from OrderEvent

Consider the case where OrderEvent has multiple implementation classes, some of which have a "timestamp" property. The next query returns the timestamp property of those implementations of the OrderEvent interface that feature the property:

select timestamp? from OrderEvent

The query as above returns a single column named "timestamp?" of type Object.

When dynamic properties are nested, then all properties under the dynamic property are also considered dynamic properties. In the below example the query asks for the "direction" property of the object returned by the "detail" dynamic property:

select detail?.direction from OrderEvent

Above is equivalent to:

select detail?.direction? from OrderEvent

The functions that are often useful in conjunction with dynamic properties are:

  • The cast function casts the value of a dynamic property (or the value of an expression) to a given type.

  • The exists function checks whether a dynamic property exists. It returns true if the event has a property of that name, or false if the property does not exist on that event.

  • The instanceof function checks whether the value of a dynamic property (or the value of an expression) is of any of the given types.

  • The typeof function returns the string type name of a dynamic property.

Dynamic event properties work with all event representations outlined next: Java objects, Map-based, Object-array-based and XML DOM-based events.

Sometimes an event can have properties that are itself events. Esper uses the term fragment and fragment type for such event pieces. The best example is a pattern that matches two or more events and the output event contains the matching events as fragments. In other words, output events can be a composite event that consists of further events, the fragments.

Fragments have the same metadata available as their enclosing composite events. The metadata for enclosing composite events contains information about which properties are fragments, or have a property value that can be represented as a fragment and therefore as an event itself.

Fragments and type metadata can allow your application to navigate composite events without the need for using the Java reflection API and reducing the coupling to the underlying event representation. The API is further described in Section 15.6, “Event and Event Type”.

For NEsper .NET also see Section C.7, “.NET Object Events”.

Plain-old Java object events are object instances that expose event properties through JavaBeans-style getter methods. Events classes or interfaces do not have to be fully compliant to the JavaBean specification; however for the Esper engine to obtain event properties, the required JavaBean getter methods must be present or an accessor-style and accessor-methods may be defined via configuration.

Esper supports JavaBeans-style event classes that extend a superclass or implement one or more interfaces. Also, Esper event pattern and EPL statements can refer to Java interface classes and abstract classes.

Classes that represent events should be made immutable. As events are recordings of a state change or action that occurred in the past, the relevant event properties should not be changeable. However this is not a hard requirement and the Esper engine accepts events that are mutable as well.

The hashCode and equals methods do not need to be implemented. The implementation of these methods by a Java event class does not affect the behavior of the engine in any way.

Please see Chapter 16, Configuration on options for naming event types represented by Java object event classes. Java classes that do not follow JavaBean conventions, such as legacy Java classes that expose public fields, or methods not following naming conventions, require additional configuration. Via configuration it is also possible to control case sensitivity in property name resolution. The relevant section in the chapter on configuration is Section 16.4.1.3, “Non-JavaBean and Legacy Java Event Classes”.

As outlined earlier, the different property types are supported by the standard JavaBeans specification, and some of which are uniquely supported by Esper:

Assume there is an NewEmployeeEvent event class as shown below. The mapped and indexed properties in this example return Java objects but could also return Java language primitive types (such as int or String). The Address object and Employee can themselves have properties that are nested within them, such as a street name in the Address object or a name of the employee in the Employee object.

public class NewEmployeeEvent {
	public String getFirstName();
	public Address getAddress(String type);
	public Employee getSubordinate(int index);
	public Employee[] getAllSubordinates();
}

Simple event properties require a getter-method that returns the property value. In this example, the getFirstName getter method returns the firstName event property of type String.

Indexed event properties require either one of the following getter-methods. A method that takes an integer-type key value and returns the property value, such as the getSubordinate method, or a method that returns an array-type, or a class that implements Iterable. An example is the getAllSubordinates getter method, which returns an array of Employee but could also return an Iterable. In an EPL or event pattern statement, indexed properties are accessed via the property[index] syntax.

Mapped event properties require a getter-method that takes a String-typed key value and returns the property value, such as the getAddress method. In an EPL or event pattern statement, mapped properties are accessed via the property('key') syntax.

Nested event properties require a getter-method that returns the nesting object. The getAddress and getSubordinate methods are mapped and indexed properties that return a nesting object. In an EPL or event pattern statement, nested properties are accessed via the property.nestedProperty syntax.

All event pattern and EPL statements allow the use of indexed, mapped and nested properties (or a combination of these) anywhere where one or more event property names are expected. The below example shows different combinations of indexed, mapped and nested properties in filters of event pattern expressions (each line is a separate EPL statement):

every NewEmployeeEvent(firstName='myName')
every NewEmployeeEvent(address('home').streetName='Park Avenue')
every NewEmployeeEvent(subordinate[0].name='anotherName')
every NewEmployeeEvent(allSubordinates[1].name='thatName')
every NewEmployeeEvent(subordinate[0].address('home').streetName='Water Street')

Similarly, the syntax can be used in EPL statements in all places where an event property name is expected, such as in select lists, where-clauses or join criteria.

select firstName, address('work'), subordinate[0].name, subordinate[1].name
from NewEmployeeEvent
where address('work').streetName = 'Park Ave'

For NEsper .NET also see Section C.8, “.NET IDictionary Events”.

Events can also be represented by objects that implement the java.util.Map interface. Event properties of Map events are the values in the map accessible through the get method exposed by the java.util.Map interface.

Similar to the Object-array event type, the Map event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types, thereby making it easier to change types at runtime or generate type information from another source.

A given Map event type can have one or more supertypes that must also be Map event types. All properties available on any of the Map supertypes are available on the type itself. In addition, anywhere within EPL that an event type name of a Map supertype is used, any of its Map subtypes and their subtypes match that expression.

Your application can add properties to an existing Map event type during runtime using the configuration operation updateMapEventType. Properties may not be updated or deleted - properties can only be added, and nested properties can be added as well. The runtime configuration also allows removing Map event types and adding them back with new type information.

After your application configures a Map event type by providing a type name, the type name can be used when defining further Map or Object-array event types by specifying the type name as a property type or an array property type.

One-to-Many relationships in Map event types are represented via arrays. A property in a Map event type may be an array of primitive, an array of Java object, an array of Map or an an array of Object-array.

The engine can process java.util.Map events via the sendEvent(Map map, String eventTypeName) method on the EPRuntime interface. Entries in the Map represent event properties. Keys must be of type java.util.String for the engine to be able to look up event property names specified by pattern or EPL statements.

The engine does not validate Map event property names or values. Your application should ensure that objects passed in as event properties match the create schema property names and types, or the configured event type information when using runtime or static configuration.

Map event properties can be of any type. Map event properties that are Java application objects or that are of type java.util.Map (or arrays thereof) or that are of type Object[] (object-array) (or arrays thereof) offer additional power:

In order to use Map events, the event type name and property names and types must be made known to the engine via Configuration or create schema EPL syntax. Please see examples in Section 5.15, “Declaring an Event Type: Create Schema” and Section 16.4.2, “Events represented by java.util.Map”.

The code snippet below defines a Map event type, creates a Map event and sends the event into the engine. The sample defines the CarLocUpdateEvent event type via runtime configuration interface (create schema or static configuration could have been used instead).

// Define CarLocUpdateEvent event type (example for runtime-configuration interface)
Map<String, Object> def = new HashMap<String, Object>;
def.put("carId", String.class);
def.put("direction", int.class);

epService.getEPAdministrator().getConfiguration().
  addEventType("CarLocUpdateEvent", def);

The CarLocUpdateEvent can now be used in a statement:

select carId from CarLocUpdateEvent.win:time(1 min) where direction = 1
// Create a CarLocUpdateEvent event and send it into the engine for processing
Map<String, Object> event = new HashMap<String, Object>();
event.put("carId", carId);
event.put("direction", direction);

epRuntime.sendEvent(event, "CarLocUpdateEvent");

The engine can also query Java objects as values in a Map event via the nested property syntax. Thus Map events can be used to aggregate multiple data structures into a single event and query the composite information in a convenient way. The example below demonstrates a Map event with a transaction and an account object.

Map event = new HashMap();
event.put("txn", txn);
event.put("account", account);
epRuntime.sendEvent(event, "TxnEvent");

An example statement could look as follows.

select account.id, account.rate * txn.amount 
from TxnEvent.win:time(60 sec) 
group by account.id

Your Map event type may declare one or more supertypes when configuring the type at engine initialization time or at runtime through the administrative interface.

Supertypes of a Map event type must also be Map event types. All property names and types of a supertype are also available on a subtype and override such same-name properties of the subtype. In addition, anywhere within EPL that an event type name of a Map supertype is used, any of its Map subtypes also matches that expression (similar to the concept of interface in Java).

This example assumes that the BaseUpdate event type has been declared and acts as a supertype to the AccountUpdate event type (both Map event types):

epService.getEPAdministrator().getConfiguration().
    addEventType("AccountUpdate", accountUpdateDef, 
    new String[] {"BaseUpdate"});

Your application EPL statements may select BaseUpdate events and receive both BaseUpdate and AccountUpdate events, as well as any other subtypes of BaseUpdate and their subtypes.

// Receive BaseUpdate and any subtypes including subtypes of subtypes
select * from BaseUpdate

Your application Map event type may have multiple supertypes. The multiple inheritance hierarchy between Maps can be arbitrarily deep, however cyclic dependencies are not allowed. If using runtime configuration, supertypes must exist before a subtype to a supertype can be added.

See Section 16.4.2, “Events represented by java.util.Map” for more information on configuring Map event types.

Strongly-typed nested Map-within-Map events can be used to build rich, type-safe event types on the fly. Use the addEventType method on Configuration or ConfigurationOperations for initialization-time and runtime-time type definition, or the create schema EPL syntax.

Noteworthy points are:

For demonstration, in this example our top-level event type is an AccountUpdate event, which has an UpdatedFieldType structure as a property. Inside the UpdatedFieldType structure the example defines various fields, as well as a property by name 'history' that holds a JavaBean class UpdateHistory to represent the update history for the account. The code snippet to define the event type is thus:

Map<String, Object> updatedFieldDef = new HashMap<String, Object>();
updatedFieldDef.put("name", String.class);
updatedFieldDef.put("addressLine1", String.class);
updatedFieldDef.put("history", UpdateHistory.class);
epService.getEPAdministrator().getConfiguration().
    addEventType("UpdatedFieldType", updatedFieldDef);

Map<String, Object> accountUpdateDef = new HashMap<String, Object>();
accountUpdateDef.put("accountId", long.class);
accountUpdateDef.put("fields", "UpdatedFieldType");	
// the latter can also be:  accountUpdateDef.put("fields", updatedFieldDef);

epService.getEPAdministrator().getConfiguration().
    addEventType("AccountUpdate", accountUpdateDef);

The next code snippet populates a sample event and sends the event into the engine:

Map<String, Object> updatedField = new HashMap<String, Object>();
updatedField.put("name", "Joe Doe");
updatedField.put("addressLine1", "40 Popular Street");
updatedField.put("history", new UpdateHistory());

Map<String, Object> accountUpdate = new HashMap<String, Object>();
accountUpdate.put("accountId", 10009901);
accountUpdate.put("fields", updatedField);

epService.getEPRuntime().sendEvent(accountUpdate, "AccountUpdate");

Last, a sample query to interrogate AccountUpdate events is as follows:

select accountId, fields.name, fields.addressLine1, fields.history.lastUpdate
from AccountUpdate

An event can also be represented by an array of objects. Event properties of Object[] events are the array element values.

Similar to the Map event type, the object-array event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types, thereby making it easier to change types at runtime or generate type information from another source.

A given Object-array event type can have only a single supertype that must also be an Object-array event type. All properties available on the Object-array supertype is also available on the type itself. In addition, anywhere within EPL that an event type name of an Object-array supertype is used, any of its Object-array subtypes and their subtypes match that expression.

Your application can add properties to an existing Object-array event type during runtime using the configuration operation updateObjectArrayEventType. Properties may not be updated or deleted - properties can only be added, and nested properties can be added as well. The runtime configuration also allows removing Object-array event types and adding them back with new type information.

After your application configures an Object-array event type by providing a type name, the type name can be used when defining further Object-array or Map event types by specifying the type name as a property type or an array property type.

One-to-Many relationships in Object-array event types are represented via arrays. A property in an Object-array event type may be an array of primitive, an array of Java object, an array of Map or an array of Object-array.

The engine can process Object[] events via the sendEvent(Object[] array, String eventTypeName) method on the EPRuntime interface. Entries in the Object array represent event properties.

The engine does not validate Object array length or value types. Your application must ensure that Object array values match the declaration of the event type: The type and position of property values must match property names and types in the same exact order and object array length must match the number of properties declared via create schema or the static or runtime configuration.

Object-array event properties can be of any type. Object-array event properties that are Java application objects or that are of type java.util.Map (or arrays thereof) or that are of type Object-array (or arrays thereof) offer additional power:

In order to use Object[] (object-array) events, the event type name and property names and types, in a well-defined order that must match object-array event properties, must be made known to the engine via configuration or create schema EPL syntax. Please see examples in Section 5.15, “Declaring an Event Type: Create Schema” and Section 16.4.3, “Events represented by Object[] (Object-array)”.

The code snippet below defines an Object-array event type, creates an Object-array event and sends the event into the engine. The sample defines the CarLocUpdateEvent event type via the runtime configuration interface (create schema or static configuration could have been used instead).

// Define CarLocUpdateEvent event type (example for runtime-configuration interface)
String[] propertyNames = {"carId", "direction"};   // order is important
Object[] propertyTypes = {String.class, int.class};  // type order matches name order

epService.getEPAdministrator().getConfiguration().
  addEventType("CarLocUpdateEvent", propertyNames, propertyTypes);

The CarLocUpdateEvent can now be used in a statement:

select carId from CarLocUpdateEvent.win:time(1 min) where direction = 1
// Send an event
Object[] event = {carId, direction};
epRuntime.sendEvent(event, "CarLocUpdateEvent");

The engine can also query Java objects as values in an Object[] event via the nested property syntax. Thus Object[] events can be used to aggregate multiple data structures into a single event and query the composite information in a convenient way. The example below demonstrates a Object[] event with a transaction and an account object.

epRuntime.sendEvent(new Object[] {txn, account}, "TxnEvent");

An example statement could look as follows:

select account.id, account.rate * txn.amount 
from TxnEvent.win:time(60 sec) 
group by account.id

Strongly-typed nested Object[]-within-Object[] events can be used to build rich, type-safe event types on the fly. Use the addEventType method on Configuration or ConfigurationOperations for initialization-time and runtime-time type definition, or the create schema EPL syntax.

Noteworthy points are:

For demonstration, in this example our top-level event type is an AccountUpdate event, which has an UpdatedFieldType structure as a property. Inside the UpdatedFieldType structure the example defines various fields, as well as a property by name 'history' that holds a JavaBean class UpdateHistory to represent the update history for the account. The code snippet to define the event type is thus:

String[] propertyNamesUpdField = {"name", "addressLine1", "history"};
Object[] propertyTypesUpdField = {String.class, String.class, UpdateHistory.class};
epService.getEPAdministrator().getConfiguration().
    addEventType("UpdatedFieldType", propertyNamesUpdField, propertyTypesUpdField);

String[] propertyNamesAccountUpdate = {"accountId", "fields"};
Object[] propertyTypesAccountUpdate = {long.class, "UpdatedFieldType"};
epService.getEPAdministrator().getConfiguration().
    addEventType("AccountUpdate", propertyNamesAccountUpdate, propertyTypesAccountUpdate);

The next code snippet populates a sample event and sends the event into the engine:

Object[] updatedField = {"Joe Doe", "40 Popular Street", new UpdateHistory()};
Object[] accountUpdate = {10009901, updatedField};

epService.getEPRuntime().sendEvent(accountUpdate, "AccountUpdate");

Last, a sample query to interrogate AccountUpdate events is as follows:

select accountId, fields.name, fields.addressLine1, fields.history.lastUpdate
from AccountUpdate

For NEsper .NET also see Section C.9, “.NET XML Events”.

Events can be represented as org.w3c.dom.Node instances and send into the engine via the sendEvent method on EPRuntime or via EventSender. Please note that configuration is required so the event type name and root element name is known. See Chapter 16, Configuration.

If a XML schema document (XSD file) can be made available as part of the configuration, then Esper can read the schema and appropriately present event type metadata and validate statements that use the event type and its properties. See Section 2.8.1, “Schema-Provided XML Events”.

When no XML schema document is provided, XML events can still be queried, however the return type and return values of property expressions are string-only and no event type metadata is available other than for explicitly configured properties. See Section 2.8.2, “No-Schema-Provided XML Events”.

In all cases Esper allows you to configure explicit XPath expressions as event properties. You can specify arbitrary XPath functions or expressions and provide a property name and type by which result values will be available for use in EPL statements. See Section 2.8.3, “Explicitly-Configured Properties”.

Nested, mapped and indexed event properties are also supported in expressions against org.w3c.dom.Node events. Thus XML trees can conveniently be interrogated via the property expression syntax.

Only one event type per root element name may be configured. The engine recognizes each event by its root element name or you may use EventSender to send events.

This section uses the following XML document as an example:

<?xml version="1.0" encoding="UTF-8"?>
<Sensor xmlns="SensorSchema">
  <ID>urn:epc:1:4.16.36</ID>
  <Observation Command="READ_PALLET_TAGS_ONLY">
    <ID>00000001</ID>
    <Tag>
      <ID>urn:epc:1:2.24.400</ID>
    </Tag>
    <Tag>
      <ID>urn:epc:1:2.24.401</ID>
    </Tag>
  </Observation>
</Sensor>

The schema for the example is:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="Sensor">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="ID" type="xs:string"/>
        <xs:element ref="Observation" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:element name="Observation">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="ID" type="xs:string"/>
        <xs:element ref="Tag" maxOccurs="unbounded" />
      </xs:sequence>
      <xs:attribute name="Command" type="xs:string" use="required" />
    </xs:complexType>
  </xs:element>

  <xs:element name="Tag">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="ID" type="xs:string"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

If you have a XSD schema document available for your XML events, Esper can interrogate the schema. The benefits are:

The engine reads a XSD schema file from an URL you provide. Make sure files imported by the XSD schema file can also be resolved.

The configuration accepts a schema URL. This is a sample code snippet to determine a schema URL from a file in classpath:

URL schemaURL = this.getClass().getClassLoader().getResource("sensor.xsd");
// For NEsper .NET use C# ResourceManager class for loading resources

Here is a sample use of the runtime configuration API, please see Chapter 16, Configuration for further examples.

epService = EPServiceProviderManager.getDefaultProvider();
ConfigurationEventTypeXMLDOM sensorcfg = new ConfigurationEventTypeXMLDOM();
sensorcfg.setRootElementName("Sensor");
sensorcfg.setSchemaResource(schemaURL.toString());
epService.getEPAdministrator().getConfiguration()
    .addEventType("SensorEvent", sensorcfg);

You must provide a root element name. This name is used to look up the event type for the sendEvent(org.w3c.Node node) method. An EventSender is a useful alternative method for sending events if the type lookup based on the root or document element name is not desired.

After adding the event type, you may create statements and send events. Next is a sample statement:

select ID, Observation.Command, Observation.ID, 
  Observation.Tag[0].ID, Observation.Tag[1].ID
from SensorEvent

As you can see from the example above, property expressions can query property values held in the XML document's elements and attributes.

There are multiple ways to obtain a XML DOM document instance from a XML string. The next code snippet shows how to obtain a XML DOM org.w3c.Document instance:

InputSource source = new InputSource(new StringReader(xml));
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
builderFactory.setNamespaceAware(true);
Document doc = builderFactory.newDocumentBuilder().parse(source);

Send the org.w3c.Node or Document object into the engine for processing:

epService.getEPRuntime().sendEvent(doc);

By setting the xpath-property-expr option the engine rewrites each property expression as an XPath expression, effectively handing the evaluation over to the underlying XPath implementation available from classpath. Most JVM have a built-in XPath implementation and there are also optimized, fast implementations such as Jaxen that can be used as well.

Set the xpath-property-expr option if you need namespace-aware document traversal, such as when your schema mixes several namespaces and element names are overlapping.

The below table samples several property expressions and the XPath expression generated for each, without namespace prefixes to keep the example simple:


For mapped properties that are specified via the syntax name('key'), the algorithm looks for an attribute by name id and generates a XPath expression as mapped[@id='key'].

Finally, here is an example that includes all different types of properties and their XPath expression equivalent in one property expression:

select nested.mapped('key').indexed[1].attribute from MyEvent

The equivalent XPath expression follows, this time including n0 as a sample namespace prefix:

/n0:rootelement/n0:nested/n0:mapped[@id='key']/n0:indexed[position() = 2]/@attribute

When providing a XSD document, the default configuration allows to transpose property values that are themselves complex elements, as defined in the XSD schema, into a new stream. This behavior can be controlled via the flag auto-fragment.

For example, consider the next query:

insert into ObservationStream
select ID, Observation from SensorEvent

The Observation as a property of the SensorEvent gets itself inserted into a new stream by name ObservationStream. The ObservationStream thus consists of a string-typed ID property and a complex-typed property named Observation, as described in the schema.

A further statement can use this stream to query:

select Observation.Command, Observation.Tag[0].ID from ObservationStream

Before continuing the discussion, here is an alternative syntax using the wildcard-select, that is also useful:

insert into TagListStream
select ID as sensorId, Observation.* from SensorEvent

The new TagListStream has a string-typed ID and Command property as well as an array of Tag properties that are complex types themselves as defined in the schema.

Next is a sample statement to query the new stream:

select sensorId, Command, Tag[0].ID from TagListStream

Please note the following limitations:

Esper automatically registers a new event type for transposed properties. It generates the type name of the new XML event type from the XML event type name and the property names used in the expression. The synposis is type_name.property_name[.property_name...]. The type name can be looked up, for example for use with EventSender or can be created in advance.

Regardless of whether or not you provide a XSD schema for the XML event type, you can always fall back to configuring explicit properties that are backed by XPath expressions.

For further documentation on XPath, please consult the XPath standard or other online material. Consider using Jaxen or Apache Axiom, for example, to provide faster XPath evaluation then your Java VM built-in XPath provider may offer.

Part of the extension and plug-in features of Esper is an event representation API. This set of classes allow an application to create new event types and event instances based on information available elsewhere, statically or dynamically at runtime when EPL statements are created. Please see Section 18.8, “Event Type And Event Object” for details.

Creating a plug-in event representation can be useful when your application has existing Java classes that carry event metadata and event property values and your application does not want to (or cannot) extract or transform such event metadata and event data into one of the built-in event representations (POJO Java objects, Map, Object-array or XML DOM).

Further use of a plug-in event representation is to provide a faster or short-cut access path to event data. For example, access to event data stored in a XML format through the Streaming API for XML (StAX) is known to be very efficient. A plug-in event representation can also provide network lookup and dynamic resolution of event type and dynamic sourcing of event instances.

Currently, EsperIO provides the following additional event representations:

  • Apache Axiom: Streaming API for XML (StAX) implementation

Please see the EsperIO documentation for details on the above.

The chapter on Section 18.8, “Event Type And Event Object” explains how to create your own custom event representation.

To summarize, an event is an immutable record of a past occurrence of an action or state change, and event properties contain useful information about an event.

The length of time an event is of interest to the event processing engine (retention time) depends on your EPL statements, and especially the data window, pattern and output rate limiting clauses of your statements.

During the retention time of an event more information about the event may become available, such as additional properties or changes to existing properties. Esper provides three concepts for handling updates to events.

The first means to handle updating events is the update istream clause as further described in Section 5.20, “Updating an Insert Stream: the Update IStream Clause”. It is useful when you need to update events as they enter a stream, before events are evaluated by any particular consuming statement to that stream.

The second means to update events is the on-merge and on-update clauses, for use with tables and named windows only, as further described in Section 6.8, “Triggered Upsert using the On-Merge Clause” and Section 6.6, “Updating Data: the On Update clause”. On-merge is similar to the SQL merge clause and provides what is known as an "Upsert" operation: Update existing events or if no existing event(s) are found then insert a new event, all in one atomic operation provided by a single EPL statement. On-update can be used to update individual properties of rows held in a table or named window.

The third means to handle updating events is the revision event types, for use with named windows only, as further described in Section 6.11, “Versioning and Revision Event Type Use with Named Windows”. With revision event types one can declare, via configuration only, multiple different event types and then have the engine present a merged event type that contains a superset of properties of all merged types, and have the engine merge events as they arrive without additional EPL statements.

Note that patterns do not reflect changes to past events. For the temporal nature of patterns, any changes to events that were observed in the past do not reflect upon current pattern state.

Your application events may consist of fairly comprehensive, coarse-grained structures or documents. For example in business-to-business integration scenarios, XML documents or other event objects can be rich deeply-nested graphs of event properties.

To extract information from a coarse-grained event or to perform bulk operations on the rows of the property graph in an event, Esper provides a convenient syntax: When specifying a filter expression in a pattern or in a select clause, it may contain a contained-event selection syntax, as further described in Section 5.19, “Contained-Event Selection”.

For NEsper .NET also see Section C.10, “.NET Event Objects Instantiated and Populated by Insert Into”.

The insert into clause can populate instantiate new instances of Java object events, java.util.Map events and Object[] (object array) events directly from the results of select clause expressions and populate such instances. Simply use the event type name as the stream name in the insert into clause as described in Section 5.10, “Merging Streams and Continuous Insertion: the Insert Into Clause”.

If instead you have an existing instance of a Java object returned by an expression, such as a single-row function or static method invocation for example, you can transpose that expression result object to a stream. This is described further in Section 5.10.7, “Transposing an Expression Result” and Section 10.4, “Select-Clause transpose Function”.

The column names specified in the select and insert into clause must match available writable properties in the event object to be populated (the target event type). The expression result types of any expressions in the select clause must also be compatible with the property types of the target event type.

If populating a POJO-based event type and the class provides a matching constructor, the expression result types of expressions in the select clause must be compatible with the constructor parameters in the order listed by the constructor. The insert into clause column names are not relevant in this case.

Consider the following example statement:

insert into com.mycompany.NewEmployeeEvent 
select fname as firstName, lname as lastName from HRSystemEvent

The above example specifies the fully-qualified class name of NewEmployeeEvent. The engine instantianes NewEmployeeEvent for each result row and populates the firstName and lastName properties of each instance from the result of select clause expressions. The HRSystemEvent in the example is assumed to have lname and fname properties, and either setter-methods and a default constructor, or a matching constructor.

Note how the example uses the as-keyword to assign column names that match the property names of the NewEmployeeEvent target event. If the property names of the source and target events are the same, the as-keyword is not required.

The next example is an alternate form and specifies property names within the insert into clause instead. The example also assumes that NewEmployeeEvent has been defined or imported via configuration since it does not specify the event class package name:

insert into NewEmployeeEvent(firstName, lastName) 
select fname, lname from HRSystemEvent

Finally, this example populates HRSystemEvent events. The example populates the value of a type property where the event has the value 'NEW' and populates a new event object with the value 'HIRED', copying the fname and lname property values to the new event object:

insert into HRSystemEvent 
select fname, lname, 'HIRED' as type from HRSystemEvent(type='NEW')

The matching of the select or insert into-clause column names to target event type's property names is case-sensitive. It is allowed to only populate a subset of all available columns in the target event type. Wildcard (*) is also allowed and copies all fields of the events or multiple events in a join.

For Java object events, your event class must provide setter-methods according to JavaBean conventions or, alternatively, a matching constructor. If the event class provides setter methods the class should also provide a default constructor taking no parameters. If the event class provides a matching constructor there is no need for setter-methods. If your event class does not have a default constructor and setter methods, or a matching constructor, your application may configure a factory method via ConfigurationEventTypeLegacy. If your event class does not have a default constructor and there is no factory method provided, the engine uses in connection with the Oracle JVM the sun.reflect.ReflectionFactory, noting that in this case member variables do not get initialized to assigned defaults.

The engine follows Java standards in terms of widening, performing widening automatically in cases where widening type conversion is allowed without loss of precision, for both boxed and primitive types and including BigInteger and BigDecimal.

When inserting array-typed properties into a Java, Map-type or Object-array underlying event the event definition should declare the target property as an array.

Please note the following limitations:

  • Event types that utilize XML org.w3c.dom.Node underlying event objects cannot be target of an insert into clause.

Each of the event representations of Java object, Map and XML document has advantages and disadvantages that are summarized in the table below: