www.espertech.comDocumentation

Chapter 3. Event Representations

3.1. Event Underlying Java Objects
3.2. Event Properties
3.2.1. Escape Characters
3.2.2. Expression as Key or Index Value
3.3. Dynamic Event Properties
3.4. Fragment and Fragment Type
3.5. Comparing Event Representations
3.5.1. Incoming Events
3.5.2. Outgoing Events
3.5.3. Schema
3.5.4. Side-by-Side
3.6. Support for Generic Tuples
3.7. Updating, Merging and Versioning Events
3.8. Coarse-Grained Events
3.9. Event Objects Instantiated and Populated by Insert Into
3.10. Event Type Uniqueness

This section outlines the different means to model and represent events.

EPL uses the term event type to describe the type information available for an event representation.

Your application may configure predefined event types using the configuration object or dynamically add event types using create schema.

The EPL create schema syntax allows declaring an event type at runtime using EPL, see Section 5.15, “Declaring an Event Type: Create Schema”.

Section 16.5.5, “Event and Event Type” explains how an event type becomes visible in statements and output events delivered by the runtime.

An event is an immutable record of a past occurrence of an action or state change. Event properties capture the state information for an event.

In EPL, an event can be represented by any of the following underlying Java objects (NEsper .NET, see Section J.6, “.NET Event Underlying Objects”):


EPL provides multiple choices for representing an event. There is no absolute need for you to create new Java classes to represent an event.

Event representations have the following in common:

  • All event representations support nested, indexed and mapped properties (aka. property expression), as explained in more detail below. There is no limitation to the nesting level.

  • All event representations provide event type metadata. This includes type metadata for nested properties.

  • All event representations allow transposing the event itself and parts or all of its property graph into new events. The term transposing refers to selecting the event itself or event properties that are themselves nestable property graphs, and then querying the event's properties or nested property graphs in further statements.

  • The Java object, Map, Object-array, JSON and Avro representations allow supertypes.

The API behavior for all event representations is the same, with minor exceptions noted.

The benefits of multiple event representations are:

  • For applications that already have events in one of the supported representations, there is no need to transform events before processing for both input and output.

  • Event representations are exchangeable, reducing or eliminating the need to change statements when the event representation changes, i.e. the EPL does not depend on whether events are Objects, Map(s), Object-array(s), JSON document(s), Avro record(s) or XML document(s).

  • Event representations are interoperable, allowing all event representations to interoperate in same or different statements.

  • The choice makes it possible to consciously trade-off performance, ease-of-use, the ability to evolve and effort needed to import or externalize events and use existing event type metadata.

Event properties capture the state information for an event. Event properties can be simple, indexed, mapped and nested event properties.

The table below outlines the different types of properties and their syntax in an event expression:


Combinations are also possible. For example, a valid combination could be person.address('home').street[0].

You may use any expression as a mapped property key or indexed property index by putting the expression within parenthesis after the mapped or index property name. Please find examples below.

If your application uses java.util.Map, Object[] (object-array) or JSON or XML to represent events, then event property names may themselves contain the dot ('.') character. The backslash ('\') character can be used to escape dot characters in property names, allowing a property name to contain dot characters.

For example, the EPL as shown below expects a property by name part1.part2 to exist on event type MyEvent:

select part1\.part2 from MyEvent

Sometimes your event properties may overlap with EPL language keywords or contain spaces or other special characters. In this case you may use the backwards apostrophe ` (aka. back tick) character to escape the property name.

The next example assumes a Quote event that has a property by name order, while order is also a reserved keyword:

select `order`, price as `price.for.goods` from Quote

When escaping mapped or indexed properties, make sure the back tick character appears outside of the map key or index.

The next EPL selects event properties that have names that contain spaces (e.g. candidate book), have the tick special character (e.g. children's books), are an indexed property (e.g. children's books[0]) and a mapped property that has a reserved keyword as part of the property name (e.g. book select('isbn')):

select `candidate book` , `children's books`[0], `book select`('isbn') from MyEventType

When referring to nested properties, escape at the individual nesting level, for example (assuming MyEventType has a payload property that is itself an event):

select payload.`candidate book`, payload.`other books`? from MyEventType

The key or index expression must be placed in parenthesis. When using an expression as key for a mapped property, the expression must return a String-typed value. When using an expression as index for an indexed property, the expression must return an int-typed value.

This example below uses Java classes to illustrate; The same principles apply to all event representations.

Assume a class declares these properties (getters not shown for brevity):

public class MyEventType {
  String myMapKey;
  int myIndexValue;
  int myInnerIndexValue;
  Map<String, InnerType> innerTypesMap;	// mapped property
  InnerType[] innerTypesArray; // indexed property
}

public class InnerType {
  String name;
  int[] ids;
}

A sample statement demonstrating expressions as map keys or indexes is:

select innerTypesMap('somekey'),  // returns map value for 'somekey'
  innerTypesMap(myMapKey),        // returns map value for myMapKey value (an expression)
  innerTypesArray[1],             // returns array value at index 1
  innerTypesArray(myIndexValue)   // returns array value at index myIndexValue (an expression)
  from MyEventType

The dot-operator can be used to access methods on the value objects returned by the mapped or indexed properties. By using the dot-operator the syntax follows the chained method invocation described at Section 9.7, “Dot Operator”.

A sample statement demonstrating the dot-operator as well as expressions as map keys or indexes is:

 select innerTypesMap('somekey').ids[1],
  innerTypesMap(myMapKey).getIds(myIndexValue),
  innerTypesArray[1].ids[2],
  innerTypesArray(myIndexValue).getIds(myInnerIndexValue)
  from MyEventType

Please note the following limitations:

  • The square brackets-syntax for indexed properties does now allow expressions and requires a constant index value.

  • When using the dot-operator with mapped or indexed properties that have expressions as map keys or indexes you must follow the chained method invocation syntax.

Dynamic (unchecked) properties are event properties that need not be known at statement compilation time. Such properties are resolved during runtime: they provide duck typing functionality.

The idea behind dynamic properties is that for a given underlying event representation you don't always know all properties in advance. An underlying event may have additional properties that are not known at statement compilation time, that you want to query on. The concept is especially useful for events that represent rich, object-oriented domain models.

The syntax of dynamic properties consists of the property name and a question mark. Indexed, mapped and nested properties can also be dynamic properties:


Dynamic properties always return the java.lang.Object type. Also, dynamic properties return a null value if the dynamic property does not exist on events processed at runtime.

As an example, consider an OrderEvent event that provides an "item" property. The "item" property is of type Object and holds a reference to an instance of either a Service or Product.

Assume that both Service and Product classes provide a property named "price". Via a dynamic property you can specify a statement that obtains the price property from either object (Service or Product):

select item.price? from OrderEvent

As a second example, assume that the Service class contains a "serviceName" property that the Product class does not possess. The following statement returns the value of the "serviceName" property for Service objects. It returns a null-value for Product objects that do not have the "serviceName" property:

select item.serviceName? from OrderEvent

Consider the case where OrderEvent has multiple implementation classes, some of which have a "timestamp" property. The next statement returns the timestamp property of those implementations of the OrderEvent interface that feature the property:

select timestamp? from OrderEvent

The statement as above returns a single column named "timestamp?" of type Object.

When dynamic properties are nested, then all properties under the dynamic property are also considered dynamic properties. In the below example the statement asks for the "direction" property of the object returned by the "detail" dynamic property:

select detail?.direction from OrderEvent

Above is equivalent to:

select detail?.direction? from OrderEvent

The functions that are often useful in conjunction with dynamic properties are:

  • The cast function casts the value of a dynamic property (or the value of an expression) to a given type.

  • The exists function checks whether a dynamic property exists. It returns true if the event has a property of that name, or false if the property does not exist on that event.

  • The instanceof function checks whether the value of a dynamic property (or the value of an expression) is of any of the given types.

  • The typeof function returns the string type name of a dynamic property.

Sometimes an event can have properties that are itself events. EPL uses the term fragment and fragment type for such event pieces. The best example is a pattern that matches two or more events and the output event contains the matching events as fragments. In other words, output events can be a composite event that consists of further events, the fragments.

Fragments have the same metadata available as their enclosing composite events. The metadata for enclosing composite events contains information about which properties are fragments, or have a property value that can be represented as a fragment and therefore as an event itself.

Fragments and type metadata can allow your application to navigate composite events without the need for using the Java reflection API and reducing the coupling to the underlying event representation. The API is further described in Section 16.5.5, “Event and Event Type”.

More information on event representations can be found in the appendix. The links are:


EPL does not require a fixed tuple structure and fully supports generic tuples. Event properties can be defined, added to existing types and queried at runtime.

The facilities for support of generic tuples are:

There is no need to explicitly create an event type for each tuple type. It is not necessary to create classes for tuple types at all. Events can be arbitrary objects.

The compiler uses the type information that is available: the compiler can verifies your statement against the known properties and types, preventing you as the EPL designer from making mistakes in EPL design. The compiler does not verify dynamic properties, which may return null at runtime. If type information is not available then properties are assumed to return java.lang.Object-typed values.

For example, let's say you need a generic tuple and you have Map events:

create schema GenericTuple()

Create statements that use dynamic properties, as the next EPL shows, which casts the timestamp value to a long-type value and outputs the hour-minute-second string:

select cast(timestamp?, long).format('hh mm ss') from GenericTuple

Send events like this:

Map<String, Object> genericEvent = new HashMap<>();
genericEvent.put("timestamp", new Date().getTime());
genericEvent.put("some_other_property", "hello");
runtime.getEventService().sendEventMap(genericEvent, "GenericTuple");

To summarize, an event is an immutable record of a past occurrence of an action or state change, and event properties contain useful information about an event.

The length of time an event is of interest to the runtime (retention time) depends on your statements, and especially the data window, pattern and output rate limiting clauses of your statements.

During the retention time of an event more information about the event may become available, such as additional properties or changes to existing properties. EPL provides two concepts for handling updates to events.

The first means to handle updating events is the update istream clause as further described in Section 5.20, “Updating an Insert Stream: The Update IStream Clause”. It is useful when you need to update events as they enter a stream, before events are evaluated by any particular consuming statement to that stream.

The second means to update events is the on-merge and on-update clauses, for use with tables and named windows only, as further described in Section 6.8, “Triggered Upsert Using the On-Merge Clause” and Section 6.6, “Updating Data: The On Update Clause”. On-merge is similar to the SQL merge clause and provides what is known as an "Upsert" operation: Update existing events or if no existing event(s) are found then insert a new event, all in one atomic operation provided by a single statement. On-update can be used to update individual properties of rows held in a table or named window.

Your application events may consist of fairly comprehensive, coarse-grained structures or documents. For example in business-to-business integration scenarios, XML documents or other event objects can be rich deeply-nested graphs of event properties.

To extract information from a coarse-grained event or to perform bulk operations on the rows of the property graph in an event, EPL provides a convenient syntax: When specifying a filter expression in a pattern or in a select clause, it may contain an contained-event selection syntax, as further described in Section 5.19, “Contained-Event Selection”.

For NEsper .NET also see Section J.10, “.NET Event Objects Instantiated and Populated by Insert Into”.

The insert into clause can instantiate and populate new instances of Java object events, java.util.Map events and Object[] (object array) events directly from the results of select clause expressions. Simply use the event type name as the stream name in the insert into clause as described in Section 5.10, “Merging Streams and Continuous Insertion: The Insert Into Clause”.

If instead you have an existing instance of a Java object returned by an expression, such as a single-row function or static method invocation for example, you can transpose that expression result object to a stream. This is described further in Section 5.10.7, “Transposing an Expression Result” and Section 10.4, “Select-Clause Transpose Function”.

The column names specified in the select and insert into clause must match available writable properties in the event object to be populated (the target event type). The expression result types of any expressions in the select clause must also be compatible with the property types of the target event type.

If populating a POJO-based event type and the class provides a matching constructor, the expression result types of expressions in the select clause must be compatible with the constructor parameters in the order listed by the constructor. The insert into clause column names are not relevant in this case.

Consider the following example statement:

insert into com.mycompany.NewEmployeeEvent 
select fname as firstName, lname as lastName from HRSystemEvent

The above example specifies the fully-qualified class name of NewEmployeeEvent. The runtime instantianes NewEmployeeEvent for each result row and populates the firstName and lastName properties of each instance from the result of select clause expressions. The HRSystemEvent in the example is assumed to have lname and fname properties, and either setter-methods and a default constructor, or a matching constructor.

Note how the example uses the as-keyword to assign column names that match the property names of the NewEmployeeEvent target event. If the property names of the source and target events are the same, the as-keyword is not required.

The next example is an alternate form and specifies property names within the insert into clause instead. The example also assumes that NewEmployeeEvent has been defined or imported via configuration since it does not specify the event class package name:

insert into NewEmployeeEvent(firstName, lastName) 
select fname, lname from HRSystemEvent

Finally, this example populates HRSystemEvent events. The example populates the value of a type property where the event has the value 'NEW' and populates a new event object with the value 'HIRED', copying the fname and lname property values to the new event object:

insert into HRSystemEvent 
select fname, lname, 'HIRED' as type from HRSystemEvent(type='NEW')

The matching of the select or insert into-clause column names to target event type's property names is case-sensitive. You can specify a subset of all available columns in the target event type. Wildcard (*) is allowed and copies all fields of the events or multiple events in a join.

For Java object events, your event class must provide setter-methods according to JavaBean conventions or, alternatively, a matching constructor. If the event class provides setter methods the class should also provide a default constructor taking no parameters. If the event class provides a matching constructor there is no need for setter-methods. If your event class does not have a default constructor and setter methods, or a matching constructor, your application may configure a factory method via ConfigurationEventTypeLegacy. If your event class does not have a default constructor and there is no factory method provided, the runtime uses in connection with the Oracle JVM the sun.reflect.ReflectionFactory, noting that in this case member variables do not get initialized to assigned defaults.

The compiler follows Java standards in terms of widening, performing widening automatically in cases where widening type conversion is allowed without loss of precision, for both boxed and primitive types and including BigInteger and BigDecimal.

When inserting array-typed properties into a Java, Map-type, Object-array or JSON underlying event the event definition should declare the target property as an array (Avro uses Collections).

Please note the following limitations:

  • Event types that utilize XML org.w3c.dom.Node underlying event objects cannot be target of an insert into clause.

Event type names of preconfigured event types are unique within both compiler and runtime.

Event type names of event types allocated by create schema are unique within both the compiler and the runtime by the combination of the deployment id and the event type name.

At runtime for internal lookups the runtime computes a CRC32 value pair. Usually you don't need to worry about CRC32 values as a collision is very unlikely and the compiler and runtime indicate relevant collisions by throwing a compile-time or deployment-time exception. For preconfigured event types the CRC32 of the type name must be unique within the runtime. For event types allocated by create schema the combination of CRC32 of the deployment id and the CRC32 of the event type name must be unique within the runtime.