Chapter 12. API Reference

12.1. API Overview

Esper has the following primary interfaces:

For EPL introductory information please see Section 4.1, “EPL Introduction” and patterns are described at Section 5.1, “Event Pattern Overview”.

The automated API documentation is also a great source for API information.

12.2. The Service Provider Interface

The EPServiceProvider interface represents an engine instance. Each instance of an Esper engine is completely independent of other engine instances and has its own administrative and runtime interface.

An instance of the Esper engine is obtained via static methods on the EPServiceProviderManager class. The GetDefaultProvider method and the GetProvider(String providerURI) methods return an instance of the Esper engine. The latter can be used to obtain multiple instances of the engine for different provider URI values. The EPServiceProviderManager determines if the provider URI matches all prior provider URI values and returns the same engine instance for the same provider URI value. If the provider URI has not been seen before, it creates a new engine instance.

The code snipped below gets the default instance Esper engine. Subsequent calls to get the default engine instance return the same instance.

EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider();

This code snippet gets an Esper engine for the provider URI RFIDProcessor1. Subsequent calls to get an engine with the same provider URI return the same instance.

EPServiceProvider epService = EPServiceProviderManager.GetProvider("RFIDProcessor1");

Since the GetProvider methods return the same cached engine instance for each URI, there is no need to statically cache an engine instance in your application.

An existing Esper engine instance can be reset via the Initialize method on the EPServiceProvider instance. This operation stops and removes all statements and resets the engine to the configuration provided when the engine instance for that URI was obtained. If no configuration is provided, an empty (default) configuration applies.

After Initialize your application must obtain new administrative and runtime services. Any administrative and runtime services obtained before the initialize are invalid and have undefined behavior.

The next code snippet outlines a typical sequence of use:

// Configure the engine, this is optional
Configuration config = new Configuration();
config.Configure("configuration.xml");	// load a configuration from file
config.XXX = ...;    // make additional configuration settings

// Obtain an engine instance
EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider(config);

// Optionally, use initialize if the same engine instance has been used before to start clean
epService.Initialize();

// Optionally, make runtime configuration changes
epService.EPAdministrator.GetConfiguration().Add...(...);

// Destroy the engine instance when no longer needed, frees up resources
epService.Dispose();

An existing Esper engine instance can be destroyed via the Dispose method on the EPServiceProvider instance. This stops and removes all statements as well as frees all resources held by the instance. After a Dispose the engine can no longer be used.

The EPServiceProvider exposes a number of events that are used to inform observers when an engine instance is about to be destroyed, after an engine instance has been initialized, when a new statement gets created and when a statement gets started, stopped or destroyed. Event handlers are registered using the events in the table below.

Table 12.1. Events For Service State Changes

Event on EPServiceProviderDescription 
ServiceDestroyRequested

Occurs after the service has been "destroyed" and all resources have been cleaned up.

 
ServiceInitialized

Occurs after the service provider has been initialized.

 
StatementCreate

Occurs after a statement has been created.

 
StatementStateChange

Occurs when a statement started, stopped or destroyed.

 

As engine instances are completely independent, your application may not send EventBean instances obtained from one engine instance into a second engine instance since the event type space between two engine instances is not shared.

12.3. The Administrative Interface

12.3.1. Creating Statements

Create event pattern expression and EPL statements via the administrative interface EPAdministrator.

This code snippet gets an Esper engine then creates an event pattern and an EPL statement.

EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider();
EPAdministrator admin = epService.EPAdministrator;

EPStatement 10secRecurTrigger = admin.CreatePattern(
  "every timer:at(*, *, *, *, *, */10)");

EPStatement countStmt = admin.CreateEPL(
  "select count(*) from MarketDataBean.win:time(60 sec)");

Note that event pattern expressions can also occur within EPL statements. This is outlined in more detail in Section 4.4.2, “Pattern-based Event Streams”.

The Create methods on EPAdministrator are overloaded and allow an optional statement name to be passed to the engine. A statement name can be useful for retrieving a statement by name from the engine at a later time. The engine assigns a statement name if no statement name is supplied on statement creation.

The createPattern and CreateEPL methods return EPStatement instances. Statements are automatically started and active when created. A statement can also be stopped and started again via the Stop and Start methods shown in the code snippet below.

countStmt.Stop();
countStmt.Start();

The Create methods on EPAdministrator also accept a user object. The user object is associated with a statement at time of statement creation and is a single, unnamed field that is stored with every statement. Applications may put arbitrary objects in this field. Use the UserObject property on EPStatement to obtain the user object of a statement.

Your application may create new statements or stop and destroy existing statements using any thread and also within event handlers or subscriber code. If using native object events, your application may not create or manage statements in the event object itself while the same event is currently being processed by a statement.

12.3.2. Receiving Statement Results

Esper provides three choices for your application to receive statement results. Your application can use all mechanisms alone or in any combination for each statement. The choices are:

Table 12.2. Choices For Receiving Statement Results

NameMethod, Property or Event on EPStatementDescription
Event HandlersEvents

Your application provides implementations of the UpdateEventHandler to the statement. Event handlers receive UpdateEventArgs instances containing statement results.

The engine continuously indicates results to all event handlers as soon they occur, and following output rate limiting clauses if specified.

Subscriber ObjectSubscriber

Your application provides a vanilla object that exposes methods to receive statement results.

The engine continuously indicates results to the single subscriber as soon they occur, and following output rate limiting clauses if specified.

This is the fastest method to receive statement results, as the engine delivers strongly-typed results directly to your application objects without the need for building an EventBean result set as in the Event Handler choice.

There can be at most 1 Subscriber Object registered per statement. If you require more than one event handler, use the event handler instead (or in addition). The Subscriber Object is bound to the statement with a strongly typed support which ensure direct delivery of new events without type conversion. This optimization is made possible because there can only be 0 or 1 Subscriber Object per statement.

Pull APIGetSafeEnumerator and GetEnumerator

Your application asks the statement for results and receives a set of events via System.Collections.IEnumerable<EventBean>.

This is useful if your application does not need continuous indication of new results in real-time.

Your application may attach one or more event handlers, zero or one single subscriber and in addition use the Pull API on the same statement. There are no limitations to the use of enumerator, subscriber or event handler alone or in combination to receive statement results.

The best delivery performance can generally be achieved by attaching a subscriber and by not attaching event handlers. The engine is aware of the event handlers and subscriber attached to a statement. The engine uses this information internally to reduce statement overhead. For example, if your statement does not have event handlers or a subscriber attached, the engine does not need to continuously generate results for delivery.

If your application attaches both a subscriber and one or more event handlers then the subscriber receives the result first before any of the event handlers.

If your application attaches more then one event handler then the event handlers receive results first in the order they were added to the statement, and To change the order of delivery among event handlers your application can add and remove event handlers at runtime.

If you have configured outbound threading, it means a thread from the outbound thread pool delivers results to the subscriber and event handlers instead of the processing or event-sending thread.

If outbound threading is turned on, we recommend turning off the engine setting preserving the order of events delivered to event handlers as described in Section 13.4.9.1, “Preserving the order of events delivered to listeners”. If outbound threading is turned on statement execution is not blocked for the configured time in the case a subscriber or listener takes too much time.

12.3.3. Setting a Subscriber Object

A subscriber object is a direct binding of query results to a CLR object. The object receives statement results via method invocation. The subscriber class does not need to implement an interface or extend a superclass. Only one subscriber object may be set for a statement.

Subscriber objects have several advantages over event handlers. First, they offer a substantial performance benefit: Query results are delivered directly to your method(s) through method calls, and there is no intermediate representation (EventBean). Second, as subscribers receive strongly-typed parameters, the subscriber code tends to be simpler.

This chapter describes the requirements towards the methods provided by your subscriber class.

The engine can deliver results to your subscriber in two ways:

  1. Each event in the insert stream results in a method invocation, and each event in the remove stream results in further method invocations. This is termed row-by-row delivery.

  2. A single method invocation that delivers all rows of the insert and remove stream. This is termed multi-row delivery.

12.3.3.1. Row-By-Row Delivery

Your subscriber class must provide a method by name Update to receive insert stream events row-by-row. The number and types of parameters declared by the Update method must match the number and types of columns as specified in the select clause, in the same order as in the select clause.

For example, if your statement is:

select orderId, price, count(*) from OrderEvent

Then your subscriber Update method looks as follows:

public class MySubscriber {
  ...
  public void Update(String orderId, double price, long count) {...}
  ...
}

Each method parameter declared by the Update method must be assignable from the respective column type as listed in the select-clause, in the order selected. The assignability rules are:

  • Widening of types follows CLR standards. For example, if your select clause selects an integer value, the method parameter for the same column can be typed int, long, float or double (or any equivalent boxed type).

  • Auto-boxing and unboxing follows CLR standards. For example, if your select clause selects a Nullable<int> value, the method parameter for the same column can be typed int. Note that if your select clause column may generate null values, an exception may occur at runtime unboxing the null value.

  • Interfaces and super-classes are honored in the test for assignability. Therefore System.Object can be used to accept any select clause column type

12.3.3.1.1. Wildcards

If your select clause contains one or more wildcards (*), then the equivalent parameter type is the underlying event type of the stream selected from.

For example, your statement may be:

select *, count(*) from OrderEvent

Then your subscriber Update method looks as follows:

public void Update(OrderEvent orderEvent, long count) {...}

In a join, the wildcard expands to the underlying event type of each stream in the join in the order the streams occur in the from clause. An example statement for a join is:

select *, count(*) from OrderEvent order, OrderHistory hist

Then your subscriber Update method should be:

public void Update(OrderEvent orderEvent, OrderHistory orderHistory, long count) {...}

The stream wildcard syntax and the stream name itself can also be used:

select hist.*, order from OrderEvent order, OrderHistory hist

The matching Update method is:

public void Update(OrderHistory orderHistory, OrderEvent orderEvent) {...}
12.3.3.1.2. Row Delivery as Map and Object Array

Alternatively, your Update method may simply choose to accept System.Collections.Generic.IDictionary as a representation for each row. Each column in the select clause is then made an entry in the resulting Map. The Map keys are the column name if supplied, or the expression string itself for columns without a name.

The Update method for Map delivery is:

public void Update(IDictionary<string, object> row) {...}

The engine also supports delivery of select clause columns as an object array. Each item in the object array represents a column in the select clause. The Update method then looks as follows:

public void Update(Object[] row) {...}
12.3.3.1.3. Delivery of Remove Stream Events

Your subscriber receives remove stream events if it provides a method named UpdateRStream. The method must accept the same number and types of parameters as the Update method.

An example statement:

select orderId, count(*) from OrderEvent.win:time(20 sec) group by orderId

Then your subscriber Update and UpdateRStream methods should be:

public void Update(String, long count) {...}
public void UpdateRStream(String orderId, long count) {...}
12.3.3.1.4. Delivery of Begin and End Indications

If your subscriber requires a notification for begin and end of event delivery, it can expose methods by name UpdateStart and UpdateEnd.

The UpdateStart method must take two integer parameters that indicate the number of events of the insert stream and remove stream to be delivered. The engine invokes the UpdateStart method immediately prior to delivering events to the Update and UpdateRStream methods.

The UpdateEnd method must take no parameters. The engine invokes the UpdateEnd method immediately after delivering events to the Update and UpdateRStream methods.

An example set of delivery methods:

// Called by the engine before delivering events to update methods
public void UpdateStart(int insertStreamLength, int removeStreamLength)

// To deliver insert stream events
public void Update(String orderId, long count) {...}

// To deliver remove stream events
public void UpdateRStream(String orderId, long count) {...}

// Called by the engine after delivering events
public void UpdateEnd() {...}

12.3.3.2. Multi-Row Delivery

In place of row-by-row delivery, your subscriber can receive all events in the insert and remove stream via a single method invocation.

The event delivery follow the scheme as described earlier in Section 12.3.3.1.2, “Row Delivery as Map and Object Array ”. The subscriber class must provide one of the following methods:

Table 12.3. Update Method for Multi-Row Delivery of Underlying Events

MethodDescription
Update(Object[][] insertStream, Object[][] removeStream)

The first dimension of each Object array is the event row, and the second dimension is the column matching the column order of the statement select clause

Update(IDictionary<string,object>[] insertStream, IDictionary<string,object>[] removeStream)

Each map represents one event, and Map entries represent columns of the statement select clause

12.3.3.2.1. Wildcards

If your select clause contains a single wildcard (*) or wildcard stream selector, the subscriber object may also directly receive arrays of the underlying events. In this case, the subscriber class should provide a method Update(Underlying[] insertStream, Underlying[] removeStream) , such that Underlying represents the class of the underlying event.

For example, your statement may be:

select * from OrderEvent.win:time(30 sec)

Your subscriber class exposes the method:

public void Update(OrderEvent[] insertStream, OrderEvent[] removeStream) {...}

12.3.4. Adding Event Handlers

Your application can subscribe to updates posted by a statement via the Events event on EPStatement. Your application must to provide an implementation of the UpdateEventHandler to the statement:

public void HandleEvents(Object sender, UpdateEventArgs updateEventArgs) { ... }
countStmt.Events += HandleEvents;

The following adds and event handler using an embedded lambda expression.

countStmt.Events += (sender, updateEventArgs) => MyMethod(updateEventArgs);

EPL statements and event patterns publish old data and new data to registered UpdateEventHandlers. New data published by statements is the events representing the new values of derived data held by the statement. Old data published by statements constists of the events representing the prior values of derived data held by the statement.

It is important to understand that UpdateEventHandlers receive multiple result rows in one invocation by the engine: the new data and old data parameters to your event handler are array parameters. For example, if your application uses one of the batch data windows, or your application creates a pattern that matches multiple times when a single event arrives, then the engine indicates such multiple result rows in one invocation and your new data array carries two or more rows.

The UpdateEventArgs object that is passed as part of the event handler is especially useful when the same event handler is registered with multiple statements, as the event argumetns contain the statement and engine instance in addition to the new and old data when the engine indicates new results to a event handler.

12.3.4.1. Subscription Snapshot and Atomic Delivery

The AddEventHandlerWithReplay method provided by EPStatement makes it possible to send a snapshot of current statement results to an event handler when the event handler is added.

When using the AddEventHandlerWithReplay method to register an event handler, the event handler receives current statement results as the first invocation of the delegate, passing in the newEvents parameter the current statement results as an array of zero or more events. Subsequent calls to the delegate of the event handler are statement results.

Current statement results are the events returned by the GetEnumerator or GetSafeEnumerator methods.

Delivery is atomic: Events occurring during delivery of current results to the event handler are guaranteed to be delivered in a separate call and not lost. The event handler implementation should thus minimize long-running or blocking operations to reduce lock times held on statement-level resources.

12.3.5. Using Enumerators

Subscribing to events posted by a statement is following a push model. The engine pushes data to event handlers when events are received that cause data to change or patterns to match. Alternatively, you need to know that statements serve up data that your application can obtain via the GetSafeEnumerator and GetEnumerator methods on EPStatement. This is called the pull API and can come in handy if your application is not interested in all new updates, and only needs to perform a frequent or infrequent poll for the latest data.

The GetSafeEnumerator method on EPStatement returns a concurrency-safe enumerator returning current statement results, even while concurrent threads may send events into the engine for processing. The safe enumerator guarantees correct results even as events are being processed by other threads. The cost is that the enumerator obtains and holds a statement lock that must be released via the Dispose method on the GetSafeEnumerator instance.

The GetEnumerator method on EPStatement returns a concurrency-unsafe enumerator. This enumerator is only useful for applications that are single-threaded, or applications that themselves perform coordination between the iterating thread and the threads that send events into the engine for processing. The advantage to this enumerator is that it does not hold a lock.

The next code snippet shows a short example of use of safe iterators:

EPStatement statement = epAdmin.CreateEPL("select avg(price) as avgPrice from MyTick");
// .. send events into the engine
// then use the pull API...
IEnumerator<EventBean> safeEnum = statement.GetSafeEnumerator();
while(safeEnum.MoveNext()) {
 // .. process event ..
 EventBean @event = safeEnum.Current;
 Console.WriteLine("avg:" + @event.Get("avgPrice");
}

This is a short example of use of the regular iterator that is not safe for concurrent event processing:

double averagePrice = (double) eplStatement.FirstOrDefault().Get("average");

The GetSafeEnumerator and GetEnumerator methods can be used to pull results out of all statements, including statements that join streams, contain aggregation functions, pattern statements, and statements that contain a where clause, group by clause, having clause or order by clause.

For statements without an order by clause, the GetEnumerator method returns events in the order maintained by the data window. For statements that contain an order by clause, the GetEnumerator method returns events in the order indicated by the order by clause.

Consider using the on-select clause and a named window if your application requires iterating over a partial result set or requires indexed access for fast iteration; Note that on-select requires that you sent a trigger event, which may contain the key values for indexed access.

Esper places the following restrictions on the pull API and usage of the GetSafeEnumerator and GetEnumerator methods:

  1. In multithreaded applications, use the GetSafeEnumerator method. Note: make sure your application closes the iterator via the Close method when done, otherwise the iterated statement stays locked and event processing for that statement does not resume.

  2. In multithreaded applications, the GetEnumerator method does not hold any locks. The iterator returned by this method does not make any guarantees towards correctness of results and fail-behavior, if your application processes events into the engine instance by multiple threads. Use the GetSafeEnumerator method for concurrency-safe iteration instead.

  3. Since the GetSafeEnumerator and GetEnumerator methods return events to the application immediately, the iterator does not honor an output rate limiting clause, if present. That is, the iterator returns results as if there is no output-rate clause for the statement in statements without grouping or aggregation. For statements with grouping or aggregation, the iterator in combintion with an output clause returns last output group and aggregation results. Use a separate statement and the insert into clause to control the output rate for iteration, if so required.

  4. When iterating a statement that selects an unbound stream (no data window declared), the iterator returns the last event. When iterating a statement that groups and aggregates values from an unbound stream, the iterated result contains only the last updated group.

12.3.6. Managing Statements

The EPAdministrator interface provides the facilities for managing statements:

  • Use the GetStatement method to obtain an existing started or stopped statement by name

  • Use the StatementNames property to obtain a list of started and stopped statement names

  • Use the StartAllStatements, StopAllStatements and DestroyAllStatements methods to manage all statements in one operation

12.3.7. Runtime Configuration

Certain configuration changes are available to perform on an engine instance while in operation. Such configuration operations are available via the GetConfiguration method on EPAdministrator, which returns a ConfigurationOperations object.

Please consult the SDK documentation of ConfigurationOperations for further information. The section Section 13.6, “Runtime Configuration” provides a summary of available configurations.

In summary, the configuration operations available on a running engine instance are as follows:

  • Add new event types for all event representations, check if an event type exists, update an existing event type, remove an event type, query a list of types and obtain a type by name.

  • Add and remove variables (get and set variable values is done via the runtime API).

  • Add a variant stream.

  • Add a revision event type.

  • Add event types for all event classes in a given namespace, using the simple type name as the event name.

  • Add import for user-defined functions.

  • Add a plug-in aggregation function, plug-in single row function, plug-in event type, plug-in event type resolution URIs.

  • Control metrics reporting.

  • Additional items please see the ConfigurationOperations interface.

For examples of above runtime configuration API functions please consider the Configuration chapter, which applies to both static configuration and runtime configuration as the ConfigurationOperations interface is the same.

12.4. The Runtime Interface

The EPRuntime interface is used to send events for processing into an Esper engine, set and get variable values and execute on-demand queries.

The below code snippet shows how to send a CLR object event to the engine. Note that the SendEvent method is overloaded. As events can take on different representation types, the SendEvent takes parameters to reflect the different types of events that can be send into the engine. The Chapter 2, Event Representations section explains the types of events accepted.

EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider();
EPRuntime runtime = epService.EPRuntime;

// Send an example event containing stock market data
runtime.SendEvent(new MarketDataBean('IBM', 75.0));		

Events, in theoretical terms, are observations of a state change that occurred in the past. Since one cannot change an event that happened in the past, events are best modelled as immutable objects.

Note that the Esper engine relies on events that are sent into an engine to not change their state. Typically, applications create a new event object for every new event, to represent that new event. Application should not modify an existing event that was sent into the engine.

Another important method in the runtime interface is the Route method. This method is designed for use by UpdateEventHandler and subscriber implementations that need to send events into an engine instance to avoid the possibility of a stack overflow due to nested calls to SendEvent.

12.4.1. Event Sender

The EventSender interface processes event objects that are of a known type. This facility can reduce the overhead of event object reflection and type lookup as an event sender is always associated to a single concrete event type.

Use the method GetEventSender(String eventTypeName) to obtain an event sender for processing events of the named type:

EventSender sender = epService.EPRuntime.GetEventSender("MyEvent");
sender.SendEvent(myEvent);

For events backed by a CLR type, the event sender ensures that the event object equals the underlying class, or implements or extends the underlying class for the given event type name.

For events backed by a System.Collections.Generic.IDictionary (Map events), the event sender does not perform any checking other then checking that the event object implements Map.

For events backed by a System.Xml.XmlNode (XML DOM events), the event sender checks that the root element name equals the root element name for the event type.

A second method to obtain an event sender is the method GetEventSender(Uri[]), which takes an array of URIs. This method is for use with plug-in event representations. The event sender returned by this method processes event objects that are of one of the types of one or more plug-in event representations. Please consult Section 15.8, “Event Type And Event Object” for more information.

12.4.2. Receiving Unmatched Events

Your application can register an implementation of the UnmatchedListener interface with the EPRuntime runtime via the SetUnmatchedListener method to receive events that were not matched by any statement.

Events that can be unmatched are all events that your application sends into the runtime via one of the SendEvent or Route methods, or that have been generated via an insert into clause.

For an event to become unmatched by any statement, the event must not match any statement's event stream filter criteria. Note that the EPL where clause or having clause are not considered part of the filter criteria for a stream, as explained by example below.

In the next statement all MyEvent events match the statement's event stream filter criteria, regardless of the value of the 'quantity' property. As long as the below statement remains started, the engine would not deliver MyEvent events to your registered UnmatchedListener instance:

select * from MyEvent where quantity > 5

In the following statement a MyEvent event with a 'quantity' property value of 5 or less does not match this statement's event stream filter criteria. The engine delivers such an event to the registered UnmatchedListener instance provided no other statement matches on the event:

select * from MyEvent(quantity > 5)

For patterns, if no pattern sub-expression is active for an event type, an event of that type also counts as unmatched in regards to the pattern statement.

12.4.3. On-Demand Snapshot Query Execution

As your application may not require streaming results and may not know each query in advance, the on-demand query facility provides for ad-hoc execution of an EPL expression.

On-demand queries are not continuous in nature: The query engine executes the query once and returns all result rows to the application. On-demand query execution is very lightweight as the engine performs no statement creation and the query leaves no traces within the engine.

Esper also provides the facility to explicitly index named windows to speed up queries. Please consult Section 4.15.10, “Explicitly Indexing Named Windows” for more information.

The following limitations apply:

  • An on-demand EPL expression only evaluates against the named windows that your application creates. On-demand queries may not specify any other streams or application event types.

  • The following clauses are not allowed in on-demand EPL: insert into and output.

  • Views and patterns are not allowed to appear in on-demand queries.

  • On-demand EPL may not perform subqueries.

  • The previous and prior functions may not be used.

12.4.3.1. On-Demand Query API

The EPRuntime provides two ways to run on-demand queries:

  1. Dynamic on-demand queries are executed once through the ExecuteQuery method.

  2. Prepared on-demand queries: The PrepareQuery method returns an EPOnDemandPreparedQuery representing the query, and the query can be performed repeatedly via the Execute method.

Prepared on-demand queries are designed for repeated execution and may perform better then the dynamic queries if running the same query multiple times. Placeholders are not allowed in prepared on-demand queries.

The next program listing runs an on-demand query against a named window MyNamedWindow and prints a column of each row result of the query:

String query = "select * from MyNamedWindow";
EPOnDemandQueryResult result = epRuntime.executeQuery(query);
for (EventBean row : result.getArray()) {
  System.out.println("name=" + row.get("name"));
}

The next code snippet demonstrates prepared on-demand queries:

EPOnDemandPreparedQuery prepared = epRuntime.prepareQuery(query);
EPOnDemandQueryResult result = prepared.execute();
// ...later ...
prepared.execute();	// execute a second time

Esper also provides the facility to explicitly index named windows to speed up queries. Please consult Section 4.15.10, “Explicitly Indexing Named Windows” for more information.

12.5. Event and Event Type

An EventBean object represents a row (event) in your continuous query's result set. Each EventBean object has an associated EventType object providing event metadata.

An UpdateEventHandler implementation receives one or more UpdateEventArgs events with each invocation. Via the GetEnumerator method on EPStatement your application can poll or read data out of statements. Statement enumerators also return EventBean instances.

Each statement provides the event type of the events it produces, available via the EventType property on EPStatement.

12.5.1. Event Type Metadata

An EventType object encapulates all the metadata about a certain type of events. As Esper supports an inheritance hierarchy for event types, it also provides information about super-types to an event type.

An EventType object provides the following information:

  • For each event property, it lists the property name and type as well as flags for indexed or mapped properties and whether a property is a fragment.

  • The direct and indirect super-types to the event type.

  • Value getters for property expressions.

  • Underlying class of the event representation.

For each property of an event type, there is an EventPropertyDescriptor object that describes the property. The EventPropertyDescriptor contains flags that indicate whether a property is an indexed (array) or a mapped property and whether access to property values require an integer index value (indexed properties only) or string key value (mapped properties only). The descriptor also contains a fragment flag that indicates whether a property value is available as a fragment.

The term fragment means an event property value that is itself an event, or a property value that can be represented as an event. The getFragmentType on EventType may be used to determine a fragment's event type in advance.

A fragment event type and thereby fragment events allow navigation over a statement's results even if the statement result contains nested events or a graph of events. There is no need to use the reflection API to navigate events, since fragments allow the querying of nested event properties or array values, including nested CLR types.

When using the Map event representation, any named Map type nested within a Map as a simple or array property is also available as a fragment. When using CLR objects either directly or within Map events, any object that is neither a primitive or boxed built-in type, and that is not an enumeration and does not implement the Map interface is also available as a fragment.

The nested, indexed and mapped property syntax can be combined to a property expression that may query an event property graph. Most of the methods on the EventType interface allow a property expression to be passed.

Your application may use an EventType object to obtain special getter-objects. A getter-object is a fast accessor to a property value of an event of a given type. All getter objects implement the EventPropertyGetter interface. Getter-objects work only for events of the same type or sub-types as the EventType that provides the EventPropertyGetter. The performance section provides additional information and samples on using getter-objects.

12.5.2. Event Object

An event object is an EventBean that provides:

  • The property value for a property given a property name or property expression that may include nested, indexed or mapped properties in any combination.

  • The event type of the event.

  • Access to the underlying event object.

  • The EventBean fragment or array of EventBean fragments given a property name or property expression.

The GetFragment method on EventBean and EventPropertyGetter return the fragment EventBean or array of EventBean, if the property is itself an event or can be represented as an event. Your application may use EventPropertyDescriptor to determine which properties are also available as fragments.

The underlying event object of an EventBean can be obtained via the Underlying property. Please see Chapter 2, Event Representations for more information on different event representations.

From a threading perspective, it is safe to retain and query EventBean and EventType objects in multiple threads.

12.5.3. Query Example

Consider a statement that returns the symbol, count of events per symbol and average price per symbol for tick events. Our sample statement may declare a fully-qualified type name as the event type: org.sample.StockTickEvent. Assume that this type exists and exposes a symbol property of type String, and a price property of type (primitive) double.

select symbol, avg(price) as avgprice, count(*) as mycount 
from org.sample.StockTickEvent 
group by symbol

The next table summarizes the property names and types as posted by the statement above:

Table 12.4. Properties offered by sample statement aggregating price

NameTypeDescriptionCode snippet
symbolSystem.StringValue of symbol event property
eventBean.Get["symbol"]
avgpriceSystem.DoubleAverage price per symbol
eventBean.Get["avgprice"]
mycountSystem.Int64Number of events per symbol
eventBean["mycount"]

A code snippet out of a possible UpdateEventHandler implementation to this statement may look as below:

String symbol = (String) newEvents[0]["symbol"];
double? price = (double?) newEvents[0]["avgprice"];
long? count= (long?) newEvents[0]["mycount"];

The engine supplies the boxed System.Double? and System.Int64? types as property values rather then primitive types. This is because aggregated values can return a null value to indicate that no data is available for aggregation. Also, in a select statement that computes expressions, the underlying event objects to EventBean instances are of type System.Collections.Generic.IDictionary.

Consider the next statement that specifies a wildcard selecting the same type of event:

select * from org.sample.StockTickEvent where price > 100

The property names and types provided by an EventBean query result row, as posted by the statement above are as follows:

Table 12.5. Properties offered by sample wildcard-select statement

NameTypeDescriptionCode snippet
symbolSystem.StringValue of symbol event property
eventBean["symbol"]
pricedoubleValue of price event property
eventBean["price"]

As an alternative to querying individual event properties via the indexer or Get methods, the Underlying property on EventBean returns the underlying object representing the query result. In the sample statement that features a wildcard-select, the underlying event object is of type org.sample.StockTickEvent:

StockTickEvent tick = (StockTickEvent) newEvents[0].Underlying;

12.5.4. Pattern Example

Composite events are events that aggregate one or more other events. Composite events are typically created by the engine for statements that join two event streams, and for event patterns in which the causal events are retained and reported in a composite event. The example below shows such an event pattern.

// Look for a pattern where BEvent follows AEvent
String pattern = "a=AEvent -> b=BEvent";
EPStatement stmt = epService.EPAdministrator.CreatePattern(pattern);
stmt.Events += HandleEvent;
// Example event handler code
  public void HandleEvent(Object sender, UpdateEventArgs e) {
    Console.WriteLine("a event={0}", e.NewData[0]["a"]);
    Console.WriteLine("b event={0}", e.NewData[0]["b"]);
  }

Note that the HandleEvent method can receive multiple events at once encapsulated in the UpdateEventArgs. For example, a time batch window may post multiple events to event handlers representing a batch of events received during a given time period.

Pattern statements can also produce multiple events delivered to update event handlers in one invocation. The pattern statement below, for instance, delivers an event for each A event that was not followed by a B event with the same id property within 60 seconds of the A event. The engine may deliver all matching A events as an array of events in a single invocation of the UpdateEventHandler delegate associated with the statement:

select * from pattern[
  every a=A -> (timer:interval(60 sec) and not B(id=a.id))]

A code snippet out of a possible UpdateEventHandler implementation to this statement that retrives the events as fragments may look as below:

EventBean a = (EventBean) newEvents[0].GetFragment("a");
// ... or using a nested property expression to get a value out of A event...
double value = (double) newEvent[0]["a.value"];

Some pattern objects return an array of events. An example is the unbound repeat operator. Here is a sample pattern that collects all A events until a B event arrives:

select * from pattern [a=A until b=B]

A possible code to retrieve different fragments or property values:

EventBean[] a = (EventBean[]) newEvents[0].GetFragment("a");
// ... or using a nested property expression to get a value out of A event...
double value = (double) newEvent[0]["a[0].value"];

12.6. Engine Threading and Concurrency

Esper is designed from the ground up to operate as a component to multi-threaded, highly-concurrent applications that require efficient use of resources. In addition, multi-threaded execution requires guarantees in predictability of results and deterministic processing. This section discusses these concerns in detail.

In Esper, an engine instance is a unit of separation. Applications can obtain and discard (initialize) one or more engine instances within the same application domain and can provide the same or different engine configurations to each instance. An engine instance efficiently shares resources between statements. For example, consider two statements that declare the same data window. The engine matches up view declarations provided by each statement and can thus provide a single data window representation shared between the two statements.

Applications can use Esper APIs to concurrently, by multiple threads of execution, perform such functions as creating and managing statements, or sending events into an engine instance for processing. Applications can use application-managed threads or thread pools or any set of same or different threads of execution with any of the public Esper APIs. There are no restrictions towards threading other then those noted in specific sections of this document.

Esper does not prescribe a specific threading model. Applications using Esper retain full control over threading, allowing an engine to be easily embedded and used as a component or library in your favorite container or process.

In the default configuration it is up to the application code to use multiple threads for processing events by the engine, if so desired. All event processing takes places within your application thread call stack. The exception is timer-based processing if your engine instance relies on the internal timer (default). If your application relies on external timer events instead of the internal timer then there need not be any Esper-managed internal threads.

The fact that event processing can take place within your application thread's call stack makes developing applications with Esper easier: Any .NET integrated development environment (IDE) can host an Esper engine instance. This allows developers to easily set up test cases, debug through listener code and inspect input or output events, or trace their call stack.

In the default configuration, each engine instance maintains a single timer thread (internal timer) providing for time or schedule-based processing within the engine. The default resolution at which the internal timer operates is 100 milliseconds. The internal timer thread can be disabled and applications can instead send external time events to an engine instance to perform timer or scheduled processing at the resolution required by an application.

Each engine instance performs minimal locking to enable high levels of concurrency. An engine instance locks on a statement level to protect statement resources.

For an engine instance to produce predictable results from the viewpoint of event handlers to statements, an engine instance by default ensures that it dispatches statement result events to event handlers in the order in which a statement produced result events. Applications that require the highest possible concurrency and do not require predictable order of delivery of events to event handlers, this feature can be turned off via configuration, see Section 13.4.9.1, “Preserving the order of events delivered to listeners”. For example, assume thread T1 processes an event applied to statement S producing output event O1. Assume thread T2 processes another event applied to statement S and produces output event O2. The engine employs a configurable latch system to ensure that event handlers to statement S receive and may complete processing of O1 before receiving O2. When using outbound threading (advanced threading options) or changing the configuration this guarantee is weakened or removed.

In multithreaded environments, when one or more statements make result events available via the insert into clause to further statements, the engine preserves the order of events inserted into the generated insert-into stream, allowing statements that consume other statement's events to behave deterministic. This feature can also be turned off via configuration, see , see Section 13.4.9.2, “Preserving the order of events for insert-into streams”. For example, assume thread T1 processes an event applied to statement S and thread T2 processes another event applied to statement S. Assume statement S inserts into into stream ST. T1 produces an output event O1 for processing by consumers of ST1 and T2 produces an output event O2 for processing by consumers of ST. The engine employs a configurable latch system such that O1 is processed before O2 by consumers of ST. When using route execution threading (advanced threading options) or changing the configuration this guarantee is weakened or removed.

We generally recommended that listener implementations block minimally or do not block at all. By implementing listener code as non-blocking code execution threads can often achieve higher levels of concurrency.

We recommended that, when using a single listener or subscriber instance to receive output from multiple statements, that the listener or subscriber code is multithread-safe. If your application has shared state between listener or subscriber instances then such shared state should be thread-safe.

12.6.1. Advanced Threading

In the default configuration the same application thread that invokes any of the SendEvent methods will process the event fully and also deliver output events to event handlers and subscribers. By default the single internal timer thread based on system time performs time-based processing and delivery of time-based results.

This default configuration reduces the processing overhead associated with thread context switching, is lightweight and fast and works well in many environments such as J2EE, server or client. Latency and throughput requirements are largely use case dependant, and Esper provides engine-level facilities for controlling concurrency that are described next.

Inbound Threading queues all incoming events: A pool of engine-managed threads performs the event processing. The application thread that sends an event via any of the SendEvent methods returns without blocking.

Outbound Threading queues events for delivery to event handlers and subscribers, such that slow or blocking event handlers or subscribers do not block event processing.

Timer Execution Threading means time-based event processing is performed by a pool of engine-managed threads. With this option the internal timer thread (or external timer event) serves only as a metronome, providing units-of-work to the engine-managed threads in the timer execution pool, pushing threading to the level of each statement for time-based execution.

Route Execution Threading means that the thread sending in an event via any of the SendEvent methods (or the inbound threading pooled thread if inbound threading is enabled) only identifies and pre-processes an event, and a pool of engine-managed threads handles the actual processing of the event for each statement, pushing threading to the level of each statement for event-arrival-based execution.

The engine starts engine-managed threads as daemon threads when the engine instance is first obtained. The engine stops engine-managed threads when the engine instance is destroyed via the Dispose method. When the engine is initialized via the Initialize method the existing engine-managed threads are stopped and new threads are created. When shutting down your application, use the Dispose method to stop engine-managed threads.

Note that the options discussed herein may introduce additional processing overhead into your system, as each option involves work queue management and thread context switching.

If your use cases require ordered processing of events or do not tolerate disorder, the threading options described herein may not be the right choice.

If your use cases require loss-less processing of events, wherein the threading options mean that events are held in an in-memory queue, the threading options described herein may not be the right choice.

Care should be taken to consider arrival rates and queue depth. Threading options utilize unbound queues or capacity-bound queues with blocking-put, depending on your configuration, and may therefore introduce an overload or blocking situation to your application. You may use the service provider interface as outlined below to manage queue sizes, if required, and to help tune the engine to your application needs. Consider throttling down the event send rate when the API (see below) indicates that events are getting queued.

All threading options are on the level of an engine. If you require different threading behavior for certain statements then consider using multiple engine instances, consider using the Route method or consider using application threads instead.

Please consult Section 13.4.9, “Engine Settings related to Concurrency and Threading” for instructions on how to configure threading options. Threading options take effect at engine initialization time.

12.6.1.1. Inbound Threading

With inbound threading an engine places inbound events in a queue for processing by one or more engine-managed threads other then the delivering application threads.

The delivering application thread uses one of the SendEvent methods on EPRuntime to deliver events or may also use the SendEvent method on a EventSender. The engine receives the event and places the event into a queue, allowing the delivering thread to continue and not block while the event is being processed and results are delivered.

Events that are sent into the engine via one of the Route methods are not placed into queue but processed by the same thread invoking the Route operation.

12.6.1.2. Outbound Threading

With outbound threading an engine places outbound events in a queue for delivery by one or more engine-managed threads other then the processing thread originating the result.

With outbound threading your listener or subscriber class receives statement results from one of the engine-managed threads in the outbound pool of threads. This is useful when you expect your listener or subscriber code to perform significantly blocking operations and you do not want to hold up event processing.

12.6.1.3. Timer Execution Threading

With timer execution threading an engine places time-based work units into a queue for processing by one or more engine-managed threads other then the internal timer thread or the application thread that sends an external timer event.

Using timer execution threading the internal timer thread (or thread delivering an external timer event) serves to evaluate which time-based work units must be processed. A pool of engine-managed threads performs the actual processing of time-based work units and thereby offloads the work from the internal timer thread (or thread delivering an external timer event).

Enable this option as a tuning parameter when your statements utilize time-based patterns or data windows. Timer execution threading is fine grained and works on the level of a time-based schedule in combination with a statement.

12.6.1.4. Route Execution Threading

With route execution threading an engine identifies event-processing work units based on the event and statement combination. It places such work units into a queue for processing by one or more engine-managed threads other then the thread that originated the event.

While inbound threading works on the level of an event, route execution threading is fine grained and works on the level of an event in combination with a statement.

12.6.1.5. Threading Service Provider Interface

The service-provider interface EPServiceProviderSPI is an extension API that allows to manage engine-level queues and thread pools .

The service-provider interface EPServiceProviderSPI is considered an extension API and subject to change between release versions.

The following code snippet shows how to obtain the BlockingQueue<Runnable> and the ThreadPoolExecutor for the managing the queue and thread pool responsible for inbound threading:

EPServiceProviderSPI spi = (EPServiceProviderSPI) epService;
int queueSize = spi.getThreadingService().getInboundQueue().size();
ThreadPoolExecutor threadpool = spi.getThreadingService().getInboundThreadPool();

12.6.2. Processing Order

12.6.2.1. Competing Statements

This section discusses the order in which N competing statements that all react to the same arriving event execute.

The engine, by default, does not guarantee to execute competing statements in any particular order unless using @Priority. We therefore recommend that an application does not rely on the order of execution of statements by the engine, since that best shields the behavior of an application from changes in the order that statements may get created by your application or by threading configurations that your application may change at will.

If your application requires a defined order of execution of competing statements, use the @Priority EPL syntax to make the order of execution between statements well-defined (requires that you set the prioritized-execution configuration setting). And the @Drop can make a statement preempt all other lowered priority ones that then won't get executed for any matching events.

12.6.2.2. Competing Events in a Work Queue

This section discusses the order of event evaluation when multiple events must be processed, for example when multiple statements use insert-into to generate further events upon arrival of an event.

The engine processes an arriving event completely before considering output events generated by insert-into or routed events inserted by event handlers or subscribers.

For example, assume three statements (1) select * from MyEvent and (2) insert into ABCStream select * from MyEvent. (3) select * from ABCStream. When a MyEvent event arrives then the event handlers to statements (1) and (2) execute first (default threading model). Event handlers to statement (3) which receive the inserted-into stream events are always executed after delivery of the triggering event.

Among all events generated by insert-into of statements and the events routed into the engine via the Route method, all events that insert-into a named window are processed first in the order generated. All other events are processed thereafter in the order they were generated.

When enabling timer or route execution threading as explained under advanced threading options then the engine does not make any guarantee to the processing order except that is will prioritize events inserted into a named window.

12.7. Controlling Time-Keeping

There are two modes for an engine to keep track of time: The internal timer based on JVM system time (the default), and externally-controlled time giving your application full control over the concept of time within an engine or isolated service.

An isolated service is an execution environment separate from the main engine runtime, allowing full control over the concept of time for a group of statements, as further described in Section 12.9, “Service Isolation”.

By default the internal timer provides time and evaluates schedules. External clocking can be used to supply time ticks to the engine instead. The latter is useful for testing time-based event sequences or for synchronizing the engine with an external time source.

The internal timer relies on the windows multimedia timer (default) for time tick events. The windows multimedia timer provides the finest granularity available on a Windows based platform (1 millisecond) but requires integration of the multimedia DLL. The next section describes timer resolution for the internal timer, by default set to 100 milliseconds but is configurable via the threading options. When using externally-controlled time the timer resolution is in your control.

To disable the internal timer and use externally-provided time instead, there are two options. The first option is to use the configuration API at engine initialization time. The second option toggles on and off the internal timer at runtime, via special timer control events that are sent into the engine like any other event.

If using a timer execution thread pool as discussed above, the internal timer or external time event provide the schedule evaluation however do not actually perform the time-based processing. The time-based processing is performed by the threads in the timer execution thread pool.

This code snippet shows the use of the configuration API to disable the internal timer and thereby turn on externally-provided time (see the Configuration section for configuration via XML file):

Configuration config = new Configuration();
config.EngineDefaults.Threading.IsInternalTimerEnabled = false;
EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider(config);

After disabling the internal timer, it is wise to set a defined time so that any statements created thereafter start relative to the time defined. Use the CurrentTimeEvent class to indicate current time to the engine and to move time forward for the engine.

This code snippet obtains the current time and sends a timer event in:

long timeInMillis = DateTimeHelper.GetCurrentTimeMillis();
CurrentTimeEvent timeEvent = new CurrentTimeEvent(timeInMillis);
epService.EPRuntime.SendEvent(timeEvent);

Alternatively, you can use special timer control events to enable or disable the internal timer. Use the TimerControlEvent class to control timer operation at runtime.

The next code snippet demonstrates toggling to external timer at runtime, by sending in a TimerControlEvent event:

EPServiceProvider epService = EPServiceProviderManager.GetDefaultProvider();
EPRuntime runtime = epService.EPRuntime;
// switch to external clocking
runtime.SendEvent(new TimerControlEvent(TimerControlEvent.ClockType.CLOCK_EXTERNAL));

Your application sends a CurrentTimeEvent event when it desires to move the time forward. All aspects of Esper engine time related to EPL statements and patterns are driven by the time provided by the CurrentTimeEvent that your application sends in.

The next example sequence of instructions sets time to zero, then creates a statement, then moves time forward to 1 seconds later and then 6 seconds later:

// Set start time at zero.
runtime.SendEvent(new CurrentTimeEvent(0));

// create a statement here
epAdministrator.CreateEPL("select * from MyEvent output every 5 seconds");

// move time forward 1 second
runtime.SendEvent(new CurrentTimeEvent(1000));

// move time forward 5 seconds
runtime.SendEvent(new CurrentTimeEvent(6000));

When sending external timer events, your application should make sure that long-type time values are ascending. That is, each long-type value should be either the same value or a larger value then the prior value provided by a CurrentTimeEvent. The engine outputs a warning if time events move back in time.

Your application may use the NextScheduledTime property in EPRuntime to determine the earliest time a schedule for any statement requires evaluation.

The following code snippet sets the current time, creates a statement and prints the next scheduled time which is 1 minute later then the current time:

// Set start time to the current time.
runtime.SendEvent(new CurrentTimeEvent(System.currentTimeMillis()));

// Create a statement.
epService.EPAdministrator.CreateEPL("select * from pattern[timer:interval(1 minute)]");

// Print next schedule time
Console.WriteLine("Next schedule at {0}", new Date(runtime.NextScheduledTime);

12.7.1. Controlling Time Using Time Span Events

With CurrentTimeEvent, as described above, your application can advance engine time to a given point in time. In addition, the NextScheduledTime property in EPRuntime returns the next scheduled time according to started statements. You would typically use CurrentTimeEvent to advance time at a relatively high resolution.

To advance time for a span of time without sending individual CurrentTimeEvent events to the engine, the API provides the class CurrentTimeSpanEvent. You may use CurrentTimeSpanEvent with or without a resolution.

If your application only provides the target end time of time span to CurrentTimeSpanEvent and no resolution, the engine advances time up to the target time by stepping through all relevant times according to started statements.

If your application provides the target end time of time span and in addition a long-typed resolution, the engine advances time up to the target time by incrementing time according to the resolution (regardless of next scheduled time according to started statements).

Consider the following example:

// Set start time to Jan.1, 2010, 00:00 am for this example
DateTime startTime = DateTime.Parse("2010 01 01 00:00:00 000", "yyyy MM dd HH:mm:ss FFF");
runtime.SendEvent(new CurrentTimeEvent(startTime.InMillis()));

// Create a statement.
EPStatement stmt = epService.EPAdministrator.CreateEPL("select current_timestamp() as ct " +
  "from pattern[every timer:interval(1 minute)]");
stmt.Events += ...;	// add an event handler

// Advance time to 10 minutes after start time
runtime.SendEvent(new CurrentTimeSpanEvent(startTime.InMillis() + 10*60*1000));

The above example advances time to 10 minutes after the time set using CurrentTimeSpanEvent. As the example does not pass a resolution, the engine advances time according to statement schedules. Upon sending the CurrentTimeSpanEvent the listener sees 10 invocations for minute 1 to minute 10.

To advance time according to a given resolution, you may provide the resolution as shown below:

// Advance time to 10 minutes after start time at 100 msec resolution
runtime.SendEvent(new CurrentTimeSpanEvent(startTime.getTime() + 10*60*1000, 100));

12.7.2. Additional Time-Related APIs

Consider using the service-provider interface EPRuntimeSPI EPRuntimeIsolatedSPI. The two interfaces are service-provider interfaces that expose additional function to manage statement schedules. However the SPI interfaces should be considered an extension API and are subject to change between release versions.

Additional engine-internal SPI interfaces can be obtained by downcasting EPServiceProvider to EPServiceProviderSPI. For example the SchedulingServiceSPI exposes schedule information per statement (downcast from SchedulingService). Engine-internal SPI are subject to change between versions.

12.8. Time Resolution

The minimum resolution that all data windows, patterns and output rate limiting operate at is the millisecond. Parameters to time window views, pattern operators or the output clause that are less then 1 millisecond are not allowed. As stated earlier, the default frequency at which the internal timer operates is 100 milliseconds (configurable).

The internal timer thread, by default, uses the call DateTime.Now to obtain system time. Please see the JIRA issue ESPER-191 Support nano/microsecond resolution for more information on system time-call performance, accuracy and drift.

The internal timer thread can be configured to use nano-second time as returned by com.espertech.esper.compat.HighResolutionTimeProvider. If configured for nano-second time, the engine computes an offset of the nano-second ticks to wall clock time upon startup to present back an accurate millisecond wall clock time. Please see section Section 13.4.15, “Engine Settings related to Time Source” to configure the internal timer thread to use HighPerformanceTimer.

Your application can achieve a higher tick rate then 1 tick per millisecond by sending external timer events that carry a long-value which is not based on milliseconds since January 1, 1970, 00:00:00 GMT. In this case, your time interval parameters need to take consideration of the changed use of engine time.

Thus, if your external timer events send long values that represents microseconds (1E-6 sec), then your time window interval must be 1000-times larger, i.e. "win:time(1000)" becomes a 1-second time window.

And therefore, if your external timer events send long values that represents nanoseconds (1E-9 sec), then your time window interval must be 1000000-times larger, i.e. "win:time(1000000)" becomes a 1-second time window.

12.9. Service Isolation

12.9.1. Overview

An isolated service allows an application to control event visibility and the concept of time as desired on a statement level: Events sent into an isolated service are visible only to those statements that currently reside in the isolated service and are not visible to statements outside of that isolated service. Within an isolated service an application can control time independently, start time at a point in time and advance time at the resolution and pace suitable for the statements added to that isolated service.

As discussed before, a single application domain may hold multiple Esper engine instances unique by engine URI. Within an Esper engine instance the default execution environment for statements is the EPRuntime engine runtime, which coordinates all statement's reaction to incoming events and to time passing (via internal or external timer).

Subordinate to an Esper engine instance, your application can additionally allocate multiple isolated services (or execution environments), uniquely identified by a name and represented by the EPServiceProviderIsolated interface. In the isolated service, time passes only when you application sends timer events to the EPRuntimeIsolated instance. Only events explicitly sent to the isolated service are visible to statements added.

Your application can create new statements that start in an isolated service. You can also move existing statements back and forth between the engine and an isolated service.

Isolation does not apply to variables: Variables are global in nature. Also, as named windows are globally visibly data windows, consumers to named windows see changes in named windows even though a consumer or the named window (through the create statement) may be in an isolated service.

An isolated service allows an application to:

  1. Suspend a statement without loosing its statement state that may have accumulated for the statement.

  2. Control the concept of time separately for a set of statements, for example to simulate, backtest, adjust arrival order or compute arrival time.

  3. Initialize statement state by replaying events, without impacting already running statements, to catch-up statements from historical events for example.

While a statement resides in an isolated runtime it receives only those events explicitly sent to the isolated runtime, and performs time-based processing based on the timer events provided to that isolated runtime.

Use the GetEPServiceIsolated method on EPServiceProvider passing a name to obtain an isolated runtime:

EPServiceProviderIsolated isolatedService = epServiceManager.GetEPServiceIsolated("name");

Set the start time for your isolated runtime via the CurrentTimeEvent timer event:

// In this example start the time at the system time
long startInMillis = DateTimeHelper.GetCurrentTimeMillis();	
isolatedService.EPRuntime.SendEvent(new CurrentTimeEvent(startInMillis));

Use the AddStatement method on EPAdministratorIsolated to move an existing statement out of the engine runtime into the isolated runtime:

// look up the existing statement
EPStatement stmt = epServiceManager.EPAdministrator.GetStatement("MyStmt");

// move it to an isolated service
isolatedService.EPAdministrator.AddStatement(stmt);

To remove the statement from isolation and return the statement back to the engine runtime, use the RemoveStatement method on EPAdministratorIsolated:

isolatedService.EPAdministrator.RemoveStatement(stmt);

To create a new statement in the isolated service, use the CreateEPL method on EPAdministratorIsolated:

isolatedService.EPAdministrator.CreateEPL(
  "@Name('MyStmt') select * from Event", null, null); 
// the example is passing the statement name in an annotation and no user object

The Dispose method on EPServiceProviderIsolated moves all currently-isolated statements for that isolated service provider back to engine runtime.

When moving a statement between engine runtime and isolated service or back, the algorithm ensures that events are aged according to the time that passed and time schedules stay intact.

To use isolated services, your configuration must have view sharing disabled as described in Section 13.4.11.1, “Sharing View Resources between Statements”.

12.9.2. Example: Suspending a Statement

By adding an existing statement to an isolated service, the statement's processing effectively becomes suspended. Time does not pass for the statement and it will not process events, unless your application explicitly moves time forward or sends events into the isolated service.

First, let's create a statement and send events:

EPStatement stmt = epServiceManager.EPAdministrator.CreateEPL("select * from TemperatureEvent.win:time(30)");
epServiceManager.EPRuntime.Send(new TemperatureEvent(...));
// send some more events over time

The steps to suspend the previously created statement are as follows:

EPServiceProviderIsolated isolatedService = epServiceManager.GetEPServiceIsolated("suspendedStmts");
isolatedService.EPAdministrator.AddStatement(stmt);

To resume the statement, move the statement back to the engine:

isolatedService.EPAdministrator.RemoveStatement(stmt);

If the statement employed a time window, the events in the time window did not age. If the statement employed patterns, the pattern's time-based schedule remains unchanged. This is because the example did not advance time in the isolated service.

12.9.3. Example: Catching up a Statement from Historical Data

This example creates a statement in the isolated service, replays some events and advances time, then merges back the statement to the engine to let it participate in incoming events and engine time processing.

First, allocate an isolated service and explicitly set it to a start time. Assuming that myStartTime is a long millisecond time value that marks the beginning of the data to replay, the sequence is as follows:

EPServiceProviderIsolated isolatedService = epServiceManager.GetEPServiceIsolated("suspendedStmts");
isolatedService.EPRuntime.SendEvent(new CurrentTimeEvent(myStartTime));

Next, create the statement. The sample statement is a pattern statement looking for temperature events following each other within 60 seconds:

EPStatement stmt = epAdmin.CreateEPL(
  "select * from pattern[every a=TemperatureEvent -> b=TemperatureEvent where timer:within(60)]");

For each historical event to be played, advance time and send an event. This code snippet assumes that currentTime is a time greater then myStartTime and reflects the time that the historical event should be processed at. It also assumes historyEvent is the historical event object.

isolatedService.EPRuntime.SendEvent(new CurrentTimeEvent(currentTime));
isolatedService.EPRuntime.Send(historyEvent);
// repeat the above advancing time until no more events

Finally, when done replaying events, merge the statement back with the engine:

isolatedService.EPAdministrator.RemoveStatement(stmt);

12.9.4. Isolation for Insert-Into

When isolating statements, events that are generated by insert into are visible within the isolated service that currently holds that insert into statement.

For example, assume the below two statements named A and B:

@Name('A') insert into MyStream select * from MyEvent
@Name('B') select * from MyStream

When adding statement A to an isolated service, and assuming a MyEvent is sent to either the engine runtime or the isolated service, a listener to statement B does not receive that event.

When adding statement B to an isolated service, and assuming a MyEvent is sent to either the engine runtime or the isolated service, a listener to statement B does not receive that event.

12.9.5. Isolation for Named Windows

When isolating named windows, the event visibility of events entering and leaving from a named window is not limited to the isolated service. This is because named windows are global data windows (a relation in essence).

For example, assume the below three statements named A, B and C:

@Name('A') create window MyNamedWindow.win:time(60) as select * from MyEvent
@Name('B') insert into MyNamedWindow select * from MyEvent
@Name('C') select * from MyNamedWindow

When adding statement A to an isolated service, and assuming a MyEvent is sent to either the engine runtime or the isolated service, a listener to statement A and C does not receive that event.

When adding statement B to an isolated service, and assuming a MyEvent is sent to either the engine runtime or the isolated service, a listener to statement A and C does not receive that event.

When adding statement C to an isolated service, and assuming a MyEvent is sent to the engine runtime, a listener to statement A and C does receive that event.

12.9.6. Runtime Considerations

Moving statements between an isolated service and the engine is an expensive operation and should not be performed with high frequency.

When using multiple threads to send events and at the same time moving a statement to an isolated service, it its undefined whether events will be delivered to a listener of the isolated statement until all threads completed sending events.

Metrics reporting is not available for statements in an isolated service. Advanced threading options are also not available in the isolated service, however it is thread-safe to send events including timer events from multiple threads to the same or different isolated service.

12.10. Exception Handling

You may register one or more exception handlers for the engine to invoke in the case it encounters an exception processing a continuously-executing statement. By default and without exception handlers the engine cancels execution of the current EPL statement that encountered the exception, logs the exception and continues to the next statement, if any. The configuration is described in Section 13.4.20, “Engine Settings related to Exception Handling”.

If your application registers exception handlers as part of engine configuration, the engine invokes the exception handlers in the order they are registered passing relevant exception information such as EPL statement name, expression and the exception itself.

Exception handlers receive any EPL statement unchecked exception such as internal exceptions or exceptions thrown by plug-in aggregation functions or plug-in views. The engine does not provide to exception handlers any exceptions thrown by static method invocations for function calls, method invocations in joins, methods on event classes and event handlers or subscriber exceptions.

An exception handler can itself throw a runtime exception to cancel execution of the current event against any further statements.

For on-demand queries the API indicates any exception directly back to the caller without the exception handlers being invoked, as exception handlers apply to continuous queries only. The same applies to any API calls other then SendEvent and the EventSender methods.

As the configuration section describes, your application registers one or more classes that implement the ExceptionHandlerFactory interface in the engine configuration. Upon engine initialization the engine obtains a factory instance from the class name that then provides the exception handler instance. The exception handler class must implement the ExceptionHandler interface.

12.11. Condition Handling

You may register one or more condition handlers for the engine to invoke in the case it encounters certain conditions, as outlined below, when executing a statement. By default and without condition handlers the engine logs the condition at informational level and continues processing. The configuration is described in Section 13.4.21, “Engine Settings related to Condition Handling”.

If your application registers condition handlers as part of engine configuration, the engine invokes the condition handlers in the order they are registered passing relevant condition information such as EPL statement name, expression and the condition information itself.

Currently the only condition indicated by this facility is raised by the pattern followed-by operator when used with the limit expression and when the limit is reached, see Section 5.5.8.1, “Limiting Sub-Expression Count”.

A condition handler may not itself throw a runtime exception or return any value.

As the configuration section describes, your application registers one or more classes that implement the ConditionHandlerFactory interface in the engine configuration. Upon engine initialization the engine obtains a factory instance from the class name that then provides the condition handler instance. The condition handler class must implement the ConditionHandler interface.

12.12. Statement Object Model

The statement object model is a set of classes that provide an object-oriented representation of an EPL or pattern statement. The object model classes are found in package com.espertech.esper.client.soda. An instance of EPStatementObjectModel represents a statement's object model.

The statement object model classes are a full and complete specification of a statement. All EPL and pattern constructs including expressions and sub-queries are available via the statement object model.

In conjunction with the administrative API, the statement object model provides the means to build, change or interrogate statements beyond the EPL or pattern syntax string representation. The object graph of the statement object model is fully navigable for easy querying by code, and is also serializable allowing applications to persist or transport statements in object form, when required.

The statement object model supports full round-trip from object model to EPL statement string and back to object model: A statement object model can be rendered into an EPL string representation via the ToEPL method on EPStatementObjectModel. Further, the administrative API allows to compile a statement string into an object model representation via the CompileEPL method on EPAdministrator.

The Create method on EPAdministrator creates and starts a statement as represented by an object model. In order to obtain an object model from an existing statement, obtain the statement expression text of the statement via the Text property on EPStatement and use the CompileEPL method to obtain the object model.

The following limitations apply:

  • Statement object model classes are not safe for sharing between threads other then for read access.

  • Between versions of Esper, the serialized form of the object model is subject to change. Esper makes no guarantees that the serialized object model of one version will be fully compatible with the serialized object model generated by another version of Esper. Please consider this issue when storing Esper object models in persistent store.

12.12.1. Building an Object Model

A EPStatementObjectModel consists of an object graph representing all possible clauses that can be part of an EPL statement.

Among all clauses, the SelectClause and FromClause objects are required clauses that must be present, in order to define what to select and where to select from.

Table 12.6. Required Statement Object Model Instances

ClassDescription
EPStatementObjectModelAll statement clauses for a statement, such as the select-clause and the from-clause, are specified within the object graph of an instance of this class
SelectClauseA list of the selection properties or expressions, or a wildcard
FromClauseA list of one or more streams; A stream can be a filter-based, a pattern-based or a SQL-based stream; Views are added to streams to provide data window or other projections

Part of the statement object model package are convenient builder classes that make it easy to build a new object model or change an existing object model. The SelectClause and FromClause are such builder classes and provide convenient Create methods.

Within the from-clause we have a choice of different streams to select on. The FilterStream class represents a stream that is filled by events of a certain type and that pass an optional filter expression.

We can use the classes introduced above to create a simple statement object model:

EPStatementObjectModel model = new EPStatementObjectModel();
model.SelectClause = SelectClause.CreateWildcard();
model.FromClause = FromClause.Create(FilterStream.Create("com.chipmaker.ReadyEvent"));

The model as above is equivalent to the EPL :

select * from com.chipmaker.ReadyEvent

Last, the code snippet below creates a statement from the object model:

EPStatement stmt = epService.EPAdministrator.Create(model);

Notes on usage:

  • Variable names can simply be treated as property names.

  • When selecting from named windows, the name of the named window is the event type name for use in FilterStream instances or patterns.

  • To compile an arbitrary sub-expression text into an Expression object representation, simply add the expression text to a where clause, compile the EPL string into an object model via the compileEPL on EPAdministrator, and obtain the compiled where from the EPStatementObjectModel via the WhereClause property.

12.12.2. Building Expressions

The EPStatementObjectModel includes an optional where-clause. The where-clause is a filter expression that the engine applies to events in one or more streams. The key interface for all expressions is the Expression interface.

The Expressions class provides a convenient way of obtaining Expression instances for all possible expressions. Please consult the SDK documentatin for detailed method information. The next example discusses sample where-clause expressions.

Use the Expressions class as a service for creating expression instances, and add additional expressions via the Add method that most expressions provide.

In the next example we add a simple where-clause to the EPL as shown earlier:

select * from com.chipmaker.ReadyEvent where line=8

And the code to add a where-clause to the object model is below.

model.WhereClause = Expressions.Eq("line", 8);

The following example considers a more complex where-clause. Assume we need to build an expression using logical-and and logical-or:

select * from com.chipmaker.ReadyEvent 
where (line=8) or (line=10 and age<5)

The code for building such a where-clause by means of the object model classes is:

model.WhereClause = Expressions.Or()
  .Add(Expressions.Eq("line", 8))
  .Add(Expressions.And()
      .Add(Expressions.Eq("line", 10))
      .Add(Expressions.Lt("age", 5))
  );

12.12.3. Building a Pattern Statement

The Patterns class is a factory for building pattern expressions. It provides convenient methods to create all pattern expressions of the pattern language.

Patterns in EPL are seen as a stream of events that consist of patterns matches. The PatternStream class represents a stream of pattern matches and contains a pattern expression within.

For instance, consider the following pattern statement.

select * from pattern [every a=MyAEvent and not b=MyBEvent]

The next code snippet outlines how to use the statement object model and specifically the Patterns class to create a statement object model that is equivalent to the pattern statement above.

EPStatementObjectModel model = new EPStatementObjectModel();
model.SelectClause = SelectClause.CreateWildcard();
PatternExpr pattern = Patterns.And()
  .Add(Patterns.EveryFilter("MyAEvent", "a"))
  .Add(Patterns.NotFilter("MyBEvent", "b"));
model.FromClause = FromClause.Create(PatternStream.Create(pattern));

12.12.4. Building a Select Statement

In this section we build a complete example statement and include all optional clauses in one EPL statement, to demonstrate the object model API.

A sample statement:

insert into ReadyStreamAvg(line, avgAge) 
select line, avg(age) as avgAge 
from com.chipmaker.ReadyEvent(line in (1, 8, 10)).win:time(10) as RE
where RE.waverId != null
group by line 
having avg(age) < 0
output every 10.0 seconds 
order by line

Finally, this code snippet builds the above statement from scratch:

EPStatementObjectModel model = new EPStatementObjectModel();
model.InsertInto = InsertIntoClause.create("ReadyStreamAvg", "line", "avgAge");
model.SelectClause = SelectClause.Create()
    .Add("line")
    .Add(Expressions.Avg("age"), "avgAge");
Filter filter = Filter.Create("com.chipmaker.ReadyEvent", Expressions.In("line", 1, 8, 10));
model.FromClause = FromClause.Create(
    FilterStream.Create(filter, "RE").AddView("win", "time", 10));
model.WhereClause = Expressions.IsNotNull("RE.waverId");
model.GroupByClause = GroupByClause.Create("line");
model.HavingClause = Expressions.Lt(Expressions.Avg("age"), Expressions.Constant(0));
model.OutputLimitClause = OutputLimitClause.Create(OutputLimitSelector.DEFAULT, Expressions.TimePeriod(null, null, null, 10.0, null));
model.OrderByClause = OrderByClause.Create("line");

12.12.5. Building a Create-Variable and On-Set Statement

This sample statement creates a variable:

create variable integer var_output_rate = 10

The code to build the above statement using the object model:

EPStatementObjectModel model = new EPStatementObjectModel();
model.CreateVariable = CreateVariableClause.Create("integer", "var_output_rate", 10));
epService.EPAdministrator.Create(model);

A second statement sets the variable to a new value:

on NewValueEvent set var_output_rate = new_rate

The code to build the above statement using the object model:

EPStatementObjectModel model = new EPStatementObjectModel();
model.OnExpr = OnClause.CreateOnSet("var_output_rate", Expressions.Property("new_rate"));
model.FromClause = FromClause.Create(FilterStream.Create("NewValueEvent"));
EPStatement stmtSet = epService.EPAdministrator.Create(model);

12.12.6. Building Create-Window, On-Delete and On-Select Statements

This sample statement creates a named window:

create window OrdersTimeWindow.win:time(30 sec) as select symbol as sym, volume as vol, price from OrderEvent

The is the code that builds the create-window statement as above:

EPStatementObjectModel model = new EPStatementObjectModel();
model.CreateWindow = CreateWindowClause.Create("OrdersTimeWindow");
model.CreateWindow.AddView("win", "time", 30);
model.SelectClause = SelectClause.Create()
		.AddWithName("symbol", "sym")
		.AddWithName("volume", "vol")
		.Add("price");
model.FromClause = FromClause.Create(FilterStream.Create("OrderEvent));

A second statement deletes from the named window:

on NewOrderEvent as myNewOrders
delete from AllOrdersNamedWindow as myNamedWindow
where myNamedWindow.symbol = myNewOrders.symbol

The object model is built by:

EPStatementObjectModel model = new EPStatementObjectModel();
model.OnExpr = OnClause.CreateOnDelete("AllOrdersNamedWindow", "myNamedWindow");
model.FromClause = FromClause.Create(FilterStream.Create("NewOrderEvent", "myNewOrders"));
model.WhereClause = Expressions.EqProperty("myNamedWindow.symbol", "myNewOrders.symbol");
EPStatement stmtOnDelete = epService.EPAdministrator.Create(model);

A third statement selects from the named window using the non-continuous on-demand selection via on-select:

on QueryEvent(volume>0) as query
select count(*) from OrdersNamedWindow as win
where win.symbol = query.symbol

The on-select statement is built from scratch via the object model as follows:

EPStatementObjectModel model = new EPStatementObjectModel();
model.OnExpr = OnClause.CreateOnSelect("OrdersNamedWindow", "win");
model.WhereClause = Expressions.EqProperty("win.symbol", "query.symbol");
model.FromClause = FromClause.Create(FilterStream.Create("QueryEvent", "query", 
  Expressions.Gt("volume", 0)));
model.SelectClause = SelectClause.Create().Add(Expressions.CountStar());
EPStatement stmtOnSelect = epService.EPAdministrator.Create(model);

12.13. Prepared Statement and Substitution Parameters

The Prepare method that is part of the administrative API pre-compiles an EPL statement and stores the precompiled statement in an EPPreparedStatement object. This object can then be used to efficiently start the parameterized statement multiple times.

Substitution parameters are inserted into an EPL statement as a single question mark character '?'. The engine assigns the first substitution parameter an index of 1 and subsequent parameters increment the index by one.

Substitution parameters can be inserted into any EPL construct that takes an expression. They are therefore valid in any clauses such as the select-clause, from-clause filters, where-clause, group-by-clause, having-clause or order-by-clause, including view parameters and pattern observers and guards. Substitution parameters cannot be used where a numeric constant is required rather then an expression and in SQL statements.

All substitution parameters must be replaced by actual values before a statement with substitution parameters can be started. Substitution parameters can be replaced with an actual value using the SetObject method for each index. Substitution parameters can be set to new values and new statements can be created from the same EPPreparedStatement object more then once.

While the SetObject method allows substitution parameters to assume any actual value including application CLR objects or enumeration values, the application must provide the correct type of substitution parameter that matches the requirements of the expression the parameter resides in.

In the following example of setting parameters on a prepared statement and starting the prepared statement, epService represents an engine instance:

String stmt = "select * from com.chipmaker.ReadyEvent(line=?)";
EPPreparedStatement prepared = epService.EPAdministrator.PrepareEPL(stmt);
prepared.SetObject(1, 8);
EPStatement statement = epService.EPAdministrator.Create(prepared);

12.14. Engine and Statement Metrics Reporting

Metrics reporting is a feature that allows an application to receive ongoing reports about key engine-level and statement-level metrics. Examples are the number of incoming events, the CPU time and wall time taken by statement executions or the number of output events per statement.

Metrics reporting is, by default, disabled. To enable reporting, please follow the steps as outlined in Section 13.4.16, “Engine Settings related to Metrics Reporting”. Metrics reporting must be enabled at engine initialization time. Reporting intervals can be controlled at runtime via the ConfigurationOperations interface available from the administrative API.

Your application receives metrics at configurable intervals via EPL statement. A metric datapoint is simply a well-defined event. The events are EngineMetric and StatementMetric and the type representing the events can be found in the client API in package com.espertech.esper.client.metric.

Since metric events are processed by the engine the same as application events, your EPL may use any construct on such events. For example, your application may select, filter, aggregate properties, sort or insert into a stream or named window all metric events the same as application events.

This example statement selects all engine metric events:

select * from com.espertech.esper.client.metric.EngineMetric

The next statement selects all statement metric events:

select * from com.espertech.esper.client.metric.StatementMetric

Make sure to have metrics reporting enabled since only then do event handlers or subscribers to a statement such as above receive metric events.

The engine provides metric events after the configured interval of time has passed. By default, only started statements that have activity within an interval (in the form of event or timer processing) are reported upon.

The default configuration performs the publishing of metric events in an Esper daemon thread under the control of the engine instance. Metrics reporting honors externally-supplied time, if using external timer events.

Via runtime configuration options provided by ConfigurationOperations, your application may enable and disable metrics reporting globally, provided that metrics reporting was enabled at initialization time. Your application may also enable and disable metrics reporting for individual statements by statement name.

Statement groups is a configuration feature that allows to assign reporting intervals to statements. Statement groups are described further in the Section 13.4.16, “Engine Settings related to Metrics Reporting” section. Statement groups cannot be added or removed at runtime.

The following limitations apply:

  • High-performance numbers are obtained through the use of a PerformanceObserver that uses the Kernel32 DLL. While this method has been exhaustively tested on Windows, we are aware that this will not work in non-Windows environments. To ensure compatability with Mono, we have included a model based on DateTime. This implementation will not provide the level of granularity or clarity that the Windows implementation will.

  • There is a performance cost to collecting and reporting metrics.

  • Not all statements may report metrics: The engine performs certain runtime optimizations sharing resources between similar statements, thereby not reporting on certain statements unless resource sharing is disabled through configuration.

12.14.1. Engine Metrics

Engine metrics are properties of EngineMetric events:

Table 12.7. EngineMetric Properties

NameDescription
engineURIThe URI of the engine instance.
timestampThe current engine time.
inputCountCumulative number of input events since engine initialization time. Input events are defined as events send in via application threads as well as insert into events.
inputCountDeltaNumber of input events since last reporting period.
scheduleDepthNumber of outstanding schedules.

12.14.2. Statement Metrics

Statement metrics are properties of StatementMetric. The properties are:

Table 12.8. StatementMetric Properties

NameDescription
engineURIThe URI of the engine instance.
timestampThe current engine time.
statementNameStatement name, if provided at time of statement creation, otherwise a generated name.
cpuTimeStatement processing CPU time (system and user) in nanoseconds.
wallTimeStatement processing wall time in nanoseconds (based on System.nanoTime).
numInputNumber of input events to the statement.
numOutputIStreamNumber of insert stream rows output to event handlers or the subscriber, if any.
numOutputRStreamNumber of remove stream rows output to event handlers or the subscriber, if any.

The totals reported are cumulative relative to the last metric report.

12.15. Event Rendering to XML and JSON

Your application may use the built-in XML and JSON formatters to render output events into a readable textual format, such as for integration or debugging purposes. This section introduces the utility classes in the client util package for rendering events to strings. Further API information can be found in the SDK documentation.

The IEventRenderer interface accessible from the runtime interface via the GetEventRenderer method provides access to JSON and XML rendering. For repeated rendering of events of the same event type or subtypes, it is recommended to obtain a JSONEventRenderer or XMLEventRenderer instance and use the Render method provided by the interface. This allows the renderer implementations to cache event type metadata for fast rendering.

In this example we show how one may obtain a renderer for repeated rendering of events of the same type, assuming that statement is an instance of EPStatement:

JSONEventRenderer jsonRenderer = epService.EPRuntime.
    EventRenderer.GetJSONRenderer(statement.EventType);

Assuming that event is an instance of EventBean, this code snippet renders an event into the JSON format:

String jsonEventText = jsonRenderer.Render("MyEvent", @event);

The XML renderer works the same:

XMLEventRenderer xmlRenderer = epService.EPRuntime.
    EventRenderer.GetXMLRenderer(statement.EventType);

...and...

String xmlEventText = xmlRenderer.Render("MyEvent", @event);

If the event type is not known in advance or if you application does not want to obtain a renderer instance per event type for fast rendering, your application can use one of the following methods to render an event to a XML or JSON textual format:

String json = epService.EPRuntime.EventRenderer.RenderJSON(@event);
String xml = epService.EPRuntime.EventRenderer.RenderXML(@event);

Use the JSONRenderingOptions or XMLRenderingOptions classes to control how events are rendered.

12.15.1. JSON Event Rendering Conventions and Options

The JSON renderer produces JSON text according to the standard documented at http://www.json.org.

The renderer formats simple properties as well as nested properties and indexed properties according to the JSON string encoding, array encoding and nested object encoding requirements.

The renderer does render indexed properties, it does not render indexed properties that require an index, i.e. if your event representation is backed by native objects and your getter method is getValue(int index), the indexed property values are not part of the JSON text. This is because the implementation has no way to determine how many index keys there are. A workaround is to have a method such as Object[] getValue() instead.

The same is true for mapped properties that the renderer also renders. If a property requires a Map key for access, i.e. your getter method is getValue(String key), such property values are not part of the result text as there is no way for the implementation to determine the key set.

12.15.2. XML Event Rendering Conventions and Options

The XML renderer produces well-formed XML text according to the XML standard.

The renderer can be configured to format simple properties as attributes or as elements. Nested properties and indexed properties are always represented as XML sub-elements to the root or parent element.

The root element name provided to the XML renderer must be the element name of the root in the XML document and may include namespace instructions.

The renderer does render indexed properties, it does not render indexed properties that require an index, i.e. if your event representation is backed by native objects and your getter method is GetValue(int index), the indexed property values are not part of the XML text. This is because the implementation has no way to determine how many index keys there are. A workaround is to have a method such as Object[] GetValue() instead.

The same is true for mapped properties that the renderer also renders. If a property requires a Map key for access, i.e. your getter method is GetValue(String key), such property values are not part of the result text as there is no way for the implementation to determine the key set.

12.16. Plug-in Loader

A plug-in loader is for general use with input adapters, output adapters or EPL code deployment or any other task that can benefits from being part of an Esper configuration file and that follows engine lifecycle.

A plug-in loader implements the com.espertech.esper.plugin.PluginLoader interface and can be listed in the configuration.

Each configured plug-in loader follows the engine instance lifecycle: When an engine instance initializes, it instantiates each PluginLoader implementation class listed in the configuration. The engine then invokes the lifecycle methods of the PluginLoader implementation class before and after the engine is fully initialized and before an engine instance is destroyed.

Declare a plug-in loader in your configuration XML as follows:

...
  <plugin-loader name="MyLoader" class-name="org.mypackage.MyLoader">
    <init-arg name="property1" value="val1"/>
  </plugin-loader>
...

Alternatively, add the plug-in loader via the configuration API:

Configuration config = new Configuration();
var props = new Dictionary<string, string>();
props["property1"] = "value1";
config.AddPluginLoader("MyLoader", "org.mypackage.MyLoader", props);

Implement the Init method of your PluginLoader implementation to receive initialization parameters. The engine invokes this method before the engine is fully initialized, therefore your implementation should not yet rely on the engine instance within the method body:

public class MyPluginLoader : PluginLoader {
  public void Init(String loaderName, Properties properties, EPServiceProviderSPI epService) {
     // save the configuration for later, perform checking
  }
  ...

The engine calls the PostInitialize method once the engine completed initialization and to indicate the engine is ready for traffic.

public void PostInitialize() {
  // Start the actual interaction with external feeds or the engine here
}
...

The engine calls the Dispose method once the engine is destroyed or initialized for a second time.

public void Dispose() {
  // Destroy resources allocated as the engine instance is being destroyed
}

12.17. Interrogating EPL Annotations

As discussed in Section 4.2.6, “Annotation” an EPL annotation is an addition made to information in an EPL statement. The API and examples to interrogate annotations are described here.

You may use the getAnnotations method of EPStatement to obtain annotations specified for an EPL statement. Or when compiling an EPL expression to a EPStatementObjectModel statement object model you may also query, change or add annotations.

The following example code demonstrates iterating over an EPStatement statement's annotations and retrieving values:

String exampleEPL = "@Tag(name='direct-output', value='sink 1') select * from RootEvent";
EPStatement stmt = epService.EPAdministrator.CreateEPL(exampleEPL);
foreach (Attribute attribute in stmt.Attributes) {
  TagAttribute tag = attribute as TagAttribute;
  if (tag != null) {
    Console.WriteLine("Tag name {0}, value {1}", tag.Name, tag.Value);
  }
}

The output of the sample code shown above is Tag name direct-output value sink 1.

12.18. CLR-Java Differences

When using Esper under .NET, one is sure to run into some differences that make it noticably different that Java. For example, Java does not have an explicit notion of properties; however, this concept is native to the CLR. Another difference is in how events are handled; in Java, it is common to expose an interface that is implemented by the event receiver; in the CLR, its common to expose an event handler that requires a delegate that the event receiver provides.


© 2011 EsperTech Inc. All Rights Reserved