Esper has the following primary interfaces:

For EPL introductory information please see Section 5.1, “EPL Introduction” and patterns are described at Section 6.1, “Event Pattern Overview”.

The JavaDoc documentation is also a great source for API information.

The below code snippet shows how to send a Java object event to the engine. Note that the sendEvent method is overloaded. As events can take on different representation classes in Java, the sendEvent takes parameters to reflect the different types of events that can be send into the engine. The Chapter 2, Event Representations section explains the types of events accepted.

EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider();
EPRuntime runtime = epService.getEPRuntime();

// Send an example event containing stock market data
runtime.sendEvent(new MarketDataBean('IBM', 75.0));		

EventSender sender = epService.getEPRuntime().getEventSender("MyEvent");
sender.sendEvent(myEvent);

select * from MyEvent where quantity > 5

select * from MyEvent(quantity > 5)

As your application may not require streaming results and may not know each query in advance, the on-demand query facility provides for ad-hoc execution of an EPL expression.

On-demand queries are not continuous in nature: The query engine executes the query once and returns all result rows to the application. On-demand query execution is very lightweight as the engine performs no statement creation and the query leaves no traces within the engine.

Esper provides the facility to explicitly index named windows to speed up on-demand and continuous queries. Please consult Section 5.15.13, “Explicitly Indexing Named Windows” for more information.

When named windows are used with contexts and context partitions, the APIs to identify, filter and select context partitions for on-demand queries can be found in Section 14.19, “Context Partition Selection”.

The EPRuntime interface provides three ways to run on-demand queries:

If your application must execute the same EPL on-demand query multiple times with different parameters use prepareQueryWithParameters.

If your application must execute the same EPL on-demand query multiple times without use either prepareQuery or prepareQueryWithParameters and specify no substitution parameters.

By using any of the prepare... methods the engine can compile an EPL query string or object model once and reuse the object and thereby speed up repeated execution.

The following limitations apply:

String query = "select * from MyNamedWindow";
EPOnDemandQueryResult result = epRuntime.executeQuery(query);
for (EventBean row : result.getArray()) {
  System.out.println("name=" + row.get("name"));
}

String query = "select * from MyNamedWindow where orderId = '123'"
EPOnDemandPreparedQuery prepared = epRuntime.prepareQuery(query);
EPOnDemandQueryResult result = prepared.execute();

// ...later on execute once more ...
prepared.execute();	// execute a second time

String query = "select * from MyNamedWindow where orderId = ?";
EPOnDemandPreparedQueryParameterized prepared = epRuntime.prepareQueryWithParameters(query);

// Set the required parameter values before each execution
prepared.setObject(1, "123");
result = epRuntime.executeQuery(prepared);

// ...execute a second time with new parameter values...
prepared.setObject(1, "456");
result = epRuntime.executeQuery(prepared);

String query = "select * from MyNamedWindow where orderId in (?) and price > ?";
EPOnDemandPreparedQueryParameterized prepared = epRuntime.prepareQueryWithParameters(query);
prepared.setObject(1, new String[] {"123", "456"});
prepared.setObject(2, 1000.0});

You may turn off context partition locking engine-wide (also read the caution notice) as described in Section 15.4.23.3, “Disable Locking”. You may disable context partition locking for a given statement by providing the @NoLock annotation as part of your EPL. Note, we provide the @NoLock annotation for the purpose of identifying locking overhead, or when your application is single-threaded, or when using an external mechanism for concurrency control or for example with virtual data windows or plug-in data windows to allow customizing concurrency for a given statement or named window. Using this annotation may have unpredictable results unless your application is taking concurrency under consideration.

For an engine instance to produce predictable results from the viewpoint of listeners to statements, an engine instance by default ensures that it dispatches statement result events to listeners in the order in which a statement produced result events. Applications that require the highest possible concurrency and do not require predictable order of delivery of events to listeners, this feature can be turned off via configuration, see Section 15.4.10.1, “Preserving the order of events delivered to listeners”. For example, assume thread T1 processes an event applied to statement S producing output event O1. Assume thread T2 processes another event applied to statement S and produces output event O2. The engine employs a configurable latch system to ensure that listeners to statement S receive and may complete processing of O1 before receiving O2. When using outbound threading (advanced threading options) or changing the configuration this guarantee is weakened or removed.

In multithreaded environments, when one or more statements make result events available via the insert into clause to further statements, the engine preserves the order of events inserted into the generated insert-into stream, allowing statements that consume other statement's events to behave deterministic. This feature can also be turned off via configuration, see , see Section 15.4.10.2, “Preserving the order of events for insert-into streams”. For example, assume thread T1 processes an event applied to statement S and thread T2 processes another event applied to statement S. Assume statement S inserts into into stream ST. T1 produces an output event O1 for processing by consumers of ST1 and T2 produces an output event O2 for processing by consumers of ST. The engine employs a configurable latch system such that O1 is processed before O2 by consumers of ST. When using route execution threading (advanced threading options) or changing the configuration this guarantee is weakened or removed.

We generally recommended that listener implementations block minimally or do not block at all. By implementing listener code as non-blocking code execution threads can often achieve higher levels of concurrency.

We recommended that, when using a single listener or subscriber instance to receive output from multiple statements, that the listener or subscriber code is multithread-safe. If your application has shared state between listener or subscriber instances then such shared state should be thread-safe.

EPServiceProviderSPI spi = (EPServiceProviderSPI) epService;
int queueSize = spi.getThreadingService().getInboundQueue().size();
ThreadPoolExecutor threadpool = spi.getThreadingService().getInboundThreadPool();

An isolated service is an execution environment separate from the main engine runtime, allowing full control over the concept of time for a group of statements, as further described in Section 14.10, “Service Isolation”.

By default the internal timer provides time and evaluates schedules. External clocking can be used to supply time ticks to the engine instead. The latter is useful for testing time-based event sequences or for synchronizing the engine with an external time source.

The internal timer relies on the java.util.concurrent.ScheduledThreadPoolExecutor class for time tick events. The next section describes timer resolution for the internal timer, by default set to 100 milliseconds but is configurable via the threading options. When using externally-controlled time the timer resolution is in your control.

To disable the internal timer and use externally-provided time instead, there are two options. The first option is to use the configuration API at engine initialization time. The second option toggles on and off the internal timer at runtime, via special timer control events that are sent into the engine like any other event.

If using a timer execution thread pool as discussed above, the internal timer or external time event provide the schedule evaluation however do not actually perform the time-based processing. The time-based processing is performed by the threads in the timer execution thread pool.

This code snippet shows the use of the configuration API to disable the internal timer and thereby turn on externally-provided time (see the Configuration section for configuration via XML file):

Configuration config = new Configuration();
config.getEngineDefaults().getThreading().setInternalTimerEnabled(false);
EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider(config);

After disabling the internal timer, it is wise to set a defined time so that any statements created thereafter start relative to the time defined. Use the CurrentTimeEvent class to indicate current time to the engine and to move time forward for the engine (a.k.a application-time model).

This code snippet obtains the current time and sends a timer event in:

long timeInMillis = System.currentTimeMillis();
CurrentTimeEvent timeEvent = new CurrentTimeEvent(timeInMillis);
epService.getEPRuntime().sendEvent(timeEvent);

Alternatively, you can use special timer control events to enable or disable the internal timer. Use the TimerControlEvent class to control timer operation at runtime.

The next code snippet demonstrates toggling to external timer at runtime, by sending in a TimerControlEvent event:

EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider();
EPRuntime runtime = epService.getEPRuntime();
// switch to external clocking
runtime.sendEvent(new TimerControlEvent(TimerControlEvent.ClockType.CLOCK_EXTERNAL));

Your application sends a CurrentTimeEvent event when it desires to move the time forward. All aspects of Esper engine time related to EPL statements and patterns are driven by the time provided by the CurrentTimeEvent that your application sends in.

The next example sequence of instructions sets time to zero, then creates a statement, then moves time forward to 1 seconds later and then 6 seconds later:

// Set start time at zero.
runtime.sendEvent(new CurrentTimeEvent(0));

// create a statement here
epAdministrator.createEPL("select * from MyEvent output every 5 seconds");

// move time forward 1 second
runtime.sendEvent(new CurrentTimeEvent(1000));

// move time forward 5 seconds
runtime.sendEvent(new CurrentTimeEvent(6000));

When sending external timer events, your application should make sure that long-type time values are ascending. That is, each long-type value should be either the same value or a larger value then the prior value provided by a CurrentTimeEvent.

Your application may use the getNextScheduledTime method in EPRuntime to determine the earliest time a schedule for any statement requires evaluation.

The following code snippet sets the current time, creates a statement and prints the next scheduled time which is 1 minute later then the current time:

// Set start time to the current time.
runtime.sendEvent(new CurrentTimeEvent(System.currentTimeMillis()));

// Create a statement.
epService.getEPAdministrator().createEPL("select * from pattern[timer:interval(1 minute)]");

// Print next schedule time
System.out.println("Next schedule at " + new Date(runtime.getNextScheduledTime());

// Set start time to Jan.1, 2010, 00:00 am for this example
SimpleDateFormat format = new SimpleDateFormat("yyyy MM dd HH:mm:ss SSS");
Date startTime = format.parse("2010 01 01 00:00:00 000");
runtime.sendEvent(new CurrentTimeEvent(startTime.getTime()));

// Create a statement.
EPStatement stmt = epService.getEPAdministrator().createEPL("select current_timestamp() as ct " +
  "from pattern[every timer:interval(1 minute)]");
stmt.addListener(...);	// add a listener

// Advance time to 10 minutes after start time
runtime.sendEvent(new CurrentTimeSpanEvent(startTime.getTime() + 10*60*1000));

// Advance time to 10 minutes after start time at 100 msec resolution
runtime.sendEvent(new CurrentTimeSpanEvent(startTime.getTime() + 10*60*1000, 100));

The internal timer thread can be configured to use nano-second time as returned by System.nanoTime(). If configured for nano-second time, the engine computes an offset of the nano-second ticks to wall clock time upon startup to present back an accurate millisecond wall clock time. Please see section Section 15.4.18, “Engine Settings related to Time Source” to configure the internal timer thread to use System.nanoTime().

The internal timer is based on java.util.concurrent.ScheduledThreadPoolExecutor (java.util.Timer does not support high accuracy VM time).

Your application can achieve a higher tick rate then 1 tick per millisecond by sending external timer events that carry a long-value which is not based on milliseconds since January 1, 1970, 00:00:00 GMT. In this case, your time interval parameters need to take consideration of the changed use of engine time.

Thus, if your external timer events send long values that represents microseconds (1E-6 sec), then your time window interval must be 1000-times larger, i.e. "win:time(1000)" becomes a 1-second time window.

And therefore, if your external timer events send long values that represents nanoseconds (1E-9 sec), then your time window interval must be 1000000-times larger, i.e. "win:time(1000000)" becomes a 1-second time window.

You may register one or more exception handlers for the engine to invoke in the case it encounters an exception processing a continuously-executing statement. By default and without exception handlers the engine cancels execution of the current EPL statement that encountered the exception, logs the exception and continues to the next statement, if any. The configuration is described in Section 15.4.24, “Engine Settings related to Exception Handling”.

If your application registers exception handlers as part of engine configuration, the engine invokes the exception handlers in the order they are registered passing relevant exception information such as EPL statement name, expression and the exception itself.

Exception handlers receive any EPL statement unchecked exception such as internal exceptions or exceptions thrown by plug-in aggregation functions or plug-in views. The engine does not provide to exception handlers any exceptions thrown by static method invocations for function calls, method invocations in joins, methods on event classes and listeners or subscriber exceptions.

An exception handler can itself throw a runtime exception to cancel execution of the current event against any further statements.

For on-demand queries the API indicates any exception directly back to the caller without the exception handlers being invoked, as exception handlers apply to continuous queries only. The same applies to any API calls other then sendEvent and the EventSender methods.

As the configuration section describes, your application registers one or more classes that implement the ExceptionHandlerFactory interface in the engine configuration. Upon engine initialization the engine obtains a factory instance from the class name that then provides the exception handler instance. The exception handler class must implement the ExceptionHandler interface.

You may register one or more condition handlers for the engine to invoke in the case it encounters certain conditions, as outlined below, when executing a statement. By default and without condition handlers the engine logs the condition at informational level and continues processing. The configuration is described in Section 15.4.25, “Engine Settings related to Condition Handling”.

If your application registers condition handlers as part of engine configuration, the engine invokes the condition handlers in the order they are registered passing relevant condition information such as EPL statement name, expression and the condition information itself.

Currently the only conditions indicated by this facility are raised by the pattern followed-by operator, see Section 6.5.8.1, “Limiting Sub-Expression Count” and see Section 6.5.8.2, “Limiting Engine-wide Sub-Expression Count”.

A condition handler may not itself throw a runtime exception or return any value.

As the configuration section describes, your application registers one or more classes that implement the ConditionHandlerFactory interface in the engine configuration. Upon engine initialization the engine obtains a factory instance from the class name that then provides the condition handler instance. The condition handler class must implement the ConditionHandler interface.

The prepare method that is part of the administrative API pre-compiles an EPL statement and stores the precompiled statement in an EPPreparedStatement object. This object can then be used to efficiently start the parameterized statement multiple times.

Substitution parameters are inserted into an EPL statement as a single question mark character '?'. The engine assigns the first substitution parameter an index of 1 and subsequent parameters increment the index by one.

Substitution parameters can be inserted into any EPL construct that takes an expression. They are therefore valid in any clauses such as the select-clause, from-clause filters, where-clause, group-by-clause, having-clause or order-by-clause, including view parameters and pattern observers and guards. Substitution parameters cannot be used where a numeric constant is required rather then an expression and in SQL statements.

All substitution parameters must be replaced by actual values before a statement with substitution parameters can be started. Substitution parameters can be replaced with an actual value using the setObject method for each index. Substitution parameters can be set to new values and new statements can be created from the same EPPreparedStatement object more then once.

While the setObject method allows substitution parameters to assume any actual value including application Java objects or enumeration values, the application must provide the correct type of substitution parameter that matches the requirements of the expression the parameter resides in.

In the following example of setting parameters on a prepared statement and starting the prepared statement, epService represents an engine instance:

String stmt = "select * from com.chipmaker.ReadyEvent(line=?)";
EPPreparedStatement prepared = epService.getEPAdministrator().prepareEPL(stmt);
prepared.setObject(1, 8);
EPStatement statement = epService.getEPAdministrator().create(prepared);

The engine can report key processing metrics through the JMX platform mbean server by setting a single configuration flag described in Section 15.4.19, “Engine Settings related to JMX Metrics”. For additional detailed reporting and metrics events, please read on.

Metrics reporting is a feature that allows an application to receive ongoing reports about key engine-level and statement-level metrics. Examples are the number of incoming events, the CPU time and wall time taken by statement executions or the number of output events per statement.

Metrics reporting is, by default, disabled. To enable reporting, please follow the steps as outlined in Section 15.4.20, “Engine Settings related to Metrics Reporting”. Metrics reporting must be enabled at engine initialization time. Reporting intervals can be controlled at runtime via the ConfigurationOperations interface available from the administrative API.

Your application can receive metrics at configurable intervals via EPL statement. A metric datapoint is simply a well-defined event. The events are EngineMetric and StatementMetric and the Java class representing the events can be found in the client API in package com.espertech.esper.client.metric.

Since metric events are processed by the engine the same as application events, your EPL may use any construct on such events. For example, your application may select, filter, aggregate properties, sort or insert into a stream or named window all metric events the same as application events.

This example statement selects all engine metric events:

select * from com.espertech.esper.client.metric.EngineMetric

The next statement selects all statement metric events:

select * from com.espertech.esper.client.metric.StatementMetric

Make sure to have metrics reporting enabled since only then do listeners or subscribers to a statement such as above receive metric events.

The engine provides metric events after the configured interval of time has passed. By default, only started statements that have activity within an interval (in the form of event or timer processing) are reported upon.

The default configuration performs the publishing of metric events in an Esper daemon thread under the control of the engine instance. Metrics reporting honors externally-supplied time, if using external timer events.

Via runtime configuration options provided by ConfigurationOperations, your application may enable and disable metrics reporting globally, provided that metrics reporting was enabled at initialization time. Your application may also enable and disable metrics reporting for individual statements by statement name.

Statement groups is a configuration feature that allows to assign reporting intervals to statements. Statement groups are described further in the Section 15.4.20, “Engine Settings related to Metrics Reporting” section. Statement groups cannot be added or removed at runtime.

The following limitations apply:

Your application may use the built-in XML and JSON formatters to render output events into a readable textual format, such as for integration or debugging purposes. This section introduces the utility classes in the client util package for rendering events to strings. Further API information can be found in the JavaDocs.

The EventRenderer interface accessible from the runtime interface via the getEventRenderer method provides access to JSON and XML rendering. For repeated rendering of events of the same event type or subtypes, it is recommended to obtain a JSONEventRenderer or XMLEventRenderer instance and use the render method provided by the interface. This allows the renderer implementations to cache event type metadata for fast rendering.

In this example we show how one may obtain a renderer for repeated rendering of events of the same type, assuming that statement is an instance of EPStatement:

JSONEventRenderer jsonRenderer = epService.getEPRuntime().
    getEventRenderer().getJSONRenderer(statement.getEventType());

Assuming that event is an instance of EventBean, this code snippet renders an event into the JSON format:

String jsonEventText = jsonRenderer.render("MyEvent", event);

The XML renderer works the same:

XMLEventRenderer xmlRenderer = epService.getEPRuntime().
    getEventRenderer().getXMLRenderer(statement.getEventType());

...and...

String xmlEventText = xmlRenderer.render("MyEvent", event);

If the event type is not known in advance or if you application does not want to obtain a renderer instance per event type for fast rendering, your application can use one of the following methods to render an event to a XML or JSON textual format:

String json = epService.getEPRuntime().getEventRenderer().renderJSON(event);
String xml = epService.getEPRuntime().getEventRenderer().renderXML(event);

Use the JSONRenderingOptions or XMLRenderingOptions classes to control how events are rendered. To render specific event properties using a custom event property renderer, specify an EventPropertyRenderer as part of the options that renders event property values to strings. Please see the JavaDoc documentation for more information.

As discussed in Section 5.2.7, “Annotation” an EPL annotation is an addition made to information in an EPL statement. The API and examples to interrogate annotations are described here.

You may use the getAnnotations method of EPStatement to obtain annotations specified for an EPL statement. Or when compiling an EPL expression to a EPStatementObjectModel statement object model you may also query, change or add annotations.

The following example code demonstrates iterating over an EPStatement statement's annotations and retrieving values:

String exampleEPL = "@Tag(name='direct-output', value='sink 1') select * from RootEvent";
EPStatement stmt = epService.getEPAdministrator().createEPL(exampleEPL);
for (Annotation annotation : stmt.getAnnotations()) {
  if (annotation instanceof Tag) {
    Tag tag = (Tag) annotation;
    System.out.println("Tag name " + tag.name() + " value " + tag.value());
  }
}

The output of the sample code shown above is Tag name direct-output value sink 1.

This chapter discusses how to select context partitions. Contexts are discussed in Chapter 4, Context and Context Partitions and the reasons for context partition selection are introduced in Section 4.8, “Operations on Specific Context Partitions”.

The section is only relevant when you declare a context. It applies to all different types of hash, partitioned, category, overlapping or other temporal contexts. The section uses a category context for the purpose of illustration. The API discussed herein is general and handles all different types of contexts including nested contexts.

Consider a category context that separates bank transactions into small, medium and large:

// declare category context
create context TxnCategoryContext 
  group by amount < 100 as small, 
  group by amount between 100 and 1000 as medium, 
  group by amount > 1000 as large from BankTxn

// retain 1 minute of events of each category separately
context TxnCategoryContext select * from BankTxn.win:time(1 minute)

In order for your application to iterate one or more specific categories it is necessary to identify which category, i.e. which context partition, to iterate. Similarly for on-demand queries, to execute on-demand queries against one or more specific categories, it is necessary to identify which context partition to execute the on-demand query against.

Your application may iterate one or more specific context partitions using either the iterate or safeIterate method of EPStatement by providing an implementation of the ContextPartitionSelector interface.

For example, assume your application must obtain all bank transactions for small amounts. It may use the API to identify the category and iterate the associated context partition:

ContextPartitionSelectorCategory categorySmall = new ContextPartitionSelectorCategory() {
    public Set<String> getLabels() {
      return Collections.singleton("small");
    }
  };
Iterator<EventBean> it = stmt.iterator(categorySmall);

Your application may execute on-demand queries against one or more specific context partitions by using the executeQuery method on EPRuntime or the execute method on EPOnDemandPreparedQuery and by providing an implementation of ContextPartitionSelector.

On-demand queries execute against named windows, therefore below EPL statement creates a named window which the engine manages separately for small, medium and large transactions according to the context declared earlier:

// Named window per category
context TxnCategoryContext create window BankTxnWindow.win:time(1 min) as BankTxn

The following code demonstrates how to fire an on-demand query against the small and the medium category:

ContextPartitionSelectorCategory categorySmallMed = new ContextPartitionSelectorCategory() {
    public Set<String> getLabels() {
      return new HashSet<String>(Arrays.asList("small", "medium"));
    }
  };
epService.getEPRuntime().executeQuery(
   "select count(*) from BankTxnWindow", 
   new ContextPartitionSelector[] {categorySmallMed});

The following limitations apply:

This chapter briefly discusses the API to manage context partitions. Contexts are discussed in Chapter 4, Context and Context Partitions.

The section is only relevant when you declare a context. It applies to all different types of hash, partitioned, category, overlapping or other temporal contexts.

The administrative API for context partitions is EPContextPartitionAdmin. Use the getContextPartitionAdmin method of the EPAdministrator interface to obtain said service.

The context partition admin API allows an application to:

Stopping individual context partitions is useful to drop state, free memory and suspend a given context partition without stopping or destroying any associated statements. For example, assume a keyed segmented context per user id. To suspend and free the memory for a given user id your application can stop the user id's context partition. The engine does not allocate a context partition for this user id again, until your application destroys or starts the context partition.

Destroying individual context partitions is useful to drop state, free memory and deregister the given context partition without stopping or destroying any associated statements. For example, assume a keyed segmented context per user id. To deregister and free the memory for a given user id your application can destroy the user id's context partition. The engine can allocate a fresh context partition for this user id when events for this user id arrive.

Please see the JavaDoc documentation for more information.