This section discusses the notion of context and its role in the Esper event processing language (EPL).

When you look up the word context in a dictionary, you may find: Context is the set of circumstances or facts that surround a particular event, situation, etc.

Context-dependent event processing occurs frequently: For example, consider a requirement that monitors banking transactions. For different customers your analysis considers customer-specific aggregations, patterns or data windows. In this example the context of detection is the customer. For a given customer you may want to analyze the banking transactions of that customer by using aggregations, data windows, patterns including other EPL constructs.

In a second example, consider traffic monitoring to detect speed violations. Assume the speed limit must be enforced only between 9 am and 5 pm. The context of detection is of temporal nature.

A context takes a cloud of events and classifies them into one or more sets. These sets are called context partitions. An event processing operation that is associated with a context operates on each of these context partitions independently. (Credit: Taken from the book "Event Processing in Action" by Opher Etzion and Peter Niblett.) A basic partitioned query was reviewed in Section 2.11, “Basic Partitioned Query”.

A context is a declaration of dimension and may thus result in one or more context partitions. In the banking transaction example the context dimension is the customer and a context partition exists per customer. In the traffic monitoring example there is a single context partition that exists only between 9 am and 5 pm and does not exist outside of that daily time period.

In an event processing glossary you may find the term event processing agent. An EPL statement is an event processing agent. An alternative term for context partition is event processing agent instance.

Esper EPL allows you to declare contexts explicitly, offering the following benefits:

Esper EPL allows you to declare a context explicitly via the create context syntax introduced below.

After you have declared a context, one or more EPL statements can refer to that context by specifying context name. When an EPL statement refers to a context, all EPL-statement related state such as aggregations, patterns or data windows etc. exists once per context partition.

If an EPL statement does not declare a context, it implicitly has a single context partition. The single context partition lives as long as the EPL statement is started and ends when the EPL statement is stopped.

You may have heard of the term session. A context partition is the same as a session.

You may have heard of the term session window to describe the duration between when a session becomes alive to when a session gets destroyed. We use the term context partition lifecycle instead.

The context declaration specifies how the engine manages context partitions (or sessions):

For the API to administrate context partitions please see Section 14.19, “Context Partition Administration”. For more information on locking and threading please see Section 14.7, “Engine Threading and Concurrency”. For performance related information please refer to Chapter 23, Performance.

The create context statement declares a context by specifying a context name and context dimension information.

A context declaration by itself does not consume any resources or perform any logic until your application starts at least one statement that refers to that context. Until then the context is inactive and not in use.

When your application creates or starts the first statement that refers to the context, the engine activates the context.

As soon as your application stops or destroys all statements that refer to the context, the context becomes inactive again.

When your application stops or destroys a statement that refers to a context, the context partitions associated to that statement also end (context partitions associated to other started statements live on).

When your application stops or destroys the statement that declared the context and does not also stop or destroy any statements that refer to the context, the context partitions associated to each such statement do not end.

When your application destroys the statement that declared the context and destroys all statements that refer to that context then the engine removes the context declaration entirely.

The create context statement posts no output events to listeners or subscribers and does not return any rows when iterated.

create context context_name partition [by]
  event_property [and event_property [and ...]] from stream_def
  [, event_property [...] from stream_def]
  [, ...]

create context SegmentedByCustomer partition by custId from BankTxn

context SegmentedByCustomer
select custId, account, sum(amount) from BankTxn group by account

context SegmentedByCustomer
select * from pattern [
  every a=BankTxn(amount > 400) -> b=BankTxn(amount > 400) where timer:within(10 minutes)
]

// Neither LoginEvent nor LogoutEvent are listed in the context declaration
context SegmentedByCustomer
select * from pattern [every a=LoginEvent -> b=LogoutEvent where timer:within(10 minutes)]

// Not valid
create context SegmentedByCustomer partition by custId from BankTxn, account from BankTxn

// Invalid: Type mismatch between properties
create context SegmentedByCustomer partition by custId from BankTxn, loginTime from LoginEvent

create context SegmentedByCustomer partition by 
  custId from BankTxn, loginId from LoginEvent, loginId from LogoutEvent

create context SegmentedByCustomer partition by 
  custId from BankTxn, loginId from LoginEvent(failed=false)

create context ByCustomerAndAccount partition by custId and account from BankTxn

context ByCustomerAndAccount select custId, account, sum(amount) from BankTxn

create context ByCustomerAndAccount partition by custId and account from BankTxn

context ByCustomerAndAccount
  select context.name, context.id, context.key1, context.key2 from BankTxn

create context ByCust partition by custId from BankTxn

context ByCust 
  select * from BankTxn as t1 unidirectional, BankTxn#time(30) t2 
  where t1.amount = t2.amount

context ByCust
  select * from SecurityEvent as t1 unidirectional, BankTxn#time(30) t2 
  where t1.customerName = t2.customerName

create context context_name coalesce [by] 
  hash_func_name(hash_func_param) from stream_def
  [, hash_func_name(hash_func_param) from stream_def ]
  [, ...]
  granularity granularity_value
  [preallocate]

create context SegmentedByCustomerHash
  coalesce by consistent_hash_crc32(custId) from BankTxn granularity 16 preallocate

context SegmentedByCustomerHash
select custId, account, sum(amount) from BankTxn group by custId, account

create context MyHashContext
  coalesce by computeHash(*) from BankTxn granularity 16 preallocate

// Neither LoginEvent nor LogoutEvent are listed in the context declaration
context SegmentedByCustomerHash
select * from pattern [every a=LoginEvent -> b=LogoutEvent where timer:within(10 minutes)]

create context HashedByCustomer as coalesce
  consistent_hash_crc32(custId) from BankTxn,
  consistent_hash_crc32(loginId) from LoginEvent, 
  consistent_hash_crc32(loginId) from LogoutEvent
  granularity 32 preallocate

create context HashedByCustomer 
  coalesce consistent_hash_crc32(loginId) from LoginEvent(failed = false) 
  granularity 1024 preallocate

create context ByCustomerHash coalesce consistent_hash_crc32(custId) from BankTxn granularity 1024

context ByCustomerHash
  select context.name, context.id from BankTxn

create context context_name
  group [by] group_expression as category_label
  [, group [by] group_expression as category_label]
  [, ...]
  from stream_def

create context CategoryByTemp
  group temp < 65 as cold,
  group temp between 65 and 85 as normal,
  group temp > 85 as large
  from SensorEvent

context CategoryByTemp select context.label, count(*) from SensorEvent

context CategoryByTemp
  select context.name, context.id, context.label from SensorEvent

create context context_name
  start (@now | start_condition) 
  [ end end_condition ]

create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

context NineToFive select * from TrafficEvent(speed >= 100)

create context PowerOutage start PowerOutageEvent end pattern [PowerOnEvent -> timer:interval(5)]

context PowerOutage select * from TemperatureEvent

create context Every15Minutes start @now end after 15 minutes

create context AlwaysOn start @now

create context MyCtx start MyStartEvent end MyEndEvent

context MyCtx select count(*) as cnt from MyEndEvent output when terminated

create context context_name
  initiated [by] [distinct (distinct_value_expr [,...])] [@now and] initiating_condition
  [ terminated [by] terminating_condition ]

create context CtxTrainEnter
  initiated by TrainEnterEvent as te
  terminated after 5 minutes

context CtxTrainEnter
select t1 from pattern [
  t1=TrainEnterEvent(trainId = context.te.trainId) -> timer:interval(5 min) and not TrainLeaveEvent(trainId = context.te.trainId)
  ]

create context CtxEachMinute
  initiated @now and pattern [every timer:interval(1 minute)]
  terminated after 1 minutes

context CtxEachMinute select avg(temp) from SensorEvent

create context CtxTrainEnter initiated by TrainEnterEvent as te

create context OrderContext
  initiated by distinct(orderId) NewOrderEvent as newOrder 
  terminated by CloseOrderEvent(closeOrderId = newOrder.orderId)

context NineToFive 
select context.name, context.startTime, context.endTime from TrafficEvent(speed >= 100)

context CtxTrainEnter
select *, context.te.trainId, context.id, context.name
from TrainLeaveEvent(trainId = context.te.trainId)

event_stream_name [(filter_criteria)] [as stream_name]

// A non-overlapping context that starts when MyStartEvent arrives and ends when MyEndEvent arrives
create context MyContext start MyStartEvent end MyEndEvent

// An overlapping context where each MyEvent with level greater zero 
// initiates a new context partition that terminates after 10 seconds
create context MyContext initiated MyEvent(level > 0) terminated after 10 seconds

// A non-overlapping context that starts when MyEvent arrives
// and ends when a matching MyEvent arrives (same id)
create context MyContext 
start MyEvent as myevent
end MyEvent(id=myevent.id)

// An overlapping context where each MyInitEvent initiates a new context partition 
// that terminates when a matching MyTermEvent arrives 
create context MyContext 
initiated by MyInitEvent as e1 
terminated by MyTermEvent(id=e1.id, level <> e1.level)

pattern [pattern_expression] [@inclusive]

// A non-overlapping context that starts when either StartEventOne or StartEventTwo arrive
// and that ends after 5 seconds.
// Here neither StartEventOne or StartEventTwo count towards any statements
// that are referring to the context.
create context MyContext 
  start pattern [StartEventOne or StartEventTwo] 
  end after 5 seconds

// Same as above. 
// Here both StartEventOne or StartEventTwo do count towards any statements
// that are referring to the context.
create context MyContext 
  start pattern [a=StartEventOne or b=StartEventTwo] @inclusive 
  end after 5 seconds

// An overlapping context where each distinct MyInitEvent initiates a new context 
// and each context partition terminates after 20 seconds
// Use @inclusive to say that the same MyInitEvent that fires the pattern
// also applies to statements that are associated to the context.
create context MyContext
  initiated by pattern [every-distinct(a.id, 20 sec) a=MyInitEvent]@inclusive
  terminated after 20 sec

// An overlapping context where each pattern match initiates a new context 
// and all context partitions terminate when MyTermEvent arrives.
// The MyInitEvent and MyOtherEvent that trigger the pattern are themselves not included 
// in any statements that are associated to the context.
create context MyContext
  initiated by pattern [every MyInitEvent -> MyOtherEvent where timer:within(5)]
  terminated by MyTermEvent

// A non-overlapping context that starts when either StartEventOne or StartEventTwo arrive
// and that ends when either a matching EndEventOne or EndEventTwo arrive
create context MyContext 
  start pattern [a=StartEventOne or b=StartEventTwo]@inclusive
  end pattern [EndEventOne(id=a.id) or EndEventTwo(id=b.id)]

// A non-overlapping context started daily between 9 am to 5 pm
// and not started outside of these hours:
create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

// An overlapping context where crontab initiates a new context every 1 minute
// and each context partition terminates after 10 seconds:
create context MyContext initiated (*, *, *, *, *) terminated after 10 seconds

after time_period_expression

// A non-overlapping context started after 10 seconds 
// that ends 1 minute after it starts and that again starts 10 seconds thereafter.
create context NonOverlap10SecFor1Min start after 10 seconds end after 1 minute

// An overlapping context that starts a new context partition every 5 seconds
// and each context partition lasts 1 minute
create context Overlap5SecFor1Min initiated after 5 seconds terminated after 1 minute

A nested context is a context that is composed from two or more contexts.

The syntax for creating a nested context is as follows:

create context context_name
  context nested_context_name [as] nested_context_definition ,
  context nested_context_name [as] nested_context_definition [, ...]

The context_name you assign to the context can be any identifier.

Following the context name is a comma-separated list of nested contexts. For each nested context specify the context keyword followed a nested context name and the nested context declaration. Any of the context declarations as outlined in Section 4.2, “Context Declaration” are allowed for nested contexts. The order of nested context declarations matters as outlined below. The nested context names have meaning only in respect to built-in properties and statements may not be assigned to nested context names.

The next statement creates a nested context NineToFiveSegmented that, between 9 am and 5 pm, allocates a new context partition for each customer id:

create context NineToFiveSegmented
  context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *),
  context SegmentedByCustomer partition by custId from BankTxn

The following statement refers to the nested context to compute a total withdrawal amount per account for each customer but only between 9 am and 5 pm:

context NineToFiveSegmented
select custId, account, sum(amount) from BankTxn group by account

Esper implements nested contexts as a context tree: The context declared first controls the lifecycle of the context(s) declared thereafter. Thereby, in the above example, outside of the 9am-to-5pm time the engine has no memory and consumes no resources in relationship to bank transactions or customer ids.

When combining segmented contexts, the set of context partitions for the nested context effectively is the Cartesian product of the partition sets of the nested segmented contexts (sample: create context MyCartSegmented context SegmentedByAccount partition by acctId from BankTxn context SegmentedByCustomer partition by custId from BankTxn).

When combining temporal contexts with other contexts, since temporal contexts may overlap and may terminate, it is important to understand that temporal contexts control the lifecycle of sub-contexts (contexts declared thereafter). The order of declaration of contexts in a nested context can thereby change resource usage and output result.

The next statement creates a context that allocates context partition only when a train enters a station and then for each hash of the tag id of a passenger as indicated by PassengerScanEvent events, and terminates all context partitions after 5 minutes:

create context CtxNestedTrainEnter
  context InitCtx initiated by TrainEnterEvent as te terminated after 5 minutes,
  context HashCtx coalesce by consistent_hash_crc32(tagId) from PassengerScanEvent
    granularity 16 preallocate

In the example above the engine does not start tracking PassengerScanEvent events or hash codes or allocate context partitions until a TrainEnterEvent arrives.

create context CtxSampleNestedContext
  context SpanA start AStart end AEnd,
  context SpanB start BStart end BEnd

context CtxSampleNestedContext select count(*) from C

create context CtxSampleNestedContext 
  start pattern[every a=AStart or every a=BStart] as mypattern 
  end pattern[every AEnd or every BEnd]

context CtxNestedTrainEnter
select context.InitCtx.te.trainId, context.HashCtx.id,
  tagId, count(*) from PassengerScanEvent group by tagId

context NineToFiveSegmented
select context.NineToFive.startTime, context.SegmentedByCustomer.key1 from BankTxn

context NineToFiveSegmented select context.name, context.id from BankTxn

You do not need to declare a context to partition data windows, aggregation values or patterns themselves individually. You may mix-and-match partitioning as needed.

The table below outlines other partitioning syntax supported by EPL:

// Length window of 2 events per customer
select * from
  BankTxn#groupwin(custId)#length(2)

select avg(price), window(*)
  from BankTxn group by custId

select * from pattern [
  every a=BankTxn -> BankTxn(custId = a.custId)...]

select * from match_recognize 
  ... partition by custId

select * from ... where a.custId = b.custId

You may use output rate limiting to trigger output when a context partition ends or terminates using output when terminated. This concept was introduced to you in Section 2.13, “Basic Partitioned and Output-Rate-Limited Query”. For more information on output rate limiting please see Section 5.7, “Stabilizing and Controlling Output: The Output Clause”.

The context CtxEachMinute initiates a new context partition every 1 minute, and each context partition expires after 5 minutes:

create context CtxEachMinute
initiated by pattern [every timer:interval(1 min)]
terminated after 5 minutes

The following statement computes an ongoing average temperature however only outputs the last value of the average temperature, together with the context partition id and the current engine time, after 5 minutes when a context partition ends:

context CtxEachMinute
select current_timestamp as endtime, context.id, avg(temp) from SensorEvent output last when terminated

The when terminated syntax can be combined with other output rates.

The next example outputs every 1 minute and also when the context partition ends:

context CtxEachMinute
select context.id, avg(temp) from SensorEvent output last every 1 minute and when terminated

In the case that the end/terminating condition of the context partition is an event or pattern, the context properties contain the information of the tagged events in the pattern or the single event that ended/terminated the context partition.

For example, consider the following context wherein the engine initializes a new context partition for each arriving MyStartEvent event and that terminates a context partition when a matching MyEndEvent arrives:

create context CtxSample
initiated by MyStartEvent as startevent
terminated by MyEndEvent(id = startevent.id) as endevent

The following statement outputs the id property of the initiating and terminating event and only outputs when a context partition ends:

context CtxSample 
select context.startevent.id, context.endevent.id, count(*) from MyEvent
output last when terminated

You may in addition specify a termination expression that the engine evaluates when a context partition terminates. Only when the termination expression evaluates to true does output occur. The expression may refer to built-in properties as described in Section 5.7.1.1, “Controlling Output Using an Expression”. The syntax is as follows:

...output when terminated and termination_expression

The next example statement outputs when a context partition ends but only if at least two events are available for output:

context CtxEachMinute
select * from SensorEvent output when terminated and count_insert >= 2

The final example EPL outputs when a context partition ends and sets the variable myvar to a new value:

context CtxEachMinute
select * from SensorEvent output when terminated then set myvar=3

Named windows are globally-visible data windows that may be referred to by multiple statements. You may refer to named windows in statements that declare a context without any special considerations, with the exception of on-action statements (latter must refer to the same context associated with the named window).

You may also create a named window and declare a context for the named window. In this case the engine in effect manages separate named windows, one for each context partition.

For example, consider the 9 am to 5 pm non-overlapping context as shown earlier:

create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

You may create a named window that only exists between 9 am and 5 pm:

context NineToFive create window SpeedingEvents1Hour#time(30 min) as TrafficEvent

You can insert into the named window:

context NineToFive on TrafficEvent(speed > 100) merge SpeedingEvents1Hour insert select *

Any on-merge, on-select, on-update and on-delete statements must declare the same context, in order to operate on partitioned named windows or tables for the same partition.

The following is not a valid statement as it does not declare the same context that was used to declare the named window:

// You must declare the same context for on-trigger statements
on TruncateEvent delete from SpeedingEvents1Hour

The following is valid:

context NineToFive on TruncateEvent delete from SpeedingEvents1Hour

For context declarations that require specifying event types, such as the hash segmented context and keyed segmented context, please provide the named window underlying event type.

The following sample EPL statements define a type for the named window, declare a context and associate the named window to the context:

create schema ScoreCycle (userId string, keyword string, productId string, score long)

create context HashByUserCtx as 
  coalesce by consistent_hash_crc32(userId) from ScoreCycle granularity 64

context HashByUserCtx create window ScoreCycleWindow#unique(productId, keyword) as ScoreCycle

Tables are globally-visible data structures that hold rows organized by primary key(s) and that may be referred to by multiple statements. You may refer to tables in statements that declare a context without any special considerations, with the exception of on-action statements (latter must refer to the same context associated with the table).

You may also create a table and declare a context for the table. In this case the engine in effect manages separate tables, one for each context partition.

For example, consider the 9 am to 5 pm non-overlapping context as shown earlier:

create context NineToFive start (0, 9, *, *, *) end (0, 17, *, *, *)

You may create a table that only exists between 9 am and 5 pm:

context NineToFive create table AverageSpeedTable (
  carId string primary key, 
  avgSpeed avg(double))

You can aggregate-into the table only if the aggregating statement declares the same context:

// declare the same context as for the table
context NineToFive into table AverageSpeedTable
select avg(speed) as avgSpeed
from TrafficEvent
group by carId

When you declare a context for a table, any select, on-merge, on-select, on-update and on-delete statements as well as statements that subquery the table must declare the same context.

For example, this EPL truncates the AverageSpeedTable:

context NineToFive on TruncateEvent delete from AverageSpeedTable

A variable is a scalar, object or event value that is available for use in all statements. Variables can be either global variables or context variables.

The value of a global variable is the same for all context partitions. The next example declares a global threshold variable:

create variable integer var_global_threshold = 100

For context variables, there is a variable value per context partition. The next example declares a context and a context variable:

create context ParkingLotContext initiated by CarArrivalEvent as cae terminated by CarDepartureEvent(lot = cae.lot)

context ParkingLotContext create variable integer var_parkinglot_threshold = 100

The variable var_parkinglot_threshold is a context variable. Each context partition can have its own value for the variable.

For more information on variables, please refer to Section 5.17, “Variables and Constants”.

Context variables can only be used in statements that associated to the same context.

The API to read and manage content partitions themselves is Section 14.19, “Context Partition Administration”.

Selecting specific context partitions and interrogating context partition state is useful for:

Esper provides APIs to identify, filter and select context partitions for statement iteration and on-demand queries. The APIs are described in detail at Section 14.18, “Context Partition Selection”.

For statement iteration, your application can provide context selector objects to the iterate and safeIterate methods on EPStatement. If your code does not provide context selectors the iteration considers all context partitions. At the time of iteration, the engine obtains the current set of context partitions and iterates each independently. If your statement has an order-by clause, the order-by clause orders within the context partition and does not order across context partitions.

For on-demand queries, your application can provide context selector objects to the executeQuery method on EPRuntime and to the execute method on EPOnDemandPreparedQuery. If your code does not provide context selectors the on-demand query considers all context partitions. At the time of on-demand query execution, the engine obtains the current set of context partitions and queries each independently. If the on-demand query has an order-by clause, the order-by clause orders within the context partition and does not order across context partitions.