www.espertech.comDocumentation
This section provides information for using JSON to represent events.
An event can be represented by a JSON-formatted string value. Event properties of JSON events are the properties of the JSON object.
The advantages for supporting JSON as an event representation are:
JSON is easy, reliable, and fast and is often used for serializing and transmitting structured data. JSON has bindings for a wide variety of programming languages and platforms and has RPC and file representations.
At time of EPL compilation, the Esper EPL compiler produces a high performance JSON parser for the specific JSON event type, achieving excellent parsing performance.
At time of EPL compilation, the Esper EPL compiler produces an internal representation of the JSON event type properties that supports high read speeds of property values without the need for document traversal, and that is memory-efficient.
Similar to the Map and object-array event type, the JSON event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types.
The runtime can process string-value events that are formatted JSON documents via the sendEventJson(String json, String jsonEventTypeName)
method on the EPEventService
interface.
The runtime does not validate JSON events. Your application must ensure that the JSON object matches the declaration of the schema in create json schema
.
A given JSON event type can have only a single supertype that must also be a JSON event type. All properties available on the JSON supertype are also available on the type itself. In addition, anywhere within EPL that an event type name of a JSON supertype is used, the JSON subtype and the subtype of the subtype match that expression.
The Esper JSON parser produces an event object that is available to your application. The event object implements the JsonEventObject
interface (package com.espertech.esper.common.client.json.util
).
The JsonEventObject
extends Map<String, Object>
allowing you to use event objects that are originating from JSON like any map.
In order to use JSON for incoming events, the event type name and JSON schema, if available, must be made known via the create json schema
EPL syntax. Please see examples in Section 5.15, “Declaring an Event Type: Create Schema”.
In the case that you don't want to define fields using EPL create json schema
, you may specify an application class that provides the JSON schema. Please refer to Section G.10, “Customizing the JSON Event Class”.
JSON event types must be defined in EPL using create json schema
. JSON event types cannot be pre-configured.
If you don't know some or all of the event properties in advance, please follow the instructions in Section G.6, “JSON Dynamic Event Properties”.
The code snippets below define a JSON event type, create a JSON event and send the event into the runtime. The sample defines the CarLocUpdateEvent
event type. The @public
means the event type
has public visibility and the @buseventtype
means your application can use sendEventJson
for this event type.
@public @buseventtype create json schema CarLocUpdateEvent(carId string, direction int)
The CarLocUpdateEvent
can now be used in a statement:
select count(*) from CarLocUpdateEvent(direction = 1)#time(1 min)
The sample code to send an event is:
String event = "{" + " \"carId\" : \"A123456\",\n" + " \"direction\" : 1\n" + "}"; runtime.getEventService().sendEventJson(event, "CarLocUpdateEvent");
JSON allows objects to be nested within other objects. Each JSON object may itself contain further JSON objects or arrays of JSON objects.
For example, consider the following JSON document:
{ "isbn": "123-456-222", "author": { "lastname": "Doe", "firstname": "Jane" }, "editor": { "lastname": "Smith", "firstname": "Jane" }, "title": "The Ultimate Database Study Guide", "category": ["Non-Fiction", "Technology"] }
The author
and editor
objects are nested objects. Please define the schema for nested objects first, and the schemas that are building on the nested schemas thereafter.
Define the schemas as follows:
create json schema Names(lastname string, firstname string); @public @buseventtype create json schema BookEvent(isbn string, author Names, editor Names, title string, category string[]);
This sample EPL outputs the isbn and the last name of the author and the editor, as well as the first category (use sendEventJson(json, "BookEvent")
to process the document):
select isbn, author.lastname as authorName, editor.lastname as editorName, category[0] as firstCategory from BookEvent
The next example JSON document is meant to demonstrate nested objects and arrays of objects:
{ "id": "0001", "type": "donut", "name": "Cake", "batters": { "machine": "machine A", "batter": [ { "id": "1001", "type": "Regular" }, { "id": "1002", "type": "Chocolate" }, { "id": "1003", "type": "Blueberry" }, { "id": "1004", "type": "Devil's Food" } ] }, "topping": [ { "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5007", "type": "Powdered Sugar" } ] }
Define the schemas as follows:
create json schema IdAndType(id string, type string); create json schema Batters(machine string, batter IdAndType[]); @public @buseventtype create json schema CakeEvent(id string, type string, name string, batters Batters, topping IdAndType[]);
The following EPL outputs the name, the batter machine, type of the first batter and the type of the first topping, the count of batters, and the count of toppings of cake events (use sendEventJson(json, "CakeEvent")
to process the document):
select name, batters.machine as batterMachine, batters.batter[0].type as firstBatterType, topping[0].type as firstToppingType, batters.batter.countOf() as countBatters, topping.countOf() as countToppings from CakeEvent
The create json schema
EPL specifies the name and type of each event property. The create json schema
may also specify no event properties or only a subset of all available event properties.
When there are JSON properties that are not known at time of compilation, please see Section G.6, “JSON Dynamic Event Properties”.
The table below shows the supported types and provides links to more information. Types not listed below are not supported for use with a JSON event type.
Table G.1. Support JSON Event Property Types
Type | More Information |
---|---|
Application-provided class | See Section G.5, “JSON Application-Provided Class” |
bigdecimal, java.math.BigDecimal | Numeric type parsed by new BigDecimal ; See Section G.4.1, “Numeric Types” |
biginteger, java.math.BigInteger | Numeric type parsed by new BigInteger ; See Section G.4.1, “Numeric Types” |
byte, java.lang.Byte | Numeric type parsed by Byte.parseByte ; See Section G.4.1, “Numeric Types” |
boolean, java.lang.Boolean | See Section G.4.2, “Boolean Type” |
double, java.lang.Double | Numeric type parsed by Double.parseDouble ; See Section G.4.1, “Numeric Types” |
float, java.lang.Float | Numeric type parsed by Float.parseFloat ; See Section G.4.1, “Numeric Types” |
int, java.lang.Integer | Numeric type parsed by Integer.parseInt ; See Section G.4.1, “Numeric Types” |
long, java.lang.Long | Numeric type parsed by Long.parseLong ; See Section G.4.1, “Numeric Types” |
null | No parsing, a value that is always null. |
short, java.lang.Short | Numeric type parsed by Short.parseShort ; See Section G.4.1, “Numeric Types” |
string, java.lang.String, char, java.lang.Character | See Section G.4.3, “String Type” |
java.util.Map | Represents any untyped JSON object. See Section G.4.4, “Map Type for Holding an Untyped JSON Object” |
Object[] | Represents any untyped JSON array. See Section G.4.5, “Object-Array Type for Holding an Untyped JSON Array” |
Object | Represents any untyped JSON value. See Section G.4.6, “Object Type for Holding an Untyped JSON Value (Any JSON Value)” |
java.net.URI | As parsed by new URI(...) |
java.net.URL | As parsed by new URL(...) |
java.time.LocalDate | As parsed by LocalDate.parse |
java.time.LocalDateTime | As parsed by LocalDateTime.parse |
java.time.OffsetDateTime | As parsed by OffsetDateTime.parse |
java.time.ZonedDateTime | As parsed by ZonedDateTime.parse |
java.util.UUID | As parsed by UUID.fromString |
The numeric types, the string
type, the boolean type and the enumeration class type allow one- and two-dimensional arrays. The Map
type does not support arrays and Object
does not support two-dimensional arrays.
An example with arrays is the schema create json schema MyEvent(prices bigdecimal[], intmatrix int[][])
.
Use [primitive]
to instruct the EPL compiler to use array of primitive values, for example create json schema MyEvent(intarray int[primitive])
.
The default value for all event properties is null
. When the JSON property of the same name is not present in the JSON event document the event property value is null
.
The null
JSON value is allowed for all types.
The JSON parser uses the respective parse method of the relevant JVM type (see JavaDoc) as listed in the above table, e.g. Integer.parseInt
for integer. If the value cannot be parsed it produces an exception.
The parsor allows both numeric and string value JSON text for all numeric types. The event property type is always a boxed type (except for arrays).
To illustrate, assume the following schema:
@public @buseventtype create json schema MyEvent(intValue int)
Here are some sample JSON documents:
Table G.2. Sample JSON Documents With an Integer-Type Value
Sample JSON | Result |
---|---|
{ "intValue": 1 } | Event has an intValue value of 1 |
{ "intValue": "2" } | Event has an intValue value of 2 (allows string values) |
{ "intValue": null } | Event has an intValue value of null |
{} | Event has an intValue value of null |
{ "intValue": "some text" } | JSON parser exception |
{ "intValue" : ...anything but above, such as JSON array or JSON object... } | Event has an intValue value of null |
The JSON parser parses true
and false
, as well as string-type "true"
and "false"
.
The event property type is always Boolean
(except for arrays of primitive boolean).
For instance, assume the following schema:
@public @buseventtype create json schema MyEvent(booleanValue boolean)
A few sample JSON documents are listed below.
Table G.3. Sample JSON Documents With a Boolean-Type Value
Sample JSON | Result |
---|---|
{ "booleanValue": true } | Event has a booleanValue value of true |
{ "booleanValue": false } | Event has a booleanValue value of false |
{ "booleanValue": "true" } | Event has a booleanValue value of true (allows string 'true') |
{ "booleanValue": "false" } | Event has a booleanValue value of false (allows string 'false') |
{ "booleanValue": null } | Event has a booleanValue value of null |
{} | Event has a booleanValue value of null |
{ "booleanValue": "some text" } | JSON parser exception |
{ "booleanValue" : ...anything but above, such as JSON array or JSON object... } | Event has an booleanValue value of null |
The JSON parser takes any string-type, number-type or boolean-type value. For character-typed values the JSON parser obtains the first character if available.
Let's say the schema is as follows:
@public @buseventtype create json schema MyEvent(stringValue string)
To show the behavior the example JSON is:
Table G.4. Sample JSON Documents With a String-Type Value
Sample JSON | Result |
---|---|
{ "stringValue": "abc" } | Event has a stringValue value of "abc" |
{ "stringValue": 1 } | Event has a stringValue value of "1" (allows numbers) |
{ "stringValue": false } | Event has a stringValue value of "false" (allows boolean) |
{ "stringValue": null } | Event has a stringValue value of null |
{} | Event has a stringValue value of null |
{ "stringValue" : ...anything but above, such as JSON array or JSON object... } | Event has an stringValue value of null |
The JSON parser, when it encounters a JSON object for the same name, populates a Map<String, Object>
that contains the JSON object properties and their nested values (deep).
The parser ignores any other JSON value type than object.
The Map<String, Object>
that the parser produces adheres to the type rules for dynamic properties below.
Define a schema like this one:
@public @buseventtype create json schema MyEvent(jsonObject java.util.Map)
The next table has sample JSON documents.
Table G.5. Sample JSON Documents With a Untyped JSON Object
Sample JSON | Result |
---|---|
{ "jsonObject": {} } | Event has a jsonObject value that is an empty map |
{ "jsonObject": { "property-one": 1, "property-two": "abc" }} | Event has a jsonObject map-type value that has two entries: key propertyOne with value 1 (an integer) and key propertyTwo with value abc (a string) |
{ "jsonObject": null } | Event has a jsonObject value of null |
{} | Event has a jsonObject value of null |
{ "jsonObject" : ...anything but an object... } | Event has a jsonObject value of null |
The JSON parser, when it encounters a JSON array for the same name, populates an object-array Object[]
that contains the JSON array values and their nested values (deep).
The parser ignores any other JSON value type than array.
The Object[]
that the parser produces adheres to the rules for dynamic properties below.
Define a schema like this one:
@public @buseventtype create json schema MyEvent(jsonArray Object[])
The sample JSON documents are listed below.
Table G.6. Sample JSON Documents With an Untyped Array
Sample JSON | Result |
---|---|
{ "jsonArray": [] } | Event has a jsonArray value that is an empty array |
{ "jsonArray": [ "value-one", 1 ] } | Event has a jsonArray value object-array Object[] -type value that has two values: The string value-one and the integer 1 |
{ "jsonArray": null } | Event has a jsonArray value of null |
{} | Event has a jsonArray value of null |
{ "jsonArray" : ...anything but an array... } | Event has a jsonArray value of null |
The JSON parser, when it encounters any JSON value for the same name, provides the value as an object Object
that contains the JSON value and its nested values (deep).
The Object
that the parser produces adheres to the rules for dynamic properties below, e.g. Map<String, Object>
for JSON objects, Object[]
for JSON arrays
and other object values for the types as listed.
Define a schema like this one:
@public @buseventtype create json schema MyEvent(jsonValue Object)
The JSON documents for example cases are:
Table G.7. Sample JSON Documents with an Untyped Value
Sample JSON | Result |
---|---|
{ "jsonValue": {} } | Event has a jsonValue value that is an empty map of type Map<String, Object> |
{ "jsonValue": { "property-one": 1, "property-two": "abc" }} | Event has a jsonValue map-type value that has two entries: key propertyOne with value 1 (an integer) and key propertyTwo with value abc (a string) |
{ "jsonValue": [] } | Event has a jsonValue value that is an empty object-array of type Object[] |
{ "jsonValue": [ "value-one", 1 ] } | Event has a jsonValue object-array Object[] -type value that has two values: The string value-one and the integer 1 |
{ "jsonValue": 1 } | Event has a jsonValue value of 1 (an integer) |
{ "jsonValue": "abc" } | Event has a jsonValue value of abc (a string) |
{ "jsonValue": true } | Event has a jsonValue value of true (a boolean) |
{ "jsonValue": null } | Event has a jsonValue value of null |
{} | Event has a jsonValue value of null |
Your application may provide a class that has public fields and a default constructor. The EPL compiler inspects the class and determines the parser and serialization for the class according to the types listed in Section G.4, “JSON Supported Types” and described herein.
In the case that you don't want to define fields using EPL create json schema
, you specify an application class that provides the JSON schema. See Section G.10, “Customizing the JSON Event Class” instead.
To illustrate, the below class has two fields that hold a string-type name and UUID-type id:
public class Person { public String name; public UUID id; }
The schema shown next has a person
event property of type Person
(below example assumes the class is part of the imports; Or specify the fully-qualified class name).
@public @buseventtype create json schema MyEvent(person Person)
A sample JSON document is:
{ "person": { "name": "Joe", "id": "ff362753-b20d-4a9f-9144-93919cb12442" } }
The JSON parser supports, in addition to all types discussed earlier, the following:
List
of application class or other type. For example: public List<Person> persons
.
One-dimensional and two-dimensional arrays. For example: public Person[] persons
or public Person[][] personMatrix
.
For customizing parsing of strings see Section G.9, “Customizing JSON Serializing and Deserializing”.
The JSON parser takes the JSON value and invokes the valueOf
method of the enum class, allowing string and number values and returning null for JSON boolean, array or object values.
Let's say the schema is as follows:
@public @buseventtype create json schema MyEvent(enumValue my.company.package.EnumClass)
To show the behavior the example JSON is:
Table G.8. Sample JSON Documents With a Enum-Type Value
Sample JSON | Result |
---|---|
{ "enumValue": "enum_1" } | Event has an enumValue value that is EnumClass.valueOf("enum_1") |
{ "enumValue": null } | Event has an enumValue value of null |
{ "enumValue": 1} | Event has an enumValue value that is EnumClass.valueOf("1") |
{ "enumValue": false} | Event has an enumValue value of null |
{} | Event has an enumValue value of null |
{ "enumValue" : ...anything but a value... } | Event has an enumValue value of null |
The event properties that are known in advance can be listed as part of create json schema
. Since the name and type is known the EPL compiler can verify EPL statements that use the predefined properties in expressions.
See Section G.4, “JSON Supported Types”.
Event properties for which the property name is known but the type is not known can be defined as one of the untyped types Object
(for any JSON value), Object[]
(for any JSON array) or Map
(for any JSON object) which will provide nested values, arrays and objects; As for instance described at Section G.4.6, “Object Type for Holding an Untyped JSON Value (Any JSON Value)”.
By default, the JSON parser discards JSON properties whose property name does not match a property name in create schema
.
You must use the @JsonSchema(dynamic=true)
to instruct the EPL compiler to not discard JSON properties.
Add @JsonSchema(dynamic=true)
to create schema
so that the JSON parser parses and the event object keeps all properties available in the JSON document.
The EPL dynamic properties allow using properties in expressions that are not predefined. Please refer to Section 3.3, “Dynamic Event Properties”.
The dynamic JSON properties are also available in the JsonEventObject
event object for use by application code.
The following table outlines the object type for each JSON value:
Table G.9. JSON Value Types to Object Type Mapping
JSON Value Type | Object Type |
---|---|
JSON String | String |
JSON Number | Integer value returned by Integer.parseInt or the double value returned by Double.parseDouble |
JSON Boolean | Boolean |
JSON Null | null |
JSON Object | Map<String, Object> |
JSON Array | Object[] |
The next EPL creates an empty schema for sensor events:
@JsonSchema(dynamic=true) @public @buseventtype create json schema SensorEvent()
The sample JSON document is (use sendEventJson(json, "SensorEvent")
to process the document):
{ "entityID":"cd9f930e", "temperature" : 70, "status" : true, "entityName":{ "english":"Cooling Water Temperature" }, "vt":["2014-08-20T15:30:23.524Z"], "flags" : null }
You can use dynamic properties to query the events. Specifically here is EPL to retrieve the properties:
select entityID? as entityId, temperature? as temperature, status? as status, entityName? as entityName, vt? as vt, flags? as flags from SensorEvent
The output event for this example is:
Table G.10. Empty JSON Schema Sample Output Event
Name | Type | Value |
---|---|---|
entityId | String | "cd9f930e" |
temperature | Integer | 70 |
status | Boolean | true |
entityName | map | Map<String, Object> containing a single entry for key english and string-type value Cooling Water Temperature |
vt | Object[] | Array with a single string-type value 2014-08-20T15:30:23.524Z |
flags | null | null |
Use the dot ('.'
) to obtain the value of nested dynamic properties. For example:
select entityName?.english as englishEntityName from SensorEvent
Use select *
to obtain the JsonEventObject
object itself (use getUnderlying()
for listeners receiving EventBean
). For example:
select * as englishEntityName from SensorEvent
You may use @JsonSchema(dynamic=true)
together with predefined properties. For example:
@JsonSchema(dynamic=true) @public @buseventtype create json schema SensorEvent(entityID string)
Esper provides an API to parse the JSON text without processing the event. This is done by following these steps:
Define the event type schema or schemas. The event type must have public visibility and bus event type visibility.
Obtain an EventSenderJson
instance for the event type.
Invoke the parse
method on the sender, which returns the populated event object.
For instance, the below EPL declares the CarLocUpdateEvent
event type:
@public @buseventtype create json schema CarLocUpdateEvent(carId string, direction int)
Obtain the EventSenderJson
like so:
EventSenderJson sender = (EventSenderJson) runtime.getEventService().getEventSender("CarLocUpdateEvent");
The parse
method returns the populated event object.
JsonEventObject eventObject = sender.parse(json);
The returned object implements the JsonEventObject
interface which extends Map<String, Object>
. The interface provides additional methods for writing JSON. Please see the JavaDoc for more information.
Esper includes a shaded copy of the MinimalJson library in package com.espertech.esper.common.client.json.minimaljson
(MIT license, see https://github.com/ralfstx/minimal-json/blob/master/LICENSE).
MinimalJson provides the base parser. The Esper EPL compiler generates a specific JSON parser class that internally uses MinimalJson.
MinimalJson allows your application to build JSON text using the API. Please consult the JavaDoc for more information. A short example is provided below:
JsonObject jsonObject = new JsonObject().add("carId", "A123456").add("direction", 1); String json = jsonObject.toString();
Use the @JsonSchemaField
annotation to customize how the runtime deserializes (parses from JSON) a given event property and serializes (writes as JSON) a given event property.
The @JsonSchemaField
annotation can be used with create json schema
and also applies to @EventRepresentation('json')
for use with insert into
or create window
.
The name
value of @JsonSchemaField
provides the event property name. The adapter
property provides the class name of the class that implements the JsonFieldAdapterString
interface.
The adapter class must have a default constructor and must implement the T parse(String value)
and void write(T value, JsonWriter writer) throws IOException
methods.
The adapter implementation must be thread-safe and the runtime may reuse the same adapter instance.
Assume a JSON document that has a date-type field formatted as dd-M-yyyy
. The sample JSON schema is below. Assume that the MyDateJSONParser
class was added to imports.
You may specify the fully-qualified class name instead, i.e. adapter='com.mycompany.MyDateJSONParser'
.
Here is the sample EPL:
@JsonSchemaField(name=myDate, adapter=MyDateJSONParser) create json schema JsonEvent(myDate Date)
A basic JSON document is:
{"myDate" : "22-09-2018"}
The class below is a sample parser and writer for date values:
public class MyDateJSONParser implements JsonFieldAdapterString<Date> { public Date parse(String value) { try { return value == null ? null : new SimpleDateFormat("dd-M-yyyy").parse(value); } catch (ParseException e) { throw new EPException("Failed to parse: " + e.getMessage(), e); } } public void write(Date value, JsonWriter writer) throws IOException { if (value == null) { writer.writeLiteral("null"); return; } writer.writeString(new SimpleDateFormat("dd-M-yyyy").format(value)); } }
Instead of providing event property names and types as part of EPL create json schema
, you may specify an application class that provides public fields instead.
You must provide the class name in the classname
annotation value of the JsonSchema
annotation.
Assume there is a predefined person event class that has two fields that hold a string-type name and UUID-type id:
public class PersonEvent { public String name; public UUID id; }
This example declares the JSON person event to point to the PersonEvent
as the event object (below example assumes the class is part of the imports; Or specify the fully-qualified class name).
@public @buseventtype @JsonSchema(className='PersonEvent') create json schema JSONPersonEvent()
A sample JSON document is:
{ "name": "Joe", "id": "ff362753-b20d-4a9f-9144-93919cb12442" }
Please also see Section G.5, “JSON Application-Provided Class” for application class support and Section G.4, “JSON Supported Types” for type information.
By specifying an application class, you may not:
Define additional fields for the schema that don't have the same name and type as provided by the application class.
Define a supertype for the event type.
Use the dynamic
flag.
The following limitations apply:
A JSON event bean cannot contain EventBean
instances.
While the EPL compiler performs best-effort assignment checking and widening, it does not actually itself verify that the JSON contains valid data, for both production of JSON and consumption of JSON.
There is no API that maps a JSON document to your own application class. For that task please consider using Jackson (https://github.com/FasterXML/jackson
) or Gson (https://github.com/google/gson
).
The JSON parser, when encountering duplicate property names, uses the last value as the final value.