www.espertech.comDocumentation

Appendix G. Event Representation: JSON Events

This section provides information for using JSON to represent events.

An event can be represented by a JSON-formatted string value. Event properties of JSON events are the properties of the JSON object.

The advantages for supporting JSON as an event representation are:

Similar to the Map and object-array event type, the JSON event type takes part in the comprehensive type system that can eliminate the need to use Java classes as event types.

The runtime can process string-value events that are formatted JSON documents via the sendEventJson(String json, String jsonEventTypeName) method on the EPEventService interface.

The runtime does not validate JSON events. Your application must ensure that the JSON object matches the declaration of the schema in create json schema.

A given JSON event type can have only a single supertype that must also be a JSON event type. All properties available on the JSON supertype are also available on the type itself. In addition, anywhere within EPL that an event type name of a JSON supertype is used, the JSON subtype and the subtype of the subtype match that expression.

The Esper JSON parser produces an event object that is available to your application. The event object implements the JsonEventObject interface (package com.espertech.esper.common.client.json.util). The JsonEventObject extends Map<String, Object> allowing you to use event objects that are originating from JSON like any map.

In order to use JSON for incoming events, the event type name and JSON schema, if available, must be made known via the create json schema EPL syntax. Please see examples in Section 5.15, “Declaring an Event Type: Create Schema”.

In the case that you don't want to define fields using EPL create json schema, you may specify an application class that provides the JSON schema. Please refer to Section G.10, “Customizing the JSON Event Class”.

Note

  • JSON event types must be defined in EPL using create json schema. JSON event types cannot be pre-configured.

  • If you don't know some or all of the event properties in advance, please follow the instructions in Section G.6, “JSON Dynamic Event Properties”.

The code snippets below define a JSON event type, create a JSON event and send the event into the runtime. The sample defines the CarLocUpdateEvent event type. The @public means the event type has public visibility and the @buseventtype means your application can use sendEventJson for this event type.

@public @buseventtype create json schema CarLocUpdateEvent(carId string, direction int)

The CarLocUpdateEvent can now be used in a statement:

select count(*) from CarLocUpdateEvent(direction = 1)#time(1 min)

The sample code to send an event is:

String event = "{" +
  "  \"carId\" : \"A123456\",\n" +
  "  \"direction\" : 1\n" +
  "}";
runtime.getEventService().sendEventJson(event, "CarLocUpdateEvent");

JSON allows objects to be nested within other objects. Each JSON object may itself contain further JSON objects or arrays of JSON objects.

For example, consider the following JSON document:

{ 
  "isbn": "123-456-222",  
  "author": 
    {
      "lastname": "Doe",
      "firstname": "Jane"
    },
  "editor": 
    {
      "lastname": "Smith",
      "firstname": "Jane"
    },
  "title": "The Ultimate Database Study Guide",  
  "category": ["Non-Fiction", "Technology"]
 }

The author and editor objects are nested objects. Please define the schema for nested objects first, and the schemas that are building on the nested schemas thereafter.

Define the schemas as follows:

create json schema Names(lastname string, firstname string);
@public @buseventtype create json schema BookEvent(isbn string, author Names, editor Names, title string, category string[]);

This sample EPL outputs the isbn and the last name of the author and the editor, as well as the first category (use sendEventJson(json, "BookEvent") to process the document):

select isbn, author.lastname as authorName, editor.lastname as editorName, 
  category[0] as firstCategory from BookEvent

The next example JSON document is meant to demonstrate nested objects and arrays of objects:

{
  "id": "0001",
  "type": "donut",
  "name": "Cake",
  "batters": 	
  {
    "machine": "machine A",
    "batter":
    [
      { "id": "1001", "type": "Regular" },
      { "id": "1002", "type": "Chocolate" },
      { "id": "1003", "type": "Blueberry" },
      { "id": "1004", "type": "Devil's Food" }
    ]
  },
  "topping":
  [
    { "id": "5001", "type": "None" },
    { "id": "5002", "type": "Glazed" },
    { "id": "5005", "type": "Sugar" },
    { "id": "5007", "type": "Powdered Sugar" }
  ]
}

Define the schemas as follows:

create json schema IdAndType(id string, type string);
create json schema Batters(machine string, batter IdAndType[]);
@public @buseventtype create json schema CakeEvent(id string, type string, name string, batters Batters, topping IdAndType[]);

The following EPL outputs the name, the batter machine, type of the first batter and the type of the first topping, the count of batters, and the count of toppings of cake events (use sendEventJson(json, "CakeEvent") to process the document):

select name, batters.machine as batterMachine, batters.batter[0].type as firstBatterType,
  topping[0].type as firstToppingType, batters.batter.countOf() as countBatters,
  topping.countOf() as countToppings from CakeEvent

The create json schema EPL specifies the name and type of each event property. The create json schema may also specify no event properties or only a subset of all available event properties. When there are JSON properties that are not known at time of compilation, please see Section G.6, “JSON Dynamic Event Properties”.

The table below shows the supported types and provides links to more information. Types not listed below are not supported for use with a JSON event type.

Table G.1. Support JSON Event Property Types

TypeMore Information
Application-provided classSee Section G.5, “JSON Application-Provided Class”
bigdecimal, java.math.BigDecimalNumeric type parsed by new BigDecimal; See Section G.4.1, “Numeric Types”
biginteger, java.math.BigIntegerNumeric type parsed by new BigInteger; See Section G.4.1, “Numeric Types”
byte, java.lang.ByteNumeric type parsed by Byte.parseByte; See Section G.4.1, “Numeric Types”
boolean, java.lang.BooleanSee Section G.4.2, “Boolean Type”
double, java.lang.DoubleNumeric type parsed by Double.parseDouble; See Section G.4.1, “Numeric Types”
float, java.lang.FloatNumeric type parsed by Float.parseFloat; See Section G.4.1, “Numeric Types”
int, java.lang.IntegerNumeric type parsed by Integer.parseInt; See Section G.4.1, “Numeric Types”
long, java.lang.LongNumeric type parsed by Long.parseLong; See Section G.4.1, “Numeric Types”
nullNo parsing, a value that is always null.
short, java.lang.ShortNumeric type parsed by Short.parseShort; See Section G.4.1, “Numeric Types”
string, java.lang.String, char, java.lang.CharacterSee Section G.4.3, “String Type”
java.util.MapRepresents any untyped JSON object. See Section G.4.4, “Map Type for Holding an Untyped JSON Object”
Object[]Represents any untyped JSON array. See Section G.4.5, “Object-Array Type for Holding an Untyped JSON Array”
ObjectRepresents any untyped JSON value. See Section G.4.6, “Object Type for Holding an Untyped JSON Value (Any JSON Value)”
java.net.URIAs parsed by new URI(...)
java.net.URLAs parsed by new URL(...)
java.time.LocalDateAs parsed by LocalDate.parse
java.time.LocalDateTimeAs parsed by LocalDateTime.parse
java.time.OffsetDateTimeAs parsed by OffsetDateTime.parse
java.time.ZonedDateTimeAs parsed by ZonedDateTime.parse
java.util.UUIDAs parsed by UUID.fromString

The numeric types, the string type, the boolean type and the enumeration class type allow one- and two-dimensional arrays. The Map type does not support arrays and Object does not support two-dimensional arrays. An example with arrays is the schema create json schema MyEvent(prices bigdecimal[], intmatrix int[][]).

Use [primitive] to instruct the EPL compiler to use array of primitive values, for example create json schema MyEvent(intarray int[primitive]).

The default value for all event properties is null. When the JSON property of the same name is not present in the JSON event document the event property value is null.

The null JSON value is allowed for all types.


Your application may provide a class that has public fields and a default constructor. The EPL compiler inspects the class and determines the parser and serialization for the class according to the types listed in Section G.4, “JSON Supported Types” and described herein.

In the case that you don't want to define fields using EPL create json schema, you specify an application class that provides the JSON schema. See Section G.10, “Customizing the JSON Event Class” instead.

To illustrate, the below class has two fields that hold a string-type name and UUID-type id:

public class Person {
  public String name;
  public UUID id;
}

The schema shown next has a person event property of type Person (below example assumes the class is part of the imports; Or specify the fully-qualified class name).

@public @buseventtype create json schema MyEvent(person Person)

A sample JSON document is:

{
  "person": {
    "name": "Joe",
    "id": "ff362753-b20d-4a9f-9144-93919cb12442"
  }
}

The JSON parser supports, in addition to all types discussed earlier, the following:

The event properties that are known in advance can be listed as part of create json schema. Since the name and type is known the EPL compiler can verify EPL statements that use the predefined properties in expressions. See Section G.4, “JSON Supported Types”.

Event properties for which the property name is known but the type is not known can be defined as one of the untyped types Object (for any JSON value), Object[] (for any JSON array) or Map (for any JSON object) which will provide nested values, arrays and objects; As for instance described at Section G.4.6, “Object Type for Holding an Untyped JSON Value (Any JSON Value)”.

Note

By default, the JSON parser discards JSON properties whose property name does not match a property name in create schema.

You must use the @JsonSchema(dynamic=true) to instruct the EPL compiler to not discard JSON properties.

Add @JsonSchema(dynamic=true) to create schema so that the JSON parser parses and the event object keeps all properties available in the JSON document.

The EPL dynamic properties allow using properties in expressions that are not predefined. Please refer to Section 3.3, “Dynamic Event Properties”. The dynamic JSON properties are also available in the JsonEventObject event object for use by application code.

The following table outlines the object type for each JSON value:


The next EPL creates an empty schema for sensor events:

@JsonSchema(dynamic=true) @public @buseventtype create json schema SensorEvent()

The sample JSON document is (use sendEventJson(json, "SensorEvent") to process the document):

{
"entityID":"cd9f930e",
"temperature" : 70,
"status" : true,
"entityName":{
  "english":"Cooling Water Temperature"
},
"vt":["2014-08-20T15:30:23.524Z"],
"flags" : null
}

You can use dynamic properties to query the events. Specifically here is EPL to retrieve the properties:

select entityID? as entityId, temperature? as temperature, status? as status, 
    entityName? as entityName, vt? as vt, flags? as flags from SensorEvent

The output event for this example is:


Use the dot ('.') to obtain the value of nested dynamic properties. For example:

select entityName?.english as englishEntityName from SensorEvent

Use select * to obtain the JsonEventObject object itself (use getUnderlying() for listeners receiving EventBean). For example:

select * as englishEntityName from SensorEvent

You may use @JsonSchema(dynamic=true) together with predefined properties. For example:

@JsonSchema(dynamic=true) @public @buseventtype create json schema SensorEvent(entityID string)

Esper provides an API to parse the JSON text without processing the event. This is done by following these steps:

For instance, the below EPL declares the CarLocUpdateEvent event type:

@public @buseventtype create json schema CarLocUpdateEvent(carId string, direction int)

Obtain the EventSenderJson like so:

EventSenderJson sender = (EventSenderJson) runtime.getEventService().getEventSender("CarLocUpdateEvent");

The parse method returns the populated event object.

JsonEventObject eventObject = sender.parse(json);

The returned object implements the JsonEventObject interface which extends Map<String, Object>. The interface provides additional methods for writing JSON. Please see the JavaDoc for more information.

Esper includes a shaded copy of the MinimalJson library in package com.espertech.esper.common.client.json.minimaljson (MIT license, see https://github.com/ralfstx/minimal-json/blob/master/LICENSE). MinimalJson provides the base parser. The Esper EPL compiler generates a specific JSON parser class that internally uses MinimalJson.

MinimalJson allows your application to build JSON text using the API. Please consult the JavaDoc for more information. A short example is provided below:

JsonObject jsonObject = new JsonObject().add("carId", "A123456").add("direction", 1);
String json = jsonObject.toString();

Use the @JsonSchemaField annotation to customize how the runtime deserializes (parses from JSON) a given event property and serializes (writes as JSON) a given event property.

The @JsonSchemaField annotation can be used with create json schema and also applies to @EventRepresentation('json') for use with insert into or create window.

The name value of @JsonSchemaField provides the event property name. The adapter property provides the class name of the class that implements the JsonFieldAdapterString interface. The adapter class must have a default constructor and must implement the T parse(String value) and void write(T value, JsonWriter writer) throws IOException methods.

Assume a JSON document that has a date-type field formatted as dd-M-yyyy. The sample JSON schema is below. Assume that the MyDateJSONParser class was added to imports. You may specify the fully-qualified class name instead, i.e. adapter='com.mycompany.MyDateJSONParser'.

Here is the sample EPL:

@JsonSchemaField(name=myDate, adapter=MyDateJSONParser)
create json schema JsonEvent(myDate Date)

A basic JSON document is:

{"myDate" : "22-09-2018"}

The class below is a sample parser and writer for date values:

public class MyDateJSONParser implements JsonFieldAdapterString<Date> {
  public Date parse(String value) {
    try {
      return value == null ? null : new SimpleDateFormat("dd-M-yyyy").parse(value);
    } catch (ParseException e) {
      throw new EPException("Failed to parse: " + e.getMessage(), e);
    }
  }

  public void write(Date value, JsonWriter writer) throws IOException {
    if (value == null) {
      writer.writeLiteral("null");
      return;
    }
    writer.writeString(new SimpleDateFormat("dd-M-yyyy").format(value));
  }
}

Instead of providing event property names and types as part of EPL create json schema, you may specify an application class that provides public fields instead. You must provide the class name in the classname annotation value of the JsonSchema annotation.

Assume there is a predefined person event class that has two fields that hold a string-type name and UUID-type id:

public class PersonEvent {
  public String name;
  public UUID id;
}

This example declares the JSON person event to point to the PersonEvent as the event object (below example assumes the class is part of the imports; Or specify the fully-qualified class name).

@public @buseventtype @JsonSchema(className='PersonEvent') create json schema JSONPersonEvent()

A sample JSON document is:

{
  "name": "Joe",
  "id": "ff362753-b20d-4a9f-9144-93919cb12442"
}

Please also see Section G.5, “JSON Application-Provided Class” for application class support and Section G.4, “JSON Supported Types” for type information.

By specifying an application class, you may not:

  • Define additional fields for the schema that don't have the same name and type as provided by the application class.

  • Define a supertype for the event type.

  • Use the dynamic flag.

The following limitations apply: