Stream Event Serializer and Deserializer

Stream Event Serializer

The stream event serializer serializes real-time event streams from various data sources and writes them to heterogeneous stream tables that can be used as the input or output table of the CEP engine.

To define a serializer, you can use the streamEventSerializer function.

Syntax

streamEventSerializer(name, eventSchema, outputTable, [eventTimeField], [commonField]

Arguments

name is a string indicating the engine name. It consists of letters, digits, and underscores(_) and must start with a letter.

eventSchema is a table or a scalar/vector of class definition of event types, indicating the data to be serialized. If it is a table, it must have the schema as follows:

Column Data Type Comment
eventType STRING The event type.
eventField STRING The field names (separated by comma) of the event type.
fieldType (optional) STRING Data types (separated by comma) of eventField.
fieldTypeId INT[] Data type IDs of eventField.
fieldFormId INT[] Data form IDs of eventField (0: scalar; 1: vector; 2: pair; 3: matrix; 4: set; 5: dictionary; 6: table). Currently, only 0 and 1 can be specified.

outputTable is a non-partitioned in-memory table or a stream table for outputting the results. The output columns are in the following order:

(1) A time column of TIMESTAMP type (if eventTimeField is specified);

(2) A SYMBOL or STRING column indicating the events;

(3) A BLOB column that stores the serialized result of each event;

(4) The column(s) with the same names and data types (if commonField specified).

eventTimeField (optional) is a string scalar or vector indicating the name(s) of time field for the event(s).

  • If all events share the same time field name, simply specify it as a single string.

  • If the time field varies among events, specify a vector with the same length as eventSchema. Each element corresponds to one time field.

commonField (optional) is a string scalar or vector indicating the field(s) with the same name and data type. If specified, the common fields can be filtered out during subscription.

Stream Event Deserializer

The CEP engine has an internal implementation of the stream event deserializer, which eliminates the need for you to define an additional deserializer. The engine automatically deserializes the data retrieved from the heterogeneous stream tables to which you have subscribed.