Maintenance and Visualization
DolphinDB provides different functions that allow you to monitor the CEP engine. You can retrieve information about the CEP engine and create customized read-only data views to monitor the data generated by the engine for easy consumption by external clients.
Additionally, DolphinDB's web interface offers a CEP Streaming Engine view, accessible through the Stream tab. This view provides a visual representation of the CEP engine's status, performance metrics, and output data.
Checking Status of CEP Engine
Checking Status of All CEP Engines
You can use getStreamEngineStat().CEPEngine
to check the status
of all CEP engines in the current session. It returns a table containing the
following columns:
Column Name | Description |
---|---|
name | the name of the CEP engine |
user | the name of the user who creates the engine |
status | CEP engine status |
lastErrorMessage | the last error message |
lastErrorTimestamp | the timestamp of the last error message |
useSystemTime | whether useSystemTime is set to true |
numOfSubEngine | the number of sub-engines |
queueDepth | the queue depth |
eventsReceived | the number of receivedevents |
eventsEmitted | the number of events emitted to the output queue |
eventsOnOutputQueue | the number of events in the output queue |
Checking Status of specific CEP Engine
You can use getCEPEngineStat
to check the status of a specific
CEP engine.
Syntax
getCEPEngineStat(engine)
Arguments
engine is the engine object returned by
createCEPEngine
.
Return Values
It returns a dictionary containing the following keys:
-
EngineStat: a dictionary with the info of engine status (same as the returned info as
getStreamEngineStat().CEPEngine
). -
eventSchema: a table with the info of all events received.
Column Name | Description |
---|---|
eventType | the event type |
eventField | the field names (separated by comma) of the event type |
fieldType | data types (separated by comma) of eventField |
fieldTypeId | data type IDs of eventField |
fieldFormId | data form IDs of eventField (0: scalar; 1: vector; 2: pair; 3: matrix; 4: set; 5: dictionary; 6: table). Currently, only 0 and 1 will be returned. |
-
subEngineStat: a table with the info of sub-engine status.
Column Name | Description |
---|---|
subEngineName | the name of the sub-engine |
eventsOnInputQueue | the number of events in the input queue |
monitorNumber | the number of monitors |
listeners | the number of listeners |
timers | the number of timers. When using
addEventListener with any of the following
parameters specified: at, wait, within, or
exceedTime, a timer will be created. |
eventsRouted | the number of events routed to the front of the input queue |
eventsSent | the number of events sent to the end of the input queue |
eventsReceived | the number of receivedevents |
eventsConsumed | the number of matched events |
lastEventTime | the timestamp of the last event |
lastErrorMessage | the last error message |
lastErrorTimestamp | the timestamp of the last error message |
-
dataViewEngines: a table with the info of data view engines' status.
Column Name | Description |
---|---|
name | the name of the data view engine |
user | the name of the user who creates the engine |
status | data view engine status |
lastErrorMessage | the last error message |
lastErrorTimestamp | the timestamp of the last error message |
keyColumns | key column(s) (separated by space) specified by parameter keyColumns. |
outputTableName | the output table name |
useSystemTime | whether useSystemTime is set to true |
throttle | the time interval between data writes to the outputTable. If throttle parameter is not specified, returns NULL. |
numItems | the row count |
memoryUsed | the memory (in bytes) occupied by data view engine |
Visualizing with Data View
The CEP engine operates on an event-driven basis. As events are continuously fed into the engine, it generates and updates numerous intermediate variables within its internal computations. Users typically want to monitor the latest values of these variables and observe their trends over time. To facilitate this, DolphinDB provides a data view engine that enables the CEP engine to write the monitored values to the data view engine during its execution.
The data view engine is responsible for maintaining an up-to-date snapshot of each monitored variable and exporting the data to a target table, usually a stream table, which can be subscribed by other clients.
Creating a Data View Engine
The createDataViewEngine
function allows you to create a data
view engine that returns a keyed table with keyColumns as the key. The
table maintains the latest record for each key.
Syntax
createDataViewEngine(name, outputTable, keyColumns, timeColumn,
[useSystemTime=true], [throttle])
Arguments
name is a string indicating the name of the data view engine. It consists of letters, digits, or underscores (_) and must start with a letter.
outputTable is an in-memory table or a DFS table. If you want to display real-time data, or to graph trends of the data, it must be a stream table.
keyColumns is a STRING scalar or vector that specifies one or more columns in the outputTable as the key columns. For each unique value in the keyColumn, only the latest record is retained.
timeColumn is a STRING scalar which specifies the time column in the output table.
useSystemTime (optional) is a Boolean value indicating whether to use the system time as the time column for the output table.
-
If the input data does not contain a time column, useSystemTime should set to true, i.e., the time column of outputTable is the system time.
-
If the input data contain a time column, useSystemTime should set to false, i.e., the time column of outputTable uses the timestamp of each record.
throttle (optional) is of DURATION type, which specifies the time interval between data writes to the outputTable.
Obtaining Data from a Data View Engine
To get the handle of data view engine defined in a specific CEP engine, you can
use getDataViewEngine
.
Syntax
getDataViewEngine([CEPEngine], dataViewEngineName)
Arguments
CEPEngine (optional) is the handle of a CEP engine.
dataViewEngineName is the name of data view engine.
Examples
dvEngine= getDataViewEngine(,`dv1)
select * from dvEngine
dvEngine= getDataViewEngine(`cep1,`dv1)
select * from dvEngine
To simply get the handle of a data view engine outside a CEP engine context, you
can use the getStreamEngine
function. For example,
select * from getStreamEngine(`dv1)
Inserting Data to Data View Engine
Use append!
, tableInsert
, insert
into
to insert data to a data view engine (with the handle returned
by getDataViewEngine
).
Updating Values of Data View
The data view engine maintains a keyed table with keyColumns (of
createDataViewEngine
) as the primary key. You can call
updateDataViewItems
to update values of a data view through
keys.
Syntax
updateDataViewItems(engine, keys, valueNames, newValues)
Arguments
engine is the handle of a data view engine.
keys is a scalar, vector, or tuple, indicating the key(s) to be updated.
valueNames is a STRING scalar or vector, indicating the columns (from the
outputTable of createDataViewEngine
) to be
updated.
newValues is a scalar, vector, or tuple, indicating the updated values.
Deleting Values from Data View
To remove values from a data view, calldeleteDataViewItems
.
Syntax
deleteDataViewItems(engine, keys)
Arguments
engine is the handle of a data view engine.
keys is a scalar, vector, or tuple, indicating the key(s) to be removed.
The updateDataViewItems
and
deleteDataViewItems
functions can be called within
a CEP engine context or outside of it.
-
Within a CEP engine: The system will first attempt to locate a data view engine handle with specified engine within the current CEP engine context. If a matching engine is found, the corresponding update and delete operation will be performed. If no such engine, the system will then search for a data view engine with the same handle outside of the CEP engine context.
-
Outside a CEP engine: The system will search for and perform corresponding operation on a data view engine handle with specified engine only from the context outside of any CEP engine.