Time Concepts

System time refers to the (local) machine time when the system processes the streaming data. When stream processing operations rely on system time, all time-based operations (e.g., time windows) are based on the local time of the machine running the program.

System time is straightforward to implement, offering low latency and high performance because calculations are triggered immediately after the end of each time window. However, in distributed or asynchronous environments, system time can be affected by issues such as network delays and varying latencies in data arrival across machines.

Event time refers to the actual time when an event occurs. Each record has an associated, fixed event time before it enters the processing system. Event time is independent of the system time of the processing machine. When calculations are based on event time, the results are consistent and reproducible, regardless of delays in data arrival at the processing server.

Factors like communication latency and scheduling delays can cause deviations between event and system time. In practice, system time offers lower latency, while event time ensures consistency between live and historical backtesting results. DolphinDB streaming engines use the parameter useSystemTime to control whether to use the system time or the event time for calculations.

• When useSystemTime = true, the engine uses the system time (with millisecond precision) when the data enters the engine.
• When useSystemTime = false (default), the engine uses the time column in the data for all time-based calculations.

When using event time, streaming engines operate under the assumption that records with timestamps earlier than the current record have already been received. Once a record arrives that exceeds the current window's boundary, the engine automatically closes the window and preserves the relevant state information within that time frame. To avoid computational delays during periods of inactivity, some engines also offer trigger parameters such as updateTime and forceTriggerTime of createTimeSeriesEngine, and maxDelayedTime of createEquiJoinEngine.