DStream::sync

Syntax

DStream::sync()

Arguments

None

Details

Waits for all parallel tasks to complete before proceeding with downstream operations. Typically used to synchronize and merge parallel processing branches.

Note: The DStream::parallelize and DStream::sync methods must be called together.

Examples

Partition the stream data into four streams based on the symbol column for downstream calculations:

use catalog test

g = createStreamGraph(name)
g.source("trade", 1024:0, `symbol`datetime`price`volume, [SYMBOL, TIMESTAMP,DOUBLE, INT])
  .parallelize("symbol", 4)
  .timeSeriesEngine(60*1000, 60*1000, <[first(price),max(price),min(price),last(price),sum(volume)]>, "datetime", false, "symbol")
  .reactiveStateEngine(<[datetime, first_price, max_price, min_price, last_price, sum_volume, mmax(max_price, 5), mavg(sum_volume, 5)]>, `symbol)
  .sync()
  .sink("output")
.g.submit()