1 to 1 Single Table Replay

Replaying a single table to a target table with the same schema is supported by the replay function. The syntax is as follows:

replay(inputTables, outputTables, [dateColumn], [timeColumn], [replayRate], [absoluteRate=true], [parallelLevel=1], [sortColumns])

For details on the parameters, refer to replay.

Example 1

The following example demonstrates how to replay a single table to a target table with the same schema at different rates.

  • Create input and output tables for replaying and writing simulated data to the input table:
    // Create an input table and insert simulated data
    n=1000
    sym = take(`IBM,n)
    date = take(2012.06.12,n)
    time = take(temporalAdd(09:30:12.000,1..500,'s'),n)
    volume = rand(100,n)
    trades = table(sym, date, time, volume)
    trades.sortBy!(`time)
    // Create an output table
    share streamTable(100:0,`sym`date`time`volume,[SYMBOL, DATE, TIME, INT]) as st
  • Replay 100 records per second. For 1,000 records in table trades, it takes about 10 seconds.
    timer replay(inputTables=trades, outputTables=st, dateColumn=`date, timeColumn=`time,replayRate=100, absoluteRate=true)
    // Time elapsed: 10001.807 ms
  • Replay at 100 times the time span of the data. The difference between the start timestamp and the end timestamp in table trades is 500 seconds, and it takes about 5 seconds to replay the table.
    timer replay(inputTables=trades,outputTables=st,dateColumn=`date,timeColumn=`time,replayRate=100,absoluteRate=false)
    // Time elapsed: 5000.036 ms
  • Replay at the maximum speed:
    timer replay(inputTables=trades,outputTables=st,dateColumn=`date,timeColumn=`time)
    // Time elapsed: 1.024 ms

Example 2

The following example uses the replayDS function to replay data in a DFS table.

  • Create input and output tables for replay and write simulated data to the input table:
    // Create an input table and insert simulated data
    n=int(60*60*6.5)
    sym = take(take(`IBM,n).join(take(`GS,n)), n*2*3)$SYMBOL
    date=take(2021.01.04..2021.01.06, n*2*3).sort!()
    time=take(09:30:00..15:59:59,n*2*3)$TIME
    volume = rand(100, n*2*3)
    t=table(sym,date,time,volume)
    // Write the input table to a database
    if(existsDatabase("dfs://test_stock")){
    dropDatabase("dfs://test_stock")
    }
    db1=database("",RANGE, 2021.01.04..2021.01.07)
    db2=database("",VALUE,`IBM`GS)
    db=database("dfs://test_stock",COMPO,[db1, db2])
    trades=db.createPartitionedTable(t,`trades,`date`sym)
    trades.append!(t)
    // Create an output table
    share streamTable(100:0,`sym`date`time`volume,[SYMBOL, DATE, TIME, INT]) as st
  • Use the replayDS function to split the data source:
    ds = replayDS(sqlObj=<select * from loadTable(db, `trades)>, dateColumn=`date, timeColumn=`time)
    // View the number of split data sources
    ds.size()
    
    // output: 3
  • Use the replay function to replay the split data sources at the maximum speed:
    timer replay(inputTables=ds1, outputTables=outTable, dateColumn=`date, timeColumn=`time)
    // Time elapsed: 20.809 ms