replayDS

replayDS(sqlObj, [dateColumn], [timeColumn], [timeRepartitionSchema])

sqlObj is metacode with SQL statements. The table object in the SQL statement is a DFS table and must use a DATE type column as one of the partitioning columns.

dateColumn (optional) must be a time column of the table queried by the SQL statement, based on which the data is sorted. It can be of DATE (most commonly used), MONTH or other temporal types. If dateColumn is specified, it must be one of the partitioning columns of the DFS table. Data sources are generated based on the time precision of the dateColumn, e.g., if the dateColumn is partitioned by day, the data source is also divided by day.

timeRepartitionSchema (optional) is a vector of temporal type. If timeColumn is specified, timeRepartitionSchema deliminates multiple data sources based on timeColumn. For example, if timeRepartitionSchema =[t1, t2, t3], then there are 4 data sources within each day: [00:00:00.000,t1), [t1,t2), [t2,t3), and [t3,23:59:59.999).

Generates a tuple of data sources from a DFS table (queried by a SQL statement) based on its time columns. It can be further divided by the parameters timeColumn and timeRepartitionSchema.

It is used as the inputs of function replay. To replay a DFS table, the replay function must be conjuncted with the replayDS function.

n=int(60*60*6.5)
sym = take(take(`IBM,n).join(take(`GS,n)), n*2*3)
date=take(2021.01.04..2021.01.06, n*2*3).sort!()
time=take(09:30:00..15:59:59,n*2*3)
volume = rand(100, n*2*3)
t=table(sym,date,time,volume)
if(existsDatabase("dfs://test_stock")){
dropDatabase("dfs://test_stock")
}
db1=database("",RANGE, 2021.01.04..2021.01.07)
db2=database("",VALUE,`IBM`GS)
db=database("dfs://test_stock",COMPO,[db1, db2])
trades=db.createPartitionedTable(t,`trades,`date`sym)
trades.append!(t);
ds = replayDS(sqlObj=<select * from loadTable(db, `trades)>, dateColumn=`date, timeColumn=`time)
ds.size();
// output
3

ds = replayDS(sqlObj=<select * from loadTable(db, `trades)>, dateColumn=`date, timeColumn=`time, timeRepartitionSchema=[11:30:00, 14:00:00])
ds.size();
// output
9