3.00.2

• In previous releases, append! could be used to append data to a tuple member, and the referenced object itself was also modified. As of this release, this operation is no longer supported.
• Added function memberModify! for modifying tuple members.

For example:

// create a tuple tp
a = [1, 2, 3, 4]
b = [6, 7, 8]
tp = [a, b]
tp // [[1, 2, 3, 4], [6, 7, 8]]

Modify the tuple with append!:

// Modify the first element of tp
tp[0].append!(5)
tp
// Output (previous releases): [[1, 2, 3, 4, 5], [6, 7, 8]]. Modified successfully.
// Output (current release):[[1, 2, 3, 4], [6, 7, 8]]. Tuple remains unchanged
a
// Output (previous releases): [1, 2, 3, 4, 5]. The value of a was also changed.
// Output (current release): [1, 2, 3, 4]. The value of a is not affected.

Modify the tuple with memberModify!:

// Modify the first element of tp
memberModify!(tp, append!, 0, 5)
tp 
// Output: [[1, 2, 3, 4, 5], [6, 7, 8]]

// Modify the second element
memberModify!(tp, append!, 1, 9)
tp 
// Output: [[1, 2, 3, 4, 5], [6, 7, 8, 9]]

• In previous releases, it would drop the entire table.
• Since this release, an error is reported "Please use truncate instead of dropPartition to delete the entire table."

// create a table t with 1000000 rows
n = 1000000;
t = table(2020.01.01T00:00:00 + 0..(n-1) as timestamp, rand(`IBM`MS`APPL`AMZN,n) as symbol, rand(10.0, n) as value)
db = database("dfs://rangedb_tradedata", RANGE, `A`F`M`S`ZZZZ)
Trades = db.createPartitionedTable(table=t, tableName="Trades", partitionColumns="symbol", compressMethods={timestamp:"delta"});
Trades.append!(t)
t1 = loadTable(db, "Trades")
select count(*) from t1
// Output: 1000000

// Call dropPartition without specifying partitionPaths
dropPartition(db, , tableName="t")
// Error (previous releases): None of the specified partitions exist. RefId: S01056'
// Error (current release): Please use truncate instead of dropPartition to delete the entire table.

// Check the number of rows in the table
select count(*) from t1
// Output (previous releases): 0. The entire table is dropped.
// Output (current release): 1000000

// Create a stream table for publishing
n = 1000
sym = take(`IBM,n)
date = take(2012.06.12,n)
time = take(temporalAdd(09:30:12.000,1..500,'s'),n)
volume = rand(100,n)
trades = table(sym, date, time, volume)
trades.sortBy!(`time)
// Create a stream table for subscribing
share streamTable(100:0,`sym`date`time`volume,[SYMBOL, DATE, TIME, INT]) as st

unsubscribeTable(,"st1","sub1")
// No result or error was returned in previous releases.
// Error (current release): The shared table st1 doesn't exist.

• In previous releases, NULL was returned.
• Since this release, the error "X must be a vector or a matrix." is reported.

percentChange(1,1)
// NULL was returned in previous releases.
// Error (current release): X must be a vector or a matrix.

sym = `A`B`C
id = 1 2 3
val = 11 22 33
t=keyedTable(`id`sym,sym,id,val)
insert into t values ("A", 1.0, 55)
t

Table t in previous releases:

Since this release, record with the same key is correctly updated:

• In previous releases, sum(first(col)) took the first element in the col group for the sum calculation.
• Since this release, sum(first(col)) takes the first element from each window in the col group for the sum calculation.

t = table(["A","B","A","B","B","A"] as
              sym,[1,2,3,4,5,6] as id,[8, 2, 3, 6, 6, 4] as val) wj(t, t, 1:10,
      <[first(val),sum((first(val))+val)]>,`sym`id)

Results in previous releases:

Results since this release:

• In previous releases, the external variable was calculated by group.
• Since this release, the aggregate function is applied to the entire external variable, regardless of grouping.

Take the avg function as an example:

t = table( [1,2,1,2,3] as id, [1,3,5,7,9] as val)
v = [1,2,3,4,5]
select avg(v) from t group by id

In previous releases, the aggregation results varied with each group: The id1 group contained the first and third rows of table t, so avg_v was the average of the first and third elements of v. Similarly, avg_v of the id2 group 2 was the average of the second and fourth elements.

Since this release, the result for each group remains the same, which is the value of avg(v), 3.

• In previous releases, isortTop was performed disregarding the groups.
• Since this release, isortTop is performed within each group.

t = table([1,1,2,2,3,3,4,4,5,5] as id, 1..10 as val)
select isortTop(val,2) from t context by id

Results in previous releases:

Results since this release:

• In previous releases, subarray was performed within each partition. The results from each partition were combined into the final output.
• Since this release, subarray is performed once on all data loaded to memory.

db = database("dfs://test",VALUE,1..3)
t = table(take(1..3,15) as id,1..15 as val)
pt = db.createPartitionedTable(t,"pt",`id)
pt.append!(t)

select subarray(val,0:3) from pt

Results in previous releases:

Results since this release:

• In previous releases, the query was executable but the result was incorrect.
• Since this release, an error is reported.

dbName = "dfs://test_distinct"
if(existsDatabase(dbName)){
	dropDB(dbName)
}
db = database(dbName,RANGE,0 100 200 300 400,,`TSDB)
x = table(take(10..15 join 105..120 join 255..260 join 360..380,10000) as id,take(1 2 4 2 3,10000) as val)
pt = createPartitionedTable(db, x, 'pt', `id,,`id).append!(x)

select funcByName(`first)(distinct val) from pt

// Error (current release): Cannot nest aggregate function. RefId: S02032

• In previous releases, the query was executable but the result was incorrect.

• Since this release, an error is reported.

arr = 1..10
id = take(1..5,10)
type1 = take(1..2, 10)
type2 = take(1..3, 10)

t = table(arr, id, type1, type2)

select rowAvg(arr, id) from t pivot by type1, type2

// Error (current release): Row reduction operation [rowAvg] cannot be used with PIVOT BY. RefId: S02029

createDailyTimeSeriesEngine when the input column type was inconsistent with that specified by dummyTable.

• In previous releases, the query was executable without any output.
• Since this release, an error is reported.

share streamTable(1000:0, `time`sym`qty`price, [TIMESTAMP, SYMBOL, INT, INT]) as trades
output1=streamTable(1:0,`time`sym`avg_qty`avg_price,[TIMESTAMP,SYMBOL,DOUBLE,DOUBLE])
agg1=createTimeSeriesAggregator(name="agg1",windowSize=30000, step=30000, metrics=<[avg(qty),avg(price)]>, 
dummyTable=trades, outputTable=output1, timeColumn=`time, keyColumn=`sym)

n = 10000
insert into agg1 values(2024.02.01T00:00:00.000+1000*(1..n),take(`A,n),take(`A,n),take(`A,n))

// Error (current release): insert into agg1 values (2024.02.01T00:00:00.000 + 1000 * (1 .. n), take("A", n), take("A", n), take("A", n)) => Failed to append data to column 'price'

• In previous releases, the query was executable but the result was incorrect.
• Since this release, an error is reported.

dbName ="dfs://test_pointManagement_append"
if(existsDatabase(dbName)) {dropDB(dbName)}
dbvalue = database(, VALUE, 1..10)
dbtime = database(, VALUE, 2012.01.01..2012.01.10)
db = database(dbName, COMPO, [dbvalue, dbtime],,`TSDB)
create table "dfs://test_pointManagement_append"."pt1"(
	par1 INT, par2 TIMESTAMP, intv INT
) 
partitioned by par1, par2, 
sortColumns=[`par1, `par2],
keepDuplicates = ALL
go
create table "dfs://test_pointManagement_append"."pt2"(
	par1 INT, par2 TIMESTAMP, intv INT
) 
partitioned by par1, par2, 
sortColumns=[`par1, `par2],
keepDuplicates = ALL
go

t1 = table(1..10 as par1, 2012.01.01..2012.01.10 as par2, 1..10 as intv)
t2 = table(1..10 as par1, 2012.01.01..2012.01.10 as par2, 1..10 as intv)

pt1 = loadTable("dfs://test_pointManagement_append","pt1")
pt2 = loadTable("dfs://test_pointManagement_append","pt2")
pt1.append!(t1)
pt2.append!(t2)

delete pt1 from ej(pt1, pt2, `par1`par2)
// Error (current release): Joiner doesn't support both tables are distributed tables when left tablet type is SINGLE.

• In previous releases, the query was executable but the result was incorrect.
• Since this release, an error is reported.

dbName = "dfs://test_analytical_function"
if(existsDatabase(dbName)){
	dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..100)
n = 1000
t = table(take(1..10,n) as id,rand(100,n) as value)
pt = db.createPartitionedTable(t, "pt", ["id"]).append!(t)
m = 1
re = select *, row_number() OVER (partition by 1 order by m) as row_col from loadTable(dbName, "pt")
select *, 1..size(re)  from loadTable(dbName, "pt")
// Error (current release): All columns must be of the same length.