mvccTable

Syntax

mvccTable(X, [X1], [X2], .....)

or

mvccTable(capacity:size, colNames, colTypes, [path], [tableName], [defaultValues], [allowNull])

Arguments

For the first scenario:

X, X1, X2 ...are vectors.

For the second scenario:

capacity is the amount of memory (in terms of the number of rows) allocated to the table. When the number of rows exceeds capacity, the system will first allocate memory of 1.2~2 times of capacity, copy the data to the new memory space, and release the original memory. For large tables, these steps may use significant amount of memory.

size is an integer no less than 0 indicating the initial size (in terms of the number of rows) of the table. If size=0, create an empty table. If size>0, the initialized values are determined by defaultValues.

colNames is a string vector of column names.

colTypes is a string vector of column types.

path is a string indicating the absolute path of the table on disk. The DFS path is not supported.

tableName is a string indicating the name of the table on disk.

defaultValues is a tuple that has the same length as colNames. It specifies the default values for all columns. If it is not specified, then the initialized values are:
  • false for Boolean type;

  • 0 for numeric, temporal, IPADDR, COMPLEX, and POINT types;

  • NULL for Literal, INT128 types.

allowNull is a Boolean vector that has the same length as colNames. It specifies whether to allow NULL values in each column. By default, elements of the Boolean vector are all true, meaning NULL values are allowed.

Details

Create an MVCC (Multi Version Concurrency Control) table. When appending, updating or deleting rows of a table, another version of the table is created so that concurrent read will not be blocked. The mvcc table is optimal for the use case with frequent read and append, but few update and delete operations.

If parameters path and tableName are specified, the table will be persisted to disk. The table on disk can be loaded into memory with function loadMvccTable.
Note:
  • When size is set to 0, if NULL values are disallowed for a column, an MVCC table can be created but NULL values cannot be appended to the column.

  • The MVCC table does not support addColumn, reorderColumns!, upsert!, drop, erase!.

Examples

Example 1. Create an MVCC table by two methods:

Method 1:

id=`XOM`GS`AAPL
x=102.1 33.4 73.6
mvccTable(id, x);
id x
XOM 102.1
GS 33.4
AAPL 73.6

Method 2:

mvccTable(200:10, `name`id`value, [STRING,INT,DOUBLE],"C:/DolphinDB/Data","t1");
name id value
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0

There will be files named t1.tbl, t1.sym and a folder named t1 under path C:/DolphinDB/Data. You should delete all these files before deleting a disk table.

Example 2. Create a partitioned MVCC table:

n=200000
colNames = `time`sym`qty`price
colTypes = [TIME,SYMBOL,INT,DOUBLE]
trades_mvcc1 = mvccTable(n:0, colNames, colTypes)
trades_mvcc2 = mvccTable(n:0, colNames, colTypes)
db=database(, VALUE, `A`D)
trades = createPartitionedTable(db,table=[trades_mvcc1, trades_mvcc2], tableName="", partitionColumns=`sym)

Before version 2.00.10.4, the update, appending and deletion operations are not supported on partitioned MVCC tables. You can only operate on the tablets trades_mvcc1 and trades_mvcc2 to modify the partitioned MVCC table trades.

insert into trades_mvcc1 values(09:30:00.001,`A,100,56.5)
insert into trades_mvcc2 values(09:30:01.001,`D,100,15.5)

insert into trades values(09:30:00.001,`D,100,26.5)
Error: Can't append data to a segmented table that contains external partitions.

select * from trades;
time sym qty price
09:30:00.001 A 100 56.5
09:30:01.001 D 100 15.5

Since 2.00.10.4, the update, appending and deletion operations can be directly applied to partitioned MVCC tables.

insert into trades values(09:30:00.001,`D,100,26.5)
select * from trades;
time sym qty price
09:30:00.001 A 100 56.5
09:30:01.001 D 100 15.5
09:30:01.001 D 100 26.5
delete from trades where sym=`A
select * from trades;
time sym qty price
09:30:01.001 D 100 15.5
09:30:01.001 D 100 26.5
update trades set price=price*10 where sym=`D
select * from trades;
time sym qty price
09:30:01.001 D 100 155
09:30:01.001 D 100 265

Note that when appending data to a tablet of a partitioned MVCC table, the system would not validate whether the data matches the table schema. If an out-of-scope record is appended to a tablet, it can lead to data corruption in the partitioned MVCC table. Therefore, it is more recommended to operate directly on the table instead of its tablets.