Add Partitions

We can add new partitions for a partitioned database with VALUE domain:

• Set the configuration parameter newValuePartitionPolicy=add. In cluster mode, it is set in cluster.cfg. In standalone mode m it is set in dolphindb.cfg.

• Use function addValuePartitions .

For a partitioned database with VALUE domain, if the new data does not belong to the existing partitioning scheme, we can specify how the new data is processed with the configuration parameter newValuePartitionPolicy.

1. By default, newValuePartitionPolicy=skip. The records in the new data that belong to the existing partitioning scheme will be saved to the database, while those that don't belong to the existing partitioning scheme will not be saved to the database.

2. newValuePartitionPolicy=fail. If the new data has at least one records that doesn't belong to the existing partitioning scheme, an exception is thrown and none of the new data will be saved to the database.

3. newValuePartitionPolicy=add. If the new data has records that doesn't belong to the existing partitioning scheme, new partitions will be automatically generated. All of the new data will be saved to the database.

If newValuePartitionPolicy=skip or fail, we can use function addValuePartitions to extend partitioning scheme. In the following example, we add new date partitions from 2017.08.12 to 2017.08.20 in database "dfs://compoDB".

n=1000000
ID=rand(100, n)
dates=2017.08.07..2017.08.11
date=rand(dates, n)
x=rand(10.0, n)
t=table(ID, date, x);

dbID=database(, RANGE, 0 50 100);
dbDate = database(, VALUE, 2017.08.07..2017.08.11)
db = database("dfs://compoDB", COMPO, [dbID, dbDate]);
pt = db.createPartitionedTable(t, `pt, `ID`date)
t.append!(t);

addValuePartitions(db,2017.08.12..2017.08.20,1);

// output
9

Please note: the database should be reloaded after adding new partitions.

db=database("dfs://compoDB")
pt=loadTable(db,"pt")
t1=table(0..99 as ID,take(2017.08.12,100) as date,rand(10.0,100) as x)
pt.append!(t1)
select count(*) from loadTable("dfs://compoDB","pt");

// output
1000100

For a partitioned database with RANGE domain, new partitions cannot be automatically generated with a configuration parameter. Data that falls outside existing partitions is discarded by default.You can use function addRangePartitions to extend the existing partitioning scheme. Note that the new partition can only be added after the last existing partition.

To add 3 new ID partitions [100,150), [150,200) and [200,250) to the example above:

addRangePartitions(db,100 150 200 250,0);

// output
3

The database should be reloaded after adding new partitions.

db=database("dfs://compoDB")
pt=loadTable(db,"pt")
t1=table(rand(100..249,10000) as ID,rand(2017.08.07..2017.08.12,10000) as date,rand(10.0,10000) as x)
pt.append!(t1)
select count(*) from loadTable("dfs://compoDB","pt");

// output
1010100