Drop Partitions

Use command dropPartition to delete data from selected partitions.

Please note that dropPartition does not change the partitioning scheme. We do not need to reestablish these partitions if we need to append new data to them.

Examples

Reminder: the script in this example should be executed on a data node of a cluster.

n=1000000
ID=rand(150, n)
dates=2017.08.07..2017.08.11
date=rand(dates, n)
x=rand(10.0, n)
t=table(ID, date, x)
dbDate = database(, VALUE, 2017.08.07..2017.08.11)
dbID = database(, RANGE, 0 50 100 150)
db = database("dfs://compoDB", COMPO, [dbDate, dbID])
pt = db.createPartitionedTable(t, `pt, `date`ID)
pt.append!(t);

The script above created a database with composite partition. The first level is a value partition with partitioning column of date, and the second level is a range partition with partitioning column of ID.

Example 1: Delete one partition.

Use either of the following ways to delete the partition "/20170807/0_50".

(1) Specify the partition path.

dropPartition(db,"/20170807/0_50");

(2) Specify the filtering condition.

dropPartition(db,[2017.08.07, 0]);

0 means the partition of [0, 50). We can choose any number from 0 to 49 to represent this partition.

Example 2: Delete a first level partition.

Use either of the following ways to delete the first level partition of 2017.08.08.

(1) Specify the path of all partitions under 2017.08.08.

partitions=["/20170808/0_50","/20170808/50_100","/20170808/100_150"]
dropPartition(db,partitions);

(2) Specify the filtering condition.

dropPartition(db,2017.08.08);

Example 3: Delete a second level partition.

Use either of the following ways to delete the second level partition of [0,50).

(1) Specify the path of all partitions of [0,50).

partitions=["/20170807/0_50","/20170808/0_50","/20170809/0_50","/20170810/0_50","/20170811/0_50"]
dropPartition(db,partitions);

(2) Specify the filtering condition.

dropPartition(db,[,[0]]);

Example 4: Delete multiple same level partitions.

To delete the second level partitions of [0,50) and [100,150):

dropPartition(db,[,[0,100]]);

Function existsPartition can be used to check whether a partition exists or not. For example, check if partition "dfs://compoDB"中"/20170807/0_50" exists:

existsPartition("dfs://compoDB/20170807/0_50")