setRetentionPolicy

Syntax

setRetentionPolicy(dbHandle, retentionHours, [retentionDimension], [hoursToColdVolume])

Arguments

dbHandle is a database handle. The data type of at least one of the partitioning columns of the database must be DATE or DATEHOUR.

retentionHours is a positive integer indicating the number of hours that data are kept.

retentionDimension (optional) is an integer indicating the layer of the temporal partition. The default value is 0 indicating the first layer of a COMPO partition is a temporal partition.

hoursToColdVolume (optional) is a positive integer indicating the number of hours that data are kept in volumes. Data stored in volumes will be migrated to the specified coldVolumes (configuration parameter) after hoursToColdVolume.

Note:

Make sure that retentionHours - hoursToColdVolume > 7 * 24 (7 days).

Details

Set the policy of data retention and tiered storage. The parameter retentionHours should be specified as large as possible for tiered storage to avoid deleting any data.

Both data retention and tiered storage are partition-based. Therefore, the interval configured by retentionHours and hoursToColdVolume must be divisible by the granularity of the partition.

The system will keep the data with the timestamp of the last retentionHours based on the system time: Data of the last hoursToColdVolume are stored in volumes; Data in [current time - hoursToColdVolumes - 10 days, current time - hoursToColdVolumes) are migrated to coldVolumes. If multiple paths are specified for coldVolumes, the data will be transferred randomly to one of the specified directories.

For other data, only data in the range of [current time - retentionHours - 10 days, current time - retentionHours) are deleted. To delete the data outside the range, you can call function dropPartition.
Note:

The function is only applied to a DFS database.

Examples

db=database("dfs://db1",VALUE,2019.06.01..date(now()))
retentionHour=9*24
hoursToColdVolume=1*24
setRetentionPolicy(db,retentionHour,0, hoursToColdVolume);

schema(db);
// output
partitionSchema->[2022.05.05,2022.05.04,2022.05.03,2022.05.02,2022.05.01,2022.04.30,2022.04.29,2022.04.28,2022.04.27,2022.04.26,...]
partitionSites->
partitionTypeName->VALUE
hoursToColdVolume->24
atomic->TRANS
databaseDir->dfs://db1
engineType->OLAP
chunkGranularity->TABLE
retentionDimension->0
partitionType->1
retentionHours->216