ops

Syntax

cancelJobEx(id=NULL)

Arguments

• id: a string indicating a background job ID, which you can get with the server function getRecentJobs().

Details

Cancels running background jobs in the cluster. If id is specified, cancel the specified job; otherwise, cancel all the background jobs in the cluster.

Example

Create 3 background jobs:

def testJob(n,id){
   for(i in 0:n){
        writeLog("demo"+id+"is working")
        sleep(1000)
   }
}
submitJob("demo1","test background job1",testJob,300,1);
submitJob("demo2","test background job2",testJob,300,2);
submitJob("demo3","test background job3",testJob,300,3);

Cancel the job "demo1" and get the status of all background jobs on the data nodes and compute nodes:

 cancelJobEx("demo1") 
 pnodeRun(getRecentJobs)

The result shows that "demo1" is marked as "The task was canceled."

To cancel all jobs, run the following script:

pnodeRun(getRecentJobs)

Syntax

closeInactiveSessions(hours=12)

Arguments

• hours: a numeric value indicating the session timeout period (in hours). The default value is 12.

Return

Returns a table containing information on all active sessions in the cluster. The table has the same schema as the table returned by the server function getSessionMemoryStat.

Details

If a session has been inactive for a time period longer than the specified hours, it is considered as timed out. Call this function to close all inactive sessions.

Note: To check the last active time of a session, call server function getSessionMemoryStat.

Examples

getSessionMemoryStat()

closeInactiveSessions(0.05)

Syntax

getDDL(database, tableName)

Arguments

• database: a string indicating the path to a distributed database, e.g., "dfs://demodb".
• tableName: a string indicating the name of a DFS table

Details

Returns the DDL statements that can be used to recreate the specified database and the DFS table, as well as the column names and column types of the DFS table.

Examples

ID=rand(10, n)
x=rand(1.0, n)
t=table(ID, x)
db=database("dfs://rangedb", RANGE,  0 5 10)
pt=db.createPartitionedTable(t, `pt, `ID)
getDDL("dfs://rangedb", "pt")
#output

db = database("dfs://rangedb")
colName = `ID`x
colType = [INT,DOUBLE]
tbSchema = table(1:0, colName, colType)
db.createPartitionedTable(table=tbSchema,tableName=`pt,partitionColumns=`ID)

Syntax

getTableDiskUsage(database, tableName, byNode=false)

Arguments

• database: a string indicating the path to a distributed database, e.g., "dfs://demodb".
• tableName: a string indicating the name of a DFS table.
• byNode: is a Boolean indicating whether to display disk usage by node. The default value is false, i.e., to display the total disk usage of all nodes.

Details

Returns a table displaying the disk usage of the specified DFS table. It has the following columns:

• node: a string indicating a node alias. It is returned only when byNode = true.
• diskGB: a DOUBLE value indicating the disk usage of the specified DFS table.

Examples

getTableDiskUsage("dfs://rangedb", "pt", true)

Syntax

dropRecoveringPartitions(dbPath , tableName="")

Arguments

• dbPath:a string indicating the path to a distributed database, e.g., "dfs://demodb".
• tableName: a string indicating the name of a DFS table. Specify it only when the database chunk granularity is at TABLE level (i.e.,database: chunkGranularity = 'TABLE').

Details

Deletes the partitions in RECOVERING status from the specified database. tableName is a required parameter when the database chunk granularity is at TABLE level.

Example

First, get the metadata of all chunks in the cluster with the following server functions:

rpc(getControllerAlias(), getClusterChunksStatus)

The result suggests that both chunk files of the "olapDemo" database are in RECOVERING status.

Execute dropRecoveringPartitions to force delete these two partitions:

dropRecoveringPartitions(database("dfs://olapDemo"));

Syntax

getAllLicenses()

Arguments

None

Details

Returns a table displaying the license expiration date of all nodes in the cluster. It has the following columns:

• nodeAlias: a string indicating a node alias.
• endDate: a date indicating the expiration date.

Examples

getAllLicenses()

Syntax

updateAllLicenses()

Arguments

None

Return

Returns a table displaying the license expiration date of all nodes in the cluster. It has the following columns:

• nodeAlias: a string indicating a node alias.
• endDate: a date indicating the expiration date.

Details

Note: Execute this function after you have replaced the license files on the nodes.

Updates the license on all nodes in a cluster without a reboot. Return license expiration information.

Example

updateAllLicenses()

Syntax

unsubscribeAll()

Arguments

None

Details

Cancels all subscriptions on the current node.

Examples

share streamTable(10:0, `id`val, [INT, INT]) as st
t = table(10:0, `id`val, [INT, INT])
subscribeTable(tableName=`st, actionName=`sub_st, handler=append!{t})
undef(st, SHARED)
#error
All subscriptions to the shared stream table [st] must be canceled before it can be undefined.

unsubscribeAll()
undef(st, SHARED)

Syntax

gatherClusterPerf(monitoringPeriod=60, scrapeInterval=15, dir="/tmp")

Arguments

• monitoringPeriod: an integer indicating the time frame (in seconds) of monitoring. The default value is 60.
• scrapeInterval: the interval (in seconds) at which the monitoring metrics are scraped. The default value is 15.
• dir: a string indicating an existing directory to save the monitoring result. The default value is "/tmp".

Note: For a Windows system, use absolute-path and forward slashes ("/") or double backslashes (\) to separate the directories.

Details

Gets cluster performance monitoring metrics based on the specified monitoring period and scrape interval. Exports the result to the specified directory in a statis.csv file. For more information about the monitoring metrics, see server function getClusterPerf.

Examples

gatherClusterPerf(30, 3, "/tmp") 
// check the result in /tmp/statis.csv after 30 seconds

Syntax

gatherStreamingStat(subNode, monitoringPeriod=60, scrapeInterval=15, dir="/tmp")

Arguments

• subNode: a string indicating the alias of a subscriber node.
• monitoringPeriod: an integer indicating the timeframe (in seconds) of the monitoring. The default value is 60.
• scrapeInterval: the interval (in seconds) at which the monitoring metrics are scraped. The default value is 15.
• dir: a string indicating an existing directory to save the monitoring result. The default value is "/tmp".

Note: For a Windows system, use absolute-path and forward slashes ("/") or double backslashes (\) to separate the directories.

Details

Gets the status of workers on the a subscriber node based on the specified monitoring period and scrape interval. Export the result to the specified directory in a sub_worker_statis.csv file. For more information about the monitoring metrics, see server function getStreamingStat.

Examples

gatherStreamingStat("subNode",30, 3, "/tmp") 
// check the result in /tmp/sub_worker_statis.csv after 30 seconds

Syntax

getDifferentData(t1, t2)

Arguments

• t1 / t2: a handle to an in-memory table.

Details

Checks if the values of t1 and t2 are identical by calling server function eqObj. t1 and t2 must be of the same size.

Return

Returns the rows that are different in the two specified tables; otherwise, print "Both tables are identical".

Examples

t1=table(1 2 3 as id, 4 5 6 as val)
t2=table(1 8 9 as id, 4 8 9 as val)
t3=table(1 2 3 as id, 4 5 6 as val)
for (row in getDifferentData(t1, t2))
  print row
#output
id val
-- ---
2  5
3  6
id val
-- ---
8  8
9  9

getDifferentData(t1, t3)
#output
Both tables are identical

Syntax

checkChunkReplicas(dbName, tableName, targetChunkId)

Arguments

• dbName: a string indicating the path to a distributed database, e.g., "dfs://demodb".
• tableName: a string indicating the name of a DFS table.
• targetChunkId: a string indicating a chunk ID which you can get with server function getTabletsMeta.

Details

Checks if the two replicas of the specified chunk are identical. This function is available only when the configuration parameter dfsReplicationFactor is set to 2 on the controller.

Return

A Boolean indicating whether the data of two chunk replicas are identical.

Examples

n=1000000
ID=rand(10, n)
x=rand(1.0, n)
t=table(ID, x)
db=database("dfs://rangedb", RANGE,  0 5 10)
pt=db.createPartitionedTable(t, `pt, `ID)
pt.append!(t)
checkChunkReplicas("dfs://rangedb", "pt", "af8268f0-151e-c18b-a84c-a77560b721e6")
#output
true

Stop a data node with the kill -9 PID command:

pt.append!(t)
checkChunkReplicas("dfs://rangedb", "pt", "af8268f0-151e-c18b-a84c-a77560b721e6")// get the chunk ID with getTabletsMeta()
#output
checkChunkReplicas: throw "colFiles on two replicas are not same" => colFiles on two replicas are not same

Reboot the data node. When the chunk recovery is complete, execute checkChunkReplicas() again:

checkChunkReplicas("dfs://rangedb", "pt", "af8268f0-151e-c18b-a84c-a77560b721e6") // chunk ID can be checked with getTabletsMeta() 
#output
true

Syntax

clearAllSubscriptions()

Arguments

None

Details

Cancels all stream table subscriptions on the current node.

Return

Returns the names and handle names of the unsubscribed stream tables, and prints "All subscriptions have been cleared!".

Example

clearAllSubscriptions()
#output
unsub: st, sub1
All subscriptions have been cleared !

Syntax

dropAllEngines()

Arguments

None

Details

Drop all streaming engine on the current node.

Return

Prints “All engines have been dropped!”.

Example

dropAllEngines()
#output
All engines have been dropped !

Syntax

existsShareVariable(names)

Arguments

• names: a string scalar or vector specifying the object name(s).

Details

Determines whether each element in a string scalar or vector is a shared variable.

Return

Returns a scalar or vector indicating whether each element in names is a shared variable.

Example

share streamTable(10000:0, `timestamp`sym`val, [TIMESTAMP, SYMBOL, INT]) as variable1
existsShareVariable("variable1")
//Output: true

Syntax

clearAllSharedTables()

Arguments

None

Details

Deletes all shared tables on the current node.

Return

Returns the names of the deleted shared tables and prints "All shared table have been cleared!".

Example

clearAllSharedTables()
#output
Drop Shared Table: st
All shared table have been cleared !

Syntax

clearAllStreamEnv()

Arguments

None

Details

Clears all stream table subscriptions, streaming engines, and shared tables.

Return

Returns a summary of the return values of clearAllSubscriptions() , dropAllEngines() , and clearAllSharedTables() .

Example

clearAllStreamEnv()
#output
unsub: st, sub1
All subscriptions have been cleared !
All engines have been dropped !
Drop Stream Table: dummyTable1
All stream table have been cleared !

Syntax

getPersistenceTableNames()

Arguments

None

Details

Retrieves the names of all persisted shared stream tables.

Return

Returns the names of all persisted shared stream tables.

Example

getPersistenceTableNames()
// Output: [st1, st2]

Syntax

getNonPersistenceTableNames()

Arguments

None

Details

Returns the names of all non-persisted shared stream tables.

Return

The names of all non-persisted shared stream tables.

Example

getNonPersistenceTableNames()
// Output: [st1, st2]

Syntax

getPersistenceStat()

Arguments

None

Details

Returns the status of all persisted shared stream tables.

Return

Returns the metadata of all persisted shared stream tables.

Example

getPersistenceStat()

Syntax

getNonPersistenceTableStat()

Arguments

None

Details

Returns the status of all non-persisted shared stream tables.

Return

Returns the metadata of all non-persisted shared stream tables.

Example

getNonPersistenceTableStat()

Syntax

clearAllPersistenceTables()

Arguments

None

Details

Deletes all persisted stream tables.

Return

None

Example

clearAllPersistenceTables()

Syntax

getDatabaseDDL(dbName)

Arguments

• dbName: a string scalar specifying the database name.

Details

Gets the creation statement for the specified database.

Return

A string scalar indicating the creation statement of the specified database.

Example

dbName = "dfs://testDatabaseRANGE"
db1 = database(dbName, RANGE, 1985.01.01 1990.01.01 1995.01.01 2000.01.01 2005.01.01 2010.01.01)
getDatabaseDDL(dbName)
#output
database(directory = 'dfs://testDatabaseRANGE', partitionType = RANGE, partitionScheme =[1985.01.01,1990.01.01,1995.01.01,2000.01.01,2005.01.01,2010.01.01], engine= `OLAP, atomic = `TRANS)

Syntax

getDBTableDDL(dbName,tbName)

Arguments

• dbName: a string scalar specifying the database name.

• tbName: a string scalar specifying the table name.

Details

Gets the create table statement for a table in the specified database.

Return

A string scalar indicating the creation statement of the specified table.

Example

dbName = "dfs://test_OLAP_RANGE"
tbName1 = "partition1"
tbName2= "dimension1"
db1 = database(dbName, RANGE, 1985.01.01 1990.01.01 1995.01.01 2000.01.01 2005.01.01 2010.01.01)
tbSchema = table(1:0, `SecurityID`DateTime`Open`High`Low`Close, [SYMBOL, DATETIME, DOUBLE, DOUBLE, DOUBLE, DOUBLE])
db = database(dbName)
tb1 = createPartitionedTable(db, tbSchema, tbName1, ["DateTime"])
tb2 = createTable(db, tbSchema, tbName2)
getDBTableDDL(dbName,tbName1)
getDBTableDDL(dbName,tbName2)
#output
createPartitionedTable(dbHandle = database('dfs://test_OLAP_RANGE'),table = table(1:0, ["SecurityID","DateTime","Open","High","Low","Close"],["SYMBOL","DATETIME","DOUBLE","DOUBLE","DOUBLE","DOUBLE"]),tableName = 'partition1',partitionColumns =["DateTime"],compressMethods = dict(["SecurityID","DateTime","Open","High","Low","Close"],["lz4","lz4","lz4","lz4","lz4","lz4"]))

createTable(dbHandle = database('dfs://test_OLAP_RANGE'),table = table(1:0, ["SecurityID","DateTime","Open","High","Low","Close"],["SYMBOL","DATETIME","DOUBLE","DOUBLE","DOUBLE","DOUBLE"]),tableName = 'dimension1',compressMethods = dict(["SecurityID","DateTime","Open","High","Low","Close"],["lz4","lz4","lz4","lz4","lz4","lz4"]))

Syntax

getObjectAccess(objName)

Arguments

• objName: a string scalar that represents one of the following:
  • DFS database path, e.g., dfs://db1
  • DFS table path, e.g., dfs://db1/pt1 (same format as used in permission functions such as grant and deny)
  • Function view name.

Details

Returns all users with permissions on the specified object and their specific permissions, each permission represented as a boolean. In DolphinDB 3.00.0 and later versions, permissions granted at the catalog level are also identified and returned.

• When querying a database: returns one row per user for each accessible table, where objectName is the table's full path.

• When querying a table or function view: returns one row per user.

• The result includes only users with permissions. The following two types of users are included in the result even if they are not explicitly granted permissions via grant:

  • The built-in admin user: has full permissions on all objects even though not returned by getUserList.
  • The table creator: has full read and write permissions on the tables they created.

Return

Returns an in-memory table with the following columns:

Example

db = database("dfs://db1", VALUE, 1..3)
t = table(1 2 3 as id, 10 20 30 as price)
db.createPartitionedTable(t, `pt1, `id).append!(t)
db.createPartitionedTable(t, `pt2, `id).append!(t)
def myView(){ return 1 }
addFunctionView(myView)
createUser("u1", "123456")
grant("u1", DB_READ, "dfs://db1") // can read the whole database
createUser("u2", "123456")
grant("u2", TABLE_READ, "dfs://db1/pt1") // can read only table pt1
createUser("u3", "123456")
grant("u3", VIEW_EXEC, "myView") // can execute myView
// Query database: one row per accessible table, objectName is the full path of the table.
getObjectAccess("dfs://db1")

Query a single table.

getObjectAccess("dfs://db1/pt1")

Query a function view.

getObjectAccess("myView")