User Access Control

DolphinDB offers a powerful, flexible, and secure access control system with the following features:

  • Access roles like admin, user, and group for convenient permission control.
  • 19 types of access privileges for various scenarios.
  • Built-in access control functions/commands.
  • Function views to provide analysis results without sacrificing data privacy.
  • Dynamic access control for scheduled jobs and streaming tasks.
  • RSA encryption for key information.
  • SSO to simplify cross-domain access.

This tutorial will cover the key components of DolphinDB's access control system and how to effectively manage database security.

1. User Roles

Before logging in DolphinDB, a user can access databases as a guest who does not have (and cannot be granted with) any access privileges. A user can log in (out) with command login (logout).

1.1 User Types

1.1.1 Users and Groups

DolphinDB uses groups to manage users with the access privileges. A user can belong to 0, 1 or multiple groups, and a group can have 0, 1 or multiple users.

A user's access privileges are determined by his/her own privileges and the privileges of the groups that he/she belongs to. You can grant or deny the permissions for users or groups. When the permission of a group changes, it applies to all members in the group. When a new member joins a group, the user inherits all privileges from the group.

1.1.2 Administrators

There are 2 types of administrators in DolphinDB: super admin and admin.

When a DolphinDB cluster is started for the first time, a super admin is created with user ID "admin" and password "123456". This super admin cannot be deleted and has all access privileges that cannot be revoked or denied.

To create an admin, you can set the parameter isAdmin to true for function createUser.

1.1.3 Administrators v.s. Users

Users other than the administrators are regular users.

Only administrators can create/delete users and groups, and grant/revoke/deny the permissions of other administrators, users and groups. The following user-management functions can only be executed by administrators: resetPwd, createUser, deleteUser, createGroup, deleteGroup, addGroupMember, and deleteGroupMember.

Users (including administrators) can change their passwords by calling changePwd. Administrators can use resetPwd to modify the passwords for other users.

The following table summarizes the differences among super admin, admin and regular user.

super adminadminuser
Need to be manually created×
Initial privilegesAll PrivilegesNoneNone
Can be deleted×
Can be listed by getUserList()×
Can create or delete admin/user/group×
Can grant or deny privileges for admin/user/group×
Can create or delete function views×
Can delete scheduled jobs submitted by other users×

1.2 Role Management

As of version 1.30.21/2.00.9, user-management functions can be executed on data nodes, while in previous versions these functions can only be called on controllers.

1.2.1 User Management

Only administrators can create users:

login(`admin, "123456")
createUser("admin1","123456",,true);

login("admin1","123456")
createUser("user1","123456",,false) // create a user
createUser("user2","123456",,true) // create an admin

All users can change their own passwords:

//user1 changes password
login("user1","123456")
changePwd("123456","123456@")
logout("user1")

login("user1","123456@") // password changed successfully

Only administrators can change other users' passwords:

// admin changes password for user1 login
login("admin1","123456")
resetPwd("user1","123456")

login("user1","123456")
// successfully logged in with new password

Only administrators can delete users:

// admin deletes user1 
login("admin1","123456")
deleteUser("user1")

login("user1","123456") 
// Error: The user name or password is incorrect

1.2.2 Group Management

Only administrators can manage user groups, including:

Creating a group:

createGroup("group1",["admin1"])

Adding a group member:

addGroupMember(["user2"],"group1")

Removing a group member:

deleteGroupMember(["user2"],"group1")

Deleting a group:

deleteGroup("group1")

When a group is deleted, the group members are not deleted, but all permissions that were applied to the group are revoked. For details on the privileges on users and groups, see 3.1 Users and Groups' Access Privileges.

1.2.3 Check User Information

  • getGroupList: Return a list of group names. It can only be executed by administrators.
  • getUsersByGroupId: Return a string vector of the user names that belong to the specified group. It can only be executed by administrators.
  • getGroupsByUserId: Return a string vector of the groups that the user belongs to. It can only be executed by administrators.
  • getGroupAccess: Return a table of privileges for one or multiple groups. It can only be executed by administrators.
  • getUserList: Return a list of user names other than the administrators. It can only be executed by administrators.
  • getUserAccess: Return privileges for individual users, without taking into account the privileges for the groups the users belong to. When userId is not specified, return the privileges for the current user. Only an administrator can use this function to get other users' privileges.

2. Access Management

Users can use commands grant, deny and revoke to set access privileges, which are marked as "allow", "deny" or "none" in the permission state.

Note: As of version 1.30.21/2.00.9, access-management functions can be executed on data nodes, while in previous versions these functions can only be called on controllers.

grant: grant specific privileges to a user/group

grant(`user2, DBOBJ_CREATE,"*")  getUserAccess("user2")

revoke: revoke specific privileges that were previously granted or denied to a use/group

revoke(`user2, DBOBJ_CREATE,"*") getUserAccess("user2")

deny: deny specific privileges to a user/group

deny(`user2, DBOBJ_CREATE,"*")  getUserAccess("user2")

3. Privilege Rules

3.1 Users and Groups

For the privilege rules discussed below, a user is viewed as belonging to a special group with himself/herself as the only member. The user's access privileges are determined by the privileges of the groups that he/she belongs to.

Different groups may set different access privileges to a user. For these situations the following rules apply:

  • If a user is granted a privilege in one group and this privilege is not denied in any other group, then the user is granted the privilege.
  • If a user is denied a privilege in at least one group, then the user's privilege is denied.

Example 1. If a user belongs to one group, and the user's permission state is deny, while the group's state is allow, then the user's permission is denied.

login("admin","123456")
createUser("user1","123456")
createGroup("group1",["user1"])
deny(`user1, DB_OWNER) // deny DB_OWNER to user1
grant(`group1, DB_OWNER) // grant DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases

Example 2. If a user belongs to one group, and the user's permission state is none, while the group's state is allow, then the user's permission is granted.

login("admin","123456")
revoke(`user1, DB_OWNER) // revoke the denied DB_OWNER
grant(`group1, DB_OWNER) // grant DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is granted

Example 3. If a user belongs to one group, and the user's permission state is none, while the group's state is deny, then the user's permission is denied.

login("admin","123456")
revoke(`user1, DB_OWNER) // revoke the denied DB_OWNER
deny(`group1, DB_OWNER) // deny DB_OWNER to group1
login("user1","123456")
database("dfs://test",VALUE,1..10)//  user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases

Example 4. If a user belongs to multiple groups, and some groups' permission state is deny, while some is allow, then the user's permission is denied.

login("admin","123456")
createGroup("group2",["user1"])
createGroup("group3",["user1"])
deny(`group1, DB_OWNER)
grant(`group2, DB_OWNER)
grant(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases

Example 5. If a user belongs to multiple groups, and some groups' permission state is none, while some is allow, then user's permission is granted.

login("admin","123456")
revoke(`group1, DB_OWNER)
grant(`group2, DB_OWNER)
grant(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is granted

Example 6. If a user belongs to multiple groups, and some groups' permission state is none, while some is deny, then the user's permission is denied.

login("admin","123456")
revoke(`group1, DB_OWNER)
deny(`group2, DB_OWNER)
deny(`group3, DB_OWNER)
login("user1","123456")
database("dfs://test",VALUE,1..10) // user1 is denied
// Error: <NoPrivilege>Not granted to create or delete databases

Example 7. If a user belongs to a group, the user retains only his/her own privileges after the group is deleted.

login("admin","123456")
createUser("user1","123456")
createGroup("group1",["user1"])
grant(`user1, DB_OWNER) //grant DB_OWNER to user1
deny(`group1, DB_OWNER) //deny DB_OWNER to group1
deleteGroup(`group1) //delete group1
login("user1","123456")
database("dfs://test",VALUE,1..10) //user1 is granted
login("admin","123456")
createGroup("group1",["user1"])
revoke(`user1, DB_OWNER) //revoke granted DB_OWNER
grant(`group1, DB_OWNER) //grant DB_OWNER to group1
deleteGroup(`group1) //delete group1
login("user1","123456")
database("dfs://test",VALUE,1..10) //user1 is denied

3.2 Access Scopes

Based on the application scope, permissions are divided into three scopes: global (*), database and table.

The following are privilege rules for determining permissions on databases and tables:

  • If you grant/revoke/deny at a smaller scope, and then manage permissions at a larger scope, the smaller scope permissions will be invalidated and overridden by the larger scope permissions.
  • If you allow at a larger scope, and then deny at a smaller scope, the deny operation will take effect while the allow permissions still apply to the objects outside of the deny scope.
  • If you allow/deny at a larger scope, and then revoke at a smaller scope, the revoke operation will be invalid while the allow permissions still apply to the original scope.
  • If you deny at a larger scope, and then grant at a smaller scope:
    • For versions before 1.30.21 and 2.00.9, the grant can take effect while the deny permissions still apply to the objects outside of the grant scope.
    • Since version 1.30.21 and 2.00.9, this operation is not allowed.
  • If you revoke in a large scope, and then manage permissions at a smaller scope, the smaller scope permissions will take effect.

Example 1. First deny TABLE_READ at table level to user 1, then grant TABLE_READ at global level. Now user1 has the TABLE_READ access to all tables:

login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)

deny("user1", TABLE_READ, dbName+"/pt")
grant("user1", TABLE_READ, "*")

login("user1", "123456")
select * from loadTable(dbName, "pt") 
// user1 is granted TABLE_READ to "dfs://test"

Example 2. First grant TABLE_READ at table level to user2, then deny TABLE_READ at global level. Now user2 is denied to access all tables:

login("admin", "123456")
createUser("user2","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)

grant("user2", TABLE_READ, dbName+"/pt")
deny("user2", TABLE_READ, "*")

login("user2", "123456")
select * from loadTable(dbName, "pt") 
// user2 is denied to access "dfs://test"

Example 3. First grant TABLE_READ at table level to user3, then revoke TABLE_READ at global level. Now the TABLE_READ permissions on all tables of user3 are revoked.

login("admin", "123456")
createUser("user3","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)

grant("user3", TABLE_READ, dbName+"/pt")
revoke("user3", TABLE_READ, "*")
login("user3", "123456")
select * from loadTable(dbName, "pt") 
// user3's privilege is revoked

Example 4. First grant TABLE_READ at global level to user1, then deny TABLE_READ on the table "dfs://test/pt" to user1. Now user1 is granted TABLE_READ access to all tables except "dfs://test/pt".

login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user1", TABLE_READ, "*")
deny("user1", TABLE_READ, dbName+"/pt")

login("user1", "123456")
select * from loadTable(dbName, "pt") 
// user1's access to "dfs://test/pt" is denied

Example 5. First grant TABLE_READ to user1, then revoke user1's TABLE_READ access to "dfs://test/pt". Now user1 is still granted TABLE_READ access to all tables.

login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)
grant("user1", TABLE_READ, "*")
revoke("user1", TABLE_READ, dbName+"/pt")

login("user1", "123456")
select * from loadTable(dbName, "pt") 
// user1 is granted to access "dfs://test/pt"

Example 6. First deny TABLE_READ at global level to user1, then revoke the denied TABLE_READ access to table "dfs://test/pt". Now user1 is still denied to read all tables.

login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt.append!(t)

deny("user1", TABLE_READ, "*")
revoke("user1", TABLE_READ, dbName+"/pt")

getUserAccess("user1")

Example 7. First deny TABLE_READ at global level to user1, then grant TABLE_READ on table "dfs://test/pt" to user1.

For versions before 1.30.21/2.00.9, user1 has the TABLE_READ access to table "dfs://test/pt", but the access to other tables is denied; For versions since 1.30.21/2.00.9, the grant operation will raise an error.

login("admin", "123456")
createUser("user1","123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id")
pt1=  db.createPartitionedTable(t, "pt1", "id")
pt.append!(t)
pt1.append!(t)

deny("user1", TABLE_READ, "*")
// As of 1.30.21/2.00.9 version, the grant operation reports 'Invalid grant: grant [dfs://test/pt] and [deny *] are in conflict'
grant("user1", TABLE_READ, dbName+"/pt")
login("user1", "123456")
select * from loadTable(dbName, "pt")
// In the previous versions, user1 is granted to access "dfs://test/pt"
select * from loadTable(dbName, "pt1")
// user1 is denied to access other tables

3.3 Compatibility

If users created before version 1.30.21/2.00.9 were granted table-level privileges like TABLE_READ and TABLE_WRITE on some tables while denied those privileges on other tables, after a server upgrade to version 1.30.21/2.00.9 or higher, they have access to all tables except those that were explicitly denied.

For example, user1 is granted TABLE_READ access to pt1, and denied TABLE_READ access to pt2 in versions before 1.30.21/2.00.9. Then user1 is only granted to access "dfs://valuedb/pt1".

login("admin","123456")
createUser("user1","123456")
grant("user1",TABLE_READ,"dfs://valuedb/pt1")
deny("user1",TABLE_READ,"dfs://valuedb/pt2")

After the server is upgraded, user1 is granted to access to all tables of database "dfs://valuedb" except pt2. The above script is equivalent to:

grant("user1",TABLE_READ,"dfs://valuedb/*")
deny("user1",TABLE_READ,"dfs://valuedb/pt2")

4. Privilege Types

4.1 Database-Level Privileges

PrivilegeDescription
DB_MANAGEDatabase level: delete databases. Table level: create/delete/rename tables, add/delete partitions, add/delete/rename/replace columns
DB_OWNERDatabase level: create databases; delete databases created by themselves. Table level (in databases owned by themselves): create/delete/rename tables, add/delete partitions, add/delete columns. Users with this privilege can grant other users the following privileges on the databases created by themselves: TABLE_READ, TABLE_WRITE, TABLE_INSERT, TABLE_UPDATE, TABLE_DELETE, DBOBJ_CREATE, DBOBJ_DELETE, DB_READ, DB_WRITE, DB_INSERT, DB_UPDATE, DB_DELETE
DBOBJ_CREATEcreate tables in specific databases
DBOBJ_DELETEdelete tables and schema from specific databases
DB_READread tables in specific databases
DB_WRITEwrite to tables in specific databases
DB_INSERTinsert into tables in specific databases
DB_UPDATEupdate tables in specific databases
DB_DELETEdelete tables from specific databases

4.2 Table-Level Privileges

PrivilegeDescription
TABLE_READread tables
TABLE_WRITEwrite to tables
TABLE_INSERTinsert into tables
TABLE_UPDATEupdate tables
TABLE_DELETEdelete tables

4.3 Other Privileges

PrivilegeDescription
VIEW_EXECexecute specific function views
SCRIPT_EXECexecute scripts
TEST_EXECexecute test scripts
QUERY_RESULT_MEM_LIMITset the memory limit for a query result
TASK_GROUP_MEM_LIMITset the memory limit for a task group

4.4 Compatibility of Privilege Types

(1) If users created before version 1.30.21/2.00.9 were only granted DB_MANAGE but not DB_OWNER, after a server upgrade they are not allowed to create databases.

(2) If users created before version 1.30.21/2.00.9 were granted TABLE_WRITE, after a server upgrade they are also granted TABLE_INSERT, TABLE_UPDATE, and TABLE_DELETE privileges.

(3) When using grant/deny/revoke with accessType=DB_OWNER, the objs parameter for databases must end with "*", indicating the prefix pattern of databases.

login("admin", "123456")
createUser("user1","123456")
grant("user1", DB_OWNER,"*") 
// user1 is granted DB_OWNER access to all databases
grant("user1", DB_OWNER,"dfs://test0*")
// user1 is granted DB_OWNER access to all databases with a predix of "dfs://test0"

(4) When using grant/deny/revoke with accessType=DB_MANAGE, the databases specified in objs must already be created. Otherwise an error will be reported.

login("admin", "123456")
createUser("user1","123456")
grant("user1", DB_MANAGE,"dfs://test0") 
// Error: The database [dfs://test0] does not exist
grant("user1", DB_MANAGE, ["dfs://db1","dfs://db2"]) 
// Error: The database [dfs://db1] does not exist

4.5 Privilege Revocation

A table-level privilege is revoked when the involved user or table is deleted, and a database-level privilege is revoked when the involved user or database is deleted. A DB_OWNER privilege is revoked only when the user and all involved databases are deleted.

For privileges applied on shared in-memory tables or streaming engines, they will not be revoked after server restarts, or when the shared tables or streaming engines are deleted. Therefore, if new shared tables or streaming engines are created with the same names, the previously-assigned privileges still take effect.

Example 1. user1 is granted TABLE_READ on table "dfs://valuedb/pt". Delete and recreate the table, and the TABLE_READ privilege is revoked from user1.

login("admin", "123456")
dbName = "dfs://valuedb"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
        dropDatabase(dbName)
} 
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id").append!(t)
​
createUser("user1","123456")
grant("user1", TABLE_READ,"dfs://valuedb/pt")
login("user1", "123456")
select * from loadTable("dfs://valuedb",`pt)
// user1 is granted to access "dfs://valuedb/pt"
login("admin", "123456")
dropTable(db,`pt)
pt=  db.createPartitionedTable(t, "pt", "id").append!(t)
login("user1", "123456")
select * from loadTable("dfs://valuedb",`pt)
// the TABLE_READ access of user1 is revoked

Example 2. user1 is granted DB_MANAGE on the database "dfs://valuedb". Delete and recreate the database, and the DB_MANAGE privilege is revoked from user1.

login("admin", "123456")
dbName = "dfs://valuedb"
t = table(1..10 as id , rand(100, 10) as val)
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id").append!(t)

createUser("user1","123456")
grant("user1", DB_MANAGE,"dfs://valuedb")
getUserAccess("user1")

Now user1 is granted the DB_MANAGE access to database "dfs://valuedb".

login("admin", "123456")
if(existsDatabase(dbName)){
  dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "id").append!(t)
getUserAccess("user1")

The DB_MANAGE privilege is revoked from user1.

Example 3. The user u1 is granted read and write privileges on the stream tables "trades", "output1" and the streaming engine "agg1":

login(`admin, `123456)
createUser(`u1, "111111");
enableTableShareAndPersistence(streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]),`trades)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as  output1
agg1 = createTimeSeriesEngine(name="agg1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)
grant("u1", TABLE_READ, "agg1")
grant("u1", TABLE_WRITE, "agg1")
grant("u1", TABLE_READ, "trades")
grant("u1", TABLE_WRITE, "trades")
grant("u1", TABLE_READ, "output1")
grant("u1", TABLE_WRITE, "output1")
getUserAccess(`u1)

If server restarts, or a user deletes and recreates the stream table "trades", "output1" or the streaming engine "agg1", access control is not applied to these objects, and any user can access them.

login(`admin, `123456)
enableTableShareAndPersistence(streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]),`trades)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as  output1
agg1 = createTimeSeriesEngine(name="agg1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)

Add access control to these objects:

addAccessControl(`trades)
addAccessControl(output1)
addAccessControl(agg1)

Now only u1 and the creator admin can access these objects:

login(`u1, "111111")
insert into agg1 values(2018.10.08T01:01:01.785,`A,10) 
insert into output1 values(2018.10.08T01:01:01.785,`A,10) 
insert into trades values(2018.10.08T01:01:01.785,`A,10) 

Other users are denied:

login(`u2, "111111")
insert into agg1 values(2018.10.08T01:01:01.785,`A,10)=> No access to table [agg1]'
insert into output1 values(2018.10.08T01:01:01.785,`A,10) => No access to table [output1]'
insert into trades values(2018.10.08T01:01:01.785,`A,10) => No access to table [trades]'

5. Privilege Verification

5.1 Privileges Required by DDL/DML Operations

The following table introduces the privileges required by DDL/DML operations before and since version 1.30.21/2.00.9.

OperationRequired Privileges (Since version 1.30.21/2.00.9)Required Privileges (Before version 1.30.21/2.00.9)
database (create databases)DB_OWNER to create databases at global level or databases with specific prefixDB_MANAGE or DB_OWNER to create databases
database (load databases)NoneNone
dropDatabaseDB_MANAGE to delete any database;DB_OWNER to delete databases created by the userDB_MANAGE to delete any database;DB_OWNER to delete databases created by the user
createTableDB_MANAGE/DBOBJ_CREATE to create tables in any database;DB_OWNER to create tables in databases created by the userDBOBJ_CREATE to create tables in any database;A user who creates the database can create tables in that database
dropTableDB_MANAGE/DBOBJ_DELETE to delete tables in any database;DB_OWNER to delete tables in databases created by the userDB_OWNER to delete any table in databases created by the user;DB_MANAGE to delete any table created by the user;A user with DBOBJ_DELETE privilege on the database can delete any table in that database
createPartitionedTableDB_MANAGE/DBOBJ_CREATE to create tables in any database; DB_OWNER to create tables in databases created by the userDBOBJ_CREATE to create tables in any database;A user who creates the database can create tables in that database
addPartitions(addValuePartitions/addRangePartitions)DB_MANAGE to add partitions to any database;DB_OWNER to add partitions to databases created by the userDB_MANAGE to add partitions to any database;DB_OWNER to add partitions to databases created by the user
configure newValuePartitionPolicy=addThe parameter is automatically added once a new partition is written by a user with write privilege.The parameter is automatically added once a new partition is written by a user with write privilege.
dropPartition (deleteSchema=false)DB_MANAGE/DB_DELETE to delete partitions from any database;DB_OWNER to delete partitions from databases created by the user;TABLE_DELETE at global level to delete partitions from any databaseDB_OWNER to delete partitions from databases created by the user; DBOBJ_DELETE to delete partitions from any database
dropPartition(deleteSchema=true)DB_MANAGE to delete partitions from any database; DB_OWNER to delete partitions from databases created by the userDB_OWNER to delete partitions from databases created by the user;DBOBJ_DELETE to delete partitions from any database
renameTableDB_MANAGE/DBOBJ_CREATE to rename tables from any database;DB_OWNER to rename tables in databases created by the userDB_OWNER to rename tables in databases created by the user; DBOBJ_DELETE to rename tables from specific databases
addColumnDB_MANAGE/DBOBJ_CREATE to add columns to tables in any database;DB_OWNER to add columns to tables in databases created by the userDB_OWNER/DB_MANAGE to add columns to tables in databases created by the user;DBOBJ_CREATE to add columns to tables in specific databases
dropColumns!DB_MANAGE/DBOBJ_DELETE to delete columns from any database;DB_OWNER/DBOBJ_CREATE to delete columns from databases created by the userDB_MANAGE to delete columns from any database; DB_OWNER to delete columns from databases created by the user
rename!DB_MANAGE/DBOBJ_CREATE to rename columns in any database; DB_OWNER to rename columns from databases created by the userDB_MANAGE to rename columns in any database; DB_OWNER to rename columns from databases created by the user
replaceColumn!DB_MANAGE/DBOBJ_CREATE to replace columns of tables from any database;DB_OWNER to replace columns of tables from databases created by the userDB_MANAGE to replace columns of tables from any database; DB_OWNER to replace columns of tables from databases created by the user
setColumnCommentDB_MANAGE/DBOBJ_CREATE to set column comments for tables in any database;DB_OWNER to set column comments for tables in databases created by the userDB_OWNER/DB_MANAGE to set column comments for tables in databases created by the user;DBOBJ_CREATE to set column comments for tables in specific databases
truncateDBOBJ_CREATE/DB_OWNER/DB_MANAGE to truncate tables from databases created by the user;TABLE_WRITE/TABLE_DELETE/DB_WRITE/DB_DELETE to truncate tables from specific databases;VIEW_EXEC to execute truncate in function viewsDBOBJ_CREATE/DB_OWNER/DB_MANAGE to truncate tables from databases created by the user;TABLE_WRITE to truncate tables from specific databases;VIEW_EXEC to execute truncate in function views
append!DBOBJ_CREATE/DB_OWNER/DB_MANAGE to append data to tables created by the user;TABLE_WRITE/TABLE_INSERT/DB_WRITE/DB_INSERT to append data to specific tables;VIEW_EXEC to execute append! in function viewsDBOBJ_CREATE/DB_OWNER/DB_MANAGE to append data to tables created by the user;TABLE_WRITE to append data to specific tables;VIEW_EXEC to execute append! in function views
upsert/SQL updateDBOBJ_CREATE/DB_OWNER/DB_MANAGE to upsert or update tables created by the user;TABLE_WRITE/TABLE_UPDATE/DB_DELETE/DB_UPDATE to upsert or update specific tables;VIEW_EXEC to execute upsert/update in function viewsDBOBJ_CREATE/DB_OWNER/DB_MANAGE to upsert or update tables created by the user;TABLE_WRITE to upsert or update specific tables;VIEW_EXEC to execute upsert/update in function views
SQL deleteDBOBJ_CREATE/DB_OWNER/DB_MANAGE to delete tables created by the user; TABLE_WRITE/TABLE_DELETE/DB_WRITE/DB_DELETE to delete specific tables;VIEW_EXEC to execute SQL delete in function viewsDBOBJ_CREATE/DB_OWNER/DB_MANAGE to delete tables created by the user; TABLE_WRITE to delete specific tables;VIEW_EXEC to execute SQL delete in function views

When calling function addColumn, dropColumns!, renameColumns, replaceColumns, setColumnComment, rename!, or renameTable, if the applied object is created by other users, the current user must be granted TABLE_READ access to the table.

For example:

user 1 creates database "dfs://test" and grants user2 DBOBJ_CREATE. Then log in as user2 to create table pt1.

login("admin", "123456")
createUser("user1","123456",,true)
createUser("user2","123456",,true)
grant(`user1, DB_OWNER,"dfs://test*")

login("user1", "123456")
dbName = "dfs://test"
if(existsDatabase(dbName)){
        dropDatabase(dbName)
}
t = table(1..10 as id , rand(100, 10) as val)
db=database(dbName, VALUE, 1..10)
grant(`user2, DBOBJ_CREATE ,"dfs://test")
login("user2", "123456")
pt1=  db.createPartitionedTable(t, "pt1",`id).append!(t)

If user1 attempts to rename a column in pt1, error of no privilege is reported:

login("user1", "123456")
rename!(pt1,`val,`val1)
// Error: <NoPrivilege> Not granted to read table dfs://test/pt1'

user1 must be granted the TABLE_READ privilege on pt1:

login("admin", "123456")
grant(`user1, TABLE_READ,"dfs://test/pt1")

Then log in as user1 to re-execute the rename operation:

login("user1", "123456")
rename!(pt1,`val,`val1)
schema(loadTable(dbName,`pt1)).colDefs
------------------------------------------
name typeString typeInt extra comment
---- ---------- ------- ----- -------
id   INT        4                    
val1 INT        4                                                 

5.2 Visibility of Functions

FunctionRequired Privileges (Since version 1.30.21/2.00.9)Required Privileges (Before version 1.30.21/2.00.9)
getAllDBsExecuted by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node;Otherwise, return databases (to which the user has DB_OWNER privileges)return all DFS databases on the current node
getTablesReturn all tables in the specific databasereturn all tables in the specific database
getClusterDFSDatabasesExecuted by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node;Otherwise, return databases (to which the user has DB_OWNER privileges)return all DFS databases on the current node
getDFSDatabasesExecuted by administrators or users with DB_MANAGE privilege to return all DFS databases on the current node;Otherwise, return databases (to which the user has DB_OWNER privileges)return all DFS databases on the current node
getDFSTablesByDatabaseExecuted by administrators or users with DB_MANAGE privilege to return tables in all DFS databases on the current node;Otherwise, return tables in databases (to which the user has DB_OWNER privileges)return tables in all DFS databases on the current node

Since version 1.30.21/2.00.9, users with VIEW_EXEC privilege only cannot get expected results when executing the above functions.

6. Privilege Management

This chapter introduces the privilege management in DFS databases, shared in-memory tables, streaming data and tasks, function views, scheduled jobs, and memory management.

6.1 DFS Databases

For DFS databases, you can use function grant/deny/revoke to set privilege access for users or groups. The parameter accessType of these functions can take all the privilege types.

  • Global or table-level privileges: TABLE_READ, TABLE_WRITE, TABLE_INSERT, TABLE_UPDATE, TABLE_DELETE
  • Global or database-level privileges: DBOBJ_CREATE, DBOBJ_DELETE, DB_READ, DB_WRITE, DB_INSERT, DB_UPDATE, DB_DELETE, DB_MANAGE, DB_OWNER

Example 1. Grant user1 TABLE_READ at global level:

grant(`user1, TABLE_READ, "*")

Example 2. Grant user1 TABLE_READ of table pt in the database "dfs://valuedb":

grant(`user1, TABLE_READ,"dfs://valuedb/pt")

Example 3. Grant user1 DBOBJ_CREATE in the database "dfs://valuedb":

grant(`user1, DBOBJ_CREATE,"dfs://valuedb")

Example 4. Grant user1 the read and write privileges in a specific database (new feature in version 1.30.21/2.00.9):

login("admin","123456")
createUser("user1","123456")
grant("user1",DB_READ,"dfs://valuedb")
grant("user1",DB_WRITE,"dfs://valuedb")

Now user1 has all access privileges to database "dfs://valuedb".

Example 5. Grant user2 the privilege to create and manage databases with prefix "dfs://db0" only.

login("admin","123456")
createUser("user2","123456")
grant("user2", DB_OWNER, "dfs://db0*")

In this case, if user2 attempts to create a database with prefix "dfs://db1", the operation will be denied.

Example 6. Grant all members of a group the read privilege on a table, and only some members with write privilege.

login("admin","123456")
createUser("user1","123456")
createUser("user2","123456")
createUser("user3","123456")
createGroup("group1",["user1","user2","user3"])
grant("group1", TABLE_READ, "dfs://valuedb/pt")
createUser("user4","123456")
addGroupMember("user4","group1")
grant("user1", TABLE_WRITE, "dfs://valuedb/pt")
grant("user4", TABLE_WRITE, "dfs://valuedb/pt")

Now user1 and user4 are granted read and write privileges on table "dfs://valuedb/pt", and user2 and user3 have only the read privilege.

Example 7. grant or deny privileges on all objects (represented by *). Grant user1 the read privilege on all DFS tables:

grant("user1",TABLE_READ,"*")  

If the grant or deny operation applies to all objects, revoke can only be used to cancel the privilege settings from all objects. If only the privilege access on a specific object is revoked, the revoke operation does not take effect:

revoke("user1",TABLE_READ,"dfs://db1/t1")

Instead, use the following command to revoke granted privileges from all objects:

revoke("user1",TABLE_READ,"*")

Similarly, after using grant or deny on a group, only revoke can be used to cancel the privilege settings. Note the revoke command can only be applied to the group. If only the access of a member is revoked, the operation does not take effect.

Example 8. Grant DB_OWNER to user1. Log in as user1 to create table "dfs://dbMT/dt" and grant user2 the read privilege on the table. Now user2 now can access the table "dfs://dbMT/dt".

createUser("user1", "123456")
createUser(`user2, "123456")
grant("user1",DB_OWNER);
login("user1", "123456")
db = database("dfs://dbMT", VALUE, 1..10)
t=table(1..1000 as id, rand(100, 1000) as x)
dt = db.createTable(t, "dt").append!(t)
grant("user2", TABLE_READ, "dfs://dbMT/dt");

6.2 Shared In-Memory Tables

In DolphinDB, the shared in-memory tables are shared across all sessions, and the read and write permissions are managed in the access system. The access types for shared in-memory tables only support TABLE_READ and TABLE_WRITE.

Note:

  • Only the table creator or administrator is allowed to set other users' access privileges on that table.
  • If the table creator or administrator calls addAccessControl to restrict other users from accessing the shared table, other users can only access the table after being granted access privileges.
  • If the table creator or administrator has set access privileges on the table for other users with grant/deny/revoke, access control will be automatically added. Users without granted privileges cannot access the table.

Examples:

Create an administrator admin1. Log in as admin1 to create a shared in-memory table st1:

login(`admin, "123456");
createUser("admin1","123456",,true)
login(`admin1, "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as st1

All users have the read and write privileges on the table st1:

createUser("user1","123456",,false)
login("user1","123456")
insert into st1 values(1970.01.01T00:00:00.001,`sym,1)
select * from st1

Grant user1 the read privilege on table st1:

login(`admin1, "123456")
grant("user1", TABLE_READ, "st1") 

After the grant operation is performed, access control is automatically added. Now user1 is granted the read privilege but not write privilege:

login("user1","123456")
select * from st1
insert into st1 values(1970.01.01T00:00:00.001,`sym,1)
// Error: No access to table [st1]

user1's privilege can be revoked:

login(`admin1, "123456")
revoke("user1", TABLE_READ, "st1")
login("user1","123456")
select * from st1 
// Error: No access to shared table [st1]

Use addAccessControl to restrict access to a shared in-memory table to the table creator and administrators.

share table(1:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as st2
addAccessControl(st2)

6.3 Streaming Data

6.3.1 Streaming Engines

Users can call addAccessControl to restrict other users from accessing the streaming engines.

Examples:

Log in as user1 to create a streaming engine "engine1":

login(`admin, `123456)
createUser("user1", "123456");
createUser("user2", "123456");
login("user1", "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as trades
output1 = table(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT])
engine1 = createTimeSeriesEngine(name="engine1", windowSize=600, step=600, metrics=<[sum(volume)]>, dummyTable=trades, outputTable=output1, timeColumn=`time, useSystemTime=false, keyColumn=`sym, garbageSize=50, useWindowStartTime=false)

Add access control to engine1:

addAccessControl(engine1)

user2 fails to ingest data to the engine:

login("user2", "123456")
insert into engine1 values(2018.10.08T01:01:01.785,`A,10) 
// ERROR: No access to table [engine1]

Grant user2 the write privilege on the engine:

login(`admin, `123456) grant("user2", TABLE_WRITE, "engine1")

user2 is allowed to ingest data to the engine:

login("user2", "123456")
insert into engine1 values(2018.10.08T01:01:01.785+1..600,take(`A,600),1..600)

6.3.2 Subscription From Stream Tables to Stream Tables

Before subscribing to a stream table, a user must have the TABLE_READ privilege to read the publishing table. Both TABLE_READ and TABLE_WRITE privileges to the stream table where the subscribed data will be saved are required.

Example 1. user1 creates stream tables pubTable and output1, and calls addAccessControl.

login(`admin, `123456)
createUser(`user1, "123456");
login(`user1, "123456")
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as pubTable
share streamTable(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT]) as output1
addAccessControl(`pubTable)
addAccessControl(`output1)

Grant user2 TABLE_READ on table pubTable, and TABLE_READ and TABLE_WRITE on the table output1.

login(`admin, `123456)
grant("user2", TABLE_READ, "pubTable")
grant("user2", TABLE_WRITE, "output1")
grant("user2", TABLE_READ, "output1")
login(`user2, "123456")
subscribeTable(tableName="pubTable", actionName="engine1", offset=0, handler=append!{output1}, msgAsTable=true); 

Example 2. When the publisher and subscriber are not on the same node, the applied object must be specified in the "nodeAlias:tableName" format.

Create pubTable on a remote NODE2 and add access control:

login(`admin, `123456)
def pubTable(){
    share (streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]), `pubTable)
    addAccessControl(`pubTable)
}
rpc(`NODE2,pubTable)
share streamTable(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT]) as output1

A local subscriber must have the TABLE_READ access privilege to pubTable on NODE2:

createUser(`user1, "123456");
grant("user1", TABLE_READ, "NODE2:pubTable")
login(`user1, "123456")
subscribeTable(server=`NODE2,tableName="pubTable", actionName="engine1", offset=0, handler=append!{output1}, msgAsTable=true);

6.3.3 Subscription From Stream Tables to DFS Tables

Before subscribing to a stream table, a user must have the TABLE_READ privilege to read the publishing table. The TABLE_WRITE privilege to the DFS table where the subscribed data will be saved is also required.

Examples:

Log in as admin to create a stream table "pubTable" and a DFS table "dfs://valuedb/pt":

login(`admin, `123456)
dbName = "dfs://valuedb"
t = table(10000:0, `time`sym`sumVolume, [TIMESTAMP, SYMBOL, INT])
insert into t values(2018.10.08T01:01:01.785+1..600,take(`A,600),1..600) 
if(existsDatabase(dbName)){
    dropDatabase(dbName)
}
db=database(dbName, VALUE, 1..10)
pt=  db.createPartitionedTable(t, "pt", "sumVolume").append!(t)
share streamTable(1000:0, `time`sym`volume, [TIMESTAMP, SYMBOL, INT]) as pubTable

Grant user2 the TABLE_WRITE privilege on the DFS table:

createUser(`user2, "123456");
grant("user2", TABLE_WRITE,"dfs://valuedb/pt")

user2 can subscribe to the table:

login(`user2, "123456")
def savepubTableToDFS(mutable dfspubTable, msg): dfspubTable.append!(msg)
subscribeTable(tableName="pubTable", actionName="agg1", offset=0, handler=savepubTableToDFS{pt}, msgAsTable=true);
login(`admin, `123456)
select * from loadTable(dbName,`pt)

6.4 Function Views

Function view provides a flexible way to manage user access on databases and tables.

  • Only administrators can create or delete function views. Granted users can get the result of a function view even if they don't have the privileges to access the involved data.
  • If a function view has the same name as a local function, the system will parse it as a local function.
  • Some DDL and DML operations can be performed with only VIEW_EXEC privilege when included in function views.
  • VIEW_EXEC can be applied to global (*), module namespace (<namespace>::*), or function view level. If functions of a module are added as views using addFunctionName(moduleName), the fully-qualified namespace can be used for specifying a module-level access control.

Example 1. Define a function view:

def countTradeAll(){  
    return exec count(*) from loadTable("dfs://TAQ","Trades")  
}
addFunctionView(countTradeAll)  
grant("user1",VIEW_EXEC,"countTradeAll")  

Log in as user1 to execute the function view:

countTradeAll()

In this example, though user1 does not have the privilege to access table "dfs://TAQ/Trades", user1 can still get the number of rows in the table with countTradeAll().

Example 2. Function views can take parameters. This example creates a function view to get the trades data of a specified stock on a specified date:

def getTrades(s, d){
	return select * from loadTable("dfs://TAQ","Trades") where sym=s, date=d
}
addFunctionView(getTrades)
grant("user1",VIEW_EXEC,"getTrades")  

Similarly, though user1 does not have the privilege to access table "dfs://TAQ/Trades", user1 can execute getTrades with specified stock symbol and date:

getTrades("IBM", 2018.07.09)

Example 3.

Declare a module myModule and define functions f1 and f2 within the module.

module dir1::myModule

def f1(){
    print "dir1::myModule::f1"
}
def f2(){
    print "dir1::myModule::f2"
}

After referencing the module with use, all functions within the module are added as function views.

use dir1::myModule
addFunctionView(moduleName="dir1::myModule")
getFunctionViews()
namebody
dir1::myModule::f1def f1(){ print("dir1::myModule::f1")}
dir1::myModule::f2def f2(){ print("dir1::myModule::f2")}

Grant VIEW_EXEC privilege on all function views within dir1::myModule to user1:

grant(userId="user1", accessType=VIEW_EXEC, objs="dir1::myModule::*")

Deny VIEW_EXEC privilege on all function views within dir1::myModule to user2:

deny(userId="user2", accessType=VIEW_EXEC, objs="dir1::myModule::*")

Note that the namespace must be specified to the module level. An error will be raised if the access control is set on dir1.

grant(userId="user1", accessType=VIEW_EXEC, objs="dir1::*")
// Error:The Module [dir1::*] does not exist.

6.5 Scheduled Jobs

Scheduled jobs are automated tasks to execute on a specific time or recurring basis.

  • A user who creates a scheduled job does not require the access privileges on the involved objects. However, if the user does not have privileges to access the objects when the scheduled job is set to be executed, the scheduled job will not be executed.
  • With command deleteScheduledJob, administrators can delete all scheduled jobs, while users can only delete scheduled jobs created by themselves.
login("user1","123456")  
def readTable(){  
	read_t1=loadTable("dfs://db1","t1")  
	return exec count(*) from read_t1  
}  
scheduleJob("readTableJob","read DFS table",readTable,minute(now()),date(now()),date(now())+1,'D');

In the example above, job readTable can be scheduled even if user1 does not have privilege to read table "dfs://db1/t1". When the job is set to be executed, however, if "user1" still does not have privilege to read table "dfs://db1/t1", the job will not be executed.

6.6 Execution of Scripts

The scope of SCRIPT_EXEC and TEST_EXEC is global.

Executing function run, or executing scripts on clients requires SCRIPT_EXEC privilege:

login(`admin, `123456)
createUser(`user1, "123456");
grant(`user1,SCRIPT_EXEC)
login(`user1, "123456")
run("test.txt")

Executing function test, or executing unit tests on clients requires SCRIPT_EXEC privilege:

login(`admin, `123456)
createUser(`user1, "123456");
grant(`user1,TEST_EXEC)
login(`user1, "123456")
test("test.txt")

6.7 Memory Usage

You can call grant and specify accessType = QUERY_RESULT_MEM_LIMIT to set the memory limit for the result of each query for a user, or accessType = TASK_GROUP_MEM_LIMIT to set the memory limit of a task group sent from the current node. It has similar functions with functions setMemLimitOfQueryResult and setMemLimitOfTaskGroupResult, but grant allows you to set the limit for a specific user (currently a group is not supported).

Set the memory limit of a query at 4 GB for user1:

grant("user1", QUERY_RESULT_MEM_LIMIT, 4)

Set the memory limit of a task group at 4 GB for user1:

grant("user1", TASK_GROUP_MEM_LIMIT, 4)

7. Secure Communication with HTTPS

DolphinDB supports HTTPS for secure communication over the web.

7.1 Enable HTTPS

There are 2 ways to enable HTTPS:

  • Add enableHTTPS=true in the configuration file of the cluster controller (controller.cfg);
  • Add -enableHTTPS true in the command line when starting the cluster controller.
./dolphindb -enableHTTPS true -home master -publicName www.psui.com -mode controller -localSite 192.168.1.30:8500:rh8500 -logFile ./log/master.log

7.2 HTTPS Certificate

You need to install server authentication certificate at each server in DolphinDB for secure connections. There should be a certificate on the controller and each of the agent nodes. Data nodes use the certificate located at the agent node on the same server.

7.2.1 Certificate Authority (CA) Certificate

Get a certificate from a certificate authority, rename it as server.crt, and copy to the folder keys under the home directory of the controller. If the folder keys does not exist, we need to create it. The certificate does not need to be installed as it is from a certificate authority. This is recommended in most use cases.

7.2.2 Self-Signed Authentication Certificate

Sometimes we may want to install a self-signed authentication certificate. For example, in a small isolated cluster or in a research (non-production) setting. Please take the following steps:

(1) Set the configuration parameter publicName as the domain name of the computer.

Add publicName=www.ABCD.com to the configuration file (controller.cfg) for the controller in a cluster, or add -publicName www.ABCD.com when starting the controller in the command line.

(2) Check if the authentication certificate and private key have been generated.

Start the controller node and check if the certificate file server.crt and the private key for the server serverPrivate.key exist in the folder keys under the home directory.

(3) Install the self-signed certificate to the certificate authority of the web browser.

In Google Chrome, choose Settings -> Advanced -> Manage certificates -> AUTHORITIES-> Import to install the self-signed certificate server.crt. In other web browsers, there might be slight differences in this process.

Now enter https://www.abcd.com:8500 in the web browser to connect to DolphinDB cluster manager, where 8500 is the port number of the controller. If you see a green lock sign and "Secure" in the address bar, the authentication certificate has been successfully installed.

8. SSO (Single Sign-On)

On the web-based cluster manager, we can click any data node to open a notebook on this node. The data node may be located on a different physical server from the controller node. DolphinDB provides SSO (Single Sign-On) so that users don't need to log in the system again when visiting different servers in a cluster.

DolphinDB provides 2 API functions for SSO:

  • getAuthenticatedUserTicket(): issue the current user's encrypted ticket
  • authenticateByTicket(ticket): use the ticket generated by getAuthenticatedUserTicket() to log in the system.

DolphinDB developers can use these API functions to develop more functionalities.