Cluster Mode

DolphinDB clusters can be deployed on a single or multiple machines. To improve performance, high availability can be enabled for a cluster.

A DolphinDB cluster consists of 4 types of nodes: controller, agent, data node, and compute node. DolphinDB configuration files are saved under the config directory.

controller.cfg: Configuration file for controllers.
agent.cfg: Configuration file for agent nodes.
cluster.cfg: Configuration file for data nodes. All configuration parameters in the reference can be used for data nodes in the cluster mode by specifying them in cluster.cfg.
cluster.nodes: Network information and node mode of each node in the cluster.

Note: All configuration files require that the first line is not a blank line.

Regular Cluster

For details about the deployment of regular clusters, refer to Single-Machine Cluster Deployment and Multi-Machine Cluster Deployment.

Note: In multi-machinemode, the machine where the controller is located must contain controller.cfg, cluster.cfg, and cluster.nodes. The machine where the agent node is located must contain agent.cfg.

There are 4 methods to specify configuration parameters in cluster.cfg:

Method 1: Specify with node alias, which is defined in cluster.nodes.

nodeA.volumes = /DFSRoot/nodeA
nodeB.volumes = /DFSRoot/nodeB

Method 2: Specify with wildcard characters "%" (representing 0, 1, or multiple characters) and "?" (representing a single character).

%8821.volumes = /DFSRoot/data8821
%8822.volumes = /DFSRoot/data8822
DFSnode?.maxMemSize=16

Method 3: Specify with the macro variable <ALIAS>, which automatically replaces specific node alias. The following script uses <ALIAS> to specify the volume path to a cluster containing two data nodes — nodeA and nodeB.

volumes = /DFSRoot/<ALIAS>

This is equivalent to:

nodeA.volumes = /DFSRoot/nodeA
nodeB.volumes = /DFSRoot/nodeB

Method 4: Specify the same value for a parameter across all nodes in the cluster.

// for Windows
maxConnections=64
// for Linux
maxConnections=512
maxMemSize=12

Note:

Method 1/2/3 can be used to specify different values for a parameter across nodes.
Method 4 can be used to specify the same value for a parameter across nodes.
The macro variable <ALIAS> cannot be used when the node is specified through Method 1 (nodeA.volumes = /DFSRoot/<ALIAS>); otherwise, the controller will be unable to start.

The following startup and shutdown scripts are provided in server/clusterDemo.


Script Type	Windows	Linux
Startup script for controller	startController.bat	startController.sh
Startup script for agent	startAgent.bat	startAgent.sh
Shutdown script	stopAll.bat	stopAll.sh

These scripts only apply to the default configuration. Users can modify them as needed.

Note: In cluster mode, the shutdown script will shut down the entire cluster. It is recommended to restart an individual node in DolphinDB Web Interface. For details, refer to Single-Machine Cluster Deployment.

High-Availability Cluster

DolphinDB HA cluster offers high availability for data (multi-replica mechanism), metadata (Raft group with multiple controllers), and API clients (automatic reconnection and switching).

Configurations for HA clusters are uniformly managed by the Raft group. The files cluster.cfg and cluster.nodes are the configuration reference only for the initial cluster startup. Once the cluster is launched, changes to these files will take no effect. To reconfigure the cluster, users must modify configuration parameters through Web Interface or APIs, which automatically synchronize the modifications to all configuration files in the cluster.

Related Information

High-availability Cluster Deployment