Graceful Shutdown

Before version 1.30.17/20.00.5, DolphinDB did not support a graceful shutdown. If the DolphinDB process is terminated by pkill -9 or kill -9, the following problems may occur when restarting the DolphinDB service:

  1. Some completed transactions may not be flushed to disk. When the data node restarts, redo log needs to be replayed to flush these transactions to disk. If there are multiple redo logs that need to be replayed, it might take a long time to restart the data node.

  2. If there is a problem in replaying the redo log on the data node, the data node will not start up.

Starting from version 1.30.17/20.00.5, DolphinDB supports graceful shutdown, which can avoid the above problems by flushing transactions to disk before the system shuts down. This section introduces the mechanism of graceful shutdown under linux system and the process of correctly disconnect from the DolphinDB service.

Graceful Shutdown

A SIGTERM signal sent by the data node through the kill -15 (kill -TERM) command will be captured by DolphinDB and processed as follows:

  • The data node stops sending heartbeats to the controller so that the controller knows this node is down. At this point, newly initiated tasks in the cluster such as read, write or recovery tasks will not be allocated to this node.

  • If there are ongoing transactions on the data node, wait for them to complete and flush to disk.

  • Transactions in the cache engine and the redo log are garbage-collected by the data node.

Graceful Shutdown in Standalone Mode

  • Front-end interactive mode

    Type "quit" in the console to exit

  • In back-end interactive mode, kill command is required:

    kill -15 process ID or pkill -15 dolphindb

Graceful Shutdown in Cluster Mode

  1. Shut down a data node: The following options are available:

    • Use the stop button in the Web-based Cluster Manager

    • Type the stopDataNode command in the Web-based Cluster Manager

    • Use the stopDataNode command on the GUI or other API clients

  2. Shut down the agent node and controller:

    The command to graceful shut down the agent node and controller is the same as for standalone mode. Alternatively, you can run the stopAll.bat script to directly kill all processes.