S01009

Error Code

S01009

Error Message

<WARNING> Liveness checking: data node <nodeDesc> has gone offline. RefId:S01009

Probable Causes

This error appears in the controller log, indicating that the corresponding node has not sent a heartbeat to the controller or the controller has not processed the heartbeat within the expected time interval. In this case, the controller will mark the node as offline. Possible offline causes include node crash and network issues.

When a node is offline, the controller may remove the replica information of that data node from meta data. Insufficient replicas or chunk recovery could be caused in this case.

Solutions

  • Check the node logs to confirm if it has crashed.
  • If the node has not crashed, check the network monitoring and the controller logs to see if there are records of the node's status changing from offline to online immediately after being marked as offline.