enableResourceTracking

Syntax

enableResourceTracking()

Arguments

None

Details

Use this function to enable resource tracking at runtime. This function can only be called by the administrator on a data node, and it only takes effect when resourceSamplingInterval is set to a positive integer.

Once resource tracking is enabled, the data nodes will track queries at intervals defined by resourceSamplingInterval.

<HomeDir>/resource/hardware.log records the CPU and memory usage in CSV format, including the following information:

  • timestamp: the sampling timestamp of NANOTIMESTAMP type.

  • userId: the login user ID.

  • cpu: the number of CPU threads used by the user.

  • memory: the memory used (in bytes) by the user.

<HomeDir>/resource/access.log records DFS SQL queries in CSV format, including the following information:

  • timestamp: the sampling timestamp of NANOTIMESTAMP type. If type is "sql", it indicates the start time of the query execution; If type is "rowCount" or "memUsage", it indicates the time when data is read.

  • rootQueryId: the root ID of a DFS query and all its sub-queries split by partition.

  • userId: the login user ID.

  • database: the queried database.

  • table: the queried table.

  • type: the message type, which can be sql, rowCount or memUsage.

  • value: the message value corresponding to each type.

    • sql: the execution count (always 1).

    • rowCount: the number of rows read.

    • memUsage: the data volume read (in bytes).

Note that the values for rowCount and memUsage are recorded based on each access. For example, after a dimension table is loaded into memory, the number of rows read and data volume read are logged each time the table is accessed, rather than the amount of data initially loaded into memory.

  • script: the SQL query script when type is sql, otherwise an empty string is logged.

Note that for DFS SQL queries, currently only the SELECT statement is supported, and tables in nested table joins that are not compatible with ANSI SQL-92 are not tracked. For example, tables t1 and t2 in the query ej(ej(t1, t2, `id), t3, `id) are not sampled.

To prevent excessive growth of the query logs, log rotation is applied once the log files reach the size threshold. The generated file names contain timestamp prefixes. Logs are automatically removed after the retention period configured with resourceSamplingLogRetentionTime regardless of whether tracking is enabled.

Related function: disableResourceTracking