input

The input plugin is developed based on libarchive and is used to decompress in-memory data or file content and return the decompressed text. The plugin can automatically recognize compression formats (currently supporting zstd and gzip) and decompress the data.

Installation

Version Requirements

DolphinDB Server: 2.00.14/3.00.2 and later, deployed on a Linux x86-64 system.

In the DolphinDB client, use the listRemotePlugins function to view plugins available for installation.
```
login("admin", "123456")
listRemotePlugins()
```
Use the installPlugin function to install the plugin.
```
installPlugin("input")
```
Use the loadPlugin function to load the plugin.
```
loadPlugin("input")
```

Method References

readMemory

Syntax

input::readMemory(data)

Details

Reads binary data (BLOB) from memory, automatically detects its compression format (supports zstd and gzip), and decompresses it.

Parameters

data: A scalar of type BLOB that specifies the data to be decompressed.

Returns

The decompressed text content, which is a scalar of type STRING.

readFile

Syntax

input::readFile(path)

Details

Reads a file from the specified path, automatically detects its compression format (supports zstd and gzip), and decompresses it.

Parameters

path: A scalar of type STRING that specifies the path of the file to be decompressed.

Returns

The decompressed text content, which is a scalar of type STRING.

blobToLong

Syntax

input::blobToLong(data)

Details

Converts binary data of fewer than 8 bytes into a LONG value.

Parameters

data: A scalar of type BLOB that specifies the data to be converted.

Returns

A scalar of type LONG.

Examples

Example 1: Read a Record Table From a Binary File

Assume that an external system periodically generates a binary file in which each record sequentially contains trade time, symbol, price, and quantity, all as fixed-length fields. The file can be read in DolphinDB as follows:

// Load the plugin
loadPlugin("input")

// Read and decompress the file from the specified path
path = "./trade.bin"
content = readFile(path)

// Convert to a table
table = parseJsonTable(content)

In the preceding example, trade.bin is a compressed JSON file. readFile reads the entire file and decompresses it into data of type STRING. The data can then be transformed using DolphinDB string parsing and processing functions.

Example 2: Read Data From Shared Memory

In some high-frequency scenarios, an external process writes the latest data to a shared memory region. readMemory reads data directly from the memory address, reducing the latency caused by disk I/O and data copying.

// Load the plugin
loadPlugin("input")

// Receive data from the message queue
// content = ...
content = readMemory(content)

// Convert to a table
table = parseJsonTable(content)

In the preceding example, the content is compressed data received through the message queue. readMemory decompresses it into data of type STRING. The data can then be transformed using DolphinDB string parsing and processing functions.