createIPCInMemoryTable

createIPCInMemoryTable(size, tableName, columnNames, columnTypes)

size is an integer indicating the number of records that can be cached. The specified size must be greater than 1,000,000.

tableName is a string indicating the name of inter-process communication (IPC) in-memory table to be created.

columnNames is a STRING vector of column names.

columnTypes is a vector indicating data types of columns specified by columnNames.

Create a handle to inter-process communication (IPC) in-memory table. IPC in-memory tables are often used as output tables for streaming. In scenarios with extremely high latency requirements, user processes can directly access shared memory to obtain data through APIs, which greatly reduces the latency of network transmissions such as TCP.

An IPC in-memory table communicates between processes through shared memory (managed by the operating system). The shared table only supports sharing within the same physical server.

Read/write is supported by the IPC in-memory table, whereas its schema cannot be changed. Writing to an IPC in-memory table is done in the same way as to a regular in-memory table. If the amount of data inserted at one time exceedes shared memory, an exception will be thrown.

• Read/write mechanism: The process that writes to the shared memory is regarded as the producer, and the process that reads data from the shared memory as the consumer. Data written in the same batch is consumed (read) as a whole, and the batches are consumed in the same order as they are written. For example, if 100 records are written for the first time, and 200 records for the second time, the 100 records will be read first, 200 records of the second batch will be read next, and so on. The reading process will be blocked if all data in the shared table has been consumed, and will resume when new consumable data arrives.

The DolphinDB system allows processes to concurrently write to the memory, or to read and write at the same time. Note that there can only be one reading process at a time. As explained earlier, reading to the IPC in-memory table is an one-off operation. Concurrent reads will obtain the data written in different batches.

Create an IPC in-memory table that serves as the output table for subscription.

share streamTable(10000:0,`timestamp`temperature, [TIMESTAMP,DOUBLE]) as pubTable
ipc_t = createIPCInMemoryTable(1000000, "ipc_table", `timestamp`temperature, [TIMESTAMP, DOUBLE])
def shm_append(mutable table, msg) {
   table.append!(msg)
}
subscribeTable(tableName="pubTable", actionName="act3", offset=0, handler=shm_append{ipc_t}, msgAsTable=true)
// data input

n = 200
timestamp = 2022.01.01T09:00:00.000 + 1..n
temp = 30 + rand(5.0,n)

tableInsert(pubTable,timestamp,temp)

Related functions: dropIPCInMemoryTable, loadIPCInMemoryTable