Data Types

Note:

• The following data types are not supported in table columns: VOID, FUNCTIONDEF, HANDLE, CODE, DATASOURCE, RESOURCE, COMPRESSED, DURATION.

• DolphinDB uses STRING, BLOB and SYMBOL data types to represent strings.

  • SYMBOL is a special STRING type. When a column is of SYMBOL type, the number of unique values must be less than 2,097,152 (2^21), otherwise the error "One symbase's size can't exceed 2097152" will be reported.

  • Version 1.30.23/2.00.11 added size constraints for STRING, BLOB, and SYMBOL data written to distributed databases:

    • Data of STRING type must be smaller than 64 KB, otherwise it will be truncated to 65,535 bytes (i.e., 64 KB - 1 byte);

    • Data of BLOB type must be smaller than 64 MB, otherwise it will be truncated to 67,108,863 bytes (i.e., 64 MB - 1 byte);

    • Data of SYMBOL type must be smaller than 255 bytes, otherwise the system will throw an exception.

  Note: Data exceeding these limits that has already been stored in the databases can still be accessed.

• SYMBOL is a special STRING type. When a column is of SYMBOL type, the number of unique values must be less than 2097152 (2^21), otherwise the error "One symbase's size can't exceed 2097152" will be reported.

• ANY DICTIONARY is the data type in DolphinDB for JSON.

• COMPRESSED can only be generated with function compress .

• Values of BLOB type do not participate in calculations.

• The DURATION type can be generated with function duration or by combining an integer with a unit of time (case sensitive): y, M, w, d, B, H, m, s, ms, us, ns. The range of a DURATION value is -2 ³¹ +1~2 ³¹ -1. If a data type overflow occurs, the data is treated as NULL value.

• DolphinDB uses IEEE 754 standard for the data types DOUBLE and FLOAT. If a data type overflow occurs, the data is treated as NULL value.

Type check

typestr 3l;
// output
LONG

type 3l;
// output
5

x=3;
if(type(x) == INT){y=10};
y;

// output
10

Data Range

x=-128c;
x;
// output
00c
typestr x;
// output
CHAR

Data Type Symbols

A data type symbol is used for declaring a data type of a constant. In the example below, without specifying a data type symbol, number 3 is stored in memory by default as an integer. If you would like to save it as a floating number, it should be declared as 3f(float) or 3F(double).

typestr 3;
// output
INT

typestr 3f;
// output
FLOAT

typestr 3F;
// output
DOUBLE

typestr 3l;
// output
LONG

typestr 3h;
// output
SHORT

typestr 3c;
// output
CHAR

typestr 3b;
// output
BOOL

Symbol and String

In some circumstances it might be optimal to save strings as SYMBOL types in DolphinDB. SYMBOL types are stored as integers in DolphinDB to allow more efficient sorting and comparison. Therefore, SYMBOL types could potentially improve operating performance and save storage space. On the other hand, mapping strings to integers (hashing) takes time and the hash table consumes memory.

The following rules could help you decide whether to use SYMBOL types or not:

Symbol Vector Creation

syms=array(SYMBOL, 0, 100);
// create an empty symbol array;

typestr syms;
// output
FAST SYMBOL VECTOR
syms.append!(`IBM`C`MS);
syms;
// output
["IBM","C","MS"]

syms=`IBM`C`MS;
typestr syms;
// output
STRING VECTOR

// converting to a symbol vector;
sym=syms$SYMBOL;

typestr sym;
// output
FAST SYMBOL VECTOR
typestr syms;
// output
STRING VECTOR

syms=`IBM`C`MS;
symRand=rand(syms, 10);
//generate a random SYMBOL vector

symRand;
// output
["IBM","IBM","IBM","MS","C","C","MS","IBM","C","MS"]
typestr symRand;
// output
FAST SYMBOL VECTOR

Integer Overflow

An overflow occurs when the result of an operation falls outside the valid range of the data type being used. In DolphinDB, an integer overflow returns null.

In the following example, the variable x is of INT type, and it is assigned the maximum INT value, i.e., 2 ³¹ - 1. The result of x+1 exceeds the upper limit of the INT type, therefore it returns NULL.

x=(pow(2,31)-1)$INT;
x+1;
// output
00i