3.00.6

Changes Made to Match Industry Practices

  • Support subtype expressions for the ANY type and columnar tuple. The outputs of typestr, type, schema, and related functions have changed:

    • Use ANY[<BasicType>] to represent subtypes of ANY, such as ANY[INT];

    • The return value of type is represented by adding 128 to the type value of the basic type.

      a = array(ANY[DECIMAL32(2)], 0, 10)
isColumnarTuple(a)
typestr(a)
// Previous versions return 'ANY VECTOR'
// New version returns 'ANY[DECIMAL32(2)] VECTOR'

type(a)
// Previous versions return int(25)
// New version returns int(165)

b = table(10:0, `a`b`c, [INT, ANY[DECIMAL32(2)], DECIMAL64(4)])
schema(b).colDefs
/* Previous versions return
name typeString   typeInt extra comment
---- ------------ ------- ----- -------
a    INT          4
b    ANY          25
c    DECIMAL64(4) 38      4
*/
/* New version returns
name typeString        typeInt extra comment
---- ----------------- ------- ----- -------
a    INT               4
b    ANY[DECIMAL32(2)] 165     2
c    DECIMAL64(4)      38      4
*/

  • INT columns in in-memory tables no longer support writing STRING data:

    • Previous versions attempt to extract digits from the string and write the converted value;

    • The new version reports an error.

      share streamTable(10000:0, `time`sym`price`qty`id, [TIMESTAMP, SYMBOL, INT, INT, DOUBLE]) as trades
n = 500
timev = temporalAdd(2012.01.01T11:24:56.245, take([1, 3, 5, 7], n).sort(), "s")
symv = take(`AA`BB`CC, n)
pricev = rand("A" + string(1..10), n)
qtyv = rand(0..50 join int(), n)
id = rand(1..100 join int(), n) * 0.01
insert into trades values(timev, symv, pricev, qtyv, id)

// Previous versions can write the data by removing the letters in pricev and converting the remaining part to numbers.
// New version reports an error: Failed to append data to column 'price' with error: Failed to convert string to int: A2

  • Deprecated legacy changelog-related interfaces, including changelogStreamTable, getStreamTableChangelog, peekAppend, StreamGraph::changelogSource, and DStream::changelogSink.

  • The return format of getStatelessMetrics has changed. The outputMetricKey and triggerOn columns in the returned table now retain double quotation marks:

    • Previous versions return values such as b:value1 and a:f1;

    • The new version returns values such as "b" : "value1" and "a" : "f1".

  • Date fields of INSTRUMENT and MKTDATA types, such as referenceDate, settlement, and termDates, now only allow the DATE type:

    • Previous versions allow STRING or DATE values;

    • The new version no longer supports STRING values.

  • When the PKEY primary-key engine creates a database, only chunkGranularity = "TABLE" is supported; chunkGranularity = "DATABASE" is no longer supported.

  • The JIT distribution and support scope have changed:

    • JIT is now dynamically linked through libjit.so. The Linux ABI=1 version enables JIT by default; use isJIT() to confirm whether it has been loaded successfully.

    • The standalone JIT installation package has been removed, and JIT is no longer compatible with Windows.

    • JIT usages supported in previous versions but not currently supported in the new version include: recursive functions, RSE streaming engine-related functionality, expressions in the form table(xxx as id), symbol vector, SQL, multi-statement assignment, accessing values of temporary variables such as a()[2], metaprogramming, module, class vector, int128-based types (including INT128, IPADDR, UUID, COMPLEX, and POINT), changing a variable's type after its type has been determined, higher-order function symbols, and some dictionary key/value types.

System Impacts Caused by Bug Fixes

  • The behavior of the in / not in operators and the in function has changed when comparing different temporal types in version 3.00.6.4:

    • Previous versions: Temporal units were not converted. Values were compared directly based on their underlying integer representations. For example, when applying in between a DATETIME value and a TIMESTAMP vector, the corresponding time could not be matched because of the difference in precision. When incompatible temporal types, such as DATE and MONTH, were compared, the result was false.

    • New version: The behavior is consistent with equality comparison (=).

      • Mutually convertible temporal types, such as DATE and DATETIME, or TIMESTAMP and NANOTIMESTAMP: values are first converted to the same temporal unit before comparison, so identical times can be matched correctly. If the higher-precision time contains additional finer-grained components, it is correctly treated as not matching.

      • Temporal type combinations that do not support implicit cross-type conversion, such as MONTH and DATE / DATETIME / TIMESTAMP, or date types and time-of-day types such as TIME / MINUTE: an Incompatible type exception is thrown directly.

    // Example 1: Comparing non-convertible temporal types (MONTH and DATE)
x = 2012.12M 2011.05M 2013.06M
in(2013.06.13, x)
// Previous versions returned: false
// New version reports an error: Incompatible type. Expected: DATE, Actual: MONTH
// Example 2: Comparing date/time types with different precisions (DATETIME and NANOTIMESTAMP)
dt = 2025.01.01T00:15:00
ts = [2025.01.01T00:15:00.000000000, 2025.01.01T00:30:00.000000000]
dt in ts
// Previous versions returned: false
// New version returns: true

t = table(
    1 2 3 as id,
    datetime(2025.01.01T00:15:00 2025.01.01T00:30:00 2025.01.01T00:45:00) as data_time
)
select * from t where data_time in (2025.01.01T00:15:00.000000000, 2025.01.01T00:30:00.000000000)

// Previous versions: Returns 0 rows because underlying integer values (seconds vs. nanoseconds) are compared directly without unit conversion.
// New version: Converts values to the same time unit before comparison, correctly returning 2 rows.

  • The return form of complex when the input is a matrix has changed:

    • Previous versions return a vector;

    • The new version returns a matrix.

      x = matrix([0.75, 0.125, 0.0, 0.125],
           [-0.25, 0.125, 0.0, 0.125],
           [-0.0, 0.0517767, -0.75, 0.3017767],
           [-0.0, -0.3017767, 0.75, -0.0517767])
typestr(complex(x, 0))

// Previous versions return 'FAST COMPLEX VECTOR'
// New version returns 'FAST COMPLEX MATRIX'

  • When loadText converts strings that do not match numeric formats to numeric types according to a specified schema, the result has changed:

    • Previous versions extract and convert the numeric prefix of the string;

    • The new version returns null values.

      t = table(["2013.06.13  13:30:10.008",
           "2012.06.13 13:30:10",
           "2012.06.13 13:30:10.008",
           "13:30:10.008007006",
           "2012.06.13 13:30:10.008007006"] as str)
saveText(t, "compat_numeric.csv")
schema = table(`str as name, `INT as type)
t1 = loadText("compat_numeric.csv", ",", schema)
print(t1)

/* Previous versions return
str
----
2013
2012
2012
13
2012
*/
// New version returns only null values

  • In 3.00.6, the build environment of the OpenBLAS library has changed, so the fallback architecture may differ from earlier versions. If the OPENBLAS_CORETYPE environment variable is not set, OpenBLAS selects an instruction set based on the machine architecture. As a result, functions that rely on OpenBLAS may produce numerical differences across machines or compared with earlier versions, such as differences in precision, eigenvector order, or sign. Affected calculations include pca, varma, vectorAR, derivative, arima, bvls, olsEx, piecewiseLinFit, and schur. To keep results consistent with the previous environment, set OPENBLAS_CORETYPE=Haswell before startup.

Upgrade Notes

  • After upgrading to version 3.00.6.3, the following downgrade restrictions apply if any access tokens have been created using createAccessToken:

    • If an access token has ever been created for the admin user, downgrading to a version earlier than 3.00.6.3 is not supported.
    • If access tokens have been created only for users other than admin, you must delete all users who have ever had an access token before downgrading to a version earlier than 3.00.6.3.

  • When upgrading from 2.00.16 or 2.00.17 to 3.00.6, if the earlier version enabled enhancedSecurityVerification and executed lockUser, unLockUser, createUser, resetPwd, changePwd, or setIPConnectionLimit, ACL log compatibility must be handled first: configure useOldAclLog=1 to start the new version, wait until all controller nodes complete the automatic checkpoint (the completion log is Checkpoint completed, ACL fixed.), then change useOldAclLog to 0 or remove the configuration and restart all nodes in the cluster.

  • For backup files generated by backupSettings in 2.00.16 or 2.00.17, when restoring them in the new version with restoreSettings, specify the Server version that generated the backup file through serverVersion, for example, serverVersion="200.16" or serverVersion="200.17".

  • If streaming high availability was enabled in a 2.00.18 or 3.00.5 environment, rollback to an earlier version is not supported after upgrading to 3.00.6 because the serialization format for streaming high availability has changed.

  • After upgrading to 3.00.6, if users are created, deleted, or modified, or if aclCheckPoint or takeMasterCheckpoint is triggered, ACL logs or checkpoint data that cannot be recognized by earlier versions may be written, and rollback to an earlier version is not supported.