2.00.12

Added parameter minPeriods to functions mcount and mrank for specifying the minimum number of observations in a window. (2.00.12.6)

Added the new function createStreamBroadcastingEngine for creating an engine that broadcasts streaming data. (2.00.12.4)

Added function fromStdJson for parsing standard JSON objects into DolphinDB variables. (2.00.12.2)

Added function parseInteger (alias parseInt) for converting strings into integers with specified numeral system. (2.00.12.2)

Added functions backupSettings and restoreSettings to back up and restore settings on users, permissions and function views for database migration. (2.00.12.2)

Added functions VaR and CVaR to calculate value at risk and conditional value at risk. (2.00.12.1)

Added functions nss and nssPredict to fit the yield curve and predict the bond yield. (2.00.12.1)

Added function matrixRank to calculate the rank of a matrix.

Added function bondAccrInt to calculate the accrued interest of securities.

Added function pinverse to find the pseudoinverse of a matrix.

Added function geoWithin to determine whether a point lies inside or outside a polygon.

Added functions getDfsRecoveryConcurrency and resetDfsRecoveryConcurrency to get and set the maximum concurrency of data recovery tasks on the controller.

Added functions getDfsRebalanceConcurrency and resetDfsRebalanceConcurrency to get and set the maximum concurrency of data rebalancing tasks on the controller.

Added function getTSDBTableIndexCacheStatus to check the memory usage of the level file indices of TSDB tables loaded on the current node.

Added function rowGroupby for aggregation grouped by rows.

Added function wslr for weighted single linear regression.

Added function digitize to bucket data based on specific bin values.

Added function lowerBound to find the first element that is greater than or equal to a specific value and return its index in an increasing object.

Added function fmin to minimize a function using a Nelder-Mead simplex algorithm.

Added functions pdfF, pdfChiSquare, and pdfNormal to calculate probability density of F, chi-square, and normal distributions.

Added function createReactiveStateLessEngine which maintains states and outputs calculation results based on associated data for each data injection.

Added function kroghInterpolate for polynomial interpolation of a set of points.

Added mTopN function: mpercentileTopN.

Added m-functions: mfirstNot, mlastNot, mLowRange, mTopRange.

Added functions minIgnoreNull and maxIgnoreNull for obtaining the min and max values with NULLs ignored.

Added tm-functions: tmLowRange, tmTopRange.

Added support for ZSTD compression algorithm.

Added function setDynamicConfig to dynamically set a specified configuration parameter.

Added function getDynamicConfig to list all dynamically-modifiable configuration parameters.

Added configuration parameter decimalRoundingMode to set the rounding method for DECIMAL values.

Added configuration parameter checkExistingPartitionNumForQuery to use the number of existing partitions obtained from the controller for queries.

Added configuration parameter enableAuditLog to enable audit logs. Added function getAuditLog to query audit logs and configuration parameter auditLogRetentionTime to set audit log retention time.

Added configuration parameter enableNullSafeJoin to determine whether NULL values are allowed to match other NULL values during join operations.

Added function setStreamTableTimestamp to attach a timestamp column to a stream table to record the data ingestion time.

The following features are supported in DolphinDB JIT version:

• Set and dictionary forms;

• STRING and SYMBOL types;

• Multiple return values;

• Creating array vectors.

The use keyword can be used to call .dom module files in scripts.

Added support for creating a matrix of DECIMAL or STRING type.

Added support for using macro variables and column series in SQL statements.

Enhanced the security of the login system by implementing IP-specific lockouts. Multiple failed login attempts occur from a particular IP address will temporarily block further login attempts only from that specific IP. (2.00.12.6)

For TSDB databases configured to retain all data (keepDuplicated set to ALL), the upsert! function no longer requires specifying the keyColNames parameter for all sort columns. (2.00.12.6)

Enahanced user access control for function views and module functions. (2.00.12.4)

Enhanced the performance of queries for finding maximum and minimum values on partitioning columns. (2.00.12.4)

The parameter groupingCol of function groupby now supports specifying a tuple with vectors of the same length. (2.00.12.2)

The parameter sortDirections of function sortBy! can be specified for a keyed table. (2.00.12.2)

The function fixedLengthArrayVector now supports concatenating fixed-length array vectors. (2.00.12.2)

Added parameter keyFilter to function createCrossSectionalEngine to enable key selection with filtering conditions. (2.00.12.2)

Optimized the performance of heterogeneous replay at maximum speed. (2.00.12.1)

Added parameter read for the license function to determine whether to verify the license file before returning the result. (2.00.12.1)

For the sortKeyMappingFunction parameter of function createPartitionedTable, if the mapping function is hashBucket, buckets now can be set as 1. (2.00.12.1)

Optimized the query performance on TSDB tables when using IN condition on sort keys. (2.00.12.1)

Enhanced system security with encrypted passwords. (2.00.12.1)

Optimized the performance of TSDB engine when reading data from cache engine.

The sortColumns parameter specified for a TSDB table now supports DECIMAL type.

Added support for setting priority queues at the database level for asynchronous replication.

Added support for tiered storage in a standalone mode.

The rebalanceChunksWithinDataNode function now can be called in a standalone mode.

Enhanced the partition pruning logic when using IN keyword within a WHERE clause.

Added alias createDimensionTable for the createTable function.

Optimized the performance of functions lasso and ElasticNet.

Added parameter ignoreNull for functions nunique and cumnunique, which specifies whether to return only the unique non-NULL elements.

Increased the maximum allowed precision for the round function to 10 digits.

Soft delete for TSDB databases now can be enabled in a CREATE TABLE statement with the newly-added parameter softDelete.

Added the execution plan for queries involving join operations in [HINT_EXPLAIN].

Added the execution plan for the WHERE clause in [HINT_EXPLAIN] to monitor the pre-filtering execution phase for queries on TSDB tables.

The following functions now can be called on a compute node: createUser, createGroup, resetPwd, changePwd, deleteUser, deleteGroup, addGroupMember, deleteGroupMember, grant, deny, revoke, setMaxJobPriority, setMaxJobParallelism, getUserList, getGroupList, getGroupsByUserId, getUsersByGroupId, getUserAccess, getGroupAccess.

Functions mifirstNot, miLastNot, mfirstNot, mlastNot, mimax, mimin now support indexed series and indexed matrix.

'DFLT' now can be set for keyword parameters to fix optional parameters as default values within a partial application.

Added parameter n for functions deltas and percentChange to enable calculations across multiple rows.

The reactive state engine now supports invoking the functions prev and move on array vector columns.

UPDATE statement now supports CSORT and HAVING clauses.

Added parameter bondType for functions fiDirtyPrice, fiDuration, and fiConvexity to support multiple bond types including zero-coupon bond and discount bond.

Array vector columns are now supported in keyed tables, keyed stream table, and indexed tables.

Added the arrayMarker parameter for functions loadText, loadTextEx, ploadText, and textChunkDS to specify an identifier to distinguish imported array vector.

Optimized the performance of cross join.

Optimized the performance when joining a dimension table and DFS table.

Added parameter anomalyDescription for function createAnomalyDetectionEngine to customize the detailed description for anomalies.

The replayDS function now can accept input tables containing DECIMAL columns.

Optimized the performance when retrieving distinct values of a single column using distinct.

Enhanced the createCrossSectionalEngine function with the following changes:

• The contextByColumn now can be a STRING vector.

• Added parameter outputElapsedMicroseconds to output the elapsed time and total rows of each calculated batch.

• Added parameter roundTime to align the window boundary when triggeringPattern='dataInterval'.

warmupStreamEngine now supports time series engine.

The func parameter of the moving function now accepts aggregate functions that return a tuple.

Added parameter map for the sql function to generate SQL statements with map keyword.

Optimized the performance of built-in aggregate functions operating on matrices.

Enhanced the createTimeSeriesEngine and createDailyTimeSeriesEngine functions with the following changes:

• Added parameter parallelism to support multi-threaded execution of metrics in the time-series engine.

• Added parameter acceptedDelay to set the maximum allowable delay for data reception when useSystemTime= true.

The print function is now supported in metrics of streaming engines.

The system now logs startup commands during the startup process.

Optimized the concurrency performance when joining multiple tables.

Added parameter usePinv for the ols function to determine whether to use pseudo-inverse method to calculate inverse of a matrix.

Added parameter excludedPeriod for the tmoving function to skip the calculation for specific time period.

Added support for updating array vector columns of DFS tables.

Modified the behavior of function at when it is served as a higher-order function. When X is a function and index is a tuple, each element of index is passed as an argument to the X.

The higher-order function each now allows a function with two or more arguments, where the first argument is a dictionary.

Optimized data rebalancing algorithms.

Added parameter updatedBeforeDays for functions rebalanceChunksAmongDataNodes and rebalanceChunksWithinDataNode, which limits the rebalancing process to only include data that was last updated before updatedBeforeDays days.

The higher-order function cross now allows a function that returns values of CODE, FUNCTIONDEF, STRING, or BLOB type.

Removed the restriction on time precision for WHERE conditions involving comparisons of time values in distributed queries.

In standalone mode, writing to distributed tables is restricted if the data size of a write operation exceeds the capacity of the cache engine.

Added support for defining vectors or tuples across multiple lines.

Enhanced the Analytic Function (Window Function) with the following changes:

• Added support for the following functions: imin, imax, iminLast, imaxLast, firstNot, lastNot, sum2, prod, percentile, wavg, wsum, corr, covar, beta, atImax, atImin.

• RANGE can be used to define a window frame based on time values.

• Added support for GROUP BY clause and allowed to nest aggregate function.

[D20-18152] Fixed an issue where interactive jobs submitted through Web interface or VS Code extension by users with lower-than-default priority (or parallelism) still used default values. (2.00.12.6)

[D20-18245] Fixed an error of the backup function on a compo-partitioned database if the filter condition matched the first-level partition and some second-level sub-partitions were empty. (2.00.12.6)

[D20-18312] Fixed a server crash when querying an array vector column with distinct. (2.00.12.6)

[D20-18356] Fixed an issue where queries failed during ej operation if the table name was not specified in the where condition. (2.00.12.6)

[D20-18423] Fixed system crashes after executing the createWindowJoinEngine function with invalid parameters. (2.00.12.6)

[D20-18122] Fixed an error where using the move function may lead to incorrect calculation results if the input data was an intermediate result. (2.00.12.6)

[D20-18219] Fixed occasional crashes when executing the mpercentile function multiple times. (2.00.12.6)

[D20-18331] Fixed an error in JIT when using the rank function due to type inference issues. (2.00.12.6)

[D20-18291] Fixed an error in JIT when using the isort function due to compatibility issues. (2.00.12.6)

Fixed incorrect equi join (ej) results of keyed tables. (2.00.12.4)

Fixed occasional redo log replay failures that occurred when function views were being added or deleted concurrently during a checkpoint operation. (2.00.12.4)

Fixed an issue where queries on a dimension table occasionally failed when dimension table cache was being released concurrently. (2.00.12.4)

Fixed an issue where asynchronous replication would get stuck in the slave cluster when multiple tables in a database were truncated concurrently. (2.00.12.4)

Fixed an issue of hanging nodes when modules loaded with configuration parameter preloadModules contained recursive function calls. (2.00.12.3)

Fixed occasional login failures during client reconnection. (2.00.12.3)

Fixed occasional server crashes when replaying redo logs for excessively time-consuming TSDB transactions. (2.00.12.3)

Fixed occasional server crashes caused by left join. (2.00.12.3)

Fixed inaccurate memory usage statistics for array vectors. (2.00.12.3)

Fixed server crashes caused by SQL queries when the first argument of interval was not a column. (2.00.12.2)

Fixed an issue where minPeriods of function mmad did not take effect. (2.00.12.2)

Fixed server crashes caused by SQL queries if a database path was not a string. (2.00.12.2)

Fixed incorrect results of distributed queries if context by clause involved more than two columns and the csort clause included these context-by columns. (2.00.12.2)

Fixed an issue where replay tasks got stuck if the number of workers was significantly smaller than the number of partitions. (2.00.12.2)

Fixed an error in SQL queries with pivot by clause when the select/exec clause contained expressions. (2.00.12.2)

Fixed an error when restoring manual backup files from Amazon S3. (2.00.12.2)

Fixed an error when querying an empty table where the select clause called function eachPre and the SQL query contained context by clause. This issue was introduced in version 2.00.11.1. (2.00.12.2)

Fixed precision loss of DECIMALs when writing streaming data to a DFS table with batchSize set to 1. (2.00.12.2)

Fixed occasional incorrect results of metaprogramming with macro variables when applying function patterns to multiple columns by group, particularly with aggregate or order-sensitive functions. (2.00.12.2)

Fixed an issue where the equal= sign in where conditions did not take effect if a STRING or SYMBOL partitioning column of a VALUE-partitioned table contained decimal points or other special characters. This issue was introduced in version 2.00.12. (2.00.12.2)

Fixed an error reported when aggregating joined columns. (2.00.12.2)

Fixed server crashes caused by SQL queries calling ols within the higher-order function moving. (2.00.12.2)

Fixed server crashes where Raft was not initialized while HA stream tables were already loaded during a node reboot. (2.00.12.2)

Fixed incorrect result of executionSet returned by getSlaveReplicationQueueStatus if a Leader was switched to Follower and then re-elected as Leader. (2.00.12.2)

Fixed Unrecognized column name error for queries using ej on DFS tables and external variables in where conditions. (2.00.12.2)

Fixed server crashes on the ARM version caused by multi-threaded queries. (2.00.12.1)

Fixed server crashes caused by concurrent select and append! operations on in-memory tables after resource tracking was enabled. (2.00.12.1)

Fixed an issue where the execution of clearCachedModules in console did not take effect. (2.00.12.1)

Fixed server crashes when using the head function in the aggs parameter of wj. (2.00.12.1)

Fixed server crashes caused by compilation failure during recursive or nested calls to JIT functions. (2.00.12.1)

Fixed server crashes caused by a distributed query with the WHERE clause containing functions that generated temporary variables. (2.00.12.1)

Fixed an issue where the analytic function could not recognize the column used in multiple joins. (2.00.12.1)

Fixed an issue where the schema function returned an incorrect partitionSchema that contained partitions already dropped with function dropPartition(with deleteSchema= true specified).

Fixed an issue where the result of the iif function was incorrectly typed as a STRING when the output was intended to be metacode.

Fixed an error for function getFunctionViews when the returned function body was excessively large.

Fixed an issue where the result of each function call was not being properly removed when the function contained cast($) operations used to convert the data type to the same type.

Fixed server crashes when the results of lshift (<<) and rshift (>>) contained NULL values.

Fixed an issue where replicas of chunks could be copied to a compute node using the copyReplicas function.

Fixed an issue where the format function returned incorrect results when applying a specified format to DECIMAL objects.

Fixed server crashes when using function med with excessively large data.

Fixed server crashes when passing a tuple with non-vector elements to the nunique function.

Fixed incorrect results when both X and Y of the asof function input vectors containing only NULL values.

Fixed an issue where appending data to a vector ignored the maximum size limit. An error will now be reported in such cases.

Fixed an issue where the lj, aj and other join-related functions did not support keyword parameters.

Fixed an issue where the database function failed to create an in-memory database with enableLocalDatabase=false configured.

Fixed an issue with the sqlDS function where partition pruning was not applied when the SQL statement specified in the sqlObj parameter contained the IN keyword in the WHERE clause.

Fixed an issue in standalone mode, where transactions were incorrectly replayed due to the edit log being written before the redo log.

Fixed incorrect results returned by the reactive state engine when calculating the cummin function with the following types of data: UUID, INT128, IPADDR, COMPLEX and POINT.

Fixed an error occurred when comments were placed between the function name and the function body.

Fixed an issue where the cleanup process for expired files of persistent stream tables would get stuck in an infinite loop if it encountered a non-existent file.

Fixed an issue where the presence of stale partitions could potentially cause data from new partitions located under the same path to be garbage collected.

Fixed server crashes when calculating DECIMAL array vectors in a query with PIVOT BY clause.

Fixed server crashes when using both CASE and WHERE NOT keywords in a single query.

Fixed an error that reported when using three or more UNION ALL keywords in a single SELECT FROM statement.

Fixed inconsistent query results when using double quote ("") and single quotes ('') in the WHERE condition to filter the CHAR column of a DFS table.

Fixed incorrect query results on a sorted keyed table

Fixed an issue where assigning a matrix to multiple output columns in a query returned incorrect results.

Fixed server crashes when using exists keyword in a query containing joins of multiple partitioned tables.

Fixed an issue where using a non-aggregate function in the SELECT statement did not report an error when grouping value-partitioned columns using the GROUP BY clause. An error will now be reported in such cases.

Fixed an error that occurred when the query result contained an array vector and the PIVOT BY clause included sort keys.

Fixed an error that occurred when using a function with multiple return values in combination with a WHERE clause in a query.

Fixed server crashes due to an out-of-bounds error when accessing an array in metrics of a time-series engine with useSystemTime=true specified.

Fixed a parsing issue in time-series engines for functions that return array vectors.

Fixed an issue where a write failure to a persisted table could disrupt data consumption for the subscription.

Fixed server crashes when the number of metricNames did not match that of factors specified in metrics for function createNarrowReactiveStateEngine.

Fixed an error when using the nullFill! function in a reactive state engine. This error was introduced in version 2.00.11.2.

Fixed server crashes when the formal parameter of the JIT function had the same name as the built-in functions.

Fixed an issue where the newly appended element to a vector within a JIT function could not be accessed or modified by indexes.

Fixed incorrect results returned by the JIT function when calculating a vector that had NULL values appended or assigned to it.