each (:E)
Syntax
or
or
or
Arguments
func is a function.
args … are the required parameters of func.
operator is a binary operator.
X / Y can be pair/vector/matrix/table/dictionary. X and Y must have the same dimensions.
Details
Apply a function (specified by func or operator) to each element of args/ X/ Y.
For matrices, calculate in each column;
For tables, calculate in each row;
For dictionaries, calculate each value.
The data type and form of the return value are determined by each calculation result. It returns a vector or matrix if all calculation results have the same data type and form, otherwise it returns a tuple.
The difference between func(X) and func :E X is that the former treats X as the one input variable while the later takes each element in X as an input variable. If func is a vector function, avoid using “:E” since element-wise operations are very slow with a large number of elements.
Examples
Suppose we need to calculate the daily compensation for 3 workers. Their working hours are stored in vector x=[9,6,8]. Their hourly rate is \(10 under 8 hours and \)20 beyond 8 hours. Consider the following function wage
:
$ x=[9,6,8]
$ def wage(x){if(x<=8) return 10*x; else return 20*x-80}
$ wage x;
The vector can't be converted to bool scalar.
wage(x) does not return a result, as x<=8, i.e., [9,6,8]<=8 returns a vector of conditions [0,1,1], not a scalar condition that is required by if
.
In contrast, consider the following solutions:
$ each(wage, x);
[100,60,80]
$ wage :E x;
[100,60,80]
$ def wage2(x){return iif(x<=8, 10*x, 20*x-80)};
// the iif function is an element-wise conditional operation
$ wage2(x);
[100,60,80]
Similarly, each
can also be applied to a function with more than one parameter.
$ def addeven(x,y){if (x%2==0) return x+y; else return 0}
$ x1=1 2 3
$ x2=4 5 6;
$ each(addeven, x1, x2);
[0,7,0]
each
with a tuple:
$ t = table(1 2 3 as id, 4 5 6 as value, `IBM`MSFT`GOOG as name);
$ t;
id |
value |
name |
---|---|---|
1 |
4 |
IBM |
2 |
5 |
MSFT |
3 |
6 |
GOOG |
$ each(max, t[`id`value]);
[3,6]
each
with matrices:
$ m=1..12$4:3;
$ m;
col1 |
col2 |
col3 |
---|---|---|
1 |
5 |
9 |
2 |
6 |
10 |
3 |
7 |
11 |
4 |
8 |
12 |
$ each(add{1 2 3 4}, m);
// note add{1 2 3 4} is a partial application, which adds [1, 2, 3, 4] to each of the 3 columns
col1 |
col2 |
col3 |
---|---|---|
2 |
6 |
10 |
4 |
8 |
12 |
6 |
10 |
14 |
8 |
12 |
16 |
$ x=1..6$2:3;
$ y=6..1$2:3;
$ x;
col1 |
col2 |
col3 |
---|---|---|
1 |
3 |
5 |
2 |
4 |
6 |
$ y;
col1 |
col2 |
col3 |
---|---|---|
6 |
4 |
2 |
5 |
3 |
1 |
$ each(**, x, y);
[16,24,16]
// e.g., 24=3*4+4*3
When there are multiple objects passed in as args/X/Y, the function takes the element at the same position from each object as arguments for each calculation.
$ m1 = matrix(1 3 6, 4 6 8, 5 -1 3)
$ m2 = matrix(3 -6 0, 2 NULL 3, 6 7 9)
$ each(corr, m1, m2)
// equal to corr(m1[0], m2[0]) join corr(m1[1], m2[1]) join corr(m1[2], m2[2])
[-0.216777, 1, -0.142857]
Starting from version 1.30.21, each supports dictionary:
$ d=dict(`a`b`c, [[1, 2, 3],[4, 5, 6], [7, 8, 9]])
$ each(sum, d)
b->15
c->24
a->6
When func is a user-defined function that operates on dictionaries whose keys are STRINGs, the each template combines each dictionary and outputs a table following these rules: - The table schema is only determined by the first dictionary whose values are appended to the first row, and keys are treated as column names. The number of keys is the number of columns. - Iterate through the remaining dictionary and append each dictionary value as a new row in the table. Specifically:
When a dictionary key matches a column name, append the corresponding value to that column.
For any extra keys in the dictionary that don’t match columns, discard those values.
For any extra column names without matching keys, fill in missing values as NULL.
days = 2023.01.01..2023.01.10
def mf(day) {
out = dict(STRING, ANY)
if(day==2023.01.05){
out["v"] = 3
}
else{
out["day"] = day
out["v"] = 1
}
return out
}
each(mf, days)
v day
1 2023.01.01
1 2023.01.02
1 2023.01.03
1 2023.01.04
3
1 2023.01.06
1 2023.01.07
1 2023.01.08
1 2023.01.09
1 2023.01.10
In the example below, we use the function call
in a partial application that applies each of [sin, log] to vector 1..3
// when "functionName" is empty, it will be filled with function names dynamically.
$ each(call{, 1..3},(sin,log));
sin |
log |
---|---|
0.841471 |
0 |
0.909297 |
0.693147 |
0.14112 |
1.098612 |
Performance Note
Template
peach
is recommended for tasks that take a long time to execute.
$ m=rand(1,20000:5000)
$ timer f=peach(mskew{,8},m)
Time elapsed: 3134.71 ms
$ timer f=mskew(m,8)
Time elapsed: 8810.485 ms
Template
:E (each)
is not recommended when there is a large number of elements. In those scenarios we should look for more efficient vector solutions.
$ x=rand(16, 1000000);
$ timer(10){each(wage, x)};
Time elapsed: 38164.9 ms
$ timer(10){iif(x<8,10*x,20*x-80)};
Time elapsed: 81.516 ms