each

Syntax

each(func, args...) (apply a function to each element of the specified parameters)

or

F :E X (apply a function to each element of X)

or

X < operator > :E Y (apply an operator to each element of X and Y)

or

func:E(args...)

Arguments

func is a function.

args are the required parameters of func.

operator is a binary operator.

X / Y can be pair/vector/matrix/table/dictionary. X and Y must have the same dimensions.

Details

Apply a function (specified by func or operator) to each element of args / X / Y.

• For matrices, calculate in each column;

• For tables, calculate in each row;

• For dictionaries, calculate each value.

The data type and form of the return value are determined by each calculation result. It returns a vector or matrix if all calculation results have the same data type and form, otherwise it returns a tuple.

The difference between func(X) and func :E X is that the former treats X as the one input variable while the later takes each element in X as an input variable. If func is a vector function, avoid using :E since element-wise operations are very slow with a large number of elements.

Examples

Suppose we need to calculate the daily compensation for 3 workers. Their working hours are stored in vector x=[9,6,8]. Their hourly rate is $10 under 8 hours and $20 beyond 8 hours. Consider the following function wage:

x=[9,6,8]
def wage(x){if(x<=8) return 10*x; else return 20*x-80}
wage x;
// output
The vector can't be converted to bool scalar.

wage(x) does not return a result, as x<=8, i.e., [9,6,8]<=8 returns a vector of conditions [0,1,1], not a scalar condition that is required by if.

In contrast, consider the following solutions:

each(wage, x);
// output
[100,60,80]

wage :E x;
// output
[100,60,80]

def wage2(x){return iif(x<=8, 10*x, 20*x-80)};
// the iif function is an element-wise conditional operation

wage2(x);
// output
[100,60,80]

Similarly, each can also be applied to a function with more than one parameter.

def addeven(x,y){if (x%2==0) return x+y; else return 0}
x1=1 2 3
x2=4 5 6;
each(addeven, x1, x2);
// output
[0,7,0]

each with a tuple:

t = table(1 2 3 as id, 4 5 6 as value, `IBM`MSFT`GOOG as name);
t;

each(max, t[`id`value]);
// output
[3,6]

each with matrices:

m=1..12$4:3;
m;

each(add{1 2 3 4}, m);
// add{1 2 3 4} is a partial application, which adds [1, 2, 3, 4] to each of the 3 columns

x=1..6$2:3;
y=6..1$2:3;
x;

y;

each(**, x, y);
// output
[16,24,16]
// e.g., 24=3*4+4*3

When there are multiple objects passed in as args / X / Y, the function takes the element at the same position from each object as arguments for each calculation.

m1 = matrix(1 3 6, 4 6 8, 5 -1 3)
m2 = matrix(3 -6 0, 2 NULL 3, 6 7 9)
each(corr, m1, m2)
// equal to corr(m1[0], m2[0]) join corr(m1[1], m2[1]) join corr(m1[2], m2[2])
// output
[-0.216777, 1, -0.142857]

each supports dictionary:

d=dict(`a`b`c, [[1, 2, 3],[4, 5, 6], [7, 8, 9]])
each(sum, d)
// output
b->15
c->24
a->6

When func is a user-defined function that operates on dictionaries whose keys are STRINGs, the each template combines each dictionary and outputs a table following these rules:

• The table schema is only determined by the first dictionary whose values are appended to the first row, and keys are treated as column names. The number of keys is the number of columns.

• Iterate through the remaining dictionary and append each dictionary value as a new row in the table. Specifically:

  • When a dictionary key matches a column name, append the corresponding value to that column.

  • For any extra keys in the dictionary that don't match columns, discard those values.

  • For any extra column names without matching keys, fill in missing values as NULL.

days = 2023.01.01..2023.01.10
def mf(day) {  
    out = dict(STRING, ANY)
    if(day==2023.01.05){
        out["v"] = 3
    }
    else{
        out["day"] = day
        out["v"] = 1
    }
    return out
}
each(mf, days)
// output
v   day
1   2023.01.01
1   2023.01.02
1   2023.01.03
1   2023.01.04
3
1   2023.01.06
1   2023.01.07
1   2023.01.08
1   2023.01.09
1   2023.01.10

Since version 2.00.12/3.00.0, when args contain a dictionary, the each function can be applied to a function with more than one arguments with the dictionary as the first argument.

t = table(1..10 as id, 2..11 as id2)
sumbars:E(t.transpose(), 3 5).transpose()

In the example below, we use the function call in a partial application that applies each of [sin, log] to vector 1..3

// when "functionName" is empty, it will be filled with function names dynamically.
each(call{, 1..3},(sin,log));

Performance Tips

• Template peach is recommended for tasks that take a long time to execute.

  m=rand(1,20000:5000)
  timer f=peach(mskew{,8},m)
  Time elapsed: 3134.71 ms
  
  timer f=mskew(m,8)
  Time elapsed: 8810.485 ms

• Template :E (each) is not recommended when there is a large number of elements. In those scenarios we should look for more efficient vector solutions.

  x=rand(16, 1000000);
  timer(10){each(wage, x)};
  Time elapsed: 38164.9 ms
  
  timer(10){iif(x<8,10*x,20*x-80)};
  Time elapsed: 81.516 ms