Ranking Expressions
This is a complete reference to the ranking expressions used to configure application specific ranking functions. For examples and an overview of how to use ranking expressions, see the ranking overview.
Ranking expressions are written in a simple language similar to ordinary functional notation. The atoms in ranking expressions are rank features and constants. These atoms can be combined by arithmetic operations and built-in functions.
| Rank Features | A rank feature is a named value calculated or looked up by vespa for each query/document combination. See the rank feature reference for a list of all the rank features available to ranking expressions. |
|---|---|
| Constants | A constant is either a floating point number, or a quoted string. Strings are only useful for equality comparison in the if function. |
Arithmetic operations
The four basic operations, addition, subtraction, multiplication and division are expressed in in-fix notation:
a + b - c * d / (e + f)Multiplication and division has higher precedence than addition and subtraction. Terms can be ordered using parentheses when it is necessary to override default precedence.
Mathematical Functions
| cosh(x) | Hyperbolic cosine of x |
| sinh(x) | Hyperbolic sine of x |
| tanh(x) | Hyperbolic tangent of x |
| cos(x) | Cosine of x |
| sin(x) | Sine of x |
| tan(x) | Tangent of x |
| acos(x) | Inverse cosine of x |
| asin(x) | Inverse sine of x |
| atan2(y, x) | Inverse tangent of y / x, using signs of both arguments to determine correct quadrant. |
| atan(x) | Inverse tangent of x |
| exp(x) | Base-e exponential function. |
| ldexp(x, exp) | Multiply x by 2 to the power of exp |
| log10(x) | Base-10 logarithm of x |
| log(x) | Base-e logarithm of x |
| pow(x, y) | Return x raised to the power of y |
| sqrt(x) | Square root of x |
| ceil(x) | Lowest integral value not less than x |
| fabs(x) | Absolute value of (floating-point) number x |
| floor(x) | Largest integral value not greater than x |
| isNan(x) | Returns 1.0 if x is NaN, 0.0 otherwise |
| fmod(x, y) | Remainder of x / y |
| min(x, y) | Smaller of x and y |
| max(x, y) | Larger of x and y |
x and y may be any ranking expression.
The if Function
The if function chooses between two sub-expressions based
on the truth value of a condition.
if (expression1 operator expression2, trueExpression, falseExpression)If the condition given in the first argument is true, the expression in argument 2 is used, otherwise argument 3. The four expressions may be any ranking expression. Conditional operators in ranking expression if functions:
| <= | Less than or equal |
| < | Less than |
| == | Equal |
| ~= | Approximately equal |
| >= | Greater than or equal |
| > | Greater than |
The in membership operator uses a slightly modified if-syntax:
if (expression1 in [expression2, expression3, ..., expressionN], trueExpression, falseExpression)If expression1 is equal to either of expression1 through expressionN, then trueExpression is used, otherwise falseExpresion.
The foreach Function
The foreach function is not really part of the expression language but implemented as a rank feature.
Tensor Functions
The following set of tensors functions are available to use in ranking expressions. The functions are grouped in primitive functions and convenience functions that can be implemented in terms of the primitive ones.
Primitive functions
| map(tensor, f(x)(expr)) |
Returns a new tensor with the lambda function defined in Arguments:
tensor. Examples:
map(t, f(x)(abs(x))) map(t, f(i)(if(i < 0, 0, i))) |
|---|---|
| reduce(tensor, aggregator, dim1, dim2, ...) |
Returns a new tensor with the Arguments:
dim1, dim2, etc.
If no dimensions are specified, reduce over all dimensions.
Available aggregators are:
reduce(t, sum) # Sum all values in tensor reduce(t, count, x) # Count number of cells along dimension x |
| join(tensor1, tensor2, f(x,y)(expr)) |
Returns a new tensor constructed from the natural join between Arguments:
tensor1 and tensor2,
with the resulting cells having the value as calculated from f(x,y)(expr),
where x is the cell value from tensor1 and
y from tensor2.
Formally, the result of the Examples:
t1 = {{x:0}: 1.0, {x:1}: 2.0}
t2 = {{x:0,y:0}: 3.0, {x:0,y:1}: 4.0, {x:1,y:0}: 5.0, {x:1,y:1}: 6.0}
join(t1, t2, f(x,y)(x * y)) = {{x:0,y:0}: 3.0, {x:0,y:1}: 4.0, {x:1,y:0}: 10.0, {x:1,y:1}: 12.0}
reduce(join(t1, t2, f(x,y)(x * y)), sum) = 29.0
|
| merge(tensor1, tensor2, f(x,y)(expr)) |
Returns a new tensor consisting of all cells from both the arguments, where the lambda function is used to produce a single value in the cases where both arguments provide a value for a cell. Arguments:
The argument tensors must have the same type, and that will be the type of the resulting tensor. Example:
t1 = tensor(key{},x[4]):{a:[1,2],b:[3,4]}
t2 = tensor(key{},x[2]):{b:[5,6],c:[7,8]}
merge(t1, t2, f(left,right)(right)) = tensor(key{},x[2]):{a:[1,2],b:[5,6],c:[7,8]}
|
| tensor(tensor-type-spec)(expr) |
Generates new tensors according to type specification and expression Arguments:
expr.
The tensor type must be an indexed tensor (e.g. tensor<float>(x[10])).
The expression in expr will be evaluated for each cell.
The arguments in the expression is implicitly the names of the dimensions defined in the type spec.
Useful for creating transformation tensors. Examples:
t1 = tensor(x[4]):[1.0, 2.0, 3.0, 4.0]
tensor<float>(x[3])(x) = [0.0, 1.0, 2.0]
tensor<float>(x[2],y[2])(x == y) = {{x:0,y:0}: 1.0, {x:0,y:1}: 0.0, {x:1,y:0}: 0.0, {x:1,y:1}: 1.0}
tensor(f[2])(t1{x: f + f()})) - Create a size 2 1d tensor by accessing t1 at two indexes given by adding
the index of it (f - which is 0 and 1) by a function also named f.
|
| rename(tensor, dim-to-rename, new-names) |
Renames one or more dimensions in the tensor. Arguments:
t1 = {{x:0,y:0}: 1.0, {x:0,y:1}: 0.0, {x:1,y:0}: 0.0, {x:1,y:1}: 1.0}
rename(t1,x,z) = {{z:0,y:0}: 1.0, {z:0,y:1}: 0.0, {z:1,y:0}: 0.0, {z:1,y:1}: 1.0}
rename(t1,(x,y),(i,j)) = {{i:0,j:0}: 1.0, {i:0,j:1}: 0.0, {i:1,j:0}: 0.0, {i:1,j:1}: 1.0}
|
| concat(tensor1, tensor2, dim) |
Concatenates two tensors along dimension Arguments:
tensor1 and
tensor2 concatenated along dimension dim. Examples:
t1 = {{x:0}: 0.0, {x:1}: 1.0}
t2 = {{x:0}: 2.0, {x:1}: 3.0}
concat(t1,t2,x) = {{x:0}: 0.0, {x:1}: 1.0}, {x:2}: 2.0, {x:3}: 3.0}}
|
| (tensor)partial-address |
Slice - returns a new tensor containing the cells matching the partial address. Arguments:
t1 = tensor(x[2]):[1.0, 2.0]]
t2 = tensor(key{}):{ key1:1.0, key2:2.0 }
t3 = tensor(key{},x[2]):{ key1:[1.0, 2.0], key2:[3.0, 4.0] }
t1[1] = {2.0}
t2{key1} = {1.0}
t3{key1} = tensor(x[2]):[1.0, 2.0]
t3[1] = tensor(key{}):{key1:2.0, key2:4.0}
t3{key:key1,x:1} = {2.0}
t3{key:key1,x:(3-2)} = {2.0}
|
| tensor-literal-form |
Returns a new tensor having the type and cell values given explicitly. Each cell value may be supplied by a lambda which can access other features. Returns a new tensor from the literal form, where the type must be specified explicitly. Each value may be supplied by a lambda, which - in contrast to all other lambdas - may refer to features and expressions from the context. Examples:
t1 = tensor(x[2]):[1.0, 2.0]]
tensor(x{}):{x1:3, x2:4} = tensor(x{}):{x1:3.0, x2:4.0}
tensor(x{}):{x1:t1[1], x2:t1[0]} = tensor(x{}):{x1:2.0, x2:1.0} - Used in conjunction with slice,
to convert an indexed tensor to mapped form.
|
| cell_cast(tensor, cell_type) |
Returns a new tensor that is the same as the argument, except that all cell values are converted to the given cell type. Arguments:
Example: t1 = tensor<float>(x[5])(x+1) t2 = tensor<bfloat16>(x[5])(x+1) t3 = cell_cast(t1, bfloat16) # t3 will be exactly the same as t2 |
Lambda functions in primitive functions
Some of the primitive functions accept lambda functions that are evaluated and applied to a set of tensor cells. The functions contain a single expression that have the same format and built-in functions as general ranking expressions. However, the atoms are the arguments defined in the argument list of the lambda.
The expression cannot access variables or data structures outside of the lambda, i.e. they are not closures.
Examples:
f(x)(abs(x)) f(x,y)(if(x < y, 0, 1))
Non-primitive functions
Non-primitive functions can be implemented by primitive functions, but are not necessarily so for performance reasons.
| abs(t) |
map(t, f(x)(abs(x))) Absolute value of all elements. |
| acos(t) |
map(t, f(x)(acos(x))) Arc cosine of all elements. |
| t1 + t2 (add) |
join(t1, t2, f(x,y)(x + y)) Join and sum tensors t1 and t2.
|
| argmax(t, dim) |
join(t, max(t, dim), f(x,y)(if (x == y, 1, 0))) Returns a tensor with cell(s) of the highest value(s) in the tensor set to 1. The dimension argument follows the same format as reduce as multiple dimensions can be given and is optional. |
| argmin(t, dim) |
join(t, min(t), f(x,y)(if (x >= y, 0, 1))) Returns a tensor with cell(s) of the lowest value(s) in the tensor set to 1. The dimension argument follows the same format as reduce as multiple dimensions can be given and is optional. |
| asin(t) |
map(t, f(x)(asin(x))) Arc sine of all elements. |
| atan(t) |
map(t, f(x)(atan(x))) Arc tangent of all elements. |
| atan2(t1,t2) |
join(t1, t2, f(x,y)(atan2(x,y))) Arctangent of t1 and t2/
|
| avg(t, dim) |
reduce(t, avg, dim) Reduce the tensor with the average aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| ceil(t) |
map(t, f(x)(ceil(x))) Ceiling of all elements. |
| count(t, dim) |
reduce(t, count, dim) Reduce the tensor with the count aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| cos(t) |
map(t, f(x)(cos(x))) Cosine of all elements. |
| cosh(t) |
map(t, f(x)(cosh(x))) Hyperbolic cosine of all elements. |
| diag(n1,n2) |
tensor(i[n1],j[n2])(if (i==j, 1.0, 0.0))) Returns a tensor with the diagonal set to 1.0. |
| t1 / t2 (div) |
join(t1, t2, f(x,y)(x / y)) Join and divide tensors t1 and t2.
|
| elu(t) |
map(t, f(x)(if(x < 0, exp(x)-1, x))) Exponential linear unit. |
| t1 == t2 (equal) |
join(t1, t2, f(x,y)(x == y)) Join and determine if each element in t1 and t2 are equal.
|
| exp(t) |
map(t, f(x)(exp(x))) Exponential function (e^x) of each element. |
| floor(t) |
map(t, f(x)(floor(x))) Floor of each element. |
| t1 > t2 (greater) |
join(t1, t2, f(x,y)(x > y)) Join and determine if each element in t1 is greater than t2.
|
| t1 >= t2 (greater or equals) |
join(t1, t2, f(x,y)(x >= y)) Join and determine if each element in t1 is greater than or equals t2.
|
| t1 < t2 (less) |
join(t1, t2, f(x,y)(x < y)) Join and determine if each element in t1 is less than t2.
|
| t1 <= t2 (less equals) |
join(t1, t2, f(x,y)(x <= y)) Join and determine if each element in t1 is less than or equals t2.
|
| l1_normalize(t, dim) |
join(t, reduce(t, sum, dim), f(x,y) (x / y)) L1 normalization: t / sum(t, dim).
|
| l2_normalize(t, dim) |
join(t, map(reduce(map(t, f(x)(x * x)), sum, dim), f(x)(sqrt(x))), f(x,y)(x / y)) L2 normalization: t / sqrt(sum(t^2, dim).
|
| log(t) |
map(t, f(x)(log(x))) Natural logarithm of each element. |
| log10(t) |
map(t, f(x)(log10(x))) Logarithm with base 10 of each element. |
| matmul(t1, t2, dim) |
reduce(join(t1, t2, f(x,y)(x * y)), sum, dim) Matrix multiplication of two tensors. This is the product of the two tensors summed along a shared dimension. |
| max(t, dim) |
reduce(t, max, dim) Reduce the tensor with the max aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| max(t1,t2) |
join(t1, t2, f(x,y)(max(x,y))) Join and return the max of t1 or t2. Arguments can be scalars.
|
| median(t, dim) |
reduce(t, median, dim) Reduce the tensor with the median aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| min(t, dim) |
reduce(t, min, dim) Reduce the tensor with the min aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| min(t1,t2) |
join(t1, t2, f(x,y)(min(x,y))) Join and return the minimum of t1 or t2. Arguments can be scalars.
|
| mod(t,constant) |
map(t, f(x)(mod(x,constant))) Modulus of constant with each element.
|
| t1 * t2 (mul) |
join(t1, t2, f(x,y)(x * y)) Join and multiply tensors t1 and t2.
|
| t1 != t2 (not equal) |
join(t1, t2, f(x,y)(x != y)) Join and determine if each element in t1 and t2 are not equal.
|
| pow(t,constant) |
map(t, f(x)(pow(x,constant))) Raise each element to the power of constant.
|
| prod(t, dim) |
reduce(t, prod, dim) Reduce the tensor with the product aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| random(n1, n2, ...) |
tensor(i1[n1],i2[n2],...)(random(1.0)) Returns a tensor with random values between 0.0 and 1.0, uniform distribution. |
| range(n) |
tensor(i[n])(i) Returns a tensor with increasing values. |
| relu(t) |
map(t, f(x)(max(0,x))) Rectified linear unit. |
| round(t) |
map(t, f(x)(round(x))) Round each element. |
| sigmoid(t) |
map(t, f(x)(1.0 / (1.0 + exp(0.0-x)))) Returns the sigmoid of each element. |
| sin(t) |
map(t, f(x)(sin(x))) Sinus of each element. |
| sinh(t) |
map(t, f(x)(sinh(x))) Hyperbolic sinus of each element. |
| sign(t) |
map(t, f(x)(if(x < 0, -1.0, 1.0))) The sign of each element. |
| softmax(t, dim) |
join(map(t, f(x)(exp(x))), reduce(map(t, f(x)(exp(x))), sum, dim), f(x,y)(x / y)) The softmax of the tensor, e.g. e^x / sum(e^x).
|
| sqrt(t) |
map(t, f(x)(sqrt(x))) The square root of each element. |
| square(t) |
map(t, f(x)(square(x))) The square of each element. |
| t1 - t2 (subtract) |
join(t1, t2, f(x,y)(x - y)) Join and subtract tensors t1 and t2.
|
| sum(t, dim) |
reduce(t, sum, dim) Reduce the tensor with the summation aggregator along dimension dim.
If the dimension argument is omitted, this reduces over all dimensions.
|
| tan(t) |
map(t, f(x)(tan(x))) The tangent of each element. |
| tanh(t) |
map(t, f(x)(tanh(x))) The hyperbolic tangent of each element. |
| xw_plus_b(x,w,b,dim) |
join(reduce(join(x, w, f(x,y)(x * y)), sum, dim), b, f(x,y)(x+y)) Matrix multiplication of x (usually a vector) and w (weights),
with b added (bias).
A typical operation for activations in a neural network layer,
e.g. sigmoid(xw_plus_b(x,w,b))).
|