A tensor is a set of named dimensions defining its order and a set of values located in the space of those dimensions:
The set of dimensions, cell values and cell address key-values can be of any size including zero.
A dimension can be either mapped or indexed.
Mapped dimensions use string identifiers as labels in the cell addresses (like a map),
while indexed dimensions use integers in the range [0,N>
(like an array),
where N is the size of the dimension.
Contained in constant or tensor field type. The dimensions of a tensor and the cell type defines its type. A tensor type contains a list of dimensions on the format:
tensor<value-type>(dimension-1,dimension-2,...,dimension-N)
The value-type is one of:
Type | Description |
---|---|
float | 32-bit IEEE 754 floating point |
double | 64-bit IEEE 754 floating point |
int8 | signed 8-bit integer - see performance considerations |
bfloat16 | first 16 bits of 32-bit IEEE 754 floating point - see performance considerations |
A dimension is specified as follows:
dimension-name{}
- a mapped dimensiondimension-name[size]
- an indexed dimensionThe tensor type for a tensor<float> with two mapped dimensions x and y looks like:
tensor<float>(x{},y{})
Example tensor with this type:
{{x:a,y:b}:10.0, {x:c,y:d}:20.1}
The tensor type for a tensor<float> with two indexed dimensions x and y with sizes 3 and 2 respectively looks like:
tensor<float>(x[3],y[2])
Example tensor with this type (representing a matrix):
{{x:0,y:0}:1, {x:0,y:1}:2.1, {x:1,y:0}:3, {x:1,y:1}:5, {x:2,y:0}:7, {x:2,y:1}:11}
Note that the labels are indexes in the range [0,dimension-size>
A tensor<double> with both mapped and indexed dimensions is mixed:
tensor<double>(key{},x[2])
Example:
{{key:a,x:0}:10, {key:b,x:0}:2.7, {key:a,x:1}:5.3, {key:b,x:1}:-7 }
The tensor literal form is used in:
The general literal form can represent any tensor and is as follows (EBNF):
literal tensor = ( tensor-type-spec ":" )? "{" cells "}" ; cells = | cell , { "," cell } ; cell = "{" address "}:" scalar ; address = | element, { "," element } ; element = dimension ":" label ; dimension = integer | identifier ; label = integer | identifier | 'string' | "string" ; identifier = ["A"-"Z","a"-"z","0"-"9","_","@"](["A"-"Z","a"-"z","0"-"9","_","@","$"])*
If no type spec is included the type is inferred from cells.
An empty tensor:
{}
A single value tensor with a single mapped dimension x:
{ {x:foo}:5.0 }
A tensor with multiple values and mapped dimensions x and y:
{ {x:foo, y:bar}:5.0, {x:foo, y:baz}:7.0 }
A tensor where type is specified explicitly with a single indexed dimension x representing a vector:
tensor<float>(x[3]):{ {x:0}:3.0, {x:1}:5.0, {x:2}:7.0 }
A tensor with a type using the default value type (double) and quoted labels:
tensor(key{}):{ {key:'key.1'}:3.0, {key:'key 2'}:5.0, {key:"key's"}:7.0 }
Tensors where all dimensions are indexed can be written as numbers wrapped in square brackets in right dimension adjacent order. This form requires an explicit tensor type.
Brackets must either be nested according to the structure of the type, where values in dimensions to the right are closer than dimensions on the left, or alternatively all values may be given (in the same order) as a flat array (the latter not supported in expressions).
A float 1d tensor in indexed form:
tensor<float>(x[3]):[3.0, 5.0, 7.0]
A matrix in indexed form. Since the values for the right-most dimension (y) are adjacent, the value 3 is here assigned to the cell {x:0,y:2}:
tensor<float>(x[2],y[3]):[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
The dimension order here is given by the order in which the dimensions are specified, not in the natural (alphabetic) order inherent in the tensor type. Since inner brackets can be omitted, the above is equivalent to
tensor<float>(x[2],y[3]):[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
Tensors representing binary data (indexed tensors with int8
cell value type) may be
represented using a hexadecimal representation, similar to the corresponding
JSON tensor feed format.
Tensors with a single mapped dimension can be written by specifying just the label in that implicit dimension instead of a full address map. This form requires a type to be specified.
tensor<float>(key{}):{ key1:1.0, key2:2.0 }
Tensors with a single mapped dimension and one or more indexed dimensions can be written by specifying the mapped dimension in the map short form and the values of each dense subspace on the indexed short form. This form requires a type to be specified.
A map of matrices:
tensor<float>(key{},x[2],y[3]):{ key1:[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]], key2:[[1.1, 2.1, 3.1], [4.1, 5.1, 6.1]] }
Tensor functions are listed in the expressions documentation.
The following rank features can be used to refer to or create tensors in ranking expressions. The tensors can come from the document, the query or a constant in the application package:
Use the following reference documentation on how use tensors in documents: