Tensor Reference

A tensor is a set of named dimensions defining its order and a set of values located in the space of those dimensions:

• Cell: A value located in the dimension space. Consists of a cell address and the value at that address.
• Address: A set of key-values where each key is a dimension from the set of dimensions of the tensor, and each value is a label (integer or string) determining the cell's location in that dimension.

The set of dimensions, cell values and cell address key-values can be of any size including zero. A dimension can be either mapped or indexed. Mapped dimensions use string identifiers as labels in the cell addresses (like a map), while indexed dimensions use integers in the range [0,N> (like an array), where N is the size of the dimension.

Tensor type spec

Contained in constant or tensor field type. The dimensions of a tensor and the cell type defines its type. A tensor type contains a list of dimensions on the format:

tensor<value-type>(dimension-1,dimension-2,...,dimension-N)

The value-type is one of:

A dimension is specified as follows:

• dimension-name{} - a mapped dimension
• dimension-name[size] - an indexed dimension

The tensor type for a tensor<float> with two mapped dimensions x and y looks like:

tensor<float>(x{},y{})

Example tensor with this type:

{{x:a,y:b}:10.0, {x:c,y:d}:20.1}

The tensor type for a tensor<float> with two indexed dimensions x and y with sizes 3 and 2 respectively looks like:

tensor<float>(x[3],y[2])

Example tensor with this type (representing a matrix):

{{x:0,y:0}:1, {x:0,y:1}:2.1,
 {x:1,y:0}:3, {x:1,y:1}:5,
 {x:2,y:0}:7, {x:2,y:1}:11}

Note that the labels are indexes in the range [0,dimension-size>

A tensor<double> with both mapped and indexed dimensions is mixed:

tensor<double>(key{},x[2])

Example:

{{key:a,x:0}:10,  {key:b,x:0}:2.7,
 {key:a,x:1}:5.3, {key:b,x:1}:-7  }

Tensor literal form

The tensor literal form is used in:

• Tensors in queries, see defining query feature types
• Constant tensors in stateless model evaluation
• Building tensors using the Java Tensor API

General literal form

The general literal form can represent any tensor and is as follows (EBNF):

literal tensor = ( tensor-type-spec ":" )? "{" cells "}" ;
cells = | cell , { "," cell } ;
cell = "{" address "}:" scalar ;
address = | element, { "," element } ;
element = dimension ":" label ;
dimension = integer | identifier ;
label = integer | identifier | 'string' | "string" ;
identifier = ["A"-"Z","a"-"z","0"-"9","_","@"](["A"-"Z","a"-"z","0"-"9","_","@","$"])*

If no type spec is included the type is inferred from cells.

General literal form examples:

An empty tensor:

{}

A single value tensor with a single mapped dimension x:

{ {x:foo}:5.0 }

A tensor with multiple values and mapped dimensions x and y:

{ {x:foo, y:bar}:5.0, {x:foo, y:baz}:7.0 }

A tensor where type is specified explicitly with a single indexed dimension x representing a vector:

tensor<float>(x[3]):{ {x:0}:3.0, {x:1}:5.0, {x:2}:7.0 }

A tensor with a type using the default value type (double) and quoted labels:

tensor(key{}):{ {key:'key.1'}:3.0, {key:'key 2'}:5.0, {key:"key's"}:7.0 }

Indexed short form

Tensors where all dimensions are indexed can be written as numbers wrapped in square brackets in right dimension adjacent order. This form requires an explicit tensor type.

Brackets must either be nested according to the structure of the type, where values in dimensions to the right are closer than dimensions on the left, or alternatively all values may be given (in the same order) as a flat array (the latter not supported in expressions).

Indexed short form examples:

A float 1d tensor in indexed form:

tensor<float>(x[3]):[3.0, 5.0, 7.0]

A matrix in indexed form. Since the values for the right-most dimension (y) are adjacent, the value 3 is here assigned to the cell {x:0,y:2}:

tensor<float>(x[2],y[3]):[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]

The dimension order here is given by the order in which the dimensions are specified, not in the natural (alphabetic) order inherent in the tensor type. Since inner brackets can be omitted, the above is equivalent to

tensor<float>(x[2],y[3]):[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]

Binary hex format

Tensors representing binary data (indexed tensors with int8 cell value type) may be represented using a hexadecimal representation, similar to the corresponding JSON tensor feed format.

Mapped short form

Tensors with a single mapped dimension can be written by specifying just the label in that implicit dimension instead of a full address map. This form requires a type to be specified.

Map short form example:

tensor<float>(key{}):{ key1:1.0,
                       key2:2.0 }

Mixed short form

Tensors with a single mapped dimension and one or more indexed dimensions can be written by specifying the mapped dimension in the map short form and the values of each dense subspace on the indexed short form. This form requires a type to be specified.

Mixed short form example:

A map of matrices:

tensor<float>(key{},x[2],y[3]):{ key1:[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]],
                                 key2:[[1.1, 2.1, 3.1], [4.1, 5.1, 6.1]] }

Tensor functions

Tensor functions are listed in the expressions documentation.

Tensor rank features

The following rank features can be used to refer to or create tensors in ranking expressions. The tensors can come from the document, the query or a constant in the application package:

• attribute(tensor_attribute)
• query(tensor_feature)
• constant(tensor_constant)
• tensorFromWeightedSet(source, dimension)
• tensorFromLabels(source, dimension)

Use the following reference documentation on how use tensors in documents:

• Tensor field in schema
• Document JSON Format