Tensor User Guide

This user guide touches upon most of the common steps needed to work with tensors in Vespa, including the following:

  • Setting up tensor fields in search definitions.
  • Feeding tensors to Vespa.
  • Querying Vespa with tensors.
  • Ranking with tensors.
  • Constant tensors.
  • Tensor Java API.
  • Some common use cases.

Most of the contents discussed here is demonstrated in the tensor sample app, and in the following relevant links will be provided into that application.

For a quick introduction to what tensors are about, please refer to Introduction to working with tensors in Vespa. Also refer to the tensor reference guide for more details.

Tensor document fields

In typical use, each document would contain one or more tensor fields to be used during ranking. For example:

field tensor_attribute type tensor(x{}) {
    indexing: attribute | summary
    attribute: tensor(x{})
}

Link to sample app

This sets up a tensor field called tensor_attribute. Each tensor requires a type, and here the type defines that the tensor contains one sparse dimensions, x (a sparse vector). Tensors can't be used in searching, only during ranking.

For more details on setting up tensor fields, please refer to the tensor field in the search definition documentation. For more details on tensor types, refer to the tensor type reference.

Feeding tensors to Vespa

After setting up the tensor fields in the search definition, feeding tensors are fairly straight forward. Basically, you have two options.

  • Feed tensors directly using the tensor JSON format.
  • Convert some other field, for instance a vector of floats, to a tensor during document processing using the tensor Java API.

To convert to tensors during document processing, please refer to the tensor Java API.

To feed to the tensor_attribute field we defined above, we use the JSON format:

{
    "fields": {
        "tensor_attribute": {
            "cells": [
                { "address" : { "x" : "0" }, "value": 1.0 },
                { "address" : { "x" : "1" }, "value": 2.0 },
                { "address" : { "x" : "2" }, "value": 3.0 },
                { "address" : { "x" : "3" }, "value": 5.0 }
            ]
        }
    }
}

Link to sample app

The field x is sparse, so the indices can be any textual value as can be seen in the JSON above. For dense dimensions, the indices are required to be numeric. For more details about the JSON format, please refer to the tensor JSON format in the document JSON put format documentation.

Querying Vespa with tensors

Typically, we would like to pass a tensor down with the query to be used during ranking. This tensor can either be supplied in the actual query string directly, or constructed from some other data or data source. In the latter case, please refer to the tensor Java API for details on how to construct tensors programmatically.

A tensor can be constructed directly from the tensor literal form. The corresponding literal form of the tensor above is:

{{x:0}: 1.0, {x:1}: 2.0, {x:2}: 3.0, {x:3}: 5.0}

This can be used in a query string by url encoding it:

http://host:port/search/?query=sddocname:music&tensor=%7B%7Bx%3A0%7D%3A1.0%2C%7Bx%3A1%7D%3A2.0%2C%7Bx%3A2%7D%3A3.0%2C%7Bx%3A3%7D%3A5.0%7D

To use this tensor to ranking, there are two things you need to do:

  • Add the tensor to the query as a rank feature.
  • Define the tensor type of the rank feature.

To add the tensor to the query as a rank feature, you need to add some code to a custom searcher as follows:

public class ExampleTensorSearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        Object tensorProperty = query.properties().get("tensor");
        if (tensorProperty != null) {
            Tensor tensor = Tensor.from(tensorProperty.toString());
            query.getRanking().getFeatures().put("query(tensor)", tensor);
        }
        return execution.search(query);
    }
}

Link to sample app

This grabs the value of the tensor query parameter, and constructs a com.yahoo.tensor.Tensor object directly from the value. It then adds this object to the query as a rank feature. You can also create the Tensor object programmatically. Please refer to the tensor Java API for details.

Vespa needs to know the type of the query(tensor) rank feature for it to be used in ranking. This is defined using a query profile type. As explained there, we create a new file search/query-profiles/types/root.xml in the application package with the following content:

<query-profile-type id="root" inherits="native">
    <field name="ranking.features.query(tensor)" type="tensor(x{})" />
</query-profile-type>

Link to sample app

This tells the ranking backend that the rank feature named query(tensor) will have type tensor(x{}), enabling the backend to effectively compile expressions that use this feature.

Ranking with tensors

Tensors are used during ranking to modify a document's rank score given the query. Typical operations are dot products between tensors of order 1 (vectors), or matrix products between tensors of order 2 (matrices). Tensors are used in rank expressions as rank features. Above, we have defined two rank features:

  • attribute(tensor_attribute): the tensor associated with the document.
  • query(tensor): the tensor sent with the query.

These can be used in rank expressions, with the caveat that the final rank score of a document must resolve into a single double value. An example is the following:

rank-profile dot_product {
    first-phase {
        expression: sum(query(tensor)*attribute(tensor_attribute))
    }
}

Link to sample app

This takes the product of the query tensor and the document tensor, and sums all fields thus resolving into a single value which is used as the rank score. In the case above, the value is 39.0.

There are some ranking functions that are specifically used for tensors:

map(tensor, f(x)(...)) Returns a new tensor with the lambda function defined in f(x)(...) applied to each cell.
reduce(tensor, aggregator, dim1, dim2, ...) Returns a new tensor with the aggregator applied across dimensions dim1, dim2, etc. If no dimensions are specified, reduce over all dimensions.
join(tensor1, tensor2, f(x,y)(...)) Returns a new tensor constructed from the natural join between tensor1 and tensor2, with the resulting cells having the value as calculated from f(x,y)(...), where x is the cell value from tensor1 and y from tensor2.

These primitives allow for a great deal of flexibility when combined. The above rank expression is equivalently:

rank-profile dot_product {
    first-phase {
        expression {
            reduce(
                join(
                    query(tensor),
                    attribute(tensor_attribute),
                    f(x,y)(x * y)
                ),
                sum
            )
        }
    }
}

...and represents the general dot product for tensors of any order. Details about tensor ranking functions including lambda expression and available aggregators can be found in the tensor reference documentation. More examples of tensor expression can be found in the tensor introduction.

Constant tensors

In addition to document tensors and query tensors, Vespa has the feature of distributing tensors along with the application package. This is useful for the case when you have constant tensors to be used in ranking expressions, for instance when you have machine learned models.

The following example shows how to define a constant tensor:

constant tensor_constant {
    file: constants/constant_tensor_file.json
    type: tensor(x{})
}

Link to sample app

This defines a new tensor rank feature with the type as defined and the contents distributed with the application package in the file constants/constant_tensor_file.json. The format of this file is the tensor JSON format.

To use this tensor in a rank expression, encapsulate the constant name with constant(...):

rank-profile use_constant_tensor {
    first-phase {
        expression: sum(query(tensor) * attribute(tensor_attribute) * constant(tensor_constant))
    }
}

Link to sample app

The above expression combines three tensors: the query tensor, the document tensor and a constant tensor. For more details, please refer to the tensor constant documentation in the search definition documentation.

Tensor Java API

The Tensor API javadoc can be found here.

Common Use Cases

In the following section we describe some common use cases that can be solved using tensor operations.

Dot Product between query and document vectors

Assume we have a set of documents where each document contains a vector of size 4. We want to calculate the dot product between the document vectors and a vector passed down with the query and rank the results according to the dot product score.

The following sd-file defines an attribute tensor field with a tensor type that has one indexed dimension x of size 4. In addition we define a rank profile that calculates the dot product.

search example {
  document example {
    field document_vector type tensor(x[4]) {
      indexing: attribute | summary
      attribute: tensor(x[4])
    }
  }
  rank-profile dot_product {
    first-phase {
      expression: sum(query(query_vector)*attribute(document_vector))
    }
  }
}

The tensor to pass down with query is defined in a query profile type with the same tensor type as the field in the document:

<query-profile-type id="myProfileType">
  <field name="ranking.features.query(query_vector)" type="tensor(x[4])" />
</query-profile-type>

Example document with the vector [1.0, 2.0, 3.0, 5.0]:

[
  { "put": "id:example:example::0", "fields": {
      "document_vector" : {
        "cells": [
          { "address" : { "x" : "0" }, "value": 1.0 },
          { "address" : { "x" : "1" }, "value": 2.0 },
          { "address" : { "x" : "2" }, "value": 3.0 },
          { "address" : { "x" : "3" }, "value": 5.0 }
        ]
      }
    }
  }
]

Example query set in a searcher with the vector [1.0, 2.0, 3.0, 5.0]:

public Result search(Query query, Execution execution) {
    query.getRanking().getFeatures().put("query(query_vector)",
        Tensor.Builder.of(TensorType.fromSpec("tensor(x[4])")).
        cell().label("x", 0).value(1.0).
        cell().label("x", 1).value(2.0).
        cell().label("x", 2).value(3.0).
        cell().label("x", 3).value(5.0).build());
    return execution.search(query);
}

Matrix Product between 1d vector and 2d matrix

Assume we have a 3x2 matrix represented in an attribute tensor field document_matrix with a tensor type tensor(x[3],y[2]) with content:

{ {x:0,y:0}:1.0, {x:1,y:0}:3.0, {x:2,y:0}:5.0, {x:0,y:1}:7.0, {x:1,y:1}:11.0, {x:2,y:1}:13.0 }
Also assume we have 1x3 vector passed down with the query as a tensor with type tensor(x[3]) with content:
{ {x:0}:1.0, {x:1}:3.0, {x:2}:5.0 }
that is set as query(query_vector) in a searcher as specified in query feature.

To calculate the matrix product between the 1x3 vector and 3x2 matrix (to get a 1x2 vector) use the following ranking expression:

sum(query(query_vector) * attribute(document_matrix),x)
This is a sparse tensor product over the shared dimension x, followed by a sum over the same dimension.