Ranking Expressions and Features

Read the ranking introduction first. This guide helps using ranking expressions and rank features, find guides and examples below.

Ranking expressions

Vespa uses ranking expressions to rank documents matching a query, computing a rank score per document. A ranking expression is stored in a rank profile.

Ranking expressions are mathematical functions. The function may contain anything from a single reference to a built-in feature to a machine-learned model. Ranking expressions support the usual operators and functions, as well as an if function - enabling decision trees and conditional business logic. They support a comprehensive set of tensor functions, which allows expressing machine-learned functions such as deep neural nets. Refer to multivalue query operators for details on using dot products, tensors and wand.

Ranking is most often the resource driver - this is where is application's logic is imlpemented. Use two-phase ranking to optimize, using an inexpensive first-phase ranking to eliminate the lowest ranked candidates, then focus the resources on a strong second-phase ranking.

Ranking expressions can be hand written - works well if the ranking is well defined enough to be easily mappable into a ranking expression. Alternatively, make the ranking expression automatically using machine learning. Ranking expressions can be large, and can be imported using file:filename.

Rank features

The primitive values used in ranking expressions are called rank features. Rank features can be tensors, multivalue fields or scalars, and one of:

  • Constants set in the application package
  • Values sent with the query or set in the document
  • Computed by Vespa from the query and the document to provide information about how well the query matched the document
Vespa's rank feature set contains a large set of low level features, as well as some higher level features. If automated training is used, all features can often just be handed to the training algorithm to let it choose which ones to use. Depending on the algorithm, it can be a good idea to leave out the unnormalized features to avoid spending learning power on having to learn to normalize these features and determine that they really represent the same information as some of the normalized features.

Include default rank features in query results by adding ranking.listFeatures to the query. This is useful for tasks like recording the rank feature values for automated training - learn more in the tutorial. If more rank features than is available in the default set is wanted, they can be added to the set in the rank profile:

rank-features: feature1 feature2 …
It is also possible to control which features to dump - add this to the rank profile:
ignore-default-rank-features
This will make the explicitly listed rank features the only ones dumped when requesting rankfeatures in the query.

Normalization

The rank features provided includes both features normalized to the range 0-1, and un-normalized features like counts and positions. Whenever possible, prefer the normalized features. They capture the same information (and more), but are easier to use because they can be combined more easily with other features. In addition, try to write ranking expressions such that the combined rank score is also normalized, for example by taking averages not sums. The resulting normalized rank scores makes it possible to implement relevance based blending, search assistance triggering when there are no good hits, and so on.

Configuration

Some features, most notably the fieldMatch features, has configuration parameters that enables the feature calculation to be tweaked per field for performance or relevance. Feature configuration values are set by adding to the rank profile:

rank-properties {
    featureName.configurationProperty: "value"
}
The values are set per field, like:
rank-properties {
    fieldMatch(title).maxAlternativeSegmentations: 10
    fieldmatch(title).maxOccurrences: 5
    fieldMatch(description).maxOccurrences: 20
}
Refer to the rank feature configuration reference.

Feature contribution functions

Vespa ranking features are linear. For example, the earliness feature is 0 if the match is at the very end of the field, 1 if the match is at the very start of the field, and 0.5 if the match is exactly in the middle of the field. In many cases, the contribution of a feature should not be linear with its "goodness". For example, earliness could decay quickly in the beginning and slowly at the end of the field. This from the intuition that it matters more if the match is of the first or the twentieth word in the field, but it doesn't matter as much if the match is at the thousands or thousand-and-twentieths.

To achieve this, pass the feature value through a function which turns the line into a curve matching the intent. This is easiest with normalized fields. The function begins and ends in the same point, f(0)=0 and f(1)=1, but which curves in between. To get the effect described above, a curve which starts almost flat and ends very steep works - example:

pow(0-x,2)
The second number decides how pronounced the curving is. A larger number will make changes to higher x values even more important relative to the same change to lower x values.

Dumping rank features for specific documents

For a training set containing judgements for certain documents, it is useful to select those documents in the query by adding a term matching the document id, but without impacting the values of any rank features. To do this, add that term with ranked set to false:

select * from mydocumenttype where myidfield contains ([ {"ranked": false} ] "mydocumentid" and ...);

Accessing feature/function values in results

Any feature can be returned in the hit producing it by adding it to the list of summary-features of the rank profile. As all functions are features this allows the result of any computation to be accessible in results. Example:

rank-profile test {

    summary-features: tensor_join join_sum

    function tensor_join() {
        expression: attribute(my_tensor_field) * query(my_query_tensor)
    }

    function join_sum() {
        expression: sum(tensor_join())
    }

}
The results of this functions will be available in the Hits of the result as follows:
import com.yahoo.search.result.FeatureData;
    ...
    FeatureData featureData = (FeatureData)hit.getField("summaryfeatures");
    Tensor tensor_join_value = featureData.getTensor("rankingExpression(tensor_join)");
    double join_sum_value = featureData.getTensor("rankingExpression(join_sum)");
You can easily do further computation on the returned tensors, such as e.g Tensor larger = tensor_join_value.map((value) -> 3 * value).

The if function and string equality tests

if can be used for other purposes than encoding MLR trained decision trees. One use is to choose different ranking functions for different types of documents in the same search. Ranking expressions are able to do string equality tests, so to choose between different ranking sub-functions based on the value of a string attribute (say, "category"), use an expression like:

if (attribute(category)=="restaurant",…restaurant function, if (attribute(category)=="hotel",…hotel function, …))
This method is also used automatically when multiple schemas are deployed to the same cluster, and all is included in the same query to choose the ranking expression from the correct schema for each document.

By using if functions, one can also implement strict tiering, ensuring that documents having some criterion always gets a higher score than the other documents. Example:

if (fieldMatch(business).fieldCompleteness==1, 0.8+document.distance*0.2,
                                               if (attribute(category)=="shop", 0.6+fieldMatch(title)*0.2,
                                                                                 match*attribute(popularity)*0.6 )
This function puts all exact matches on business names first, sorted by geographical distance, followed by all shops sorted by title match, followed by everything else sorted by the overall match quality and popularity.

Using constants

Ranking expressions can refer to constants defined in a constants clause:

first-phase {
    expression: myConst1 + myConst2
}
constants {
    myConst1: 1.5
    myConst2: 2.5
    ...
}
Constants lists are inherited and can be overridden in sub-profiles. This is useful to create a set of rank profiles that use the same broad ranking but differs by constants values.

For performance, always prefer constants to query variables (see below) whenever the constant values to use can be enumerated in a set of ranking profiles. Constants are applied to ranking expressions at configuration time, and the resulting constant parts of expressions calculated, which may lead to reduced execution cost, especially with tensor constants.

Using query variables

As ranking expressions can refer to any feature by name, one can use query features as ranking variables. These variables can be used for example to allow the query to specify the degree of importance to various parts of a ranking expression, or to quickly search large parameter spaces to find a good ranking, by trying different values in each query. These variables can be assigned default values in the rank profile by adding:

rank-properties {
    query(myvalue): 0.5
}
to the ranking profile. Also, these variables can be overridden in the query by adding:
ranking.features.query(myvalue)=0.1
to the query - see the Query API.

Query tensors must be defined in application package In order to use request tensors one must define the tensor type of the query feature in a query profile type and modify the default query-profile to use the correct query profile type. In the following example the tensor type of the tensor query(user_profile) is defined to have one mapped dimension cat and is of type float. Also see the Sample app user tensor configuration

In search/query-profiles/types/ one need a file named root.xml with the tensor type definitions:

<query-profile-type id="root">
    <field name="ranking.features.query(user_profile)" type="tensor&lt;float&gt;(cat{})" />
</query-profile-type>
and search/query-profiles/default.xml must be updated to specify the correct query profile type:
<query-profile id="default" type="root">
</query-profile>
Without the correct tensor type and definition the passed query(user_profile) is handled as a string rank feature which is converted to a scalar. Refer to query profile field type reference.

Use the tensor literal form to send a tensor - here send user_profile as {{cat:pop}:0.8,{cat:rock}:0.2,{cat:jazz}:0.1}:

ranking.features.query(user_profile)=%7B%7Bcat%3Apop%7D%3A0.8%2C%7Bcat%3Arock%7D%3A0.2%2C%7Bcat%3Ajazz%7D%3A0.1%7D

A query tensor can also be made in a Searcher. The same restriction applies for programmatic generated tensors, their type must be defined in the application package as described in the section above. Set the tensor value using the RankFeatures instance associated with Query instance. This example makes a tensor with a single cell with value 500:

package com.yahoo.example;

import com.yahoo.search.Query;
import com.yahoo.search.Result;
import com.yahoo.search.Searcher;
import com.yahoo.search.searchchain.Execution;
import com.yahoo.tensor.MappedTensor;
import com.yahoo.tensor.TensorType;

public class TensorInQuerySearcher extends Searcher {
    @Override
    public Result search(Query query, Execution execution) {
        //The Tensor type must also be in the application package as described in the above section
        query.getRanking().getFeatures().put("query(tensor_feature)",
            new MappedTensor.Builder(TensorType.fromSpec("tensor<float>(x{})")).cell().label("x", "foo").value(500).build());
        return execution.search(query);
    }
}
Refer to the Tensor Java API for details on how to construct tensors programmatically.

Function snippets

When using machine learned ranking, we are searching a function space which is much more limited than the space of functions supported by ranking expressions. We can increase the space of functions available to MLR because the primitive features used in MLR training do not need to be primitive features in Vespa - they can just as well be ranking expression snippets. If there are certain mathematical combinations of features believed to be useful in an application, these can be pre-calculated from the actual primitive features of Vespa and given to MLR as primitives. Such primitives can then be replaced textually by the corresponding rank expression snippet, before the learned expression is deployed on Vespa.

Vespa supports expression functions. Functions having zero arguments can be used as summary- and rank-features. For example, the function "myfeature":

rank-profile myrankprofile inherits default {
    function myfeature() {
      expression: fieldMatch(title).completeness * pow(0 - fieldMatch(title).earliness, 2)
    }
}
becomes available as a feature as follows:
summary-features {
    myfeature
}

Tracking relevance variations over time

Vespa comes with a few simple metrics for relevance that enables applications to see how relevance changes over time, either as a result of changes to how relevance is computed, changes to query construction, changes to the content ingested, or as a result of changing user behavior.

The relevance metrics are relevance.at_1, relevance.at_3 and relevance.at_10. See metrics for more information.

Examples

If the user is underage, assign 0 to adult content and use the average of match quality in the title field and popularity among kids. If the user is not, use the match quality in the title field:

if ( query(userage) < 18,
     if ( attribute(adultness) > 0.1, 0 , (fieldMatch(title)+attribute(kidspopularity)) / 2 ),
     fieldMatch(title) )
Use a weighted average of the match quality in some fields, multiplied by 1-exp of the document age:
( 10*fieldMatch(title) + 5*fieldMatch(description) +
  7*attributeMatch(tags).normalizedWeight ) /22 * ( 1 - age(creationtime) )