Sorting Reference

A sorting specification in a query consists of one or more sorting expressions. Each sorting expression is an optional sort order followed by an attribute name, a function over an attribute, or a rank feature. Multiple expressions are separated by a single SPACE character.

Using more than one sort expression will give you multilevel sorting. In this case, the most significant expression is the first, while subsequent expressions detail sorting within groups of equal values for the previous expression.

Sorting       ::= SortExpr ( ' ' SortExpr )*
SortExpr      ::= [SortOrder] ( SortObject | MissingExpr | FeatureExpr )
FeatureExpr   ::= FEATURE '(' FeatureName ')'
MissingExpr   ::= MISSING '(' SortObject ',' MissingPolicy ')'
MissingPolicy ::= FIRST |
                  LAST |
                  AS ',' MissingValue
SortObject    ::= SortAttribute | Function
Function      ::= LOWERCASE '(' SortAttribute ')' |
                  RAW '(' SortAttribute ')' |
                  UCA '(' SortAttribute [ ',' Locale [ ',' Strength] ] ')'
MISSING       ::= 'missing'
FIRST         ::= 'first'
LAST          ::= 'last'
AS            ::= 'as'
LOWERCASE     ::= 'lowercase'
UCA           ::= 'uca'
RAW           ::= 'raw'
FEATURE       ::= 'feature'
FeatureName   ::= [A-Za-z_][A-Za-z0-9_]*             /* A name listed in the rank profile's sort-features */
Locale        ::= An identifier following unicode locale identifiers, fx 'en_US'.
Strength      :: 'PRIMARY' | 'SECONDARY' | 'TERTIARY' | 'QUATERNARY' | 'IDENTICAL'
SortOrder     ::= '+' | '-'
SortAttribute ::= ID | ID.key | ID.value            /* A valid attribute name, with key/value appended for maps */
MissingValue  ::= QuotedString | Token
Token         ::= TokenChar*
TokenChar     ::= any non Delimiter
QuotedString  ::= '"' ( StringChar | '\' QuotedChar )* '"'
StringChar    ::= any non QuotedChar
Delimiter     ::= ' ' | ',' | '(' | ')' | QuotedChar
QuotedChar    ::= '\' | '"'

See Geo search for sorting by distance. Refer to YQL Vespa reference for how to set sorting parameters in YQL.

Sort order

+ denotes ascending sorting order, while - gives descending order. Ascending order is defined as lowest values first for numerical attributes. Strings are sorted according to the sort function chosen. Descending order is the reverse of ascending order.

Note: + in query URLs must be encoded as %2B - for consistency, - can be encoded as %2D.

Default sort order

If +/- is omitted, the default is used, either the system-wide default of + or any override in schema. Default sort order is + or ascending, except for [rank] or the special builtin [relevance], which has - or descending. feature(name) defaults to ascending, independent of attribute sorting settings in the schema.

Sort attribute

The sorting attribute in a sort expression is the name of an attribute in the index. Attribute names will often be the same as field names. In the schema, an attribute is indicated by the indexing language fragment for a field having an attribute statement.

When sorting only on attributes, it is recommended to use the built-in unranked rank-profile. This allows the search kernel to execute the query significantly faster than execution the ranking framework for many hits and then finally ignore this score and sort by the specified sorting specification.

Multivalue sort attribute

When sorting on a multivalue attribute (map, array, or weightedset) one of the values for the document is selected to be used for sorting. Ascending sort order uses the lowest value while descending sort order uses the highest value. See the missing policies section for how a document without values in the attribute is treated.

Sort function

Refer to function.

Sorting by rank feature

Use feature(name) to sort by a numeric rank feature or a rank-profile function without arguments. The name must be listed in sort-features in the rank profile selected by the query's ranking parameter. Features with arguments or named outputs must be wrapped in a function, as shown for bm25(title) below.

This example assumes an indexed title field with enable-bm25, and numeric year and price attributes:

rank-profile sorting {
    function decade() {
        expression: floor(attribute(year) / 10)
    }
    function title_bm25() {
        expression: bm25(title)
    }
    first-phase {
        expression: title_bm25
    }
    sort-features {
        decade
        title_bm25
        nativeRank
    }
}

Sort by decade ascending, then by BM25 score descending, then by price ascending:

vespa query \
  'yql=select * from sources * where title contains "alpha"' \
  'ranking=sorting' \
  'sorting=+feature(decade) -feature(title_bm25) +price'

The equivalent YQL order by clause uses the function annotation on each feature name:

select * from sources * where title contains "alpha"
order by {function: "feature"}decade asc, {function: "feature"}title_bm25 desc, price asc

Set ranking=sorting for this query as well. Feature sort keys can appear at any level, mixed with attributes and special sorting attributes.

Feature sorting and ranking

Selecting a sort feature does not itself activate first- or second-phase ranking. If the query does not otherwise require ranking, hits have relevance: 0, even when sorting by a feature such as title_bm25. Include [relevance] (or its alias [rank]) to compute relevance scores and use them in sorting. For example:

+feature(decade) -[relevance]

Sort features are computed during matching on the content nodes for each hit considered for sorting. Only features selected by the query are evaluated for sorting; unused sort-features entries only add deployment-time compilation cost but don't add to query time. For sorting directly by an attribute value, use attribute sorting to avoid feature evaluation overhead.

Declaring a sort feature does not return its value in the response. To return feature values, also list them in match-features or summary-features.

Feature sorting restrictions

All schemas searched must use indexed mode, and the selected rank profile in each schema must allow the requested features. Streaming search does not support feature sorting.
Each feature must produce a numeric scalar, represented as a double. Reduce tensor expressions to a scalar before using them as sort features.
feature(name) requires a bare identifier matching [A-Za-z_][A-Za-z0-9_]*. Expressions such as feature(bm25(title)) and feature(foo.output) are invalid.
Missing policies cannot be applied to feature sorting. A feature produces a value for each hit; handle missing inputs in its ranking expression. NaN and infinite values are converted to negative infinity, sorting before finite values in ascending order and after them in descending order.
Queries using feature sorting are rejected when global-phase reranking is enabled. To use a profile that declares global-phase, set ranking.globalPhase.rerankCount=0 to disable global-phase reranking for the query.

Invalid or disallowed feature names cause a query error. If a content node cannot obtain the requested sort-feature values, it fails the query on that node.

Special sorting attributes

Three special attributes are available for sorting in addition to the index specific attributes:

[relevance]	The document's relevance score for this query. This is the same as the default ordering when no sort specification is given ([rank] is a legacy alias for the same thing).
[source]	The document's source name. This is only relevant when querying multiple sources.
[docid]	The document's identification in the search backend. This will typically give you the documents in indexing order. Keep in mind that this id is unique only to the backend node. The same document might have different id on a different node. The same way a different document might have the same id on another node. This is just intended as a cheap way of getting an almost stable sort order.

These special attributes are most useful as secondary sort expressions in a multilevel sort. This will allow you to sort groups of equal values for the primary expression in either relevancy or indexing order. Without this additional sort expression, the order within each equal group is not deterministic.

Important: In YQL, using order by, the special sorting attributes must be enclosed in quotes.

Missing policies

A document might not have a value in the attribute. One of the following missing policies will then be applied:

Policy	Example	Description
default	`+attr`	If the sort order is ascending and the attribute is single-valued then the document is sorted before any documents with values in the attribute. If the attribute is multi-valued or the sort order is descending then the document is sorted after any documents with values in the attribute.
first	`+missing(attr,first)`	The document is sorted before any documents with values in the attribute.
last	`+missing(attr,last)`	The document is sorted after any documents with values in the attribute.
as	`+missing(attr,as,42)`	The document is sorted as if it had the missing value specified in the sorting specification. If the missing value cannot be converted to the attribute data type then an error is reported (query is aborted for indexed search, parts of the sort spec is ignored for streaming search).

Note that missing policies can be combined with attribute sort functions, e.g. +missing(lowercase(attr),as,"nothing here"). They cannot be combined with feature(...).

Limitations

Attribute only

Sorting directly by a field value requires an attribute. An index or summary field without an associated attribute cannot be used as an attribute sort key. To sort by a computed value, use feature sorting.

Also note that match-phase can be enabled automatically when sorting on attributes. It is not enabled automatically when the primary sort key is feature(...).

Optimization causing incorrect total hit count

When the primary sort key is a single-value numeric attribute with fast-search Vespa will by default activate an optimization which makes delivering sorted results much faster, but with inaccurate total-hit count. To disable this optimization, set the query parameter sorting.degrading to false (in the request or a query profile). See the reference for details. This optimization is not activated when the primary sort key is feature(...).

Examples

Sort by surname in ascending order:

+surname

Sort by surname in ascending order after lowercasing the surname:

+lowercase(surname)

Sort by surname in ascending English US locale collation order.

+uca(surname,en_US)

Sort by surname in ascending Norwegian 'Bokmål' locale collation order:

+uca(surname,nb_NO)

Sort by surname in ascending Norwegian 'Bokmål' locale collation order, but more attributes of a character are used to distinguish. Now it is case-sensitive and 'aa' and 'å' are different:

+uca(surname,nb_NO,TERTIARY)

Sort by surname, with the youngest ones first when the names are equal:

+surname -yearofbirth

Sort in ascending order yearofbirth groups, and sort by relevancy within each group of equal values with the highest relevance first:

+yearofbirth -[relevance]