Query Profiles

A query profile is a named set of query parameters, defined in the application package. When a query request names a profile, the parameters in that profile are applied to the request. This has two main benefits:

Clients send only the profile name, instead of having to manage and send every parameter a use case needs.
The parameters for a use case can be changed by redeploying the application, without changing any clients.

Query profiles also make it easy to treat different groups of queries differently. For example, use them to run bucket tests, where a part of the traffic gets experimental settings, or to vary behavior by customer, user type, region, or frontend.

This guide explains how to create and use query profiles. For the full syntax, see the query profile reference.

Using a Query Profile

A query profile is an XML file that lists parameter names and their values:

<query-profile id="MyProfile">
    <field name="hits">20</field>
    <field name="maxHits">2000</field>
    <field name="unique">merchantid</field>
</query-profile>

See the Query API reference for a list of the built-in query properties.

Important: Use full property names in query profiles. Aliases such as input.query(...) are only supported in requests and in programmatic lookup.

To deploy a query profile:

Create a file for the profile in the format above, named [my-profile-name].xml, e.g. MyProfile.xml (replace any / in the name with _).
Put the file in the search/query-profiles directory in the application package root.
Redeploy the application package.

You can add any number of query profile files to this directory. If a query profile contains errors, such as invalid syntax or infinite reference loops, deployment fails.

To use a query profile in a query request, pass its name in the queryProfile parameter:

queryProfile=MyProfile

If the request does not specify a query profile, the profile named default is used. If there is no default profile, no profile is used. If queryProfile is set but does not match an existing profile, an error is returned. For example, this sets a timeout of 200 ms for all queries that don't specify a query profile:

$ cat search/query-profiles/default.xml

<query-profile id="default">
    <field name="timeout">0.2</field>
</query-profile>

Parameter values, whether they come from a query profile or from the request, are available as query properties. To look up a value in a Searcher component, use:

query.properties().get("myVariable")

Property names are case-sensitive.

Example

This query profile defines a YQL query that filters on city using the IN operator, with the list of cities inserted by substitution of the cities parameter:

<query-profile id="city-filter">
    <field name="yql">select * from restaurant where userQuery() and city in (%{cities})</field>
    <field name="cities">""</field>
</query-profile>

The IN operator requires the field to use match: word:

field city type string {
    indexing: summary | index
    match:    word
}

A query that passes the cities to substitute into the YQL string:

$ vespa query \
  queryProfile=city-filter \
  cities='"berlin","paris"' \
  query='what the user typed'

The application can then use the city-filter profile when the user has selected cities, and a no-filter profile otherwise:

<query-profile id="no-filter">
    <field name="yql">select * from restaurant where userQuery()</field>
</query-profile>

$ vespa query \
  queryProfile=no-filter \
  query='what the user typed'

Overrides

The values in a query profile are used as if they were set directly in the request. If a parameter is set both in the request and in the profile, the request value takes precedence by default. To make a profile value take precedence instead, set the overridable attribute of the field to false:

<query-profile id="default">
    <field name="timeout" overridable="false">0.2</field>
</query-profile>

Nested Structure

To give structure to the parameters, a query profile value can be a reference to another query profile instead of a string. The referenced profile then acts as a map (or a struct, if types are used) within the referencing profile. The parameter names in the nested profile are prefixed by the name of the referencing field and a dot. For example:

<query-profile id="MyProfile">
    <field name="hits">10</field>
    <field name="unique">merchantid</field>
    <field name="user"><ref>MyUserProfile</ref></field>
</query-profile>

Where the referenced profile might look like:

<query-profile id="MyUserProfile">
    <field name="age">20</field>
    <field name="profession">student</field>
</query-profile>

A query that uses MyProfile now gets these parameters:

hits=10
unique=merchantid
user.age=20
user.profession=student

References can be nested to any depth, producing parameter names with multiple dots. Like any other parameter, a dotted parameter can be overridden in the request by using its full dotted name.

The request can also set which profile a reference points to. This lets the request choose not only the top-level profile, but also any of the nested profiles. For example, this request replaces the user reference in the example above with another profile:

queryProfile=MyProfile&user=ref:MyOtherUserProfile

The ref: prefix is required. It tells Vespa to set user to the query profile with the given id, rather than to the string MyOtherUserProfile.

Inheritance

A query profile can inherit one or more other query profiles. This is useful when several use cases share a common set of parameters, and only a smaller set varies between them. To inherit another query profile, reference it in the inherits attribute:

<query-profile id="MyProfile" inherits="MyBaseProfile">
    …
</query-profile>

When this profile is used, the parameters of MyBaseProfile are included exactly as if they were written in this profile.

To inherit multiple profiles, list their ids separated by spaces. The order matters: if a parameter is set in more than one inherited profile, the first value found in a depth-first, left-to-right search is used.

A parameter set in the inheriting profile always overrides the same parameter in an inherited one.

Value Substitution

Query profile values can contain substitution strings of the form %{property-name}. For example:

<query-profile id="MyProfile">
    <field name="message">Hello %{world}!</field>
    <field name="world">Earth</field>
</query-profile>

Looking up message returns Hello Earth!.

Global resolution

A substitution string is replaced by the value of query.properties().get("property-name") at the time of the lookup. This means the substituted value can come from a variant, an inherited profile, a value set at run time, or a referenced query profile. Details:

No substitution is performed in values set at run time.
If the referenced value is null, it is replaced by the empty string.
Unclosed substitutions cause an error at deploy time, but unknown properties do not, since they may be set at run time. If they are not, they are replaced by the empty string.
Recursive substitution works as expected, but there is no loop detection.

Local substitution

To substitute a value from the same query profile (or variant), prefix the property name with a dot:

<query-profile id="MyProfile">
    <field name="message">Hello %{.world}!</field>
    <field name="world">Earth</field>
</query-profile>

Local substitutions are verified at deploy time, and deployment fails if the referenced property is not found.

Query Profile Variants

Sometimes the values in a query profile should depend on the values of some parameters in the request. For example, the values may depend on the region the request comes from (region), the kind of device (model), and the bucket the request belongs to (bucket).

All of this can be expressed in a single query profile, by defining nested variants of the profile for the relevant combinations of request values. The dimensions element names the request parameters to vary over, and the for attribute of each nested profile gives the values it applies to. A complete example:

<query-profile id="multiprofile1"> <!-- A regular profile may define "virtual" children within itself -->

    <!-- Names of the request parameters defining the variant profiles of this. Order matters as described below.
         Each individual value looked up in this profile is resolved from the most specific matching virtual
         variant profile -->
    <dimensions>region,model,bucket</dimensions>

    <!-- Values may be set in the profile itself as usual, this becomes the default values given no matching
         virtual variant provides a value for the property -->
    <field name="a">My general a value</field>

    <!-- The "for" attribute in a child profile supplies values in order for each of the dimensions -->
    <query-profile for="us,nokia,test1">
        <field name="a">My value of the combination us-nokia-test1-a</field>
    </query-profile>

    <!-- Same as [us,*,*]  - trailing "*"'s may be omitted -->
    <query-profile for="us">
        <field name="a">My value of the combination us-a</field>
        <field name="b">My value of the combination us-b</field>
    </query-profile>

    <!-- Given a request which matches both the below, the one which specifies concrete values to the left
         gets precedence over those specifying concrete values to the right
         (i.e the first one gets precedence here) -->
    <query-profile for="us,nokia,*">
        <field name="a">My value of the combination us-nokia-a</field>
        <field name="b">My value of the combination us-nokia-b</field>
    </query-profile>
    <query-profile for="us,*,test1">
        <field name="a">My value of the combination us-test1-a</field>
        <field name="b">My value of the combination us-test1-b</field>
    </query-profile>

</query-profile>

Variants and Inheritance

Variants can be defined at several levels in an inheritance hierarchy. The variant dimensions are inherited from parent to child with the usual precedence rules (depth-first, left-to-right), so a parent profile can define the dimensions, and a child the values that vary over them.

Variant resolution within a profile takes precedence over resolution in its parents. For example, a default value for a property in a child profile is chosen over a perfect variant match in an inherited profile.

Variants can specify their own inherited profiles:

<query-profile id="multiprofile1">
    …
    <query-profile for="us,nokia,test1" inherits="parent1 parent2">
        …
    </query-profile>
</query-profile>

Values are then resolved from this profile and its inherited profiles, interleaved in the variant resolution order (by default, most specific first). In decreasing priority:

1.   Value in the highest-priority variant
2.   Value inherited by the highest-priority variant
3.   Value in the next highest-priority variant
4.   Value inherited by the next highest-priority variant
…
n.   Value defined at the top level of the profile
n+1. Value inherited by the profile

Query Profile Types

Query profiles can optionally be type checked. To turn on type checking, reference a query profile type from the query profile. The type lists which parameters the profile can contain, whether other parameters are allowed, and so on.

Reference a query profile type with the type attribute:

<query-profile id="MyProfile" type="MyProfileType">
    …
</query-profile>

The type itself is defined like this:

<query-profile-type id="MyProfileType">
    <field name="age" type="integer"/>
    <field name="profession" type="string"/>
    <field name="user" type="query-profile:MyUserProfileType"/>
</query-profile-type>

This type allows profiles to contain these three parameters, and specifies that user must reference a query profile of type MyUserProfileType.

A type can also make parameters mandatory, disallow parameters that are not listed (strict), inherit other types, and so on. See the query profile reference for the full syntax. If the base profile type is strict, it must inherit a built-in query profile type, see the strict reference documentation.

To deploy a query profile type, add a file named [query-profile-type-name].xml to the search/query-profiles/types directory in the application package.

Query profile types are useful even if query profiles are not used to set any values. Since a type defines the names, types, and structure of the parameters a request can contain, it can be used to define, restrict, and validate query requests. When a typed query profile is used, the built-in Query API parameters are also type checked, so types can restrict which parameters a request can set, or require that some are always set. The built-in parameters are defined in a set of query profile types that are always present, and that application-defined types can inherit and reference. These built-in types are described in the Query API reference.

Path Matching

By adding <match path="true"> to a profile type, profiles of that type are looked up by path matching instead of the default exact name matching. Path matching treats a profile name as a slash-separated path, so a reference to a more specific path matches a profile with a less specific path. The most specific match is used. For example:

Given the query profile names:
  a1
  a1/b1

Then:
  a1/b1/c1/d1  resolves to a1/b1
  a1/b         resolves to a1
  a            does not resolve

This lets you give every client or bucket its own query profile id without creating a separate profile for each one. If a client or bucket later needs its own configuration, you can add a profile for it without changing the client, since it already sends a specific id.

Versioning

Query profiles (and types) can exist in multiple versions at the same time. Wherever a query profile (or type) is referenced by name, a version can be appended after a colon, e.g. MyProfile:1.2.3. The version is resolved: if no version is given, the highest known version is used. If the version is only partially specified, as in MyProfile:1, the highest version starting with 1 is used.

Where a query profile (or type) is defined, the id can include a version after a colon:

<query-profile id="MyProfile:1.2.3">
  …
</query-profile>

In a definition, omitted parts of the version are taken to be 0, so id="MyProfile:1" is the same as id="MyProfile:1.0.0".

A query profile (or type) that specifies a version in its id must have a file name that includes the same version after the name, separated by a dash, e.g. MyProfile-1.2.3.xml.

For more information on versions, see component versioning.

Dump Tool

To inspect how query profiles resolve, dump them offline with vespa-query-profile-dump-tool. Run it without arguments to see usage:

$ vespa-query-profile-dump-tool