Self-managedThis content is applicable to self-managed Vespa systems.
Routing
Routing is used to configure the paths that documents and updates written
to Vespa take through the system. Vespa will automatically set up a routing
configuration which is appropriate for most cases, so no explicit routing
configuration is necessary. However, explicit routing can be used in advanced use
cases such as sending different document streams to different document processing
clusters, or through multiple consecutive clusters etc.
There are other, more in-depth, articles on routing:
Use vespa-route
to inspect routes and services of a Vespa application, like in the
example
In Vespa, there is a transport layer and a programming interface that
are available to clients that wish to communicate with a Vespa application.
The transport layer is Message Bus.
Document API is
implemented on top of Message Bus.
Configuring the interface therefore exposes some features available in Message Bus.
Refer to the Vespa APIs and interfaces
for clients using the Document API.
The atoms in Vespa routing are routes and hops.
document-processing is an example of custom document processing, and useful for testing routing.
A route is a sequence of hops
The sequence of hosts, routers, bridges, gateways, and other devices
that network traffic takes, or could take, from its source to its
destination is what is classically termed a route.
As a verb, to route means to determine the link down which to send a packet,
that will minimize its total journey time according to some routing algorithm.
In Vespa, a route is simply a sequence of named hops.
Instead of leaving selection logic to a route,
the responsibility of resolving recipients is given to
the hops' selectors.
A hop can do more or less whatever it wants
to change a message's journey through your application;
it can slightly alter itself by choosing among some predefined recipients,
it can change itself completely by either rewriting or looking up another hop,
or it can even modify the entire route from that branch onwards.
In effect, a route can end up branching at several points along its path,
resulting in complex routes.
As the figure suggests, Message Bus supports
both unicasting
and multicasting -
Message Bus allows for arbitrarily complex routes.
Each node in the above graph represents a Vespa service:
A hop is a point-to-point transmission
In telecommunication, a hop is one step, from one router to the next,
on the path of a packet on an Internet Protocol network.
It is a direct host-to-host connection forming part of the route between
two hosts in a routed network such as the Internet.
In more general terms, a hop is a point-to-point transmission in a series
required to get a message from point A to point B.
With Message Bus the concept of hops was introduced as the smallest
steps of the transmission of a message.
A hop consists of a name that is used by the messaging clients to select it,
a list of recipient services that it may transmit to,
and a selector that is used to select among those recipients.
Unlike traditional hops, in Vespa a hop is a transmission from one sender to many recipients.
Well, the above is only partially true;
it is the easiest way to understand the hop concept.
In fact, a hop's recipient list is nothing more than a configured list of strings
that is made available to all routing policies
that are named in the selector string.
See selection logic below for details.
A hop's recipient is the service name of a Message Bus client that has
been registered in Vespa's service location broker (vespa-slobrok).
These names are well-defined once their derivation logic is understood;
they are "/"-separated sets of address-components whose values
are given by a service's role in the application.
An example of a recipient is:
search/cluster.foo/*/feed-destination
The marked components of the above recipient, /search/cluster.foo/*,
resolves to a host's symbolic name.
This is the name with which a Message Bus instance was configured.
The unmarked component, feed-destination,
is the local name of the running service that the hop transmits to,
i.e. the name of the session created on the running Message Bus instance.
The Active Configuration page in Vespa's administration interface
gives an insight into what symbolic names exist for any given
application by looking at its current configuration subscriptions.
All available Message Bus services use their ConfigId as their host's symbolic name.
See vespa-route for how to inspect this,
or use the config API.
A hop can be prefixed using the special character "?" to force it to behave as if its
ignore-result attribute was configured to "true".
Asterisk
A service identifier may include the special character "*" as an address component.
A recipient that contains this character
is a request for the network to choose any one service that matches it.
Routing policies
A routing policy is a protocol-specific algorithm that chooses among a
list of candidate recipients for a single address component -
see hop description above.
These policies are designed and implemented as key parts of a Message Bus protocol.
E.g. for the "Document" protocol these are what make up the routing behavior for document transmission.
Without policies, a hop would only be able to match verbatim to a recipient,
and thus the only advanced selection logic would be that of the asterisk.
In addition to implementing a selection algorithm,
a routing policy must also implement a merging algorithm that combines the replies
returned from each selected recipient into a single sensible reply.
This is needed because a client does not necessarily know
whether a message has been sent to one or multiple recipients,
and Message Bus guarantees a single reply for every message.
More formally, a routing policy is an arbitrarily large (or small),
named, stand-alone piece of code registered with a Message Bus protocol.
As discussed above,
an instance of a policy is run both when resolving a route to recipients,
and when merging replies.
The policy is passed a RoutingContext object
that pretty much allows it to do whatever it pleases to the route and replies.
The same policy object and the same context object is used for both selection and merging.
When Message Bus is about to route a message, at the last possible time,
it inspects the first hop of the message's route to resolve a set of recipients.
First, all of its policies are resolved.
Second, the output service name is matched to the routing table to see if it maps to another hop or route.
Finally, the message is sent to all chosen recipient services.
Because each policy can select multiple recipients,
this can give rise to an arbitrarily complex routing tree.
There are, of course, safeguards within Message Bus to prevent infinite recursions
due to circular dependencies or misconfiguration.
Note:
It is possible to develop a different protocol with other
policies to run in the application,
but since all of Vespa's component only support the "Document" protocol,
it makes little sense to do so.
1. Resolve Policy Directives
The logic run at this step is actually simple;
as long as the hop string contains a policy directive,
i.e. some arbitrary string enclosed in square brackets,
Message Bus will create and run an instance of that policy for the protocol of the message being routed.
The above hop is probably the simplest hop you will encounter in Vespa;
it has a single policy directive contained in a string
that closely resembles service names discussed above, and it has no recipients.
When resolving this hop,
Message Bus creates an instance of the "DocumentRouteSelector" policy
and invokes its select() method.
The "Distributor" policy will replace its own directive
with a proper distributor identifier,
yielding a hop string that is now an unambiguous service identifier.
This hop has a selector which is nothing more than a single policy directive,
"[DocumentRouteSelector]", and it has two configured recipients,
"search/cluster.music" and "search/cluster.books".
This policy expands the hop to zero, one or two new routes
by replacing its own directive with the content of the recipient routes.
Each of these routes may have one or more hops themselves.
In turn, these will be processed independently.
When replies are available from all chosen recipients,
the policy's merge() method is invoked,
and the resulting reply is passed upwards.
This hop has a selector but no recipients.
The reason for this is best explained in
the routing policies reference,
but it serves as an example of a hop that has no configured recipients.
Notice how the policy directive contains a colon (":")
which denotes that the remainder of the directive is a parameter to the policy constructor.
This policy replaces the whole route of the message
with the set of routes named in the parameter string.
What routing policies are available depends on what protocol is currently running.
As of this version the only supported protocol is "Document".
This offers a set of routing policies discussed
below.
2. Resolve Hop- and Route names
As soon as all policy directives have been resolved,
Message Bus makes sure that the resulting string is,
in fact, a service name and not the name of another hop or route (in that order)
configured for the running protocol. The outcome is either:
The string is recognized as a hop name - The current hop
is replaced by the named one, and processing returns
to step 1.
The string is recognized as a route name - The current
route, including all the hops following this, is replaced by the named one.
Processing returns to step 1.
The string is accepted as a service name - This terminates
the current branch of the routing tree. If all branches are terminated,
processing proceeds to step 3.
Because hop names are checked before route names,
Message Bus also supports a "route:" prefix that forces the remainder of the string
to resolve to a configured route or fail.
3. Send to Services
When the route resolver reaches this point,
the first hop of the message being sent has been resolved to an arbitrarily complex routing tree.
Each leaf of this tree represents a service that is to receive the message,
unless some policy has already generated a reply for it.
No matter how many recipients are chosen, the message is serialized only once,
and the network transmission is able to share the same chunk of memory between all recipients.
As replies to the message arrive at the sender they are handed over to
the corresponding leaf nodes of the routing tree,
but merging will not commence until all leaf nodes are ready.
Route resolving happens just before network transmission, after all resending logic.
This means that if the route configuration changes while there are messages scheduled for resending,
these will adhere to the new routes.
If the resolution of a recipient passed through a hop that was configured
to ignore results,
the network layer will reply immediately with a synthetic "OK".
Example: Reconfigure the default route
Assume that the application requires both search and storage capabilities,
but that the default feed should only pass through to search.
An imaginary scenario for this would be a system where there
is a continuous feed being passed into Vespa with no filtering on spam.
You would like a minimal storage-only cluster that stores a URL blocklist
that can be used by a custom document processor to block incoming documents from offending sites.
Apart from the blocklist and the document processor, add the following:
This overrides the default route to pass through any available
blocklisting document processor before being indexed.
If the document processor decides to block a message,
it must respond with an appropriate ok reply,
or your client software needs to accept whatever error reply you decide to return when blocking.
When feeding blocklist information to storage,
your application need only use the already available storage hop.
See #13193
for a discussion on using default as a name.
The Document API
With the current implementation of Document API running on Message bus,
the configuration of the API implies configuration of the latter.
Most clients will only ever route through this API.
To use the Document API, you need to instantiate a class
that implements the DocumentAccess interface.
At the time of writing only MessageBusDocumentAccess exists,
and it requires a parameter set for creation.
These parameters are contained in an instance of MessageBusDocumentAccessParam
that looks somewhat like the following:
class MessageBusDocumentAccessParams {
String documentManagerConfigId; // The id to resolve to document manager config.
String oosServerPattern; // The service pattern to resolve to fleet controller
// services.
String appConfigId; // The id to resolve to application config.
String slobrokConfigId; // The id to resolve to slobrok config.
String routingConfigId; // The id to resolve to messagebus routing config.
String routeName; // The name of the route to send to.
int traceLevel; // The trace level to use when sending.
class SourceSessionParams {
int maxPending; // Maximum number of pending messages.
int maxPendingSize; // Maximum size of pending messages.
double timeout; // Default timeout in seconds for messages
// that have no timeout set.
double requestTimeoutA; // Default request timeout in seconds, using
double requestTimeoutB; // the equation 'requestTimeout = a * retry + b'.
double retryDelay; // Number of seconds to wait before resending.
};
}
The most obvious configuration parameter is routeName,
which informs the MessageBusDocumentAccess object the
name of the route to use when sending documents and updates.
The second parameter is traceLevel,
which allows a client to see exactly how the data was transmitted.
Note:
Tracing can be enabled on a level from 1-9,
where a higher number means more tracing.
Because the concept of tracing is not exposed by the Document API itself,
its data will simply be printed to standard output when a reply arrives for the sender.
This should therefore not be used in production,
but can be helpful when debugging.
Contained in routing.
Specifies a routing table for a specific protocol.
Attribute
Required
Value
Default
Description
protocol
required
Configure which protocol to use.
Only the protocol document is defined,
so if you define a routing table for an unsupported protocol,
the application will just log an INFO entry that contains the name of that protocol.
Contained in routingtable.
Specifies a route for a message to its destination through a set of intermediate hops.
If at least one hop in a route does not exist,
the application will fail to start and issue an error that contains the name of that hop.
Attribute
Required
Value
Default
Description
name
required
Route name.
hops
required
A whitespace-separated list of hop names, where each name must be a valid hop.
Contained in routingtable.
Specifies a single hop that can be used to construct one or more routes.
A hop must have a name that is unique within the routing table to which it belongs.
A hop contains a selector string and a list of recipient sessions.
Attribute
Required
Value
Default
Description
name
required
Hop name.
selector
required
Selector string.
ignore-result
optional
If set to true, specifies that the result of routing through that hop should be ignored.
Contained in hop.
Specifies a recipient session of a hop.
Attribute
Required
Value
Default
Description
session
required
This attribute must correspond to a running instance of a
service that can be routed to.
All session identifiers consist of a location part and a
name. A search node is always given a session name on the form
search/cluster.name/g#/r#/c#/feed-destination,
whereas a document processor service is always named
docproc/cluster.name/docproc/#/feed-processor.
Contained in routing.
Specifies a set of services available for a specific protocol.
At the moment the only supported protocol is document.
The services specified are used by the route verification step
to allow hops and routes to reference services known to exist,
but that can not be derived from services.xml.
Attribute
Required
Value
Default
Description
protocol
required
Configure which protocol to use. Only the protocol document is defined.
This article contains detailed descriptions of the behaviour of all routing policies available in Vespa.
The Document protocol is currently the only Message Bus protocol supported by Vespa.
Furthermore, all routing policies that are part of this protocol
share a common code path for merging replies.
The policies offered by the protocol are:
Content - Selects a content cluster distributor based on system state
MessageType - Selects a next hop based on message type
Extern - Selects a recipient by querying a remote Vespa application
LocalService - Selects a recipient based on ip address
RoundRobin - Selects one from the configured recipients in round-robin order
SubsetService - Selects only among a subset of all matching services
LoadBalancer - A round-robin policy that chooses between the recipients
by generating a weight according to their performance
Common Document merge() logic
The shared merge logic of most Document routing policies is an attempt to
do the "right" thing when merging multiple replies into one.
It works by first stepping through all replies, storing their content as either:
OK replies,
IGNORE replies, or
ERROR replies
If at least one ERROR reply is found, return a new reply that contains all the errors of the others.
If there is at least one OK reply, return the first OK reply,
but transfer all feed information from the others to this
(this is specific data for start- and end-of-feed messages).
Otherwise, return a new reply that contains all the IGNORE errors. Pseudocode:
for each reply, do
if reply has no errors, do
store reply in OK list
else, do
if reply has only IGNORE errors
copy all errors from reply to IGNORE list
else, do
copy all errors from reply to ERROR list
if ERROR list is not empty, do
return new reply with all errors
else, do
if OK list is not empty, do
return first reply with all feed answers
else, do
return new reply with all IGNORE errors
Routing policies reference
Policy
Description
AND
This is a mostly a convenience policy
that allows the user to fork a message's route to all configured recipients.
It is not message-type aware, and will simply always select all recipients.
Replies are merged according to the shared logic.
The optional string parameter is parsed as a space-separated list of hops.
Configured recipients have precedence over parameter-given recipients,
although this is likely to be changed in the future.
DocumentRouteSelector
This policy is responsible for selecting among the policy's recipients
according to the subscription rules defined by a content cluster's documents element
in services.xml.
If the "selection" attribute is set in the "documents" element,
its value is processed as a document select string,
and run on documents and document updates to determine routes.
If the "feedname" attribute is set, all feed commands are filtered through it.
The recipient list of this policy is required to map directly to route names.
E.g. if a recipient is "search/cluster.music", and a message is appropriate according to the selection criteria,
the message is routed to the "search/cluster.music" route.
If the route does not exist, this policy will reply with an error.
In short, this policy selects one or more recipient routes based on document content and configured criteria.
If more than one route is chosen, its replies are merged according to the shared logic.
This policy does not support any parameters.
The configuration for this is "documentrouteselectorpolicy" available from config id "routing/documentapi".
Important:
Because GET messages do not contain any document on which to run the selection criteria,
this policy returns an IGNORED reply that the merging logic processes.
You can see this by attempting to retrieve a document from an application that does not have a content cluster.
Content
This policy allows you to send a message to a content cluster.
The policy uses a system state retrieved from the cluster in question
in conjunction with slobrok information to pick the correct distributor for your message.
In short; use this policy when communicating with document storage.
This policy supports multiple parameters, up to one each of:
cluster
The name of the cluster you want to reach. Example: cluster=mycluster
config
A comma-separated list of config servers or proxies
you want to use to fetch configuration for the policy.
This can be used to communicate with other clusters than the one you're currently in.
Example: config=tcp/myadmin1:19070,tcp/myadmin2:19070
Separate each parameter with a semicolon.
MessageType
This policy will select the next hop based on the type of the message.
You configure where all messages should go (defaultroute).
Then you configure what messages types should be overridden and sent to alternative routes.
It is currently only used internally by vespa when using the
content element.
Extern
This policy implements the necessary logic to communicate with an external Vespa application
and resolve a single service pattern using that other application's slobrok servers.
Keep in mind that there might be some delay from the moment this policy is initially created
and when it receives the response to its service query,
so using this policy might cause a message to be resent a few times until it is resolved.
If you disable retries, this policy might cause all messages to fail for the first seconds.
This policy uses its parameter for both the address of the extern slobrok server to connect to,
and also the pattern to use for querying.
The parameter is required to be on the form <spec>;<service>,
where spec is a comma-separated list of slobrok connection specs
on the form "tcp/hostname:port", and service is a service running on the remote Vespa application.
Important:
The remote application needs to have a version of both message bus and the document api
that is binary compatible with the application sending from.
This can be a problem even between patch releases,
so keep the application versions in sync when using this policy.
LocalService
This policy is used to select among all matching services,
but preferring those running on the same host as the current one.
The pattern used when querying for available services is the current one,
but replacing the policy directive with an asterisk.
E.g. the hop "docproc/cluster.default/[LocalService]/chain.default" would prefer local services
among all those that match the pattern "docproc/cluster.default/*/chain.default".
If there are multiple matching services that run locally,
this policy will do simple round-robin load balancing between them.
If no matching services run locally,
this policy simply returns the asterisk as a match to allow the underlying network logic
to do load balancing among all available.
This policy accepts an optional parameter which overrides the local hostname.
Use this if you wish the hop to prefer some specific host.
Important:
There is no additional logic to replace other policy directives with an asterisk,
meaning that if other policies directives are present in the hop string
after "[LocalService]",
no services can possibly be matched.
RoundRobin
This policy is used to select among a configured set of recipients.
For each configured recipient, this policy determines what online services are matched,
and then selects one among all of those in round-robin order.
If none of the configured recipients match any available service,
this policy returns an error that indicates to the sender that it should retry later.
Because this policy only selects a single recipient, it contains no merging logic.
SubsetService
This policy is used to select among a subset of all matching services,
and is used to minimize number of connections in the system.
The pattern used when querying for available services is the current one,
but replacing the policy directive with an asterisk.
E.g. the hop "docproc/cluster.default/[SubsetService:3]/chain.default"
would select among a subset of all those that match the pattern "docproc/cluster.default/*/chain.default".
Given that the pattern returns a set of matches,
this policy stores a subset of these based on the hash-value of the running message bus'
connection string (this is unique for each instance).
If there are no matching services,
this policy returns the asterisk as a match to allow the underlying network logic to fail gracefully.
This policy parses its optional parameter as the size of the subset.
If none is given, the subset defaults to size 5.
Important:
There is no additional logic to replace other policy directives with an asterisk,
meaning that if other policies directives are present in the hop string
after "[SubsetService]",
no services can possibly be matched.
LoadBalancer
This policy is used to send to a stateless cluster such as docproc,
where any node can be chosen to process any message.
Messages are sent between the nodes in a round-robin fashion,
but each node is assigned a weight based on its performance.
The weights are calculated by measuring the number of times the node had a full input-queue
and returned a busy response.
Use this policy to send to docproc clusters that have nodes with different performance characteristics.
This policy supports multiple parameters, up to one each of:
cluster
The name of the cluster you want to reach.
Example: cluster=docproc/cluster.default (mandatory)
session
The destination session you want to reach.
In the case of docproc, the name of the docproc chain.
Example: session=chain.mychain (mandatory)
config
A comma-separated list of config servers or proxies
you want to use to fetch configuration for the policy.
This can be used to communicate with other clusters than the one you're currently in.
Example: config=tcp/myadmin1:19070,tcp/myadmin2:19070
Separate each parameter with a semicolon.
By default, this policy will use the current Vespa cluster for configuration.
Routing for indexing
A normal Vespa configuration has container and content cluster(s),
with one or more document types defined in schemas.
Routing document writes means routing documents to the indexing container cluster,
then the right content cluster.
Given this configuration, Vespa knows which is the container cluster used for indexing,
and which content cluster that stores the music document type.
Use vespa-route
to display routing generated from this configuration:
$ vespa-route
There are 6 route(s):
1. default
2. default-get
3. music
4. music-direct
5. music-index
6. storage/cluster.music
There are 2 hop(s):
1. container/chain.indexing
2. indexing
Note the default route. This route is auto-generated by Vespa,
and is used when no other route is used when using /document/v1.
default points to indexing:
$ vespa-route --route default
The route 'default' has 1 hop(s):
1. indexing
$ vespa-route --hop indexing
The hop 'indexing' has selector:
[DocumentRouteSelector]
And 1 recipient(s):
1. music
$ vespa-route --route music
The route 'music' has 1 hop(s):
1. [MessageType:music]
In short, the default route handles documents of type music.
Vespa will route to the container cluster with document-api -
note the chain.indexing above.
This is a set of built-in document processors that does the indexing (below).
By default, the indexing chain is set up on the first container cluster in services.xml.
When having multiple container clusters, it is recommended to configure this explicitly, see
multiple container clusters.
Document selection
The document
can have a selection string,
normally used to expire documents.
This is also evaluated during feeding, so documents that would immediately expire are dropped.
This is not an error, the document API will report 200 - but can be confusing.
The evaluation is done in the
DocumentRouteSelector at the feeding endpoint - before any processing/indexing.
I.e. the document is evaluated using the selection string (drop it or not),
then where to route it, based on document type.
Example: the selection is configured to not match the document being fed:
$ vespa-feeder --trace 6 doc.json
<trace>
[1564576570.693] Source session accepted a 4096 byte message. 1 message(s) now pending.
[1564576570.713] Sequencer sending message with sequence id '-1163801147'.
[1564576570.721] Recognized 'default' as route 'indexing'.
[1564576570.727] Recognized 'indexing' as HopBlueprint(selector = { '[DocumentRouteSelector]' }, recipients = { 'music' }, ignoreResult = false).
[1564576570.811] Running routing policy 'DocumentRouteSelector'.
[1564576570.822] Policy 'DocumentRouteSelector' assigned a reply to this branch.
[1564576570.828] Sequencer received reply with sequence id '-1163801147'.
[1564576570.828] Source session received reply. 0 message(s) now pending.
</trace>
Messages sent to vespa (route default) :
----------------------------------------
PutDocument: ok: 0 msgs/sec: 0.00 failed: 0 ignored: 1 latency(min, max, avg): 9223372036854775807, -9223372036854775808, 0
Without the selection (i.e. everything matches):
$ vespa-feeder --trace 6 doc.json
<trace>
[1564576637.147] Source session accepted a 4096 byte message. 1 message(s) now pending.
[1564576637.168] Sequencer sending message with sequence id '-1163801147'.
[1564576637.176] Recognized 'default' as route 'indexing'.
[1564576637.180] Recognized 'indexing' as HopBlueprint(selector = { '[DocumentRouteSelector]' }, recipients = { 'music' }, ignoreResult = false).
[1564576637.256] Running routing policy 'DocumentRouteSelector'.
[1564576637.268] Component '[MessageType:music]' selected by policy 'DocumentRouteSelector'.
...
</trace>
Messages sent to vespa (route default) :
----------------------------------------
PutDocument: ok: 1 msgs/sec: 1.05 failed: 0 ignored: 0 latency(min, max, avg): 845, 845, 845
In the last case, in the
DocumentRouteSelector routing policy,
the document matched the selection string / there was no selection string,
and the document was forward to the nex hop in the route.
Document processing
Add custom processing of documents using document processing.
The normal use case is to add document processors in the default route, before indexing. Example:
Note that a new hop default/chain.default is added,
and the default route is changed to include this:
$ vespa-route
There are 6 route(s):
1. default
2. default-get
3. music
4. music-direct
5. music-index
6. storage/cluster.music
There are 3 hop(s):
1. default/chain.default
2. default/chain.indexing
3. indexing
$ vespa-route --route default
The route 'default' has 2 hop(s):
1. default/chain.default
2. indexing
Note that the document processing chain must be called default
to automatically be included in the default route.
Inherit indexing chain
An alternative to the above is inheriting the indexing chain - use this when getting this error:
Indexing cluster 'XX' specifies the chain 'default' as indexing chain.
As the 'default' chain is run by default, using it as the indexing chain will run it twice.
Use a different name for the indexing chain.
Call the chain something else than default, and let it inherit indexing:
Vespa can be configured to use more than one container cluster.
Use cases can be to separate search and document processing
or having different document processing clusters due to capacity constraints or dependencies.
Example with separate search and feeding/indexing container clusters:
The indexing route is explicit using
document-processing
elements from the content to the container cluster
Set up document-api on the same cluster as indexing to avoid network hop
from feed endpoint to indexing processors
If no document-processing is configured,
it defaults to a container cluster named default.
When using multiple container clusters,
it is best practice to explicitly configure document-processing.
Observe the container-indexing/chain.indexing hop,
and the indexing chain is set up on the container-indexing cluster:
$ vespa-route
There are 6 route(s):
1. default
2. default-get
3. music
4. music-direct
5. music-index
6. storage/cluster.music
There are 2 hop(s):
1. container-indexing/chain.indexing
2. indexing
