Routing

Routing is used to configure the paths that documents and updates written to Vespa take through the system. Vespa will automatically set up a routing configuration which is appropriate for most cases, so no explicit routing configuration is necessary. However, explicit routing can be used in advanced use cases such as sending different document streams to different document processing clusters, or through multiple consecutive clusters etc.

There are other, more in-depth, articles on routing:

• Use vespa-route to inspect routes and services of a Vespa application, like in the example
• Routing policies reference. See the routing policies note for complex routes and default routing

In Vespa, there is a transport layer and a programming interface that are available to clients that wish to communicate with a Vespa application. The transport layer is Message Bus. Document API is implemented on top of Message Bus. Configuring the interface therefore exposes some of the features available in Message Bus. Refer to the Vespa APIs and interfaces for clients using the Document API. The atoms in Vespa routing are routes and hops.

A route is a sequence of hops

The sequence of hosts, routers, bridges, gateways, and other devices that network traffic takes, or could take, from its source to its destination is what is classically termed a route. As a verb, to route means to determine the link down which to send a packet, that will minimize its total journey time according to some routing algorithm.

In Vespa, a route is simply a sequence of named hops. Instead of leaving selection logic to a route, the responsibility of resolving recipients is given to the hops' selectors. A hop can do more or less whatever it wants to change a message's journey through your application; it can slightly alter itself by choosing among some predefined recipients, it can change itself completely by either rewriting or looking up another hop, or it can even modify the entire route from that branch onwards. In effect, a route can end up branching at several points along its path, resulting in complex routes. As the figure suggests, Message Bus supports both unicasting and multicasting - Message Bus allows for arbitrarily complex routes. Each node in the above graph represents a Vespa service:

A hop is a point-to-point transmission

In telecommunication, a hop is one step, from one router to the next, on the path of a packet on an Internet Protocol network. It is a direct host-to-host connection forming part of the route between two hosts in a routed network such as the Internet. In more general terms, a hop is a point-to-point transmission in a series required to get a message from point A to point B.

With Message Bus the concept of hops was introduced as the smallest steps of the transmission of a message. A hop consists of a name that is used by the messaging clients to select it, a list of recipient services that it may transmit to, and a selector that is used to select among those recipients. Unlike traditional hops, in Vespa a hop is a transmission from one sender to many recipients.

Well, the above is only partially true; it is the easiest way to understand the hop concept. In fact, a hop's recipient list is nothing more than a configured list of strings that is made available to all routing policies that are named in the selector string. See selection logic below for details.

A hop's recipient is the service name of a Message Bus client that has been registered in Vespa's service location broker (vespa-slobrok). These names are well-defined once their derivation logic is understood; they are "/"-separated sets of address-components whose values are given by a service's role in the application. An example of a recipient is:

search/cluster.foo/*/feed-destination

The marked components of the above recipient, /search/cluster.foo/*, resolves to a host's symbolic name. This is the name with which a Message Bus instance was configured. The unmarked component, feed-destination, is the local name of the running service that the hop transmits to, i.e. the name of the session created on the running Message Bus instance.

The Active Configuration page in Vespa's administration interface gives an insight into what symbolic names exist for any given application by looking at its current configuration subscriptions. All available Message Bus services use their ConfigId as their host's symbolic name. See vespa-route for how to inspect this, or use the config API.

A hop can be prefixed using the special character "?" to force it to behave as if its ignore-result attribute was configured to "true".

Asterisk

A service identifier may include the special character "*" as an address component. A recipient that contains this character is a request for the network to choose any one service that matches it.

Routing policies

A routing policy is a protocol-specific algorithm that chooses among a list of candidate recipients for a single address component - see hop description above. These policies are designed and implemented as key parts of a Message Bus protocol. E.g. for the "Document" protocol these are what make up the routing behavior for document transmission. Without policies, a hop would only be able to match verbatim to a recipient, and thus the only advanced selection logic would be that of the asterisk.

In addition to implementing a selection algorithm, a routing policy must also implement a merging algorithm that combines the replies returned from each selected recipient into a single sensible reply. This is needed because a client does not necessarily know whether a message has been sent to one or multiple recipients, and Message Bus guarantees a single reply for every message.

More formally, a routing policy is an arbitrarily large (or small), named, stand-alone piece of code registered with a Message Bus protocol. As discussed above, an instance of a policy is run both when resolving a route to recipients, and when merging replies. The policy is passed a RoutingContext object that pretty much allows it to do whatever it pleases to the route and replies. The same policy object and the same context object is used for both selection and merging.

Refer to the routing policy reference.

Selection logic

When Message Bus is about to route a message, at the last possible time, it inspects the first hop of the message's route to resolve a set of recipients. First, all of its policies are resolved. Second, the output service name is matched to the routing table to see if it maps to another hop or route. Finally, the message is sent to all chosen recipient services. Because each policy can select multiple recipients, this can give rise to an arbitrarily complex routing tree. There are, of course, safe-guards within Message Bus to prevent infinite recursions due to circular dependencies or misconfiguration.

1. Resolve Policy Directives

The logic run at this step is actually simple; as long as the hop string contains a policy directive, i.e. some arbitrary string enclosed in square brackets, Message Bus will create and run an instance of that policy for the protocol of the message being routed.

Name:        storage/cluster.backup
Selector:    storage/cluster.backup/distributor/[Distributor]/default
Recipients:  -

The above hop is probably the simplest hop you will encounter in Vespa; it has a single policy directive contained in a string that closely resembles service names discussed above, and it has no recipients. When resolving this hop, Message Bus creates an instance of the "DocumentRouteSelector" policy and invokes its select() method. The "Distributor" policy will replace its own directive with a proper distributor identifier, yielding a hop string that is now an unambiguous service identifier.

Name:        indexing
Selector:    [DocumentRouteSelector]
Recipients:  search/cluster.music
             search/cluster.books

This hop has a selector which is nothing more than a single policy directive, "[DocumentRouteSelector]", and it has two configured recipients, "search/cluster.music" and "search/cluster.books". This policy expands the hop to zero, one or two new routes by replacing its own directive with the content of the recipient routes. Each of these routes may have one or more hops themselves. In turn, these will be processed independently. When replies are available from all chosen recipients, the policy's merge() method is invoked, and the resulting reply is passed upwards.

Name:        default
Selector:    [AND:indexing storage/cluster.backup]
Recipients:  -

This hop has a selector but no recipients. The reason for this is best explained in the policies article, but it serves as an example of a hop that has no configured recipients. Notice how the policy directive contains a colon (":") which denotes that the remainder of the directive is a parameter to the policy constructor. This policy replaces the whole route of the message with the set of routes named in the parameter string.

What routing policies are available depends on what protocol is currently running. As of this version the only supported protocol is "Document". This offers a set of routing policies discussed in this article.

2. Resolve Hop- and Route names

As soon as all policy directives have been resolved, Message Bus makes sure that the resulting string is, in fact, a service name and not the name of another hop or route (in that order) configured for the running protocol. The outcome is either:

1. The string is recognized as a hop name - The current hop is replaced by the named one, and processing returns to step 1.
2. The string is recognized as a route name - The current route, including all the hops following this, is replaced by the named one. Processing returns to step 1.
3. The string is accepted as a service name - This terminates the current branch of the routing tree. If all branches are terminated, processing proceeds to step 3.

Because hop names are checked before route names, Message Bus also supports a "route:" prefix that forces the remainder of the string to resolve to a configured route or fail.

3. Send to Services

When the route resolver reaches this point, the first hop of the message being sent has been resolved to an arbitrarily complex routing tree. Each leaf of this tree represents a service that is to receive the message, unless some policy has already generated a reply for it. No matter how many recipients are chosen, the message is serialized only once, and the network transmission is able to share the same chunk of memory between all recipients.

As replies to the message arrive at the sender they are handed over to the corresponding leaf nodes of the routing tree, but merging will not commence until all leaf nodes are ready.

Route resolving happens just before network transmission, after all resending logic. This means that if the route configuration changes while there are messages scheduled for resending, these will adhere to the new routes.

If the resolution of a recipient passed through a hop that was configured to ignore results, the network layer will reply immediately with a synthetic "OK".

Example: Reconfigure the default route

Assume that the application requires both search and storage capabilities, but that the default feed should only pass through to search. An imaginary scenario for this would be a system where there is a continuous feed being passed into Vespa with no filtering on spam. You would like a minimal storage-only cluster that stores a URL blocklist that can be used by a custom document processor to block incoming documents from offending sites.

Apart from the blocklist and the document processor, add the following:

<routing version="1.0">
    <routingtable protocol="document">
        <route name="default" hops="docproc/cluster.blocklist/*/chain.blocklist indexing" />
    </routingtable>
</routing>

This overrides the default route to pass through any available blocklisting document processor before being indexed. If the document processor decides to block a message, it must respond with an appropriate ok reply, or your client software needs to accept whatever error reply you decide to return when blocking.

When feeding blocklist information to storage, your application need only use the already available storage hop.

The Document API

With the current implementation of Document API running on Message bus, the configuration of the API implies configuration of the latter. Most clients will only ever route through this API. To use the Document API, you need to instantiate a class that implements the DocumentAccess interface. At the time of writing only MessageBusDocumentAccess exists, and it requires a parameter set for creation. These parameters are contained in an instance of MessageBusDocumentAccessParam that looks somewhat like the following:

class MessageBusDocumentAccessParams {
    String documentManagerConfigId; // The id to resolve to document manager config.
    String oosServerPattern;        // The service pattern to resolve to fleet controller
                                    // services.
    String appConfigId;             // The id to resolve to application config.
    String slobrokConfigId;         // The id to resolve to slobrok config.
    String routingConfigId;         // The id to resolve to messagebus routing config.

    String routeName;               // The name of the route to send to.
    int    traceLevel;              // The trace level to use when sending.

    class SourceSessionParams {
        int    maxPending;          // Maximum number of pending messages.
        int    maxPendingSize;      // Maximum size of pending messages.
        double timeout;             // Default timeout in seconds for messages
                                    // that have no timeout set.
        double requestTimeoutA;     // Default request timeout in seconds, using
        double requestTimeoutB;     // the equation 'requestTimeout = a * retry + b'.
        double retryDelay;          // Number of seconds to wait before resending.
    };
}

The most obvious configuration parameter is routeName, which informs the MessageBusDocumentAccess object the name of the route to use when sending documents and updates. The second parameter is traceLevel, which allows a client to see exactly how the data was transmitted.

Refer to the Document API JavaDoc.