[+] expand all

Application Packages

An application package is a set of files in a specific structure that defines a deployable application. It contains all the configuration, components and machine-learned models that is necessary to deploy and run the application: No configuration is ever done locally on Vespa nodes or over remote API's.

The application package is a directory, containing at minimum services.xml. Additionally, services.xml might consume other files or directories from the application package - see the reference for a full list.

Most application packages are stored as source code in a code repository. However, some resources are generated and/or too large to store in a code repository, like models or an FSA. See below for how to use the url config type to download resources to container nodes.

Use the Vespa CLI to build and deploy your application package. Refer to the bundle plugin for how to build an application package with Java components. Also, find more details in the config introduction.

Code and config consistency

A change to code and configuration is atomically deployed to running instances. To ensure code and config consistency, config definition files are compiled to Java code, discrepancies causes build failures, which is easier to manage than production errors. Read more in configuring components.

Deploy

Deploy the application package using vespa-deploy. The application package is validated at deploy, and destructive changes blocked. To allow such changes to pass the validation in deploy prepare, add validation-overrides.xml. Making changes to schemas (e.g. add a field) is followed by a deployment to the application instance. Most such changes do not require restarts or re-indexing, if they do, deployment fails, and a validation override is required. Use deploy to:

prepare: Upload, validate and distribute to nodes
activate: Waits for prepare to complete before activation. Activate on nodes - this reloads the application bundles, while serving.

Find more details in the reference. Deploying application changes to production is safe: Vespa will reject the change if it will break something and manage the process of making it effective otherwise.

During development it can also be handy to just wipe the state of Vespa completely and start over:

Delete all config server state on all config servers
Run vespa-remove-index to wipe content nodes

File distribution

The application package can have components and other large files. At vespa-deploy prepare, these files are distributed to the nodes:

Components (i.e bundles)
Files with type path and url in config, see Adding files to the component configuration
Constant tensors

When new components or files specified in config are distributed, the container gets a new file reference, waits for it to be available and switches to new config when all files are available.

Nodes get config from a config server cluster

Use vespa-status-filedistribution to check if files have been distributed to all the hosts.

Deploying remote models

Machine learned models in Vespa, either TensorFlow, ONNX, XGBoost, or LightGBM, are stored in the application package under the models directory. This might be inconvenient for some applications, for instance for models that are frequently retrained on some remote system. Also, models might be too large to fit within the constraints of the version control system.

The solution is to download the models from the remote location during the application package build. This is simply implemented by adding a step in pom.xml (see example):

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.4.0</version>
            <executions>
                <execution>
                    <id>download-model</id>
                    <phase>generate-resources</phase>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                    <configuration>
                        <executable>bin/download_models.sh</executable>
                        <arguments>
                            <argument>target/application/models</argument>
                            <argument>MODEL-URL</argument>
                        </arguments>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

bin/download_model.sh example:

#!/bin/bash

DIR="$1"
URL="$2"

echo "[INFO] Downloading $URL into $DIR"

mkdir -p $DIR
pushd $DIR
    curl -O $URL
popd

Any necessary credentials for authentication and authorization should be added to this script, as well as any unpacking of archives (for TensorFlow models for instance).

services.xml

services.xml specifies the services that makes the application - each top-level element specifies one service. Example:

<?xml version="1.0" encoding="utf-8" ?>
<services version="1.0">

    <container id="default" version="1.0">
        <processing/>      <!-- Request-response processors go here. -->
        <search/>          <!-- Searchers go here. -->
        <docproc/>         <!-- DocumentProcessors go here. -->
        <nodes>            <!-- Nodes in the container cluster -->
            <node hostalias="node1"/>
            <node hostalias="node2"/>
            <node hostalias="node3"/>
        </nodes/>
    </container>

    <content id="my-content" version="1.0">
        <redundancy>1</redundancy>
        <documents>         <!-- Add document schemas here -->
            <document type="my-searchable-type" mode="index"/>
            <document type="my-other-type"      mode="index"/>
        </documents>
        <nodes>             <!-- # nodes in the content cluster -->
            <node hostalias="node4" distribution-key="0" />
            <node hostalias="node5" distribution-key="1" />
            <node hostalias="node6" distribution-key="2" />
        </nodes/>
    </content>

</services>

Refer to the services.xml reference for different service types and configuration.

Component configuration

The application's custom Java code (in components) is configured in services.xml. Example, a configured port number for a remote service:

    <container id="default" version="1.0">
        <handler id="com.myapp.vespatest.ConfiguredHandler">
            <config name="vespatest.port">
                <port>12345</port>
            </config>