Our goal with this series is to set up a Vespa application for personalized news recommendations. We will do this in stages, starting with a simple news search system and gradually adding functionality as we go through the tutorial parts.
The parts are:
There are different entry points to this tutorial. This one is for getting started using Docker on your local machine. Getting started on cloud.vespa.ai is coming soon. We will also have a version for pyvespa soon. For atomic model updates, see the Models hot swap tutorial.
In this part we will start with a minimal Vespa application to get used to some basic operations for running the application on Docker. In the next part of the tutorial, we'll start developing our application.
Prerequisites:
NO_SPACE - the vespaengine/vespa container image + headroom for data requires disk space.
Read more.
curl to download the dataset and run the Vespa health-checks.In upcoming parts of this series, we will have some additional python dependencies - we use PyTorch to train vector representations for news and users and train machine learning models for use in ranking.
This tutorial uses Vespa-CLI, Vespa CLI is the official command-line client for Vespa.ai. It is a single binary without any runtime dependencies and is available for Linux, macOS and Windows.
$ brew install vespa-cli
This tutorial has a companion sample application.
Throughout the tutorial we will be using support code from this application.
Also, the final state of each tutorial can be found in the various app-... subdirectories.
Let's start by cloning the sample application:
$ vespa clone -f news news && cd news
The above downloads the news directory from the Vespa
sample apps repository and
places the contents in a folder called news. Use --help to see documentation
for the vespa-cli utility:
$ vespa clone --help
In the news directory there are several pre-configuration applications packages.
The app-1-getting-started directory contains a minimal Vespa application.
There are two files there:
services.xml - defines the services the application consists ofschemas/news.sd - defines the schema for searchable content.We will get back to these files in the next part of the tutorial.
This application doesn't contain much at the moment, let's start up the application anyway by starting a Docker container to run it:
$ docker pull vespaengine/vespa $ docker run --detach --name vespa --hostname vespa-tutorial \ --publish 8080:8080 --publish 19071:19071 --publish 19092:19092 \ vespaengine/vespa
First, we pull the latest vespa-image
from the Docker hub, then we
start it with the name vespa. This starts the Docker container and the
initial Vespa services to be able to deploy an application.
Starting the container can take a short while. Before continuing, make sure
that the configuration service is running by using vespa status.
$ vespa status deploy --wait 300
With the config server up and running, deploy the application using vespa-cli:
$ vespa deploy --wait 300 app-1-getting-started
The command uploads the application and verifies the content. If anything is wrong with the application, this step will fail with a failure description, otherwise this switches the application to a live status.
Whenever you have a new version of your application, run the same command to deploy the application. In most cases, there is no need to restart services. Vespa takes care of reconfiguring the system. If a restart of services is required in some rare case, however, the output will notify which services needs restart to make the change effective.
In the upcoming parts of the tutorials, we'll frequently deploy the application changes in this manner.
We must index data before we can search for it. This is called "feeding", and we'll get back to that in more detail in the next part of the tutorial. For now, to test that everything is up and running, we'll feed in a single test document:
$ vespa feed -t http://localhost:8080 doc.json
The -v option will make vespa-cli print the http request:
$ vespa document -v doc.json
We can also feed using Vespa document api directly.
Once the feed operation is ack'ed by Vespa, the operation is visible in search.
We can query the endpoint using the vespa-cli's support for performing queries.
It uses the Vespa query api to query vespa,
including -v in the command we can see the exact endpoint and url request parameters used.
$ vespa query -v 'yql=select * from news where true'
This example uses YQL (Vespa Query Language) to
search for all documents of type news. This query request will return 1 result, which is the document we fed above.
$ vespa query \ 'yql=select * from news where userQuery()' \ 'query=hello world' \ 'default-index=title'
Another query language example that searches for hello or world in the title.
$ vespa query \
'yql=select * from news where title contains phrase("hello","world")'
Another query language example that searches for the phrase "hello world" in the title. In the next part of the tutorial we'll demonstrate more query examples, and also ranking and grouping of results.
Run the following to remove the document from the index:
$ vespa document -v remove id:news:news::1
Well done!
Keep Vespa running to continue with next steps in this tutorial set (skip the below).
To stop Vespa, we can run the following commands:
$ docker exec vespa vespa-stop-services $ docker exec vespa vespa-stop-configserver
Likewise, to start the Vespa services:
$ docker exec vespa vespa-start-configserver $ docker exec vespa vespa-start-services
If a restart is required due to change in the application package, these two steps are what you need to do.
To wipe the index and restart:
$ docker exec vespa sh -c ' \ vespa-stop-services && \ vespa-remove-index -force && \ vespa-start-services'
You can stop and kill the Vespa container application like this:
$ docker stop vespa; docker rm -f vespa
This will delete the Vespa application, including all data and configuration. See container tuning for production.
Our simple application should now be up and running. In the next part of the tutorial, we'll start building from this foundation.