[+] expand all

vespa-feed-client

Java library and command line client for feeding document operations using Document v1 over HTTP/2.
Asynchronous, high-performance Java implementation, with retries and dynamic throttling.
Supports a JSON array of feed operations, as well as JSONL: one operation JSON per line.

Installing

Java library

The Java library is available as a Maven JAR artifact at Maven Central.

Command line client

Two alternatives:

Install vespa-clients/vespa RPM package.
Download vespa-feed-client zip artifact from Maven Central.

Download example:

$ F_REPO="https://repo1.maven.org/maven2/com/yahoo/vespa/vespa-feed-client-cli" && \
  F_VER=$(curl -Ss "${F_REPO}/maven-metadata.xml" | sed -n 's/.*<release>\(.*\)<.*>/\1/p') && \
  curl -SsLo vespa-feed-client-cli.zip ${F_REPO}/${F_VER}/vespa-feed-client-cli-${F_VER}-zip.zip && \
  unzip -o vespa-feed-client-cli.zip

Enable feed endpoint in Vespa

Requirements:

Document API must be enabled on container.

HTTP/2 over TLS is optional but recommended from a security perspective.

Example services.xml with TLS:

<?xml version="1.0" encoding="utf-8" ?>
<services version="1.0">
    <container version="1.0" id="default">
        <http>
            <server id="default" port="443">
                <ssl>
                    <private-key-file>/path/to/private-key.pem</private-key-file>
                    <certificate-file>/path/to/certificate.pem</certificate-file>
                    <ca-certificates-file>/path/ca-certificates.pem</ca-certificates-file>
                </ssl>
            </server>
        </http>
        <document-api/>
    </container>
</services>

Example services.xml without TLS:

<?xml version="1.0" encoding="utf-8" ?>
<services version="1.0">
    <container version="1.0" id="default">
        <document-api/>
    </container>
</services>

Using the client

The Javadoc for the programmatic API is available at javadoc.io. See output of $ vespa-feed-client --help for usage.

Use --speed-test for bandwidth testing.

Example Java

Add vespa-feed-client as dependency to your Maven (or other build system using Maven for dependency management):

<dependency>
    <groupId>com.yahoo.vespa</groupId>
    <artifactId>vespa-feed-client</artifactId>
    <version>8.91.2</version>
</dependency>

Code examples are listed in the vespa-feed-client source code on GitHub.

Example command line

HTTP/2 over TLS:

$ vespa-feed-client \
  --connections 4 \
  --certificate cert.pem --private-key key.pem --ca-certificates ca.pem \
  --file /path/to/json/file \
  --endpoint https://container-endpoint:443/

The input must be either a proper JSON array, or a series, of JSON feed operations (JSONL), in the format described for the Vespa feed client here.

HTTP/2 without TLS:

$ vespa-feed-client \
  --connections 4 \
  --file /path/to/json/file \
  --endpoint http://container-endpoint:8080/

Feed using Hadoop, Pig, Oozie

See the documentation for vespa-hadoop library.