• [+] expand all

HTTP/2

This document contains HTTP/2 performance considerations on the container—see Container tuning for general tuning of container clusters.

Enabling HTTP/2 on container

HTTP/2 is enabled on a container for all connectors with TLS/HTTPS configured (requires Vespa 7.418.23 or newer). Both HTTP/1.1 and HTTP/2 will be served over the same connector using the TLS ALPN Extension. The Application-Layer Protocol Negotiation (ALPN) extension allows the client to send a list of supported protocols during TLS handshake. The container selects a supported protocol from that list.

The HTTP/2 specification dictates multiple requirements for the TLS connection. Vespa may enforce some or all of these restrictions. See the HTTP/2 specification for the full list. The most significant are listed below:

  • Client must use at least TLSv1.2.
  • Client must provide target domain with the TLS Server Name Indication (SNI) Extension.
  • Client must not use any of the banned TLSv1.2 ciphers.

Feeding over HTTP/2

One of the major improvements with HTTP/2 is multiplexing of multiple concurrent requests over a single TCP connection. This allows for high-throughput feeding through the Document v1 HTTP API, with a simple one-operation–one-request model, but without the overhead of hundreds of parallel connections that HTTP/1.1 would require for sufficient concurrency. HTTP/2 + Document v1 is a great alternative to the Vespa HTTP Client, which implements a custom protocol, and requires a JDK 8+ client environment.

Vespa provides a Java implementation of HTTP/2 + Document v1 in vespa-feed-client. This client is available both as a Java API and a command line client.

Performance tuning

Client

The number of multiple concurrent requests per connection is typically adjustable in HTTP/2 clients/libraries. Document v1 API is designed for high concurrency and can easily handle thousands of concurrent requests. Its implementation is asynchronous and max concurrency is not restricted by a thread pool size, so configure your client to allow enough concurrent requests/streams to saturate the feed container. Other APIs such as the Query API is backed by a synchronous implementation, and max concurrency is restricted by the underlying thread pool size. Too many concurrent streams may result in the container rejecting requests with 503 responses.

There are also still some reasons to use multiple TCP connections—even with HTTP/2:

  • Utilize multiple containers. A single container may not saturate the content layer. A client may have to use more connections than container nodes if the containers are behind a load balancer.
  • Higher throughput. Many clients allow only for a single thread to operate each connection. Multiple connections may be required for utilizing several CPU cores.

Client recommendations

Use vespa-feed-client for feeding through Document v1 API (JDK8+).

We recommend the h2load benchmarking tool for load testing. vespa-fbench does not support HTTP/2 at the moment.

For Java there are 4 good alternatives:

  1. Jetty Client
  2. OkHttp
  3. Apache HttpClient 5.x
  4. java.net.http.HttpClient (JDK11+)