These notes documents the changes between Vespa major versions 6 and 7. As documented in Vespa versions, major versions are used to mark breaking compatibility, not to release significant new functionality (which instead happens on minor versions). However, even on major versions, compatibility is broken only in two specific ways: Changes to defaults means that applications may need to set some option explicitly to preserve earlier behavior, and removal of deprecated functionality means that applications which use functionality that has earlier been deprecated need to change to keep working.
Most deprecated functionality causes warning during compilation (Java API deprecations) or deployment (application package deprecations), however with web service APIs there is no way to emit deprecation warnings, and we have to rely on marking these as deprecated in the documentation.
Given this, application owners need to do 3 tasks to be compatible with Vespa 7:
As Vespa 7 does not introduce new functionality, it is as safe and mature as the versions of Vespa 6 preceding it. Upon release of Vespa 7, no further releases will be made of Vespa 6 for any reason.
The following sections lists the changes on moving from Vespa 6 to Vespa 7 which must be reviewed by applications.
The following defaults have changed:
| Change | Configuration required to avoid change on Vespa 7 |
|---|---|
stemming: shortest changed to stemming: best |
Add stemming: shortest
to the schema block of all schemas. |
| Default linguistics component changed from SimpleLinguistics to OpenNlpLinguistics, including language detection using Optimaize turned on by default. | Configure com.yahoo.language.simple.SimpleLinguistics as a component in services.xml as described in
linguistics in Vespa
|
| The default format accepted by the Java HTTP client is changed from XML to JSON | To keep using XML:
|
| Query timeout changed from 5000 ms to 500 ms. | Set the timeout parameter explicitly in requests or query profiles. |
| ranking.softtimeout.enable changed to default true | Set to false in requests or a query profile. |
| The default access log format is changed to JSON. | To keep the old proprietary format, set accesslog type=vespa in services.xml as described in the accesslog reference. |
| Default return format in vespa-visit and vespa-get is changed to JSON | To get XML output specify the --xmloutput method |
Java components must be rebuilt with JDK 11 for the Vespa bundle-plugin to generate the correct set of imported packages for your OSGi bundles.
Classes and methods that were marked as deprecated in Vespa 6 are removed. If you get deprecation warnings for Vespa APIs when building your application, they must be fixed before migrating to Vespa 7.
The following maven artifacts are no longer provided runtime:
If you need any of these dependencies, they must be embedded in your bundle by adding them in scope 'compile' in pom.xml.
The following HTTP APIs are removed:
| Name | Replacement |
|---|---|
Legacy HTTP apis for document feeding:
|
The /document/v1/ web service API, or (for high throughput) the vespa-http-client. |
The following HTTP API parameters are removed
| Name | Replacement |
|---|---|
The defidx parameter in the search API |
Use a custom searcher if this functionality is needed. |
The following command line tools are removed:
| Name | Replacement |
|---|---|
| Vespa spooler | Custom client using the Java HTTP client |
The following settings are removed from schemas:
| Name | Replacement |
|---|---|
| header | None. This setting doesn't have any effect |
| body | None. This setting doesn't have any effect |
The following tags and attributes are removed from services.xml:
| Name | Replacement |
|---|---|
| ‘rotationScheme’ attribute in <container><accesslog> | None, rotation scheme ‘date’ will always be used |
| <container><filter> tag | <container><http><filtering> |
The following metrics are removed:
| Name | Replacement |
|---|---|
| free/used/totalMemoryBytes | mem.heap.free/used/total |
| http.in.bytes | serverBytesReceived |
| http.out.bytes | serverBytesSent |
| http.requests | serverNumRequests |
| http.latency | serverTotalSuccessfulResponseLatency |
| http.out.firstbytetime | serverTimeToFirstByte |
| proc.uptime | serverStartedMillis |
| proton.* | content.proton.* (note that metrics might have different structure and names in new namespace) |
| vds.filestor.spi.* | vds.filestor.alldisks.allthreads.* |
| Character | Percent-encoding |
|---|---|
| \ | %5C |
| ^ | %5E |
| ` | %60 |
| { | %7B |
| | | %7C |
| } | %7D |
The content of fields of type position in the default JSON query result format was rendered as XML on Vespa 6 but is rendered as JSON.
Specifically, the content of a position field was rendered as a string like
<position x="-121996000" y="37401000" latlong="N37.401000;W121.996000"/>
but is now instead rendered as a JSON map:
{
"y": 37401000,
"x": -121996000,
"latlong": "N37.401000;W121.996000"
}
The following metrics are renamed:
| Old Name | New Name |
|---|---|
| 95p_query_latency | query_latency.95percentile |
| 99p_query_latency | query_latency.99percentile |
| active_queries | active_queries.average |
| athenz-tenant-cert.expiry.seconds | athenz-tenant-cert.expiry.seconds.last |
| bytes | vds.datastored.alldisks.bytes.average |
| configserver.cacheChecksumElems | configserver.cacheChecksumElems.last |
| configserver.cacheConfigElems | configserver.cacheConfigElems.last |
| configserver.delayedResponses | configserver.delayedResponses.count |
| configserver.failedRequests | configserver.failedRequests.count |
| configserver.hosts | configserver.hosts.last |
| configserver.latency | configserver.latency.average |
| configserver.requests | configserver.requests.count |
| configserver.sessionChangeErrors | configserver.sessionChangeErrors.count |
| configserver.zkAvgLatency | configserver.zkAvgLatency.last |
| configserver.zkConnections | configserver.zkConnections.last |
| configserver.zkMaxLatency | configserver.zkMaxLatency.last |
| configserver.zkOutstandingRequests | configserver.zkOutstandingRequests.last |
| configserver.zkZNodes | configserver.zkZNodes.last |
| content.cluster-controller.cluster-state-change.count | cluster-controller.cluster-state-change.count |
| content.proton.memoryusage.max | content.proton.documentdb.memory_usage.allocated_bytes.max |
| content.proton.transport.docsum.latency.average | content.proton.docsum.latency.average |
| degraded_queries | degraded_queries.rate |
| deletefailed | vds.idealstate.delete_bucket.done_failed.rate |
| deleteok | vds.idealstate.delete_bucket.done_ok.rate |
| deletepending | vds.idealstate.delete_bucket.pending.average |
| diskqueuesize | vds.filestor.alldisks.queuesize.average |
| diskqueuewait | vds.filestor.alldisks.averagequeuewait.sum.average |
| diskusage | content.proton.documentdb.disk_usage.last |
| docs | vds.datastored.alldisks.docs.average |
| document_requests | content.proton.docsum.docs.rate |
| documents_active | content.proton.documentdb.documents.active.last |
| documents_inmemory | content.proton.documentdb.index.docs_in_memory.last |
| documents_processed | documents_processed.rate |
| documents_ready | content.proton.documentdb.documents.ready.last |
| documents_removed | content.proton.documentdb.documents.removed.last |
| documents_total | content.proton.documentdb.documents.total.last |
| empty_results | empty_results.rate |
| error.backend_communication_error | error.backend_communication_error.rate |
| error.backends_oos | error.backends_oos.rate |
| error.empty_document_summaries | error.empty_document_summaries.rate |
| error.internal_server_error | error.internal_server_error.rate |
| error.invalid_query_parameter | error.invalid_query_parameter.rate |
| error.invalid_query_transformation | error.invalid_query_transformation.rate |
| error.misconfigured_server | error.misconfigured_server.rate |
| error.plugin_failure | error.plugin_failure.rate |
| error.result_with_errors | error.result_with_errors.rate |
| error.timeout | error.timeout.rate |
| error.unhandled_exception | error.unhandled_exception.rate |
| error.unspecified | error.unspecified.rate |
| failed_queries | failed_queries.rate |
| handled.requests | handled.requests.count |
| hits_per_query | hits_per_query.average |
| joinfailed | vds.idealstate.join_bucket.done_failed.rate |
| joinok | vds.idealstate.join_bucket.done_ok.rate |
| joinpending | vds.idealstate.join_bucket.pending.average |
| logd.processed.lines | logd.processed.lines.count |
| max_query_latency | query_latency.max |
| mean_query_latency | query_latency.average |
| mergefailed | vds.idealstate.merge_bucket.done_failed.rate |
| mergeok | vds.idealstate.merge_bucket.done_ok.rate |
| mergepending | vds.idealstate.merge_bucket.pending.average |
| peak_qps | peak_qps.max |
| queries | queries.rate |
| query_latency | content.proton.transport.query.latency.average |
| query_requests | content.proton.transport.query.count.rate |
| search_connections | search_connections.average |
| sentinel.uptime | sentinel.uptime.last |
| slobrok.heartbeats.failed | slobrok.heartbeats.failed.count |
| splitfailed | vds.idealstate.split_bucket.done_failed.rate |
| splitok | vds.idealstate.split_bucket.done_ok.rate |
| splitpending | vds.idealstate.split_bucket.pending.average |
| totalhits_per_query | totalhits_per_query.average |
| visit | vds.visitor.allthreads.created.sum.rate |
| visitorlifetime | vds.visitor.allthreads.averagevisitorlifetime.sum.average |
| visitorqueuewait | vds.visitor.allthreads.averagequeuewait.sum.average |
Vespa will not any longer implicitly load the "search" components" in containers which load the "document-api" components.
If your application depends on "search" functionality in a container specifying the <document-api> tag in services.xml,
make sure this container also specifies the <search> tag.