# Data management and backup

[](/en/operations/data-management.html.md "View as Markdown") 

This guide covers data management operations for Vespa Cloud applications, including automated backups, document export, feed, and bulk updates and removals.

## Automated Backups

Depending on [plan](https://vespa.ai/pricing/), content clusters are automatically backed up when a [`<backup>`](../reference/applications/deployment.html#backup) element is specified in _deployment.xml_. Vespa Cloud manages the backup schedule, storage, and lifecycle with no external tooling required. Backups will run at the configured frequency while also respecting any [maintenance block windows](../reference/applications/deployment.html#block-change) (`<block-change maintenance="true">`) defined for the instance.

```
```
<instance id="default">
    <backup frequency="7d" />
    <prod>
        <region>aws-us-east-1c</region>
    </prod>
</instance>
```
```

The first backup of a cluster is not created immediately when backups are enabled. A cluster becomes eligible for its first backup only once a content node has been running for at least one full backup interval — for example, 7 days for a `7d` frequency, measured from when the node was first allocated. Subsequent backups then follow at the configured frequency. Backup eligibility is evaluated approximately once per hour, so the first backup may appear up to an hour after the cluster becomes eligible.

Backups are retained for three backup intervals (e.g. 21 days for a 7-day frequency). The most recent fully completed backup is always retained regardless of age. See [Restore from Backup](#restore) for how to request a restore.

If you prefer to manage backups yourself, documents can be exported manually using `vespa visit` as shown in the [Google Cloud Function example](https://github.com/vespa-engine/sample-apps/tree/master/examples/google-cloud/cloud-functions#backup---experimental).

## Restore from Backup

Restoring from a backup is handled by Vespa Cloud. To initiate a restore, contact [Vespa Support](https://vespa.ai/support/). Response time and priority handling are governed by your [support plan](https://vespa.ai/pricing/).

Restore requires a deployed target cluster with:

- The same number of content nodes as the backup.
- At least equivalent disk capacity per node as at the time of the backup.

Note that content redistribution is usually required after restoration. See [backup reference](../reference/applications/deployment.html#backup) for details.

## Export documents

 **Note:** The examples below use the [Vespa CLI](../clients/vespa-cli.html). Ensure you have the latest version installed.

To export documents, configure the application to export from, then select zone, container cluster and schema - example:

```
$ vespa config set application vespa-team.vespacloud-docsearch.default

$ vespa visit --zone prod.aws-us-east-1c --cluster default --selection doc | head
```

Some of the parameters above are redundant if unambiguous. Here, the application is set up using a template found in [multinode-HA](https://github.com/vespa-engine/sample-apps/tree/master/examples/operations/multinode-HA) with multiple container clusters. This example [visit](../writing/visiting.html) documents from the `doc` schema.

Use a [fieldset](../schemas/documents.html#fieldsets) to export document IDs only:

```
$ vespa visit --zone prod.aws-us-east-1c --cluster default --selection doc --field-set '[id]' | head
```

 **Note:** Configuring the [`documentid`](../reference/schemas/schemas.html#documentid) field to be an attribute in the schema avoids that this requires disk access and, hence, speeds up the exporting process.

As the name implies, fieldsets are useful to select a subset of fields to export. Note that, if disk access is required to fetch a field from the fieldset, selecting fewer fields does not speed up the exporting process as the same amount of data is read from the index. The data transfer out of the Vespa application is smaller with fewer fields.

For copying documents between applications, see [cloning applications and data](cloning).

## Feed

If a document feed is generated with `vespa visit` (above), it is already in [JSON Lines](https://jsonlines.org/) feed-ready format by default:

```
$ vespa visit | vespa feed - -t $ENDPOINT
```

Find more examples in [cloning applications and data](cloning).

A document export generated using [/document/v1](../writing/document-v1-api-guide.html) is slightly different from the .jsonl output from `vespa visit` (e.g., fields like a continuation token are added). Extract the `document` objects before feeding:

```
$ gunzip -c docs.gz |[jq](https://stedolan.github.io/jq/)'.documents[]' | \
  vespa feed - -t $ENDPOINT
```

## Delete

To remove all documents in a Vespa deployment—or a selection of them—run a _deletion visit_. Use the `DELETE` HTTP method, and fetch only the continuation token from the response:

```
#!/bin/bash

set -x

# The ENDPOINT must be a regional endpoint, do not use '*.g.vespa-app.cloud/'
ENDPOINT="https://vespacloud-docsearch.vespa-team.aws-us-east-1c.z.vespa-app.cloud"
NAMESPACE=open
DOCTYPE=doc
CLUSTER=documentation

# doc.path =~ "^/old/" -- all documents under the /old/ directory:
SELECTION='doc.path%3D~%22%5E%2Fold%2F%22'

continuation=""

while
  token=$( curl -X DELETE -s \
           --cert data-plane-public-cert.pem \
           --key data-plane-private-key.pem \
           "${ENDPOINT}/document/v1/${NAMESPACE}/${DOCTYPE}/docid?selection=${SELECTION}&cluster=${CLUSTER}&${continuation}" \
           | tee >( jq . > /dev/tty ) | jq -re .continuation )
do
  continuation="continuation=${token}"
done
```

Each request will return a response after roughly one minute—change this by specifying _timeChunk_ (default 60).

To purge all documents in a document export (above), generate a feed with `remove`-entries for each document ID, like:

```
$ gunzip -c docs.gz | jq '[.documents[] | {remove: .id} ]' | head

[
  {
    "remove": "id:open:doc::open/documentation/schemas.html"
  },
  {
    "remove": "id:open:doc::open/documentation/securing-your-vespa-installation.html"
  },
```

Complete example for a single chunk:

```
$ gunzip -c docs.gz | jq '[.documents[] | {remove: .id} ]' | \
  vespa feed - -t $ENDPOINT
```

## Update

To update all documents in a Vespa deployment—or a selection of them—run an _update visit_. Use the `PUT` HTTP method, and specify a partial update in the request body:

```
#!/bin/bash

set -x

# The ENDPOINT must be a regional endpoint, do not use '*.g.vespa-app.cloud/'
ENDPOINT="https://vespacloud-docsearch.vespa-team.aws-us-east-1c.z.vespa-app.cloud"
NAMESPACE=open
DOCTYPE=doc
CLUSTER=documentation

# doc.inlinks == "some-url" -- the weightedset<string> inlinks has the key "some-url"
SELECTION='doc.inlinks%3D%3D%22some-url%22'

continuation=""

while
  token=$( curl -X PUT -s \
           --cert data-plane-public-cert.pem \
           --key data-plane-private-key.pem \
           --data '{ "fields": { "inlinks": { "remove": { "some-url": 0 } } } }' \
           "${ENDPOINT}/document/v1/${NAMESPACE}/${DOCTYPE}/docid?selection=${SELECTION}&cluster=${CLUSTER}&${continuation}" \
           | tee >( jq . > /dev/tty ) | jq -re .continuation )
do
  continuation="continuation=${token}"
done
```

Each request will return a response after roughly one minute—change this by specifying _timeChunk_ (default 60).

## Using /document/v1/ api

To get started with a document export, find the _namespace_ and _document type_ by listing a few IDs. Hit the [/document/v1/](../reference/api/document-v1.html) ENDPOINT. Restrict to one CLUSTER, see [content clusters](../reference/applications/services/content.html):

```
$ curl \
  --cert data-plane-public-cert.pem \
  --key data-plane-private-key.pem \
  "$ENDPOINT/document/v1/?cluster=$CLUSTER"
```

For ID export only, use a [fieldset](../schemas/documents.html#fieldsets):

```
$ curl \
  --cert data-plane-public-cert.pem \
  --key data-plane-private-key.pem \
  "$ENDPOINT/document/v1/?cluster=$CLUSTER&fieldSet=%5Bid%5D"
```

From an ID, like _id:open:doc::open/documentation/schemas.html_, extract

- NAMESPACE: open
- DOCTYPE: doc

Example script:

```
#!/bin/bash

set -x

# The ENDPOINT must be a regional endpoint, do not use '*.g.vespa-app.cloud/'
ENDPOINT="https://vespacloud-docsearch.vespa-team.aws-us-east-1c.z.vespa-app.cloud"
NAMESPACE=open
DOCTYPE=doc
CLUSTER=documentation

continuation=""
idx=0

while
  ((idx+=1))
  echo "$continuation"
  printf -v out "%05g" $idx
  filename=${NAMESPACE}-${DOCTYPE}-${out}.data.gz
  echo "Fetching data..."
  token=$( curl -s \
           --cert data-plane-public-cert.pem \
           --key data-plane-private-key.pem \
           "${ENDPOINT}/document/v1/${NAMESPACE}/${DOCTYPE}/docid?wantedDocumentCount=1000&concurrency=4&cluster=${CLUSTER}&${continuation}" \
           | tee >( gzip > ${filename} ) | jq -re .continuation )
do
  continuation="continuation=${token}"
done
```

If only a few documents are returned per response, _wantedDocumentCount_ (default 1, max 1024) can be specified for a lower bound on the number of documents per response, if that many documents still remain.

Specifying _concurrency_ (default 1, max 100) increases throughput, at the cost of resource usage. This also increases the number of documents per response, and _could_ lead to excessive memory usage in the HTTP container when many large documents are buffered to be returned in the same response.

 Copyright © 2026 - [Cookie Preferences](#)

### On this page:

- [Data management and backup](#page-title)
- [Automated Backups](#backup)
- [Restore from Backup](#restore)
- [Export documents](#export-documents)
- [Feed](#feed)
- [Delete](#delete)
- [Update](#update)
- [Using /document/v1/ api](#using-document-v1-api)