Testing with Java JUnit tests

This is the Vespa Testing reference for Vespa application system tests written in Java, as JUnit 5 unit test.

These tests verify the behaviour of a Vespa application by using its HTTP interfaces. To write tests without Java, see basic HTTP test reference.

See below for how to run the tests.

Test suites

The testing documentation defines three test scenarios, comprised of four test code categories. The system test framework in com.yahoo.vespa:tenant-cd-api uses Java annotations to declare what category a JUnit test class belongs to. To run tests with Maven belonging to a specific category, a JUnit 5 test tag must be specified:

$ mvn test -D test.categories=system -D vespa.test.config=/path-to/test-config.json

For an example including system and staging tests, check out this sample application test suite.

Running tests

The Vespa CLI makes it easy to set up a test deployment on the local machine, and tests can then be run by pointing the test runtime at this deployment:

$ vespa deploy --wait 600
$ mvn test -D test.categories=system -D vespa.test.config=/path-to/test-config.json

The test config file should define the endpoints for each of the clusters in services.xml as fields under a localEndpoints JSON object, like:

{
    "localEndpoints": {
        "query-service": "http://localhost:8080/",
        "feed-service" : "http://localhost:8081/"
    }
}

To run a staging test, set up a test deployment with old code, run setup, upgrade the deployment, run verification:

$ # load old application code, e.g., using git, then deploy it
$ vespa deploy --wait 600
$ mvn test -D test.categories=staging-setup -D vespa.test.config=/path-to/test-config.json
$ # make changes to the application, or use, e.g., git to check out the new code, then deploy again
$ vespa deploy --wait 120
$ mvn test -D test.categories=staging -D vespa.test.config=/path-to/test-config.json