This is the Vespa Testing reference for Vespa application system tests written in Java, as JUnit 5 unit test.
These tests verify the behaviour of a Vespa application by using its HTTP interfaces. To write tests without Java, see basic HTTP test reference.
See below for how to run the tests.
The testing documentation defines three test scenarios,
comprised of four test code categories. The system test framework in
com.yahoo.vespa:tenant-cd-api
uses Java annotations to declare what category a JUnit test class belongs to.
To run tests with Maven belonging to a specific category, a JUnit 5 test tag must be specified:
$ mvn test -D test.categories=system -D vespa.test.config=/path-to/test-config.json
Category | Annotation | JUnit tag | Description |
---|---|---|---|
System test | @SystemTest | system | Independent, functional tests |
Staging setup | @StagingSetup | staging-setup | Set state before upgrade |
Staging test | @StagingTest | staging | Verify state after upgrade |
Production test | @ProductionTest | production | Verify domain specific metrics |
For an example including system and staging tests, check out this sample application test suite.
The Vespa CLI makes it easy to set up a test deployment on the local machine, and tests can then be run by pointing the test runtime at this deployment:
$ vespa deploy --wait 600 $ mvn test -D test.categories=system -D vespa.test.config=/path-to/test-config.json
The test config file should define the endpoints for each of the clusters in
services.xml as fields under a localEndpoints
JSON object, like:
{ "localEndpoints": { "query-service": "http://localhost:8080/", "feed-service" : "http://localhost:8081/" } }
To run a staging test, set up a test deployment with old code, run setup, upgrade the deployment, run verification:
$ # load old application code, e.g., using git, then deploy it $ vespa deploy --wait 600 $ mvn test -D test.categories=staging-setup -D vespa.test.config=/path-to/test-config.json $ # make changes to the application, or use, e.g., git to check out the new code, then deploy again $ vespa deploy --wait 120 $ mvn test -D test.categories=staging -D vespa.test.config=/path-to/test-config.json