Testing with Java JUnit tests

This is the Vespa Testing reference for Vespa application system tests written in Java, as JUnit 5 unit test.

These tests verify the behaviour of a Vespa application by using its HTTP interfaces. To write tests without Java, see basic HTTP test reference.

See the testing guide for examples of how to run the tests.

Test suites

The testing documentation defines three test scenarios, comprised of four test code categories. The system test framework in com.yahoo.vespa:tenant-cd-api uses Java annotations to declare what category a JUnit test class belongs to. To run tests with Maven belonging to a specific category, a JUnit 5 test tag must be specified:

$ mvn test -D test.categories=system -D vespa.test.config=/path-to/test-config.json

For an example including system and staging tests, check out the sample application test suite.

TestNG

Combining Vespa JUnit 5 test suites with unit tests in TestNG is possible. You'll need to explicitly configure Maven's surefire plugin to enable integration for both frameworks. To execute the Vespa test suites specify -D test.categories=[tag], where [tag] is one of the values listed in Test suites.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit-platform</artifactId>
            <version>${surefire.vespa.tenant.version)</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>${surefire.vespa.tenant.version)</version>
        </dependency>
    </dependencies>
</plugin>