Inspecting Java services

Debug interface

The following RPC commands are available in the container for debugging purposes:

NameArgumentsResult
debug.dump-query-packets cluster-name, 0/1 (disable, enable) Dumps query packets to $VESPA_HOME/logs/vespa/qrs/qrs.<server-index>.<cluster-name>.query.dump debug.dump-result-packets cluster-name, 0/1 (disable, enable) Dumps query packets to$VESPA_HOME/logs/vespa/qrs/qrs.<server-index>.<cluster-name>.result.dump
debug.output-search-chain search-chain-name Returns a human readable description of a given search chain and the search chains it forwards to
debug.backend-statistics cluster-name Returns (backend, active connections, total connections) tuples separated with the same order into three arrays
The commands can be called by using vespa-rpc-invoke (call it without arguments to get usage information). Example - output the searchers in a search chain:
$vespa-rpc-invoke tcp/localhost:19102 debug.output-search-chain s:vespa  JConsole Determine the state of each running Java Vespa service using JConsole. JConsole is distributed along with the Java developer kit. It can be started as follows: $ jconsole <host>:<port>

where the host and port determine which service to attach to. For security purposes the JConsole tool can not directly attach to Vespa services from external machines.

Connecting to a Vespa instance

To attach a JConsole to a Vespa service running on another host, create a tunnel from the JConsole host to the Vespa service host. This can for example be done by setting up two SSH tunnels as follows:

$ssh -N -L<port1>:localhost:<port1> <service-host> &$ ssh -N -L<port2>:localhost:<port2> <service-host> &

where port1 and port2 are determined by the type of service (see below). A JConsole can then be attached to the service as follows:
$jconsole localhost:<port1>  Port numbers: Service Port 1 Port 2 QRS 19015 19016 Docproc 19123 19124 Updated port information can be found by running: $ vespa-model-inspect service <servicename>

where the resulting RMIREGISTRY and JMX lines determine port1 and port2, respectively.