| vds.datastored.alldisks.buckets |
Number of buckets managed |
bucket |
| vds.datastored.alldisks.docs |
Number of documents stored |
document |
| vds.datastored.alldisks.bytes |
Number of bytes stored |
byte |
| vds.datastored.alldisks.activebuckets |
Number of active buckets on the node |
bucket |
| vds.datastored.alldisks.readybuckets |
Number of ready buckets on the node |
bucket |
| vds.visitor.allthreads.averagevisitorlifetime |
Average lifetime of a visitor |
millisecond |
| vds.visitor.allthreads.averagequeuewait |
Average time an operation spends in input queue. |
millisecond |
| vds.visitor.allthreads.queuesize |
Size of input message queue. |
operation |
| vds.visitor.allthreads.completed |
Number of visitors completed |
operation |
| vds.visitor.allthreads.created |
Number of visitors created. |
operation |
| vds.visitor.allthreads.failed |
Number of visitors failed |
operation |
| vds.visitor.allthreads.averagemessagesendtime |
Average time it takes for messages to be sent to their target (and be replied to) |
millisecond |
| vds.visitor.allthreads.averageprocessingtime |
Average time used to process visitor requests |
millisecond |
| vds.visitor.allthreads.aborted |
Number of visitors aborted. |
instance |
| vds.visitor.allthreads.averagevisitorcreationtime |
Average time spent creating a visitor instance |
millisecond |
| vds.visitor.allthreads.destination_failure_replies |
Number of failure replies received from the visitor destination |
instance |
| vds.filestor.queuesize |
Size of input message queue. |
operation |
| vds.filestor.averagequeuewait |
Average time an operation spends in input queue. |
millisecond |
| vds.filestor.active_operations.size |
Number of concurrent active operations |
operation |
| vds.filestor.active_operations.latency |
Latency (in ms) for completed operations |
millisecond |
| vds.filestor.throttle_window_size |
Current size of async operation throttler window size |
operation |
| vds.filestor.throttle_waiting_threads |
Number of threads waiting to acquire a throttle token |
thread |
| vds.filestor.throttle_active_tokens |
Current number of active throttle tokens |
instance |
| vds.filestor.allthreads.mergemetadatareadlatency |
Time spent in a merge step to check metadata of current node to see what data it has. |
millisecond |
| vds.filestor.allthreads.mergedatareadlatency |
Time spent in a merge step to read data other nodes need. |
millisecond |
| vds.filestor.allthreads.mergedatawritelatency |
Time spent in a merge step to write data needed to current node. |
millisecond |
| vds.filestor.allthreads.mergeavgdatareceivedneeded |
Amount of data transferred from previous node in chain that we needed to apply locally. |
byte |
| vds.filestor.allthreads.mergebuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.mergebuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.mergebuckets.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.mergelatencytotal |
Latency of total merge operation, from master node receives it, until merge is complete and master node replies. |
millisecond |
| vds.filestor.allthreads.put_latency |
Latency of individual puts that are part of merge operations |
millisecond |
| vds.filestor.allthreads.remove_latency |
Latency of individual removes that are part of merge operations |
millisecond |
| vds.filestor.allstripes.throttled_rpc_direct_dispatches |
Number of times an RPC thread could not directly dispatch an async operation directly to Proton because it was disallowed by the throttle policy |
instance |
| vds.filestor.allstripes.throttled_persistence_thread_polls |
Number of times a persistence thread could not immediately dispatch a queued async operation because it was disallowed by the throttle policy |
instance |
| vds.filestor.allstripes.timeouts_waiting_for_throttle_token |
Number of times a persistence thread timed out waiting for an available throttle policy token |
instance |
| vds.filestor.allstripes.averagequeuewait |
Average time an operation spends in input queue. |
millisecond |
| vds.filestor.allthreads.put.count |
Number of requests processed. |
operation |
| vds.filestor.allthreads.put.failed |
Number of failed requests. |
operation |
| vds.filestor.allthreads.put.test_and_set_failed |
Number of operations that were skipped due to a test-and-set condition not met |
operation |
| vds.filestor.allthreads.put.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.put.request_size |
Size of requests, in bytes |
byte |
| vds.filestor.allthreads.remove.count |
Number of requests processed. |
operation |
| vds.filestor.allthreads.remove.failed |
Number of failed requests. |
operation |
| vds.filestor.allthreads.remove.test_and_set_failed |
Number of operations that were skipped due to a test-and-set condition not met |
operation |
| vds.filestor.allthreads.remove.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.remove.request_size |
Size of requests, in bytes |
byte |
| vds.filestor.allthreads.remove.not_found |
Number of requests that could not be completed due to source document not found. |
request |
| vds.filestor.allthreads.get.count |
Number of requests processed. |
operation |
| vds.filestor.allthreads.get.failed |
Number of failed requests. |
operation |
| vds.filestor.allthreads.get.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.get.request_size |
Size of requests, in bytes |
byte |
| vds.filestor.allthreads.get.not_found |
Number of requests that could not be completed due to source document not found. |
request |
| vds.filestor.allthreads.update.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.update.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.update.test_and_set_failed |
Number of requests that were skipped due to a test-and-set condition not met |
request |
| vds.filestor.allthreads.update.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.update.request_size |
Size of requests, in bytes |
byte |
| vds.filestor.allthreads.update.latency_read |
Latency of the source read in the request. |
millisecond |
| vds.filestor.allthreads.update.not_found |
Number of requests that could not be completed due to source document not found. |
request |
| vds.filestor.allthreads.createiterator.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.createiterator.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.createiterator.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.visit.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.visit.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.visit.docs |
Number of entries read per iterate call |
document |
| vds.filestor.allthreads.visit.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.remove_location.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.remove_location.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.remove_location.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.splitbuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.splitbuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.splitbuckets.latency |
Latency of successful requests. |
request |
| vds.filestor.allthreads.joinbuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.joinbuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.joinbuckets.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.deletebuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.deletebuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.deletebuckets.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.setbucketstates.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.setbucketstates.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.setbucketstates.latency |
Latency of successful requests. |
millisecond |
| vds.mergethrottler.averagequeuewaitingtime |
Time merges spent in the throttler queue |
millisecond |
| vds.mergethrottler.queuesize |
Length of merge queue |
instance |
| vds.mergethrottler.active_window_size |
Number of merges active within the pending window size |
instance |
| vds.mergethrottler.bounced_due_to_back_pressure |
Number of merges bounced due to resource exhaustion back-pressure |
instance |
| vds.mergethrottler.locallyexecutedmerges.ok |
The number of successful merges for 'locallyexecutedmerges' |
instance |
| vds.mergethrottler.locallyexecutedmerges.failures.aborted |
The number of merges that failed because the storage node was (most likely) shutting down |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.bucketnotfound |
The number of operations that failed because the bucket did not exist |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.busy |
The number of merges that failed because the storage node was busy |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.exists |
The number of merges that were rejected due to a merge operation for their bucket already being processed |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.notready |
The number of merges discarded because distributor was not ready |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.other |
The number of other failures |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.rejected |
The number of merges that were rejected |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.timeout |
The number of merges that failed because they timed out towards storage |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.total |
Sum of all failures |
operation |
| vds.mergethrottler.locallyexecutedmerges.failures.wrongdistribution |
The number of merges that were discarded (flushed) because they were initiated at an older cluster state than the current |
operation |
| vds.mergethrottler.mergechains.ok |
The number of successful merges for 'mergechains' |
operation |
| vds.mergethrottler.mergechains.failures.busy |
The number of merges that failed because the storage node was busy |
operation |
| vds.mergethrottler.mergechains.failures.total |
Sum of all failures |
operation |
| vds.mergethrottler.mergechains.failures.exists |
The number of merges that were rejected due to a merge operation for their bucket already being processed |
operation |
| vds.mergethrottler.mergechains.failures.notready |
The number of merges discarded because distributor was not ready |
operation |
| vds.mergethrottler.mergechains.failures.other |
The number of other failures |
operation |
| vds.mergethrottler.mergechains.failures.rejected |
The number of merges that were rejected |
operation |
| vds.mergethrottler.mergechains.failures.timeout |
The number of merges that failed because they timed out towards storage |
operation |
| vds.mergethrottler.mergechains.failures.wrongdistribution |
The number of merges that were discarded (flushed) because they were initiated at an older cluster state than the current |
operation |
| vds.server.network.tls-handshakes-failed |
Number of client or server connection attempts that failed during TLS handshaking |
operation |
| vds.server.network.peer-authorization-failures |
Number of TLS connection attempts failed due to bad or missing peer certificate credentials |
failure |
| vds.server.network.client.tls-connections-established |
Number of secure mTLS connections established |
connection |
| vds.server.network.server.tls-connections-established |
Number of secure mTLS connections established |
connection |
| vds.server.network.client.insecure-connections-established |
Number of insecure (plaintext) connections established |
connection |
| vds.server.network.server.insecure-connections-established |
Number of insecure (plaintext) connections established |
connection |
| vds.server.network.tls-connections-broken |
Number of TLS connections broken due to failures during frame encoding or decoding |
connection |
| vds.server.network.failed-tls-config-reloads |
Number of times background reloading of TLS config has failed |
failure |
| vds.bouncer.unavailable_node_aborts |
Number of operations that were aborted due to the node (or target bucket space) being unavailable |
operation |
| vds.changedbucketownershiphandler.avg_abort_processing_time |
Average time spent aborting operations for changed buckets |
millisecond |
| vds.changedbucketownershiphandler.external_load_ops_aborted |
Number of outdated external load operations aborted |
operation |
| vds.changedbucketownershiphandler.ideal_state_ops_aborted |
Number of outdated ideal state operations aborted |
operation |
| vds.communication.bucket_space_mapping_failures |
Number of messages that could not be resolved to a known bucket space |
operation |
| vds.communication.convertfailures |
Number of messages that failed to get converted to storage API messages |
operation |
| vds.communication.exceptionmessageprocesstime |
Time transport thread uses to process a single message that fails with an exception thrown into communication manager |
millisecond |
| vds.communication.messageprocesstime |
Time transport thread uses to process a single message |
millisecond |
| vds.communication.messagequeue |
Size of input message queue. |
item |
| vds.communication.sendcommandlatency |
Average ms used to send commands to MBUS |
millisecond |
| vds.communication.sendreplylatency |
Average ms used to send replies to MBUS |
millisecond |
| vds.communication.toolittlememory |
Number of messages failed due to too little memory available |
operation |
| vds.datastored.bucket_space.active_buckets |
Number of active buckets in the bucket space |
bucket |
| vds.datastored.bucket_space.bucket_db.memory_usage.allocated_bytes |
The number of allocated bytes |
byte |
| vds.datastored.bucket_space.bucket_db.memory_usage.dead_bytes |
The number of dead bytes (<= used_bytes) |
byte |
| vds.datastored.bucket_space.bucket_db.memory_usage.onhold_bytes |
The number of bytes on hold |
byte |
| vds.datastored.bucket_space.bucket_db.memory_usage.used_bytes |
The number of used bytes (<= allocated_bytes) |
byte |
| vds.datastored.bucket_space.buckets_total |
Total number buckets present in the bucket space (ready + not ready) |
bucket |
| vds.datastored.bucket_space.bytes |
Bytes stored across all documents in the bucket space |
byte |
| vds.datastored.bucket_space.docs |
Documents stored in the bucket space |
document |
| vds.datastored.bucket_space.ready_buckets |
Number of ready buckets in the bucket space |
bucket |
| vds.datastored.fullbucketinfolatency |
Amount of time spent to process a full bucket info request |
millisecond |
| vds.datastored.fullbucketinforeqsize |
Amount of distributors answered at once in full bucket info requests. |
node |
| vds.datastored.simplebucketinforeqsize |
Amount of buckets returned in simple bucket info requests |
bucket |
| vds.filestor.allthreads.applybucketdiff.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.applybucketdiff.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.applybucketdiff.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.applybucketdiffreply |
Number of applybucketdiff replies that have been processed. |
request |
| vds.filestor.allthreads.bucketfixed |
Number of times bucket has been fixed because of corruption |
bucket |
| vds.filestor.allthreads.bucketverified.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.bucketverified.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.bucketverified.latency |
Latency of successful requests. |
request |
| vds.filestor.allthreads.bytesmerged |
Total number of bytes merged into this node. |
byte |
| vds.filestor.allthreads.createbuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.createbuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.createbuckets.latency |
Latency of successful requests. |
request |
| vds.filestor.allthreads.failedoperations |
Number of operations throwing exceptions. |
operation |
| vds.filestor.allthreads.getbucketdiff.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.getbucketdiff.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.getbucketdiff.latency |
Latency of successful requests. |
request |
| vds.filestor.allthreads.getbucketdiffreply |
Number of getbucketdiff replies that have been processed. |
request |
| vds.filestor.allthreads.internaljoin.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.internaljoin.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.internaljoin.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.movedbuckets.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.movedbuckets.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.movedbuckets.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.operations |
Number of operations processed. |
operation |
| vds.filestor.allthreads.readbucketinfo.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.readbucketinfo.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.readbucketinfo.latency |
Latency of successful requests. |
request |
| vds.filestor.allthreads.readbucketlist.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.readbucketlist.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.readbucketlist.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.recheckbucketinfo.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.recheckbucketinfo.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.recheckbucketinfo.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.revert.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.revert.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.revert.latency |
Latency of successful requests. |
millisecond |
| vds.filestor.allthreads.revert.not_found |
Number of requests that could not be completed due to source document not found. |
request |
| vds.filestor.allthreads.stat_bucket.count |
Number of requests processed. |
request |
| vds.filestor.allthreads.stat_bucket.failed |
Number of failed requests. |
request |
| vds.filestor.allthreads.stat_bucket.latency |
Latency of successful requests. |
request |
| vds.filestor.bucket_db_init_latency |
Time taken (in ms) to initialize bucket databases with information from the persistence provider |
millisecond |
| vds.filestor.directoryevents |
Number of directory events received. |
operation |
| vds.filestor.diskevents |
Number of disk events received. |
operation |
| vds.filestor.partitionevents |
Number of partition events received. |
operation |
| vds.filestor.pendingmerge |
Number of buckets currently being merged. |
bucket |
| vds.filestor.waitingforlockrate |
Amount of times a filestor thread has needed to wait for lock to take next message in queue. |
operation |
| vds.mergethrottler.mergechains.failures.aborted |
The number of merges that failed because the storage node was (most likely) shutting down |
operation |
| vds.mergethrottler.mergechains.failures.bucketnotfound |
The number of operations that failed because the bucket did not exist |
operation |
| vds.server.memoryusage |
Amount of memory used by the storage subsystem |
byte |
| vds.server.memoryusage_visiting |
Message use from visiting |
byte |
| vds.server.message_memory_use.highpri |
Message use from high priority storage messages |
byte |
| vds.server.message_memory_use.lowpri |
Message use from low priority storage messages |
byte |
| vds.server.message_memory_use.normalpri |
Message use from normal priority storage messages |
byte |
| vds.server.message_memory_use.total |
Message use from storage messages |
byte |
| vds.server.message_memory_use.veryhighpri |
Message use from very high priority storage messages |
byte |
| vds.state_manager.invoke_state_listeners_latency |
Time spent (in ms) propagating state changes to internal state listeners |
millisecond |
| vds.visitor.cv_queueevictedwaittime |
Milliseconds waiting in create visitor queue, for visitors that was evicted from queue due to higher priority visitors coming |
millisecond |
| vds.visitor.cv_queuefull |
Number of create visitor messages failed as queue is full |
operation |
| vds.visitor.cv_queuesize |
Size of create visitor queue |
item |
| vds.visitor.cv_queuetimeoutwaittime |
Milliseconds waiting in create visitor queue, for visitors that timed out while in the visitor quueue |
millisecond |
| vds.visitor.cv_queuewaittime |
Milliseconds waiting in create visitor queue, for visitors that was added to visitor queue but scheduled later |
millisecond |
| vds.visitor.cv_skipqueue |
Number of times we could skip queue as we had free visitor spots |
operation |
| vds.server.network.rpc-capability-checks-failed |
Number of RPC operations that failed to due one or more missing capabilities |
failure |
| vds.server.network.status-capability-checks-failed |
Number of status page operations that failed to due one or more missing capabilities |
failure |
| vds.server.fnet.num-connections |
Total number of connection objects |
connection |