| Name | Description | Unit |
|---|---|---|
vds.datastored.alldisks.buckets |
Number of buckets managed | bucket |
vds.datastored.alldisks.docs |
Number of documents stored | document |
vds.datastored.alldisks.bytes |
Number of bytes stored | byte |
vds.datastored.alldisks.activebuckets |
Number of active buckets on the node | bucket |
vds.datastored.alldisks.readybuckets |
Number of ready buckets on the node | bucket |
vds.visitor.allthreads.averagevisitorlifetime |
Average lifetime of a visitor | millisecond |
vds.visitor.allthreads.averagequeuewait |
Average time an operation spends in input queue. | millisecond |
vds.visitor.allthreads.queuesize |
Size of input message queue. | operation |
vds.visitor.allthreads.completed |
Number of visitors completed | operation |
vds.visitor.allthreads.created |
Number of visitors created. | operation |
vds.visitor.allthreads.failed |
Number of visitors failed | operation |
vds.visitor.allthreads.averagemessagesendtime |
Average time it takes for messages to be sent to their target (and be replied to) | millisecond |
vds.visitor.allthreads.averageprocessingtime |
Average time used to process visitor requests | millisecond |
vds.visitor.allthreads.aborted |
Number of visitors aborted. | instance |
vds.visitor.allthreads.averagevisitorcreationtime |
Average time spent creating a visitor instance | millisecond |
vds.visitor.allthreads.destination_failure_replies |
Number of failure replies received from the visitor destination | instance |
vds.filestor.queuesize |
Size of input message queue. | operation |
vds.filestor.averagequeuewait |
Average time an operation spends in input queue. | millisecond |
vds.filestor.active_operations.size |
Number of concurrent active operations | operation |
vds.filestor.active_operations.latency |
Latency (in ms) for completed operations | millisecond |
vds.filestor.throttle_window_size |
Current size of async operation throttler window size | operation |
vds.filestor.throttle_waiting_threads |
Number of threads waiting to acquire a throttle token | thread |
vds.filestor.throttle_active_tokens |
Current number of active throttle tokens | instance |
vds.filestor.allthreads.mergemetadatareadlatency |
Time spent in a merge step to check metadata of current node to see what data it has. | millisecond |
vds.filestor.allthreads.mergedatareadlatency |
Time spent in a merge step to read data other nodes need. | millisecond |
vds.filestor.allthreads.mergedatawritelatency |
Time spent in a merge step to write data needed to current node. | millisecond |
vds.filestor.allthreads.mergeavgdatareceivedneeded |
Amount of data transferred from previous node in chain that we needed to apply locally. | byte |
vds.filestor.allthreads.mergebuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.mergebuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.mergebuckets.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.mergelatencytotal |
Latency of total merge operation, from master node receives it, until merge is complete and master node replies. | millisecond |
vds.filestor.allthreads.put_latency |
Latency of individual puts that are part of merge operations | millisecond |
vds.filestor.allthreads.remove_latency |
Latency of individual removes that are part of merge operations | millisecond |
vds.filestor.allstripes.throttled_rpc_direct_dispatches |
Number of times an RPC thread could not directly dispatch an async operation directly to Proton because it was disallowed by the throttle policy | instance |
vds.filestor.allstripes.throttled_persistence_thread_polls |
Number of times a persistence thread could not immediately dispatch a queued async operation because it was disallowed by the throttle policy | instance |
vds.filestor.allstripes.timeouts_waiting_for_throttle_token |
Number of times a persistence thread timed out waiting for an available throttle policy token | instance |
vds.filestor.allstripes.averagequeuewait |
Average time an operation spends in input queue. | millisecond |
vds.filestor.allthreads.put.count |
Number of requests processed. | operation |
vds.filestor.allthreads.put.failed |
Number of failed requests. | operation |
vds.filestor.allthreads.put.test_and_set_failed |
Number of operations that were skipped due to a test-and-set condition not met | operation |
vds.filestor.allthreads.put.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.put.request_size |
Size of requests, in bytes | byte |
vds.filestor.allthreads.remove.count |
Number of requests processed. | operation |
vds.filestor.allthreads.remove.failed |
Number of failed requests. | operation |
vds.filestor.allthreads.remove.test_and_set_failed |
Number of operations that were skipped due to a test-and-set condition not met | operation |
vds.filestor.allthreads.remove.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.remove.request_size |
Size of requests, in bytes | byte |
vds.filestor.allthreads.remove.not_found |
Number of requests that could not be completed due to source document not found. | request |
vds.filestor.allthreads.get.count |
Number of requests processed. | operation |
vds.filestor.allthreads.get.failed |
Number of failed requests. | operation |
vds.filestor.allthreads.get.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.get.request_size |
Size of requests, in bytes | byte |
vds.filestor.allthreads.get.not_found |
Number of requests that could not be completed due to source document not found. | request |
vds.filestor.allthreads.update.count |
Number of requests processed. | request |
vds.filestor.allthreads.update.failed |
Number of failed requests. | request |
vds.filestor.allthreads.update.test_and_set_failed |
Number of requests that were skipped due to a test-and-set condition not met | request |
vds.filestor.allthreads.update.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.update.request_size |
Size of requests, in bytes | byte |
vds.filestor.allthreads.update.latency_read |
Latency of the source read in the request. | millisecond |
vds.filestor.allthreads.update.not_found |
Number of requests that could not be completed due to source document not found. | request |
vds.filestor.allthreads.createiterator.count |
Number of requests processed. | request |
vds.filestor.allthreads.createiterator.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.createiterator.failed |
Number of failed requests. | request |
vds.filestor.allthreads.visit.count |
Number of requests processed. | request |
vds.filestor.allthreads.visit.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.visit.docs |
Number of entries read per iterate call | document |
vds.filestor.allthreads.visit.failed |
Number of failed requests. | request |
vds.filestor.allthreads.remove_location.count |
Number of requests processed. | request |
vds.filestor.allthreads.remove_location.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.remove_location.failed |
Number of failed requests. | request |
vds.filestor.allthreads.splitbuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.splitbuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.splitbuckets.latency |
Latency of successful requests. | request |
vds.filestor.allthreads.joinbuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.joinbuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.joinbuckets.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.deletebuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.deletebuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.deletebuckets.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.remove_by_gid.count |
Number of requests processed. | request |
vds.filestor.allthreads.remove_by_gid.failed |
Number of failed requests. | request |
vds.filestor.allthreads.remove_by_gid.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.setbucketstates.count |
Number of requests processed. | request |
vds.filestor.allthreads.setbucketstates.failed |
Number of failed requests. | request |
vds.filestor.allthreads.setbucketstates.latency |
Latency of successful requests. | millisecond |
vds.mergethrottler.averagequeuewaitingtime |
Time merges spent in the throttler queue | millisecond |
vds.mergethrottler.queuesize |
Length of merge queue | instance |
vds.mergethrottler.active_window_size |
Number of merges active within the pending window size | instance |
vds.mergethrottler.estimated_merge_memory_usage |
An estimated upper bound of the memory usage (in bytes) of the merges currently in the active window | byte |
vds.mergethrottler.bounced_due_to_back_pressure |
Number of merges bounced due to resource exhaustion back-pressure | instance |
vds.mergethrottler.locallyexecutedmerges.ok |
The number of successful merges for 'locallyexecutedmerges' | instance |
vds.mergethrottler.locallyexecutedmerges.failures.aborted |
The number of merges that failed because the storage node was (most likely) shutting down | operation |
vds.mergethrottler.locallyexecutedmerges.failures.bucketnotfound |
The number of operations that failed because the bucket did not exist | operation |
vds.mergethrottler.locallyexecutedmerges.failures.busy |
The number of merges that failed because the storage node was busy | operation |
vds.mergethrottler.locallyexecutedmerges.failures.exists |
The number of merges that were rejected due to a merge operation for their bucket already being processed | operation |
vds.mergethrottler.locallyexecutedmerges.failures.notready |
The number of merges discarded because distributor was not ready | operation |
vds.mergethrottler.locallyexecutedmerges.failures.other |
The number of other failures | operation |
vds.mergethrottler.locallyexecutedmerges.failures.rejected |
The number of merges that were rejected | operation |
vds.mergethrottler.locallyexecutedmerges.failures.timeout |
The number of merges that failed because they timed out towards storage | operation |
vds.mergethrottler.locallyexecutedmerges.failures.total |
Sum of all failures | operation |
vds.mergethrottler.locallyexecutedmerges.failures.wrongdistribution |
The number of merges that were discarded (flushed) because they were initiated at an older cluster state than the current | operation |
vds.mergethrottler.mergechains.ok |
The number of successful merges for 'mergechains' | operation |
vds.mergethrottler.mergechains.failures.busy |
The number of merges that failed because the storage node was busy | operation |
vds.mergethrottler.mergechains.failures.total |
Sum of all failures | operation |
vds.mergethrottler.mergechains.failures.exists |
The number of merges that were rejected due to a merge operation for their bucket already being processed | operation |
vds.mergethrottler.mergechains.failures.notready |
The number of merges discarded because distributor was not ready | operation |
vds.mergethrottler.mergechains.failures.other |
The number of other failures | operation |
vds.mergethrottler.mergechains.failures.rejected |
The number of merges that were rejected | operation |
vds.mergethrottler.mergechains.failures.timeout |
The number of merges that failed because they timed out towards storage | operation |
vds.mergethrottler.mergechains.failures.wrongdistribution |
The number of merges that were discarded (flushed) because they were initiated at an older cluster state than the current | operation |
vds.server.network.tls-handshakes-failed |
Number of client or server connection attempts that failed during TLS handshaking | operation |
| Number of TLS connection attempts failed due to bad or missing peer certificate credentials | failure | |
vds.server.network.client.tls-connections-established |
Number of secure mTLS connections established | connection |
vds.server.network.server.tls-connections-established |
Number of secure mTLS connections established | connection |
vds.server.network.client.insecure-connections-established |
Number of insecure (plaintext) connections established | connection |
vds.server.network.server.insecure-connections-established |
Number of insecure (plaintext) connections established | connection |
vds.server.network.tls-connections-broken |
Number of TLS connections broken due to failures during frame encoding or decoding | connection |
vds.server.network.failed-tls-config-reloads |
Number of times background reloading of TLS config has failed | failure |
| Number of operations that were aborted due to the node (or target bucket space) being unavailable | operation | |
vds.changedbucketownershiphandler.avg_abort_processing_time |
Average time spent aborting operations for changed buckets | millisecond |
vds.changedbucketownershiphandler.external_load_ops_aborted |
Number of outdated external load operations aborted | operation |
vds.changedbucketownershiphandler.ideal_state_ops_aborted |
Number of outdated ideal state operations aborted | operation |
vds.communication.bucket_space_mapping_failures |
Number of messages that could not be resolved to a known bucket space | operation |
vds.communication.convertfailures |
Number of messages that failed to get converted to storage API messages | operation |
vds.communication.exceptionmessageprocesstime |
Time transport thread uses to process a single message that fails with an exception thrown into communication manager | millisecond |
vds.communication.messageprocesstime |
Time transport thread uses to process a single message | millisecond |
vds.communication.messagequeue |
Size of input message queue. | item |
vds.communication.sendcommandlatency |
Average ms used to send commands to MBUS | millisecond |
vds.communication.sendreplylatency |
Average ms used to send replies to MBUS | millisecond |
vds.communication.toolittlememory |
Number of messages failed due to too little memory available | operation |
vds.datastored.bucket_space.active_buckets |
Number of active buckets in the bucket space | bucket |
vds.datastored.bucket_space.bucket_db.memory_usage.allocated_bytes |
The number of allocated bytes | byte |
vds.datastored.bucket_space.bucket_db.memory_usage.dead_bytes |
The number of dead bytes (<= used_bytes) | byte |
vds.datastored.bucket_space.bucket_db.memory_usage.onhold_bytes |
The number of bytes on hold | byte |
vds.datastored.bucket_space.bucket_db.memory_usage.used_bytes |
The number of used bytes (<= allocated_bytes) | byte |
vds.datastored.bucket_space.buckets_total |
Total number buckets present in the bucket space (ready + not ready) | bucket |
vds.datastored.bucket_space.entries |
Number of entries (documents + tombstones) stored in the bucket space | document |
vds.datastored.bucket_space.bytes |
Bytes stored across all documents in the bucket space | byte |
vds.datastored.bucket_space.docs |
Documents stored in the bucket space | document |
vds.datastored.bucket_space.ready_buckets |
Number of ready buckets in the bucket space | bucket |
vds.datastored.fullbucketinfolatency |
Amount of time spent to process a full bucket info request | millisecond |
vds.datastored.fullbucketinforeqsize |
Amount of distributors answered at once in full bucket info requests. | node |
vds.datastored.simplebucketinforeqsize |
Amount of buckets returned in simple bucket info requests | bucket |
vds.filestor.allthreads.applybucketdiff.count |
Number of requests processed. | request |
vds.filestor.allthreads.applybucketdiff.failed |
Number of failed requests. | request |
vds.filestor.allthreads.applybucketdiff.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.applybucketdiffreply |
Number of applybucketdiff replies that have been processed. | request |
vds.filestor.allthreads.bucketfixed |
Number of times bucket has been fixed because of corruption | bucket |
vds.filestor.allthreads.bucketverified.count |
Number of requests processed. | request |
vds.filestor.allthreads.bucketverified.failed |
Number of failed requests. | request |
vds.filestor.allthreads.bucketverified.latency |
Latency of successful requests. | request |
vds.filestor.allthreads.bytesmerged |
Total number of bytes merged into this node. | byte |
vds.filestor.allthreads.createbuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.createbuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.createbuckets.latency |
Latency of successful requests. | request |
vds.filestor.allthreads.failedoperations |
Number of operations throwing exceptions. | operation |
vds.filestor.allthreads.getbucketdiff.count |
Number of requests processed. | request |
vds.filestor.allthreads.getbucketdiff.failed |
Number of failed requests. | request |
vds.filestor.allthreads.getbucketdiff.latency |
Latency of successful requests. | request |
vds.filestor.allthreads.getbucketdiffreply |
Number of getbucketdiff replies that have been processed. | request |
vds.filestor.allthreads.internaljoin.count |
Number of requests processed. | request |
vds.filestor.allthreads.internaljoin.failed |
Number of failed requests. | request |
vds.filestor.allthreads.internaljoin.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.movedbuckets.count |
Number of requests processed. | request |
vds.filestor.allthreads.movedbuckets.failed |
Number of failed requests. | request |
vds.filestor.allthreads.movedbuckets.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.operations |
Number of operations processed. | operation |
vds.filestor.allthreads.readbucketinfo.count |
Number of requests processed. | request |
vds.filestor.allthreads.readbucketinfo.failed |
Number of failed requests. | request |
vds.filestor.allthreads.readbucketinfo.latency |
Latency of successful requests. | request |
vds.filestor.allthreads.readbucketlist.count |
Number of requests processed. | request |
vds.filestor.allthreads.readbucketlist.failed |
Number of failed requests. | request |
vds.filestor.allthreads.readbucketlist.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.recheckbucketinfo.count |
Number of requests processed. | request |
vds.filestor.allthreads.recheckbucketinfo.failed |
Number of failed requests. | request |
vds.filestor.allthreads.recheckbucketinfo.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.revert.count |
Number of requests processed. | request |
vds.filestor.allthreads.revert.failed |
Number of failed requests. | request |
vds.filestor.allthreads.revert.latency |
Latency of successful requests. | millisecond |
vds.filestor.allthreads.revert.not_found |
Number of requests that could not be completed due to source document not found. | request |
vds.filestor.allthreads.stat_bucket.count |
Number of requests processed. | request |
vds.filestor.allthreads.stat_bucket.failed |
Number of failed requests. | request |
vds.filestor.allthreads.stat_bucket.latency |
Latency of successful requests. | request |
vds.filestor.bucket_db_init_latency |
Time taken (in ms) to initialize bucket databases with information from the persistence provider | millisecond |
vds.filestor.directoryevents |
Number of directory events received. | operation |
vds.filestor.diskevents |
Number of disk events received. | operation |
vds.filestor.partitionevents |
Number of partition events received. | operation |
vds.filestor.pendingmerge |
Number of buckets currently being merged. | bucket |
vds.filestor.waitingforlockrate |
Amount of times a filestor thread has needed to wait for lock to take next message in queue. | operation |
vds.mergethrottler.mergechains.failures.aborted |
The number of merges that failed because the storage node was (most likely) shutting down | operation |
vds.mergethrottler.mergechains.failures.bucketnotfound |
The number of operations that failed because the bucket did not exist | operation |
vds.server.memoryusage |
Amount of memory used by the storage subsystem | byte |
vds.server.memoryusage_visiting |
Message use from visiting | byte |
vds.server.message_memory_use.highpri |
Message use from high priority storage messages | byte |
vds.server.message_memory_use.lowpri |
Message use from low priority storage messages | byte |
vds.server.message_memory_use.normalpri |
Message use from normal priority storage messages | byte |
vds.server.message_memory_use.total |
Message use from storage messages | byte |
vds.server.message_memory_use.veryhighpri |
Message use from very high priority storage messages | byte |
vds.state_manager.invoke_state_listeners_latency |
Time spent (in ms) propagating state changes to internal state listeners | millisecond |
vds.visitor.cv_queueevictedwaittime |
Milliseconds waiting in create visitor queue, for visitors that was evicted from queue due to higher priority visitors coming | millisecond |
vds.visitor.cv_queuefull |
Number of create visitor messages failed as queue is full | operation |
vds.visitor.cv_queuesize |
Size of create visitor queue | item |
vds.visitor.cv_queuetimeoutwaittime |
Milliseconds waiting in create visitor queue, for visitors that timed out while in the visitor quueue | millisecond |
vds.visitor.cv_queuewaittime |
Milliseconds waiting in create visitor queue, for visitors that was added to visitor queue but scheduled later | millisecond |
vds.visitor.cv_skipqueue |
Number of times we could skip queue as we had free visitor spots | operation |
vds.server.network.rpc-capability-checks-failed |
Number of RPC operations that failed to due one or more missing capabilities | failure |
vds.server.network.status-capability-checks-failed |
Number of status page operations that failed to due one or more missing capabilities | failure |
vds.server.fnet.num-connections |
Total number of connection objects | connection |