Monitoring

Monitor nxtStore cluster health, OSD status, and I/O performance through nxtRadar integration — key metrics, alert thresholds, and observability configuration.

Overview

nxtStore cluster health and performance data is exported to nxtRadar for centralized monitoring and alerting. This page covers the key metrics to monitor, recommended alert thresholds, and how to configure the integration.

Administrator Access Required — This operation requires the admin role. Contact your NxtSight administrator if you do not have sufficient permissions.

Prerequisites

• Administrator credentials with the admin role
• nxtRadar deployed and accessible (see nxtRadar Admin Guide)
• Metric scrape target configured for the nxtStore cluster metrics endpoint

---

Key Metrics and Alert Thresholds

Metric	Namespace	Alert Threshold	Action
Cluster health	NxtSight_storage_health	HEALTH_WARN	Investigate immediately
OSD down count	NxtSight_storage_osd_down	> 0	Replace or recover failed OSD
Pool used %	NxtSight_storage_pool_used_pct	> 70%	Plan capacity expansion
Recovery I/O rate	NxtSight_storage_recovery_bytes_sec	Sustained > 200 MB/s	Consider I/O throttling
PG inconsistent count	NxtSight_storage_pg_inconsistent	> 0	Run ceph health detail and repair
Replication lag (RGW)	NxtSight_storage_rgw_sync_lag_sec	Sustained > 30s	Check network bandwidth to RGW
OSD apply latency	NxtSight_storage_osd_apply_latency_ms	> 20 ms	Investigate OSD or disk health

---

Configuring nxtRadar Integration

1. Verify metrics endpoint

   The nxtStore cluster exposes metrics on the management node. Verify the endpoint is reachable:

   Check metrics endpoint

   curl http://<MGMT_NODE_IP>:9283/metrics | head -20
   

   Port 9283 is the default metrics exporter port deployed by nxtDeploy.

2. Add scrape target in nxtRadar

   Navigate to Monitoring → Administration → Scrape Targets → Add Target:

   Field	Value
   URL	http://<MGMT_NODE_IP>:9283/metrics
   Scrape Interval	60s (storage metrics don’t need sub-minute resolution)
   Labels	service=nxtStore, cluster=<CLUSTER_NAME>

   Or via CLI:

   Add nxtStore scrape target

   nxtradar target add \
     --url http://<MGMT_NODE_IP>:9283/metrics \
     --interval 60s \
     --label service=nxtStore \
     --label cluster=prod-storage
   
3. Create alert rules

   Navigate to Monitoring → Alerting → Alert Rules and create rules for each threshold in the table above.

   Example alert rule for pool utilization:

   alert-storage-capacity.yaml

   name: nxtStore-pool-capacity-warning
   metric: NxtSight_storage_pool_used_pct
   condition: ">"
   threshold: 70
   evaluation_period: 10m
   severity: warning
   notification_channels:
     - ops-email
   

   Alert rules appear in the Active Rules list and evaluate against live storage metrics.

---

Built-In Dashboards

The nxtRadar portal includes pre-built nxtStore dashboards. Navigate to Monitoring → Dashboards and search for “nxtStore” or “Storage”:

Dashboard	Shows
nxtStore Cluster Overview	Health state, OSD counts, capacity, recovery activity
nxtStore Pool Utilization	Per-pool used %, available bytes, PG counts
nxtStore OSD Performance	Per-OSD latency, IOPS, throughput
nxtStore Recovery	Active recovery operations, estimated completion time

Pin the “nxtStore Cluster Overview” dashboard to your nxtRadar home screen for constant visibility during on-call rotations.

---

Cluster CLI Health Check

For quick health checks without opening the nxtRadar portal, use the management CLI directly from a cluster node:

Quick health overview

ceph status

OSD performance snapshot

ceph osd perf

Pool I/O statistics (5-second window)

ceph osd pool stats

Active slow requests

ceph health detail | grep -i slow

---

Next Steps

nxtRadar Admin Guide

Configure the monitoring platform that collects and displays nxtStore metrics

Capacity Planning

Use utilization metrics to plan cluster expansion before thresholds are reached

Troubleshooting

Diagnose the issues surfaced by monitoring alerts

nxtRadar Alert Rules

Configure notification channels for storage health alerts