Configuring Alerts for ETCD Nodes

The ETCD runs on separate nodes in a multi-node environment as opposed to a container within Kubernetes environment. A Node-Exporter runs on each of the ETCD nodes to obtain host level metrics. Also, the CEE Prometheus starts scrapping the metrics automatically after deployment.

You can create alerting rules based on the ETCD Node-Exporter metrics. To configure alerting rules based on ETCD Node-Exporter metrics, use the ETCD Node IP as the instance label instead of the Pod name in the expression.

Important

The Node-Exporter on ETCD is not running as a Kuberenets Pod.

The following examples configure alerting rules based on ETCD Node-Exporter metrics.

Example:

The following expression configures alerts based on the availability of host memory (less than 30%):

((node_memory_MemAvailable_bytes{{{instance="<ETCD-Node-IP>:9100"}}} / node_memory_MemTotal_bytes{{{instance="<ETCD-Node-IP>:9100"}}}) < 30

The following expression configures alerts based on the average CPU usage for five minutes. (greater than 70%):

sum(avg without (cpu)(irate(node_cpu_seconds_total{instance="<ETCD-Node-IP>:9100",mode!="idle"}[5m]))) * 100 > 70