Monitoring and Alerting
rancher-monitoring application, you can quickly deploy leading open-source monitoring and alerting solutions onto your cluster.
Prometheus lets you view metrics from your Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or Grafana, which is an analytics viewing platform deployed along with Prometheus.
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, or restore crashed servers.
rancher-monitoring operator, introduced in Rancher v2.5, is powered by Prometheus, Grafana, Alertmanager, the Prometheus Operator, and the Prometheus adapter.
The monitoring application allows you to:
- Monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments
- Define alerts based on metrics collected via Prometheus
- Create custom Grafana dashboards
- Configure alert-based notifications via Email, Slack, PagerDuty, etc. using Prometheus Alertmanager
- Defines precomputed, frequently needed or computationally expensive expressions as new time series based on metrics collected via Prometheus
- Expose collected metrics from Prometheus to the Kubernetes Custom Metrics API via Prometheus Adapter for use in HPA
How Monitoring Works
For an explanation of how the monitoring components work together, see this page.
Default Components and Deployments
By default, the monitoring application deploys Grafana dashboards (curated by the kube-prometheus project) onto a cluster.
It also deploys an Alertmanager UI and a Prometheus UI. For more information about these tools, see Built-in Dashboards.
Default Metrics Exporters
By default, Rancher Monitoring deploys exporters (such as node-exporter and kube-state-metrics).
These default exporters automatically scrape metrics for CPU and memory from all components of your Kubernetes cluster, including your workloads.
The monitoring application deploys some alerts by default. To see the default alerts, go to the Alertmanager UI and click Expand all groups.
Components Exposed in the Rancher UI
For a list of monitoring components exposed in the Rancher UI, along with common use cases for editing them, see this section.
Role-based Access Control
For information on configuring access to monitoring, see this page.
- Enable monitoring
- Uninstall monitoring
- Monitoring workloads
- Customizing Grafana dashboards
- Persistent Grafana dashboards
- Debugging high memory usage
- Migrating from Monitoring V1 to V2
Configuring Monitoring Resources in Rancher
The configuration reference assumes familiarity with how monitoring components work together. For more information, see How Monitoring Works.
Configuring Helm Chart Options
For more information on
rancher-monitoring chart options, including options to set resource limits and requests, see this page.
Windows Cluster Support
Available as of v2.5.8
When deployed onto an RKE1 Windows cluster, Monitoring V2 will now automatically deploy a windows-exporter DaemonSet and set up a ServiceMonitor to collect metrics from each of the deployed Pods. This will populate Prometheus with
windows_ metrics that are akin to the
node_ metrics exported by node_exporter for Linux hosts.
To be able to fully deploy Monitoring V2 for Windows, all of your Windows hosts must have a minimum wins version of v0.1.0.
For more details on how to upgrade wins on existing Windows hosts, refer to the section on Windows cluster support for Monitoring V2.
There is a known issue that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend setting
prometheus.prometheusSpec.resources.memory.limit to 2500 Mi and
prometheus.prometheusSpec.resources.memory.request to 1750 Mi.
For tips on debugging high memory usage, see this page.