# Prometheus

An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

# Prometheus Operator

Following CRDs are available right now:

  • Prometheus, which defines a desired Prometheus deployment.
  • Alertmanager, which defines a desired Alertmanager deployment.
  • ThanosRuler, which defines a desired Thanos Ruler deployment.
  • ServiceMonitor, which declaratively specifies how groups of Kubernetes services should be monitored. The Operator automatically generates Prometheus scrape configuration based on the current state of the objects in the API server.
  • PodMonitor, which declaratively specifies how group of pods should be monitored. The Operator automatically generates Prometheus scrape configuration based on the current state of the objects in the API server.
  • Probe, which declaratively specifies how groups of ingresses or static targets should be monitored. The Operator automatically generates Prometheus scrape configuration based on the definition.
  • PrometheusRule, which defines a desired set of Prometheus alerting and/or recording rules. The Operator generates a rule file, which can be used by Prometheus instances.
  • AlertmanagerConfig, which declaratively specifies subsections of the Alertmanager configuration, allowing routing of alerts to custom receivers, and setting inhibit rules.

NOTE: Unfortunately there is no CRD for AlertManager silences right now (GitHub Feature request).

# Queries

# Troubleshooting Kubernetes

# Find Containers without configured Resources

sum by (namespace,pod)(count by (namespace,pod,container)(kube_pod_container_info{container!="",namespace="<NAMESPACE>"}) unless sum by (namespace,pod,container)(kube_pod_container_resource_limits{resource="cpu"}))

More information's: Kubernetes resource limits

# Alertmanager

Unfortunately Google's chat webhooks are not compatible with Alertmanager's webhooks. But there is a community project called calert that makes this possible.

NOTE: If you use calert the Alertmanager's receiver have to be named as the Google Chat name.