运维别卷系列 - 云原生监控平台 之 02.prometheus exporter 实践

文章目录

• • @[toc]
  • exporter 简介
  • 常用的 exporter
  • node-exporter 实践

exporter 简介

常用的 exporter

• Elasticsearch exporter
• MySQL server exporter
• Kafka exporter
• Gluster exporter
• Ceph exporter
• Node/system metrics exporter
• Blackbox exporter
• kube-state-metrics
• cAdvisor

node-exporter 实践

• 同 Prometheus，这里的 node-exporter 也是在 k8s 内部署的，部署的 node-exporter 版本是 v1.8.0

创建 svc

---
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    app: node-exporter
  name: node-exporter-svc
  namespace: monitor
spec:
  clusterIP: None
  ports:
  - name: http
    port: 9100
    protocol: TCP
  selector:
    app: node-exporter
  type: ClusterIP

创建 daemonset

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  annotations:
  labels:
    app: node-exporter
  name: node-exporter
  namespace: monitor
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
      annotations:
        prometheus.io/port: "9100"
        prometheus.io/scrape: "true"
        prometheus.io/type: "node_exporter"
    spec:
      hostNetwork: true
      hostPID: true
      hostIPC: true
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.8.0
        args:
        - "--path.rootfs=/rootfs"
        ports:
        - containerPort: 9100
          hostPort: 9100
          name: http
        volumeMounts:
        - mountPath: /rootfs
          name: root
      volumes:
      - hostPath:
          path: /
        name: root

prometheus 配置服务发现

global:
  scrape_interval: 30s
  evaluation_interval: 30s
  scrape_timeout: 10s
  external_labels:
    prometheus: monitor/k8s
rule_files:
- /etc/prometheus/rules/*.yml
scrape_configs:
- job_name: prometheus
  metrics_path: '/metrics'
  static_configs:
  - targets: ['prometheus-svc.monitor.svc.cluster.local:9090']

- job_name: 'kubernetes-pods'
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
    target_label: __address__
  - action: labelmap
    regex: __meta_kubernetes_pod_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    action: replace
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_pod_name]
    action: replace
    target_label: kubernetes_pod_name

curl -XPOST http://192.168.11.167:31090/-/reload