运维别卷系列 - 云原生监控平台 之 06.prometheus pushgateway 实践

文章目录

• • @[toc]
  • Pushgateway 简介
  • Pushgateway 部署
  • Pushgateway 测试

Pushgateway 简介

• 官方建议在某些有限的情况下使用 Pushgateway。盲目地使用 Pushgateway 而不是 Prometheus 通常的拉取模型进行常规指标收集时，存在几个陷阱：
  • 当通过单个 Pushgateway 监控多个实例时，Pushgateway 既是单点故障，又是潜在的瓶颈。
  • 将失去 Prometheus 通过 up 指标（每次抓取生成）的自动实例运行状况监控。
  • Pushgateway 永远不会清除推送给它的时间序列，并将永远将它们暴露给 Prometheus，除非通过 Pushgateway 的 API 手动删除这些时间序列。

Pushgateway 部署

创建 svc

---
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    app: pushgateway
  name: pushgateway-svc
  namespace: monitor
spec:
  type: NodePort
  ports:
  - name: gateway
    port: 9091
    protocol: TCP
    targetPort: 9091
    nodePort: 31091
  selector:
    app: pushgateway

创建 deployment

---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
  labels:
    app: pushgateway
  name: pushgateway
  namespace: monitor
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pushgateway
  template:
    metadata:
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: "9091"
        prometheus.io/scrape: "true"
        prometheus.io/type: pushgateway
      labels:
        app: pushgateway
    spec:
      containers:
      - args:
        - "--web.listen-address=:9091"
        image: prom/pushgateway:v1.8.0
        livenessProbe:
          failureThreshold: 60
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: gateway
          timeoutSeconds: 1
        name: pushgateway
        ports:
        - containerPort: 9091
          name: gateway
          protocol: TCP
        readinessProbe:
          failureThreshold: 60
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: gateway
          timeoutSeconds: 1
      terminationGracePeriodSeconds: 0

Pushgateway 测试

echo "hello_world 2024" | curl --data-binary @- http://192.168.11.167:31091/metrics/job/test-pushgateway/instance/192.168.11.167/study/pushgateway

删除 Pushgateway 上对应 lable 的数据

echo "hello_world_2024 2024" | curl --data-binary @- http://192.168.11.167:31091/metrics/job/test-pushgateway/instance/192.168.11.167/study/test_delete
echo "hello_world_2025 2025" | curl --data-binary @- http://192.168.11.167:31091/metrics/job/test-pushgateway/instance/192.168.11.167/study/test_delete
echo "hello_world_2025 2025" | curl --data-binary @- http://192.168.11.167:31091/metrics/job/test-pushgateway/instance/192.168.11.167/study/test_delete_2

curl -XDELETE http://192.168.11.167:31091/metrics/job/test-pushgateway/instance/192.168.11.167/study/test_delete