问题描述
我目前正在尝试使用Prometheus和Prometheus适配器运行自动缩放演示,而我想知道是否有一种方法可以根据Prometheus从另一个部署中删除的指标来自动缩放我的一个部署.
我现在有2种不同的部署,kafka-consumer-application(我要扩展)和kafka-exporter(公开了我将用于扩展的kafka指标).我知道如果在同一部署中将它们都作为容器都可以自动缩放,但是问题是kafka-exporter也可以自动缩放并且不理想,所以我想将它们分开.我尝试使用以下HPA,但无法正常工作:
种类:HorizontalPodAutoscalerapiVersion:自动缩放/v2beta1元数据:名称:consumer-hpa规格:scaleTargetRef:apiVersion:apps/v1种类:部署名称:kafka-consumer-applicationminReplicas:1maxReplicas:10指标:-类型:物件目的:目标:kafka出口商metricName:"kafka_consumergroup_lag"targetValue:5
我不确定我是不是做错了什么,或者这不是我可以做的,所以任何建议都值得赞赏.
谢谢!
注意:我正在使用此配置运行适配器:
规则:默认值:false资源:{}风俗:-seriesQuery:"kafka_consumergroup_lag"资源:覆盖:kubernetes_namespace:{资源:名称空间"}kubernetes_pod_name:{资源:"pod"}名称:匹配:"kafka_consumergroup_lag"作为:"kafka_consumergroup_lag"metricsQuery:"avg_over_time(kafka_consumergroup_lag {topic ="my-topic",consumergroup ="we-consume"} [1m])"``
在 kubernetes文档,您可以阅读:
因此,使用外部指标,您的HPA yaml可能如下所示:
种类:HorizontalPodAutoscalerapiVersion:自动缩放/v2beta2元数据:名称:consumer-hpa规格:scaleTargetRef:apiVersion:apps/v1种类:部署名称:kafka-consumer-applicationminReplicas:1maxReplicas:10指标:-类型:外部外部的:指标:名称:kafka_consumergroup_lag#selector:#matchLabels:#主题:我的主题"目标:类型:平均值平均值:5
如果您有多个kafka出口商,则可以使用选择器
对其进行过滤(源):
还可以查看此Stack问题.
Im currently trying to run an autoscaling demo using prometheus and the prometheus adapter, and i was wondering if there is a way to autoscale one of my deployments based on metrics that prometheus scrapes from another deployment.
What i have right now are 2 different deployments, kafka-consumer-application (which i want to scale) and kafka-exporter (which exposes the kafka metrics that I'll be using for scaling). I know that if I have both of them as containers in the same deployment the autoscaling works, but the issue is that the kafka-exporter also gets autoscaled and its not ideal, so i want to separate them. I tried with the following HPA but i could not get it to work:
kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta1
metadata:
name: consumer-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: kafka-consumer-application
minReplicas: 1
maxReplicas: 10
metrics:
- type: object
object:
target: kafka-exporter
metricName: "kafka_consumergroup_lag"
targetValue: 5
Im not sure if im doing something wrong or if this is just not something i can do, so any advice is appreciated.
Thanks!
Note: im running the adapter with this config:
rules:
default: false
resource: {}
custom:
- seriesQuery: 'kafka_consumergroup_lag'
resources:
overrides:
kubernetes_namespace: {resource: "namespace"}
kubernetes_pod_name: {resource: "pod"}
name:
matches: "kafka_consumergroup_lag"
as: "kafka_consumergroup_lag"
metricsQuery: 'avg_over_time(kafka_consumergroup_lag{topic="my-topic",consumergroup="we-consume"}[1m])'
``
In kubernetes documentation you can read:
So using external metrics, your HPA yaml could look like following:
kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2beta2
metadata:
name: consumer-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: kafka-consumer-application
minReplicas: 1
maxReplicas: 10
metrics:
- type: External
external:
metric:
name: kafka_consumergroup_lag
#selector:
# matchLabels:
# topic: "my-topic"
target:
type: AverageValue
averageValue: 5
If you have more than one kafka-exporter you can use selector
to filter it (source):
Also have a look at this Stack question.
这篇关于Kubernetes HPA使用来自另一个部署的指标的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!