如何在Kubernetes上的Elasticsearch中禁用交换?

问题描述

根据官方es文档，禁用交换功能是Elasticsearch可获得的最佳性能提升之一.

As per the official es docs, disabling swapping is one of the best performance boosts available to Elasticsearch.

但是，事实证明配置起来很困难.我花了很多时间研究并尝试使用不同的方法来使用Kubernetes上的官方ES docker镜像来禁用交换.

However, it's proving to be difficult to configure. I've spent a number of hours researching and attempting different methods to disable swapping using the official ES docker image on Kubernetes.

将bootstrap.memory_lock: true设置为env变量时，映像无法启动，并显示错误:Unable to lock JVM Memory: error=12, reason=Cannot allocate memory. This can result in part of the JVM being swapped out. Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536.正如文档所指出的那样，这是意料之中的.我什至安装了带有设置的自定义/etc/security/limits.conf，但是失败了.

When setting bootstrap.memory_lock: true as an env variable, the image fails to boot up with the error: Unable to lock JVM Memory: error=12, reason=Cannot allocate memory. This can result in part of the JVM being swapped out. Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536. As the docs point out, this is kind of expected. I've even mounted a custom /etc/security/limits.conf with the settings, but that's failed.

在k8s上使用官方es映像时，建议的禁用交换的方法是什么?

What is the recommended way to disable swapping when using the official es image on k8s?

而且，这是我的Yaml的相关部分

And, here are the relevant sections of my yaml

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: elastic-data
spec:
  serviceName: elastic-data
  replicas: 1
  template:
    spec:
      securityContext:
        runAsUser: 0
        fsGroup: 0
      containers:
      - name: elastic-data
        image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.4.0
        env:
        - name: ES_JAVA_OPTS
          value: "-Xms2g -Xmx2g"
        - name: cluster.name
          value: "elastic-devs"
        - name: node.name
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: discovery.zen.ping.unicast.hosts
          value: "elastic-master.default.svc.cluster.local"
        - name: node.master
          value: "false"
        - name: node.ingest
          value: "false"
        - name: node.data
          value: "true"
        - name: network.host
          value: "0.0.0.0"
        - name: path.data
          value: /usr/share/elasticsearch/data
        - name: indices.memory.index_buffer_size
          value: "512MB"
        - name: bootstrap.memory_lock
          value: "true"
        resources:
          requests:
            memory: "3Gi"
          limits:
            memory: "3Gi"
        ports:
        - containerPort: 9300
          name: transport
        - containerPort: 9200
          name: http
        volumeMounts:
        - name: data-volume
          mountPath: /usr/share/elasticsearch/data
        - name: swappiness-config
          mountPath: /etc/security/limits.conf
          subPath: limits.conf
      volumes:
      - name: data-volume
        persistentVolumeClaim:
          claimName: pvc-es
     - name: swappiness-config
       configMap:
         name: swappiness-config
         items:
          - key: limits.conf
             path: limits.conf

limits.conf

limits.conf

elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch hard nofile 65536
elasticsearch soft nofile 65536

推荐答案

我认为，我的Yaml中的ulimit未被识别，因此我遵循了这篇文章，并创建了一个带有用于设置设置的自定义入口点的图像.

I think, the ulimits in my yaml weren't being recognized, so I followed this post and created an image with a custom entrypoint that set the settings.

这篇关于如何在Kubernetes上的Elasticsearch中禁用交换?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！