在源代码控制中存储kubernetes配置的最佳实践

问题描述

在Kubernetes文档站点的几个地方，他们建议您将配置YAML文件存储在源代码管理中，以便轻松进行版本跟踪，回滚和部署。

我和我的同事正在试图决定我们的git存储库的结构。

• 我们已经决定，由于配置可以在不更改应用程序代码的情况下更改，我们希望将配置存储在单独的共享存储库。


• 我们可能需要多个版本的某些组件在给定环境（群集）内并行运行。这些版本可能有不同的配置。

似乎有很多潜在的变化，并且它们都有缺点。构建这样一个仓库的可接受的方式是什么？

解决方案

我相信还没有确立的标准。我发现helm的图表太复杂了，特别是在k8s集群上运行另一个非托管组件。这是一个我们遵循的工作流程，对于15个微服务和5个不同环境（devx2，staging，qa，prod）的设置非常有效。

2键想法：

1. 将kubernetes配置存储在具有
   其他构建工具的源代码仓库中。例如：除了具有构建/释放特定微服务的工具的微服务源代码外。


2. 根据您定位的环境模拟kubernetes配置，并根据您定位的环境呈现模板。
   

这个工具可以通过编写一些bash脚本或者与Makefile等进行整合来解决。

编辑：回答一些问题在注释中

应用程序源代码库被用作单一来源。所以这意味着如果一切正常，应该永远不要将更改从kubernetes集群移动到存储库。

在我们的工作流程中禁止直接在服务器上进行更改。如果它确实发生过，我们必须手动确保它们再次进入应用程序库。

再一次，只需要注意存储在源代码中的配置实际上是模板并使用非常宽松。这意味着某些配置是从CI工具中提取的，因为它们被渲染，有些来自只存在于集群上的秘密（如数据库密码，API令牌等）。

In several places on the Kubernetes documentation site they recommend that you store your configuration YAML files inside source control for easy version-tracking, rollback, and deployment.

My colleagues and I are currently in the process of trying to decide on the structure of our git repository.

• We have decided that since configuration can change without any changes to the app code, that we would like to store configurations in a separate, shared repository.
• We may need multiple versions of some components running side-by-side within a given environment (cluster). These versions may have different configurations.

There seem to be a lot of potential variations, and all of them have shortcomings. What is the accepted way to structure such a repository?

解决方案

There is no established standard yet, I believe. I find helm's charts too complicated to start with, especially having another unmanaged component running on the k8s cluster. This is a workflow that we follow that works quite well for a setup of 15ish microservices, and 5 different environments (devx2, staging, qa, prod).

The 2 key ideas:

1. Store kubernetes configurations in the same source repo that has theother build tooling. Eg: alongside the microservice source code which has the tooling for building/releasing that particular microservice.
2. Template the kubernetes configuration with something like jinja and render the templates according to the environment you're targeting.

The tooling is reasonably straightforward to figure out by putting together a few bash scripts or integrating with a Makefile etc.

EDIT: to answer some of the questions in the comment

The application source code repository is used as the single source of truth. So that means that if everything works as it should, changes should never be moved from the kubernetes cluster to the repository.

Changes directly on the server are prohibited in our workflow. If it ever does happen, we have to manually make sure they enter the application repository again.

Again, just want to note that the configurations stored in the source code are actually templates and use secretKeyRef quite liberally. This means that some configurations are coming in from the CI tooling as they are rendered and some are coming in from secrets that live only on the cluster (like database passwords, API tokens etc.).

这篇关于在源代码控制中存储kubernetes配置的最佳实践的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！