Kubernetes实战总结 - DevOps实现

一、概述

Git：一个开源的分布式版本控制系统，可以有效、高速地处理从很小到非常大的项目版本管理。

Jenkins：一个开源软件项目，是基于Java开发的一种持续集成工具，用于监控持续重复的工作。

Maven：项目对象模型(POM)，可以通过一小段描述信息来管理项目的构建，报告和文档的项目管理工具软件。

Harbor：构建企业级私有Docker镜像的仓库的开源解决方案。

Kubernetes：一个开源的，用于管理云平台中多个主机上的容器化的应用。

 

---

二、Java镜像构建

启动脚本run.sh：

#!/bin/sh

java ${JVM_OPTS} ${JAVA_OPTS} -jar *.jar ${CLI_OPTS} 2>&1 | tee -a ${OUTPUT}

构建脚本Dockerfile：

# 基础镜像
FROM hub.jhmy.com/base/centos

# 维护者
MAINTAINER leozhang

# 切换目录
WORKDIR /root

# 添加环境变量
ENV JDK_VERSION=jdk1.8.0_191
ENV JAVA_HOME=/usr/local/$JDK_VERSION
ENV PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/lib/

ENV JAVA_OPTS='' CLI_OPTS='' OUTPUT='logs/output.log'
ENV JVM_OPTS='-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=2'

# 设置时区语言
ENV TZ='Asia/Shanghai'
ENV LANG='C.UTF-8' LANGUAGE='C.UTF-8' LC_ALL='C.UTF-8'

# 添加程序包
ADD $JDK_VERSION.tar.gz /usr/local/
COPY run.sh .

# 执行命令
RUN chmod a+x run.sh && rm -rf `ls | grep -v "^run.sh$"` && mkdir -p /root/logs

# 构建镜像示例
# docker build -t hub.jhmy.com/base/java:1.8 .
# 启动容器示例
# docker run -dit --name=java hub.jhmy.com/base/java:1.8

环境变量说明：

 

---

三、Java项目镜像构建

增加项目依赖脚本addPlugins.py：

# -*- coding:utf-8 -*-
# ****************************
# Author      :leozhang
# Date        :2020/7/16
# File        :addPlugins.py
# Description :增加POM文件依赖
# ****************************
import os
pomplug=os.environ["PLUGINS"]

newpom = ""
with open("pom.xml", "r+") as fp:
    for line in fp.readlines():
        if(line.find("<build>") != -1):
            line += "\n" + pomplug + "\n"
        newpom += line

with open("pom.xml", "r+") as f:
    f.write(newpom)print(newpom)

构建脚本Dockerfile：

# 基础镜像
FROM hub.jhmy.com/base/java:1.8

# 维护者
MAINTAINER leozhang

# 工作目录
WORKDIR /root

# 复制文件到镜像
COPY *.jar .
COPY lib/ lib/

# 查看当前目录
RUN ls -l .

# 容器启动时运行命令
CMD ["./run.sh"]

# 构建镜像示例
# docker build -t hub.jhmy.com/test/appservice .
# 启动容器示例
# docker run -dit --name=appservice hub.jhmy.com/test/appservice

镜像构建脚本buildImage.sh：

#!/bin/sh
# creator: leozhang
# up-date: 2020/03/21
# description: buildImage.sh


# 公共参数
env=$1
name=$2
tag=$3
file=$4

# 设置仓库地址和账户
harbor=hub.jhmy.com
username=jenkins
password=$(echo Q2hpc2NkY0AxMjMK | base64 -d)

# 修改镜像名称为小写
image=${harbor}/${env}/$(echo ${name} | tr '[A-Z]' '[a-z]')

# 默认名Dockerfile
if  [ ! "${file}" ] ;then
    file=Dockerfile
fi

# 构建镜像
echo "************* build image ***************"
docker build -t ${image}:${tag} -f ${file} .

echo "************* push image ***************"
# 登录仓库
docker login -u ${username} -p ${password} https://${harbor}

# 上传镜像
docker push ${image}:${tag}

# 如果非latest标签，则更新latest标签镜像
if [ "${tag}" != "latest" ]; then
    docker tag ${image}:${tag} ${image}:latest
    docker push ${image}:latest
fi
# 登出仓库
docker logout https://${harbor}

 

---

四、Java项目容器化部署

Deployment文件：

apiVersion: apps/v1     # 资源版本
kind: Deployment    #资源类型
metadata:       #资源元数据：名称、命名空间等
  name: appservice
  namespace: default

spec:
  replicas: 3   #副本数
  selector:     #选择器
    matchLabels:
      app: appservice

  template:     #Pod模板
    metadata:   #Pod元数据：标签、注解等
      labels:
        app: appservice
      annotations:
        version: "20200715"

    spec:       #Pod配置：名称、镜像、拉取策略、端口等
      containers:
      - name: appservice
        image: hub.jhmy.com/test/appservice:latest
        imagePullPolicy: Always

        env:    #环境变量
        - name: JAVA_OPTS
          value: "-Dprops=/root/appservice.properties -Dlogpath=/root/logs"
        ports:
        - name: rest
          containerPort: 20114

        resources:  #资源限制：cpu、mem
          limits:
            memory: 2Gi
            cpu: 1

        livenessProbe:     #就绪检测&生存检测：初始时间、超时、检测方式等
          initialDelaySeconds: 60
          timeoutSeconds: 10
          tcpSocket:
            port: 20114
        readinessProbe:
          initialDelaySeconds: 20
          timeoutSeconds: 10
          tcpSocket:
            port: 20114

        volumeMounts:       #存储挂载
        - name: config
          mountPath: /root/appservice.properties
          subPath: appservice.properties
        - name: log
          mountPath: /root/logs

      volumes:      #存储声明：服务配置、持久化存储请求、本地存储
      - name: config
        configMap:
          name: appservice
      - name: log
        hostPath:
          path: /home/ymt/logs/appservice
          type: DirectoryOrCreate

      #affinity:    #节点亲和性
      #  nodeAffinity:
      #    requiredDuringSchedulingIgnoredDuringExecution:
      #      nodeSelectorTerms:
      #      - matchExpressions:
      #        - key: app
      #          operator: In
      #          values:
      #          - appservice

ConfigMap文件：

apiVersion: v1
kind: ConfigMap
metadata:
  name: appservice
  namespace: default
data:
  appservice.properties: |-
    server.port=20114
　　 ......
    ......
    ......

Service文件：

apiVersion: v1
kind: Service
metadata:
  name: apprestserver
  namespace: default
spec:
  selector:
    app: apprestserver
  ports:
  - name: rest
    port: 20114

 

---

五、Jenkins持续部署

Pipeline任务脚本：

def NOW = new Date().format("yyyyMMddHHmmss")
def SERVER = env.JOB_BASE_NAME.toLowerCase()
pipeline {
   agent any
//   parameters {
//         string(name: 'TAG', defaultValue: 'latest', description: '镜像标签' )
//         string(name: 'BRANCH', defaultValue: 'test_0629', description: '分支名称' )
//         string(name: 'URL', defaultValue: 'http://10.88.88.226:8090/jhmy/source/wx_v1/appdataservicetemp.git', description: '项目地址' )
//         string(name: 'PLUGINS', defaultValue: '<plugins> <plugin> ...... </plugin> </plugins> ', description: '相关依赖' )
//

   environment {
        ENV="test"
        MVN="/application/mvnBranch/bin/mvn"
        SCRIPT="${JENKINS_HOME}/workspace/Docker/updateConfigs/script"
    }

   stages {
      stage('Git Clone') {
         steps {
             git branch: '${BRANCH}', credentialsId: 'yf3b_gitlab', url: '${URL}'
         }
      }

      stage('Clean Package') {
         steps {
            sh "python ${SCRIPT}/addPlugins.py"
            sh "${MVN} clean package -Dmaven.test.skip=true"
         }
         post { success { archiveArtifacts "target/*.jar" } }
      }

      stage('Build Image') {
          steps {
            dir('target') {
                sh "cp ${SCRIPT}/Dockerfile-jar Dockerfile"
                sh "sh ${SCRIPT}/buildImage.sh ${ENV} ${SERVER} ${TAG}"
            }
          }
      }

      stage('Rolling Update') {
          steps {
            sh "kubectl config use-context k8s-ymt"
            sh """kubectl patch deployment ${SERVER} --patch '{"spec":{"template":{"metadata":{"annotations":{"version":"${NOW}"}}}}}'"""
            sh "sleep 30 && kubectl get pod -o wide | grep ${SERVER}"
          }
      }

      stage('Clean up') {
          steps { deleteDir() }
      }
   }
}