说明：本实验在该篇章https://www.cnblogs.com/douyi/p/12031600.html集群基础上模拟真实的一个线上环境，在k8s集群中部署LNMP环境运行Discuz论坛
一、下载公共镜像【deploy上操作】
1：下载mysql5.7镜像
1 [root@master ~]# docker pull mysql:5.7 2 5.7: Pulling from library/mysql 3 d599a449871e: Pull complete 4 f287049d3170: Pull complete 5 08947732a1b0: Pull complete 6 96f3056887f2: Pull complete 7 871f7f65f017: Pull complete 8 1dd50c4b99cb: Pull complete 9 5bcbdf508448: Pull complete 10 02a97db830bd: Pull complete 11 c09912a99bce: Pull complete 12 08a981fc6a89: Pull complete 13 818a84239152: Pull complete 14 Digest: sha256:5779c71a4730da36f013a23a437b5831198e68e634575f487d37a0639470e3a8 15 Status: Downloaded newer image for mysql:5.7
2：下载nginx-php-fpm镜像
1 [root@master ~]# docker pull richarvey/nginx-php-fpm 2 Using default tag: latest 3 latest: Pulling from richarvey/nginx-php-fpm 4 9d48c3bd43c5: Pull complete 5 4bf02c0a37c8: Pull complete 6 9ce49f939c6f: Pull complete 7 2fa33c09831c: Pull complete 8 64d703475f7c: Pull complete 9 687e14517432: Pull complete 10 5d8ec717fece: Pull complete 11 9fe12d3f6add: Pull complete 12 e860070fe546: Pull complete 13 36baa049bdfe: Pull complete 14 4073f665d7d8: Pull complete 15 42a3d0ae6c56: Pull complete 16 682e06ece949: Pull complete 17 3c3d9faab7c5: Pull complete 18 258d52d772e4: Pull complete 19 41336c37b6cd: Pull complete 20 9ef84aa1b775: Pull complete 21 94c642e23a42: Pull complete 22 f601338f6a53: Pull complete 23 0c5f20947667: Pull complete 24 50f6eb03fd0b: Pull complete 25 75174a25ab08: Pull complete 26 8e16258eb3ba: Pull complete 27 447fabc9be1b: Pull complete 28 a485db129518: Pull complete 29 de30cb0b9f89: Pull complete 30 5724050c9b48: Pull complete 31 c20deb9ffa63: Pull complete 32 52031a5fa502: Pull complete 33 6aea67ff4e17: Pull complete 34 Digest: sha256:f8109029deb401ee83c4fcd1e737cc496f42aca6a6e8f445f8501ea977be6784 35 Status: Downloaded newer image for richarvey/nginx-php-fpm:latest
二、上传镜像到harbor【deploy上操作】
说明：harbor安装方法参考次篇文章https://www.cnblogs.com/douyi/p/12057218.html
1：mysql镜像push到harbor上
1 [root@master ~]# docker tag mysql:5.7 harbor.ghs.com/lnmp/mysql:5.7 2 [root@master ~]# docker push harbor.ghs.com/lnmp/mysql:5.7 3 The push refers to repository [harbor.ghs.com/lnmp/mysql] 4 94a471180601: Pushed 5 934de0c0816e: Pushed 6 fc12e028de3b: Pushed 7 398ef8a407f7: Pushed 8 7055b7f82e4c: Pushed 9 2a9aab74013a: Pushed 10 414373ffccb4: Pushed 11 6599033b2ab2: Pushed 12 51734435c93c: Pushed 13 5a8a245abd1c: Pushed 14 99b5261d397c: Pushed 15 5.7: digest: sha256:9e02c7c9a87d363588e85c87b8c6f637254c5c67b915b1666482f54121bb0926 size: 2622
2：nginx-php-fpm镜像push到harbor上
1 [root@master ~]# docker tag richarvey/nginx-php-fpm harbor.ghs.com/lnmp/nginx-php-fpm:latest 2 [root@master ~]# docker push harbor.ghs.com/lnmp/nginx-php-fpm:latest 3 The push refers to repository [harbor.ghs.com/lnmp/nginx-php-fpm] 4 2624e7202e4b: Pushed 5 eb30cdb90ca1: Pushed 6 ae93c067c849: Pushed 7 9801a5392a22: Pushed 8 7b2d66211161: Pushed 9 9d4627716f06: Pushed 10 d3cf99377a74: Pushed 11 f647f7714c77: Pushed 12 8ed73ab2d11b: Pushed 13 525bffddcce1: Pushed 14 d93022710c58: Pushed 15 28e868b9a611: Pushed 16 ebbffb8bead8: Pushed 17 c4ffba011c55: Pushed 18 9ac09047b17d: Pushed 19 3fefc401e422: Pushed 20 3b73d52fb480: Pushed 21 73f0a2737684: Pushed 22 bb46af78e4a5: Pushed 23 5d02268e182d: Pushed 24 84bd64e5da78: Pushed 25 14d91eeebaab: Pushed 26 67ee9b354071: Pushed 27 d822f3d53ee3: Pushed 28 a0c85631af8f: Pushed 29 7c1ad7fa7002: Pushed 30 03ecc22184eb: Pushed 31 1581c77670a3: Pushed 32 14ee8729351c: Pushed 33 03901b4a2ea8: Pushed 34 latest: digest: sha256:f8109029deb401ee83c4fcd1e737cc496f42aca6a6e8f445f8501ea977be6784 size: 6576
三、安装NFS【master2上操作】
1：创建web和DB目录
##/data/k8s下的两个目录用于集群中运行LNMP环境存放的数据，web目录为nginx网站的根目录，DB为数据库存放的数据目录
[root@master2 ~]# mkdir -p /data/k8s/{web,db}
2：yum安装nfs
[root@master2 ~]# yum -y install nfs-utils
3：设置共享目录
[root@master2 ~]# vim /etc/exportfs
/data/k8s 192.168.1.0/24(sync,rw,no_root_squash)
4：启动服务
[root@master2 ~]# systemctl start nfs
[root@master2 ~]# systemctl enable nfs
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
四、安装Mysql服务【deploy上操作】
1：创建存放mysql服务的yaml文件目录
[root@master2 ~]# mkdir -p /home/k8s/lnmp/mysql
2：创建secret，将mysql数据库的密码存放到secret中
[root@master mysql]# kubectl create secret generic mysql-pass --from-literal=password=abc123
3：创建mysql-pv
1 [root@master mysql]# vim mysql-pv.yaml 2 apiVersion: v1 3 kind: PersistentVolume 4 metadata: 5 name: mysql-pv 6 spec: 7 capacity: 8 storage: 10Gi 9 accessModes: 10 - ReadWriteMany 11 nfs: 12 path: /data/k8s/db ##该参数指定你的NFS端的共享目录 13 server: 192.168.1.201 ##指定nfs端的IP 14 [root@master mysql]# kubectl create -f mysql-pv.yaml 15 persistentvolume/mysql-pv created
4：创建mysql-pvc
1 [root@master mysql]# vim mysql-pvc.yaml 2 apiVersion: v1 3 kind: PersistentVolumeClaim 4 metadata: 5 name: mysql-claim 6 labels: 7 app: discuz 8 spec: 9 accessModes: 10 - ReadWriteMany 11 resources: 12 requests: 13 storage: 10Gi 14 [root@master mysql]# kubectl create -f mysql-pvc.yaml 15 persistentvolumeclaim/mysql-claim created
5：查看pv，pvc信息
##下面可以看出状态Bound（绑定）正常，pv中的CLAIM列绑定的信息为pvc定义的name，pvc中VOLUME列的信息显示绑定了mysql-pv的name
1 [root@master mysql]# kubectl get pv,pvc 2 NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE 3 persistentvolume/mysql-pv 10Gi RWX Retain Bound default/mysql-claim 71s 4 5 NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 6 persistentvolumeclaim/mysql-claim Bound mysql-pv 10Gi RWX 15s
6：创建mysql-deploy
1 [root@master mysql]# vim mysql-dp.yaml 2 apiVersion: apps/v1 3 kind: Deployment 4 metadata: 5 name: dz-mysql 6 labels: 7 app: discuz 8 spec: 9 selector: 10 matchLabels: 11 app: discuz 12 tier: mysql 13 strategy: 14 type: Recreate 15 template: 16 metadata: 17 labels: 18 app: discuz 19 tier: mysql 20 spec: 21 imagePullSecrets: 22 - name: my-secret 23 containers: 24 - image: harbor.ghs.com/lnmp/mysql:5.7 ##此处指定你的harbor需要拉取的镜像 25 name: mysql 26 env: 27 - name: MYSQL_ROOT_PASSWORD 28 valueFrom: 29 secretKeyRef: 30 name: mysql-pass ##此模块是调用secret中存放的mysql密码 31 key: password 32 ports: 33 - containerPort: 3306 34 name: dz-mysql 35 volumeMounts: 36 - name: mysql-persistent-storage 37 mountPath: /var/lib/mysql 38 volumes: 39 - name: mysql-persistent-storage 40 persistentVolumeClaim: ##此模块参数调用定义创建的pvc名称 41 claimName: mysql-claim 42 [root@master mysql]# kubectl create -f mysql-dp.yaml 43 deployment.apps/dz-mysql created
7：创建mysql-service
1 [root@master mysql]# vim mysql-svc.yaml 2 apiVersion: v1 3 kind: Service 4 metadata: 5 name: dz-mysql 6 labels: 7 app: discuz 8 spec: 9 ports: 10 - port: 3306 11 selector: 12 app: discuz 13 tier: mysql 14 [root@master mysql]# kubectl create -f mysql-svc.yaml 15 service/dz-mysql created
8：查看mysql服务的pod和svc信息
##下面信息可以看出，dz-mysql状态Running（运行）正常，分配的cluster-ip=10.68.235.35，该IP在集群中可以访问，外部无法访问该IP
1 [root@master mysql]# kubectl get pod,svc 2 NAME READY STATUS RESTARTS AGE 3 pod/dz-mysql-787586b8d6-9kl5b 1/1 Running 0 5m35s 4 pod/nginx-7c45b84548-4998z 1/1 Running 2 20d 5 6 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 7 service/dz-mysql ClusterIP 10.68.235.35 <none> 3306/TCP 5m13s 8 service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 20d 9 service/nginx ClusterIP 10.68.241.175 <none> 80/TCP 20d
五、安装Nginx服务【deploy上操作】
1：创建nginx-pv
1 [root@master nginx]# vim web-pv.yaml 2 apiVersion: v1 3 kind: PersistentVolume 4 metadata: 5 name: web-pv 6 spec: 7 capacity: 8 storage: 10Gi 9 accessModes: 10 - ReadWriteMany 11 nfs: 12 path: /data/k8s/web 13 server: 192.168.1.201 14 [root@master nginx]# kubectl create -f web-pv.yaml 15 persistentvolume/web-pv created
2：创建nginx-pvc
1 [root@master nginx]# vim web-pvc.yaml 2 apiVersion: v1 3 kind: PersistentVolumeClaim 4 metadata: 5 name: web-claim 6 labels: 7 app: discuz 8 spec: 9 accessModes: 10 - ReadWriteMany 11 resources: 12 requests: 13 storage: 10Gi 14 [root@master nginx]# kubectl create -f web-pvc.yaml 15 persistentvolumeclaim/web-claim created
3：查看nginx的pv，pvc信息
1 [root@master nginx]# kubectl get pv,pvc 2 NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE 3 persistentvolume/mysql-pv 10Gi RWX Retain Bound default/mysql-claim 34m 4 persistentvolume/web-pv 10Gi RWX Retain Bound default/web-claim 114s 5 6 NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 7 persistentvolumeclaim/mysql-claim Bound mysql-pv 10Gi RWX 33m 8 persistentvolumeclaim/web-claim Bound web-pv 10Gi RWX 108s
4：创建nginx-deploy
1 [root@master nginx]# vim web-dp.yaml 2 apiVersion: apps/v1 3 kind: Deployment 4 metadata: 5 name: dz-web 6 labels: 7 app: discuz 8 spec: 9 replicas: 1 10 selector: 11 matchLabels: 12 app: discuz 13 tier: nginx-php 14 template: 15 metadata: 16 labels: 17 app: discuz 18 tier: nginx-php 19 spec: 20 imagePullSecrets: 21 - name: my-secret 22 containers: 23 - image: harbor.ghs.com/lnmp/nginx-php-fpm:latest 24 name: dz-web 25 ports: 26 - containerPort: 9000 27 - containerPort: 80 28 name: dz-web 29 volumeMounts: 30 - name: mysql-persistent-storage 31 mountPath: /var/www/html/ 32 volumes: 33 - name: mysql-persistent-storage 34 persistentVolumeClaim: 35 claimName: web-claim 36 [root@master nginx]# kubectl create -f web-dp.yaml 37 deployment.apps/dz-web created
5：创建nginx-service
##映射端口30001提供集群外部的机器访问web网站
1 [root@master nginx]# vim web-svc.yaml 2 apiVersion: v1 3 kind: Service 4 metadata: 5 name: dz-web 6 labels: 7 app: discuz 8 spec: 9 type: NodePort 10 ports: 11 - port: 80 12 nodePort: 30001 13 selector: 14 app: discuz 15 tier: nginx-php 16 [root@master nginx]# kubectl create -f web-svc.yaml 17 service/dz-web created
6：查看nginx的pod,svc信息
1 [root@master nginx]# kubectl get pod,svc 2 NAME READY STATUS RESTARTS AGE 3 pod/dz-mysql-787586b8d6-9kl5b 1/1 Running 1 38m 4 pod/dz-web-d4968cf8d-clppp 1/1 Running 0 3m37s 5 pod/nginx-7c45b84548-4998z 1/1 Running 2 20d 6 7 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 8 service/dz-mysql ClusterIP 10.68.235.35 <none> 3306/TCP 38m 9 service/dz-web NodePort 10.68.53.228 <none> 80:30001/TCP 11m 10 service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 20d 11 service/nginx ClusterIP 10.68.241.175 <none> 80/TCP 20d
六、安装Discuz【NFS上操作】
1：下载Discuz
1 [root@master2 ~]# cd /data/ 2 [root@master2 data]# git clone https://gitee.com/ComsenzDiscuz/DiscuzX.git 3 正克隆到 'DiscuzX'... 4 remote: Counting objects: 9085, done. 5 remote: Compressing objects: 100% (5400/5400), done. 6 remote: Total 9085 (delta 3895), reused 8535 (delta 3632) 7 接收对象中: 100% (9085/9085), 12.96 MiB | 166.00 KiB/s, done. 8 处理 delta 中: 100% (3895/3895), done.
2：将Discuz项目中upload目录下所有文件，放到web中
1 [root@master2 data]# mv DiscuzX/upload/* k8s/web/ 2 [root@master2 data]# cd k8s/web/ 3 [root@master2 web]# chmod 100 data uc_server/data uc_client/data config
3：进入mysql，创建dz库和授权用户
1 [root@master ~]# kubectl exec -it dz-mysql-787586b8d6-9kl5b -- mysql -u root -p 2 Welcome to the MySQL monitor. Commands end with ; or \g. 3 Your MySQL connection id is 4 4 Server version: 5.7.28 MySQL Community Server (GPL) 5 6 Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. 7 8 Oracle is a registered trademark of Oracle Corporation and/or its 9 affiliates. Other names may be trademarks of their respective 10 owners. 11 12 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. 13 14 mysql> create database dz; 15 Query OK, 1 row affected (0.19 sec) 16 17 mysql> grant all on dz.* to 'dz'@'%' identified by 'abc123'; 18 Query OK, 0 rows affected, 1 warning (0.11 sec
4：浏览器访问web安装页面
实验成功
5：通过设置Nginx代理供外部访问
##目前nginx服务是运行在kubernetes集群里，node节点以及master节点上是可以通过cluster-ip访问到，但是外部的客户端就不能访问了。所以，可以在任意一台node或者master上建一个nginx反向代理即可访问到集群内的nginx。 kubectl get svc dz-web //查看cluster-ip，我的ip是10.68.235.35
nginx代理配置文件内容如下：
1 server { 2 listen 80; 3 server_name dz.yuankeedu.com; 4 5 location / { 6 proxy_pass http://10.68.235.35; 7 proxy_set_header Host $host; 8 proxy_set_header X-Real-IP $remote_addr; 9 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 10 } 11 }