目录
Service 类型
- ClusterIP 只在集群内部访问，无法跨越集群边界
- NodePort 提供集群外部流量进入到集群内部，是在clusterIP前提之上增强的
- LoadBalancer 集群部署在公有云或私有云之上，提供给公有云或私有云的负载均衡中间使用
- ExternelName 把集群外部的服务映射给集群内部使用， 是一个FQDN的名称，一个CNAME名称，这个CNAM指向公网的FQDN
No ClusterIP 成为 Headless Sercie
ServiceName -> PodIP
Service 是一个四层调度器
也就是说，当请求为HTTPS请求的时候，则无法卸载证书。
namespace 名称空间
；
创建 namespace 如下：
[root@master manifests]# kubectl create namespace dev
namespace/dev created
[root@master manifests]# kubectl get ns
NAME STATUS AGE
default Active 20d
dev Active 4s # 新创建的namespace
kube-node-lease Active 20d
kube-public Active 20d
kube-system Active 20d
删除 namespace 如下：
[root@master manifests]# kubectl delete ns/dev
namespace "dev" deleted
[root@master manifests]# kubectl get ns
NAME STATUS AGE
default Active 20d
kube-node-lease Active 20d
kube-public Active 20d
kube-system Active 20d
Ingress Controller
举例
当kubernetes集群有上千甚至跟多个节点的时候，此时需要特有的web七层代理
如在集群其中的四个节点上打上污点，这四个节点上只运行web七层代理所对应的Pod
由此Pod来代理集群内部的Service，Service再把流量转发给集群内部对应的Pod。
这就叫做 Ingress Controller
Ingress Controller 是基于 DaemonSet 控制器来实现。
DaemonSet 是实现保证集群内部每个节点上都运行一个指定的Pod。
Kubernetes 中三种常用七层代理:
- Nginx # 常规代理
- Traefik # 微服务前段负载均衡
- Envoy # 微服务代理负载均衡，使用的多
Ingress
Ingress 和 Ingress Controller 是两回事。
Ingress 原理
Ingress-nginx 进行测试
本次使用的是相对简单部署 ingress-nginx 的配置清单。
本文创建的基本流程：创建前端service 创建后端service和pod，创建ingress。
ingress-nginx 单独的详情页面
因为是直接在物理机上直接安装的kubernetes集群，所以需要两个配置清单。
[root@master ingress]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
还需要一个Ingress-name名称空间中创建的NodePort Service，以便引入集群外流量
需要根据实际情况修改一下：
[root@master ingress]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
[root@master ingress]# cat service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080 # 手动指定node对外的http端口
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443 # 手动指定node对外的https端口
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
[root@master ingress]#kubectl apply -f service-nodeport.yaml
service/ingress-nginx created
查看创建的ingress-nginx Pod
[root@master ingress]# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-7995bd9c47-b4fzh 1/1 Running 0 34s 10.244.1.27 node03.kubernetes <none> <none>
[root@master ingress]# kubectl get svc -n ingress-nginx # 这里是第二次执行在Ingress-name名称空间中创建的NodePort Service，以便引入集群外流量
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.111.201.108 <none> 80:30080/TCP,443:30443/TCP 44s
创建对应的后端Pod和Service
查看创建的ingress-myapp
[root@master ingress]# cat myapp-deploy.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp # Service 名称
namespace: default
spec:
selector:
app: myapp
release: canary
ports:
- name: http
targetPort: 80 # 指定容器端口
port: 80 # Service 自己开房的端口
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: myapp
release: canary
template:
metadata:
labels:
app: myapp
release: canary
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v2
ports:
- name: http
containerPort: 80
[root@master ingress]# kubectl apply -f myapp-deplay.yaml
service/myapp created
deployment.apps/myapp-deploy unchanged
查看
[root@master ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-55b78d8548-b4c2c 1/1 Running 0 42h
myapp-deploy-55b78d8548-gpdw8 1/1 Running 0 42h
myapp-deploy-55b78d8548-hr6vx 1/1 Running 0 42h
[root@master ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 22d
myapp ClusterIP 10.111.44.7 <none> 80/TCP 6s
创建 Ingress
[root@master ingress]# cat ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx" # 自动生成 nginx 相关的匹配和修改规则
spec:
rules:
- host: myapp.beijingfc.com # 使用 host 主机模式来访问
http:
paths:
- path:
backend:
serviceName: myapp # 这里指定刚刚创建的后端Pod对应的Service名称
servicePort: 80
[root@master ingress]# kubectl apply -f ingress-myapp.yaml
ingress.extensions/ingress-myapp created
[root@master ingress]# kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.beijingfc.com 80 6s
查看 Ingress-controller 对应的Pod配置信息
上面一系列都配置完成之后，此时在看查看 Ingress-controller 的Pod中nginx的配置信息包含如下:
[root@master ingress]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-7995bd9c47-b4fzh -- /bin/sh
$ cat nginx.conf
server {
server_name myapp.beijingfc.com ;
listen 80;
set $proxy_upstream_name "-";
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;
location / {
set $namespace "default";
set $ingress_name "ingress-myapp";
set $service_name "myapp";
set $service_port "80";
set $location_path "/";
... ...
... ...
访问测试
此时在集群外部访问测试：
注意，临时测试，上面使用的host模式来访问，所以在测试的集群外部机器上手动添加hosts记录
[root@xinguan-test-node1 ~]# grep myapp /etc/hosts
10.0.20.20 myapp.beijingfc.com
[root@xinguan-test-node1 ~]# curl myapp.beijingfc.com:30080
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@xinguan-test-node1 ~]# curl myapp.beijingfc.com:30080/hostname.html
myapp-deploy-55b78d8548-hr6vx
模拟测试 Ingress 后端 Tomcat 访问
还是基于上面的配置，一切都不动，增加tomcat后端Pod和Service 以及 Ingress
创建 tomcat 的 Pod 和Service
[root@master ingress]# cat tomcat-deploy.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat
namespace: default
spec:
selector:
app: tomcat
release: canary
ports:
- name: http
targetPort: 8080
port: 8080
- name: ajp
targetPort: 8009
port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-depoly
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
release: canary
template:
metadata:
labels:
app: tomcat
release: canary
spec:
containers:
- name: tomcat
image: tomcat:8.5.32-jre8-alpine
ports:
- name: http
containerPort: 8080
- name: ajp
containerPort: 8009
创建和查看
[root@master ingress]# kubectl apply -f tomcat-deploy.yaml
service/tomcat created
deployment.apps/tomcat-depoly created
[root@master ingress]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myapp-deploy-55b78d8548-b4c2c 1/1 Running 0 43h 10.244.3.29 node01.kubernetes <none> <none>
myapp-deploy-55b78d8548-gpdw8 1/1 Running 0 43h 10.244.1.26 node03.kubernetes <none> <none>
myapp-deploy-55b78d8548-hr6vx 1/1 Running 0 43h 10.244.2.22 node02.kubernetes <none> <none>
tomcat-depoly-579d97b849-66wrh 1/1 Running 0 7s 10.244.3.30 node01.kubernetes <none> <none>
tomcat-depoly-579d97b849-rh5r6 1/1 Running 0 7s 10.244.1.28 node03.kubernetes <none> <none>
tomcat-depoly-579d97b849-sd49w 1/1 Running 0 7s 10.244.2.23 node02.kubernetes <none> <none>
[root@master ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 22d
myapp ClusterIP 10.111.44.7 <none> 80/TCP 76m
tomcat ClusterIP 10.106.39.78 <none> 8080/TCP,8009/TCP 3m18s
创建 tomcat-ingress
[root@master ingress]# cat ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: tomcat.beijingfc.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
创建和查看
[root@master ingress]# kubectl apply -f ingress-tomcat.yaml
ingress.extensions/ingress-tomcat created
[root@master ingress]# kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.beijingfc.com 80 53m
ingress-tomcat tomcat.beijingfc.com 80 4s
测试访问 tomcat
注意，同样写Hosts 之后进行测试
[root@xinguan-test-node1 ~]# grep tomcat /etc/hosts
10.0.20.20 myapp.beijingfc.com tomcat.beijingfc.com
[root@xinguan-test-node1 ~]# curl tomcat.beijingfc.com:30080
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Apache Tomcat/8.5.32</title>
<link href="favicon.ico" rel="icon" type="image/x-icon" />
<link href="favicon.ico" rel="shortcut icon" type="image/x-icon" />
<link href="tomcat.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="wrapper">
<div id="navigation" class="curved container">
<span id="nav-home"><a href="http://tomcat.apache.org/">Home</a></span>
<span id="nav-hosts"><a href="/docs/">Documentation</a></span>
<span id="nav-config"><a href="/docs/config/">Configuration</a></span>
<span id="nav-examples"><a href="/examples/">Examples</a></span>
<span id="nav-wiki"><a href="http://wiki.apache.org/tomcat/FrontPage">Wiki</a></span>
<span id="nav-lists"><a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></span>
<span id="nav-help"><a href="http://tomcat.apache.org/findhelp.html">Find Help</a></span>
<br class="separator" />
</div>
<div id="asf-box">
<h1>Apache Tomcat/8.5.32</h1>
</div>
... ...
... ...
模拟测试 Https
模拟测试 通过Ingress 的七层代理，访问Https，通过Ingress卸载证书后访问后端
所有先需要手动自签SSL证书
自签SSL证书
[root@master https]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
...............................+++
.....................................................................................................................................+++
e is 65537 (0x10001)
[root@master https]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=BeiJing/L=BeiJing/O=DefOps/CN=tomcat.beijingfc.com
[root@master https]# ll
total 8
-rw-r--r-- 1 root root 1302 Aug 1 11:33 tls.crt
-rw-r--r-- 1 root root 1675 Aug 1 11:31 tls.key
自签证书完成
创建secret
注意，此证书不能直接帖进去，需要先创建成secret，才能应用到Ingress中
[root@master https]# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
secret/tomcat-ingress-secret created
[root@master https]# kubectl get secret
NAME TYPE DATA AGE
default-token-bc86p kubernetes.io/service-account-token 3 22d
tomcat-ingress-secret kubernetes.io/tls 2 10s
[root@master https]# kubectl describe secret tomcat-ingress-secret
Name: tomcat-ingress-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.key: 1675 bytes
tls.crt: 1302 bytes
创建tls的https清单文件
[root@master https]# cat ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat-tls
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- tomcat.beijingfc.com
secretName: tomcat-ingress-secret # 这里指定刚刚创建的secret名称
rules:
- host: tomcat.beijingfc.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
创建和查看
[root@master https]# kubectl apply -f ingress-tomcat-tls.yaml
ingress.extensions/ingress-tomcat-tls created
[root@master https]# kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.beijingfc.com 80 174m
ingress-tomcat tomcat.beijingfc.com 80 121m
ingress-tomcat-tls tomcat.beijingfc.com 80, 443 4s # 这里看到被创建成功
[root@master https]# kubectl describe ingress ingress-tomcat-tls
Name: ingress-tomcat-tls
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
tomcat-ingress-secret terminates tomcat.beijingfc.com
Rules:
Host Path Backends
---- ---- --------
tomcat.beijingfc.com
tomcat:8080 (10.244.1.28:8080,10.244.2.23:8080,10.244.3.30:8080)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-tomcat-tls","namespace":"default"},"spec":{"rules":[{"host":"tomcat.beijingfc.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}],"tls":[{"hosts":["tomcat.beijingfc.com"],"secretName":"tomcat-ingress-secret"}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 26s nginx-ingress-controller Ingress default/ingress-tomcat-tls
测试访问
此时就已经完成，可以测试访问
因前面测试hosts已经写好