关于我们
技术分享
技术分享
kubernetes之Ingress部署
kubernetes之Ingress部署
2020-12-06
1，如何访问K8S中的服务：
1，Ingress介绍
Kubernetes 暴露服务的方式目前只有三种：LoadBlancer Service、NodePort Service、Ingress；前两种估计都应该很熟悉，下面详细的了解下这个 Ingress
Ingress由两部分组成：Ingress Controller 和 Ingress 服务。
Ingress Contronler 通过与 Kubernetes API 交互，动态的去感知集群中 Ingress 规则变化，然后读取它，按照自定义的规则，规则就是写明了哪个域名对应哪个service，生成一段 Nginx 配置，再写到 Nginx-ingress-control的 Pod 里，这个 Ingress Contronler 的pod里面运行着一个nginx服务，控制器会把生成的nginx配置写入/etc/nginx.conf文件中，然后 reload 一下 使用配置生效。以此来达到域名分配置及动态更新的问题。
看个简单的图方便理解：
ingress控制器有两种：nginx和haproxy 这里是以nginx为讲解。
2，部署一个Nginx Ingress
ingress的部署文件在github Ingress 仓库找到. 针对官方配置我们单独添加了 nodeselector 指定，绑定LB地址 以方便DNS 做解析。
主要用到的文件：
$ ls default-backend.yaml jenkins-ingress.yml nginx-ingress-controller-rbac.yml nginx-ingress-controller.yaml - - - default-backend.yaml：这是官方要求必须要给的默认后端，提供404页面的。它还提供了一个http检测功能，检测nginx-ingress-controll健康状态的，通过每隔一定时间访问nginx-ingress-controll的/healthz页面，如是没有响应就 返回404之类的错误码。 nginx-ingress-controller-rbac.yml:这ingress的RBAC授权文件 nginx-ingress-controller.yaml：这是控制器的部署文件。 jenkins-ingress.yml：这是Ingress服务文件，这个可以是任意web程序，里面配置域名与service的对应关系，Ingress称之为规则。
第一个是要部署RBAC文件：
cat nginx-ingress-controller-rbac.yml #apiVersion: v1 #kind: Namespace #metadata: #这里是创建一个namespace，因为此namespace早有了就不用再创建了 # name: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount #创建一个serveerAcount namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole #这个ServiceAcount所绑定的集群角色 rules: - apiGroups: - "" resources: #此集群角色的权限，它能操作的API资源 - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "extensions" resources: - ingresses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role #这是一个角色，而非集群角色 namespace: kube-system rules: #角色的权限 - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get - create - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding #角色绑定 metadata: name: nginx-ingress-role-nisa-binding namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount #绑定在这个用户 namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding #集群绑定 metadata: name: nginx-ingress-clusterrole-nisa-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount #集群绑定到这个serviceacount namespace: kube-system #集群角色是可以跨namespace，但是这里只指明给这个namespce来使用
创建：
$ kubectl create -f nginx-ingress-controller-rbac.yml serviceaccount "nginx-ingress-serviceaccount" created clusterrole "nginx-ingress-clusterrole" created role "nginx-ingress-role" created rolebinding "nginx-ingress-role-nisa-binding" created clusterrolebinding "nginx-ingress-clusterrole-nisa-binding" created
RBAC创建完后，就创建default backend服务：
$ cat default-backend.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: default-http-backend labels: k8s-app: default-http-backend namespace: kube-system spec: replicas: 1 template: metadata: labels: k8s-app: default-http-backend spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissable as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: gcr.io/google_containers/defaultbackend:1.0 livenessProbe: httpGet: path: /healthz #这个URI是 nginx-ingress-controller中nginx里配置好的localtion port: 8080 scheme: HTTP initialDelaySeconds: 30 #30s检测一次/healthz timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi nodeSelector: #指定调度到些Node, 以便后面DNS解析 kubernetes.io/hostname: 10.3.1.17 --- apiVersion: v1 kind: Service #为default backend 创建一个service metadata: name: default-http-backend namespace: kube-system labels: k8s-app: default-http-backend spec: ports: - port: 80 targetPort: 8080 selector: k8s-app: default-http-backend
创建：
$ kubectl create -f default-backend.yaml deployment "default-http-backend" created service "default-http-backend" created
创建之后查看：
root@ubuntu15:/data/ingress# kubectl get rs,pod,svc -n kube-system NAME DESIRED CURRENT READY AGE rs/default-http-backend-857b544d94 1 1 1 1m NAME READY STATUS RESTARTS AGE po/default-http-backend-857b544d94-bwgjd 1/1 Running 0 1m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/default-http-backend ClusterIP 10.254.208.144 <none> 80/TCP 1m
创建好default backend后就要创建nginx-ingress-controller了：
$ cat nginx-ingress-controller.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-ingress-controller labels: k8s-app: nginx-ingress-controller namespace: kube-system spec: replicas: 1 template: metadata: labels: k8s-app: nginx-ingress-controller spec: # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used # like with kubeadm # hostNetwork: true #注释表示不使用宿主机的80口， terminationGracePeriodSeconds: 60 hostNetwork: true #表示容器使用和宿主机一样的网络 serviceAccountName: nginx-ingress-serviceaccount #引用前面创建的serviceacount containers: - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.1 #容器使用的镜像 name: nginx-ingress-controller #容器名 readinessProbe: #启动这个服务时要验证/healthz 端口10254会在运行的node上监听。 httpGet: path: /healthz port: 10254 scheme: HTTP livenessProbe: httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 #每隔10做健康检查 timeoutSeconds: 1 ports: - containerPort: 80 hostPort: 80 #80映射到80 - containerPort: 443 hostPort: 443 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend # - --default-ssl-certificate=$(POD_NAMESPACE)/ingress-secret #这是启用Https时用的 nodeSelector: #指明运行在哪，此IP要和default backend是同一个IP kubernetes.io/hostname: 10.3.1.17 #上面映射到了hostport80，确保此IP80，443没有占用.
这个控制器就是一个deployment ，里面运行一个容器gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.1 ，有点像nginx容器，现在创建：
$ kubectl create -f nginx-ingress-controller.yaml deployment "nginx-ingress-controller" created
root@ubuntu15:/data/ingress# kubectl get rs,pod,svc -n kube-system NAME DESIRED CURRENT READY AGE rs/default-http-backend-857b544d94 1 1 1 12m rs/nginx-ingress-controller-8576d4545d 1 1 0 27s NAME READY STATUS RESTARTS AGE po/default-http-backend-857b544d94-bwgjd 1/1 Running 0 12m po/nginx-ingress-controller-8576d4545d-9tjnv 0/1 ContainerCreating 0 27s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/default-http-backend ClusterIP 10.254.208.144 <none> 80/TCP 12m
现在ingress controller 控制器已部署好了，那么如何使用了，那就要写一个ingress规则了，此处就以已存在的jenkins服务为例，配置如何使用域名访问这个service:
$ kubectl get svc,ep NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/jenkinsservice NodePort 10.254.70.47 <none> 8080:30002/TCP 3h NAME ENDPOINTS AGE ep/jenkinsservice 172.30.10.15:8080,172.30.11.7:8080 3h
现在写个jenkins service的Ingress 规则：
$ cat jenkins-ingress.yml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: jenkins-ingress namespace: default #服务在哪个空间内就写哪个空间 annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: ingress.jenkins.com #此service的访问域名 http: paths: - backend: serviceName: jenkinsservice servicePort: 8080
创建它：
$ kubectl create -f jenkins-ingress.yml ingress "jenkins-ingress" created $ kubectl get ingress NAME HOSTS ADDRESS PORTS AGE jenkins-ingress ingress.jenkins.com 80 10s
到这里就已经部署完成了，配置好域名后，就可以用此域名来访问了：
部署完成了，现在看下nginx-ingress-controller 里nginx配置文件发生了哪些变化：
upstream default-jenkinsservice-8080 {
least_conn;
server 172.30.10.15:8080 max_fails=0 fail_timeout=0;
server 172.30.11.7:8080 max_fails=0 fail_timeout=0;
}
upstream upstream-default-backend {
least_conn;
server 172.30.11.6:8080 max_fails=0 fail_timeout=0;
}
server {
server_name ingress.jenkins.com;
listen [::]:80;
location / {
...
proxy_pass http://default-jenkinsservice-8080;
...
}
}这些配置都是ingress-controller 自已写入的，动态更新就是它能通过K8S API感知到service的endpoint 发生了变化，然后修改nginx配置并执行reload.
至此，部署完成。
Ingress还有很多部署方式，比如配置https访问的, 以后再写。
- 标签：
-
技术分享
您可能感兴趣的新闻 换一批
热门文章
现在下载，可享30天免费试用
