Hands-on Lab – 使用Helm部署OpenShift应用

前言

Helm 是 Kubernetes 的包管理器。包管理器类似于我们在 Ubuntu 中使用的apt、Centos中使用的yum 或者Python中的 pip 一样，能快速查找、下载和安装软件包。Helm 由客户端组件 helm 和服务端组件 Tiller 组成, 能够将一组K8S资源打包统一管理, 是查找、共享和使用为Kubernetes构建的软件的最佳方式。

解决问题

能对多个模板文件（yaml文件）进行统一管理，设想一下，如果没有helm，如何执行多个模板文件呢？那就得一个个的执行kubectl apply -f xxx.yaml。而如今有了helm，只需要将文件放入chart包中，然后一句helm install xxx path便可以执行所有的模板文件。当然helm的好处不仅于此，还有其他便利之处，有待大家挖掘。

   //创建一个chart包模板
   helm create xxx(chart包名)

   //安装
   helm install xxx -n namespace

   //卸载
   helm uninstall xxx -n namespace

   //预填充变量/参数
   helm install xxx -n namespace --dry-run

chart包结构

架构介绍

组件

Helm 包含两个组件，分别是 helm 客户端和 Tiller 服务器：

helm 是一个命令行工具，用于本地开发及管理chart，chart仓库管理等
Tiller 是 Helm 的服务端。Tiller 负责接收 Helm 的请求，与 k8s 的 apiserver 交互，根据chart 来生成一个 release 并管理 release，Tiller 是和k8s沟通的组件，helm是客户端命令，Tiller是服务器端程序，helm发送命令到Tiller，然后Tiller操作k8s完成任务。
chart Helm的打包格式叫做chart，所谓chart就是一系列文件, 它描述了一组相关的 k8s 集群资源
release 使用 helm install 命令在 Kubernetes 集群中部署的 Chart 称为 Release
Repoistory Helm chart 的仓库，Helm 客户端通过 HTTP 协议来访问存储库中 chart 的索引文件和压缩包

原理

下面图描述了 Helm 的几个关键组件 Helm（客户端）、Tiller（服务器）、Repository（Chart 软件仓库）、Chart（软件包）之间的关系以及它们之间如何通信：

创建release

helm 客户端从指定的目录或本地tar文件或远程repo仓库解析出chart的结构信息
helm 客户端指定的 chart 结构和 values 信息通过 gRPC 传递给 Tiller
Tiller 服务端根据 chart 和 values 生成一个 release
Tiller 将install release请求直接传递给 kube-apiserver

删除release

helm 客户端从指定的目录或本地tar文件或远程repo仓库解析出chart的结构信息
helm 客户端指定的 chart 结构和 values 信息通过 gRPC 传递给 Tiller
Tiller 服务端根据 chart 和 values 生成一个 release
Tiller 将delete release请求直接传递给 kube-apiserver

更新release

helm 客户端将需要更新的 chart 的 release 名称 chart 结构和 value 信息传给 Tiller
Tiller 将收到的信息生成新的 release，并同时更新这个 release 的 history
Tiller 将新的 release 传递给 kube-apiserver 进行更新

Helm架构及其组件

在OpenShift 4中内置了Helm 3，我们可以使用Helm Chart部署OpenShift应用。Helm使用Chart记录要部署资源的配置，这非常类似OpenShift自带的Template：

下载helm客户端

下载helm客户端，并查看版本。

$ sudo curl -L https://mirror.openshift.com/pub/openshift-v4/clients/helm/latest/helm-linux-amd64 -o /usr/local/bin/helm
$ sudo chmod +x /usr/local/bin/helm
$ helm version
version.BuildInfo{Version:"v3.6.2+5.el8", GitCommit:"eb607dd4f123eaedab662cef21008d177f2c3426", GitTreeState:"clean", GoVersion:"go1.15.13"}

在控制台用Helm Chart安装部署应用

1.创建项目

$ oc new-project helm-nodejs

2.进入OpenShift控制台的“开发者”视图，点击“+添加”菜单，然后找到“Helm Chart”进入。

3.通过在“Languages”中查找“nodejs”，找到下面的Helm Chart，然后点击进入。

4.在右滑的安装界面中点击“安装Helm Chart”按钮，然后在“安装Helm Chart”页面中接受缺省配置，最后点击“安装”按钮，此时页面将跳转到“拓扑”。

5.此时OpenShift会根据这个Helm Chart拉取Nodejs应用代码，然后通过Build构建应用镜像，最后再部署运行。

6.部署完成后如下图，此时可通过Route访问部署好的应用了。

7.进入“Helm”菜单，在列表中进入“nodejs”，可在“资源”中查看所有通过Helm Chart创建的OpenShift资源。

通过外部Helm Repo安装部署应用

1.创建项目。

$ oc new-project helm-nginx

2.执行命令，查看包含nginx的helm hub的Repo源。

$ helm search hub nginx
URL CHART VERSION APP VERSION DESCRIPTION
https://artifacthub.io/packages/helm/test-nginx... 0.1.0 1.16.0 A Helm chart for Kubernetes
https://artifacthub.io/packages/helm/shubhamtat... 0.1.12 1.19.6 Nginx Helm chart for Kubernetes
https://artifacthub.io/packages/helm/dysnix/nginx 7.1.8 1.19.4 Chart for the nginx server
https://artifacthub.io/packages/helm/mirantis/n... 0.1.0 1.16.0 A NGINX Docker Community based Helm chart for K...
https://artifacthub.io/packages/helm/bitnami/nginx 9.4.2 1.21.1 Chart for the nginx server
https://artifacthub.io/packages/helm/bitnami-ak... 9.4.2 1.21.1 Chart for the nginx server
https://artifacthub.io/packages/helm/krakazyabr... 1.0.0 1.19.0 Nginx Helm chart for Kubernetes
https://artifacthub.io/packages/helm/wiremind/n... 2.1.1 An NGINX HTTP server
https://artifacthub.io/packages/helm/t3n/nginx 0.1.1 Simple nginx deployment usable for deploying pr...
https://artifacthub.io/packages/helm/slamdev/nginx 0.0.17 1.19.10 Helm chart to deploy [nginx](https://www.nginx....
https://artifacthub.io/packages/helm/cocainefar... 1.0.1 latest A chart to do nginx things
https://artifacthub.io/packages/helm/douban/nginx 0.1.2 0.16.0 A Helm chart for Kubernetes
https://artifacthub.io/packages/helm/ingress-ng... 4.0.0 1.0.0-beta.1 Ingress controller for Kubernetes using NGINX a...
https://artifacthub.io/packages/helm/nginx-edge... 0.0.0-edge 1.12.0 NGINX Ingress Controller
https://artifacthub.io/packages/helm/nginx/ngin... 0.10.0 1.12.0 NGINX Ingress Controller
https://artifacthub.io/packages/helm/okteto/ngi... 1.41.2 v0.34.1 An nginx Ingress controller that uses ConfigMap...
https://artifacthub.io/packages/helm/hkube/ngin... 1.31.1002 0.29.0 An nginx Ingress controller that uses ConfigMap...
https://artifacthub.io/packages/helm/wenerme/in... 4.0.0 1.0.0-beta.1 Ingress controller for Kubernetes using NGINX a...
https://artifacthub.io/packages/helm/api/ingres... 3.29.1 0.45.0 Ingress controller for Kubernetes using NGINX a...
https://artifacthub.io/packages/helm/ngrok-ingr... 0.3.0 1.16.0 Nginx example demonstrates usage of a helm char...
https://artifacthub.io/packages/helm/nexclipper... 0.1.3 0.1.3 A Helm chart for the Prometheus NGINX Exporter
https://artifacthub.io/packages/helm/drpsychick... 0.0.2 8-fpm-alpine A chart for an nginx pod with multiple phpfpm pods
https://artifacthub.io/packages/helm/ygqygq2/ng... 1.1.0 1.14.2 Chart for the nginx server
https://artifacthub.io/packages/helm/bitnami-ak... 5.2.4 1.17.10 Chart for the nginx server
https://artifacthub.io/packages/helm/ygqygq2/fa... 1.0.2 V5.11 FastDFS-Nginx chart for Kubernetes
https://artifacthub.io/packages/helm/cloudposse... 0.1.8 A Helm chart for Nginx Ingress
https://artifacthub.io/packages/helm/psu-swe/ng... 0.2.0 1.16.1 An nginx chart built to simply provide redirect...
https://artifacthub.io/packages/helm/cocainefar... 0.7.0 latest A Dockerfile for nginx-rtmp-module + FFmpeg fro...
https://artifacthub.io/packages/helm/newrelic/s... 1.1.1 1.1 A Helm chart for installing a simple nginx
https://artifacthub.io/packages/helm/cloudposse... 0.5.0 A Helm chart for nginx-default-backend to be us...
https://artifacthub.io/packages/helm/microfunct... 7.5.0 0.44.0 Chart for the nginx Ingress controller
https://artifacthub.io/packages/helm/bitnami/ng... 7.6.18 0.48.1 Chart for the nginx Ingress controller
。。。

3.用浏览器查看查询结果中的https://artifacthub.io/packages/helm/bitnami/nginx

4.执行命令，将上图的https://charts.bitnami.com/bitnami加入到当前helm的repo中。

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

5.查看本地Helm Repo列表，确认有名为bitnami的Repo，并且其中包含nginx的Chart。

$ helm repo list
NAME    URL
bitnami https://charts.bitnami.com/bitnami
$ helm search repo bitnami/nginx
NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
bitnami/nginx                           9.4.2           1.21.1          Chart for the nginx server
bitnami/nginx-ingress-controller        7.6.18          0.48.1          Chart for the nginx Ingress controller

6.根据bitnami/nginx的Chart创建应用资源。

$ helm install my-nginx bitnami/nginx --set service.type=ClusterIP
NAME: my-nginx
LAST DEPLOYED: Thu Dec 24 13:03:47 2020
NAMESPACE: helm-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **
 
NGINX can be accessed through the following DNS name from within your cluster:
 
    my-nginx.helm-nginx.svc.cluster.local (port 80)
 
To access NGINX from outside the cluster, follow the steps below:
 
1. Get the NGINX URL by running these commands:
 
    export SERVICE_PORT=$(kubectl get --namespace helm-nginx -o jsonpath="{.spec.ports[0].port}" services my-nginx)
    kubectl port-forward --namespace helm svc/my-nginx ${SERVICE_PORT}:${SERVICE_PORT} &
    echo "http://127.0.0.1:${SERVICE_PORT}"

7.执行命令，查看当前project中通过helm部署的Chart，确认有名为my-ngin的Chart资源。

$ helm ls
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
my-nginx        helm-nginx      1               2020-12-24 13:03:47.342110531 +0000 UTC deployed        nginx-9.4.2     1.21.1

8.确认此时Deployment、Pod、Service等资源已经部署好。

$ oc get deployment -n helm-nginx
NAME       READY   UP-TO-DATE   AVAILABLE   AGE
my-nginx   1/1     1            1           33m
$ oc get pods -n helm-nginx
NAME                       READY   STATUS      RESTARTS   AGE
my-nginx-f5c84bc49-dnp7b   1/1     Running     0          21m$ 
$ oc get services -n helm-nginx
NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
my-nginx       ClusterIP   172.30.134.59    <none>        80/TCP,443/TCP   22m

9.根据Service生成Route，然后用浏览器访问Route地址。

$ oc expose svc/my-nginx -n helm-nginx
route.route.openshift.io/my-nginx exposed
$ oc get routes -n helm-nginx
NAME       HOST/PORT                                                                   PATH   SERVICES   PORT   TERMINATION   WILDCARD
my-nginx   my-nginx-helm-nginx.apps.cluster-pek-99bc.pek-99bc.sandbox309.opentlc.com          my-nginx   http                 None

10.删除my-nginx的Chart包括的相关资源以及Route资源。

$ helm uninstall my-nginx
release "my-nginx" uninstalled
$ oc delete route my-nginx -n helm-nginx
$ oc get all -n helm-nginx
No resources found in helm-nginx namespace.

使用本地Helm Repo安装部署应用

1.创建新项目

$ oc new-project helm-mysql

2.下载项目Repo到本地

$ git clone https://github.com/liuxiaoyu-git/helm-repo
Cloning into 'helm-repo'...
remote: Enumerating objects: 39, done.
remote: Total 39 (delta 0), reused 0 (delta 0), pack-reused 39
Unpacking objects: 100% (39/39), done.

3.根据本地Repo安装mysql应用

$ helm install mysql ./helm-repo/incubator/mysql
NAME: mysql                                                                                                                                      
LAST DEPLOYED: Mon Mar  2 07:02:53 2020                                                                                                          
NAMESPACE: helm-mysql                                                                                                                           
STATUS: deployed                                                                                                                                 
REVISION: 1                                                                                                                                      
TEST SUITE: None                                                                                                                                 
NOTES:                                                                                                                                           
MySQL can be accessed via port 3306 on the following DNS name from within your cluster:                                                          
mysql-mysql.helm-mysql.svc.cluster.local                                                                                                        
                                                                                                                                                 
To get your root password run:                                                                                                                   
                                                                                                                                                 
    MYSQL_ROOT_PASSWORD=$(oc get secret --namespace helm-mysql mysql-mysql -o jsonpath="{.data.database-root-password}" | base64 --decode; echo)
                                                                                                                                                 
To know your database:                                                                                                                           
                                                                                                                                                 
    MYSQL_DATABASE=$(oc get secret --namespace helm-mysql mysql-mysql -o jsonpath="{.data.database-name}" | base64 --decode; echo)              
                                                                                                                                                 
To connect to your database:                                                                                                                     
                                                                                                                                                 
(1). Run an Centos pod that you can use as a client:                                                                                               
                                                                                                                                                 
    oc run -i --rm --tty centos --image=centos/mysql-57-centos7 --restart=Never -- bash -il                                                      
                                                                                                                                                 
(2). Connect using the mysql cli, then provide your password:                                                                                      
    $ mysql -h mysql-mysql.helm-mysql.svc.cluster.local -u root -p${MYSQL_ROOT_PASSWORD} ${MYSQL_DATABASE}                                      
                                                                                                                                                 
To connect to your database directly from outside the K8s cluster:                                                                               
    MYSQL_HOST=127.0.0.1                                                                                                                         
    MYSQL_PORT=3306                                                                                                                              
                                                                                                                                                 
    ## Execute the following commands to route the connection:                                                                                    
    export POD_NAME=$(oc get pods --namespace helm-mysql -l "app=mysql-mysql" -o jsonpath="{.items[0].metadata.name}"; echo)                    
    oc port-forward --namespace helm-mysql $POD_NAME 3306:3306                                                                                  
                                                                                                                                                 
    mysql -h ${MYSQL_HOST} -P${MYSQL_PORT} -u root -p${MYSQL_ROOT_PASSWORD} ${MYSQL_DATABASE}

4.查看应用的Pod运行部署状态

$ oc get pod -n helm-mysql
NAME                           READY   STATUS    RESTARTS   AGE
mysql-mysql-7dd478658f-cxjhh   1/1     Running   0          34m

定制Helm Chart(创建新的Helm Chart)

1.在my-chart-nginx项目中创建名为my-chart-nginx的Chart。

$ oc new-project my-chart-nginx
$ helm create my-chart-nginx

2.查看Helm创建的my-chart目录包括的资源。

$ tree my-chart-nginx
my-chart-nginx
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml
 
3 directories, 10 files

3.为了简化，可以删除以下不必要的配置文件。

$ rm my-chart-nginx/templates/hpa.yaml
$ rm my-chart-nginx/templates/ingress.yaml
$ rm my-chart-nginx/templates/NOTES.txt
$ rm my-chart-nginx/templates/serviceaccount.yaml

4.修改my-chart-nginx/templates/deployment.yaml文件，确保文件为以下内容：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-chart-nginx.fullname" . }}
  labels:
    app.kubernetes.io/name: {{ include "my-chart-nginx.name" . }}
    helm.sh/chart: {{ include "my-chart-nginx.chart" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ include "my-chart-nginx.name" . }}
      app.kubernetes.io/instance: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app.kubernetes.io/name: {{ include "my-chart-nginx.name" . }}
        app.kubernetes.io/instance: {{ .Release.Name }}
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP

5.修改my-chart-nginx/templates/service.yaml文件，确保文件为以下内容：

apiVersion: v1
kind: Service
metadata:
  name: {{ include "my-chart-nginx.fullname" . }}
  labels:
    app.kubernetes.io/name: {{ include "my-chart-nginx.name" . }}
    helm.sh/chart: {{ include "my-chart-nginx.chart" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: {{ include "my-chart-nginx.name" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}

6.编辑my-chart-nginx/values.yaml文件，确保image部分的内容为以下内容。

replicaCount: 1
image:
  repository: "bitnami/nginx"
  tag: "latest"
  pullPolicy: IfNotPresent
service:
  type: NodePort
  port: 80

7.执行命令检查Chart目录中资源。

$ helm lint ./my-chart-nginx/
==> Linting ./my-chart-nginx/
[INFO] Chart.yaml: icon is recommended
 
1 chart(s) linted, 0 chart(s) failed

8.执行命令，根据my-chart创建资源。

$ helm install my-chart-nginx ./my-chart-nginx
NAME: my-chart-nginx
LAST DEPLOYED: Fri Dec 25 03:01:02 2020
NAMESPACE: my-chart-nginx
STATUS: deployed
REVISION: 1

9.执行命令，查看helm部署的Chart。

$ helm ls 
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
my-chart-nginx  my-chart-nginx  1               2020-12-25 03:01:02.016558706 +0000 UTC deployed        my-chart-nginx-0.1.0    1.16.0

10.查看通过Chart部署的资源。

$ oc get all -n my-chart-nginx
NAME                                  READY   STATUS    RESTARTS   AGE
pod/my-chart-nginx-6d5876c89b-svx9w   1/1     Running   0          108s
 
NAME                     TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/my-chart-nginx   NodePort   172.30.83.106   <none>        80:31047/TCP   108s
 
NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-chart-nginx   1/1     1            1           108s
 
NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/my-chart-nginx-6d5876c89b   1         1         1       108s

11.生成Route，然后用浏览器访问route

$ oc expose svc my-chart-nginx -n my-chart-nginx
route.route.openshift.io/my-chart-nginx exposed
$ oc get route -n my-chart-nginx
NAME             HOST/PORT                                                                             PATH   SERVICES         PORT   TERMINATION   WILDCARD
my-chart-nginx   my-chart-nginx-my-chart-nginx.apps.cluster-pek-99bc.pek-99bc.sandbox309.opentlc.com

修改更新Helm Chart

1.删除刚刚手动创建的Route对象

$  oc delete route my-chart-nginx  -n my-chart-nginx

2.新建内容如下的my-chart-nginx/templates/routes.yaml文件

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: {{ include "my-chart-nginx.fullname" . }}
  labels:
    {{- include "my-chart-nginx.labels" . | nindent 4 }}
spec:
  port:
    targetPort: http
  to:
    kind: Service
    name: {{ include "my-chart-nginx.fullname" . }}
    weight: 100
  wildcardPolicy: None

3.执行命令，更新Helm Chart

$ helm upgrade my-chart-nginx ./my-chart-nginx
Release "my-chart-nginx" has been upgraded. Happy Helming!
NAME: my-chart-nginx
LAST DEPLOYED: Fri Dec 25 03:11:19 2020
NAMESPACE: my-chart-nginx
STATUS: deployed
REVISION: 2

4.确认已经Helm创建出Route类型对象。

$ oc get route -n my-chart-nginx
NAME             HOST/PORT                                                                             PATH   SERVICES         PORT   TERMINATION   WILDCARD
my-chart-nginx   my-chart-nginx-my-chart-nginx.apps.cluster-pek-99bc.pek-99bc.sandbox309.opentlc.com          my-chart-nginx   http                 None

回退Helm Chart配置

1.进入OpenShift的“开发者”视图的“Helm”菜单，然后进入my-chart-nginx，在“Helm 修改历史”中可以看到Helm每次更新的记录。

2.在my-chart-nginx的“资源”中可查看通过Helm Chart创建的资源。

3.点击上图“操作”下拉菜单中的“回滚”项目，此时界面将显示下图。选中“修订”为“1”的版本，然后点击“回滚”按钮。

4.此时再次查看my-chart-nginx的“资源”，确认Route资源已经没有了。这说明Helm Chart已经回退到最开始的配置了。

5.再次查看my-chart-nginx的“Helm 修改历史”，确认“修订”当前版本为“3”。根据描述，“3”版是从“1”版回滚的。

92IT

Just love IT.

Hands-on Lab – 使用Helm部署OpenShift应用