1. 说明

在k8s上部署jenkins，并用 jenkins.k8s-t2.com访问
在namespace为devops下安装在指定节点k8s-master-1，有指定持久化的PV/PVC/SC
CI/DI 实践

2. 步骤

2.1 准备工作

设置代理，不然去git拿文件的时候会遇到麻烦

git config --global http.proxy 'socks5://192.168.0.108:1080'
git config --global https.proxy 'socks5://192.168.0.108:1080'

git config --global --unset http.proxy
git config --global --unset https.proxy

编辑客户机hosts, 映射子域名

192.168.0.103 jenkins.k8s-t2.com

2.2 安装

2.2.1 用jenkins原站for k8s的安装仓方法安装

获取

git clone https://github.com/scriptcamp/kubernetes-jenkins

建个namespace

kubectl create ns devops-tools

编辑 volume.yaml，设置/data0/jenkins-volume为存储地, 节点 k8s-master-0

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv-volume
  labels:
    type: local
spec:
  storageClassName: local-storage
  claimRef:
    name: jenkins-pv-claim
    namespace: devops-tools
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: /data0/jenkins-volume
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-master-0

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
  namespace: devops-tools
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

建serviceAccount.yaml

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-admin
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-admin
  namespace: devops-tools

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins-admin
subjects:
- kind: ServiceAccount
  name: jenkins-admin
  namespace: devops-tools

建deployment.yaml ，为使得jenkins插件能科学安装，需进行deployment中的环境代理设置，否则牙痛 : )

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  namespace: devops-tools
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins-server
  template:
    metadata:
      labels:
        app: jenkins-server
    spec:
      securityContext:
            fsGroup: 1000 
            runAsUser: 1000
      serviceAccountName: jenkins-admin
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts
          resources:
            limits:
              memory: "2Gi"
              cpu: "1000m"
            requests:
              memory: "500Mi"
              cpu: "500m"
          ports:
            - name: httpport
              containerPort: 8080
            - name: jnlpport
              containerPort: 50000
          livenessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 90
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 5
          readinessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
          volumeMounts:
            - name: jenkins-data
              mountPath: /var/jenkins_home   
          env:
          - name: http_proxy
            value: http://192.168.0.108:1081
          - name: https_proxy
            value: http://192.168.0.108:1081
          - name: no_proxy
            value: aliyuncs.com,huaweicloud.com,k8s-master-0,k8s-master-1,k8s-worker-0,localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
      volumes:
        - name: jenkins-data
          persistentVolumeClaim:
              claimName: jenkins-pv-claim

建service.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: devops-tools
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path:   /
      prometheus.io/port:   '8080'
spec:
  selector: 
    app: jenkins-server
  type: NodePort  
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 32000

执行脚本k8s-jenkins.sh

#!/bin/bash

kubectl label node k8s-master-0 app=jenkins-server

kubectl create namespace devops-tools

kubectl apply -f /k8s_apps/kubernetes-jenkins/serviceAccount.yaml

kubectl create -f /k8s_apps/kubernetes-jenkins/volume.yaml

kubectl apply -f /k8s_apps/kubernetes-jenkins/deployment.yaml

kubectl apply -f /k8s_apps/kubernetes-jenkins/service.yaml

成功后可查pod日志获取admin密码

2.2.2 Helm 安装

添加jenkins来源:

helm repo add jenkinsci https://charts.jenkins.io
helm repo update

helm search repo jenkinsci

可知当前版本为:

NAME                    CHART VERSION   APP VERSION     DESCRIPTION
jenkinsci/jenkins       4.3.26          2.401.1         Jenkins - Build great things at any scale! The ...

获取到本地，并解压

helm fetch jenkinsci/jenkins
tar -zxvf jenkins.tgz

编辑 values.yaml：

ingress:
  #enabled: false
=>
ingress:
  enabled: true
  # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
  # 需注明用的是哪个ingress class，因为之前装的是ingress-nginx, 所以这里填nginx
  ingressClassName: nginx
    # Set this path to jenkinsUriPrefix above or use annotations to rewrite path
  hostName: jenkins.k8s-t2.com

注意ingress需指定对应的 ingressClassName

执行安装

helm upgrade --install jenkins --namespace default \
    -f values.yaml \
    jenkins/jenkins

# 过程大概要30分钟

NOTES:
1. 获取admin登录密码 Get your 'admin' user password by running:
  kubectl exec --namespace default -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/additional/chart-admin-password && echo
2. Visit http://jenkins.k8s-t2.com

3. Login with the password from step 1 and the username: admin
4. Configure security realm and authorization strategy
5. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http://jenkins.k8s-t2.com/configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos

获知部署后的情况

orangepi@k8s-master-1:/k8s_apps/jenkins$ kubectl describe ingress jenkins
Name:             jenkins
Labels:           app.kubernetes.io/component=jenkins-controller
                  app.kubernetes.io/instance=jenkins
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=jenkins
                  helm.sh/chart=jenkins-4.3.23
Namespace:        default
Address:
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host                Path  Backends
  ----                ----  --------
  jenkins.k8s-t2.com
                      /jenkins   jenkins:8080 (10.244.2.7:8080)
Annotations:          kubernetes.io/ingress.class: nginx
                      meta.helm.sh/release-name: jenkins
                      meta.helm.sh/release-namespace: default
Events:
  Type    Reason  Age                  From                      Message
  ----    ------  ----                 ----                      -------
  Normal  Sync    11s (x2 over 5m36s)  nginx-ingress-controller  Scheduled for sync

然后就可以欢快地访问 jenkins.k8s-t2.com了