ubuntu kubeasz 部署高可用k8s 集群

测试环境
- 主机列表
- 软件清单
kubeasz 部署高可用 kubernetes
- 配置源
- 配置host文件
- 安装 ansible 并进行 ssh 免密登录：
- 下载 kubeasz 项⽬及组件
部署集群
- 部署各组件
- 开始安装
- 修改 config 配置文件
增加 master 节点
增加 kube_node 节点
登录dashboard
- 查看dashboard容器运行在那个节点及端口了
- 通过浏览器访问dashboard
- 生成Token登录
安装持久化存储CSI解决方案-LongHorn

测试环境

kubeasz与K8S版本对比
在这里插入图片描述

主机列表

在这里插入图片描述

软件清单

操作系统: Ubuntu server 25.04
k8s版本 1.32.3
calico 3.28.3
kubeasz 3.6.6
etcd  v3.5.20

kubeasz 部署高可用 kubernetes

部署节点基本配置
此处部署节点是：192.168.8.19，部署节点的功能如下：

1从互联⽹下载安装资源
2可选将部分镜像修改tag后上传到公司内部镜像仓库服务器
3对master进⾏初始化
4对node进⾏初始化
5后期集群维护，包括：添加及删除master节点;添加就删除node节点;etcd数据备份及恢复

配置源

1，配置本地光盘源
将系统光盘挂载到光驱, mount到/mnt下

mount /dev/cdrom /mnt
echo "edeb file:///mnt plucky main restricted" >/etc/apt/sources.list.d/cdrom.list

2，配置ali源

sudo cp /etc/apt/sources.list.d/ubuntu.sources /etc/apt/sources.list.d/ubuntu.sources.bak
cat > /etc/apt/sources.list.d/ubuntu.sources << EOF
Types: deb
URIs: http://mirrors.aliyun.com/ubuntu/
Suites: noble noble-updates noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
EOF

配置host文件

root@uasz1:~# cat /etc/hosts
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.8.13 umaster1.meng.com umaster1
192.168.8.14 umaster2.meng.com umaster2
192.168.8.15 umaster3.meng.com umaster3
192.168.8.16 uworker1.meng.com uworker1
192.168.8.17 uworker2.meng.com uworker2
192.168.8.18 uworker3.meng.com uworker3
192.168.8.19 uasz1.meng.com uasz1

安装 ansible 并进行 ssh 免密登录：

apt 安装 ansieble，并将部署节点的公钥拷贝至 master、node、etcd 节点，部署节点能免密登录其他主机
注：在部署节点执行

root@uasz1:~# apt update &&　apt install ansible -y
root@uasz1:~# ssh-keygen    # 生成密钥对，一路回车即可
root@uasz1:~# apt install sshpass -y # 如果已经安装不需要执行

#!/bin/bash
#目标主机列表
IP="
192.168.8.13
192.168.8.14
192.168.8.15
192.168.8.16
192.168.8.17
192.168.8.18
"
REMOTE_PORT="22"
REMOTE_USER="root"
REMOTE_PASS="root"
for REMOTE_HOST in ${IP};do
  REMOTE_CMD="echo ${REMOTE_HOST} is successfully!"
  ssh-keyscan -p "${REMOTE_PORT}" "${REMOTE_HOST}" >> ~/.ssh/known_hosts   #在本地添加远程主机的公钥信息，避免交互式应答
  sshpass -p "${REMOTE_PASS}" ssh-copy-id "${REMOTE_USER}@${REMOTE_HOST}"
  if [ $? -eq 0 ];then
    echo ${REMOTE_HOST} 免秘钥配置完成!
    ssh ${REMOTE_HOST} ln -sv /usr/bin/python3 /usr/bin/python
  else
    echo "免密钥配置失败!"
  fi
done
# 验证免密登录
ssh 192.168.3.100

下载 kubeasz 项⽬及组件

root@uasz1:mkdir -p /data/kubeasz
root@uasz1:cd /data/kubeasz
root@uasz1:/data/kubeasz# wget https://github.com/easzlab/kubeasz/releases/download/3.6.6/ezdown
root@uasz1:/data/kubeasz# chmod +x ezdown
#下载kubeasz代码、二进制、默认容器镜像,会运行一个redistry镜像仓库，将下载的镜像push到仓库
root@uasz1:/data/kubeasz# ./ezdown -D
root@uasz1:/data/kubeasz# ll /etc/kubeasz/       #kubeasz所有文件和配置路径
total 148
drwxrwxr-x  13 root root  4096 May 19 21:47 ./
drwxr-xr-x 112 root root  4096 May 19 22:24 ../
-rw-rw-r--   1 root root 20304 Mar 23 20:24 ansible.cfg
drwxr-xr-x   5 root root  4096 May 19 21:14 bin/
drwxr-xr-x   3 root root  4096 May 19 21:47 clusters/
drwxrwxr-x   8 root root  4096 Mar 23 20:30 docs/
drwxr-xr-x   3 root root  4096 May 19 21:27 down/
drwxrwxr-x   2 root root  4096 Mar 23 20:30 example/
-rwxrwxr-x   1 root root 25868 Mar 23 20:24 ezctl*
-rwxrwxr-x   1 root root 32772 Mar 23 20:24 ezdown*
drwxrwxr-x   4 root root  4096 Mar 23 20:30 .github/
-rw-rw-r--   1 root root   301 Mar 23 20:24 .gitignore
drwxrwxr-x  10 root root  4096 Mar 23 20:30 manifests/
drwxrwxr-x   2 root root  4096 Mar 23 20:30 pics/
drwxrwxr-x   2 root root  4096 Mar 23 20:30 playbooks/
-rw-rw-r--   1 root root  6404 Mar 23 20:24 README.md
drwxrwxr-x  22 root root  4096 Mar 23 20:30 roles/
drwxrwxr-x   2 root root  4096 Mar 23 20:30 tools/

部署集群

root@uasz1:/data/kubeasz# cd /etc/kubeasz/
root@uasz1:/etc/kubeasz# ./ezctl new k8s-cluster01         # 新建管理集群
2025-05-19 21:47:03 [ezctl:145] DEBUG generate custom cluster files in /etc/kubeasz/clusters/k8s-cluster01     #集群使用相关配置路径
2025-05-19 21:47:04 [ezctl:151] DEBUG set versions
2025-05-19 21:47:04 [ezctl:179] DEBUG cluster k8s-cluster01: files successfully created.
2025-05-19 21:47:04 [ezctl:180] INFO next steps 1: to config '/etc/kubeasz/clusters/k8s-cluster01/hosts'
2025-05-19 21:47:04 [ezctl:181] INFO next steps 2: to config '/etc/kubeasz/clusters/k8s-cluster01/config.yml'

配置用于集群管理的 ansible hosts 文件

root@uasz1:/etc/kubeasz/clusters/k8s-cluster01# cat hosts
# 'etcd' cluster should have odd member(s) (1,3,5,...)
[etcd]
192.168.8.13
192.168.8.14
192.168.8.15

# master node(s), set unique 'k8s_nodename' for each node
# CAUTION: 'k8s_nodename' must consist of lower case alphanumeric characters, '-' or '.',
# and must start and end with an alphanumeric character
[kube_master]
192.168.8.13 k8s_nodename='umaster1'
192.168.8.14 k8s_nodename='umaster2'
192.168.8.15 k8s_nodename='umaster3'

# work node(s), set unique 'k8s_nodename' for each node
# CAUTION: 'k8s_nodename' must consist of lower case alphanumeric characters, '-' or '.',
# and must start and end with an alphanumeric character
[kube_node]
192.168.8.16 k8s_nodename='uworker1'
192.168.8.17 k8s_nodename='uworker2'
192.168.8.18 k8s_nodename='uworker3'

# [optional] harbor server, a private docker registry
# 'NEW_INSTALL': 'true' to install a harbor server; 'false' to integrate with existed one
[harbor]
#192.168.1.8 NEW_INSTALL=false

# [optional] loadbalance for accessing k8s from outside
[ex_lb]
#192.168.1.6 LB_ROLE=backup EX_APISERVER_VIP=192.168.1.250 EX_APISERVER_PORT=8443
#192.168.1.7 LB_ROLE=master EX_APISERVER_VIP=192.168.1.250 EX_APISERVER_PORT=8443

# [optional] ntp server for the cluster
[chrony]
#192.168.1.1

[all:vars]
# --------- Main Variables ---------------
# Secure port for apiservers
SECURE_PORT="6443"

# Cluster container-runtime supported: docker, containerd
# if k8s version >= 1.24, docker is not supported
CONTAINER_RUNTIME="containerd"

# Network plugins supported: calico, flannel, kube-router, cilium, kube-ovn
CLUSTER_NETWORK="calico"

# Service proxy mode of kube-proxy: 'iptables' or 'ipvs'
PROXY_MODE="ipvs"

# K8S Service CIDR, not overlap with node(host) networking
SERVICE_CIDR="10.68.0.0/16"

# Cluster CIDR (Pod CIDR), not overlap with node(host) networking
CLUSTER_CIDR="172.20.0.0/16"

# NodePort Range
NODE_PORT_RANGE="30000-32767"

# Cluster DNS Domain
CLUSTER_DNS_DOMAIN="cluster.local"

# -------- Additional Variables (don't change the default value right now) ---
# Binaries Directory
bin_dir="/opt/kube/bin"

# Deploy Directory (kubeasz workspace)
base_dir="/etc/kubeasz"

# Directory for a specific cluster
cluster_dir="{
   { base_dir }}/clusters/k8s-cluster01"

# CA and other components cert/key Directory
ca_dir="/etc/kubernetes/ssl"

# Default 'k8s_nodename' is empty
k8s_nodename=''

# Default python interpreter
ansible_python_interpreter=/usr/bin/python

config.yml 是用于配置 K8S 集群的具体配置

root@uasz1:/etc/kubeasz/clusters/k8s-cluster01# cat config.yml
############################
# prepare
############################
# 可选离线安装系统软件包 (offline|online)
INSTALL_SOURCE: "online"

# 可选进行系统安全加固 github.com/dev-sec/ansible-collection-hardening
# (deprecated) 未更新上游项目，未验证最新k8s集群安装，不建议启用
OS_HARDEN: false


############################
# role:deploy
############################
# default: ca will expire in 100 years
# default: certs issued by the ca will expire in 50 years
CA_EXPIRY: "876000h"
CERT_EXPIRY: "876000h"

# force to recreate CA and other certs, not suggested to set 'true'
CHANGE_CA: false

# kubeconfig 配置参数
CLUSTER_NAME: "cluster1"
CONTEXT_NAME: "context-{
   { CLUSTER_NAME }}"

# k8s version
K8S_VER: "1.32.3"

# set unique 'k8s_nodename' for each node, if not set(default:'') ip add will be used
# CAUTION: 'k8s_nodename' must consist of lower case alphanumeric characters, '-' or '.',
# and must start and end with an alphanumeric character (e.g. 'example.com'),
# regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'
K8S_NODENAME: "{%- if k8s_nodename != '' -%} \
                    {
   { k8s_nodename|replace('_', '-')|lower }} \
               {%- else -%} \
                    k8s-{
   { inventory_hostname|replace('.', '-') }} \
               {%- endif -%}"

# use 'K8S_NODENAME' to set hostname
ENABLE_SETTING_HOSTNAME: true


############################
# role:etcd
############################
# 设置不同的wal目录，可以避免磁盘io竞争，提高性能
ETCD_DATA_DIR: "/var/lib/etcd"
ETCD_WAL_DIR: ""


############################
# role:runtime [containerd,docker]
############################
# [.]启用拉取加速镜像仓库
ENABLE_MIRROR_REGISTRY: true

# [.]添加信任的私有仓库
# 必须按照如下示例格式，协议头'http://'和'https://'不能省略
INSECURE_REG:
  - "http://easzlab.io.local:5000"
#  - "https://reg.your