基于Kubernetes的Apache Pulsar云原生架构解析与集群部署指南（下）

文章目录

k8s安装部署Pulsar集群
前期准备
- 版本要求
安装 Pulsar Helm chart
管理pulsar
- Clusters
- Brokers
- Topic

k8s安装部署Pulsar集群

前期准备

版本要求

Kubernetes 集群，版本 1.14 或更高版本
Helm v3（3.0.2 或更高版本）
数据持久化（本文使用NFS CSI动态供给）

K8s集群信息

[root@master ~]# kubectl get node
NAME      STATUS   ROLES           AGE     VERSION
master    Ready    control-plane   77d     v1.24.2
worker1   Ready    <none>          77d     v1.24.2
worker2   Ready    <none>          77d     v1.24.2
worker3   Ready    <none>          4d18h   v1.24.2

Helm安装

wget https://get.helm.sh/helm-v3.17.2-linux-amd64.tar.gz
tar xf helm-v3.17.2-linux-amd64.tar.gz
chown root:root helm
mv linux-amd64/helm /usr/local/sbin/
helm version

NFS server部署
本文使用yum 包部署nfs server

nfs server主机安装nfs相关软件包

yum install nfs-utils -y

启动nfs server 相关服务

sudo systemctl start rpcbind
systemctl enable rpcbind
systemctl start nfs-server
systemctl enable nfs-server

配置nfs共享目录导出
（/nfsdata目录为主机单独一块磁盘的挂载点，需提前配置完成。）

vim /etc/exports
/nfsdata 192.168.61.0/24(rw,sync,no_subtree_check,no_root_squash)

导出共享目录
exportfs -ra

为数据持久化部署csi storageclass
本文使用nfs csi storageclass，但nfs不具有高可用性，生产环境建议使用ceph，glusterfs分布式的存储解决方案。
在 Kubernetes 中，NFS CSI（Container Storage Interface）StorageClass 提供了一种标准化的方式来动态供应基于 NFS（Network File System）的持久化存储卷。通过使用 NFS CSI 驱动程序和相应的 StorageClass，用户可以更方便地请求、分配和管理持久化存储资源，而无需手动创建 PersistentVolume（PV）。

nfs csi storageclass安装部署步骤参考：
https://github.com/kubernetes-csi/csi-driver-nfs

安装 Pulsar Helm chart

添加 Pulsar repo

helm repo add apache https://pulsar.apache.org/charts
helm repo update

列出可部署的版本，可以看出chart和pulsar的版本并非一一对应。

本文要安装的pulsar版本为2.10.2，因此选择pulsar heml对应版本为3.0.0

[root@master pulsar-helm-chart]# helm search repo apache/pulsar --versions
NAME            CHART VERSION   APP VERSION     DESCRIPTION                            
apache/pulsar   4.0.0           4.0.3           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.9.0           4.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.8.0           4.0.1           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.7.0           4.0.0           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.6.0           3.0.7           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.5.0           3.0.6           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.4.1           3.0.5           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.4.0           3.0.4           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.3.1           3.0.3           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.3.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.2.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.1.0           3.0.2           Apache Pulsar Helm chart for Kubernetes
apache/pulsar   3.0.0           2.10.2          Apache Pulsar Helm chart for Kubernetes
apache/pulsar   2.9.4           2.9.3           Apache Pulsar Helm chart for Kubernetes

从helm仓库中下载指定版本的pulsar helm chart 安装包

helm pull apache/pulsar --version 3.0.0
pulsar-3.0.0.tgz

tar xf pulsar-3.0.0.tgz
cd pulsar

ls -l

总用量 68K

-rw-r--r-- 1 root root  253 10月 21  2022 Chart.lock
drwxr-xr-x 3 root root 4.0K  4月 23 22:01 charts
-rw-r--r-- 1 root root  529 10月 21  2022 Chart.yaml
-rw-r--r-- 1 root root  13K 10月 21  2022 LICENSE
-rw-r--r-- 1 root root  167 10月 21  2022 NOTICE
drwxr-xr-x 2 root root 4.0K  4月 23 22:01 templates
-rw-r--r-- 1 root root  32K  4月 24 10:48 values.yaml

通过修改配置文件指定pulsar版本

vim scripts/pulsar/common_auth.sh
…
PULSAR_VERSION="2.10.2"
…

参考以下内容修改values.yaml文件

namespace: "pulsar"
initialize: true
volumes:
  persistence: true
  local_storage: false
…
components:
  # zookeeper
  zookeeper: true
…
  # pulsar manager
  pulsar_manager: true
…
images:
  zookeeper:
    repository: apachepulsar/pulsar-all
    # uses defaultPulsarImageTag when unspecified
    tag: 2.10.2
    pullPolicy: IfNotPresent
  bookie:
    repository: apachepulsar/pulsar-all
    # uses defaultPulsarImageTag when unspecified
    tag: 2.10.2
    pullPolicy: IfNotPresent
…
  pulsar_manager:
    repository: apachepulsar/pulsar-manager
    tag: v0.3.0
    pullPolicy: IfNotPresent
hasCommand: false
…
zookeeper:
volumes:
  persistence: true
  data:
    storageClassName: nfs-csi
…
bookkeeper:
volumes:
persistence: true
journal:
      name: journal
      size: 10Gi
      local_storage: false
  storageClassName: nfs-csi
    ledgers:
      name: ledgers
      size: 50Gi
      local_storage: false
      storageClassName: nfs-csi
…
kube-prometheus-stack:
  enabled: false
…

使用 Pulsar Helm将 Pulsar 集群安装到 Kubernetes。

helm install \
	--values values.yaml \
	--set initialize=true \
	--version 3.0.0 \
	pulsar01 apache/pulsar

查看pod的状态

[root@master pulsar]# kubectl get pod -n pulsar 
NAME                                      READY   STATUS      RESTARTS      AGE
pulsar01-bookie-0                         1/1     Running     0             13h
pulsar01-bookie-1                         1/1     Running     0             13h
pulsar01-bookie-2                         1/1     Running     0             13h
pulsar01-bookie-init-zb9gt                0/1     Completed   0             13h
pulsar01-broker-0                         1/1     Running     3 (12h ago)   13h
pulsar01-broker-1                         1/1     Running     3 (12h ago)   13h
pulsar01-broker-2                         1/1     Running     1 (13h ago)   13h
pulsar01-proxy-0                          1/1     Running     0             13h
pulsar01-pulsar-init-6zcx5                0/1     Completed   0             13h
pulsar01-pulsar-manager-f856cd9c6-fzj72   1/1     Running     0             33m
pulsar01-recovery-0                       1/1     Running     0             13h
pulsar01-toolset-0                        1/1     Running     0             13h
pulsar01-zookeeper-0                      1/1     Running     0             13h
pulsar01-zookeeper-1                      1/1     Running     0             13h
pulsar01-zookeeper-2                      1/1     Running     0             13h

查看svc

[root@master pulsar]# kubectl get svc -n pulsar 
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
pulsar01-bookie           ClusterIP      None            <none>        3181/TCP,8000/TCP                     13h
pulsar01-broker           ClusterIP      None            <none>        8080/TCP,6650/TCP                     13h
pulsar01-proxy            LoadBalancer   10.96.97.171    <pending>     80:31682/TCP,6650:30689/TCP           13h
pulsar01-pulsar-manager   LoadBalancer   10.100.16.216   <pending>     9527:31991/TCP                        33m
pulsar01-recovery         ClusterIP      None            <none>        8000/TCP                              13h
pulsar01-toolset          ClusterIP      None            <none>        <none>                                13h
pulsar01-zookeeper        ClusterIP      None            <none>        8000/TCP,2888/TCP,3888/TCP,2181/TCP   13h

查看pvc

[root@master pulsar]# kubectl get pvc -n pulsar 
NAME                                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pulsar01-bookie-journal-pulsar01-bookie-0      Bound    pvc-c86bb6dd-d98e-4b0f-b9f0-c67c169a5255   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-1      Bound    pvc-e7c2935d-d4cf-4cad-8d63-5dc87dbcae5a   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-2      Bound    pvc-9dfa73ec-8512-495a-ad6d-d3141a97128d   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-journal-pulsar01-bookie-3      Bound    pvc-1f038b9a-4d0d-46c8-89ca-dc8f630ade76   10Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-0      Bound    pvc-8fcb5205-eef2-4903-90e4-68499746c8d8   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-1      Bound    pvc-4412bca2-4314-4fd6-9fb0-85bf1472fd53   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-2      Bound    pvc-a0fae489-99f9-4f39-ba7d-a543d2691502   50Gi       RWO            nfs-csi        13h
pulsar01-bookie-ledgers-pulsar01-bookie-3      Bound    pvc-0124e56b-0c28-4b24-92be-019acb8331a8   50Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-0   Bound    pvc-81219d17-27f6-4ee0-864c-9f1a296f85cd   20Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-1   Bound    pvc-8571b0c0-9ad9-4a5c-9582-0a81fda4c6ff   20Gi       RWO            nfs-csi        13h
pulsar01-zookeeper-data-pulsar01-zookeeper-2   Bound    pvc-ab41f541-c9a1-475b-ab07-cb1b90b47c87   20Gi       RWO            nfs-csi        13h

在修改values.yaml后可以使用upgrade更新部署

[root@master pulsar]# helm upgrade --values values.yaml --set initialize=true --version 3.0.0 pulsar01 apache/pulsar
Release "pulsar01" has been upgraded. Happy Helming!
NAME: pulsar01
LAST DEPLOYED: Thu Apr 24 10:48:19 2025
NAMESPACE: default
STATUS: deployed
REVISION: 4
TEST SUITE: None

c如何删除集群

[root@master pulsar]# helm list
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
pulsar01        default         4               2025-04-24 10:48:19.873604446 +0800 CST deployed        pulsar-3.0.0    2.10.2    

[root@master pulsar]# helm uninstall pulsar01 --namespace pulsar
release "pulsar01 " uninstalled

[root@master pulsar]# helm list --namespace pulsar
NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION

管理pulsar

Clusters

获取现有集群的配置

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin clusters get pulsar01
{
  "serviceUrl" : "http://pulsar01-broker.pulsar.svc.cluster.local:8080/",
  "serviceUrlTls" : "https://pulsar01-broker.pulsar.svc.cluster.local:8443/",
  "brokerServiceUrl" : "pulsar://pulsar01-broker.pulsar.svc.cluster.local:6650/",
  "brokerServiceUrlTls" : "pulsar+ssl://pulsar01-broker.pulsar.svc.cluster.local:6651/",
  "brokerClientTlsEnabled" : false,
  "tlsAllowInsecureConnection" : false,
  "brokerClientTlsEnabledWithKeyStore" : false,
  "brokerClientTlsTrustStoreType" : "JKS"
}

获取 Pulsar实例中所有集群的列表

I have no name!@pulsar01-toolset-0:/pulsar/bin$./pulsar-admin clusters list
pulsar01

I have no name!@pulsar01-toolset-0:/pulsar/bin$ curl -X GET "http://pulsar01-proxy.pulsar.svc.cluster.local/admin/v2/clusters"v2/clusters"
["pulsar01"]
Tenants

列出所有租户

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants list
public
pulsar

I have no name!@pulsar01-toolset-0:/pulsar/bin$ curl -X GET "http://pulsar01-proxy.pulsar.svc.cluster.local/admin/v2/tenants"
["public","pulsar"]

新创建1个名为my-tenant的租户

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants create my-tenant

创建租户时，您可以选择使用-r/–admin-roles 标志分配管理员角色，并使用-c/–

allowed-clusters标志分配集群。您可以以逗号分隔的列表形式指定多个值。
I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants create my-tenant2 -r role1 -c pulsar01

curl -X PUT "http://pulsar01-proxy/admin/v2/tenants/my-tenant3" \
> -H "Content-Type: application/json" \
> -d '{
>   "allowedClusters": ["pulsar01"],
>   "adminRoles": ["role1"]
> }'

获取现有租户的配置。

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants get my-tenant2                                                
{
  "adminRoles" : [ "role1" ],
  "allowedClusters" : [ "pulsar01" ]
}

I have no name!@pulsar01-toolset-0:/pulsar$ curl -X GET "http://pulsar01-proxy/admin/v2/tenants/my-tenant3"
{"adminRoles":["role1"],"allowedClusters":["pulsar01"]}

删除指定租户

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin tenants delete my-tenant

I have no name!@pulsar01-toolset-0:/pulsar$ curl -X DELETE "http://pulsar01-proxy/admin/v2/tenants/my-tenant"

Brokers

可以通过以下方式管理broker：

工具brokers的命令pulsar-admin
/admin/v2/brokers管理REST API的端点

列出所有活跃的broker

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers list use 
pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar01-broker-1.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar01-broker-0.pulsar01-broker.pulsar.svc.cluster.local:8080

curl -s -X GET "http://pulsar01-proxy.pulsar/admin/v2/brokers/pulsar01"    
["pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080","pulsar01-broker-1.pulsar01-broker.pulsar.svc.cluster.local:8080","pulsar01-broker-0.pulsar01-broker.pulsar.svc.cluster.local:8080"]

列出指定broker拥有和服务的所有命名空间。

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers namespaces use --url pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080
pulsar/pulsar01/pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080/0x00000000_0xffffffff    [broker_assignment=shared is_controlled=false is_active=true]
pulsar/pulsar01-broker-2.pulsar01-broker.pulsar.svc.cluster.local:8080/0x00000000_0xffffffff    [broker_assignment=shared is_controlled=false is_active=true]
public/functions/0x40000000_0x80000000    [broker_assignment=shared is_controlled=false is_active=true]
public/default/0x20000000_0x30000000    [broker_assignment=shared is_controlled=false is_active=true]

获取broker的监控指标数据

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin broker-stats monitoring-metrics

此命令输出的内容较多，为json格式，建议使用jq格式化，更易阅读。
涵盖了 Pulsar Broker 的多个关键方面，包括：

Managed Ledger 缓存性能。
Topic 加载时间。
ZooKeeper 交互延迟。
JVM 性能和资源使用。
Broker 客户端连接状态。
负载均衡资源使用。
查看broker的健康状态

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers healthcheck
Ok

查看broker的版本：

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin brokers version    
2.10.2

Topic

创建topic

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics create persistent://public/default/my-topic-1

创建3分区的topic

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics create-partitioned-topic persistent://public/default/my-partitioned-topic --partitions 3

查看topic状态

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics stats persistent://public/default/my-topic-1

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics  partitioned-stats persistent://public/default/my-partitioned-topic --per-partition

Topic增加分区

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics update-partitioned-topic persistent://public/default/my-partitioned-topic --partitions 6

分区数量应与预期的吞吐量需求相匹配。例如：

如果你预计每秒需要处理 100,000 条消息，并且单个分区的吞吐量为 10,000 条/秒，则至少需要 10 个分区。
如果你的集群有 5 个 Broker，那么 10 个分区可以均匀分布在这些 Broker 上，每个 Broker 处理 2 个分区
查看指定topic的ledgerId

I have no name!@pulsar01-toolset-0:/pulsar/bin$ ./pulsar-admin topics info-internal persistent://public/default/my-topic
{
  "version": 7,
  "creationDate": "2025-04-23T14:29:59.347Z",
  "modificationDate": "2025-04-25T03:54:25.812Z",
  "ledgers": [
    {
      "ledgerId": 11,
      "entries": 36,
      "size": 1870,
      "isOffloaded": false
    },
    {
      "ledgerId": 18,
      "entries": 1,
      "size": 48,
      "isOffloaded": false
    },
    {
      "ledgerId": 28,
      "isOffloaded": false
    }
  ],