Ansible自动化部署Harbor 2.14.0高可用集群实战指南在容器化技术普及的今天企业级私有镜像仓库Harbor已成为DevOps工具链中不可或缺的一环。然而传统的手动部署方式不仅耗时费力更难以保证多环境的一致性。本文将展示如何通过Ansible实现Harbor集群的全自动化部署让您从繁琐的配置工作中解放出来。1. 环境规划与准备1.1 基础设施需求构建高可用Harbor集群需要考虑以下关键因素节点角色划分负载均衡节点2台避免单点故障Harbor服务节点至少3台确保服务冗余数据库集群PostgreSQL建议3节点Redis集群建议3节点哨兵模式硬件规格建议节点类型CPU内存磁盘网络带宽负载均衡2核4GB50GB1GbpsHarbor服务节点8核16GB500GB SSD1Gbps数据库节点4核8GB1TB SSD1GbpsRedis节点2核4GB100GB1Gbps提示生产环境建议所有节点使用SSD存储特别是数据库和Redis节点1.2 证书管理方案安全通信是高可用集群的基础我们采用以下证书策略# 证书目录结构示例 /etc/harbor-certs/ ├── ca.crt ├── harbor01.example.com.crt ├── harbor01.example.com.key ├── harbor02.example.com.crt └── harbor02.example.com.key使用OpenSSL生成证书的Ansible任务示例- name: Generate CA certificate openssl_certificate: path: /etc/harbor-certs/ca.crt privatekey_path: /etc/harbor-certs/ca.key csr_path: /etc/harbor-certs/ca.csr provider: selfsigned selfsigned_not_after: 3650d subject_alt_name: DNS:harbor-cluster.example.com2. Ansible角色设计与实现2.1 核心角色结构我们的Ansible Playbook采用模块化设计主要包含以下角色roles/ ├── common/ # 基础环境配置 ├── postgresql-ha/ # 数据库集群 ├── redis-ha/ # Redis集群 ├── harbor/ # Harbor服务 │ ├── tasks/ │ │ ├── main.yml │ │ ├── install.yml │ │ └── config.yml │ └── templates/ │ ├── harbor.yml.j2 │ └── docker-compose.yml.j2 └── loadbalancer/ # 负载均衡配置2.2 关键配置模板Harbor的核心配置文件通过Jinja2模板动态生成{# roles/harbor/templates/harbor.yml.j2 #} hostname: {{ harbor_domain }} http: port: 80 https: port: 443 certificate: /etc/harbor/ssl/{{ inventory_hostname }}.crt private_key: /etc/harbor/ssl/{{ inventory_hostname }}.key external_database: host: {{ postgresql_vip }} port: 5432 db_name: registry username: harbor password: {{ postgresql_harbor_password }} external_redis: host: {{ redis_vip }} password: {{ redis_password }}2.3 数据库高可用配置PostgreSQL集群采用Patronietcd实现自动故障转移- name: Configure Patroni template: src: patroni.yml.j2 dest: /etc/patroni/patroni.yml vars: node_name: {{ inventory_hostname }} etcd_hosts: {% for host in groups[etcd] %}{{ host }}:2379{% if not loop.last %},{% endif %}{% endfor %} postgresql_parameters: max_connections: 500 shared_buffers: 4GB3. 部署流程自动化3.1 主Playbook设计完整的部署流程通过一个主Playbook协调# harbor-ha-deploy.yml - hosts: etcd roles: - etcd - hosts: postgresql roles: - postgresql-ha - hosts: redis roles: - redis-ha - hosts: harbor serial: 1 # 滚动更新避免服务中断 roles: - harbor - hosts: loadbalancer roles: - loadbalancer3.2 证书分发机制使用Ansible Vault保护敏感证书实现安全分发# 加密证书文件 ansible-vault encrypt roles/harbor/files/ssl/*.key # Playbook中安全部署证书 - name: Deploy SSL certificates copy: src: {{ item }} dest: /etc/harbor/ssl/ mode: 0600 loop: - ssl/{{ inventory_hostname }}.crt - ssl/{{ inventory_hostname }}.key become: yes no_log: true # 避免日志泄露敏感信息4. 高可用架构验证4.1 故障转移测试验证集群的容错能力是关键步骤数据库主节点故障# 手动停止主节点PostgreSQL服务 ansible postgresql[0] -b -m service -a namepatroni statestopped # 观察故障转移 curl http://etcd:2379/v2/keys/service/postgresql/leader | jq .Harbor节点宕机# 停止一个Harbor节点 ansible harbor[0] -b -m service -a namedocker statestopped # 测试服务连续性 while true; do curl -k https://harbor-cluster.example.com/api/v2.0/ping; sleep 1; done4.2 性能基准测试使用Docker基准测试工具验证集群性能# 并发推送测试 docker run --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ -e DOCKER_USERadmin \ -e DOCKER_PASSHarbor12345 \ -e REGISTRY_URLharbor-cluster.example.com \ registry-benchmark \ --concurrency 20 \ --total 1000 \ --size small测试结果参考指标指标单节点集群(3节点)镜像推送(QPS)120320镜像拉取(QPS)200550API响应时间(ms)150805. 运维与扩展5.1 监控集成建议配置Prometheus监控关键指标# roles/harbor/tasks/monitoring.yml - name: Configure Prometheus targets lineinfile: path: /etc/prometheus/targets/harbor.yml line: - targets: [{{ inventory_hostname }}:9090] insertafter: EOF state: present notify: reload prometheus - name: Deploy Grafana dashboard uri: url: http://grafana:3000/api/dashboards/db method: POST body_format: json body: {{ lookup(file, files/grafana-dashboard.json) | from_json }} headers: Authorization: Bearer {{ grafana_api_token }}5.2 版本升级策略采用蓝绿部署实现无缝升级部署新版本Harbor到备用节点组将负载均衡流量逐步切换到新节点验证新版本稳定性下线旧版本节点对应的Ansible Playbook片段- hosts: harbor-new roles: - harbor vars: harbor_version: 2.14.1 - name: Switch load balancer backend uri: url: http://loadbalancer:8080/api/backend method: PUT body_format: json body: old_backend: harbor-old new_backend: harbor-new status_code: 2006. 完整Playbook示例以下是核心部署任务的完整示例# roles/harbor/tasks/main.yml - include_tasks: prerequisites.yml tags: always - include_tasks: certificates.yml when: harbor_https_enabled - include_tasks: database.yml when: harbor_external_database - include_tasks: install.yml tags: install - include_tasks: config.yml tags: config - include_tasks: service.yml tags: service - include_tasks: verify.yml tags: verify对应的install.yml关键内容- name: Download Harbor offline package get_url: url: https://github.com/goharbor/harbor/releases/download/v{{ harbor_version }}/harbor-offline-installer-v{{ harbor_version }}.tgz dest: /tmp/harbor-offline-installer-v{{ harbor_version }}.tgz checksum: sha256:{{ harbor_checksum }} - name: Extract Harbor package unarchive: src: /tmp/harbor-offline-installer-v{{ harbor_version }}.tgz dest: /opt remote_src: yes extra_opts: [--strip-components1] - name: Generate configuration template: src: harbor.yml.j2 dest: /opt/harbor/harbor.yml notify: Restart Harbor - name: Run installer command: ./install.sh --with-trivy --with-chartmuseum args: chdir: /opt/harbor environment: HTTP_PROXY: {{ proxy.http_proxy | default() }} HTTPS_PROXY: {{ proxy.https_proxy | default() }}在实际项目中我们通过这种自动化部署方式将Harbor集群部署时间从原来的4小时缩短到15分钟且完全消除了人为配置错误。特别是在多环境部署场景下只需调整inventory文件即可快速完成测试、预生产和生产环境的部署。