02-大模型部署之KubernetesvLLM安装大模型和容器调度1. Kubernetes基础与vLLM集成概述1.1 为什么使用Kubernetes部署vLLMKubernetes提供了企业级的容器编排能力特别适合vLLM部署的以下场景弹性伸缩根据负载自动调整vLLM实例数量高可用性自动故障恢复和负载均衡资源管理精细化的GPU资源分配和调度多租户隔离不同模型或用户之间的资源隔离版本管理无缝的模型版本升级和回滚1.2 Kubernetes与vLLM架构┌─────────────────────────────────────────────────────────────┐ │ Kubernetes集群 │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Master节点 │ │ Worker节点1 │ │ Worker节点2 │ │ │ │ │ │ │ │ │ │ │ │ API Server │ │ vLLM Pod 1 │ │ vLLM Pod 2 │ │ │ │ Scheduler │ │ (GPU 0,1) │ │ (GPU 2,3) │ │ │ │ Controller │ │ │ │ │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ └─────────────────────────────────────────────────────────────┘2. 环境准备与依赖安装2.1 Kubernetes集群要求硬件要求Master节点至少2 CPU4GB内存Worker节点至少4 CPU16GB内存1-2张NVIDIA GPU网络节点间万兆网络推荐软件要求Kubernetes 1.24NVIDIA GPU OperatorContainer Runtime (containerd或Docker)kubectl命令行工具2.2 NVIDIA GPU Operator安装# 添加NVIDIA Helm仓库helm repoaddnvidia https://nvidia.github.io/gpu-operator helm repo update# 安装GPU Operatorhelminstallgpu-operator nvidia/gpu-operator\--namespacegpu-operator\--create-namespace\--setdriver.enabledtrue\--settoolkit.enabledtrue\--setdevicePlugin.enabledtrue2.3 验证GPU资源# 检查GPU节点kubectl get nodes-lgputrue# 查看GPU资源kubectl describenodeworker-node-name|grep-igpu# 验证NVIDIA设备插件kubectl get pods-ngpu-operator3. vLLM容器镜像构建3.1 基础DockerfileFROM nvidia/cuda:11.8-devel-ubuntu20.04 # 设置环境变量 ENV DEBIAN_FRONTENDnoninteractive ENV PYTHONUNBUFFERED1 # 安装系统依赖 RUN apt-get update apt-get install -y \ python3.9 \ python3.9-pip \ python3.9-dev \ git \ wget \ rm -rf /var/lib/apt/lists/* # 创建软链接 RUN ln -s /usr/bin/python3.9 /usr/bin/python \ ln -s /usr/bin/pip3 /usr/bin/pip # 升级pip RUN pip install --upgrade pip # 安装vLLM RUN pip install vllm0.2.5 torch2.0.1 # 创建应用目录 WORKDIR /app # 复制启动脚本 COPY start_vllm.sh /app/ RUN chmod x /app/start_vllm.sh # 暴露端口 EXPOSE 8000 # 启动命令 CMD [/app/start_vllm.sh]3.2 启动脚本 (start_vllm.sh)#!/bin/bash# 设置模型路径MODEL_PATH${MODEL_PATH:-meta-llama/Llama-2-7b-chat-hf}# 设置GPU内存使用率GPU_MEMORY_UTILIZATION${GPU_MEMORY_UTILIZATION:-0.9}# 设置张量并行大小TENSOR_PARALLEL_SIZE${TENSOR_PARALLEL_SIZE:-1}# 启动vLLM API服务器python-mvllm.entrypoints.api_server\--model${MODEL_PATH}\--host0.0.0.0\--port8000\--gpu-memory-utilization${GPU_MEMORY_UTILIZATION}\--tensor-parallel-size${TENSOR_PARALLEL_SIZE}\--max-num-batched-tokens8192\--max-num-seqs2563.3 构建和推送镜像# 构建镜像dockerbuild-tyour-registry/vllm-server:latest.# 推送到镜像仓库dockerpush your-registry/vllm-server:latest4. Kubernetes资源配置4.1 Namespace创建# namespace.yamlapiVersion:v1kind:Namespacemetadata:name:vllm4.2 ConfigMap配置# configmap.yamlapiVersion:v1kind:ConfigMapmetadata:name:vllm-confignamespace:vllmdata:MODEL_PATH:meta-llama/Llama-2-7b-chat-hfGPU_MEMORY_UTILIZATION:0.85TENSOR_PARALLEL_SIZE:1MAX_NUM_BATCHED_TOKENS:8192MAX_NUM_SEQS:2564.3 Secret配置用于模型访问# secret.yamlapiVersion:v1kind:Secretmetadata:name:huggingface-secretnamespace:vllmtype:Opaquedata:# echo -n your-huggingface-token | base64HF_TOKEN:eW91ci1odWdnaW5nZmFjZS10b2tlbg4.4 PVC配置用于模型缓存# pvc.yamlapiVersion:v1kind:PersistentVolumeClaimmetadata:name:vllm-model-cachenamespace:vllmspec:accessModes:-ReadWriteOnceresources:requests:storage:100GistorageClassName:fast-ssd4.5 Deployment配置# deployment.yamlapiVersion:apps/v1kind:Deploymentmetadata:name:vllm-deploymentnamespace:vllmspec:replicas:2selector:matchLabels:app:vllmtemplate:metadata:labels:app:vllmspec:nodeSelector:gpu:truecontainers:-name:vllm-containerimage:your-registry/vllm-server:latestimagePullPolicy:Alwaysports:-containerPort:8000env:-name:MODEL_PATHvalueFrom:configMapKeyRef:name:vllm-configkey:MODEL_PATH-name:GPU_MEMORY_UTILIZATIONvalueFrom:configMapKeyRef:name:vllm-configkey:GPU_MEMORY_UTILIZATION-name:TENSOR_PARALLEL_SIZEvalueFrom:configMapKeyRef:name:vllm-configkey:TENSOR_PARALLEL_SIZE-name:HF_TOKENvalueFrom:secretKeyRef:name:huggingface-secretkey:HF_TOKENresources:requests:nvidia.com/gpu:1memory:16Gicpu:4limits:nvidia.com/gpu:1memory:32Gicpu:8volumeMounts:-name:model-cachemountPath:/root/.cache/huggingfacereadinessProbe:httpGet:path:/healthport:8000initialDelaySeconds:60periodSeconds:10livenessProbe:httpGet:path:/healthport:8000initialDelaySeconds:120periodSeconds:30volumes:-name:model-cachepersistentVolumeClaim:claimName:vllm-model-cachetolerations:-key:nvidia.com/gpuoperator:Existseffect:NoSchedule4.6 Service配置# service.yamlapiVersion:v1kind:Servicemetadata:name:vllm-servicenamespace:vllmspec:selector:app:vllmports:-port:80targetPort:8000protocol:TCPtype:ClusterIP4.7 Ingress配置# ingress.yamlapiVersion:networking.k8s.io/v1kind:Ingressmetadata:name:vllm-ingressnamespace:vllmannotations:nginx.ingress.kubernetes.io/rewrite-target:/nginx.ingress.kubernetes.io/proxy-body-size:100mspec:rules:-host:vllm.example.comhttp:paths:-path:/pathType:Prefixbackend:service:name:vllm-serviceport:number:805. 部署与管理5.1 部署vLLM服务# 创建命名空间kubectl apply-fnamespace.yaml# 应用配置kubectl apply-fconfigmap.yaml kubectl apply-fsecret.yaml kubectl apply-fpvc.yaml# 部署应用kubectl apply-fdeployment.yaml kubectl apply-fservice.yaml kubectl apply-fingress.yaml# 检查部署状态kubectl get pods-nvllm kubectl get services-nvllm5.2 扩缩容操作手动扩缩容# 扩容到3个副本kubectl scale deployment vllm-deployment--replicas3-nvllm# 缩容到1个副本kubectl scale deployment vllm-deployment--replicas1-nvllm自动扩缩容配置# hpa.yamlapiVersion:autoscaling/v2kind:HorizontalPodAutoscalermetadata:name:vllm-hpanamespace:vllmspec:scaleTargetRef:apiVersion:apps/v1kind:Deploymentname:vllm-deploymentminReplicas:1maxReplicas:10metrics:-type:Resourceresource:name:cputarget:type:UtilizationaverageUtilization:70-type:Resourceresource:name:memorytarget:type:UtilizationaverageUtilization:805.3 滚动更新# 更新镜像版本kubectlsetimage deployment/vllm-deployment\vllm-containeryour-registry/vllm-server:v2.0-nvllm# 查看更新状态kubectl rollout status deployment/vllm-deployment-nvllm# 回滚到上一版本kubectl rollout undo deployment/vllm-deployment-nvllm6. 监控与日志6.1 Prometheus监控配置# servicemonitor.yamlapiVersion:monitoring.coreos.com/v1kind:ServiceMonitormetadata:name:vllm-monitornamespace:vllmspec:selector:matchLabels:app:vllmendpoints:-port:httppath:/metricsinterval:30s6.2 日志收集# fluentd-config.yamlapiVersion:v1kind:ConfigMapmetadata:name:fluentd-confignamespace:vllmdata:fluent.conf:|source type tail path /var/log/containers/*vllm*.log pos_file /var/log/fluentd-vllm.log.pos tag kubernetes.* format json /sourcematch kubernetes.**type elasticsearch host elasticsearch.logging.svc.cluster.local port 9200 index_name vllm-logs /match6.3 健康检查端点# 在vLLM容器中添加健康检查端点fromflaskimportFlask,jsonifyimportpsutil appFlask(__name__)app.route(/health)defhealth_check():健康检查端点try:# 检查GPU状态importtorch gpu_availabletorch.cuda.is_available()gpu_memory_usedtorch.cuda.memory_allocated()/1024**3ifgpu_availableelse0# 检查系统资源cpu_percentpsutil.cpu_percent()memory_percentpsutil.virtual_memory().percentreturnjsonify({status:healthy,gpu_available:gpu_available,gpu_memory_used_gb:gpu_memory_used,cpu_percent:cpu_percent,memory_percent:memory_percent})exceptExceptionase:returnjsonify({status:unhealthy,error:str(e)}),500if__name____main__:app.run(host0.0.0.0,port8001)7. 故障排查7.1 常见问题GPU资源不足# 检查GPU资源分配kubectl describe podpod-name-nvllm# 检查GPU节点资源kubectltopnodes kubectl describenodenode-name模型加载失败# 查看Pod日志kubectl logspod-name-nvllm# 进入容器调试kubectlexec-itpod-name-nvllm -- /bin/bash网络连接问题# 测试服务连通性kubectl run test-pod--imagebusybox--rm-it-- /bin/sh# 在测试Pod中执行wget-qO- http://vllm-service.vllm.svc.cluster.local/health7.2 性能调优资源限制优化resources:requests:nvidia.com/gpu:1memory:24Gi# 根据模型大小调整cpu:6limits:nvidia.com/gpu:1memory:48Gi# 避免OOMcpu:12调度策略优化# 使用节点亲和性spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:-matchExpressions:-key:gpu-typeoperator:Invalues:[A100,V100]8. 最佳实践8.1 安全配置镜像安全使用最小化基础镜像网络策略限制Pod间通信RBAC最小权限原则密钥管理使用Kubernetes Secret管理敏感信息8.2 成本优化资源配额合理设置资源请求和限制节点池使用GPU专用节点池自动扩缩容根据负载动态调整实例数Spot实例非关键服务使用Spot实例8.3 运维建议版本管理使用Git管理Kubernetes配置渐进式部署使用金丝雀发布策略备份策略定期备份重要配置和数据文档维护保持部署文档的及时更新通过以上配置和实践可以在Kubernetes环境中成功部署和管理vLLM大语言模型服务实现高可用、可扩展的模型推理能力。