K8S中ingress-nginx通过header路由到不同后端
公司使用ingress-nginx作为网关的项目，需要在相同域名、uri，根据header将请求转发到不同的后端中 在稳定发布的情况下，ingress-nginx是没有语法直接支持根据header做转发的。但是这个可以利用灰度发布的特性实现header路由功能 package main
import "github.com/gin-gonic/gin"
func main ( ) {
r := gin. Default ( )
r. GET ( "/app" , func ( context * gin. Context) {
context. JSON ( 200 , gin. H{ "message" : "app1" } )
} )
r. Run ( ":8080" )
}
使用Dockerfile构建镜像
这里构建 goapp1:v1，goapp2:v1两个镜像（goapp2请将main.go修改 “message”: “app2”） FROM golang:1.17.13
RUN mkdir -p /go/app/; \
cd /go/app/; \
go mod init app1;\
GOPROXY="https://goproxy.cn,direct" go get github.com/gin-gonic/gin@v1.6.3
WORKDIR /go/app/
COPY main.go /go/app
EXPOSE 8080
CMD go run main.go
使用灰度发布的特性进行header的路由
此解决方案参考：https://v2-1.docs.kubesphere.io/docs/zh-CN/quick-start/ingress-canary/ 注：本人使用低版本ingress-nginx，高版本的请大家自行修改不同之处 首先部署goapp1:v1 和 goapp2:v1 的deployment和service
apiVersion : apps/v1
kind : Deployment
metadata :
name : goapp1
namespace : default
spec :
replicas : 1
selector :
matchLabels :
app : goapp1
template :
metadata :
labels :
app : goapp1
spec :
containers :
- image : goapp1: v1
imagePullPolicy : IfNotPresent
name : goapp1
ports :
- containerPort : 80
protocol : TCP
---
apiVersion : v1
kind : Service
metadata :
name : goapp1
namespace : default
spec :
ports :
- port : 8080
protocol : TCP
targetPort : 8080
selector :
app : goapp1
部署稳定发布版本的ingress，路由至goapp1 apiVersion : networking.k8s.io/v1
kind : Ingress
metadata :
name : goapp1
namespace : default
annotations :
kubernetes.io/ingress.class : nginx
spec :
rules :
- host : test.com
http :
paths :
- path : /app
pathType : Prefix
backend :
service :
name : goapp1
port :
number : 8080
部署canary版本的ingress，路由至goapp2
这里可见 域名都是 test.com，uri都是 /app 注解：
nginx.ingress.kubernetes.io/canary: “true” # 启用canary灰度发布特性 nginx.ingress.kubernetes.io/canary-by-header: canary # 通过header可选择是否转发至canary版本的后端 apiVersion : networking.k8s.io/v1
kind : Ingress
metadata :
name : goapp2
namespace : default
annotations :
kubernetes.io/ingress.class : nginx
nginx.ingress.kubernetes.io/canary : "true"
nginx.ingress.kubernetes.io/canary-by-header : canary
spec :
rules :
- host : test.com
http :
paths :
- path : /app
pathType : Prefix
backend :
service :
name : goapp2
port :
number : 8080
for i in { 1 .. 20 } ;
do curl test.com:31132/app -H "canary: never" ;
echo -e "" ;
done
for i in { 1 .. 20 } ;
do curl test.com:31132/app -H "canary: always" ;
echo -e "" ;
done
效果如下，可以看到可以通过header控制发送请求到不同后端，能够满足需求 第二种方法可通过在k8s集群部署一个nginx， 通过nginx进行分流
流量路径如下： ingress-nginx --> nginx --> goapp1或goapp2 这里nginx写法有比较多，我选择最简单的通过if判断$http_my_header 在使用$http_my_header之前，需要对ingress-nginx和nginx添加参数，允许header中存在下划线
kubectl edit cm ingress-nginx-controller
------------------
apiVersion: v1
data:
allow-snippet-annotations: "true"
enable-underscores-in-headers: "true"
ignore-invalid-headers: "false"
kind: ConfigMap
部署nginx，nginx中开启允许header下划线的参数：underscores_in_headers on; apiVersion : apps/v1
kind : Deployment
metadata :
name : nginx
namespace : default
spec :
replicas : 1
selector :
matchLabels :
app : nginx
template :
metadata :
labels :
app : nginx
spec :
containers :
- name : nginx
image : nginx: 1.24.0
ports :
- containerPort : 80
volumeMounts :
- name : nginx
mountPath : /etc/nginx/nginx.conf
subPath : nginx.conf
volumes :
- name : nginx
configMap :
name : nginx
items :
- key : nginx.conf
path : nginx.conf
---
apiVersion : v1
kind : Service
metadata :
name : nginx
namespace : default
spec :
selector :
app : nginx
ports :
- protocol : TCP
port : 80
targetPort : 80
---
apiVersion : v1
data :
nginx.conf : |
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
upstream upstream_server1 {
server goapp1: 8080;
}
upstream upstream_server2 {
server goapp2: 8080;
}
include /etc/nginx/mime.types;
default_type application/octet- stream;
log_format main '$remote_addr - $remote_user [ $time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" "$http_my_header"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
server {
underscores_in_headers on;
listen 80;
server_name test.com;
location /app {
if ($http_my_header = "value1") {
proxy_pass http: //upstream_server1;
}
if ($http_my_header = "value2") {
proxy_pass http: //upstream_server2;
}
}
}
}
kind : ConfigMap
metadata :
name : nginx
namespace : default
上面的配置判断 $http_my_header是 value1 还是 value2，再转发到不同的upstream 测试 curl test.com/app -H "my_header:value1"
curl test.com/app -H "my_header:value2"
