nginx模块
nginx作为项目的7层代理入口,对于http请求的过滤,如sql注入,xss攻击等过滤功能较弱,研究了下开源的一些waf,完全开源的https://github.com/xsec-lab/x-waf,利用lua来过滤请求,同时拥有一个管理控制台,添加规则和代理,但使用过程中,缺少文档,碰到一些问题,需要调试和增强,促使用window编译通过源代码lua和使用x-waf。
lua-nginx-module模块
ngx_ http_lua_ module-将 Lua 的强大功能嵌入到 Nginx HTTP 服务器中。这个模块是 OpenResty 的核心组件,如果您正在使用这个模块,那么您实际上就是在使用 OpenResty。这个模块没有随 Nginx 源代码一起发布。这是 OpenResty 的一个核心组件,如果你正在使用这个模块,那么你实际上是在使用 OpenResty:)
x-waf
X-WAF是一款适用中、小企业的云WAF系统,让中、小企业也可以非常方便地拥有自己的免费云WAF。
 项目已经5年未更新,可以作为研究目的,可以通过安全攻击过程分析,自行拓展规则。
 文档地址:https://github.com/xsec-lab/x-waf
主要特性
- 支持对常见WEB攻击的防御,如sql注入、xss、路径穿越,阻断扫描器的扫描等
- 对持对CC攻击的防御
- waf为反向模式,后端保护的服务器可直接用内网IP,不需暴露在公网中
- 支持IP、URL、Referer、User-Agent、Get、Post、Cookies参数型的防御策略
- 安装、部署与维护非常简单
- 支持在线管理waf规则
- 支持在线管理后端服务器
- 多台waf的配置可自动同步
- 跨平台,支持在linux、unix、mac和windows操作系统中部署
x-waf安装
linux安装
安装openresty
下载对应系统的安装包
 http://openresty.org/cn/linux-packages.html
 debian系统安装参考:http://openresty.org/cn/linux-packages.html#debian
 或者通过源码安装参考:
 http://openresty.org/cn/installation.html
 安装完假设目录为:
 –prefix=/usr/local/openresty
安装x-waf
克隆x-waf到/usr/local/openresty/nginx/conf目录
cd /usr/local/openresty/nginx/conf && git clone https://github.com/xsec-lab/x-waf
下载下来是一个x-waf 的文件夹,里面的 nginx_conf/nginx.conf 文件是一个配置好了lua和x-waf规则目录的模板文件,可以拷贝到 /usr/local/openresty/nginx/conf/nginx.conf 直接覆盖原有openresty的配置文件
cp /usr/local/openresty/nginx/conf/x-waf/nginx_conf/nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
作者的代码里面url白名单部分有个错误,需要修正一下,否则会因为找不到url白名单文件,而导致白名单失效
vi /usr/local/openresty/nginx/conf/x-waf/waf.lua
找到 writeurl.rule,替换为 whiteUrl.rule
 
 建立虚拟主机配置文件目录(这里主要是nginx.conf里include了,管理后台生成的代理文件目录)
mkdir -p /usr/local/openresty/nginx/conf/vhosts
修改配置文件
vi /usr/local/openresty/nginx/conf/x-waf/config.lua
local _M = {
    -- 开启WAF
    config_waf_enable = "on",
    -- WAF防护日志目录,需要保证openresty的运行用户对该目录有访问权限,可通过修改目录的所有者为openresty的运行用户来实现,或者修改权限为777,注意执行:chmod 777 /opt/waf
    config_log_dir = "/opt/waf",
    -- rule setting
    config_rule_dir = "/usr/local/openresty/nginx/conf/x-waf/rules",
    -- 启用网址白名单过滤
    config_white_url_check = "on",
    -- 启用IP白名单过滤
    config_white_ip_check = "on",
    -- 启用IP黑名单过滤
    config_black_ip_check = "on",
    -- 启用url过滤
    config_url_check = "on",
    -- 启用url参数过滤
    config_url_args_check = "on",
    -- 启用浏览器用户代理过滤
    config_user_agent_check = "on",
    -- 启用cookie过滤
    config_cookie_check = "on",
    -- 启用CC攻击检测
    config_cc_check = "on",
    -- CC攻击检测阈值,10次/60秒
    config_cc_rate = "10/60",
    -- enable/disable post filtering
    config_post_check = "on",
    -- 检测攻击后给攻击者的输出,默认html文本串,通过config_output_html配置,或者设置为url,则通过config_waf_redirect_url配置
    config_waf_model = "html",
    -- if config_waf_output ,setting url
    config_waf_redirect_url = "http://xxx.com",
    config_expire_time = 600,
    config_output_html = [[
    <html>
    <head>
    <meta charset="UTF-8">
    <title>非法访问</title>
    </head>
      <body>
        <div>
      <div class="table">
        <div>
          <div class="cell">
            非法访问,您的IP为: %s
          </div>
          <div class="cell">
            如需帮助请联系客服
          </div>
        </div>
      </div>
    </div>
      </body>
    </html>
    ]],
}
安装x-waf-admin
管理后台安装,管理后台使用GO语言编写,可以直接下载编译好的版本直接运行即可
 https://github.com/xsec-lab/x-waf-admin/releases/download/x-waf-admin0.1/x-waf-admin0.1-linux-amd64.tar.gz
解压
tar -xzf x-waf-admin0.1-linux-amd64.tar.gz
编辑配置文件
vi x-waf-admin/conf/app.ini
RUN_MODE = dev #开发环境使用
;RUN_MODE = prod #生产环境使用
[server]
HTTP_PORT = 5000 #管理后台端口
API_KEY = xsec.io||secdevops.cn
NGINX_BIN = /usr/local/openresty/nginx/sbin/nginx #指定openresty可执行文件位置
NGINX_VHOSTS = /usr/local/openresty/nginx/conf/vhosts/ #指定虚拟主机配置文件位置
API_SERVERS = 127.0.0.1, 你自己的IP  #指定管理后台的IP地址,加上你自己的服务器IP即可
[database]
USER = 数据库用户名
PASSWD = 数据库密码
HOST = 127.0.0.1:3306 #数据库地址和端口  ,数据库用来保存用户和自定义规则信息
NAME = waf #数据库名
[waf]
RULE_PATH = /usr/local/openresty/nginx/conf/x-waf/rules/
已后台进程方式启动管理后台,启动后会自动往MySQL数据库写入配置表,如果没有配置好MySQL,管理后台会因为找不到数据库里面的用户而无法登录
nohup ./server >> x-waf.log 2>&1 &
tail -f x-waf.log 查看运行日志和启动启动
然后就可以访问管理后台 http://ip:5000/login/ 了,
 默认的管理后台用户是admin,密码是 x@xsec.io,生产环境一点要修改账户密码
测试默认规则是否拦截(select.+(from|limit) )
 http://10.10.0.117/?id=select * from dual
 
在管理后台新增规则,比如参数数有helloworld直接拦截
 
 http://10.10.0.117/?id=hellowold发现拦击
开发环境安装
nginx源码编译
nginx源码在clion编译参考:nginx编译
 请确保按照上面的编译通过nginx能正常运行,这里假设
 我的window代码安装在:D:\code1\nginx-master
 cygwin显示的路径为:/cygdrive/d/code1/nginx-master
安装luajit
官网:http://luajit.org/install.html
 下载openresty最新2.1版本:不要下载luajit官网的,否则报错http://luajit.org/download.html
git clone https://github.com/openresty/luajit2
Cygwin64 Terminal进入解压目录,编译
 cd luajit2 && make install PREFIX=/usr/local/LuaJIT 
编译过程注意 cygwin下编译出的是dll,实际命令中 还是执行拷贝的是so文件
$ make install PREFIX=/usr/local/LuaJIT
==== Installing LuaJIT 2.1.0-beta3 to /usr/local/LuaJIT ====
mkdir -p /usr/local/LuaJIT/bin /usr/local/LuaJIT/lib /usr/local/LuaJIT/include/luajit-2.1 /usr/local/LuaJIT/share/man/man1 /usr/local/LuaJIT/lib/pkgconfig /usr/local/LuaJIT/share/luajit-2.1.0-beta3/jit /usr/local/LuaJIT/share/lua/5.1 /usr/local/LuaJIT/lib/lua/5.1
cd src && install -m 0755 luajit /usr/local/LuaJIT/bin/luajit-2.1.0-beta3
cd src && test -f libluajit.a && install -m 0644 libluajit.a /usr/local/LuaJIT/lib/libluajit-5.1.a || :
rm -f /usr/local/LuaJIT/lib/libluajit-5.1.so.2.1.0 /usr/local/LuaJIT/lib/libluajit-5.1.so /usr/local/LuaJIT/lib/libluajit-5.1.so.2
cd src && test -f libluajit.so && \
  install -m 0755 libluajit.so /usr/local/LuaJIT/lib/libluajit-5.1.so.2.1.0 && \
  ldconfig -n /usr/local/LuaJIT/lib && \
  ln -sf libluajit-5.1.so.2.1.0 /usr/local/LuaJIT/lib/libluajit-5.1.so && \
  ln -sf libluajit-5.1.so.2.1.0 /usr/local/LuaJIT/lib/libluajit-5.1.so.2 || :
cd etc && install -m 0644 luajit.1 /usr/local/LuaJIT/share/man/man1
cd etc && sed -e "s|^prefix=.*|prefix=/usr/local/LuaJIT|" -e "s|^multilib=.*|multilib=lib|" luajit.pc > luajit.pc.tmp && \
  install -m 0644 luajit.pc.tmp /usr/local/LuaJIT/lib/pkgconfig/luajit.pc && \
  rm -f luajit.pc.tmp
cd src && install -m 0644 lua.h lualib.h lauxlib.h luaconf.h lua.hpp luajit.h /usr/local/LuaJIT/include/luajit-2.1
cd src/jit && install -m 0644 bc.lua bcsave.lua dump.lua p.lua v.lua zone.lua dis_x86.lua dis_x64.lua dis_arm.lua dis_arm64.lua dis_arm64be.lua dis_ppc.lua dis_mips.lua dis_mipsel.lua dis_mips64.lua dis_mips64el.lua vmdef.lua /usr/local/LuaJIT/share/luajit-2.1.0-beta3/jit
==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local/LuaJIT ====
看到这句
 rm -f /usr/local/LuaJIT/lib/libluajit-5.1.so.2.1.0 /usr/local/LuaJIT/lib/libluajit-5.1.so /usr/local/LuaJIT/lib/libluajit-5.1.so.2
 最后拷贝lib文件到/usr/local/LuaJIT/lib/libluajit-5.1.so ,但实际src下生成的是cyglua51.dll
cp src/cyglua51.dll /usr/local/LuaJIT/lib/libluajit-5.1.so
cp src/cyglua51.dll /usr/local/LuaJIT/lib/libluajit-5.1.a
/etc/profile添加两个环境变量
export LUAJIT_LIB=/usr/local/LuaJIT/lib
export LUAJIT_INC=/usr/local/LuaJIT/include/luajit-2.1
生效:source /etc/profile
添加lua-nginx-module
下载ngx_devel_kit源代码到nginx源代码extend目录
 https://github.com/simplresty/ngx_devel_kit/tags
 下载lua-nginx-module源代码到ngxin的extend目录
 https://github.com/openresty/lua-nginx-module/tags
 
nginx下执行
./auto/configure --with-ld-opt="-Wl,-rpath,/usr/local/LuaJIT/lib" --add-module=./extend/ngx_devel_kit-0.3.0 --add-module=./extend/lua-nginx-module-0.10.22
生成CmakeLists.txt中添加Makefile中的編譯參數
 
set(CMAKE_C_FLAGS "-I/usr/local/LuaJIT/include/luajit-2.1  -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g  -DNDK_SET_VAR")

 clion运行看不到错误,编译通过后在cygwin运行,nginx运行
liaomin@DESKTOP-FSEDE3P /cygdrive/d/code1/nginx-master/objs
$ ./nginx.exe -c /cygdrive/d/code1/nginx-master/conf/nginx.conf
D:/code1/nginx-master/objs/nginx.exe: error while loading shared libraries: cyglua51.dll: cannot open shared object file: No such file or directory
拷贝luajit/src目录下cyglua51.dll到nginx源代码objs目录,如果在idea中需要拷贝到cmake-build-debug中,继续运行
liaomin@DESKTOP-FSEDE3P /cygdrive/d/code1/nginx-master/objs
$ ./nginx.exe -c /cygdrive/d/code1/nginx-master/conf/nginx.conf
nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html)
nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: module 'resty.core' not found:
        no field package.preload['resty.core']
        no file './resty/core.lua'
        no file '/usr/local/share/luajit-2.1.0-beta3/resty/core.lua'
        no file '/usr/local/share/lua/5.1/resty/core.lua'
        no file '/usr/local/share/lua/5.1/resty/core/init.lua'
        no file './resty/core.so'
        no file '/usr/local/lib/lua/5.1/resty/core.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file './resty.so'
        no file '/usr/local/lib/lua/5.1/resty.so'
        no file '/usr/local/lib/lua/5.1/loadall.so') in /cygdrive/d/code1/nginx-master/conf/nginx.conf:118
说明运行需要resty.core,继续参考https://github.com/openresty/lua-nginx-module#installation
 
 lua-resty-core和lua-resty-lrucache其实可以理解为lua-nginx-module调用初始化的一些lua脚本,负责初始化ngx变化调用nginx。
 默认安装的目录:
安装lua-resty-core
安装lua-resty-core,下载源码后,默认安装在/usr/local/lib/下
liaomin@DESKTOP-FSEDE3P /cygdrive/d/test/lua-resty-core-master
$ make install
install -d /usr/local/lib/lua//resty/core/
install -d /usr/local/lib/lua//ngx/
install -d /usr/local/lib/lua//ngx/ssl
install lib/resty/*.lua /usr/local/lib/lua//resty/
install lib/resty/core/*.lua /usr/local/lib/lua//resty/core/
install lib/ngx/*.lua /usr/local/lib/lua//ngx/
install lib/ngx/ssl/*.lua /usr/local/lib/lua//ngx/ssl/
安装lua-resty-lrucache
安装lua-resty-lrucache,下载源码后,默认安装在/usr/local/lib/下
$ make install PREFIX=usr/local   //这里注意前面别加/否则会多个/路径会报错
install -d /usr/local/lib/lua//resty/lrucache
install lib/resty/*.lua /usr/local/lib/lua//resty/
install lib/resty/lrucache/*.lua /usr/local/lib/lua//resty/lrucache/
解决初始化脚本兼容异常
lua脚本中某些调用linux系统的变量在window中不存在,导致异常,需要做处理。
 再次启动nginx,报错
D:\code1\nginx-master\cmake-build-debug\nginx.exe -c /cygdrive/d/code1/nginx-master/conf/nginx.conf
nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: module 'resty.core' not found:
	no field package.preload['resty.core']
	no file './resty/core.lua'
	no file '/usr/local/LuaJIT/share/luajit-2.1.0-beta3/resty/core.lua'
	no file '/usr/local/share/lua/5.1/resty/core.lua'
	no file '/usr/local/share/lua/5.1/resty/core/init.lua'
	no file '/usr/local/LuaJIT/share/lua/5.1/resty/core.lua'
	no file '/usr/local/LuaJIT/share/lua/5.1/resty/core/init.lua'
	no file './resty/core.so'
	no file '/usr/local/lib/lua/5.1/resty/core.so'
	no file '/usr/local/LuaJIT/lib/lua/5.1/resty/core.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file './resty.so'
	no file '/usr/local/lib/lua/5.1/resty.so'
	no file '/usr/local/LuaJIT/lib/lua/5.1/resty.so'
	no file '/usr/local/lib/lua/5.1/loadall.so') in /cygdrive/d/code1/nginx-master/conf/nginx.conf:125
说是没有安装resty.core,但是已经安装了,需要在nginx.conf配置初始化lua脚本,http快中添加:
   lua_package_path "/usr/local/lib/lua/?.lua;;";
继续启动,然后又报错
D:\code1\nginx-master\cmake-build-debug\nginx.exe -c /cygdrive/d/code1/nginx-master/conf/nginx.conf
nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: /usr/local/lib/lua/resty/core/worker.lua:35: No such file or directory) in /cygdrive/d/code1/nginx-master/conf/nginx.conf:125
worker.lua:35报错,注释掉第35行。
    ngx_lua_ffi_worker_id = C.ngx_http_lua_ffi_worker_id
    ngx_lua_ffi_worker_pid = C.ngx_http_lua_ffi_worker_pid
    --ngx_lua_ffi_worker_pids = C.ngx_http_lua_ffi_worker_pids
    ngx_lua_ffi_worker_count = C.ngx_http_lua_ffi_worker_count
    ngx_lua_ffi_worker_exiting = C.ngx_http_lua_ffi_worker_exiting
继续运行,报错
D:\code1\nginx-master\cmake-build-debug\nginx.exe -c /cygdrive/d/code1/nginx-master/conf/nginx.conf
nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: /usr/local/lib/lua/resty/core/time.lua:47: No such file or directory) in /cygdrive/d/code1/nginx-master/conf/nginx.conf:125
注释掉time.lua:47
修改配置后,可以正常启动,添加测试lua脚本
location /hello {
           default_type 'text/plain';
           content_by_lua 'ngx.say("hello, lua")';
           charset utf-8;
        }
访问http://localhost/hello显示hello,lua
x-waf源码安装
将x-waf源码拷贝到conf目录下,将config.lua的config_rule_dir配置为绝对路径
 
 拷贝x-waf下修改nginx_conf/nginx.conf到项目conf/nginx中,将x-waf lua加入到lua_package_path
#user  nobody;
worker_processes  1;
error_log  /cygdrive/d/code1/nginx-master/objs/logs/error.log;
#error_log  logs/error.log  notice;
error_log  /cygdrive/d/code1/nginx-master/objs/logs/info.log  info;
#pid        logs/nginx.pid;
events {
    worker_connections  24;
}
daemon off;
master_process off;
http {
    include       mime.types;
    default_type  application/octet-stream;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    lua_package_path "/cygdrive/d/code1/nginx-master/conf/x-waf/?.lua;/usr/local/lib/lua/?.lua;;";
    #gzip  on;
    init_by_lua_file /cygdrive/d/code1/nginx-master/conf/x-waf/init.lua;
    access_by_lua_file /cygdrive/d/code1/nginx-master/conf/x-waf/access.lua;
    server {
        listen       80;
        server_name  localhost;
        #charset koi8-r;
        #access_log  logs/host.access.log  main;
        location / {
            proxy_pass http://192.168.1.35:8888/;    #这里随便代理到一个可用的web站点,http://www.baidu.com也行
            #root   html;
            #index  index.html index.htm;
        }
        location /hello {
           default_type 'text/plain';
           content_by_lua 'ngx.say("hello, lua")';
           charset utf-8;
        }
        #error_page  404              /404.html;
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}
注意因为window不支持共享内存,cc_attack_check这个是使用ngx.shared.limit,注释掉waf.lua _M.check()函数中_M.cc_attack_check()
 
lua-cjson安装
配置好后,访问nginx直接错误,缺少cjson.safe模块,需要安装
 cjson官网:https://github.com/mpx/lua-cjson/tags ,下载最新的2.1.0 tags
 修改makefile
#CJSON_LDFLAGS =     -shared   指定luajit的lib目录
CJSON_LDFLAGS =     -shared -L/usr/local/LuaJIT/lib -llua51
#LUA_INCLUDE_DIR =   $(PREFIX)/include 指定头文件目录
LUA_INCLUDE_DIR =   /usr/local/LuaJIT/include/luajit-2.1
修改完成后依然有个错误
make PREFIX=/usr/local/LuaJIT/
cc -c -O3 -Wall -pedantic -DNDEBUG  -I/usr/local/include -I/usr/local/LuaJIT/include/luajit-2.1 -fpic -o lua_cjson.o lua_cjson.c
lua_cjson.c:1299:1: 错误:对‘luaL_setfuncs’的静态声明出现在非静态声明之后
 1299 | {
修改lua_cjson.c
 找到行数luaL_setfuncs 去掉static 编译通过
make PREFIX=/usr/local/LuaJIT/ install
验证
访问:http://localhost/?id=select * from dual
 



















