ModSecurity memory leak

[vncloudsco]

Dear: dev team
I have to use modsec with nginx (nginx version 1.16.1) However, it takes about 2 weeks for us to run out of memory, the system did not self-release the ram. How can I fix it?

lsof command show

[zimmerle]

Hi @vncloudsco,

What is your ModSecurity version?

[vncloudsco]

Dear: @zimmerle
I am using modsecurity v3 and nginx 1.16.1

I used the build method of @ejhayes

#117 (comment)

I cannot use the option --with-compat

[vncloudsco]

We run the system for about 2 weeks as it runs out of memory.

[zimmerle]

@vncloudsco is that version 3.0.4? are you happens to perform webserver reloads? Did you ever observed if the leaks are related to the reload?

[martinhsv]

Hi @vncloudsco ,

There are some known memory leaks that have already been fixed in 3.1 development code, but are not yet in v3/master or in an official release.

Per @zimmerle's question if you're on 3.0.4, please mention whether you mean the official release (from Jan. 10, 2020) or if you mean current v3/master.

[Michal256]

Similar problem here. Problem occurs with v1.0.1 connector, modsecurity version 3.0.4 on nginx 1.16.1.
Whenever the nginx server has a modsecurity module attached in configuration file for ex.

server {
    listen       8080;
    server_name  localhost;

    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/rules.conf;
}

and nginx -s reload command was run, x (M) * number of nginx servers (with modsecurity module attached) is the amount of memory that nginx is taking permanently. Without connecting the module to any of nginx servers, the problem does not occur.
Systemctl restart nginx command makes memory available again.

[zimmerle]

Similar problem here. Problem occurs with v1.0.1 connector, modsecurity version 3.0.4 on nginx 1.16.1.

Running any particular ruleset? custom rules?

[Michal256]

Yes I got some custom ruleset and I'm basing on owasp-modsecurity-crs-3.2.0. I also just checked that after disabling the custom ruleset, problem still seems to exist.

[vncloudsco]

@martinhsv My problem is the same with @Michal256 . When start running, the system has no problem. But after a period of use, the memory is overflowed specifically after 2 weeks of operation. I have tested it with many rule (comodo modsecurity and owasp-modsecurity-crs-3.2.0 ) sets but to no avail.

[martinhsv]

If your memory leaks are not associated with rule reload, In v3.0.4 I am aware of the following less-used ModSecurity features that will leak memory:

setenv (action)
validateDTD (operator)
validateSchema (operator)

To my knowledge CRS does not use any of these, but depending on which rule sets you are using you
could try checking for them.

(All three of the above have been corrected in 3.1-experimental code -- i.e. not yet available in v3/master)

It may well be, however, that you are experiencing something not previously identified.

[vncloudsco]

Dear: @martinhsv

Something is wrong, let me ask how can I build the library?
I run make command on modsec v3.1-experimental but got an error.

/bin/sh ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I.  -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL       -DPCRE_HAVE_JIT    -I/usr/include/libxml2 -DWITH_LIBXML2    -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -o parser/libmodsecurity_la-seclang-parser.lo `test -f 'parser/seclang-parser.cc' || echo './'`parser/seclang-parser.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL -DPCRE_HAVE_JIT -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c parser/seclang-parser.cc  -fPIC -DPIC -o parser/.libs/libmodsecurity_la-seclang-parser.o
In file included from ../headers/modsecurity/anchored_set_variable.h:31:0,
                 from ../headers/modsecurity/transaction.h:44,
                 from ../headers/modsecurity/modsecurity.h:188,
                 from ../src/rule_unconditional.h:28,
                 from seclang-parser.yy:28,
                 from parser/seclang-parser.cc:41:
../headers/modsecurity/string_view.hpp:443:38: error: redeclaration 'bpstd::basic_string_view<CharT, Traits>::npos' differs in 'constexpr'
     basic_string_view<CharT,Traits>::npos;
                                      ^
../headers/modsecurity/string_view.hpp:85:32: error: from previous declaration 'bpstd::basic_string_view<CharT, Traits>::npos'
     static constexpr size_type npos = size_type(-1);
                                ^
../headers/modsecurity/string_view.hpp:443:38: error: declaration of 'constexpr const size_type bpstd::basic_string_view<CharT, Traits>::npos' outside of class is not definition [-fpermissive]
     basic_string_view<CharT,Traits>::npos;
                                      ^
make[3]: *** [parser/libmodsecurity_la-seclang-parser.lo] Error 1
make[3]: Leaving directory `/opt/ModSecurity/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/opt/ModSecurity/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/ModSecurity/src'
make: *** [all-recursive] Error 1

[martinhsv]

Hi @vncloudsco ,

I have not seen that particular compile error.

One thing to pay attention to is the output of the ./configure step. The output may tell you about some critical dependency that is missing (although in this case I suspect that it won't).

One possibility is that what you are seeing is related to compiler version.

What is your compiler and version? (And what O/S and distribution are you on?)

If you are using gcc and the versioning seems to match, you may be hitting this compiler bug:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58541

Which is asserted to have been fixed in 7.1.0.

If that all seems to line up, you could try upgrading your compiler version.

[vncloudsco]

@martinhsv
this configure setup

my os:

[root@nginx ModSecurity]# cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

I cannot compile it. though can run in v3.0 version with no problems

[martinhsv]

Hi @vncloudsco ,

Did you try checking your compiler version? E.g.

gcc --version