Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve CPU cores affinity to NUMA nodes v8 #12336

Closed
wants to merge 10 commits into from
Closed

improve CPU cores affinity to NUMA nodes v8 #12336

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6b05390
runmodes: query the active runmode with a function call
Dec 8, 2024
b2ee7e2
dpdk: move DPDK socket retrieval to utils
Dec 6, 2024
ffc2c4c
github-ci: install hwloc as a mandatory dependency
Sep 3, 2024
75aba5f
actions: test hwloc build
Dec 19, 2024
9c1c2e4
threading: refactor CPU affinity code
Jan 2, 2025
c4679b6
util-affinity: move properties of *-cpu-set node one layer up in YAML
Dec 6, 2024
1db10cd
threading: transform *-cpu-set nodes from list items to nodes
Dec 6, 2024
6cd2d1a
threading: support previous threading configuration format
Jan 3, 2025
f11ca6f
doc: remove title in threading section with no content
Jan 3, 2025
a717c60
threading: support thread autopinning and interface-specific affinity
Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 64 additions & 1 deletion .github/workflows/builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -258,6 +260,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -353,6 +357,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -504,6 +510,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -601,6 +609,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -693,6 +703,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
hiredis-devel \
jansson-devel \
jq \
Expand Down Expand Up @@ -791,6 +803,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -889,6 +903,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -987,6 +1003,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1080,6 +1098,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1165,6 +1185,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1261,6 +1283,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
libasan \
libtool \
libyaml-devel \
Expand Down Expand Up @@ -1382,6 +1406,8 @@ jobs:
coccinelle \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
libcap-ng-dev \
libevent-dev \
Expand Down Expand Up @@ -1454,6 +1480,8 @@ jobs:
clang-14 \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -1591,6 +1619,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -1696,6 +1726,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -1835,6 +1867,8 @@ jobs:
cbindgen \
clang-18 \
git \
hwloc \
libhwloc-dev \
jq \
libc++-dev \
libc++abi-dev \
Expand Down Expand Up @@ -1922,6 +1956,8 @@ jobs:
cbindgen \
clang-18 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -2023,6 +2059,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
libc++-dev \
libc++abi-dev \
Expand Down Expand Up @@ -2122,6 +2160,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2205,6 +2245,8 @@ jobs:
apt -y install \
build-essential \
curl \
hwloc \
libhwloc-dev \
libtool \
libpcap-dev \
libnet1-dev \
Expand Down Expand Up @@ -2274,6 +2316,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2358,6 +2402,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
libtool \
libpcap-dev \
libnet1-dev \
Expand Down Expand Up @@ -2423,6 +2469,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2562,6 +2610,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2624,7 +2674,7 @@ jobs:
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-dpdk
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-dpdk --enable-hwloc
- run: make -j ${{ env.CPUS }}
- run: make check
# IDS config
Expand Down Expand Up @@ -2660,6 +2710,8 @@ jobs:
curl \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2763,6 +2815,8 @@ jobs:
cmake \
curl \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2844,6 +2898,8 @@ jobs:
curl \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2928,6 +2984,8 @@ jobs:
ccache \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libpcre2-dev \
libpcap-dev \
Expand Down Expand Up @@ -3001,6 +3059,8 @@ jobs:
ccache \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libpcre2-dev \
libpcap-dev \
Expand Down Expand Up @@ -3065,6 +3125,7 @@ jobs:
cbindgen \
curl \
hiredis \
hwloc \
jansson \
jq \
libmagic \
Expand Down Expand Up @@ -3294,6 +3355,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
libtool \
libyaml-devel \
Expand Down
28 changes: 28 additions & 0 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,33 @@
exit 1
fi

LIBHWLOC=""
AC_ARG_ENABLE(hwloc,
AS_HELP_STRING([--enable-hwloc], [Enable hwloc support [default=no]]),
[enable_hwloc=$enableval],[enable_hwloc=no])
AS_IF([test "x$enable_hwloc" = "xyes"], [
PKG_CHECK_MODULES([HWLOC], [hwloc >= 2.0.0],
[AC_DEFINE([HAVE_HWLOC], [1], [Define if hwloc library is present and meets version requirements])],
LIBHWLOC="no")

if test "$LIBHWLOC" = "no"; then
echo
echo " ERROR! hwloc library version > 2.0.0 not found, go get it"
echo " from https://www.open-mpi.org/projects/hwloc/ "
echo " or your distribution:"
echo
echo " Ubuntu: apt-get install hwloc libhwloc-dev"
echo " Fedora: dnf install hwloc hwloc-devel"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/install hwloc/install hwloc

echo " CentOS/RHEL: yum install hwloc hwloc-devel"
echo
exit 1
else
CFLAGS="${CFLAGS} ${HWLOC_CFLAGS}"
LDFLAGS="${LDFLAGS} ${HWLOC_LIBS}"
enable_hwloc="yes"
fi
])

# libpthread
AC_ARG_WITH(libpthread_includes,
[ --with-libpthread-includes=DIR libpthread include directory],
Expand Down Expand Up @@ -2561,6 +2588,7 @@ SURICATA_BUILD_CONF="Suricata Configuration:
JA4 support: ${enable_ja4}
Non-bundled htp: ${enable_non_bundled_htp}
Hyperscan support: ${enable_hyperscan}
Hwloc support: ${enable_hwloc}
Libnet support: ${enable_libnet}
liblz4 support: ${enable_liblz4}
Landlock support: ${enable_landlock}
Expand Down
12 changes: 6 additions & 6 deletions doc/userguide/capture-hardware/dpdk.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,12 @@ management and worker CPU set.
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
- receive-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
- worker-cpu-set:
cpu: [ 2,4,6,8 ]
management-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
receive-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
worker-cpu-set:
cpu: [ 2,4,6,8 ]
...

Interrupt (power-saving) mode
Expand Down
Loading
Loading