Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add clang sanitizers #20

Merged
merged 13 commits into from
Nov 27, 2024
38 changes: 38 additions & 0 deletions .github/workflows/clang-sanitizers-linux-nix-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Build and Test sanitizers on Linux with clang

on:
workflow_call:

jobs:
build-and-test:
name: "Build and test Linux with clang"
runs-on: [self-hosted, Linux, X64, aws_autoscaling]
continue-on-error: true
steps:
# https://github.com/actions/checkout/issues/1552
- name: Clean up after previous checkout
run: chmod +w -R ${GITHUB_WORKSPACE}; rm -rf ${GITHUB_WORKSPACE}/*;

- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Run checks
run: |
nix build -L .?#checks.x86_64-linux.all-sanitizers
mkdir results
cp result/test-logs/*_test.xml results/
continue-on-error: true
env:
NIX_CONFIG: |
cores = 4

- name: Publish Test Results
uses: EnricoMi/publish-unit-test-result-action/linux@v2
with:
check_name: "Sanitizers Test Results"
files: "results/*.xml"
comment_mode: ${{ github.event.pull_request.head.repo.fork && 'off' || 'always' }} # Don't create PR comment from fork runs
action_fail_on_inconclusive: true # fail, if no reports

45 changes: 45 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Nightly Testing

on:
schedule:
- cron: '0 0 * * *'

jobs:
test-linux-sanitizers:
name: Linux placeholder testing and sanitize with clang
uses: ./.github/workflows/clang-sanitizers-linux-nix-check.yml
if: |
always() && !cancelled()
secrets: inherit

post-telemetry:
name: Post test results in Open Telemetry format
runs-on: [self-hosted, Linux, X64, aws_autoscaling]
needs:
- test-linux-sanitizers
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Post logs to Sig Noz
run: |
ls -l -a
nix build -L .?#checks.x86_64-linux.all-sanitizers
cat ./result/test-logs/test_errors.txt
export UndefinedBehaviorSanitizer=$(grep UndefinedBehaviorSanitizer result/test-logs/test_errors.txt | wc -l)
export AddressSanitizer=$(grep AddressSanitizer result/test-logs/test_errors.txt | wc -l)
export LeakSanitizer=$(grep LeakSanitizer result/test-logs/test_errors.txt | wc -l)
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
python3 --version
sudo yum update -y
sudo yum install -y python3-pip
pip3 install -r requirements.txt
/home/ec2-user/.local/bin/opentelemetry-instrument \
--traces_exporter console,otlp \
--metrics_exporter console,otlp \
--logs_exporter console,otlp \
--service_name nightly-build \
python3 ./parse_tests.py

1 change: 0 additions & 1 deletion .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,3 @@ jobs:
if: |
always() && !cancelled()
secrets: inherit

1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -626,7 +626,6 @@ callgrind.*
/*.c
/*.cpp
/*.h
/*.py
/*.key
/*.pem
/*.der
Expand Down
6 changes: 5 additions & 1 deletion crypto3.nix
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,13 @@ in stdenv.mkDerivation {
checkPhase = ''
# JUNIT file without explicit file name is generated after the name of the master test suite inside `CMAKE_CURRENT_SOURCE_DIR`
export BOOST_TEST_LOGGER=JUNIT:HRF
cd crypto3 && ctest --verbose --output-on-failure -R && cd ..
cd crypto3
# remove || true after all tests are fixed under clang-sanitizers check:
ctest --verbose --output-on-failure -R > test_errors.txt || true
cd ..
mkdir -p ${placeholder "out"}/test-logs
find .. -type f -name '*_test.xml' -exec cp {} ${placeholder "out"}/test-logs \;
cp crypto3/test_errors.txt ${placeholder "out"}/test-logs \
'';

shellHook = ''
Expand Down
24 changes: 21 additions & 3 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,17 @@
crypto3-debug-tests = (pkgs.callPackage ./crypto3.nix {
enableDebug = true;
runTests = true;
sanitize = true;
akokoshn marked this conversation as resolved.
Show resolved Hide resolved
});
crypto3-sanitize = (pkgs.callPackage ./crypto3.nix {
enableDebug = true;
runTests = true;
enableDebug = false;
sanitize = true;
});
crypto3-clang-sanitize = (pkgs.callPackage ./crypto3.nix {
akokoshn marked this conversation as resolved.
Show resolved Hide resolved
stdenv = pkgs.llvmPackages_19.stdenv;
runTests = true;
enableDebug = false;
sanitize = true;
});
crypto3-clang-debug = (pkgs.callPackage ./crypto3.nix {
Expand Down Expand Up @@ -90,7 +97,7 @@
# fetched from the cache.
all = pkgs.symlinkJoin {
name = "all";
paths = [ crypto3 proof-producer];
paths = [ crypto3 parallel-crypto3 proof-producer];
};
default = all;
};
Expand Down Expand Up @@ -121,6 +128,11 @@
runTests = true;
enableDebug = false;
});
parallel-crypto3-clang-sanitize = (pkgs.callPackage ./parallel-crypto3.nix {
stdenv = pkgs.llvmPackages_19.stdenv;
runTests = true;
enableDebug = false;
});

proof-producer-gcc = (pkgs.callPackage ./proof-producer.nix {
runTests = true;
Expand All @@ -131,14 +143,20 @@
runTests = true;
enableDebug = false;
});
proof-producer-sanitize = (pkgs.callPackage ./proof-producer.nix {
akokoshn marked this conversation as resolved.
Show resolved Hide resolved
stdenv = pkgs.llvmPackages_19.stdenv;
runTests = true;
enableDebug = false;
sanitize = true;
});

all-clang = pkgs.symlinkJoin {
name = "all";
paths = [ crypto3-clang parallel-crypto3-clang proof-producer-clang ];
};
all-sanitizers = pkgs.symlinkJoin {
akokoshn marked this conversation as resolved.
Show resolved Hide resolved
name = "all";
paths = [ crypto3-clang-sanitize ];
paths = [ crypto3-clang-sanitize parallel-crypto3-clang-sanitize proof-producer-sanitize ];
};
all-gcc = pkgs.symlinkJoin {
name = "all";
Expand Down
10 changes: 9 additions & 1 deletion parallel-crypto3.nix
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
enableDebugging,
enableDebug ? false,
runTests ? false,
sanitize? false,
}:
let
inherit (lib) optional;
Expand All @@ -32,13 +33,20 @@ in stdenv.mkDerivation {
[
(if runTests then "-DBUILD_PARALLEL_CRYPTO3_TESTS=TRUE" else "")
(if enableDebug then "-DCMAKE_BUILD_TYPE=Debug" else "-DCMAKE_BUILD_TYPE=Release")
(if sanitize then "-DSANITIZE=ON" else "-DSANITIZE=OFF")
"-DPARALLEL_CRYPTO3_ENABLE=TRUE"
];

doCheck = runTests; # tests are inside parallel-crypto3-tests derivation

checkPhase = ''
cd parallel-crypto3 && ctest --verbose --output-on-failure -R && cd ..
# JUNIT file without explicit file name is generated after the name of the master test suite inside `CMAKE_CURRENT_SOURCE_DIR`
export BOOST_TEST_LOGGER=JUNIT:HRF
cd parallel-crypto3
ctest --verbose --output-on-failure -R
cd ..
mkdir -p ${placeholder "out"}/test-logs
find .. -type f -name '*_test.xml' -exec cp {} ${placeholder "out"}/test-logs \;
'';

shellHook = ''
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ BOOST_AUTO_TEST_CASE(vector_multiplication_test) {
}, nil::crypto3::ThreadPool::PoolLevel::HIGH));

for (std::size_t i = 0; i < size; ++i) {
BOOST_CHECK(v[i] == i * i);
if(v[i] != i * i) // If uncomment, test result parser will log an error: CData section too big found
akokoshn marked this conversation as resolved.
Show resolved Hide resolved
BOOST_CHECK(v[i] == i * i);
}
}

Expand Down
42 changes: 42 additions & 0 deletions parse_tests.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import logging, json
from junitparser import JUnitXml
import glob, os
from opentelemetry import trace

undefined_behavior_sanitizer=os.environ['UndefinedBehaviorSanitizer']
address_sanitizer=os.environ['AddressSanitizer']
leak_sanitizer=os.environ['LeakSanitizer']

aggregated_test_results = JUnitXml();
for file in glob.glob("result/test-logs/*.xml"):
try:
aggregated_test_results.append(JUnitXml.fromfile(file))
except Exception as ex:
print("Error processing {}".format(file))
print(ex)

succeeded = aggregated_test_results.tests - \
aggregated_test_results.failures - \
aggregated_test_results.errors - \
aggregated_test_results.skipped

result = {
"tests" : aggregated_test_results.tests,
"failures" : aggregated_test_results.failures,
"errors" : aggregated_test_results.errors,
"skipped" : aggregated_test_results.skipped,
"succeeded" : succeeded,
"execution_time" : aggregated_test_results.time,
"undefined_behavior_sanitizer" : int(undefined_behavior_sanitizer),
"address_sanitizer" : int(address_sanitizer),
"leak_sanitizer" : int(leak_sanitizer),
}

print("Resulting JSON: {}".format(json.dumps(result)))

tracer = trace.get_tracer_provider().get_tracer(__name__)
with tracer.start_as_current_span("nightly_span"):
current_span = trace.get_current_span()
current_span.add_event("Nightly build finished")
logging.getLogger().error(json.dumps(result))

6 changes: 5 additions & 1 deletion proof-producer.nix
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
gtest,
enableDebug ? false,
runTests ? false,
sanitize? false,
}:
let
inherit (lib) optional;
Expand All @@ -34,6 +35,7 @@ in stdenv.mkDerivation {
"-DCMAKE_INSTALL_PREFIX=${placeholder "out"}"
(if enableDebug then "-DCMAKE_BUILD_TYPE=Debug" else "-DCMAKE_BUILD_TYPE=Release")
(if runTests then "-DENABLE_TESTS=ON" else "-DENABLE_TESTS=OFF")
(if sanitize then "-DSANITIZE=ON" else "-DSANITIZE=OFF")
"-DPROOF_PRODUCER_ENABLE=TRUE"
"-G Ninja"
];
Expand All @@ -43,7 +45,9 @@ in stdenv.mkDerivation {
checkPhase = ''
# JUNIT file without explicit file name is generated after the name of the master test suite inside `CMAKE_CURRENT_SOURCE_DIR`
export BOOST_TEST_LOGGER=JUNIT:HRF
cd proof-producer && ctest --verbose --output-on-failure -R && cd ..
cd proof-producer
ctest --verbose --output-on-failure -R
cd ..
mkdir -p ${placeholder "out"}/test-logs
find .. -type f -name '*_test.xml' -exec cp {} ${placeholder "out"}/test-logs \;
'';
Expand Down
14 changes: 14 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
requests==2.32.2
opentelemetry-distro[otlp]==0.45b0
opentelemetry-instrumentation-requests==0.45b0
opentelemetry-util-http==0.45b0
opentelemetry-instrumentation==0.45b0
opentelemetry-semantic-conventions==0.45b0
opentelemetry-instrumentation==0.45b0
opentelemetry-semantic-conventions==0.45b0
opentelemetry-util-http==0.45b0
opentelemetry-instrumentation==0.45b0
opentelemetry-semantic-conventions==0.45b0
opentelemetry-util-http==0.45b0
opentelemetry-sdk
junitparser==3.2.0
9 changes: 7 additions & 2 deletions zkevm-framework.nix
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,13 @@ in stdenv.mkDerivation rec {
doCheck = runTests;

checkPhase = ''
cd zkevm-framework && ctest
cd .. && ninja executables_tests
# JUNIT file without explicit file name is generated after the name of the master test suite inside `CMAKE_CURRENT_SOURCE_DIR`
export BOOST_TEST_LOGGER=JUNIT:HRF
cd zkevm-framework
ctest --verbose --output-on-failure -R
cd ..
mkdir -p ${placeholder "out"}/test-logs
find .. -type f -name '*_test.xml' -exec cp {} ${placeholder "out"}/test-logs \;
'';

shellHook = ''
Expand Down
Loading