Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout support to PoC for adding JDK's built-in support for Unix Domain Sockets on Java 16+ #8314

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Jan 30, 2025

What Does This Do

This PR continues on #8197 to add support for timeouts. Currently, I am trying to add tests to confirm that reading from a channel with no data will block instead of respecting the timeout period.

Motivation

For Java 17+, we want to replace third party jnr-unixsocket support of unix domain sockets with JDK's built-in support.

Additional Notes

The PR (including this comment) is a WIP. Tests need to be run with ./gradlew :utils:socket-utils:test -PtestJvm=17

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Comment on lines +33 to +35
TunnelingJdkSocket(final Path path) {
this.unixSocketAddress = UnixDomainSocketAddress.of(path);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Consider adding super() or this() to your constructor (...read more)

In Java, it is suggested to call super() in an extended class. This rule will report a violation if both a call to super() and an overloaded constructor are absent.

View in Datadog  Leave us feedback  Documentation

@pr-commenter
Copy link

pr-commenter bot commented Jan 30, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-timeout-support
git_commit_date 1738333475 1738362759
git_commit_sha 8d0bb34 c239caf
release_version 1.47.0-SNAPSHOT~8d0bb34e88 1.47.0-SNAPSHOT~c239caf3cb
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738365988 1738365988
ci_job_id 790178803 790178803
ci_pipeline_id 54636994 54636994
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-yhuzh6uy-project-304-concurrent-0-71c6w5io 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-yhuzh6uy-project-304-concurrent-0-71c6w5io 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 29 performance improvements and 9 performance regressions! Performance is the same for 17 metrics, 8 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start better
[-187.448ms; -147.512ms] or [-15.866%; -12.486%]
1.014s 1.181s
scenario:startup:insecure-bank:iast:GlobalTracer better
[-113.449ms; -106.460ms] or [-48.547%; -45.556%]
123.736ms 233.691ms
scenario:startup:insecure-bank:iast:Remote Config worse
[+910.889µs; +965.624µs] or [+145.177%; +153.901%]
1565.689µs 627.432µs
scenario:startup:insecure-bank:iast:Telemetry better
[-7.388ms; -7.057ms] or [-84.066%; -80.307%]
1.565ms 8.788ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start better
[-160.243ms; -131.106ms] or [-13.740%; -11.242%]
1.021s 1.166s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer better
[-107.298ms; -103.674ms] or [-46.653%; -45.078%]
124.503ms 229.989ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:AppSec better
[-41.721ms; -36.415ms] or [-78.265%; -68.311%]
14.239ms 53.307ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config worse
[+935.816µs; +969.488µs] or [+151.093%; +156.530%]
1572.016µs 619.364µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry better
[-7.409ms; -7.043ms] or [-84.521%; -80.346%]
1.540ms 8.766ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start better
[-174.703ms; -137.621ms] or [-14.931%; -11.762%]
1.014s 1.170s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer better
[-110.625ms; -104.390ms] or [-47.735%; -45.045%]
124.241ms 231.749ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:IAST worse
[+1.482ms; +2.716ms] or [+6.643%; +12.177%]
24.401ms 22.303ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:AppSec better
[-43.416ms; -42.154ms] or [-76.224%; -74.009%]
14.173ms 56.958ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config worse
[+919.972µs; +971.136µs] or [+150.699%; +159.081%]
1556.022µs 610.468µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry better
[-7.171ms; -6.838ms] or [-83.925%; -80.026%]
1.540ms 8.545ms
scenario:startup:insecure-bank:tracing:Agent.start better
[-166.939ms; -141.728ms] or [-16.086%; -13.657%]
0.883s 1.038s
scenario:startup:insecure-bank:tracing:GlobalTracer better
[-117.215ms; -113.013ms] or [-48.991%; -47.235%]
124.142ms 239.257ms
scenario:startup:insecure-bank:tracing:AppSec better
[-33.657ms; -31.906ms] or [-60.965%; -57.793%]
22.426ms 55.207ms
scenario:startup:insecure-bank:tracing:Remote Config worse
[+871.709µs; +914.247µs] or [+122.346%; +128.316%]
1605.473µs 712.495µs
scenario:startup:petclinic:appsec:Agent better
[-385.533ms; -222.081ms] or [-3.595%; -2.071%]
10.421s 10.725s
scenario:startup:petclinic:appsec:Agent.start better
[-286.885ms; -263.979ms] or [-24.287%; -22.348%]
0.906s 1.181s
scenario:startup:petclinic:appsec:GlobalTracer better
[-115.690ms; -110.796ms] or [-48.646%; -46.588%]
124.575ms 237.818ms
scenario:startup:petclinic:appsec:AppSec better
[-159.628ms; -154.830ms] or [-91.172%; -88.432%]
17.855ms 175.084ms
scenario:startup:petclinic:appsec:IAST worse
[+0.839ms; +1.366ms] or [+3.913%; +6.376%]
22.533ms 21.430ms
scenario:startup:petclinic:iast:Agent.start better
[-158.773ms; -149.987ms] or [-13.594%; -12.842%]
1.014s 1.168s
scenario:startup:petclinic:iast:GlobalTracer better
[-108.033ms; -105.710ms] or [-46.846%; -45.838%]
123.743ms 230.615ms
scenario:startup:petclinic:iast:AppSec better
[-43.694ms; -39.010ms] or [-78.555%; -70.135%]
14.270ms 55.621ms
scenario:startup:petclinic:iast:Remote Config worse
[+946.363µs; +971.841µs] or [+156.724%; +160.943%]
1562.943µs 603.841µs
scenario:startup:petclinic:iast:Telemetry better
[-7.325ms; -7.008ms] or [-84.086%; -80.441%]
1.545ms 8.712ms
scenario:startup:petclinic:profiling:Agent.start better
[-166.053ms; -157.099ms] or [-13.192%; -12.481%]
1.097s 1.259s
scenario:startup:petclinic:profiling:GlobalTracer better
[-116.047ms; -112.996ms] or [-33.104%; -32.234%]
236.030ms 350.551ms
scenario:startup:petclinic:profiling:AppSec better
[-41.810ms; -38.544ms] or [-75.999%; -70.063%]
14.836ms 55.013ms
scenario:startup:petclinic:profiling:Remote Config worse
[+878.260µs; +904.788µs] or [+129.286%; +133.191%]
1570.839µs 679.315µs
scenario:startup:petclinic:profiling:Telemetry better
[-7.512ms; -7.316ms] or [-84.490%; -82.283%]
1.477ms 8.891ms
scenario:startup:petclinic:tracing:Agent.start better
[-162.786ms; -155.773ms] or [-15.670%; -14.995%]
0.880s 1.039s
scenario:startup:petclinic:tracing:GlobalTracer better
[-116.878ms; -115.466ms] or [-48.730%; -48.141%]
123.678ms 239.850ms
scenario:startup:petclinic:tracing:AppSec better
[-34.320ms; -33.413ms] or [-61.679%; -60.049%]
21.777ms 55.644ms
scenario:startup:petclinic:tracing:Remote Config worse
[+864.556µs; +894.538µs] or [+121.541%; +125.756%]
1590.872µs 711.326µs
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.038 s) : 0, 1037796
Total [baseline] (8.618 s) : 0, 8618166
Agent [candidate] (883.463 ms) : 0, 883463
Total [candidate] (8.506 s) : 0, 8505529
section iast
Agent [baseline] (1.181 s) : 0, 1181423
Total [baseline] (9.219 s) : 0, 9219255
Agent [candidate] (1.014 s) : 0, 1013943
Total [candidate] (9.052 s) : 0, 9052409
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.166 s) : 0, 1166245
Total [baseline] (9.198 s) : 0, 9197883
Agent [candidate] (1.021 s) : 0, 1020570
Total [candidate] (9.071 s) : 0, 9071237
section iast_TELEMETRY_OFF
Agent [baseline] (1.17 s) : 0, 1170080
Total [baseline] (9.19 s) : 0, 9189537
Agent [candidate] (1.014 s) : 0, 1013918
Total [candidate] (9.092 s) : 0, 9091957
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent iast 1.181 s 143.627 ms (13.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.166 s 128.449 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.17 s 132.283 ms (12.7%)
Total tracing 8.618 s -
Total iast 9.219 s 601.09 ms (7.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.198 s 579.717 ms (6.7%)
Total iast_TELEMETRY_OFF 9.19 s 571.371 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 883.463 ms -
Agent iast 1.014 s 130.48 ms (14.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.021 s 137.108 ms (15.5%)
Agent iast_TELEMETRY_OFF 1.014 s 130.455 ms (14.8%)
Total tracing 8.506 s -
Total iast 9.052 s 546.879 ms (6.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.071 s 565.708 ms (6.7%)
Total iast_TELEMETRY_OFF 9.092 s 586.428 ms (6.9%)
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.622 ms) : 0, 714622
BytebuddyAgent [candidate] (718.551 ms) : 0, 718551
GlobalTracer [baseline] (239.257 ms) : 0, 239257
GlobalTracer [candidate] (124.142 ms) : 0, 124142
AppSec [baseline] (55.207 ms) : 0, 55207
AppSec [candidate] (22.426 ms) : 0, 22426
Remote Config [baseline] (712.495 µs) : 0, 712
Remote Config [candidate] (1.605 ms) : 0, 1605
Telemetry [baseline] (12.808 ms) : 0, 12808
Telemetry [candidate] (1.549 ms) : 0, 1549
section iast
BytebuddyAgent [baseline] (842.717 ms) : 0, 842717
BytebuddyAgent [candidate] (832.874 ms) : 0, 832874
GlobalTracer [baseline] (233.691 ms) : 0, 233691
GlobalTracer [candidate] (123.736 ms) : 0, 123736
IAST [baseline] (26.103 ms) : 0, 26103
IAST [candidate] (24.792 ms) : 0, 24792
AppSec [baseline] (54.178 ms) : 0, 54178
AppSec [candidate] (14.203 ms) : 0, 14203
Remote Config [baseline] (627.432 µs) : 0, 627
Remote Config [candidate] (1.566 ms) : 0, 1566
Telemetry [baseline] (8.788 ms) : 0, 8788
Telemetry [candidate] (1.565 ms) : 0, 1565
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (831.84 ms) : 0, 831840
BytebuddyAgent [candidate] (838.556 ms) : 0, 838556
GlobalTracer [baseline] (229.989 ms) : 0, 229989
GlobalTracer [candidate] (124.503 ms) : 0, 124503
IAST [baseline] (26.564 ms) : 0, 26564
IAST [candidate] (24.856 ms) : 0, 24856
AppSec [baseline] (53.307 ms) : 0, 53307
AppSec [candidate] (14.239 ms) : 0, 14239
Remote Config [baseline] (619.364 µs) : 0, 619
Remote Config [candidate] (1.572 ms) : 0, 1572
Telemetry [baseline] (8.766 ms) : 0, 8766
Telemetry [candidate] (1.54 ms) : 0, 1540
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.57 ms) : 0, 834570
BytebuddyAgent [candidate] (832.882 ms) : 0, 832882
GlobalTracer [baseline] (231.749 ms) : 0, 231749
GlobalTracer [candidate] (124.241 ms) : 0, 124241
IAST [baseline] (22.303 ms) : 0, 22303
IAST [candidate] (24.401 ms) : 0, 24401
AppSec [baseline] (56.958 ms) : 0, 56958
AppSec [candidate] (14.173 ms) : 0, 14173
Remote Config [baseline] (610.468 µs) : 0, 610
Remote Config [candidate] (1.556 ms) : 0, 1556
Telemetry [baseline] (8.545 ms) : 0, 8545
Telemetry [candidate] (1.54 ms) : 0, 1540
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1038859
Total [baseline] (10.437 s) : 0, 10437349
Agent [candidate] (879.58 ms) : 0, 879580
Total [candidate] (10.384 s) : 0, 10384471
section appsec
Agent [baseline] (1.181 s) : 0, 1181238
Total [baseline] (10.725 s) : 0, 10725184
Agent [candidate] (905.806 ms) : 0, 905806
Total [candidate] (10.421 s) : 0, 10421377
section iast
Agent [baseline] (1.168 s) : 0, 1167945
Total [baseline] (10.949 s) : 0, 10948816
Agent [candidate] (1.014 s) : 0, 1013566
Total [candidate] (10.844 s) : 0, 10843535
section profiling
Agent [baseline] (1.259 s) : 0, 1258730
Total [baseline] (10.849 s) : 0, 10848947
Agent [candidate] (1.097 s) : 0, 1097154
Total [candidate] (10.567 s) : 0, 10567226
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent appsec 1.181 s 142.379 ms (13.7%)
Agent iast 1.168 s 129.086 ms (12.4%)
Agent profiling 1.259 s 219.87 ms (21.2%)
Total tracing 10.437 s -
Total appsec 10.725 s 287.834 ms (2.8%)
Total iast 10.949 s 511.467 ms (4.9%)
Total profiling 10.849 s 411.597 ms (3.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 879.58 ms -
Agent appsec 905.806 ms 26.226 ms (3.0%)
Agent iast 1.014 s 133.986 ms (15.2%)
Agent profiling 1.097 s 217.574 ms (24.7%)
Total tracing 10.384 s -
Total appsec 10.421 s 36.906 ms (0.4%)
Total iast 10.844 s 459.064 ms (4.4%)
Total profiling 10.567 s 182.755 ms (1.8%)
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (713.28 ms) : 0, 713280
BytebuddyAgent [candidate] (715.886 ms) : 0, 715886
GlobalTracer [baseline] (239.85 ms) : 0, 239850
GlobalTracer [candidate] (123.678 ms) : 0, 123678
AppSec [baseline] (55.644 ms) : 0, 55644
AppSec [candidate] (21.777 ms) : 0, 21777
Remote Config [baseline] (711.325 µs) : 0, 711
Remote Config [candidate] (1.591 ms) : 0, 1591
Telemetry [baseline] (14.197 ms) : 0, 14197
Telemetry [candidate] (1.566 ms) : 0, 1566
section appsec
BytebuddyAgent [baseline] (733.069 ms) : 0, 733069
BytebuddyAgent [candidate] (733.53 ms) : 0, 733530
GlobalTracer [baseline] (237.818 ms) : 0, 237818
GlobalTracer [candidate] (124.575 ms) : 0, 124575
IAST [baseline] (21.43 ms) : 0, 21430
IAST [candidate] (22.533 ms) : 0, 22533
AppSec [baseline] (175.084 ms) : 0, 175084
AppSec [candidate] (17.855 ms) : 0, 17855
Remote Config [baseline] (651.03 µs) : 0, 651
Remote Config [candidate] (1.727 ms) : 0, 1727
Telemetry [baseline] (8.648 ms) : 0, 8648
Telemetry [candidate] (1.622 ms) : 0, 1622
section iast
BytebuddyAgent [baseline] (832.988 ms) : 0, 832988
BytebuddyAgent [candidate] (832.497 ms) : 0, 832497
GlobalTracer [baseline] (230.615 ms) : 0, 230615
GlobalTracer [candidate] (123.743 ms) : 0, 123743
IAST [baseline] (24.187 ms) : 0, 24187
IAST [candidate] (24.821 ms) : 0, 24821
AppSec [baseline] (55.621 ms) : 0, 55621
AppSec [candidate] (14.27 ms) : 0, 14270
Remote Config [baseline] (603.841 µs) : 0, 604
Remote Config [candidate] (1.563 ms) : 0, 1563
Telemetry [baseline] (8.712 ms) : 0, 8712
Telemetry [candidate] (1.545 ms) : 0, 1545
section profiling
BytebuddyAgent [baseline] (705.426 ms) : 0, 705426
BytebuddyAgent [candidate] (704.377 ms) : 0, 704377
GlobalTracer [baseline] (350.551 ms) : 0, 350551
GlobalTracer [candidate] (236.03 ms) : 0, 236030
AppSec [baseline] (55.013 ms) : 0, 55013
AppSec [candidate] (14.836 ms) : 0, 14836
Remote Config [baseline] (679.315 µs) : 0, 679
Remote Config [candidate] (1.571 ms) : 0, 1571
Telemetry [baseline] (8.891 ms) : 0, 8891
Telemetry [candidate] (1.477 ms) : 0, 1477
ProfilingAgent [baseline] (95.637 ms) : 0, 95637
ProfilingAgent [candidate] (96.725 ms) : 0, 96725
Profiling [baseline] (95.661 ms) : 0, 95661
Profiling [candidate] (96.739 ms) : 0, 96739
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-31T22:56:54 2025-01-31T23:03:56
git_branch master sarahchen6/add-timeout-support
git_commit_date 1738333475 1738362759
git_commit_sha 8d0bb34 c239caf
release_version 1.47.0-SNAPSHOT~8d0bb34e88 1.47.0-SNAPSHOT~c239caf3cb
start_time 2025-01-31T22:56:40 2025-01-31T23:03:43
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738364963 1738364963
ci_job_id 790178804 790178804
ci_pipeline_id 54636994 54636994
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kmuzcns6-project-304-concurrent-0-35mbm85q 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kmuzcns6-project-304-concurrent-0-35mbm85q 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 9 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 18 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast better
[-89.246µs; -42.536µs] or [-17.584%; -8.381%]
unstable
[-2159.029op/s; +3301.886op/s] or [-26.988%; +41.274%]
441.660µs 8571.429op/s 507.551µs 8000.000op/s
scenario:load:insecure-bank:iast_FULL better
[-328.374µs; -278.432µs] or [-44.175%; -37.456%]
unstable
[+472.510op/s; +5241.776op/s] or [+8.269%; +91.731%]
439.947µs 8571.429op/s 743.350µs 5714.286op/s
scenario:load:insecure-bank:iast_GLOBAL better
[-134.822µs; -85.573µs] or [-23.820%; -15.119%]
unstable
[-1285.910op/s; +3428.768op/s] or [-17.145%; +45.717%]
455.800µs 8571.429op/s 565.998µs 7500.000op/s
scenario:load:insecure-bank:iast_HARDCODED_SECRET_DISABLED better
[-88.168µs; -38.581µs] or [-17.449%; -7.636%]
unstable
[-1505.616op/s; +3967.154op/s] or [-18.820%; +49.589%]
441.901µs 9230.769op/s 505.276µs 8000.000op/s
scenario:load:petclinic:appsec better
[-359.699µs; -307.711µs] or [-20.603%; -17.625%]
unstable
[+74.738op/s; +1074.687op/s] or [+2.709%; +38.957%]
1.412ms 3333.333op/s 1.746ms 2758.621op/s
scenario:load:petclinic:appsec_no_iast better
[-355.118µs; -302.792µs] or [-20.341%; -17.344%]
unstable
[-104.612op/s; +987.371op/s] or [-3.792%; +35.792%]
1.417ms 3200.000op/s 1.746ms 2758.621op/s
scenario:load:petclinic:iast better
[-110.739µs; -56.246µs] or [-7.386%; -3.752%]
unstable
[-324.632op/s; +837.453op/s] or [-10.551%; +27.217%]
1.416ms 3333.333op/s 1.499ms 3076.923op/s
scenario:load:petclinic:profiling better
[-146.316µs; -91.333µs] or [-9.486%; -5.922%]
unstable
[-440.489op/s; +686.643op/s] or [-14.316%; +22.316%]
1.424ms 3200.000op/s 1.542ms 3076.923op/s
scenario:load:petclinic:tracing better
[-110.165µs; -57.447µs] or [-7.362%; -3.839%]
unstable
[-596.362op/s; +596.362op/s] or [-18.636%; +18.636%]
1.413ms 3200.000op/s 1.496ms 3200.000op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.362 ms) : 1343, 1382
.   : milestone, 1362,
appsec (1.746 ms) : 1722, 1769
.   : milestone, 1746,
appsec_no_iast (1.746 ms) : 1721, 1770
.   : milestone, 1746,
iast (1.499 ms) : 1474, 1524
.   : milestone, 1499,
profiling (1.542 ms) : 1517, 1568
.   : milestone, 1542,
tracing (1.496 ms) : 1471, 1522
.   : milestone, 1496,
section candidate
no_agent (1.342 ms) : 1323, 1362
.   : milestone, 1342,
appsec (1.412 ms) : 1387, 1437
.   : milestone, 1412,
appsec_no_iast (1.417 ms) : 1393, 1441
.   : milestone, 1417,
iast (1.416 ms) : 1390, 1441
.   : milestone, 1416,
profiling (1.424 ms) : 1398, 1449
.   : milestone, 1424,
tracing (1.413 ms) : 1389, 1436
.   : milestone, 1413,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.362 ms [1.343 ms, 1.382 ms] -
appsec 1.746 ms [1.722 ms, 1.769 ms] 383.565 µs (28.2%)
appsec_no_iast 1.746 ms [1.721 ms, 1.77 ms] 383.545 µs (28.2%)
iast 1.499 ms [1.474 ms, 1.524 ms] 136.985 µs (10.1%)
profiling 1.542 ms [1.517 ms, 1.568 ms] 180.11 µs (13.2%)
tracing 1.496 ms [1.471 ms, 1.522 ms] 134.226 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.362 ms] -
appsec 1.412 ms [1.387 ms, 1.437 ms] 69.742 µs (5.2%)
appsec_no_iast 1.417 ms [1.393 ms, 1.441 ms] 74.472 µs (5.5%)
iast 1.416 ms [1.39 ms, 1.441 ms] 73.374 µs (5.5%)
profiling 1.424 ms [1.398 ms, 1.449 ms] 81.168 µs (6.0%)
tracing 1.413 ms [1.389 ms, 1.436 ms] 70.301 µs (5.2%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.336 µs) : 360, 399
.   : milestone, 379,
iast (507.551 µs) : 486, 529
.   : milestone, 508,
iast_FULL (743.35 µs) : 721, 765
.   : milestone, 743,
iast_GLOBAL (565.998 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (505.276 µs) : 482, 528
.   : milestone, 505,
iast_INACTIVE (456.498 µs) : 435, 478
.   : milestone, 456,
iast_TELEMETRY_OFF (499.479 µs) : 476, 523
.   : milestone, 499,
tracing (461.976 µs) : 440, 484
.   : milestone, 462,
section candidate
no_agent (378.807 µs) : 359, 398
.   : milestone, 379,
iast (441.66 µs) : 420, 463
.   : milestone, 442,
iast_FULL (439.947 µs) : 416, 464
.   : milestone, 440,
iast_GLOBAL (455.8 µs) : 432, 479
.   : milestone, 456,
iast_HARDCODED_SECRET_DISABLED (441.901 µs) : 419, 465
.   : milestone, 442,
iast_INACTIVE (422.542 µs) : 400, 445
.   : milestone, 423,
iast_TELEMETRY_OFF (432.194 µs) : 409, 456
.   : milestone, 432,
tracing (420.325 µs) : 398, 443
.   : milestone, 420,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.336 µs [359.665 µs, 399.007 µs] -
iast 507.551 µs [485.657 µs, 529.444 µs] 128.215 µs (33.8%)
iast_FULL 743.35 µs [721.431 µs, 765.268 µs] 364.014 µs (96.0%)
iast_GLOBAL 565.998 µs [543.8 µs, 588.196 µs] 186.662 µs (49.2%)
iast_HARDCODED_SECRET_DISABLED 505.276 µs [482.17 µs, 528.381 µs] 125.94 µs (33.2%)
iast_INACTIVE 456.498 µs [435.439 µs, 477.558 µs] 77.162 µs (20.3%)
iast_TELEMETRY_OFF 499.479 µs [476.11 µs, 522.848 µs] 120.143 µs (31.7%)
tracing 461.976 µs [440.304 µs, 483.647 µs] 82.64 µs (21.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.807 µs [359.203 µs, 398.411 µs] -
iast 441.66 µs [420.147 µs, 463.172 µs] 62.853 µs (16.6%)
iast_FULL 439.947 µs [415.522 µs, 464.371 µs] 61.14 µs (16.1%)
iast_GLOBAL 455.8 µs [432.25 µs, 479.349 µs] 76.993 µs (20.3%)
iast_HARDCODED_SECRET_DISABLED 441.901 µs [418.926 µs, 464.877 µs] 63.095 µs (16.7%)
iast_INACTIVE 422.542 µs [399.71 µs, 445.374 µs] 43.735 µs (11.5%)
iast_TELEMETRY_OFF 432.194 µs [408.71 µs, 455.679 µs] 53.388 µs (14.1%)
tracing 420.325 µs [398.048 µs, 442.602 µs] 41.519 µs (11.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/add-timeout-support
git_commit_date 1738333475 1738362759
git_commit_sha 8d0bb34 c239caf
release_version 1.47.0-SNAPSHOT~8d0bb34e88 1.47.0-SNAPSHOT~c239caf3cb
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1738365576 1738365576
ci_job_id 790178805 790178805
ci_pipeline_id 54636994 54636994
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-yhuzh6uy-project-304-concurrent-1-ce050hfo 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-yhuzh6uy-project-304-concurrent-1-ce050hfo 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 4 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-547.214µs; -454.962µs] or [-23.139%; -19.238%]
1.864ms 2.365ms
scenario:dacapo:tomcat:iast better
[-163.372µs; -46.578µs] or [-7.764%; -2.214%]
1.999ms 2.104ms
scenario:dacapo:tomcat:iast_GLOBAL better
[-171.258µs; -53.490µs] or [-8.002%; -2.499%]
2.028ms 2.140ms
scenario:dacapo:tomcat:tracing better
[-133.625µs; -42.933µs] or [-6.824%; -2.192%]
1.870ms 1.958ms
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.631 s) : 15631000, 15631000
.   : milestone, 15631000,
appsec (15.116 s) : 15116000, 15116000
.   : milestone, 15116000,
iast (18.797 s) : 18797000, 18797000
.   : milestone, 18797000,
iast_GLOBAL (18.195 s) : 18195000, 18195000
.   : milestone, 18195000,
profiling (15.587 s) : 15587000, 15587000
.   : milestone, 15587000,
tracing (15.096 s) : 15096000, 15096000
.   : milestone, 15096000,
section candidate
no_agent (15.633 s) : 15633000, 15633000
.   : milestone, 15633000,
appsec (14.904 s) : 14904000, 14904000
.   : milestone, 14904000,
iast (17.082 s) : 17082000, 17082000
.   : milestone, 17082000,
iast_GLOBAL (18.096 s) : 18096000, 18096000
.   : milestone, 18096000,
profiling (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
tracing (15.043 s) : 15043000, 15043000
.   : milestone, 15043000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.631 s [15.631 s, 15.631 s] -
appsec 15.116 s [15.116 s, 15.116 s] -515.0 ms (-3.3%)
iast 18.797 s [18.797 s, 18.797 s] 3.166 s (20.3%)
iast_GLOBAL 18.195 s [18.195 s, 18.195 s] 2.564 s (16.4%)
profiling 15.587 s [15.587 s, 15.587 s] -44.0 ms (-0.3%)
tracing 15.096 s [15.096 s, 15.096 s] -535.0 ms (-3.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.633 s [15.633 s, 15.633 s] -
appsec 14.904 s [14.904 s, 14.904 s] -729.0 ms (-4.7%)
iast 17.082 s [17.082 s, 17.082 s] 1.449 s (9.3%)
iast_GLOBAL 18.096 s [18.096 s, 18.096 s] 2.463 s (15.8%)
profiling 15.131 s [15.131 s, 15.131 s] -502.0 ms (-3.2%)
tracing 15.043 s [15.043 s, 15.043 s] -590.0 ms (-3.8%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~c239caf3cb, baseline=1.47.0-SNAPSHOT~8d0bb34e88
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.365 ms) : 2321, 2408
.   : milestone, 2365,
iast (2.104 ms) : 2050, 2159
.   : milestone, 2104,
iast_GLOBAL (2.14 ms) : 2085, 2195
.   : milestone, 2140,
profiling (1.972 ms) : 1928, 2017
.   : milestone, 1972,
tracing (1.958 ms) : 1916, 2001
.   : milestone, 1958,
section candidate
no_agent (1.48 ms) : 1468, 1491
.   : milestone, 1480,
appsec (1.864 ms) : 1822, 1906
.   : milestone, 1864,
iast (1.999 ms) : 1945, 2053
.   : milestone, 1999,
iast_GLOBAL (2.028 ms) : 1973, 2082
.   : milestone, 2028,
profiling (1.893 ms) : 1848, 1937
.   : milestone, 1893,
tracing (1.87 ms) : 1828, 1912
.   : milestone, 1870,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 2.365 ms [2.321 ms, 2.408 ms] 890.213 µs (60.4%)
iast 2.104 ms [2.05 ms, 2.159 ms] 629.534 µs (42.7%)
iast_GLOBAL 2.14 ms [2.085 ms, 2.195 ms] 665.571 µs (45.1%)
profiling 1.972 ms [1.928 ms, 2.017 ms] 497.569 µs (33.7%)
tracing 1.958 ms [1.916 ms, 2.001 ms] 483.539 µs (32.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.468 ms, 1.491 ms] -
appsec 1.864 ms [1.822 ms, 1.906 ms] 383.981 µs (25.9%)
iast 1.999 ms [1.945 ms, 2.053 ms] 519.415 µs (35.1%)
iast_GLOBAL 2.028 ms [1.973 ms, 2.082 ms] 548.053 µs (37.0%)
profiling 1.893 ms [1.848 ms, 1.937 ms] 412.687 µs (27.9%)
tracing 1.87 ms [1.828 ms, 1.912 ms] 390.117 µs (26.4%)

@sarahchen6 sarahchen6 force-pushed the sarahchen6/add-timeout-support branch from 1802662 to 3bb147c Compare January 30, 2025 19:07
@sarahchen6 sarahchen6 force-pushed the sarahchen6/add-timeout-support branch from 75d65cd to c239caf Compare January 31, 2025 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants