Skip to content

feat: Add some benches and integrate with CodSpeed#62

Draft
larseggert wants to merge 9 commits into
mainfrom
feat-codspeed
Draft

feat: Add some benches and integrate with CodSpeed#62
larseggert wants to merge 9 commits into
mainfrom
feat-codspeed

Conversation

@larseggert

Copy link
Copy Markdown
Collaborator

No description provided.

Copilot AI review requested due to automatic review settings March 27, 2026 11:32
@codecov-commenter

codecov-commenter commented Mar 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.03%. Comparing base (2b341bf) to head (4bcf894).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #62   +/-   ##
=======================================
  Coverage   97.03%   97.03%           
=======================================
  Files           2        2           
  Lines         776      776           
  Branches      776      776           
=======================================
  Hits          753      753           
  Misses         21       21           
  Partials        2        2           
Flag Coverage Δ
linux 97.03% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed-hq

codspeed-hq Bot commented Mar 27, 2026

Copy link
Copy Markdown

Congrats! CodSpeed is installed 🎉

🆕 8 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a benchmark suite for the happy-eyeballs state machine and wires it into CI via CodSpeed to track performance over time.

Changes:

  • Added divan-style benchmarks covering several Happy Eyeballs scenarios (IPv6-only, dual-stack racing, HTTPS/ECH, many endpoints).
  • Integrated CodSpeed-compatible Divan dependency and registered the happy_eyeballs bench target.
  • Updated CI to run CodSpeed benchmarks and expanded coverage command to include all targets.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File Description
benches/happy_eyeballs.rs New Divan/CodSpeed-compatible benchmark scenarios for the state machine.
Cargo.toml Adds CodSpeed Divan compat dev-dependency and declares the bench target (no harness).
Cargo.lock Locks new transitive dependencies introduced by CodSpeed/Divan compat.
.github/workflows/rust.yml Runs cargo bench --bench happy_eyeballs via CodSpeed; adjusts coverage run to --all-targets.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread benches/happy_eyeballs.rs Outdated
Comment thread .github/workflows/rust.yml Fixed
@larseggert larseggert force-pushed the feat-codspeed branch 2 times, most recently from b12ebab to dd4addb Compare March 27, 2026 11:50
@larseggert

Copy link
Copy Markdown
Collaborator Author

@mxinden ready from my side. Can tweak the actual benches in follow-up PRs.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good addition — the benchmarks are well-structured, cover the key state machine paths (single-family, dual-stack racing, HTTPS/ECH, many-endpoint exhaustion), and the CodSpeed integration is straightforward.

A few things to address:

  • Action version mismatch: The benchmarks job still pins mozilla/actions/rust at v1.0.4 while the rest of the workflow was bumped to v1.0.11 via main. See inline comment.

  • --all-targets in coverage job: This change causes the benchmark binary to also execute during coverage runs (divan falls back to walltime mode outside CodSpeed). Worth considering whether that's intentional — see inline comment.

  • PR title typo: "Cosdpeed" → "CodSpeed".

  • Re: Copilot's None-in-loop finding (comment): This was already addressed — the current code correctly groups None with the break arm (lines 253–256).

  • Re: zizmor permissions finding (comment): Already addressed — id-token: write has the explanatory comment on line 50.

Comment thread .github/workflows/rust.yml Outdated
Comment thread .github/workflows/rust.yml Outdated
Comment thread benches/happy_eyeballs.rs Outdated
Comment thread Cargo.toml Outdated
@larseggert

Copy link
Copy Markdown
Collaborator Author

@mxinden I don't think this will cause vendoring issues for Gecko.

@larseggert

larseggert commented Apr 17, 2026

Copy link
Copy Markdown
Collaborator Author

@mxinden OK to merge?

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good set of benchmarks — they cover the key state machine paths (single-family, dual-stack racing, HTTPS/ECH, many-endpoint exhaustion) and the CodSpeed integration is clean.

Most findings from prior reviews have been addressed in the latest push: --all-targets removed from coverage (replaced with cargo check --benches), String::new() for failure messages, trailing newline in Cargo.toml, and None correctly breaks in the many_endpoints loop.

One remaining item: the mozilla/actions/rust pin in the benchmarks job is still behind — v1.0.11 vs v1.1.0 in the rest of the workflow (see inline comment).

PR title typo (noted by the prior review but still present): "Cosdpeed" → "CodSpeed".

Comment thread .github/workflows/rust.yml Outdated
Comment thread benches/happy_eyeballs.rs
@larseggert larseggert changed the title feat: Add some benches and integrate with Cosdpeed feat: Add some benches and integrate with CodSpeed Apr 17, 2026
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Comment thread src/lib.rs Outdated
Comment thread src/lib.rs Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-designed benchmark suite covering the key state machine paths (single-family, dual-stack racing, HTTPS/ECH, many-endpoint exhaustion). The CodSpeed integration and CI wiring are clean.

Blocking: The latest commit ("Fix") breaks CI — cargo check --benches and cargo codspeed build both fail. See inline comment for the fix.

Previously flagged issues — now resolved:

  • mozilla/actions/rust pin: now v1.1.0, consistent with the rest of the workflow
  • --all-targets in coverage: replaced with a separate cargo check --benches step
  • String::new() for failure messages (avoids allocation noise)
  • Trailing newline in Cargo.toml
  • None correctly breaks in the many_endpoints loop

Comment thread benches/happy_eyeballs.rs

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-structured benchmark suite covering the key state machine paths — good selection of scenarios (IPv6-only baseline, dual-stack racing, HTTPS/ECH with hints, many-endpoint exhaustion).

Status of prior review findings — all addressed in latest push:

  • assert_matches import (prior comment): Fixed — switched to stable assert!(matches!(…)), removing the broken use std::assert_matches that prevented compilation.
  • mozilla/actions/rust pin: Now v1.1.0, consistent with the rest of the workflow.
  • --all-targets in coverage: Replaced with a dedicated cargo check --benches step — cleaner separation of compile-check vs. coverage.
  • String::new() for failure messages: Eliminates per-iteration allocation noise.
  • Trailing newline in Cargo.toml: Present.

No new blocking issues found. A few observations:

  • The many_endpoints loop driver (lines 228–241) is well-designed: it correctly breaks on terminal outputs (Succeeded/Failed) and None, and the VecDeque-based in-flight tracking naturally handles the interleaved AttemptConnection/Timer cadence the state machine produces.
  • The code duplication between benches/happy_eyeballs.rs and tests/common/mod.rs (constants, input constructors) was noted in a prior review — still present but non-blocking and the bench helpers are intentionally leaner (no env_logger, drain_dns_queries baked into setup).
  • CI permissions are properly scoped (id-token: write only on the benchmarks job, with an explanatory comment). The workflow_dispatch trigger for CodSpeed backtesting is standard.

@mxinden

mxinden commented Apr 17, 2026

Copy link
Copy Markdown
Member

Sorry for the delay.

The state machine is quite simple. The most expensive operations are a couple of string allocations.

I'm not convinced these benchmarks provide much value at this point. If the result is X, what does X tell us relative to? We haven't identified any improvement opportunities to measure against.

I'd suggest holding off on merging until we've identified something concrete to improve, e.g. if happy eyeballs shows up in a Firefox profile. Until then, I suggest we keep this as a pull request only. I should have suggested this earlier, sorry about that.

@larseggert

Copy link
Copy Markdown
Collaborator Author

ACK, I'll make it a draft.

@larseggert larseggert marked this pull request as draft April 17, 2026 09:33
@larseggert

Copy link
Copy Markdown
Collaborator Author

(Wondering if the memory benchmarks would be more useful...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants