Add CodSpeed performance benchmarks and CI#63
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 13 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
Greptile SummaryThis PR adds CodSpeed continuous performance benchmarking for the
Confidence Score: 5/5Safe to merge — all changes are purely additive (new dev-only dependency, new bench file, new CI workflow) with zero effect on production code paths. No production code is modified. The benchmark harness is correctly structured using black_box, the CI workflow follows the repository's existing action-pinning convention, and simulation mode is the appropriate choice for GitHub-hosted runners. The Cargo changes are scoped to dev-dependencies and do not affect published build artifacts. No files require special attention. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant GH as GitHub Actions
participant SR as moonrepo/setup-rust
participant CC as cargo codspeed build
participant CA as CodSpeedHQ/action
participant CR as cargo codspeed run
participant CS as CodSpeed (OIDC)
GH->>SR: Install Rust toolchain + cargo-codspeed binary
SR-->>GH: Toolchain + cache ready
GH->>CC: cargo codspeed build --workspace
CC-->>GH: Benchmark binaries in target/codspeed/
GH->>CA: "Trigger with mode=simulation"
CA->>CR: cargo codspeed run
CR-->>CA: Benchmark results (wallclock, simulation)
CA->>CS: Upload results via OIDC token
CS-->>CA: Acknowledgement / performance report
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant GH as GitHub Actions
participant SR as moonrepo/setup-rust
participant CC as cargo codspeed build
participant CA as CodSpeedHQ/action
participant CR as cargo codspeed run
participant CS as CodSpeed (OIDC)
GH->>SR: Install Rust toolchain + cargo-codspeed binary
SR-->>GH: Toolchain + cache ready
GH->>CC: cargo codspeed build --workspace
CC-->>GH: Benchmark binaries in target/codspeed/
GH->>CA: "Trigger with mode=simulation"
CA->>CR: cargo codspeed run
CR-->>CA: Benchmark results (wallclock, simulation)
CA->>CS: Upload results via OIDC token
CS-->>CA: Acknowledgement / performance report
Reviews (3): Last reviewed commit: "fix(docs): use working CodSpeed badge en..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- Fix CodSpeed badge to project-specific endpoint URL in both READMEs - Correct make_scores range comment in benchmark (0.001, ~1.0]
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
The project-specific badge.json path returns 404; CodSpeed's badge uses the generic app.codspeed.io/badge.json endpoint with a project-specific link.
|
3 participants
Overview
This PR sets up continuous performance measurement with CodSpeed for the Rust core crate (
code-search-please, libcsp). It adds a Criterion-based benchmark suite covering the hot paths and a GitHub Actions workflow that runs the benchmarks on every push and pull request.What was added
crates/csp/benches/core.rs) usingcriterionbacked bycodspeed-criterion-compat, so the same file runs under vanilla Criterion locally and under CodSpeed instrumentation in CI. Benchmarks cover:tokens::tokenize— identifier tokenization run over every indexed file.tokens::split_identifier— camelCase / snake_case / PascalCase splitting.ranking::penalties::file_path_penalty— per-path penalty matching.ranking::penalties::rerank_top_k— top-k reranking over 500 candidates, with and without path penalties.codspeed-criterion-compat(aliased ascriterion) as a dev-dependency and a[[bench]]target incrates/csp/Cargo.toml.Cargo.lockis updated accordingly..github/workflows/codspeed.yml): runs onpushtomain,pull_request, andworkflow_dispatch, using OIDC authentication andsimulationmode. Actions are pinned by commit hash to match the repository's existing convention.README.mdandREADME.ko.md.Verification
Built and ran the suite locally under simulation mode — all 5 benchmarks were measured and uploaded successfully:
file_path_penalty/mixed_pathssplit_identifier/mixed_batchtokenize/source_blockrerank_top_k/500_candidates_top10_penalisedrerank_top_k/500_candidates_top10_no_penaltycargo fmt --checkandcargo clippy --benches -D warningsboth pass on the new benchmark.Next steps
main.indexing::index,search) as those code paths stabilize.Summary by cubic
Adds CodSpeed benchmarks and CI to track Rust core performance on every push and PR. Includes a Criterion-compatible suite for key hot paths and a GitHub Actions workflow in simulation mode.
New Features
crates/csp/benches/core.rsusingcodspeed-criterion-compat(aliased ascriterion), covering tokenize, split_identifier, file_path_penalty, and rerank_top_k (with/without penalties)..github/workflows/codspeed.ymlforpushtomain,pull_request, andworkflow_dispatch; builds withcargo codspeed buildand runs withcargo codspeed runusing OIDC and simulation mode.criteriondev-dependency and a[[bench]]target incrates/csp/Cargo.toml; updatedCargo.lock.Bug Fixes
badge.jsonendpoint with a project-specific link in both READMEs.make_scoresbenchmark helper.Written for commit 44a69e6. Summary will update on new commits.