Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions crates/core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,7 @@ harness = false
[[bench]]
name = "gaussian_pyramid_bench"
harness = false

[[bench]]
name = "kpm_bench"
harness = false
81 changes: 60 additions & 21 deletions crates/core/benches/BENCHMARKS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# WebARKitLib.rs Core Benchmarks

**Last Updated**: 2026-06-10 (chore: criterion 0.5.1 → 0.8 — #174)
**Last Updated**: 2026-07-15 (perf: add dedicated `kpm_bench.rs` — #225)

This document tracks the performance of critical image processing and pattern matching functions in the `webarkitlib_rs` core crate.

Expand Down Expand Up @@ -39,6 +39,17 @@ cargo bench --features simd --bench simd_bench
cargo bench --bench simd_bench
```

The crate registers these benches (all `harness = false`):

| Bench | Measures |
| :--- | :--- |
| `simd_bench` | Scalar vs SIMD kernels — the table above (`--features simd-x86-sse41`) |
| `marker_bench` | `ar_detect_marker` + pose — barcode/template marker pipeline |
| `feature_map_bench` | `ar2_gen_feature_map` — NFT marker generation (`--features log-helpers`) |
| `pyramid_bench` | Box-filter pyramid downsample (kept-but-unused reference, #203) |
| `gaussian_pyramid_bench` | Gaussian scale-space pyramid build (#200/#201/#207) |
| `kpm_bench` | `KpmHandle::kpm_matching` — KPM/NFT detection (#225; see below) |

## Setup Details

- **Tooling**: [Criterion.rs](https://github.com/bheisler/criterion.rs) `0.8`
Expand All @@ -48,20 +59,51 @@ cargo bench --bench simd_bench
- **Toolchain**: `rustc 1.94.0 (stable)`
- **SIMD activation**: SSE4.1 intrinsics are gated by `#[cfg(target_feature = "sse4.1")]` — set `RUSTFLAGS="-C target-feature=+sse4.1"` (or `target-cpu=native`) when running `simd_bench` to exercise the SIMD paths.

## KPM / NFT performance (M9-3 status)
## KPM / NFT performance

Issue #142's acceptance criterion calls for the pure-Rust NFT pipeline
to run within 20% of the C++ backend on `pinball-demo`. As of M9-3,
**there is no dedicated benchmark exercising the KPM / FreakMatcher
path**. The existing `marker_bench` measures `ar_detect_marker`
(barcode/template marker detection), which doesn't touch the
FreakMatcher and therefore can't distinguish pure-Rust from the C++
FFI backend.
to run within 20% of the C++ backend on `pinball-demo`. `marker_bench`
measures `ar_detect_marker` (barcode/template marker detection), which
never touches the FreakMatcher and therefore can't distinguish the
pure-Rust backend from the C++ FFI one — hence the dedicated
`kpm_bench.rs` below, added in #225.

### `kpm_bench.rs` — dedicated KPM wall-clock (#225)

Times a single `KpmHandle::kpm_matching` query — the per-frame NFT
detection path — using the pure-Rust `RustFreakMatcher` backend.

| | |
|---|---|
| **Measures** | one `KpmHandle::kpm_matching` call (detection only) |
| **Reference marker** | `pinball.fset3`, assigned to page 0 |
| **Query image** | `pinball-demo.jpg` — 2000×1500, converted to luma |
| **Fixtures** | `crates/core/examples/Data/` (shared with the `simple_nft` example and the KPM regression tests) |
| **Criterion config** | `sample_size = 10`, 15 s measurement — a full query is heavy |

```sh
cargo bench -p webarkitlib-rs --bench kpm_bench
```

Setup — reference-data load and handle construction — happens once
**outside** the measured loop, so only `kpm_matching` is timed.

### Functional parity evidence (in lieu of wall-clock numbers)
Baseline (release build, x86_64):

The Rust and C++ backends agree on the meaningful outputs across
several test suites:
| Backend | Query time (median) |
|---------|---------------------|
| Rust (`RustFreakMatcher`) | ~0.30 s |

To produce the C++ side of the #142 within-20% comparison, build with
`--features ffi-backend` and swap `RustFreakMatcher` for
`CppFreakMatcher` in the bench. Wall-clock numbers are
hardware-dependent, so treat the committed figure as an
order-of-magnitude reference rather than a hard CI gate.

### Functional parity evidence

Beyond wall-clock timing, the Rust and C++ backends agree on the
meaningful outputs across several test suites:

| Test | What it asserts | Status post-#170 |
|------|------------------|------------------|
Expand All @@ -70,13 +112,10 @@ several test suites:
| `cross_stack_parity` (jsartoolkitNFT#584 Track 2) | C++ FFI and Rust pose agree with jsartoolkitNFT-Node within rot 0.08 / trans 10 mm | ✅ green |
| `kpm_regression::test_full_pipeline_pose` | Linux C++ pose matches committed numerical baseline to 1e-2 | ✅ green |

The within-20% perf target is treated as **deferred, not failed**:
the functional evidence shows Rust meets parity by every quality
metric we measure, and #142 explicitly permits deferring the
quantitative perf check to a follow-up: *"If slower, open a follow-up
performance issue rather than blocking this PR."*

A future PR will add a KPM-specific Criterion bench (`kpm_bench.rs`)
that loads `pinball.fset3` + `pinball-demo.jpg` and times
`kpm_matching` with each backend, producing the dedicated wall-clock
comparison.
Taken together: Rust meets parity by every quality metric we measure,
and `kpm_bench` now provides the wall-clock baseline needed to catch
performance regressions in the FreakMatcher pipeline. The formal
within-20% comparison against C++ remains a manual, opt-in run
(`--features ffi-backend`) rather than a CI gate, per #142: *"If
slower, open a follow-up performance issue rather than blocking this
PR."*
124 changes: 124 additions & 0 deletions crates/core/benches/kpm_bench.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/*
* kpm_bench.rs
* WebARKitLib-rs
*
* This file is part of WebARKitLib-rs - WebARKit.
*
* WebARKitLib-rs is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* WebARKitLib-rs is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with WebARKitLib-rs. If not, see <http://www.gnu.org/licenses/>.
*
* As a special exception, the copyright holders of this library give you
* permission to link this library with independent modules to produce an
* executable, regardless of the license terms of these independent modules, and to
* copy and distribute the resulting executable under terms of your choice,
* provided that you also meet, for each linked independent module, the terms and
* conditions of the license of that module. An independent module is a module
* which is neither derived from nor based on this library. If you modify this
* library, you may extend this exception to your version of the library, but you
* are not obligated to do so. If you do not wish to do so, delete this exception
* statement from your version.
*
* Copyright 2026 WebARKit.
*
* Author(s): Walter Perdan @kalwalt https://github.com/kalwalt
*
*/

//! End-to-end KPM (Keypoint Matching) detection benchmark.
//!
//! Measures a single `KpmHandle::kpm_matching` query — the per-frame NFT
//! detection path — using the pure-Rust `RustFreakMatcher` backend against
//! the `pinball` reference marker on the `pinball-demo.jpg` query image.
//! `marker_bench` only covers barcode/template marker detection, so this is
//! the regression signal for the FreakMatcher pipeline (deferred from #142,
//! tracked in #225).
//!
//! Fixtures live in `crates/core/examples/Data/` (the same assets the
//! `simple_nft` example and the KPM regression tests use). Setup — loading
//! the reference data set and building the handle — is done once outside the
//! measured loop; only `kpm_matching` is timed.
//!
//! To also compare against the C++ FreakMatcher, build with
//! `--features ffi-backend` and swap in `CppFreakMatcher` (see #225 goal:
//! pure-Rust should stay within ~20% of C++).

use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;
use std::io::Cursor;
use std::path::Path;
use std::sync::Arc;
use std::time::Duration;

use webarkitlib_rs::kpm::ref_data_set::KPM_CHANGE_PAGE_NO_ALL_PAGES;
use webarkitlib_rs::kpm::types::KpmRefDataSet;
use webarkitlib_rs::kpm::{KpmHandle, RustFreakMatcher};
use webarkitlib_rs::types::{ARParam, ARParamLT};

fn kpm_matching_benchmark(c: &mut Criterion) {
let data_dir = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("examples")
.join("Data");

// Query image (pinball-demo) → grayscale. Exact luma values are
// irrelevant for timing, so we use the `image` crate directly.
let img = image::open(data_dir.join("pinball-demo.jpg"))
.expect("failed to open pinball-demo.jpg")
.to_luma8();
let width = img.width() as i32;
let height = img.height() as i32;
let luma: Vec<u8> = img.into_raw();

// Camera parameters, scaled to the image size (mirrors simple_nft).
let param_bytes =
std::fs::read(data_dir.join("camera_para.dat")).expect("failed to read camera_para.dat");
let mut param =
ARParam::load(Cursor::new(&param_bytes)).expect("failed to parse camera params");
let sx = width as f64 / param.xsize as f64;
let sy = height as f64 / param.ysize as f64;
for col in 0..4 {
param.mat[0][col] *= sx;
param.mat[1][col] *= sy;
}
param.xsize = width;
param.ysize = height;

// Reference data set (.fset3), assigned to page 0.
let mut ref_data =
KpmRefDataSet::load(&data_dir.join("pinball.fset3")).expect("failed to load pinball.fset3");
ref_data.change_page_no(KPM_CHANGE_PAGE_NO_ALL_PAGES, 0);

// KpmHandle over the pure-Rust FreakMatcher backend.
let param_lt = Arc::new(ARParamLT::new_basic(param));
let backend = RustFreakMatcher::new(width, height).expect("failed to create RustFreakMatcher");
let mut kpm_handle = KpmHandle::new(width, height, Some(param_lt), Box::new(backend));
kpm_handle
.set_ref_data_set(ref_data)
.expect("failed to set ref data set");

// A full KPM query is heavy; keep the sample count and measurement time
// modest so the bench stays runnable in CI without dominating the suite.
let mut group = c.benchmark_group("kpm");
group.sample_size(10);
group.measurement_time(Duration::from_secs(15));
group.bench_function(format!("kpm_matching_pinball_{width}x{height}"), |b| {
b.iter(|| {
kpm_handle
.kpm_matching(black_box(&luma))
.expect("kpm_matching failed");
})
});
group.finish();
}

criterion_group!(benches, kpm_matching_benchmark);
criterion_main!(benches);
Loading