Description:
To move purecv toward a production-ready AR library, we need to implement the geometric pipeline that accounts for lens characteristics and image perspective.
Scope of Work:
Performance & WASM SIMD Requirements
Functions like remap and warpPerspective are pixel-heavy operations. To ensure real-time execution (60fps) on mobile browsers, the implementation must adhere to the following architecture:
Context:
Currently, solvePnP assumes a pinhole camera model without distortion. Adding remap and initUndistortRectifyMap will allow us to calibrate against real-world mobile device sensors, significantly reducing tracking "jitter" at the frame edges. warpPerspective provides the final step for mapping content onto detected planes.
Technical Note for Implementation
Since these functions involve significant pixel-looping, they are prime candidates for SIMD-accelerated interpolation (bilinear/nearest-neighbor).
For remap, consider defining a generic interpolation trait:
pub trait Interpolator {
fn interpolate(&self, src: &Mat, x: f32, y: f32) -> Vec<u8>;
}
This allows you to implement BilinearInterpolator (which is standard for high-quality AR) efficiently using WASM SIMD to process multiple pixels per instruction.
Description:
To move purecv toward a production-ready AR library, we need to implement the geometric pipeline that accounts for lens characteristics and image perspective.
Scope of Work:
remap: Implement the core function for generic image remapping.initUndistortRectifyMap: Create a function to generate the map matrices required for remap. This is crucial for correcting radial and tangential lens distortion.warpPerspective: Implement the transformation function to apply the homography matrix (obtained from findHomography) to an image, enabling planar surface rendering.findFundamentalMat: Implement the algorithm to estimate the fundamental matrix, adding robust outlier rejection (RANSAC) to improve feature matching accuracy in non-planar scenes.examples: Provide a sample demonstration (examples/rectification.rs) showcasing camera undistortion and perspective warping.Performance & WASM SIMD Requirements
Functions like remap and warpPerspective are pixel-heavy operations. To ensure real-time execution (60fps) on mobile browsers, the implementation must adhere to the following architecture:
SIMD Acceleration: The core interpolation loops (e.g., bilinear interpolation) must be optimized using
pulpSIMD to process multiple pixels simultaneously per CPU cycle. (Implemented generic SIMD dispatch forf32,f64, andu8elements).Scalar Fallback: A standard scalar implementation (one pixel at a time) must be provided as a fallback to prevent crashes on older devices or environments that lack native WebAssembly SIMD support.
Context:
Currently,
solvePnPassumes a pinhole camera model without distortion. AddingremapandinitUndistortRectifyMapwill allow us to calibrate against real-world mobile device sensors, significantly reducing tracking "jitter" at the frame edges.warpPerspectiveprovides the final step for mapping content onto detected planes.Technical Note for Implementation
Since these functions involve significant pixel-looping, they are prime candidates for SIMD-accelerated interpolation (bilinear/nearest-neighbor).
For
remap, consider defining a generic interpolation trait:This allows you to implement
BilinearInterpolator(which is standard for high-quality AR) efficiently using WASM SIMD to process multiple pixels per instruction.