Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

quic-apreetam · 2025-04-29T20:19:22Z

Detailed Description

This PR introduces FastCV DSP Extension APIs within the 'cv::fastcv::dsp' namespace.
The following APIs have been added:

fcvdspinit: Initializes the FastCV DSP environment.
fcvdspdeinit: Deinitializes the FastCV DSP environment.
sumOfAbsoluteDiffs: Computes the sum of absolute differences of an image against an 8x8 template.
thresholdOtsu: Binarizes a grayscale image using Otsu's method.
FFT: Computes the 1D or 2D Fast Fourier Transform of a real-valued matrix.
IFFT: Computes the 1D or 2D Inverse Fast Fourier Transform of a complex-valued matrix.
canny: Applies the Canny edge detector to an 8-bit grayscale image.
filter2D: Applies a generic 2D filter to an image.

The QcAllocator has been added to manage memory allocations on Qualcomm's Chipsets. This allocator ensures that matrices are allocated using the Qualcomm hardware memory allocator, providing efficient DSP operations.

Requires binary from opencv/opencv_3rdparty#95

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

asmorkalov · 2025-04-30T06:52:23Z

modules/fastcv/include/opencv2/fastcv/edges_dsp.hpp

+ * @param apertureSize  The Sobel kernel size for calculating gradient. Supported sizes are 3, 5 and 7.
+ * @param L2gradient    L2 Gradient or L1 Gradient
+*/
+CV_EXPORTS_W void canny(InputArray _src, OutputArray _dst, int lowThreshold, int highThreshold, int apertureSize = 3, bool L2gradient = false);


Canny is name. Please use capital letter.

Sure, will update

asmorkalov · 2025-04-30T06:55:44Z

modules/fastcv/src/allocator.cpp

+    if(data0)
+        u->flags |= cv::UMatData::USER_ALLOCATED;
+
+    u->userdata = new std::string("QCOM");


The solution leads to memory leak. Something like u->userdata = "QCOM"; should be enough.

Will compare Mat::allocator field with DSP allocator address, I will remove this

asmorkalov · 2025-04-30T06:56:04Z

modules/fastcv/src/allocator.cpp

+    if (u->userdata)
+    {
+        delete static_cast<std::string*>(u->userdata);
+        u->userdata = nullptr;


String is not freed.

Will compare Mat::allocator field with DSP allocator address, I will remove this as it won't be needed

asmorkalov · 2025-04-30T07:00:29Z

modules/fastcv/src/blur_dsp.cpp

+            CV_Assert(CV_MAT_DEPTH(kernel.type()) == CV_8S);
+            parallel_for_(Range(0, src.rows), FcvFilter2DLoop_Invoker(src, dst, kernel), nStripes);
+            break;
+        }
+        case CV_32F:
+        {
+            CV_Assert(CV_MAT_DEPTH(kernel.type()) == CV_32F);
+            parallel_for_(Range(0, src.rows), FcvFilter2DLoop_Invoker(src, dst, kernel), nStripes);
+            break;


DSP is a shared resource. Are you sure that parallel_for_ makes sense here? Also I would say that we need parallel scheme according to amount of DSP cores/units rather than CPU cores.

We can remove the parallel_for_ for now, we have parallelization within DSP API.

asmorkalov · 2025-04-30T07:03:21Z

modules/fastcv/src/precomp.hpp

+    #define IS_FASTCV_ALLOCATED(mat) \
+    ((mat.u && mat.u->userdata && \
+        *static_cast<std::string*>(mat.u->userdata) == "QCOM") ? true : \
+        (CV_Error(cv::Error::StsBadArg, cv::format("Matrix '%s' not allocated with FastCV allocator. " \


You can just compare Mat::allocator field with DSP allocator address. User data is redundant.

yeah, thanks for the input

asmorkalov · 2025-04-30T07:05:48Z

modules/fastcv/src/sad_dsp.cpp

+    // Check DSP initialization status and initialize if needed
+    FASTCV_CHECK_DSP_INIT();


I would say, that it should be at the very beginning of the function. If IS_FASTCV_ALLOCATED returned true, then DSP is already initialized, right?

In case the user fails to allocate the buffer using QcAllocator then we do not need to Check or Initialize the DSP. IS_FASTCV_ALLOCATED does not mean that the DSP is initialized. DSP will be initialized only when fcvdspinit is called or during FASTCV_CHECK_DSP_INIT();

asmorkalov · 2025-04-30T09:30:37Z

Java bindings generator does not handle namespaces well:

[255/3358] Generating opencv-4120.jar
FAILED: CMakeFiles/dephelper/opencv_java_jar bin/opencv-4120.jar 
cd /home/ci/build/modules/java/jar/opencv && /bin/ant -noinput -k jar && /usr/bin/cmake -E touch /home/ci/build/CMakeFiles/dephelper/opencv_java_jar
Buildfile: /home/ci/build/modules/java/jar/opencv/build.xml

jar:
    [javac] Compiling 299 source files to /home/ci/build/modules/java/jar/opencv/build/classes
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:715: error: method filter2D(Mat,Mat,int,Mat) is already defined in class Fastcv
    [javac]     public static void filter2D(Mat _src, Mat _dst, int ddepth, Mat _kernel) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:798: error: method FFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void FFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:818: error: method IFFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void IFFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 3 errors
Target 'jar' failed with message 'Compile failed; see the compiler error output for details.'.

BUILD FAILED

If I understand correctly, DSP part is not usable from Java code (no way to manage allocators in Java). I propose to replace CV_EXPORTS_W with CV_EXPORTS.

quic-apreetam · 2025-05-02T09:35:57Z

Java bindings generator does not handle namespaces well:

[255/3358] Generating opencv-4120.jar
FAILED: CMakeFiles/dephelper/opencv_java_jar bin/opencv-4120.jar 
cd /home/ci/build/modules/java/jar/opencv && /bin/ant -noinput -k jar && /usr/bin/cmake -E touch /home/ci/build/CMakeFiles/dephelper/opencv_java_jar
Buildfile: /home/ci/build/modules/java/jar/opencv/build.xml

jar:
    [javac] Compiling 299 source files to /home/ci/build/modules/java/jar/opencv/build/classes
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:715: error: method filter2D(Mat,Mat,int,Mat) is already defined in class Fastcv
    [javac]     public static void filter2D(Mat _src, Mat _dst, int ddepth, Mat _kernel) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:798: error: method FFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void FFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] /home/ci/build/modules/java/jar/opencv/java/org/opencv/fastcv/Fastcv.java:818: error: method IFFT(Mat,Mat) is already defined in class Fastcv
    [javac]     public static void IFFT(Mat src, Mat dst) {
    [javac]                        ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 3 errors
Target 'jar' failed with message 'Compile failed; see the compiler error output for details.'.

BUILD FAILED

If I understand correctly, DSP part is not usable from Java code (no way to manage allocators in Java). I propose to replace CV_EXPORTS_W with CV_EXPORTS.

sure @asmorkalov , will change CV_EXPORTS_W with CV_EXPORTS

quic-apreetam added 6 commits April 30, 2025 00:16

Add QcAllocator and DSP Init and Deinit Functions

7c342d6

Added sumOfAbsoluteDiffs DSP API with Accuracy and Perf Test

2bed4bc

Add thresholdOtSu DSP API with Accuracy and Perf Test

2ea338a

Add FFT and IFFT DSP API with Accuracy and Perf Test

51f92b0

Add canny DSP API with Accuracy and Perf Test

ffa4c84

Add filter2D DSP API with Accuracy and Perf Test

350f7f9

asmorkalov reviewed Apr 30, 2025

View reviewed changes

asmorkalov added the category: fastcv label Apr 30, 2025

asmorkalov assigned gursimarsingh Apr 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

quic-apreetam commented Apr 29, 2025 •

edited

Loading

asmorkalov Apr 30, 2025

quic-apreetam May 2, 2025

asmorkalov Apr 30, 2025

quic-apreetam May 2, 2025

asmorkalov Apr 30, 2025

quic-apreetam May 2, 2025

asmorkalov Apr 30, 2025

qc-apreetam May 2, 2025

asmorkalov Apr 30, 2025

quic-apreetam May 2, 2025

asmorkalov Apr 30, 2025

quic-apreetam May 2, 2025

asmorkalov commented Apr 30, 2025

quic-apreetam commented May 2, 2025

		// Check DSP initialization status and initialize if needed
		FASTCV_CHECK_DSP_INIT();

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

Are you sure you want to change the base?

Add FastCV DSP Initialization, QcAllocator and FastCV DSP Extension APIs #3931

Conversation

quic-apreetam commented Apr 29, 2025 • edited Loading

Pull Request Readiness Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asmorkalov commented Apr 30, 2025

quic-apreetam commented May 2, 2025

quic-apreetam commented Apr 29, 2025 •

edited

Loading