CameraView CameraCaptureMode Stream

[qz2rg4]

Hi @pictos ,

I am looking to use CameraView with an EMGU OpenCV with a Face Landmark Detection + DNN.

I looked at the code and when the photo capture mode is selected you set up an image reader SetupImageReader to capture the image.

The idea is to add a StreamCaptureMode which in turn will set a capture method and delegate to capture and process the image buffer so that we can do real-time object detection without the need of taking the image first.

Please let me know your thoughts

Anton

[pictos]

@qz2rg4 I'm sorry if my questions (now and in the future) look naive, it's because I'm not very familiar with the Image recognition area. So what are you asking if for a way to process the image in real-time? Like if we are recording a video? I know that using the CameraView as a video recorder will not work because you just have access to the stream at the end of the recording. Do you have a code sample of it? Can be on Java.

[qz2rg4]

@pictos

Thanks for getting back to me.

Have a look at libemgucv-pro-4.4.0.4099.zip If anyone wants a copy of the lib email EMGUCV Support and they will send you an evaluation copy

Under libemgucv-pro-4.4.0.4099\Solution\Android\Emgu.CV.Android.Example.sln

Here they capture the stream directly from the camera.

OnYuvImageAvailableListener.cs

[pictos]

@qz2rg4 I grab the lib.

[qz2rg4]

In addition to my previous comment. We are looking at the Face Landmark Detection (DNN Module) - using Camera instead of photo or photo from library.

In the example they capture the stream from the camera and process one image/frame at a time. So the idea is to add the ability to CameraView to capture a frame and implementing a delegate or event to to handle the processing so that we don't have to use a specific library like OpenCV for processing, but we can roll our own.

This way we can do object recognition without the need to take a picture first.

I want to collaborate in this idea if accepted.

[pictos]

@qz2rg4 that sounds reasonable. For now, we are just talking about Android, do you know if it's feasible in UWP and iOS? Also, I need to take a look at the CameraX API. It's still in beta but maybe this change can help us to focus more on this kind of improvement.

[qz2rg4]

@pictos iOS will be a requirement with the project I am busy with. And seeing the CoreML integration into Xamarin.iOS. It will make sense to implement something like this. Unfortunately the example included in the EMGU lib does not show stream capture from the camera in ios or UWP. I am not sure that this is possible for iOS. Hoping you can shed some light on this.

I'll do more research and revert.

[pictos]

That sounds like a plan. TBH right now I can't have time to look into this, and you don't need to worry to do it very fast because this will take a while to be added. I can't say to you an EAT for now, but I promise to do it ASAP. And thanks again for this proposal!

[qz2rg4]

@pictos Cool beans, I must say CameraX will make life much easier

https://github.com/venetasoft/Xamarin.CameraX

[pictos]

Yeah, this is planned for the future. CameraX is alpha (google side) and because of that, we didn't use it in the first place.