Set preview size based on camera capabilities

[omares]

I would like to set the preview size depending on the camera capabilities. As example always use the middle size of the supported sizes. Unfortunately the size information is only available when the camera object is present, in the case of android-vision this is after CameraSource.Builder.Build() was called. As the builder requires the parameters before calling build you end up with a dead end.

Setting the preview size directly on the camera, after the the camera source class was build, currently leads to errors as the expected byte sizes differ.

Is there currently any possibility/hack to solve the issue above?

[pm0733464]

If you need more fine grained control of the camera, one thing to consider would be to make your own class that manages the camera. CameraSource class isn't closely tied to the rest of the mobile vision implementation, so the rest of the API could be used without it. It wouldn't be too hard for you to replace CameraSource with your own camera managing class (which selects the preview size differently). The main interaction between the camera source and the detector (and associated pipeline) is this method for sending preview frames into the detector:

https://developers.google.com/android/reference/com/google/android/gms/vision/Detector.html#receiveFrame(com.google.android.gms.vision.Frame)

You'd just need to have your camera managing class call this to pump preview frames into the detector/pipeline.

[omares]

Hm ok good to know, but as the CameraSource and CameraSource.Builder take away a lot of pita work when setting up the camera, controlling the buffers and what not, and reinventing the wheel in general is bad :), it would be great to get some kind of hook or callback that allows users todo their custom camera setup stuff.

[omares]

Btw would you mind open sourcing your CameraSource.Builder and CameraSource code?

[pm0733464]

We will consider it.

[omares]

@pm0733464 Could you give me some insights what parameters the SetPreviewSize methods effects besides the Camera.Parameters.SetPreviewSize option?

[pm0733464]

When you set the desired preview size, it will affect the selection of both the preview size and the picture size (i.e., the resolution when taking a photo). It shouldn't affect any other parameters.

The preview size and the picture size must be selected to both have the same aspect ratio, otherwise the camera's preview image will appear distorted on some devices. CameraSource chooses the preview size that most closely matches the desired preview size while also having a corresponding picture size of the same aspect ratio.

However, note that there is a bug currently that it selects the lowest resolution picture size, even if there are higher resolution picture sizes available at the same aspect ratio. This will be fixed in a future release.

[omares]

Ok thanks. Do you also set the camera display orientation parameter based on the preview size automatically?

[omares]

Thats weird. I am getting errors after setting the Preview- and PictureSize manually to the highest supported camera resolution. For example i am setting the preview size on my Htc one m8 to Width 1920 and Height 1088, i directly get an error after the first frames got processed:

[Camera-JNI] Callback buffer was too small! Expected 3133440 bytes, but got 1036808 bytes!
[Camera-JNI] Couldn't allocate byte array for JPEG data

Could you check if that is an issue on your side?

[omares]

Duh, found the issue in theory. I am setting the preview size while the preview is already running, need to find a way to stop and start the preview. Right now when stoping and starting the preview, by accessing the private camera field, no barcode is read afterwards.

Would be super awesome if you could help me out with your insights on how to stop and restart the current preview without loosing the current camera object and the preview callback. While still using CameraSource.

[pm0733464]

The camera display orientation parameter is not based on the preview size.

[claywilkinson]

@omares - The CameraSource source code has been added to the barcode-reader sample.

[omares]

@claywilkinson that is awesome. i guess that is not the version which is bundled with the vision library?

[pm0733464]

@omares This version of CameraSource was copied from what is bundled with the vision library. But it contains some features/fixes that aren't yet available from the current Google Play Services release (e.g., it has an auto-focus option).

[omares]

Cool!

Going through the code i noticed a comment stating that the play services v 8.1. (or higher) are required. Could you elaborate on that? The changelog of the play services does not provide any information on the changes of indirect buffers.

[pm0733464]

The previous version of the API used direct byte buffers exclusively, and the Frame class would not accept indirect byte buffers (intentionally throwing an exception if it were passed one). This had a slight performance advantage for calling into native code.

We discovered that using direct byte buffers in CameraSource was a problem, because the camera API could not account for the byte offset associated with direct byte buffers. The result was that CameraSource images could be shifted up to seven pixels to the left when using the face detector.

In 8.1, the API was changed to accept either indirect byte buffers or direct byte buffers. CameraSource was changed to use indirect byte buffers to avoid the pixel shift bug.

[omares]

Sorry for all the questions but did that only affect the face detector? It seems i am currently "stuck" with the 7.8 play release

[pm0733464]

Just curious, why are you stuck with 7.8?

Most users are automatically upgraded to the latest version of Google Play Services every time that there is an upgrade. Although an app can be compiled with an earlier version (and existing apps can continue to use earlier versions), in practice this will mean that users will be running a mix of the old version (in the client app) and the new version (in Google Play Services). Which is fine to do in general, but will be slightly off due to the earlier bug.

If you switch to using direct byte buffers in CameraSource in order to stick with 7.8, the position reported for your barcodes will be shifted to the left a bit (e.g., four pixels). But I'm guessing that probably isn't a big deal for most barcode apps.

[omares]

We are using Xamarin, and as of right now they do not provide any bindings for the newest google play services. I will update to the latest services as soon as these are available but this will take a few weeks and i want to ship the new scanner as fast as possible.

Thanks a lot for the insights! I created a custom CameraSource with a configuration hook that enables the user to inject custom configuration. :) As soon as the code is ready i will share it here.

[omares]

As announced i created a custom CameraSource:

• Added a ConfiguratorCallback which is called before the preview starts, thus giving full access and control to the camera features.
• Moved the existing configuration logic from the Builder into the DefaultConfigurator.
• Partitioned various methods into smaller ones to allow finer control over the camera. The start and stop methods still behave same. (open, startPreview, stopPreview, startFrameProcessor, stopFrameProcessor)

If requested i would create PR to incorporate the features into the sample.

[omares]

@pm0733464 Just so that i might can remove this task from my list: Would you guys be interested in a PR including the above changes?

[pm0733464]

Thanks. I can see how it would be useful for some applications, but my concern would be that it may make the API appear more complicated for others who don't need that level of customization. I'd wait to see if there's enough demand first.

[urlAkash]

@pm0733464 I want picture of scanned barcode , and want to display it to resulting activity, there are some methods like takePicture in cameraSource but how to use that in BarcodeCaptureActivity and where to call that method and how to get that captured image and store it??
Please I need an instant help with coding. thanks in advance

[pm0733464]

You'd have to decide where in your app it makes sense to call takePicture. Once called, it will deliver a binary jpeg image to the PictureCallback object that you supplied to takePicture:

https://developers.google.com/android/reference/com/google/android/gms/vision/CameraSource#takePicture(com.google.android.gms.vision.CameraSource.ShutterCallback, com.google.android.gms.vision.CameraSource.PictureCallback)

https://developers.google.com/android/reference/com/google/android/gms/vision/CameraSource.PictureCallback

[urlAkash]

thank you so much for replying

But I m not able to get it!!
when the barcode detection is going on and whenever the ROI generated around the barcode then at that moment i want to capture the image but i am not able to do that because i dont know where to call take picture, how to call take picture and how to get image as a result along with decoded barcode string onto calling activity and how to display the image....
Because I have implemented that , whenever the first barcode decoded it directly goes back to calling activity with the decoded barcode value without calling onTap using this link -

http://stackoverflow.com/questions/33371331/how-to-get-detected-barcode-qr-automatically/34102714#34102714

so if one can help me with previews and codes....because i m new in android!