Set preview size based on camera capabilities

[omares]

I would like to set the preview size depending on the camera capabilities. As example always use the middle size of the supported sizes. Unfortunately the size information is only available when the camera object is present, in the case of android-vision this is after CameraSource.Builder.Build() was called. As the builder requires the parameters before calling build you end up with a dead end.

Setting the preview size directly on the camera, after the the camera source class was build, currently leads to errors as the expected byte sizes differ.

Is there currently any possibility/hack to solve the issue above?

[pm0733464]

If you need more fine grained control of the camera, one thing to consider would be to make your own class that manages the camera. CameraSource class isn't closely tied to the rest of the mobile vision implementation, so the rest of the API could be used without it. It wouldn't be too hard for you to replace CameraSource with your own camera managing class (which selects the preview size differently). The main interaction between the camera source and the detector (and associated pipeline) is this method for sending preview frames into the detector:

https://developers.google.com/android/reference/com/google/android/gms/vision/Detector.html#receiveFrame(com.google.android.gms.vision.Frame)

You'd just need to have your camera managing class call this to pump preview frames into the detector/pipeline.

[omares]

Hm ok good to know, but as the CameraSource and CameraSource.Builder take away a lot of pita work when setting up the camera, controlling the buffers and what not, and reinventing the wheel in general is bad :), it would be great to get some kind of hook or callback that allows users todo their custom camera setup stuff.

[omares]

Btw would you mind open sourcing your CameraSource.Builder and CameraSource code?

[pm0733464]

We will consider it.

[omares]

@pm0733464 Could you give me some insights what parameters the SetPreviewSize methods effects besides the Camera.Parameters.SetPreviewSize option?

[pm0733464]

When you set the desired preview size, it will affect the selection of both the preview size and the picture size (i.e., the resolution when taking a photo). It shouldn't affect any other parameters.

The preview size and the picture size must be selected to both have the same aspect ratio, otherwise the camera's preview image will appear distorted on some devices. CameraSource chooses the preview size that most closely matches the desired preview size while also having a corresponding picture size of the same aspect ratio.

However, note that there is a bug currently that it selects the lowest resolution picture size, even if there are higher resolution picture sizes available at the same aspect ratio. This will be fixed in a future release.

[omares]

Ok thanks. Do you also set the camera display orientation parameter based on the preview size automatically?

[omares]

Thats weird. I am getting errors after setting the Preview- and PictureSize manually to the highest supported camera resolution. For example i am setting the preview size on my Htc one m8 to Width 1920 and Height 1088, i directly get an error after the first frames got processed:

[Camera-JNI] Callback buffer was too small! Expected 3133440 bytes, but got 1036808 bytes!
[Camera-JNI] Couldn't allocate byte array for JPEG data

Could you check if that is an issue on your side?

[omares]

Duh, found the issue in theory. I am setting the preview size while the preview is already running, need to find a way to stop and start the preview. Right now when stoping and starting the preview, by accessing the private camera field, no barcode is read afterwards.

Would be super awesome if you could help me out with your insights on how to stop and restart the current preview without loosing the current camera object and the preview callback. While still using CameraSource.

[pm0733464]

The camera display orientation parameter is not based on the preview size.

[claywilkinson]

@omares - The CameraSource source code has been added to the barcode-reader sample.

[omares]

@claywilkinson that is awesome. i guess that is not the version which is bundled with the vision library?

[pm0733464]

@omares This version of CameraSource was copied from what is bundled with the vision library. But it contains some features/fixes that aren't yet available from the current Google Play Services release (e.g., it has an auto-focus option).

[omares]

Cool!

Going through the code i noticed a comment stating that the play services v 8.1. (or higher) are required. Could you elaborate on that? The changelog of the play services does not provide any information on the changes of indirect buffers.

[pm0733464]

The previous version of the API used direct byte buffers exclusively, and the Frame class would not accept indirect byte buffers (intentionally throwing an exception if it were passed one). This had a slight performance advantage for calling into native code.

We discovered that using direct byte buffers in CameraSource was a problem, because the camera API could not account for the byte offset associated with direct byte buffers. The result was that CameraSource images could be shifted up to seven pixels to the left when using the face detector.

In 8.1, the API was changed to accept either indirect byte buffers or direct byte buffers. CameraSource was changed to use indirect byte buffers to avoid the pixel shift bug.