Read image brightness while streaming?

[elaw01887]

I'm trying to write a video-streaming program and this library and in particular the "mjpeg_server_2.py" example seem like a great start.

But I do have one interesting challenge... the lens I'm using on my camera has a motorized iris. I know how to control the iris from the Pi but I'd like to do it automatically which means I somehow need to be able to tell what "brightness" (for lack of a better term) the image sensor is seeing while the streaming is taking place.

Has anyone here ever attempted anything like that, or have any ideas how I might go about doing that?

[davidplowman]

I would consider looking at the "lux" value in the image metadata (for example, obtained by picam2.capture_metadata()['lux'], or if you have captured a request already then request.get_metadata()['lux']).

This should give you an absolute measure of the scene brightness - maybe you can use a simple lookup to translate that into an iris value. Of course, the AGC/AEC algortihm (assuming it's running) will take a moment to catch up when the iris moves.

The other way might be to read the "ExposureTime" and "AnalogueGain" out of the metadata instead. You'd multiply the two together and then that would give a similar "absolute brightness" value (though larger now means "darker"). But I expect the "lux" value might be easier to use, and should change less from sensor to sensor.

[elaw01887]

Thanks!
I'm realizing that part of what I need to figure out is how to have two streams going at once... one being served to web browsers, and the other going to code that can analyze it and pull out values like you're talking about. It's gonna be interesting...

[davidplowman]

You should be able to set up a stream that gets sent to web browser using the start_recording() call. This will run transparently in the background, and your main application thread doesn't have to do anything.

That leaves the main application thread free to do analysis, either on the same images that you're streaming, or on the other Picamera2 stream.

For streaming to web browsers, I've used both MJPEG (e.g. mjpeg_server_2.py, though that's relatively "old fashioned" these days) and MediaMTX for WebRTC or RTSP streams. There are probably other options too...