PyavOutput and VLC compatibility

[nudefireninja]

Hi folks... do you have any tips to improve compatibility with VLC or other media players? I'm working on a "wildlife" camera for the birds that frequent my window.

Previously I was recording .mjpeg files and later converting them to .mp4 with ffmpeg, and those files would play fine on macOS, even in QuickLook. Then I wanted to fix the inaccurate timestamps/framerate issue due to MJPEG not having support for variable frame durations, and somewhere along the line I discovered PyavOutput.

Now I'm using PyavOutput to make .mp4 files directly from H264Encoder and CircularOutput2, and I was even able to add audio for free. Nice! Not to mention I don't have to transfer the large .mjpeg files to the PC for the intensive processing anymore. However, playback only seems to work in IINA.app (and ffplay) on macOS. Both QuickTime and the latest version of VLC (3.0.21) can't play them.

I appreciate any suggestions.

---

More info:

I tried changing format from YUV420 to RGB888 but it didn't help.

The VLC 'Messages' window shows one error:

h26x error: this doesn't look like a hevc ES stream, continuing anyway

and a few hundred warnings, all the same:

hevc warning: Forbidden zero bit not null, corrupted NAL

Couldn't find an answer to that.

My setup code looks like this:

from picamera2 import Picamera2
from picamera2.encoders import MJPEGEncoder, H264Encoder, Quality
from picamera2.outputs import FileOutput, CircularOutput2
from libcamera import Transform

tuning = Picamera2.load_tuning_file("imx708_noir.json")
algo = Picamera2.find_tuning_algo(tuning, "rpi.agc")
algo["channels"][0]["metering_modes"]["custom"] = {"weights": [0,0,0,0,15,1,0,1,0,0,1,0,0,1,0]}

picam2 = Picamera2(tuning=tuning)
picam2.pre_callback = apply_timestamp

config = picam2.create_video_configuration(
    main={"size": (1280, 720), "format": "RGB888"}, # <---------- setting format
    lores={"size": (640, 360)},
    transform=Transform(hflip=True, vflip=True),
    buffer_count=10,
    encode="main")
picam2.configure(config)

picam2.set_controls({"FrameRate": 25})
picam2.set_controls({"NoiseReductionMode": 1})

capture_encoder = H264Encoder(bitrate=10_000_000) # <-------- encoder for .mp4's
capture_encoder.size = config["main"]["size"]
capture_encoder.format = config["main"]["format"]
capture_encoder.audio = True

circ = CircularOutput2(buffer_duration_ms=60_000)
capture_encoder.output = [circ]

stream_encoder = MJPEGEncoder()
stream_output = StreamingOutput()
picam2.start_recording(stream_encoder, FileOutput(stream_output), quality=Quality.MEDIUM)
picam2.start_encoder(capture_encoder, name="main")

And for capture I have basically this:

from time import strftime
from copy import copy
from picamera2.outputs import CircularOutput2, PyavOutput

# clone the circular output to allow overlapping captures
is_list = isinstance(capture_encoder.output, list)
if is_list:
    circ = next(o for o in capture_encoder.output if not o._output)
else:
    circ = capture_encoder.output

clone = CircularOutput2(buffer_duration_ms=circ.buffer_duration_ms)
clone._circular = copy(circ._circular)
clone._streams = circ._streams
clone.start()

if is_list:
    capture_encoder.output.append(clone)
else:
    capture_encoder.output = [circ, clone]

timestamp = strftime("%Y-%m-%d_%H-%M-%S")
pyavout = PyavOutput(f"{timestamp}.mp4") # <-------- creating PyavOutput
circ.open_output(pyavout)

# later...
circ.stop()

(Thank you so much btw for all the example codes that have helped me so much!)

[davidplowman]

I'm not sure I fully understand the code. I can see that you're writing to the PyavOutput from the initial CircularOutput2 "circ", but what is the "clone" doing? If you take that out do you still have the same problem?

It might be worth taking a little time to create the simplest example you can that shows the problem. For example, is that "clone" necessary? Can you remove the audio? Presumably the tuning file changes aren't relevant? Probably the streaming_output is not relevant either, so that could be removed? If you can create a slightly simpler example, I'd be happy to take a look.

If I had to guess, it might be that updating the capture_encoder's outputs while it's running isn't safe, but let's see if we can do anything to pin down the cause of the problem first.