UVC Video encoder (zephyr,videoenc) + H264 support + STM32 STM32N6 conf files #97425
+923
−235
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The UVC class was deciding itself which formats were sent to the host. Remove this logic out of the UVC class and introduce uvc_add_format() to give the application the freedom of which format to list. Signed-off-by: Josuah Demangeon <[email protected]>
The UVC class now lets the application select the format list sent to the host. Leverage this in the sample to filter out any format that is not expected to work (buffer too big, rarely supported formats). Signed-off-by: Josuah Demangeon <[email protected]>
Add USB UVC device's new uvc_add_format() function to the release note, and document the semantic changes of UVC APIs in the migration guide. Signed-off-by: Josuah Demangeon <[email protected]>
Currently the DCMIPP driver rely on a Kconfig in order to
select the right sensor resolution / format to pick.
This also makes the exposure of caps easier since it can
be exposed as:
DUMP pipe: same caps as mentioned in Kconfig
MAIN pipe: any format supported on this pipe and resolution
starting at sensor selected resolution down to
64 times smaller (which is the maximum of the
downscale)
AUX pipe: same as MAIN except without the semi-planar and
planar formats
Signed-off-by: Alain Volmat <[email protected]>
Some devices allow for downscale / upscale via the set_selection compose API. When using it, it is necessary to perform a set_selection of the compose target prior to setting the format. In order to allow non-compose aware application to benefit from it, introduce a helper which take care of setting the compose prior to setting the format. Signed-off-by: Alain Volmat <[email protected]>
Simplify the code by using the video_set_compose_format helper. Signed-off-by: Alain Volmat <[email protected]>
Honor the CONFIG_VIDEO_BUFFER_POOL_ALIGN config by using the video_buffer_aligned_alloc function instead of video_buffer_alloc in order to provide properly aligned buffers to drivers. Signed-off-by: Alain Volmat <[email protected]>
Use the helper video_set_compose_format in order to allow controlling the compose. Signed-off-by: Alain Volmat <[email protected]>
Select from commonly used resolution when the video device advertise capabilities using range. Signed-off-by: Alain Volmat <[email protected]>
Add board specific conf files for the stm32n6570_dk Signed-off-by: Alain Volmat <[email protected]>
zephyrbot
added
platform: STM32
zephyrbot
requested review from
FRASTM,
GeorgeCGV,
cfriedt,
djiatsaf-st,
erwango,
etienne-lms,
gautierg-st,
jfischer-no,
jhedberg,
josuah,
loicpoulain,
marwaiehm-st,
mathieuchopstm,
ngphibang and
tmon-nordic
|
Moving assignee to Video subsystem maintainer, as it seems more appropriate to the content of the RFC |
|
Adding a DNM flag for PRs dependencies. |
josuah
previously approved these changes
Oct 14, 2025
Comment on lines
250
to
+369
| ret = video_stream_start(video_dev, VIDEO_BUF_TYPE_OUTPUT); | ||
| if (ret != 0) { | ||
| LOG_ERR("Failed to start %s", video_dev->name); | ||
| return ret; | ||
| } | ||
| ret = video_stream_start(uvc_src_dev, VIDEO_BUF_TYPE_INPUT); | ||
| if (ret != 0) { | ||
| LOG_ERR("Failed to start %s", uvc_src_dev->name); | ||
| return ret; | ||
| } | ||
| #endif |
There was a problem hiding this comment.
How about using videoenc_dev for both here?
Then if a videoscaler_dev is introduced, this becomes another #if DT_HAS_CHOSEN(zephyr_videoscaler) that uses the associated videoscaler_dev for both input/output?
Comment on lines
+371
to
+375
| ret = video_stream_start(uvc_src_dev, VIDEO_BUF_TYPE_OUTPUT); | ||
| if (ret != 0) { | ||
| LOG_ERR("Failed to start %s", video_dev->name); | ||
| return ret; | ||
| } |
There was a problem hiding this comment.
How about using video_dev here?
| #if DT_HAS_CHOSEN(zephyr_videoenc) | ||
| vbuf_enc_in = &(struct video_buffer){.type = VIDEO_BUF_TYPE_OUTPUT}; | ||
|
|
||
| if (video_dequeue(video_dev, &vbuf_enc_in, K_NO_WAIT) == 0) { |
There was a problem hiding this comment.
This works well for this sample but this is where things will start to become difficult.
One way to solve this maybe is to have a configuration table at the top:
video_dst = video_dev;
prev = &video_dst;
#if DT_HAS_CHOSEN(zephyr_videoisp)
videoisp_src = *prev;
*prev = videoisp_dev;
prev = &videoisp_dst;
#endif
#if DT_HAS_CHOSEN(zephyr_videoscaler)
videoscaler_src = *prev;
*prev = videoscaler_dev;
prev = &videoscaler_dst;
#endif
#if DT_HAS_CHOSEN(zephyr_videoenc)
videoenc_src = *prev;
*prev = videoenc_dev;
prev = &videoenc_dst;
#endif
videousb_src = *prev;
*prev = videousb_dev;Then in each block, it is possible to just use _src and _dst.
Of course a component-based system is better, but this would be a different sample for libMP. :]
There was a problem hiding this comment.
This is just for discussion and not necessarily to implement on this PR, thoough, as there is only one M2M device.
added 8 commits
October 19, 2025 17:30
Allow creating a pipeline as follow camera receiver -> encoder -> uvc In the chosen zephyr,videoenc is available, the sample will pipe the camera receiver to the encoder and then the UVC device instead of directly the camera receiver to the UVC. In order to making the change as simple as possible, the source device of the UVC device is renamed uvc_src_dev instead of video_dev previously since, depending on the configuration, the UVC source might be either the video_dev or the encoder_dev. Current implementation has several points hardcoded for the time being: 1. intermediate pixel format between the camera receiver and encoder is set to NV12. This is temporary until proper analysis of video_dev caps and encoder caps is done, allowing to select the common format of the two devices. 2. it is considered that encoder device do NOT perform any resolution change and that encoder output resolution is directly based on the camera receiver resolution. Thanks to this, UVC exposed formats are thus the encoder output pixel format & camera receiver resolutions. This has been tested using the STM32N6-DK and the JPEG codec, leading to the following pipe: IMX335 -> CSI/DCMIPP -> JPEG -> UVC Signed-off-by: Alain Volmat <[email protected]>
This commit prepares introduction of the UVC Frame Based support by using the struct uvc_frame_descriptor as parameter of most of the UVC functions. struct uvc_frame_descriptor contains the common fields for all supported frame type and then depending on the DescriptorSubtype the pointer is casted in the correct struct definition. Signed-off-by: Alain Volmat <[email protected]>
The frame_based descriptors differ from the frame descriptors in that there is no dwMaxVideoFrameBufferSize field. In order to do that, add a new uvc_frame_based_discrete_descriptor structure to be used to fill in proper information into the frame descriptor. In addition to that, a new format descriptor is also added for frame based transfer. Signed-off-by: Alain Volmat <[email protected]>
Correct handling of the return value within the enqueue ops. Signed-off-by: Alain Volmat <[email protected]>
Add proper check of the return value of video_enqueue / video_dequeue. Signed-off-by: Alain Volmat <[email protected]>
Add H264 in the supported format in order to be able to advertise H264 encoded stream. Signed-off-by: Alain Volmat <[email protected]>
Add board specific conf files for the stm32n6570_dk when using the JPEG encoder. Signed-off-by: Alain Volmat <[email protected]>
Add board specific conf files for the stm32n6570_dk when using the VENC H264 encoder. Signed-off-by: Alain Volmat <[email protected]>
avolmat-st
force-pushed
the
pr-uvc-videoenc-support
branch
from
75eec12 to
3edf0b3
Compare
|
avolmat-st
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow creating a pipeline as follow
camera receiver -> encoder -> uvc
I post this PR while there are still some points to improve (cf hardcoded stuff detailed below) in order to get some first feedbacks. Since this depends on the UVC PR (PR #93192) and DCMIPP UVC PR (PR #94562), there are lots of commits in this PR. However only the LAST 8 COMMITS is relevant for this PR.
If the chosen zephyr,videoenc is available, the sample will pipe the camera receiver to the encoder and then the UVC device instead of directly the camera receiver to the UVC.
In order to making the change as simple as possible, the source device of the UVC device is renamed uvc_src_dev instead of video_dev previously since, depending on the configuration, the UVC source might be either the video_dev or the encoder_dev.
Current implementation has several points hardcoded for the time being:
1. intermediate pixel format between the camera receiver and encoder
is set to NV12. This is temporary until proper analysis of video_dev
caps and encoder caps is done, allowing to select the common
format of the two devices.
2. it is considered that encoder device do NOT perform any resolution
change and that encoder output resolution is directly based on the
camera receiver resolution. Thanks to this, UVC exposed formats
are thus the encoder output pixel format & camera receiver
resolutions.
This has been tested using the STM32N6-DK and the JPEG codec, leading to the following pipe:
IMX335 -> CSI/DCMIPP -> JPEG -> UVC
compiled via:
And also VENC codec, leading to the following pipe:
IMX335 -> CSI/DCMIPP -> VENC -> UVC
compiled via: