Fill HDR Side Data for HEVC Decoding

[softworkz]

Structures for HDR side data have recently been added for HEVC encoding: a9685bf :

• mfxExtContentLightLevelInfo
• mfxExtMasteringDisplayColourVolume

What missing is to fill those structures when decoding HEVC video.

Could this be added?

[dmitryermilov]

@softworkz , just to be on the same page: would you like to have this reporting from DecodeHeader and together with each output frame (if there're per frame SEI), right?

[softworkz]

@softworkz , just to be on the same page: would you like to have this reporting from DecodeHeader and together with each output frame (if there're per frame SEI), right?

Correct. From all videos that I've seen so far, even with static HDR (like HDR10), the data is in every frame (SEI type 137 and 144 IIRC).

Purposes:

• Re-Encoding while keeping HDR data
  e.g. just scaling down while retaining color format and HDR (you've already done the encoding side)
• The data is required for tone mapping
  • It's a prerequisite for this: Support for Tone Mapping #1949
  • But even right now it's a problem when doing tone mapping for example with OpenCL, because QuickSync decoders (unlike VAAPI) do bitstream and codec parsing, so it's not possible to get at this data (e.g. using ffmpeg)

[dmitryermilov]

Thanks for details. Yes, we can do it but please expect some delay, ~2 weeks.

[softworkz]

2 weeks? That's not a delay - that would be awesome 😆

[wangyan-intel]

@pengxin99 Could you please take a look? Thanks.

[softworkz]

Thanks for details. Yes, we can do it but please expect some delay, ~2 weeks.

@dmitryermilov @pengxin99 - Is there any progress in sight on this issue?

[wangyan-intel]

@softworkz Sorry for slow response. As you know, we are transferring from MSDK legacy to oneVPL (https://github.com/oneapi-src/oneVPL-intel-gpu). For legacy branch, it has been implemented by @pengxin99 with extension buffer. But this flow cannot be used in VPL architecture. We are evaluating the path and will update it ASAP.

[softworkz]

Only the "legacy" path is of interest at this time. Will the implementation from @pengxin99 be merged?

[softworkz]

For Linux we could easily merge this patch ourselves, but not for Windows - that's the big problem.
Or is there a way how we could build our own libmfxhw64.dll for Windows?

[wangyan-intel]

@softworkz
In general, we will not add new feature into legacy path and just bug fix. Of course, you could use @pengxin99's legacy PR. Windows code isn't included in current repo (https://github.com/Intel-Media-SDK). @dmitryermilov @onabiull Could we provide binary build beyond the normal release build?

[softworkz]

Or even better - could we get the Windows code (maybe under NDA) to build our own binaries?
Thanks,
sw