Skip to content

Commit 4be6c3c

Browse files
authored
Support HEVC/H.265 video with hev1 flavor (in addition to hvc1) (#7569)
### What * Details over on rerun-io/re_mp4#9 In short: we supported some kinds of hevc and not others. <img width="295" alt="image" src="https://github.com/user-attachments/assets/570115c5-bbd0-4624-bf55-eaf9a9679e52"> <img width="295" alt="image" src="https://github.com/user-attachments/assets/a85bc5d7-7b4d-43e6-a1d3-972c6bbed3b0"> Test videos: https://rerun.io/viewer/pr/7569?url=https://static.rerun.io/rrd/0.19.0/hvc1_5b55f7b0f38dc9ce3fca58b8e0ec9f1e084aa8a9.rrd https://rerun.io/viewer/pr/7569?url=https://static.rerun.io/rrd/0.19.0/hev1_1132ff3c2853de110ae7f323af10bbc1a556ac17.rrd TODO gather testing data from... * [x] Windows Chrome * [x] Windows Firefox * [x] Linux Chrome * [x] Linux Firefox * [x] More linux Chrome samples - the note there is tentative so far. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7569?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7569?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7569) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
1 parent e6f93d8 commit 4be6c3c

File tree

4 files changed

+19
-10
lines changed

4 files changed

+19
-10
lines changed

Cargo.lock

+1-1
Original file line numberDiff line numberDiff line change
@@ -5272,7 +5272,7 @@ dependencies = [
52725272
[[package]]
52735273
name = "re_mp4"
52745274
version = "0.1.0"
5275-
source = "git+https://github.com/rerun-io/re_mp4?rev=3236c76f9228cf6ab0b2bfb1b8f9ffcde975ea05#3236c76f9228cf6ab0b2bfb1b8f9ffcde975ea05"
5275+
source = "git+https://github.com/rerun-io/re_mp4?rev=4705e85f62ddb47c32d9c091d8f0662068211bc8#4705e85f62ddb47c32d9c091d8f0662068211bc8"
52765276
dependencies = [
52775277
"byteorder",
52785278
"bytes",

Cargo.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -550,7 +550,7 @@ missing_errors_doc = "allow"
550550

551551
# commit on `rerun-io/mp4` `master` branch: https://github.com/rerun-io/re_mp4/tree/master
552552
# https://github.com/rerun-io/mp4/commit/3236c76f9228cf6ab0b2bfb1b8f9ffcde975ea05
553-
re_mp4 = { git = "https://github.com/rerun-io/re_mp4", rev = "3236c76f9228cf6ab0b2bfb1b8f9ffcde975ea05" }
553+
re_mp4 = { git = "https://github.com/rerun-io/re_mp4", rev = "4705e85f62ddb47c32d9c091d8f0662068211bc8" }
554554

555555
# commit on `rerun-io/re_arrow2` `main` branch
556556
# https://github.com/rerun-io/re_arrow2/commit/e4717d6debc6d4474ec10db8f629f823f57bad07

crates/store/re_video/src/mp4.rs

+5-2
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
use super::{Config, Sample, Segment, Time, Timescale, VideoData, VideoLoadError};
44

55
pub fn load_mp4(bytes: &[u8]) -> Result<VideoData, VideoLoadError> {
6-
let mp4 = re_mp4::read(bytes)?;
6+
let mp4 = re_mp4::Mp4::read_bytes(bytes)?;
77

88
let mp4_tracks = mp4.tracks().iter().map(|(k, t)| (*k, t.kind)).collect();
99

@@ -86,7 +86,10 @@ pub fn load_mp4(bytes: &[u8]) -> Result<VideoData, VideoLoadError> {
8686

8787
fn unknown_codec_fourcc(mp4: &re_mp4::Mp4, track: &re_mp4::Track) -> re_mp4::FourCC {
8888
let stsd = &track.trak(mp4).mdia.minf.stbl.stsd;
89-
stsd.unknown.first().copied().unwrap_or_default()
89+
match &stsd.contents {
90+
re_mp4::StsdBoxContent::Unknown(four_cc) => *four_cc,
91+
_ => Default::default(),
92+
}
9093
}
9194

9295
/// Returns whether a buffer is MP4 video data.

docs/content/reference/video.md

+12-6
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,15 @@ Video playback is done using the browser's own video decoder, so the supported c
4040

4141
Overall, we recommend using Chrome or another Chromium-based browser, as it seems to have the best video support as of writing.
4242

43-
When choosing a codec, we recommend [AV1](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#av1), as that seems to have the best overall playback support. Since AV1 is patent-free, it is also likely the first codec we will support in the native viewer.
43+
When choosing a codec, we recommend [AV1](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#av1),
44+
as it seems to have the best overall playback support while also having very high compression quality.
45+
Since AV1 is patent-free, it is also likely the first codec we will support in the native viewer.
4446

4547
For decoding video in the Web Viewer, we use the [WebCodecs API](https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API).
4648
This API enables us to take advantage of the browser's hardware accelerated video decoding capabilities.
4749
It is implemented by all modern browsers, but with varying levels of support for different codecs, and varying levels of quality.
4850

49-
With that in mind, here are the browsers which we have tested and verified to work:
51+
With that in mind, here are the browsers which we have tested and verified to generally work:
5052

5153
| | Linux | macOS | Windows |
5254
| ---------- | ------ | ----- | ------- |
@@ -64,12 +66,16 @@ which codecs are supported by which browser, see [Video codecs on MDN](https://d
6466

6567
At the moment, we test the following codecs:
6668

67-
| | Linux | macOS | Windows |
68-
| ----- | ----- | ------- | ------- |
69-
| AV1 || 🚧[^4] ||
70-
| H.264 ||||
69+
| | Linux Firefox | Linux Chrome | macOS Firefox | macOS Chrome | macOS Safari | Windows Firefox | Windows Chrome |
70+
| ---------- | ------------- | ------------ | ------------- | ------------ | ------------ | --------------- | -------------- |
71+
| AV1 ||||| 🚧[^4] |||
72+
| H.264/avc ||||||||
73+
| H.265/hevc ||||| 🚧[^6] || 🚧[^7] |
7174

7275
[^4]: Safari/WebKit does not support AV1 decoding except on [Apple Silicon devices with hardware support](https://webkit.org/blog/14445/webkit-features-in-safari-17-0/).
76+
[^5]: Firefox does not support H.265 decoding on any platform.
77+
[^6]: Safari/WebKit has been observed suttering when playing `hvc1` but working fine with `hevc1`. Despite support being advertised Safari 16.5 has been observed not support H.265 decoding.
78+
[^7]: Only supported if hardware encoding is available. Therefore always affected by Windows stuttering issues, see [^3].
7379

7480
## Links
7581
* [Web video codec guide, by Mozilla](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs)

0 commit comments

Comments
 (0)