Skip to content

Add VP8 and VP9 Android gRPC stream options - #57

Draft
krystofwoldrich-agent wants to merge 4 commits into
expo:mainfrom
krystofwoldrich-agent:codex/vp8-vp9-android-streaming
Draft

krystofwoldrich-agent wants to merge 4 commits into
expo:mainfrom
krystofwoldrich-agent:codex/vp8-vp9-android-streaming

Conversation

@krystofwoldrich-agent

@krystofwoldrich-agent krystofwoldrich-agent commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • expose H.264, VP8, and VP9 controls for Android Emulator gRPC WebSocket streaming
  • teach the Hub client to consume codec-bearing video generations and SEMU v1/v2 frame metadata, then configure WebCodecs for H.264, VP8, or VP9
  • keep legacy H.264-only serve-emu connections working when codec announcements are absent
  • route standalone CLI codec selection into serve-emu and reject VPx with the WebRTC transport
  • automatically use WebSocket for VP8/VP9, disable the incompatible WebRTC control, and restore WebRTC availability for H.264
  • compose codec selection with the newly merged gRPC input-source setting so either can change without resetting the other
  • add documentation, focused contract/UI tests, and a release changeset
  • pin the feature-only VP8/VP9 serve-emu head

Depends on expo/serve-emu#21. The submodule is pinned to feature-only commit 3f5cbc8 from that branch.

The independent MMAP scheduler and encoder-write cadence correction is tracked in expo/serve-emu#23 and expo-device-hub#60; it is intentionally not included here.

The pinned scrcpy 4.0 server cannot emit VP8 or VP9, so those options remain scoped to the gRPC screenshot source.

Validation

  • bun run typecheck — 11/11 tasks passed
  • bun run lint — 4/4 tasks passed with no errors or warnings
  • bun run test — 5/5 package tasks passed
  • bun run build — 7/7 tasks passed, including vendored serve-emu and the standalone Hub bundle
  • focused root integration suite: 78 tests passed, 0 failed
  • serve-emu bun run check
    • 868 tests passed, 0 failed, 4,702 assertions
    • coverage, all TypeScript checks, documentation sync, production build, and package smoke passed
  • agent-browser CLI against a Pixel_10 AVD on Android 17 / Chrome
    • Android Emulator 37.2.7, including the Apple Silicon MMAP fix
    • verified live H.264, VP8, and VP9 switching and changing 436×980 WebCodecs frames over gRPC MMAP
    • verified VP8/VP9 disable WebRTC and H.264 re-enables it
  • current head ab281d3 is rebased on current main and is mergeable

Feature-only 980px gRPC+MMAP comparison

These measurements were captured on the patch-equivalent pre-rebase serve-emu feature head d5879f0 with a deterministic high-motion workload, one viewer, 8 Mbps, a 60 FPS target, and 436×980 output.

Codec Usable MMAP FPS Encoded FPS Viewer FPS
H.264 37.793 39.890 37.800
VP8 37.753 37.738 37.450
VP9 37.065 37.118 36.325

All codecs were constrained by the same pre-existing, codec-independent MMAP pacing ceiling. The table therefore validates comparable codec behavior rather than final 60 FPS capacity. The pipeline diagnosis, measurements, tests, and fixes live in the standalone cadence PRs linked above.

Review

The codec implementation, VP8 tuning, and the input-source integration were reviewed against the repository standards and request. Identified issues were fixed before the final rebase.

@krystofwoldrich-agent
krystofwoldrich-agent force-pushed the codex/vp8-vp9-android-streaming branch from 9bc8af7 to e47d8b8 Compare September 4, 2026 11:52
Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Krystof Woldrich <31292499+krystofwoldrich@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
@krystofwoldrich-agent
krystofwoldrich-agent force-pushed the codex/vp8-vp9-android-streaming branch from e47d8b8 to ab281d3 Compare September 4, 2026 12:11
@krystofwoldrich-agent

Copy link
Copy Markdown
Contributor Author

Post-rebase verification for feature-only head ab281d3 (serve-emu pin 3f5cbc8):

@krystofwoldrich
krystofwoldrich marked this pull request as draft September 4, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant