Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add screen share decryption #4

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

RealKeyboardWarrior
Copy link

@RealKeyboardWarrior RealKeyboardWarrior commented Nov 13, 2022

This PR adds support for decryption of the screen share video feed. This works quite, well and will spit out recording files by default if you enable it, it's disabled by default.

There are quite a few changes in this PR, some breaking and honestly I haven't tested the rebased version.

Here's a sample of the outputted H264 stream (had to change the extension to log to allow upload). It's unplayable in its current form, I hope someone else out there figures out how to actually decode the H264 stream. I managed to decode a a tiny bit of a frame, enough to see some letters but otherwise it's complete garbage.
2022-10-09-18-45-27.h264.log
note: constraint_set7_flag, 1111 1110 = 0xFE - ffmpeg will reject, rewriting to 0 does help a bit

It took me a few months to get to this stage, the ZoomRtpEncoder is untested, never got around to that. They seem to be using their own version of RTP Frame marking as well.

Copy link
Author

@RealKeyboardWarrior RealKeyboardWarrior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was building on your codebase for a while, it was a lot of fun looking into this.
There might be more mistakes in the code, might not even compile & run because I had to rebase all these changes to the refactored version you published in august.


if you do want to do that, you may find this of use (add the following to https://github.com/zoom/sample-app-web/blob/master/Local/js/meeting.js):
// The value 2 is required or you will simply never receive a video stream.
values.Set("as_type", "2")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this type to 2 by default, this is probably breaking.

Comment on lines +12 to +16
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if these are needed as well

@chris124567
Copy link
Owner

This is pretty amazing! I wanted to add this originally but it seemed orders of magnitude more complicated than chat support. I will review & merge into this when I get time later this week. Surprised to see someone using this repository.

@chris124567
Copy link
Owner

chris124567 commented Nov 24, 2022

Apologies for the delay I had some work and school stuff that got in the way. Very impressive work. I can't seem to get it to record me screensharing though. I have the bot join the meeting, then I enable screen sharing, and the recording file (.h264) is created but is always empty.
2022-11-24-12-25-06.log

Also, joining with as_type=2 will break how chat is currently handled (plaintext) but I will fix that as soon as I get this to work.

@RealKeyboardWarrior
Copy link
Author

@chris124567

  1. You must first create the ZoomSession
  2. Add the participant to the ParticipantRoster which is used for deriving the secret key used in encryption / decryption session.ParticipantRoster.AddParticipant(person.ID, person.ZoomID)
  3. Listen for SharingStatusIndication and subscribe to the screenshare stream

Another thing you might to try is re-enable the raw json message logging, because I had to upgrade the version to mimic 2.5.0 because my WebSocket were being forcibly closed!

Over the past weekend, I've successfully managed to decrypt and decode the normal webcam streams! I've resorted to just push the changes to my fork, and then once I consider my efforts finished, I will try and upstream the changes.

@taylorchu
Copy link

It will be really helpful if you can describe how rtp works in sdk if someone wants to contribute to audio decoding .

@RealKeyboardWarrior
Copy link
Author

@taylorchu I have a working audio decryptor which I will push to my fork by 2023 but I have not figured out how to package the audio into a format that ffmpeg or fflpay can decode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants