Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement WebRTC #114

Merged
merged 7 commits into from
Jan 12, 2024
Merged

Implement WebRTC #114

merged 7 commits into from
Jan 12, 2024

Conversation

amalnanavati
Copy link
Contributor

Describe this pull request. Link to relevant GitHub issues, if any.

Joint with ada_feeding#154.

In service of #112 , this PR implements a WebRTC connection for streaming video from the robot to the client. It implements "Option A" from #112 . Specifically, it launches a headless browser on lovelace to be one end of the P2P connection. Although this should not be necessary (see Option B in #112 ), it was the most straightforward to implement given that is the approach stretch_teleop_interface used.

As such, this PR changes the web app's launch procedure on lovelace. Whereas earlier one had to launch the web app and the web_video_server, now they have to run:

  1. The web app: npm run start (or cd build; python3 -m http.server 3000 if you have already run npm run build).
  2. The WebRTC signaling server: node --env-file=.env server.js
  3. The robot browser: node --env-file=.env start_robot_browser.js
    (NOTE: The robot browser must be the last of the web app commands to run!)

Explain how this pull request was tested, including but not limited to the below checkmarks.

Setup:

  • Pull this PR and ada_feeding#154.
  • cd into feedingwebapp
  • npm install --legacy-peer-deps
  • npx playwright install (Note: you may have to run node --env-file=.env start_robot_browser.js first)

Testing:
The below tests were run with the real web app running (see launch procedure above), the real perception launchfile, but the dummy motion action servers.

  • Load the app on a smartphone (Amal's iPhone). For every screen with a video (i.e., BiteSelection, DetectingFace, VideoModal), move your hand/face in front of the camera and verify there is minimal lag.
  • Go through 3 consecutive bites end-to-end and verify that the video stays up-to-date.
    • Importantly, verify that BiteSelection works and the app receives the response from the action server.
  • Load the app on a desktop computer (weebo) at the same time. Verify that there is no noticeable lag between the smartphone compared to weebo.

Before creating a pull request

  • Format React code with npm run format
  • [N/A] Format Python code by running python3 -m black . in the top-level of this repository
  • Thoroughly test your code's functionality, including unintended uses.
  • Fully test the responsiveness of the feature as documented in the Responsiveness Testing Guidelines. If you deviate from those guidelines, document above why you deviated and what you did instead.
  • [N/A] Consider the user flow between states that this feature introduces, consider different situations that might occur for the user, and ensure that there is no way for the user to get stuck in a loop.

Before merging a pull request

  • Squash all your commits into one (or Squash and Merge)

@amalnanavati amalnanavati merged commit c293b3e into main Jan 12, 2024
4 checks passed
@amalnanavati amalnanavati deleted the amaln/webrtc branch January 12, 2024 03:09
@amalnanavati amalnanavati mentioned this pull request Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant