Bare metal fails over virtual network #100

Activity

1. HoustonHuff commented on Jun 25, 2023

   

   HoustonHuff

   on Jun 25, 2023

   More actions

   Hi, I'm ezbrooks's colleague and have been troubleshooting a different part of the system, namely the packet receiving while communicating over Ubuntu network namespace (using the provided demo); we're seeing a report of each of the video frames being dropped in the webrtc.log, such as below:

   (remote_estimator_proxy.cc:151): {"mediaInfo":{"audioInfo":{"audioJitterBufferDelay":1.7976931348623157e+308,"audioJitterBufferEmitedCount":18446744073709551615,"concealedSamples":18446744973709551615,"concealmentEvents":18446744073709551615,"echoReturnLoss":1.7976931348623157e+308,"echoReturnLossEnhancement":1.7976931348623157e+308,"estimatedPlayoutTimestamp":9223372036854775807,"totalSamplesReceived":18446744073709551615,,"totalSamplesSent":18446744073709551615},"videoInfo":{"framesCaptured":18446744073709551615,"framesDecoded":18446744073709551615,"framesDroped":18446744073709551615,"framesReceived":18446744073709551615,"framesSent":18446744073709551615,"fullFramesLost":18446744073709551615,"hugeFramesReceived":18446744073709551615,"hugeFramesSent":18446744073709551615,"keyFramesReceived":18446744073709551615,"keyFramesSender":18446744073709551615,"partialFramesLost":18446744073709551615,"videoJitterBufferDelay":1.7976931348623157e+308,"videoJitterBufferEmittedCount":18446744073709551615}},"pacerPacingRate":100000.0,"pacerPaddingRate":100000.0,"packetInfo":{"arrivalTimeMs":1687529358363,"header":{"headerLength":28,"paddingLength":0,"payloadType":125,"sendTimestamp":45865,"sequenceNumber":1115,"ssrc":1665672282},"lossRates":0.0,"payloadSize":1079}}

   I've been tracing back through the project to see where I could find where these StatCollect values are originally incremented so that I can debug the frame dropping from there, but as far as I can tell though those keys as they're named by StatCollect aren't found anywhere else besides the build files, though I'm fairly certain that what I'm looking for is in the core files somewhere. Would you happen to know which files are directly responsible for the receiver-sender stream? That's ultimately what I've been looking for.
2. jeongyooneo

   added

   question

   on Jun 26, 2023
3. jeongyooneo commented on Jun 26, 2023

   

   jeongyooneo

   on Jun 26, 2023

   Last edited by jeongyooneo

   Collaborator

   More actions

   Hey @HoustonHuff, thanks for the issue!

   The metrics framesDroped, fullFramesLost, partialFramesLost seem to be initialized with the default values of StatsCollect (each entry of stats_collect_ is initialized with default values which corresponds to ULONG_MAX, i.e. 18446744073709551615).

   You're right that StatsCollect values are not incremented anywhere else. You need to measure the metrics you'd like to instrument and selectively populate the entries of stats_collect_, if you want to.

   However, I'd highly recommend you to use the set of WebRTC-instrumented metrics printed on the receiver-side logs, such as WebRTC.Video.DroppedFrames.Receiver.

   I've added #101 to clear up the confusion. Thanks again for reporting this!
4. jeongyooneo commented on Jun 26, 2023

   

   jeongyooneo

   on Jun 26, 2023

   Last edited by jeongyooneo

   Collaborator

   More actions

   Thanks for trying AlphaRTC, @ezbrooks!

   The SignalEncoderTimedOut error in your receiver log is common when the sender starts to send the video stream, and itself does not show any call failure.

   To check the call quality, you can use the receiver- and sender-side statistics at receive_log.txt and
   send_log.txt. For example, observing the sender-side statistics from your send_log.txt, the call seems to be run with 300Kbps bandwidth (from (bitrate_allocator.cc:394): Current BWE 300000), and the sender-side media bitrate WebRTC.Video.MediaBitrateSentInBps is avg. 255976, looks like there's no special issue w.r.t. the call.

   (send_statistics_proxy.cc:667): WebRTC.Video.InputFramesPerSecond periodic_samples:9, {min:10, avg:10, max:11}
   WebRTC.Video.SentFramesPerSecond periodic_samples:9, {min:10, avg:10, max:10}
   WebRTC.Video.SentToInputFpsRatioPercent 100
   WebRTC.Video.NumberOfPauseEvents 0
   WebRTC.Video.PausedTimeInPercent 0
   WebRTC.Video.BitrateSentInBps periodic_samples:9, {min:175528, avg:264104, max:278944}
   WebRTC.Video.MediaBitrateSentInBps periodic_samples:9, {min:169592, avg:255976, max:270384}
   WebRTC.Video.PaddingBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
   WebRTC.Video.RetransmittedBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
   WebRTC.Video.FecBitrateSentInBps periodic_samples:9, {min:0, avg:0, max:0}
   Frames encoded 196
   WebRTC.Video.DroppedFrames.Capturer 0
   WebRTC.Video.DroppedFrames.EncoderQueue 0
   WebRTC.Video.DroppedFrames.Encoder 1
   WebRTC.Video.DroppedFrames.Ratelimiter 0
   WebRTC.Video.DroppedFrames.CongestionWindow 0
   

   Feel free to re-open this issue if you need more help!
5. jeongyooneo

   closed this as completedon Jun 26, 2023