#493: Data for measuring audio-video synchronization and end-to-end delay in realtime communications

Visit on Github.

Opened Mar 31, 2020

Hello TAG!

I'm requesting a TAG review of captureTimestamp and senderCaptureTimeOffset.

These two new data fields are added to RTCRtpContributingSources as WebRTC extensions. They are used for measuring how synchronized the audio and video tracks are, and the end-to-end delay, in real-time communication applications. They are particularly desired by real-time communication systems, where an intermediate steam regenerator that terminates the streams originating from senders, an audio mixer as an example, is involved.

Real applications, Google Hangouts and Meet, for example, have been asking for reliable audio video synchronization and end-to-end delay measurements for several years, which we can back up with discussions from 2017, see https://github.com/w3c/webrtc-stats/issues/158.

Further details:

[] I have reviewed the TAG's API Design Principles

  • Relevant time constraints or deadlines: n/a

  • The group where the work on this specification is currently being done: WebRTC W3C WG

  • The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): WebRTC W3C WG

  • Major unresolved issues with or opposition to this specification: N/A

  • This work is being funded by: Google

We'd prefer the TAG provide feedback as (please delete all but the desired option): 🐛 open issues in our GitHub repo for each point of feedback

Discussions

2020-04-13

Minutes

David: Was hoping Sangwhan had some comments.

Peter: I'm a little bit concerned that we'll miss important comments because this is outside of our field of expertise.

Sangwhan: So the original question I had was are there any security implications of having a somewhat disentaglable sum of delays which map to your hop route - and I don't know. I'm sure someone with the right security experience can find a way to abuse this but all I know is that given small enough hops and the sums of maybe 2-3 users it seems numerically possible to unmask, whether or not hat that is useful.. no idea.

Re-enforce the original "please discuss with IETF" after repeating the original question