#493: Data for measuring audio-video synchronization and end-to-end delay in realtime communications
Discussions
2020-04-13
David: Was hoping Sangwhan had some comments.
Peter: I'm a little bit concerned that we'll miss important comments because this is outside of our field of expertise.
Sangwhan: So the original question I had was are there any security implications of having a somewhat disentaglable sum of delays which map to your hop route - and I don't know. I'm sure someone with the right security experience can find a way to abuse this but all I know is that given small enough hops and the sums of maybe 2-3 users it seems numerically possible to unmask, whether or not hat that is useful.. no idea.
Re-enforce the original "please discuss with IETF" after repeating the original question
OpenedMar 31, 2020
Hello TAG!
I'm requesting a TAG review of captureTimestamp and senderCaptureTimeOffset.
These two new data fields are added to RTCRtpContributingSources as WebRTC extensions. They are used for measuring how synchronized the audio and video tracks are, and the end-to-end delay, in real-time communication applications. They are particularly desired by real-time communication systems, where an intermediate steam regenerator that terminates the streams originating from senders, an audio mixer as an example, is involved.
Explainer¹ (minimally containing user needs and example code): https://github.com/w3c/webrtc-extensions/blob/master/explainer.md
Specification URL: https://w3c.github.io/webrtc-extensions/#rtcrtpcontributingsource-dictionary
Tests:
Tests for captureTimestamp: https://cs.chromium.org/chromium/src/third_party/blink/web_tests/external/wpt/webrtc-extensions/RTCRtpSynchronizationSource-captureTimestamp.html
Tests for senderCaptureTimeOffset: will be added
Primary contacts (and their relationship to the specification):
Organization(s)/project(s) driving the specification: Google Hangouts
Key pieces of existing multi-stakeholder review or discussion of this specification: The proposal has been presented to W3C WG and browser implementers. While there is no signal from other implementers to implement this yet, the proposal has been reviewed without concerns raised.
Real applications, Google Hangouts and Meet, for example, have been asking for reliable audio video synchronization and end-to-end delay measurements for several years, which we can back up with discussions from 2017, see https://github.com/w3c/webrtc-stats/issues/158.
Further details:
[] I have reviewed the TAG's API Design Principles
Relevant time constraints or deadlines: n/a
The group where the work on this specification is currently being done: WebRTC W3C WG
The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): WebRTC W3C WG
Major unresolved issues with or opposition to this specification: N/A
This work is being funded by: Google
We'd prefer the TAG provide feedback as (please delete all but the desired option): 🐛 open issues in our GitHub repo for each point of feedback