#517: Allow audio packet rate to adapt in realtime communication.

Visit on Github.

Opened May 28, 2020

Hello TAG!

I'm requesting a TAG review of allowing audio packet rate to adapt in realtime communication.

As the packet rate is a big determining factor to the overall bitrate of an audio stream, an optimal congestion control should be allowed to adapt the packet rate. The audio packet rate is analogous to the video frame rate, which also plays an important role in the video bitrate adaptation.

Although adaptive packet rate may be ubiquitously beneficial, we propose to add a flag for applications to enable or disable it to avoid potential interoperability problems.

Further details:

  • I have reviewed the TAG's API Design Principles

  • The group where the incubation/design work on this is being done (or is intended to be done in the future): WebRTC W3C WG

  • The group where standardization of this work is intended to be done ("unknown" if not known): WebRTC W3C WG

  • Existing major pieces of multi-stakeholder review or discussion of this design: -- interoperability: adapting audio packet rate does not violate existing specifications but may introduce interoperability problems, as some current implementations may take a fixed packet rate as an assumption and thus fail or perform sub-optimally when packet rate changes. We resolved the problem by introducing the adaptivePTime flag as proposed in this review.

    -- format: whether to surface an API for a more detailed control of packet rate than just a boolean to allow a browser built-in adaptation scheme. As it will be difficult for Web developers that have no RTC expertise to make adaptation algorithms from scratch.

  • Major unresolved issues with or opposition to this design: none

  • This work is being funded by: Google

Discussions

Comment by @torgo Jun 9, 2020 (See Github)

Just one question – it's possible there are no security & privacy issues, but could you note that in the request if that's the case? Thanks.

Discussed Jun 15, 2020 (See Github)

Sangwhan: I'm totally OK with this. it's very webrtc specific - it's a minor change. i don't see why we should say no to this. The best explanation is it's mpeg-dash for webrtc audio streams...

Yves: one of the main issues - they're defining something network related in the encoding... but they provided explanation of why they needed to do it there - can't change the settings on the connection metadata. So for them this is the easiest way and it actually makes sense. No security & privacy things added. Minor change that makes sense. Not at the ideal place but most convenient place. We should say "go ahead" and close it.

Alice: i care about audio quality of video calls - how will this effect user experience?

Sangwhan: it will make it lower quality in exchange for less choppy.

Yves: it will modulate the bandwidth used for bandwidth - meaning it can decided to send less packets. Will allow the session to change to a lower quality encoding and also send less packets so use less bandwidth.

Alice: would that be responsive to for example turning off video feed?

Yves: depends on implementation. Goal is to reduce the bandwidth so people don't hear robots.

[closed by consensus

Comment by @cynthia Jun 16, 2020 (See Github)

We discussed this in our call today, and we are happy to see this move forward. Thanks for bringing this to our attention!

Comment by @ylafon Jun 16, 2020 (See Github)

As a followup, we also noted that the parameter is not at the ideal place, it would make more sense to have this parameter at the SDP level, and have ways to dynamically change this at the level, but the rationale provided to have it at the encoding level make sense as a practical way to deploy this. Thanks again.

Comment by @minyuel Jun 16, 2020 (See Github)

Just one question – it's possible there are no security & privacy issues, but could you note that in the request if that's the case? Thanks.

Yes, added to the request. Thanks for mentioning!

As a followup, we also noted that the parameter is not at the ideal place, it would make more sense to have this parameter at the SDP level, and have ways to dynamically change this at the level, but the rationale provided to have it at the encoding level make sense as a practical way to deploy this. Thanks again.

Thanks for the feedback! We considered SDP parameter as an alternative, but we think "the effort of standardizing a new SDP parameter is large. Another drawback is that the SDP negotiation lacks dynamic nature, as it is difficult to re-configure during a call" (as has been written in the explainer: https://w3c.github.io/webrtc-extensions/explainer#a-new-flag-in-rtcrtpencodingparameters-for-adaptive-packet-rate)