#148: WebRTC Stats review

Visit on Github.

Opened Dec 20, 2016

Hello TAG!

I'm requesting a TAG review of:

Further details (optional):

You should also know that this spec is a companion document to the main WebRTC API

We'd prefer the TAG provide feedback as (please select one):

  • open issues in our Github repo for each point of feedback
  • open a single issue in our Github repo for the entire review
  • leave review feedback as a comment in this issue and @-notify [github usernames]

Discussions

Comment by @torgo Feb 8, 2017 (See Github)

Taken up at Feb 2017 f2f.

Comment by @hadleybeeman Feb 8, 2017 (See Github)

They haven't yet gotten to the security and privacy questionnaire, but they have an open issue reminding them to do it. I left a comment on their issue asking for their thoughts specifically on fingerprinting: https://github.com/w3c/webrtc-stats/issues/99

Comment by @torgo Apr 29, 2017 (See Github)

Paging @dontcallmedom - we are trying to figure out how to progress this.

Comment by @travisleithead May 9, 2017 (See Github)

In the two examples 1 & 2, the value of the type member is inconsistent. In one case the value of outboundrtp is used, while in the other it's outbound-rtp.

Comment by @travisleithead May 9, 2017 (See Github)

In example 2, now.id is used like an index into the stats array, but in example 1, it doesn't look like id is meant to be an index?

Comment by @travisleithead May 9, 2017 (See Github)

In Section 4.1 (Guidelines) you may link to our Design principals document for your note about camel-case naming of members ;-)

Comment by @travisleithead May 9, 2017 (See Github)

In 4.3, I'm concerned that the advice given doesn't align with the design expectations around dictionaries. The document states:

The general rule is that stats objects, once created, exist for the lifetime of the PeerConnection that contains them, even when the underlying object they report on is stopped or deleted. This is important in order to report consistently on short-lived objects and to be able to report totals over the lifetime of a PeerConnection. When an object is closed or deleted, the timestamp member of the stats object stops updating; it is frozen at the time when the object stopped.

Yet, the design of a dictionary type in WebIDL is such that it is always a value-copy of the information put into it, thus once created, it has no intrinsic tie back to any platform object, and exists as a pure data object, subject to collection by the GC as soon as it is no longer referenced.

It would be counter-design to have the timestamp member of the dictionary get updated either proactively (by the platform) or reactively (on demand, by the platform) because it would require the platform to keep track of each dictionary returned from the getStats call. If this behavior is desired, than an interface type is required (not a dictionary).

Comment by @travisleithead May 9, 2017 (See Github)

Example 2 seems to have fallen out-of step with Example 8 from the core WebRTC spec. The latter has a get method that looks helpful...

Comment by @travisleithead May 9, 2017 (See Github)

Yes, you'll want to sync-up the examples with the maplike<..> definition in the core WebRTC spec.

Comment by @travisleithead May 9, 2017 (See Github)

It looks like this document's stats "infrastructure" has fallen out-of-step with the parent WebRTC spec... In looking over the various specific stats, nothing jumps out at me as very concerning. Perhaps there is a bit of a privacy concern regarding surfacing all the network statistics, however, this is only information available after a connection is in progress, which means at least the two parties much have had some degree of mutual desire to connect--which means sharing these stats may be just fine.

Comment by @torgo Jul 25, 2017 (See Github)

We've followed up. If we don't hear anything back by 8th of August then we will close this issue.

Comment by @torgo Aug 8, 2017 (See Github)

@dontcallmedom last chance to comment on this?

Comment by @dontcallmedom Aug 8, 2017 (See Github)

just discussed this with the chairs - we'll make it happen in the next few weeks, sorry for the delay but we've been focusing on getting the main WebRTC spec out of the door (ie to CR)

Comment by @torgo Sep 26, 2017 (See Github)

@dontcallmedom any update on this? we're just reviewing it at our f2f. thanks! :)

Comment by @vr000m Sep 26, 2017 (See Github)

@torgo we are discussing this shortly, should have an update on the next steps shortly.

Comment by @dontcallmedom Sep 26, 2017 (See Github)

@torgo as @vr000m reported, we've just been discussing this; we now have a work plan, but we will need a couple more weeks before we have something usefully reviewable by the TAG.

Comment by @alvestrand Oct 25, 2017 (See Github)

Roll-up reply to comments.

Travis' comments:

  • Examples have been fixed / reworked since the review
  • Linking to "Design principles": Will do.
  • Text about "the general rule" has been reworked in response to another issue. Return-by-value was the paradigm that was intended all along.
  • Examples are already reworked, and infrastructure realigned with webrtc-pc
  • Privacy concerns are in PR https://github.com/w3c/webrtc-stats/pull/251

Remaining change (link): https://github.com/w3c/webrtc-stats/pull/266

Comment by @annevk Oct 25, 2017 (See Github)

Why does this not require secure contexts?

Comment by @travisleithead Jan 31, 2018 (See Github)

Why does this not require secure contexts?

@annevk, I'm not sure this applies specifically to the review of WebRTC stats as all this spec defines are dictionaries and enums… but I think your comment is more generally applicable to Web RTC.

Comment by @travisleithead Jan 31, 2018 (See Github)

@alvestrand the changes made above look good to me! Thanks for looking into them!

Comment by @travisleithead Jan 31, 2018 (See Github)

Why does this not require secure contexts?

@annevk, I'm not sure this applies specifically to the review of WebRTC stats as all this spec defines are dictionaries and enums… but I think your comment is more generally applicable to Web RTC.

Assuming the above is true, we're tracking this with a new issue so that we can close this one :-)

See #228

Comment by @travisleithead Jan 31, 2018 (See Github)

Closing now. @alvestrand and @dontcallmedom thanks for working with the TAG!

Comment by @annevk Jan 31, 2018 (See Github)

It applies more generally, but it would be good to get it right for new things from the start.