It is pretty hard to dig out from the example and a long Github discussion.

A small explainer showing the new API and explaining the use-cases would be much more welcome. We are all busy people and having to manually dig through a discussion thread is a bit too much. An explainer could summarize all this. You could also write a summary in the issue and link to that instead.

FWIW Chrome has been using a private extension(webcam_private.idl) to satisfy this use case.

• Does this API differ from the private and presumable battle tested extension?
• pan and tilt values goes from what? 1-100? 0-1?
• Can all cameras tilt and pan as much? or is there a way to query their limits?
• It feels a bit weird that a pan value is considered a constrain and not just a control value.
• what was the conclusion of the "- Pan, Roll, Tilt vs Pan and Tilt" discussion

Thanks @kenchris for the feedback. I have added code snippet and a brief explainer/motivation in the issue. Summary of the long Github discussion was mainly -

• Pan, Roll, Tilt vs Pan and Tilt
• Units to use, specifically - Arc seconds vs degrees.

Units to use, specifically - Arc seconds vs degrees.

So which one are you using?

Also can you give a short summary on the pan/roll/tilt vs no roll

FWIW Chrome has been using a private extension(webcam_private.idl) to satisfy this use case.

• Does this API differ from the private and presumable battle tested extension?

Presumably battle tested API gave Pan Tilt Zoom (PTZ). Zoom is already available from the start of MediaStream-ImageCapture API. Pan and Tilt are the recent additions and hence this issue

• pan and tilt values goes from what? 1-100? 0-1?

depends on camera

• Can all cameras tilt and pan as much? or is there a way to query their limits?

You can query the range. Different camera have different ranges. For example electronic Pan/Tilt do not pan more than 10 degrees.

• It feels a bit weird that a pan value is considered a constrain and not just a control value.
• what was the conclusion of the "- Pan, Roll, Tilt vs Pan and Tilt" discussion

Roll isn't commonly available in consumer webcams. Even though Pan/Tilt may be niche, we felt that Roll is even more niche.

depends on camera

Actually reading the PR I read this: 1/3600th of a degree. Values are in the range from –180x3600 arc seconds to +180x3600 arc seconds

Units to use, specifically - Arc seconds vs degrees.

So which one are you using?

arc-seconds. I have summarized this in this comment

Also can you give a short summary on the pan/roll/tilt vs no roll

Image Capture provides a way to query the supported the MediaTrackCapabilities, by querying that on a video track you'll get for both pan and tilt their allowed ranges and steps, e.g for pan (same for tilt):

const trackCapabilities = imageCapturer.track.getCapabilities();
if (trackCapabilities.pan === undefined) {
  console.error('pan not supported, boo!');
} else {
  const maxPan = trackCapabilities.pan.max;
  const minPan = trackCapabilities.pan.min;
  const stepPan = trackCapabilities.pan.step;
}

All those numerical values will be in arc-seconds.

So we had a brief discussion of the potential privacy issues in today's TAG meeting -- if hardware capabilities were to change substantially (e.g., laptop cameras that rotate a lot more than today), it feels like an implementation might want to add a separate permission prompt for the user to grant access to pan/tilt. So we'd like to make sure that the specification is designed in a way that would allow that to happen later on. It sounds (from a very brief look/discussion today) like the API is sufficiently asynchronous that that's the case, but it's worth thinking about a little more carefully.

@dbaron / TAG : Just for clarification, does this mean a permission prompt first for getUserMedia() to access camera, and then another permission prompt to access the pan/tilt feature? Suppose a developer is making a PWA camera where she wants to use both pan/tilt feature and other MediaStream properties, does she ask user for 2 separate permission prompts?

I think it's up to the user-agent / implementation to decide how to structure the prompts -- the key part is that the API should be designed in a way that allows for appropriate choices. This generally means it needs to involve promises rather than being synchronous so that a user agent might resolve the promise after prompting the user. It may also mean that the intent to use both the camera and pan/tilt should be present at the same time in case the user-agent wants to combine them into one prompt.

Of course, these demands may have tradeoffs with other desired characteristics of the API, so there might be reasons not to satisfy them. But they should be considered.

I presume that pan/tilt can be applied to any video track regardless of whether or not the imagecapture feature is being used? If so, it seems odd that it's defined here (along with most (all?) of the other capabilities/constraints defined)

These new constrains apply to the live video feed, and that's covered in the Spec by making these an extension to the MediaStreamTrack: they are MediaTrackSettings, whereas others are only seen upon takePhoto() and they are PhotoSettings.

Btw, the permission thing @dbaron mentions also applies to "zoom"

@riju Sorry this took so long. It's the first time we mechanically (or logically, depending on the hardware implementation) allowed to "move" things in the physical world from the web, and firsts are always a bit scary.

We've discussed this in quite a bit of detail, and the group opinion (after a bit of back and forth) is that capture and control should be modeled in a way that it can be two distinct permissions; ideally requestable in a single call. Cases are where you would be fine showing one fixed (tidy) part of a room that is conference safe, and not allow access to the other side; this in native is covered by the conference software, but in the web we can't assume the software is trustworthy and will respect the user's preference.

The plumbing for implementations to be able to provide a way for the user to opt-out of this feature, while giving permission to the video stream seems like a valid use case, and we'd like to see this covered.

Thanks TAG @kenchris, @cynthia, @dbaron for the feedback.

We're changing the process of how we manage the reviews; this one is considered stalled because we have checked the relevant chromium.org ML thread and didn't see a clear resolution. (Last reply seems like from a couple weeks ago)

We'll leave this open so you can let us know what the resolution on your side is.

We have contacted the chromium UX/Privacy team for their feedback on TAG's recommendation. We will update this thread when we get a response.

@riju haven't heard back from you folks in a while, has anything happened in the meanwhile or were our concerns deemed irrelevant? Let us know if you have anything to share.

@cynthia: we shipped the Pan/Tilt feature behind a flag. The other stakeholders like Chrome's UX and privacy teams are evaluating. We also need to evaluate how many users do have pan/tilt cameras. I can give TAG folks a demo in Tokyo/Fukuoka :)

Hi @riju we will be discussing this in a breakout at 15:30 tomorrow at our Tokyo f2f (most likely in the cafeteria on 43).

@riju you promised us feedback from stakeholders but we haven't heard anything back and it has been a while.

I don't like the name pan and tilt, it is really camera motion controls - maybe we could call it something like that instead

The terminology might be weird, but it's somewhat well-known. e.g. https://en.wikipedia.org/wiki/Pan%E2%80%93tilt%E2%80%93zoom_camera

PTZ seems to be the name that people use, so ptz: true might make more sense than Pan-Tilt-Zoom: true as I saw in one of the PRs

@riju would it be possible to write an explainer which is a requirement today. Also @torgo has some privacy concerns, which conflicts with all the NO answers in the questionnaire (high value data indirectly).

Can you read (or deduct) the angle using this API? Things like that would be very valuable information in an explainer

Looking at this again. The security & privacy questionnaire answers seem questionable. In particular, (3.2) the technology may allow access to data that the user had not intended to show and (3.4) the tech may allow access to persistent state from our read of the spec. Can you please update the questionnaire and let us know what the mitigation strategy is for these aspects? @riju

@torgo @kenchris : Thanks for your patience, we worked with UX/Privacy folks (from Chrome team) and looks like we need to make some changes to the API in terms of letting users know that the camera supports "pan/Tilt/zoom". For example, the model we are experimenting now is -

Camera permission without PTZ (obtained using an older version of the user agent, or with another camera) is not implicitly upgraded to Camera permission with PTZ (even if the hardware supports it). The permission has to be re-requested.

We will come up with a solution in a few weeks and then ping TAG.

This was on our breakout schedule but as there is still no explainer, this is stalled for now

Friendly ping @riju

The TAG has decided that if we don't hear back before our next meeting we will close this with [resolution: unsatisfied] to get it off our radar and concentrate on more active tasks.

@kenchris I will be drafting the owed explainer in the following days.

May I ask you kindly to keep this issue opened as I'm collecting info? The current COVID-19 situation doesn't help. Thank you for your understanding.

Sure, sounds good - good to know you are working on this. @torgo let's postpone looking at this for a couple of weeks

The PTZ explainer is now available at https://github.com/w3c/mediacapture-image/blob/master/ptz-explainer.md

Hi folks - we had a good discussion on today's TAG breakout with @kenchris and I just had a few followup questions.

1. it was good to hear that there are mitigations against privacy issues being discussed, in particular a distinct permission prompt for pan/tilt/zoom and the idea that the feature will be disabled when the tab is not in focus.

2. the explainer includes some of this info but not everything - can you please make it more clear?

3. the explainer lacks a privacy & security considerations section (and lacks the word "privacy") and really needs to have this info explicitly called out for such a powerful API

4. what aspects of any of the above are intended for the spec as opposed to the implementation?

Hi @riju - we are just following up on this one on our TAG breakout call today. Has there been any update / progress on the questions listed above?

Hi @torgo, not much. We just landed the platform support for Windows and CrOS. Linux was working fine. We are still discussing with the Privacy folks about the details and then we can update this audience.

Hi @riju – As far as the API design design goes, we are OK. However, we're going to mark this as "unsatisfied" because it really doesn't look like the security & privacy issues we've raised are being taken seriously. I would really encourage you to re-read our security & privacy self check and to add some explicit info to the explainer and to the spec covering abuse scenarios and mitigations against these scenarios.

Thanks @torgo for the thumps up on the API design. We will take another look at the explainer soon.

@torgo We've finally added Security and Fingerprinting sections to the explainer.

@torgo : We have had some long discussions regarding this API in the WebRTC group and looks like now there's overall consensus among the stakeholders. Hopefully TAG is now satisfied with Privacy and Security information we have added in the PTZ explainer .